Commit Graph

931 Commits

Author SHA1 Message Date
Lukas Petersson d1e21cef79 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 444d26582121550a371f14e17d614ba6984cc10a
2022-05-18 19:59:30 -07:00
HaraldSchafer 3494ec7bbd Revert "Joint model (#24521)" (#24581)
This reverts commit 9fc732468ee4d4fd135d679a919c748633f06d02.
old-commit-hash: e2e5378dd3dbcc35777495a578bd842ba3096fac
2022-05-18 10:56:45 -07:00
Adeeb Shihadeh 1d9d8d874b controlsd cpu has crept up
old-commit-hash: 3771ccc6550dfe4f0797bb249354f3f52cbf53a3
2022-05-17 19:42:58 -07:00
HaraldSchafer efa559f9be Revert torque control (#24565)
* torque reversal start

* Fix carmodel tests

* Update ref

* update ref

* Elif is better than if
old-commit-hash: 9f8b03753dbdb131fac7bdc021e841cb30b3bf2c
2022-05-17 19:00:57 -07:00
George Hotz 3ee5b1bf07 make debayer faster (#24557)
* remove local caching

* remove local caching

* reduce camerad power

* break stupid imx390 black level support

* handle edges

* now 13ms, vignetting is 'slightly' less correct

* halfs->floats and inlines, down to 12.9ms

* oops, fix float

* val from 12 is ushort

* don't decide vignetting in the debayer kernel

* 7.77 ms

* adding back black level support was free

* Revert "adding back black level support was free"

This reverts commit a841d17727886807a040dcf856ab22480fa93eff.

* minor

* rip out unused gain and black level, remove print

* save 150mW

* fix replay test

* fix top/bottom rows

* lame left right edge fix

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
old-commit-hash: 72b52f8b7d74bf3ea6fe8bd4d59af1c44c83afdb
2022-05-17 10:18:36 -07:00
HaraldSchafer 78c2e16a03 Joint model (#24521)
* update models

* wip lanelines mhp parsing

* typos

* newer models

* 95995a49-db0c-4261-8776-b90780dc2a8c/600

* Use laneful policy to prevent planner aggression

* c8c3ab69-bd2c-4d1c-a7f8-9e8457e4827b/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* 7205c5af-7532-4215-ad9b-678789054db7/995

* bfb34fa3-1d72-4987-854f-43d42ee01015/950

* change cost

* 8ae2477a-b452-4601-b04c-d3af66ea5e98/950

* Update heading cost

* 9b991314-94f5-4f0b-872e-58d95266e4d0/333

* 9b991314-94f5-4f0b-872e-58d95266e4d0/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* undo lateral planner changes

* rm whitespace

* update ref

* c0acc19d-f2f1-47ae-8838-5382c6c6c7f5/950

* c0acc19d-f2f1-47ae-8838-5382c6c6c7f5/950

* update model refg

* Update model ref

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
old-commit-hash: 7996fd37ef127c76aa62afa27f97ad4898fe1898
2022-05-16 17:26:23 -07:00
Willem Melching 0a2c2a6b28 Add unittest for alert callbacks (#24551)
add alert callback test
old-commit-hash: 31af13101ba2ad3b6282140c70c07e5298ccffce
2022-05-16 16:25:52 +02:00
Joost Wooning 4b3954a21f debayering: update test ref commit
old-commit-hash: 2294fcdbe7c87cec6572f813e8f7f31e94841102
2022-05-16 13:37:16 +02:00
Adeeb Shihadeh 29a516bed8 replay: cleanup unused RGB buffers (#24537)
* replay: cleanup unused RGB buffers

* update replay cam

* little more
old-commit-hash: ab75c53792718d32e51d34336be3fae568051d2b
2022-05-14 19:55:12 -07:00
Shane Smiskol 874ba219dc Fix process replay regen (#24517)
* fix regen issues

* compress like uploader
old-commit-hash: d6c6275163fde581b9c32059561d2f720063e808
2022-05-14 19:53:08 -07:00
George Hotz a053d070be encoderd: enable (#24492)
* enable encoderd

* correct enable line

* fix loggerd tests

* fix power draw and cpu tests

* correct cpu for encoderd

* fix a bug, video_writer is shared

* fix issue with not recording dcam

* add recording state

* wooo tests pass. encode id keeps counting

* core 3

* loggerd then encoderd

* stop loggerd first

* core 3 always online

* see the camera when we see encoder packet

* encoderd on small core uses 37%

* remove encoder logic from loggerd

* delete unit test that doesn't really make sense anymore

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 94b9972eb798408a3c412716b8d9eb8ddb9354bc
2022-05-13 17:56:02 -07:00
Joost Wooning da6d0d878a camerad: combine debayering and rgb_to_yuv opencl kernels (#24452)
* camerad: combine debayering and rgb_to_yuv opencl kernels

* fix border

* fix snapshot

* rename function

* update camerad cpu usage

* update camerad power draw

* vignetting equal to previously

* test other local worksize

* use less floats

* reduce amount of code

* move barrier back

* make faster

* fix corners

* cleanup

* cleanup

* allow more jitter on driverState timing

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 489fbb74b0f532c738cf8c06afe1a5f9ce1e1651
2022-05-13 16:49:31 +02:00
Adeeb Shihadeh aefc73be1b Revert "Joint Model (#24213)"
This reverts commit f6aaafb8d9332c934373fc6fde27abd596b7049f.

old-commit-hash: ef81afc35488905b00f1a0a5d0c0fc36b2bc856f
2022-05-12 22:52:45 -07:00
Adeeb Shihadeh 99e7d3f26c jenkins: workaround lfs bug
old-commit-hash: c643d7460938e0dee04e3578670f0d523cb2f4ba
2022-05-12 22:08:25 -07:00
YassineYousfi 4709fb5f61 Joint Model (#24213)
* update models

* wip lanelines mhp parsing

* typos

* newer models

* 95995a49-db0c-4261-8776-b90780dc2a8c/600

* Use laneful policy to prevent planner aggression

* c8c3ab69-bd2c-4d1c-a7f8-9e8457e4827b/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* 7205c5af-7532-4215-ad9b-678789054db7/995

* bfb34fa3-1d72-4987-854f-43d42ee01015/950

* change cost

* 8ae2477a-b452-4601-b04c-d3af66ea5e98/950

* Update heading cost

* 9b991314-94f5-4f0b-872e-58d95266e4d0/333

* 9b991314-94f5-4f0b-872e-58d95266e4d0/950

* ede8f75a-df3e-41fb-bc56-63e2f33858e5/950

* undo lateral planner changes

* rm whitespace

* update ref

Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
old-commit-hash: eff08cb17f2b0d00d24cd13729b8d16ec394c82c
2022-05-12 19:47:33 -07:00
Adeeb Shihadeh 7a0bddab89 jenkins: prune lfs objects
old-commit-hash: affeb2ef73169282a8c450bdd89101c9860e535d
2022-05-12 19:38:58 -07:00
Shane Smiskol 28dfaa0abe process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
old-commit-hash: d686f9e527f235d513f856e70b60cc0f37cb5559
2022-05-12 19:19:47 -07:00
Shane Smiskol 5a0778518d process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
old-commit-hash: 934bc1e6fb2ab9b9476438b2f9aea45a4dcb5298
2022-05-12 17:05:22 -07:00
Shane Smiskol f1f447d092 Add desired curvature to controlsState (#24510)
* Add desired_curvature to controlsState

* clean up

* bump

* bump

* update refs from GitHub!

Co-authored-by: Jason Shuler <jshuler@gmail.com>
old-commit-hash: 6518ecca8ff75475c6f79aac1af660c1ccc74da9
2022-05-12 16:59:52 -07:00
Joost Wooning 6059fe131b debayering: update ref commit for debayer test
old-commit-hash: 7ff3e32c2b74860d73057dc6bc632f127b26dffa
2022-05-12 14:56:32 +02:00
grekiki 7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cdd9322918e9d5d388f7fa47aa30e9b7
2022-05-12 14:08:11 +02:00
Shane Smiskol 024f42bae5 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d2d3b7b823bac0d24e2136f7a97c3aa4c6d9ae47
2022-05-11 15:03:05 -07:00
HaraldSchafer 3427eb411b Latcontrol torque: fix integrator induced ping pong (#24458)
* Latcontrol torque: fix integrator induced ping pong

* Reset on disengage since unwind resets anywayh

* Might be overkill

* rm whitespace

* update ref
old-commit-hash: 29afd53d88f5c98efdac8727f3df1e4679790cde
2022-05-06 22:17:21 -07:00
Joost Wooning 9ffe2ceb0e debayering: fill outer image borders (#24441)
* debayering: fill outer image borders

* correct conditions

* try something else

* remove global check

* update ref

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 49e10dc773780884f2aab309b7dc2436fe6f473a
2022-05-06 18:27:41 +02:00
Willem Melching 7a855eff47 sanity check on debayer time in CI (#24453)
* check debayer time

* set actual threshold

* also print mean and max
old-commit-hash: 9ef17a877fa0cb4b0d9e7017c152dadbfb94a9aa
2022-05-06 17:27:54 +02:00
Willem Melching 1dab87e416 test_debayer.py some PC fixes (#24449)
* test_debayer.py some PC fixes

* fix rgb shape
old-commit-hash: 85975e3dd91421ac3fffd03f75428257ce42b5b8
2022-05-06 16:32:44 +02:00
Joost Wooning 8dea381a77 test for comparing debayer results to ref commit (#24420)
* test for comparing debayering results to ref commit

* fix run on device

* print more useful diff

* limit to diff output

* remove env var

* ref

* more useful diff

* use real local work size

* remove pyopencl install

* update ref

* include rgb_to_yuv

* fix diff

* get kernel build out of loop

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 9da9b773eba344e9e59aeaa8a2c0235aadbd1f44
2022-05-05 16:05:37 +02:00
ntegan1 af82274bac LatControlTorque: log desired/actual lateral accel (#24406)
* log torque lateral accel

* bump cereal

* update refs

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 9a44d4c4e0895f88dd5cd716ce3e2d6ebeab3457
2022-05-03 14:18:46 -07:00
Adeeb Shihadeh b9311fa3f4 body: log charging status (#24409)
* body: log charging status

* update refs
old-commit-hash: 986a991a09cc38375ded9f9809388a4384abf169
2022-05-03 13:27:19 -07:00
Willem Melching be08693d5c delay bz2 compression from logging to uploading (#24392)
* remove log_name

* log without compression

* fix tests

* remove extension for bootlog

* another test fix

* uploader compresses

* also compress in athena

* only compress qlog

* more generic check in do_upload

* fix bootlog compression

* lower loggerd cpu usage

* dont link against bz2

* set core affinity to little cluster

* handle old files
old-commit-hash: 77a6f3d034a7b17d70b750b9ea454310f4b8edf1
2022-05-03 14:09:17 +02:00
HaraldSchafer a5ed9358a5 Latcontrol torque: update tuning (#24357)
* Little more chill

* Update ref

* Update refs
old-commit-hash: 8cdeaf9aee158bf11869132f82863bec4489e4f9
2022-05-02 14:42:18 -07:00
Shane Smiskol 121b708ba1 Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
old-commit-hash: 4190d2608a536e1b28fb999a72bbf5b80810743d
2022-05-02 13:34:31 -07:00
Keyvan Fatehi 3faf33147e body: add fuelGauge to body carState (#24386)
old-commit-hash: 0bbfc63c9c11bab25767ef10cea5b70b520cba7f
2022-05-01 18:41:50 -07:00
Adeeb Shihadeh 247c153a35 controlsd: better alert precedence for system malfunctions (#24366)
* controlsd: better alert precedence for system malfunctions

* down to 20

* move that

* check can flags

* update refs

* update refs
old-commit-hash: be748c0d6a1a931a26b4ec741bbba71d4269b821
2022-04-29 13:36:25 -07:00
Willem Melching 2a5eae5a56 ui: combine OnroadHud into NvgWindow (#24369)
* ui: combine OnroadHUD and NVGWindow

* draw hud first

* cleanup

* removed commented line

* fix text rendering

* increase cpu usage
old-commit-hash: a3cfa444d37bacdac5278335ac7a49b6cb5b81f0
2022-04-29 10:55:44 -07:00
George Hotz 3a6a5485f2 camerad: less ioctls try 2 (#24371)
* premap the buffers

* memory manager

* free buffers properly, alignment seems okay

* update camerad CPU usage

* cam_sync_fd

* useless line, and use the define

* cheap prereqs for multistream

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: b7ddce8bacde6de2f708af0b70ea8a20d61396f2
2022-04-29 18:09:12 +02:00
George Hotz e7aac92b21 Revert "camerad: don't remap everything every time (#24334)" (#24370)
This reverts commit 30e9a8aab8337d6b5661f4acdb0f2cf8c1421504.
old-commit-hash: 5ddd2000b23ca3fc3c67a5a29331a28cffaa2a9e
2022-04-29 07:59:03 -07:00
Shane Smiskol 754b93997a Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
old-commit-hash: c1caca104f8553d485334e33446bd7b8ec1fb8ca
2022-04-28 22:37:46 -07:00
Shane Smiskol 32075d4681 safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
old-commit-hash: 66dd8f934cc84cdbd3e48e7b49836ec95f82c23a
2022-04-28 17:56:34 -07:00
George Hotz 0c2a527b3e camerad: don't remap everything every time (#24334)
* premap the buffers

* memory manager

* free buffers properly, alignment seems okay

* update camerad CPU usage

* cam_sync_fd

* useless line, and use the define

* cheap prereqs for multistream

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: fb7d84875bc993beaf75645a7e4b311704742f02
2022-04-28 11:32:32 -07:00
Adeeb Shihadeh 211fc8fc68 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
old-commit-hash: 3e098c9ba5fd8f6958f38dc500c357db34e83ffe
2022-04-27 12:08:11 -07:00
Shane Smiskol 3e9694d3f3 LatControlTorque: fix deadzone and missing steer saturated warning (#24294)
* fix steer saturated alert and deadzone

* fix and formatting

* update refs

* update refs

* not needed

* remove
old-commit-hash: e834b4d9bed65e672c25c6e9ba67d41f2a1c8329
2022-04-27 01:18:05 -07:00
Shane Smiskol 5469375c91 HKG: only allow engagement on user button press (#24140)
* use deque for prev_cruise_buttons

* stash

* disallow engagement if resume or decel isn't pressed

* fix buttons

* reduce chances of causing 30-frame fault

Fix clu11

drive down cancel times

* Revert "reduce chances of causing 30-frame fault"

This reverts commit 0f54c051bf5a22b1f584718ec46e78bc3a82a0dd.

* consider pause/resume button

* stash

* Revert "stash"

This reverts commit 551ca7be6c45729f0747abc81cd81894cd621c32.

* sadly some cars need 8 op frames (identical to 4 updates from CLU11)

* add main button

* 6 should be safe

* use max

* clean up

* change to 4 samples and process all messages received, like panda

* bump panda

* test: replay segments with known re-engagements on-device

* Revert "test: replay segments with known re-engagements on-device"

This reverts commit 9730c3c14f942b82b6ed5ef2e81b8ae0126f3006.

* need prev_cruise_buttons as we don't run every CAN message through carstate

* more generic

* extend

* Update selfdrive/car/interfaces.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* 3% faster at 1000000 loops

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: efa98d2a0694b9a33a2fda3bebfae50fb4ba33cc
2022-04-26 13:27:28 -07:00
Shane Smiskol b3e7498f27 onnxruntime: initialize CUDA provider before predictions (#24330)
* initialize CUDA runtime before predictions

* dmond pauses modeld execution...

* done in other PR
old-commit-hash: 60b7114024181f79819c9a8b14863a8f23fb0252
2022-04-26 11:49:32 -07:00
Shane Smiskol ed58f3edbd regen: send vipc_client_extra frames (#24331)
* send extra frames to fix regen

* for sanity

* not used

* minimal changes
old-commit-hash: a8e632bb98288c09a2292bb83da9d1fb7d5dff1f
2022-04-26 00:15:09 -07:00
Joost Wooning 31f9e8af0f UI: remove memcpy on frame receive (#24318)
* UI: remove frame memcpy

* fix freezing

* fix onroad test
old-commit-hash: f84fdc86bbd8223f22158abfa37e368fa18bedf6
2022-04-25 13:45:55 -07:00
Adeeb Shihadeh 8eb8cf95ec Revert "UI: remove RGB buffer usage (#24298)"
This reverts commit 2a13d8446cd19340bfb67f14470b7c35f643e8b8.

old-commit-hash: 13e1c0b3393617a47bd1f9acf20485a1993f83e9
2022-04-24 11:56:30 -07:00
Shane Smiskol 82e5b382f1 longActive: consider longitudinal control (#24295)
* consider longitudinal control

* update refs
old-commit-hash: f4d1703b424a9bc9864fb86e9feb705e8b947937
2022-04-22 20:32:36 -07:00
Adeeb Shihadeh 4fa266c035 tici downclock (#24088)
* tici downclock

* fix that

* gp ref

* lower that a bit

* that was a scam

* perf

* update power

* increase that

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: b88fb037aba5582a4c669fe972ddde17d0c4d9b6
2022-04-22 14:14:59 -07:00
Joost Wooning e0c973fe70 UI: remove RGB buffer usage (#24298)
* ui: use yuv buffer

* cleanup

* fix watch3

* replay always send yuv

* fix drawing of path

* fix onroad test

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: b9907e86b2dde54263d55d36346bb5609950ab99
2022-04-22 12:55:37 -07:00