Commit Graph

253 Commits

Author SHA1 Message Date
Adeeb Shihadeh 608c16007e Rewrite logcatd in Python (#36111)
* Add Python logcatd implementation

* lil more
2025-09-07 11:32:44 -07:00
Adeeb Shihadeh 275abc1eb5 Rewrite proclogd in Python (#36110)
* Rewrite proclogd in Python

* lil more

* lil more

* Update system/proclogd.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update system/proclogd.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update system/proclogd.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-07 11:13:39 -07:00
Maxime Desroches 8258257658 ci: modernize test onroad (#36059)
* start

* fix

* better

* more

* test

* Revert "test"

This reverts commit 17066ac123668cb7280cf85e3f21a3043b4785b0.

* remove
2025-08-26 15:34:34 -07:00
Maxime Desroches 63441c048c test_onroad: relax first fid assertion (#36032)
* fid

* test

* Revert "test"

This reverts commit 38e6635dd0b0b9fb9c08bcc3a74b9283207b0c2f.

* r

* Revert "r"

This reverts commit 4037a321f89af137a645345a0fffb73da6071c72.
2025-08-19 22:30:48 -07:00
Jimmy d7b0a5fa7e Record feedback with LKAS button (#35888)
* record feedback with LKAS button

* fix alert test

* slightly simplify feedbackd

* "Audio Feedback Saved" upon time expiration or early stop

* earlySend --> earlyStop

* userFlag --> userBookmark

* RecordAudioFeedback param/toggle

* add audioFeedback test

* simplify feedbackd

* send bookmark regardless of toggle, show feedback event with higher priority

* add userBookmark to selfdrived sm

* fix mispelled param name

* default off and move to main

* segmentNum --> blockNum, earlyStop --> lastBlock

* preserve audioFeedback

* get rid of lastBlock and just send bookmark saved at the end

* update raylib side

* update toggle description and add raylib toggle

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2025-08-05 13:41:41 -07:00
Maxime Desroches bc5336d805 params: auto decode based on type (#35794)
* type

* test

* more

* might as well use this

* one more

* live

* athena

* b

* also

* more

* now

* ah

* pigeon
2025-07-22 21:58:06 -07:00
Adeeb Shihadeh 2f80854644 sensord: rewrite in Python (#35353)
* py sensord

* fix up mmc

* temp

* port over accel

* lil more

* kinda works

* rm that

* gpiochip

* mostly there

* lil more

* lil more

* irq timestamps

* fix ts

* fix double deg2rad

* test passes

* fix up mypy

* rm one more

* exception

* lint:

* read in all events

* bump that

* get under budget:

* accel self test

* gyro self-test

* keep these readable

* give it more cores

* debug

* valid

* rewrite that

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-05-30 13:31:07 -07:00
Adeeb Shihadeh 67486ff92d bump tinygrad (#35208)
* bump tinygrad

* fix

* why is mac different?

* fix sim

* relax that
2025-05-13 16:59:35 -07:00
Kacper Rączy f237649a7a Reapply "Online lateral lag learning" (#34975)
* Online lateral lag learning (#34974)

This reverts commit b4cc9e68d1.

* pad to the best size for fft

* Fix static analysis

* Add typing

* Fix typing

* MAX_LAG

* Calculate cross correlation regardless if the points are valid

* Back to lagd

* Add lagd to process_config

* Lagd in test onroad

* Move lag estimator for lagd

* Remove duplicate entry from test_onroad

* Update process replay

* pre-fill the data

* Update cpu usage

* 25sec window

* Change the meaning of lateralDelayEstimate

* No newline

* Fix typing

* Prefill

* Update ref commit

* Add a unit test

* Fix static issues

* Time limit

* Or timeout

* Use mocker

* Update estimate every time

* empty test

* DT const

* enable RIVIAN again

* Update ref commit

* Update that again

* Improve the tests

* Fix static

* Add masking test

* Increase timeout

* Add liveDelay to selfdrived

* Add liveDelay to selfdrived in process_replay

* Fix block_avg restore after num_blocks

* regen most

* Update bolt

* Update ref commit

* Change the key name

* Add assert

* True weighted average
2025-04-09 22:31:07 +00:00
Kacper Rączy 049193b178 test_onroad: test liveParameters service timing (#34977)
Add live params to test onroad
2025-04-04 03:31:55 +00:00
Adeeb Shihadeh 2382f1cab0 Run DM model twice, prep for bigger models (#34894)
* twice

* swap

* adjust
2025-03-24 15:54:42 -07:00
Adeeb Shihadeh ffd54e41ee bump manager starting time after 519ea9d 2025-03-17 15:19:34 -07:00
Adeeb Shihadeh 3f6fd3da78 test_onroad threshold tuning 2025-03-11 16:03:00 -07:00
Adeeb Shihadeh 65bf2ec0ea test_onroad: add sanity check on ecode outputs (#34784)
* test_onroad: add sanity check on ecode outputs

* exact
2025-03-05 15:58:03 -08:00
Adeeb Shihadeh 821148150b Revert "make test_onroad faster (#34704) (#34783)
* Revert "make test_onroad faster (#34704)"

This reverts commit 470ec46830.

* try this

* add there

* cleanup

* try this

* lil more

* list

* classic

* don't skip test
2025-03-05 15:42:22 -08:00
Maxime Desroches 470ec46830 make test_onroad faster (#34704)
* revert that

* again

* port over

* clean

* round
2025-02-25 10:32:35 -08:00
Adeeb Shihadeh 952def7abf start logging services first (#34646)
* start logging services first

* fix test

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-20 16:53:07 -08:00
Adeeb Shihadeh d9a8fab3e6 test_onroad: run same checks on both EncodeIdx and CameraState (#34638)
* try this

* cleanup

* lil more

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-20 15:46:19 -08:00
Adeeb Shihadeh 52b66a1c0c Stricter cam sync tests (#34609)
* strict

* fix that

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-20 14:15:38 -08:00
Adeeb Shihadeh a30d3f7d72 test_onroad: cleanup with time series LogReader (#34613) 2025-02-17 21:26:21 -08:00
Adeeb Shihadeh 73266a016c test_onroad: always show model exec timings (#34606)
Co-authored-by: Comma Device <device@comma.ai>
2025-02-17 09:18:41 -08:00
Adeeb Shihadeh 958c8d1ce3 Beautiful camera timings (#34558)
* the start of something beautiful

* mv that

* cleanup

* cleanup output

* the good stuff

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-14 21:33:50 -08:00
Adeeb Shihadeh fbf918b311 test_onroad fixups (#34572)
* test_onroad fixpus

* fix
2025-02-11 20:15:30 -08:00
Dean Lee 9ec54f59c6 loggerd: switch logging from raw file format to zstd compressed files (#34549)
* switch logging from raw file format to zstd compressed files

* more zst suffix

* compress bootlog

* remove class RawFile

* Optimize ZstdFileWriter by adding input caching

* use ZSTD_compressStream2

* cleanup

* LOG_COMPRESSION_LEVEL=10

* space

* add zst suffix to LOGS_SIZE_RATE
2025-02-08 12:55:31 -08:00
Adeeb Shihadeh 9bc35c0919 Revert "athena upload: reduce memory usage and improve efficiency with streaming (#34528)"
This reverts commit 4c65f51a55.
2025-02-06 11:50:04 -08:00
Dean Lee 4c65f51a55 athena upload: reduce memory usage and improve efficiency with streaming (#34528)
* reduce memory usage and improve upload efficiency with streaming

* space
2025-02-04 13:10:13 -08:00
Maxime Desroches 89204b5464 test_onroad: adjust rlog size (#34336)
s
2025-01-07 19:45:53 -08:00
Harald Schäfer 17ca6389e1 Tinygrad runner (#34261)
* squash

* dmonitoringmodeld: use cl transform (#34235)

* needs cleanup

* only if tici

* bump tinygrad

* check width

* base modelframe

* .

* need to be args

* more cleanup

* no _frame in base

* tici only

* its DrivingModelFrame

* .6 is fair

---------

Co-authored-by: Comma Device <device@comma.ai>

* Update tinygrad

* tg upstream

* bump tg

* bump tg

* debug

* attr

* misc cleanup

* whitespace

* remove

* Add TODOs to make python proc for modelrunners

* whitespace

---------

Co-authored-by: ZwX1616 <zwx1616@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-12-18 11:58:59 -08:00
Harald Schäfer f6885dcbec Revert Tinygrad (#34243)
* Revert "dmonitoringmodeld: use cl transform (#34235)"

This reverts commit 684b0b9d4d.

* Revert "load model before calling convert_fp16_to_fp32"

This reverts commit 31606a7d15.

* Revert "bump tinygrad"

This reverts commit 44f58ff758.

* Revert "Tinygrad runner (#34171)"

This reverts commit 7b5a4fbb03.

* Allow init buffer

* typo
2024-12-14 12:15:36 -08:00
Harald Schäfer 7b5a4fbb03 Tinygrad runner (#34171)
* squash

* bump tg

* bump tg

* debump tinygrad

* bump tinygrad

* bump tg

* Skip init iteration

* fixes

* cleanups

* skip first test sample

* typos

* linter unhappy

* update cpu usage

* OPENCL just zeros for now

* imports

* Try printing

* Runs again, but slower

* unused import

* Allow more buffer with tg and all on gpu

* bump tinygrad

* seems ok

* stricter timings for driving looser for dm

* try llvm

* check nvidia

* More timeout for now

* make test pass

* Revert "try llvm"

This reverts commit ef136e478320101fea262bae3579e558da991902.

* small fixes

* whitespace

* revert test timeout

* No model runners

* Always CPU always fast

* No onnx runtime GPU

* more cores

* cleanup

* Is this faster

* Is this faster

* at least runs

* FP32 is faster than 16

* fix deps

* whitespace

* comment

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2024-12-11 23:15:20 -08:00
Maxime Desroches 00c10f6851 ci: adjust pandad cpu usage (#34077)
more
2024-11-21 11:32:06 -08:00
Maxime Desroches b206879e4d ci: more robust memory usage test in test_onroad (#34067)
metric
2024-11-19 21:07:22 -08:00
Maxime Desroches bf21e10d81 ci: move manager test_startup_time to test_onroad (#34062)
* get

* fix

* now

* try

* better sign

* better

* better

* clean

* space

* fix

* more

* msg
2024-11-19 14:27:15 -08:00
Adeeb Shihadeh d9d57e5d6f Revert ISP image processing + tinygrad bump (#34020)
* Revert "Replace ThneedModel with TinygradModel (#33532)"

This reverts commit da952e9b64.

* Revert "camerad: move E + D cams image pipelines to the IFE (#33959)"

This reverts commit f2a1cce42b.
2024-11-13 19:27:11 -08:00
Maxime Desroches 4948f910f0 ci: fix test_onroad (#34016)
fix
2024-11-13 12:02:17 -08:00
Mitchell Goff da952e9b64 Replace ThneedModel with TinygradModel (#33532)
* squash

* bump tg

* bump tg

* debump tinygrad

* bump tinygrad

* bump tg

* Skip init iteration

* fixes

* cleanups

* skip first test sample

* typos

* linter unhappy

* update cpu usage

* OPENCL just zeros for now

* imports

* Try printing

* Runs again, but slower

* unused import

* Allow more buffer with tg and all on gpu

* bump tinygrad

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
2024-11-11 13:32:21 -08:00
Maxime Desroches 700a5651bd ci: nicer output for test_onroad (#33961)
* viz

* viz

* fix

* show more

* fix

* failure

* cleanup
2024-11-07 21:45:51 -08:00
Maxime Desroches 6a2ae346ab ci: relax test_onroad cpu usage bounds (#33951)
bounds
2024-11-06 20:30:38 -08:00
Maxime Desroches 32c5254415 ci: faster test_onroad (#33906)
* 30s

* ruff

* remove

* timeout

* global

* value

* get value

* better

* format

* cleaner
2024-11-02 21:19:05 -07:00
Maxime Desroches fcbdd39aaf test_onroad: lower bound for soundd cpu usage (#33883)
lb
2024-10-28 17:25:48 -07:00
Shane Smiskol abdd3ad2e9 lower card CPU usage threshold
from removing capnp conversion!
2024-10-14 14:07:48 -07:00
Shane Smiskol 7556233cca Reapply "move car.capnp to opendbc (#33722)" (#33728)
* Reapply "move car.capnp to opendbc" (#33725)

This reverts commit 9d52a5b485.

* why can't i repro?!

* Revert "why can't i repro?!"

This reverts commit 0435d218f790faf7b7aaed27d05ab9ee67b087e6.

* does this cause card to try and read it?

* better place

* wtf

* Reapply "why can't i repro?!"

This reverts commit d24fd5a0abf454f47d5591e3b39039fdc4d0251c.

* also here
2024-10-03 22:47:03 -07:00
Shane Smiskol af774d894e Deprecate car onroadEvents (#33687)
* new OnroadEvent struct

* and migrate python

* more forgotten

* re-index new OnroadEvent struct

* fix!

* more missing

* migrate onroadEvents

migrate onroadEvents

* migrate dm events

* hacks to get proc replay to say succeeded

* Revert "hacks to get proc replay to say succeeded"

This reverts commit 0bb8803e5755d606ae9f09da5395d9f50678c7c7.

* update refs
2024-10-03 15:20:58 -07:00
Adeeb Shihadeh af609212f7 bump qcamera too 2024-10-02 10:10:11 -07:00
Adeeb Shihadeh c87f953396 test_onroad: bump up hevc size 2024-10-01 23:21:33 -07:00
Maxime Desroches d826b08369 ci: lower soundd CPU budget (#33684)
tr
2024-09-30 18:59:46 -07:00
Adeeb Shihadeh 3b6f1a0b39 test_onroad: update CPU usages after recent changes 2024-09-29 10:55:39 -07:00
ZwX1616 876f192112 exec DM model with gpu (#33609)
* half

old-commit-hash: 9f72eca003d4637ca7fe847ebaf925c694fc2e84

* optimed

old-commit-hash: 6e36e2a12e09275ec21d1590012a92b05ca52ff5

* thneed

old-commit-hash: 419a06c09c0c767d828bcd1e118bc575898c343a

* exec

old-commit-hash: 0059c27ec11b076a37f65d604ed135ea6541b1a6

* runner

old-commit-hash: 34232ada94450ce541eaef546197fa219810a891

* runs but

old-commit-hash: 3db37c00b6a64908293b4de8d8b56e80308cd8f2

* it is 01

old-commit-hash: a160d81eb1a7e77abbef959b44f602610f68f665

* np

old-commit-hash: c1caff6ba648cc2c0094c71b2ea074f01c3c2dc8

* module url

old-commit-hash: 6f4902c4d384263a53e2c1d14d93b5ff864b6a5f

* new

old-commit-hash: 779ae79b1bc3df6374fb6663ac8592e107a6e504

* ds fast

* is this work

* corcention

* real timing

* no reg

* interim gather

* 0e4a9c7b

* fa69be01, and halve

* list

* cleanup

* slighly faster

* setprotlt

* expected

* replay ref

* more powar

* reluctantly

* bump tg

* 8

* less

* less

* bump tg

* better than exp

* closer

* cc

* see diff

* commits

* was right

* to 32 cast

* remove dlc file

* support both

* dspExecutionTime -> gpuExecutionTime

* ignore

* time ref

* ref commit

* last

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-09-26 16:40:44 -07:00
ZwX1616 ffb6e11f96 Revert "exec DM model with gpu" (#33604)
Revert "exec DM model with gpu (#33397)"

This reverts commit f79aca8e1e.
2024-09-19 20:55:11 -07:00
ZwX1616 f79aca8e1e exec DM model with gpu (#33397)
* half

old-commit-hash: 9f72eca003d4637ca7fe847ebaf925c694fc2e84

* optimed

old-commit-hash: 6e36e2a12e09275ec21d1590012a92b05ca52ff5

* thneed

old-commit-hash: 419a06c09c0c767d828bcd1e118bc575898c343a

* exec

old-commit-hash: 0059c27ec11b076a37f65d604ed135ea6541b1a6

* runner

old-commit-hash: 34232ada94450ce541eaef546197fa219810a891

* runs but

old-commit-hash: 3db37c00b6a64908293b4de8d8b56e80308cd8f2

* it is 01

old-commit-hash: a160d81eb1a7e77abbef959b44f602610f68f665

* np

old-commit-hash: c1caff6ba648cc2c0094c71b2ea074f01c3c2dc8

* module url

old-commit-hash: 6f4902c4d384263a53e2c1d14d93b5ff864b6a5f

* new

old-commit-hash: 779ae79b1bc3df6374fb6663ac8592e107a6e504

* ds fast

* is this work

* corcention

* real timing

* no reg

* interim gather

* 0e4a9c7b

* fa69be01, and halve

* list

* cleanup

* slighly faster

* setprotlt

* expected

* replay ref

* more powar

* reluctantly

* bump tg

* 8

* less

* less

* bump tg

* better than exp

* closer

* cc

* see diff

* commits

* was right

* to 32 cast

* remove dlc file

* support both

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-09-19 17:42:44 -07:00