Commit Graph

160 Commits

Author SHA1 Message Date
Maxime Desroches 5772683432 ci: faster process replay (#35578)
* waste

* update

* again

* we love tesla

* again again
2025-06-19 11:00:41 -07:00
Shane Smiskol 8473774b65 process replay: add Tesla route (#35199)
* add tesla

* update refs

* run tesla on more procs

* update refs
2025-05-12 21:18:51 -07:00
Kacper Rączy f237649a7a Reapply "Online lateral lag learning" (#34975)
* Online lateral lag learning (#34974)

This reverts commit b4cc9e68d1.

* pad to the best size for fft

* Fix static analysis

* Add typing

* Fix typing

* MAX_LAG

* Calculate cross correlation regardless if the points are valid

* Back to lagd

* Add lagd to process_config

* Lagd in test onroad

* Move lag estimator for lagd

* Remove duplicate entry from test_onroad

* Update process replay

* pre-fill the data

* Update cpu usage

* 25sec window

* Change the meaning of lateralDelayEstimate

* No newline

* Fix typing

* Prefill

* Update ref commit

* Add a unit test

* Fix static issues

* Time limit

* Or timeout

* Use mocker

* Update estimate every time

* empty test

* DT const

* enable RIVIAN again

* Update ref commit

* Update that again

* Improve the tests

* Fix static

* Add masking test

* Increase timeout

* Add liveDelay to selfdrived

* Add liveDelay to selfdrived in process_replay

* Fix block_avg restore after num_blocks

* regen most

* Update bolt

* Update ref commit

* Change the key name

* Add assert

* True weighted average
2025-04-09 22:31:07 +00:00
Kacper Rączy dec42839e3 process_replay: remove GM (volt) (#35000)
Remove volt segment
2025-04-09 04:13:02 +00:00
Kacper Rączy 0b72ae3bc2 process_replay: fix GM and replace TOYOTA3 (#34996)
* Add volt and bolt param migration

* Remove mazda due to invalid sensor timings

* Fix the import

* Add mazda to excluded

* Bring back mazda, but add todo

* New segment for TOYOTA3

* Fix

* Update ref commit
2025-04-09 03:12:52 +00:00
Shane Smiskol 949aaaba2c regenerate problem segments 2025-03-03 19:34:49 -08:00
Shane Smiskol 5bde16fc0f Rivian in release (#34700)
* bump

* bump

* fix

* bump

* fix footnotes

* bump

* docs

* add rivian to proc replay

* update
2025-02-24 22:09:22 -08:00
Shane Smiskol 8e19185cee Bump submodules (#34616)
* bump

* now libsafety

* skip rivian for now
2025-02-18 13:48:28 -08:00
Shane Smiskol 01a6dccc86 use a tuple 2024-11-12 14:34:43 -08:00
Maxime Desroches 4170534c02 ci: faster process_replay (#33989)
* try

* fix

* try

* try

* try

* monitor

* no

* how fast

* more

* timeout

* remove

* ruff

* try

* less

* cons

* less

* cons

* revert

* more and less

* more?

* more?

* cons

* cons

* cons

* cons

* cons
2024-11-12 14:26:21 -08:00
Shane Smiskol 1e3f8bec46 Ford Q3: enable long by default (#33945)
* bump

* update docs

* release notes

ctrl

* op

* migrate for regen

* regen and update refs

* only ford
2024-11-06 16:21:44 -08:00
Shane Smiskol aed1eaede5 Deprecate carState.events (#33693)
* add lkas bools

* switch these two over

* bump

* deprecate low speed lockout

* add lowSpeedAlert bool

bump

* GM vehicle speed is now signed!

* reimagine

* rm

* do event

* bump

* STASH

* comment

* bump

* no out!

* format

* move almost everything to selfdrived

* add back CC_prev for cruise initialization

* ok

* errors are passed to radarState as well as freq check

* deprecate!

* use selfdrived for test models events

* we only want noEntry from car events, not system, have to check pedalPressed

* no more events

* regen with buttonEvents set properly

* update refs
2024-10-02 21:32:56 -07:00
Jason Young ba350aac4c merge Tesla without a replay route (#33691)
* bump opendbc

* no Tesla replay route for now

* bump opendbc to head
2024-10-01 19:48:19 -04:00
Adeeb Shihadeh e04455cbaa selfdrived: controlsd only does controls (#33485)
* selfdrived

* process replay

* lil more

* set the valids

* rename that
2024-09-06 17:16:32 -07:00
Kacper Rączy 012162e816 Deprecate liveLocationKalman (#33405)
* Remove usages of llk

* Deprecate it

* Add scenarios back

* Fix orientationNED valid

* Regenerate

* Increase tolerances and update segment

* Remove calibration
old-commit-hash: 282eae3c309c32a293f26ed82cb623afe0fe5ca7
2024-08-30 02:07:16 -07:00
Kacper Rączy 786aff27da azure_container: pass the filename explicitely when using get_url (#33407)
* Pass filename explicitely when using azure container

* Do not update segments in test_processes

* openpilotci_source_zst

* Fix static analysisc
old-commit-hash: b5958ebb600f503f585a1080236a0f26b4f5c6a0
2024-08-29 21:51:55 -07:00
Shane Smiskol 4f019b5f60 move selfdrive/car to opendbc (#32630)
* move most of /car

* move some car tests

move some car tests

* fix selfdrive/car/tests

* fix selfdrive/controls tests

* fix the rest of the selfdrive tests

* bump opendbc

* fix all tests

* few more non-test references

* remove opcar and move docs to car

fix these debugging scripts

fix docs

* bump opendbc and panda

forgot panda
old-commit-hash: e735a7f3796e208cdc1ecc651ace09e1daf6009a
2024-08-17 00:54:51 -07:00
Shane Smiskol f8ef09fcb2 rm tesla (#33300)
* rm tesla

* more rm

* ugh we should remove dynamic imports soon
old-commit-hash: b9dec5e3b51cf60f5544e93ca208b15d47b99ef0
2024-08-14 15:06:00 -07:00
Shane Smiskol 7ce33fbe2b process replay: use zst (#33138)
* test

* update refs to zst

* update
old-commit-hash: dfd387520e88fa68f0485b6ce8dcf5befa82bafa
2024-07-30 01:46:03 -07:00
Kacper Rączy 130d0d2e3a process_replay: regenerate locationd logs (#33134)
* Filter out non-relevant services in most_messages_valid check

* Set as comprehension

* Update segments

* Update ref commit

* New check for hyundai2
old-commit-hash: 6e185f4eea470fe56fc6a33de9e81779f70941ab
2024-07-29 17:12:05 -07:00
Shane Smiskol 539c828861 tools: keep re pattern file small (#33084)
remove bz2 from re file
old-commit-hash: e68bb26e14d4e85f61ab3f58ac9667ca9d6ea565
2024-07-25 22:05:10 -07:00
Shane Smiskol 19d9e7ac6b revert marking some files as unexecutable (#32613)
* this is not a test

* these are not tests!
old-commit-hash: c7b3ece37ef4445bcc18403b6a9e0217db4abd96
2024-06-03 17:19:27 -07:00
Adeeb Shihadeh 86b6103657 test_*.py files are no longer executable (#32610)
* test_*.py files are no longer executable

* cleanup empty lines

* update precommit

* fixme

* enable

* build
old-commit-hash: c59887c8109dd8d33ea2e7169995184fbe38cc3a
2024-06-03 15:48:56 -07:00
Kacper Rączy 5e70ea98d2 process_replay: most messages valid check (#32521)
* check_most_messages_valid impl

* Add to both regen and test_processes

* Refactor

* Bring back carOutput

* Use Counter

* Use get(k, 0)
old-commit-hash: e0d20d2cf33a4398426e7ca4886bf6231d1220ad
2024-05-23 15:30:21 -07:00
Shane Smiskol ee03d13529 card: process that abstracts car interface and CAN (#32380)
* format card

* standalone process

* no class member CS, there's no point

also can be confusing; what else could be using this?

* rename CoS

* Update selfdrive/controls/controlsd.py

* never works first time :D

* canRcvTimeout is bool

* hack

* add cpu

* see what testing closet comes up with

* first

* some clean up

* support passable CI, fix test models

* fix startup alert

* process replay changes

* test_fuzzy

* gate carOutput valid on carControl valid

* we should publish after we update carOutput

* controlsd was using actuatorsOutput from 2 frames ago for torque, not the most up to date

* check all checks for carControl in case controlsd dies

* log more timestamps

* more generic latency logger; needs some clean up

latency_logger.py was difficult to understand and modify

* card polls on can and carControl to get latest carControl possible

* temp try to send earlier

* add log

* remove latencylogger

* no mpld3!

* old loop

* detect first event

* normal send

* revert "card polls on can and carControl to get latest carControl possible"

how it was is best

* sheesh! update should be first

* first timestamp

* temp comment ( timestamp is slow :( )

* more final ordering, and make polling on/off test repeatable

* Received can

* new plot timestamps

* clean up

* no poll

* add controllers (draft)

* Revert "add controllers (draft)"

This reverts commit e2c3f01b2fadcff74347bac90c8a5cc1ef4e27b3.

* fix that

* conventions

* just use CS

* consider controlsd state machine in card: not fully done

* hmm it's just becoming controlsd

* rm debugging

* Revert "hmm it's just becoming controlsd"

This reverts commit 534a357ee95bec4ed070667186af55d59421bbc7.

* Revert "just use CS"

This reverts commit 9fa7406f30c86200f20457f7b9ff95e731201bf9.

* add vCruise

* migrate car state

* Revert "migrate car state"

This reverts commit 4ae86ca163c6920070f410f608f7644ab632850b.

* Revert "add vCruise"

This reverts commit af247a8da41c3626ada4231b98042da1a1ae4633.

* simple state machine in card (doesn't work as is)

* Revert "simple state machine in card (doesn't work as is)"

This reverts commit b4af8a9b0a2e17fdfc89d344c64678ef51305c24.

* poll carState without conflate

* bump

* remove state transition

* fix

* update refs

* ignore cumLagMs and don't ignore valid

* fix controls mismatch; controlsd used to set alt exp

* controlsd_config_callback not needed for card

* revert ref temp

* update refs

* no poll

* not builder!

* test fix

* need to migrate initialized

* CC will be a reader

* more as_reader!

* fix None

* init after publish like before - no real difference

* controlsd clean up

* remove redundant check and check passive for init

* stash

* flip

* migrate missing carOutput for controlsd

* Update ref_commit

* bump cereal

* comment

* no class params

* no class

* Revert "no class"

This reverts commit 5499b83c2dcb5462070626f8523e3aec6f4c209d.

* add todo

* regen and update refs

* fix

* update refs

* and fix that

* should be controlsstate

* remove controlsState migration

CoS.initialized isn't needed yet

* fix

* flip!

* bump

* fix that

* update refs

* fix

* if canValid goes false, controlsd would still send

* bump

* rm diff

* need to be very careful with initializing

* update refs
old-commit-hash: 71f5c441fe32184d94a9f26565a36c661e2ccf28
2024-05-21 01:18:10 -07:00
Shane Smiskol b3fc3f25ce process replay: regen routes (#32464)
* regen

* regen

* update

* undo

* update refs

* fix

* real fix

* test

* test2

* test3

* stash

* Revert "stash"

This reverts commit bf6765f526b48426f49a4b29c4042097f57fc4b0.

* rk.lagging can not be trusted. BODY segment has radarFault while modelV2 is not valid, causing radarState to not be valid

* order

* update refs again

* we never logged carOutput!

* bump

* add back checks
old-commit-hash: 220fcc16753ce87c8997275733f62bc385342a7e
2024-05-19 01:07:12 -07:00
Shane Smiskol 681f5c6a34 process replay: check missing services (#32452)
* add check to check

* don't forget to raise

* skip this segment
old-commit-hash: ac81467c5c2c7ca0f340ee8d2aff81e330eb803a
2024-05-16 21:11:55 -07:00
Justin Newberry b899592b18 cars: platform enums -> prepend brand name (#31927)
* with brand name

* migrate

* Fix

* fixes

* more

* passes

* fix

* fix the doc

* collects

* these too

* more stuff

* body exception :/

* more

* hardcode i guess

* update ref

* toyota

* more toyota

* and here

* final!

* fix notebooks and ccs

* move this here
old-commit-hash: b0eb3ba4f477142797624580eb3fdb6448f068d6
2024-03-19 17:29:50 -07:00
Justin Newberry a5bc36ea9d move git commands to common/git.py (#31921)
move git into common/git.py
old-commit-hash: 4fbc8a389662f82018eeabd03294cc761b87ce75
2024-03-19 12:39:09 -07:00
thenhnn 77d6d1a8f1 PlatformConfig: automatically get platform_str from the enum name (#31868)
* get platform_str from the enum name

* fix tests

* add migration table

* remove impossible todo

* Add link to PR in MIGRATION table

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Remove useless brand name comments and rename RAM_1500 to RAM_1500_5TH_GEN

* rename RAM_HD to RAM_HD_5TH_GEN

* rename references to RAM_HD and RAM_1500

* change "mock" to "MOCK" and rename torque data of Nissan Leaf 2018 IC

* remove MOCK from fingerprints.py

* change hard-coded car model in test_can_fingerprint.py/test_timing

* migration

* update ref

* space

* prius

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: justin newberry <justin@comma.ai>
Co-authored-by: Justin Newberry <jnewberry0502@gmail.com>
old-commit-hash: 489528dcae3888311f92075ba9e0f68626725121
2024-03-19 11:16:02 -07:00
Cameron Clough 220ac6c422 test_processes: fix unclosed file (#31644)
old-commit-hash: 251eee46644c7975d732cb557af419d211532895
2024-02-29 11:17:13 -05:00
Cameron Clough 9bd90112d0 use pyupgrade to update to new typing syntax (#31580)
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae4967943ee33a3699a1d89a7a770d95e9
2024-02-24 16:41:23 -08:00
Adeeb Shihadeh b4c03185d4 bump cereal (#31392)
* bump cereal

* update those

* update refs

* bump cereal

* bump

* bump cereal

* bump

* fix

* bump

* typo:
old-commit-hash: daceb171bde5aef4ea483e8054456187772afe92
2024-02-09 21:44:23 -08:00
Greg Hogan 14e12980f5 simplify git remote is comma check (#31284)
* simplify git remote is comma check

* cast to str

* eliminate default and always return string

* add type annotation for cache decorator

* fix up default handling
old-commit-hash: 7affba06d89b366d98f42c4ea3a2f904e6ef3357
2024-02-02 21:23:32 -08:00
Justin Newberry 4b7edec2b0 LogReader: move opci to tools/lib (#31021)
move opci
old-commit-hash: 7d400112df6692ef026245b03951345cc36dd83e
2024-01-16 10:58:51 -08:00
Justin Newberry 059aa6ffd9 Revert "Process Replay: move to pytest (#30260)" (#30687)
This reverts commit eb8f3f7f568aa0e294b32226ae5c5ed5e1b75e9d.
old-commit-hash: 7b5d4d031adb0e2fecf36772bbbb95e2989b9cc7
2023-12-11 14:46:56 -08:00
Shane Smiskol f5d1c4f371 ruff: check newline (#30668)
* check newline

* nothing catches this
old-commit-hash: 97da129e11e58b07d5d136093827cd1c62311fea
2023-12-09 13:24:18 -08:00
Justin Newberry 337b156518 Pytest: load by group by default (#30466)
* wip

* use a class marker

* this one should be grouped too
old-commit-hash: 31c682d40743f4b102a70d72e6993b383d85a040
2023-11-14 17:33:47 -08:00
Justin Newberry 6af8e72604 Process Replay: move to pytest (#30260)
* process replay pytest

* enable long diff

* readd job name

* make it executable

* cleanup imports

* retrigger
old-commit-hash: 90c873ab1def0756f6f91fcb7f2d6fad503e3f49
2023-11-13 13:50:23 -08:00
YassineYousfi 91751770a8 (New) Lemon Pie Model 🍋 (#30209)
* 6f6e3749-1b7c-42e8-a33b-03929b7fc476/700

* oops deleted too much

* 1b4308b7-a659-4ebd-b4c6-c81c1c3890f8/700

* 1be192f3-f407-4217-9757-78b9ad92750a/700

* remove some todos

* more cleanup in lat planner

* vego > min_speed

* regen and update process replay refs

* update model replay ref

* update model replay ref commit again

* Revert "update model replay ref commit again"

This reverts commit 922cb796b8dfe264b0cce7a18206bb898b18bcb3.

* update again

* bump cereal
old-commit-hash: 4c2bd853e4410db7f0d80e258291408e94c6c068
2023-11-02 14:01:00 -07:00
Kacper Rączy 82009093f5 regen: update refs with dummy dcamera (#30347)
Update refs
old-commit-hash: e5eea7276f82f0e8467c43e52545ca06701526fa
2023-10-30 15:33:17 -07:00
Justin Newberry f2dcaf6f79 compare_logs: better names for short/long diffs (#30263)
* better names

* dont need that

* those too
old-commit-hash: def2b938950def9f4bd6e44d8f16a172983428a6
2023-10-16 17:04:14 -07:00
Justin Newberry f4365f014e test_processes: assert that mazda is tested (#30258)
mazda left dashcam mode
old-commit-hash: d23f5d2ec544f2bfec99586a897a7bf8bc766260
2023-10-16 12:18:25 -07:00
Shane Smiskol 338987a5d2 scripts: format diff when comparing logs (#30174)
* move format diff to compare_logs to pretty print

* smaller
old-commit-hash: a1ca5dc484132122b8b9ba7e2ff73da7c86bf1a2
2023-10-04 22:59:03 -07:00
Lee Jong Mun 89e5ee45ca cleanup remnants of LQR and INDI tuning (#29860)
* not use 'indi' cleanup

* bump cereal

* rm more

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 238f9d65dbc17613c2a045b5741483c3f9a7659d
2023-09-10 16:30:13 -07:00
Adeeb Shihadeh 0d4beff6a0 process replay: parallel by default (#29633)
* process replay: parallel by default

* more is helpful
old-commit-hash: 4d6bb32df250a65ef92ad82dce812b91a73896c4
2023-08-25 14:41:43 -07:00
Adeeb Shihadeh 16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b69af19306143fc4def02fb5769405d2
2023-08-20 20:49:55 -07:00
Harald Schäfer 92d1e3fabb Proc replay: field tolerance unused (#29479)
field tolerance unused
old-commit-hash: 1dea20e57485a79ff4e1515d892e589ed4a4c65e
2023-08-20 13:05:18 -07:00
Justin Newberry 3ee0786de3 CI: Pylint to ruff (#29294)
* pylint to riff

* pylint to riff

* pylint to riff

* revert more

* undo exclude removal

* exclude tinygrad

* set line length

* pylint exclusions to ruff

* same excludes as old linter

* fix tools QA

* remove unrequired check

* revert linting third_party

* ignore e402
old-commit-hash: 7fdd9fc37c1c5febf1cd3dc86441ebf0dec7f370
2023-08-09 13:26:54 -07:00
Justin Newberry ff96892511 Revert "CI: pylint to ruff (#29276)" (#29284)
This reverts commit ffad7279cbad3c4f2ea786287b4624d3a3aa18d7.
old-commit-hash: a91764954b0c9bfe5ccfc8e64321af6bb6c56536
2023-08-08 14:24:04 -07:00