Commit Graph

194 Commits

Author SHA1 Message Date
Adeeb Shihadeh cc4f53e6c6 manager: move to system/ (#32538)
* manager: move to system/

* one more
old-commit-hash: accdade4cfe73cdca41f5afa73cf4ad085802930
2024-05-25 12:41:17 -07:00
Kacper Rączy 5e70ea98d2 process_replay: most messages valid check (#32521)
* check_most_messages_valid impl

* Add to both regen and test_processes

* Refactor

* Bring back carOutput

* Use Counter

* Use get(k, 0)
old-commit-hash: e0d20d2cf33a4398426e7ca4886bf6231d1220ad
2024-05-23 15:30:21 -07:00
ZwX1616 bd0cabd88d Update process replay refs with updated submaster alive behavior (#32508)
* add cereal

* update cereal

* cereal master

* lol really

* ref com

* ref
old-commit-hash: 749455c91dc27ed876d1b194c1d84bd481eefb00
2024-05-22 15:47:11 -07:00
Shane Smiskol ee03d13529 card: process that abstracts car interface and CAN (#32380)
* format card

* standalone process

* no class member CS, there's no point

also can be confusing; what else could be using this?

* rename CoS

* Update selfdrive/controls/controlsd.py

* never works first time :D

* canRcvTimeout is bool

* hack

* add cpu

* see what testing closet comes up with

* first

* some clean up

* support passable CI, fix test models

* fix startup alert

* process replay changes

* test_fuzzy

* gate carOutput valid on carControl valid

* we should publish after we update carOutput

* controlsd was using actuatorsOutput from 2 frames ago for torque, not the most up to date

* check all checks for carControl in case controlsd dies

* log more timestamps

* more generic latency logger; needs some clean up

latency_logger.py was difficult to understand and modify

* card polls on can and carControl to get latest carControl possible

* temp try to send earlier

* add log

* remove latencylogger

* no mpld3!

* old loop

* detect first event

* normal send

* revert "card polls on can and carControl to get latest carControl possible"

how it was is best

* sheesh! update should be first

* first timestamp

* temp comment ( timestamp is slow :( )

* more final ordering, and make polling on/off test repeatable

* Received can

* new plot timestamps

* clean up

* no poll

* add controllers (draft)

* Revert "add controllers (draft)"

This reverts commit e2c3f01b2fadcff74347bac90c8a5cc1ef4e27b3.

* fix that

* conventions

* just use CS

* consider controlsd state machine in card: not fully done

* hmm it's just becoming controlsd

* rm debugging

* Revert "hmm it's just becoming controlsd"

This reverts commit 534a357ee95bec4ed070667186af55d59421bbc7.

* Revert "just use CS"

This reverts commit 9fa7406f30c86200f20457f7b9ff95e731201bf9.

* add vCruise

* migrate car state

* Revert "migrate car state"

This reverts commit 4ae86ca163c6920070f410f608f7644ab632850b.

* Revert "add vCruise"

This reverts commit af247a8da41c3626ada4231b98042da1a1ae4633.

* simple state machine in card (doesn't work as is)

* Revert "simple state machine in card (doesn't work as is)"

This reverts commit b4af8a9b0a2e17fdfc89d344c64678ef51305c24.

* poll carState without conflate

* bump

* remove state transition

* fix

* update refs

* ignore cumLagMs and don't ignore valid

* fix controls mismatch; controlsd used to set alt exp

* controlsd_config_callback not needed for card

* revert ref temp

* update refs

* no poll

* not builder!

* test fix

* need to migrate initialized

* CC will be a reader

* more as_reader!

* fix None

* init after publish like before - no real difference

* controlsd clean up

* remove redundant check and check passive for init

* stash

* flip

* migrate missing carOutput for controlsd

* Update ref_commit

* bump cereal

* comment

* no class params

* no class

* Revert "no class"

This reverts commit 5499b83c2dcb5462070626f8523e3aec6f4c209d.

* add todo

* regen and update refs

* fix

* update refs

* and fix that

* should be controlsstate

* remove controlsState migration

CoS.initialized isn't needed yet

* fix

* flip!

* bump

* fix that

* update refs

* fix

* if canValid goes false, controlsd would still send

* bump

* rm diff

* need to be very careful with initializing

* update refs
old-commit-hash: 71f5c441fe32184d94a9f26565a36c661e2ccf28
2024-05-21 01:18:10 -07:00
Shane Smiskol b3fc3f25ce process replay: regen routes (#32464)
* regen

* regen

* update

* undo

* update refs

* fix

* real fix

* test

* test2

* test3

* stash

* Revert "stash"

This reverts commit bf6765f526b48426f49a4b29c4042097f57fc4b0.

* rk.lagging can not be trusted. BODY segment has radarFault while modelV2 is not valid, causing radarState to not be valid

* order

* update refs again

* we never logged carOutput!

* bump

* add back checks
old-commit-hash: 220fcc16753ce87c8997275733f62bc385342a7e
2024-05-19 01:07:12 -07:00
Adeeb Shihadeh f60cbbb04b modeld: prep for camera transform refactor (#31820)
* modeld: prep for camera transform refactor

* update refs

* add pub

* do setup
old-commit-hash: 7436aa8b053aeccfc9bacb06bfaf4a42b3ec5f78
2024-03-10 16:39:40 -07:00
Harald Schäfer 2f0d283183 Wheeled body (#31614)
* Wheeled body

* 100hz only for balance

* No carparams in locationd no more

* Update ref
old-commit-hash: c05b37979d49c0a586b56db9294aa061c2c079e2
2024-02-27 13:23:04 -08:00
Cameron Clough a09300385b mypy: use implicit-optional (#31590)
* mypy: set implicit-optional = true

* find and replace '| None = None' -> '= None' in function args
old-commit-hash: 80da3aee147867e46f566b63a96f4fe199e0628d
2024-02-25 13:29:18 -08:00
Cameron Clough 9bd90112d0 use pyupgrade to update to new typing syntax (#31580)
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae4967943ee33a3699a1d89a7a770d95e9
2024-02-24 16:41:23 -08:00
Adeeb Shihadeh 66adafe850 radard: remove sleep for cars without radar (#31467)
* radard: remove sleep for cars without radar

* fix

* update refs
old-commit-hash: 6f905ed979cb8e217c182c653528df3a6094015b
2024-02-14 13:12:54 -08:00
Harald Schäfer 66dbadb029 Delete lat planner (#31089)
* Initial commit

* Fixup

* typo

* ignore lateral plan

* Update cereal

* Remove lateralPlan

* Fix release build

* Fix release build

* give car params

* Add carParams to include_all_types

* Write car param in powerdraw test

* add demo mode

* Update model regf

* proc replay ref commit

* Try

* Move enum definition

* Update cereal

* typo

* Write car param for modeld test

* Update ref

* Update model ref again

---------

Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
old-commit-hash: e6c97c384671b448f307a7ed91416886f2186d80
2024-01-21 12:09:48 -08:00
Adeeb Shihadeh 972650f0eb remove dashcam branch (#31081)
* remove dashcam branch

* fix

* more cleanup

* remove param

* one more
old-commit-hash: 7cd9568423a2e78187d16018b70a78f56c02027d
2024-01-19 14:05:03 -08:00
Kacper Rączy 214b38bc69 controlsd: require joystick mode param for the body (#30680)
* Update joystick mode in step

* Stop putting the body in joystick mode by default

* Toggle JoystickDebugMode by default for notCars in process_replay

* Remove subsection about web joystick

* Update param only in non-cars

* Make joystickDebug non-static event, and add it in a loop

* Add comment
old-commit-hash: e62756596b82e4e40d02a30b34d3f3c9960bffe6
2023-12-12 14:14:51 -08:00
Adeeb Shihadeh b761d0efb7 process replay: check valid flag (#30588)
old-commit-hash: a2b48efa2083c86638b536df180fbb638cab7404
2023-12-02 10:47:08 -08:00
Adeeb Shihadeh f38648f84b rename carEvents -> onroadEvents (#30577)
* rename carEvents -> onroadEvents

* bump cereal

* bump cereal
old-commit-hash: b97e5b0e03a5e22ef0ae6d6f5c2523592ee25540
2023-12-01 22:48:29 -08:00
Vivek Aithal f8dce94688 params: Remove separate CarParams from each daemon (#30405)
* remove separate previous route carparams from each daemon and add centrally to controlsd

* extract out sigint handler

* make process replay work for torqued

* don't write param if None
old-commit-hash: 2eb487c9a5a905b502433db2e7c5e52a52224f39
2023-11-08 15:25:41 -08:00
Kacper Rączy 1027a60851 process_replay: use frame dimensions from FrameReader (#30332)
Use dimensions from FrameReader to setup vipc server
old-commit-hash: 4094d584f3b7788d50eb90110b6f3d7651c96863
2023-10-25 15:46:16 -07:00
Harald Schäfer 7c4d29c3d1 Rm laikad (#30299)
* rm laika

* Rm laika

* Needed this

* More rm

* More rm
old-commit-hash: 56dea7b0b42e7624a39fef0986cd0db151bb9512
2023-10-23 00:15:39 -07:00
Justin Newberry ef1acd4e6c Proc Replay: cleanup iterable log typing (#30261)
* cleanup log typing

* use iterable

* fix those

* Cleanup

* cleanup

* missed that one
old-commit-hash: 1820b5f25930ae68e760157d6ff026516e3811ed
2023-10-17 12:58:28 -07:00
Kacper Rączy c33f68047b process_replay: migrate panda/peripheral/manager states (#30141)
* Migrate pandaStates

* Fix import

* Make pandaStates migration optional, so process_replay still works on logs without carParams

* peripheralState migration if its not available

* Do nothing if log file has peripheralStates

* Add migrate_managerState

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 898acb13f27d69291e8b7ecfa8a6ea577b330520
2023-10-04 13:41:50 -07:00
Kacper Rączy 71ad6ce228 process_replay: fix support for single cam segments (#30143)
* Fix assertion for vision_pubs

* Get camera states from meta

* Convert deviceType to string

* Use neo instead of eon for frame sizes dict

* update vision_pubs of process config when creating vipc
old-commit-hash: 9c5bbe1a91cc56df71d6f71a37b801aee24d073b
2023-10-02 19:36:47 -07:00
Adeeb Shihadeh 9c8de3e08b simplify sensor alert (#29824)
* simplify

* update proc replay
old-commit-hash: a4aa801fbe924ace2ebd493eeac7eb0c596f81b5
2023-09-26 16:47:21 -07:00
Shane Smiskol 13381ed446 services: capitalize constant service list (#30005)
* rename

* bump

* Apply suggestions from code review
old-commit-hash: 7f073483bb2586053d6c5508ef3f420947941d90
2023-09-21 18:25:15 -07:00
Adeeb Shihadeh 53e0f5eb49 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7b6afbc1623b8c159a92f6a5a951fb4d688045ee
2023-09-05 18:52:40 -07:00
Kacper Rączy a40c0c9b6e process_replay: clean env vars after run (#29694)
Clean environ vars at the end of replay
old-commit-hash: bb14ba4e5e5bd229bde42a14d1c9a7bbc02297b5
2023-08-28 19:44:40 -07:00
Harald Schäfer 3a28e97a95 Revert: Locationd: enable laikad (#29687)
* Revert: Locationd: enable laikad

* update ref
old-commit-hash: 55b391509d94c45000df6754f173ad446dc1b34c
2023-08-28 16:27:11 -07:00
Harald Schäfer bc017ab936 Locationd: enable laikad (#29570)
* experiment

* llk back

* update refs

* update ref

* no internet

* Update ref
old-commit-hash: 8205590624913b927d7abb1d214ea25886d7c1ea
2023-08-24 16:51:45 -07:00
Harald Schäfer a967592f8f Revert laikad llk (#29535)
* Revert "bump laika (#29522)"

This reverts commit 19ab859e302d52b4f1667f05543ac622ad259e57.

* Revert "Locationd: switch to laikad (#29278)"

This reverts commit 5dda7be8ac08ef6b51283eda74569517bba905f6.
old-commit-hash: fda3106c28af236e8b3dc78bfdca7a1365904d1e
2023-08-22 14:12:51 -07:00
Harald Schäfer 0174c5f21c Locationd: switch to laikad (#29278)
* use_laika

* Update test

* fix spacing

* fix proc replay

* small fix

* Try correct std

* cast correctly

* experimental

* simpler

* update reset thresholds

* improve

* Ready for live running

* outlier reject

* update laika

* add fine speeds

* more experiments

* fixes

* all in laika

* more small tweaks

* outlier rejection not needed

* fix conflict

* no more gpslocation

* update test

* refactor to laika

* runs again

* Update ref
old-commit-hash: fd3b8a3d7511ad3d7c5aa0b82a1e2c28505e5a7f
2023-08-21 22:51:02 -07:00
Adeeb Shihadeh 16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b69af19306143fc4def02fb5769405d2
2023-08-20 20:49:55 -07:00
Harald Schäfer 92d1e3fabb Proc replay: field tolerance unused (#29479)
field tolerance unused
old-commit-hash: 1dea20e57485a79ff4e1515d892e589ed4a4c65e
2023-08-20 13:05:18 -07:00
Shane Smiskol 1a2818d775 Revert "radard: tie radard frequency to modelV2 vol. 2 (#29240)" (#29474)
* Revert "static analysis fixes"

This reverts commit 2271d0adf9a02aee1359717c47cb6280893a6b9b.

* Revert "radard: tie radard frequency to modelV2 vol. 2 (#29240)"

This reverts commit 6757e90415766e84ad1b6c921a2edb29ceb3bbbe.

* update refs
old-commit-hash: 1a08c2986e5600dd4e84bfeaed26c74e679085de
2023-08-18 22:56:43 -07:00
Kacper Rączy deb8257322 process_replay: prevent creating orphaned processes if replay fails during setup (#29426)
Fix the ability to cleanup, if process container fails during start
old-commit-hash: f07e4fb49a20b915e887f6a834ff3bc3636aa9cf
2023-08-16 10:37:19 -07:00
Kacper Rączy 4017b1707c radard: tie radard frequency to modelV2 vol. 2 (#29240)
* radard: tie radard frequency to modelV2

Accumulate parsed messages until state is updated (toyota)

Same for honda

Rename rr to something more descriptive

* Change _update method name, since signature has changed

* Update ref commit

* Some renames

* Check for number of cans
old-commit-hash: 460f5c8e0b2fd2c420e211b09b70d1d79834a929
2023-08-11 18:07:06 -07:00
Justin Newberry 715f028a0e Ruff: enable flake8-builtins (#29315)
* enable flake8-builtins

* replace any with contains

* fix typo in pack

* fix type

* format is from the parent module, has to be enabled

* item_id

* fix item_id

* disable for id since that's what the remote server returns
old-commit-hash: 8793cbff40662a92bff16d75b51479c80517305a
2023-08-10 18:58:22 -07:00
Justin Newberry a14f162bc7 Ruff: comprehensions conventions (#29317)
ignore c408
old-commit-hash: 4a9afd7554060ac5353e96f1d7feb06d3a3671e9
2023-08-10 16:40:30 -07:00
Justin Newberry 3ee0786de3 CI: Pylint to ruff (#29294)
* pylint to riff

* pylint to riff

* pylint to riff

* revert more

* undo exclude removal

* exclude tinygrad

* set line length

* pylint exclusions to ruff

* same excludes as old linter

* fix tools QA

* remove unrequired check

* revert linting third_party

* ignore e402
old-commit-hash: 7fdd9fc37c1c5febf1cd3dc86441ebf0dec7f370
2023-08-09 13:26:54 -07:00
Justin Newberry 7b677910df Linter: Fix >160 line length test (#29287)
* test

* fix line length

* fix line length

* fix file size

* revert poetry

* cleanup for PR
old-commit-hash: cf64b27168614f851087adab2d7f766425922f64
2023-08-08 17:13:35 -07:00
Justin Newberry ff96892511 Revert "CI: pylint to ruff (#29276)" (#29284)
This reverts commit ffad7279cbad3c4f2ea786287b4624d3a3aa18d7.
old-commit-hash: a91764954b0c9bfe5ccfc8e64321af6bb6c56536
2023-08-08 14:24:04 -07:00
Justin Newberry 99006365a3 CI: pylint to ruff (#29276)
* pylint to riff

* pylint to riff

* pylint to riff

* revert more

* undo exclude removal

* exclude tinygrad
old-commit-hash: cb31a53d8a93cf6cb1f8bd2fff1431716b3a47e4
2023-08-08 13:41:30 -07:00
Kacper Rączy c92e7265a6 Revert "radard: tie radard frequency to modelV2 (#29063)" (#29243)
This reverts commit 4b221cb4fd271664ab7d39c5cfefb4225ca07c2a.
old-commit-hash: 1774b4fc61c8f92fae16545a68854238660737e6
2023-08-04 20:25:40 -07:00
Kacper Rączy 79fa322d4a radard: tie radard frequency to modelV2 (#29063)
old-commit-hash: 255f5c13f2d8c6796fd960b248856f11a3c1b05a
2023-08-03 22:10:21 -07:00
Kacper Rączy 67ac96c8b4 process_replay: capture process output (#29027)
* Add ProcessOutputProxy

* Move launcher to its own field

* Move ProcessOutputCapture to its own file

* Return itself from __enter__ of OpenpilotPrefix

* Integrate ProcessOutputCapture into process_replay

* Add note about capture_output_store to README

* ipykernel import is optional

* Disable type checking for link_with_current_proc

* Remove assertion

* Decode outputs to utf-8

* read(self): return empty buf if its none

* Fix type annotations

* Replace fifo with regular file, to avoid hitting fifo size limit
old-commit-hash: 547a033a3c32f05f2dd678280cee4d7378b60b83
2023-07-31 16:30:58 -07:00
Kacper Rączy d50f057f02 process_replay: ability to permanently unlock some sockets (#29070)
Add unlocked_pubs for keeping some sockets permanently unlocked
old-commit-hash: 81e8cf414e25650ef1e01b3d45c2440ece0e4923
2023-07-20 16:03:58 -07:00
Kacper Rączy 4c6efa638c process_replay: support for replaying multiple processes simultaneously (#28829)
* first working ProcessContainer and _replay_multi_process implementation

* Setup controlsd config_callback

* Add vision ipc support

* Remove old code

* add todo

* Update estimated processing times

* Use deepcopy when getting configs

* Rework the API to support passing multiple names or configs

* Add get_custom_params_from_lr

* Add more typing

* Enable simulation for controlsd

* Fix typing issues

* Regen refactor

* Fix linter issues

* Make wideRoadCameraState optional

* Move DummySocket to helpers

* Improve typing in regen

* Minor improvements

* Maintain sort order of internal_pub_queue using heapq

* Move ProcessContainer elsewhere

* Fix internal heap

* Change comment

* Remove subtest_name from ProcessConfig

* Update cereal

* Add newline
old-commit-hash: b2d0f74ec3cc48faa6b54c26267d53ac554e140f
2023-07-18 14:56:24 -07:00
Kacper Rączy fe84dadaa5 controlsd: support for SIMULATION mode for process replay (#28895)
* Support for simulation flag while replay

* Update cereal

* Enable simulation for controlsd

* Update ref commit

* Update cereal
old-commit-hash: 72579a0e1081ecec5c414250831bedcecdc3bdc4
2023-07-12 10:32:02 -07:00
Kacper Rączy 1fa42d0b97 process_replay: modeld and dmonitoringmodeld support (#28463)
* vipc support. modeld and dmonitoringmodeld configs

* Update cereal

* Add vision meta

* Use vision meta in process replay

* Exclude modeld and dmonitoringmodeld from test_processes

* Refactor other pieces of code to use vision meta

* remove unnecessary imports

* Add MAX_FRAMES_PER_SEGMENT

* cameraState migration

* Move camera state migration to migration module

* model_replay refactor

* Fix lint issues

* Reduce number of vipc buffers to 2

* Update spinner message

* Update model replay ref

* Support the new log order

* Replace CameraFrameSizes with dict. Add encode_index field

* Migrate camera states based on encode indices

* Refactor process replay

* Refactor model replay

* Remove unnecessary print

* Update model replay ref

* Make camera state migration optional

* Remove unused import

* Add ignore_alive_pubs field

* Add assertion for carParams inside fingerprinting callback

* Remove WideCameraOnly param setup

* Refactor ModeldCameraSyncRcvCallback

* Update model replay ref commit

* Fix camera state migration

* Only migrate camera states when process has vision pubs

* Refactor model replay again

* Fix linter issues

* One more linter fix

* Update model replay ref commit
old-commit-hash: 2c617e0c7a082aeb3217b14e812e5d3c0c26cd4a
2023-06-22 17:36:25 -07:00
Kacper Rączy 0b2580b20a process_replay: custom params (#28624)
* Add custom params argument

* Fix default arguments

* Fix typing issues
old-commit-hash: 3cae32a85fb534020a3f84fdb257edff91c12ac8
2023-06-21 20:16:14 -07:00
Dean Lee eddc84d104 automatically detect available camera streams (#27640)
* remove WideCameraOnly

* check in set_initial_state

* no block

* remove try block

* apply reviews
old-commit-hash: 4ae0378f639826df5cc1518574842b61b118f371
2023-06-12 13:40:43 -07:00
Kacper Rączy 0541586636 process_replay: deprecated message migration (#28456)
* Move migration code to new file

* Migrate messages in process replay. Remove old code

* Fix old_logtime

* Keep old logtimes on migrated messages

* Update ref commit
old-commit-hash: 838346d8a3a139a160e3cc602ba38d6f3b09a00a
2023-06-09 19:17:50 -07:00