Commit Graph

1661 Commits

Author SHA1 Message Date
Dean Lee 2119f7774e cereal/SubMaster: encapsulate frequency management in FrequencyTracker (#33252)
* encapsulate frequency management

* apply reviews

* early return, avoiding unnecessary calculations

* simplify avg freq calc
old-commit-hash: b14fca78e07af4e334af1097fe7e328103226e21
2024-08-19 16:53:13 -07:00
commaci-public 155934bd49 [bot] Update Python packages (#33327)
* Update Python packages

* fix opendbc

---------

Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 9b2cafe597c0ab871b06ec30e28444773ba1c3a6
2024-08-19 15:25:05 -07:00
Adeeb Shihadeh ce9bea56ea update test
old-commit-hash: 08f64ae30ae263710697cf712958f40751fec15c
2024-08-19 14:19:12 -07:00
Shane Smiskol 4f019b5f60 move selfdrive/car to opendbc (#32630)
* move most of /car

* move some car tests

move some car tests

* fix selfdrive/car/tests

* fix selfdrive/controls tests

* fix the rest of the selfdrive tests

* bump opendbc

* fix all tests

* few more non-test references

* remove opcar and move docs to car

fix these debugging scripts

fix docs

* bump opendbc and panda

forgot panda
old-commit-hash: e735a7f3796e208cdc1ecc651ace09e1daf6009a
2024-08-17 00:54:51 -07:00
Shane Smiskol 95224db413 selfdrive/car: ban cereal and capnp (#33208)
* ban cereal and msgq

* common too

* do toyota/values.py

* do all fingerprints

* example without builder

* this still works, but no type checking anymore

* stash

* wtf, how does this work

* okay actually not bad

* safe

* epic!

* stash data_structures.py

* some clean up

* hell yeah

* clean up old file

* add to delete

* delete

This reverts commit 90239b7797ace31ee647a2fdbd67e0c3faa98dcf.

* switch more CarParams stuff over

remove unused

* fix car tests by removing cereal! mypy forgets about dataclass if we wrap it :(

* fix this too

* fix this too

* remove more cereal and add some good hyundai tests

* bunch more typing

* override default with 20hz radar

* temp capnp converter helper

* more lateralTuning

* small union replicator is better than what i was trying, and fixes mypy dynamic typing issues

* can keep all this the same now!

* type ret: CarParams, add more missing structs, revert lateralTuning changes (smaller diff!)

* revert more

* get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :(

would have been `CarParams.NetworkLocation()`

* Revert "get first enum automatically, but ofc mypy doesn't pick up the new metaclass so can't use :("

This reverts commit bb28b228becba932052d2fc5a4389784027435b1.

* remove cereal from car_helpers (TODO: caching)

* remove a bunch of temp lines

* use dataclass_transform!

* remove some car.CarParams from the interfaces

* remove rest of car.CarParams from the interfaces

* same which() API

* sort

* from cereal/cache from fingerprinting!

* more typing

* dataclass to capnp helper for CarParams, cached it since it's kinda slow

* (partial) fix process replay fingerprintig for new API

* latcontrollers take capnp

* forgot this

* fix test_models

* fix unit tests

* not here

* VehicleModel and controller still takes capnp CP since they get it from Params()

* fix modeld test

* more fix

* need to namespace to structs, since CarState is both class and struct

* this was never in the base class?!

* clean that up again

* fix import error

fix import error

* cmts and more structs

* remove some more cereal from toyota + convert CarState to capnp

* bruh this was wrong

* replace more cereal

* EventName is one of the last things...

* replace a bunch more cereal.car

* missing imports

* more

* can fix this typing now

* proper toyota+others CS typing!

* mypy can detect return type of CS.update() now

* fix redeclaration of cruise_buttons type

* mypy is only complaining about events now

* temp fix

* add carControl struct

* replace CarControl

i hope there's no circular imports in hyundai's CC

* fine now

* lol this was wrong too

* fix crash

* include my failed attempts at recursively converting to dataclass (doesn't implicitly convert types/recursively :( )

but attrs does, maybe will switch in the future

* clean up

* try out attr.s for its converter (doesn't work recursively yet, but interesting!)

* Revert "try out attr.s for its converter (doesn't work recursively yet, but interesting!)"

This reverts commit ff2434f7bbd45a4d4bfb21f7d6712d1f1c3bcde9.

* test processes doesn't fail anymore (on toyota)!

* fix honda crash

* stash

* Revert "stash"

This reverts commit c1762af4e776790e4ad1322ad4ce0610157346e0.

* remove a bunch more cereal!

* LET'S GOOO

* fix these tests

* and these

* and that

* stash, something is wrong with hyundai enable

* Revert "stash, something is wrong with hyundai enable"

This reverts commit 39cf327def258e2959fe23cd7a550a858f6d8f03.

* forgot these

* remove cereal from fw_versions

* Revert "remove cereal from fw_versions"

This reverts commit 232b37cd409b55d04b1afc90d4a80c49e710eb56.

* remove rest of the cereal exceptions!

* fix that

* add typing to radard since I didn't realize RI.update() switched from cereal to structs

* and here too!

* add TODO for slots

* needed CS to be capnp, fix comparisons, and type hint car_specific so it's easier to catch type issues (capnp isn't detected by mypy :( )

* remove the struct converter

* save ~4-5% CPU at 100hz, we don't modify after so no need to deepcopy

btw pickle.loads(pickle.dumps()) is faster by ~1% CPU

* deepcopy -> copy: we can technically make a reference, but copy is almost free and less error-prone

saves ~1% CPU

* add non-copying asdict function

* should save ~3% CPU (still 4% above baseline)

* fix that, no dict support

* ~27% decrease in time for 20k iterations on 3X (3.37857 -> 2.4821s)

* give a better name

* fix

* dont support none, capitalize

* sheesh, this called type() on every field

* remove CS.events, clean up

* bump card %

* this was a bug on master!

* add a which enum

* default to pid

* revert

* update refs

* not needed, but consistent

* just Ecu

* don't need to do this in this pr

* clean up

* no cast

* consistent typing

* rm

* fix

* can do this if we're desperate for the last few %

* Revert "can do this if we're desperate for the last few %"

This reverts commit 18e11ac7883a0a56583750b1cc5a2b13011e7299.

* type this

* don't need to convert carControl

* i guess don't support set either

* fix CP type hint

* simplify that
old-commit-hash: 6a15c42143a4764e79df851d797f950a6212f464
2024-08-16 15:13:00 -07:00
Shane Smiskol f8ef09fcb2 rm tesla (#33300)
* rm tesla

* more rm

* ugh we should remove dynamic imports soon
old-commit-hash: b9dec5e3b51cf60f5544e93ca208b15d47b99ef0
2024-08-14 15:06:00 -07:00
Maxime Desroches dc396b6301 lint.sh: better shebang lint (#33296)
-e
old-commit-hash: 585f362738908bb4c98e87fac0cd7cb64a78d6b4
2024-08-14 11:34:52 -07:00
Maxime Desroches c551f1d239 fix bash + python3 shebangs (#33287)
fix shebangs
old-commit-hash: 1dcd66081531f4568a8de69637fc7a3f4ef3214f
2024-08-13 21:22:13 -07:00
Kacper Rączy 80bc61dc6c controlsd: use livePose (#33283)
* Pose calibrator

* Fix static analysis

* Fix static

* Fix test_latcontrol

* Fix test_latcontrol

* Update services in process replay

* Fix static

* Matmul not mul

* Add assertion

* Move pose calibration to data_sample

* Update ref commit

* Remove llk from cycle alerts

* Deprecated nogps event

* Switch power_draw to lp

* Bring back noGps alert

* Add handling code back

* get_bool

* Bring inputsok back
old-commit-hash: 9734015bbb6d448bb6b0fb453370ec702fa73106
2024-08-13 21:11:16 -07:00
Shane Smiskol 162436088f move capnp CarParams decoding out of car_helpers (#33251)
* move capnp byte decoding out of car_helpers

* always wanted to type this

* and process replay

* this does nothing

* might as well
old-commit-hash: ee62b9c88baf4a53f6db88381da2b8052c3fab17
2024-08-10 00:06:56 -07:00
Shane Smiskol a7db075f08 selfdrive/car: generic CAN send/receive callbacks (#33215)
* start

* hmm API doesn't seem great

* better

* rm line

* sendcan -> can_send callable for best static type coverage, list -> tuple msg

TODO: logcan

* more sendcan -> can_send references

* remove pandad's capnp from selfdrive/car

* fix and remove cereal from test_can_fingerprint

* test_fw_fingerprint: remove pandad, less cereal

* comment done too

* better comment

* fix test_models test

* niceeee it works

* move to can_definitions

* can't come up with a better name :(

* I think we can remove SimpleNamespace soon

* fix test_can_fingerprint.py

* maintain previous behavior

* Revert "maintain previous behavior"

This reverts commit f848fd32132391692c6191a305bb38f74091ec91.

* can test comment

* no need for get_one_can now!

* big clean up: no SimpleNamespace

* now self explanatory!

* not needed

* use empty can again since this is now real

* cmt
old-commit-hash: 9880b1393c7bfc2b71aec01b6b91caffb72f9cc0
2024-08-07 23:49:25 -07:00
Shane Smiskol 8131716c9e CAN fingerprinting: count empty CAN packets as frames (#33232)
* content of can packet won't affect can fingerprint time

* remove get_one_can

* fix comment

* actually we still want to be able to send empty can (xx fingerprint func does this if it runs out of msgs)

* fix process replay
old-commit-hash: 87f183f43cccdb07f6fb9d0947f4ada2d806e294
2024-08-07 23:12:47 -07:00
Shane Smiskol c0f721180f get_car: returns just CI (#33204)
just return CI
old-commit-hash: d00a53983089be6bed0e037474abfea823f2fc3e
2024-08-06 00:30:25 -07:00
Shane Smiskol 1a74993ee2 selfdrive/car: ban params (#33198)
* ban params too with a callback

* all sorts of messed up

* use cloudlog

* consistent order

* order

* better type hint

* format

* this is a bit nicer

* hmm

* fix PLR1704

* no carvin

* fix process replay
old-commit-hash: 5a1596a32299d6f78d89c89469ab6893b8d5e083
2024-08-06 00:10:53 -07:00
Shane Smiskol a1c2971f13 selfdrive/car: OBD callback (#33200)
* finish

* pass empty cb

* last fix!

* i keep messing this up

* even more coverage!

* not needed
old-commit-hash: 86aeb123bcdac9544db046a2db7e2974b5a687e1
2024-08-05 23:30:52 -07:00
Adeeb Shihadeh fc469e5f80 Revert "Remove setproctitle (#32716)" (#33195)
* Revert "Remove `setproctitle` (#32716)"

This reverts commit 9020d1931f4cd72f8162e4402278bdf76e3746f2.

* uv lock

* old name
old-commit-hash: 29b58d4f2fcc982d5c06ed8391e368b1b293a96c
2024-08-05 16:42:22 -07:00
Shane Smiskol 11d18b7aca test_onroad: lower lower log size bound
old-commit-hash: 0fe143e4a75e7657f4d815a5616fc29f70de5ad5
2024-07-30 14:07:47 -07:00
Shane Smiskol 7ce33fbe2b process replay: use zst (#33138)
* test

* update refs to zst

* update
old-commit-hash: dfd387520e88fa68f0485b6ce8dcf5befa82bafa
2024-07-30 01:46:03 -07:00
Kacper Rączy 7ea8d9e91d torqued: use livePose (#33136)
* Use livePose

* Replace it in process replay

* Add liveCalibration to messages

* Update ref commit
old-commit-hash: 86d8d1d99617a0808e0a17031965a056cdc5a1b8
2024-07-29 20:20:31 -07:00
Kacper Rączy 3740693956 paramsd: use livePose (#33099)
* Use livePose instead  of llk

* Update process replay sockets

* Fix import

* Fix calib

* Fix field name

* Dont store device_from_calib

* Update ref commit
old-commit-hash: 84cff4fc03423acde611ba29b3b2ff9436ff2580
2024-07-29 19:51:21 -07:00
Kacper Rączy 130d0d2e3a process_replay: regenerate locationd logs (#33134)
* Filter out non-relevant services in most_messages_valid check

* Set as comprehension

* Update segments

* Update ref commit

* New check for hyundai2
old-commit-hash: 6e185f4eea470fe56fc6a33de9e81779f70941ab
2024-07-29 17:12:05 -07:00
Shane Smiskol 7b67994a05 use zstandard library (#33110)
* use zstandard

* run 10

* bye bye

* this was a little overzealous

* clean up
old-commit-hash: 1b3b5ab75857465f58ec8a687db04c8ee0470e05
2024-07-28 02:03:47 -07:00
Shane Smiskol 09df2fbb14 test_onroad: use zstd compression (#33100)
* use zstd in test_onroad

* debug

* now leans towards 0.4 instead of 0.5

* 5x runs

* better

* more

* Update selfdrive/test/test_onroad.py

* revert Jenkinsfile

* and this
old-commit-hash: 4caecf21439eb65cedb25df589b140c9c09e26fc
2024-07-26 21:14:06 -07:00
Shane Smiskol 539c828861 tools: keep re pattern file small (#33084)
remove bz2 from re file
old-commit-hash: e68bb26e14d4e85f61ab3f58ac9667ca9d6ea565
2024-07-25 22:05:10 -07:00
Dean Lee 757cb2eb6c mcid.py: optimize FFT and A-weighting calculations (#33057)
* Precomputing weighting

* add comments back

* use cache

* spacing

spacing

* clean up

* lower by diff

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 313a2826c268a7974ce7422084ab294767b7c9bd
2024-07-25 13:36:44 -07:00
Harald Schäfer 28a91c4e23 livePose log (#32868)
* add livepose

* Formatting

* Add to sevices

* Update locationd to publish livePose

* Remove fields and increase decimation

* Fix field indices

* Remove the line

* Add livePose to pubmaster

* Fix llk decimation

* Update ref commit

* XYZ measurements instead of lists

* Update locationd

* Update ref commit

* Lower the qlog size in test_onroad

* Update lower and upper boundary

---------

Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
old-commit-hash: 996bced674e6943976f1d6dba37075e113bf9288
2024-07-17 16:17:23 -07:00
Shane Smiskol a878118b00 Revert "selfdrive/car: ban imports from external modules" (#32993)
Revert "selfdrive/car: ban imports from external modules (#32973)"

This reverts commit c1e8ff2dbc47ccb07fb41a4e88bfb66af2669254.
old-commit-hash: 35a4a773f1ff6c7b171b7acdb2fd4a65279edebd
2024-07-15 11:14:04 -07:00
Dean Lee 58df39a2ba selfdrive/car: ban imports from external modules (#32973)
* lint import

* Update selfdrive/car/__init__.py

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 83d4623590f87dcb7df2b458a8814d5ad6ea711b
2024-07-15 11:12:24 -07:00
Shane Smiskol bb67139ca7 torqued: check steer override to current time (#32963)
* lat active/steer override: check up to now

* lint

* Update ref_commit
old-commit-hash: b247c3caaa5b5d503a9c1aa3336a7a206765167c
2024-07-10 17:25:39 -07:00
Shane Smiskol 8e3b18c3a0 Ford: use more reliable gear message (#32886)
* todo

* nope

* not this one

* yep

* passes

* clean up

* not used!

* oop

* update refs
old-commit-hash: 7de2aac3c93caa20213894e574f06bd45f789335
2024-07-02 22:22:56 -07:00
Mitchell Goff e0c89a36da Notre Dame model (#32884)
* New model: 6f23a03f-486b-4d3e-a314-19d149644c7c/700

* Bump refs
old-commit-hash: 01c2174d5968266b87f1d1fecefce5affaeaa624
2024-07-01 13:59:44 -07:00
Kacper Rączy 02a35949dc drivingModelData: frameDropPerc (#32871)
* Smokes! lets go!

* Ignore it in model replay
old-commit-hash: 417f1e2139ba6037c9fd6494f68194e36e243540
2024-06-29 02:31:55 -07:00
Kacper Rączy 102188c245 drivingModelData: new model packet for qlogs (#32821)
* Mini model v2 prototype

* Apply suggestions

* Add meta

* Support for new packet in model

* Support in process replay

* Meta

* Add it to services

* Update model replay

* Add drivingModelData in model_replay

* Update ref

* MAX_FRAMES*3

* No modelv2, but drivingModelData at 2hz

* Polypath

* Dont use keyword

* Update model ref commit

* xyz coeff

* Fix field name

* Update ref commit

* Min qlog size 0.6

---------

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 2059f986dff46254419c889eadcc82ecea1bce94
2024-06-28 19:27:58 -07:00
Adeeb Shihadeh 6745358478 remove mapbox references (#32811)
old-commit-hash: bccccde568c05a7a641110c90d4ed95047cdea4e
2024-06-22 18:01:47 -07:00
Adeeb Shihadeh 3b8ed67aa3 remove navigation (#32773)
* remove navigation

* lil more

* fix cabana

* revert for now

* fix docs

* move that over

* lock

* rm assets
old-commit-hash: e491c9d964a2aef07eeaa7b47f4aa9f992130482
2024-06-19 19:14:13 -07:00
Harald Schäfer 47d5601d8a long control: new API (#32706)
* Simplify long control

* Seperate

* Rename

* Try new api for toyota

* rm v_pid everywhere

* No speed in reset

* 0 is better default

* unassigned variable

* Update other cars

* Update gm

* SIMPLIFY

* simplify more

* fix API boundry

* Fix stopping bug

* Small fixes

* Update ref
old-commit-hash: bc303df6a0a4a9d52a2e92e5976b6c7aa5138b95
2024-06-14 00:08:58 -07:00
Adeeb Shihadeh 9cfdd6200d Remove setproctitle (#32716)
* Custom setproctitle (#32667)

* add custom setproctitle

* add test

* Update poetry.lock

* fix lint

* support only Linux

* test only Linux

* final lint

* Update test_setproctitle.py

* Update setproctitle.py

* convert to threadnames

* delete proctitles

* Check str len and use PR_GET_NAME

* fix poetry.lock

* lint fix

* Update common/threadname.py

---------

Co-authored-by: reddyn12 <nikhilr.ssm@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* revert that for now

* use last 15

* fix

* use name

* update those

* and modeld

* rm

---------

Co-authored-by: schlimeszn <138847413+schlimeszn@users.noreply.github.com>
Co-authored-by: reddyn12 <nikhilr.ssm@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 83ac80c6c899dbf4b5b2d422947b02b57d83bccd
2024-06-11 21:57:32 -07:00
Shane Smiskol 272216b2a8 remove uiPlan (#32721)
* remove uiPlan from ui

* remove publishing

* Update log.capnp

* Update services.py

* no uiPlan here

* Update ref_commit
old-commit-hash: 684209e077d77da0c3a12e8fda528649c5fe9edc
2024-06-11 21:31:10 -07:00
Mauricio Alvarez Leon 008008b03a Update to Python 3.12 (#32548)
* 3.12

* pprofile is broken

* use modified metadrivepy3-12

* 0.3.0 metadrive

* add metadrive/commaai git dependency

* metadrive git set

* pin sounddevice 0.4.6

* datetime.utcnow() deprecation

* poetry lock

* make datetime not aware

* poetry lock

* pin pytools

* google_crc32c wheel

* unpin sounddevice

* clean metadrive

* use python crc

* mypy

* 3.12.4

* allow python3.11

* test

* no pip

* poetry

* better

* better

* merge

* remove

* try

* test

* try  this

* snok

* python

* simpler

* setuptools

* lower

* try

* try

* work?

* ubuntu deps

* ubuntu

* try

* remove

* move

* remove this

* names

* Update .github/workflows/tools_tests.yaml

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* python<4

* <3.13

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
old-commit-hash: 148eaf8fa6c9b23cdd346e926f58fad61f79ff1d
2024-06-11 20:45:15 +00:00
Harald Schäfer d9fb61da16 Long control: Double delay is super complicated (#32694)
* Double delay is super complicated

* No more upper bound

* DEAD

* Update ref
old-commit-hash: 965415135ac7db4c67ea248ac52535eac537f3bf
2024-06-10 22:29:17 -07:00
Adeeb Shihadeh 328079f8ad Revert "test_onroad: update modeld proc name"
This reverts commit 65f37bb46982c73d8c84ca576d7dd48edacea5d1.

old-commit-hash: 00388ca7958d7dd9b4d505927ee57df77bdb16d7
2024-06-10 19:55:48 -07:00
Adeeb Shihadeh 83350bbd22 test_onroad: update modeld proc name
old-commit-hash: 4684651dc55dd0fb95fe33686ad519309f988bfd
2024-06-10 19:45:03 -07:00
Shane Smiskol 196aaf01d6 fix hypothesis warning (#32678)
fix
old-commit-hash: 43ebf367ef37d949952301f2e240359b451a7043
2024-06-10 00:28:43 -07:00
Adeeb Shihadeh 8472df65ea tools/sim: remove special docker image (#32659)
old-commit-hash: 541b5cbf1387df07e76da55d3e332170d9e756f0
2024-06-08 09:18:28 -07:00
Harald Schäfer f870a968e9 Split cereal into cereal/msgq (#32631)
* squash

* fix doc

* compile device

* compile device

* Update ref

* add msgq to precommit exclusions

* No service ports

* fix compile

* address comments

* More comments

* Delete logger

* Update opendbc

* Linting

* bump msgq
old-commit-hash: e71ec4e8d7e39a56235f590227e7cb361cd8b79d
2024-06-06 14:31:56 -07:00
Dean Lee 6f9e28d8f2 fuzzy_generation.py: improve performance (#32591)
* improve performance

* remove DEPRECATED

* formatting

* catch kjException

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 3a43f5d7844611cd5c770748b7d235974c3eb97d
2024-06-05 23:48:20 -07:00
Adeeb Shihadeh 30c41788dc thermald is hardwared (#32637)
* thermald is deviced

* hardwared

* rename
old-commit-hash: 51fdf2330453fe863597db0eb8daf87c2ec4ff8d
2024-06-05 15:58:00 -07:00
Adeeb Shihadeh 22bfc435a9 jenkins: fix unsafe checkout when changing submodules (#32635)
do it twice
old-commit-hash: 95263340493ed516c9a2f84ce770276560007b89
2024-06-05 14:00:55 -07:00
Adeeb Shihadeh 2c6829ed67 boardd is pandad (#32628)
* boardd is pandad

* rename tests
old-commit-hash: 34e329649d7a388d4cd1f25dac9dda9d5409bf4a
2024-06-04 19:16:55 -07:00
Vivek Aithal cf8885b596 [torqued] Reduce friction for some Hyundai cars (#32578)
* reduce friction coeff for ev6 and ioniq6

* Update ref

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
old-commit-hash: 8a4bfcf5b4f4adc5cf9f0c91895d621421e994a2
2024-06-04 11:10:04 -07:00