Commit Graph

100 Commits

Author SHA1 Message Date
Adeeb Shihadeh 86b6103657 test_*.py files are no longer executable (#32610)
* test_*.py files are no longer executable

* cleanup empty lines

* update precommit

* fixme

* enable

* build
old-commit-hash: c59887c8109dd8d33ea2e7169995184fbe38cc3a
2024-06-03 15:48:56 -07:00
Shane Smiskol ee03d13529 card: process that abstracts car interface and CAN (#32380)
* format card

* standalone process

* no class member CS, there's no point

also can be confusing; what else could be using this?

* rename CoS

* Update selfdrive/controls/controlsd.py

* never works first time :D

* canRcvTimeout is bool

* hack

* add cpu

* see what testing closet comes up with

* first

* some clean up

* support passable CI, fix test models

* fix startup alert

* process replay changes

* test_fuzzy

* gate carOutput valid on carControl valid

* we should publish after we update carOutput

* controlsd was using actuatorsOutput from 2 frames ago for torque, not the most up to date

* check all checks for carControl in case controlsd dies

* log more timestamps

* more generic latency logger; needs some clean up

latency_logger.py was difficult to understand and modify

* card polls on can and carControl to get latest carControl possible

* temp try to send earlier

* add log

* remove latencylogger

* no mpld3!

* old loop

* detect first event

* normal send

* revert "card polls on can and carControl to get latest carControl possible"

how it was is best

* sheesh! update should be first

* first timestamp

* temp comment ( timestamp is slow :( )

* more final ordering, and make polling on/off test repeatable

* Received can

* new plot timestamps

* clean up

* no poll

* add controllers (draft)

* Revert "add controllers (draft)"

This reverts commit e2c3f01b2fadcff74347bac90c8a5cc1ef4e27b3.

* fix that

* conventions

* just use CS

* consider controlsd state machine in card: not fully done

* hmm it's just becoming controlsd

* rm debugging

* Revert "hmm it's just becoming controlsd"

This reverts commit 534a357ee95bec4ed070667186af55d59421bbc7.

* Revert "just use CS"

This reverts commit 9fa7406f30c86200f20457f7b9ff95e731201bf9.

* add vCruise

* migrate car state

* Revert "migrate car state"

This reverts commit 4ae86ca163c6920070f410f608f7644ab632850b.

* Revert "add vCruise"

This reverts commit af247a8da41c3626ada4231b98042da1a1ae4633.

* simple state machine in card (doesn't work as is)

* Revert "simple state machine in card (doesn't work as is)"

This reverts commit b4af8a9b0a2e17fdfc89d344c64678ef51305c24.

* poll carState without conflate

* bump

* remove state transition

* fix

* update refs

* ignore cumLagMs and don't ignore valid

* fix controls mismatch; controlsd used to set alt exp

* controlsd_config_callback not needed for card

* revert ref temp

* update refs

* no poll

* not builder!

* test fix

* need to migrate initialized

* CC will be a reader

* more as_reader!

* fix None

* init after publish like before - no real difference

* controlsd clean up

* remove redundant check and check passive for init

* stash

* flip

* migrate missing carOutput for controlsd

* Update ref_commit

* bump cereal

* comment

* no class params

* no class

* Revert "no class"

This reverts commit 5499b83c2dcb5462070626f8523e3aec6f4c209d.

* add todo

* regen and update refs

* fix

* update refs

* and fix that

* should be controlsstate

* remove controlsState migration

CoS.initialized isn't needed yet

* fix

* flip!

* bump

* fix that

* update refs

* fix

* if canValid goes false, controlsd would still send

* bump

* rm diff

* need to be very careful with initializing

* update refs
old-commit-hash: 71f5c441fe32184d94a9f26565a36c661e2ccf28
2024-05-21 01:18:10 -07:00
cl0cks4fe 639a634f7c Unittest to pytest (#32366)
* add pytest-asyncio

* switch common

* switch selfdrive

* switch system

* switch tools

* small fixes

* fix setUp and valgrind pytest

* switch to setup

* fix random

* switch mock

* switch test_lateral_limits

* revert test_ui

* fix poetry.lock

* add unittest to banned-api

* add inline ignores to remaining unittest imports

* revert test_models

* revert check_can_parser_performance

* one more skip

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: dd9d5d4528ff0a61c7699abc1b63ba8fd282632f
2024-05-17 11:01:44 -07:00
Shane Smiskol 97dc134a42 card: create pedal pressed event (#32417)
* card: create pedal pressed event (#32393)

* move pedalPressed to card

* rm

* needs to be a builder

* move these

* clean up

* reader later

* Update ref_commit

* moved to card
old-commit-hash: d0d44a51a0ff66c1b2da56b3d9864e10ecc2174d
2024-05-13 16:18:55 -07:00
Shane Smiskol 40fea75562 debug test_models script fixes (#32362)
* some things were renamed without checking

* fix that
old-commit-hash: e5107b1eabc3a04cb097f0f17d5cf587eb387453
2024-05-06 15:01:06 -07:00
Cameron Clough aa56650862 test_models: migrate fingerprint (#31936)
old-commit-hash: 90ff0dd04728c55237a71ec99ece7adcd03c1e21
2024-03-20 10:57:28 -07:00
Justin Newberry b899592b18 cars: platform enums -> prepend brand name (#31927)
* with brand name

* migrate

* Fix

* fixes

* more

* passes

* fix

* fix the doc

* collects

* these too

* more stuff

* body exception :/

* more

* hardcode i guess

* update ref

* toyota

* more toyota

* and here

* final!

* fix notebooks and ccs

* move this here
old-commit-hash: b0eb3ba4f477142797624580eb3fdb6448f068d6
2024-03-19 17:29:50 -07:00
Justin Newberry 1f178307f0 move getting platform to get_params (#31871)
* better

* string

* not here
old-commit-hash: ca5a2ed942c7560f11dd85b812c914f944318250
2024-03-14 21:58:25 -07:00
Justin Newberry 3105613e69 fix car test routes typing (#31773)
* Fix typing

* and fix test_car_model

* fix
old-commit-hash: a919d27afc9a225978737e86241adbdba336bbdd
2024-03-07 11:33:40 -08:00
Justin Newberry 1a5b057983 Honda: move to platformconfig (#31637)
* first

* Fixes

* not

* not

* Fixes

* fix

* cleanup + use sets for now

* update ref

* update ref
old-commit-hash: 5734d7c2bf13c217e469d37c4900321cba469121
2024-02-29 11:29:40 -08:00
Cameron Clough 9bd90112d0 use pyupgrade to update to new typing syntax (#31580)
* add pyupgrade hook

* run pyupgrade (pre-commit run -a)

* ruff --fix

* Revert "add pyupgrade hook"

This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.

* revert changes to third_party/

* manual type fixes

* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae4967943ee33a3699a1d89a7a770d95e9
2024-02-24 16:41:23 -08:00
Justin Newberry c564806520 Subaru: move carspecs to PlatformConfig (#31550)
* subaru platform config

* forester wrong dbc

* spacing

* subaru car specs

* someday!

* more red diff

* all brands can be done like this

* but this should be done first and thats subaru specific

* that seems very low but we shouldn't change it here

* as long as it subclasses str its fine

* fix
old-commit-hash: 1161d33c1892b32980a222b7c5b2ad9d6b05621e
2024-02-23 13:23:40 -08:00
Adeeb Shihadeh 66adafe850 radard: remove sleep for cars without radar (#31467)
* radard: remove sleep for cars without radar

* fix

* update refs
old-commit-hash: 6f905ed979cb8e217c182c653528df3a6094015b
2024-02-14 13:12:54 -08:00
Shane Smiskol 1f67acfed9 test models: more examples in CI (#31444)
* more examples

* how is it so fast?!

* 500

* is there a hypothesis cache?

all

* Revert "is there a hypothesis cache?"

This reverts commit e628ba73d5b639e36f95df0ab9049697b34a58b8.

* same as jenkins
old-commit-hash: 9479686fa399ed95232d9419c1e566fb51ce586f
2024-02-13 23:00:44 -08:00
Justin Newberry 572744128e test_models: cleanup by using new logreader (#31267)
* cleanup

* only check error

* this segment only

* fix

* fix

* keep this

* fix internal
old-commit-hash: 66dc7030de1b39bd814cc20091d22ec38976523b
2024-02-06 16:50:30 -08:00
Justin Newberry f447062b66 CommaCarSegments: public database of segments for each platform (#31114)
* comma car segments

* comma car segments for test_models

* oneliner

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* better name

* not used here

* sort

* remove print

* better comment

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: caa9153974c21fd827d0e991f1789cf3e81b5c3b
2024-01-22 17:33:03 -08:00
Justin Newberry 4080bec11d pytest: add marker for sharing the download cache (#31082)
* fix cache

* with a marker
old-commit-hash: 3846130d8e15ea71e9576ae62b69918757edc7b8
2024-01-22 16:30:59 -08:00
Justin Newberry b29eea47ce sanitizer: sanitize vin number (#31090)
old-commit-hash: 2c86c023fe2d3072910ed33ff97cd6d3ddf2915d
2024-01-21 12:22:32 -05:00
Justin Newberry eeaf219cce test_models: sanitize segment before running test (#31026)
to ensure sanetized routes will work
old-commit-hash: 5c24527683cd4a9d338fe64dd9e98179e903316e
2024-01-16 15:08:44 -08:00
Justin Newberry 4b7edec2b0 LogReader: move opci to tools/lib (#31021)
move opci
old-commit-hash: 7d400112df6692ef026245b03951345cc36dd83e
2024-01-16 10:58:51 -08:00
Justin Newberry 1fe1523fc2 add read_seglist helper (#30908)
add helper
old-commit-hash: 1148c65d232fdfe7aa46b80c078721d78876da4a
2024-01-04 17:52:07 -08:00
Shane Smiskol dac5a594aa test_models: fallback to public route when route isn't uploaded to CI bucket (#30893)
* not sure why this is a test

* Revert "not sure why this is a test"

This reverts commit c13e16311eb882df65d216bbd59acb5702e985e6.

* no ruff for this pattern :(

* this is always caught

* should fail

* simpler

* Revert "should fail"

This reverts commit 21bd15275c46cd42fd19b058245334bae83cce77.

* fix

* cleanup this

* better comment

* better

* this can be more localized

* better names

* better names

* should fail

* Revert "should fail"

This reverts commit c58495e8b3417ce351df8048752ec60177827cb3.

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7f398e1cf9b0fd83bb1238b9b347a21d8868ee0f
2024-01-03 11:54:22 -08:00
Shane Smiskol 52150bbb02 Revert "test_models: fallback to public route when route isn't uploaded to CI bucket (#30794)" (#30896)
* fails if no skip function

* Revert "fails if no skip function"

This reverts commit ff3d697d38dce56057c83f06033a689f2301a29d.

* Revert "test_models: fallback to public route when route isn't uploaded to CI bucket (#30794)"

This reverts commit f60e70d4729efe7db36888a0a750a0e4706cc943.
old-commit-hash: 8c4930d2d300a439aaf7dd9ca4324271cc0c7b52
2024-01-02 17:22:06 -08:00
Justin Newberry bf8e8f8338 test_models: fallback to public route when route isn't uploaded to CI bucket (#30794)
* wip

* add public comment

* fix that

* wip

* retrigger
old-commit-hash: 05dc0b51e6597fb83aed14f5a390fa4254751c98
2023-12-20 11:29:19 -08:00
Shane Smiskol f65ccca8e0 test_models: fuzz test panda and CarState (#30443)
* pre-hypothesis

* some hypothesis junk

* this kinda works but is really slow due to counter check

* choose addrs from fingerprint

* stash

* honda nidec brake pressed mismatches fixed

* bump panda

* stash

* tesla: use DI_torque2 (panda msg)

* run

* run

* ah this honda mismatch too

* no more multi can msgs

* clean up, remove old file

* add todo

* prob can remove urandom

* stash, huge examples

* fix pq standstill mismatch

* yuge

* yup there's a leak somewhere

* try to find leak

* skip dashcam (pq and tesla)

* PR comments

* bump

* draft stash

* fix alt brake hondas

* bump

* bump

* bump

* some clean up

* minor clean up

* more clean up

* stash

* fix honda bug

* more

* 100 examples

* revert tesla

* no memory leak any more?

* bring back tests with skips

* parameterize max_examples

* skip interceptor

* is jenkins on my branch?

* ooh that's fast

* 50 is not bad for GH CI

* 300 might be better with rest of test_models

* no more detection

* bump

* need CS_prev to catch bugs where openpilot changes and panda doesn't (eg. not setting interceptor safety mode)

* need to simplify all this

* need a warm up first, since some signals are 1 by default (toyota's gas_released!=1)

* changes

* set honda safety param

* set toyota safety param

* bump panda

* clean up honda

* rm interceptor

* thought interleaving addrs might help, but we can fine tune later

* Revert "thought interleaving addrs might help, but we can fine tune later"

This reverts commit 153301384b48c9f33f9e2af3c224241eaeec41c1.

* get size from dict

* what

* add nocapture marker

* clean up

* try to raise logging level

* need to run last as pytest_runtest_call, since it starts capturing

* get capman conditionally

* mark

* type fingerprint

* should use gen_empty_fingerprint

* no longer needed

* draft

* no longer need gc

* clean that up

* test everything!

* more clean up

* more

* no point

* fix that

* fix errors

* bump

* nice even 300 examples for 300 segs

* final bump :fingers_crossed:

* better import order

* remove debugging prints

* warm up kinda works

* Revert "warm up kinda works"

This reverts commit 7fc77b07d592edb13eadca77deb49540954a7d69.

* random seed

* revert

* strat

strat

* add expl comment

* cmt

* check controls allowed

* Revert "check controls allowed"

This reverts commit e82a0e5396810dd4670e6847aa555194a709e10f.

* not unittests

* run tests!

* run tests 2!

* run tests 3!

* seed unused

* revert

* add shrink phase, and remove health check suppression

* hello

* oncemore

* Update selfdrive/car/tests/test_models.py
old-commit-hash: 5052b55c440f3a778f9db9c854fa385077dd7ab9
2023-12-19 03:18:54 -06:00
Shane Smiskol 3c7fcbb839 test_models: use gen_empty_fingerprint (#30752)
use gen_empty_fingerprint
old-commit-hash: d9a8846d58af2a3a9a598fcf9fb1acf89e4b54f6
2023-12-15 17:16:19 -08:00
Shane Smiskol 19572df478 Revert "test_models: check consitency between panda and openpilot for velocity (#30750)" (#30753)
This reverts commit 19c040150dcec8bfa4b70aa870ae5564739c278a.
old-commit-hash: 123ef46af3694afc30b4ec28ce8f995d9fa92cd1
2023-12-15 17:16:01 -08:00
Justin Newberry ffae1ce377 test_models: check consitency between panda and openpilot for velocity (#30750)
* sanity check on velocity

* import from there

* use raw

* just check last

* ignore body

* reduce tolerance and bump panda

* its not average anymore

* slightly higher tolerance
old-commit-hash: 1ca1cf39223557955cee58e18a418914faf472c3
2023-12-15 14:50:37 -08:00
Shane Smiskol e46c20ac61 test_models: randomize internal segment list (#30653)
* randomize internal seg list segments

* random

* pytest-randomly sets random.seed to a consistent value for all the workers/processes

* noeol

* update

* Revert "update"

This reverts commit aff9a69c4e5e3934deebaa33986b42f44b55b002.

* lock

* don't randomize by default

* remove random-order

* strict

* random

* one fix

* test

* does nothing

* rm tests

* Revert "rm tests"

This reverts commit b548e3fcd48e60538695506888d863c01b459d27.

* (can't repro locally) just athena should be fine

* bs1

* bs2

* bs3

* bs4

* bs5

* wrong way

* no controls

* no car

* no board

* controls?

* crazy -common

* Revert "crazy -common"

This reverts commit 02365d712b3d09cab1893cce2261a4b418bb3851.

* test athena

* test athena 2

* test athena 3

* test athena 4

* test athena 5

* test athena 6

* test athena 7

* test athena 8

* test athena 9

* ??

* in one commit

* common?

* car and board

* -controls -board

* random-order

* no board

* revert

* car/tests

* least likely

* try

* try 2

* draft

* draft

* so much better

* cmt

* use randomly

* not needed here

* directly modify option works

* bb

* test time

* Revert "test time"

This reverts commit 2c5caabe2b470b47b7322e37800680b92773fccc.

* tmut

* i concur

* revert
old-commit-hash: ac83318ac4f54c46deef02c69d46c549bd741cb3
2023-12-09 14:28:32 -08:00
Justin Newberry 62d0647ab3 pytest: group test_models by route instead of car name (#30625)
* group by route

* dont need those
old-commit-hash: f16df8e4b7901f5e38b6f90a8900eef02faab504
2023-12-06 16:00:59 -08:00
Adeeb Shihadeh b526644eb1 falsify rx checks (#30564)
old-commit-hash: 613ccf4b74ce70ac4bff75d1dbdd5ee7f8f4acdc
2023-11-30 14:53:33 -08:00
Adeeb Shihadeh ef707d3226 bump panda (#30501)
* bump panda

* bump

* bump

* fix renamed variables

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: afd61352d0a09c7ff169cda5b3b1bb6a94af4d38
2023-11-20 14:45:51 -08:00
Shane Smiskol 559b620b20 test_models: check if in relay-breaking safety mode (#30476)
* remove initData, remove tolerance

* can check faster now

* don't
old-commit-hash: fc8cfe128046e741d28b127bac03b7a691aa39f3
2023-11-15 18:57:37 -08:00
Shane Smiskol de63cac4c7 test_models: remove ignored address check list (#30475)
* remove ignore_addr_checks_valid

* lessgoooo
old-commit-hash: e44d84a7f998022c6bbe8543c69ea294ec1313ae
2023-11-15 18:50:39 -08:00
Justin Newberry 337b156518 Pytest: load by group by default (#30466)
* wip

* use a class marker

* this one should be grouped too
old-commit-hash: 31c682d40743f4b102a70d72e6993b383d85a040
2023-11-14 17:33:47 -08:00
Justin Newberry ef75f0ab12 CI: use "slow" marker for pytest (#30063)
* skip to mark

* fix quotes
old-commit-hash: 0251742079b206ce40a3b4d8ae9cec1381494d4a
2023-09-26 13:44:18 -07:00
Shane Smiskol 600c5e2644 test_models: check controlsd events (#29907)
* test_moedls: check controlsd events

* clean up

* sort

* controlsd modifies CP

* fix

* can use any segment now

* final fix

* use constant
old-commit-hash: 1410a1150c9909bc643df4e2cacbefa721d3b6cb
2023-09-14 20:20:49 -07:00
Lee Jong Mun 89e5ee45ca cleanup remnants of LQR and INDI tuning (#29860)
* not use 'indi' cleanup

* bump cereal

* rm more

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 238f9d65dbc17613c2a045b5741483c3f9a7659d
2023-09-10 16:30:13 -07:00
Shane Smiskol 9fefac8bcc test_models: detect when OBD multiplexing/fingerprinting (#29759)
* wait to leave elm mode before checking relay malfunction

* clean up

* clean up

* pycharm is best

* clean up

* more

* use pandaStatesDEPRECATED

* fix

* use constant

* only radar

* cmt

* set inline

* comment and re-arrange

* enable for relay malfunc check

* down here

* Update selfdrive/car/tests/test_models.py

* rm

* don't affect other tests

* up here

* one line

* update cmt

* no tolerance

* Revert "no tolerance"

This reverts commit 41b1c7e8beda87a878c3bb1f37ec96b256965966.

* comment is real solution

---------

Co-authored-by: Kacper Rączy <gfw.kra@gmail.com>
old-commit-hash: 5d1550de871b0ac036b1dd44545fd7ff17fb91cd
2023-09-05 21:22:36 -07:00
Justin Newberry 6537493450 Tests: disable laika tests in CI (#29742)
skip laika
old-commit-hash: 4c039ab02230c7fcf2439b56fe04ad79df911ad4
2023-08-31 17:17:07 -07:00
Justin Newberry c4f322ce2d Linter: remove pylint (#29611)
remove pylint
old-commit-hash: 1ee6ed4b16e2ade0dd887c0687243aee769043e5
2023-08-24 16:30:54 -07:00
Adeeb Shihadeh 16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b69af19306143fc4def02fb5769405d2
2023-08-20 20:49:55 -07:00
Shane Smiskol 98b3087fbb test_models: remove body standstill exception (#29453)
* remove body exception

* bump to master
old-commit-hash: 3563292e6538a9aaf98800e28f9a6557994d8968
2023-08-17 16:26:30 -07:00
Shane Smiskol d87466556c Hyundai: match panda standstill check (#29442)
* bump panda

* clean up

* common?

* Revert "common?" (not now)

This reverts commit 197af7cdbb6a8c8e58fcbccca94f7db004ce91e4.

* fix

* no failures!

* remove time

* bump

* bump

* undo
old-commit-hash: 9d0d59ac9b40dabeb5984afc301917242da2515a
2023-08-17 00:06:38 -07:00
Shane Smiskol dc6af2820a test_models: check panda cruise state (#29423)
* cruise state test

* add todo

* rename

* clean up

* remove
old-commit-hash: 2d636deb4eabf54044b086d87305fc898f833711
2023-08-15 23:55:53 -07:00
Shane Smiskol 7e716dbc87 CANParser: perf benchmark on route script (#29324)
* timing test

* test

* fix

* print

* loop

* clean up

* wider range

* Update selfdrive/car/tests/test_models.py

* Apply suggestions from code review

* run many times

* Update selfdrive/car/tests/test_models.py

* run for many its

* run with unittest and print as list

* Update .github/workflows/selfdrive_tests.yaml

* Update .github/workflows/selfdrive_tests.yaml

* total time is super inconsistent (body)

* Update selfdrive/car/tests/test_models.py

* clean up

* clean up

* clean up

* this works!

* draft

* test suite not as modular

* try something like this

* can do kb, but not too representative

* clean up

* remove kb? it depends on signals

* clean up

* more clean up

* rename

* just measure all CANParsers

* can do all this manually

* but this is way simpler

* comment

* fix mypy

* this is correct too

* 3 seconds instead of 1.5
old-commit-hash: e2910d0720f0b65496234382847da1ab0d086e4d
2023-08-15 20:06:15 -07:00
Shane Smiskol 6c6fcd8b22 test_models: more typing (#29420)
* not type checked, but we set this in test_car_model

* type this
old-commit-hash: 6ec00f5ee2d07ca0b1b98cf4dc1f19487c08a98f
2023-08-15 20:01:33 -07:00
Shane Smiskol 4eadeff03f test_models: some typing (#29413)
* type

* also type check this func

* unused
old-commit-hash: 936f8b98177d98e75ebe50ed2a98a4b677b76dc8
2023-08-15 15:25:36 -07:00
Shane Smiskol 6be9e66073 jenkins: re-enable test_models (#29394)
* re-enable test_models

* now?

* Remove bad Volt segment

* Update Jenkinsfile

* use first n segments
old-commit-hash: fb235238addb73e04f59fca941040e3785be103e
2023-08-14 16:08:34 -07:00
Shane Smiskol 1945ef0441 test_models: match online fingerprinting behavior (#29306)
* draft

* 1x

* clean up

* final clean up

* fix

* self explan

* space

* space
old-commit-hash: 001d4e8729bdab999f9bd948b162091465a125f6
2023-08-11 18:33:11 -07:00