Commit Graph

74 Commits

Author SHA1 Message Date
Adeeb Shihadeh 2f7b2ce4d0 Ship EV6 (#25095)
* Ship EV6

* bump opendbc
old-commit-hash: d08a231774
2022-07-08 22:46:20 -07:00
Adeeb Shihadeh 733cacdd8f process replay: add Ram route (#25063)
old-commit-hash: 8d98d8c657
2022-07-07 10:28:55 -07:00
Gijs Koning 2a11354d28 Process replay: Fix subtest diff (#25054)
Fix subtest diff
old-commit-hash: edf170103e
2022-07-07 02:14:31 -07:00
Shane Smiskol 4eabd3b9d9 FW fingerprinting: log all FW versions (#25042)
* get_fw_versions returns all fw versions with request's brand

* keep track of everything received

* debug

* need to regen or write a hack in build_fw_dict

* to be safe, still replace old responses within same brands (hyundai responds to two queries, can fix later)

to be safe, still replace old responses within same brands (hyundai responds to two queries, can fix later)

* update test_fw_query_on_routes

* clean up

* better name

* slightly cleaner

* fix test_startup unit test

del

* fix imports

* fix test_fw_fingerprint

fix test_fw_fingerprint

fix

* fingerprint on all FW_VERSIONS, not just brands with requests

* support old routes in test_fw_query_on_routes

* regen and update refs

* similar function style to before

* better comment

* space

switch name

* try to exact match first

* useless else

* fix debug script

* simpler dictionary

* bump cereal to master
old-commit-hash: ea241bf3dc
2022-07-06 19:42:58 -07:00
Gijs Koning 76aaef497b Add laikadOffline subtest to process replay. (#24995)
* Add subtests to process replay.
Adds laikadOffline subtest

* Update cpp.

* Update ref

* Update ref again

* Update ref again

* update ref

* Fix disabling fetching orbits

* Add proc name to event exception

* update ref

* Update setup_env

* Fix offline test and update refs
old-commit-hash: c49f997be5
2022-06-30 08:39:12 -07:00
Shane Smiskol fda3609f6a process replay: clean up common code (#24855)
* regen and process replay clean up

* test_fuzzy actually uses fingerprint hardcoding

fix

* revert

* revert

* this can be a url or path so just print full variable
old-commit-hash: 170ed3d761
2022-06-14 12:03:30 -07:00
Adeeb Shihadeh 3c48bc47b7 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
old-commit-hash: 0fce5d9045
2022-06-11 23:19:27 -07:00
Lukas Petersson 35f277516e process replay: upload-only bug (#24722)
dont look for lr with upload only
old-commit-hash: 50a8b30580
2022-06-02 17:25:44 -07:00
Shane Smiskol 6e6f8422fc process replay: fix tested_cars
old-commit-hash: 6a21cc83ff
2022-06-02 14:20:19 -07:00
Shane Smiskol 490a836dd6 process replay: fix updating refs (#24716)
fix updating refs
old-commit-hash: 90af436a12
2022-06-02 13:49:21 -07:00
Lukas Petersson a16a20005b process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e
2022-06-01 22:02:42 -07:00
ClockeNessMnstr eeccd04c11 process replay: only download used logs (#24661)
* only download wanted logs

* cleanup

* cache as an option

* cleanup

* Readme

* Revert "cache as an option"

This reverts commit 060ed4ade548696505e3185fc7d20f1897072ded.

* cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 3fb22c0289
2022-06-01 21:46:45 -07:00
Shane Smiskol 222d8818d7 process replay: fix updating refs after regen (#24656)
* fix updating refs after regen

* match spacing

* same as before

* fix

* minimize diff

* update

* not used
old-commit-hash: dff70876c3
2022-05-26 00:39:33 -07:00
Lukas Petersson 81af5dacdf process replay: update regen body route (#24648)
update to a preserved route
old-commit-hash: 3e975eb3aa
2022-05-24 17:07:10 -07:00
Lukas Petersson d1e21cef79 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 444d265821
2022-05-18 19:59:30 -07:00
Shane Smiskol 28dfaa0abe process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
old-commit-hash: d686f9e527
2022-05-12 19:19:47 -07:00
Shane Smiskol 5a0778518d process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
old-commit-hash: 934bc1e6fb
2022-05-12 17:05:22 -07:00
grekiki 7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cd
2022-05-12 14:08:11 +02:00
Shane Smiskol 024f42bae5 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d2d3b7b823
2022-05-11 15:03:05 -07:00
Shane Smiskol 121b708ba1 Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
old-commit-hash: 4190d2608a
2022-05-02 13:34:31 -07:00
Shane Smiskol 754b93997a Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
old-commit-hash: c1caca104f
2022-04-28 22:37:46 -07:00
Shane Smiskol 32075d4681 safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
old-commit-hash: 66dd8f934c
2022-04-28 17:56:34 -07:00
Adeeb Shihadeh 211fc8fc68 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
old-commit-hash: 3e098c9ba5
2022-04-27 12:08:11 -07:00
Shane Smiskol ebec632ea2 Make safetyParam unsigned (#24160)
* bump panda & cereal

* make unsigned in boardd

* update health packet version

bump panda

* fix definition

* bump

* regen TOYOTA2 route

* upload route

* oh, ndog param wasn't being set

* regenerate routes with non-zero safety params

* update refs

* regen routes and update refs

* bump panda

* update refs

* update refs

* bump submodules to master
old-commit-hash: 6e62df65d7
2022-04-21 00:01:05 -07:00
Shane Smiskol 500034211d regen: check if regenerated route is enabled (#24207)
* check if regenerated route is enabled

* elif

* initialize CUDA runner before all other processes

* Revert "initialize CUDA runner before all other processes"

This reverts commit 124e3dd4d3408b6649f40b3e6e7f217ca347f59f.

* add back fixes

* fix typo
old-commit-hash: 1a5ddfaedd
2022-04-13 18:01:16 -07:00
Adeeb Shihadeh a8546e8bf9 body: set fault state (#24125)
* body: set fault state

* fix that

* update refs
old-commit-hash: 6c5b0685da
2022-04-04 23:05:40 -07:00
Adeeb Shihadeh 9e0a09868a body: only control while engaged (#24114)
* body: only control while engaged

* bump panda

* remove that

* bump panda

* update refs

* update refs
old-commit-hash: 00ebd10b87
2022-04-04 19:37:21 -07:00
Adeeb Shihadeh 78bb03345e better support non-car robots (#24106)
* better support non-car robots

* bump cereal

* update refs
old-commit-hash: e40c36f22b
2022-04-01 20:03:20 -07:00
Igor Biletskyy df806cdbeb car port: comma body (#24019)
* body FPv2

* ..

* ..

* ..

* Temp, REVERT!

* more cleanup

* typo

* ..

* del eyes

* should work?

* fix

* new dbc

* ..

* fixes

* static analysis

* cln balancing code

* no test route

* excluded_interfaces

* THE DOCS!

* comments on steer/speed mixin

* switch to bus 0

* less UDS

* FAKE bus 0 vin and fingerprint

* FAKE locationd

* Keep steady and remove handcoded offset

* Improve startup sequence, get closer to stock openpilot

* Forgot to define angle

* lowercase

* revert that

* little cleanup

* change safety model

* update refs

* body is gold

* handle no llk

* oops

* litte more

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 031f79ba88
2022-03-31 17:16:15 -07:00
Willem Melching 962201fff1 Ensure controlsd can engage again in process replay (#23556)
* hacks to make process replay engage again

* dont change that

* enable engagement check again

* chmod +x

* first working regen

* proper logMonoTime

* fix video framerate consistency

* mpc is valid again

* proper alive checks

* revert loggerd change

* ensure engaged in regen, fix managerState, peripheralState

* ubloxRaw is unused

* add submaster config for gm

* regen all services we can run without HW

* fix loggerd

* loggerd: matroska without the extension

* update ref
old-commit-hash: d3d10e014a
2022-01-21 13:13:34 +01:00
Ewout ter Hoeven f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a82
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh e0456c4989 Mazda: move CX-9 out of dashcam mode (#22717)
* move cx9 out of dashcam

* no alc below 28

* process replay route
old-commit-hash: 2252c6bba7
2021-10-31 22:02:36 -07:00
Robbe Derks 58c99158a1 Make pandaState and safetyMode a list (#22454)
* wip: move to pandaStates

* bump cereal

* wip: SafetyMode struct

* move to safetyMode

* fix typo

* this can be None

* fix potential empty pandaStates list

* fix thermald

* fix controlsd

* rename safetyModes to safetyConfigs

* update process_replay

* fix test_models

* bump cereal
old-commit-hash: 91987f38d4
2021-10-08 17:54:34 +02:00
Adeeb Shihadeh 1244074094 add TSS2 replay route (#21953)
* add TSS2 replay route

* skip on sim

* do not set on import time

* update ref

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 86f68ff22e
2021-08-31 10:48:21 -07:00
Adeeb Shihadeh bc0803868f update prius test route (#21791)
old-commit-hash: f7dc898853
2021-07-29 20:04:07 -07:00
Harald Schafer 6aca5f9f38 update
old-commit-hash: 1979005e82
2021-07-09 16:31:34 -07:00
HaraldSchafer 7e87930a20 Refactor lateral lag compensation (#21334)
* add T_IDXS

* refactor

* fix test

* unused

* typo

* needs casting

* Update selfdrive/controls/lib/drive_helpers.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* deprecate field

* regen all

* new segs

* add todo

* split back

* clean

* bad names

* do in controls

* add arg

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 6838e1c82c
2021-06-30 14:19:39 -07:00
Adeeb Shihadeh d31397b84b segment regen (#21148)
* start seg regen

* send vipc

* fix up some paths

* set fp

* fix no entries

* engages

* seperate camera procs

* send real frames

* regen test routes

* nice argparser

* fix valgrind test

* move that

* fix that
old-commit-hash: 4a1aec27ef
2021-06-28 21:44:40 -07:00
robbederks a1157cc762 Tesla AP2 port (#20074)
* squashed the PR

* remove AP passthrough

* disengage on steer override

* bump panda

* add test routes

* bump panda

* tesla is still dashcam only

* dashcam only

* rerun CI

* add to release files
old-commit-hash: cdfba105ed
2021-06-23 14:56:59 +02:00
Adeeb Shihadeh b700c1bda3 Update VW process replay route (#21142)
old-commit-hash: 7d8480cffe
2021-06-04 15:29:13 -07:00
Adeeb Shihadeh 9531bf9d88 fix process replay on local files (#20758)
old-commit-hash: c45dbbc737
2021-04-26 20:50:47 -07:00
Jason Young 8df4311821 VW MQB: Update to Audi A3 Mk3 (#20489)
* Add A3 and A3 e-tron FPv2

* Update README

* Catch up with model renaming

* Trim duplicate FW values

* Add 2014 A3 (bankster)

* Update model-year range

* Revert more model name changes

* Revert more model name changes

* More FW for 2017 A3 e-tron (fastzeek)

* Update A3 models and packages in README

* Update A3 packages again

* AUDI_A3 -> AUDI_A3_MK3

* Update test route reference

* CAN valid should pass for VW now

* Update process_replay route

* Expand tested A3 range (we have known 2014-15,17)

* Revert "CAN valid should pass for VW now"

This reverts commit 73a4b4886f572045ada05cd13e60790ca3e17a94.

* revert route changes and update refs

* update refs again

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 48fa8ffb33
2021-03-30 14:49:46 -07:00
Adeeb Shihadeh aa194a3cbb VW: update process replay route (#20239)
* VW: update process replay route

* update refs
old-commit-hash: f3f44e43e6
2021-03-04 14:58:28 -08:00
HaraldSchafer 5092214913 Different lag comp (#20024)
* more sensible

* different names for different segs

* new ref

* this is checked somewhere
old-commit-hash: c3707328f5
2021-02-05 14:01:22 -08:00
HaraldSchafer 5c2c6691b2 remove old model packet (#19769)
* remove publisher

* move to V2

* radard new model

* fix plant

* change packety

* need hack here too

* change to new

* this has been wrong all along

* no more model msg

* subscribe to new model

* not needed anymore

* make work

* need to ignore that too

* should pass tests, needs car test

* fix process replay

* no more poly

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 124100d0fa
2021-01-19 17:15:16 -08:00
Adeeb Shihadeh 77817dbd50 update VW and nissan test routes
old-commit-hash: 13ea3ccf6d
2021-01-19 14:22:50 -08:00
HaraldSchafer 61c5e9e690 Update Subaru and GM test routes (#19828)
* 2 more

* update refs

* note?

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: a4098d7bb0
2021-01-18 15:54:02 -08:00
HaraldSchafer c6d9b9565a Mpc rework2 (#19660)
* start again

* need that too

* this actually works

* not needed

* do properly

* still works

* still works

* still good

* all G without ll

* still works

* all still good

* cleanup building

* cleanup sconscript

* new lane planner

* how on earth is this silent too....

* update

* add rotation radius

* update

* pathplanner first pass

* misc fixes

* fix

* need deep_interp

* local again

* fix

* fix test

* very old

* new replay

* interp properly

* correct length

* another horrible silent bug

* like master

* fix that

* do doubles

* different delay compensation

* make robust to empty msg

* make pass with hack for now

* add some extra

* update ref for increased leg

* test cpu usage on this pr

* tiny bit faster

* purge numpy

* update ref

* not needed

* ready for merge

* try again after recompile

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 158210cde8
2021-01-14 18:43:50 -08:00
HaraldSchafer 3b911d61bc Extra test segs (#19749)
* new routes

* update refs

* new ref
old-commit-hash: 95eb5b6fcb
2021-01-13 21:56:29 -08:00