Commit Graph

68 Commits

Author SHA1 Message Date
Adeeb Shihadeh 3c48bc47b7 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
old-commit-hash: 0fce5d90459b77bf2cfa70f55f322f0e1fb8d01c
2022-06-11 23:19:27 -07:00
Lukas Petersson 35f277516e process replay: upload-only bug (#24722)
dont look for lr with upload only
old-commit-hash: 50a8b3058037351519c8bfa558527d225f0d97c6
2022-06-02 17:25:44 -07:00
Shane Smiskol 6e6f8422fc process replay: fix tested_cars
old-commit-hash: 6a21cc83ffdae067550542e8dad45ea0cc2fe2b4
2022-06-02 14:20:19 -07:00
Shane Smiskol 490a836dd6 process replay: fix updating refs (#24716)
fix updating refs
old-commit-hash: 90af436a121164a51da9fa48d093c29f738adf6a
2022-06-02 13:49:21 -07:00
Lukas Petersson a16a20005b process replay: logreader as bytes (#24610)
* willem's changes

* classmethod for bytes

* submodules

* submodules

* Update tools/lib/logreader.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* add back files

* little cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 86f73a507e2dba56bc8c3a1e753818147fb0eb0f
2022-06-01 22:02:42 -07:00
ClockeNessMnstr eeccd04c11 process replay: only download used logs (#24661)
* only download wanted logs

* cleanup

* cache as an option

* cleanup

* Readme

* Revert "cache as an option"

This reverts commit 060ed4ade548696505e3185fc7d20f1897072ded.

* cleanup

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 3fb22c0289d1c108aa0a5eb2064e7d87cc997bb9
2022-06-01 21:46:45 -07:00
Shane Smiskol 222d8818d7 process replay: fix updating refs after regen (#24656)
* fix updating refs after regen

* match spacing

* same as before

* fix

* minimize diff

* update

* not used
old-commit-hash: dff70876c3a41e66f218b3ecd9849034482139c4
2022-05-26 00:39:33 -07:00
Lukas Petersson 81af5dacdf process replay: update regen body route (#24648)
update to a preserved route
old-commit-hash: 3e975eb3aa02636aaf734f8268f23a3ba9391e04
2022-05-24 17:07:10 -07:00
Lukas Petersson d1e21cef79 process replay: support running in parallel (#24534)
* prefix params

* set env

* prefix in manager

* filesystem except

* dont delete manager folder

* Update selfdrive/common/params.h

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug same path

* remove cleanup + same default

* dont use filesystem lib

* param symlink path

* prefix helpers

* path

* dont delete d

* parallel loop

* refactor loop

* msgq

* clean msgs

* spelling

* nestable pool

* spelling

* logreaders in parallel

* bugfix

* assert msgq

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* assert in cereal

* folder exists

* create dirs in test

* bump cereal

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* PR feedback

* params path in basedir

* ref commit

* param path

* bugfix

* upload_only

* param path

* Update selfdrive/test/process_replay/process_replay.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* msgq path name

* python concurrency.features

* progress bar

* remove progress bar from compare logs

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* defaultdict

* context manager

* update refs

* dont get logs if upload only

* upload refs in parallel

* cleanup

* Update selfdrive/test/process_replay/test_processes.py

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* cleanup

* text

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Shane Smiskol <shane@smiskol.com>
old-commit-hash: 444d26582121550a371f14e17d614ba6984cc10a
2022-05-18 19:59:30 -07:00
Shane Smiskol 28dfaa0abe process replay: save logs to folder (#24509)
* save logs to folder for proc replay

* use fakedata dir

* fix

* only once and where needed
old-commit-hash: d686f9e527f235d513f856e70b60cc0f37cb5559
2022-05-12 19:19:47 -07:00
Shane Smiskol 5a0778518d process replay: don't print diff when only uploading (#24514)
when only uploading no need to print success status or update diff

don't print succeed/fail when only uploading
old-commit-hash: 934bc1e6fb2ab9b9476438b2f9aea45a4dcb5298
2022-05-12 17:05:22 -07:00
grekiki 7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cdd9322918e9d5d388f7fa47aa30e9b7
2022-05-12 14:08:11 +02:00
Shane Smiskol 024f42bae5 process replay: automatically push refs on fail (#24414)
* test failure()

* let's just change a tune here

* debug

revert

* debug

* use current commit, not ref_commit

fix

* need to figure out better place for this

* quick test

* test without upload

* temp

* use azure token

* fixes

* shouldn't need this

* debug

* debug

* not getting anything?

* does this mean nothing gets envvars?

* add azure token to docker environment variables

* quote

* move back

* clean up a bit

* more clean up

* like this sorting better

* replace flags with CI and clean up

* test FULL_TEST and minimalize diff a bit

* now test all

* revert tests comments

* remove flags

* revert this

revert this

* now make it fail

* now update ref_commit to last commit (make sure we can re-start this test if we commit before last one finishes uploading)

* fix

fix

fix

fix

* bad commit

* why is it not throwing an exception?

* debug

* URLFile returns empty bytes if using cache and remote file doesn't exist

* we always need to download anyway

* debug...

* duh, wrong file. but neither should have it

* add that back and just check explicitly

* check both

* clean up and make a diff

* stylize

* see if this is a better diff on files changed

* update refs

* revert changes

* only for owners or members

* if we have token access

* if we have token access

* if we have token access

* move up

* clean up

* revert

* move update refs to test_processes

* clean up

* update messages

* update msg

* update README and delete update_refs

* this isn't possible to reach anymore

* fix readme

* better help message

better help message

better help message

* only show basename when uploading, only if failed to find

* test diff

* fix printing old ref commit

* change to using

* update refs

* Revert "update refs"

This reverts commit 2e352a736a6de68e2c7064daa4e2e9409ce77686.

* revert

* ref refers to reference commit/logs, cur refers to current logs/commit (future ref)

* like for better

* Apply suggestions from code review

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update selfdrive/test/process_replay/test_processes.py

* every time lol

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: d2d3b7b823bac0d24e2136f7a97c3aa4c6d9ae47
2022-05-11 15:03:05 -07:00
Shane Smiskol 121b708ba1 Change safetyParam to uint16_t (#24376)
* bump panda & cereal

* bump panda

* rest of references

* regen TOYOTA with safety param 578, TOYOTA2 with 329

* regen rest of routes

* update ref

* bump cereal
old-commit-hash: 4190d2608a536e1b28fb999a72bbf5b80810743d
2022-05-02 13:34:31 -07:00
Shane Smiskol 754b93997a Support RAV4 Hybrid 2022 with stock longitudinal (#23969)
* add panda flag for toyota stock long with camera

* clean up

* Add 2022 RAV4 Hybrid from Philly

* fix wrong fw in interface, did this ever work?

* Must be a hybrid

* no radar parsing

* fix can error

* move to own platform

* generate docs

* fix

* Add 2022 Rav4 XSE Australia fingerprint parameters (#24303)

* Update values.py

Add 2022 Rav4 XSE Australia

* add commas

Co-authored-by: Shane Smiskol <shane@smiskol.com>

* bump panda

* wait, the camera doesn't even send 0x343, right?

* use a set instead, more obvious

* don't test without a parser

* bump panda

* flip panda flag

* bump panda

* add commas

* regen and update refs

* set to none by default

* revert parenthesis

* update comment

* bump panda

* regen and update refs

* add test models and update readme

* bump to master

Co-authored-by: BrettLynch123 <34538435+BrettLynch123@users.noreply.github.com>
old-commit-hash: c1caca104f8553d485334e33446bd7b8ec1fb8ca
2022-04-28 22:37:46 -07:00
Shane Smiskol 32075d4681 safety param: make uint16_t (#24362)
* only use half of the available bytes

* move to set_safety_model

* regen and update refs

* last two

* bump to master
old-commit-hash: 66dd8f934cc84cdbd3e48e7b49836ec95f82c23a
2022-04-28 17:56:34 -07:00
Adeeb Shihadeh 211fc8fc68 Toyota: add safety param flag for alt brake message (#24341)
* Toyota: add safety param flag for alt brake message

* update refs

* update refs

* bump panda
old-commit-hash: 3e098c9ba5fd8f6958f38dc500c357db34e83ffe
2022-04-27 12:08:11 -07:00
Shane Smiskol ebec632ea2 Make safetyParam unsigned (#24160)
* bump panda & cereal

* make unsigned in boardd

* update health packet version

bump panda

* fix definition

* bump

* regen TOYOTA2 route

* upload route

* oh, ndog param wasn't being set

* regenerate routes with non-zero safety params

* update refs

* regen routes and update refs

* bump panda

* update refs

* update refs

* bump submodules to master
old-commit-hash: 6e62df65d722bf2013c42514bf56cbe5c838ad63
2022-04-21 00:01:05 -07:00
Shane Smiskol 500034211d regen: check if regenerated route is enabled (#24207)
* check if regenerated route is enabled

* elif

* initialize CUDA runner before all other processes

* Revert "initialize CUDA runner before all other processes"

This reverts commit 124e3dd4d3408b6649f40b3e6e7f217ca347f59f.

* add back fixes

* fix typo
old-commit-hash: 1a5ddfaedd40d07db31a84f31e76bd5f684e3fd9
2022-04-13 18:01:16 -07:00
Adeeb Shihadeh a8546e8bf9 body: set fault state (#24125)
* body: set fault state

* fix that

* update refs
old-commit-hash: 6c5b0685da7384102b64883756716b79a5bc97a5
2022-04-04 23:05:40 -07:00
Adeeb Shihadeh 9e0a09868a body: only control while engaged (#24114)
* body: only control while engaged

* bump panda

* remove that

* bump panda

* update refs

* update refs
old-commit-hash: 00ebd10b87d6d96f4b801879c281927cab82d27c
2022-04-04 19:37:21 -07:00
Adeeb Shihadeh 78bb03345e better support non-car robots (#24106)
* better support non-car robots

* bump cereal

* update refs
old-commit-hash: e40c36f22ba93200a5fc83211005b28fca3fa728
2022-04-01 20:03:20 -07:00
Igor Biletskyy df806cdbeb car port: comma body (#24019)
* body FPv2

* ..

* ..

* ..

* Temp, REVERT!

* more cleanup

* typo

* ..

* del eyes

* should work?

* fix

* new dbc

* ..

* fixes

* static analysis

* cln balancing code

* no test route

* excluded_interfaces

* THE DOCS!

* comments on steer/speed mixin

* switch to bus 0

* less UDS

* FAKE bus 0 vin and fingerprint

* FAKE locationd

* Keep steady and remove handcoded offset

* Improve startup sequence, get closer to stock openpilot

* Forgot to define angle

* lowercase

* revert that

* little cleanup

* change safety model

* update refs

* body is gold

* handle no llk

* oops

* litte more

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 031f79ba88c58f9fbc087df0a1e2ba5ddc3f80db
2022-03-31 17:16:15 -07:00
Willem Melching 962201fff1 Ensure controlsd can engage again in process replay (#23556)
* hacks to make process replay engage again

* dont change that

* enable engagement check again

* chmod +x

* first working regen

* proper logMonoTime

* fix video framerate consistency

* mpc is valid again

* proper alive checks

* revert loggerd change

* ensure engaged in regen, fix managerState, peripheralState

* ubloxRaw is unused

* add submaster config for gm

* regen all services we can run without HW

* fix loggerd

* loggerd: matroska without the extension

* update ref
old-commit-hash: d3d10e014a69f77d3f67c8d01100b03fde8ab034
2022-01-21 13:13:34 +01:00
Ewout ter Hoeven f206ebd054 Pyupgrade 3.6: Update syntax with Python 3.6+ features (#23305)
Updated Python code with Python 3.6+ features:
- utf-8 encoding is now the default (PEP 3120)
- Replace list comprehensions by Generator Expressions (PEP 289)
- Replace yield loop by yield from (PEP 380)
- Remove the (object) subclass when defining a class
- Replace the IOError alias by OSError (PEP 3151)
- Define sets with curly braces {} instead of set()
- Remove "r" parameter from open function, which is default

Co-Authored-By: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-Authored-By: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>

Co-authored-by: Adeeb Shihadeh <8762862+adeebshihadeh@users.noreply.github.com>
Co-authored-by: GregorKikelj <96022003+GregorKikelj@users.noreply.github.com>
old-commit-hash: 332f568a8241fba9459cb70c76840b9670e6993a
2021-12-24 11:18:39 -08:00
Ewout ter Hoeven a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f36f49da3896e687ac5530f40c1d150
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh e0456c4989 Mazda: move CX-9 out of dashcam mode (#22717)
* move cx9 out of dashcam

* no alc below 28

* process replay route
old-commit-hash: 2252c6bba76c15916dcc91aa4da95fa77be65302
2021-10-31 22:02:36 -07:00
Robbe Derks 58c99158a1 Make pandaState and safetyMode a list (#22454)
* wip: move to pandaStates

* bump cereal

* wip: SafetyMode struct

* move to safetyMode

* fix typo

* this can be None

* fix potential empty pandaStates list

* fix thermald

* fix controlsd

* rename safetyModes to safetyConfigs

* update process_replay

* fix test_models

* bump cereal
old-commit-hash: 91987f38d425e2a5dc61d09571933cffb9536113
2021-10-08 17:54:34 +02:00
Adeeb Shihadeh 1244074094 add TSS2 replay route (#21953)
* add TSS2 replay route

* skip on sim

* do not set on import time

* update ref

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 86f68ff22ed184dc40bfe58761a554aea1e3995e
2021-08-31 10:48:21 -07:00
Adeeb Shihadeh bc0803868f update prius test route (#21791)
old-commit-hash: f7dc8988538eb8a7c511544975af6743854f5890
2021-07-29 20:04:07 -07:00
Harald Schafer 6aca5f9f38 update
old-commit-hash: 1979005e8215990541170aaf6e1ae8c9f9cdbdbb
2021-07-09 16:31:34 -07:00
HaraldSchafer 7e87930a20 Refactor lateral lag compensation (#21334)
* add T_IDXS

* refactor

* fix test

* unused

* typo

* needs casting

* Update selfdrive/controls/lib/drive_helpers.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* deprecate field

* regen all

* new segs

* add todo

* split back

* clean

* bad names

* do in controls

* add arg

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 6838e1c82c5abb9f347228006219d96744b3fbb5
2021-06-30 14:19:39 -07:00
Adeeb Shihadeh d31397b84b segment regen (#21148)
* start seg regen

* send vipc

* fix up some paths

* set fp

* fix no entries

* engages

* seperate camera procs

* send real frames

* regen test routes

* nice argparser

* fix valgrind test

* move that

* fix that
old-commit-hash: 4a1aec27efb48bdba1d267c86ef66c8a044c4986
2021-06-28 21:44:40 -07:00
robbederks a1157cc762 Tesla AP2 port (#20074)
* squashed the PR

* remove AP passthrough

* disengage on steer override

* bump panda

* add test routes

* bump panda

* tesla is still dashcam only

* dashcam only

* rerun CI

* add to release files
old-commit-hash: cdfba105ed86ec37bd41565cb95796d653ee53bc
2021-06-23 14:56:59 +02:00
Adeeb Shihadeh b700c1bda3 Update VW process replay route (#21142)
old-commit-hash: 7d8480cffec40f899ac78fab7e96cb74ccf024e1
2021-06-04 15:29:13 -07:00
Adeeb Shihadeh 9531bf9d88 fix process replay on local files (#20758)
old-commit-hash: c45dbbc737c95a68de3b1f9c1bc6b1e08a23cb36
2021-04-26 20:50:47 -07:00
Jason Young 8df4311821 VW MQB: Update to Audi A3 Mk3 (#20489)
* Add A3 and A3 e-tron FPv2

* Update README

* Catch up with model renaming

* Trim duplicate FW values

* Add 2014 A3 (bankster)

* Update model-year range

* Revert more model name changes

* Revert more model name changes

* More FW for 2017 A3 e-tron (fastzeek)

* Update A3 models and packages in README

* Update A3 packages again

* AUDI_A3 -> AUDI_A3_MK3

* Update test route reference

* CAN valid should pass for VW now

* Update process_replay route

* Expand tested A3 range (we have known 2014-15,17)

* Revert "CAN valid should pass for VW now"

This reverts commit 73a4b4886f572045ada05cd13e60790ca3e17a94.

* revert route changes and update refs

* update refs again

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 48fa8ffb33dbc49d441519f3d496441a65e8a066
2021-03-30 14:49:46 -07:00
Adeeb Shihadeh aa194a3cbb VW: update process replay route (#20239)
* VW: update process replay route

* update refs
old-commit-hash: f3f44e43e6c8fc3b70011a2364730d5f5b43f94c
2021-03-04 14:58:28 -08:00
HaraldSchafer 5092214913 Different lag comp (#20024)
* more sensible

* different names for different segs

* new ref

* this is checked somewhere
old-commit-hash: c3707328f51c4cec051b375a195b41d7456d27a6
2021-02-05 14:01:22 -08:00
HaraldSchafer 5c2c6691b2 remove old model packet (#19769)
* remove publisher

* move to V2

* radard new model

* fix plant

* change packety

* need hack here too

* change to new

* this has been wrong all along

* no more model msg

* subscribe to new model

* not needed anymore

* make work

* need to ignore that too

* should pass tests, needs car test

* fix process replay

* no more poly

* update refs

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 124100d0fa42485edaf86cb25f77f27e8917eed1
2021-01-19 17:15:16 -08:00
Adeeb Shihadeh 77817dbd50 update VW and nissan test routes
old-commit-hash: 13ea3ccf6dd2108d16e2d520c8e1b3bf9bf7851d
2021-01-19 14:22:50 -08:00
HaraldSchafer 61c5e9e690 Update Subaru and GM test routes (#19828)
* 2 more

* update refs

* note?

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: a4098d7bb0a0f82bad0c4ab78bade8eed413e6d7
2021-01-18 15:54:02 -08:00
HaraldSchafer c6d9b9565a Mpc rework2 (#19660)
* start again

* need that too

* this actually works

* not needed

* do properly

* still works

* still works

* still good

* all G without ll

* still works

* all still good

* cleanup building

* cleanup sconscript

* new lane planner

* how on earth is this silent too....

* update

* add rotation radius

* update

* pathplanner first pass

* misc fixes

* fix

* need deep_interp

* local again

* fix

* fix test

* very old

* new replay

* interp properly

* correct length

* another horrible silent bug

* like master

* fix that

* do doubles

* different delay compensation

* make robust to empty msg

* make pass with hack for now

* add some extra

* update ref for increased leg

* test cpu usage on this pr

* tiny bit faster

* purge numpy

* update ref

* not needed

* ready for merge

* try again after recompile

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 158210cde8e689daa04bcaa1e502727cf7bfddb6
2021-01-14 18:43:50 -08:00
HaraldSchafer 3b911d61bc Extra test segs (#19749)
* new routes

* update refs

* new ref
old-commit-hash: 95eb5b6fcb17b89844f9f880b33f9be22d4ec7df
2021-01-13 21:56:29 -08:00
HaraldSchafer 9c8660d0e2 Test segs 08 (#19701)
* start updating routes

* update that too

* new ref commit

* try again
old-commit-hash: 307ba858db376003f9bad3ed0db9786d84a2adff
2021-01-08 16:39:47 -08:00
Adeeb Shihadeh c64a8356fe move to github container registry + CI speedup (#19621)
old-commit-hash: 59d7c5e64ac3c8666c54c3443f912e4e0db92eb9
2020-12-31 14:57:01 -08:00
Adeeb Shihadeh 6e078827b3 check for safety mode mismatch (#2443)
old-commit-hash: 89d02dcf97d60585f9c627f11fdd75387d632b83
2020-11-04 11:12:15 -08:00
Adeeb Shihadeh 68b4380545 update process replay test routes (#2482)
* update bosch route

* update rfs

* bump refs
old-commit-hash: 37959207502c366d783972243a0d02f6ba0a91df
2020-11-03 18:44:35 -08:00
Adeeb Shihadeh e94809f5bd add cars with LQR and INDI lat control to process replay (#2363)
* add cars that use LQR and INDI lat control to process replay

* bump ref commit
old-commit-hash: 516da927f9a1f04be4cbc281bc1b3c73092a4a36
2020-10-19 21:18:50 -07:00
Adeeb Shihadeh a58dc92f20 paramsd process replay test (#2118)
* paramsd process replay test

* cleanup
old-commit-hash: 1e5a935ff00916efe65046f50d46a77d091c2e29
2020-09-01 18:31:26 -07:00