Commit Graph

40 Commits

Author SHA1 Message Date
Justin Newberry e525f47141 model_replay: allow skipping modeld/dmodeld (#30365)
* skip model

* sa

---------

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: c52f2ec19b9d0cfc98aaa66b459d02f0d87116cb
2023-11-02 18:10:11 -07:00
Justin Newberry f2dcaf6f79 compare_logs: better names for short/long diffs (#30263)
* better names

* dont need that

* those too
old-commit-hash: def2b938950def9f4bd6e44d8f16a172983428a6
2023-10-16 17:04:14 -07:00
Shane Smiskol 338987a5d2 scripts: format diff when comparing logs (#30174)
* move format diff to compare_logs to pretty print

* smaller
old-commit-hash: a1ca5dc484132122b8b9ba7e2ff73da7c86bf1a2
2023-10-04 22:59:03 -07:00
Adeeb Shihadeh 7a0a5c604e model replay: add cameraOdometry (#30007)
* model replay: add back cameraOdometry coverage

* update refs

* try again

* fix cmp log
old-commit-hash: 5f318faf93d7a9acb31fa694633c3881e9e5283f
2023-09-22 17:10:36 -07:00
Adeeb Shihadeh 16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b69af19306143fc4def02fb5769405d2
2023-08-20 20:49:55 -07:00
Shane Smiskol 0a6b589f23 dmonitoringmodeld: initialize model first (#29080)
* hacky fix

* even better

* ordering

* clean up

* fix model replay

* add comment

* catch sigint
old-commit-hash: 12ed894362dd22513a21de6689a31511b791e37b
2023-07-23 09:28:22 -07:00
Kacper Rączy 1fa42d0b97 process_replay: modeld and dmonitoringmodeld support (#28463)
* vipc support. modeld and dmonitoringmodeld configs

* Update cereal

* Add vision meta

* Use vision meta in process replay

* Exclude modeld and dmonitoringmodeld from test_processes

* Refactor other pieces of code to use vision meta

* remove unnecessary imports

* Add MAX_FRAMES_PER_SEGMENT

* cameraState migration

* Move camera state migration to migration module

* model_replay refactor

* Fix lint issues

* Reduce number of vipc buffers to 2

* Update spinner message

* Update model replay ref

* Support the new log order

* Replace CameraFrameSizes with dict. Add encode_index field

* Migrate camera states based on encode indices

* Refactor process replay

* Refactor model replay

* Remove unnecessary print

* Update model replay ref

* Make camera state migration optional

* Remove unused import

* Add ignore_alive_pubs field

* Add assertion for carParams inside fingerprinting callback

* Remove WideCameraOnly param setup

* Refactor ModeldCameraSyncRcvCallback

* Update model replay ref commit

* Fix camera state migration

* Only migrate camera states when process has vision pubs

* Refactor model replay again

* Fix linter issues

* One more linter fix

* Update model replay ref commit
old-commit-hash: 2c617e0c7a082aeb3217b14e812e5d3c0c26cd4a
2023-06-22 17:36:25 -07:00
Kacper Rączy 2edd65ad4e process_replay: helpers (#28367)
* process_replay helper

* Refactor

* refactor check_enabled uses

* fix __init__

* noqa in __init__ imports

* Move compare_logs outside of process_replay

* Move save_logs to tools.lib.helpers

* Remove save_log from compare_logs

* fix lint issues
old-commit-hash: 64377a88f94527f56ba8e9cd7c01c97100681455
2023-06-06 10:31:13 -07:00
Mitchell Goff 3dc3978a10 Test navmodel (#28272)
* Test navmodel

* removed comment

* Updated model_replay ref

* read tiles from azure bucket

* updated ref commit
old-commit-hash: 2d34083cd8a507ed06f0139f496930cfe4bde820
2023-05-30 20:33:01 -07:00
Adeeb Shihadeh a4af0fca74 jenkins fixups
old-commit-hash: 5f66a9a6213f6c42161584979a2f760f9ad9df47
2023-01-10 16:20:27 -08:00
Mitchell Goff b3626c8ecd model_replay: use last N frames of segment so navRoute is valid (#26790)
* model_replay: use last N frames of segment so navRoute is valid

* Use llk logMonoTime for MapRenderState.locationMonoTime

* Record mapRenderState

* update refs

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 5ab2fc16ab5b883f3fa151d0e7362bc73545ca32
2022-12-20 14:19:21 -08:00
Adeeb Shihadeh 44098cc024 navmodel: check images in replay test (#26747)
* move position

* log images

* enable test

* update refs

* ignore

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: f3a711953afea1fcebb4c10b1e2462c4032ac147
2022-12-09 22:56:32 -08:00
Adeeb Shihadeh 01db07375c nav model replay (#26697)
* nav model replay

* set token

* fix gh actions

* update refs

* fix pc
old-commit-hash: 1da6f6a6e213280979dcf64b4d3e90f291bda4a3
2022-12-07 19:36:30 -08:00
Adeeb Shihadeh 7dac75758d model replay speedup (#26181)
* model replay speedup

* less frames too

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 188c2c1c5e1c06d68ae957ab16bcb77e29052f4b
2022-10-20 20:58:58 -07:00
HaraldSchafer a8ec6f4cfd Rocket Launcher Model (#25963)
* 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/300

* compute only 9 tokens: 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/300

* tinygrad: cleanup gather

* 1456d261-d232-4654-8885-4d9fde883894/440 6b7d7cec-ead8-40f3-86cc-86d52c9b03fe/700

* empty commit for tests

* bump tinygrad

* dont use tinygrad matmul for now

* bump tinygrad

* 1456d261-d232-4654-8885-4d9fde883894/440 e63ab895-2222-4abd-a9a5-af86bb70e260/700

* float16 1456d261-d232-4654-8885-4d9fde883894/440 e63ab895-2222-4abd-a9a5-af86bb70e260/700

* increase steer rate cost

* Revert "increase steer rate cost"

This reverts commit 74ce9ab9be7ef17ecfec931f96851b12f37f2336.

* fork tinygrad

* empty commit for tests

* basics

* Kinda works

* new lat

* new tuning

* Move LATMPCN so scons compiles

* Update long weights

* Add tinygrad optim

* Update model ref

* update weights

* Update ref

* Try

* Error message for field ignore

* update model regf

* ref commit

* Fix onnx test

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
old-commit-hash: cb0b7375b728d1b6e92db68c9ba55f0f54c09a3f
2022-10-05 21:43:38 -07:00
Adeeb Shihadeh 35ff65121e pre-commit: add codespell (#25571)
old-commit-hash: 6590fb2b93baedb2e1c5267b4f191f8e20fcd1d2
2022-08-30 11:20:55 -07:00
Shane Smiskol 64195b3c99 process replay: print correct ref path (#25448)
* fix log paths ref printing in process replay

* rm that

* this is easier to read

* fix model_replay
old-commit-hash: 4602e5a8f9e23f173635e9e00a60ab3b73deb254
2022-08-16 00:32:42 -07:00
HaraldSchafer 48522500ea Falcon Punch Model: turn cutting improvements (#25413)
* simplified change to mpc dynamics

* add jerk pts

* increase jerk cost

* increase jerk pts multipler to master value

* Add final commit

* 1456d261-d232-4654-8885-4d9fde883894/440 ac1a6744-85b0-4ec6-8ba7-608d0717b8f1/750

* some copies are useful

* update model replay ref

* less frames in model replay onnx cpu

* 1456d261-d232-4654-8885-4d9fde883894/440 264b67f5-3f52-4b58-b11f-58dd8aaf08bf/950

* 1456d261-d232-4654-8885-4d9fde883894/440 236fc556-fba3-4255-8ccf-684b22637160/950

* c9d10c64-bea4-41ec-8ca3-d8c886fda172/440 26d73dd2-862a-44ae-bbdd-32cc4f397ad7/900

* Fix couple tests

* Update ref

* Unused for now

* Add lateral factor comment

* Unused variable

Co-authored-by: nuwandavek <vivekaithal44@gmail.com>
Co-authored-by: Bruce Wayne <yassine@comma.ai>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
Co-authored-by: Bruce Wayne <batman@gpu06.internal>
old-commit-hash: 041458f632f197de3c954ed4a08650fbcb24690a
2022-08-12 00:47:59 -07:00
YassineYousfi 6709805020 Model file too big: Revert "New model: minor fixes" (#25390)
Revert "New model: minor fixes (#25368)"

This reverts commit 4e69dfab07507510eb52f532e7038b01f83471b7.
old-commit-hash: 3afe7464dfc6a356f8eed4b30e3c79a2c32a5cf0
2022-08-08 12:42:46 -07:00
HaraldSchafer f8a05617a3 New model: minor fixes (#25368)
* 1456d261-d232-4654-8885-4d9fde883894/440 ac1a6744-85b0-4ec6-8ba7-608d0717b8f1/750

* some copies are useful

* update model replay ref

* less frames in model replay onnx cpu

Co-authored-by: Bruce Wayne <yassine@comma.ai>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
old-commit-hash: 55bf9385047ec876643cdebc197fba4f05cc4695
2022-08-05 17:28:15 -07:00
HaraldSchafer e5589e572f Cleanup calibration code (#25119)
* First attempt

* worksish

* tests pass

* cleanup

* get rid of garbahe

* fix name

* Still used in xx

* add debug functions

* used

* Revert "used"

This reverts commit 276e8ebab06d2d4f0e9927ba32b7d8aca2bf88c3.

* Update ref
old-commit-hash: 772b748689cff4d6dc6aab6a8a20247b29a86056
2022-08-01 20:35:14 -07:00
Adeeb Shihadeh e53cd5ac1f model_replay: update format_diff call
old-commit-hash: ef00bf76c1db2ac85f8033e48e89213f1f846d44
2022-07-25 00:08:30 -07:00
ZwX1616 cd2a98aa11 fullframe DM model (#24860)
* Revert "put cereal on master"

This reverts commit 4646c132bae7358079c9b2867725f8382906c1e5.

* Revert "Revert fullframe DM model (#24812)"

This reverts commit 59e8af4c3101785cead69a9880cc03e0a18081e1.

* revert revert cereal

* clip6

* 0.8 is fair

* Fiction compensation should be based on error

* Update refs

* Add deadzone

* not that

* good mg

* ref

* ref

* ee8f

* minor tweak

* ref

* recompile

* ref

* cereal

* match driverstatus

* new ref

* new ref

* pass token through jenkins credentials

* quote

* fix snpe dead weights

* final ref

Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 1f2f9ea9c9dc37bdea9c6e32e4cb8f88ea0a34bf
2022-06-20 16:24:51 -07:00
Adeeb Shihadeh 3c48bc47b7 Move a bunch of stuff to system/ part 3 (#24829)
* move swaglog.py

* timezoned

* logmessaged

* version.py

* fix linter
old-commit-hash: 0fce5d90459b77bf2cfa70f55f322f0e1fb8d01c
2022-06-11 23:19:27 -07:00
Adeeb Shihadeh 9a45a3b99b Move selfdrive/hardware/ to system/ (#24725)
* move hardware to system/

* fix mypy
old-commit-hash: 1139fe507b01f34de9714c99228f411558b44231
2022-06-11 16:38:24 -07:00
ZwX1616 c73d4e5fc6 Revert fullframe DM model (#24812)
* Revert "fullframe DM: flip RHD yaw to use matching thresholds"

This reverts commit ce7daabc8847d18ba46e5d1879f5a6958d04ccc7.

* Revert "fullframe DM model (#24762)"

This reverts commit 817be81fb19004f4873881f6b29dcdfffbe7e3a8.

* revert cereal
old-commit-hash: c646eeee0ac54925db5afc51b95c5d869d6dba68
2022-06-10 16:16:46 -07:00
ZwX1616 02f0f39b52 fullframe DM model (#24762)
* get log

* simplify two nonsense

* not needed

* libyuv is a joke

* clean up

* try small

* fast but not bad

* working

* clean up driverview

* simplified

* thats mirrored

* smol

* tweak

* ref is screen

* w/ ee

* update camera model

* no if TICI

* start

* update pose thresh

* less cpu more dsp

* new libyuv

* new snpe

* add files

* test

* should be fast

* update out len

* trigger test

* use master snpe

* add cereal

* update cereal

* refactor parsing

* missing ;

* get

* wrong type

* test model

* use driver data

* 10829278-72fe-4283-a118-2cef959ce174/1550

* no pf

* adapt driverview

* ;

* rhd learner

* update libyuv buildi x64

* ad4337ea

* remove blink slack

* test

* no

* use toggle

* b16

* fix for nv12

* 5b02cff5 both

* update test

* update cereal

* update cereal

* update cereal

* v2 packets

* revert libyuv

* no /

* update snpemodel

* ;

* memcpy

* fix test

* use toggle in driverview

* update power

* update replay

* Revert "update replay"

This reverts commit 1d0979ca59dbc89bc5890656e9501e83f0556d50.

* update model ref

* halve cpu

* fake 8bit onnx runner

* same thresh as report

* cereal master

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: d6c07a6b158595c1ab10256dc6eee4fbdf902379
2022-06-08 20:13:46 -07:00
HaraldSchafer 7c4645f227 Fix onnx model replay (#24702)
* No more C2

* Add test

* Various fixups

* Use ONNXCPU

* rm prints

* LFS pull

* Use sim docker

* Try sim docker again

* Print counter

* Make sure it can fail

* Fails correctly

* comments

* Add more install to opencl base docker

* vim not needed in base
old-commit-hash: a2c6b30c982cc4b5ebea890ab7cdd937dea9b4fa
2022-06-03 10:43:09 -07:00
George Hotz 2a6ffaf71d nv12: encoderd avoids a full frame copy (#24519)
* rgb to nv12

* nv12 works (w memcpy)

* correct now

* no copy

* fix nv12 with fast debayer

* reverts of unused stuff

* ui use nv12

* comment out thumbnails for now

* rebase fix

* dm read nv12

* model read nv12

* fix ffmpeg encoder

* thumbnails from nv12

* replay to nv12

* python framereader support nv12

* remove hardcoded frame/buffer sizes

* fix build

* ffmpeg encoder fix buffers

* small cleanup

* reduce power usage test

* fix cpu usage test

* fix snapshot

* fix loggerd test

* bump cereal

Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Joost Wooning <jwooning@gmail.com>
old-commit-hash: ea5b8cdfb10686c972d8d2bc1778e6e6567e45a9
2022-06-01 17:18:28 +02:00
Adeeb Shihadeh 3e8625e50b visionipc python shim (#24660)
old-commit-hash: 32a9ccfc37d97e6692435153ac9f0fe72feb1e9f
2022-05-25 20:49:56 -07:00
grekiki 7cd23af9b1 Mypy fixes for --check-untyped-defs (#24372)
more type fixes
old-commit-hash: 17e33978cdd9322918e9d5d388f7fa47aa30e9b7
2022-05-12 14:08:11 +02:00
Adeeb Shihadeh 5a9644261f remove comma two support (#24248)
* remove comma two support

* cleanup release files

* little more

* more libs

* no more gralloc

* add snpe back
old-commit-hash: 5c48e7bc86a621dd35f4ff25da7a26ee506bac76
2022-04-18 17:55:23 -07:00
HaraldSchafer fc042cd85a Fix tici bigmodel (#23817)
* Fix bigmodel on tici: cam intrinsics arent static

* Give TICI route own name for import

* Make function useable in offline test

* typo

* change tici ecam focal to trained focal

* No shadow variable

* Fixed ref

* Force update this time

* Update model replay ref commit
old-commit-hash: 48154703a7ebad8cfea4b9cfd0e37eee132fc32f
2022-02-21 19:57:04 -08:00
Willem Melching ad5ccabce3 bigmodel (#23684)
* Added wide cam vipc client and bigmodel transform logic

* Added wide_frame to ModelState, should still work normally

* Refactored image input into addImage method, should still work normally

* Updated thneed/compile.cc

* Bigmodel, untested: 44f83118-b375-4d4c-ae12-2017124f0cf4/200

* Have to initialize extra buffer in SNPEModel

* Default paramater value in the wrong place I think

* Move USE_EXTRA to SConscript

* New model: 6c34d59a-acc3-4877-84bd-904c10745ba6/250

* move use extra check to runtime, not on C2

* this is always true

* more C2 checks

* log if frames are out of sync

* more logging on no frame

* store in pointer

* print sof

* add sync logic

* log based on sof difference as well

* keep both models

* less assumptions

* define above thneed

* typo

* simplify

* no need for second client is main is already wide

* more comments update

* no optional reference

* more logging to debug lags

* add to release files

* both defines

* New model: 6831a77f-2574-4bfb-8077-79b0972a2771/950

* Path offset no longer relevant

* Remove duplicate execute

* Moved bigmodel back to big_supercombo.dlc

* add wide vipc stream

* Tici must be tici

* Needs state too

* add wide cam support to model replay

* handle syncing better

* ugh, c2

* print that

* handle ecam lag

* skip first one

* so close

* update refs

Co-authored-by: mitchellgoffpc <mitchellgoffpc@gmail.com>
Co-authored-by: Harald Schafer <harald.the.engineer@gmail.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 85efde269da2680a77c75c248aafc8c29435ca98
2022-02-19 16:06:31 -08:00
Ewout ter Hoeven a962365292 Convert format strings strings to f-strings (#23241)
* Convert all text strings to f-strings

Reformats all the text from the old "%-formatted" and .format(...) format to the newer f-string format, as defined in PEP 498. This requires Python 3.6+.

Flynt 0.69 was used to reformat the strings. 120 f-strings were created in 51 files.

F-strings are in general more readable, concise and performant. See also: https://www.python.org/dev/peps/pep-0498/#rationale

* revert pyextra changes

* revert ublox.py

Co-authored-by: Willem Melching <willem.melching@gmail.com>
old-commit-hash: 55390d273f36f49da3896e687ac5530f40c1d150
2021-12-16 14:58:17 +01:00
Adeeb Shihadeh 24c730f083 model replay: more consistent replay (#23237)
* first cleanup

* don't send calib and desire

* should always be valid

* little more
old-commit-hash: 66ee3e9ef98f240f424228953c307be8daaf5597
2021-12-15 16:12:24 -08:00
Dean Lee 9b0fcaeb84 rename yuv streams (#23071)
* rename yuv streams

* bump cereal

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 348d2d2b0d169b0d181a51eb0e23b1f9b8fc6793
2021-11-29 18:14:59 -08:00
Willem Melching 28dd415759 Do not run code on version.py import (#23063)
* Do not run code on version.py import

* fix athena
old-commit-hash: f7c46c694985456e569ce2313b29e43607c62a12
2021-11-29 19:38:55 +01:00
ZwX1616 97175e78a1 test: Replay both models on C2/C3 (#23019)
* add dcam stream

* delete model_test

* both in jenkins

* update spinner

* get filename fn

* they are not random names

* update route

* new ref commit

* temporarily resize tmp

* clean up unlog send

* need calib 0

* revert device

* cleanup

* arg

Co-authored-by: Comma Device <device@comma.ai>
old-commit-hash: 1ec8f03ec642fb5117727eaf31df4841625e0d7b
2021-11-24 15:59:19 -08:00
Adeeb Shihadeh c7c49e8e35 fix up model tests + tools (#21071)
* unlogger: send yuv stream

* fix up model test tools

* fix unlogger

* rename model replay

* bump cereal

* test in actions

* no ci for now
old-commit-hash: cf6d133638caac0d974209f92be9c200cdbcbf3f
2021-05-28 23:08:08 -07:00