13 Commits

Author SHA1 Message Date
Kacper Rączy 338b7c915f regen: use ci bucket for files (#34766)
* Remove route meta mode

* use get_url

* Remove useless import
2025-03-04 03:10:48 +00:00
Justin Newberry 059aa6ffd9 Revert "Process Replay: move to pytest (#30260)" (#30687)
This reverts commit eb8f3f7f568aa0e294b32226ae5c5ed5e1b75e9d.
old-commit-hash: 7b5d4d031adb0e2fecf36772bbbb95e2989b9cc7
2023-12-11 14:46:56 -08:00
Justin Newberry 6af8e72604 Process Replay: move to pytest (#30260)
* process replay pytest

* enable long diff

* readd job name

* make it executable

* cleanup imports

* retrigger
old-commit-hash: 90c873ab1def0756f6f91fcb7f2d6fad503e3f49
2023-11-13 13:50:23 -08:00
Kacper Rączy bacaff6eda regen: dummy dcamera support (#30323)
* DummyFrameReader

* dummy_dcam_if_missing flag

* Blank dcamera for all neo segments, as dmonitoringd does not support neo camera feeds

* Remove trailing whitespace

* zero_dcamera staticmethod

* enable dmonitoring in test_regen

* Simplify. User decides wether to use dummy or not

* Moving some stuff

* rename to dummy_driver_cam
old-commit-hash: 7a5d85232fee1b4792f1bfcfcea38a2f57c0b0ea
2023-10-30 15:17:32 -07:00
Kacper Rączy 2878a3a876 CI: add regen job (#30157)
* Ability to whitelist/blacklist cars in regen_all

* Add CI job for regen, running on 2 segments

* Run regen_all, not regen

* Use coverage run

* Add test_regen

* Use test_regen in ci test

* Add test case names

* ONNXCPU = 1

* Add mazda segment

* Use RUN_CL

* build cl image before running

* unset PYTHONWARNINGS

* Create regen cache

* Replace daemons with processes

* Skip ford

* Skip mazda

* Add comment about commented segments

* Update selfdrive/test/process_replay/test_regen.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Remove unset pythonwarnings

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 3aa74c28fcd426f73da2201b2fdd7ac2be21ba71
2023-10-05 13:59:00 -07:00
Adeeb Shihadeh 53e0f5eb49 pytest: use a clean environment for all tests (#29788)
* pytest: use a clean environment for all tests

* rm that

* fix pj

* put build back

* fix params

* fix that

* handle no key

* that was removed

---------

Co-authored-by: Justin Newberry <justin@comma.ai>
old-commit-hash: 7b6afbc1623b8c159a92f6a5a951fb4d688045ee
2023-09-05 18:52:40 -07:00
Adeeb Shihadeh 16ec56398c add openpilot prefix to imports (#29498)
* add openpilot prefix to imports

* more

* more

* fix docs

* fix linter

* bump submodules

* fix patched tests

* update dynamic imports

* debug

* Revert "debug"

This reverts commit db5e13b9911cc74438bee123bc3430da6c31b24b.

* fix pm test
old-commit-hash: a9626f95b69af19306143fc4def02fb5769405d2
2023-08-20 20:49:55 -07:00
Vivek Aithal 51d25b2011 Live torque (#25456)
* wip torqued

* add basic logic

* setup in manager

* check sanity and publish msg

* add first order filter to outputs

* wire up controlsd, and update gains

* rename intercept to offset

* add cloudlog, live values are not updated

* fix bugs, do not reset points for now

* fix crashes

* rename to main

* fix bugs, works offline

* fix float in cereal bug

* add latacc filter

* randomly choose points, approx for iid

* add variable decay

* local param to capnp instead of dict

* verify works in replay

* use torqued output in controlsd

* use in controlsd; use points from past routes

* controlsd bugfix

* filter before updating gains, needs to be replaced

* save all points to ensure smooth transition across routes, revert friction factor to 1.5

* add filters to prevent noisy low-speed data points; improve fit sanity

* add engaged buffer

* revert lat_acc thresh

* use paramsd realtime process config

* make latacc-to-torque generic, and overrideable

* move freq to 4Hz, avoid storing in np.array, don't publish points in the message

* float instead of np

* remove constant while storing pts

* rename slope, offset to lat_accet_factor, offset

* resolve issues

* use camelcase in all capnp params

* use camelcase everywhere

* reduce latacc threshold or sanity, add car_sane todo, save points properly

* add and check tag

* write param to disk at end of route

* remove args

* rebase op, cereal

* save on exit

* restore default handler

* cpu usage check

* add to process replay

* handle reset better, reduce unnecessary computation

* always publish raw values - useful for debug

* regen routes

* update refs

* checks on cache restore

* check tuning vals too

* clean that up

* reduce cpu usage

* reduce cpu usage by 75%

* cleanup

* optimize further

* handle reset condition better, don't put points in init, use only in corolla

* bump cereal after rebasing

* update refs

* Update common/params.cc

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* remove unnecessary checks

* Update RELEASES.md

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 4fa62f146426f76c9c1c2867d9729b33ec612b59
2022-09-19 15:19:26 -07:00
Shane Smiskol 9f395d26d2 regen: send wideRoadCameraState (#24863)
* fix camera malfunction

* revert

* send in one process to fix frames out of sync

* not used

* revert

* fix and add --no-upload for CI testing

* fingerprint if source is fw

* no FW versions
old-commit-hash: 5dce006a82879f133ab69355b6c9b8c74d2b4d33
2022-06-16 22:41:13 -07:00
Shane Smiskol fda3609f6a process replay: clean up common code (#24855)
* regen and process replay clean up

* test_fuzzy actually uses fingerprint hardcoding

fix

* revert

* revert

* this can be a url or path so just print full variable
old-commit-hash: 170ed3d761a58222a626c86b9706f9be4900d058
2022-06-14 12:03:30 -07:00
Lukas Petersson 4171e45e9c process replay: regen in parallel (#24628)
* regen in parallel

* prefixes

* clean regen

* clean output

* tqdm loc

* del swp file

* add routes back

* cleanup

* disable tqdm

* unique dirs

* unique dirs

* outdir in regen_all

* formatting when played from other dirs

* prefix dongle id

* local disable_tqdm

* formatting

* bug fix

* dont spam fakedata

* 16 char fake dongle ids

* formatting

* formatting

* more descriptive dongle

* fix azure path

* couple more fixes

* handle failures nicely

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 397da56c85d4d4932d25435dcb1d57f2cd2d8c4b
2022-06-06 14:21:12 -07:00
Willem Melching 962201fff1 Ensure controlsd can engage again in process replay (#23556)
* hacks to make process replay engage again

* dont change that

* enable engagement check again

* chmod +x

* first working regen

* proper logMonoTime

* fix video framerate consistency

* mpc is valid again

* proper alive checks

* revert loggerd change

* ensure engaged in regen, fix managerState, peripheralState

* ubloxRaw is unused

* add submaster config for gm

* regen all services we can run without HW

* fix loggerd

* loggerd: matroska without the extension

* update ref
old-commit-hash: d3d10e014a69f77d3f67c8d01100b03fde8ab034
2022-01-21 13:13:34 +01:00
HaraldSchafer 7e87930a20 Refactor lateral lag compensation (#21334)
* add T_IDXS

* refactor

* fix test

* unused

* typo

* needs casting

* Update selfdrive/controls/lib/drive_helpers.py

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* deprecate field

* regen all

* new segs

* add todo

* split back

* clean

* bad names

* do in controls

* add arg

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
old-commit-hash: 6838e1c82c5abb9f347228006219d96744b3fbb5
2021-06-30 14:19:39 -07:00