Compare commits

...

269 Commits

Author SHA1 Message Date
James Vecellio-Grant 7a3550358e modeld_v2: create a compiled pkl -> gpu kernel artifact converter (#2053) 2026-10-17 13:35:54 -04:00
James Vecellio-Grant 7df636ac01 modeld_v2: compile models to plain gpu memory artifact (#2050) 2026-10-17 13:31:04 -04:00
Jason Wen ca0202c972 Merge commit '35e78a9d9518855ec0dacec919e719624a28131f' into sync-20261016
# Conflicts:
#	.gitattributes
#	.gitignore
#	.gitmodules
#	opendbc_repo
#	openpilot/common/params_keys.h
#	openpilot/common/version.py
#	openpilot/selfdrive/car/card.py
#	openpilot/selfdrive/controls/lib/longitudinal_planner.py
#	openpilot/selfdrive/controls/plannerd.py
#	openpilot/selfdrive/controls/tests/test_latcontrol.py
#	openpilot/selfdrive/modeld/modeld.py
#	openpilot/selfdrive/pandad/pandad.py
#	openpilot/selfdrive/selfdrived/alerts_offroad.json
#	openpilot/selfdrive/selfdrived/selfdrived.py
#	openpilot/selfdrive/test/longitudinal_maneuvers/plant.py
#	openpilot/selfdrive/ui/layouts/settings/developer.py
#	openpilot/selfdrive/ui/layouts/settings/toggles.py
#	openpilot/selfdrive/ui/mici/layouts/home.py
#	openpilot/selfdrive/ui/mici/layouts/onboarding.py
#	openpilot/selfdrive/ui/mici/onroad/model_renderer.py
#	openpilot/selfdrive/ui/onroad/model_renderer.py
#	openpilot/selfdrive/ui/soundd.py
#	openpilot/selfdrive/ui/translations/app.pot
#	openpilot/selfdrive/ui/translations/app_de.po
#	openpilot/selfdrive/ui/translations/app_en.po
#	openpilot/selfdrive/ui/translations/app_es.po
#	openpilot/selfdrive/ui/translations/app_fr.po
#	openpilot/selfdrive/ui/translations/app_ja.po
#	openpilot/selfdrive/ui/translations/app_ko.po
#	openpilot/selfdrive/ui/translations/app_pt-BR.po
#	openpilot/selfdrive/ui/translations/app_th.po
#	openpilot/selfdrive/ui/translations/app_tr.po
#	openpilot/selfdrive/ui/translations/app_uk.po
#	openpilot/selfdrive/ui/translations/app_zh-CHS.po
#	openpilot/selfdrive/ui/translations/app_zh-CHT.po
#	openpilot/selfdrive/ui/ui_state.py
#	openpilot/system/hardware/chestnut/status.py
#	openpilot/system/manager/process_config.py
#	openpilot/system/ui/README.md
#	panda
#	tinygrad_repo
2026-10-17 12:38:37 -04:00
Adeeb Shihadeh 35e78a9d95 show offroad alert for ignored uploads (#39104) 2026-10-16 19:19:47 -07:00
royjr 27c132984d bug: fix duplicate MADS no-entry alerts (#1949)
* alert-dedup

* Update selfdrived.py

* Update selfdrived.py

* one liner

---------

Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-10-16 12:56:24 -04:00
Harald Schäfer 0d57eadd74 Remove obsolete Chill mode code (#39097)
* Remove obsolete Chill mode code

* Remove obsolete Chill mode test variants

* Use measured ego motion for stock ACC planning

* Restore roadmap entry

* Preserve stock ACC path colors and camera view

* remove allowThrottle

* Deprecate unused experimentalMode field

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2026-10-16 09:47:48 -07:00
royjr f20d9ea5e6 chestnut: show icon with mads engagement (#1980)
* ui engaged

* try simple

* Revert "try simple"

This reverts commit b951385080.

* move to sp

* Update opendbc_repo

* Revert "Update opendbc_repo"

This reverts commit d3d8a351ea.

* Update opendbc_repo

* Update opendbc_repo
2026-10-16 12:33:43 -04:00
Kumar 5ec0c02195 Sunset Dynamic Experimental Mode (DEC) (#2056) 2026-10-16 07:03:15 -07:00
Shane Smiskol 90932d9b5c releases tweaks 2026-10-15 21:21:28 -07:00
Shane Smiskol fcf4b0d395 Initial cruise set speed scroller ui (#39098)
* Revert " A New Dawn"

This reverts commit f50108b4af.

* Revert "Add configurable initial max speed (#39081)"

This reverts commit 24c26487f4.

* add back some stuff and new ui

* exp mode removal again

* restore tici

* rm exp mode and add new designs, fix color bug for tici

* dedup

* more dedup

* round values when toggling between metric!

* nick design tm

* inlining

* simplification

* fix

* cmts

* clr

* clr

* remove taparea

* rm

* rm

* clean up
2026-10-15 20:51:41 -07:00
Harald Schäfer f5bcd10a2a Fix typo in Autospeed feature name 2026-10-15 16:54:10 -07:00
Harald Schäfer cd3d5b0057 Update RELEASES.md for version 0.11.3 changes
Added details about the new auto-speed feature in version 0.11.3, including its capabilities and transition from beta mode.
2026-10-15 16:52:13 -07:00
Shane Smiskol 4429ec0c8c update translations (#39101) 2026-10-15 16:04:04 -07:00
Shane Smiskol 41acce17fa Rename to following distance (#39100)
rename
2026-10-15 15:59:39 -07:00
James Vecellio-Grant f7bc31f208 ci: no hook (#2055) 2026-10-15 07:41:21 -07:00
Shane Smiskol 75b2ccca60 Minor initial set speed fixes (#39099)
* fix bugs

* cmt
2026-10-15 01:22:01 -07:00
Harald Schäfer f50108b4af A New Dawn 2026-10-14 21:33:00 -07:00
Adeeb Shihadeh fe7ccf9587 Ratekeeper: resync after a stall instead of running catch-up frames (#39095) 2026-10-14 17:11:30 -07:00
Harald Schäfer 24c26487f4 Add configurable initial max speed (#39081)
* Add configurable initial cruise speed

* Use five-unit steps for initial max speed

* Adjust initial max speed directly in settings

* Limit initial max speed to supported cars and a 25 mph minimum

* scroller

* no spring

* Revert "no spring"

This reverts commit bf104344a6b93a9b4c574da6e049657f5767d227.

* Revert "scroller"

This reverts commit e4255e895edfd46d06676f3048eeeae2a781d689.

* Clarify initial cruise speed parameter units

* Simplify initial max speed controls

* Repeat initial max speed adjustments while held

* Snap initial cruise speed adjustments to five-unit steps

* Clean up initial cruise speed initialization and repeat callback

* Update toggles.py

* Fix initial max speed setting description

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2026-10-14 16:45:37 -07:00
Harald Schäfer 6eb63e0454 pigeond: set minimum satellite elevation to 0 degrees (#39083)
Co-authored-by: haraschax <6804392+haraschax@users.noreply.github.com>
2026-10-14 16:27:29 -07:00
Shane Smiskol 690fa743b9 ui: check raylib version for keyboard gradient (#39036) 2026-10-14 15:06:26 -07:00
Daniel Koepping 4f3626dc9b bump to 0.11.3 (#39093) 2026-10-14 14:27:52 -07:00
Harald Schäfer 1e219d7785 MD v3 (#39087)
1365126a-e71e-4778-be00-8ab92dc82c09/12864

Co-authored-by: haraschax <6804392+haraschax@users.noreply.github.com>
2026-10-14 13:43:30 -07:00
Daniel Koepping 6939d172e8 modeld: retry big model loading once (#39086)
modeld: retry big model load once with an onroad cycle
2026-10-14 11:54:07 -07:00
Daniel Koepping 66752e769f bump panda (#39092) 2026-10-14 11:53:36 -07:00
Daniel Koepping 21edbbcfb9 bump tinygrad (#39085) 2026-10-14 11:53:15 -07:00
Harald Schäfer 2b09bc94a9 modeld: seed previous curvature when lateral control is inactive (#39091)
* modeld: seed previous curvature when lateral control is inactive

* modeld: reuse current curvature from carControl

* Revert "modeld: reuse current curvature from carControl"

This reverts commit 2f63062fd1b70e6e6f128b320bc7a8265a7d36e7.
2026-10-14 10:20:22 -07:00
Adeeb Shihadeh cb1d6f11a7 pandad: 1s wait for panda to come up (#39090) 2026-10-14 09:32:32 -07:00
Adeeb Shihadeh d3f0aab96f pandad: simplify flashing loop (#39089) 2026-10-14 08:47:35 -07:00
Adeeb Shihadeh 50319fa702 simple SPI (#39076) 2026-10-14 08:19:41 -07:00
James Vecellio-Grant 9bebe42f73 default models: fix big (#2046)
* fix big

* whoops
2026-10-13 23:45:39 -04:00
Harald Schäfer 1623441a9f pigeond: aid startup with last GPS position (#39079)
* pigeond: aid startup with last GPS position

* pigeond: aid position after receiver setup

* pigeond: fixed 100km position accuracy, skip stale positions

* store LastGPSPosition as capnp bytes

* ubloxd: save LastGPSPosition

* pigeond: send time and position once system time is valid

* pigeond: use datetime for saved position age

* pigeond: remove position accuracy comment

* pigeond: send position assistance during initialization

* Update ubloxd.py

---------

Co-authored-by: haraschax <6804392+haraschax@users.noreply.github.com>
2026-10-13 16:56:59 -07:00
Adeeb Shihadeh 675d38213e ssh.py: try local route first (#39082) 2026-10-13 16:29:53 -07:00
DimitriChrysafis a47d1c56f6 remove unused code (#39080) 2026-10-13 11:55:22 -07:00
Harald Schäfer 4c53557764 pigeond: overwrite cached AssistNow time with host UTC (#39071)
* pigeond: fix stale AssistNow time assistance

* pigeond: save GNSS state after stopping receiver

* pigeond: preserve fractional UTC assistance

* Revert "pigeond: save GNSS state after stopping receiver"

This reverts commit 5bf5ea84d8a995f7561a58bdf6d071cc78759703.

* pigeond: send current time after AssistNow assistance

* pigeond: simplify AssistNow initialization filter

* pigeond: send host time only with AssistNow

* pigeond: check host time validity inside send_time

* Update pigeond.py

---------

Co-authored-by: haraschax <6804392+haraschax@users.noreply.github.com>
2026-10-13 11:45:14 -07:00
Daniel Koepping afcbb171f6 chestnut: flash before going onroad (#39078) 2026-10-13 11:25:55 -07:00
Harald Schäfer 42ccc5cebd Remove experimental mode initial cruise speed override (#39077) 2026-10-13 09:55:11 -07:00
Adeeb Shihadeh 2713630e79 Remove car mass (#39072) 2026-10-12 20:49:45 -07:00
Adeeb Shihadeh 4c362ca327 nav alpha (#39066)
* nav is back

* start cleanup

* prefetch all tiles

* one map

* debug

* lil more

* lil more
2026-10-11 17:49:50 -07:00
Warren 6f4c0faf1f Mountain Dew Model (#39047)
* big model: desires-final (ea1600cb/12864)

* big model: mountain dew (870a4823/1284)

* big model: MD retrained (8d736683/12864)

* rebase and recompile

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
Co-authored-by: YassineYousfi <yyousfi1@binghamton.edu>
2026-10-10 11:07:36 -07:00
Daniel Koepping 64be6c80d3 fix downgrade to last release (#39063)
* fix downgrade to releases before the openpilot/ move

* trigger ci
2026-10-10 09:47:34 -07:00
Daniel Koepping 0679ff36c6 chestnut: cut VBUS for fallback test (#39062)
cut VBUS for fallback test
2026-10-10 09:44:30 -07:00
Kacper Rączy b9c815d56a tools: use log migration in cpp tools (#38571)
* Universal PyProcess module runner. Migration cli

* print to stderr

* Migration in jotpluggler

* Do -

* Improvements

* Explicit stdout

* Existing master pyprocess changes

* PyTools

* Fix runtime warning
2026-10-10 01:42:59 +00:00
Harald Schäfer 263e8d1966 longitudinal planner: set A_CRUISE_MIN to -0.5 (#39060) 2026-10-09 15:32:15 -07:00
YassineYousfi 655bfdebc7 model replay pr bot: show both (#39055)
* model replay pr bot: show both

* jenkins
2026-10-08 21:39:24 -07:00
Daniel Koepping 0e5718116a bump opendbc (#39057) 2026-10-08 20:38:08 -07:00
Daniel Koepping 043759da36 add chestnut model fallback test (#39058)
* test_onroad: chestnut usb unplug fallback

* test_onroad: fix chestnut unplug test

* test_onroad: chestnut fallback

* check that small model is running

* fix small model check
2026-10-08 20:38:06 -07:00
Daniel Koepping d05c2d971b bump tinygrad (#39054)
* bump tinygrad

* Revert "modeld: drop chestnut on fallback (#39044)"

* rebuild big models
2026-10-08 19:26:56 -07:00
stef 9b585d58f5 network: force metered on comma prime webbing sim (#38756)
* force metered on comma prime webbing sim

* rearrange

---------

Co-authored-by: Daniel Koepping <elkoled@gmail.com>
2026-10-08 19:19:05 -07:00
Daniel Koepping 948ad05c6c chestnut: fix offroad alert (#39056)
chestnut: fix offroad alert
2026-10-08 19:18:57 -07:00
Adeeb Shihadeh f804d3468e jp: zoom improvements 2026-10-08 13:54:23 -07:00
Adeeb Shihadeh 6846d733d4 jp: add segment and duration 2026-10-08 13:16:38 -07:00
Daniel Koepping 28917bb3ab soundd: louder warning sounds (#39052) 2026-10-08 11:15:34 -07:00
Daniel Koepping 8d1865bdc6 ui: sync gpu with offroad state (#39053)
ui: show chestnut icon before selfdriveState
2026-10-08 11:13:52 -07:00
Daniel Koepping a742df6182 chestnut: bump fw (#39050)
bump chestnut fw
2026-10-07 19:24:08 -07:00
Daniel Koepping f89ce23f9d chestnut: align alerts (#39051)
chestnut: reword alerts
2026-10-07 19:23:52 -07:00
Daniel Koepping 3f06ed3bc7 chestnut: show switch branch alert only on release (#39049)
chestnut: only show branch alert on release branches
2026-10-07 15:13:02 -07:00
Harald Schäfer f21bfc36be Enable Experimental Mode by default without confirmation (#39043)
* Enable Experimental Mode by default without confirmation

* Update toggles.py

* Fix formatting in End-to-End Longitudinal Control description
2026-10-07 07:59:43 -07:00
Adeeb Shihadeh 657d3a4f7c Fix Athena test port collision (#39048) 2026-10-06 23:43:16 -07:00
Warren f59056e5b6 Revert "Mountain Dew Model" (#39046)
Revert "Mountain Dew Model (#39045)"

This reverts commit 0e0c7c711d.
2026-10-06 19:34:21 -07:00
Warren 0e0c7c711d Mountain Dew Model (#39045)
big model: desires-final (ea1600cb/12864)
2026-10-06 19:27:13 -07:00
Daniel Koepping a0d47bcd08 modeld: drop chestnut on fallback (#39044) 2026-10-06 18:43:50 -07:00
Daniel Koepping 035a447d1f ui: longitudinal policy indicator (#39040)
* ui: mici longitudinal icons

* ui: rm orange state

* ui: lead policy indicator

* fix

* show on op long

* make own widget

* sync long indicator with dmoji

* fast fade in with dmoji
2026-10-06 15:51:25 -07:00
stef 5b2a59d288 ui: bookmark fill (#38631)
* bookmark fill

* fade and sticky click

* fade out animation

* no funny animation

* clean

---------

Co-authored-by: Daniel Koepping <elkoled@gmail.com>
2026-10-06 15:42:39 -07:00
James Vecellio-Grant 3323aafb54 Sync: commaai/openpilot:master -> sunnypilot/sunnypilot:master (#2036) 2026-10-06 13:32:30 -07:00
Harald Schäfer ab8c84c398 Gas override boost (#39015)
* Add gas override boost, mici boost bar, and Rivian maximum set speed

* Refactor acceleration boost and gate accumulation on model limit

* Fade acceleration boost with model acceleration

* Limit boost buildup per gas override

* Latch boost eligibility before gas override

* Move model limiting check into AccelBoost

* Use DT_MDL directly in AccelBoost

* Clarify acceleration boost state names

* Scale acceleration boost by speed instead of decaying it

* Enable Experimental Mode by default

* Scale boost buildup with the application speed ramp

* Fix boost serialization and model-limit eligibility

* Remove boost UI and Experimental Mode default changes

* Combine boost update and application
2026-10-06 12:01:13 -07:00
Adeeb Shihadeh c933446486 jp: fix file resolving 2026-10-05 19:55:43 -07:00
Adeeb Shihadeh 2b7c34a5c3 jp: show text fields (#39042) 2026-10-05 18:42:04 -07:00
Adeeb Shihadeh fc61640782 jotpluggler: fix subset time bounds (#39041) 2026-10-05 18:31:10 -07:00
Adeeb Shihadeh 84f9bdbc10 jp: add initData tab (#39039) 2026-10-05 18:19:04 -07:00
Adeeb Shihadeh cce54f20d5 deprecate unused initData fields 2026-10-05 18:05:05 -07:00
Harald Schäfer 4bcf73282e ResAction (#39037)
* 7a933c1c-b3ba-4b31-be45-d2a03ca9fa9d/12864

* compiled
2026-10-05 17:58:45 -07:00
Daniel Koepping 910c08e082 Revert #38464 (#39038)
* Revert #38464

Reverts aa0ac919ba.
2026-10-05 15:31:58 -07:00
Shane Smiskol 01a670d6b2 ui: fix keyboard text overflow gradient 2026-10-05 12:04:16 -07:00
Harald Schäfer ec95db3f1f Bump opendbc (#39035) 2026-10-02 18:02:42 -07:00
Daniel Koepping 78dccf0b48 ui: lead bar (#39033)
* ui: lead bar

* cleanup

* add width

* show on brake
2026-10-01 16:19:08 -07:00
Daniel Koepping 7ee974ab5b fix low speed curvature saturation alert (#39034)
fix curvature saturation alert
2026-10-01 12:03:32 -07:00
discountchubbs 16322aef16 models: per model tg ref 2026-10-01 08:44:25 -07:00
Robbe Derks d2c0ce91ad pandad: log NMI and hardfault resets (#38737) 2026-10-01 13:43:30 +02:00
discountchubbs 5b75e1709c Sync: commaai/openpilot:master -> sunnypilot/sunnypilot:master 2026-09-30 12:59:15 -07:00
Daniel Koepping c8fb906815 build nightly-chestnut-dev (#39031) 2026-09-28 19:18:41 -07:00
Daniel Koepping 0c464e5c66 rm chestnut uncompiled alert (#39030) 2026-09-28 18:24:41 -07:00
Daniel Koepping 134517ceb3 AGNOS 19.9 (#39026) 2026-09-28 16:21:29 -07:00
Adeeb Shihadeh 49bbba371c add color to modeld warning print 2026-09-25 16:15:25 -07:00
Adeeb Shihadeh f474f05928 modeld: fix failed build on PCs without LLVM (#39019) 2026-09-25 16:12:28 -07:00
Daniel Koepping b78331e042 chestnut: offroad alert for model error (#39018)
chestnut: offroad alert for big model errors
2026-09-25 15:29:22 -07:00
Amy Jeanes 46b97e6195 ui: match tizi wide/narrow cam speed gates to mici (#39016)
ui: match the comma 3X road camera speed gates to the comma 4

Experimental Mode switches the road view to the wide camera below 5 m/s (11 mph) and back to the narrow camera above 10 m/s (22 mph) on both devices. Also corrects the comma 4's mph comments (were 10 and 25 mph).
2026-09-25 11:22:22 -07:00
Daniel Koepping 027770d49c modeld: fix 3s chestnut timeout (#39012)
modeld: restore 3s chestnut timeout
2026-09-24 14:59:23 -07:00
Daniel Koepping bbabd94c11 modeld: wait for chestnut to enumerate (#39011)
* modeld: wait for chestnut to enumerate

* trigger ci
2026-09-24 13:24:58 -07:00
Daniel Koepping f2c772ff11 hardwared: log usb-c connected (#39010)
hardwared: log usb cable connected
2026-09-24 12:19:05 -07:00
Daniel Koepping afa47035ba modeld: check model.chestnut in fallback (#39008) 2026-09-23 16:36:47 -07:00
Daniel Koepping c5543b1a05 modeld: set realtime scheduling after model load (#39007)
modeld: defer realtime scheduling until models are loaded
2026-09-23 15:13:47 -07:00
Daniel Koepping 4dbdd66e0c fix AGNOS updater keyboard crash (#39005)
* fix AGNOS updater keyboard across raylib versions

* ui: handle raylib 5 keyboard argument errors
2026-09-23 10:43:03 -07:00
Daniel Koepping f00d226d39 add max critical alert sound (#39006)
Add max critical alert sound after eight seconds
2026-09-22 19:24:51 -07:00
Shane Smiskol a86351c3ef process replay: new Y route (#39000)
* new

* PIPE?!

* bump

---------

Co-authored-by: Daniel Koepping <elkoled@gmail.com>
2026-09-22 17:27:51 -07:00
Daniel Koepping 8377c40da1 process replay: push new refs (#39004) 2026-09-22 17:25:29 -07:00
Shane Smiskol 841076f6d1 Revert "process replay: new Y route" (#38999)
Revert "process replay: new Y route (#38998)"

This reverts commit 27fa4fca8e.
2026-09-22 16:21:36 -07:00
Shane Smiskol 27fa4fca8e process replay: new Y route (#38998)
* new

* PIPE?!

* bump
2026-09-22 16:16:21 -07:00
Shane Smiskol 93eec90077 Tesla: auto-detect 3-bit steering control type (#38997)
bump
2026-09-22 15:24:40 -07:00
Harald Schäfer e1862e2e17 Bump opendbc and panda for Rivian alpha longitudinal (#38995)
* Bump opendbc and panda for Rivian alpha longitudinal

* Bump opendbc to include Tesla reverts

---------

Co-authored-by: Shane Smiskol <shane@smiskol.com>
2026-09-22 15:21:09 -07:00
Shane Smiskol ec9241f296 Bump submodules (#38996)
* bump opendbc

* bump panda
2026-09-22 15:16:09 -07:00
Harald Schäfer 66fd3d1bd4 Ship precompiled Chestnut camera warps (#38991) 2026-09-22 10:02:10 -07:00
Harald Schäfer 082c43da39 Chestnut: default GPU power limit to 100W (#38992)
modeld: default Chestnut GPU power limit to 100W
2026-09-22 10:01:32 -07:00
Harald Schäfer 521db4c825 locationd: reduce initial gyro bias uncertainty (#38981) 2026-09-20 18:49:17 -07:00
Trey Moen 3b2a75a4e0 cabana: resize heatmap and prevent signal clipping (#38975)
* cabana: allow heatmap resizing without clipping

* cabana: keep outer grid lines inside the heatmap

* cabana: add a visible heatmap resize handle

* cabana: use the standard pane divider for resizing
2026-09-19 18:41:58 -07:00
Trey Moen 1b1b60ba93 cabana: generate dbc files during builds (#38974) 2026-09-19 16:54:47 -07:00
Trey Moen 95ed0213ec cabana: refine signal heatmap grid (#38964)
* cabana: refine signal heatmap grid

* cabana: restore direct signal color blending

* cabana: strengthen heatmap signal saturation

* Revert "cabana: strengthen heatmap signal saturation"

This reverts commit 07ee9664d533f260b5bbfaed1afc4391c795603c.

* cabana: remove signal cell gradients
2026-09-19 16:00:59 -07:00
Trey Moen ea0afb67a8 cabana: preserve activity with a brighter baseline (#38971)
* cabana: preserve activity above a brighter baseline

* cabana: apply signal baseline in both themes
2026-09-19 15:24:14 -07:00
Trey Moen 680aa6f868 cabana: revert quiet signal brightness clamp (#38970)
Revert "cabana: brighten quiet signals in dark mode (#38969)"

This reverts commit d375e7ff5d.
2026-09-19 15:02:54 -07:00
Trey Moen d375e7ff5d cabana: brighten quiet signals in dark mode (#38969) 2026-09-19 14:58:18 -07:00
Trey Moen 2eee697f7d cabana: improve theme contrast (#38965)
* cabana: centralize accessible theme and data colors

* cabana: remove color contrast tests

* cabana: simplify theme color helpers

* cabana: soften inactive message rows in dark mode

* cabana: retain original signal tooltip behavior
2026-09-19 09:47:49 -07:00
ZwX1616 5ae0da0e46 DM: Super Leicht Model (#38942) 2026-09-18 17:35:52 -07:00
stef a39561058f ui: allow delay of scroller start (#38958)
* delay scroller after pairing QR code

* clean
2026-09-18 14:58:48 -07:00
stef 8abd133637 offroad alerts: pairing notification (#38948)
* Show pairing provider and Prime status in device settings

* remove unnecessary

* simplify

* bespoke human code

* clean

* readme

* prime layout

* sepearte qr code stuff

* move icons

* move qr code into its own widget

* clean

* Fix prime menu lint

* Update widget leak test for prime menu

* fix lint

* update to new theme

* change names

* fix font size

* get rid of subtitle

* adjust commacare icon

* drop manage, its cleaner

* edit text

* clean

* for github and apple always return generic

* pairing notification

* update copy and remove duplicates

* don't write too often

* remove

* make sure it doesn't show up on 3X

* reorg

* remove redundancies
2026-09-18 14:44:45 -07:00
Daniel Koepping e691acbf34 chestnut stresstest (#38858)
chestnut stress test
2026-09-18 11:21:39 -07:00
Trey Moen 7f6f13997c replay: fix ranges with omitted start (#38963)
* replay: fix ranges with omitted start

* replay: keep range tests scoped to omitted start
2026-09-18 06:44:39 -07:00
stef d06711be19 ui: prime menu (#38860)
* Show pairing provider and Prime status in device settings

* remove unnecessary

* simplify

* bespoke human code

* clean

* readme

* prime layout

* sepearte qr code stuff

* move icons

* move qr code into its own widget

* clean

* Fix prime menu lint

* Update widget leak test for prime menu

* fix lint

* update to new theme

* change names

* fix font size

* get rid of subtitle

* adjust commacare icon

* drop manage, its cleaner

* edit text

* clean

* for github and apple always return generic

* updates

* go back to original

* noice

* better overwrite

* close on unpairing if in management section

* missing 20 px when icon is there

* clean diff

* fix lint
2026-09-17 23:58:15 -07:00
Trey Moen 9a95fdc1a2 cabana: fix stale message size warnings (#38961) 2026-09-17 22:56:42 -07:00
Harald Schäfer c6d13eb723 Cinque v3 (#38932)
Use f78ed37d for the precompiled eGPU driving model
2026-09-17 22:44:49 -07:00
Trey Moen c8d56a1990 cabana: filter multiplexed signals in binary grid (#38960)
* cabana: filter multiplexed signals in binary grid

* cabana: refresh overlap warnings on branch changes
2026-09-17 22:28:30 -07:00
Harald Schäfer df7e0e5e77 Use upstream tinygrad disk tensors for model loading (#38956) 2026-09-17 22:18:13 -07:00
Trey Moen 6c69ebed7f cabana: improve heatmap readability (#38955)
* cabana: improve heatmap readability

* cabana: use shared colors for the heatmap

* cabana: brighten dark theme text
2026-09-17 22:09:33 -07:00
Trey Moen c504b92aba cabana: clarify signal button states (#38953)
* cabana: clarify plot button states

* cabana: strengthen signal action hover states
2026-09-17 22:07:01 -07:00
stef 1090e2824b ui: add pairing provider to prime state (#38950)
* changes

* clean and lock sensible changes

* don't write to disk every loop

* clean

* clean diff

* clean indents
2026-09-17 22:06:22 -07:00
Trey Moen b55e37b9c9 cabana: right-align numeric message values (#38959) 2026-09-17 22:05:10 -07:00
Trey Moen e77f6e56f9 cabana: show cumulative byte and bit heatmaps (#38954) 2026-09-17 22:04:45 -07:00
Trey Moen 60b3ccf39e cabana: accept prefixed hex values in id filters (#38952) 2026-09-17 21:57:48 -07:00
Trey Moen bca3b04720 cabana: improve dark theme contrast (#38957) 2026-09-17 21:56:31 -07:00
stef 7635dec618 ui: qr widget (#38951)
* rewrite pairing dialog into nice qr widget

* simplify, move refresh to device pairing dialog

* fix

* refactor
2026-09-17 21:53:39 -07:00
Harald Schäfer 3cd1b6e483 Update tinygrad and use retargetable model artifacts (#38933)
* Update tinygrad to latest upstream with minimal openpilot patches

* Rebase tinygrad on latest upstream with one remaining patch

* Use retargetable model and warp artifacts

* Pack compiled model weights to reduce startup time

* Memory-map out-of-band model weights

* Use unmodified upstream tinygrad master

* Keep locally built model artifacts native
2026-09-17 20:43:51 -07:00
stef 7dcb524058 ui: abstract info stack (#38949)
* abstract info stack

* better seperation in device py

* clean
2026-09-17 19:02:45 -07:00
stef d643860afa ui: increase info subtext size (#38946)
increase to 36
2026-09-17 18:10:57 -07:00
Adeeb Shihadeh f992a60bbd op setup: 2x faster on Linux (warm venv) (#38945) 2026-09-17 17:43:25 -07:00
Harald Schäfer 1328ace068 Remove model chunking and use LFS for Chestnut releases (#38941)
* Remove model chunking and use LFS for Chestnut releases

* Check Git blob sizes once during release packaging

* Download the big model for Chestnut nightly builds

* Reuse published LFS models when pushing releases

* Preserve LFS configuration when copying Chestnut releases

* Detect release builds independently of LFS attributes
2026-09-17 16:34:52 -07:00
Daniel Koepping 7db7735598 rm chestnut power test (#38943)
* test power draw with chestnut

* revert chestnut power draw test

* Revert "add powertest for mici (#38843)"

This reverts commit f174e39d7a.
2026-09-17 15:58:41 -07:00
Adeeb Shihadeh cd1490af4b modeld: 2x faster chestnut build (#38656)
* not worth 1m

* saves 24s

* save 13s

* parallelize model compilation

* explain compiler affinity

* full cpufreq

* rm that for now

* cmt

---------

Co-authored-by: elkoled <elkoled@gmail.com>
2026-09-17 14:31:42 -07:00
Shane Smiskol bd176cb6a7 ui: remove question marks (#38938)
fix question marks
2026-09-17 00:59:12 -07:00
stef 4d9d1bc341 ui: not paired bookmark alert (#38936)
* check connect alert

* remove test

* simplify

* clean diff

* fit with bookmark icon out

* clean
2026-09-16 19:34:52 -07:00
Daniel Koepping cab53438ae AGNOS 19.8 (#38935) 2026-09-16 19:10:00 -07:00
Harald Schäfer 6080cc6168 Use a precompiled eGPU driving model (#38930)
* Ship precompiled eGPU model and camera warps

Compile f78ed37d-afad-4dbc-8050-40ea885eedde/12864 through xx/ml_tools/openpilot_compile using the pinned tinygrad version.

* Precompile the existing master driving model

Use the unchanged master ONNX (SHA-256 6fee5937923c74848df4a63f6239eb6331c6274dd4bdb7a5d6ec0388a8b543d5) instead of updating the trained model.

* Compile camera warps on device

* Remove obsolete ONNX chunking and big model build check

* Chunk model artifacts only during release packaging

* Require model and camera warps for Chestnut readiness

* Recompile precompiled CPU helpers for the runtime host

* Ship the eGPU model with an ARM submission helper

* Exempt model pickles from the build product size limit
2026-09-16 08:13:45 -07:00
Harald Schäfer 81ae1a2e2d Use tinygrad generic ONNX compiler artifacts (#38926)
* Use tinygrad generic ONNX compiler artifacts

* Keep compiler temporary model buffers off device tmpfs

* Keep recurrent model state updates on the GPU

* Use compiled output buffers and swap recurrent state inputs

* Note when ONNX chunk reassembly can be removed

* Bump tinygrad to rebased compilers and AMD queue fix

* Keep recurrent buffers fixed and use faster USB argument updates

* Bump tinygrad for faster model startup
2026-09-15 18:01:25 -07:00
Daniel Koepping f15544b482 rm pyserial from updater bundle (#38925)
rm pyserial dependency from updater package
2026-09-15 13:20:49 -07:00
Harald Schäfer b751b04cda Use tinygrad ONNX and warp compilers for driving and DM (#38922)
* modeld: use tinygrad ONNX and warp compilers for driving and DM

* Bump tinygrad with current openpilot model compile tests

* modeld: prioritize tinygrad compiler imports in build environment

* Use compiler branch based on openpilot's pinned tinygrad

* modeld: invoke tinygrad compilers without firmware patch
2026-09-15 11:21:47 -07:00
Harald Schäfer fa9f56ed76 Compile warp and onnx separately (#38864)
* Extract shared model warp routines

* Move DM warp compilation into compile_warp

* Configure shared warp compilation through SCons flags

* Move the driving warp wrapper into compile_warp

* Build driving warps separately from the model

* Compile model inputs directly from ONNX

* modeld: simplify split input setup

* modeld: extract input packing setup

* modeld: simplify packed views and share firmware setup
2026-09-14 23:07:26 -07:00
Harald Schäfer 64f9b47b6e modeld: move history queues into ONNX (#38916)
* modeld: move history queues into ONNX

* Move ONNX history export into xx exporter

* Rename ONNX image input to new_img

* Combine ONNX model history state

* Revert model history README addition

* Keep float32 input packing from master

* Rename model input specs to input shapes
2026-09-14 20:44:33 -07:00
Harald Schäfer 7b469af438 ci: run docs builds on Namespace (#38921) 2026-09-14 20:21:16 -07:00
Harald Schäfer 2c88d1ed64 Revert "bump msgq (#38836)" (#38920)
This reverts commit 1e0914051f.
2026-09-14 20:20:53 -07:00
stef ffef0e6d3b ui: custom alert icons (#38917)
* custom alerts and alert pill better

* add comment back

* fix
2026-09-14 20:03:25 -07:00
stef e6aa405e08 ui: fix touch validity on alerts menu (#38915)
fix touch validity on alerts menu
2026-09-14 17:14:38 -07:00
Matt Purnell 2689917757 locationd: fix critical_services spelling (#38912) 2026-09-14 10:43:43 -07:00
Amy Jeanes a5f44653d7 mici: add a refresh models button to the models panel (#2018)
models: add a refresh models button to mici, gate and show progress on both panels

Adds the refresh-models tile to the mici models panel (it was missing there),
factoring the sync-key trigger + in-progress check into refresh_model_list()
and refresh_in_progress() shared with the big UI.

On both UIs the refresh button is now gated on offroad + not-downloading +
not-refreshing (the manager runs offroad-only and its per-tick manifest fetch
sits above a blocking download loop, so a refresh queued in either state would
stick), and shows progress while the manager refetches: mici shows "fetching..."
on the tile, the big UI flips its button from REFRESH to "FETCHING..." to match
its FETCHING.../SELECT/CLEAR label style (as in the OSM panel), replacing the
old fire-and-forget popup.


Claude-Session: https://claude.ai/code/session_01EGMnVnSk5inGTrd7kuDVG9

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-09-14 08:33:48 -07:00
Trey Moen 5f50bda7ba cabana: improve signal panel layout (#38910)
* cabana: improve signal panel layout

* cabana: match row actions to selection colors

* cabana: preserve master remove action

* cabana: match remove icon to selected row
2026-09-13 17:06:52 -07:00
Trey Moen 5b34f151de cabana: balance signal action spacing (#38911) 2026-09-13 17:01:04 -07:00
Trey Moen 64e5d1e5e3 cabana: preserve selected menu color on hover (#38909)
* cabana: unify menu highlight colors

* cabana: preserve selected menu color on hover
2026-09-13 16:54:34 -07:00
Trey Moen d3106c2b4b cabana: unify video and live stream titles (#38908)
* cabana: unify video and live stream titles

* cabana: put stream before dbc in window title

* cabana: show fingerprint in route details
2026-09-13 16:43:53 -07:00
Trey Moen 885792de3f cabana: remove message table outer border (#38907) 2026-09-13 16:36:16 -07:00
Trey Moen 5319474ee5 cabana: disable tool window collapse (#38906) 2026-09-13 16:31:19 -07:00
Trey Moen 164a3a1e30 cabana: stop chart tooltips over menus (#38905)
* cabana: keep chart tooltips below menus

* cabana: exclude popup menus from plot hover
2026-09-13 16:27:40 -07:00
Trey Moen c715ef4762 cabana: flatten signal panel containers (#38904)
* cabana: flatten signal panel containers

* cabana: keep messages and logs switch border
2026-09-13 16:15:25 -07:00
Trey Moen 2ae3598b51 cabana: prevent signal panel collapse (#38903)
* cabana: keep signal scrolling inside the list

* cabana: reserve space for the signal panel
2026-09-13 15:56:21 -07:00
Trey Moen 4e60afe70d cabana: show persistent fps counter (#38898)
* cabana: show persistent UI FPS

* cabana: render whole fps in monospace

* cabana: keep fps aligned beside centered progress

* cabana: move fps to top right

* Revert "cabana: move fps to top right"

This reverts commit 51a5334aec79807f8102b9a6f1f902f22814d5c7.
2026-09-13 15:48:35 -07:00
Trey Moen 5c365d3398 cabana: remove inner video container (#38901) 2026-09-13 15:46:29 -07:00
Trey Moen e6f9b3c6c4 cabana: preserve menu bar border on highlight (#38902)
* cabana: preserve menu bar border on highlight

* cabana: remove border drawing comment
2026-09-13 15:46:14 -07:00
Trey Moen 94f71fce00 cabana: add browser sign-in for remote routes (#38894)
* cabana: add browser sign-in for remote routes

* cabana: use shared dialog styling for browser sign-in

* cabana: support spacious centered action buttons

* cabana: vertically center sign-in states

* cabana: keep sign-in layout stable between states

* cabana: center sign-in error text

* cabana: avoid hint flash when retrying sign-in
2026-09-13 15:23:22 -07:00
Trey Moen b6918cb31b cabana: fix download bar layout (#38900)
* cabana: fix download bar layout

* cabana: reduce download label font size
2026-09-13 15:22:13 -07:00
Trey Moen 11093e743e cabana: improve dark slider track contrast (#38899) 2026-09-13 14:44:13 -07:00
Trey Moen ebb202f29d tools: share cancellable browser sign-in (#38893)
* tools: share cancellable browser sign-in

* tools: invoke auth JSON mode directly from Cabana

* tools: require an explicit Python module

* tools: order module constants alphabetically

* tools: use shared unauthorized handling for devices
2026-09-13 14:35:46 -07:00
Trey Moen 9c054e285d replay: retry failed segment loads (#38897)
* replay: retry failed segment loads

* replay: rename test helper for spellcheck

* replay: include portable temporary directory declaration

* replay: name attempt limit and shorten test delays

* replay: drop timing-dependent retry tests
2026-09-13 14:19:59 -07:00
Trey Moen c5cf29cf5e cabana: contain tooltips and capture plot drags (#38896)
* cabana: contain chart tooltips within columns

* cabana: refresh tooltip bounds after chart resize

* cabana: capture plot drags to prevent window movement
2026-09-13 13:53:35 -07:00
Amy Jeanes 63a2a3868e models: don't freeze the ui on an unset LagdToggleDelay (#2026)
models: don't block the ui on an unset LagdToggleDelay

Params.get's second positional is `block`, not a fallback value, so
get("LagdToggleDelay", "0.2") passes block=True and does a blocking read.
When the param is unset this spins the ui thread until it appears, freezing
the models panel (the description is rebuilt every frame). Read it the same
way livedelay/lagd_toggle.py does.


Claude-Session: https://claude.ai/code/session_01EGMnVnSk5inGTrd7kuDVG9

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-09-13 13:31:13 -07:00
Matt Purnell 5484f7f4a7 modeld: set the valid flag on modelDataV2SP (#2017)
* modeld: set the valid flag on modelDataV2SP

modelDataV2SP was published with new_message's default valid=False, so
the message was permanently invalid. Nothing acts on that today because
selfdrived lists it under ignore_valid, but it shows up as invalid in
every commIssue dump and hides any real problem behind a false one.

Copy modelV2's flag, the same way fill_model_msg already does for
drivingModelData.

Assisted-by: Claude
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* modeld_v2: set the valid flag on modelDataV2SP

The sunnypilot model runner publishes the same message and had the same
gap. Copy modelV2's flag here too, so both daemons agree.

Assisted-by: Claude
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-09-13 12:49:59 -07:00
dzid26 347238b307 camera offset: use real horizon for the shear center (#2016) 2026-09-13 12:38:20 -07:00
Amy Jeanes c57f9a7f4e workflows: let forks run their own model builds (#2009)
build-single-tinygrad-model and build-all-tinygrad-models gain a docs_repo
input (default sunnypilot/sunnypilot-models) so a fork can run either against
its own gh-pages catalog, next to the existing hf_repo input for the dataset.

build-all's setup job now checks out the repo and branch it was dispatched
from instead of sunnypilot/sunnypilot's default branch. That keeps the
manifest's tinygrad_ref tied to the code that compiled the models, and lets a
fork run the full rebuild against its own dataset and docs repo.


Claude-Session: https://claude.ai/code/session_01EGMnVnSk5inGTrd7kuDVG9

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-09-13 10:31:27 -07:00
Amy Jeanes 3c24eeea25 ci: remove stale disabled workflows (#1994)
Both are disabled in the Actions tab and have not run in months:

- Release Drafter (release-drafter.yml) and its config
  .github/release-drafter.yml: last run 2025-12-18
- Debug Discourse Posting (test-discourse.yaml.yml): one-off debug
  workflow from #1435, last run 2025-10-28

The post-to-discourse composite action is kept; the prebuilt workflow
still uses it. docs, stale and jenkins scan are also disabled here but
are inherited from commaai/openpilot and left in place to avoid
modify/delete conflicts on every upstream sync.


Claude-Session: https://claude.ai/code/session_01WAnUCRcA7Hp68qxyL11Wvm

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
2026-09-13 10:16:44 -07:00
Trey Moen 45a7747ca3 cabana: fix detached panel menus (#38890)
* cabana: keep dropdowns above detached panels and within the screen

* cabana: shorten dropdown comments
2026-09-13 09:46:43 -07:00
Harald Schäfer 064a51fe59 model replay: use a mici segment (#38878)
* model replay: use a mici segment

* process replay: wait for vision listener startup
2026-09-13 09:38:23 -07:00
Trey Moen e5a6a8b73e cabana: make the Signals pane a native dock panel, fix pad (#38889)
* cabana: align center tabs and keep signal panel docked

* cabana: prevent panels from docking into the tabless center

* cabana: use native docking for the Signals panel

* cabana: migrate dock tabs when loading the layout
2026-09-13 09:19:48 -07:00
Amy Jeanes 7430f245c7 mici: add a clear cache button to the models panel (#2008)
models: add a clear cache button to mici, gate and show progress on both panels

Adds the clear-cache tile to the mici models panel (trash slide-to-confirm),
factoring the cache-size math into model_cache_size_mb() shared with the big UI.

On both UIs the clear button is now gated on offroad + not-downloading +
not-clearing (the manager runs offroad-only, so a clear queued onroad would
never be serviced and would stick), and shows progress while the manager works:
mici shows "clearing..." on the tile, the big UI flips its button to
"CLEARING..." to match its FETCHING.../SELECT/CLEAR label style (as in the OSM
panel).


Claude-Session: https://claude.ai/code/session_01EGMnVnSk5inGTrd7kuDVG9

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-13 08:12:56 -07:00
Daniel Koepping d38264c42c log chestnut telemetry in hardwared (#38866)
log chestnut hardware state independently of modeld
2026-09-12 23:57:49 -07:00
Trey Moen 0d4a4ab910 cabana: disable collapse for dockable panels (#38885) 2026-09-12 23:48:57 -07:00
Trey Moen 44914a7cf4 cabana: remove custom focus-loss handling (#38884)
* cabana: fix focus handling across docked and floating windows

* cabana: remove custom focus-loss handling
2026-09-12 22:45:32 -07:00
Trey Moen 4c0799eb8e cabana: make Charts a native dockable panel (#38883)
* cabana: make charts a native dockable panel

* cabana: remove redundant charts docking button
2026-09-12 21:40:31 -07:00
Trey Moen 1aa97338c4 Cabana: standardize floating dropdown menus (#38874)
* Cabana: standardize floating dropdown menus

* Cabana: use uniform items in menus and combo lists

* Cabana: remove dropdown test and explanatory comments
2026-09-12 20:56:08 -07:00
Trey Moen 20064d9681 cabana: show placeholder for unresolved timestamps (#38870) 2026-09-12 20:46:51 -07:00
Trey Moen dc338c2831 cabana: standardize button spacing (#38873)
* cabana: standardize button spacing and control alignment

* cabana: center shared step controls and inset chart content

* cabana: use uniform per-chart content insets
2026-09-12 20:37:01 -07:00
Daniel Koepping 298e51010c print jenkins crash log (#38880) 2026-09-12 19:48:31 -07:00
Trey Moen 7e84a61668 cabana: use ImGui decorations for floating panes (#38871)
* cabana: redraw floating panes during native resizing

* cabana: keep rendering while native resize is held

* cabana: use ImGui decorations to avoid native resize loops

* cabana: draw docking previews only on the target viewport
2026-09-12 17:45:14 -07:00
Trey Moen f147094cb7 cabana: restore Qt palettes and contrast (#38875)
* cabana: restore Qt dark palette and contrast

* cabana: restore Qt standard light palette and selection text

* cabana: drive component colors from shared style values

* cabana: remove historical palette comments
2026-09-12 17:44:43 -07:00
Adeeb Shihadeh f235784042 Revert tinygrad update (#38879) 2026-09-12 17:26:45 -07:00
Harsh Singh 39146cb240 test_runner: collect parameterized_class tests instead of skipping them with their base (#38820) 2026-09-12 14:30:51 -07:00
commaci-public 19f0f69d85 [bot] Update Python packages (#38698)
* Update Python packages

* Fix pandad exports

---------

Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2026-09-12 14:28:19 -07:00
Adeeb Shihadeh 64b4dbf115 add shellcheck-like static analysis (#38877) 2026-09-12 14:14:39 -07:00
Kacper Rączy ad5c0b946d tools: optimize route parsing (#38714) 2026-09-12 14:13:08 -07:00
David 5cad99bf7a fix(tools): allow spaces in workspace path for op.sh (#38839) 2026-09-12 14:06:20 -07:00
Adeeb Shihadeh f1ba169fb5 remove old profiler setups 2026-09-12 13:45:51 -07:00
Daniel Koepping dd226ac791 update chestnut fw in CI (#38876)
Update Chestnut firmware before CI build
2026-09-12 12:41:14 -07:00
James Vecellio-Grant b67898fac4 models: test tinygrad concurrency (#2006) 2026-09-11 20:24:12 -07:00
Adeeb Shihadeh 0cf294d85f ui: add descriptions for settings (#38867) 2026-09-11 20:13:00 -07:00
Daniel Koepping 6965492e66 clean up after jenkins disconnect (#38865)
* fix SSH build cleanup

* simplify SSH cleanup
2026-09-11 17:25:33 -07:00
Bhavya Gada 73720bfd08 Get MetaDrive simulator working on macOS (#38830) 2026-09-11 15:29:13 -07:00
Harald Schäfer 6107da1cd6 Cinque v2 (#38850)
1a421175-db71-4e3d-9d62-e2166421b02b/12864
2026-09-10 21:24:09 -07:00
Daniel Koepping 6c410eee37 validate compiled model (#38855)
* validate serialized models after writing
2026-09-10 21:06:22 -07:00
Daniel Koepping fc4af50824 enable alpha long on nightly-chestnut (#38854)
panda debug build
2026-09-10 20:57:57 -07:00
Daniel Koepping f87e170bee reject incomplete model buffers (#38852) 2026-09-10 18:17:40 -07:00
Daniel Koepping 2a70b40cdc rebuild models when helpers change (#38853)
rebuild models when serialization helpers change
2026-09-10 18:14:23 -07:00
Daniel Koepping e6f401b5cf build nightly-chestnut (#38849)
build nightly-chestnut
2026-09-10 17:38:04 -07:00
Daniel Koepping e9711bc51d longer SSH keepalive for chestnut model load (#38851)
ci: allow longer SSH keepalive gaps
2026-09-10 17:26:33 -07:00
Harald Schäfer 9b899fdaa2 Reapply Git LFS hosting on Hugging Face (#38841) 2026-09-10 16:47:40 -07:00
Daniel Koepping 511c1f1e4e chestnut updater: close files explicitly (#38846)
chestnut: close USB sysfs files after reading
2026-09-10 14:28:12 -07:00
Daniel Koepping 443967ddad enable caching in chestnut CI (#38845)
enable caching
2026-09-10 14:17:21 -07:00
Daniel Koepping 633117b2a9 test_onroad in chestnut CI (#38844)
test chestnut with live camera frames
2026-09-10 14:02:18 -07:00
Daniel Koepping f174e39d7a add powertest for mici (#38843) 2026-09-10 13:19:44 -07:00
Zeph 68d829c7c9 selfdrived: no localizer alerts from capnp defaults (#38840)
selfdrived: don't alert on a message that was never received

posenetInvalid, locationdTemporaryError and paramsdTemporaryError are raised
off the capnp defaults of deviceMotion and vehicleParameters before either
message has ever arrived. When modeld is down, locationd and paramsd never
publish, the defaults read as a nan posenet speed, and that NoEntryAlert can
hide the process not running alert that names modeld.

Same guard as the big model alert in #38500.
2026-09-10 12:14:54 -07:00
Daniel Koepping 0cd3f0f20b remove old manifest before rebuild (#38842)
invalidate manifest before rebuilt
2026-09-10 11:18:54 -07:00
Trey Moen b97a2d8201 eSIM: require internet to delete a profile (#38838)
* Guard eSIM profile deletion with internet connectivity check

* Simplify eSIM delete connectivity guard
2026-09-09 20:21:52 -07:00
Harald Schäfer d9ecb0b678 Revert "Move Git LFS hosting to Hugging Face" (#38837) 2026-09-09 18:14:47 -07:00
Harald Schäfer ff2f78c3e8 Move Git LFS hosting to Hugging Face (#38824)
* Move LFS asset transfers to xet-core

* Limit migration to Git LFS hosting on Hugging Face

---------

Co-authored-by: Harald Schäfer <6804392+haraschax@users.noreply.github.com>
2026-09-09 14:31:06 -07:00
Adeeb Shihadeh 1e0914051f bump msgq (#38836) 2026-09-09 09:57:22 -07:00
Amy Jeanes 20bddff857 Use LF line endings
Split from https://github.com/commaai/openpilot/pull/38810.
2026-09-09 09:40:15 -07:00
Adeeb Shihadeh 14617070ab lil more comment 2026-09-09 09:33:57 -07:00
Trey Moen 802a231bd2 mici: simplify eSIM connecting status (#38833) 2026-09-08 23:34:37 -07:00
Trey Moen 179718dcb0 esim: mici profile download UI (#38813)
* esim: MICI profile download UI

* Show temporary feedback for non-LPA QR codes

* Scan eSIM QR codes every 250 ms

* Shrink eSIM error text and log displayed errors

* Activate downloaded eSIM through the profile tap flow

* Let the eSIM connectivity error wrap naturally

* Release add-profile button feedback immediately

* Check internet before opening the eSIM QR scanner

* Use UI network state and standard connectivity dialog for eSIM

* Drop LPA prime settings change from eSIM download UI

* Scope eSIM UI constants to their owning classes

* Remove unused ot codespell exception

* esim: simplify profile download operation and UI state

* esim: share activation code validation with QR scanner

* esim: match QR scanner video to cabin preview

* esim: show errors in a scrollable text dialog

* esim: require nonblank profile nicknames
2026-09-08 23:16:17 -07:00
Trey Moen d70df67366 mici: process eSIM notifications after profile deletion (#38832) 2026-09-08 21:39:24 -07:00
Trey Moen 66b4dbe2a1 qrcode: add QR decoder (#38814)
* qrcode: add QR decoder

Decodes a QR code from a grayscale image or a module matrix: adaptive
binarization, finder pattern search by run ratios, perspective sampling
with alignment pattern refinement, format info, unmasking, block
de-interleaving, Reed-Solomon correction, and numeric, alphanumeric,
byte (with ECI), and Kanji segments.

Fixtures are packed matrices from python-qrcode 8.2 covering every
version and level, plus Segno 1.6.6 matrices for the ECI cases.

* qrcode: tune decoder for the device

Measured on a comma mici with real cabin frames downsampled to 672x380,
the size the eSIM screen scans at. Per frame: 30 ms -> 8 ms with no
code in view, 42 ms -> 13 ms with a code.

- binarize: fixed two-radius fill from one integral image instead of an
  iterative outward fill (a dark cabin is almost all flat tiles), tile
  stats on a contiguous layout, flat tiles decided whole so sensor noise
  cannot become speckle, uint8 threshold compare
- finder search: scan every 4th row, build column runs only at
  candidate columns
- alignment search starts at a 2-module radius
- Reed-Solomon syndromes through table lookups, placement order cached

* qrcode: simplify decoder

Fold the function-module mask into _data_coords and the format
coordinates into _read_format, evaluate masks directly on the data
coordinates, compact the ECI, numeric, and alphanumeric parsing, and
flatten the retry loop in decode. No behavior change.
2026-09-09 04:23:55 +00:00
Trey Moen ef769c173d qrcode: generalize EC, format, and GF tables (#38828)
Replace the level-L-only Reed-Solomon parameters, the single hardcoded
format word, and the bitwise GF(256) multiply with the full EC table,
the 32 format words, and log/antilog tables. Block splitting and
interleaving move into _block_lengths and _interleaved. This is the
structure a decoder needs to read any version, level, and mask; the
encoder's module matrices are unchanged and test_encoder pins them.
2026-09-08 19:15:39 -07:00
Trey Moen a4a24d5d87 qrcode: fix alignment pattern positions for version 32 (#38827)
The closed-form step between alignment patterns rounds the wrong way
for version 32 (28 instead of 26). Use the form that is right for every
version. The encoder only emits versions 1-20, so its output is
unchanged.
2026-09-08 19:05:18 -07:00
Trey Moen 1fb3edb7dd qrcode: test alignment pattern positions (#38829) 2026-09-08 18:53:33 -07:00
Daniel Koepping dd4615850c replay chestnut in CI (#38826)
run model replay on chestnut in CI
2026-09-08 18:05:03 -07:00
Daniel Koepping f23e65768c ui: resize status icons (#38749)
* ui: resize status icons

* Update orange GPU icon to 108px height

* Trigger CI after LFS upload

* ci
2026-09-08 17:06:12 -07:00
Adeeb Shihadeh 96be8c7eb5 check dependency footprint (#38825) 2026-09-08 16:46:19 -07:00
Daniel Koepping 23d0474b95 compile big model in jenkins CI (#38822)
compile chestnut in ci
2026-09-08 15:37:12 -07:00
ZwX1616 60d8ae2d4f modeld/SConscript: fix chunk_targets estimate (#38821) 2026-09-08 15:27:11 -07:00
Trey Moen 7bfe6ba4d0 pandad: allow disabling driver camera IR offroad (#38812) 2026-09-08 15:21:03 -07:00
Pramish 9946c521f7 modeld: build DM warp and model JITs for all camera configs (#38767)
* modeld: build DM warp and model JITs for all camera configs

In #38684, camera_configs on comma_arm64 was restricted to only the host device
running SCons (selecting either _os_fisheye or _ar_ox_fisheye).

However, prebuilt release and nightly bundles are built on TICI/tizi runners
in Jenkins and distributed to both Comma 3/3X (TICI) and Comma 4 (MICI) devices.
Because the build machine was TICI, dm_warp_1344x760_tinygrad.pkl was omitted
from the packaged bundle, causing dmonitoringmodeld to fail with FileNotFoundError
on Comma 4 and leaving driver-monitoring calibration stuck at 0%.

Restore CAMERA_CONFIGS so both camera resolutions (1928x1208 and 1344x760)
are built for all platforms.

Fixes #38762

* simplify

---------

Co-authored-by: ZwX1616 <zwx1616@gmail.com>
2026-09-08 14:16:21 -07:00
Trey Moen 25d9d41c90 esim: comma four profile management UI (#37844)
* esim: MICI eSIM profile management UI

* esim: align rename button position regardless of delete button

* esim: skip profile UI when SIM is not an eUICC

Probe is_euicc() on the first profile poll and cache it; non-eUICC SIMs
avoid list_profiles/process_notifications (which hang on a plain SIM) and
the eSIM button shows ICCID/MCC-MNC metadata instead of opening the
profile management screen.

* esim: satisfy ruff E731 in action_pressed helper

* esim: get modem info from modem.py state instead of shelling out

* esim: simplify switch lifecycle, drop active flag and settle window

* esim: only show 'switching...' on the target profile

* esim: read cell strength directly from HARDWARE

* esim: hide checkmark and dim cell icon during switch; keep rename available

* esim: disable active profile button (rename still clickable as overlay)

* esim: stop rename/delete buttons and labels from flashing during operations

* esim: show 'comma prime' on network button for comma profile

* esim: move display_name and is_comma onto Profile dataclass

* esim: anchor rename button to rightmost slot to prevent shift

* esim: include iccid prefix check in Profile.is_comma

* esim: drop process_notifications from cellular manager

* esim: disable network button when SIM isn't an eUICC

* esim: rename ESim* classes to Esim*

* esim: sort imports

* esim: default to 'loading...' instead of 'no active profile'

* esim: rename PROFILE_POLL_INTERVAL to PROFILE_POLL_INTERVAL_S

* esim: slim EsimNetworkButton

* esim: use DEFAULT_TEXT_COLOR; load delete dialog texture in __init__

* esim: revert cell-icon index trick, name each NetworkStrength explicitly

* esim: tighten delete/rename spacing so 'switch' label fits on one line

* esim: shrink delete/rename buttons by 25%

* esim: keep rename button at full size, only delete shrinks

* Revert "disable modem.py for now"

This reverts commit 1eeba86ec1.

* Revert "modem.py is disabled"

This reverts commit d238a1ccc4.

* lpa: inline comma iccid prefix

* ui/cellular_manager: clear switching state when LPA returns

* ui/cellular_manager: lock callback queue, drop poll log noise

* esim: sort NetworkStrength/NetworkType imports

* esim: drop switching_iccid concept

* esim: show 'switching...' on the clicked profile button

* esim: optimistic switch, skip post-switch list_profiles to avoid flicker

* esim: only style profile button from local op flags, not global busy

* esim: remove deleting/switching state, collapse profile button branches

* esim: show GSM settings on full prime when non-comma profile is active

* esim: expose CellularManager.active_profile, dedup callers

* esim: apply comma prime defaults on switch (roaming, metered, no APN)

* esim: restore re_sort on show_event to avoid first-frame jank

* lpa: apply comma prime defaults inside TiciLPA.switch_profile

* lpa: lazy-import Params to break circular import

* lpa: treat +CME ERROR 13 (SIM failure) as non-eUICC in is_euicc

* esim: show 'obtaining IP...' on non-eUICC path while connecting

* Revert "lpa: treat +CME ERROR 13 (SIM failure) as non-eUICC in is_euicc"

This reverts commit 475ca448ea914ff8f9892b4cbf4525d961d7618b.

* esim: poll profiles every 5s

* lpa: tighten Profile.is_comma to require both Webbing provider and comma BIN

* lpa: fall back to '<unnamed>' instead of iccid prefix in display_name

* esim: re-probe is_euicc each poll for runtime SIM swaps

* esim_ui: rename EsimUIMici to EsimUI

* esim: simplify cellular manager and profile UI

- optimistic profile switch at click time so the active button no longer bounces back
- unify LPA worker threads, refresh_profiles resets the poll timer
- deterministic profile ordering on show
- reuse wifi ForgetButton for delete, drop DeleteButton
- construct CellularManager inside NetworkLayoutMici
- drop comma prime param defaults from LPA.switch_profile (moved to a separate PR)

* esim: drop eUICC state change log

* ui: gate cellular settings on prime subscription

* ui: disable eSIM management for full prime

* ui: keep eSIM management disabled for full prime

* ui: unify eSIM profile action buttons

* ui: tighten eSIM profile action spacing

* ui: place rename before delete in eSIM actions

* esim: restrict individual profiles and process switch notifications

* esim: defer poll after operations and confirm eUICC loss before clearing profiles

* ui: allow cellular settings for unregistered and unpaired devices

* ui: disable all profile switching for full prime

* ui: simplify cellular access to not full prime

* ui: make profile poll interval a CellularManager constant

* ui: read modem state on the profile poll cadence

* ui: scope eSIM profile colors to their class

* ui: inline modem state read in cellular polling

* ui: rely on hardware modem state fallback
2026-09-07 21:55:28 -07:00
Trey Moen 8f2d66d0e5 cabana: round only the outer video frame (#38808)
* cabana: remove video frame corner rounding

* cabana: round the outer video frame including letterboxing
2026-09-07 21:26:53 -07:00
Trey Moen 3eafb658bf mici: remove duplicate pairing button from main settings (#38809) 2026-09-07 17:48:33 -07:00
Trey Moen 10b9e73859 cabana: fill event bar to camera edges (#38807) 2026-09-07 17:06:01 -07:00
Trey Moen bb86bee688 cabana: clean up UI wording and formatting (#38803) 2026-09-07 14:56:19 -07:00
Trey Moen 3e54af9db8 cabana: playback controls for live streaming (#38802)
* Hide Cabana playback controls during live streaming

* cabana: retain pause and go-live controls for live streams
2026-09-07 14:47:20 -07:00
Trey Moen 20ae4ac44d cabana: standardize settings step buttons (#38801)
* cabana: align and brighten settings step buttons

* cabana: use standard settings icon button color
2026-09-07 14:08:32 -07:00
Trey Moen 7c1f224443 cabana: stack the undo/redo zoom buttons in the chart menu (#38800) 2026-09-07 13:03:12 -07:00
Trey Moen 2b4d050e9c cabana: fix video startup and desktop interactions (#38798)
* cabana: fix video startup and desktop interactions

* cabana: remove util comments
2026-09-07 12:43:41 -07:00
Trey Moen 1bb019521e cabana: fix chart ranges and control layout (#38797) 2026-09-07 12:42:41 -07:00
Trey Moen 624d5b9995 cabana: fix stream and replay lifetimes (#38796) 2026-09-07 12:33:14 -07:00
Trey Moen 0f9ec7158b cabana: validate message IDs and include recorded CAN messages (#38799) 2026-09-07 12:32:22 -07:00
Trey Moen 9c02eebb72 cabana: fix clipped control outlines and filter sizing (#38795)
* cabana: fix clipped control outlines and filter sizing

* cabana: remove added comments

* cabana: document control helper API usage
2026-09-07 12:05:26 -07:00
Trey Moen cb70ba89aa cabana: smooth startup layout (#38794)
* cabana: smooth startup layout

* cabana: remove camera tab startup comment

* cabana: initialize dock geometry on the first frame

* cabana: remove route loading placeholder
2026-09-07 11:50:13 -07:00
Trey Moen 65c411433b cabana: reduce downloader startup and dense chart rendering overhead (#38793)
* cabana: reduce downloader startup and dense chart rendering overhead

* cabana: remove downloader spawn tests
2026-09-07 01:22:23 -07:00
Adeeb Shihadeh 6bbdf8ad18 remove sentry (#38790) 2026-09-06 21:20:39 -07:00
Trey Moen 9d1f0d4171 cabana: restyle (#38789)
* cabana: connect palette theme, rounded cards, floating page switch

Move the jotpluggler port's connect palette into a shared theme module
(ui/theme.h/.cc) and apply it to all of cabana: Palette struct for dark
and light, applyTheme fills ImGui and ImPlot styles, readableCurveColor
and sectionTitle for reuse by the analysis workspace. Drop the old
DarkTheme constants and per-theme color branches in widgets.

Docked panels are borderless with content in rounded bordered cards;
custom-drawn rects use the style radii. The detail widget's bottom tab
bar is replaced by a floating Messages/Logs pill.

* cabana: scrollbar and menu contrast, page switch below the page

The scrollbar grab uses the border color on the window-colored track.
Selection colors are stronger so an open menu reads as blue in both
themes and the hovered item is visible in light mode. The Messages/Logs
pill sits in a strip below the page instead of covering its last rows.

* cabana: even spacing around the page switch, steady top menu highlight

* cabana: inner spacing before the clear button of a clearable input

* cabana: center the page switch below the page, space the signal row buttons

* cabana: logs table cells padded and bordered like the messages table

* cabana: recess the signals card in the window color

* cabana: one button vocabulary

Three button shapes, all one frame height with the style padding and
rounding: ImGui::Button for framed text, iconButton for a framed square
icon, toolButton for a flat square icon (or flat text). Adjacent buttons
are ItemInnerSpacing apart, groups ItemSpacing apart. Tool bars drop
their private padding and 1 px spacing, the signal row, chart header,
detail header, logs export and signal selector buttons all use the
shared sizes, and the page switch is built from the same style values.

* cabana: center the glyph ink in square icon buttons

* cabana: framed tool buttons and plain icon glyphs

toolButton is now iconButton with an optional text, so every button in
the app is framed. Boxed glyphs (plus-square, x-square, dash-square,
arrow-*-square, window-stack) are replaced by their plain variants
(plus-lg, trash, arrows-collapse, box-arrow-*, window-plus), the light
x/plus/dash by their -lg forms, the chart menu by three-dots-vertical
and the toolbar overflow by chevron-double-right.

* cabana: draw square button glyphs at 80% so full bleed icons keep a margin

* cabana: signal values in the mono font

* cabana: center square button icons on their atlas ink, snapped to framebuffer pixels

* cabana: draw square button glyph quads directly, AddText truncates the position

* cabana: right align the messages and signals tool bars to the table edge, full size collapse button

* cabana: detail header and tab scroll buttons flush with the right edge

* cabana: header and footer strips, gap above the panels, video splitter as a separator

The menu bar and the status bar are strips in the surface color with a
separator line towards the panels, and the dockspace sits one item
spacing below the menu bar, matching the spacing above the status bar.
The video/charts splitter draws the same 1 px separator line as the
dock splitters, centered in a gap as tall as the side padding, and a
double click snaps the video back to its natural height.

* cabana: top menus flush with the menu bar, 2 px video splitter without double click

* cabana: dropdowns continue the menu bar, splitter in the border color, row button gap

Top level menus draw no top border and square top corners, and the menu
bar leaves its separator out under the open dropdown. The video splitter
uses the border color like the dock splitters. The signal row buttons
keep one inner spacing before the scrollbar.

* cabana: dropdowns share the bar's 1 px border line, messages toolbar spacing matches the video card

* cabana: tool bar groups, framed dropdown buttons, chart content flush with the tool bar

Tool bar items are ItemSpacing apart and the items of a group (a button
pair, a separator and its neighbor) ItemInnerSpacing apart, instead of
a private 1 px spacing. The dropdown buttons are framed like the other
buttons. The video time is a plain mono readout that toggles on click.
Chart cards have no horizontal margin so their header and plot line up
with the tool bar above.

* cabana: chart list spacing: gap above the scroll area, charts clear of the scrollbar, even legend gaps

* cabana: loop and speed buttons grouped like the playback buttons

* cabana: survive narrow panels and small windows

Stress tested by dragging the column and video splitters to their
limits and shrinking the window to 640x480. Fixes: the messages tool
bar and the message detail header overflow into the >> menu instead of
clipping (the view button and the name stay); the binary view keeps a
minimum cell width and scrolls sideways; the Messages/Logs switch drops
to icons when the strip is too narrow; the camera keeps its last frame
across reconnects, so a video restored from a collapsed splitter while
paused is no longer black.

* cabana: range slider spans the charts >> menu, remove all and float as menu entries

* cabana: the charts card keeps its tool bar row or collapses, like the video at the other end

* cabana: uniform >> menus: dropdowns become submenus, the time readout an action entry

* cabana: video/charts splitter gap matches the column splitters

* cabana: drop unused lambda capture in chart type menu

Claude-Session: https://claude.ai/code/session_01XTFaJpf235LeHBmyFRgeJ1

* cabana: the menu bar and the status bar stay in full screen

* cabana: simplify theme helpers and toolbar actions

* cabana: preserve panel width for overflow edit dialog

* cabana: keep message editor usable in narrow panels

* cabana: group heatmap modes in a dropdown

* cabana: trim redundant restyle comments

* cabana: fix review findings from the restyle

- set the caret and other unset colors so the light theme has no white caret
- tolerate float error in the toolbar fit check to stop spurious overflow
- restore the full screen chrome hiding
- clear stale camera frames when the server changes, keep them across a collapse
- cache icon glyph ink bounds instead of scanning the atlas every frame
- drop dead plot_bg and ImPlot globals, add a badge palette role
- add toolbarMenu and use toolbarAction for the remove message item
- derive the signal view button size seed from the font and style

* cabana: remove remaining redundant comments
2026-09-06 19:46:02 -07:00
YassineYousfi c51e3e5a71 cinque terre model 🇮🇹 (#38771)
a4c5f1d1-1f5d-4807-9593-54afa27099d5/12864
2026-09-05 19:03:12 -07:00
Trey Moen 3bc6701f34 cabana: fill video pane and reset video height (#38786) 2026-09-05 17:02:05 -07:00
Trey Moen cb0da74b41 cabana: center speed dropdown label, add divider after loop button (#38785) 2026-09-05 15:34:00 -07:00
Trey Moen a4f7c50d2a cabana: fix sparkline edges and stroke rendering (#38784)
cabana: preserve sparkline edges and improve stroke rendering
2026-09-05 14:54:17 -07:00
Trey Moen 444be0a649 cabana: pace frames instead of relying on vsync (#38783)
On Wayland a vsynced glfwSwapBuffers blocks until the compositor sends a
frame callback, which it stops doing while the window is on another
workspace or otherwise off-screen. The main thread never gets back to
glfwPollEvents, so Hyprland reports cabana as not responding.

Run with swap interval 0 and pace the loop to the monitor refresh rate.
2026-09-05 14:49:37 -07:00
Trey Moen a989bc0b50 cabana: misc UI tweaks (#38782) 2026-09-05 14:13:15 -07:00
Trey Moen f9dacd0d6b cabana: build binary directly, drop wrapper script (#38781) 2026-09-05 13:45:12 -07:00
Trey Moen 0ec3a082c7 op: support all the linuxes! (#38776) 2026-09-04 21:10:04 -07:00
Trey Moen c8603fb3ce op: point git-lfs config at the vendored binary (#38775)
git-lfs is vendored in the venv, but `git lfs install` writes filters and hooks that invoke a bare `git-lfs`, which is only on PATH inside the venv. With filter.lfs.required set, any checkout from a plain shell fails.

git runs filters and hooks from the worktree root, so a relative path into the venv resolves from anywhere.
2026-09-04 21:09:40 -07:00
Trey Moen 69ad376fef Revert "op: support all the linuxes!" (#38774)
Revert "op: support all the linuxes! (#38773)"

This reverts commit 33790c855c.
2026-09-04 19:09:45 -07:00
Trey Moen 33790c855c op: support all the linuxes! (#38773) 2026-09-04 18:30:45 -07:00
Adeeb Shihadeh 675ff56981 tools: add op docs command (#38770) 2026-09-04 15:20:43 -07:00
YassineYousfi e3def37695 revert 38742 (#38760) 2026-09-03 09:45:51 -07:00
Trey Moen 3a13b67b6f bye bye Qt! (#38753)
* cabana: remove Qt implementation
2026-09-02 20:44:11 -07:00
Trey Moen 7bade9a67a cabana: halve dock panel min width (#38759) 2026-09-02 20:28:53 -07:00
Trey Moen c163743e32 cabana: better colors (#38758) 2026-09-02 20:20:23 -07:00
Trey Moen 06d76bdb24 cabana: imgui port (#38725)
* cabana: imgui frontend base infra (ui/)

* cabana: line-for-line imgui ports of the Qt widgets and dialogs into ui/

* cabana ui: fixes from the side by side parity test

* cabana ui: deqt.md status after the line-for-line ports

* cabana ui: fixes from the line-for-line review

* cabana ui: parity fixes from the end to end matrix, event driven frame pacing

* cabana ui: preserve F1 help formatting

* cabana ui: parity fixes from the end to end matrix, event driven frame pacing

* cabana ui: deqt.md status

* cabana: fix macOS build of imgui frontend

- link Security.framework (libusb's darwin backend needs
  SecTaskCreateFromSelf/SecTaskCopyValueForEntitlement)
- drop unused loop counter in SignalView::updateChartState
- avoid -Wshadow on Sparkline::size and SignalView::highlight

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: parity round 3 fixes, thread pool and one malloc arena for flat memory

* cabana ui: toolbar overflow budget, menu button carets, help overlay colors, signal editor validation

* cabana ui: deqt.md status

* cabana: let vsync pace the frame loop

The loop threw away frames with glfwWaitEventsTimeout() to hit a 30fps
(10fps idle) target while glfwSwapBuffers() was already blocking on
vsync, so two clocks beat against each other. Input events also reset
frameInterval() to 0 for a few frames, so a click swung the cadence from
~33ms to ~13ms and back -- the camera view redraws on the UI's schedule,
so that showed up as a visible hitch on every message selection.

Poll and draw every iteration and let glfwSwapInterval(1) do the pacing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana: fix macOS build of thread pool / malloc arena changes

- malloc.h and mallopt(M_ARENA_MAX) are glibc only; guard on __GLIBC__.
  macOS has a single allocator zone, so there is nothing to cap.
- capturing structured bindings in a lambda is a C++20 extension and
  clang rejects it under -Werror; bind the event range to plain locals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: chart axis label precision, signal list click on empty space clears the selection

* cabana ui: codespell

* cabana ui: taller signal rows, system theme detection, full dark palette

- signal rows are 1.5x a frame tall with the sparkline sized to the row,
  so the graphs are readable. The expanded sub-rows keep the Qt height.
- "Automatic" resolved to the light theme regardless of the system: Qt
  gets this from QStyle::standardPalette(), which follows the macOS
  appearance, so read AppleInterfaceStyle and match it. Other platforms
  stay light, like standardPalette() does there.
- widgets that tested settings.theme == DARK_THEME themselves went light
  under AUTO_THEME while the style went dark; they now ask isDarkTheme()
  for the theme applyTheme() actually resolved.
- the dark branch only set 10 ImGuiCol_ entries, so buttons, scrollbars,
  tabs, title bars, separators, tables and grips still came from imgui's
  stock dark theme. Derive the whole palette from DarkTheme instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: drop the dead y axis alignment chain and chart cache stub

ImPlot::BeginAlignedPlots aligns the chart y axes, so the hand-rolled
alignment protocol ported from Qt no longer feeds anything: align_to was
write-only. Remove the whole chain (y_label_width and its measuring loop
in draw(), axisYLabelWidthChanged, align_timer, alignCharts), which
reduces updatePlotArea(int, bool) to updatePlotArea().

resetChartCache() has been an empty stub since the port (imgui redraws
every frame); drop it and its four call sites. Also replace the empty
move_icon_rect branch in mousePressEvent with an early return.

No behavior change. updateAxisY's "|| y_label_width == 0" guard only
fired after a unit change, and y_precision/y_tick_count depend solely on
min_y/max_y/tick_count, so the recompute it triggered was a no-op.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: stop the main window from scrolling

The host window that holds the dockspace uses zero WindowPadding but the
default ItemSpacing, so the dockspace and the status bar below it summed
to ItemSpacing.y (5px) more than the viewport, leaving the whole UI
scrollable by a few pixels.

Reserve the spacing along with the status bar height, and mark the host
window NoScrollbar/NoScrollWithMouse: it is pinned to the viewport work
area and should never scroll, as QMainWindow does not. Full screen is
unaffected, it draws no status bar and reserves no height.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: inset the status bar so its ends are not clipped

A borderless BeginChild gets WindowPadding forced to zero, so the status
bar text sat flush against both edges and the first glyph of "For Help,
Press F1" was cut off. Inset both ends by WindowPadding.x, which lines
the text up with the content of the docked panels above it.

Right align the cached minutes / FPS label to the captured child width
rather than calling GetContentRegionAvail() again after the message
text, which only returned the full width because the text had already
wrapped the cursor to a new line.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: drop the leftover imgui object census log

The block printed an imgui/implot object count to stderr every 10s,
which was instrumentation for tracking down a leak, not something the
port needs. It was the only user of implot_internal.h in mainwin.cc, so
drop that include with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: keep the selected row highlighted while hovered

Selectable() picks its background as HeaderActive when held, else
HeaderHovered when hovered, else Header. The hovered test comes before
the selected one, so a selected row under the cursor takes the hover
color. Both tables push a transparent HeaderHovered to suppress the hover
highlight that QTreeView/QTableView do not have, which also blanked the
selection: clicking a row walked press (HeaderActive, visible), release
(HeaderHovered, transparent, looks unselected), then mouse off the row
(Header, visible), so the selection appeared to flicker until the cursor
moved away.

Only make HeaderHovered transparent for rows that are not selected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: draw the binary view hex column in bold and in the text color

The Qt delegate never sets a pen for the hex column, so it inherited the
black QPainter default and the two halves of a row did not match. Use
paletteText(is_message_active), the same color the bit columns use, so
both halves read as one row and dim together when the message goes
inactive.

JetBrains Mono ships no bold variant and imgui has no embolden option, so
give drawStaticText a bold flag that redraws the glyphs a fraction of a
pixel to the right. That keeps the monospace advance, which switching to
the proportional bold face would not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: highlight the whole row in the speed menu

MenuItem spans from the cursor to the right edge, so the Indent() that
made room for the radio bullet also pushed the highlight in by one
FontSize and left the bullet column outside it. QMenu highlights the
whole item, indicator included.

Draw each row as one full width Selectable with the bullet and the label
inside it. Every row declares the same width, so the popup is as wide as
the widest entry and the highlights reach both edges; a label-less
Selectable sized zero would declare no layout width and collapse the
auto-sizing popup. Selectable auto-closes its parent popup like MenuItem
does, so clicking still works in the dropdown and in the Speed submenu of
the toolbar overflow menu, which shares this function.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: fix help menu overlay, Ctrl+F11, time label clipping, toolbar overflow and tool table parity

* cabana ui: click to edit signal cells, light theme selection, logs cell selection and filter fixes

* cabana ui: chart x label margin, zoom context menu items, video splitter collapse, signal editor close

* cabana ui: signal editor lifetime and close, logs cell hover, tool table headers

* cabana ui: branch indicator clicks, editor swallows shortcut and button clicks, selection colors

* cabana ui: float dock windows into real OS windows

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: toolbar menu arrow spacing, only open the editor on editable cells

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* op: add cabana2 to run the imgui cabana

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana: cabana2 launcher script builds _cabana_ui before running it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: only float the dock panels into their own OS windows

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: open the video pane at the camera's natural aspect ratio

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana: default fps to 30

* cabana ui: fusion palette, shared fusion slider, chart grid colors

* cabana ui: smaller checkbox indicator, closer to the Fusion size

* cabana ui: video toolbar height and item centering, separator extent and menu arrow like Fusion

* cabana ui: message and signal row heights match Qt

* cabana ui: timeline slider height, groove and stacking match the Qt video layout

* cabana ui: fix crash opening a live stream, the video pane default height read the missing camera widget

* cabana: drop the FPS setting, the streams update the UI at a fixed 30 fps

* cabana ui: video and charts pane run edge to edge with a uniform toolbar margin like the Qt frames

* cabana ui: binary view M/L marker size and timeline event fills match Qt

* cabana ui: signal view keeps its sparklines while filtering, number label size matches Qt

* cabana ui: no modal dim fade, QDialog does not animate in

* cabana ui: collapse all is an auto-raise tool button with the Qt icon size

* cabana ui: dialogs and tool windows open as real OS windows like QDialog

* cabana ui: hold back a focus loss while a mouse button is down so tearing a panel off does not abort the drag and create its window twice

* cabana ui: speed menu rows pad the label on the right like QMenu

* cabana ui: raise the dark theme contrast, the Darcula grays all sat on top of the background

the chart y axis guides are opaque and 2 px in dark mode, and text, outlines,
separators, table borders, scrollbar and slider grabs move away from the window
and base grays. light theme is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: simplify comments, drop the Qt references and the ones that only restate the code

* cabana ui: keep the sparkline antialiased, thin peaks sparkled as the window slid

Qt dropped antialiasing above 500 points because it rasterized into a pixmap. drawing
straight into the frame, an aliased 1 px segment between two columns rounds into one of
them and the rounding flips as the window advances, so the peaks of the dense signals
popped between columns. updateState runs at STREAM_UPDATE_FPS, which at a 30s range moves
the curve well under a pixel per update, so the flip is all that is visible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: anchor the sparkline to the time window, it jittered sideways as samples expired

x was measured from the first sample still inside the window, which changes whenever the
oldest one falls out, so every update slid the whole curve sideways by the sample spacing.
measuring from the start of the window pins the right edge and lets the samples flow left
with the clock. the density heuristic wanted the data span, which is no longer the x of the
last point, and the flat line started at x 0 instead of at its first sample.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: scroll the sparkline at the frame rate, it stepped at the message rate

the window ended at the last message of the id and updateState only runs when one arrives,
so a slow message held the sparkline still for several frames and then jumped it. the window
now ends at the playback clock, and draw() slides the rendered polyline by the time that has
passed since it was built, clipped to the cell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: keep the signal value on the right of the column before there is a sparkline

it was drawn left aligned from the start of the column, so the value sat next to the name
until the first samples arrived and then jumped across the row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: read a little past the sparkline window so the oldest points slide out of it

a sample used to disappear the moment it aged out, which on a fixed rate signal is a point
popping off the left edge. the query reaches back a bit further and the clip rect hides it
instead. the lead in must not move the scale, so it is left out of the min and max.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: use the macOS fullscreen space, taking over the monitor rendered at the old size

glfwSetWindowMonitor switched the display mode and returned a drawable at that mode, so the
ui was rendered at the windowed size and stretched over the screen. NSWindow toggleFullScreen
keeps the backing scale. the green button goes through the same space, so the state is read
back from the style mask rather than tracked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: let the signal value column shrink again, it held the widest value ever seen

max_value_width only ever grew, so one long value left the sparklines ending
well short of the text for the rest of the message. Track the current widest
value instead, growing on demand and giving room back once it is a couple of
characters too wide, which keeps the width from jittering per sample.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: make the signal rows a quarter taller so the sparklines read better

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: lift the binary view heatmap alphas in dark mode, the tints washed out

The alphas were tuned against a white background; over the dark base the same
values left the cells barely colored. The floor comes up and the ramp gets a
gamma in dark mode only, which keeps the flip counts in the same order.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: saturate the sparkline color in light mode, the pastel lines vanished

Signal colors are low saturation and high value, which is aimed at dark fills;
a 1 px line of one of them on white is hard to see. Draw the sparkline with
twice the saturation and less value when the theme is light.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: give the docked panels a minimum width, the dividers ran over content

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* replay: capture the downloader's stderr so the progress lines reach the handler

file_downloader.py writes PROGRESS:<cur>:<total> to stderr while it streams a
file, but runPython() only piped stdout and left stderr attached to the parent,
so the progress handler only ever fired on failure and cabana downloaded a
segment with no progress bar. Pipe stderr as well, turn the PROGRESS lines into
handler calls, and pass everything else through to the parent's stderr.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* replay: read the downloader's stderr on a thread, the two fd select was overkill

getline() handles the line splitting, partial lines and EOF, so the manual
splitter, the second drain loop and the multiplexing bookkeeping go away and
the stdout loop goes back to what it was.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: one toolButton helper in imgui_util.h

* cabana ui: one toImU32/toImVec4/paletteBrightText in imgui_util.h

* cabana ui: string helpers live in utils/strings.h

* cabana ui: share the fusion slider handle from imgui_util.h

* cabana ui: style.cc declarations move to imgui_util.h

* cabana ui: validatedInput and the qt validators move to imgui_util.h

* cabana ui: viewSelectable helper for the item view rows

* cabana ui: drop the comboBox pass-through

* cabana ui: tabbar moves to its own file, ready for the charts port

* cabana ui: find signal searches on the main thread again, deferred one frame like QTimer::singleShot(0)

* cabana ui: find signal computes its button and label state from the model instead of caching it

* cabana ui: share the queued popup owner protocol between the message box and the file dialog

* cabana ui: one beginDialog helper for the three centered modals, with the reopen recovery

* cabana ui: tool dialogs share the title, begin/end and escape handling, and own their connections

* cabana ui: comboBox helper replaces the hand rolled zero separated item buffers

* cabana ui: drop the unused dialog isOpen accessors

* cabana ui: one validatedText helper for the ip, double and int line edits

* cabana ui: route info reads the route segments instead of caching a row copy

* cabana ui: settings drops the phantom fps label, the constant spin flags and the raw log path buffer

* cabana ui: find similar bits drops the no-op find button disable and the address round trip

* cabana ui: message box warning overload without the empty detailed text

* cabana ui: help overlay only collects rects while it is up, skips torn off panels, and the fingerprint compare is ordered

* cabana ui: drop the glfw callbacks that only forward to the imgui backend, which already installs them

* cabana ui: elided label renders with RenderTextEllipsis instead of a paint time cache

* cabana ui: one save loop, inline cached minutes label, and drop the dead help parser branches

* cabana ui: signal cell bands are built from the two corner notches instead of a region sweep

* cabana ui: binary view items go through itemAt and updateItem assigns unconditionally

* cabana ui: message bytes read the cell padding inline and colored bytes keep the qt default pen

* cabana ui: detail widget drops createToolBar, the welcome widget flag and the unused whatsThis

* cabana ui: messages widget owns its view, header and model directly, fixed size filter editors, from_chars range parsing

* cabana ui: colored hex bytes keep the row pen, the qt delegate does not reset it per cell

* cabana ui: closeEditor drops the queued commit, its item is about to be deleted

* cabana ui: rowsInserted carries the insert position so the selected row follows its message

* cabana ui: the log filter and the message name editors use the shared validators

* cabana ui: the log header sizing and painting are free functions, no HeaderView object

* cabana ui: the visible row range is the change detector, no mirrored scroll position

* cabana ui: the line editor takes the focus flag as an argument and lets InputText revert on escape

* cabana ui: signal view drops the unreachable column parameters, unused item values and repeated sparkline tests

* cabana ui: rows outside the tree viewport are measured but not painted

* cabana ui: one toolbar item table, playback state read in draw()

* cabana ui: draw the stored thumbnail scaled by AddImage, no cpu prescaled copy

* cabana ui: one radioMenuItem helper for the speed and series type menus

* cabana ui: drop dead camera members, hoist the jpeg colour space branch, decode qlog thumbnails on the thread pool

* cabana ui: stop the vipc thread when the splitter collapses the video pane

* cabana ui: the toolbar strings and enabled state are computed in drawToolBar

* cabana ui: chartswidget uses the shared tabbar widget

* cabana ui: the chart header layout is recomputed in resizeEvent, no updateTitle

* cabana ui: updateLayout has no force flag

* cabana ui: one condition hides the value tip

* cabana ui: signal selector uses the shared dialog buttons and row indexes

* cabana ui: the chart tile geometry is one Layout struct saved and restored by drawGhost

* cabana ui: drop the empty theme branch, the unused size hint and the charts container indirections

* cabana ui: the chart holds its tip label by value

* cabana ui: one toolbar item list for both toolbar groups

* cabana ui: the sparkline polyline is emitted through the draw list path

* cabana ui: the tab bar latches a programmatic selection and defers the close request past EndTabBar

* cabana ui: validatedText refuses an invalid edit inside the imgui buffer and the ip field keeps its char filter

* cabana ui: drop the ambiguous three argument warning overload

* cabana ui: combobox reports a real change, menu rows span the popup, one popup protocol for the dialogs

* cabana ui: the startup stream is owned by the window until the first frame opens it

* cabana ui: the slider toolbar index has an out of range sentinel

* cabana ui: the find signal search runs on the frame after the pending state is painted

* cabana ui: the log clears its selection when the rows are dropped and cells are identified by their message

* cabana ui: the open editor survives a scroll, collapse all commits it and escape in the size editor does not

* cabana ui: the header display order is filled with iota

* cabana ui: qlog thumbnails decode on their own threads, the hover thumbnail is mipmapped and the time tooltip appears on the first toggle

* cabana ui: a collapsed video dock stops the vipc thread

* cabana ui: one TOOLBAR_ITEM_SPACING, the signal view used the QCommonStyle 4 instead of Fusion's 1

* cabana ui: sparkline includes imgui_util.h for isDarkTheme

* cabana ui: imgui_util.h becomes util.h and util.cc

* cabana: delete deqt.md

* cabana ui: panel min width, matching video panel padding, centered tabs

The dock panels stop shrinking at the width where the signal view tool bar
squishes instead of at a hardcoded 440. The video/charts panel keeps the
standard window padding so it lines up with the other panels, and the
Msg/Logs tabs are centered in their bar. The cached minutes label is gone
from the status bar.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: persist imgui state in cabana.json and migrate the Qt frontend's layout blobs

* cabana ui: 2px sparkline stroke

a 1px antialiased line is all fringe and no solid core, so the curve
faded and shimmered as the window scrolled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana: drop the Automatic color theme

it only did anything on macOS: the Qt frontend fell through to
standardPalette() for both Automatic and Light, so the two were
identical everywhere else.

LIGHT_THEME and DARK_THEME keep their values so a persisted dark theme
still reads as dark; the old Automatic 0 falls back to light on load.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: window color between the binary and signal views, fixed split

the binary view sits at its size hint and no longer resizes, and the gap
below it takes the window color instead of leaving a white seam.

also border the video and chart panels like the signal view.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: bigger tab bar scroll buttons

imgui's built in scroll arrows are fixed at FontSize - 2 wide with the
glyph drawn in a FontSize box, so they are small and sit off center.
draw chevron buttons at the frame height instead, spaced and disabled at
the ends of the scroll range like the rest of the tool buttons.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* cabana ui: 1.5px sparkline stroke

* cabana ui: stable sparkline scrolling, one sample per column, aliased spikes

Scroll the polyline by whole physical pixels with each sample's subpixel
phase fixed to its timestamp, thin to one sample per pixel column, and
draw near-vertical segments aliased so spikes stay crisp while smooth
curves keep antialiasing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: fix spelling

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: size the value column for the widest value a signal can produce

Sizing it to the values in the last message moved the sparklines every
time a value changed length.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: center the tab bar scroll chevrons and give the buttons a frame

The icon font glyph sits off center in its padded advance, so the
chevron is drawn in the button rect. The buttons take the theme's
button background and border like the tool buttons next to them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: darker grid lines between table cells

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: the gap between the video and the charts matches the side padding

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: plus-square icon for the new chart button, like the remove all button

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: video toolbar buttons as tall as the charts toolbar buttons

The buttons carried 10 px of vertical padding, so the hover rect was
much taller than the glyph and the row looked off center.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: the time display in the mono font at the ui font size

With proportional digits the time changed width as it ticked and the
items after it moved.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: closing a floated side panel docks it back, no hide tab bar button, the charts container never scrolls

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: the message tabs use the TabBar widget, chevron scroll buttons for both tab bars

The chevron scroll buttons move into a scrollable tab bar widget that
both use. The wheel scrolls the tabs, both tab bars get Close Other
Tabs, and the Msg tab is labeled Messages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: deliver a focus loss right away on macOS, holding it back swallowed the first click in a popup

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: draw the remote routes dialog at the modal's level, opened inside the tab's child window it never appeared

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* cabana ui: --no-cache turns off the local route file cache, same as replay

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* cabana: parse merged segments on replay's merge thread, the parse on the main thread dropped frames on every merge while downloading

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* cabana ui: Esc leaves full screen on all platforms

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* cabana ui: an app bundle on macOS so the menu bar and the dock show Cabana

A bare binary shows its file name there. The build wraps _cabana_ui in
a minimal Cabana.app with an Info.plist and the launcher runs that.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* Revert "cabana ui: an app bundle on macOS so the menu bar and the dock show Cabana"

This reverts commit 1a45540125847ca46fcd4d7ac0522918e315057d.

* cabana ui: the app menu on macOS says Cabana

A bare binary gets an info dictionary with its file name in it, which
glfw reads for the app menu. The name is set there before glfw brings
up cocoa.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: macapp moves into ui/util

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: setMacAppName lives in ui/util.cc

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: apply the code review

Bugs: an empty decode batch dereferenced vals.front(), the chart x comparator
took a float, reserve used capacity(), the bit text went black while resizing
a signal, byte cells under-sized non-multiple-of-8 payloads, the CAN speed
lookup read the data speed table, the export tooltip never showed while
disabled, and the thumbnail parser and the signal search joined ad-hoc
threads on the render thread. Both now run off the pool; the search fans out
over it from its own thread.

Shared helpers in ui/util: the tool bar with its overflow menu, the menu
button, modal dialog setup, Escape inside dialogButtons, drawText,
drawElidedText, color markers, alignRight, clearableInput,
disabledItemTooltip, inputTextMultiline, tableHeadersRow with the right
click, withAlpha, boldFont, the Cocoa full screen glue, the messages panel
id. utils gets hexByte, guarded and nonEmptyDBCFiles; threadpool gets
parallelFor.

The help overlay leaves mainwin.cc for its own file, the stream teardown is
one releaseStream(), the charts own their views through unique_ptr, the
find-similar-bits message list follows the source bus, the clipboard copies
every open DBC, and the Qt vestiges (event names, model API, protected
sections, restating comments, stale TODOs, mixed member naming) are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: collapse the model/view/delegate split

The Qt-era model, view and delegate classes were ported one to one, but
imgui draws a table in one function. MessageView, MessageViewHeader and
MessageListModel's string API fold into MessagesWidget over a plain
MessageList; HistoryLogModel folds into LogsWidget; BinaryViewModel and
BinaryItemDelegate fold into BinaryView; SignalItemDelegate folds into
SignalView, and SignalModel keeps only the item tree with three predicates
instead of the flags bitmask; FindSignalModel becomes SignalSearch. The
byte cell painter is a set of free functions shared by the two tables.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* cabana ui: load the route behind the window

The route file listing is an HTTPS round trip to the comma API and took
the whole first second of startup before the window existed. The replay
branch of main() now hands run() a loader that a worker executes after
the first frame; the window maps in ~0.1 s instead of ~1 s. On a failed
load the app stays open in the No Stream state with the error box
instead of exiting before a window ever appeared.

* cabana ui: fix the live stream video controls layout

* cabana ui: only copy dbc to clipboard with a single file

* cabana ui: report route loading failures instead of hanging

* cabana: launch the imgui frontend by default, --legacy for Qt

---------

Co-authored-by: Trey Moen <trey@Treys-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 08:06:41 -07:00
YassineYousfi 50fb67f718 bump tg + TC_MIN_GLOBALS (#38751)
bump op + TC min globals
2026-09-01 23:12:28 -07:00
502 changed files with 23017 additions and 13651 deletions
+3 -1
View File
@@ -1,8 +1,10 @@
* text=auto
* text=auto eol=lf
# to move existing files into LFS:
# git add --renormalize .
*.onnx filter=lfs diff=lfs merge=lfs -text
openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl filter=lfs diff=lfs merge=lfs -text
openpilot/sunnypilot/modeld_v2/models/*.pkl filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
-43
View File
@@ -1,43 +0,0 @@
exclude-labels:
- 'no-changelog'
categories:
- title: '🚀 Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
collapse-after: 5
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧰 Maintenance'
collapse-after: 5
label: 'chore'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
change-title-escapes: '\<*_&'
replacers:
- search: '/[Ss][Uu][Nn][Nn][Yy][Pp][Ii][Ll][Oo][Tt]/g'
replace: 'sunnypilot'
- search: '/\b[Ss][Pp]\b/g'
replace: 'SP'
version-resolver:
major:
labels:
- 'major'
minor:
labels:
- 'minor'
patch:
labels:
- 'patch'
default: patch
name-template: 'v$RESOLVED_VERSION 🚀'
tag-template: 'v$RESOLVED_VERSION'
version-template: "0.$MAJOR.$MINOR.$PATCH" # The day OP becomes v1, we need to bump this
tag-prefix: "v0." # The day OP becomes v1, we need to bump this
prerelease-identifier: "staging"
template: |
## Changes
$CHANGES
@@ -20,6 +20,11 @@ on:
required: false
type: string
default: 'sunnypilot/sunnypilot_models_v1'
docs_repo:
description: 'GitHub repo holding the driving_models JSON on its gh-pages branch'
required: false
type: string
default: 'sunnypilot/sunnypilot-models'
jobs:
setup:
@@ -34,9 +39,9 @@ jobs:
- name: Checkout sunnypilot repo
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot
path: sunnypilot
submodules: recursive
fetch-depth: 1
- name: Get tinygrad_repo ref
id: get-tinygrad-ref
@@ -47,13 +52,14 @@ jobs:
echo "tinygrad_ref=$ref" >> $GITHUB_OUTPUT
echo "tinygrad_ref is $ref"
- name: Checkout docs repo (sunnypilot-models, gh-pages)
- name: Checkout docs repo (gh-pages)
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot-models
repository: ${{ inputs.docs_repo }}
ref: gh-pages
path: docs
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
fetch-depth: 1
- name: Get next JSON version to use (from GitHub docs repo)
id: get-json
@@ -118,6 +124,7 @@ jobs:
json_version: ${{ needs.setup.outputs.json_version }}
target_hardware: ${{ github.event.inputs.target_hardware }}
hf_repo: ${{ github.event.inputs.hf_repo }}
docs_repo: ${{ inputs.docs_repo }}
set_min_version: ${{ github.event.inputs.set_min_version }}
tinygrad_ref: ${{ needs.setup.outputs.tinygrad_ref }}
secrets: inherit
@@ -162,6 +169,7 @@ jobs:
target_hardware: ${{ github.event.inputs.target_hardware }}
artifact_suffix: -retry
hf_repo: ${{ github.event.inputs.hf_repo }}
docs_repo: ${{ inputs.docs_repo }}
set_min_version: ${{ github.event.inputs.set_min_version }}
tinygrad_ref: ${{ needs.setup.outputs.tinygrad_ref }}
secrets: inherit
+77 -107
View File
@@ -46,8 +46,8 @@ jobs:
if [ "${{ inputs.target }}" = "big" ]; then
NAME=$(python3 -c "from openpilot.sunnypilot.models.model_name import DEFAULT_BIG_MODEL; print(DEFAULT_BIG_MODEL)")
ONNX_PATH="openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx"
HF_DEFAULTS_PATH="models/defaults/big"
ONNX_PATH="openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl"
elif [ "${{ inputs.target }}" = "dm" ]; then
ONNX_PATH="openpilot/selfdrive/modeld/models/dmonitoring_model.onnx"
HF_DEFAULTS_PATH="models/defaults/dm"
@@ -57,8 +57,9 @@ jobs:
ONNX_PATH="openpilot/selfdrive/modeld/models/driving_supercombo.onnx"
HF_DEFAULTS_PATH="models/defaults/small"
fi
ONNX_REF=""
[ -n "$ONNX_PATH" ] && ONNX_REF=$(git log -1 --format='%H' -- "$ONNX_PATH")
ONNX_REF=$(git log -1 --format='%H' -- "$ONNX_PATH")
TINYGRAD_REF=$(python3 openpilot/sunnypilot/models/tinygrad_ref.py)
if [ -z "$TINYGRAD_REF" ]; then
echo "::error::Failed to resolve tinygrad ref"
@@ -78,6 +79,8 @@ jobs:
if: ${{ inputs.target == 'small' }}
runs-on: [self-hosted, tici]
env:
MODELS_DIR: openpilot/selfdrive/modeld/models
COMPILER: tinygrad_repo/examples/openpilot
SMALL_ONNX: openpilot/selfdrive/modeld/models/driving_supercombo.onnx
SMALL_PKL: openpilot/selfdrive/modeld/models/driving_tinygrad.pkl
steps:
@@ -85,9 +88,10 @@ jobs:
with:
submodules: recursive
- name: Pull ONNX via LFS
- name: Pull small ONNX via LFS
run: git lfs pull -I "${{ env.SMALL_ONNX }}"
- name: Set environment variables
run: |
source /etc/profile
@@ -103,38 +107,15 @@ jobs:
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
- name: Compile small model with stock compiler
- name: Compile small model from ONNX
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}/tinygrad_repo:${{ github.workspace }}"
MODEL_SIZE=$(python3 -c "from openpilot.common.transformations.model import MEDMODEL_INPUT_SIZE as s; print(f'{s[0]}x{s[1]}')")
CAMERA_RES=$(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')")
FRAME_SKIP=$(python3 -c "from openpilot.selfdrive.modeld.constants import ModelConstants as MC; print(MC.MODEL_RUN_FREQ // MC.MODEL_CONTEXT_FREQ)")
TG_FLAGS="DEV=QCOM IMAGE=1 FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 OPENPILOT_HACKS=1"
env ${TG_FLAGS} python3 \
${{ github.workspace }}/openpilot/selfdrive/modeld/compile_modeld.py \
--onnx ${{ github.workspace }}/${{ env.SMALL_ONNX }} \
--model-size $MODEL_SIZE \
--camera-resolutions $CAMERA_RES \
--frame-skip $FRAME_SKIP \
--output ${{ github.workspace }}/${{ env.SMALL_PKL }}
- name: Chunk small pkl
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH=${{ github.workspace }}
python3 -c "
from openpilot.common.file_chunker import chunk_file, get_chunk_targets
import os
pkl = '${{ github.workspace }}/${{ env.SMALL_PKL }}'
size = os.path.getsize(pkl)
targets = get_chunk_targets(pkl, size)
chunk_file(pkl, targets)
print(f'Chunked into {len(targets)} files')
"
env ${TG_FLAGS} python3 ${{ github.workspace }}/${{ env.COMPILER }}/compile_onnx.py \
${{ github.workspace }}/${{ env.SMALL_ONNX }} \
${{ github.workspace }}/${{ env.SMALL_PKL }} \
--device-input "*" --out-of-band --benchmark-runs 1
- name: Prepare output
env:
@@ -147,14 +128,16 @@ jobs:
PKL_BASE="driving_tinygrad.pkl"
mkdir -p "$OUTPUT_DIR"
cp "$MODELS_DIR/${PKL_BASE}".chunk* "$OUTPUT_DIR/"
cp "$MODELS_DIR/${PKL_BASE}.chunkmanifest" "$OUTPUT_DIR/"
cp "$MODELS_DIR/${PKL_BASE}" "$OUTPUT_DIR/"
ONNX_SHA256=$(git cat-file blob "$(git ls-files -s "${{ github.workspace }}/${{ env.SMALL_ONNX }}" | awk '{print $2}')" | grep '^oid sha256:' | cut -d: -f2)
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
--model-dir "$MODELS_DIR" \
--output-dir "$OUTPUT_DIR" \
--custom-name "$MODEL_NAME" \
--upstream-branch "${{ needs.resolve.outputs.onnx_ref }}"
--upstream-branch "${{ needs.resolve.outputs.onnx_ref }}" \
--onnx-sha256 "$ONNX_SHA256"
echo "model-${MODEL_NAME}-${{ github.run_number }}" > "$OUTPUT_DIR/artifact_name.txt"
@@ -181,15 +164,16 @@ jobs:
if: ${{ inputs.target == 'big' }}
runs-on: [self-hosted, chestnut]
env:
BIG_ONNX: openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx
MODELS_DIR: openpilot/selfdrive/modeld/models
COMPILER: tinygrad_repo/examples/openpilot
BIG_PKL: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Pull big ONNX via LFS
run: git lfs pull -I "${{ env.BIG_ONNX }}"
- name: Pull big PKL via LFS
run: git lfs pull -I "${{ env.BIG_PKL }}"
- name: Set environment variables
run: |
@@ -222,38 +206,33 @@ jobs:
raise RuntimeError('Chestnut PCIe link not ready after 10 attempts')
"
- name: Compile big model with stock compiler
- name: Compile big warp
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}/tinygrad_repo:${{ github.workspace }}"
CHESTNUT_TG_FLAGS="DEBUG=1 DEV=USB+AMD:LLVM FRAME_DEV=CPU FLOAT16=1 JIT_BATCH_SIZE=0 GMMU=0 TC_OPT=2 TC_MIN_GLOBALS=32"
COMPILER="${{ github.workspace }}/tinygrad_repo/examples/openpilot"
MODELS_DIR="${{ github.workspace }}/openpilot/selfdrive/modeld/models"
MODEL_SIZE=$(python3 -c "from openpilot.common.transformations.model import MEDMODEL_INPUT_SIZE as s; print(f'{s[0]}x{s[1]}')")
CAMERA_RES=$(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')")
FRAME_SKIP=$(python3 -c "from openpilot.selfdrive.modeld.constants import ModelConstants as MC; print(MC.MODEL_RUN_FREQ // MC.MODEL_CONTEXT_FREQ)")
TG_FLAGS="DEBUG=2 DEV=USB+AMD:LLVM WARP_DEV=QCOM FLOAT16=1 JIT_BATCH_SIZE=0 GMMU=0 TC_OPT=2"
env ${TG_FLAGS} python3 \
${{ github.workspace }}/openpilot/selfdrive/modeld/compile_modeld.py \
--onnx ${{ github.workspace }}/${{ env.BIG_ONNX }} \
--model-size $MODEL_SIZE \
--camera-resolutions $CAMERA_RES \
--frame-skip $FRAME_SKIP \
--output ${{ github.workspace }}/${{ env.BIG_PKL }}
- name: Chunk big pkl
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH=${{ github.workspace }}
python3 -c "
from openpilot.common.file_chunker import chunk_file, get_chunk_targets
import os
pkl = '${{ github.workspace }}/${{ env.BIG_PKL }}'
size = os.path.getsize(pkl)
targets = get_chunk_targets(pkl, size)
chunk_file(pkl, targets)
print(f'Chunked into {len(targets)} files')
"
for res in $(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')"); do
CAM_W=$(echo $res | cut -d x -f1)
CAM_H=$(echo $res | cut -d x -f2)
STRIDE_INFO=$(python3 -c "
import sys
from openpilot.system.camerad.cameras.nv12_info import get_nv12_info
s,y,u,sz = get_nv12_info(int(sys.argv[1]), int(sys.argv[2]))
print(f'{s},{y},{u},{sz}')
" $CAM_W $CAM_H)
WARP_PKL="${MODELS_DIR}/big_driving_warp_${res}_tinygrad.pkl"
env ${CHESTNUT_TG_FLAGS} python3 "${COMPILER}/compile_warp.py" \
--frame ${CAM_W},${CAM_H},${STRIDE_INFO} \
--warp-to ${MODEL_SIZE} \
--layout yuv420 \
--frames 2 \
--output "${WARP_PKL}"
done
- name: Prepare output
env:
@@ -266,14 +245,18 @@ jobs:
PKL_BASE="big_driving_tinygrad.pkl"
mkdir -p "$OUTPUT_DIR"
cp "$MODELS_DIR/${PKL_BASE}".chunk* "$OUTPUT_DIR/"
cp "$MODELS_DIR/${PKL_BASE}.chunkmanifest" "$OUTPUT_DIR/"
cp "$MODELS_DIR/${PKL_BASE}" "$OUTPUT_DIR/"
PKL_LFS_SHA256=$(git cat-file blob "$(git ls-files -s "${{ github.workspace }}/${{ env.BIG_PKL }}" | awk '{print $2}')" | grep '^oid sha256:' | cut -d: -f2)
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
--model-dir "$MODELS_DIR" \
--output-dir "$OUTPUT_DIR" \
--custom-name "$MODEL_NAME" \
--upstream-branch "${{ needs.resolve.outputs.onnx_ref }}"
--upstream-branch "${{ needs.resolve.outputs.onnx_ref }}" \
--onnx-sha256 "$PKL_LFS_SHA256"
cp "$MODELS_DIR"/big_driving_warp_* "$OUTPUT_DIR/" || true
echo "model-${MODEL_NAME}-${{ github.run_number }}" > "$OUTPUT_DIR/artifact_name.txt"
@@ -311,7 +294,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Pull ONNX via LFS
- name: Pull LFS for resolved ONNX files
if: ${{ needs.resolve.outputs.onnx_path != '' }}
run: git lfs pull -I "${{ needs.resolve.outputs.onnx_path }}"
- name: Install huggingface_hub
@@ -376,38 +360,28 @@ jobs:
from datetime import datetime, UTC
dm_dir = Path('dm_output')
manifest = list(dm_dir.glob('*.chunkmanifest'))
assert manifest, 'No chunkmanifest found'
pkl_name = manifest[0].name.removesuffix('.chunkmanifest')
num_chunks = int(manifest[0].read_text().strip())
chunks = []
for i in range(num_chunks):
chunk = dm_dir / f'{pkl_name}.chunk{i+1:02d}of{num_chunks:02d}'
chunks.append({
'file_name': chunk.name,
'sha256': hashlib.sha256(chunk.read_bytes()).hexdigest()
})
matches = sorted(dm_dir.glob('dmonitoring_*_tinygrad.pkl'))
assert len(matches) == 1, f'expected one DM pkl in {dm_dir}, found {matches}'
pkl_name = matches[0].name
digest = hashlib.sha256()
for c in chunks:
with open(dm_dir / c['file_name'], 'rb') as f:
while block := f.read(1024*1024):
digest.update(block)
with open(dm_dir / pkl_name, 'rb') as f:
while block := f.read(1024*1024):
digest.update(block)
metadata = {
'bundles': [{
'short_name': 'DMMODEL',
'display_name': '${{ needs.resolve.outputs.model_name }}',
'ref': '${{ needs.resolve.outputs.onnx_ref }}',
'tinygrad_ref': '${{ needs.resolve.outputs.tinygrad_ref }}',
'runner': 'tinygrad',
'build_time': datetime.now(UTC).strftime('%Y-%m-%dT%H:%M:%SZ'),
'models': [{
'type': 'chunked',
'artifact': {
'file_name': pkl_name,
'download_uri': {'url': '', 'sha256': digest.hexdigest()},
'chunks': chunks
'download_uri': {'url': '', 'sha256': digest.hexdigest()}
}
}]
}]
@@ -470,42 +444,39 @@ jobs:
${{ github.workspace }}/${{ env.DM_ONNX }} \
${{ github.workspace }}/${{ env.DM_PKL }}
- name: Chunk DM pkl
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH=${{ github.workspace }}
python3 -c "
from openpilot.common.file_chunker import chunk_file, get_chunk_targets
import os
pkl = '${{ github.workspace }}/${{ env.DM_PKL }}'
size = os.path.getsize(pkl)
targets = get_chunk_targets(pkl, size)
chunk_file(pkl, targets)
print(f'Chunked {pkl} into {len(targets)} chunks')
"
- name: Compile DM warp
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}/tinygrad_repo:${{ github.workspace }}"
TG_FLAGS="DEV=QCOM IMAGE=1 FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 OPENPILOT_HACKS=1"
MODEL_DIR="${{ github.workspace }}/openpilot/selfdrive/modeld"
COMPILER="${{ github.workspace }}/tinygrad_repo/examples/openpilot"
MODELS_DIR="${{ github.workspace }}/openpilot/selfdrive/modeld/models"
DM_SIZE=$(python3 -c "from openpilot.common.transformations.model import DM_INPUT_SIZE as s; print(f'{s[0]}x{s[1]}')")
for res in $(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')"); do
WARP_PKL="${MODEL_DIR}/models/dm_warp_${res}_tinygrad.pkl"
taskset -c 7 env ${TG_FLAGS} python3 ${MODEL_DIR}/compile_dm_warp.py \
--camera-resolution ${res} \
CAM_W=$(echo $res | cut -d x -f1)
CAM_H=$(echo $res | cut -d x -f2)
STRIDE_INFO=$(python3 -c "
import sys
from openpilot.system.camerad.cameras.nv12_info import get_nv12_info
s,y,u,sz = get_nv12_info(int(sys.argv[1]), int(sys.argv[2]))
print(f'{s},{y},{u},{sz}')
" $CAM_W $CAM_H)
WARP_PKL="${MODELS_DIR}/dm_warp_${res}_tinygrad.pkl"
taskset -c 7 env ${TG_FLAGS} python3 "${COMPILER}/compile_warp.py" \
--frame ${CAM_W},${CAM_H},${STRIDE_INFO} \
--warp-to ${DM_SIZE} \
--output ${WARP_PKL}
--layout luma \
--border-fill 16 \
--transform-device NPY \
--output "${WARP_PKL}"
done
- name: Prepare DM output
run: |
mkdir -p dm_output
cp ${{ github.workspace }}/${{ env.DM_PKL }}.chunk* dm_output/
cp ${{ github.workspace }}/${{ env.DM_PKL }}.chunkmanifest dm_output/
cp ${{ github.workspace }}/${{ env.DM_PKL }} dm_output/
cp ${{ github.workspace }}/openpilot/selfdrive/modeld/models/dm_warp_* dm_output/
- name: Upload DM artifact
@@ -519,4 +490,3 @@ jobs:
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
@@ -39,6 +39,11 @@ on:
required: false
type: string
default: 'sunnypilot/sunnypilot_models_v1'
docs_repo:
description: 'GitHub repo holding the driving_models JSON on its gh-pages branch'
required: false
type: string
default: 'sunnypilot/sunnypilot-models'
set_min_version:
description: 'Minimum selector version'
required: false
@@ -107,6 +112,11 @@ on:
required: false
type: string
default: 'sunnypilot/sunnypilot_models_v1'
docs_repo:
description: 'GitHub repo holding the driving_models JSON on its gh-pages branch'
required: false
type: string
default: 'sunnypilot/sunnypilot-models'
env:
RECOMPILED_DIR: recompiled${{ inputs.recompiled_dir }}
JSON_FILE: docs/docs/driving_models_${{ inputs.target_hardware == 'chestnut' && 'chestnut_v' || 'v' }}${{ inputs.json_version }}.json
@@ -124,19 +134,17 @@ jobs:
publish_model:
if: ${{ !cancelled() && needs.build_model.result == 'success' }}
concurrency:
group: hf-push-${{ inputs.recompiled_dir }}
cancel-in-progress: false
needs: build_model
runs-on: ubuntu-latest
permissions:
actions: read
id-token: write
contents: write
steps:
- name: Checkout docs repo
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot-models
repository: ${{ inputs.docs_repo }}
ref: gh-pages
path: docs
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
@@ -172,18 +180,21 @@ jobs:
echo "artifact_name=$ARTIFACT_NAME" >> $GITHUB_OUTPUT
- name: Download and extract model artifact
uses: actions/download-artifact@v4
with:
name: ${{ steps.read-artifact-name.outputs.artifact_name }}
path: output
env:
GH_TOKEN: ${{ github.token }}
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
run: |
id=$(gh api -X GET "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/artifacts" -f name="$ARTIFACT_NAME" --jq '.artifacts[0].id')
url=$(curl -sf -o /dev/null -w '%{redirect_url}' -H "Authorization: Bearer $GH_TOKEN" "https://api.github.com/repos/$GITHUB_REPOSITORY/actions/artifacts/$id/zip")
sudo apt-get install -y -qq aria2 > /dev/null
aria2c -q -x16 -s16 -k8M --max-tries=5 --file-allocation=none -o model.zip "$url" && unzip -q model.zip -d output && rm model.zip
- name: Create models folder
env:
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
run: |
mkdir -p "local_models/${RECOMPILED_DIR}/${ARTIFACT_NAME}"
cp -r output/* "local_models/${RECOMPILED_DIR}/${ARTIFACT_NAME}/"
rm -f "local_models/${RECOMPILED_DIR}/${ARTIFACT_NAME}/artifact_name.txt"
cp output/metadata.json "local_models/${RECOMPILED_DIR}/${ARTIFACT_NAME}/"
- name: Upload to Hugging Face
env:
@@ -195,13 +206,12 @@ jobs:
"models/${RECOMPILED_DIR}/${ARTIFACT_NAME}/" \
--repo-type=dataset
- name: Pull gh-pages
- name: Update and push JSON to docs repo
run: |
cd docs
git pull origin gh-pages
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
- name: Run json_parser.py to update JSON
run: |
FOLDER="${{ inputs.model_folder }}"
if [ "$FOLDER" = "Other" ]; then
FOLDER="${{ inputs.custom_model_folder }}"
@@ -214,18 +224,27 @@ jobs:
[ -n "${{ inputs.version }}" ] && ARGS="$ARGS --version \"${{ inputs.version }}\""
[ -n "${{ inputs.set_min_version }}" ] && ARGS="$ARGS --set-min-version \"${{ inputs.set_min_version }}\""
[ -n "${{ inputs.tinygrad_ref }}" ] && ARGS="$ARGS --tinygrad-ref \"${{ inputs.tinygrad_ref }}\""
eval python3 docs/json_parser.py \
--json-path "$JSON_FILE" \
--recompiled-dir "local_models/$RECOMPILED_DIR" \
--sort-by-date \
$ARGS
- name: Push updated JSON to GitHub docs repo
run: |
cd docs
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git checkout gh-pages
git add docs/"$(basename $JSON_FILE)"
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || echo "No changes to commit"
git push origin gh-pages
for attempt in $(seq 1 10); do
git fetch origin gh-pages
git reset --hard origin/gh-pages
eval python3 json_parser.py \
--json-path "docs/$(basename $JSON_FILE)" \
--recompiled-dir "../local_models/$RECOMPILED_DIR" \
--sort-by-date \
$ARGS
git add docs/"$(basename $JSON_FILE)"
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || { echo "No changes to commit"; exit 0; }
if git push origin gh-pages; then
echo "Push succeeded on attempt $attempt"
exit 0
fi
echo "Push failed (attempt $attempt/10), retrying after backoff..."
sleep $((RANDOM % 10 + 3))
done
echo "All 10 push attempts failed"
exit 1
@@ -0,0 +1,71 @@
name: Compile warp for sunnypilot modeld
on:
workflow_dispatch:
jobs:
compile_warps:
runs-on: [self-hosted, chestnut]
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Set environment variables
run: |
source /etc/profile
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export UV_PYTHON_PREFERENCE=managed
export UV_PYTHON_INSTALL_DIR=${HOME}/uv/python
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
uv sync --frozen
printenv >> $GITHUB_ENV
- name: Disable powersave
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
- name: Compile Warp Kernels
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}/tinygrad_repo:${{ github.workspace }}"
mkdir -p openpilot/sunnypilot/modeld_v2/models/
MODEL_SIZE=$(python3 -c "from openpilot.common.transformations.model import MEDMODEL_INPUT_SIZE as s; print(f'{s[0]}x{s[1]}')")
CAMERA_RES=$(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')")
echo "Model size: $MODEL_SIZE"
echo "Camera resolutions: $CAMERA_RES"
for res in $CAMERA_RES; do
FRAME_PARAM=$(python3 -c "from openpilot.system.camerad.cameras.nv12_info import get_nv12_info; w, h = map(int, '${res}'.split('x')); stride, y_h, uv_h, _ = get_nv12_info(w, h); print(f'{w},{h},{stride},{y_h},{uv_h},{stride * (y_h + uv_h)}')")
WARP_PKL="openpilot/sunnypilot/modeld_v2/models/driving_warp_${res}_tinygrad.pkl"
echo "Compiling $WARP_PKL on QCOM"
taskset -c 7 env DEV=QCOM IMAGE=1 FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 OPENPILOT_HACKS=1 python3 openpilot/sunnypilot/modeld_v2/compile_warp.py \
--frame ${FRAME_PARAM} --warp-to ${MODEL_SIZE} --output "${WARP_PKL}"
BIG_WARP_PKL="openpilot/sunnypilot/modeld_v2/models/big_driving_warp_${res}_tinygrad.pkl"
echo "Compiling $BIG_WARP_PKL on AMD"
taskset -c 7 env DEV=USB+AMD:LLVM FRAME_DEV=CPU FLOAT16=1 JIT_BATCH_SIZE=0 GMMU=0 TC_OPT=2 TC_MIN_GLOBALS=32 python3 openpilot/sunnypilot/modeld_v2/compile_warp.py \
--frame ${FRAME_PARAM} --warp-to ${MODEL_SIZE} --output "${BIG_WARP_PKL}"
done
- name: Re-enable powersave
if: always()
run: |
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
- name: Create Pull Request
uses: peter-evans/create-pull-request@9153d834b60caba6d51c9b9510b087acf9f33f83
with:
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "[bot] Update warp pkl for modeld_v2"
title: "[bot] Update modeld_v2 Warp"
branch: "auto/compile-warp-kernels"
base: "master"
delete-branch: true
labels: bot
add-paths: |
openpilot/sunnypilot/modeld_v2/models/*.pkl
+9 -8
View File
@@ -18,24 +18,25 @@ concurrency:
env:
GIT_CONFIG_COUNT: 1
GIT_CONFIG_KEY_0: lfs.fetchexclude
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
jobs:
docs:
name: build docs
runs-on: ubuntu-24.04
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
steps:
- uses: commaai/timeout@v1
- uses: actions/checkout@v7
with:
submodules: true
- run: ./tools/op.sh setup
# Build
- name: Build docs
run: |
git lfs pull
python docs/serve.py --build
run: ./tools/op.sh docs --build
# Push to docs.comma.ai
- uses: actions/checkout@v7
@@ -1,5 +1,5 @@
name: Download HF model chunks
description: Resolve and download model chunks from HuggingFace in parallel
name: Download HF model
description: Resolve and download models from HuggingFace in parallel
inputs:
hf_repo:
@@ -11,16 +11,21 @@ inputs:
dest_dir:
description: Destination directory for downloaded chunks
required: true
tinygrad_ref:
description: 'Tinygrad ref to match bundle against'
required: false
default: ''
runs:
using: composite
steps:
- name: Download model chunks
- name: Download models
shell: bash
env:
HF_REPO: ${{ inputs.hf_repo }}
MODELS_JSON: ${{ inputs.models }}
DEST_DIR: ${{ inputs.dest_dir }}
TINYGRAD_REF: ${{ inputs.tinygrad_ref }}
run: |
set -eo pipefail
DOWNLOAD_LIST=$(mktemp)
@@ -28,30 +33,41 @@ runs:
resolve_chunks() {
local HF_PATH="$1" ONNX_HASH="$2" CANONICAL="$3" DEST_DIR="$4"
local JSON_URL="https://huggingface.co/datasets/${HF_REPO}/resolve/main/${HF_PATH}/default_models.json"
local DEFAULTS BUNDLE ARTIFACT BASE_URL NUM_CHUNKS
local DEFAULTS BUNDLE ARTIFACT FILE_URL BASE_URL
DEFAULTS=$(curl -fsSL "$JSON_URL")
BUNDLE=$(echo "$DEFAULTS" | jq --arg hash "$ONNX_HASH" '.bundles[] | select(.onnx_sha256 == $hash)')
if [ -n "$TINYGRAD_REF" ]; then
BUNDLE=$(echo "$DEFAULTS" | jq --arg hash "$ONNX_HASH" --arg tg "$TINYGRAD_REF" '
. as $root | [.bundles[] | select(.onnx_sha256 == $hash and (.tinygrad_ref == $tg or (.tinygrad_ref == null and $root.tinygrad_ref == $tg)))] | .[0] // empty
')
else
BUNDLE=$(echo "$DEFAULTS" | jq --arg hash "$ONNX_HASH" '
[.bundles[] | select(.onnx_sha256 == $hash)] | .[0] // empty
')
fi
if [ -z "$BUNDLE" ] || [ "$BUNDLE" = "null" ]; then
echo "::error::No model bundle found matching ONNX hash $ONNX_HASH (tinygrad ref: $TINYGRAD_REF) in $HF_PATH"
return 1
fi
ARTIFACT=$(echo "$BUNDLE" | jq -r '.models[0].artifact')
BASE_URL=$(echo "$ARTIFACT" | jq -r '.download_uri.url' | sed 's|/[^/]*$||')
NUM_CHUNKS=$(echo "$ARTIFACT" | jq -r '.chunks | length')
FILE_URL=$(echo "$ARTIFACT" | jq -r '.download_uri.url')
BASE_URL=$(echo "$FILE_URL" | sed 's|/[^/]*$||')
mkdir -p "$DEST_DIR"
while IFS= read -r CHUNK_NAME; do
CHUNK_IDX=$(echo "$CHUNK_NAME" | grep -oP 'chunk\K[0-9]+of[0-9]+' || true)
if [ -z "$CHUNK_IDX" ]; then
echo "::error::Failed to parse chunk index from: $CHUNK_NAME"
return 1
fi
ENCODED_URL=$(python3 -c "import urllib.parse; print(urllib.parse.quote('${BASE_URL}/${CHUNK_NAME}', safe=':/'))")
printf '%s\t%s\n' "$ENCODED_URL" "${DEST_DIR}/${CANONICAL}.chunk${CHUNK_IDX}" >> "$DOWNLOAD_LIST"
done < <(echo "$ARTIFACT" | jq -r '.chunks[].file_name')
echo "$NUM_CHUNKS" > "${DEST_DIR}/${CANONICAL}.chunkmanifest"
ENCODED_URL=$(python3 -c "import urllib.parse; print(urllib.parse.quote('${FILE_URL}', safe=':/'))")
printf '%s\t%s\n' "$ENCODED_URL" "${DEST_DIR}/${CANONICAL}" >> "$DOWNLOAD_LIST"
if [ "$CANONICAL" = "dmonitoring_model_tinygrad.pkl" ]; then
for warp in dm_warp_1928x1208_tinygrad.pkl dm_warp_1344x760_tinygrad.pkl; do
ENCODED_URL=$(python3 -c "import urllib.parse; print(urllib.parse.quote('${BASE_URL}/${warp}', safe=':/'))")
printf '%s\t%s\n' "$ENCODED_URL" "${DEST_DIR}/${warp}" >> "$DOWNLOAD_LIST"
done
elif [ "$CANONICAL" = "big_driving_tinygrad.pkl" ]; then
for warp in big_driving_warp_1928x1208_tinygrad.pkl big_driving_warp_1344x760_tinygrad.pkl; do
ENCODED_URL=$(python3 -c "import urllib.parse; print(urllib.parse.quote('${BASE_URL}/${warp}', safe=':/'))")
printf '%s\t%s\n' "$ENCODED_URL" "${DEST_DIR}/${warp}" >> "$DOWNLOAD_LIST"
done
fi
}
@@ -63,7 +79,7 @@ runs:
done
TOTAL=$(wc -l < "$DOWNLOAD_LIST")
echo "Downloading $TOTAL chunks with 8 parallel connections..."
echo "Downloading $TOTAL files with 8 parallel connections..."
xargs -P8 -d'\n' -I{} bash -c '
URL="${1%% *}"
DEST="${1#* }"
+7 -9
View File
@@ -1,8 +1,8 @@
name: Sync comma's LFS
env:
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
LFS_URL: 'https://huggingface.co/sunnypilot/sunnypilot-lfs.git/info/lfs'
LFS_PUSH_URL: 'https://huggingface.co/sunnypilot/sunnypilot-lfs.git/info/lfs'
on:
schedule:
@@ -40,14 +40,12 @@ jobs:
git config --global user.name 'GitHub Action'
git config --global user.email 'action@github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
- name: Setup hf
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
git config --global credential.helper store
echo "https://user:${HF_TOKEN}@huggingface.co" > ~/.git-credentials
- name: Ensure branch
run: |
-28
View File
@@ -1,28 +0,0 @@
name: Release Drafter
on:
push:
branches:
- master
tags:
- 'v*'
pull_request_target:
types: [opened, reopened, synchronize]
workflow_dispatch:
permissions:
contents: read
jobs:
update_release_draft:
permissions:
contents: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v6
with:
config-name: release-drafter.yml
prerelease: ${{ !startsWith(github.ref, 'refs/tags/v') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+5 -1
View File
@@ -7,7 +7,7 @@ on:
env:
GIT_CONFIG_COUNT: 1
GIT_CONFIG_KEY_0: lfs.fetchexclude
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
jobs:
build___nightly:
@@ -34,3 +34,7 @@ jobs:
- run: ./tools/op.sh setup
- name: Push __nightly
run: BRANCH=__nightly tools/release/build_stripped.sh
- name: Push chestnut nightly
run: |
git lfs pull --exclude=''
INCLUDE_BIG_MODEL=1 BRANCH=__nightly-chestnut tools/release/build_stripped.sh
+1 -1
View File
@@ -11,7 +11,7 @@ env:
PYTHONPATH: ${{ github.workspace }}
GIT_CONFIG_COUNT: 1
GIT_CONFIG_KEY_0: lfs.fetchexclude
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
jobs:
package_updates:
+108 -70
View File
@@ -100,24 +100,56 @@ jobs:
commit_date=$(git log -1 --format=%cd --date=format:'%B %d, %Y')
echo "model_date=${commit_date}" >> $GITHUB_OUTPUT
cat $GITHUB_OUTPUT
- run: |
- name: Fetch Models
run: |
cd ${{ github.workspace }}/openpilot/openpilot
fetch_lfs() {
sudo apt-get install -y -qq aria2 > /dev/null # Parallel ranges because gh is flaky
LFS_URL=$(git config -f "$(git rev-parse --show-toplevel)/.lfsconfig" lfs.url)
for f in "$@"; do
grep -qs "^version https://git-lfs" "$f" || continue
OID=$(awk -F'sha256:' '/^oid/{print $2}' "$f"); SIZE=$(awk '/^size/{print $2}' "$f")
HREF=$(curl -fsSL --retry 3 -X POST -H "Accept: application/vnd.git-lfs+json" -H "Content-Type: application/vnd.git-lfs+json" "$LFS_URL/objects/batch" \
-d "{\"operation\":\"download\",\"transfer\":[\"basic\"],\"objects\":[{\"oid\":\"$OID\",\"size\":$SIZE}]}" | jq -r '.objects[0].actions.download.href')
aria2c -x 16 -s 16 -k 16M --max-tries=5 --retry-wait=5 --file-allocation=none --summary-interval=30 --allow-overwrite=true \
-d "$(dirname "$f")" -o "$(basename "$f")" "$HREF"
echo "$OID $f" | sha256sum -c -
done
}
UPSTREAM_PKL=selfdrive/modeld/models/big_driving_tinygrad.pkl
if [ "${{ inputs.target_hardware }}" != "chestnut" ]; then
git lfs pull -X "**/selfdrive/modeld/models/big_*.onnx,**/selfdrive/modeld/models/dmonitoring_*.onnx"
rm -f selfdrive/modeld/models/big_*.onnx selfdrive/modeld/models/dmonitoring_*.onnx
git lfs pull -X "**/selfdrive/modeld/models/big_*.onnx,**/selfdrive/modeld/models/dmonitoring_*.onnx,**/selfdrive/modeld/models/big_*.pkl,**/selfdrive/modeld/models/dmonitoring_*.pkl"
rm -f selfdrive/modeld/models/big_*.onnx selfdrive/modeld/models/dmonitoring_*.onnx selfdrive/modeld/models/big_*.pkl selfdrive/modeld/models/dmonitoring_*.pkl
elif compgen -G "selfdrive/modeld/models/big_*.onnx" > /dev/null; then
fetch_lfs selfdrive/modeld/models/big_*.onnx selfdrive/modeld/models/big_*.pkl
find selfdrive/modeld/models -type f \( -name "*.onnx" -o -name "*.pkl" \) ! -name "big_*.onnx" ! -name "big_*.pkl" -delete
elif [ -f "$UPSTREAM_PKL" ]; then
echo "No big model ONNX tracked: converting commaai's precompiled $UPSTREAM_PKL, with the tinygrad it was built with"
fetch_lfs "$UPSTREAM_PKL"
find selfdrive/modeld/models -type f \( -name "*.onnx" -o -name "*.pkl" \) ! -path "$UPSTREAM_PKL" -delete
GITMODULES="$(git rev-parse --show-toplevel)/.gitmodules"
TG_NAME=$(git config -f "$GITMODULES" --get-regexp '^submodule\..*\.path$' | awk '$2 == "tinygrad_repo" {sub(/^submodule\./, "", $1); sub(/\.path$/, "", $1); print $1}')
TG_URL=$(git config -f "$GITMODULES" "submodule.$TG_NAME.url" | sed 's|^\.\./\.\./|https://github.com/|')
echo "$TG_URL $(git rev-parse HEAD:tinygrad_repo) $(sha256sum "$UPSTREAM_PKL" | cut -d' ' -f1)" > selfdrive/modeld/models/upstream_pkl.txt
else
git lfs pull -I "**/selfdrive/modeld/models/big_*.onnx" -X ""
find selfdrive/modeld/models -name "*.onnx" ! -name "big_*.onnx" -delete
echo "::error::no big model ONNX or precompiled big_driving_tinygrad.pkl at this ref"
exit 1
fi
if grep -lIF "version https://git-lfs.github.com/spec/v1" selfdrive/modeld/models/*.onnx; then
echo "::error::the ONNX files above are still LFS pointers, not real models"
if grep -lIF "version https://git-lfs.github.com/spec/v1" selfdrive/modeld/models/*.onnx selfdrive/modeld/models/*.pkl 2>/dev/null; then
echo "::error::the ONNX or PKL files above are still LFS pointers, not real models"
exit 1
fi
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: models-${{ env.REF }}${{ inputs.artifact_suffix }}
path: ${{ github.workspace }}/openpilot/openpilot/selfdrive/modeld/models/*.onnx
compression-level: 0
path: |
${{ github.workspace }}/openpilot/openpilot/selfdrive/modeld/models/*.onnx
${{ github.workspace }}/openpilot/openpilot/selfdrive/modeld/models/*.pkl
${{ github.workspace }}/openpilot/openpilot/selfdrive/modeld/models/upstream_pkl.txt
if-no-files-found: error
build_model:
@@ -132,7 +164,6 @@ jobs:
fetch-depth: 1
submodules: recursive
- name: Set environment variables
id: set-env
run: |
@@ -144,9 +175,6 @@ jobs:
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
uv sync --frozen
printenv >> $GITHUB_ENV
if [[ "${{ runner.debug }}" == "1" ]]; then
cat $GITHUB_OUTPUT
fi
- name: Setup build environment
run: |
@@ -158,10 +186,6 @@ jobs:
echo "VERSION: ${{ steps.set-env.outputs.version }}"
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
echo "-------"
if [[ "${{ runner.debug }}" == "1" ]]; then
printenv
fi
source ${UV_PROJECT_ENVIRONMENT}/bin/activate
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
rm -rf ${{ env.MODELS_DIR }}/*.onnx*
@@ -184,11 +208,11 @@ jobs:
MODEL_SIZE=$(python3 -c "from openpilot.common.transformations.model import MEDMODEL_INPUT_SIZE as s; print(f'{s[0]}x{s[1]}')")
CAMERA_RES=$(python3 -c "from openpilot.common.transformations.camera import _ar_ox_fisheye as a, _os_fisheye as o; print(f'{a.width}x{a.height} {o.width}x{o.height}')")
TG_FLAGS_QCOM="DEV=QCOM IMAGE=1 FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 OPENPILOT_HACKS=1"
TG_FLAGS_QCOM="DEV=QCOM IMAGE=1 FLOAT16=1 NOLOCALS=1 OPENPILOT_HACKS=1"
if [ "${{ inputs.target_hardware }}" == "chestnut" ]; then
echo "CHESTNUT build"
export CHESTNUT=1
TG_FLAGS="DEBUG=1 DEV=USB+AMD:LLVM FLOAT16=1 JIT_BATCH_SIZE=0 GMMU=0 TC_OPT=2 TC_OCCUPANCY_OPT=1"
TG_FLAGS="DEBUG=1 DEV=USB+AMD:LLVM FRAME_DEV=CPU FLOAT16=1 GMMU=0 TC_OPT=2 TC_MIN_GLOBALS=32"
OUTPUT_PKL="${{ env.MODELS_DIR }}/big_driving_tinygrad.pkl"
else
echo "QCOM build"
@@ -196,64 +220,77 @@ jobs:
OUTPUT_PKL="${{ env.MODELS_DIR }}/driving_tinygrad.pkl"
fi
# Generate metadata for all ONNX files
find "${{ env.MODELS_DIR }}" -maxdepth 1 -name '*.onnx' | while IFS= read -r onnx_file; do
echo "Generating metadata: $onnx_file"
env ${TG_FLAGS_QCOM} python3 "${{ env.MODELS_DIR }}/../get_model_metadata.py" "$onnx_file" || true
done
NATIVE_PKL=$(find "${{ env.MODELS_DIR }}" -maxdepth 1 -name "*.pkl" -print -quit)
# Detect model type and build compile args
VISION_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_vision.onnx" "${{ env.MODELS_DIR }}/big_driving_vision.onnx"; do
[ -f "$f" ] && VISION_ONNX="$f" && break
done
if [ -f "${{ env.MODELS_DIR }}/upstream_pkl.txt" ]; then
read -r TG_URL TG_SHA _ < "${{ env.MODELS_DIR }}/upstream_pkl.txt"
echo "Converting commaai's precompiled model with tinygrad $TG_SHA"
UPSTREAM_TG="${{ runner.temp }}/upstream_tinygrad"
rm -rf "$UPSTREAM_TG" && git init -q "$UPSTREAM_TG" && git -C "$UPSTREAM_TG" fetch -q --depth 1 "$TG_URL" "$TG_SHA" && git -C "$UPSTREAM_TG" checkout -q FETCH_HEAD
CONVERTED="${{ runner.temp }}/big_driving_tinygrad_plain.pkl"
env DEV=USB+AMD PYTHONPATH="$UPSTREAM_TG:${{ github.workspace }}" taskset -c 7 python3 openpilot/sunnypilot/modeld_v2/convert_upstream_pkl.py \
"${{ env.MODELS_DIR }}/big_driving_tinygrad.pkl" "$CONVERTED"
mv "$CONVERTED" "$OUTPUT_PKL"
elif NEW_ARCH_ONNX=$(find "${{ env.MODELS_DIR }}" -maxdepth 1 -name "${{ inputs.target_hardware == 'chestnut' && 'big_' || '' }}*driving*.onnx" -print -quit) &&
[ -n "$NEW_ARCH_ONNX" ] && grep -qa new_img "$NEW_ARCH_ONNX"; then # the new architecture: one ONNX taking the warped frames as new_img
echo "Compiling new architecture ONNX: $NEW_ARCH_ONNX"
env ${TG_FLAGS} taskset -c 7 python3 openpilot/sunnypilot/modeld_v2/compile_modeld_v2.py \
"$NEW_ARCH_ONNX" "$OUTPUT_PKL" --device-input "*" --benchmark-runs 1
else
echo "Detected onnx architecture. Routing to native compile_modeld.py."
# Detect model type and build compile args
VISION_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_vision.onnx" "${{ env.MODELS_DIR }}/big_driving_vision.onnx"; do
[ -f "$f" ] && VISION_ONNX="$f" && break
done
POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_policy.onnx"; do
[ -f "$f" ] && POLICY_ONNX="$f" && break
done
POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_policy.onnx"; do
[ -f "$f" ] && POLICY_ONNX="$f" && break
done
OFF_POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_off_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_off_policy.onnx"; do
[ -f "$f" ] && OFF_POLICY_ONNX="$f" && break
done
OFF_POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_off_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_off_policy.onnx"; do
[ -f "$f" ] && OFF_POLICY_ONNX="$f" && break
done
ON_POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_on_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_on_policy.onnx"; do
[ -f "$f" ] && ON_POLICY_ONNX="$f" && break
done
ON_POLICY_ONNX=""
for f in "${{ env.MODELS_DIR }}/driving_on_policy.onnx" "${{ env.MODELS_DIR }}/big_driving_on_policy.onnx"; do
[ -f "$f" ] && ON_POLICY_ONNX="$f" && break
done
SUPERCOMBO_ONNX=""
for f in "${{ env.MODELS_DIR }}/supercombo.onnx" "${{ env.MODELS_DIR }}/driving_supercombo.onnx" "${{ env.MODELS_DIR }}/big_supercombo.onnx" "${{ env.MODELS_DIR }}/big_driving_supercombo.onnx"; do
[ -f "$f" ] && SUPERCOMBO_ONNX="$f" && break
done
SUPERCOMBO_ONNX=""
for f in "${{ env.MODELS_DIR }}/supercombo.onnx" "${{ env.MODELS_DIR }}/driving_supercombo.onnx" "${{ env.MODELS_DIR }}/big_supercombo.onnx" "${{ env.MODELS_DIR }}/big_driving_supercombo.onnx"; do
[ -f "$f" ] && SUPERCOMBO_ONNX="$f" && break
done
MODEL_TYPE="" ONNX_ARGS="" OUTPUT_NAME=""
if [ -f "$VISION_ONNX" ]; then
ONNX_ARGS="--vision-onnx $VISION_ONNX"
if [ -f "$ON_POLICY_ONNX" ] && [ -f "$OFF_POLICY_ONNX" ]; then
MODEL_TYPE=vision_multi_policy
ONNX_ARGS="$ONNX_ARGS --off-policy-onnx $OFF_POLICY_ONNX --on-policy-onnx $ON_POLICY_ONNX"
elif [ -f "$OFF_POLICY_ONNX" ] && [ -f "$POLICY_ONNX" ]; then
MODEL_TYPE=vision_multi_policy
ONNX_ARGS="$ONNX_ARGS --policy-onnx $POLICY_ONNX --off-policy-onnx $OFF_POLICY_ONNX"
elif [ -f "$POLICY_ONNX" ]; then
MODEL_TYPE=vision_policy
ONNX_ARGS="$ONNX_ARGS --policy-onnx $POLICY_ONNX"
MODEL_TYPE="" ONNX_ARGS="" OUTPUT_NAME=""
if [ -f "$VISION_ONNX" ]; then
ONNX_ARGS="--vision-onnx $VISION_ONNX"
if [ -f "$ON_POLICY_ONNX" ] && [ -f "$OFF_POLICY_ONNX" ]; then
MODEL_TYPE=vision_multi_policy
ONNX_ARGS="$ONNX_ARGS --off-policy-onnx $OFF_POLICY_ONNX --on-policy-onnx $ON_POLICY_ONNX"
elif [ -f "$OFF_POLICY_ONNX" ] && [ -f "$POLICY_ONNX" ]; then
MODEL_TYPE=vision_multi_policy
ONNX_ARGS="$ONNX_ARGS --policy-onnx $POLICY_ONNX --off-policy-onnx $OFF_POLICY_ONNX"
elif [ -f "$POLICY_ONNX" ]; then
MODEL_TYPE=vision_policy
ONNX_ARGS="$ONNX_ARGS --policy-onnx $POLICY_ONNX"
fi
elif [ -f "$SUPERCOMBO_ONNX" ]; then
MODEL_TYPE=supercombo
ONNX_ARGS="--supercombo-onnx $SUPERCOMBO_ONNX"
fi
elif [ -f "$SUPERCOMBO_ONNX" ]; then
MODEL_TYPE=supercombo
ONNX_ARGS="--supercombo-onnx $SUPERCOMBO_ONNX"
fi
if [ -n "$MODEL_TYPE" ]; then
echo "Detected: $MODEL_TYPE -> $OUTPUT_PKL"
env ${TG_FLAGS} python3 "$COMPILE_MODELD" \
--model-type $MODEL_TYPE \
--model-size $MODEL_SIZE \
--camera-resolutions $CAMERA_RES \
$ONNX_ARGS \
--output "$OUTPUT_PKL"
if [ -n "$MODEL_TYPE" ]; then
echo "Detected: $MODEL_TYPE -> $OUTPUT_PKL"
env ${TG_FLAGS} taskset -c 7 python3 "$COMPILE_MODELD" \
--model-type $MODEL_TYPE \
--model-size $MODEL_SIZE \
--camera-resolutions $CAMERA_RES \
$ONNX_ARGS \
--output "$OUTPUT_PKL"
fi
fi
- name: Prepare Output
@@ -264,8 +301,7 @@ jobs:
# Copy the model files
rsync -avm \
--include='*.dlc' \
--include='*.chunk*' \
--include='*.chunkmanifest' \
--include='/${{ inputs.target_hardware == 'chestnut' && 'big_' || '' }}driving_tinygrad.pkl' \
--exclude='*' \
--delete-excluded \
--chown=comma:comma \
@@ -276,6 +312,7 @@ jobs:
--output-dir "${{ env.OUTPUT_DIR }}" \
--custom-name "${{ env.MODEL_NAME }}" \
--upstream-branch "${{ inputs.upstream_branch }}" \
$([ -f "${{ env.MODELS_DIR }}/upstream_pkl.txt" ] && echo "--onnx-sha256 $(cut -d' ' -f3 "${{ env.MODELS_DIR }}/upstream_pkl.txt")") \
${{ inputs.is_20hz && '--is-20hz' || '' }}
- name: Write artifact name to file
@@ -287,6 +324,7 @@ jobs:
with:
name: model-${{ env.MODEL_NAME }}${{ inputs.artifact_suffix }}-${{ github.run_number }}
path: ${{ env.OUTPUT_DIR }}
compression-level: 0
- name: Upload artifact name file
uses: actions/upload-artifact@v4
@@ -170,6 +170,9 @@ jobs:
scons -j1 cache_dir="$SCONS_CACHE" --minimal \
openpilot/selfdrive/locationd openpilot/sunnypilot/selfdrive/locationd
echo "Building rest of sunnypilot"
sudo rm -rf /tmp/* || true
mkdir -p "$BUILD_DIR/tmp"
export TMPDIR="$BUILD_DIR/tmp"
SKIP_TINYGRAD_COMPILE=1 /usr/bin/time -v scons -j$(nproc) cache_dir="$SCONS_CACHE" --minimal
touch ${BUILD_DIR}/prebuilt
if [[ "${{ runner.debug }}" == "1" ]]; then
@@ -221,37 +224,40 @@ jobs:
cancel-in-progress: false
outputs:
onnx_sha256: ${{ steps.resolve.outputs.onnx_sha256 }}
tinygrad_ref: ${{ steps.resolve.outputs.tinygrad_ref }}
env:
GH_REPO: ${{ github.repository }}
HF_REPO: sunnypilot/sunnypilot_models_v1
HF_DEFAULTS_PATH: models/defaults/big
steps:
- name: Resolve ONNX hash and tinygrad ref via API
- name: Resolve tinygrad ref via API
id: resolve
run: |
REF="${{ github.head_ref || github.ref_name }}"
BLOB_SHA=$(gh api "repos/${GH_REPO}/contents/openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx?ref=${REF}" --jq '.sha')
BLOB_SHA=$(gh api "repos/${GH_REPO}/contents/openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl?ref=${REF}" --jq '.sha')
[ -n "$BLOB_SHA" ] || { echo "::error::Failed to resolve big_driving_tinygrad.pkl blob SHA"; exit 1; }
ONNX_HASH=$(gh api "repos/${GH_REPO}/git/blobs/${BLOB_SHA}" --jq '.content' | base64 -d | grep '^oid sha256:' | cut -d: -f2)
echo "ONNX hash: $ONNX_HASH"
[ -n "$ONNX_HASH" ] || { echo "::error::Failed to extract ONNX hash"; exit 1; }
echo "onnx_sha256=$ONNX_HASH" >> $GITHUB_OUTPUT
echo "big PKL hash: $ONNX_HASH"
[ -n "$ONNX_HASH" ] || { echo "::error::Failed to extract big PKL hash"; exit 1; }
echo "onnx_sha256=$ONNX_HASH" >> "$GITHUB_OUTPUT"
TINYGRAD_REF=$(gh api "repos/${GH_REPO}/contents/tinygrad_repo?ref=${REF}" --jq '.sha')
echo "tinygrad ref: $TINYGRAD_REF"
echo "tinygrad_ref=$TINYGRAD_REF" >> "$GITHUB_OUTPUT"
JSON_URL="https://huggingface.co/datasets/${HF_REPO}/resolve/main/${HF_DEFAULTS_PATH}/default_models.json"
check_defaults() {
DEFAULTS=$(curl -fsSL "${JSON_URL}?t=$(date +%s)" 2>/dev/null) || return 1
TINYGRAD_MATCH=$(echo "$DEFAULTS" | jq -r --arg ref "$TINYGRAD_REF" '.tinygrad_ref == $ref' 2>/dev/null)
[ "$TINYGRAD_MATCH" = "true" ] || return 1
BUNDLE=$(echo "$DEFAULTS" | jq --arg hash "$ONNX_HASH" '.bundles[] | select(.onnx_sha256 == $hash)' 2>/dev/null)
BUNDLE=$(echo "$DEFAULTS" | jq --arg hash "$ONNX_HASH" --arg ref "$TINYGRAD_REF" '
. as $root | [.bundles[] | select(.onnx_sha256 == $hash and (.tinygrad_ref == $ref or (.tinygrad_ref == null and $root.tinygrad_ref == $ref)))] | .[0] // empty
' 2>/dev/null)
[ -n "$BUNDLE" ] && [ "$BUNDLE" != "null" ]
}
if check_defaults; then
echo "HF defaults match repo ONNX hash and tinygrad ref"
echo "HF defaults match repo tinygrad ref"
exit 0
fi
@@ -304,12 +310,13 @@ jobs:
cancel-in-progress: false
outputs:
driving_onnx_sha256: ${{ steps.resolve.outputs.driving_onnx_sha256 }}
tinygrad_ref: ${{ steps.resolve.outputs.tinygrad_ref }}
env:
GH_REPO: ${{ github.repository }}
HF_REPO: sunnypilot/sunnypilot_models_v1
HF_DEFAULTS_PATH: models/defaults/small
steps:
- name: Resolve ONNX hash and tinygrad ref via API
- name: Resolve tinygrad ref via API
id: resolve
run: |
REF="${{ github.head_ref || github.ref_name }}"
@@ -318,23 +325,24 @@ jobs:
DRIVING_HASH=$(gh api "repos/${GH_REPO}/git/blobs/${BLOB_SHA}" --jq '.content' | base64 -d | grep '^oid sha256:' | cut -d: -f2)
echo "Driving ONNX hash: $DRIVING_HASH"
[ -n "$DRIVING_HASH" ] || { echo "::error::Failed to extract driving ONNX hash"; exit 1; }
echo "driving_onnx_sha256=$DRIVING_HASH" >> $GITHUB_OUTPUT
echo "driving_onnx_sha256=$DRIVING_HASH" >> "$GITHUB_OUTPUT"
TINYGRAD_REF=$(gh api "repos/${GH_REPO}/contents/tinygrad_repo?ref=${REF}" --jq '.sha')
echo "tinygrad ref: $TINYGRAD_REF"
echo "tinygrad_ref=$TINYGRAD_REF" >> "$GITHUB_OUTPUT"
JSON_URL="https://huggingface.co/datasets/${HF_REPO}/resolve/main/${HF_DEFAULTS_PATH}/default_models.json"
check_defaults() {
DEFAULTS=$(curl -fsSL "${JSON_URL}?t=$(date +%s)" 2>/dev/null) || return 1
TINYGRAD_MATCH=$(echo "$DEFAULTS" | jq -r --arg ref "$TINYGRAD_REF" '.tinygrad_ref == $ref' 2>/dev/null)
[ "$TINYGRAD_MATCH" = "true" ] || return 1
DRIVING=$(echo "$DEFAULTS" | jq --arg hash "$DRIVING_HASH" '.bundles[] | select(.onnx_sha256 == $hash)' 2>/dev/null)
[ -n "$DRIVING" ] && [ "$DRIVING" != "null" ] || return 1
DRIVING=$(echo "$DEFAULTS" | jq --arg hash "$DRIVING_HASH" --arg ref "$TINYGRAD_REF" '
. as $root | [.bundles[] | select(.onnx_sha256 == $hash and (.tinygrad_ref == $ref or (.tinygrad_ref == null and $root.tinygrad_ref == $ref)))] | .[0] // empty
' 2>/dev/null)
[ -n "$DRIVING" ] && [ "$DRIVING" != "null" ]
}
if check_defaults; then
echo "HF defaults match repo ONNX hash and tinygrad ref"
echo "HF defaults match repo tinygrad ref"
exit 0
fi
@@ -387,12 +395,13 @@ jobs:
cancel-in-progress: false
outputs:
dm_onnx_sha256: ${{ steps.resolve.outputs.dm_onnx_sha256 }}
tinygrad_ref: ${{ steps.resolve.outputs.tinygrad_ref }}
env:
GH_REPO: ${{ github.repository }}
HF_REPO: sunnypilot/sunnypilot_models_v1
HF_DEFAULTS_PATH: models/defaults/dm
steps:
- name: Resolve ONNX hash and tinygrad ref via API
- name: Resolve tinygrad ref via API
id: resolve
run: |
REF="${{ github.head_ref || github.ref_name }}"
@@ -401,23 +410,24 @@ jobs:
DM_HASH=$(gh api "repos/${GH_REPO}/git/blobs/${BLOB_SHA}" --jq '.content' | base64 -d | grep '^oid sha256:' | cut -d: -f2)
echo "DM ONNX hash: $DM_HASH"
[ -n "$DM_HASH" ] || { echo "::error::Failed to extract DM ONNX hash"; exit 1; }
echo "dm_onnx_sha256=$DM_HASH" >> $GITHUB_OUTPUT
echo "dm_onnx_sha256=$DM_HASH" >> "$GITHUB_OUTPUT"
TINYGRAD_REF=$(gh api "repos/${GH_REPO}/contents/tinygrad_repo?ref=${REF}" --jq '.sha')
echo "tinygrad ref: $TINYGRAD_REF"
echo "tinygrad_ref=$TINYGRAD_REF" >> "$GITHUB_OUTPUT"
JSON_URL="https://huggingface.co/datasets/${HF_REPO}/resolve/main/${HF_DEFAULTS_PATH}/default_models.json"
check_defaults() {
DEFAULTS=$(curl -fsSL "${JSON_URL}?t=$(date +%s)" 2>/dev/null) || return 1
TINYGRAD_MATCH=$(echo "$DEFAULTS" | jq -r --arg ref "$TINYGRAD_REF" '.tinygrad_ref == $ref' 2>/dev/null)
[ "$TINYGRAD_MATCH" = "true" ] || return 1
DM=$(echo "$DEFAULTS" | jq --arg hash "$DM_HASH" '.bundles[] | select(.onnx_sha256 == $hash)' 2>/dev/null)
[ -n "$DM" ] && [ "$DM" != "null" ] || return 1
DM=$(echo "$DEFAULTS" | jq --arg hash "$DM_HASH" --arg ref "$TINYGRAD_REF" '
. as $root | [.bundles[] | select(.onnx_sha256 == $hash and (.tinygrad_ref == $ref or (.tinygrad_ref == null and $root.tinygrad_ref == $ref)))] | .[0] // empty
' 2>/dev/null)
[ -n "$DM" ] && [ "$DM" != "null" ]
}
if check_defaults; then
echo "HF defaults match DM ONNX hash and tinygrad ref"
echo "HF defaults match DM tinygrad ref"
exit 0
fi
@@ -493,11 +503,12 @@ jobs:
mkdir -p ${{ env.OUTPUT_DIR }}
tar xzf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }}
- name: Download model chunks from HF
uses: ./.github/workflows/download-hf-model-chunks
- name: Download models from HF
uses: ./.github/workflows/download-hf-model
with:
hf_repo: sunnypilot/sunnypilot_models_v1
dest_dir: ${{ env.OUTPUT_DIR }}/openpilot/selfdrive/modeld/models
tinygrad_ref: ${{ needs.prepare_dm_model.outputs.tinygrad_ref }}
models: |
[
{"hf_path": "models/defaults/small", "onnx_hash": "${{ needs.prepare_small_model.outputs.driving_onnx_sha256 }}", "canonical": "driving_tinygrad.pkl"},
@@ -562,11 +573,12 @@ jobs:
mkdir -p ${{ env.OUTPUT_DIR }}
tar xzf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }}
- name: Download model chunks from HF
uses: ./.github/workflows/download-hf-model-chunks
- name: Download models from HF
uses: ./.github/workflows/download-hf-model
with:
hf_repo: sunnypilot/sunnypilot_models_v1
dest_dir: ${{ env.OUTPUT_DIR }}/openpilot/selfdrive/modeld/models
tinygrad_ref: ${{ needs.prepare_dm_model.outputs.tinygrad_ref }}
models: |
[
{"hf_path": "models/defaults/small", "onnx_hash": "${{ needs.prepare_small_model.outputs.driving_onnx_sha256 }}", "canonical": "driving_tinygrad.pkl"},
-78
View File
@@ -1,78 +0,0 @@
name: Debug Discourse Posting
on:
push:
jobs:
test-discourse-post:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Post test message to Discourse
uses: ./.github/workflows/post-to-discourse
with:
discourse-url: ${{ vars.DISCOURSE_URL }}
api-key: ${{ secrets.DISCOURSE_API_KEY }}
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
message: |
## 🧪 Test Post from GitHub Actions
**This is a test post to verify Discourse integration**
- **Workflow**: ${{ github.workflow }}
- **Run Number**: #${{ github.run_number }}
- **Branch**: `${{ github.ref_name }}`
- **Commit**: ${{ github.sha }}
- **Actor**: @${{ github.actor }}
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
---
### Fake Build Info (for testing)
- **Version**: 0.9.8-test
- **Build**: #42
- **Branch**: release-test
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
*This is an automated test message. Drive safe! 🚗💨*
- name: Create topic on Discourse
uses: ./.github/workflows/post-to-discourse
with:
discourse-url: ${{ vars.DISCOURSE_URL }}
api-key: ${{ secrets.DISCOURSE_API_KEY }}
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
#topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
category-id: 4
title: "This is a test of a new topic instead of a reply"
message: |
## 🧪 Test Post from GitHub Actions
**This is a test post to verify Discourse integration**
- **Workflow**: ${{ github.workflow }}
- **Run Number**: #${{ github.run_number }}
- **Branch**: `${{ github.ref_name }}`
- **Commit**: ${{ github.sha }}
- **Actor**: @${{ github.actor }}
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
---
### Fake Build Info (for testing)
- **Version**: 0.9.8-test
- **Build**: #42
- **Branch**: release-test
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
*This is an automated test message. Drive safe! 🚗💨*
- name: Display results
if: always()
run: |
echo "::notice::Discourse post test completed"
echo "Check your Discourse topic to verify the post appeared correctly"
+69
View File
@@ -0,0 +1,69 @@
name: Test Models Compatibility With Tinygrad Changes
on:
pull_request:
paths:
- 'tinygrad_repo'
workflow_dispatch:
jobs:
generate-matrix:
runs-on: ubuntu-latest
outputs:
models: ${{ steps.set-matrix.outputs.models }}
steps:
- uses: actions/checkout@v4
- name: Fetch and Parse json
id: set-matrix
run: |
python3 -c '
import json, urllib.request, os, re
with open("openpilot/sunnypilot/models/fetcher.py", "r") as f:
urls = re.findall(r"MODEL_URL(?:_CHESTNUT)?\s*=\s*[\"'"'"']([^\"'"'"']+)[\"'"'"']", f.read())
artifacts = []
for url in urls:
data = json.loads(urllib.request.urlopen(url).read())
for bundle in data.get("bundles", []):
for model in bundle.get("models", []):
if "artifact" in model:
artifacts.append(model["artifact"])
with open(os.environ["GITHUB_OUTPUT"], "a") as f:
f.write(f"models={json.dumps(artifacts)}\n")
'
test-model:
name: Test ${{ matrix.artifact.file_name }}
needs: generate-matrix
runs-on: ubuntu-latest
container: ghcr.io/commaai/openpilot-base:latest
strategy:
fail-fast: false
matrix:
artifact: ${{ fromJson(needs.generate-matrix.outputs.models) }}
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: Download model
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
MODEL_URL: ${{ matrix.artifact.download_uri.url }}
MODEL_FILE: ${{ matrix.artifact.file_name }}
shell: bash
run: |
mkdir -p /tmp/models
AUTH=()
if [ -n "$HF_TOKEN" ]; then AUTH=(-H "Authorization: Bearer $HF_TOKEN"); fi
curl --retry 3 --retry-all-errors -s -S -f -L "${AUTH[@]}" -o "/tmp/models/$MODEL_FILE" "$MODEL_URL"
- name: Run Model Compatibility Test
env:
MODEL_BASE_NAME: ${{ matrix.artifact.file_name }}
MODEL_DIR: "/tmp/models"
run: |
export PYTHONPATH="$GITHUB_WORKSPACE/tinygrad_repo:$GITHUB_WORKSPACE:$PYTHONPATH"
python3 -m pip uninstall -y tinygrad || uv pip uninstall -y tinygrad --system || true
python3 -m pytest openpilot/sunnypilot/modeld_v2/tests/test_models.py
+1 -1
View File
@@ -22,7 +22,7 @@ env:
PYTHONPATH: ${{ github.workspace }}
GIT_CONFIG_COUNT: 1
GIT_CONFIG_KEY_0: lfs.fetchexclude
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx
GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
jobs:
build_release:
+2
View File
@@ -50,6 +50,8 @@ st[0-9A-Za-z][0-9A-Za-z][0-9A-Za-z][0-9A-Za-z][0-9A-Za-z][0-9A-Za-z]
*.stats
*.pkl
*.pkl*
!openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
!openpilot/sunnypilot/modeld_v2/models/*.pkl
config.json
compile_commands.json
compare_runtime*.html
+1
View File
@@ -19,3 +19,4 @@
[submodule "sunnypilot/neural_network_data"]
path = openpilot/sunnypilot/neural_network_data
url = https://github.com/sunnypilot/neural-network-data.git
+3 -2
View File
@@ -1,4 +1,5 @@
[lfs]
url = https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git
url = https://huggingface.co/sunnypilot/sunnypilot-lfs.git/info/lfs
pushurl = https://huggingface.co/sunnypilot/sunnypilot-lfs.git/info/lfs
locksverify = false
Vendored
+62 -4
View File
@@ -12,16 +12,17 @@ def retryWithDelay(int maxRetries, int delay, Closure body) {
def device(String ip, String step_label, String cmd) {
withCredentials([file(credentialsId: 'id_rsa', variable: 'key_file')]) {
def ssh_cmd = """
ssh -o ControlMaster=auto -o ControlPath=/tmp/ssh_control_%C -o ControlPersist=yes -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec /usr/bin/bash <<'END'
ssh -o ControlMaster=no -o ControlPath=none -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=12 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec setpriv --pdeathsig HUP /usr/bin/bash <<'END'
set -e
export TERM=xterm-256color
shopt -s huponexit # kill all child processes when the shell exits
trap 'kill 0' HUP # stop this process group on SSH disconnect
export CI=1
export PYTHONWARNINGS=error
export PYTHONFAULTHANDLER=1
export COMMA_CACHE=/data/tmp/comma_download_cache
#export LOGPRINT=debug # this has gotten too spammy...
export TEST_DIR=${env.TEST_DIR}
@@ -69,7 +70,8 @@ export LD_LIBRARY_PATH="\$(python -c 'import ffmpeg; print(ffmpeg.LIB_DIR)'):/us
ln -snf ${env.TEST_DIR} /data/pythonpath
cd ${env.TEST_DIR} || true
time ${cmd}
time ( ${cmd} ) &
wait \$!
END"""
sh script: ssh_cmd, label: step_label
@@ -159,6 +161,14 @@ def step(String name, String cmd, Map args = [:]) {
return [name, cmd, args]
}
def chestnutStage(String name = "chestnut") {
deviceStage(name, "mici-chestnut-ci", ["UNSAFE=1", "CHESTNUT=1"], [
step("build", "./openpilot/selfdrive/test/chestnut.sh"),
step("model replay", "openpilot/selfdrive/test/process_replay/model_replay.py --chestnut"),
step("onroad tests", "./openpilot/selfdrive/test/test_onroad.py TestChestnutOnroad", [timeout: 120]),
])
}
node {
env.CI = "1"
env.PYTHONWARNINGS = "error"
@@ -169,7 +179,37 @@ node {
env.GIT_BRANCH = checkout(scm).GIT_BRANCH
env.GIT_COMMIT = checkout(scm).GIT_COMMIT
def excludeBranches = ['__nightly', 'devel', 'devel-staging',
if (env.JOB_BASE_NAME == 'chestnut-stresstest') {
sh 'git fetch --no-tags origin nightly-chestnut'
env.GIT_COMMIT = sh(script: 'git show FETCH_HEAD:git_src_commit', returnStdout: true).trim()
if (!(env.GIT_COMMIT ==~ /[0-9a-f]{40}/)) {
error("invalid nightly-chestnut source commit")
}
def results = []
try {
timeout(time: 12, unit: 'HOURS') {
for (int run = 1; run <= 100; run++) {
results << "run ${run}: INCOMPLETE"
try {
chestnutStage("chestnut ${run}/100")
results[run - 1] = "run ${run}: PASS"
} catch (hudson.AbortException e) {
results[run - 1] = "run ${run}: FAIL"
echo e.toString()
}
}
}
} finally {
writeFile file: 'chestnut_stress_report.txt', text: "commit: ${env.GIT_COMMIT}\ncompleted: ${results.count { !it.endsWith('INCOMPLETE') }}/100\n" + results.join('\n') + '\n'
archiveArtifacts artifacts: 'chestnut_stress_report.txt'
}
if (results.any { !it.endsWith('PASS') }) {
error("chestnut stress test failed")
}
return
}
def excludeBranches = ['__nightly', '__nightly-chestnut', 'devel', 'devel-staging',
'release-tizi', 'release-tizi-staging', 'release-mici', 'release-mici-staging', 'testing-closet*', 'hotfix-*']
def excludeRegex = excludeBranches.join('|').replaceAll('\\*', '.*')
@@ -201,6 +241,21 @@ node {
)
}
if (env.BRANCH_NAME == '__nightly-chestnut') {
parallel (
'nightly-chestnut': {
deviceStage("build nightly-chestnut", "mici-chestnut-ci", ["CHESTNUT=1"], [
step("build nightly-chestnut", "SCONSFLAGS=-j4 INCLUDE_BIG_MODEL=1 RELEASE_BRANCH=nightly-chestnut $SOURCE_DIR/tools/release/build_release.sh TestChestnutOnroad"),
])
},
'nightly-chestnut-dev': {
deviceStage("build nightly-chestnut-dev", "mici-chestnut-ci", ["CHESTNUT=1"], [
step("build nightly-chestnut-dev", "SCONSFLAGS=-j4 INCLUDE_BIG_MODEL=1 PANDA_DEBUG_BUILD=1 RELEASE_BRANCH=nightly-chestnut-dev $SOURCE_DIR/tools/release/build_release.sh TestChestnutOnroad"),
])
},
)
}
if (!env.BRANCH_NAME.matches(excludeRegex)) {
parallel (
'onroad tests': {
@@ -252,6 +307,9 @@ node {
step("test amp", "./openpilot/common/hardware/comma/tests/test_amplifier.py"),
])
},
'chestnut': {
chestnutStage()
},
)
}
+11 -1
View File
@@ -1,4 +1,14 @@
Version 0.11.2 (2026-08-12)
Version 0.11.3 (2026-XX-XX)
========================
* Autospeed
* On supported cars, openpilot will intelligently decide what speed to drive based on the scene, including:
* stopping at red lights, and going at green lights
* slowing down for curves
* adjusting speed based on surrounding traffic
* This behavior was previously available with openpilot's "Experimental mode", but is now out of beta and always enabled
* With openpilot deciding which speed to drive, the car's cruise speed is treated as a maximum speed openpilot will not exceed
Version 0.11.2 (2026-10-14)
=======================
* New driving model
* Big model with 880M parameters
+2 -13
View File
@@ -20,7 +20,7 @@ SetOption('num_jobs', max(1, int(os.cpu_count()/(1 if "CI" in os.environ else 2)
AddOption('--ccflags', action='store', type='string', default='', help='pass arbitrary flags over the command line')
AddOption('--verbose', action='store_true', default=False, help='show full build commands')
release = not os.path.exists(File('#.gitattributes').abspath) # file absent on release branch, see release_files.py
release = not os.path.exists(File('#.gitmodules').abspath) # file absent on release branch, see release_files.py
AddOption('--minimal',
action='store_false',
dest='extras',
@@ -87,7 +87,6 @@ acados_include_dirs = [
# vendored in commaai/dependencies.
allowed_system_libs = {
"EGL", "GLESv2", "GL",
"Qt5Charts", "Qt5Core", "Qt5Gui", "Qt5Widgets",
"dl", "drm", "gbm", "m", "pthread",
}
@@ -121,6 +120,7 @@ def _libflags(target, source, env, for_signature):
env = Environment(
ENV={
"PATH": os.environ['PATH'],
"TMPDIR": os.environ.get('TMPDIR', '/tmp'),
"PYTHONPATH": os.pathsep.join(submodule_python_paths),
"ACADOS_SOURCE_DIR": acados.DIR,
"ACADOS_PYTHON_INTERFACE_PATH": acados.TEMPLATE_DIR,
@@ -316,8 +316,6 @@ def count_scons_nodes(nodes):
if node in seen:
continue
seen.add(node)
if hasattr(node, 'has_builder') and node.has_builder():
build_product_nodes.add(node)
executor = node.get_executor()
if executor is not None:
stack += executor.get_all_prerequisites() + executor.get_all_children()
@@ -326,7 +324,6 @@ def count_scons_nodes(nodes):
progress_interval = 5
progress_count = 0
build_product_nodes = set()
progress_total = max(1, count_scons_nodes(env.arg2nodes(BUILD_TARGETS or [Dir('.')], env.fs.Entry)))
def progress_function(node):
@@ -342,11 +339,3 @@ def progress_function(node):
Progress(progress_function, interval=progress_interval)
AddPostAction(BUILD_TARGETS or [Dir('.')], prune_cache_dir)
def check_build_product_size(target, source, env):
limit = 50 * 1024 * 1024 # GitHub max size
for t in target:
if hasattr(t, 'isfile') and t.isfile() and (size := os.path.getsize(t.abspath)) > limit:
raise SCons.Errors.UserError(f"{t} is {size / (1024 * 1024):.1f} MiB, exceeding the {limit / (1024 * 1024):.1f} MiB limit")
if not GetOption('extras'):
AddPostAction(list(build_product_nodes), Action(check_build_product_size, None))
+7 -7
View File
@@ -34,7 +34,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Chrysler|Pacifica Hybrid 2019-25|Adaptive Cruise Control (ACC)|Stock|0 mph|39 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 FCA connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Chrysler Pacifica Hybrid 2019-25">Buy Here</a></sub></details>|||
|comma|body|All|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|None|<a href="https://youtu.be/VT-i3yRsX2s?t=2736" target="_blank"><img height="18px" src="assets/icon-youtube.svg" /></a>||
|CUPRA[<sup>12</sup>](#footnotes)|Ateca 2018-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=CUPRA Ateca 2018-23">Buy Here</a></sub></details>|||
|CUPRA[<sup>12</sup>](#footnotes)|Born 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=CUPRA Born 2021-23">Buy Here</a></sub></details>|||
|CUPRA|Born 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW MEB connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=CUPRA Born 2021-23">Buy Here</a></sub></details>|||
|Dodge|Durango 2020-21|Adaptive Cruise Control (ACC)|Stock|0 mph|39 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 FCA connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Dodge Durango 2020-21">Buy Here</a></sub></details>|||
|Ford|Bronco Sport 2021-24|Co-Pilot360 Assist+|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Ford Q3 connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Ford Bronco Sport 2021-24">Buy Here</a></sub></details>|||
|Ford|Escape 2020-22|Co-Pilot360 Assist+|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Ford Q3 connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Ford Escape 2020-22">Buy Here</a></sub></details>|||
@@ -99,7 +99,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Honda|Fit 2018-20|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda Fit 2018-20">Buy Here</a></sub></details>|||
|Honda|Freed 2020|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda Freed 2020">Buy Here</a></sub></details>|||
|Honda|HR-V 2019-22|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda HR-V 2019-22">Buy Here</a></sub></details>|||
|Honda|HR-V 2023-25|All|openpilot available[<sup>1,5</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch B connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda HR-V 2023-25">Buy Here</a></sub></details>|||
|Honda|HR-V 2023-27|All|openpilot available[<sup>1,5</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch B connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda HR-V 2023-27">Buy Here</a></sub></details>|||
|Honda|Insight 2019-22|All|openpilot available[<sup>1,5</sup>](#footnotes)|0 mph|3 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch A connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda Insight 2019-22">Buy Here</a></sub></details>|||
|Honda|Inspire 2018|All|openpilot available[<sup>1,5</sup>](#footnotes)|0 mph|3 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch A connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda Inspire 2018">Buy Here</a></sub></details>|||
|Honda|N-Box 2018|All|openpilot available[<sup>1,5</sup>](#footnotes)|0 mph|11 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch A connector<br>- 1 OBD-C cable (2 ft)<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Honda N-Box 2018">Buy Here</a></sub></details>|||
@@ -268,8 +268,8 @@ A supported vehicle is one that just works when you install a comma device. All
|Škoda[<sup>12</sup>](#footnotes)|Superb 2015-22[<sup>15</sup>](#footnotes)|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Škoda Superb 2015-22">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model 3 (with HW3) 2019-23[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla A connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model 3 (with HW3) 2019-23">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model 3 (with HW4) 2024-25[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla B connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model 3 (with HW4) 2024-25">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model Y (with HW3) 2020-23[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla A connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model Y (with HW3) 2020-23">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model Y (with HW4) 2024-25[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla B connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model Y (with HW4) 2024-25">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model Y (with HW3) 2020-24[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla A connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model Y (with HW3) 2020-24">Buy Here</a></sub></details>|||
|Tesla[<sup>10</sup>](#footnotes)|Model Y (with HW4) 2023-25[<sup>9</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Tesla B connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Tesla Model Y (with HW4) 2023-25">Buy Here</a></sub></details>|||
|Toyota|Alphard 2019-20|All|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Toyota A connector<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Toyota Alphard 2019-20">Buy Here</a></sub></details>|||
|Toyota|Alphard Hybrid 2021|All|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Toyota A connector<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Toyota Alphard Hybrid 2021">Buy Here</a></sub></details>|||
|Toyota|Avalon 2016|Toyota Safety Sense P|Stock|19 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 Toyota A connector<br>- 1 comma four<br>- 1 comma power v3<br>- 1 harness box<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Toyota Avalon 2016">Buy Here</a></sub></details>|||
@@ -335,8 +335,8 @@ A supported vehicle is one that just works when you install a comma device. All
|Volkswagen[<sup>12</sup>](#footnotes)|Golf R 2015-19|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Golf R 2015-19">Buy Here</a></sub></details>|||
|Volkswagen[<sup>12</sup>](#footnotes)|Golf SportsVan 2015-20|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Golf SportsVan 2015-20">Buy Here</a></sub></details>|||
|Volkswagen[<sup>12</sup>](#footnotes)|Grand California 2019-24|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|31 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Grand California 2019-24">Buy Here</a></sub></details>|<a href="https://youtu.be/4100gLeabmo" target="_blank"><img height="18px" src="assets/icon-youtube.svg" /></a>||
|Volkswagen[<sup>12</sup>](#footnotes)|ID.4 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen ID.4 2021-23">Buy Here</a></sub></details>|||
|Volkswagen[<sup>12</sup>](#footnotes)|ID.4 2024-25|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen ID.4 2024-25">Buy Here</a></sub></details>|||
|Volkswagen|ID.4 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW MEB connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen ID.4 2021-23">Buy Here</a></sub></details>|||
|Volkswagen|ID.4 2024-25|Adaptive Cruise Control (ACC) & Lane Assist|openpilot[<sup>16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW MEB connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen ID.4 2024-25">Buy Here</a></sub></details>|||
|Volkswagen[<sup>12</sup>](#footnotes)|Jetta 2019-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Jetta 2019-23">Buy Here</a></sub></details>|||
|Volkswagen[<sup>12</sup>](#footnotes)|Jetta GLI 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Jetta GLI 2021-23">Buy Here</a></sub></details>|||
|Volkswagen|Passat 2015-22[<sup>14</sup>](#footnotes)|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,16</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 OBD-C cable (2 ft)<br>- 1 VW J533 connector<br>- 1 comma four<br>- 1 harness box<br>- 1 long OBD-C cable (9.5 ft)<br>- 1 mount<br><a href="https://comma.ai/shop/comma-3x?harness=Volkswagen Passat 2015-22">Buy Here</a></sub></details>|||
@@ -363,7 +363,7 @@ A supported vehicle is one that just works when you install a comma device. All
<sup>6</sup>See more setup details for <a href="https://github.com/commaai/openpilot/wiki/nissan" target="_blank">Nissan</a>. <br />
<sup>7</sup>In the non-US market, openpilot requires the car to come equipped with EyeSight with Lane Keep Assistance. <br />
<sup>8</sup>Enabling longitudinal control (alpha) will disable all EyeSight functionality, including AEB, LDW, and RAB. <br />
<sup>9</sup>Some 2023 model years have HW4. To check which hardware type your vehicle has, look for <b>Autopilot computer</b> under <b>Software -> Additional Vehicle Information</b> on your vehicle's touchscreen. See <a href="https://www.notateslaapp.com/news/2173/how-to-check-if-your-tesla-has-hardware-4-ai4-or-hardware-3">this page</a> for more information. <br />
<sup>9</sup>Model years 2023 and 2024 can have either hardware type, depending on build date and factory. To check which hardware type your vehicle has, look for <b>Autopilot computer</b> under <b>Software -> Additional Vehicle Information</b> on your vehicle's touchscreen. See <a href="https://www.notateslaapp.com/news/2173/how-to-check-if-your-tesla-has-hardware-4-ai4-or-hardware-3">this page</a> for more information. <br />
<sup>10</sup>See more setup details for <a href="https://github.com/commaai/openpilot/wiki/tesla" target="_blank">Tesla</a>. <br />
<sup>11</sup>openpilot operates above 28mph for Camry 4CYL L, 4CYL LE and 4CYL SE which don't have Full-Speed Range Dynamic Radar Cruise Control. <br />
<sup>12</sup>The J533 harness plugs in at the CAN gateway under the dashboard, just above the steering column. More information can be found at <a href="https://docs.howtocomma.com/docs/j533-harness-install" target="_blank">this guide</a>. <br />
+2 -2
View File
@@ -5,10 +5,10 @@ The site is updated on pushes to master by this [workflow](../.github/workflows/
**1. Build the site**
``` bash
python docs/serve.py --build
op docs --build
```
**2. Run the site locally** (rebuilds on change)
``` bash
python docs/serve.py
op docs
```
+12 -10
View File
@@ -18,21 +18,21 @@ function agnos_init {
sudo chmod 660 /dev/adsprpc-smd /dev/ion /dev/kgsl-3d0
# Check if AGNOS update is required
if [ $(< /VERSION) != "$AGNOS_VERSION" ]; then
if [ "$(< /VERSION)" != "$AGNOS_VERSION" ]; then
AGNOS_PY="$DIR/openpilot/common/hardware/comma/agnos.py"
MANIFEST="$DIR/openpilot/system/hardware/comma/agnos.json"
if $AGNOS_PY --verify $MANIFEST; then
if "$AGNOS_PY" --verify "$MANIFEST"; then
sudo reboot
fi
while true; do
$DIR/openpilot/common/hardware/comma/updater $AGNOS_PY $MANIFEST
"$DIR/openpilot/common/hardware/comma/updater" "$AGNOS_PY" "$MANIFEST"
done
fi
}
function launch {
# Remove orphaned git lock if it exists on boot
[ -f "$DIR/.git/index.lock" ] && rm -f $DIR/.git/index.lock
[ -f "$DIR/.git/index.lock" ] && rm -f "$DIR/.git/index.lock"
# Check to see if there's a valid overlay-based update available. Conditions
# are as follows:
@@ -44,7 +44,7 @@ function launch {
# that completed successfully and synced to disk.
if [ -f "${DIR}/.overlay_init" ]; then
find ${DIR}/.git -newer ${DIR}/.overlay_init | grep -q '.' 2> /dev/null
find "${DIR}/.git" -newer "${DIR}/.overlay_init" | grep -q '.' 2> /dev/null
if [ $? -eq 0 ]; then
echo "${DIR} has been modified, skipping overlay update installation"
else
@@ -53,9 +53,11 @@ function launch {
echo "Valid overlay update found, installing"
LAUNCHER_LOCATION="${BASH_SOURCE[0]}"
mv $DIR /data/safe_staging/old_openpilot
mv "${STAGING_ROOT}/finalized" $DIR
cd $DIR
mv "$DIR" /data/safe_staging/old_openpilot
mv "${STAGING_ROOT}/finalized" "$DIR"
cd "$DIR"
# releases before the openpilot/ move can't swap AGNOS on newer AGNOS, use ours
[ ! -L system/hardware/tici/agnos.json ] && /data/safe_staging/old_openpilot/openpilot/common/hardware/comma/agnos.py --verify system/hardware/tici/agnos.json && sudo reboot
echo "Restarting launch script ${LAUNCHER_LOCATION}"
unset AGNOS_VERSION
@@ -69,7 +71,7 @@ function launch {
fi
# handle pythonpath
ln -sfn $(pwd) /data/pythonpath
ln -sfn "$(pwd)" /data/pythonpath
export PYTHONPATH="$PWD"
# submodule package symlinks for PYTHONPATH imports on device.
@@ -90,7 +92,7 @@ function launch {
# start manager
cd openpilot/system/manager
if [ ! -f $DIR/prebuilt ]; then
if [ ! -f "$DIR/prebuilt" ]; then
./build.py
fi
./manager.py
+1 -1
View File
@@ -16,7 +16,7 @@ export VECLIB_MAXIMUM_THREADS=1
export QCOM_PRIORITY=12
if [ -z "$AGNOS_VERSION" ]; then
export AGNOS_VERSION="19.7"
export AGNOS_VERSION="19.9"
fi
export STAGING_ROOT="/data/safe_staging"
+5 -2
View File
@@ -196,7 +196,6 @@ struct ModelManagerSP @0xaedffd8f31e7b55d {
}
struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
dec @0 :DynamicExperimentalControl;
longitudinalPlanSource @1 :LongitudinalPlanSource;
smartCruiseControl @2 :SmartCruiseControl;
speedLimit @3 :SpeedLimit;
@@ -205,7 +204,11 @@ struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
events @6 :List(OnroadEventSP.Event);
e2eAlerts @7 :E2eAlerts;
struct DynamicExperimentalControl {
deprecated :group {
dec @0 :DynamicExperimentalControlDEPRECATED;
}
struct DynamicExperimentalControlDEPRECATED {
state @0 :DynamicExperimentalControlState;
enabled @1 :Bool;
active @2 :Bool;
+22 -16
View File
@@ -128,6 +128,7 @@ struct OnroadEvent @0xc4fa6047f024e718 {
personalityChanged @91;
aeb @92;
userBookmark @95;
userBookmarkNotPaired @104;
excessiveActuation @96;
bigModelLoading @100;
bigModelFailed @102;
@@ -166,12 +167,7 @@ struct InitData {
gitSrcCommit @23 :Text;
gitSrcCommitDate @24 :Text;
androidProperties @16 :Map(Text, Text);
pandaInfo @8 :PandaInfo;
dirty @9 :Bool;
passive @12 :Bool;
params @17 :Map(Text, Data);
commands @19 :Map(Text, Data);
@@ -189,6 +185,17 @@ struct InitData {
mici @7; # comma four
}
deprecated :group {
gctx @1 :Text;
androidBuildInfo @5 :Deprecated.AndroidBuildInfo;
androidSensors @6 :List(Deprecated.AndroidSensor);
chffrAndroidExtra @7 :ChffrAndroidExtra;
pandaInfo @8 :PandaInfo;
passive @12 :Bool;
iosBuildInfo @14 :Deprecated.IosBuildInfo;
androidProperties @16 :Map(Text, Text);
}
struct PandaInfo {
hasPanda @0 :Bool;
dongleId @1 :Text;
@@ -199,14 +206,6 @@ struct InitData {
struct ChffrAndroidExtra {
allCameraCharacteristics @0 :Map(Text, Text);
}
deprecated :group {
gctx @1 :Text;
androidBuildInfo @5 :Deprecated.AndroidBuildInfo;
androidSensors @6 :List(Deprecated.AndroidSensor);
chffrAndroidExtra @7 :ChffrAndroidExtra;
iosBuildInfo @14 :Deprecated.IosBuildInfo;
}
}
struct FrameData {
@@ -572,6 +571,9 @@ struct PandaState @0xa7649e2575e4591e {
controlsAllowedLateral @38 :Bool;
controlsAllowedLongitudinal @39 :Bool;
nmiReset @40 :Bool;
hardfaultReset @41 :Bool;
enum FaultStatus {
none @0;
faultTemp @1;
@@ -694,6 +696,7 @@ struct PeripheralState {
struct UsbState {
devices @0 :List(Device);
connected @1 :Bool;
struct Device {
busnum @0 :UInt8;
@@ -819,7 +822,6 @@ struct SelfdriveState {
alertHudVisual @12 :Car.CarControl.HUDControl.VisualAlert;
# configurable driving settings
experimentalMode @10 :Bool;
personality @11 :LongitudinalPersonality;
enum AudibleAlert {
@@ -863,6 +865,7 @@ struct SelfdriveState {
deprecated :group {
alertSound @8 :Car.CarControl.HUDControl.AudibleAlert;
experimentalMode @10 :Bool;
}
}
@@ -1237,6 +1240,7 @@ struct LongitudinalPlan @0xe00b5b3eba12876c {
shouldStop @37: Bool;
allowThrottle @38: Bool;
allowBrake @39: Bool;
accelBoost @40 :Float32;
solverExecutionTime @35 :Float32;
@@ -2234,6 +2238,7 @@ struct DriverMonitoringState {
pose @0: Bool;
eye @1: Bool;
phone @2: Bool;
sleep @3: Bool;
}
struct Pose {
@@ -2594,6 +2599,7 @@ struct Event {
clocks @35 :Clocks;
deviceState @6 :DeviceState;
chestnutState @152 :ChestnutState;
chestnutGpuState @153 :ChestnutState;
logMessage @18 :Text;
errorLogMessage @85 :Text;
@@ -2704,8 +2710,8 @@ struct Event {
driverMonitoringStateDEPRECATED @71 :DriverMonitoringStateDEPRECATED;
gpsNMEADEPRECATED @3 :GPSNMEAData;
uploaderStateDEPRECATED @79 :UploaderState;
navInstructionDEPRECATED @82 :NavInstruction;
navRouteDEPRECATED @83 :NavRoute;
navInstruction @82 :NavInstruction;
navRoute @83 :NavRoute;
navThumbnailDEPRECATED @84 :Thumbnail;
gnssMeasurementsDEPRECATED @91 :GnssMeasurements;
mapRenderStateDEPRECATED @105: MapRenderState;
+4 -1
View File
@@ -25,7 +25,8 @@ _services: dict[str, tuple] = {
"accelerometer": (True, 104., 104),
"temperatureSensor": (True, 2., 200),
"deviceState": (True, 2., 1),
"chestnutState": (True, 10., 10),
"chestnutState": (True, 10., 1),
"chestnutGpuState": (False, 10.),
"touch": (True, 20., 1),
"can": (True, 100., 2053, QueueSize.BIG), # decimation gives ~3 msgs in a full segment
"controlsState": (True, 100., 10, QueueSize.MEDIUM),
@@ -51,6 +52,8 @@ _services: dict[str, tuple] = {
"procLog": (True, 0.5, 15, QueueSize.BIG),
"gpsLocationExternal": (True, 10., 10),
"gpsLocation": (True, 1., 1),
"navInstruction": (True, 1., 10),
"navRoute": (True, 0.),
"ubloxGnss": (True, 10.),
"qcomGnss": (True, 2.),
"clocks": (True, 0.1, 1),
+4
View File
@@ -16,6 +16,10 @@ class CV:
DEG_TO_RAD = np.pi / 180.
RAD_TO_DEG = 1. / DEG_TO_RAD
# Length
METER_TO_FOOT = 3.28084
METER_TO_MILE = 1. / 1609.344
# Mass
LB_TO_KG = 0.453592
+10 -1
View File
@@ -1,6 +1,8 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
WEBBING_ICCID_PREFIX = "8985235"
class LPAError(RuntimeError):
pass
@@ -19,7 +21,14 @@ class Profile:
@property
def is_comma(self) -> bool:
return self.provider == 'Webbing' and self.iccid.startswith('8985235')
return self.provider == 'Webbing' and self.iccid.startswith(WEBBING_ICCID_PREFIX)
@property
def display_name(self) -> str:
if self.is_comma:
return "comma prime"
name = self.nickname or self.provider or "<unnamed>"
return f"{name} (...{self.iccid[-4:]})"
class LPABase(ABC):
+1 -1
View File
@@ -613,7 +613,7 @@ def parse_lpa_activation_code(activation_code: str) -> tuple[str, str]:
if not activation_code.startswith("LPA:"):
raise ValueError("Invalid activation code format")
parts = activation_code[4:].split("$")
if len(parts) != 3:
if len(parts) != 3 or not all(parts):
raise ValueError("Invalid activation code format")
return parts[1], parts[2]
+4 -11
View File
@@ -24,6 +24,7 @@ def chunk_file(path, targets):
manifest_path, *chunk_paths = targets
actual_num_chunks = max(1, math.ceil(os.path.getsize(path) / CHUNK_SIZE))
assert len(chunk_paths) >= actual_num_chunks, f"Allowed {len(chunk_paths)} chunks but needs at least {actual_num_chunks}, for path {path}"
Path(manifest_path).unlink(missing_ok=True)
with open(path, 'rb') as f:
for chunk_path in chunk_paths:
with open(chunk_path, 'wb') as out:
@@ -31,14 +32,6 @@ def chunk_file(path, targets):
Path(manifest_path).write_text(str(len(chunk_paths)))
os.remove(path)
def get_existing_chunks(path):
if os.path.isfile(path):
return [path]
if os.path.isfile(manifest := get_manifest_path(path)):
num_chunks = int(Path(manifest).read_text().strip())
return _chunk_paths(path, num_chunks)
raise FileNotFoundError(path)
class ChunkStream(io.RawIOBase):
def __init__(self, paths):
self._paths = iter(paths)
@@ -66,11 +59,11 @@ class ChunkStream(io.RawIOBase):
def open_file_chunked(path):
manifest_path = get_manifest_path(path)
if os.path.isfile(manifest_path):
if os.path.isfile(path):
paths = [path]
elif os.path.isfile(manifest_path):
num_chunks = int(Path(manifest_path).read_text().strip())
paths = [get_chunk_name(path, i, num_chunks) for i in range(num_chunks)]
elif os.path.isfile(path):
paths = [path]
else:
raise FileNotFoundError(path)
return io.BufferedReader(ChunkStream(paths))
-10
View File
@@ -27,16 +27,6 @@ def gpio_read(pin: int) -> bool | None:
return val
def gpio_export(pin: int) -> None:
if os.path.isdir(f"/sys/class/gpio/gpio{pin}"):
return
try:
with open("/sys/class/gpio/export", 'w') as f:
f.write(str(pin))
except Exception:
print(f"Failed to export gpio {pin}")
@cache
def get_irq_action(irq: int) -> list[str]:
try:
+3
View File
@@ -145,6 +145,9 @@ class HardwareBase(ABC):
def get_modem_temperatures(self):
return []
def get_modem_state(self) -> dict:
return {}
def initialize_hardware(self):
pass
+11 -11
View File
@@ -56,28 +56,28 @@
},
{
"name": "boot",
"url": "https://commadist.azureedge.net/agnosupdate/boot-6ecf6f987cd11968104abcccabbe268485d329cdb73012dfd3c381a6b8deb27d.img.xz",
"hash": "6ecf6f987cd11968104abcccabbe268485d329cdb73012dfd3c381a6b8deb27d",
"hash_raw": "6ecf6f987cd11968104abcccabbe268485d329cdb73012dfd3c381a6b8deb27d",
"size": 46897152,
"url": "https://commadist.azureedge.net/agnosupdate/boot-b9c9b9261fca0cef9e6d9b198fa4b563767e6342afc7a61ab137b2b519260615.img.xz",
"hash": "b9c9b9261fca0cef9e6d9b198fa4b563767e6342afc7a61ab137b2b519260615",
"hash_raw": "b9c9b9261fca0cef9e6d9b198fa4b563767e6342afc7a61ab137b2b519260615",
"size": 46962688,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "d12e1e5b9455b62a1464558716493b33e470d7a7e88da1c4105a3b21d0961808"
"ondevice_hash": "3ef4769bb1cc73bfc31f96f073b46af42dcc0bad4ab2dde32d11929745d0cd28"
},
{
"name": "system",
"url": "https://commadist.azureedge.net/agnosupdate/system-3c271e2b3d20d2f0a8bf6555a1319f3efb12845490967d6151195174a01e912f.img.xz",
"hash": "74ffc9c551e1f29cda897ace8a69080fe644f8039977c6885f2b48362e39b744",
"hash_raw": "3c271e2b3d20d2f0a8bf6555a1319f3efb12845490967d6151195174a01e912f",
"url": "https://commadist.azureedge.net/agnosupdate/system-80d9373348ddff2a5d1f534fab3420383939c96f3eba60baab6b15339fc602c7.img.xz",
"hash": "9aebca09b9d142718e71efbfe44b723010eec6e70067c043c63186067291739d",
"hash_raw": "80d9373348ddff2a5d1f534fab3420383939c96f3eba60baab6b15339fc602c7",
"size": 4718592000,
"sparse": true,
"full_check": false,
"has_ab": true,
"ondevice_hash": "6a992680183685eea9db99d915219a37935f45989330d9b619e880450257f448",
"ondevice_hash": "270b76c5c5568828e3883390b83ea445425a104dd42119d68f06093ce6b6fcef",
"alt": {
"hash": "3c271e2b3d20d2f0a8bf6555a1319f3efb12845490967d6151195174a01e912f",
"url": "https://commadist.azureedge.net/agnosupdate/system-3c271e2b3d20d2f0a8bf6555a1319f3efb12845490967d6151195174a01e912f.img",
"hash": "80d9373348ddff2a5d1f534fab3420383939c96f3eba60baab6b15339fc602c7",
"url": "https://commadist.azureedge.net/agnosupdate/system-80d9373348ddff2a5d1f534fab3420383939c96f3eba60baab6b15339fc602c7.img",
"size": 4718592000
}
}
@@ -8,6 +8,7 @@ from functools import cached_property, lru_cache
from pathlib import Path
from openpilot.cereal import log
from openpilot.common.esim.base import WEBBING_ICCID_PREFIX
from openpilot.common.utils import sudo_read, sudo_write
from openpilot.common.gpio import gpio_set, gpio_init, get_irqs_for_action
from openpilot.common.esim.base import LPABase
@@ -207,6 +208,8 @@ class HardwareComma(HardwareBase):
def get_network_metered(self, network_type) -> bool:
if network_type in (NetworkType.cell2G, NetworkType.cell3G, NetworkType.cell4G, NetworkType.cell5G):
if self.get_modem_state().get("iccid", "").startswith(WEBBING_ICCID_PREFIX):
return True
from openpilot.common.params import Params
return Params().get_bool("GsmMetered")
try:
+2 -1
View File
@@ -17,6 +17,8 @@ from ipaddress import IPv4Address, AddressValueError
from enum import Enum
from openpilot.common.esim.base import WEBBING_ICCID_PREFIX
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s.%(msecs)03d %(levelname)-7s modem: %(message)s",
@@ -44,7 +46,6 @@ NETWORK_TYPE = {0: "gsm", 1: "gsm", 3: "gsm", 8: "gsm",
11: "nr", 12: "nr", 13: "nr"}
DIAL_CID = 1
WEBBING_ICCID_PREFIX = "8985235"
PPPD_CMD = [
"sudo", "pppd", PPP_PORT, "460800", "noauth", "nodetach", "noipdefault", "usepeerdns",
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a94ab8395f20d20a9d5a2a2bacca0694f072df8421cf13adca6250d28065bdc
size 24709205
oid sha256:f92c0a9b41613e063b6a1cd086729f5555d9df6482249e171ed1e276272cad7d
size 24709520
+6 -1
View File
@@ -1,7 +1,7 @@
import os
from pathlib import Path
CHESTNUT_FW_VERSION = "ed4e39b7"
CHESTNUT_FW_VERSION = "8516b768"
CHESTNUT_USB_IDS = ((0xADD1, 0x0001), (0x3801, 0x0001))
CHESTNUT_ROM_USB_IDS = ((0x174C, 0x2464), (0x174C, 0x2463))
CHESTNUT_USB_PRODUCT = f"custom {CHESTNUT_FW_VERSION}-CLEAN"
@@ -36,6 +36,10 @@ def read_int(path: Path, base: int = 10) -> int:
return 0
def cable_connected() -> bool:
return read_int(TYPEC_CC_ORIENTATION_PATH) != 0
def usb_devices() -> list[Path]:
try:
devices = (d for d in USB_DEVICES_PATH.glob("*") if (d / "idVendor").exists())
@@ -91,3 +95,4 @@ def set_usb_state(device_state, devices: list[dict]) -> None:
chestnut_present = True
device_state.chestnutPresent = chestnut_present
device_state.usbState.connected = cable_connected()
+10 -2
View File
@@ -28,6 +28,7 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"ControlsReady", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"CurrentBootlog", {PERSISTENT, STRING}},
{"CurrentRoute", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, STRING}},
{"DisableDriverCameraIR", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"DisableLogging", {CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION, BOOL}},
{"DisablePowerDown", {PERSISTENT | BACKUP, BOOL}},
{"DisableUpdates", {PERSISTENT | BACKUP, BOOL, "0"}},
@@ -138,7 +139,15 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"UptimeOnroad", {PERSISTENT, FLOAT, "0.0"}},
{"ChestnutActive", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION | CLEAR_ON_IGNITION_ON, BOOL}},
{"ChestnutLoading", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION | CLEAR_ON_IGNITION_ON, BOOL}},
{"ChestnutModelError", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION | CLEAR_ON_IGNITION_ON, BOOL}},
{"ChestnutRetry", {CLEAR_ON_MANAGER_START | CLEAR_ON_IGNITION_ON, BOOL}},
{"InitialCruiseSpeedKph", {PERSISTENT, INT, "40"}},
{"NavDestination", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, JSON}},
{"NavigationEnabled", {PERSISTENT, BOOL, "0"}},
{"Offroad_ChestnutModelError", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_Pairing", {CLEAR_ON_MANAGER_START, JSON}},
{"Offroad_UploadsIgnored", {CLEAR_ON_MANAGER_START, JSON}},
{"PairingEmail", {PERSISTENT, STRING}},
{"PairingProvider", {PERSISTENT, STRING}},
{"Version", {PERSISTENT, STRING}},
// --- sunnypilot params --- //
@@ -240,7 +249,6 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"ToyotaEnforceStockLongitudinal", {PERSISTENT | BACKUP, BOOL, "0"}},
{"ToyotaStopAndGoHack", {PERSISTENT | BACKUP, BOOL, "0"}},
{"DynamicExperimentalControl", {PERSISTENT | BACKUP, BOOL, "0"}},
{"BlindSpot", {PERSISTENT | BACKUP, BOOL, "0"}},
// sunnypilot model params
+556 -35
View File
@@ -1,16 +1,47 @@
"""Small QR encoder for the UI's byte-mode, error-correction-level-L codes."""
"""QR code encoding, decoding, and UI textures."""
import functools
import itertools
import numpy as np
import pyray as rl
# Indexes are QR versions. These are the only two Reed-Solomon parameters needed
# for error-correction level L.
_ECC_LEN = (0, 7, 10, 15, 20, 26, 18, 20, 24, 30, 18, 20, 24, 26, 30, 22, 24, 28, 30, 28, 28)
_NUM_BLOCKS = (0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6, 6, 6, 7, 8)
# (ec codewords per block, block count) for levels L, M, Q, H, versions 1-40
_EC = [
((7, 1), (10, 1), (13, 1), (17, 1)), ((10, 1), (16, 1), (22, 1), (28, 1)), ((15, 1), (26, 1), (18, 2), (22, 2)),
((20, 1), (18, 2), (26, 2), (16, 4)), ((26, 1), (24, 2), (18, 4), (22, 4)), ((18, 2), (16, 4), (24, 4), (28, 4)),
((20, 2), (18, 4), (18, 6), (26, 5)), ((24, 2), (22, 4), (22, 6), (26, 6)), ((30, 2), (22, 5), (20, 8), (24, 8)),
((18, 4), (26, 5), (24, 8), (28, 8)), ((20, 4), (30, 5), (28, 8), (24, 11)), ((24, 4), (22, 8), (26, 10), (28, 11)),
((26, 4), (22, 9), (24, 12), (22, 16)), ((30, 4), (24, 9), (20, 16), (24, 16)), ((22, 6), (24, 10), (30, 12), (24, 18)),
((24, 6), (28, 10), (24, 17), (30, 16)), ((28, 6), (28, 11), (28, 16), (28, 19)), ((30, 6), (26, 13), (28, 18), (28, 21)),
((28, 7), (26, 14), (26, 21), (26, 25)), ((28, 8), (26, 16), (30, 20), (28, 25)), ((28, 8), (26, 17), (28, 23), (30, 25)),
((28, 9), (28, 17), (30, 23), (24, 34)), ((30, 9), (28, 18), (30, 25), (30, 30)), ((30, 10), (28, 20), (30, 27), (30, 32)),
((26, 12), (28, 21), (30, 29), (30, 35)), ((28, 12), (28, 23), (28, 34), (30, 37)), ((30, 12), (28, 25), (30, 34), (30, 40)),
((30, 13), (28, 26), (30, 35), (30, 42)), ((30, 14), (28, 28), (30, 38), (30, 45)), ((30, 15), (28, 29), (30, 40), (30, 48)),
((30, 16), (28, 31), (30, 43), (30, 51)), ((30, 17), (28, 33), (30, 45), (30, 54)), ((30, 18), (28, 35), (30, 48), (30, 57)),
((30, 19), (28, 37), (30, 51), (30, 60)), ((30, 19), (28, 38), (30, 53), (30, 63)), ((30, 20), (28, 40), (30, 56), (30, 66)),
((30, 21), (28, 43), (30, 59), (30, 70)), ((30, 22), (28, 45), (30, 62), (30, 74)), ((30, 24), (28, 47), (30, 65), (30, 77)),
((30, 25), (28, 49), (30, 68), (30, 81)),
]
# 15 format-info bits for level L (01) with mask 0: ((0x08 << 10) | bch_remainder) ^ 0x5412
_FORMAT_BITS = 0b111011111000100
# GF(256) with the QR polynomial x^8 + x^4 + x^3 + x^2 + 1: powers of alpha and their logs
_EXP = [1]
for _ in range(254):
_EXP.append(_EXP[-1] << 1 ^ (0x11D if _EXP[-1] & 0x80 else 0))
_LOG = {v: i for i, v in enumerate(_EXP)}
def _bch_format(data: int) -> int:
v = data << 10
for shift in range(14, 9, -1):
if v >> shift & 1:
v ^= 0x537 << (shift - 10)
return (data << 10 | v) ^ 0x5412
# 15-bit format info indexed by (level bits << 3 | mask). Level bits: L=01, M=00, Q=11, H=10.
_FORMATS = [_bch_format(d) for d in range(32)]
def _raw_modules(version: int) -> int:
@@ -21,8 +52,24 @@ def _raw_modules(version: int) -> int:
return result - (36 if version >= 7 else 0)
def _block_lengths(version: int, level: int) -> list[int]:
"""Data codewords per Reed-Solomon block. The last blocks may be one longer."""
ec, nblocks = _EC[version - 1][level]
total = _raw_modules(version) // 8 - ec * nblocks
return [total // nblocks + (i >= nblocks - total % nblocks) for i in range(nblocks)]
def _interleaved(version: int, level: int) -> list[tuple[int, int]]:
"""(block, index within block) of each transmitted codeword: data column-major, then ECC column-major."""
ec, nblocks = _EC[version - 1][level]
lens = _block_lengths(version, level)
data = [(b, i) for i in range(max(lens)) for b in range(nblocks) if i < lens[b]]
ecc = [(b, lens[b] + i) for i in range(ec) for b in range(nblocks)]
return data + ecc
def _capacity(version: int) -> int:
return _raw_modules(version) // 8 - _ECC_LEN[version] * _NUM_BLOCKS[version]
return sum(_block_lengths(version, 0))
def _append_bits(bits: list[int], value: int, length: int) -> None:
@@ -49,37 +96,18 @@ def _data_codewords(data: bytes, version: int) -> bytes:
def _codewords(data: bytes, version: int) -> bytes:
"""Split data codewords into Reed-Solomon blocks and interleave data + ECC."""
data = _data_codewords(data, version)
num_blocks = _NUM_BLOCKS[version]
ecc_len = _ECC_LEN[version]
raw_codewords = _raw_modules(version) // 8
short_len = raw_codewords // num_blocks
num_short = num_blocks - raw_codewords % num_blocks
divisor = _divisor(ecc_len)
blocks: list[tuple[bytes, bytes]] = []
divisor = _divisor(_EC[version - 1][0][0])
blocks = []
offset = 0
for i in range(num_blocks):
length = short_len - ecc_len + (0 if i < num_short else 1)
for length in _block_lengths(version, 0):
block = data[offset:offset + length]
blocks.append((block, _remainder(block, divisor)))
blocks.append(block + _remainder(block, divisor))
offset += length
result = bytearray()
for i in range(short_len - ecc_len + 1):
for block, _ in blocks:
result.extend(block[i:i + 1])
for i in range(ecc_len):
for _, ecc in blocks:
result.append(ecc[i])
return bytes(result)
return bytes(blocks[b][i] for b, i in _interleaved(version, 0))
def _multiply(x: int, y: int) -> int:
result = 0
for _ in range(8):
result = (result << 1) ^ (0x11D if result & 0x80 else 0)
if y & 0x80:
result ^= x
y <<= 1
return result
return _EXP[(_LOG[x] + _LOG[y]) % 255] if x and y else 0
def _divisor(degree: int) -> bytes:
@@ -108,7 +136,7 @@ def _alignment_positions(version: int) -> list[int]:
if version == 1:
return []
count = version // 7 + 2
step = ((version * 4 + count * 2 + 1) // (count * 2 - 2)) * 2
step = (version * 8 + count * 3 + 5) // (count * 4 - 4) * 2
return [6] + [version * 4 + 10 - step * i for i in range(count - 1)][::-1]
@@ -171,7 +199,7 @@ class _Qr:
def _format(self) -> None:
for i in range(15):
bit = ((_FORMAT_BITS >> i) & 1) != 0
bit = ((_FORMATS[1 << 3 | 0] >> i) & 1) != 0 # level L, mask 0
y_pos = i if i < 6 else i + 1 if i < 8 else self.size - 15 + i
self._set_function(8, y_pos, bit)
x_pos = self.size - 1 - i if i < 8 else 15 - i if i < 9 else 14 - i
@@ -216,3 +244,496 @@ def make_texture(data: str, inverted: bool = False) -> rl.Texture:
rl_image.mipmaps = 1
rl_image.format = rl.PixelFormat.PIXELFORMAT_UNCOMPRESSED_R8G8B8A8
return rl.load_texture_from_image(rl_image)
# ---- Symbol structure for decoding ----
class QRError(Exception):
pass
_LEVELS = (1, 0, 3, 2) # format info level bits -> column in _EC
_MASKS = [
lambda i, j: (i + j) % 2 == 0,
lambda i, j: i % 2 == 0,
lambda i, j: j % 3 == 0,
lambda i, j: (i + j) % 3 == 0,
lambda i, j: (i // 2 + j // 3) % 2 == 0,
lambda i, j: (i * j) % 2 + (i * j) % 3 == 0,
lambda i, j: ((i * j) % 2 + (i * j) % 3) % 2 == 0,
lambda i, j: ((i + j) % 2 + (i * j) % 3) % 2 == 0,
]
_ALIGNMENT = np.ones((5, 5), dtype=bool)
_ALIGNMENT[1:4, 1:4] = False
_ALIGNMENT[2, 2] = True
def _gf_inv(a: int) -> int:
return _EXP[-_LOG[a] % 255]
@functools.lru_cache
def _data_coords(version: int) -> tuple[np.ndarray, np.ndarray]:
"""(rows, cols) of the data and error correction modules in placement order: two-column zigzag from the right."""
dim = version * 4 + 17
func = np.zeros((dim, dim), dtype=bool) # finder, timing, alignment, format, and version modules
func[:9, :9] = func[:9, dim - 8:] = func[dim - 8:, :9] = True
func[6, :] = func[:, 6] = True
positions = _alignment_positions(version)
for r, c in itertools.product(positions, positions):
if (r, c) not in ((6, 6), (6, dim - 7), (dim - 7, 6)):
func[r - 2:r + 3, c - 2:c + 3] = True
if version >= 7:
func[:6, dim - 11:dim - 8] = func[dim - 11:dim - 8, :6] = True
ys = np.arange(dim)
rows, cols = [], []
# the vertical timing column is skipped, so the pairs left of it start at odd columns
for i, right in enumerate(col if col > 6 else col - 1 for col in range(dim - 1, 0, -2)):
r = np.repeat(ys[::-1] if i % 2 == 0 else ys, 2)
c = np.tile((right, right - 1), dim)
keep = ~func[r, c]
rows.append(r[keep])
cols.append(c[keep])
return np.concatenate(rows), np.concatenate(cols)
# ---- Matrix decoding ----
def _poly_eval(p: list[int], x: int) -> int:
# p is highest degree first
y = 0
for c in p:
y = _multiply(y, x) ^ c
return y
_EXP_TABLE = np.array(_EXP)
_LOG_TABLE = np.array([_LOG.get(v, 0) for v in range(256)])
def _syndromes(msg: list[int], nsym: int) -> list[int]:
"""syn[i] = msg(alpha^i), msg highest degree first."""
m = np.array(msg)
exponents = np.arange(nsym)[:, None] * (len(msg) - 1 - np.arange(len(msg)))
return np.bitwise_xor.reduce(_EXP_TABLE[(_LOG_TABLE[m] + exponents) % 255] * (m != 0), axis=1).tolist()
def _rs_correct(msg: list[int], nsym: int) -> list[int]:
"""Corrects up to nsym // 2 errors in a Reed-Solomon codeword, in place."""
n = len(msg)
syn = _syndromes(msg, nsym)
if not any(syn):
return msg
# Berlekamp-Massey, sigma is lowest degree first
sigma, prev, L, m, b = [1], [1], 0, 1, 1
for r in range(nsym):
d = syn[r]
for i in range(1, L + 1):
d ^= _multiply(sigma[i], syn[r - i])
if d == 0:
m += 1
continue
coef = _multiply(d, _gf_inv(b))
shifted = [0] * m + prev
saved = sigma[:]
sigma = sigma + [0] * max(0, len(shifted) - len(sigma))
for i, c in enumerate(shifted):
sigma[i] ^= _multiply(coef, c)
if 2 * L <= r:
L, prev, b, m = r + 1 - L, saved, d, 1
else:
m += 1
sigma = sigma[:L + 1]
if 2 * L > nsym:
raise QRError("too many errors")
# Chien search: codeword position p has locator alpha^(n-1-p)
positions = [p for p in range(n) if _poly_eval(sigma[::-1], _EXP[(p - n + 1) % 255]) == 0]
if len(positions) != L:
raise QRError("error locator mismatch")
# solve syn[i] = sum_k e_k * X_k^i for the magnitudes e_k
xlog = [(n - 1 - p) % 255 for p in positions]
A = [[_EXP[(xlog[k] * i) % 255] for k in range(L)] + [syn[i]] for i in range(L)]
for col in range(L):
piv = next((r for r in range(col, L) if A[r][col]), None)
if piv is None:
raise QRError("singular")
A[col], A[piv] = A[piv], A[col]
inv = _gf_inv(A[col][col])
A[col] = [_multiply(inv, v) for v in A[col]]
for r in range(L):
if r != col and A[r][col]:
f = A[r][col]
A[r] = [a ^ _multiply(f, c) for a, c in zip(A[r], A[col], strict=True)]
for k, p in enumerate(positions):
msg[p] ^= A[k][L]
if any(_syndromes(msg, nsym)):
raise QRError("uncorrectable")
return msg
def _read_format(m: np.ndarray) -> int:
"""Returns the closest format info (level bits << 3 | mask) from either copy."""
dim = m.shape[0]
copies = ([(8, i) for i in range(6)] + [(8, 7), (8, 8), (7, 8)] + [(5 - i, 8) for i in range(6)],
[(dim - 1 - i, 8) for i in range(7)] + [(8, dim - 8 + i) for i in range(8)]) # (row, col), msb first
candidates = []
for coords in copies:
bits = int("".join(str(int(m[r, c])) for r, c in coords), 2)
candidates += [((bits ^ f).bit_count(), i) for i, f in enumerate(_FORMATS)]
distance, fmt = min(candidates)
if distance > 3:
raise QRError("bad format info")
return fmt
_ALNUM = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ $%*+-./:"
_ECI_ENCODINGS = {
0: "cp437", 2: "cp437", 1: "iso8859-1", 3: "iso8859-1",
**{i + 2: f"iso8859-{i}" for i in range(2, 17) if i != 12},
20: "shift_jis", 21: "cp1250", 22: "cp1251", 23: "cp1252", 24: "cp1256",
25: "utf-16-be", 26: "utf-8", 27: "ascii", 170: "ascii", 28: "big5", 29: "gb18030", 30: "euc_kr",
}
class _Bits:
def __init__(self, data: list[int]):
self._value = int.from_bytes(bytes(data), "big")
self.remaining = len(data) * 8
def read(self, n: int) -> int:
if n > self.remaining:
raise QRError("bitstream underflow")
self.remaining -= n
return self._value >> self.remaining & (1 << n) - 1
def read_below(self, n: int, limit: int) -> int:
v = self.read(n)
if v >= limit:
raise QRError("value out of range")
return v
def _parse_data(data: list[int], version: int) -> str:
bits = _Bits(data)
out: list[str] = []
encoding = None
band = 0 if version <= 9 else 1 if version <= 26 else 2
while bits.remaining >= 4:
mode = bits.read(4)
if mode == 0:
break
if mode == 7: # ECI character set assignment
first = bits.read(8)
extra = 0 if first < 0x80 else 8 if first < 0xC0 else 16 if first < 0xE0 else -1 # 1, 2, or 3 byte assignment
if extra < 0:
raise QRError("bad ECI assignment")
assignment = (first & 0x7F >> extra // 8) << extra | bits.read(extra)
encoding = _ECI_ENCODINGS.get(assignment)
if encoding is None:
raise QRError(f"unsupported ECI assignment {assignment}")
elif mode == 1:
n = bits.read((10, 12, 14)[band])
while n > 0:
k = min(n, 3) # 3 digits in 10 bits, the last 2 or 1 in 7 or 4
out.append(f"{bits.read_below((4, 7, 10)[k - 1], 10 ** k):0{k}d}")
n -= k
elif mode == 2:
n = bits.read((9, 11, 13)[band])
while n > 0:
k = min(n, 2) # 2 characters in 11 bits, a last one in 6
v = bits.read_below((6, 11)[k - 1], 45 ** k)
out.append(_ALNUM[v // 45] * (k - 1) + _ALNUM[v % 45])
n -= k
elif mode == 4:
n = bits.read((8, 16, 16)[band])
segment = bytes(bits.read(8) for _ in range(n))
try:
out.append(segment.decode(encoding or "utf-8"))
except UnicodeDecodeError as e:
if encoding is not None:
raise QRError("invalid ECI byte segment") from e
out.append(segment.decode("latin-1"))
elif mode == 8:
n = bits.read((8, 10, 12)[band])
for _ in range(n):
v = bits.read(13)
c = (v // 0xC0) << 8 | v % 0xC0
c += 0x8140 if c < 0x1F00 else 0xC140
try:
out.append(c.to_bytes(2, "big").decode("shift_jis"))
except UnicodeDecodeError as e:
raise QRError("invalid Kanji character") from e
else:
raise QRError(f"unsupported mode {mode}")
return "".join(out)
def decode_matrix(m: np.ndarray) -> str:
"""Decodes a square boolean module matrix (True = dark) without a quiet zone."""
dim = m.shape[0]
if m.shape != (dim, dim) or dim % 4 != 1 or not 21 <= dim <= 177:
raise QRError("bad matrix size")
version = (dim - 17) // 4
fmt = _read_format(m)
level = _LEVELS[fmt >> 3]
rows, cols = _data_coords(version)
bits = m[rows, cols] ^ _MASKS[fmt & 7](rows, cols)
codewords = np.packbits(bits[:len(bits) // 8 * 8]).tolist()
ec, _ = _EC[version - 1][level]
lens = _block_lengths(version, level)
blocks = [[0] * (n + ec) for n in lens]
for (b, i), codeword in zip(_interleaved(version, level), codewords, strict=True):
blocks[b][i] = codeword
data: list[int] = []
for block, n in zip(blocks, lens, strict=True):
data += _rs_correct(block, ec)[:n]
return _parse_data(data, version)
# ---- Image decoding ----
def _box_sums(a: np.ndarray, radii: tuple[int, ...]) -> list[np.ndarray]:
"""Sums over (2r + 1)^2 neighborhoods of the last two axes, edge padded, from one integral image."""
P = max(radii)
lead = [(0, 0)] * (a.ndim - 2)
cs = np.pad(np.cumsum(np.cumsum(np.pad(a, lead + [(P, P), (P, P)], mode="edge"), -2), -1), lead + [(1, 0), (1, 0)])
H, W = a.shape[-2:]
out = []
for r in radii:
lo, hi = P - r, P + r + 1
out.append(cs[..., hi:hi + H, hi:hi + W] - cs[..., lo:lo + H, hi:hi + W] - cs[..., hi:hi + H, lo:lo + W] + cs[..., lo:lo + H, lo:lo + W])
return out
def _binarize(gray: np.ndarray) -> np.ndarray:
"""Adaptive threshold: each pixel against the mean of the surrounding tiles that have contrast."""
h, w = gray.shape
if h < 21 or w < 21:
raise QRError("image too small")
B = max(8, min(h, w) // 128 * 2)
H, W = -(-h // B), -(-w // B)
padded = np.pad(gray, ((0, H * B - h), (0, W * B - w)), mode="edge")
# block statistics from a subsample are plenty
sub = np.ascontiguousarray(padded[::2, ::2].reshape(H, B // 2, W, B // 2).transpose(0, 2, 1, 3)).reshape(H, W, -1)
blocks = sub.sum(axis=2, dtype=np.uint32) / sub.shape[2]
known = sub.max(axis=2) - sub.min(axis=2) >= 32
# Flat tiles cannot estimate their own threshold: use the tiles with contrast nearby, then
# further out, then the global midrange. A flat tile is then all dark or all light.
est = np.full((H, W), (blocks.min() + blocks.max()) / 2)
filled = np.zeros((H, W), dtype=bool)
for total, count in _box_sums(np.stack((known * blocks, known.astype(float))), (2, 6)):
fill = ~filled & (count > 0)
est[fill] = total[fill] / count[fill]
filled |= fill
thr = np.where(known, np.minimum(est, 254) + 1, np.where(blocks <= est, 255, 0)).astype(np.uint8)
return (padded.reshape(H, B, W, B) < thr[:, None, :, None]).reshape(H * B, W * B)[:h, :w]
class _Runs:
"""Run-length table of a padded, flattened binary image with a per-pixel run index."""
def __init__(self, padded: np.ndarray):
self.flat = padded.ravel()
self.lines, self.stride = padded.shape
change = self.flat[1:] != self.flat[:-1]
self.starts = np.concatenate(([0], np.flatnonzero(change) + 1))
self.lengths = np.diff(np.append(self.starts, self.flat.size)).astype(np.int32)
def run_at(self, line: np.ndarray, pos: np.ndarray) -> np.ndarray:
"""Index of the run containing the pixel at `pos` along `line`."""
return np.searchsorted(self.starts, line * self.stride + pos + 1, side="right") - 1
@staticmethod
def _match(lengths: list[np.ndarray], ratios: tuple[int, ...]) -> tuple[np.ndarray, np.ndarray]:
"""Checks windows of runs against the ratios, given the length of each run. Returns (ok, module size)."""
S = sum(ratios)
total = sum(lengths[1:], start=lengths[0])
ok = total >= 2 * S # modules need to be at least 2 px
for L, r in zip(lengths, ratios, strict=True):
ok &= np.abs(2 * S * L - 2 * r * total) <= r * total # integer form of |L - r * total / S| <= r * total / (2 * S)
return ok, total / S
def scan(self, ratios: tuple[int, ...]) -> np.ndarray:
"""Returns the indices of all dark runs starting a window of runs matching the ratios."""
n = len(ratios)
N = len(self.lengths) - n + 1
if N <= 0:
return np.zeros(0, dtype=int)
ok, _ = self._match([self.lengths[k:N + k] for k in range(n)], ratios)
ok &= self.flat[self.starts[:N]]
first = np.flatnonzero(ok)
return first[self.starts[first] // self.stride == self.starts[first + n - 1] // self.stride]
def check(self, first: np.ndarray, ratios: tuple[int, ...]) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Checks the run windows starting at run index `first`. Returns (ok, center position along the line, module size)."""
n, half = len(ratios), len(ratios) // 2
ok = (first >= 0) & (first + n <= len(self.starts))
idx = np.clip(first[:, None] + np.arange(n), 0, len(self.starts) - 1)
matched, module = self._match([self.lengths[idx[:, k]] for k in range(n)], ratios)
ok &= matched & self.flat[self.starts[idx[:, 0]]]
ok &= self.starts[idx[:, 0]] // self.stride == self.starts[idx[:, -1]] // self.stride
center = self.starts[idx[:, half]] % self.stride - 1 + self.lengths[idx[:, half]] / 2
return ok, center, module
def _find_patterns(binary: np.ndarray, ratios: tuple[int, ...]) -> list[tuple[float, float, float]]:
"""Finds dark/light run patterns with the given module ratios. Returns (x, y, module size)."""
half = len(ratios) // 2
step = 2 # the center rows of a 2 px finder pattern still get scanned twice
rows_t = _Runs(np.pad(binary[::step], ((0, 0), (1, 1))))
first = rows_t.scan(ratios)
if len(first) == 0:
return []
_, cx, hmod = rows_t.check(first, ratios)
row = rows_t.starts[first] // rows_t.stride * step
xi = cx.astype(int)
xs, col = np.unique(xi, return_inverse=True)
cols_t = _Runs(np.pad(binary[:, xs].T, ((0, 0), (1, 1))))
ok, cy, vmod = cols_t.check(cols_t.run_at(col, row) - half, ratios)
ok &= (0.5 <= vmod / hmod) & (vmod / hmod <= 2)
line = np.clip(np.rint(cy / step), 0, rows_t.lines - 1).astype(int)
ok2, cx2, hmod2 = rows_t.check(rows_t.run_at(line, xi) - half, ratios)
ok &= ok2 & (0.5 <= hmod2 / vmod) & (hmod2 / vmod <= 2)
found: list[list[float]] = [] # [x, y, module, count]
for x, y, module in zip(cx2[ok], cy[ok], (hmod2[ok] + vmod[ok]) / 2, strict=True):
for f in found:
if abs(f[0] - x) <= f[2] and abs(f[1] - y) <= f[2] and 0.5 <= f[2] / module <= 2:
c = f[3]
f[0], f[1], f[2], f[3] = (f[0] * c + x) / (c + 1), (f[1] * c + y) / (c + 1), (f[2] * c + module) / (c + 1), c + 1
break
else:
found.append([x, y, module, 1])
found.sort(key=lambda f: -f[3])
return [(f[0], f[1], f[2]) for f in found if f[3] >= 2]
def _pick_finders(patterns: list[tuple[float, float, float]]) -> tuple[np.ndarray, np.ndarray, np.ndarray, float]:
"""Returns (top-left, top-right, bottom-left) centers and the module size of the most square-looking triple."""
best = None
for a, b, c in itertools.combinations(patterns[:10], 3):
mods = sorted((a[2], b[2], c[2]))
if mods[2] / mods[0] > 1.5:
continue
pts = [np.array(p[:2]) for p in (a, b, c)]
d = [np.linalg.norm(pts[(i + 1) % 3] - pts[(i + 2) % 3]) for i in range(3)]
tl = int(np.argmax(d)) # opposite the hypotenuse
p1, p2 = pts[(tl + 1) % 3], pts[(tl + 2) % 3]
v1, v2 = p1 - pts[tl], p2 - pts[tl]
n1, n2 = np.linalg.norm(v1), np.linalg.norm(v2)
if n1 == 0 or n2 == 0:
continue
cos = abs(np.dot(v1, v2)) / (n1 * n2)
if cos > 0.35 or not 0.6 <= n1 / n2 <= 1.6:
continue
score = cos + abs(np.log(n1 / n2)) + np.log(mods[2] / mods[0])
if best is not None and score >= best[0]:
continue
if v1[0] * v2[1] - v1[1] * v2[0] < 0:
p1, p2 = p2, p1
best = (score, pts[tl], p1, p2, float(sum(mods) / 3))
if best is None:
raise QRError("no finder patterns")
return best[1:]
def _perspective(src: np.ndarray, dst: np.ndarray) -> np.ndarray:
"""Homography mapping the four src points onto the four dst points."""
A = [row for (x, y), (u, v) in zip(src, dst, strict=True)
for row in ([x, y, 1, 0, 0, 0, -u * x, -u * y], [0, 0, 0, x, y, 1, -v * x, -v * y])]
try:
h = np.linalg.solve(np.array(A, dtype=float), np.asarray(dst, dtype=float).ravel())
except np.linalg.LinAlgError as e:
raise QRError("degenerate geometry") from e
return np.append(h, 1).reshape(3, 3)
def _transform(H: np.ndarray, pts: np.ndarray) -> np.ndarray:
p = np.column_stack((pts, np.ones(len(pts)))) @ H.T
return p[:, :2] / p[:, 2:3]
def _match_alignment(binary: np.ndarray, est: np.ndarray, offs: np.ndarray, r: int, module: float) -> np.ndarray | None:
h, w = binary.shape
dy = np.arange(max(0, int(est[1]) - r), min(h, int(est[1]) + r)) - est[1]
dx = np.arange(max(0, int(est[0]) - r), min(w, int(est[0]) + r)) - est[0]
if len(dy) == 0 or len(dx) == 0:
return None
y = np.rint(est[1] + dy[:, None, None] + offs[None, None, :, 1]).astype(int)
x = np.rint(est[0] + dx[None, :, None] + offs[None, None, :, 0]).astype(int)
valid = ((y >= 0) & (y < h) & (x >= 0) & (x < w)).all(axis=2)
samples = binary[np.clip(y, 0, h - 1), np.clip(x, 0, w - 1)]
score = np.where(valid, (samples == _ALIGNMENT.ravel()).sum(axis=2), 0)
if score.max() < 23:
return None
hits = np.argwhere(score == score.max())
centers = np.column_stack((est[0] + dx[hits[:, 1]], est[1] + dy[hits[:, 0]]))
closest = centers[np.argmin(np.linalg.norm(centers - est, axis=1))]
return centers[np.linalg.norm(centers - closest, axis=1) <= module / 2].mean(axis=0)
def _locate_alignment(binary: np.ndarray, H: np.ndarray, center: float, module: float) -> np.ndarray | None:
"""Template matches the 5x5 alignment pattern around its position estimated from H."""
grid = np.mgrid[-2:3, -2:3].reshape(2, -1).T[:, ::-1] + center # (25, 2) module coords (x, y)
pts = _transform(H, grid)
# the affine estimate can be off in both position and local scale under perspective
for radius in (2, 4, 8, 16):
for scale in (1.0, 0.8, 1.25, 0.65, 1.5):
found = _match_alignment(binary, pts[12], (pts - pts[12]) * scale, int(module * radius), module)
if found is not None:
return found
return None
def _sample(binary: np.ndarray, tl: np.ndarray, tr: np.ndarray, bl: np.ndarray, module: float, dim: int, use_alignment: bool) -> np.ndarray:
src = np.array([(3.5, 3.5), (dim - 3.5, 3.5), (3.5, dim - 3.5), (dim - 3.5, dim - 3.5)])
dst = np.array([tl, tr, bl, tr + bl - tl])
H = _perspective(src, dst)
if use_alignment and dim > 21:
align = _locate_alignment(binary, H, dim - 6.5, module)
if align is not None:
src[3], dst[3] = (dim - 6.5, dim - 6.5), align
H = _perspective(src, dst)
rows, cols = np.mgrid[0:dim, 0:dim]
pts = _transform(H, np.column_stack((cols.ravel() + 0.5, rows.ravel() + 0.5)))
xy = np.rint(pts).astype(int)
h, w = binary.shape
if (xy < 0).any() or (xy[:, 0] >= w).any() or (xy[:, 1] >= h).any():
raise QRError("code extends outside image")
return binary[xy[:, 1], xy[:, 0]].reshape(dim, dim)
def decode(gray: np.ndarray) -> str | None:
"""Decodes the QR code in a 2D uint8 grayscale image. Modules need to be at least 2 px.
Returns None if nothing could be decoded."""
try:
binary = _binarize(gray)
tl, tr, bl, module = _pick_finders(_find_patterns(binary, (1, 1, 3, 1, 1)))
except QRError:
return None
d = (np.linalg.norm(tr - tl) + np.linalg.norm(bl - tl)) / 2
dim = int(round((d / module + 7 - 17) / 4)) * 4 + 17
dims = [cand for cand in (dim, dim - 4, dim + 4) if 21 <= cand <= 177]
for cand, use_alignment, transpose in itertools.product(dims, (True, False), (False, True)):
try:
m = _sample(binary, tl, tr, bl, module, cand, use_alignment)
return decode_matrix(m.T if transpose else m)
except QRError:
pass
return None
+7 -2
View File
@@ -91,8 +91,13 @@ class Ratekeeper:
self.avg_dt.add_value(self._last_monitor_time - prev)
lagged = False
remaining = self._next_frame_time - time.monotonic()
self._next_frame_time += self._interval
now = time.monotonic()
remaining = self._next_frame_time - now
if remaining < -self._interval:
# more than a frame behind (e.g. a stall): resync instead of running catch-up frames back-to-back
self._next_frame_time = now + self._interval
else:
self._next_frame_time += self._interval
if self._print_delay_threshold is not None and remaining < -self._print_delay_threshold:
print(f"{self._process_name} lagging by {-remaining * 1000:.2f} ms")
lagged = True
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+178
View File
@@ -0,0 +1,178 @@
import hashlib
import math
from pathlib import Path
import numpy as np
from openpilot.common import qrcode as qr
from openpilot.common.test import OpenpilotTestCase
LPA = "LPA:1$rsp.truphone.com$QRF-BETTERROAMING-PMRDGIR2EARDEIT5"
# Matrices generated with python-qrcode 8.2, covering all versions and EC levels.
# Packed fixtures keep the decoder tests independent of our encoder.
FIXTURES = {}
for path in Path(__file__).with_name("fixtures").glob("qrcode_*.npz"):
with np.load(path) as fixtures:
FIXTURES.update({key: fixtures[key] for key in fixtures.files})
def fixture(key: str) -> np.ndarray:
bits = np.unpackbits(FIXTURES[key])
size = math.isqrt(len(bits))
return bits[:size * size].reshape(size, size).astype(bool)
def render(matrix: np.ndarray, box: int = 6, border: int = 4) -> np.ndarray:
img = np.repeat(np.repeat(np.pad(matrix, border), box, axis=0), box, axis=1)
return np.where(img, 0, 255).astype(np.uint8)
def make(data: str, version: int | None = None, level: int = 0, box: int = 6, border: int = 4):
matrix = fixture(hashlib.sha256(f"{version}:{level}:{data}".encode()).hexdigest())
return matrix, render(matrix, box, border)
def warp(img: np.ndarray, H: np.ndarray) -> np.ndarray:
"""Bilinear resampling through the output -> input homography H, white outside the image."""
h, w = img.shape
rows, cols = np.mgrid[0:h, 0:w]
pts = qr._transform(H, np.column_stack((cols.ravel() + 0.5, rows.ravel() + 0.5))) - 0.5
x0, y0 = np.floor(pts[:, 0]).astype(int), np.floor(pts[:, 1]).astype(int)
fx, fy = pts[:, 0] - x0, pts[:, 1] - y0
padded = np.pad(img.astype(float), 1, constant_values=255)
def at(y, x):
return padded[np.clip(y + 1, 0, h + 1), np.clip(x + 1, 0, w + 1)]
out = at(y0, x0) * (1 - fx) * (1 - fy) + at(y0, x0 + 1) * fx * (1 - fy) + at(y0 + 1, x0) * (1 - fx) * fy + at(y0 + 1, x0 + 1) * fx * fy
return np.clip(out, 0, 255).astype(np.uint8).reshape(h, w)
def rotate(img: np.ndarray, angle: float) -> np.ndarray:
h, w = img.shape
t = np.radians(angle)
R = np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
center = np.array([w / 2, h / 2])
corners = np.array([(0, 0), (w, 0), (w, h), (0, h)], dtype=float)
return warp(img, qr._perspective((corners - center) @ R.T + center, corners))
class TestQRCode(OpenpilotTestCase):
def test_alignment_positions(self):
assert qr._alignment_positions(7) == [6, 22, 38]
assert qr._alignment_positions(32) == [6, 34, 60, 86, 112, 138]
assert qr._alignment_positions(40) == [6, 30, 58, 86, 114, 142, 170]
def test_all_versions(self):
for version in range(1, 41):
for level in range(4):
with self.subTest(version=version, level=level):
data = "".join(chr(ord("a") + i % 26) for i in range(version))
matrix, img = make(data, version, level, box=3)
assert qr.decode_matrix(matrix) == data
assert qr.decode(img) == data
def test_modes(self):
for data in ["0123456789012345", "HELLO WORLD $1.50", LPA, "こんにちは", "ünïcødé", "mixed 123 ABC xyz"]:
with self.subTest(data=data):
matrix, img = make(data)
assert qr.decode_matrix(matrix) == data
assert qr.decode(img) == data
def test_error_correction(self):
matrix, _ = make(LPA, level=2)
rng = np.random.default_rng(0)
flipped = matrix.copy()
for r, c in rng.integers(9, matrix.shape[0] - 9, size=(40, 2)):
flipped[r, c] ^= True
assert qr.decode_matrix(flipped) == LPA
def test_large_modules(self):
for data in ["0123456789012345", "HELLO WORLD $1.50", LPA, "mixed 123 ABC xyz"]:
for box in [16, 20, 24, 32]:
for dark, light in [(0, 255), (60, 200), (140, 250)]:
with self.subTest(data=data, box=box, dark=dark):
_, img = make(data, box=box)
img = np.where(img == 0, dark, light).astype(np.uint8)
assert qr.decode(img) == data
def test_image_edges(self):
# a code touching the image edge must not lose the rows and columns left over from tiling
matrix, _ = make(LPA)
for size in (200, 203):
with self.subTest(size=size):
img = np.full((size, size), 255, dtype=np.uint8)
code = render(matrix, box=5, border=0)
img[size - code.shape[0]:, size - code.shape[1]:] = code
assert qr.decode(img) == LPA
def test_eci(self):
# qrcode_eci.npz: packed Segno 1.6.6 matrices, generated with mode='byte',
# eci=True, micro=False and the named encoding. Mixed also includes numeric,
# alphanumeric, and Kanji segments after changing the byte encoding twice.
cases = {
"iso8859-5": "Привет", "utf-16-be": "héllo", "utf-8": "こんにちは",
"shift_jis": "日本語", "cp1251": "Привет", "iso8859-1": "héllo",
"mixed": "hélloПривет日本語123ABC漢字",
}
for encoding, expected in cases.items():
with self.subTest(encoding=encoding):
matrix = fixture(encoding)
assert qr.decode_matrix(matrix) == expected
assert qr.decode(render(matrix)) == expected
def test_parse_data(self):
def parse(stream: str) -> str:
stream += '0' * (-len(stream) % 8)
return qr._parse_data([int(stream[i:i + 8], 2) for i in range(0, len(stream), 8)], 1)
def eci(assignment: str, payload: bytes = b'A') -> str:
return parse('0111' + assignment + '0100' + f'{len(payload):08b}' + ''.join(f'{b:08b}' for b in payload) + '0000')
# ASCII assignment 170 uses the two-byte ECI representation.
assert eci('1000000010101010') == 'A'
for assignment in ['00001110', '1000001111100111', '110000010000000000000000', '11100000']:
with self.subTest(assignment=assignment), self.assertRaises(qr.QRError):
eci(assignment)
with self.assertRaises(qr.QRError):
eci('00011010', b'\xff') # Invalid UTF-8 must not fall back to Latin-1.
# out-of-range numeric, alphanumeric, and Kanji values are format errors, not crashes
for stream in ['0001' + '0000000011' + '1111111111', '0001' + '0000000010' + '1111111',
'0010' + '000000010' + '11111111111', '0010' + '000000001' + '111111',
'1000' + '00000001' + '0000000111111']:
with self.subTest(stream=stream), self.assertRaises(qr.QRError):
parse(stream)
def test_rotation(self):
for angle in [0, 90, 180, 270, 25, 110]:
with self.subTest(angle=angle):
_, img = make(LPA, box=8, border=12)
assert qr.decode(rotate(img, angle)) == LPA
def test_mirrored(self):
_, img = make(LPA)
assert qr.decode(img[:, ::-1]) == LPA
def test_perspective_and_noise(self):
_, img = make(LPA, box=10, border=8)
h, w = img.shape
corners = np.array([(40, 60), (w - 20, 30), (w - 60, h - 40), (30, h - 90)])
arr = warp(img, qr._perspective(corners, np.array([(0, 0), (w, 0), (w, h), (0, h)]))).astype(float)
rng = np.random.default_rng(1)
arr = arr * 0.6 + 60 + rng.normal(0, 12, arr.shape) # low contrast + noise
# uneven lighting
arr += np.linspace(-40, 40, w)[None, :]
assert qr.decode(np.clip(arr, 0, 255).astype(np.uint8)) == LPA
def test_no_code(self):
rng = np.random.default_rng(2)
assert qr.decode(rng.integers(0, 256, size=(240, 320), dtype=np.uint8)) is None
assert qr.decode(np.full((240, 320), 200, dtype=np.uint8)) is None
def test_encoder_roundtrip(self):
for version in range(1, 21):
with self.subTest(version=version):
assert qr.decode_matrix(np.array(qr._Qr(version, b"hello").modules)) == "hello"
-15
View File
@@ -5,7 +5,6 @@ import contextlib
import subprocess
import time
import functools
from subprocess import Popen, PIPE, TimeoutExpired
import zstandard as zstd
LOG_COMPRESSION_LEVEL = 10 # little benefit up to level 15. level ~17 is a small step change
@@ -157,20 +156,6 @@ def run_cmd_default(cmd: list[str], default: str = "", cwd=None, env=None) -> st
return default
@contextlib.contextmanager
def managed_proc(cmd: list[str], env: dict[str, str]):
proc = Popen(cmd, env=env, stdout=PIPE, stderr=PIPE)
try:
yield proc
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except TimeoutExpired:
proc.kill()
def tabulate(tabular_data, headers=(), tablefmt="simple", floatfmt="g", stralign="left", numalign=None):
rows = [list(row) for row in tabular_data]
+1 -1
View File
@@ -1 +1 @@
#define COMMA_VERSION "0.11.2"
#define COMMA_VERSION "0.11.3"
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:911f59f248015600da7ecc689398103d47dfc57f6be17ac8c8e543a726a6c64b
size 3311
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd5d8d3fce5e30662797f7eeed224f5775b2cabcc055f01f5ebf6e7b2657b616
size 1801
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1e58deb1778cf2826339f27e9f09eecc79ea137c1436210c14b71c352a649c77
size 34953
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b7ab989c9fe22e7d2119bac5284cdccfba2889477d69f5f6c6b4470c7ee0aab3
size 1801
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f37a02dd914405c6f86f415700dd5985eb976b923e7abd6580d2da76533594e
size 9466
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:90d441310f8e1833c661d8a4f0546aab2eb50eb21cc21beb0aff0d27b5ee6066
size 1571
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b2dad33fead9f064c3a548651d6ef37daf82b6127c329683f538ec6e986ecbc
size 11204
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:82648254da89edb0cf65ef63fc5e6e0741414051bfea8a40449132fd61b1ed0f
size 1533
Binary file not shown.
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:01841b602632c66ab14a8e52b874a1623f09641dc2ef0620f4e2d00bb4a913f3
size 16243
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0936a3d890a3ad68ffe021e4905b9b9e59ffc2ae19ee9c703daa3e7c92e7c78
size 1793
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:57c7ba72642b0350ee764c924586fa66ff94915be4aba1813797e67eb4b74376
size 33215
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71dc069bd3dc5b11ba8e1b52eb3a1025ee0ad8c82834f94b4b2269ec0d401920
size 237
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73a6fe02cf19105c0051a6664a424c1bf2ede364c6cf3791f235c86ffa203f39
size 10348
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4f500f118dca3eedbd86fc6cc943ad8be67fe5d4f7b474d5b452cd757ad805c3
size 282
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a49996f0ce069a048abe94ba9424de655dcff53745676ab35e045204af88a3c
size 12103
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8542374779f1908dc52c8a126bc041efea4c4d605afdd34e3bf8e520a7e78bf2
size 320
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:21e656b387c1a7e8de54d3e1b1c246f442cffe3c5e0924273d6d6d897c00b568
size 13868
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd91685bf656e828648acf035a4737acb2c4709e8514cf0aa0a10fa470a9bb60
size 11580
oid sha256:92431d37a844206570d73d48f39dd8d934539a89668fe8d5f4cccae38f7b301b
size 11530
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3efe7d12ae82e0f5a1bd773129cf1eafe3c8c36ff5cba43a6c95f295706062b5
size 19724
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dfb645e59636687d61d20283bae65b68ddda6f7b2d4fda01a6e613a842e3b41f
size 823
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:df07383a6ddfe8cd7eeb2f22271cbd13068327ca0978873f8d10368860562e40
size 111083
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7518f3409e2ff8f7e99fb7f0c06afa8bdf0b0a506147b5cf9d4931b820a1a75
size 1107
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a18e3de5f455b4089a824da8d5d78d5d532ef71facb05cc75c178b24d5beeb7
size 920
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:710e74b526adb5bbac15cd8915a828dd673dd8ce15b2fd694c64c182eef6f56f
size 13744
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:25db08a9038b5faa3b200ef5625cb216d7f4b31ea3d49fb0395f9b726bfaf8e6
size 13135
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:547f466cc59d29647f3bf131d8444d7f22b63b186bcea55ceb8109c68ca74fe8
size 2345
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:55050096cd98e7f4ea3e843d927c18815e458dfbd5e7d100b503db92075d061b
size 2373
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f9c439ed89bc0357e86c005d5dbb6f1ad2b8d72cdc0890e3201d842d8c3928f1
size 2443
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7aeccd3e0d8517f74eb34f7d73632c0b29b0ad5df5b9686c0f509bb6b136561e
size 1728
+2 -2
View File
@@ -23,8 +23,8 @@ done
# sudo apt install inkscape
for svg in $(find $DIR -type f | grep svg$); do
bunx svgo $svg --multipass --pretty --indent 2
for svg in $(find "$DIR" -type f | grep svg$); do
bunx svgo "$svg" --multipass --pretty --indent 2
# convert to PNG
png="${svg%.svg}.png"
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5d02369e21d3c78a3a5ece7e78deedb82343bc3228a28db73da48b5c27860004
size 52374
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3bef898e4e784baf98f3ceca53cbe95b1020da5e01162cdaa14b79b1ce0b1340
size 52374
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1bb440f424c989c06e203bfba0b32d39aede8f6c78a9b335e5e778460202b601
oid sha256:884e828e5c51d980d41b6e2400fcb4f4a926275018a00674d8239184e1eb96db
size 73026
+3 -9
View File
@@ -127,9 +127,6 @@ class Car:
set_alternative_experience(self.CP, self.CP_SP, self.params)
set_car_specific_params(self.CP, self.CP_SP, self.params)
# Dynamic Experimental Control
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
openpilot_enabled_toggle = self.params.get_bool("OpenpilotEnabledToggle")
controller_available = self.CI.CC is not None and openpilot_enabled_toggle and not self.CP.dashcamOnly
self.CP.passive = not controller_available or self.CP.dashcamOnly
@@ -181,14 +178,11 @@ class Car:
self.v_cruise_helper = VCruiseHelper(self.CP, self.CP_SP)
self.is_metric = self.params.get_bool("IsMetric")
self.experimental_mode = self.params.get_bool("ExperimentalMode")
self.initial_cruise_speed = self.params.get("InitialCruiseSpeedKph", return_default=True)
# card is driven by can recv, expected at 100Hz
self.rk = Ratekeeper(100, print_delay_threshold=None)
# log fingerprint in sentry
sunnypilot_interfaces.log_fingerprint(self.CP)
def state_update(self) -> tuple[car.CarState, custom.CarStateSP, structs.RadarDataT | None]:
"""carState update loop, driven by can"""
@@ -217,7 +211,7 @@ class Car:
self.v_cruise_helper.update_v_cruise(CS, self.sm['carControl'].enabled, self.is_metric)
if self.sm['carControl'].enabled and not self.CC_prev.enabled:
# Use CarState w/ buttons from the step selfdrived enables on
self.v_cruise_helper.initialize_v_cruise(self.CS_prev, self.experimental_mode, self.dynamic_experimental_control)
self.v_cruise_helper.initialize_v_cruise(self.CS_prev, self.initial_cruise_speed)
# TODO: mirror the carState.cruiseState struct?
CS.vCruise = float(self.v_cruise_helper.v_cruise_kph)
@@ -303,9 +297,9 @@ class Car:
while not evt.is_set():
self.is_metric = self.params.get_bool("IsMetric")
self.experimental_mode = self.params.get_bool("ExperimentalMode") and self.CP.openpilotLongitudinalControl
self.initial_cruise_speed = self.params.get("InitialCruiseSpeedKph", return_default=True)
# sunnypilot
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
self.v_cruise_helper.read_custom_set_speed_params()
time.sleep(0.1)
+2 -6
View File
@@ -13,7 +13,6 @@ V_CRUISE_MIN = 8
V_CRUISE_MAX = 145
V_CRUISE_UNSET = 255
V_CRUISE_INITIAL = 40
V_CRUISE_INITIAL_EXPERIMENTAL_MODE = 105
IMPERIAL_INCREMENT = round(CV.MPH_TO_KPH, 1) # round here to avoid rounding errors incrementing set speed
ButtonEvent = car.CarState.ButtonEvent
@@ -138,17 +137,14 @@ class VCruiseHelper(VCruiseHelperSP):
self.button_timers[b.type.raw] = 1 if b.pressed else 0
self.button_change_states[b.type.raw] = {"standstill": CS.cruiseState.standstill, "enabled": enabled}
def initialize_v_cruise(self, CS, experimental_mode: bool, dynamic_experimental_control: bool) -> None:
def initialize_v_cruise(self, CS, initial_speed: int) -> None:
# initializing is handled by the PCM
if self.CP.pcmCruise:
return
initial_experimental_mode = experimental_mode and not dynamic_experimental_control
initial = V_CRUISE_INITIAL_EXPERIMENTAL_MODE if initial_experimental_mode else V_CRUISE_INITIAL
if any(b.type in (ButtonType.accelCruise, ButtonType.resumeCruise) for b in CS.buttonEvents) and self.v_cruise_initialized:
self.v_cruise_kph = self.v_cruise_kph_last
else:
self.v_cruise_kph = int(round(np.clip(CS.vEgo * CV.MS_TO_KPH, initial, V_CRUISE_MAX)))
self.v_cruise_kph = int(round(np.clip(max(CS.vEgo * CV.MS_TO_KPH, initial_speed), V_CRUISE_INITIAL, V_CRUISE_MAX)))
self.v_cruise_cluster_kph = self.v_cruise_kph
@@ -14,7 +14,7 @@ ButtonEvent = car.CarState.ButtonEvent
ButtonType = car.CarState.ButtonEvent.Type
def run_cruise_simulation(cruise, e2e, personality, t_end=20.):
def run_cruise_simulation(cruise, personality, t_end=20.):
man = Maneuver(
'',
duration=t_end,
@@ -24,7 +24,6 @@ def run_cruise_simulation(cruise, e2e, personality, t_end=20.):
cruise_values=[cruise],
prob_lead_values=[0.0],
breakpoints=[0.],
e2e=e2e,
personality=personality,
)
valid, output = man.evaluate()
@@ -32,8 +31,7 @@ def run_cruise_simulation(cruise, e2e, personality, t_end=20.):
return output[-1, 3]
@parameterized_class(("e2e", "personality", "speed"), itertools.product(
[True, False], # e2e
@parameterized_class(("personality", "speed"), itertools.product(
log.LongitudinalPersonality.schema.enumerants, # personality
[5,35])) # speed
class TestCruiseSpeed(OpenpilotTestCase):
@@ -41,7 +39,7 @@ class TestCruiseSpeed(OpenpilotTestCase):
print(f'Testing {self.speed} m/s')
cruise_speed = float(self.speed)
simulation_steady_state = run_cruise_simulation(cruise_speed, self.e2e, self.personality)
simulation_steady_state = run_cruise_simulation(cruise_speed, self.personality)
self.assertAlmostEqual(simulation_steady_state, cruise_speed, delta=.01, msg=f'Did not reach {self.speed} m/s')
@@ -59,16 +57,16 @@ class TestVCruiseHelper(OpenpilotTestCase):
for _ in range(2):
self.v_cruise_helper.update_v_cruise(car.CarState(cruiseState={"available": False}), enabled=False, is_metric=False)
def enable(self, v_ego, experimental_mode, dynamic_experimental_control):
def enable(self, v_ego):
# Simulates user pressing set with a current speed
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), experimental_mode, dynamic_experimental_control)
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), V_CRUISE_INITIAL)
def test_adjust_speed(self):
"""
Asserts speed changes on falling edges of buttons.
"""
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS)
for btn in (ButtonType.accelCruise, ButtonType.decelCruise):
for pressed in (True, False):
@@ -92,7 +90,7 @@ class TestVCruiseHelper(OpenpilotTestCase):
CS.buttonEvents = [ButtonEvent(type=ButtonType.decelCruise, pressed=pressed)]
self.v_cruise_helper.update_v_cruise(CS, enabled=enabled, is_metric=False)
if pressed:
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS)
# Expected diff on enabling. Speed should not change on falling edge of pressed
assert (not pressed) == (self.v_cruise_helper.v_cruise_kph == self.v_cruise_helper.v_cruise_kph_last)
@@ -102,7 +100,7 @@ class TestVCruiseHelper(OpenpilotTestCase):
Asserts we don't increment set speed if user presses resume/accel to exit cruise standstill.
"""
self.enable(0, False, False)
self.enable(0)
for standstill in (True, False):
for pressed in (True, False):
@@ -122,7 +120,7 @@ class TestVCruiseHelper(OpenpilotTestCase):
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS)
# first decrement speed, then perform gas pressed logic
expected_v_cruise_kph = self.v_cruise_helper.v_cruise_kph - IMPERIAL_INCREMENT
@@ -143,12 +141,10 @@ class TestVCruiseHelper(OpenpilotTestCase):
Asserts allowed cruise speeds on enabling with SET.
"""
for experimental_mode in (True, False):
for dynamic_experimental_control in (True, False):
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego), experimental_mode, dynamic_experimental_control)
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego))
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized
@@ -0,0 +1,38 @@
import numpy as np
from openpilot.common.constants import CV
from openpilot.common.realtime import DT_MDL
ACCEL_BOOST_MAX = 0.2
ACCEL_BOOST_RATE = 0.025
ACCEL_BOOST_PER_OVERRIDE = 0.05
ACCEL_BOOST_MIN_SPEED = 10 * CV.MPH_TO_MS
class AccelBoost:
def __init__(self):
self.total_boost = 0.0
self.boost_this_override = 0.0
self.boost_eligible = False
def update(self, sm, output_a_target_e2e, output_a_target_mpc, a_cruise):
enabled = sm['selfdriveState'].enabled
gas_pressed = sm['carState'].gasPressed
v_ego = sm['carState'].vEgo
speed_scale = np.interp(v_ego, [ACCEL_BOOST_MIN_SPEED, 2 * ACCEL_BOOST_MIN_SPEED], [0.0, 1.0])
boost_scale = speed_scale * np.interp(output_a_target_e2e, [-1.0, -0.5, 5.0], [0.0, 1.0, 1.0], right=0.0)
model_limited = output_a_target_e2e + self.total_boost * boost_scale < min(output_a_target_mpc, a_cruise) - 0.1
if not enabled or not gas_pressed:
self.boost_this_override = 0.0
# Latch eligibility before the override changes the lead plan.
self.boost_eligible = enabled and model_limited
if not enabled:
self.total_boost = 0.0
elif gas_pressed and self.boost_eligible:
increase = min(ACCEL_BOOST_RATE * DT_MDL * speed_scale,
ACCEL_BOOST_PER_OVERRIDE - self.boost_this_override, ACCEL_BOOST_MAX - self.total_boost)
self.total_boost += increase
self.boost_this_override += increase
return output_a_target_e2e + self.total_boost * boost_scale
@@ -4,7 +4,6 @@ from openpilot.common.realtime import DT_CTRL, DT_MDL
MIN_SPEED = 1.0
CONTROL_N = 17
CAR_ROTATION_RADIUS = 0.0
# This is a turn radius smaller than most cars can achieve
MAX_CURVATURE = 0.2
MIN_STABLE_DELAY = 0.3
@@ -1,11 +1,13 @@
import math
import numpy as np
from collections import deque
from openpilot.cereal import log
from openpilot.common.pid import PIDController
from openpilot.selfdrive.controls.lib.latcontrol import LatControl
from openpilot.selfdrive.controls.lib.drive_helpers import MAX_CURVATURE
CURVATURE_SATURATION_THRESHOLD = 1e-3 # 1/m
LAT_ACCEL_SATURATION_THRESHOLD = 0.5 # m/s^2
class LatControlCurvature(LatControl):
@@ -20,6 +22,8 @@ class LatControlCurvature(LatControl):
else:
self.pid = None
self.kf = 1.
self.desired_curvature_buffer_len = int(1. / self.dt)
self.desired_curvature_buffer = deque([0.] * self.desired_curvature_buffer_len, maxlen=self.desired_curvature_buffer_len)
def reset(self):
super().reset()
@@ -30,6 +34,9 @@ class LatControlCurvature(LatControl):
curvature_log = log.ControlsState.LateralCurvatureState.new_message()
actual_curvature = -VM.calc_curvature(math.radians(CS.steeringAngleDeg - params.angleOffsetDeg), CS.vEgo, params.roll)
error = desired_curvature - actual_curvature
self.desired_curvature_buffer.append(desired_curvature)
delay_frames = int(np.clip(lat_delay / self.dt + 1, 1, self.desired_curvature_buffer_len))
expected_curvature = self.desired_curvature_buffer[-delay_frames]
if not active:
output_curvature = 0.0
@@ -53,6 +60,6 @@ class LatControlCurvature(LatControl):
curvature_log.actualCurvature = float(actual_curvature)
curvature_log.desiredCurvature = float(desired_curvature)
curvature_log.output = float(output_curvature)
curvature_log.saturated = bool(self._check_saturation(abs(error) > CURVATURE_SATURATION_THRESHOLD, CS,
curvature_log.saturated = bool(self._check_saturation(abs(expected_curvature - actual_curvature) * CS.vEgo ** 2 > LAT_ACCEL_SATURATION_THRESHOLD, CS,
False, curvature_limited))
return 0.0, float(output_curvature), curvature_log
@@ -8,6 +8,7 @@ from openpilot.common.constants import CV
from openpilot.common.filter_simple import FirstOrderFilter
from openpilot.common.realtime import DT_MDL
from openpilot.selfdrive.modeld.constants import ModelConstants
from openpilot.selfdrive.controls.lib.accel_boost import AccelBoost
from openpilot.selfdrive.controls.lib.longcontrol import LongCtrlState
from openpilot.selfdrive.controls.lib.longitudinal_mpc_lib.long_mpc import LongitudinalMpc, LongitudinalPlanSource
from openpilot.selfdrive.controls.lib.longitudinal_mpc_lib.long_mpc import T_IDXS as T_IDXS_MPC
@@ -25,7 +26,6 @@ CONTROL_N_T_IDX = ModelConstants.T_IDXS[:CONTROL_N]
ALLOW_THROTTLE_THRESHOLD = 0.4
MIN_ALLOW_THROTTLE_SPEED = 2.5
# Lookup table for turns
_A_TOTAL_MAX_V = [1.7, 3.2]
_A_TOTAL_MAX_BP = [20., 40.]
@@ -33,7 +33,7 @@ def get_max_accel(v_ego):
return np.interp(v_ego, A_CRUISE_MAX_BP, A_CRUISE_MAX_VALS)
def get_coast_accel(pitch):
return np.sin(pitch) * -5.65 - 0.3 # fitted from data using xx/projects/allow_throttle/compute_coast_accel.py
return np.sin(pitch) * -5.65 - 0.3
def get_cruise_accel(e2e, v_cruise, v_ego, a_cruise_prev, angle_steers, CP, dt, accel_coast, allow_throttle):
max_accel = ACCEL_MAX if e2e else get_max_accel(v_ego)
@@ -62,6 +62,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
LongitudinalPlannerSP.__init__(self, self.CP, CP_SP, self.mpc)
self.fcw = False
self.dt = dt
self.accel_boost = AccelBoost()
self.allow_throttle = True
self.v_desired_filter = FirstOrderFilter(init_v, 2.0, self.dt)
@@ -89,18 +90,18 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
long_control_off = sm['controlsState'].longControlState == LongCtrlState.off
# Reset current state when not engaged, or user is controlling the speed
reset_state = long_control_off if self.CP.openpilotLongitudinalControl else not sm['selfdriveState'].enabled
# PCM cruise speed may be updated a few cycles later, check if initialized
v_cruise_initialized = sm['carState'].vCruise != V_CRUISE_UNSET
reset_state = reset_state or not v_cruise_initialized
throttle_probs = sm['modelV2'].meta.disengagePredictions.gasPressProbs
throttle_prob = throttle_probs[1] if len(throttle_probs) > 1 else 1.0
self.allow_throttle = throttle_prob > ALLOW_THROTTLE_THRESHOLD or v_ego <= MIN_ALLOW_THROTTLE_SPEED
steer_angle_without_offset = sm['carState'].steeringAngleDeg - sm['vehicleParameters'].angleOffsetDeg
# Reset current state when not engaged, or user is controlling the speed
reset_state = long_control_off if self.CP.openpilotLongitudinalControl else not sm['selfdriveState'].enabled
# PCM cruise speed may be updated a few cycles later, check if initialized
v_cruise_initialized = sm['carState'].vCruise != V_CRUISE_UNSET
reset_state = reset_state or not v_cruise_initialized
if reset_state:
self.v_desired_filter.x = v_ego
self.output_a_target = np.clip(sm['carState'].aEgo, ACCEL_MIN, ACCEL_MAX)
@@ -116,7 +117,10 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
v_cruise, self.output_a_target = LongitudinalPlannerSP.update_targets(self, sm, self.v_desired_filter.x, self.output_a_target, v_cruise)
self.mpc.set_weights(prev_accel_constraint, personality=sm['selfdriveState'].personality)
self.mpc.set_cur_state(self.v_desired_filter.x, self.output_a_target)
if self.CP.openpilotLongitudinalControl:
self.mpc.set_cur_state(self.v_desired_filter.x, self.output_a_target)
else:
self.mpc.set_cur_state(v_ego, sm['carState'].aEgo)
self.mpc.update(sm['radarState'], personality=sm['selfdriveState'].personality)
self.v_desired_trajectory = np.interp(CONTROL_N_T_IDX, T_IDXS_MPC, self.mpc.v_solution)
@@ -138,16 +142,18 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
output_a_target_e2e = sm['modelV2'].action.desiredAcceleration
output_should_stop_e2e = sm['modelV2'].action.shouldStop
is_e2e = self.is_e2e(sm)
experimental_mode = sm['selfdriveState'].deprecated.experimentalMode
self.a_cruise = get_cruise_accel(is_e2e, v_cruise, v_ego,
self.a_cruise = get_cruise_accel(experimental_mode, v_cruise, v_ego,
self.a_cruise, steer_angle_without_offset, self.CP, self.dt,
accel_coast, self.allow_throttle)
cruise_should_stop = should_stop(v_ego, self.a_cruise)
output_a_target_e2e = self.accel_boost.update(sm, output_a_target_e2e, output_a_target_mpc, self.a_cruise)
candidates = [(output_a_target_mpc, self.mpc.source, output_should_stop_mpc),
(self.a_cruise, LongitudinalPlanSource.cruise, cruise_should_stop)]
if is_e2e:
if experimental_mode:
candidates.append((output_a_target_e2e, LongitudinalPlanSource.e2e, output_should_stop_e2e))
output_a_target, self.mpc.source, _ = min(candidates, key=lambda c: c[0])
@@ -175,6 +181,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
longitudinalPlan.fcw = self.fcw
longitudinalPlan.aTarget = float(self.output_a_target)
longitudinalPlan.accelBoost = float(self.accel_boost.total_boost)
longitudinalPlan.shouldStop = bool(self.output_should_stop)
longitudinalPlan.allowBrake = True
longitudinalPlan.allowThrottle = bool(self.allow_throttle)
@@ -13,7 +13,7 @@ def desired_follow_distance(v_ego, v_lead, t_follow=None):
t_follow = get_T_FOLLOW()
return get_safe_obstacle_distance(v_ego, t_follow) - get_stopped_equivalence_factor(v_lead)
def run_following_distance_simulation(v_lead, t_end=100.0, e2e=False, personality=0):
def run_following_distance_simulation(v_lead, t_end=100.0, personality=0):
man = Maneuver(
'',
duration=t_end,
@@ -22,7 +22,6 @@ def run_following_distance_simulation(v_lead, t_end=100.0, e2e=False, personalit
initial_distance_lead=100,
speed_lead_values=[v_lead],
breakpoints=[0.],
e2e=e2e,
personality=personality,
)
valid, output = man.evaluate()
@@ -30,8 +29,7 @@ def run_following_distance_simulation(v_lead, t_end=100.0, e2e=False, personalit
return output[-1,2] - output[-1,1]
@parameterized_class(("e2e", "personality", "speed"), itertools.product(
[True, False], # e2e
@parameterized_class(("personality", "speed"), itertools.product(
[log.LongitudinalPersonality.relaxed, # personality
log.LongitudinalPersonality.standard,
log.LongitudinalPersonality.aggressive],
@@ -39,9 +37,8 @@ def run_following_distance_simulation(v_lead, t_end=100.0, e2e=False, personalit
class TestFollowingDistance(OpenpilotTestCase):
def test_following_distance(self):
v_lead = float(self.speed)
simulation_steady_state = run_following_distance_simulation(v_lead, e2e=self.e2e, personality=self.personality)
simulation_steady_state = run_following_distance_simulation(v_lead, personality=self.personality)
correct_steady_state = desired_follow_distance(v_lead, v_lead, get_T_FOLLOW(self.personality))
err_ratio = 0.2 if self.e2e else 0.1
abs_err_margin = 0.5 if v_lead > 0.0 else 1.15
self.assertAlmostEqual(simulation_steady_state, correct_steady_state,
delta=err_ratio * correct_steady_state + abs_err_margin)
delta=0.2 * correct_steady_state + abs_err_margin)
@@ -8,12 +8,14 @@ from opendbc.car.honda.values import CAR as HONDA
from opendbc.car.toyota.values import CAR as TOYOTA
from opendbc.car.nissan.values import CAR as NISSAN
from opendbc.car.gm.values import CAR as GM
from opendbc.car.volkswagen.values import CAR as VOLKSWAGEN
from opendbc.car.vehicle_model import VehicleModel
from openpilot.common.realtime import DT_CTRL
from openpilot.selfdrive.car.helpers import convert_to_capnp
from openpilot.selfdrive.controls.lib.latcontrol_pid import LatControlPID
from openpilot.selfdrive.controls.lib.latcontrol_torque import LatControlTorque
from openpilot.selfdrive.controls.lib.latcontrol_angle import LatControlAngle
from openpilot.selfdrive.controls.lib.latcontrol_curvature import LatControlCurvature
from openpilot.selfdrive.locationd.helpers import Pose
from openpilot.common.mock.generators import generate_deviceMotion
from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfaces
@@ -22,7 +24,8 @@ from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfac
class TestLatControl(OpenpilotTestCase):
@parameterized.expand([(HONDA.HONDA_CIVIC, LatControlPID), (TOYOTA.TOYOTA_RAV4, LatControlTorque),
(NISSAN.NISSAN_LEAF, LatControlAngle), (GM.CHEVROLET_BOLT_EUV, LatControlTorque)])
(NISSAN.NISSAN_LEAF, LatControlAngle), (GM.CHEVROLET_BOLT_EUV, LatControlTorque),
(VOLKSWAGEN.VOLKSWAGEN_ID4_MK2, LatControlCurvature)])
def test_saturation(self, car_name, controller):
CarInterface = interfaces[car_name]
CP = CarInterface.get_non_essential_params(car_name)
+6 -6
View File
@@ -278,12 +278,12 @@ def main():
estimator = LocationEstimator(DEBUG)
filter_initialized = False
critcal_services = ["accelerometer", "gyroscope", "cameraOdometry"]
critical_services = ["accelerometer", "gyroscope", "cameraOdometry"]
observation_input_invalid = defaultdict(int)
input_invalid_limit = {s: round(INPUT_INVALID_LIMIT * (SERVICE_LIST[s].frequency / 20.)) for s in critcal_services}
input_invalid_threshold = {s: input_invalid_limit[s] - 0.5 for s in critcal_services}
input_invalid_decay = {s: calculate_invalid_input_decay(input_invalid_limit[s], INPUT_INVALID_RECOVERY, SERVICE_LIST[s].frequency) for s in critcal_services}
input_invalid_limit = {s: round(INPUT_INVALID_LIMIT * (SERVICE_LIST[s].frequency / 20.)) for s in critical_services}
input_invalid_threshold = {s: input_invalid_limit[s] - 0.5 for s in critical_services}
input_invalid_decay = {s: calculate_invalid_input_decay(input_invalid_limit[s], INPUT_INVALID_RECOVERY, SERVICE_LIST[s].frequency) for s in critical_services}
initial_pose_data = params.get("LocationFilterInitialState")
if initial_pose_data is not None:
@@ -313,7 +313,7 @@ def main():
if valid:
t = log_mono_time * 1e-9
res = estimator.handle_log(t, which, msg)
if which not in critcal_services:
if which not in critical_services:
continue
if res == HandleLogResult.TIMING_INVALID:
@@ -328,7 +328,7 @@ def main():
filter_initialized = sm.all_checks() and sensor_all_checks(acc_msgs, gyro_msgs, sensor_valid, sensor_recv_time, sensor_alive, SIMULATION)
if sm.updated["cameraOdometry"]:
critical_service_inputs_valid = all(observation_input_invalid[s] < input_invalid_threshold[s] for s in critcal_services)
critical_service_inputs_valid = all(observation_input_invalid[s] < input_invalid_threshold[s] for s in critical_services)
inputs_valid = sm.all_valid() and critical_service_inputs_valid
sensors_valid = sensor_all_checks(acc_msgs, gyro_msgs, sensor_valid, sensor_recv_time, sensor_alive, SIMULATION)
@@ -80,7 +80,6 @@ class CarKalman(KalmanFilter):
}
global_vars = [
'mass',
'rotational_inertia',
'center_to_front',
'center_to_rear',
@@ -99,7 +98,10 @@ class CarKalman(KalmanFilter):
# globals
global_vars = [sp.Symbol(name) for name in CarKalman.global_vars]
m, j, aF, aR, cF_orig, cR_orig = global_vars
j, aF, aR, cF_orig, cR_orig = global_vars
# Tire stiffness and rotational inertia are already divided by vehicle mass, which cancels out of the model.
# Use unit mass to preserve the standard bicycle-model equations below.
m = 1
# make functions and jacobians with sympy
# state variables
@@ -166,8 +168,7 @@ class CarKalman(KalmanFilter):
self.filter = EKF_sym_pyx(generated_dir, CarKalman.name, CarKalman.Q, CarKalman.initial_x, CarKalman.P_initial,
dim_state, dim_state_err, global_vars=CarKalman.global_vars, logger=cloudlog)
def set_globals(self, mass, rotational_inertia, center_to_front, center_to_rear, stiffness_front, stiffness_rear):
self.filter.set_global("mass", mass)
def set_globals(self, rotational_inertia, center_to_front, center_to_rear, stiffness_front, stiffness_rear):
self.filter.set_global("rotational_inertia", rotational_inertia)
self.filter.set_global("center_to_front", center_to_front)
self.filter.set_global("center_to_rear", center_to_rear)
@@ -40,7 +40,7 @@ class PoseKalman(KalmanFilter):
initial_P = np.diag([0.01**2, 0.01**2, 0.01**2,
10**2, 10**2, 10**2,
1**2, 1**2, 1**2,
1**2, 1**2, 1**2,
0.1**2, 0.1**2, 0.1**2,
100**2, 100**2, 100**2,
0.01**2, 0.01**2, 0.01**2])
+3 -4
View File
@@ -36,12 +36,11 @@ class VehicleParamsLearner:
self.P_initial = P_initial if P_initial is not None else CarKalman.P_initial
self.kf.set_globals(
mass=CP.mass,
rotational_inertia=CP.rotationalInertia,
rotational_inertia=CP.rotationalInertiaPerMass,
center_to_front=CP.centerToFront,
center_to_rear=CP.wheelbase - CP.centerToFront,
stiffness_front=CP.tireStiffnessFront,
stiffness_rear=CP.tireStiffnessRear
stiffness_front=CP.tireStiffnessFrontPerMass,
stiffness_rear=CP.tireStiffnessRearPerMass
)
self.min_sr, self.max_sr = 0.5 * CP.steerRatio, 2.0 * CP.steerRatio

Some files were not shown because too many files have changed in this diff Show More