Commit Graph

21 Commits

Author SHA1 Message Date
Jason Wen 309304a352 Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control (#695)
* init

* more init

* keep it alive

* fixes

* more fixes

* more fix

* new submodule for nn data

* bump submodule

* update path to submodule

* spacing???

* update submodule path

* update submodule path

* bump

* dump

* bump

* introduce params

* Add Neural Network Lateral Control toggle to developer panel

This introduces a new toggle for enabling Neural Network Lateral Control (NNLC), providing detailed descriptions of its functionality and compatibility. It includes UI integration, car compatibility checks, and feedback links for unsupported vehicles.

* decouple even more

* static

* codespell

* remove debug

* in structs

* fix import

* convert to capnp

* fixes

* debug

* only initialize if NNLC is enabled or allow to enable

* oops

* fix initialization

* only allow engage if nnlc is off

* fix toggle param

* fix tests

* lint

* fix more test

* capnp test

* try this out

* validate if it's not None

* make it 33 to match

* align

* share the same friction input calculation

* return stock values if not enabled

* unused

* split base and child

* space

* rename

* NeuralNetworkFeedForwardModel

* less

* just use file name

* try this

* more explicit

* rename

* move it

* child class for additional controllers

* rename

* time to split out custom lateral acceleration

* move around

* space

* fix

* TODO-SP

* TODO-SP

* update regardless, it's an extension now

* update name and expose toggle

* ui: sunnypilot Panel -> Steering Panel

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h

* merge

* move to steering panel

* no need for this

* live params in a thread

* no live for now

* new structs

* more ui

* more flexible

* more ui

* no longer needed

* another ui

* cereal changes

* bump opendbc

* simplify checks

* all in one place

* split Enhanced Lat Accel

* handle unrecognized platform

* test for fingerprinting

* fix fingerprint

* NNLC: Mock data for unrecognized cars

* fix fingerprints

* test to verify model loading

* bump neural_network_data

* use pytest

* use different one for now

* fix ui preview alignments

* typing

* more type

* show a platform

* show match

* init params for tests

* ok ruff

* make sure to loop through tests

* ok again ruff

* ok we need this lol

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-22 09:40:29 -04:00
Jason Wen 0990212edc Revert "Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control" (#694)
Revert "Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Ac…"

This reverts commit ecb4026269.
2025-03-21 16:13:50 -04:00
Jason Wen ecb4026269 Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control (#667)
* init

* more init

* keep it alive

* fixes

* more fixes

* more fix

* new submodule for nn data

* bump submodule

* update path to submodule

* spacing???

* update submodule path

* update submodule path

* bump

* dump

* bump

* introduce params

* Add Neural Network Lateral Control toggle to developer panel

This introduces a new toggle for enabling Neural Network Lateral Control (NNLC), providing detailed descriptions of its functionality and compatibility. It includes UI integration, car compatibility checks, and feedback links for unsupported vehicles.

* decouple even more

* static

* codespell

* remove debug

* in structs

* fix import

* convert to capnp

* fixes

* debug

* only initialize if NNLC is enabled or allow to enable

* oops

* fix initialization

* only allow engage if nnlc is off

* fix toggle param

* fix tests

* lint

* fix more test

* capnp test

* try this out

* validate if it's not None

* make it 33 to match

* align

* share the same friction input calculation

* return stock values if not enabled

* unused

* split base and child

* space

* rename

* NeuralNetworkFeedForwardModel

* less

* just use file name

* try this

* more explicit

* rename

* move it

* child class for additional controllers

* rename

* time to split out custom lateral acceleration

* move around

* space

* fix

* TODO-SP

* TODO-SP

* update regardless, it's an extension now

* update name and expose toggle

* ui: sunnypilot Panel -> Steering Panel

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h

* merge

* move to steering panel

* no need for this

* live params in a thread

* no live for now

* new structs

* more ui

* more flexible

* more ui

* no longer needed

* another ui

* cereal changes

* bump opendbc

* simplify checks

* all in one place

* split Enhanced Lat Accel

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-21 15:38:31 -04:00
Jason Wen bf2731b470 MADS: Steering Mode on Brake Pedal Press (#687)
* reimplement

* bump opendbc

* Apply suggestions from code review

Co-authored-by: DevTekVE <devtekve@gmail.com>

* update toggle positions

* translations from bot later

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-19 18:24:56 -04:00
Jason Wen dd37a35b59 Revert "MADS: Steering Mode on Brake Pedal Press" (#686)
* Revert "MADS: Steering Mode on Brake Pedal Press (#684)"

This reverts commit 4525659a7e.

* bump
2025-03-19 16:02:45 -04:00
Jason Wen 4525659a7e MADS: Steering Mode on Brake Pedal Press (#684)
* MADS: cleanup param assignments

* MADS: Steering Mode on Brake Pedal Press

* bump opendbc

* translations

* make sure to initialize

* ci try this

* Revert "ci try this"

This reverts commit da975f61c3.

* Revert "translations"

This reverts commit a65aad5911.
2025-03-19 13:57:15 -04:00
Jason Wen 5c38aeae0b Longitudinal: Dynamic Experimental Control (#572)
* init dec

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* fix static test

* ff

* fix static test

* unitee testt

* Refactor test_dynamic_controller and fix formatting issues

Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency.

* Refactor test indentation for dynamic controller tests

Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code.

* Migrated to pytest using claude

* Integrate radar parameter into dynamic controller's pytest tests

Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases.

* Disabling unittest file to allow checks on the pipeline to succeed.

Pending to remove this, but leaving it to validate the move to pytest is okay before merging

* Replace unittest with pytest for dynamic controller tests

Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage.

* new line...

* Refactor and modularize DynamicExperimentalController logic

Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports.

* Add missing import for messaging in helpers.py

The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code.

* Format

* Formatting

* rebase fix

* Refactor MpcSource definition and update references.

Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality.

* Format

* Refactor DEC into a dedicated longitudinal planner class

Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity.

* **Refactor DEC module structure for better organization**

Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes.

* static test

* static

* had moved to car_state

* cleanup

* some more

* static method

* move around

* more cleanup

* stuff

* into their own

* rename

* check live param

* sync with stock

* type hint

* unused

* smoother trans

* window time

* fix type hint

* pass sm.frame from plannerd

* more fixes

* more

* more explicit

* fix test

* Revert "fix test"

This reverts commit 635b15f2bc.

* Revert "pass sm.frame from plannerd"

This reverts commit a8deaa69b8.

* use internal frame

* update name

* never used

* this is why it was never using DEC

* more logs

* slight cleanup

* remove to fail test

* update name

* more

* rename

* move around

* explicit type hints

* move to constants py

* Revert "explicit type hints"

This reverts commit c205497b

* more

* don't set to exp mode initial if DEC is active

* use walrus for None

* Revert "use walrus for None"

This reverts commit 5f2396d490.

* fix wrong typing and variable name

* use walrus (needs cleanup)

* fix tests

* revert smooht lead for now

* dec: how good is FirstOrderFilter?

* Update dec.py

* dec: faster ?

* Revert "dec: faster ?"

This reverts commit 40259cd22a.

* Revert "Update dec.py"

This reverts commit 3f29ccbd99.

* Revert "dec: how good is FirstOrderFilter?"

This reverts commit 01e06df542.

* Update slow-down logic and constants for improved behavior

Adjust the slowdown scaling factor and anomaly handling to refine behavior without abrupt resets. Modify constants to increase window size and adjust probabilities and distances for smoother adaptation. Update version to reflect the new changes.

* Fix lead detection fallback for weighted average check.

Add a fallback value of -1 when computing the weighted average to prevent errors caused by invalid or None values. This ensures robust lead detection and avoids potential crashes or undefined behavior.

* visuals for DEC

* try this

* add opacity

* should be active and dimmer

* even dimmer

* Update dec.py

* Update constants.py

* use another method for drawing

* migrate to sp only

* fix

* init

---------

Co-authored-by: rav4kumar <meetkumardesai@gmail.com>
Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-19 12:04:54 -05:00
Kumar c3bfd17028 Revert "Longitudinal: Dynamic Experimental Control" (#571)
Revert "Longitudinal: Dynamic Experimental Control (#564)"

This reverts commit bba3c39e2f.
2025-01-13 05:17:15 +00:00
Kumar bba3c39e2f Longitudinal: Dynamic Experimental Control (#564)
* init dec

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* fix static test

* ff

* fix static test

* unitee testt

* Refactor test_dynamic_controller and fix formatting issues

Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency.

* Refactor test indentation for dynamic controller tests

Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code.

* Migrated to pytest using claude

* Integrate radar parameter into dynamic controller's pytest tests

Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases.

* Disabling unittest file to allow checks on the pipeline to succeed.

Pending to remove this, but leaving it to validate the move to pytest is okay before merging

* Replace unittest with pytest for dynamic controller tests

Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage.

* new line...

* Refactor and modularize DynamicExperimentalController logic

Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports.

* Add missing import for messaging in helpers.py

The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code.

* Format

* Formatting

* rebase fix

* Refactor MpcSource definition and update references.

Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality.

* Format

* Refactor DEC into a dedicated longitudinal planner class

Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity.

* **Refactor DEC module structure for better organization**

Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes.

* static test

* static

* had moved to car_state

* cleanup

* some more

* static method

* move around

* more cleanup

* stuff

* into their own

* rename

* check live param

* sync with stock

* type hint

* unused

* smoother trans

* window time

* fix type hint

* pass sm.frame from plannerd

* more fixes

* more

* more explicit

* fix test

* Revert "fix test"

This reverts commit 635b15f2bc.

* Revert "pass sm.frame from plannerd"

This reverts commit a8deaa69b8.

* use internal frame

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-12 19:11:58 -05:00
Jason Wen 8edeb458d4 Merge branch 'openpilot/master' into sync-20250108
# Conflicts:
#	common/base.py
#	common/comma_connect.py
#	opendbc_repo
#	panda
2025-01-08 23:42:52 -05:00
Jason Wen ec044790fb Revert "Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new" (#543)
Revert "Sync: `commaai/openpilot:master` into `sunnypilot/sunnypilot:master-n…"

This reverts commit e682957101.
2025-01-08 23:37:41 -05:00
Jason Wen e682957101 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#533)
* cabana: enhance message heatmap visualization (#34239)

* enhance message heatmap visualization

* TODO

* improve log_factor

* typo

* bit_flip_counts

* Openpilot webcam support improved (#34215)

* control webcam with ENV vars

* WIP: actual instructions

* wording

* file no longer exists

* this is expected behavior, just untested

* more readable

* tested on fresh install

* wording tweaks

* explicit USE_WEBCAM toggle required

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug-ability improved

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* newline removed

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update metadrive wheel (#34292)

* test

* new wheel

* fix IR power scaling (#34293)

* fix IR power scaling

* Update system/hardware/tici/hardware.h

* replay: Update video immediately after seek when paused. (#34237)

replay: Update video immediately after seeking when paused.

Otherwise, if paused then have to resume playback for the video
frame to update and show the new location.

Implemented by temporarily un-pausing replay for a single
frame time.

* cabana: add live and time-window heatmap modes for enhanced signal analysis (#34296)

add live and time-window heatmap modes

* timed: diff against absolute value of timedelta (#34299)

* cabana: miscellaneous bug fixes and enhancements (#34297)

* toHexString

* use QToolBar

* fix incorrect groove rect

* limit CAN_MAX_DATA_BYTES

* add series type selector to chart toolbar

* dim inactive messages

* rename

* add help to chart

* cleanup

* cabana: real-time cursor and video frame sync for chart and video (#34301)

* sync cursor and thumbnail between chart and video

* Revert "replay: Update video immediately after seek when paused. (#34237)"

This reverts commit 3363881844.

* use thumbnails while scrubing

* draw alert

* no update on resume

* draw timestamp

* cleanup

* replay:  fix various synchronization and event handling issues (#34254)

fix various synchronization and event handling issues

* cabana: fix crash in live streaming mode by skipping thumbnail display (#34302)

resolve crash in live streaming mode

* bump panda

* [bot] Update Python packages (#34304)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* cleanup touch_replay (#34305)

mathematics

* uv from brew doesn't have self update

* Skip registration on newer devices (#34316)

* tici: fix cpp device type (#34315)

fix cpp

* Tinygrad upstream master (#34325)

Upstream master

* [bot] Update Python packages (#34320)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* cabana: fix missing transmitter after undoing DBC message removal (#34329)

fix missing transmitter after undoing DBC message removal

* Quick GC pass heading into 2025 (#34330)

* first pass

* bye bye snpe

* [bot] Update Python packages (#34334)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* Notre Dame model in tinygrad (#34324)

* release model: 6f23a03f-486b-4d3e-a314-19d149644c7c/700

* old style model in tinygrad

* fix desire

* tg hack

* 20Hz

* no gas probs

* No gas here

* better indexing

---------

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>

---------

Co-authored-by: Dean Lee <deanlee3@gmail.com>
Co-authored-by: Mike Busuttil <31480000+MikeBusuttil@users.noreply.github.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
Co-authored-by: Angus Gratton <gus@projectgus.com>
Co-authored-by: commaci-public <60409688+commaci-public@users.noreply.github.com>
Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Harald Schäfer <harald.the.engineer@gmail.com>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2025-01-07 06:28:20 +00:00
DevTekVE ee5c1c4507 Modular Assistive Driving System (MADS) (#446)
* allow re-regage

* bump opendbc

* bump panda

* apply pause/resume fix for hyundai (should do this in a separate PR)

* bump opendbc

* fix

* rename

* Fix?

* make sure to disengage for allow always cars

* fix

* combine

* more fix

* not needed

* check if engagement is from openpilot's state machine

* Rename

* fix panda safety

* fix

* no fake lfa button for @devtekve ;)

* fix non drive gear re-engage

* fix settings

* combine

* add replace method

* use replace

* remoev already checks if it exists

* fix

* group

* add todo

* reserve events

* cleaner

* hyundai: only allow for cars with lfa button

* sunnyParams

* make sure it's car only

* Move car-specific changes to opendbc

* no need

* bump opendbc

* more fixes

* no more available

* more!

* final?

* always emit user disable

* no longer needed

* move unit test

* add sunnypilot to unit tests

* bump opendbc

* use new cereal

* bump opendbc

* static analysis

* no unittest

* no need available

* UI border update

* show MADS updates

* Add TODO

* no longer needed

* fix changed events

* fix cluster enabled

* don't add pre enable if not long

* should use enabled

* enabled <-> active

* better format

* bump opendbc

* static analysis

* static analysis

* Rename test as collector was dying

* Show our overriding

* Revert "show MADS updates"

This reverts commit daf0ad62

Revert "fix changed events"

This reverts commit 31d8c97f

* ignoring reserved events

* adjusting creation delays

* back to stock

removing allow_cancel

* should be enabled

* revert

* silent lkas disable

* no need

* user disable tests

* just warning

* MUST REMOVE test process replay

* fix no entry

* fixme

* bump opendbc

* need this check

* cleanup

* allow entering paused state if no entry from disabled

* brake hold should apply to all

* in lists

* update unit test

* simpler

* unused

* same thing

* fix

* only mads in enabled state and long in disabled state

* unify silent enable

* do this for dlob

* bump submodules

* fix

* bump submodules

* bump opendbc

* less frequent

* more events

* fix

* allow no entry to paused for non-drive gears

* fix

* use cereal

* Revert "allow no entry to paused for non-drive gears"

This reverts commit 6d64a4dd9c.

* allow in all

* Revert "allow in all"

This reverts commit 6375f14891.

* should not be all!

* rename for clarity

* silent park brake

* flipped

* bump submodules

* Bump to latest mads-new panda

* bump panda

* more nissan

* bump panda

* bump msgq

* bump panda

* bump submodules

* bump opendbc

* bump opendbc

* improving the state

* Revert "PlayStation® model (#34133)"

This reverts commit 5160bee543.

* should be none

* bump panda

* bump opendbc

* Apply suggestions from code review

* bump panda

* bump ref panda

* add todo-sp

* bump panda ref

* bump more panda

* changing refs

* nuke nuke nuke

* use sunny's newer states

* bump with new panda

* bump panda

* Parse more flags from alt exp, more tests, hyundai main cruise allowed

* Parse more flags from alt exp, more tests, hyundai main cruise allowed

* missed

* mutation for controls allowed rising edge

* ford mutation

* license

* remove

* unused

* bump submodules

* use always allowed mads button alt exp

* fix

* whitelist jason's lastname to codespell

* test_processes: update ref logs to 82c0278

* bump submodules

* bump submodules

* bump submodules

* bump panda

* add controls mismatch lateral event

* Simplify lateral disengagement logic for MADS configuration

Reversed the conditional to align the logic with the `disengage_lateral_on_brake` parameter. This ensures that lateral disengagement behavior is more intuitive and matches the expected configuration. Improves code readability and reduces potential misconfigurations.

* remove unified engagement mode in panda

* controls allow should be allowed at all times

* squash! treat MADS button as user entry

* heartbeat for mads

* heartbeat mismatch exit control

* remove always allow mads button from alt

* move to safety_mads

* remove main cruise allowed from alt

* bump panda

* heartbeat engaged mads mismatch mutation test

* bump panda

* use mads the third panda

* ignore pre enable with mads

* only force exit if actually actuating

* use brake signal instead of pedal events when dlob is active

* fix tests

* fix panda tests

* bump panda

* new events to retain long blocks

* format

* uem: do not engage mads if long is engaged

* bump submodules

* fix not allowed engaged bug

* block uem from engaging

* flipped

* use different heartbeat check if dlob

* hard code to skip heartbeat check

* remove toyota lta status for lkas, causes weird behaviors

* block tesla

* bump panda

* bump to merged panda

* bump opendbc

* bump opendbc

* bump opendbc

* bump opendbc

* Apply suggestions from code review

* code ignore spells

* needs to be in carstate

* Bump opendbc

* Update MADS toggle descriptions for clarity.

Added notes to clarify behavior of the "MadsMainCruiseAllowed" setting, particularly its impact on vehicles without LFA/LKAS buttons. This ensures users are informed about potential implications when disabling this feature.

* Updating translations + Adding spanish

* Disengage Lateral on Brake -> Pause Lateral on Brake

* test_processes: update ref logs to dd41005

* Apply suggestions from code review

* fix mads button not allowed

* bump submodules

* bump submodule

* test_processes: update ref logs to 0a0b998

* has multiple lists

* Revert "has multiple lists"

This reverts commit a37c1d26fe.

* base

* Reapply "has multiple lists"

This reverts commit d1cd8dcc81.

* migrate mads toggles to sp panel

* this is why it keeps crashing

* house keeping

* more housekeeping

* more housekeeping

* don't show description by default (yet)

* reset to main panel when clicked away

* more

* some more with interactions

* don't stretch cause it looks weird with descriptions

* simpler to handle offroad transition

* some are toggleable while onroad

* remove unused event

* slight cleanup

* default to true for HKG main cruise toggle

* append to list after

* add Customize MADS to UI preview

* simpler

* move to sp list

* how tf was this removed

* update mads settings button on show event

* test_processes: update ref logs to efa9c32

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-04 20:48:02 -05:00
DevTekVE 993c4a0d2a Driving Model Manager (#457)
* Introduce Model Manager to handle downloads and verification

This commit introduces a new Model Manager responsible for handling model downloads, including driving and navigation application models. The manager also verifies file hashes and communicates download progress for an improved user experience.

The Model Manager is asynchronous and utilizes asyncio and aiohttp for enhanced performance, including robust error handling.

Impacted files in the 'cereal', 'common', 'sunnypilot', and 'system' directories have been updated accordingly. The 'ModelsFetcher' process configuration has been modified to run only when off-road, ensuring optimum resource management.

This update aims to enhance code clarity, improve performance, and streamline the handling of model downloads.

* "Update model management and fetching for SunnyPilot"

This update refactors the model management, downloading and cache verification mechanisms of SunnyPilot. New functionalities, such as smart cache handling, have been implemented in ModelFetcher in sunnypilot/models/model_fetcher.py. Also, the model downloading process has been moved to a separate async function _download_bundle in ModelManagerSP in sunnypilot/models/model_manager.py. Hash verification of files is now performed in an async function verify_file in sunnypilot/models/model_helper.py. Changes in system parameters related to model management have been reflected in system/manager/manager.py.

* Integrate download ETA calculations in model manager

This commit introduces a new feature that calculates and tracks the Estimated Time of Arrival (ETA) for downloading models in the model manager component. The 'eta' property in the 'DownloadProgress' structure in 'custom.capnp' is changed from 'Float32' to 'UInt32'.

In the 'model_manager.py' file, a new method '_calculate_eta' has been added to perform ETA calculations. An additional dictionary '_download_start_times' has been created to keep track of the start time of each model download. The ETA is calculated every time a portion of the model file is downloaded, and it gets updated in the 'DownloadProgress' structure. Finally, the start time is cleared after the download completes.

In 'model_manager_audit.py', an additional check is added to only print downloadProgress for the downloads currently in progress.

* format

* no default model cache {} because it can be considered a valid json, we do not want that

* Refactor typing annotations to use PEP 604 syntax.

Updated type hints to adopt PEP 604 union syntax (`X | None`) and replaced `List` and `Dict` with modern built-in `list` and `dict`. This change improves consistency and readability while aligning with Python 3.10+ standards.

* Simplify logging messages and remove unused imports.

Removed an unused import from `model_manager.py` to improve clarity and maintainability. Also refined a log message in `model_fetcher.py` by removing unnecessary formatting for consistency.

* Refactor model handling and simplify cache fallback logic.

Updated type annotations for `selected_bundle` in `model_manager.py` for clarity. Streamlined cache fallback logic in `model_fetcher.py` by removing redundant conditionals while preserving functionality. These changes improve code readability and maintainability.

* "Fix formatting for ModelManager_DownloadIndex retrieval

Condensed parameter alignment in the get method for improved readability and adherence to style guidelines. This change does not affect functionality but ensures consistent code formatting."

* Need to have main defined for process_config to be able to run it

* Refactor model management to support active bundle tracking

Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.

* UI commit (#515)

* Refactor model management to support active bundle tracking

Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.

* Add new driving model selection feature to settings

This commit introduces a new feature to the settings that allows users to select different driving models. It fetches available models and displays their download progress. The created UI also suggests a calibration reset after model download. The changes include the creation of 'SoftwarePanelSP' within 'settings.' Additionally, 'sunnypilot/SConscript' has been updated to include 'settings.cc' and 'software_panel.cc'. Changes also include localization for this feature.

* Show model description during download status

This update ensures the model description is displayed when a model is in the downloading state. It improves the user interface by providing real-time feedback during the download process.

* Update translations for multiple languages

Added new and updated translation strings in several language files, including Spanish, Arabic, Chinese (Simplified and Traditional), Turkish, Korean, Thai, Japanese, and Brazilian Portuguese. These updates include placeholder translations for new UI elements and features.

* Refactor model name handling and add generation check.

Replaced `bundleName` with `model_name` for better clarity in status messages. Added a generation mismatch check before showing the reset parameters dialog to avoid unnecessary prompts.

* Update model handling in SoftwarePanelSP

Remove unused "common/model.h" and replace "CURRENT_MODEL" with "..." as the default return value in GetModelName. Adjust logic to check for active bundles instead of selected bundles for improved accuracy. Minor text change for clarity in UI label.

* Rename `GetModelName` to `GetActiveModelName` for clarity.

The new name better reflects the function's purpose of retrieving the active model name, improving code readability. All relevant calls and references have been updated to ensure consistency across the codebase.

* Update sunnypilot/models/model_helper.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Refactor download status handling and add 'cached' state

Introduce 'cached' as a new download status and adjust relevant logic across components to support it. Simplify and streamline model status handling in the software panel for better readability and maintainability. Ensure consistent status reporting for all model types.

* Update translations for multiple languages

Refined and expanded translations across various languages, replacing placeholders with meaningful text. This improves clarity and user experience in the multilingual interface.

* Update terminology from 'bundle' to 'model' in UI texts

Replaced occurrences of 'bundle' with 'model' in button labels, dialog titles, and messages in the SoftwarePanelSP code. This improves clarity and aligns terminology with current functionality.

* Update translation placeholders for model fetching texts

Replaced "Fetching bundles" with "Fetching models" across multiple languages to align text placeholders with the updated functionality. Adjusted related background download messages for clarity and consistency.

* cleanup

* not used, and likely not needed

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* cleaning up

* Update system/manager/process_config.py

* Simplify model parsing and index handling logic.

Refactored `ModelManager_DownloadIndex` retrieval to use the walrus operator, streamlining the conditional logic. Additionally, restructured model list initialization in `_parse_bundle` for improved readability and maintainability. These changes enhance code clarity and reduce redundancy.

* `Improve error handling in model cache retrieval`

Revised the `get` method to ensure it returns an empty dictionary on errors or missing data, avoiding potential `None`-related issues. Added logging for clearer diagnostics when cached model data is unavailable or retrieval fails. This improves reliability and debuggability of the model fetching process.

* Fix cached model data handling by parsing JSON response

Previously, cached model data was returned as a raw string, causing potential issues when using the data. The change ensures the cached data is properly parsed into JSON format before returning, improving reliability and consistency.

* Adjust modelManagerSP rate and Ratekeeper frequency

Reduced the rate for modelManagerSP in services and aligned the Ratekeeper frequency in model_manager.py to 0.1.

* Update model fetcher URL and adjust modelManagerSP rate

Updated the model fetcher URL to point to the correct resource for driving models. Adjusted the rate of modelManagerSP in both its service definition and the corresponding Ratekeeper initialization to 1 Hz for improved consistency.

* Refactor model download logic for clarity and efficiency

Simplify the logic for finding the model to download by combining redundant constructs into a single line. This improves code readability and reduces unnecessary variable assignments.

* Fix cache keys for manual prebuilt actions because they were missing the cache when manually built

* no need to log

* formatting

* revert ci changes

* Refactor and restructure `modeld` to `models` module.

Renamed `modeld` directory to `models` for clarity and consistency. Updated all references and imports to reflect the new structure. This improves maintainability and aligns with naming conventions.

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-04 15:05:21 -05:00
Adeeb Shihadeh 9cf02ca8db Skip registration on newer devices (#34316) 2024-12-26 20:22:44 -08:00
DevTekVE 8f7bbe4ee3 macos: use /tmp instead of /dev/shm (#34097)
* No /dev/shm on MacOS so we go to tmp instead given that the /dev/shm makes tests not really work fine on mac

* Cleanups

Fix cereal messaging tests

* keep msgq stock

More shm updates for macos

* typo

* fix return

* Semicolon...
2024-12-04 10:51:09 -08:00
Adeeb Shihadeh 484aaf4307 no PC anymore
old-commit-hash: 4e2624783f
2024-08-27 12:05:05 -07:00
Adeeb Shihadeh 854d03a819 manager: remove LastUpdateTime default
old-commit-hash: 39c64d44a1
2024-08-27 12:03:55 -07:00
Mauricio Alvarez Leon 008008b03a Update to Python 3.12 (#32548)
* 3.12

* pprofile is broken

* use modified metadrivepy3-12

* 0.3.0 metadrive

* add metadrive/commaai git dependency

* metadrive git set

* pin sounddevice 0.4.6

* datetime.utcnow() deprecation

* poetry lock

* make datetime not aware

* poetry lock

* pin pytools

* google_crc32c wheel

* unpin sounddevice

* clean metadrive

* use python crc

* mypy

* 3.12.4

* allow python3.11

* test

* no pip

* poetry

* better

* better

* merge

* remove

* try

* test

* try  this

* snok

* python

* simpler

* setuptools

* lower

* try

* try

* work?

* ubuntu deps

* ubuntu

* try

* remove

* move

* remove this

* names

* Update .github/workflows/tools_tests.yaml

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* python<4

* <3.13

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
old-commit-hash: 148eaf8fa6
2024-06-11 20:45:15 +00:00
Adeeb Shihadeh 2c6829ed67 boardd is pandad (#32628)
* boardd is pandad

* rename tests
old-commit-hash: 34e329649d
2024-06-04 19:16:55 -07:00
Adeeb Shihadeh cc4f53e6c6 manager: move to system/ (#32538)
* manager: move to system/

* one more
old-commit-hash: accdade4cf
2024-05-25 12:41:17 -07:00