* init
* desc
* llk welcome back
* more
* new param to write
* update mapd
* no migration
* no refactor for now
* exec
* rename
* bearing
* fix test
* lint
* Add modelDataV2SP and lane turn logic implementation
Note: still need to hook up to other modeld's create unit test, fix stuff, and do the UI for it
* add unit tests for lane turn logic
* Add lane turn desire controls to models panel
* use `events_sp` instead of `events`
* integrate modelDataV2SP messaging to the other modeld controllers
* move this to that
* use min for general population here, on custom branches, change this to max :)
* Update events.py
Co-authored-by: royjr <royjr96@gmail.com>
* Update events.py
Co-authored-by: royjr <royjr96@gmail.com>
* refactor lane turn value control into one method
* Update selfdrive/ui/sunnypilot/qt/offroad/settings/models_panel.cc
* add integration tests for lane turn desire
* 10 updates is possibly more representative of real life
* real objects ofc
* desc: add toggle description for clarity
---------
Co-authored-by: royjr <royjr96@gmail.com>
* record feedback with LKAS button
* fix alert test
* slightly simplify feedbackd
* "Audio Feedback Saved" upon time expiration or early stop
* earlySend --> earlyStop
* userFlag --> userBookmark
* RecordAudioFeedback param/toggle
* add audioFeedback test
* simplify feedbackd
* send bookmark regardless of toggle, show feedback event with higher priority
* add userBookmark to selfdrived sm
* fix mispelled param name
* default off and move to main
* segmentNum --> blockNum, earlyStop --> lastBlock
* preserve audioFeedback
* get rid of lastBlock and just send bookmark saved at the end
* update raylib side
* update toggle description and add raylib toggle
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
* excessive actuation
* text
* software
* check
* logic
* whoops
* dont want to lose alert unless user interacts with it
* implement
* try x2
* counter
* try to false trigger
* use livepose
* need to check for livePose noise
* cmt
* nl
* add back
* organization
* setVisible and isVisible consecutively don't work
* style
* cant do this sadly
* actually we can!
* clean up
* clean up
* clean up
* need to match torqued, paramsd, lagd, etc. (fix op sim)
* store/send mic audio with toggle
* script to extract audio from logs
* change description and add translation placeholders
* microphone icon
* apply toggle in loggerd
* add legnth and counter
* startFrameIdx counter
* Revert "change description and add translation placeholders"
This reverts commit 7baa1f6de99c6ebe9f9906193da7e83dad79511a.
* send mic data first and then calc
* restore changed description/icon after revert
* adjust fft samples to keep old time window
* remove extract_audio.py since audio is now stored in qcam isntead of rlog
* qt microphone recording icon
* Revert "remove extract_audio.py since audio is now stored in qcam isntead of rlog"
This reverts commit 7a3a75bd8db5376d1e442a3ba931c67550b5f132.
* move extract_audio script and output file by default
* remove length field
* recording indicator swaps sides based on lhd/rhd
* use record icon from comma body
* Update toggle description
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
* update raylib toggle desc cause I did earlier
* microphone --> soundPressure, audioData --> rawAudioData
* cleanup unused var
* update README
* sidebar mic indicator instead of annotated camera
* improve logic readability
* remove startFrameIdx and sequenceNum
* use Q_PROPERTY/setProperty so that update() is actually called on value change
* specify old id for SoundPressure
* fix typo
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
* Add calPerc field and torque progress
* Fix torqued test style and CarParams usage
* test: remove unused numpy import
* move here
* trying all combinations to see what's most linear
* clean up with best method
* no no
* epic
* clean up
* last min not needed
* doesn't hurt
* list comp
* init
* some fixes
* move
* more
* old navd helpers
* bring back cereal
* fix linting
* more
* add to cereal first
* sp events
* lint
* implement in long plan
* fixme-sp
* refactor state machine
* wrong state
* start refactor controller
* some type hints
* init these
* enable debug print
* ui? ui!
* print them out
* fix spinner import
* fix path
* let's use gps chips directly for now
* service missing
* publish events
* no nav for now
* need to sub
* no car state speed yet
* missed event
* Car: `CarStateSP`
* fix tests
* bring back car state speed limit
* fix
* use old controller for now
* fix
* fix source
* type hints
* none for now
* formatting
* more
* create directory if does not exist
* mypy my bt
* policy param catch exceptions
* handle all params with exceptions
* more
* single method
* define types in init
* rename
* simpler op enabled check
* more mypy stuff
* rename
* no need for brake pressed
* don't reset if gas pressed
* type hint all
* type hint all
* back to upstream
* in another pr
* no longer need data type
* qlog
* slc in another pr
* use horizontal accuracy
* set core affinity for all realtime processes
* unused
* sort
* unused
* type hint and slight cleanup
* from old implementation
* use directly
* combine pm
* slight more cleanup
* type hints
* even more type hint
* lint
* more cleanup
* even less
* license
* feat: Show alert on user flag event
* feat: Temporarily update border status on user flag to match connect timeline
* Revert "feat: Temporarily update border status on user flag to match connect timeline"
This reverts commit f1da6a4f5f555d4db50b47fb68ce74ae7737b8af.
* feat: Increase alert duration to 1.5 seconds
* remove audible alert for bookmark event
* refactor: Use NormalPermamentAlert for user flag alert
* fix: Update userFlag enum value in OnroadEvent struct
* fix: Handle userFlag event even in dashcam mode
* don't need to ignore that anymore
* remove 'userFlag' from ignore list
---------
Co-authored-by: Jason Young <jyoung8607@gmail.com>
* Add struct and services
* Proof of concept
* Use vego
* Process replay support
* Fix issues
* Max not min
* Remove print
* Make lag positive
* Points and updates
* Sliding cross corr
* Stuff
* Remove redundant param
* Move to lagd
* Update alert
* Something that works
* Report isEstimated
* Corr field
* Use skimage masked corr
* Fixes
* Move to lat_accel_loc
* Use the delay it in the model
* Mark as executable
* Add 0.2 to initial
* add install line
* Back to 5m window
* Move the import up
* corr window 120
* Show is estimated
* Sort messages
* New impl
* Fix
* Params
* Set initial_lag elsewhere
* Add param
* Rename Param
* Fix type isues
* More blocks
* Masked fft NCC implementation
* Remove package installation
* Use 20hz data and interp the correlation peak
* Move ncc code to separate function
* steering_saturated support
* Constants
* Add recovery buffer
* Fix static
* min_valid_block_count nad liveDelay
* lateralDelayEstimate
* Fix comm issues. Do estimations at 4hz
* Fix
* Display the estimate
* Increase the block size
* Cache every minute
* DEBUG flag
* Add progress to the alert
* Fix crash
* points only in debug
* Fix BlockAverage restoration
* Comment
* Move estimators into estimator directory
* Remove lagd
* Feed only relevant messages
* Rewrite as a set literal
* np.diag with P_initial
* Fix static
* Struct renames
* Update ref commit
* Make it nicer
* Some renames
* Refactor OnroadEventSP structure to contain list of events
A restructuring of the OnroadEventSP structure has been undertaken to accommodate a list of 'Event' substructures. The change is reflected in different files where OnroadEventSP is used. This update allows for more efficient management of multiple events by grouping them together under the revised OnroadEventSP structure.
* Rename `OnroadEventSP` to `OnroadEventsSP` across codebase.
Updated all references to `OnroadEventSP` to ensure consistency with the renamed struct `OnroadEventsSP`. This change improves code clarity and aligns naming conventions across modules.
* Add optional debug logging to schema validation script
Introduced a `DEBUG` flag and a `print_debug` function to streamline debug output management. This replaces direct `print` calls with conditional logging to control verbosity during execution.
Refactor structural validation logic in cereal test
Simplify the iteration over read_instances to streamline structural validation. Removed redundant comparisons and improved error handling to detect unreadable fields more effectively. Updated error messages for better clarity during debugging.
Update build command to include 'cereal' target in CI
Modified the scons build command in selfdrive_tests workflow to explicitly build the 'cereal' target. This ensures necessary components are included during the CI process, improving reliability and consistency.
Added workflow for cereal validation artifacts generation and validation against upstream
This commit encompasses significant changes to .github/workflows/selfdrive_tests.yaml, including the addition of two new jobs. One is responsible for 'Generating cereal validation artifacts' and the other for 'Validating cereal with Upstream'. This includes generating cereal schemas, building openpilot, and running validation schema instances against master. Furthermore, a new Python script (validate_sp_cereal_upstream.py) was also added to perform cereal schema instance generation and validation. These changes aim to enhance the testing process, ensuring schema compatibility and integration quality.
* Relocate cereal validation to a dedicated GitHub workflow
This commit introduces a distinct GitHub workflow for cereal validation named 'cereal_validation.yaml'. This workflow includes two jobs: one for generating cereal validation artifacts and another for validating cereal with the upstream project. Previously, these operations were included as separate jobs in 'selfdrive_tests.yaml'. However, the decoupling in this commit allows for a better organization of GitHub workflows within the project. Additionally, this separation allows these workflows to be individually configured and run, providing a greater degree of flexibility in managing our continuous integration activities.
* Rename workflow to "cereal validation" for clarity.
Updated the workflow name in the GitHub Actions configuration to better reflect its purpose. This change improves maintainability and ensures clearer identification of the workflow's function.
* Add LFS configuration and GitLab SSH setup to workflow
Integrate GitLab LFS handling by configuring LFS URLs and enabling SSH setup. This includes adding public GitLab keys and updating the workflow to support secure connections for LFS operations. Ensures proper handling of large files and seamless integration with GitLab.
* rename
* format
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* Add AES encryption and utility methods
This commit introduces a new AESCipher class that can be used for AES encryption with support for both AES-128 and AES-256. It also adds a set of utility functions, including methods for RSA to AES key derivation, file decryption and compression, and encryption and decompression. These changes provide fundamental cryptographic functionalities for data security within the system.
* Revised backup management system with new structures
This update introduces important revisions to the backup management system. A new struct named 'BackupManagerSP' has been integrated into the 'custom.capnp' file, replacing 'CustomReserved6'. This new struct includes several variables that help keep track of backup and restore status, progress and related messages.
Additional code modifications were made in 'base.py' and 'api.py' to include a 'json' parameter in the 'api_get' method. Lastly, 'manager.py' has been updated with functions to manage device configuration backups to and from sunnylink.
The changes promote better handling and management of data backup and restoration processes. The update is specifically designed to ensure that the backup information is well integrated into the system, with properly tracked status and progress.
* Refactor `allKeys` to filter by parameter key type
Updated the `allKeys` method to support filtering by `ParamKeyType`, allowing more specific key retrieval. Added a default value for backward compatibility and updated related bindings and keys to reflect this change.
* Improve Backup and Restore mechanisms
The commit refactors and improves several aspects of the backup and restore mechanisms in the `BackupManagerSP` class.
These improvements include removing redundant status tracking variables and replacing them with unified ones, updating the messaging system to handle all changes correctly, and including an enumeration `OperationType` to keep track of the type of operation currently being processed.
This commit also applies stricter conditions for restore operations, such that it only restores parameters that are currently marked as backupable, and skips those that are not. This is a preventive measure against potential issues when restoring parameters that are no longer relevant or could conflict with current versions.
Also, the encryption and decryption methods were updated to use AES-256 for more security. These changes have increased the robustness and reliability of the backup and restoration processes.
* copyright
* Add backup_manager process to offroad sunnypilot tasks
The backup_manager process is introduced to handle backups during offroad mode when SunnyLink is ready. This ensures proper backup management functionality integrated into the system.
* Simplify backup endpoint construction in restore method
Replaced conditional expression with a more concise and readable `or` operation for constructing the backup endpoint. This makes the code cleaner and easier to maintain while preserving functionality.
* Added support for backing up and restoring sunnypilot settings
An update to the sunnypilot functionality now provides two new features that allow users to backup and restore their sunnypilot settings. The changes include the addition of UI controls for initiating backup and restore operations, and the creation of a system-wide state management function for tracking these operations. This enhancement significantly improves the user experience by providing a safety net for user settings in case of software failures, bugs, or unintended changes.
* Refactor type hints to use PEP 604 syntax for clarity
Replaced `Optional` and `Dict` type hints with `|` and `dict` syntax for improved readability and compliance with Python 3.10+. Updated related imports and adjusted list comprehension for cleaner code.
* Update import path for hardware module in utils.py
Replaced the import path for the `Paths` module to align with the new directory structure under `openpilot`. This ensures compatibility with recent project reorganization and avoids import errors.
* Improve RSA key handling and fix backup status comparison
Added explicit RSA key type checks to handle invalid key formats. Enhanced type safety in `manager.py` by ensuring the backup status comparison returns a boolean. These changes improve robustness and error handling in backups.
* format
* more
* Improve backup and restore flow with progress tracking and fixes
Added proper progress tracking and cleanup logic during restore operations. Enhanced restore experience by resetting progress after completion and introducing confirmation dialogs for errors and completion. Updated API compatibility with a version query parameter for backups.
* Enable backup button only when restore process is complete
Previously, the backup button could be enabled during a restore operation, which might cause unintended behavior. This update introduces a check to ensure the backup button remains disabled while a restore process is active. This improves user experience and prevents potential conflicts.
* Fix restore button state handling during restore process
Ensure the restore button is disabled consistently when a restore is in progress. This prevents user interaction issues and aligns the button state with the restore operation status.
* "Refactor restore logic and improve button state handling"
Replaced `is_restoring` with `restore_request_pending` for clarity and better state management. Adjusted button behavior to immediately disable upon user action, ensuring improved UX and preventing repeat inputs. Refined restore completion flow for better consistency and reliability.
* Refine restore process logic for SunnyLink settings.
Introduced `restore_request_started` to improve handling of restore states and ensure accurate UI updates during the process. Adjusted case handling to enhance clarity and maintain proper behavior when restoring settings, especially during ongoing or completed requests.
* revert
* move around
* fix enabled states for different statuses
* add prompt to notify backup is complete
* same states as restore
* disable buttons if sunnylink is off
* can use the same texts
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* temp unavailable
* fix
* replay multiple
* clean up
* bump
* rename
* reason is in liveTracks
* more rename
* bump
* fix
* bump
* fix
* need to totally ignore it, or it will throw commIssues for radarErrors or not alive
* this is just simpler -- good thing kept struct
* rm
* bool fields
* liveTracks.valid is already set using errors
* fix
* clean up
* same but diff
* fix
* fix
* fix test
* clean up
* rm radarErrors
* whoops
* no struct
* Revert "no struct"
This reverts commit ed8ec02493673f850800344c73c969508fc38991.
* bump
* update
* carControlSP and move MADS to outside structs
* publish it
* apply to all car controller
* migrate sunnypilotParams
* migrate madsEnabled
* tldr
* convert to capnp
* unused
* wrong module
* fix name
* cancer is right (all tests should be passing now)
* bump opendbc
* sp flags
* pass CP_SP to card and car interfaces
* CP_SP in radar interface
* bump opendbc
* use dataclass like old times
* bump opendbc
* write to params for controls
* fix test models
* fix
* need to use copy instead
* fix data type
* add service
* more
* fix
* Revert "fix"
This reverts commit 74723d7fb2.
* Revert "fix data type"
This reverts commit 02355f44df.
* missed
* more
* no more lagging
* Reapply "fix data type"
This reverts commit dbf1b8583f.
* Reapply "fix"
This reverts commit 9cbce9968a.
* Revert "Reapply "fix""
This reverts commit 1871919b63.
* Revert "Reapply "fix data type""
This reverts commit 5e95752fd5.
* no longer
* Revert "no longer"
This reverts commit 66ee1ba151.
* Reapply "Reapply "fix data type""
This reverts commit 670a384333.
* Reapply "Reapply "fix""
This reverts commit 42f09f955c.
* only for car params sp
* rename
* fix more test
* no need for process replay
* pass stock car params to sp set car params
* pass stock car params to sp set car params
* deprecate CarParams.sunnypilotFlags to CarParamsSP.flags
* missed arg
* fix tests
* tests fixed
* need to pass this too
* must generate cp_sp!
* fix typing
* must be initialized prior can comm callback!
* no more cancer (@devtekve)
* remove more cancer
* Refactor `get_non_essential_params_sp` to simplify arguments (#612)
* Refactor 'get_non_essential_params_sp' function calls in tests
In both `test_latcontrol.py` and `process_replay.py`, simplified the function calls to 'get_non_essential_params_sp'. Removed an unnecessary call to 'get_non_essential_params'. This change makes the code cleaner and more efficient by reducing redundancy in the function calls. This modification also ensures consistency across different code files.
* Refactor get_non_essential_params_sp to take car_params.
Simplify parameters by modifying `get_non_essential_params_sp` to use `car_params` as input. Adjust related calls in test files and process replay to match the updated method signature. This improves code clarity and reduces redundancy.
* bump opendbc
* Refactor parameter handling for `get_params_sp`.
Removed unnecessary reassignment of `car_params` in calls to `get_params_sp`, ensuring a cleaner and more streamlined code structure. This change improves code clarity and eliminates redundant operations. All relevant assertions and behavior remain unaffected.
* bumping opedbc
* bump opendbc
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* Events: Migrate sunnypilot onroad events to its own cereal
* more
* slightly more
* typing
* fix more
* fix mads state machine tests
* readjust order
* fix event
* abstract
* need these
* move around
* let's make sure it cleared on every loop
* Update selfdrive/selfdrived/alertmanager.py
Co-authored-by: DevTekVE <devtekve@gmail.com>
* use upstream custom struct
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* Events: Migrate sunnypilot onroad events to its own cereal
* more
* slightly more
* typing
* fix more
* fix mads state machine tests
* readjust order
* fix event
* abstract
* need these
* move around
* let's make sure it cleared on every loop
* Update selfdrive/selfdrived/alertmanager.py
Co-authored-by: DevTekVE <devtekve@gmail.com>
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* Longitudinal: Distance button hold to toggle Chill/Experimental Mode
* unused
* fix
* no need
* Refactor: Introduce ButtonHoldTracker to manage button hold durations (#593)
Add ButtonHoldTracker for button hold logic and tests
Introduce a new `ButtonHoldTracker` class to manage button hold durations, replacing manual timer handling in `ExperimentalSwitcher`. Updated `ExperimentalSwitcher` to leverage this implementation for cleaner and more modular code. Added comprehensive unit tests for both `ButtonHoldTracker` and `ExperimentalSwitcher` to ensure functionality and edge case coverage.
* Revert "Refactor: Introduce ButtonHoldTracker to manage button hold durations (#593)"
This reverts commit 7ee7e73ce7.
* less in selfdrived
* pass carparams into child
* tests for cruisehelper
* rename these bad bois (happy now @devtekve? xD)
* Apply suggestions from code review
Co-authored-by: DevTekVE <devtekve@gmail.com>
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* init dec
* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* fix static test
* ff
* fix static test
* unitee testt
* Refactor test_dynamic_controller and fix formatting issues
Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency.
* Refactor test indentation for dynamic controller tests
Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code.
* Migrated to pytest using claude
* Integrate radar parameter into dynamic controller's pytest tests
Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases.
* Disabling unittest file to allow checks on the pipeline to succeed.
Pending to remove this, but leaving it to validate the move to pytest is okay before merging
* Replace unittest with pytest for dynamic controller tests
Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage.
* new line...
* Refactor and modularize DynamicExperimentalController logic
Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports.
* Add missing import for messaging in helpers.py
The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code.
* Format
* Formatting
* rebase fix
* Refactor MpcSource definition and update references.
Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality.
* Format
* Refactor DEC into a dedicated longitudinal planner class
Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity.
* **Refactor DEC module structure for better organization**
Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes.
* static test
* static
* had moved to car_state
* cleanup
* some more
* static method
* move around
* more cleanup
* stuff
* into their own
* rename
* check live param
* sync with stock
* type hint
* unused
* smoother trans
* window time
* fix type hint
* pass sm.frame from plannerd
* more fixes
* more
* more explicit
* fix test
* Revert "fix test"
This reverts commit 635b15f2bc.
* Revert "pass sm.frame from plannerd"
This reverts commit a8deaa69b8.
* use internal frame
* update name
* never used
* this is why it was never using DEC
* more logs
* slight cleanup
* remove to fail test
* update name
* more
* rename
* move around
* explicit type hints
* move to constants py
* Revert "explicit type hints"
This reverts commit c205497b
* more
* don't set to exp mode initial if DEC is active
* use walrus for None
* Revert "use walrus for None"
This reverts commit 5f2396d490.
* fix wrong typing and variable name
* use walrus (needs cleanup)
* fix tests
* revert smooht lead for now
* dec: how good is FirstOrderFilter?
* Update dec.py
* dec: faster ?
* Revert "dec: faster ?"
This reverts commit 40259cd22a.
* Revert "Update dec.py"
This reverts commit 3f29ccbd99.
* Revert "dec: how good is FirstOrderFilter?"
This reverts commit 01e06df542.
* Update slow-down logic and constants for improved behavior
Adjust the slowdown scaling factor and anomaly handling to refine behavior without abrupt resets. Modify constants to increase window size and adjust probabilities and distances for smoother adaptation. Update version to reflect the new changes.
* Fix lead detection fallback for weighted average check.
Add a fallback value of -1 when computing the weighted average to prevent errors caused by invalid or None values. This ensures robust lead detection and avoids potential crashes or undefined behavior.
* visuals for DEC
* try this
* add opacity
* should be active and dimmer
* even dimmer
* Update dec.py
* Update constants.py
* use another method for drawing
* migrate to sp only
* fix
* init
---------
Co-authored-by: rav4kumar <meetkumardesai@gmail.com>
Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: DevTekVE <devtekve@gmail.com>
Jason Wen