Compare commits

...

233 Commits

Author SHA1 Message Date
Jason Wen
e78b361444 wrong btn 2025-04-15 01:50:45 -04:00
Jason Wen
74540873bf bye 2025-04-15 01:38:50 -04:00
Jason Wen
f3ef8ff6ec try this 2025-04-15 01:28:30 -04:00
Jason Wen
79a4f25f77 even more 2025-04-15 01:25:08 -04:00
Jason Wen
efef671a3f gotta add it 2025-04-15 01:23:05 -04:00
Jason Wen
ded6049301 ty lint 2025-04-15 01:21:06 -04:00
Jason Wen
51f5193ec5 events 2025-04-15 01:14:20 -04:00
Jason Wen
5db7dbee6b UI this up! 2025-04-15 01:14:16 -04:00
Jason Wen
aab08b8527 init 2025-04-15 00:56:44 -04:00
Jason Wen
4c8ed80304 Update Python packages (#819) 2025-04-15 00:04:57 -04:00
Jason Wen
3615b1a203 DM: Enforce monitoring of MADS state in driver monitoring (#818)
* DM: Enforce monitoring of MADS state in driver monitoring

Added `selfdriveStateSP.mads.enabled` to track MADS engagement. This ensures monitoring logic accounts for MADS alongside the existing selfdrive state checks, improving feature integration and event handling.

* how
2025-04-14 14:50:21 -04:00
DevTekVE
b41ace34cc sunnylink: fix sunnylink backup restore version parsing (#816)
* improvement

* fix: Improve version parsing logic for sunnypilot

---------

Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-04-14 14:18:05 -04:00
DevTekVE
b9d584245f ci: update squash script for external contributors (#811)
* Get head ref info

* Simpify adding the comment with the newest flagg by gh cli

* Dynamic origin

* Enhance PR processing: add label handling for forked PRs and improve variable naming

* Refactor PR label handling: use constant for trust-fork label and improve comments

* Fix remote addition in PR processing: change subprocess call to not check for errors

* Refactor PR comment handling: support multiple comments and improve clarity

* Refactor PR processing: streamline comment handling and improve error management

* Refactor add_pr_comments function: specify comments type as list of strings

* Maybe we don't prevent "behind" branches to be merged as lon as the PR itself is mergeable

* lint, leave me alone

---------

Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>
2025-04-14 19:29:33 +02:00
DevTekVE
6bd47c4eec Models: point to v2 of driving models json for recompiled models (#817)
Use newer model version json since they have been recompiled
2025-04-13 21:39:26 +02:00
Jason Wen
43eefed514 Hyundai longitudinal: Parse lead info for camera-based SCC platforms (#809)
* Hyundai longitudinal: Parse lead info for camera-based SCC platforms

* fix

* update

* bump

* update tests

* lol why is this here

* bump

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>
2025-04-12 12:22:42 -04:00
Nayan
f1d703e6e4 Device: Customizable Max Time Offroad (#796)
* Max Time Offroad

* Refactor & Fix param

* Error Handling

* rename SP variable

* Update selfdrive/ui/sunnypilot/qt/widgets/controls.h

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/widgets/controls.h

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/widgets/controls.h

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/widgets/controls.h

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/widgets/controls.cc

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/max_time_offroad.h

Co-authored-by: DevTekVE <devtekve@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/max_time_offroad.cc

Co-authored-by: DevTekVE <devtekve@gmail.com>

* UI layout changes for better alignment

selector is not big enough -> make it bigger ;)

OptionControlSP now includes a QMap argument to allow actual values to be set in param directly

* Rebase & resolve reviews

* change default to be closer to OP default

* me dumb

* MaxTimeOffroad: Add support for 30h limit and improve label formatting

* power_monitoring: Refactor MaxTimeOffroad parameter handling for clarity

* test: Add unit tests for MaxTimeOffroad parameter handling

* power_monitoring: Update MaxTimeOffroad handling to use seconds and improve shutdown logic

* power_monitoring: Improve exception handling and remove redundant shutdown check for MaxTimeOffroad

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-04-12 16:33:17 +02:00
Jason Wen
a598d385f2 ci: Remove redundant Panda build step from prebuilt workflow (#810)
The Panda build step was unnecessary as it is not utilized in this workflow. This change simplifies the workflow and reduces redundant actions, improving efficiency.
2025-04-12 11:47:00 +02:00
Nayan
92707e8912 UI: Device & Sunnylink Panels - Standardize push button size & alignment (#806)
* layout adjustments

* sunnylink_panel

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-04-11 22:21:49 +02:00
Jason Wen
fb87ba681a MADS: transition to soft-disable for specific gear states (#791)
* MADS: transition to soft-disable for specific gear states

This commit enhances the condition checking functionality of the Modular Assistive Driving System (MADS) to implement a 'soft-disable' feature during the vehicle's active motion when a non-forward drive gear is engaged. It includes utilizing structs to reference various car state attributes and modifying a function definition to improve clarity. This adjustment boosts the system's reaction to gear shifts, increasing the safety and efficiency of the driving assist system.

* structs
2025-04-11 16:06:45 -04:00
Jason Wen
6a00ac9cd0 Radard: vision-based yRel for Hyundai single-lead systems (#805)
Introduced processing for custom yRel values using HyundaiFlagsSP when the enhanced SCC flag is enabled. Updated `radard` to handle `CarParamsSP` and make necessary adjustments for Hyundai vehicles with specific SCC configurations.
2025-04-11 14:28:59 -04:00
Nayan
0a2fd7bd61 UI: Update AbstractControlSP_SELECTOR and OptionControlSP (#800)
* controls

* Adjust label width dynamically based on layout type.

Updated the label's fixed width to be conditional on the layout type, improving adaptability for different inline layouts. Additionally, corrected indentation in the width calculation loop for consistency.

* Refactor OptionControlSP to improve parameter value handling and encapsulate logic in dedicated methods

* Refactor getParamValue to return an integer and ensure value is updated correctly in button click handler

* Trying to unify a bit the logic. still WIP

* Reducing a bit the change footprint

* Refactor spacing item handling to prevent duplicate insertion and improve layout management

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-04-10 15:20:41 -04:00
Jason Wen
1490a24378 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#798) 2025-04-07 21:40:48 -04:00
DevTekVE
a2761ba731 Merge remote-tracking branch 'comma/master' into sync-20250407
# Conflicts:
#	opendbc_repo
#	panda
2025-04-07 21:11:47 -04:00
DevTekVE
d7d4523e49 Readme: update with links to new branches and info about rewrite and reflash agnos (#795)
* Adding the new branches info and adding a warning to reflash agnos

* Using more blockquote

* it

---------

Co-authored-by: Stupefacient <brianbrownt@gmail.com>
2025-04-06 12:42:15 +02:00
Jason Wen
a08a29d5e6 MADS: prevent wrongCarMode from disabling MADS enabled state (#792)
* MADS: prevent `wrongCarMode` from disabling MADS enabled state

Add wrongCarModeNoEntry as a NO_ENTRY event to replace wrongCarMode (USER_DISABLE)
that would incorrectly disable MADS when in the wrong car mode. Provides
contextual guidance to users based on car brand, showing appropriate
instructions for re-engagement without requiring MADS to be re-enabled.

* move down

* warning only?

* cereal SIKE

* remove duplicate

* try this out

* final
2025-04-06 04:29:13 +00:00
Jason Wen
976655b599 CI: Cache macOS Homebrew and Scons (#788)
* CI: Cache macOS builds

* try caching now

* close

* revert

* save homebrew cache

* final
2025-04-05 00:11:05 -04:00
Jason Wen
2105344725 Revert "MADS: Soft disable for non-Drive forward gear if in motion" (#790)
Revert "MADS: Soft disable for non-Drive forward gear if in motion (#784)"

This reverts commit b934aa37d4.
2025-04-04 22:59:23 -04:00
Jason Wen
8dec4ea5d7 Revert "MADS: Steering Mode on Brake Pedal Press (#687)" (#789)
This reverts commit bf2731b4
2025-04-04 22:43:00 -04:00
Jason Wen
658444ca77 CI: trigger UI report cache 2025-04-04 15:42:50 -04:00
Jason Wen
b934aa37d4 MADS: Soft disable for non-Drive forward gear if in motion (#784)
* MADS: Soft disable for non-Drive forward gear if moving

* test

* revert

* do this first

* Revert "do this first"

This reverts commit 0a4c938b7e.
2025-04-04 15:39:36 -04:00
Stupefacient
84b934033c README for sunnypilot (#577)
* be gone thot

* hhhmm what is this

a little dusty but lets see if we can freshen it up

* some of these

* bye bye

* remove xml bs

* add doc link

* i cant type

* remove patron and old subnote

* remove broken link

* update licensing

* update

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-04-03 23:19:34 -04:00
Jason Wen
87ff98dc7c ui: update names (#678)
* ui: update names

* Revert "ui: update names"

This reverts commit 313510c8c0.

* gone

* init

* Changed all occurrences of 'openpilot' to 'sunnypilot'

All instances of 'openpilot' in the user interface, documentation, and prompts have been updated to 'sunnypilot'. These changes include source strings, warning messages, and the prompts shown to users. The adjustment is part of the rebranding process and aims to avoid any uncertainty or confusion for users.

* vanish translations

* translations

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-04-03 23:11:27 -04:00
Jason Wen
6c8b2290fa Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#783) 2025-04-03 22:12:41 -04:00
Jason Wen
9dba62e686 Merge branch 'upstream/openpilot/master' into sync-20250403
# Conflicts:
#	.github/workflows/auto_pr_review.yaml
#	.github/workflows/selfdrive_tests.yaml
#	.github/workflows/ui_preview.yaml
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/controls/controlsd.py
2025-04-03 21:53:04 -04:00
Discountchubbs
63ba5e864f car: abstract sunnypilot interfaces (#721)
* Method abstracting radar tracks in interfaces.py This makes the code more maintainable for future implementations to this file

* Method abstracting radar tracks in interfaces.py This makes the code more maintainable for future implementations to this file

* Add type annotation to params in initialize_car_interface_sp

* rename

* just internal

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-30 02:45:15 +00:00
DevTekVE
6b3f75bbf0 CI: refactor Squash and Merge with simplified branch merging (#726)
* Refactor squash and merge script for improved simplicity

Simplified the squash_and_merge.py script by replacing redundant utility functions and consolidating logic. Enhanced usability by aligning command-line arguments and leveraging streamlined git operations to improve maintainability and reliability.

* Fix argument names in squash PR script

Renamed CLI arguments from '--base' and '--source' to '--target' and '--base' to align with expected input format. This ensures the script runs correctly with proper argument mapping.

* Fix incorrect base branch argument in squash script

    Updated the `--base` argument to use `source_branch` instead of `branch` to ensure the squash script processes the correct base branch. Also adjusted the command to include `branch` as a separate argument for clarity and correctness.

* Reset to a clean state after squash error.

Add a `git reset --hard` command to ensure the repository returns to a clean state after encountering errors during the squash and merge process. This prevents lingering changes from affecting subsequent operations.

* Improve error handling in squash_and_merge_prs.py

Capture and display both stdout and stderr in error cases to provide more informative feedback. Adjust the PR comment to include available output for better debugging.

* Refactor PR squash process to enhance error handling.

Modify subprocess handling to use `result.returncode` for error checks instead of relying on exceptions. Consolidate error output retrieval and logging for better clarity, while maintaining the workflow for resetting changes on failure.

* Fix incorrect return in PR processing loop

Replaced `return` with `continue` to ensure all PRs in the loop are processed before exiting. This prevents premature termination of the function and ensures accurate success count reporting.

* Simplify subprocess output handling in squash_and_merge.py

Replaced labeled print statements with direct output of stdout and stderr. This change ensures cleaner logs and remains consistent with the function's purpose of output handling during subprocess execution.

* Update subprocess.run calls to use capture_output parameter

Replaced `stdout` and `stderr` with the `capture_output` parameter for cleaner and more concise subprocess handling. Also removed extraneous whitespace for improved code readability.

* testing moving the squash script given that it's called iteratively and switching branch might miss it

* format

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-29 21:54:49 +00:00
DevTekVE
4268d7a19c Events: Refactor OnroadEventSP structure and add upstream cereal validation (#722)
* Refactor OnroadEventSP structure to contain list of events

A restructuring of the OnroadEventSP structure has been undertaken to accommodate a list of 'Event' substructures. The change is reflected in different files where OnroadEventSP is used. This update allows for more efficient management of multiple events by grouping them together under the revised OnroadEventSP structure.

* Rename `OnroadEventSP` to `OnroadEventsSP` across codebase.

Updated all references to `OnroadEventSP` to ensure consistency with the renamed struct `OnroadEventsSP`. This change improves code clarity and aligns naming conventions across modules.

* Add optional debug logging to schema validation script

Introduced a `DEBUG` flag and a `print_debug` function to streamline debug output management. This replaces direct `print` calls with conditional logging to control verbosity during execution.

Refactor structural validation logic in cereal test

Simplify the iteration over read_instances to streamline structural validation. Removed redundant comparisons and improved error handling to detect unreadable fields more effectively. Updated error messages for better clarity during debugging.

Update build command to include 'cereal' target in CI

Modified the scons build command in selfdrive_tests workflow to explicitly build the 'cereal' target. This ensures necessary components are included during the CI process, improving reliability and consistency.

Added workflow for cereal validation artifacts generation and validation against upstream

This commit encompasses significant changes to .github/workflows/selfdrive_tests.yaml, including the addition of two new jobs. One is responsible for 'Generating cereal validation artifacts' and the other for 'Validating cereal with Upstream'. This includes generating cereal schemas, building openpilot, and running validation schema instances against master. Furthermore, a new Python script (validate_sp_cereal_upstream.py) was also added to perform cereal schema instance generation and validation. These changes aim to enhance the testing process, ensuring schema compatibility and integration quality.

* Relocate cereal validation to a dedicated GitHub workflow

This commit introduces a distinct GitHub workflow for cereal validation named 'cereal_validation.yaml'. This workflow includes two jobs: one for generating cereal validation artifacts and another for validating cereal with the upstream project. Previously, these operations were included as separate jobs in 'selfdrive_tests.yaml'. However, the decoupling in this commit allows for a better organization of GitHub workflows within the project. Additionally, this separation allows these workflows to be individually configured and run, providing a greater degree of flexibility in managing our continuous integration activities.

* Rename workflow to "cereal validation" for clarity.

Updated the workflow name in the GitHub Actions configuration to better reflect its purpose. This change improves maintainability and ensures clearer identification of the workflow's function.

* Add LFS configuration and GitLab SSH setup to workflow

Integrate GitLab LFS handling by configuring LFS URLs and enabling SSH setup. This includes adding public GitLab keys and updating the workflow to support secure connections for LFS operations. Ensures proper handling of large files and seamless integration with GitLab.

* rename

* format

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-29 17:34:31 -04:00
DevTekVE
245605dc55 ci: Add LFS configuration and GitLab SSH to squash & merge script (#725)
Integrate GitLab LFS handling by configuring LFS URLs and enabling SSH setup. This includes adding public GitLab keys and updating the workflow to support secure connections for LFS operations. Ensures proper handling of large files and seamless integration with GitLab.
2025-03-29 12:42:54 +01:00
DevTekVE
e15974345b modeld: Enforce float type for lag_adjusted_curvature (#714)
Convert safe_desired_curvature to float before returning.

This ensures the returned value is explicitly a float, avoiding an issue when serializing it on capnp as it does not recognize numpy.float

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-26 08:26:21 +01:00
Jason Wen
65853f0111 NNLC: Fix exact match consistency (#716)
Move the exact_match calculation after updating model_path and max_similarity to ensure consistency. This prevents potential discrepancies when rechecking NN paths and enhances code maintainability.
2025-03-26 01:08:22 -04:00
royjr
5ce03e6ecf ui: display fingerprint as fallback in platform selector (#709)
Update platform_selector.cc
2025-03-25 08:23:21 +01:00
Jason Wen
4055efdf4b ci: add commit SHA to build notifications (#712)
This change includes the current commit SHA in the workflow outputs of the sunnypilot-build-prebuilt.yaml file. It provides better traceability for builds, ensuring each workflow run is linked to the exact commit it was triggered from.
2025-03-25 00:25:04 -04:00
royjr
da79b6d494 ui: new icon for Lateral panel (#707)
* Create icon_lateral.svg

* Update settings.cc

* Delete icon_lateral.svg

* Create icon_lateral.png

* Update settings.cc

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-25 03:05:20 +00:00
Jason Wen
45c8f5c9d5 ci: Update PR title formatting for Squash and Merge (#711)
Changed the PR reference format from (`#123`) to (PR-123) in squash commit messages
to prevent GitHub from automatically adding reference comments to PRs when
`master-dev-c3-new` is force pushed.
2025-03-24 22:56:19 -04:00
Kumar
34bbdf4d7f Controls: Automatic lane change (#653)
* init alc controller

* only for sunny

* rebase fix

* ui

* add ui preview

* Update common/params_keys.h

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/SConscript

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.h

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.cc

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* review sugg

* code review

* renaming

* move around

* style

* fix types and params exceptions handling

* take out magic numbers

* more

* rename

* shorter

* make sure reset happens at the end of every DH loop

* split into multiple updates

* just 3 seconds

* use default states

* oops

* more readable

* oops

* some space and lines

* run in DH loop directly

* adjust ui preview

* nudgeless should process immediately

* check option instead

* more explicit

* even more explicit

* tests

* brake pedal release should not allow auto lane change (caught by test)

* unnecessary

* no continuous auto lane change

* Revert "unnecessary"

This reverts commit 93d135b54a.

* more tests

* less

* less less

* update again

* more cleanup

* better

* AutoLaneChangeState -> AutoLaneChangeMode

* update

* lint

* unused

* test all states

* license

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-24 22:26:48 -04:00
Jason Wen
cc1b233277 ui: vertical space for ListWidget (#706) 2025-03-24 00:32:31 -04:00
Kumar
029a601674 Device: Quiet Mode (#654)
* init quiet mode

* only for sunny

* static

* toggle

* let's back this up

* review sugg

* oh okay

* review

* fix: ensure boolean conversion for QuietDrive parameter

* Refactor return statement to use boolean conversion for clarity in quiet mode logic

* Update selfdrive/ui/sunnypilot/quietmode.py

* rename

* sunny

* Revert "sunny"

This reverts commit 6ac4cf4b8d.

* sunny

* Revert "sunny"

This reverts commit c2bffddc05.

* sunny

* ui: support dynamic state updates for `PushButtonSP`

* test btn

* override mouse release events

* Revert "test btn"

This reverts commit cd9c9dde9a.

* Reapply "test btn"

This reverts commit 9b36b2e085.

* abstract param flipping

* Revert "Reapply "test btn""

This reverts commit 8104a262b0.

* use new button state for PushButtonSP

* Quiet Drive -> Quiet Mode

* driver camera btn moved

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-23 23:15:11 -04:00
Jason Wen
982674b4a7 ui: support dynamic state updates for PushButtonSP (#708)
* ui: support dynamic state updates for `PushButtonSP`

* test btn

* override mouse release events

* Revert "test btn"

This reverts commit cd9c9dde9a.

* Reapply "test btn"

This reverts commit 9b36b2e085.

* abstract param flipping

* Revert "Reapply "test btn""

This reverts commit 8104a262b0.

* update device panel handling for the new param
2025-03-23 22:54:40 -04:00
royjr
96d73fe2b3 NNLC: bump max similarity for higher accuracy (#704)
Update helpers.py

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-23 20:04:12 +00:00
Jason Wen
a307a08591 ui: fix offset with OptionControlSP for macOS (#705)
ui: fix offset with `OptionControlSP`
2025-03-23 15:50:21 -04:00
Jason Wen
e6177c49aa athenad: avoid infinite loop waiting for comma registration (#701)
Athenad: avoid infinite loop waiting for comma registration
2025-03-22 22:48:58 -04:00
Jason Wen
ed39aaddbc ui: use allKeys to reset sunnypilot settings (#699)
* ui: exclude certain params from deletion during settings reset

* clear all and exclude directly and only reboot after confirmation

* cleaner

* handle this instead

* send it

* 1-liner

* nuke nuke

* space

* less
2025-03-22 22:41:42 -04:00
Tim Wilson
bab2d776b6 NNLC: use torque substitutes as fuzzy fingerprints (#700)
* NNLC: use torque substitutes as fuzzy fingerprints

* substitutes are not exact matches

* update nnlc helpers

for extra nnlc model path checks, take the first one that passes.

* Update sunnypilot/selfdrive/controls/lib/nnlc/helpers.py

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-22 23:49:05 +00:00
Jason Wen
002a37cdc4 ui: show branch selector in all branches (#698) 2025-03-22 16:20:19 -04:00
DevTekVE
43e43849ad sunnylink: Settings backup & restore (#681)
* Add AES encryption and utility methods

This commit introduces a new AESCipher class that can be used for AES encryption with support for both AES-128 and AES-256. It also adds a set of utility functions, including methods for RSA to AES key derivation, file decryption and compression, and encryption and decompression. These changes provide fundamental cryptographic functionalities for data security within the system.

* Revised backup management system with new structures

This update introduces important revisions to the backup management system. A new struct named 'BackupManagerSP' has been integrated into the 'custom.capnp' file, replacing 'CustomReserved6'. This new struct includes several variables that help keep track of backup and restore status, progress and related messages.

Additional code modifications were made in 'base.py' and 'api.py' to include a 'json' parameter in the 'api_get' method. Lastly, 'manager.py' has been updated with functions to manage device configuration backups to and from sunnylink.

The changes promote better handling and management of data backup and restoration processes. The update is specifically designed to ensure that the backup information is well integrated into the system, with properly tracked status and progress.

* Refactor `allKeys` to filter by parameter key type

Updated the `allKeys` method to support filtering by `ParamKeyType`, allowing more specific key retrieval. Added a default value for backward compatibility and updated related bindings and keys to reflect this change.

* Improve Backup and Restore mechanisms

The commit refactors and improves several aspects of the backup and restore mechanisms in the `BackupManagerSP` class.

These improvements include removing redundant status tracking variables and replacing them with unified ones, updating the messaging system to handle all changes correctly, and including an enumeration `OperationType` to keep track of the type of operation currently being processed.

This commit also applies stricter conditions for restore operations, such that it only restores parameters that are currently marked as backupable, and skips those that are not. This is a preventive measure against potential issues when restoring parameters that are no longer relevant or could conflict with current versions.

Also, the encryption and decryption methods were updated to use AES-256 for more security. These changes have increased the robustness and reliability of the backup and restoration processes.

* copyright

* Add backup_manager process to offroad sunnypilot tasks

The backup_manager process is introduced to handle backups during offroad mode when SunnyLink is ready. This ensures proper backup management functionality integrated into the system.

* Simplify backup endpoint construction in restore method

Replaced conditional expression with a more concise and readable `or` operation for constructing the backup endpoint. This makes the code cleaner and easier to maintain while preserving functionality.

* Added support for backing up and restoring sunnypilot settings

An update to the sunnypilot functionality now provides two new features that allow users to backup and restore their sunnypilot settings. The changes include the addition of UI controls for initiating backup and restore operations, and the creation of a system-wide state management function for tracking these operations. This enhancement significantly improves the user experience by providing a safety net for user settings in case of software failures, bugs, or unintended changes.

* Refactor type hints to use PEP 604 syntax for clarity

Replaced `Optional` and `Dict` type hints with `|` and `dict` syntax for improved readability and compliance with Python 3.10+. Updated related imports and adjusted list comprehension for cleaner code.

* Update import path for hardware module in utils.py

Replaced the import path for the `Paths` module to align with the new directory structure under `openpilot`. This ensures compatibility with recent project reorganization and avoids import errors.

* Improve RSA key handling and fix backup status comparison

Added explicit RSA key type checks to handle invalid key formats. Enhanced type safety in `manager.py` by ensuring the backup status comparison returns a boolean. These changes improve robustness and error handling in backups.

* format

* more

* Improve backup and restore flow with progress tracking and fixes

Added proper progress tracking and cleanup logic during restore operations. Enhanced restore experience by resetting progress after completion and introducing confirmation dialogs for errors and completion. Updated API compatibility with a version query parameter for backups.

* Enable backup button only when restore process is complete

Previously, the backup button could be enabled during a restore operation, which might cause unintended behavior. This update introduces a check to ensure the backup button remains disabled while a restore process is active. This improves user experience and prevents potential conflicts.

* Fix restore button state handling during restore process

Ensure the restore button is disabled consistently when a restore is in progress. This prevents user interaction issues and aligns the button state with the restore operation status.

* "Refactor restore logic and improve button state handling"

Replaced `is_restoring` with `restore_request_pending` for clarity and better state management. Adjusted button behavior to immediately disable upon user action, ensuring improved UX and preventing repeat inputs. Refined restore completion flow for better consistency and reliability.

* Refine restore process logic for SunnyLink settings.

Introduced `restore_request_started` to improve handling of restore states and ensure accurate UI updates during the process. Adjusted case handling to enhance clarity and maintain proper behavior when restoring settings, especially during ongoing or completed requests.

* revert

* move around

* fix enabled states for different statuses

* add prompt to notify backup is complete

* same states as restore

* disable buttons if sunnylink is off

* can use the same texts

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-22 15:58:04 +00:00
Jason Wen
efb44aeecd ui: Fix NNLC toggle not staying in persistent state (#697)
The `showEvent` method in `NeuralNetworkLateralControl` was removed as it duplicated functionality now handled in `LateralPanel`. The unused `refresh` method was also removed to streamline the code and improve maintainability.
2025-03-22 11:18:32 -04:00
Jason Wen
c70d3db1e6 Params: support filtering by ParamKeyType for allKeys (#696) 2025-03-22 10:29:24 -04:00
Jason Wen
309304a352 Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control (#695)
* init

* more init

* keep it alive

* fixes

* more fixes

* more fix

* new submodule for nn data

* bump submodule

* update path to submodule

* spacing???

* update submodule path

* update submodule path

* bump

* dump

* bump

* introduce params

* Add Neural Network Lateral Control toggle to developer panel

This introduces a new toggle for enabling Neural Network Lateral Control (NNLC), providing detailed descriptions of its functionality and compatibility. It includes UI integration, car compatibility checks, and feedback links for unsupported vehicles.

* decouple even more

* static

* codespell

* remove debug

* in structs

* fix import

* convert to capnp

* fixes

* debug

* only initialize if NNLC is enabled or allow to enable

* oops

* fix initialization

* only allow engage if nnlc is off

* fix toggle param

* fix tests

* lint

* fix more test

* capnp test

* try this out

* validate if it's not None

* make it 33 to match

* align

* share the same friction input calculation

* return stock values if not enabled

* unused

* split base and child

* space

* rename

* NeuralNetworkFeedForwardModel

* less

* just use file name

* try this

* more explicit

* rename

* move it

* child class for additional controllers

* rename

* time to split out custom lateral acceleration

* move around

* space

* fix

* TODO-SP

* TODO-SP

* update regardless, it's an extension now

* update name and expose toggle

* ui: sunnypilot Panel -> Steering Panel

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h

* merge

* move to steering panel

* no need for this

* live params in a thread

* no live for now

* new structs

* more ui

* more flexible

* more ui

* no longer needed

* another ui

* cereal changes

* bump opendbc

* simplify checks

* all in one place

* split Enhanced Lat Accel

* handle unrecognized platform

* test for fingerprinting

* fix fingerprint

* NNLC: Mock data for unrecognized cars

* fix fingerprints

* test to verify model loading

* bump neural_network_data

* use pytest

* use different one for now

* fix ui preview alignments

* typing

* more type

* show a platform

* show match

* init params for tests

* ok ruff

* make sure to loop through tests

* ok again ruff

* ok we need this lol

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-22 09:40:29 -04:00
Jason Wen
0990212edc Revert "Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control" (#694)
Revert "Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Ac…"

This reverts commit ecb4026269.
2025-03-21 16:13:50 -04:00
Jason Wen
ecb4026269 Controls: Neural Network Lateral Control (NNLC) for Torque Lateral Accel Control (#667)
* init

* more init

* keep it alive

* fixes

* more fixes

* more fix

* new submodule for nn data

* bump submodule

* update path to submodule

* spacing???

* update submodule path

* update submodule path

* bump

* dump

* bump

* introduce params

* Add Neural Network Lateral Control toggle to developer panel

This introduces a new toggle for enabling Neural Network Lateral Control (NNLC), providing detailed descriptions of its functionality and compatibility. It includes UI integration, car compatibility checks, and feedback links for unsupported vehicles.

* decouple even more

* static

* codespell

* remove debug

* in structs

* fix import

* convert to capnp

* fixes

* debug

* only initialize if NNLC is enabled or allow to enable

* oops

* fix initialization

* only allow engage if nnlc is off

* fix toggle param

* fix tests

* lint

* fix more test

* capnp test

* try this out

* validate if it's not None

* make it 33 to match

* align

* share the same friction input calculation

* return stock values if not enabled

* unused

* split base and child

* space

* rename

* NeuralNetworkFeedForwardModel

* less

* just use file name

* try this

* more explicit

* rename

* move it

* child class for additional controllers

* rename

* time to split out custom lateral acceleration

* move around

* space

* fix

* TODO-SP

* TODO-SP

* update regardless, it's an extension now

* update name and expose toggle

* ui: sunnypilot Panel -> Steering Panel

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h

* merge

* move to steering panel

* no need for this

* live params in a thread

* no live for now

* new structs

* more ui

* more flexible

* more ui

* no longer needed

* another ui

* cereal changes

* bump opendbc

* simplify checks

* all in one place

* split Enhanced Lat Accel

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-21 15:38:31 -04:00
Jason Wen
75c9db260f ui: reset all sunnypilot settings (#692)
* ui: reset all sunnypilot settings

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/device_panel.cc

Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>

---------

Co-authored-by: Discountchubbs <159560811+Discountchubbs@users.noreply.github.com>
2025-03-21 01:02:09 -04:00
Jason Wen
a2628fe4f6 ui: sunnypilot Panel -> Steering Panel (#691)
* ui: sunnypilot Panel -> Steering Panel

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral_panel.h
2025-03-20 17:01:47 -04:00
Jason Wen
d21e351003 Controls: Lateral Accel Torque Control Extension (#690)
* init

* more init

* keep it alive

* fixes

* more fixes

* more fix

* new submodule for nn data

* bump submodule

* update path to submodule

* spacing???

* update submodule path

* update submodule path

* bump

* dump

* bump

* introduce params

* Add Neural Network Lateral Control toggle to developer panel

This introduces a new toggle for enabling Neural Network Lateral Control (NNLC), providing detailed descriptions of its functionality and compatibility. It includes UI integration, car compatibility checks, and feedback links for unsupported vehicles.

* decouple even more

* static

* codespell

* remove debug

* in structs

* fix import

* convert to capnp

* fixes

* debug

* only initialize if NNLC is enabled or allow to enable

* oops

* fix initialization

* only allow engage if nnlc is off

* fix toggle param

* fix tests

* lint

* fix more test

* capnp test

* try this out

* validate if it's not None

* make it 33 to match

* align

* share the same friction input calculation

* return stock values if not enabled

* unused

* split base and child

* space

* rename

* NeuralNetworkFeedForwardModel

* less

* just use file name

* try this

* more explicit

* rename

* move it

* child class for additional controllers

* rename

* time to split out custom lateral acceleration

* move around

* space

* fix

* TODO-SP

* TODO-SP

* split nnlc and custom lat accel

* more

* not yet

* comment

* fix

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-20 16:01:08 -04:00
Discountchubbs
47c8ac2431 ui: Developer Panel Fixes (#688)
* Add error log to developer panel

* Add lang files

* Refactor GitHub runner control to local variable

Replaced the `enableGithubRunner` member variable with a local variable, as it is not used elsewhere in the class. Simplified visibility management for developer panel toggles in release branches.

* Modifying visibility rules for developer panel controls

This commit modifies the visibility settings for controls within the developer panel such as `enableGithubRunner`, `errorLogBtn`, `joystickToggle` and `longManeuverToggle`. The visibility of these controls is now dependent on whether the software is running in release mode. This update ensures that certain settings are hidden from the end user in the release version, enhancing the user interface by simplifying it and preventing potential user confusion.

* Translation files

* we already handle this in stock

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-19 19:16:54 -04:00
Jason Wen
bf2731b470 MADS: Steering Mode on Brake Pedal Press (#687)
* reimplement

* bump opendbc

* Apply suggestions from code review

Co-authored-by: DevTekVE <devtekve@gmail.com>

* update toggle positions

* translations from bot later

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-03-19 18:24:56 -04:00
DevTekVE
01998f864d sunnylink: Improve API initialization and enhance WebSocket security (#662)
Refactor sunnylinkd for better readability and SSL support

Reorganized variable handling for Sunnylink API initialization. Updated WebSocket connection to use Authorization header and added SSL configuration based on the connection type. This enhances code clarity and ensures secure connections when not on localhost.

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-19 21:47:39 +00:00
Discountchubbs
d2846741bc ui: Error log button to Developer panel (#627)
* Add error log to developer panel

* Testing debug

* Fix condition for retry handling in action workflow

Updated the logic to properly check for 'NONE' author association instead of 'FIRST_TIME_CONTRIBUTOR' when determining retry limits. This ensures the workflow behaves as intended for contributors without prior activity.

* Clean

* Refine workflow condition logic for PR checks

Updated the `if` condition in the lfs-maintenance workflow to improve clarity and ensure proper evaluation. This change aligns the logical grouping for better readability and correctness in forked PR handling.

* Add lang files

* Update selfdrive/ui/qt/offroad/developer_panel.cc

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Refactor GitHub runner control to local variable

Replaced the `enableGithubRunner` member variable with a local variable, as it is not used elsewhere in the class. Simplified visibility management for developer panel toggles in release branches.

* Updating translations

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-19 21:06:42 +00:00
Jason Wen
dd37a35b59 Revert "MADS: Steering Mode on Brake Pedal Press" (#686)
* Revert "MADS: Steering Mode on Brake Pedal Press (#684)"

This reverts commit 4525659a7e.

* bump
2025-03-19 16:02:45 -04:00
Jason Wen
4525659a7e MADS: Steering Mode on Brake Pedal Press (#684)
* MADS: cleanup param assignments

* MADS: Steering Mode on Brake Pedal Press

* bump opendbc

* translations

* make sure to initialize

* ci try this

* Revert "ci try this"

This reverts commit da975f61c3.

* Revert "translations"

This reverts commit a65aad5911.
2025-03-19 13:57:15 -04:00
Jason Wen
ad67bdc853 ci: disable prebuilts on pull requests (#685) 2025-03-19 10:11:07 -04:00
Jason Wen
6ad4ebdae6 MADS: cleanup param assignments (#683) 2025-03-19 00:45:53 -04:00
Jason Wen
31b3897418 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#675) 2025-03-18 01:23:54 -04:00
DevTekVE
f0abacbc59 Merge branch 'refs/heads/master-at-17032025' into sync-17032025
# Conflicts:
#	opendbc_repo
#	selfdrive/car/card.py
#	selfdrive/car/tests/test_car_interfaces.py
#	selfdrive/car/tests/test_models.py
#	selfdrive/controls/controlsd.py
#	selfdrive/controls/lib/tests/test_latcontrol.py
#	selfdrive/pandad/panda.h
#	system/manager/process_config.py
Sync: `commaai/opendbc:master` into `sunnypilot/opendbc:master-new`
2025-03-18 01:05:34 -04:00
Jason Wen
3177142bfc ci: update existing comment during Squash and Merge (#674)
* format

* ci: update existing comment during Squash and Merge

* dynamic target branch

* test label

* try again

* no f

* test

* should be edit-last

* quote

* fix arg

* final

* bold it

* revert
2025-03-17 03:49:27 -04:00
Jason Wen
4a04819d96 Redirect license to LICENSE.md (#672)
* Redirect license to LICENSE.md

* bump
2025-03-17 00:36:32 -04:00
Jason Wen
b95140666b ci: remove sunnylink pairing QR code from UI preview (#673) 2025-03-17 00:20:13 -04:00
Jason Wen
42871f7a9d pandad: dedicated aligned buffer for deserialization (#671)
* pandad: dedicated aligned buffer for deserialization

* ide pls
2025-03-16 19:53:53 -04:00
DevTekVE
cac8527115 sunnylink: pairing and sponsorship (#523)
* Refactor sunnylink panel code for clarity and initialization fixes.

Replaced explicit pointer types with `auto` for cleaner code and added proper initialization for the `offroad` boolean member. Simplified toggle logic by consolidating description updates for enabling/disabling sunnylink. These changes improve code readability and maintainability.

* Add Sunnylink sponsor and GitHub pairing functionality

This update introduces a feature to manage sponsorship-based roles and GitHub account pairing for Sunnylink. It includes new sponsor popups, sponsor-specific widgets, QR code logic, and backend API integrations. Additionally, new models and services support sponsor tier management and user-role synchronization.

* Translation files

* Param keys

* Add setup functions for SunnyLink sponsor and pair buttons

Introduce `setup_settings_sunnylink_sponsor_button` and `setup_settings_sunnylink_pair_button` to handle specific SunnyLink UI interactions. These functions streamline button clicks for sponsor and pairing actions within SunnyLink settings.

* Add new SunnyLink test cases for sponsor and pair buttons

Added `settings_sunnylink_sponsor_button` and `settings_sunnylink_pair_button` to the UI test case dictionary. This extends the SunnyLink test coverage to include sponsor and pairing functionalities.

* No need to import sunnylink from here, and it causes just circular dependency

* Enhance SunnylinkPanel functionality in off-road settings

This commit enhances the functionality of the SunnylinkPanel in the off-road settings of the SunnyPilot user interface. A paramWatcher is added to the SunnylinkPanel to observe "SunnylinkEnabled" parameter changes. Update functionalities are enhanced to handle showing and hiding of components based on various circumstances, such as whether the system is 'on-road' or 'off-road', and whether Sunnylink is enabled or not. The stopSunnylink and startSunnylink functions were also added to start or stop processes accordingly when Sunnylink is enabled or disabled. Additionally, the ui.h file is updated to efficiently handle Sunnylink roles and device users.

* Refactor SunnylinkPanel initialization and handling.

Reorganized SunnylinkPanel to improve structure and clarity by separating sunnylink client initialization and list widget setup. Enabled automatic sunnylink startup when the feature is enabled. Added minor formatting fixes for label display consistency.

* Add missing include for <optional> in ui.h

Including <optional> ensures compatibility with standard C++ features and prevents potential compilation errors. This addition aligns with best practices for maintaining robust and clean code.

* Updated setup_settings_sunnylink_sponsor_button and setup_settings_sunnylink_pair_button function signatures

Added an optional 'scroll' parameter to the setup_settings_sunnylink_sponsor_button and setup_settings_sunnylink_pair_button functions in the test_ui module. The modifications were made to allow for more flexible function usage by potentially enabling scroll operations during the execution of these UI setup steps.

* Enable Sunnylink initialization on panel show event

Begin Sunnylink connection automatically when the panel is displayed, ensuring the feature is active if enabled. Additionally, update the sponsor button text formatting for more concise styling.

* Translations

* Added checks for new UI files in PRs

The git workflow script `ui_preview.yaml` has been modified. The script now checks if the master branch contains a file corresponding to a UI file present in the PR. If a UI file in the PR does not have a match on the master branch, it is marked as new. These enhancements improve the comparison of UI changes between the master and PR branches, particularly with the identification of new UI files.

* cleanup

* duh

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-16 16:22:14 -04:00
DevTekVE
89e385b692 CI: update default screenshot in UI preview (#669)
Jason Jason...
2025-03-16 13:01:31 -04:00
DevTekVE
61b9d81ba6 CI: identify new screenshots in UI preview (#668)
Added checks for new UI files in PRs

The git workflow script `ui_preview.yaml` has been modified. The script now checks if the master branch contains a file corresponding to a UI file present in the PR. If a UI file in the PR does not have a match on the master branch, it is marked as new. These enhancements improve the comparison of UI changes between the master and PR branches, particularly with the identification of new UI files.
2025-03-16 17:08:21 +01:00
DevTekVE
4fbf7a81b4 sunnylink: Fix IMEI fetching during registration (#661)
Fix getting the imei for the registration
2025-03-16 11:12:43 -04:00
Jason Wen
f50d7a1842 ui: Update FirehosePanel with sunnypilot ui updates (#666) 2025-03-16 02:34:25 -04:00
Jason Wen
4ef46251a7 ui: Trim panel name for Firehose panel redirect (#664)
* ui: Trim panel name for Firehose panel redirect

* unused

* extra
2025-03-16 00:09:11 -04:00
Jason Wen
7f669ec1e7 MADS: Disable for Rivian (#663) 2025-03-15 22:28:32 -04:00
Jason Wen
1be2643dfd Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#656) 2025-03-15 21:24:44 -04:00
Jason Wen
db0ce99c01 Car interface: custom safety params (#659)
* init

* do it for hyundai

* Bump submodules

* no complaints

* use latest opendbc pr ref

* don't need this for leaf for now

* bump opendbc

* gotta set the sp safety flags in test models and safety replay!

* actually fetch CP_SP

* set safety param for lda button availability (lol forgot)

* use GET_FLAG

* Add TODO-SP

* bump submodules
2025-03-15 18:22:53 -04:00
Jason Wen
4879e2f5ec Bump opendbc 2025-03-15 18:22:53 -04:00
Jason Wen
70f8225efc Merge branch 'upstream/openpilot/master' into sync-20250312
# Conflicts:
#    opendbc_repo
#    selfdrive/car/tests/test_models.py
2025-03-13 02:41:16 -04:00
Jason Wen
6ed459d7cd Bump opendbc 2025-03-12 23:10:44 -04:00
DevTekVE
8c4f5f1ff4 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new` (#652) 2025-03-09 21:43:37 +01:00
DevTekVE
0798ebab07 Bumping refs 2025-03-09 21:15:25 +01:00
DevTekVE
141a7f5f72 Use numpy directly instead of numpy_fast for clip and interp
Replaced occurrences of `openpilot.common.numpy_fast` with direct imports from `numpy` across multiple files. This simplifies dependencies and ensures consistency with standard Python library usage. Adjusted tests to mock `numpy` functions accordingly.
2025-03-09 19:07:59 +01:00
DevTekVE
ba3acbce0a Add cl_print_build_errors to handle OpenCL build errors
This function logs detailed build failure information, including the status and build log retrieved from OpenCL. It provides better debugging support for diagnosing issues with OpenCL program compilation.
2025-03-09 18:29:32 +01:00
DevTekVE
425f5ebb2e Add clutil_legacy for OpenCL program support
Introduce `clutil_legacy` to handle OpenCL program creation from binaries and error string mapping. This improves modularity and prepares for enhanced OpenCL compatibility across platforms.
2025-03-09 17:57:59 +01:00
DevTekVE
1779a76d3c Update default model to "Filet o Fish"
Changed the DEFAULT_MODEL constant and updated the associated model hash to reflect the new default. This ensures consistency between the code and the new model configuration.
2025-03-09 15:00:12 +01:00
DevTekVE
a4ef5ad982 Merge remote-tracking branch 'comma/master' into sync-20250309
# Conflicts:
#	.github/workflows/ui_preview.yaml
#	common/params_keys.h
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/ui/qt/offroad/firehose.h
#	selfdrive/ui/tests/test_ui/run.py
#	system/manager/process_config.py
2025-03-09 14:53:03 +01:00
Jason Wen
bc65c8bef7 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#640) 2025-03-09 01:26:15 -05:00
Jason Wen
290383128e Sync: Fix conflicts 2025-03-09 00:53:06 -05:00
Jason Wen
f9952d7cb7 CI: Disable process_replay and remove fakedata (#651)
* CI: Disable process_replay

* remove fakedata

* don't bump panda yet
2025-03-08 23:30:44 -05:00
DevTekVE
d0f590b55e CI: use one label for runs-on (#650)
* test

* Fix runs-on so that things acutally run otherwise they get stuck due to gh change

* Add repository condition to process_replay in CI workflow

Restrict the process_replay job to run only for the commaai/openpilot repository. This change temporarily blocks execution for forks or other repositories.

* in another PR

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-08 23:06:26 -05:00
DevTekVE
7ddaf52b2b CI: adjust allow 1 retry for first time contributor and fork and skip sync lfs on fork PRs (#647)
* Update workflow condition to skip forked PRs during sync

This change ensures the LFS maintenance workflow does not run for pull requests originating from forked repositories. It enhances efficiency by avoiding unnecessary sync actions in these cases.

* Modified retry conditions in GitHub workflow file

Adjusted the retry conditions in the GitHub workflow setup file. The updated condition allows for a second attempt if the pull request originator is a 'FIRST_TIME_CONTRIBUTOR' from a forked repository. This modification lends more flexibility to first-time contributors by providing them with an additional chance to pass the tests.

* test

* test2

* test3

* Fix conditional logic for retry check in workflow script

The condition now correctly checks if the pull request is not from a fork using 'false' instead of 'true'. This ensures proper validation of retry attempts based on the contributor and repository source.

* Fix placement of event JSON logging in workflow script

Moved the `toJSON(github.event)` logging statement to ensure it runs after retry logic validation. This improves the logical flow of the script and maintains appropriate debugging output.

* Add debug logs for PR metadata and run attempt values

This change introduces debug logs to output key PR metadata, such as fork status, author association, and the GitHub run attempt value. These logs will aid in debugging and provide better visibility into workflow execution.

* Add debug log for PR fork and author association check

This change adds a debug log to display the evaluation of the fork and author association condition in the workflow. It helps in troubleshooting PR handling logic and ensures better visibility into the condition being assessed.

* Fix logical negation for fork checks in workflow script

Correct the evaluation of fork status using proper negation syntax. This ensures accurate logic in pull request conditions, improving the reliability of the workflow execution.

* Add log for first-time contributor check in workflow

This adds a debug statement to log whether the PR author is a first-time contributor. It helps improve observability and makes it easier to debug workflows involving author associations.

* Simplify log output in GitHub action script

Removed unnecessary debug echo statements to streamline the script and improve readability. Retained essential logic to handle retries and enforce restrictions on test flakiness.

* Simplify retry logic for pull request contributors.

Updated the condition to correctly identify and handle retries for first-time contributors and forked repositories. This ensures a clearer and more accurate retry policy in the workflow.

* Refine workflow conditions for PR checks and retries

Adjusted conditional logic in `lfs-maintenance.yaml` to improve readability and ensure proper handling of draft PRs. Updated retry logic in `setup/action.yaml` to reflect correct author association for stricter validations.
2025-03-06 15:02:29 +01:00
DevTekVE
a51ed062ca CI: allow 1 retry for first time contributor and fork and skip sync lfs on fork PRs (#646)
* Update workflow condition to skip forked PRs during sync

This change ensures the LFS maintenance workflow does not run for pull requests originating from forked repositories. It enhances efficiency by avoiding unnecessary sync actions in these cases.

* Modified retry conditions in GitHub workflow file

Adjusted the retry conditions in the GitHub workflow setup file. The updated condition allows for a second attempt if the pull request originator is a 'FIRST_TIME_CONTRIBUTOR' from a forked repository. This modification lends more flexibility to first-time contributors by providing them with an additional chance to pass the tests.
2025-03-06 10:24:14 +01:00
DevTekVE
17d292c773 CI: Protecting /release/ci with CODEOWNERS 2025-03-06 09:38:58 +01:00
DevTekVE
8c3f8064b2 Uploading codeowners (#645) 2025-03-06 09:34:33 +01:00
DevTekVE
b791d4ed55 Model: Handle missing 'sim_pose' gracefully in model outputs. (#644)
Handle missing 'sim_pose' gracefully in model outputs.

Check for the presence of 'sim_pose' in output data before processing. If absent, fallback to using 'plan' data to populate temporal pose fields, ensuring robustness and preventing potential runtime errors.
2025-03-02 20:52:44 +01:00
DevTekVE
1a8dd310ae Model: split modeld into it's own contained modeld implementation (#642)
* Add support for TinyGrad model runner processing

Introduced a new function `is_tinygrad_model` to detect TinyGrad as an active model runner. Updated the `is_stock_model` logic to account for TinyGrad models and added a new process entry for TinyGrad in the model manager. This enables handling TinyGrad models alongside existing configurations.

adding modeld back

Add support for `modeld_v2` and update paths for consistency

Updated `SConscript` files to integrate `modeld_v2` alongside `modeld` and adjusted script paths for correct metadata handling. Adjusted various configurations and scripts, such as `labeler.yaml` and `build_release.sh`, to include `modeld_v2` and ensure cohesive project structure.

Refactor imports to use updated `modeld_v2` paths.

Replaced outdated `modeld` references with their `modeld_v2` counterparts for consistency and clarity across the codebase. Also updated `.gitignore` to accommodate new directory structure. This change ensures better maintainability and alignment with the new directory schema.

Refactor and reorganize modeld to sunnypilot/modeld_v2 structure.

Moved and renamed `modeld` components to the new `sunnypilot/modeld_v2` directory for better organization and modularity. Updated imports and file references to align with the new structure, ensuring compatibility and functionality. Streamlined project structure to improve maintainability and future development.

* typo

* Use `stock` model runner and refactor model checks.

Replaces outdated model detection logic with unified `stock` runner integration, simplifying the decision flow for model selection. Includes `stock` as a new enum in the `Runner` type and updates affected references accordingly.

* Handle missing 'sim_pose' in model outputs gracefully.

Added conditional checks to ensure the code handles cases where 'sim_pose' is absent in the model outputs. Fallback behaviors use 'plan' data when 'sim_pose' is unavailable, preventing potential errors and enhancing robustness.
2025-03-02 20:49:30 +01:00
DevTekVE
cce0f14976 CI: Fix runner install script (#639)
Not sure why it stopped working before but this should work now
2025-03-01 15:29:31 +01:00
Jason Wen
d5aa4ba997 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#635) 2025-02-17 00:07:06 -05:00
Jason Wen
564914c3a4 test_processes: update ref logs to a0a1635 2025-02-16 23:43:51 -05:00
Jason Wen
a0a1635c55 Merge branch 'upstream/openpilot/master' into sync-20250215
# Conflicts:
#	common/params.cc
#	opendbc_repo
#	panda
#	selfdrive/car/tests/test_models.py
#	selfdrive/modeld/fill_model_msg.py
#	selfdrive/test/process_replay/process_replay.py
#	selfdrive/test/process_replay/ref_commit
#	selfdrive/ui/qt/offroad/developer_panel.cc
#	selfdrive/ui/tests/test_ui/run.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_fr.ts
#	selfdrive/ui/translations/main_ja.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
#	system/athena/athenad.py
2025-02-16 22:10:32 -05:00
Jason Wen
bb407ae9bf MADS: mute canBusMissing if vehicle is in Park and enabled (#613)
* Car: mute `canBusMissing` if vehicle is in Park

* only when mads is enabled
2025-02-01 22:26:19 -05:00
Jason Wen
d7087d312c MADS: Properly handle gear events (#628)
* MADS: Properly handle gear events

* use referenced object directly

* fix tests

* unused

* rename
2025-02-01 22:11:10 -05:00
Jason Wen
17b3092e76 modeld: script to generate new default model hash and name (#629)
* modeld: script to generate new default model hash and name

* break CI intentionally to trigger new changes

* more verbose and fix

* more verbose hehe
2025-02-01 02:55:52 -05:00
Jason Wen
977b164dd3 Vehicle Selector (#619)
* QFrame

* symlink for OP

* symlink dat

* search with user input

* single list is fine

* bump opendbc

* set actual fingerprint

* more precision search

* different prompt in onroad/offroad

* confirm/cancel

* set while initializing

* tldr

* show demo car

* old qt

* need that param too

* use toList

* Need to be str

* Encoding

* generate `car.CarParams.brand` in json

* start cleanup

* generate no car docs platforms

* add brand to parser

* include all dashcamOnly platforms

* pass all fields to a platform in json

* introduce `getPlatformBundle`

* extract platform from CarPlatformBundle directly

* Fix name

* do not include dashcamOnly yet

* slight cleanup

* bump opendbc

* it's a json now

* split set platform

* move loadPlatformList to sp util

* add year list to bundle

* bros

* missed header

* invoke platform selector on show event
2025-02-01 00:55:09 -05:00
Jason Wen
767880ffaf ui: Display default driving model name (#623)
* ui: Display default driving model if in use

* add ref commit and tests

* fix commit

* update msg

* update msg

* fix lint

* use sha256 hash instead
2025-01-27 22:49:28 -05:00
Jason Wen
3abe1c9168 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#622) 2025-01-25 23:14:35 -05:00
Jason Wen
8801d1eb8e test_processes: update ref logs to 7e5a84b 2025-01-25 22:51:46 -05:00
Jason Wen
7e5a84b897 Merge branch 'upstream/openpilot/master' into sync-20250125
# Conflicts:
#	opendbc_repo
#	panda
#	selfdrive/modeld/fill_model_msg.py
#	selfdrive/pandad/pandad.cc
#	selfdrive/test/process_replay/ref_commit
#	selfdrive/ui/qt/home.h
#	selfdrive/ui/qt/offroad/developer_panel.cc
#	selfdrive/ui/qt/offroad/developer_panel.h
#	selfdrive/ui/tests/test_ui/run.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
2025-01-25 22:50:03 -05:00
Jason Wen
c51e74e6af ui: Vehicle panel in settings (#617)
* ui: Vehicle panel in settings

* fix click
2025-01-24 21:57:46 -05:00
DevTekVE
c392b2b269 modeld: legacy MLSIM driving models support (#595)
* Add buffer length parameter for enhanced frame handling

Introduce a configurable `buffer_length` parameter to `DrivingModelFrame` to support dynamic buffer sizes, enabling better handling of different frame rates like 20Hz. Updates include necessary adjustments in buffer initialization, copying logic, and related model inputs for improved compatibility and flexibility.

* Rename variable `len` to `length` to avoid shadowing built-in.

Replaced the usage of `len` with `length` across the code to prevent conflicts with Python's built-in `len` function. This improves code clarity and reduces potential errors or misunderstandings in variable usage.

* Fix spacing inconsistency in modeld.py

    Added a missing newline for better code readability and consistency. This change has no impact on functionality but improves code formatting.

* Move numpy_inputs initialization to correct position

Repositioned the `numpy_inputs` initialization to align with the input shape processing logic. This ensures consistency in buffer management and clarifies the flow of code execution related to input handling.

* Add 20Hz model state, smart input, and model switcher classes

Introduce `ModelState20Hz`, `ModelSmartInput`, and `ModelSwitcher` for enhanced modularity and flexibility in modeld. Refactor `ModelState` to inherit from these new classes, enabling support for 20Hz processing and smart input initialization. Update associated files to handle the new buffer length parameter and metadata management.

* Refactor `modeld` to streamline feature handling logic

Simplified feature processing for both standard and "smart input" modes by consolidating logic into reusable methods. Updated variable naming, formatting, and spacing for consistency and readability. This refactor enhances maintainability and reduces redundancy in feature update operations.

* Silence debug print statements and use cloudlog for warnings.

Commented out a debug print statement in `commonmodel.cc` to reduce noise. Replaced `print` statements with `cloudlog.warning` in `model_smart_input.py` for improved logging consistency and better integration with the logging system.

* Clean up formatting and fix minor style inconsistencies

Removed unnecessary blank lines and adjusted spacing to standardize code style across the file. These changes improve readability without altering functionality or logic.

* Refactor modeld logic and remove unused 20Hz and smart inputs

Eliminated `ModelSmartInput`, `ModelSwitcher`, and `ModelState20Hz` classes, simplifying model state handling. Centralized model processing within a unified `ModelState` class and moved related code into `sunnypilot/modeld_20hz`. This improves maintainability by removing unused features and consolidating model execution logic, aligning with current system requirements.

* clean

* Remove debug print statement in commonmodel.cc

The `printf` statement logging buffer movement details was removed as it is unnecessary for release builds. This helps streamline the code and avoid excessive console output during execution.

* Refactor model handling for 20Hz and introduce model runners

Introduce ModelRunner abstraction with TinygradRunner and ONNXRunner to streamline model handling for TICI and non-TICI hardware. Added support for dynamic input preparation and 20Hz models while simplifying the model parsing logic. This improves modularity, readability, and extensibility for future updates.

* Remove unused import and fix import order in model_runner.py

This commit removes the unused 'dtypes' import from tinygrad.tensor and adjusts the import order for cleaner code. These changes enhance readability and maintain coding standards.

* Add is20hz field to custom.capnp schema

Introduce a new boolean field `is20hz` to the `custom.capnp` schema. This allows the system to identify 20Hz-specific configurations or data processing. No changes to existing behavior are introduced for non-20Hz cases.

* Add Meta20hz class for 20Hz model message handling.

Introduces a new Meta20hz class for filling 20Hz model messages, encapsulating functionality for curvature, lane lines, road edges, and more. Refactored `modeld.py` to utilize the new class, improving modularity and maintainability. Minor adjustments were made to initialize and handle model metadata.

* Refactor import paths to align with `openpilot` structure.

Updated several import statements to use the `openpilot` namespace for better consistency and organization. This aligns the sunnypilot components more closely with the overall project structure.

* Refactor modeld to support 20Hz models and modularize runners

Replaced legacy runner logic with a unified ONNX and Tinygrad runner to support 20Hz models. Centralized model metadata management and optimized input preparation for adaptability. Updated curvature handling and output parsing for improved modularity and maintainability.

* Add 20Hz metadata handling for model predictions

Introduce `Meta20hz` class for 20Hz-specific metadata and implement dynamic loading of meta model classes in `meta_helper.py`. Update `fill_model_msg.py` to use the new metadata structure, ensuring seamless integration with 20Hz models. Adjust imports in `model_runner.py` to align with project structure.

* "Refactor modeld_20hz to modeld_v2 with cleanup"

Refactored `modeld_20hz` module to `modeld_v2` for improved clarity and consistency. Removed unused code and aligned imports across modules to reflect the new structure. Enhanced maintainability by restructuring model-related files and updating references accordingly.

* Refactor variable names and adjust imports for clarity.

Renamed `len` to `length` to avoid conflict with the built-in function and improve readability. Reorganized imports in `fill_model_msg.py` for better structure and consistency.

* "Add missing newline at end of file in __init__.py

Ensure proper formatting by adding a newline at the end of the file. This adheres to POSIX standards and improves compatibility with some tools and version control systems."

* Handle model runner initialization errors gracefully

Wrap the model runner initialization in a try-except block to catch and log exceptions. This ensures that failures during initialization are logged with detailed information, improving debugging and error tracing.

* Refactor curvature calculation for clarity and reuse.

Introduce a dedicated `get_curvature_from_output` function to handle desired curvature retrieval, improving code readability and reusability. Replace redundant logic in curvature calculation with the new function to streamline the flow.

* Make 20Hz-specific variables conditional in modeld.py

Moved the initialization of 20Hz-specific variables to be conditional based on the `is_20hz` flag. This ensures that unnecessary memory allocations are avoided when the model is not running at 20Hz, improving efficiency and clarity.

* cleanup

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-24 05:01:14 +00:00
Jason Wen
213b977774 Car: Migrate sunnypilot CarControl to its own cereal (#606)
* carControlSP and move MADS to outside structs

* publish it

* apply to all car controller

* migrate sunnypilotParams

* migrate madsEnabled

* tldr

* convert to capnp

* unused

* wrong module

* fix name

* cancer is right (all tests should be passing now)

* bump opendbc
2025-01-23 10:47:16 -05:00
Jason Wen
f1837b8502 Sentry: sets environment tag (#605)
* Sentry: sets environment tag

* master channel

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-22 15:05:10 -05:00
Jason Wen
3e7240516e Car: Migrate sunnypilot CarParams to its own cereal (#604)
* sp flags

* pass CP_SP to card and car interfaces

* CP_SP in radar interface

* bump opendbc

* use dataclass like old times

* bump opendbc

* write to params for controls

* fix test models

* fix

* need to use copy instead

* fix data type

* add service

* more

* fix

* Revert "fix"

This reverts commit 74723d7fb2.

* Revert "fix data type"

This reverts commit 02355f44df.

* missed

* more

* no more lagging

* Reapply "fix data type"

This reverts commit dbf1b8583f.

* Reapply "fix"

This reverts commit 9cbce9968a.

* Revert "Reapply "fix""

This reverts commit 1871919b63.

* Revert "Reapply "fix data type""

This reverts commit 5e95752fd5.

* no longer

* Revert "no longer"

This reverts commit 66ee1ba151.

* Reapply "Reapply "fix data type""

This reverts commit 670a384333.

* Reapply "Reapply "fix""

This reverts commit 42f09f955c.

* only for car params sp

* rename

* fix more test

* no need for process replay

* pass stock car params to sp set car params

* pass stock car params to sp set car params

* deprecate CarParams.sunnypilotFlags to CarParamsSP.flags

* missed arg

* fix tests

* tests fixed

* need to pass this too

* must generate cp_sp!

* fix typing

* must be initialized prior can comm callback!

* no more cancer (@devtekve)

* remove more cancer

* Refactor `get_non_essential_params_sp` to simplify arguments (#612)

* Refactor 'get_non_essential_params_sp' function calls in tests

In both `test_latcontrol.py` and `process_replay.py`, simplified the function calls to 'get_non_essential_params_sp'. Removed an unnecessary call to 'get_non_essential_params'. This change makes the code cleaner and more efficient by reducing redundancy in the function calls. This modification also ensures consistency across different code files.

* Refactor get_non_essential_params_sp to take car_params.

Simplify parameters by modifying `get_non_essential_params_sp` to use `car_params` as input. Adjust related calls in test files and process replay to match the updated method signature. This improves code clarity and reduces redundancy.

* bump opendbc

* Refactor parameter handling for `get_params_sp`.

Removed unnecessary reassignment of `car_params` in calls to `get_params_sp`, ensuring a cleaner and more streamlined code structure. This change improves code clarity and eliminates redundant operations. All relevant assertions and behavior remain unaffected.

* bumping opedbc

* bump opendbc

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-22 14:22:32 -05:00
DevTekVE
403d77ff3f ci: squash and merge script update to allow merging less restrictively (#610)
Fix branch status check for merge readiness

Previously, any non-"CLEAN" status prevented merging. The check now specifically fails only if the branch is "BEHIND", allowing other statuses to proceed if valid. This ensures more accurate merge validations.
2025-01-22 13:41:56 +01:00
DevTekVE
9fe4d7ecc7 ui: Add Wi-Fi scan button to network settings (#608)
* Implemented custom Networking class for sunnypilot UI

This modification introduced a new 'NetworkingSP' class for the sunnypilot user interface. It's based on the existing Networking class, but tailored to the specific needs of the sunnypilot UI. This class adds a new 'Scan' button to the Wi-Fi screen and implements an additional layout to accommodate both 'Scan' and 'Advanced' buttons. It also contains updates to the file inclusions in 'settings.cc' to use the new class. Moreover, the accessibility of several member variables in the original Networking class has been updated from 'private' to 'protected'. This change enhances the modifiability and extensibility of the class structure.

* Add Spanish translations for networking scan messages

Added translations for "Scan" and "Scanning..." in the Spanish localization file. These updates ensure proper display and functionality in the networking scan feature for Spanish-speaking users.

* updating lang files

* Refactor networking component and clean up unused code.

Removed unnecessary comments and unused includes to enhance readability and maintainability. Refactored variable declarations for consistency and streamlined layout adjustments in the networking UI code.
2025-01-22 12:04:11 +01:00
DevTekVE
8d7315fa28 ci: Add support for "settings_network" and "settings_network_advanced" scene in UI tests (#609)
* Add support for "settings_network" scene in UI tests

Updated the workflow and test script to include the "settings_network" scene. Introduced a new setup function for "settings_network" and registered it in the scene-to-function mapping. This ensures proper handling and testing of network settings in the UI.

* Static analysis lol

* Add support for "settings_network_advanced" scene

Extend UI tests and workflow to include the "settings_network_advanced" scene. Updated the YAML workflow and test script to handle this new scene for comprehensive coverage.
2025-01-22 11:35:12 +01:00
DevTekVE
3e4be4a393 ci: Add branch reset workflow and improve squash script (#579)
* Add nightly branch reset workflow and improve squash script

Introduced a GitHub Actions workflow to reset and squash PRs nightly for the `master-dev-c3-new-test` branch. Enhanced `squash_and_merge.py` to handle more specific exit codes and `squash_and_merge_prs.py` to streamline PR processing. Updated argument handling in scripts and added validation for squash script execution.

* Forcing to show up

* UnForcing to show up

* Refactor branch handling to use inputs/environments directly

Removed intermediate step for setting branch variables and updated logic to use `inputs` or environment variables directly. Simplifies the workflow and improves maintainability by reducing redundancy and reliance on unnecessary outputs.

* Fix Python script invocation in CI workflow.

Replaced implicit script execution with an explicit `python3` command to ensure compatibility and consistency in the workflow. This change resolves potential issues with shell defaults or system configurations.

* Update branch reset logic in workflow script

Replaces checkout-based branch reset with deletion and recreation to ensure the target branch correctly points to the source branch. This change handles cases where the target branch may already exist.

* Refactor PR data handling to parse JSON input.

Updated the script to parse and handle PR data as JSON, ensuring proper data structure during processing. Adjusted functions to operate on parsed JSON instead of raw arguments for improved clarity and error handling.

* Refactor PR processing to streamline branch handling

Removed redundant `fetch_pr_branches` function and integrated branch fetching directly into `process_pr`. Simplified subprocess calls for clarity and added branch cleanup to prevent conflicts. This improves code maintainability and execution efficiency.

* Add PR number to squash and merge commit titles

This change appends the PR number to the title used in squash and merge commits, improving traceability and clarity in the commit history. The modification ensures easier identification of commits linked to specific pull requests.

* Enhance PR processing with sorting and additional checks

Implemented sorting of PRs by creation date and added checks for merge conflicts, commit data availability, and status check completion. Updated nightly squash script to provide detailed feedback via PR comments for skipped PRs. These changes improve the reliability and traceability of the merge process.

* Add traceback logging to error handling in squash_and_merge_prs

Enhanced error reporting by including full traceback details when an exception occurs in the `process_pr` function. This aids in debugging by providing more context on failures.

* Refactor and add debug output to PR merge check logic

Simplified multi-line statements into single lines for clarity and added debug `print` statements to log `merge_status` and its output. These changes enhance readability and facilitate debugging of the merge conflict check process.

* Add GITHUB_TOKEN to environment for CI workflow

Ensure the workflow has access to the GITHUB_TOKEN for authentication. This is necessary for interacting securely with the GitHub API during the CI process. Without this, some steps may fail due to lack of authorization.

* Enable scheduled workflow execution at midnight UTC

Reactivates the cron schedule for the workflow to run daily at midnight UTC. This ensures the workflow executes automatically without manual triggers, maintaining regular updates or checks.

* test

* Update workflow to trigger and monitor selfdrive tests

Replaced the direct triggering of the prebuilt workflow with a step to trigger and wait for the completion of selfdrive tests. Ensures prebuilt workflow runs only if selfdrive tests succeed, improving reliability of the CI process.

* Refine workflow trigger to fetch run URL and ID

Updated the workflow script to extract the run URL and derive the workflow ID from it. This ensures more accurate handling and tracking of GitHub Actions runs.

* Simplify selfdrive test triggering in workflow

Replaces custom script with a reusable GitHub Action to trigger and wait for selfdrive test completion. This improves maintainability and reduces complexity in the workflow file. Adjusts subsequent prebuilt workflow trigger to ensure compatibility.

* Remove duplicate prebuilt workflow trigger step

The redundant step for triggering the sunnypilot prebuilt workflow has been removed. This cleanup avoids unnecessary duplication and ensures a more streamlined workflow definition.

* Update selfdrive test trigger to use GitHub CLI commands

Replaced the third-party action with GitHub CLI for triggering and monitoring selfdrive tests. This change improves maintainability and reduces reliance on external dependencies. Updated related steps to ensure compatibility with the new approach.

* Add delay to ensure selfdrive tests workflow starts

Introduce a 120-second sleep before fetching the latest run ID to allow sufficient time for the selfdrive tests action to initialize. This prevents potential issues caused by attempting to retrieve the run ID too early.

* Improve push step to check for diffs before execution

Added logic to verify if there are differences between local and remote branches before attempting a push. This prevents unnecessary pushes and skips subsequent workflows when no changes are detected. Updated dependent steps to conditionally run based on the presence of changes.

* Update target branch and improve squash comment clarity

Renamed the default target branch from `master-dev-c3-new-test` to `nightly` in the workflow configuration. Enhanced squash process comments to dynamically reflect the `target_branch` value for better clarity and consistency.

* Add missing newline at end of file

Ensures the file complies with POSIX standards by including a newline at the end. This improves consistency and prevents potential issues with some tools or systems.

* Update default target branch and disable nightly schedule

Changed the default target branch to 'master-dev-c3-new' for workflow consistency. Commented out the nightly schedule to temporarily disable automated runs. No functional changes were made to other parts of the workflow.

* Refactored squash and merge scripts for improved PR handling

In this commit, significant updates have been made to the 'squash_and_merge.py' and 'squash_and_merge_prs.py' scripts related to how pull requests (PRs) are processed.

A 'source-branch' argument has been added to the argument parser. The merging command has been changed from 'merge' to 'rebase'.

The PR processing function has been refined. Specifically, PR validation is now a separate function confirming the conditions 'branch name', 'commit data', 'check pass status', and 'mergeability'. Now, any failures under these conditions result in skipping the PR with an appropriate warning.

The target branch is deleted if it exists, before a new one is created from the source branch. The squash script now runs with more structured arguments.

These changes generally improve PR handling in CI/CD pipelines, making them more efficient and error-resistant.

* Add 'PullRequest' to .codespellignore

This update includes 'PullRequest' in the .codespellignore file to prevent it from being flagged as a spelling error. It helps streamline code reviews and reduces false positives during spell checks.
2025-01-22 09:30:53 +01:00
Jason Wen
d08fd25784 Events: Migrate sunnypilot onroad events to its own cereal (#603)
* Events: Migrate sunnypilot onroad events to its own cereal

* more

* slightly more

* typing

* fix more

* fix mads state machine tests

* readjust order

* fix event

* abstract

* need these

* move around

* let's make sure it cleared on every loop

* Update selfdrive/selfdrived/alertmanager.py

Co-authored-by: DevTekVE <devtekve@gmail.com>

* use upstream custom struct

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-20 22:18:19 -05:00
Jason Wen
4730a192b1 Revert "Events: Migrate sunnypilot onroad events to its own cereal" (#602)
Revert "Events: Migrate sunnypilot onroad events to its own cereal (#598)"

This reverts commit c9961f1590.
2025-01-20 21:50:54 -05:00
Jason Wen
5f10529a88 Device: Offroad Mode (#596)
* try scrolling

* Revert "try scrolling"

This reverts commit 18cc0828c0.

* init

* event

* add logic

* last bit

* expose toggle

* update toggle

* add offroad btn to it

* fix

* update
2025-01-20 16:59:54 -05:00
Jason Wen
bc67effb6d ui: Keep power buttons clickable while onroad (#601) 2025-01-20 16:04:56 -05:00
Jason Wen
c9961f1590 Events: Migrate sunnypilot onroad events to its own cereal (#598)
* Events: Migrate sunnypilot onroad events to its own cereal

* more

* slightly more

* typing

* fix more

* fix mads state machine tests

* readjust order

* fix event

* abstract

* need these

* move around

* let's make sure it cleared on every loop

* Update selfdrive/selfdrived/alertmanager.py

Co-authored-by: DevTekVE <devtekve@gmail.com>

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-20 13:23:44 -05:00
Jason Wen
d58b0f403f bump submodules 2025-01-20 01:37:58 -05:00
Jason Wen
837eea06a4 MADS: remove controlsAllowedLat to maintain compatibility with stock cereal (#597)
* MADS: remove `controlsAllowedLat` to maintain compatibility with stock cereal

* unused

* bump panda
2025-01-19 20:48:16 -05:00
Jason Wen
485ac32250 Longitudinal: Distance button hold to toggle Chill/Experimental Mode (#576)
* Longitudinal: Distance button hold to toggle Chill/Experimental Mode

* unused

* fix

* no need

* Refactor: Introduce ButtonHoldTracker to manage button hold durations (#593)

Add ButtonHoldTracker for button hold logic and tests

Introduce a new `ButtonHoldTracker` class to manage button hold durations, replacing manual timer handling in `ExperimentalSwitcher`. Updated `ExperimentalSwitcher` to leverage this implementation for cleaner and more modular code. Added comprehensive unit tests for both `ButtonHoldTracker` and `ExperimentalSwitcher` to ensure functionality and edge case coverage.

* Revert "Refactor: Introduce ButtonHoldTracker to manage button hold durations (#593)"

This reverts commit 7ee7e73ce7.

* less in selfdrived

* pass carparams into child

* tests for cruisehelper

* rename these bad bois (happy now @devtekve? xD)

* Apply suggestions from code review

Co-authored-by: DevTekVE <devtekve@gmail.com>

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-19 20:23:51 +00:00
Jason Wen
5c38aeae0b Longitudinal: Dynamic Experimental Control (#572)
* init dec

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* fix static test

* ff

* fix static test

* unitee testt

* Refactor test_dynamic_controller and fix formatting issues

Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency.

* Refactor test indentation for dynamic controller tests

Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code.

* Migrated to pytest using claude

* Integrate radar parameter into dynamic controller's pytest tests

Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases.

* Disabling unittest file to allow checks on the pipeline to succeed.

Pending to remove this, but leaving it to validate the move to pytest is okay before merging

* Replace unittest with pytest for dynamic controller tests

Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage.

* new line...

* Refactor and modularize DynamicExperimentalController logic

Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports.

* Add missing import for messaging in helpers.py

The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code.

* Format

* Formatting

* rebase fix

* Refactor MpcSource definition and update references.

Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality.

* Format

* Refactor DEC into a dedicated longitudinal planner class

Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity.

* **Refactor DEC module structure for better organization**

Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes.

* static test

* static

* had moved to car_state

* cleanup

* some more

* static method

* move around

* more cleanup

* stuff

* into their own

* rename

* check live param

* sync with stock

* type hint

* unused

* smoother trans

* window time

* fix type hint

* pass sm.frame from plannerd

* more fixes

* more

* more explicit

* fix test

* Revert "fix test"

This reverts commit 635b15f2bc.

* Revert "pass sm.frame from plannerd"

This reverts commit a8deaa69b8.

* use internal frame

* update name

* never used

* this is why it was never using DEC

* more logs

* slight cleanup

* remove to fail test

* update name

* more

* rename

* move around

* explicit type hints

* move to constants py

* Revert "explicit type hints"

This reverts commit c205497b

* more

* don't set to exp mode initial if DEC is active

* use walrus for None

* Revert "use walrus for None"

This reverts commit 5f2396d490.

* fix wrong typing and variable name

* use walrus (needs cleanup)

* fix tests

* revert smooht lead for now

* dec: how good is FirstOrderFilter?

* Update dec.py

* dec: faster ?

* Revert "dec: faster ?"

This reverts commit 40259cd22a.

* Revert "Update dec.py"

This reverts commit 3f29ccbd99.

* Revert "dec: how good is FirstOrderFilter?"

This reverts commit 01e06df542.

* Update slow-down logic and constants for improved behavior

Adjust the slowdown scaling factor and anomaly handling to refine behavior without abrupt resets. Modify constants to increase window size and adjust probabilities and distances for smoother adaptation. Update version to reflect the new changes.

* Fix lead detection fallback for weighted average check.

Add a fallback value of -1 when computing the weighted average to prevent errors caused by invalid or None values. This ensures robust lead detection and avoids potential crashes or undefined behavior.

* visuals for DEC

* try this

* add opacity

* should be active and dimmer

* even dimmer

* Update dec.py

* Update constants.py

* use another method for drawing

* migrate to sp only

* fix

* init

---------

Co-authored-by: rav4kumar <meetkumardesai@gmail.com>
Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-19 12:04:54 -05:00
Jason Wen
177bfc9ba7 ui: Model Selector: display prompt with rich text widget (#591) 2025-01-19 10:00:05 +01:00
Jason Wen
155f9ee3b9 bump submodules 2025-01-19 02:50:02 -05:00
Jason Wen
073887221d MADS: Honda: controls, events, and cluster updates (#589)
* MADS: Honda: Cluster icons and events updates

* bump opendbc

* nuke them all to test

* try this

* bring them back

* fix icons

* MADS: Honda: Allow steering

* rename

* match stock

* bump submodules
2025-01-19 02:48:53 -05:00
Jason Wen
36453f2014 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#590) 2025-01-18 19:19:51 -05:00
Jason Wen
2792ce472a Merge branch 'upstream/openpilot/master' into sync-20240118
# Conflicts:
#	cereal/custom.capnp
#	cereal/log.capnp
2025-01-18 18:54:56 -05:00
Jason Wen
4e6cc1f3bc ui: Hide regulatory button on non-comma devices (#585)
Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-18 20:22:15 +00:00
DevTekVE
f5764c2edf ci: release drafter sort by default (time)
Using sort by title made things look out of order. If we want to organize them a bit furter, we can do that manually before moving forward with a release.
2025-01-18 15:16:43 +01:00
DevTekVE
027023a7ff CI: Update release drafter config (#588)
* Remove contributors section from release drafter template

The contributors section has been removed to simplify the release notes format. This change ensures a cleaner and more focused release draft template.

* Update release-drafter config with new labels and options

Added 'exclude-labels' to filter out 'no-changelog' items and improved category settings with 'collapse-after'. Introduced 'sort-by' for title-based sorting to enhance organization.
2025-01-18 15:10:13 +01:00
DevTekVE
da980312d9 Add Release Drafter configuration and workflow (#586)
* Add configuration files for Release Drafter

This commit introduces two configuration files for the Release Drafter. These include release-drafter.yml which will help categorize changes into features, bug fixes, and maintenance, and workflows/release-drafter.yml which defines a GitHub actions workflow to update release drafts.

* Work on both master branches

* Meh static and i dont need this tbh
2025-01-18 14:29:20 +01:00
Jason Wen
1a3f86a542 ui: Transition offroad state for Device panel buttons (#584) 2025-01-18 02:38:29 +00:00
Jason Wen
ac255742b6 MADS: Toyota: remove LKAS button support (#583)
* MADS: Toyota: remove LKAS button support

* remove mads button test

* bump submodules

* bump submodules

* test_processes: update ref logs to 4b12690
2025-01-17 19:22:50 -07:00
Jason Wen
d15d56b161 sentry: log fingerprints and save exceptions (#581)
* sentry: log fingerprints and save exceptions

* no point

* remove unused imports

* use deprecated

* have to do this

* remove

* cap

* no duplicate

* typo

* don't use deprecated method

* wrap them around

* do this instead

* new endpoint

* no need
2025-01-17 17:22:45 +00:00
Jason Wen
429b461eae Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#580) 2025-01-16 18:00:26 -05:00
Jason Wen
ce3c55c691 test_processes: update ref logs to afde277 2025-01-16 17:13:53 -05:00
Jason Wen
afde2771f8 Merge branch 'upstream/openpilot/master' into sync-20250116
# Conflicts:
#    .github/workflows/ui_preview.yaml
#    opendbc_repo
#    panda
#    release/release_files.py
#    selfdrive/test/process_replay/ref_commit
#    selfdrive/ui/tests/test_ui/run.py
2025-01-16 17:12:53 -05:00
Jason Wen
01c5dbdc4c Hyundai CAN: auto-enable radar tracks on applicable Mando radar (#561)
* more

* event and checks

* comments

* missed events

* retry 2 times is enough

* rename to radar tracks

* fix data type

* more rename

* bump opendbc

* drain first

* put it behind a toggle lol

* re-enable

* update comments

* revert lead smoothing

* Revert "revert lead smoothing"

This reverts commit 872267970c.

* real events and radard engagement

* only show up for hyundai with mando

* update translations

* bump opendbc

* fix event name

* update description

* move above

* translations

---------

Co-authored-by: rav4kumar <meetkumardesai@gmail.com>
2025-01-13 16:14:00 -05:00
Kumar
c3bfd17028 Revert "Longitudinal: Dynamic Experimental Control" (#571)
Revert "Longitudinal: Dynamic Experimental Control (#564)"

This reverts commit bba3c39e2f.
2025-01-13 05:17:15 +00:00
Kumar
bba3c39e2f Longitudinal: Dynamic Experimental Control (#564)
* init dec

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* fix static test

* ff

* fix static test

* unitee testt

* Refactor test_dynamic_controller and fix formatting issues

Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency.

* Refactor test indentation for dynamic controller tests

Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code.

* Migrated to pytest using claude

* Integrate radar parameter into dynamic controller's pytest tests

Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases.

* Disabling unittest file to allow checks on the pipeline to succeed.

Pending to remove this, but leaving it to validate the move to pytest is okay before merging

* Replace unittest with pytest for dynamic controller tests

Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage.

* new line...

* Refactor and modularize DynamicExperimentalController logic

Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports.

* Add missing import for messaging in helpers.py

The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code.

* Format

* Formatting

* rebase fix

* Refactor MpcSource definition and update references.

Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality.

* Format

* Refactor DEC into a dedicated longitudinal planner class

Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity.

* **Refactor DEC module structure for better organization**

Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes.

* static test

* static

* had moved to car_state

* cleanup

* some more

* static method

* move around

* more cleanup

* stuff

* into their own

* rename

* check live param

* sync with stock

* type hint

* unused

* smoother trans

* window time

* fix type hint

* pass sm.frame from plannerd

* more fixes

* more

* more explicit

* fix test

* Revert "fix test"

This reverts commit 635b15f2bc.

* Revert "pass sm.frame from plannerd"

This reverts commit a8deaa69b8.

* use internal frame

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-12 19:11:58 -05:00
Jason Wen
4a60eb50e5 ui: new design for Device panel buttons (#557)
* init btns

* better design

* force offroad in another PR

* old calibration design for now

* revert

* ci: fix position

* try this

* check ci

* Revert "check ci"

This reverts commit 80795cc700.

* test more ci

* Revert "test more ci"

This reverts commit 05670b5ad6.

* nuke more on CI

* unlikely but why not

* Revert "unlikely but why not"

This reverts commit 72f45a7516.

* Revert "nuke more on CI"

This reverts commit 7f406efc24.

* try this

* Revert "try this"

This reverts commit 3ce6768067.

* same across
2025-01-12 15:29:08 -05:00
DevTekVE
2f2b13382b CI: update install script to allow restore of github runner after agnos update (#562)
* Refactor GitHub runner installation script

Enhance the script with modular functions for better readability and maintenance. Add support for restoration mode, enabling the recovery of existing runner configurations. Improve error handling, argument parsing, and system configuration setup.

* Ensure runner directory existence before restoration

Add a prerequisite check to verify the runner directory exists, preventing restoration failures due to a missing directory. Adjust exit code logic for proper handling when no restoration is required.

* rw not needed until a bit later

* Refactor system configuration handling and remount logic.

Reorganized system setup functions and centralized remount logic for consistency and clarity. Improved script structure by separating utility and setup functions, ensuring proper remounting behavior with a robust exit trap. Reduced redundancy and enhanced maintainability.

* silly change
2025-01-12 11:37:02 +01:00
DevTekVE
9ccc5cc3ea models: retain pre-20hz drive model support (#531)
* modeld: Retain pre-20hz drive model support

* Method not available anymore on OP

* some fixes

* Revert "Long planner get accel: new function args (#34288)"

* Revert "Fix low-speed allow_throttle behavior in long planner (#33894)"

* Revert "long planner: allow throttle reflects usage (#33792)"

* Revert "Gate acceleration on model gas press predictions (#33643)"

* Reapply "Gate acceleration on model gas press predictions (#33643)"

This reverts commit 76b08e37cb.

* Reapply "long planner: allow throttle reflects usage (#33792)"

This reverts commit c75244ca4e.

* Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)"

This reverts commit b2b7d21b7b.

* Reapply "Long planner get accel: new function args (#34288)"

This reverts commit 74dca2fccf.

* don't need

* retain snpe

* wrong

* they're symlinks

* remove

* put back into VCS

* add back

* don't include built

* Refactor model runner retrieval with caching support

Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions.

* Update model runner determination logic with caching fix

Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks.

* parse inputs via metadata

* load model and metadata dynamically

* cherry pick from devtekve as base

* lateral_control_params & prev_desired_curv: MLSIM V0 to Null Pointer

* old desired_curv data: MLSIM V1 to Postal Service

* Bringing what was on master back then

* Cleaning up

* Refactor model pipeline for modularity and dynamic input handling

Refactored the model pipeline by introducing helper functions to modularize model loading, metadata extraction, and input preparation. Improved flexibility in handling dynamic input keys and parsing outputs based on model configuration. Removed deprecated or unused code segments for cleaner and more maintainable structure.

* Push NDv2 because why not and fix modeld

* `Refactor model parsing and clean unused code dependencies`

Simplified `parse_outputs` by removing unnecessary `input_keys` parameter, ensuring cleaner logic. Updated `PROCESS_NAME` for standardization and eliminated deprecated `Pathlib` dependencies in model paths. Minor adjustments improve input handling for lateral control parameters.

* Refactor model and metadata loading functions.

Simplified and clarified `load_model` by renaming it to `get_model_path` and removing redundant variable assignments. Streamlined `load_metadata` by directly returning the loaded metadata without intermediate variables. These changes improve code readability and maintainability.

* Refactor modeld process selection based on SNPE support.

Introduce conditional logic to determine and start the appropriate modeld process (SNPE or default) based on hardware support. This improves flexibility and ensures correct process management.

* Walrus baby

* Update longitudinal_planner.py

* Improve model download progress handling and translations

Refactored model download status handling logic for better clarity and added mechanisms to track status changes efficiently. Updated UI text and translations across multiple languages to reflect consistent and accurate model download states.

* Revert "Update longitudinal_planner.py"

This reverts commit b44a687e4c.

* Fix variable naming for curvature size in modeld.py

Renamed the variable `len` to `length` to avoid conflict with the built-in `len()` function, improving code clarity and preventing potential errors. Also removed trailing spaces in commented-out sections for better formatting consistency.

* sync with upstream

* some are different

* should work

* sim_pose only exists in in ndv3 and prior

* dynamic meta constants

* fix

* uiview: disable power saving

* fix this

* ain't coasting for y'all

* Static analysis

* Refactor initialization of model inputs for clarity.

Removed redundant pre-initialization of `driving_style`, `nav_features`, and `nav_instructions` variables. Instead, directly initialize these within their respective conditional blocks for better readability and reduced memory usage.

* default to none

* enable in next PR

* more

* Revert "more"

This reverts commit f5a4220588.

* Revert "enable in next PR"

This reverts commit 621cc4f18e.

* no need to cast bool

* nuke

* fix desired curvature for pre LAv1 models

* mypy

* static

* fix

* new json

* Going to a test branch with a different model json list

* renamed model json

* Update model runner handling in custom.capnp and helpers

Refactored model runner logic by introducing a `runner` field in `custom.capnp` and simplifying `get_active_model_runner` logic. Removed deprecated function `get_model_runner_by_filename` and added a temporary filter in `fetcher.py` to enforce `snpe` until full tinygrad support is implemented.

* Revert "Update model runner handling in custom.capnp and helpers"

This reverts commit f34d872c13.

* Revert "renamed model json"

This reverts commit 15c6ed303b.

* Revert "Going to a test branch with a different model json list"

This reverts commit 4c1408fee5.

* Reapply "renamed model json"

This reverts commit c6fec6912a.

Reapply "Going to a test branch with a different model json list"

This reverts commit 83e253e9a3.

* Add 'runner' property to 'ModelBundle' and update relevant functions

The 'ModelBundle' class in 'custom.capnp' has been extended to include a 'runner' property. This required updating 'fetcher.py' to handle the 'runner' property when parsing model bundles. Additionally, the helper function 'get_model_runner_by_filename' has been removed from 'helpers.py' as it is no longer needed because the 'runner' property provides this information. The 'get_active_model_runner' function has also been updated in light of these changes.

* Refine bundle selection logic in SoftwarePanelSP

Improve logic for determining which model bundle to display by considering download status and failure state. Remove unnecessary function call to enhance clarity and maintainability.

* tmp

* Add retrieval of active model bundle in manager loop

Introduce a call to `get_active_bundle` to fetch the active model bundle and store it in `self.active_bundle`. This ensures the active bundle is always up-to-date during the model management process.

* Add "Use Default" option for model selection

Introduced a "Use Default" option in the model selection dropdown, allowing users to reset to the default model. Adjusted logic to handle default selection and ensure proper parameter updates. Fixed bundle status handling during the download process in the model manager.

* Refactor type hint in active_bundle assignment.

Removed an unnecessary type hint in the active_bundle assignment for cleaner and more maintainable code. This change does not affect functionality but improves code readability.

* no nested

* update json url

* split out

* format

* condense

* more

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com>
2025-01-11 22:32:01 -05:00
DevTekVE
898b7f28a8 Better model builder (#559)
* Add job to fetch and upload model artifacts

Introduced a new `get_model` job to retrieve and upload model artifacts as part of the workflow. Updated the `build_model` job to download and clean up model files before building, ensuring an isolated and streamlined process. These changes improve modularity and artifact handling in the CI pipeline.

* Refine model file handling in build workflow

Restrict deletion and download commands to ".onnx" files to prevent unintended removal or overwriting of non-ONNX files. This ensures a more targeted and safer workflow for handling model artifacts.

* Update model artifact handling in build workflow

Changed artifact upload path to include only .onnx files and commented out the deletion of existing models. Also added a dependency for the build stage to ensure proper execution sequence.

* Update build workflow to refine model artifact handling

Removed unused code for deleting models and fixed the artifact path. Added a reference file logging custom name and upstream branch for better traceability. These changes streamline the workflow and improve clarity.

* Set dynamic run-name for sunnypilot build workflow

Added a customizable `run-name` field in the sunnypilot build model workflow. This allows more informative and dynamic naming based on input parameters such as `custom_name` or `upstream_branch`. Improves clarity and traceability in workflow runs.

* Add customization options and metadata generation to build

Introduce new inputs for file name prefix and 20Hz model specification in the workflow. Enable renaming of model files, generation of file hashes, and creation of a `metadata.json` file containing build details. This improves flexibility and enhances artifact information management.

* Refactor CI workflow to remove redundant get_model job

The `get_model` job has been removed, and its relevant steps have been merged into the `build_model` job to streamline the workflow. Additionally, a patch for `SConstruct` has been added to support an arbitrary cache directory configuration. This simplifies the pipeline while maintaining functionality.

* powersave ok ok

* Revert "powersave ok ok"

This reverts commit 716b9f1e2d.

* Update build process to use .thneed files over .pkl

Replaced .pkl file references with .thneed in the sunnypilot build workflow. Adjusted file renaming, hashing, and metadata handling to align with the new file format. This ensures compatibility with the updated file structure.

* let's seeeeeeeeee

* Remove unused CXXFLAGS from build step in workflow

CXXFLAGS options `-Wno-vla-extension` and `-Wno-vla-cxx-extension` were removed from the `scons` command as they are no longer necessary. This simplifies the build configuration and ensures cleaner workflows.
2025-01-11 19:23:00 -05:00
Jason Wen
72a4e0970a MADS: Filter disengage events without lateral control impact (#556) 2025-01-10 19:27:41 -05:00
Jason Wen
acd46aa94b modeld: retain SNPE and thneed drive model support (#555)
* modeld: Retain pre-20hz drive model support

* Method not available anymore on OP

* some fixes

* Revert "Long planner get accel: new function args (#34288)"

* Revert "Fix low-speed allow_throttle behavior in long planner (#33894)"

* Revert "long planner: allow throttle reflects usage (#33792)"

* Revert "Gate acceleration on model gas press predictions (#33643)"

* Reapply "Gate acceleration on model gas press predictions (#33643)"

This reverts commit 76b08e37cb.

* Reapply "long planner: allow throttle reflects usage (#33792)"

This reverts commit c75244ca4e.

* Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)"

This reverts commit b2b7d21b7b.

* Reapply "Long planner get accel: new function args (#34288)"

This reverts commit 74dca2fccf.

* don't need

* retain snpe

* wrong

* they're symlinks

* remove

* put back into VCS

* add back

* don't include built

* Refactor model runner retrieval with caching support

Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions.

* Update model runner determination logic with caching fix

Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks.

* default to none

* enable in next PR

* more

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-10 18:34:06 -05:00
Jason Wen
7329128325 ci: generate Trips panel for ui preview (#549)
* ci: generate trips panel for ui preview

* fix name
2025-01-10 17:48:04 -05:00
Jason Wen
402ee444b7 Revert "modeld: retain SNPE and thneed drive model support" (#551)
Revert "modeld: retain SNPE and thneed drive model support (#530)"

This reverts commit 56aa07844a.
2025-01-10 16:25:16 -05:00
Jason Wen
5b99c8b185 Revert "ci: generate Trips settings panel for ui preview" (#548)
Revert "ci: generate Trips settings panel for ui preview (#546)"

This reverts commit b29117c02a.
2025-01-10 11:25:32 -05:00
Jason Wen
b29117c02a ci: generate Trips settings panel for ui preview (#546)
ci: generate trips panel for ui preview
2025-01-10 10:59:11 -05:00
Jason Wen
56aa07844a modeld: retain SNPE and thneed drive model support (#530)
* modeld: Retain pre-20hz drive model support

* Method not available anymore on OP

* some fixes

* Revert "Long planner get accel: new function args (#34288)"

* Revert "Fix low-speed allow_throttle behavior in long planner (#33894)"

* Revert "long planner: allow throttle reflects usage (#33792)"

* Revert "Gate acceleration on model gas press predictions (#33643)"

* Reapply "Gate acceleration on model gas press predictions (#33643)"

This reverts commit 76b08e37cb.

* Reapply "long planner: allow throttle reflects usage (#33792)"

This reverts commit c75244ca4e.

* Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)"

This reverts commit b2b7d21b7b.

* Reapply "Long planner get accel: new function args (#34288)"

This reverts commit 74dca2fccf.

* don't need

* retain snpe

* wrong

* they're symlinks

* remove

* put back into VCS

* add back

* don't include built

* Refactor model runner retrieval with caching support

Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions.

* Update model runner determination logic with caching fix

Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks.

* default to none

* enable in next PR

* more

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-10 08:56:10 -05:00
DevTekVE
d5f5887830 CI: Increase test report creation timeout and improve build times (#545)
* Increase timeout for test report creation by 1 minute

Adjusted the `timeout-minutes` value to allow more time for the
test report creation step.

* Add script to disable power save in build workflow

This commit integrates the `disable-powersave.py` script into the sunnypilot build workflow. It ensures power save mode is disabled during the build process, improving reliability and consistency.

* Set PYTHONPATH before running disable-powersave.py

The change ensures the correct Python module path is set when executing the disable-powersave script. This update prevents potential import issues by including the GitHub workspace in the PYTHONPATH.

* Refactor powersave handling with a dedicated script

Replaced inline powersave disable logic with a new script `manage-powersave.py` to handle enabling and disabling power saving mode. Updated the CI workflow to use this script and added a step to re-enable powersave after builds. This improves clarity and modularity for power management operations.

* Enable CPU core count display in power save script

Introduce multiprocessing to show the number of CPU cores available. Added messages to indicate power save mode state and core count, improving script feedback and user clarity.

* Refine power save mode logging output.

Improve clarity of logging messages in `manage-powersave.py` by indicating CPU core counts before and after applying changes. Simplified the power save state message for better readability.

* Updated scons cache key restore logic in GitHub workflows

This commit updates the restore key logic in two GitHub workflow config files (sunnypilot-build-model.yaml and sunnypilot-build-prebuilt.yaml). The restore key sequence has been revised for improved accuracy and consistency. This should optimize the cache hit rate and speed up the subsequent builds.

* Add comments on GitHub Actions cache isolation behavior

Clarifies the cache isolation enforced by GitHub Actions for security reasons. Notes that only caches from the default branch are shared across all builds, and this behavior cannot be changed. This improves maintainability and understanding of the workflow configuration.
2025-01-10 10:04:38 +01:00
DevTekVE
9e6656fd2e models: Fix cache expiration logic in fetcher.py
Ensure the cache is treated as expired when last_sync is zero. This prevents potential issues with uninitialized or missing sync timestamps, enhancing reliability.
2025-01-09 10:51:17 +01:00
Jason Wen
04ddb10c1d Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#544) 2025-01-09 00:11:52 -05:00
Jason Wen
8edeb458d4 Merge branch 'openpilot/master' into sync-20250108
# Conflicts:
#	common/base.py
#	common/comma_connect.py
#	opendbc_repo
#	panda
2025-01-08 23:42:52 -05:00
Jason Wen
ec044790fb Revert "Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new" (#543)
Revert "Sync: `commaai/openpilot:master` into `sunnypilot/sunnypilot:master-n…"

This reverts commit e682957101.
2025-01-08 23:37:41 -05:00
Jason Wen
1b92f6ba3b ui: add trips panel in settings (#542)
* ui: add trips panel in settings

* adjust ui preview clicks

* ci: add optional scroll parameter to setup functions in UI preview

* add trips to ui preview and scrolling for dev panel

* fix keyboard
2025-01-09 03:49:48 +00:00
Jason Wen
b434043db7 ci: add optional scroll parameter to setup functions in UI preview (#541) 2025-01-09 03:18:07 +00:00
Jason Wen
d0f11e59ad ui: deprecate RETURN_IF_SUNNYPILOT macro and cleanup (#540) 2025-01-09 02:49:55 +00:00
Jason Wen
1b7066eb12 ui: add drive stats and move comma Prime status on offroad home (#537)
ui: add drive stats to offroad home screen
2025-01-09 02:22:09 +00:00
Jason Wen
8ee28e4f79 ui: split offroad home in its own class (#538)
* ui: split offroad home in its own class

* fix
2025-01-08 20:15:53 -05:00
Jason Wen
19f3cfd3d7 ui: update experimental mode button settings index (#539) 2025-01-08 19:57:06 -05:00
Jason Wen
d6ec84c068 System: fix dynamic pathing for crash logs (#535) 2025-01-07 09:48:29 -05:00
Jason Wen
e682957101 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#533)
* cabana: enhance message heatmap visualization (#34239)

* enhance message heatmap visualization

* TODO

* improve log_factor

* typo

* bit_flip_counts

* Openpilot webcam support improved (#34215)

* control webcam with ENV vars

* WIP: actual instructions

* wording

* file no longer exists

* this is expected behavior, just untested

* more readable

* tested on fresh install

* wording tweaks

* explicit USE_WEBCAM toggle required

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* debug-ability improved

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* newline removed

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>

* Update metadrive wheel (#34292)

* test

* new wheel

* fix IR power scaling (#34293)

* fix IR power scaling

* Update system/hardware/tici/hardware.h

* replay: Update video immediately after seek when paused. (#34237)

replay: Update video immediately after seeking when paused.

Otherwise, if paused then have to resume playback for the video
frame to update and show the new location.

Implemented by temporarily un-pausing replay for a single
frame time.

* cabana: add live and time-window heatmap modes for enhanced signal analysis (#34296)

add live and time-window heatmap modes

* timed: diff against absolute value of timedelta (#34299)

* cabana: miscellaneous bug fixes and enhancements (#34297)

* toHexString

* use QToolBar

* fix incorrect groove rect

* limit CAN_MAX_DATA_BYTES

* add series type selector to chart toolbar

* dim inactive messages

* rename

* add help to chart

* cleanup

* cabana: real-time cursor and video frame sync for chart and video (#34301)

* sync cursor and thumbnail between chart and video

* Revert "replay: Update video immediately after seek when paused. (#34237)"

This reverts commit 3363881844.

* use thumbnails while scrubing

* draw alert

* no update on resume

* draw timestamp

* cleanup

* replay:  fix various synchronization and event handling issues (#34254)

fix various synchronization and event handling issues

* cabana: fix crash in live streaming mode by skipping thumbnail display (#34302)

resolve crash in live streaming mode

* bump panda

* [bot] Update Python packages (#34304)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* cleanup touch_replay (#34305)

mathematics

* uv from brew doesn't have self update

* Skip registration on newer devices (#34316)

* tici: fix cpp device type (#34315)

fix cpp

* Tinygrad upstream master (#34325)

Upstream master

* [bot] Update Python packages (#34320)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* cabana: fix missing transmitter after undoing DBC message removal (#34329)

fix missing transmitter after undoing DBC message removal

* Quick GC pass heading into 2025 (#34330)

* first pass

* bye bye snpe

* [bot] Update Python packages (#34334)

Update Python packages

Co-authored-by: Vehicle Researcher <user@comma.ai>

* Notre Dame model in tinygrad (#34324)

* release model: 6f23a03f-486b-4d3e-a314-19d149644c7c/700

* old style model in tinygrad

* fix desire

* tg hack

* 20Hz

* no gas probs

* No gas here

* better indexing

---------

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>

---------

Co-authored-by: Dean Lee <deanlee3@gmail.com>
Co-authored-by: Mike Busuttil <31480000+MikeBusuttil@users.noreply.github.com>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
Co-authored-by: Angus Gratton <gus@projectgus.com>
Co-authored-by: commaci-public <60409688+commaci-public@users.noreply.github.com>
Co-authored-by: Vehicle Researcher <user@comma.ai>
Co-authored-by: Harald Schäfer <harald.the.engineer@gmail.com>
Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2025-01-07 06:28:20 +00:00
Jason Wen
c13f159a77 Revert "models: retain SNPE runner support" (#529)
Revert "models: retain SNPE runner support (#522)"

This reverts commit f84d27c4ee.
2025-01-06 22:06:50 +00:00
DevTekVE
8970a7aa5b ui: prevent driving model change with offroad transition (#524)
* Update model manager logic and handle offroad transitions

Added is_onroad state tracking in SoftwarePanelSP to handle offroad transitions. Updated model manager conditions for improved bundle validation. Removed unnecessary clear operation for ModelManager_DownloadIndex during offroad transitions to optimize behavior.

* Using is_onroad softwarePanel

* Enable model label button only when conditions are met

Previously, the button's state update was misplaced, leading to potential issues with its interactive availability. The logic has been adjusted to ensure it is properly enabled or disabled based on onroad status and download progress. This change improves UX consistency and prevents unintended actions.

* Remove redundant setEnabled call for currentModelLblBtn

The setEnabled call was unnecessary as its functionality was not required in this context. Cleaning up this code improves readability and removes redundant operations. No changes to functionality or behavior were introduced.

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-05 18:58:10 -05:00
Jason Wen
f84d27c4ee models: retain SNPE runner support (#522)
* tinygrad with snpe

* force with snpe to validate

* fix path

* fix more paths

* Adjust modeld execution logic based on active model runner

Introduced a check to conditionally execute `modeld` based on the active model runner. Added support for distinguishing between SNPE and TinyGrad runners using new helper functions and updated `custom.capnp` definitions. This change optimizes process management by ensuring compatibility with the selected model runner.

* Refactor modeld process function checks.

Introduce `is_stock_model` to clarify logic and replace direct uses of `is_snpe_model` where the stock model condition is needed. Additionally, rename the duplicate "modeld" process in sunnyPilot to "modeld_snpe" for clarity and consistency.

* ignore tg

* fix process name

* ruff

* fix thneed paths

* mypy

* remove our own

* use upstream compile3

* fix thneed

* try this

* Revert "remove our own"

This reverts commit 1cf4f57502.

* try using compile2.py again

* add back symlink

* fix path

* more fix

* wrong path again

* Revert "wrong path again"

This reverts commit f5301c19d5.

* update

* hardcode path to our submodule

* force path

* try this

* fix file name

* try this

* again

* Revert "again"

This reverts commit 17c8cd7376.

* Revert "try this"

This reverts commit 767f78bbcf.

* Revert "fix file name"

This reverts commit 485eef68da.

* Revert "try this"

This reverts commit 41fef87680.

* Revert "force path"

This reverts commit 5c3b408937.

* Revert "hardcode path to our submodule"

This reverts commit 5ee1950b6f.

* Revert "update"

This reverts commit fb313bd7fb.

* Reapply "wrong path again"

This reverts commit 309639aeb3.

* Revert "wrong path again"

This reverts commit f5301c19d5.

* Revert "more fix"

This reverts commit 23dd423e78.

* Revert "fix path"

This reverts commit 75d338f2bd.

* Revert "add back symlink"

This reverts commit 9f71ad0b8a.

* Revert "try using compile2.py again"

This reverts commit 914117d2e1.

* Reapply "remove our own"

This reverts commit b1996377b3.

* don't even compile anymore

* need it for default snpe model

* add to lfs

* bring onnx back for sim

* must add this back

* need this

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-05 17:47:27 -05:00
DevTekVE
c12c22eac7 ui: onroad skeleton (#521)
* onroad init

* init model renderer

* Add default virtual destructors to HudRenderer and AnnotatedCameraWidget

This ensures proper cleanup of derived classes if they override these destructors. Adding default destructors promotes better memory safety and adheres to modern C++ best practices.

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-05 05:14:18 -05:00
DevTekVE
5b1bfc26db ui: add sunnylink settings and sidebar status (#503)
* Add Sunnylink integration for improved device communication

This commit introduces Sunnylink support, including modules for API interactions, device registration, logging, and uploader processes. Key changes involve adding Sunnylink-related components, such as sunnylinkd, manage_sunnylinkd, and associated utilities, along with seamless integration into process management.

* Refactor Sunnylink modules and update import paths

Standardize parameter handling in Sunnylink functions by initializing Params within functions as needed. Update imports to use fully-qualified paths for better clarity and consistency. Also, refactor logging messages for improved readability and maintainability.

* Add Sunnylink support and improve log handling

Introduced Sunnylink-specific functionality, including compression for oversized logs and platform-specific socket handling for macOS. Improved logging mechanisms, refactored log queue management, and fixed exception handling in sunnylinkd.

* Refactor and fix minor coding style inconsistencies

Remove unnecessary string concatenation, adjust spacing for better readability, and ensure cleaner code in `athenad.py` and `sunnylink.py`. Added a macOS-specific comment for TCP_KEEPALIVE configuration to improve code clarity.

* Replace platform system check with sys platform in athenad.py

To check for macOS platform, the code in athenad.py has been altered. Originally, the platform.system() function was used. However, the function has been replaced with sys.platform for a more consistent and preferable syntax. Particularly, this has been modified in the context of setting socket options.

* Apply suggestions from code review

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Simplify imports and reformat API function.

Removed unused `platform` import for cleanup in `athenad.py`. Improved readability of `api_get` in `__init__.py` by reformatting the long return statement into multiple lines.

* Adjust backoff logic and refactor API call formatting.

Introduce randomness to backoff calculation in Sunnylink API to reduce synchronization issues. Minor code refactoring improves readability in the API call logic.

* Refactor Sunnylink network check logic.

Removed hardware-based network check due to performance concerns and replaced it with a real-time device state monitoring loop. This improves efficiency and ensures accurate online status before proceeding with Sunnylink registration.

* Apply suggestions from code review

* `Refactor saveParams error handling and simplify logic`

Removed redundant try-except block wrapping the entire method for clarity. Moved error logging directly inside the loop to handle individual parameter exceptions more effectively. Simplified dictionary construction and improved error logging format.

* Add BACKUP flag to select persistent parameters

This commit introduces a new BACKUP flag and applies it to specific persistent parameters in `params.cc` and `params.h`. The BACKUP flag enhances data retention by designating parameters for inclusion in backups, ensuring crucial information is preserved across sessions.

* Simplify Sunnypilot params formatting

Removed unnecessary blank lines and adjusted the Sunnypilot comment format for better readability and consistency. No functional changes were made.

* SP: Move Sunnypilot-related code to sunnypilot/sunnylink (#504)

* Refactor and relocate sunnylink-related modules

sunnylink components have been reorganized for better modularity and clarity, with files moved under `sunnypilot/sunnylink`. Unused code was removed, and reusable utilities were separated for easier maintenance. Adjusted references across the project to reflect these changes.

* Permissions

* adding init py

* Add sunnylink toggle to developer panel and translations

This commit introduces a new toggle for enabling or disabling sunnylink in the developer panel. Corresponding translation entries have been added for all supported languages to ensure compatibility across the UI.

* Add SunnyLink integration and multi-language support updates

Enhanced SidebarSP with SunnyLink connection status and temperature display. Extended translations for multiple languages, including new strings. Updated build scripts and added utility functions for SunnyPilot-specific features.

* Need it this way as it's intentionally shortened. Sorry

* format

* only block drawing

* format

* format

* fix path

* cleanup translations

* sunnylink panel

* offroad transition

* remove stretch

* add panel to ui preview

* Translating to spanish

* just reorder params

* Refactor sidebar drawing method names and remove unused code.

Renamed `paintSidebar` to `drawSidebar` for better clarity across both `Sidebar` and `SidebarSP` classes. Removed unused utility functions `drawRoundedRect` and `interpColor` to streamline the codebase and improve maintainability.

* Updating translations

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-05 10:47:28 +01:00
DevTekVE
0b1d97fc72 sunnylink support (#499)
* Add Sunnylink integration for improved device communication

This commit introduces Sunnylink support, including modules for API interactions, device registration, logging, and uploader processes. Key changes involve adding Sunnylink-related components, such as sunnylinkd, manage_sunnylinkd, and associated utilities, along with seamless integration into process management.

* Refactor Sunnylink modules and update import paths

Standardize parameter handling in Sunnylink functions by initializing Params within functions as needed. Update imports to use fully-qualified paths for better clarity and consistency. Also, refactor logging messages for improved readability and maintainability.

* Add Sunnylink support and improve log handling

Introduced Sunnylink-specific functionality, including compression for oversized logs and platform-specific socket handling for macOS. Improved logging mechanisms, refactored log queue management, and fixed exception handling in sunnylinkd.

* Refactor and fix minor coding style inconsistencies

Remove unnecessary string concatenation, adjust spacing for better readability, and ensure cleaner code in `athenad.py` and `sunnylink.py`. Added a macOS-specific comment for TCP_KEEPALIVE configuration to improve code clarity.

* Replace platform system check with sys platform in athenad.py

To check for macOS platform, the code in athenad.py has been altered. Originally, the platform.system() function was used. However, the function has been replaced with sys.platform for a more consistent and preferable syntax. Particularly, this has been modified in the context of setting socket options.

* Apply suggestions from code review

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Simplify imports and reformat API function.

Removed unused `platform` import for cleanup in `athenad.py`. Improved readability of `api_get` in `__init__.py` by reformatting the long return statement into multiple lines.

* Adjust backoff logic and refactor API call formatting.

Introduce randomness to backoff calculation in Sunnylink API to reduce synchronization issues. Minor code refactoring improves readability in the API call logic.

* Refactor Sunnylink network check logic.

Removed hardware-based network check due to performance concerns and replaced it with a real-time device state monitoring loop. This improves efficiency and ensures accurate online status before proceeding with Sunnylink registration.

* Apply suggestions from code review

* `Refactor saveParams error handling and simplify logic`

Removed redundant try-except block wrapping the entire method for clarity. Moved error logging directly inside the loop to handle individual parameter exceptions more effectively. Simplified dictionary construction and improved error logging format.

* Add BACKUP flag to select persistent parameters

This commit introduces a new BACKUP flag and applies it to specific persistent parameters in `params.cc` and `params.h`. The BACKUP flag enhances data retention by designating parameters for inclusion in backups, ensuring crucial information is preserved across sessions.

* Simplify Sunnypilot params formatting

Removed unnecessary blank lines and adjusted the Sunnypilot comment format for better readability and consistency. No functional changes were made.

* SP: Move Sunnypilot-related code to sunnypilot/sunnylink (#504)

* Refactor and relocate sunnylink-related modules

sunnylink components have been reorganized for better modularity and clarity, with files moved under `sunnypilot/sunnylink`. Unused code was removed, and reusable utilities were separated for easier maintenance. Adjusted references across the project to reflect these changes.

* Permissions

* adding init py

* more

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-05 02:27:44 -05:00
Jason Wen
9c58fa1020 ui: remove stretch in sunnypilot panel (#520)
* no need to stretch for listwidget

* button moved
2025-01-05 01:59:30 -05:00
DevTekVE
ee5c1c4507 Modular Assistive Driving System (MADS) (#446)
* allow re-regage

* bump opendbc

* bump panda

* apply pause/resume fix for hyundai (should do this in a separate PR)

* bump opendbc

* fix

* rename

* Fix?

* make sure to disengage for allow always cars

* fix

* combine

* more fix

* not needed

* check if engagement is from openpilot's state machine

* Rename

* fix panda safety

* fix

* no fake lfa button for @devtekve ;)

* fix non drive gear re-engage

* fix settings

* combine

* add replace method

* use replace

* remoev already checks if it exists

* fix

* group

* add todo

* reserve events

* cleaner

* hyundai: only allow for cars with lfa button

* sunnyParams

* make sure it's car only

* Move car-specific changes to opendbc

* no need

* bump opendbc

* more fixes

* no more available

* more!

* final?

* always emit user disable

* no longer needed

* move unit test

* add sunnypilot to unit tests

* bump opendbc

* use new cereal

* bump opendbc

* static analysis

* no unittest

* no need available

* UI border update

* show MADS updates

* Add TODO

* no longer needed

* fix changed events

* fix cluster enabled

* don't add pre enable if not long

* should use enabled

* enabled <-> active

* better format

* bump opendbc

* static analysis

* static analysis

* Rename test as collector was dying

* Show our overriding

* Revert "show MADS updates"

This reverts commit daf0ad62

Revert "fix changed events"

This reverts commit 31d8c97f

* ignoring reserved events

* adjusting creation delays

* back to stock

removing allow_cancel

* should be enabled

* revert

* silent lkas disable

* no need

* user disable tests

* just warning

* MUST REMOVE test process replay

* fix no entry

* fixme

* bump opendbc

* need this check

* cleanup

* allow entering paused state if no entry from disabled

* brake hold should apply to all

* in lists

* update unit test

* simpler

* unused

* same thing

* fix

* only mads in enabled state and long in disabled state

* unify silent enable

* do this for dlob

* bump submodules

* fix

* bump submodules

* bump opendbc

* less frequent

* more events

* fix

* allow no entry to paused for non-drive gears

* fix

* use cereal

* Revert "allow no entry to paused for non-drive gears"

This reverts commit 6d64a4dd9c.

* allow in all

* Revert "allow in all"

This reverts commit 6375f14891.

* should not be all!

* rename for clarity

* silent park brake

* flipped

* bump submodules

* Bump to latest mads-new panda

* bump panda

* more nissan

* bump panda

* bump msgq

* bump panda

* bump submodules

* bump opendbc

* bump opendbc

* improving the state

* Revert "PlayStation® model (#34133)"

This reverts commit 5160bee543.

* should be none

* bump panda

* bump opendbc

* Apply suggestions from code review

* bump panda

* bump ref panda

* add todo-sp

* bump panda ref

* bump more panda

* changing refs

* nuke nuke nuke

* use sunny's newer states

* bump with new panda

* bump panda

* Parse more flags from alt exp, more tests, hyundai main cruise allowed

* Parse more flags from alt exp, more tests, hyundai main cruise allowed

* missed

* mutation for controls allowed rising edge

* ford mutation

* license

* remove

* unused

* bump submodules

* use always allowed mads button alt exp

* fix

* whitelist jason's lastname to codespell

* test_processes: update ref logs to 82c0278

* bump submodules

* bump submodules

* bump submodules

* bump panda

* add controls mismatch lateral event

* Simplify lateral disengagement logic for MADS configuration

Reversed the conditional to align the logic with the `disengage_lateral_on_brake` parameter. This ensures that lateral disengagement behavior is more intuitive and matches the expected configuration. Improves code readability and reduces potential misconfigurations.

* remove unified engagement mode in panda

* controls allow should be allowed at all times

* squash! treat MADS button as user entry

* heartbeat for mads

* heartbeat mismatch exit control

* remove always allow mads button from alt

* move to safety_mads

* remove main cruise allowed from alt

* bump panda

* heartbeat engaged mads mismatch mutation test

* bump panda

* use mads the third panda

* ignore pre enable with mads

* only force exit if actually actuating

* use brake signal instead of pedal events when dlob is active

* fix tests

* fix panda tests

* bump panda

* new events to retain long blocks

* format

* uem: do not engage mads if long is engaged

* bump submodules

* fix not allowed engaged bug

* block uem from engaging

* flipped

* use different heartbeat check if dlob

* hard code to skip heartbeat check

* remove toyota lta status for lkas, causes weird behaviors

* block tesla

* bump panda

* bump to merged panda

* bump opendbc

* bump opendbc

* bump opendbc

* bump opendbc

* Apply suggestions from code review

* code ignore spells

* needs to be in carstate

* Bump opendbc

* Update MADS toggle descriptions for clarity.

Added notes to clarify behavior of the "MadsMainCruiseAllowed" setting, particularly its impact on vehicles without LFA/LKAS buttons. This ensures users are informed about potential implications when disabling this feature.

* Updating translations + Adding spanish

* Disengage Lateral on Brake -> Pause Lateral on Brake

* test_processes: update ref logs to dd41005

* Apply suggestions from code review

* fix mads button not allowed

* bump submodules

* bump submodule

* test_processes: update ref logs to 0a0b998

* has multiple lists

* Revert "has multiple lists"

This reverts commit a37c1d26fe.

* base

* Reapply "has multiple lists"

This reverts commit d1cd8dcc81.

* migrate mads toggles to sp panel

* this is why it keeps crashing

* house keeping

* more housekeeping

* more housekeeping

* don't show description by default (yet)

* reset to main panel when clicked away

* more

* some more with interactions

* don't stretch cause it looks weird with descriptions

* simpler to handle offroad transition

* some are toggleable while onroad

* remove unused event

* slight cleanup

* default to true for HKG main cruise toggle

* append to list after

* add Customize MADS to UI preview

* simpler

* move to sp list

* how tf was this removed

* update mads settings button on show event

* test_processes: update ref logs to efa9c32

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-04 20:48:02 -05:00
DevTekVE
993c4a0d2a Driving Model Manager (#457)
* Introduce Model Manager to handle downloads and verification

This commit introduces a new Model Manager responsible for handling model downloads, including driving and navigation application models. The manager also verifies file hashes and communicates download progress for an improved user experience.

The Model Manager is asynchronous and utilizes asyncio and aiohttp for enhanced performance, including robust error handling.

Impacted files in the 'cereal', 'common', 'sunnypilot', and 'system' directories have been updated accordingly. The 'ModelsFetcher' process configuration has been modified to run only when off-road, ensuring optimum resource management.

This update aims to enhance code clarity, improve performance, and streamline the handling of model downloads.

* "Update model management and fetching for SunnyPilot"

This update refactors the model management, downloading and cache verification mechanisms of SunnyPilot. New functionalities, such as smart cache handling, have been implemented in ModelFetcher in sunnypilot/models/model_fetcher.py. Also, the model downloading process has been moved to a separate async function _download_bundle in ModelManagerSP in sunnypilot/models/model_manager.py. Hash verification of files is now performed in an async function verify_file in sunnypilot/models/model_helper.py. Changes in system parameters related to model management have been reflected in system/manager/manager.py.

* Integrate download ETA calculations in model manager

This commit introduces a new feature that calculates and tracks the Estimated Time of Arrival (ETA) for downloading models in the model manager component. The 'eta' property in the 'DownloadProgress' structure in 'custom.capnp' is changed from 'Float32' to 'UInt32'.

In the 'model_manager.py' file, a new method '_calculate_eta' has been added to perform ETA calculations. An additional dictionary '_download_start_times' has been created to keep track of the start time of each model download. The ETA is calculated every time a portion of the model file is downloaded, and it gets updated in the 'DownloadProgress' structure. Finally, the start time is cleared after the download completes.

In 'model_manager_audit.py', an additional check is added to only print downloadProgress for the downloads currently in progress.

* format

* no default model cache {} because it can be considered a valid json, we do not want that

* Refactor typing annotations to use PEP 604 syntax.

Updated type hints to adopt PEP 604 union syntax (`X | None`) and replaced `List` and `Dict` with modern built-in `list` and `dict`. This change improves consistency and readability while aligning with Python 3.10+ standards.

* Simplify logging messages and remove unused imports.

Removed an unused import from `model_manager.py` to improve clarity and maintainability. Also refined a log message in `model_fetcher.py` by removing unnecessary formatting for consistency.

* Refactor model handling and simplify cache fallback logic.

Updated type annotations for `selected_bundle` in `model_manager.py` for clarity. Streamlined cache fallback logic in `model_fetcher.py` by removing redundant conditionals while preserving functionality. These changes improve code readability and maintainability.

* "Fix formatting for ModelManager_DownloadIndex retrieval

Condensed parameter alignment in the get method for improved readability and adherence to style guidelines. This change does not affect functionality but ensures consistent code formatting."

* Need to have main defined for process_config to be able to run it

* Refactor model management to support active bundle tracking

Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.

* UI commit (#515)

* Refactor model management to support active bundle tracking

Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.

* Add new driving model selection feature to settings

This commit introduces a new feature to the settings that allows users to select different driving models. It fetches available models and displays their download progress. The created UI also suggests a calibration reset after model download. The changes include the creation of 'SoftwarePanelSP' within 'settings.' Additionally, 'sunnypilot/SConscript' has been updated to include 'settings.cc' and 'software_panel.cc'. Changes also include localization for this feature.

* Show model description during download status

This update ensures the model description is displayed when a model is in the downloading state. It improves the user interface by providing real-time feedback during the download process.

* Update translations for multiple languages

Added new and updated translation strings in several language files, including Spanish, Arabic, Chinese (Simplified and Traditional), Turkish, Korean, Thai, Japanese, and Brazilian Portuguese. These updates include placeholder translations for new UI elements and features.

* Refactor model name handling and add generation check.

Replaced `bundleName` with `model_name` for better clarity in status messages. Added a generation mismatch check before showing the reset parameters dialog to avoid unnecessary prompts.

* Update model handling in SoftwarePanelSP

Remove unused "common/model.h" and replace "CURRENT_MODEL" with "..." as the default return value in GetModelName. Adjust logic to check for active bundles instead of selected bundles for improved accuracy. Minor text change for clarity in UI label.

* Rename `GetModelName` to `GetActiveModelName` for clarity.

The new name better reflects the function's purpose of retrieving the active model name, improving code readability. All relevant calls and references have been updated to ensure consistency across the codebase.

* Update sunnypilot/models/model_helper.py

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* Refactor download status handling and add 'cached' state

Introduce 'cached' as a new download status and adjust relevant logic across components to support it. Simplify and streamline model status handling in the software panel for better readability and maintainability. Ensure consistent status reporting for all model types.

* Update translations for multiple languages

Refined and expanded translations across various languages, replacing placeholders with meaningful text. This improves clarity and user experience in the multilingual interface.

* Update terminology from 'bundle' to 'model' in UI texts

Replaced occurrences of 'bundle' with 'model' in button labels, dialog titles, and messages in the SoftwarePanelSP code. This improves clarity and aligns terminology with current functionality.

* Update translation placeholders for model fetching texts

Replaced "Fetching bundles" with "Fetching models" across multiple languages to align text placeholders with the updated functionality. Adjusted related background download messages for clarity and consistency.

* cleanup

* not used, and likely not needed

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

* cleaning up

* Update system/manager/process_config.py

* Simplify model parsing and index handling logic.

Refactored `ModelManager_DownloadIndex` retrieval to use the walrus operator, streamlining the conditional logic. Additionally, restructured model list initialization in `_parse_bundle` for improved readability and maintainability. These changes enhance code clarity and reduce redundancy.

* `Improve error handling in model cache retrieval`

Revised the `get` method to ensure it returns an empty dictionary on errors or missing data, avoiding potential `None`-related issues. Added logging for clearer diagnostics when cached model data is unavailable or retrieval fails. This improves reliability and debuggability of the model fetching process.

* Fix cached model data handling by parsing JSON response

Previously, cached model data was returned as a raw string, causing potential issues when using the data. The change ensures the cached data is properly parsed into JSON format before returning, improving reliability and consistency.

* Adjust modelManagerSP rate and Ratekeeper frequency

Reduced the rate for modelManagerSP in services and aligned the Ratekeeper frequency in model_manager.py to 0.1.

* Update model fetcher URL and adjust modelManagerSP rate

Updated the model fetcher URL to point to the correct resource for driving models. Adjusted the rate of modelManagerSP in both its service definition and the corresponding Ratekeeper initialization to 1 Hz for improved consistency.

* Refactor model download logic for clarity and efficiency

Simplify the logic for finding the model to download by combining redundant constructs into a single line. This improves code readability and reduces unnecessary variable assignments.

* Fix cache keys for manual prebuilt actions because they were missing the cache when manually built

* no need to log

* formatting

* revert ci changes

* Refactor and restructure `modeld` to `models` module.

Renamed `modeld` directory to `models` for clarity and consistency. Updated all references and imports to reflect the new structure. This improves maintainability and aligns with naming conventions.

---------

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-04 15:05:21 -05:00
DevTekVE
63f0237e7a prebuilt: Squash and merge script (#497)
* Add script for squash and merge workflow automation

Introduced `squash_and_merge.py` to automate the process of squashing a source branch and merging it into a target branch. The script handles backups, commit message creation, stashing, cleanup, and includes optional push functionality. Provides user prompts and error handling to simplify complex merge workflows.

* temp1

* Switch to PEP 604 style for optional and generic type hints

Replaces `Optional` and `List` with the simpler `X | None` and `list[X]` syntax introduced in Python 3.10. Updated all function signatures accordingly for consistency and modernity. Also made minor string formatting consistency adjustments.

* Fix type annotation for temp_branch variable

Updated the temp_branch variable to include a proper type annotation (`str | None`). This change improves code clarity and aligns with Python typing standards.

* more hints
2025-01-04 09:43:03 +01:00
Jason Wen
a3ab6db7eb ui: sunnypilot panel in settings (#513)
* ui: sunnypilot panel in settings

* add panel to comment

* test populate screenshot

* Revert "test populate screenshot"

This reverts commit 426b6c26c5.
2025-01-03 17:18:38 -05:00
Jason Wen
3a64efe52f Bump submodules 2025-01-03 15:07:10 -05:00
Jason Wen
3966599e9d ui: sunnypilot offroad UI (#512)
* Revert "Revert "ui: sunnypilot offroad UI" (#511)"

This reverts commit 0e264e1b05.

* Revert "move files to sp dir"

This reverts commit c72d732259.

* remove drive stats for now

* update translation

* update onboarding

* remove sp onboarding for now

* rearrange

* remove more

* shorter license
2025-01-03 15:04:09 -05:00
Jason Wen
0e264e1b05 Revert "ui: sunnypilot offroad UI" (#511)
Revert "ui: sunnypilot offroad UI (#500)"

This reverts commit 3717a111af.
2025-01-03 11:02:19 -05:00
Jason Wen
3717a111af ui: sunnypilot offroad UI (#500)
* add sp flag

* return if sp macro

* controls

* drive stats

* some more

* more

* skip the whole thing

* missed

* more

* more main

* not ready to push yet but sure

* get them icons

* later

* own icons

* Revert "own icons"

This reverts commit e07bd8e670.

* blank icon

* render differently on mac

* static

* set toggle and title space properly

* format

* remove prior objects

* good spacing for ButtonParamControlSP

* more space formatting

* on device fix

* update home wifi prompt

* handle AbstractControlSP_SELECTOR hide events properly

* use sp

* ignore name

* more

* keep spacing

* better flag

* settings button moved

* small cleanup

* move files to sp dir

* remove for now

* developer different icon

* add scrolling to ui test

* Revert "add scrolling to ui test"

This reverts commit c8d1c65d89.

* format

* make them virtual

* do we still need this?

* Apply suggestions from code review

Co-authored-by: DevTekVE <devtekve@gmail.com>

* revert for now

* shorter license

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2025-01-03 09:36:36 -05:00
DevTekVE
262d17ead2 Tools: Setup CLion run configs and external tools (#486)
* Add CLion run configs and external tools setup

Introduced CLion configuration files for "Build Debug" and "Build Release," along with external tool definitions for Poetry SCons commands. Additionally, commented out `.idea` in `.gitignore` to track necessary project-specific settings.

* rename

* only exclude specific files from gitignore

* wildcard

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2024-12-24 01:14:06 -05:00
DevTekVE
0bfa9fca88 SP: GitHub runner as service and able to toggle via developer panel (#494)
* Add support for GitHub Actions runner management

Introduce a new process and configuration to manage a GitHub Actions runner. Added a persistent "EnableGithubRunner" parameter and a script to control the runner service. Integrated the feature into the system's process manager logic.

* Restrict GitHub runner usage on metered networks.

This update modifies the `use_github_runner` function to include a check for metered networks using `HARDWARE.get_network_metered()`. This ensures the GitHub runner is not enabled when the network is metered, improving network usage efficiency.

* Add GitHub runner service toggle to developer panel

Introduces a new toggle in the developer panel to enable or disable the GitHub runner service. This provides developers with a convenient way to control the service from the UI.

* translations

* Refactor network condition check for GitHub runner.

Updated `use_github_runner` to include network type in metered network checks. This ensures more accurate validation by considering specific network types when determining metered status.

* Mark as executable

* Update paths and shebangs for consistency across scripts

Modified file paths to align with the new directory structure under `/data/media/0/github` and updated the shebang in `github_runner.sh` for better environment compatibility. Adjusted the `BUILD_DIR` in the GitHub workflow to reflect the new path.

* Fix string translation for GitHub runner parameter text

Added translation support for the GitHub runner service description text. This ensures consistent localization across the UI.

* Remove gitlab_runner.sh from Sunnypilot blacklist.

The script is no longer required to be blacklisted, likely due to updates or changes in its usage. This improves the maintainability of the blacklist by removing unnecessary entries.

* lang stuff

* Update BASE_DIR determination based on mount point

Refactored scripts to dynamically set BASE_DIR depending on whether /data/media is a valid mount point. This ensures compatibility with different environments and improves robustness of path resolution.

* Refactor GitHub runner logic in process_config.

Simplified enabling conditions for the GitHub runner by removing dependency on hardware network checks and adding a logical combination of offroad and runner-related functions. This improves code readability and reduces hardware coupling.
2024-12-23 09:19:26 +01:00
DevTekVE
cd73feec57 ci: Update version identifier in workflow for staging and release (#496)
Update version identifier in workflow for staging and release

Appended "-staging" and "-release" to the VERSION variable in the GitHub Actions workflow to clearly distinguish build types. This ensures clarity in versioning for staging and release environments.
2024-12-23 01:47:41 +01:00
DevTekVE
bd49be185f ci: enable macos runners again (#495)
no need to block macos anymore. Turns out runners are free on github public repos
2024-12-23 01:03:21 +01:00
DevTekVE
37e493ce0d CI: Add GitHub Action to build models from upstream (#491)
* Add support for custom model naming in workflow

This update introduces a new optional input `custom_name` to the `sunnypilot-build-model` GitHub workflow. If provided, it customizes the artifact name; otherwise, the default naming scheme is used. This enhances flexibility for build artifact identification.

* fix name

* Update cache key logic in build workflow

Replaced `github.ref_name` with a fallback to `github.head_ref || github.ref_name` in cache keys to handle branches and pull requests more reliably. This ensures consistent cache retrieval and improves workflow efficiency.

* Simplify artifact upload logic in CI workflow

Consolidated redundant artifact upload steps into a single action, leveraging a fallback mechanism for naming. This streamlines the workflow, reducing duplication and improving maintainability.

* Make publish step resilient and add conditional notify

Added `continue-on-error` to the publish step to allow workflow progression even if it fails. Introduced a condition to notify step that depends on publish outcome being successful. These changes improve workflow reliability and error handling.

* Fix conditional syntax in GitHub Actions workflow

Updated the conditional statement in the notify job to use the correct GitHub Actions expression syntax. This ensures proper evaluation of the publish step's success outcome.

* Update condition for notify step in GitHub Actions workflow

Replaced the deprecated syntax `${{ steps.publish.outcome == 'success' }}` with the recommended `success()` function. This ensures compatibility with newer GitHub Actions features and improves maintainability.

* Update build workflow to handle PR drafts and improve concurrency

Added support for pull request draft events to trigger the workflow, ensuring drafts are not missed. Removed `continue-on-error` in the publish step to improve reliability and prevent silent failures. Simplifies and enhances workflow robustness.

* Update pull request workflows and add draft check logic

Removed unnecessary pull request event types and drafts settings, consolidating logic. Added a condition to skip publishing for non-draft pull requests, ensuring better control over the deployment process.
2024-12-23 00:37:02 +01:00
DevTekVE
e114dc5a6f CI: Use environments for manual approvals on GH Actions for feature branches (#493)
* We no longer need to dispatch manually a workflow, it will wait for the publish to be approved for feature branches.
* The notification template has been extracted to a variable for the repo, meaning no need to do commit push to update it! (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/DISCORD_GENERAL_UPDATE_NOTICE)
* The configuration for what can be auto deployed is now also a variable, no need to commit a change to update that (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/AUTO_DEPLOY_PREBUILT_BRANCHES)
* The configuration for what sends a message on `#dev-feedback` channel on discord is also configurable (https://github.com/sunnypilot/sunnypilot/settings/variables/actions/DEV_FEEDBACK_NOTIFICATION_BRANCHES)
2024-12-22 18:18:21 +01:00
DevTekVE
ad537fcb89 CI: Build prebuilts and models from github and self hosted runners (#476)
* Create a GitHub Actions workflow for synchronizing the repository to GitLab and enhance GitLab CI settings

A new GitHub Actions workflow for mirroring the current repository to GitLab is added. This workflow is triggered by both push and delete events in addition to manual triggering. The role of GitHub actions runner is defined through a series of steps. A new '.gitlab-ci.yml' build configuration file is also introduced, with a more comprehensive definition of variables, jobs, and pipeline rules for its utilization in the GitLab CI/CD pipeline.

Further, other changes include the addition of scripts for installing and uninstalling GitLab CI runner, along with modifying the SCons build system configuration file to include custom cache directory. Moreover, 'release_files.py' has been revised to include additional blacklisted and whitelisted files specific to Sunnypilot, ensuring suitable settings for the CI flow.

The improvements facilitate a smoother integration between GitHub and GitLab, powerfully harnessing the capabilities of both platforms for more efficient and effective CI/CD pipelines and version control management.

* not needed for this

* Update workflow to build model from upstream repository

Revised the CI workflow to build directly from the upstream `commaai/openpilot` repository. Simplified configuration, removed unused steps, and added support for specifying the upstream branch dynamically via inputs.

* Update SConstruct to allow passing arbitrary cache_dir

Modified the SConstruct file to enable setting a custom cache directory via arguments. This enhances flexibility in configuring cache paths during the build process.

* Refactor build workflow to improve branch handling logic.

Reorganized conditions for setting environment variables, replacing repository_dispatch with workflow_dispatch for prebuilt builds. Added a fallback error message for unsupported triggers to improve robustness. This enhances clarity and ensures compatibility with the intended workflow triggers.

* test test test

* test test test

* test test test

* Enable publication flag during build configurations

Added `SHOULD_PUBLISH=true` to all relevant build configuration steps to ensure proper handling of publishing logic. Updated environment variables to include this flag for downstream usage. Removed the error-check step for unsupported configurations.

* Simplify publish condition in workflow logic

Replaced the previous condition for publishing with a single output variable, `should_publish`, to streamline logic and improve maintainability. This change reduces redundancy and makes the workflow more adaptable to future updates.

* Simplify restore key patterns in build workflow.

Removed unnecessary trailing dashes in SCons cache restore keys to streamline and slightly improve key matching logic. This ensures better consistency with the current workflow setup.

* Update cache key usage in build workflow

Replaced `github.ref_name` with `github.head_ref` for cache keys to ensure accurate branch-specific caching. Added fallback restore-keys for master branches to improve cache efficiency and reduce redundant builds.

* Improved debug logging in GitHub actions workflow

This commit refines the debug logging in our GitHub Actions workflow for the Sunnypilot build. This change provides more granularity, enabling logging only during debug mode, which helps to keep the runtime logs less cluttered during normal operations. This includes the conditionally displaying of environmental variables, GitHub output contents, and directory listings.

Additionally, debug mode verbosity was added to rsync commands to aid troubleshooting file transfers during the build process.

Blank lines were also reduced for better readability and cleaner code presentation.

* test diff path

* Refactor SCons cache handling in build workflow

Replaced hardcoded cache directory paths with an environment variable (`SCONS_CACHE_DIR`) for better maintainability and flexibility. Updated related workflow steps to utilize the new variable and adjusted cache key usage. Removed unused `BASE_BUILD_NUMBER` variable to clean up the configuration.

* Update cache key to include commit SHA in workflow

This change adds the commit SHA to the cache key in the GitHub Actions workflow. It ensures more precise caching by differentiating builds based on the specific commit, reducing potential conflicts.

* clean

Update GitHub runner service to set environment variables

Updated the ExecStart command to explicitly set HOME, USER, LOGNAME, and MAIL environment variables. This ensures the runner operates with the correct environment configuration, improving reliability and compatibility.

Refactor GitHub runner service ExecStart command.

Replaced direct command execution with running the service as a specific user using `su`. This improves compatibility and aligns with best practices for user-based execution. No functional changes are expected.

Update GitHub runner service to set environment variables

The ExecStart command now sets HOME, USER, LOGNAME, and MAIL environment variables for the runner process. This ensures proper environment initialization for the designated user, improving compatibility and reliability during execution.

Refactor GitHub runner service template handling.

Revised the `modify_service_template` function to create a properly structured service template for the GitHub runner. Updated service permissions, execution parameters, and enabled the function call to ensure usage during runner setup.

* Refactor GitHub Runner installer to improve argument parsing.

Reworked the script to implement flexible and explicit command-line argument parsing using flags like `--token`, `--repo`, and `--start-at-boot`. Added support for setting default values and enabling/disabling auto-start based on the `--start-at-boot` flag. Improved error handling and usage messaging for better user experience.

* Update cache keys in sunnypilot build workflow

Modified the cache keys to include `github.ref_name` for more precise caching and restore behavior. This improves build consistency by better differentiating between branches and refs. No changes to the overall workflow logic.

* Remove 'tinygrad/*' from release file exclusions

This change modifies the release file exclusions by removing 'tinygrad/*'. The adjustment ensures that files in the 'tinygrad' directory are now included in the release process, aligning with updated packaging requirements.

* Simplify release file exclusions list.

Removed redundant and unnecessary file patterns from the exclusions list in `release_files.py`. This streamlines the file handling process and reduces maintenance overhead.

* Refactor SCons cache path and key structure.

Updated the SCons cache directory path to `SCONS_CACHE_DIR` for clarity and consistency. Improved the caching key structure to include `github.head_ref` for better cache differentiation and restore hierarchy. Adjusted related build instructions to reflect the new environment variable.

* Refactor branch configuration in CI workflow

Standardize branch name handling by replacing hardcoded values with environment variables. This improves maintainability and simplifies updates to branch names across the workflow. Updated references to use the new dynamic environment variable approach.
2024-12-21 20:19:48 +01:00
Jason Wen
cf74e6416c Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#488) 2024-12-19 10:57:30 -05:00
devtekve
7592669d74 test_processes: update ref logs to a20d7a4 2024-12-19 15:46:00 +00:00
DevTekVE
a20d7a4279 Bump panda for sync 2024-12-19 16:30:02 +01:00
DevTekVE
e4fc6ffe7a Merge remote-tracking branch 'comma/master' into sync-20241219-3
# Conflicts:
#	panda
#	selfdrive/test/process_replay/ref_commit
2024-12-19 16:29:02 +01:00
DevTekVE
cf30110d65 CI: cherry pick maxime's fix for pipelines docker (#487)
set python upper bound to 3.13 (#34286)

* try

* test

* ...

* wow

* y

* docker

* <

(cherry picked from commit 9c9b273a3e)

Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-12-19 15:50:20 +01:00
DevTekVE
a48d43ec2b CI: adding recurring sync to lfs (#489)
Allows us to automatically keep our LFS in sync with comma's and also to manually perform a sync if we need to. Even able to sync the LFS from a given commit hash or a given branch. Useful for model stuff.
2024-12-19 15:32:40 +01:00
Jason Wen
c8fe86c552 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#484) 2024-12-17 07:04:03 -05:00
Jason Wen
952354c847 test_processes: update ref logs to fffa98e 2024-12-17 06:49:31 -05:00
Jason Wen
fffa98ee85 Merge branch 'upstream/master' into sync-20241712
# Conflicts:
#	.github/workflows/ci_weekly_run.yaml
#	opendbc_repo
#	panda
#	selfdrive/test/process_replay/ref_commit
2024-12-17 06:27:31 -05:00
Jason Wen
c69cd934af Setup sunnypilot package (#483)
Setup sunnypilot as a package
2024-12-16 19:47:29 -05:00
DevTekVE
437663ff98 CI: Fix process replay to properly work with local routes (#469)
Updating test_processes.py to easily upload ref routes locally to git instead of azure
2024-12-11 10:22:45 +01:00
Jason Wen
8b54fb8372 Revert "ci: Skip simulator failures" (#475)
Revert "ci: Skip simulator failures (#470)"

This reverts commit 705dd83a2f.
2024-12-08 20:26:35 -05:00
Jason Wen
0f74e0f760 Sync: commaai/openpilot:master into sunnypilot/openpilot:master-new (#474) 2024-12-08 20:12:29 -05:00
Jason Wen
4cccd07fd6 Merge branch 'upstream/master' into sync-20241208
# Conflicts:
#	.github/workflows/tools_tests.yaml
#	opendbc_repo
2024-12-08 19:40:20 -05:00
Jason Wen
b42c997f83 LFS: Sync refs from upstream (#472)
* Adding script to pull from comma's LFS before pushing to ours

* updating the script a little to allow pulling all when needed if needed

* static ..

* format

* Fuck the simulator always failing

* Apply suggestions from code review

* Apply suggestions from code review

---------

Co-authored-by: devtekve <devtekve@gmail.com>
2024-12-08 22:03:47 +01:00
Jason Wen
6aaa245e65 Revert "LFS: Sync refs from upstream" (#471)
Revert "LFS: Sync refs from upstream (#467)"

This reverts commit 8132fe9f0e.
2024-12-08 15:41:04 -05:00
DevTekVE
705dd83a2f ci: Skip simulator failures (#470)
bai sim for now
2024-12-08 15:11:38 -05:00
DevTekVE
8132fe9f0e LFS: Sync refs from upstream (#467)
* Adding script to pull from comma's LFS before pushing to ours

* updating the script a little to allow pulling all when needed if needed

* static ..

* format

* Fuck the simulator always failing

* Apply suggestions from code review

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2024-12-08 14:39:35 -05:00
DevTekVE
5dc5b6accb ci: Save cache for master-new (#468)
Adding master-new to the whitelisted refs for caching stuff
2024-12-08 13:33:32 -05:00
Jason Wen
cc1bfcf12e Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#465) 2024-12-05 18:12:23 -05:00
Jason Wen
bd6d207c8a Merge branch 'upstream/master' into sync-20241205 2024-12-05 17:48:01 -05:00
Jason Wen
777ff8dcb4 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#459) 2024-11-26 23:44:36 -05:00
Jason Wen
9446dd60d1 Merge branch 'upstream/master' into sync-20241126
# Conflicts:
#	opendbc_repo
2024-11-26 22:54:12 -05:00
Jason Wen
1deae82f12 Sync: commaai/openpilot:master into sunnypilot/sunnypilot:master-new (#458) 2024-11-23 12:28:14 -05:00
Jason Wen
0649653947 Merge branch 'upstream/master' into sync-20241122 2024-11-23 10:26:18 -05:00
DevTekVE
ab65b19ba5 Hyundai: Enhanced Smart Cruise Control (ESCC) (#443)
* initial updates to the actions to run for sp

* ignore license file

* more updates

* undoing some of the changes because I was blocking the runs on

* allowing the submodule check as well

* Allowing macos builds

* test adding cache key

* don't attempt build_release for selfdrive for the time being.

* Blocking macos builds as well since they have a 10x miltiplier for GH aciton minutes, waaaay too much!

* lol nice typo codespell

* change ref commit id to check if replay passes

* Sync up submodules for ESCC

* Remove ESCC base and interfaces classes

Consolidated ESCC functionalities directly into the Hyundai ESCC module, removing redundant base and interface classes. This simplified the class structure and improved code maintainability.

* Removing hints because they were causing a circular dependency

* bump opendbc

* bump opendbc

* Bump opendbc

* Fixing inconsistencies thanks to the tests!

* Bumping opendbc after fixing tests

* update opendbc

* Updating to the new flags ordinal on cereal

* Reverting some of the misc changes, clarifying naming and passing the scc12 and fca11 values to the methods as they are defined out of order

* undoing more naming changes to simplify diff

* More naming and tiny adjustments

* use constant for msg id

* bump opendbc

* bump submodules

* already added

* bump opendbc

* bump opendbc

* bump opendbc

* fix init

* bump submodules

* bump submodules

* unit test!

* rename

* always use true radar tracks if available

* update comment

* bump submodules

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2024-11-22 09:07:39 -05:00
DevTekVE
36f8192612 ci: sunnypilot UI GH Action fix ensure push to master-ui artifact (#453)
Fixes the GitHub action that was failing when the pipeline was running as a result of a push to master-new because it was not properly recognized as "master"
2024-11-17 09:49:37 +01:00
DevTekVE
4b66cd0577 ci: fix repo permissions in ui preview (#450)
* more fix

* forcing a change on UI to test

* undo ui change for test

* temp add branch

* adding more ways to set the master uo

* more bruteforce

* undoing force
2024-11-16 22:27:29 -05:00
DevTekVE
ad742515f1 ci: sunnypilot show UI diff on PRs (#449)
add master-new as well to the triggers for the diff ui
2024-11-16 23:45:12 +01:00
Jason Wen
2426354d72 bump opendbc (#448)
* bump opendbc

* Bump opendbc to our changed car params

* New Int16 test

* spFlags

* original name?

* Testing more carparams combinations

* Another crazy idea

* typo

* use this field

* bump

* bump opendbc

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
2024-11-16 17:04:12 -05:00
DevTekVE
cf55992c26 CI: sunnypilot configuration (#447)
* initial updates to the actions to run for sp

* ignore license file

* more updates

* undoing some of the changes because I was blocking the runs on

* allowing the submodule check as well

* Allowing macos builds

* test adding cache key

* don't attempt build_release for selfdrive for the time being.

* Blocking macos builds as well since they have a 10x miltiplier for GH aciton minutes, waaaay too much!

* lol nice typo codespell

* change ref commit id to check if replay passes
2024-11-16 10:24:47 -05:00
DevTekVE
3da0dfd47f Updating submodules 2024-11-15 08:16:52 +01:00
Jason Wen
6edaf619bf Add Custom MIT License (#438) 2024-11-15 08:11:51 +01:00
457 changed files with 34340 additions and 1717 deletions

3
.codespellignore Normal file
View File

@@ -0,0 +1,3 @@
Wen
REGIST
PullRequest

3
.github/CODEOWNERS vendored Normal file
View File

@@ -0,0 +1,3 @@
* @sunnypilot/dev-internal
/.github/ @devtekve @sunnyhaibin
/release/ci/ @devtekve @sunnyhaibin

View File

@@ -24,4 +24,4 @@ multilanguage:
autonomy:
- changed-files:
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit}"
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit,sunnypilot/modeld*/models/**}"

43
.github/release-drafter.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
exclude-labels:
- 'no-changelog'
categories:
- title: '🚀 Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
collapse-after: 5
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧰 Maintenance'
collapse-after: 5
label: 'chore'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
change-title-escapes: '\<*_&'
replacers:
- search: '/[Ss][Uu][Nn][Nn][Yy][Pp][Ii][Ll][Oo][Tt]/g'
replace: 'sunnypilot'
- search: '/\b[Ss][Pp]\b/g'
replace: 'SP'
version-resolver:
major:
labels:
- 'major'
minor:
labels:
- 'minor'
patch:
labels:
- 'patch'
default: patch
name-template: 'v$RESOLVED_VERSION 🚀'
tag-template: 'v$RESOLVED_VERSION'
version-template: "0.$MAJOR.$MINOR.$PATCH" # The day OP becomes v1, we need to bump this
tag-prefix: "v0." # The day OP becomes v1, we need to bump this
prerelease-identifier: "staging"
template: |
## Changes
$CHANGES

View File

@@ -24,7 +24,7 @@ jobs:
# Check PR target branch
- name: check branch
uses: Vankka/pr-target-branch-action@def32ec9d93514138d6ac0132ee62e120a72aed5
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
@@ -37,16 +37,16 @@ jobs:
# Welcome comment
- name: "First timers PR"
uses: actions/first-interaction@v1
if: github.event.pull_request.head.repo.full_name != 'commaai/openpilot'
if: github.event.pull_request.head.repo.full_name != 'sunnypilot/sunnypilot'
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
pr-message: |
<!-- _(run_id **${{ github.run_id }}**)_ -->
Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:
* Convert your PR to a draft unless it's ready to review
* Read the [contributing docs](https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md)
* Read the [contributing docs](https://github.com/sunnypilot/sunnypilot/blob/master/docs/CONTRIBUTING.md)
* Before marking as "ready for review", ensure:
* the goal is clearly stated in the description
* all the tests are passing
* the change is [something we merge](https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md#what-gets-merged)
* the change is [something we merge](https://github.com/sunnypilot/sunnypilot/blob/master/docs/CONTRIBUTING.md#what-gets-merged)
* include a route or your device' dongle ID if relevant

View File

@@ -13,7 +13,7 @@ jobs:
badges:
name: create badges
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
permissions:
contents: write
steps:
@@ -29,7 +29,7 @@ jobs:
git checkout --orphan badges
git rm -rf --cached .
git config user.email "badge-researcher@comma.ai"
git config user.email "badge-researcher@sunnypilot.ai"
git config user.name "Badge Researcher"
git add translation_badge.svg

View File

@@ -0,0 +1,77 @@
name: cereal validation
on:
push:
branches:
- master
- master-new
pull_request:
paths:
- 'cereal/**'
workflow_dispatch:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: cereal-validation-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
PYTHONWARNINGS: error
BASE_IMAGE: openpilot-base
BUILD: selfdrive/test/docker_build.sh base
RUN: docker run --shm-size 2G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e CI=1 -e PYTHONWARNINGS=error -e FILEREADER_CACHE=1 -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v $GITHUB_WORKSPACE/.ci_cache/scons_cache:/tmp/scons_cache -v $GITHUB_WORKSPACE/.ci_cache/comma_download_cache:/tmp/comma_download_cache -v $GITHUB_WORKSPACE/.ci_cache/openpilot_cache:/tmp/openpilot_cache $BASE_IMAGE /bin/bash -c
jobs:
generate_cereal_artifact:
name: Generate cereal validation artifacts
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc) cereal"
- name: Generate the log file
run: |
${{ env.RUN }} "cereal/messaging/tests/validate_sp_cereal_upstream.py -g -f schema_instances.bin" && \
ls -la
ls -la cereal/messaging/tests
- name: 'Prepare artifact'
run: |
mkdir -p "cereal/messaging/tests/cereal_validations"
cp cereal/messaging/tests/validate_sp_cereal_upstream.py "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py"
cp schema_instances.bin "cereal/messaging/tests/cereal_validations/schema_instances.bin"
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: cereal_validations
path: cereal/messaging/tests/cereal_validations
validate_cereal_with_upstream:
name: Validate cereal with Upstream
runs-on: ubuntu-24.04
needs: generate_cereal_artifact
steps:
- uses: actions/checkout@v4
with:
repository: 'commaai/openpilot'
submodules: true
ref: "refs/heads/master"
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc) cereal"
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: cereal_validations
path: cereal/messaging/tests/cereal_validations
- name: 'Run the validation'
run: |
chmod +x cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py
${{ env.RUN }} "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py -r -f cereal/messaging/tests/cereal_validations/schema_instances.bin"

View File

@@ -15,7 +15,7 @@ env:
jobs:
setup:
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
runs-on: ubuntu-latest
outputs:
ci_runs: ${{ steps.ci_runs_setup.outputs.matrix }}
@@ -31,7 +31,7 @@ jobs:
strategy:
fail-fast: false
matrix: ${{fromJSON(needs.setup.outputs.ci_runs)}}
uses: commaai/openpilot/.github/workflows/ci_weekly_run.yaml@master
uses: sunnypilot/sunnypilot/.github/workflows/ci_weekly_run.yaml@master
with:
run_number: ${{ matrix.run_number }}

View File

@@ -12,6 +12,6 @@ concurrency:
jobs:
selfdrive_tests:
uses: commaai/openpilot/.github/workflows/selfdrive_tests.yaml@master
uses: sunnypilot/sunnypilot/.github/workflows/selfdrive_tests.yaml@master
with:
run_number: ${{ inputs.run_number }}

View File

@@ -15,7 +15,7 @@ runs:
scons -j$(nproc) --cache-populate"
- name: Save scons cache
uses: actions/cache/save@v4
if: github.ref == 'refs/heads/master'
if: (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new')
with:
path: .ci_cache/scons_cache
key: scons-${{ runner.arch }}-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}

View File

@@ -35,13 +35,13 @@ jobs:
# Push to docs.comma.ai
- uses: actions/checkout@v4
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
with:
path: openpilot-docs
ssh-key: ${{ secrets.OPENPILOT_DOCS_KEY }}
repository: commaai/openpilot-docs
repository: sunnypilot/sunnypilot-docs
- name: Push
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
run: |
set -x

72
.github/workflows/lfs-maintenance.yaml vendored Normal file
View File

@@ -0,0 +1,72 @@
name: Sync comma's LFS
env:
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
on:
schedule:
- cron: '0 0 * * *' # Runs at 00:00 UTC every day
push:
branches:
- 'master-new'
pull_request:
branches:
- 'master-new'
workflow_dispatch: # enables manual triggering
inputs:
upstream_branch:
default: 'master'
type: string
jobs:
sync:
runs-on: ubuntu-latest
# Skip if PR is in draft mode
if: (github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false)) && !github.event.pull_request.head.repo.fork
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
repository: 'commaai/openpilot'
ref: ${{ inputs.upstream_branch }}
- name: LFS Fetch
run: |
git lfs fetch
- name: Set up Git
run: |
git config --global user.name 'GitHub Action'
git config --global user.email 'action@github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Ensure branch
run: |
if git symbolic-ref -q HEAD >/dev/null; then
echo "Already on a branch, proceeding with push"
else
echo "Detached HEAD state detected, creating temporary branch"
git checkout -b temp_branch
fi
- name: Update LFS Config
run: |
echo '[lfs]' > .lfsconfig
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
echo ' locksverify = false' >> .lfsconfig
- name: Push LFS
id: sync-and-commit
run: |
git lfs ls-files -l
git lfs push --all origin

View File

@@ -12,7 +12,7 @@ jobs:
build_prebuilt:
name: build prebuilt
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
env:
PUSH_IMAGE: true
permissions:

29
.github/workflows/release-drafter.yml vendored Normal file
View File

@@ -0,0 +1,29 @@
name: Release Drafter
on:
push:
branches:
- master-new
- master
tags:
- 'v*'
pull_request_target:
types: [opened, reopened, synchronize]
workflow_dispatch:
permissions:
contents: read
jobs:
update_release_draft:
permissions:
contents: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v6
with:
config-name: release-drafter.yml
prerelease: ${{ !startsWith(github.ref, 'refs/tags/v') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -13,7 +13,7 @@ jobs:
container:
image: ghcr.io/commaai/openpilot-base:latest
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
permissions:
checks: read
contents: write

View File

@@ -37,7 +37,7 @@ jobs:
runs-on: ubuntu-latest
container:
image: ghcr.io/commaai/openpilot-base:latest
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
steps:
- uses: actions/checkout@v4
with:

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- master
- master-new
pull_request:
workflow_dispatch:
workflow_call:
@@ -14,10 +15,11 @@ on:
type: string
concurrency:
group: selfdrive-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
group: selfdrive-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
REPORT_NAME: report-${{ inputs.run_number || '1' }}-${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && 'master' || github.event.number }}
PYTHONWARNINGS: error
BASE_IMAGE: openpilot-base
AZURE_TOKEN: ${{ secrets.AZURE_COMMADATACI_OPENPILOTCI_TOKEN }}
@@ -31,10 +33,10 @@ env:
jobs:
build_release:
if: github.repository == 'commaai/openpilot' # build_release blocked for the time being to only comma as we may have a different process.
name: build release
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
env:
STRIPPED_DIR: /tmp/releasepilot
steps:
@@ -52,7 +54,7 @@ jobs:
run: TARGET_DIR=$STRIPPED_DIR release/build_devel.sh
- uses: ./.github/workflows/setup-with-retry
- name: Check submodules
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
timeout-minutes: 3
run: release/check-submodules.sh
- name: Build openpilot and run checks
@@ -68,8 +70,7 @@ jobs:
build:
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
@@ -104,6 +105,12 @@ jobs:
env:
# package install has DeprecationWarnings
PYTHONWARNINGS: default
- name: Save Homebrew cache
uses: actions/cache/save@v4
if: (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new')
with:
path: ~/Library/Caches/Homebrew
key: brew-macos-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
- run: git lfs pull
- name: Getting scons cache
uses: ./.github/workflows/auto-cache
@@ -115,12 +122,17 @@ jobs:
scons-${{ runner.arch }}-macos
- name: Building openpilot
run: . .venv/bin/activate && scons -j$(nproc)
- name: Save scons cache
uses: actions/cache/save@v4
if: (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new')
with:
path: /tmp/scons_cache
key: scons-${{ runner.arch }}-macos-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
static_analysis:
name: static analysis
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-latest'
env:
PYTHONWARNINGS: default
steps:
@@ -135,8 +147,7 @@ jobs:
unit_tests:
name: unit tests
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
@@ -161,9 +172,9 @@ jobs:
process_replay:
name: process replay
if: github.repository == 'commaai/openpilot' # disable process_replay for forks
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
@@ -215,8 +226,7 @@ jobs:
test_cars:
name: cars
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
strategy:
fail-fast: false
matrix:
@@ -307,8 +317,7 @@ jobs:
simulator_driving:
name: simulator driving
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
if: (github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
steps:
- uses: actions/checkout@v4
@@ -329,8 +338,7 @@ jobs:
# This job name needs to be the same as UI_JOB_NAME in ui_preview.yaml
name: Create UI Report
runs-on:
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
@@ -345,7 +353,7 @@ jobs:
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc)"
- name: Create Test Report
timeout-minutes: ${{ ((steps.frames-cache.outputs.cache-hit == 'true') && 1 || 3) }}
timeout-minutes: ${{ ((steps.frames-cache.outputs.cache-hit == 'true') && 2 || 4) }}
run: >
${{ env.RUN }} "PYTHONWARNINGS=ignore &&
source selfdrive/test/setup_xvfb.sh &&
@@ -354,5 +362,5 @@ jobs:
- name: Upload Test Report
uses: actions/upload-artifact@v4
with:
name: report-${{ inputs.run_number || '1' }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
name: ${{ env.REPORT_NAME }}
path: selfdrive/ui/tests/test_ui/report_1/screenshots

View File

@@ -19,7 +19,7 @@ runs:
- shell: bash
name: No retries!
run: |
if [ "${{ github.run_attempt }}" -gt 1 ]; then
if [ "${{ github.run_attempt }}" -gt ${{ github.event.pull_request.head.repo.fork && github.event.pull_request.author_association == 'NONE' && 2 || 1}} ]; then
echo -e "\033[0;31m##################################################"
echo -e "\033[0;31m Retries not allowed! Fix the flaky test! "
echo -e "\033[0;31m##################################################\033[0m"

View File

@@ -20,7 +20,7 @@ jobs:
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
stale-pr-label: stale
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'commaai/openpilot' }} # only delete branches on the main repo
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'sunnypilot/sunnypilot' }} # only delete branches on the main repo
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE }}
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}

View File

@@ -0,0 +1,133 @@
name: Build Model from Upstream
env:
BUILD_DIR: "/data/openpilot"
OUTPUT_DIR: ${{ github.workspace }}/output
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
UPSTREAM_REPO: "commaai/openpilot"
on:
workflow_dispatch:
inputs:
upstream_branch:
description: 'Upstream branch to build from'
required: true
default: 'master'
type: string
custom_name:
description: 'Custom name for the model'
required: false
type: string
file_name:
description: 'File name prefix for the model files'
required: false
type: string
is_20hz:
description: 'Is this a 20Hz model'
required: false
type: boolean
default: false
run-name: Build model [${{ inputs.custom_name || inputs.upstream_branch }}] from ref [${{ inputs.upstream_branch }}]
jobs:
build_model:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
with:
repository: ${{ env.UPSTREAM_REPO }}
ref: ${{ github.event.inputs.upstream_branch }}
submodules: recursive
- run: git lfs pull
- name: Cache SCons
uses: actions/cache@v4
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}
- name: Setup build environment
run: |
mkdir -p "${BUILD_DIR}/"
sudo find $BUILD_DIR/ -mindepth 1 -delete
echo "Starting build stage..."
echo "Building from: ${{ env.UPSTREAM_REPO }} branch: ${{ github.event.inputs.upstream_branch }}"
- name: Patch SConstruct to pass arbitrary cache
run: |
sed -i.bak 's#cache_dir =#default_cache_dir =#' ${{ github.workspace }}/SConstruct
printf '/default_cache_dir/a\\\ncache_dir = ARGUMENTS.get("cache_dir", default_cache_dir)\n' | sed -i.bak -f - ${{ github.workspace }}/SConstruct
cat ${{ github.workspace }}/SConstruct
- name: Build Model
run: |
source /etc/profile
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
scons -j$(nproc) cache_dir=${{ env.SCONS_CACHE_DIR }} ${{ github.workspace }}/selfdrive/modeld
- name: Prepare Output
run: |
sudo rm -rf ${OUTPUT_DIR}
mkdir -p ${OUTPUT_DIR}
# Copy the model files
rsync -avm \
--include='*.dlc' \
--include='*.thneed' \
--include='*.pkl' \
--include='*.onnx' \
--exclude='*' \
--delete-excluded \
--chown=comma:comma \
./selfdrive/modeld/models/ ${OUTPUT_DIR}/
# Rename files if file_name is provided
if [ ! -z "${{ inputs.file_name }}" ]; then
mv ${OUTPUT_DIR}/supercombo.thneed ${OUTPUT_DIR}/supercombo-${{ inputs.file_name }}.thneed
mv ${OUTPUT_DIR}/supercombo_metadata.pkl ${OUTPUT_DIR}/supercombo-${{ inputs.file_name }}_metadata.pkl
fi
# Calculate SHA256 hashes
HASH=$(sha256sum ${OUTPUT_DIR}/supercombo*.thneed | cut -d' ' -f1)
METADATA_HASH=$(sha256sum ${OUTPUT_DIR}/supercombo*_metadata.pkl | cut -d' ' -f1)
# Create metadata.json
cat > ${OUTPUT_DIR}/metadata.json << EOF
{
"display_name": "${{ inputs.custom_name || inputs.upstream_branch }}",
"full_name": "${{ inputs.file_name || 'default' }}",
"is_20hz": ${{ inputs.is_20hz || false }},
"files": {
"drive_model": {
"file_name": "$(basename ${OUTPUT_DIR}/supercombo*.thneed)",
"sha256": "${HASH}"
},
"metadata": {
"file_name": "$(basename ${OUTPUT_DIR}/supercombo*_metadata.pkl)",
"sha256": "${METADATA_HASH}"
}
},
"ref": "${{ inputs.upstream_branch }}",
"build_time": "$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
}
- name: Upload Build Artifacts
uses: actions/upload-artifact@v4
with:
name: model-${{ github.event.inputs.custom_name || github.event.inputs.upstream_branch }}-${{ github.run_number }}
path: ${{ env.OUTPUT_DIR }}

View File

@@ -0,0 +1,271 @@
name: sunnypilot prebuilt action
env:
BUILD_DIR: "/data/openpilot"
OUTPUT_DIR: ${{ github.workspace }}/output
CI_DIR: ${{ github.workspace }}/release/ci
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
PUBLIC_REPO_URL: "https://github.com/sunnypilot/sunnypilot"
# Branch configurations
MASTER_BRANCH: "master"
MASTER_NEW_BRANCH: "master-new"
DEV_C3_SOURCE_BRANCH: "master-dev-c3-new"
# Target branch configurations
STAGING_TARGET_BRANCH: "staging-c3-new"
DEV_TARGET_BRANCH: "dev-c3-new"
RELEASE_TARGET_BRANCH: "release-c3-new"
on:
push:
branches: [ master, master-new, master-dev-c3-new ]
tags: [ '*' ]
workflow_dispatch:
inputs:
extra_version:
description: 'Extra version identifier'
required: false
default: ''
jobs:
build:
concurrency:
group: build-${{ github.head_ref || github.ref_name }}
cancel-in-progress: false
runs-on: self-hosted
outputs:
new_branch: ${{ steps.set-env.outputs.new_branch }}
version: ${{ steps.set-env.outputs.version }}
extra_version_identifier: ${{ steps.set-env.outputs.extra_version_identifier }}
commit_sha: ${{ steps.set-env.outputs.commit_sha }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- run: git lfs pull
- name: Cache SCons
uses: actions/cache@v4
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}
- name: Set Configuration
run: |
if [[ "${{ github.ref_name }}" == "${{ env.DEV_C3_SOURCE_BRANCH }}" ]]; then
# Dev configuration
echo "BRANCH_TYPE=dev" >> $GITHUB_ENV
echo "NEW_BRANCH=${{ env.DEV_TARGET_BRANCH }}" >> $GITHUB_ENV
echo "VERSION=$(date '+%Y.%m.%d')-${{ github.run_number }}" >> $GITHUB_ENV
echo "EXTRA_VERSION_IDENTIFIER=${{ github.run_number }}" >> $GITHUB_ENV
elif [[ "${{ github.ref_name }}" == "${{ env.MASTER_BRANCH }}" || "${{ github.ref_name }}" == "${{ env.MASTER_NEW_BRANCH }}" ]]; then
# Master configuration
echo "BRANCH_TYPE=master" >> $GITHUB_ENV
echo "NEW_BRANCH=${{ env.STAGING_TARGET_BRANCH }}" >> $GITHUB_ENV
echo "EXTRA_VERSION_IDENTIFIER=staging" >> $GITHUB_ENV
echo "VERSION=$(cat common/version.h | grep COMMA_VERSION | sed -e 's/[^0-9|.]//g')-staging" >> $GITHUB_ENV
elif [[ "${{ github.ref }}" == refs/tags/* ]]; then
# Tag configuration
echo "BRANCH_TYPE=tag" >> $GITHUB_ENV
echo "NEW_BRANCH=${{ env.RELEASE_TARGET_BRANCH }}" >> $GITHUB_ENV
echo "EXTRA_VERSION_IDENTIFIER=release" >> $GITHUB_ENV
echo "VERSION=$(cat common/version.h | grep COMMA_VERSION | sed -e 's/[^0-9|.]//g')-release" >> $GITHUB_ENV
else
# Feature branch configuration
echo "BRANCH_TYPE=dispatch" >> $GITHUB_ENV
echo "NEW_BRANCH=${{ github.head_ref || github.ref_name }}-prebuilt" >> $GITHUB_ENV
echo "VERSION=$(date '+%Y.%m.%d')-${{ github.run_number }}" >> $GITHUB_ENV
fi
- name: Set environment variables
id: set-env
run: |
# Write to GITHUB_OUTPUT from environment variables
echo "new_branch=$NEW_BRANCH" >> $GITHUB_OUTPUT
[[ ! -z "$EXTRA_VERSION_IDENTIFIER" ]] && echo "extra_version_identifier=$EXTRA_VERSION_IDENTIFIER" >> $GITHUB_OUTPUT
[[ ! -z "$VERSION" ]] && echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "commit_sha=${{ github.sha }}" >> $GITHUB_OUTPUT
# Set up common environment
source /etc/profile;
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
printenv >> $GITHUB_ENV
if [[ "${{ runner.debug }}" == "1" ]]; then
cat $GITHUB_OUTPUT
fi
- name: Setup build environment
run: |
mkdir -p "${BUILD_DIR}/"
sudo find $BUILD_DIR/ -mindepth 1 -delete
echo "Starting build stage..."
echo "BUILD_DIR: ${BUILD_DIR}"
echo "CI_DIR: ${CI_DIR}"
echo "VERSION: ${{ steps.set-env.outputs.version }}"
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
echo "-------"
if [[ "${{ runner.debug }}" == "1" ]]; then
printenv
fi
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
- name: Build Main Project
run: |
export PYTHONPATH="$BUILD_DIR"
./release/release_files.py | sort | uniq | rsync -rRl${RUNNER_DEBUG:+v} --files-from=- . $BUILD_DIR/
cd $BUILD_DIR
sed -i '/from .board.jungle import PandaJungle, PandaJungleDFU/s/^/#/' panda/__init__.py
scons -j$(nproc) cache_dir=${{env.SCONS_CACHE_DIR}} --minimal
touch ${BUILD_DIR}/prebuilt
if [[ "${{ runner.debug }}" == "1" ]]; then
ls -la ${BUILD_DIR}
fi
- name: Prepare Output
run: |
sudo rm -rf ${OUTPUT_DIR}
mkdir -p ${OUTPUT_DIR}
rsync -am${RUNNER_DEBUG:+v} \
--include='**/panda/board/' \
--include='**/panda/board/obj' \
--include='**/panda/board/obj/panda.bin.signed' \
--include='**/panda/board/obj/panda_h7.bin.signed' \
--include='**/panda/board/obj/bootstub.panda.bin' \
--include='**/panda/board/obj/bootstub.panda_h7.bin' \
--exclude='.sconsign.dblite' \
--exclude='*.a' \
--exclude='*.o' \
--exclude='*.os' \
--exclude='*.pyc' \
--exclude='moc_*' \
--exclude='*.cc' \
--exclude='Jenkinsfile' \
--exclude='supercombo.onnx' \
--exclude='**/panda/board/*' \
--exclude='**/panda/board/obj/**' \
--exclude='**/panda/certs/' \
--exclude='**/panda/crypto/' \
--exclude='**/release/' \
--exclude='**/.github/' \
--exclude='**/selfdrive/ui/replay/' \
--exclude='**/__pycache__/' \
--exclude='**/selfdrive/ui/*.h' \
--exclude='**/selfdrive/ui/**/*.h' \
--exclude='**/selfdrive/ui/qt/offroad/sunnypilot/' \
--exclude='${{env.SCONS_CACHE_DIR}}' \
--exclude='**/.git/' \
--exclude='**/SConstruct' \
--exclude='**/SConscript' \
--exclude='**/.venv/' \
--delete-excluded \
--chown=comma:comma \
${BUILD_DIR}/ ${OUTPUT_DIR}/
- name: 'Tar.gz files'
run: |
tar czf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }} .
ls -la prebuilt.tar.gz
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: prebuilt
path: prebuilt.tar.gz
- name: Re-enable powersave
if: always()
run: |
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
publish:
concurrency:
group: publish-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true
if: ${{ github.event_name != 'pull_request' || github.event_name == 'pull_request' && github.event.pull_request.draft }}
needs: build
runs-on: ubuntu-24.04
environment: ${{ contains(fromJSON(vars.AUTO_DEPLOY_PREBUILT_BRANCHES), github.head_ref || github.ref_name) && 'auto-deploy' || 'feature-branch' }}
steps:
- uses: actions/checkout@v4
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: prebuilt
- name: Untar prebuilt
run: |
mkdir -p ${{ env.OUTPUT_DIR }}
tar xzf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }}
- name: Configure Git
run: |
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"
- name: Publish to Public Repository
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo '${{ toJSON(needs.build.outputs) }}'
ls -la ${{ env.OUTPUT_DIR }}
${{ env.CI_DIR }}/publish.sh \
"${{ github.workspace }}" \
"${{ env.OUTPUT_DIR }}" \
"${{ needs.build.outputs.new_branch }}" \
"${{ needs.build.outputs.version }}" \
"https://x-access-token:${{github.token}}@github.com/sunnypilot/sunnypilot.git" \
"-${{ needs.build.outputs.extra_version_identifier }}"
echo ""
echo "---- To update the list of branches that auto deploy prebuilts -----"
echo ""
echo "1. Go to: ${{ github.server_url }}/${{ github.repository }}/settings/variables/actions/AUTO_DEPLOY_PREBUILT_BRANCHES"
echo "2. Current value: ${{ vars.AUTO_DEPLOY_PREBUILT_BRANCHES }}"
echo "3. Update as needed (JSON array with no spaces)"
notify:
needs: [ build, publish ]
runs-on: ubuntu-24.04
if: success()
steps:
- uses: actions/checkout@v4
- name: Setup Alpine Linux environment
uses: jirutka/setup-alpine@v1.2.0
with:
packages: 'jq gettext curl'
- name: Send Discord Notification
env:
DISCORD_WEBHOOK: ${{ contains(fromJSON(vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES), github.head_ref || github.ref_name) && secrets.DISCORD_DEV_FEEDBACK_CHANNEL_WEBHOOK || secrets.DISCORD_DEV_PRIVATE_CHANNEL_WEBHOOK }}
run: |
TEMPLATE='${{ vars.DISCORD_GENERAL_UPDATE_NOTICE }}'
export EXTRA_VERSION_IDENTIFIER="${{ needs.build.outputs.extra_version_identifier }}"
export VERSION="${{ needs.build.outputs.version }}"
export branch_name=${{ github.head_ref || github.ref_name }}
export new_branch=${{ needs.build.outputs.new_branch }}
export extra_version_identifier=${{ needs.build.outputs.extra_version_identifier || github.run_number}}
echo ${TEMPLATE} | envsubst | jq -c '.' | tee payload.json
curl -X POST -H "Content-Type: application/json" -d @payload.json $DISCORD_WEBHOOK
echo ""
echo "---- To update the list of branches that notify to dev-feedback -----"
echo ""
echo "1. Go to: ${{ github.server_url }}/${{ github.repository }}/settings/variables/actions/DEV_FEEDBACK_NOTIFICATION_BRANCHES"
echo "2. Current value: ${{ vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES }}"
echo "3. Update as needed (JSON array with no spaces)"
shell: alpine.sh {0}

View File

@@ -0,0 +1,193 @@
name: Nightly Branch Reset and PR Squash
env:
DEFAULT_SOURCE_BRANCH: "master-new"
DEFAULT_TARGET_BRANCH: "nightly"
PR_LABEL: "dev-c3"
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
on:
workflow_dispatch:
inputs:
source_branch:
description: 'Source branch to reset from'
required: true
default: 'master-new'
type: string
target_branch:
description: 'Target branch to reset and squash into'
required: true
default: 'master-dev-c3-new'
type: string
# schedule:
# - cron: '0 0 * * *' # Run at midnight UTC for nightly
jobs:
reset-and-squash:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches
token: ${{ secrets.GITHUB_TOKEN }}
- name: Configure Git
run: |
git config --global user.name 'github-actions[bot]'
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install PyGithub
- name: Check branches exist
run: |
# Check if source branch exists
if ! git ls-remote --heads origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} | grep -q "${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"; then
echo "Source branch ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} does not exist!"
exit 1
fi
# Make sure we have the latest source branch
git fetch origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
# Check if target branch exists
if ! git ls-remote --heads origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} | grep -q "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"; then
echo "Target branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} does not exist, creating it from ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
git checkout -b ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
git push origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
else
# Fetch target branch if it exists
git fetch origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
fi
- name: Reset target branch
run: |
echo "Resetting ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} to match ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
# Delete if exists and recreate pointing to source
git branch -D ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} || true
git branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
- name: Get PRs to squash
id: get-prs
run: |
# Use GitHub API to get PRs with specific label, ordered by creation date
PR_LIST=$(gh api graphql -f query='
query($label:String!) {
search(query: $label, type:ISSUE, first:100) {
nodes {
... on PullRequest {
number
headRefName
title
createdAt
labels(last:10) {
nodes {
name
}
}
headRepository {
name
nameWithOwner
url
isFork
}
commits(last: 1) {
nodes {
commit {
statusCheckRollup {
state
}
}
}
}
}
}
}
}' -F label="is:pr is:open label:${PR_LABEL} sort:created-asc")
echo "PR_LIST=${PR_LIST}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Process PRs
run: |
cp ${{ github.workspace }}/release/ci/squash_and_merge.py /tmp/squash_and_merge.py && \
chmod +x /tmp/squash_and_merge.py && \
python3 ${{ github.workspace }}/release/ci/squash_and_merge_prs.py \
--pr-data '${{ steps.get-prs.outputs.PR_LIST }}' \
--target-branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} \
--squash-script-path '/tmp/squash_and_merge.py'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Update LFS Config
run: |
echo '[lfs]' > .lfsconfig
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
echo ' locksverify = false' >> .lfsconfig
- name: Push changes if there are diffs
id: push-changes # Add an id so we can reference this step
run: |
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
# Fetch the latest from remote
git fetch origin $TARGET_BRANCH
# Check for diffs between local and remote
if git diff $TARGET_BRANCH origin/$TARGET_BRANCH --quiet; then
echo "No changes to push - local and remote branches are identical"
echo "has_changes=false" >> $GITHUB_OUTPUT
exit 0
fi
# If we get here, there are diffs, so push
if ! git push origin $TARGET_BRANCH --force; then
echo "Failed to push changes to $TARGET_BRANCH"
exit 1
fi
echo "Branch $TARGET_BRANCH has been reset and updated with squashed PRs"
echo "has_changes=true" >> $GITHUB_OUTPUT
- name: Trigger and wait for selfdrive tests
if: steps.push-changes.outputs.has_changes == 'true'
run: |
echo "Triggering selfdrive tests..."
gh workflow run selfdrive_tests.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
echo "Sleeping for 120s to give plenty of time for the action to start and then we wait"
sleep 120
echo "Getting latest run ID..."
RUN_ID=$(gh run list --workflow=selfdrive_tests.yaml --branch="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}" --limit=1 --json databaseId --jq '.[0].databaseId')
echo "Watching run ID: $RUN_ID"
gh run watch "$RUN_ID"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Trigger prebuilt workflow
if: success() && steps.push-changes.outputs.has_changes == 'true'
run: |
gh workflow run sunnypilot-build-prebuilt.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -3,23 +3,25 @@ on:
push:
branches:
- master
- master-new
pull_request_target:
types: [assigned, opened, synchronize, reopened, edited]
branches:
- 'master'
- 'master-new'
paths:
- 'selfdrive/ui/**'
workflow_dispatch:
env:
UI_JOB_NAME: "Create UI Report"
REPORT_NAME: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
SHA: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.sha || github.event.pull_request.head.sha }}
REPORT_NAME: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && 'master' || github.event.number }}
SHA: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && github.sha || github.event.pull_request.head.sha }}
BRANCH_NAME: "openpilot/pr-${{ github.event.number }}"
jobs:
preview:
if: github.repository == 'commaai/openpilot'
if: github.repository == 'sunnypilot/sunnypilot'
name: preview
runs-on: ubuntu-latest
timeout-minutes: 20
@@ -58,13 +60,13 @@ jobs:
- name: Getting master ui
uses: actions/checkout@v4
with:
repository: commaai/ci-artifacts
repository: sunnypilot/ci-artifacts
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
path: ${{ github.workspace }}/master_ui
ref: openpilot_master_ui
- name: Saving new master ui
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
if: (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && github.event_name == 'push'
working-directory: ${{ github.workspace }}/master_ui
run: |
git checkout --orphan=new_master_ui
@@ -101,7 +103,7 @@ jobs:
DIFF="${DIFF}<table>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF} <td> <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}</table>"
@@ -118,13 +120,13 @@ jobs:
DIFF="${DIFF}<table>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> master <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_master_ref.png\"> </td>"
DIFF="${DIFF} <td> proposed <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF} <td> master <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_master_ref.png\"> </td>"
DIFF="${DIFF} <td> proposed <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> diff <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.png\"> </td>"
DIFF="${DIFF} <td> composite diff <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.gif\"> </td>"
DIFF="${DIFF} <td> diff <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.png\"> </td>"
DIFF="${DIFF} <td> composite diff <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.gif\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}</table>"
@@ -137,7 +139,7 @@ jobs:
if [[ $INDEX -eq 0 ]]; then
TABLE="${TABLE}<tr>"
fi
TABLE="${TABLE} <td> <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
TABLE="${TABLE} <td> <img src=\"https://raw.githubusercontent.com/sunnypilot/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
if [[ $INDEX -eq 1 || $(($i + 1)) -eq ${#A[*]} ]]; then
TABLE="${TABLE}</tr>"
fi

8
.gitignore vendored
View File

@@ -75,6 +75,9 @@ comma*.sh
selfdrive/modeld/thneed/compile
selfdrive/modeld/models/*.thneed
selfdrive/modeld/models/*.pkl
sunnypilot/modeld*/thneed/compile
sunnypilot/modeld*/models/*.thneed
sunnypilot/modeld*/models/*.pkl
*.bz2
*.zst
@@ -104,3 +107,8 @@ Pipfile
# Ignore all local history of files
.history
.ionide
### JetBrains ###
!.idea/customTargets.xml
!.idea/tools/*
!.run/*

13
.gitmodules vendored
View File

@@ -1,18 +1,21 @@
[submodule "panda"]
path = panda
url = ../../commaai/panda.git
url = https://github.com/sunnyhaibin/panda.git
[submodule "opendbc"]
path = opendbc_repo
url = ../../commaai/opendbc.git
url = https://github.com/sunnypilot/opendbc.git
[submodule "msgq"]
path = msgq_repo
url = ../../commaai/msgq.git
url = https://github.com/sunnypilot/msgq.git
[submodule "rednose_repo"]
path = rednose_repo
url = ../../commaai/rednose.git
url = https://github.com/commaai/rednose.git
[submodule "teleoprtc_repo"]
path = teleoprtc_repo
url = ../../commaai/teleoprtc
url = https://github.com/commaai/teleoprtc
[submodule "tinygrad"]
path = tinygrad_repo
url = https://github.com/commaai/tinygrad.git
[submodule "sunnypilot/neural_network_data"]
path = sunnypilot/neural_network_data
url = https://github.com/sunnypilot/neural-network-data.git

25
.idea/customTargets.xml generated Normal file
View File

@@ -0,0 +1,25 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CLionExternalBuildManager">
<target id="a62f99e8-5ec4-434c-8122-49efed5af108" name="uv Scons Build Debug" defaultType="TOOL">
<configuration id="b93ec964-16e5-4962-a12e-3ed360ce8f02" name="uv Scons Build Debug">
<build type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Build Debug" />
</build>
<clean type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Clean" />
</clean>
</configuration>
</target>
<target id="edd8ad9d-183b-467c-a355-0d9a0ecab026" name="uv Scons Build Release" defaultType="TOOL">
<configuration id="09523339-5ce3-4223-ab9e-904f38ad7752" name="uv Scons Build Release">
<build type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Build Release" />
</build>
<clean type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Clean" />
</clean>
</configuration>
</target>
</component>
</project>

23
.idea/tools/External Tools.xml generated Normal file
View File

@@ -0,0 +1,23 @@
<toolSet name="External Tools">
<tool name="uv Scons Build Debug" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -u -j$(nproc) --compile_db --ccflags=\&quot;-fno-inline\&quot;&quot;" />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
<tool name="uv Scons Clean" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -c&quot; " />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
<tool name="uv Scons Build Release" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -u -j$(nproc) --compile_db&quot; " />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
</toolSet>

View File

@@ -1,4 +1,4 @@
[lfs]
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
url = https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git
locksverify = false

4
.lfsconfig-comma Normal file
View File

@@ -0,0 +1,4 @@
[lfs]
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
locksverify = false

10
.run/Build Debug.run.xml Normal file
View File

@@ -0,0 +1,10 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build Debug" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Debug" CONFIG_NAME="uv Scons Build Debug" RUN_PATH="ui">
<envs>
<env name="QT_DBL_CLICK_DIST" value="150" />
</envs>
<method v="2">
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
</method>
</configuration>
</component>

View File

@@ -0,0 +1,10 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build Release" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Release" CONFIG_NAME="uv Scons Build Release" RUN_PATH="ui">
<envs>
<env name="QT_DBL_CLICK_DIST" value="150" />
</envs>
<method v="2">
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
</method>
</configuration>
</component>

21
LICENSE.md Normal file
View File

@@ -0,0 +1,21 @@
# Custom MIT License
Copyright (c) 2024, Haibin Wen, SUNNYPILOT LLC
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to view and modify the Software, subject to the following conditions:
1. **Permission Required**: Permission Required for Commercial, For-Profit, or Closed Source Use: Use of the Software, in whole or in part, for any commercial purposes, for-profit projects, or in closed source projects requires explicit written permission from the original author(s).
2. **Redistribution**: Any redistribution of the Software, modified or unmodified, must retain this license notice and the following acknowledgment:
"This software is licensed under a custom license requiring permission for use."
3. **Visibility**: Any project that uses the Software must visibly mention the following acknowledgment:
"This project uses software from Haibin Wen and SUNNYPILOT LLC and is licensed under a custom license requiring permission for use."
4. **No Warranty**: THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Contact sunnypilot Support <support@sunnypilot.ai> for permission requests.
---
Haibin Wen, SUNNYPILOT LLC

188
README.md
View File

@@ -1,107 +1,119 @@
<div align="center" style="text-align: center;">
![](https://user-images.githubusercontent.com/47793918/233812617-beab2e71-57b9-479e-8bff-c3931347ca40.png)
<h1>openpilot</h1>
## 🌞 What is sunnypilot?
[sunnypilot](https://github.com/sunnyhaibin/sunnypilot) is a fork of comma.ai's openpilot, an open source driver assistance system. sunnypilot offers the user a unique driving experience for over 300+ supported car makes and models with modified behaviors of driving assist engagements. sunnypilot complies with comma.ai's safety rules as accurately as possible.
<p>
<b>openpilot is an operating system for robotics.</b>
<br>
Currently, it upgrades the driver assistance system in 275+ supported cars.
</p>
## 💭 Join our Discord
Join the official sunnypilot Discord server to stay up to date with all the latest features and be a part of shaping the future of sunnypilot!
* https://discord.gg/sunnypilot
<h3>
<a href="https://docs.comma.ai">Docs</a>
<span> · </span>
<a href="https://docs.comma.ai/contributing/roadmap/">Roadmap</a>
<span> · </span>
<a href="https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md">Contribute</a>
<span> · </span>
<a href="https://discord.comma.ai">Community</a>
<span> · </span>
<a href="https://comma.ai/shop">Try it on a comma 3X</a>
</h3>
![](https://dcbadge.vercel.app/api/server/wRW3meAgtx?style=flat) ![Discord Shield](https://discordapp.com/api/guilds/880416502577266699/widget.png?style=shield)
Quick start: `bash <(curl -fsSL openpilot.comma.ai)`
## Documentation
https://docs.sunnypilot.ai/ is your one stop shop for everything from features to installation to FAQ about the sunnypilot
![openpilot tests](https://github.com/commaai/openpilot/actions/workflows/selfdrive_tests.yaml/badge.svg)
[![codecov](https://codecov.io/gh/commaai/openpilot/branch/master/graph/badge.svg)](https://codecov.io/gh/commaai/openpilot)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![X Follow](https://img.shields.io/twitter/follow/comma_ai)](https://x.com/comma_ai)
[![Discord](https://img.shields.io/discord/469524606043160576)](https://discord.comma.ai)
## 🚘 Running on a dedicated device in a car
* A supported device to run this software
* a [comma three](https://comma.ai/shop/products/three) or a [C3X](https://comma.ai/shop/comma-3x)
* This software
* One of [the 300+ supported cars](https://github.com/commaai/openpilot/blob/master/docs/CARS.md). We support Honda, Toyota, Hyundai, Nissan, Kia, Chrysler, Lexus, Acura, Audi, VW, Ford and more. If your car is not supported but has adaptive cruise control and lane-keeping assist, it's likely able to run sunnypilot.
* A [car harness](https://comma.ai/shop/products/car-harness) to connect to your car
</div>
Detailed instructions for [how to mount the device in a car](https://comma.ai/setup).
<table>
<tr>
<td><a href="https://youtu.be/NmBfgOanCyk" title="Video By Greer Viau"><img src="https://github.com/commaai/openpilot/assets/8762862/2f7112ae-f748-4f39-b617-fabd689c3772"></a></td>
<td><a href="https://youtu.be/VHKyqZ7t8Gw" title="Video By Logan LeGrand"><img src="https://github.com/commaai/openpilot/assets/8762862/92351544-2833-40d7-9e0b-7ef7ae37ec4c"></a></td>
<td><a href="https://youtu.be/SUIZYzxtMQs" title="A drive to Taco Bell"><img src="https://github.com/commaai/openpilot/assets/8762862/05ceefc5-2628-439c-a9b2-89ce77dc6f63"></a></td>
</tr>
</table>
## Installation
Please refer to [Recommended Branches](#-recommended-branches) to find your preferred/supported branch. This guide will assume you want to install the latest `release-c3` branch.
* sunnypilot not installed or you installed a version before 0.8.17?
1. [Factory reset/uninstall](https://github.com/commaai/openpilot/wiki/FAQ#how-can-i-reset-the-device) the previous software if you have another software/fork installed.
2. After factory reset/uninstall and upon reboot, select `Custom Software` when given the option.
3. Input the installation URL per [Recommended Branches](#-recommended-branches). Example: ```release-c3.sunnypilot.ai```.
4. Complete the rest of the installation following the onscreen instructions.
* sunnypilot already installed and you installed a version after 0.8.17?
1. On the comma three, go to `Settings` ▶️ `Software`.
2. At the `Download` option, press `CHECK`. This will fetch the list of latest branches from sunnypilot.
3. At the `Target Branch` option, press `SELECT` to open the Target Branch selector.
4. Scroll to select the desired branch per Recommended Branches (see below). Example: `release-c3`
| Branch | Installation URL |
|:------------:|:--------------------------------:|
| `release-c3` | https://release-c3.sunnypilot.ai |
| `staging-c3` | https://staging-c3.sunnypilot.ai |
| `dev-c3` | https://dev-c3.sunnypilot.ai |
### If you want to use our newest branches (our rewrite)
> [!TIP]
>You can see the rewrite state on our [rewrite project board](https://github.com/orgs/sunnypilot/projects/2), and to install the new branches, you can use the following links
Using openpilot in a car
------
> [!IMPORTANT]
> It is recommended to [re-flash AGNOS](https://flash.comma.ai/) if you intend to downgrade from the new branches.
> You can still restore the latest sunnylink backup made on the old branches.
To use openpilot in a car, you need four things:
1. **Supported Device:** a comma 3/3X, available at [comma.ai/shop](https://comma.ai/shop/comma-3x).
2. **Software:** The setup procedure for the comma 3/3X allows users to enter a URL for custom software. Use the URL `openpilot.comma.ai` to install the release version.
3. **Supported Car:** Ensure that you have one of [the 275+ supported cars](docs/CARS.md).
4. **Car Harness:** You will also need a [car harness](https://comma.ai/shop/car-harness) to connect your comma 3/3X to your car.
| Branch | Installation URL |
|:----------------:|:---------------------------------------------:|
| `staging-c3-new` | `https://staging-c3-new.sunnypilot.ai` |
| `dev-c3-new` | `https://dev-c3-new.sunnypilot.ai` |
| `custom-branch` | `https://install.sunnypilot.ai/{branch_name}` |
| `release-c3-new` | **Not yet available**. |
We have detailed instructions for [how to install the harness and device in a car](https://comma.ai/setup). Note that it's possible to run openpilot on [other hardware](https://blog.comma.ai/self-driving-car-for-free/), although it's not plug-and-play.
> [!TIP]
> Do you require further assistance with software installation? Join the [sunnypilot Discord server](https://discord.sunnypilot.com) and message us in the `#installation-help` channel.
### Branches
| branch | URL | description |
|------------------|----------------------------------------|-------------------------------------------------------------------------------------|
| `release3` | openpilot.comma.ai | This is openpilot's release branch. |
| `release3-staging` | openpilot-test.comma.ai | This is the staging branch for releases. Use it to get new releases slightly early. |
| `nightly` | openpilot-nightly.comma.ai | This is the bleeding edge development branch. Do not expect this to be stable. |
| `nightly-dev` | installer.comma.ai/commaai/nightly-dev | Same as nightly, but includes experimental development features for some cars. |
## 🎆 Pull Requests
We welcome both pull requests and issues on GitHub. Bug fixes are encouraged.
To start developing openpilot
------
Pull requests should be against the most current `master-new` branch.
openpilot is developed by [comma](https://comma.ai/) and by users like you. We welcome both pull requests and issues on [GitHub](http://github.com/commaai/openpilot).
## 📊 User Data
* Join the [community Discord](https://discord.comma.ai)
* Check out [the contributing docs](docs/CONTRIBUTING.md)
* Check out the [openpilot tools](tools/)
* Read about the [development workflow](docs/WORKFLOW.md)
* Code documentation lives at https://docs.comma.ai
* Information about running openpilot lives on the [community wiki](https://github.com/commaai/openpilot/wiki)
By default, sunnypilot uploads the driving data to comma servers. You can also access your data through [comma connect](https://connect.comma.ai/).
Want to get paid to work on openpilot? [comma is hiring](https://comma.ai/jobs#open-positions) and offers lots of [bounties](https://comma.ai/bounties) for external contributors.
sunnypilot is open source software. The user is free to disable data collection if they wish to do so.
Safety and Testing
----
* openpilot observes [ISO26262](https://en.wikipedia.org/wiki/ISO_26262) guidelines, see [SAFETY.md](docs/SAFETY.md) for more details.
* openpilot has software-in-the-loop [tests](.github/workflows/selfdrive_tests.yaml) that run on every commit.
* The code enforcing the safety model lives in panda and is written in C, see [code rigor](https://github.com/commaai/panda#code-rigor) for more details.
* panda has software-in-the-loop [safety tests](https://github.com/commaai/panda/tree/master/tests/safety).
* Internally, we have a hardware-in-the-loop Jenkins test suite that builds and unit tests the various processes.
* panda has additional hardware-in-the-loop [tests](https://github.com/commaai/panda/blob/master/Jenkinsfile).
* We run the latest openpilot in a testing closet containing 10 comma devices continuously replaying routes.
Licensing
------
openpilot is released under the MIT license. Some parts of the software are released under other licenses as specified.
Any user of this software shall indemnify and hold harmless Comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys fees and costs) which arise out of, relate to or result from any use of this software by user.
**THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT.
YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS.
NO WARRANTY EXPRESSED OR IMPLIED.**
User Data and comma Account
------
By default, openpilot uploads the driving data to our servers. You can also access your data through [comma connect](https://connect.comma.ai/). We use your data to train better models and improve openpilot for everyone.
openpilot is open source software: the user is free to disable data collection if they wish to do so.
openpilot logs the road-facing cameras, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs.
sunnypilot logs the road-facing camera, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs.
The driver-facing camera is only logged if you explicitly opt-in in settings. The microphone is not recorded.
By using openpilot, you agree to [our Privacy Policy](https://comma.ai/privacy). You understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma for the use of this data.
By using this software, you understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma for the use of this data.
## Licensing
sunnypilot is released under the [MIT License](LICENSE). This repository includes original work as well as significant portions of code derived from [openpilot by comma.ai](https://github.com/commaai/openpilot), which is also released under the MIT license with additional disclaimers.
The original openpilot license notice, including comma.ais indemnification and alpha software disclaimer, is reproduced below as required:
> openpilot is released under the MIT license. Some parts of the software are released under other licenses as specified.
>
> Any user of this software shall indemnify and hold harmless Comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys fees and costs) which arise out of, relate to or result from any use of this software by user.
>
> **THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT.
> YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS.
> NO WARRANTY EXPRESSED OR IMPLIED.**
For full license terms, please see the [`LICENSE`](LICENSE) file.
## 💰 Support sunnypilot
If you find any of the features useful, consider becoming a [sponsor on GitHub](https://github.com/sponsors/sunnyhaibin) to support future feature development and improvements.
By becoming a sponsor, you will gain access to exclusive content, early access to new features, and the opportunity to directly influence the project's development.
<h3>GitHub Sponsor</h3>
<a href="https://github.com/sponsors/sunnyhaibin">
<img src="https://user-images.githubusercontent.com/47793918/244135584-9800acbd-69fd-4b2b-bec9-e5fa2d85c817.png" alt="Become a Sponsor" width="300" style="max-width: 100%; height: auto;">
</a>
<br>
<h3>PayPal</h3>
<a href="https://paypal.me/sunnyhaibin0850" target="_blank">
<img src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" alt="PayPal this" title="PayPal - The safer, easier way to pay online!" border="0" />
</a>
<br></br>
Your continuous love and support are greatly appreciated! Enjoy 🥰
<span>-</span> Jason, Founder of sunnypilot

View File

@@ -70,6 +70,12 @@ AddOption('--minimal',
default=os.path.exists(File('#.lfsconfig').abspath), # minimal by default on release branch (where there's no LFS)
help='the minimum build to run openpilot. no tests, tools, etc.')
AddOption('--stock-ui',
action='store_true',
dest='stock_ui',
default=False,
help='Build stock openpilot UI instead of sunnypilot UI')
## Architecture name breakdown (arch)
## - larch64: linux tici aarch64
## - aarch64: linux pc aarch64
@@ -107,6 +113,7 @@ if arch == "larch64":
]
libpath += [
"#third_party/snpe/larch64",
"#third_party/libyuv/larch64/lib",
"/usr/lib/aarch64-linux-gnu"
]
@@ -145,6 +152,14 @@ else:
"/usr/local/lib",
]
if arch == "x86_64":
libpath += [
f"#third_party/snpe/{arch}"
]
rpath += [
Dir(f"#third_party/snpe/{arch}").abspath,
]
if GetOption('asan'):
ccflags = ["-fsanitize=address", "-fno-omit-frame-pointer"]
ldflags = ["-fsanitize=address"]
@@ -159,6 +174,10 @@ else:
if arch != "Darwin":
ldflags += ["-Wl,--as-needed", "-Wl,--no-undefined"]
if not GetOption('stock_ui'):
cflags += ["-DSUNNYPILOT"]
cxxflags += ["-DSUNNYPILOT"]
ccflags_option = GetOption('ccflags')
if ccflags_option:
ccflags += ccflags_option.split(' ')
@@ -188,6 +207,7 @@ env = Environment(
"#third_party/libyuv/include",
"#third_party/json11",
"#third_party/linux/include",
"#third_party/snpe/include",
"#third_party",
"#msgq",
],
@@ -223,7 +243,8 @@ if GetOption('compile_db'):
env.CompilationDatabase('compile_commands.json')
# Setup cache dir
cache_dir = '/data/scons_cache' if AGNOS else '/tmp/scons_cache'
default_cache_dir = '/data/scons_cache' if AGNOS else '/tmp/scons_cache'
cache_dir = ARGUMENTS.get('cache_dir', default_cache_dir)
CacheDir(cache_dir)
Clean(["."], cache_dir)
@@ -370,6 +391,8 @@ SConscript(['third_party/SConscript'])
SConscript(['selfdrive/SConscript'])
SConscript(['sunnypilot/SConscript'])
if Dir('#tools/cabana/').exists() and GetOption('extras'):
SConscript(['tools/replay/SConscript'])
if arch != "larch64":

View File

@@ -10,25 +10,197 @@ $Cxx.namespace("cereal");
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
struct CustomReserved0 @0x81c2f05a394cf4af {
struct ModularAssistiveDrivingSystem {
state @0 :ModularAssistiveDrivingSystemState;
enabled @1 :Bool;
active @2 :Bool;
available @3 :Bool;
enum ModularAssistiveDrivingSystemState {
disabled @0;
paused @1;
enabled @2;
softDisabling @3;
overriding @4;
}
}
struct CustomReserved1 @0xaedffd8f31e7b55d {
struct SelfdriveStateSP @0x81c2f05a394cf4af {
mads @0 :ModularAssistiveDrivingSystem;
}
struct CustomReserved2 @0xf35cc4560bbf6ec2 {
struct ModelManagerSP @0xaedffd8f31e7b55d {
activeBundle @0 :ModelBundle;
selectedBundle @1 :ModelBundle;
availableBundles @2 :List(ModelBundle);
struct DownloadUri {
uri @0 :Text;
sha256 @1 :Text;
}
enum Type {
drive @0;
navigation @1;
metadata @2;
}
struct Model {
fullName @0 :Text;
fileName @1 :Text;
downloadUri @2 :DownloadUri;
downloadProgress @3 :DownloadProgress;
type @4 :Type;
}
enum DownloadStatus {
notDownloading @0;
downloading @1;
downloaded @2;
cached @3;
failed @4;
}
struct DownloadProgress {
status @0 :DownloadStatus;
progress @1 :Float32;
eta @2 :UInt32;
}
enum Runner {
snpe @0;
tinygrad @1;
stock @2;
}
struct ModelBundle {
index @0 :UInt32;
internalName @1 :Text;
displayName @2 :Text;
models @3 :List(Model);
status @4 :DownloadStatus;
generation @5 :UInt32;
environment @6 :Text;
runner @7 :Runner;
is20hz @8 :Bool;
}
}
struct CustomReserved3 @0xda96579883444c35 {
struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
dec @0 :DynamicExperimentalControl;
struct DynamicExperimentalControl {
state @0 :DynamicExperimentalControlState;
enabled @1 :Bool;
active @2 :Bool;
enum DynamicExperimentalControlState {
acc @0;
blended @1;
}
}
}
struct CustomReserved4 @0x80ae746ee2596b11 {
struct OnroadEventSP @0xda96579883444c35 {
events @0 :List(Event);
struct Event {
name @0 :EventName;
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
}
enum EventName {
lkasEnable @0;
lkasDisable @1;
manualSteeringRequired @2;
manualLongitudinalRequired @3;
silentLkasEnable @4;
silentLkasDisable @5;
silentBrakeHold @6;
silentWrongGear @7;
silentReverseGear @8;
silentDoorOpen @9;
silentSeatbeltNotLatched @10;
silentParkBrake @11;
controlsMismatchLateral @12;
hyundaiRadarTracksConfirmed @13;
experimentalModeSwitched @14;
wrongCarModeAlertOnly @15;
}
}
struct CustomReserved5 @0xa5cd762cd951a455 {
struct CarParamsSP @0x80ae746ee2596b11 {
flags @0 :UInt32; # flags for car specific quirks in sunnypilot
safetyParam @1 : Int16; # flags for sunnypilot's custom safety flags
neuralNetworkLateralControl @2 :NeuralNetworkLateralControl;
struct NeuralNetworkLateralControl {
model @0 :Model;
fuzzyFingerprint @1 :Bool;
struct Model {
path @0 :Text;
name @1 :Text;
}
}
}
struct CustomReserved6 @0xf98d843bfd7004a3 {
struct CarControlSP @0xa5cd762cd951a455 {
mads @0 :ModularAssistiveDrivingSystem;
}
struct BackupManagerSP @0xf98d843bfd7004a3 {
backupStatus @0 :Status;
restoreStatus @1 :Status;
backupProgress @2 :Float32;
restoreProgress @3 :Float32;
lastError @4 :Text;
currentBackup @5 :BackupInfo;
backupHistory @6 :List(BackupInfo);
enum Status {
idle @0;
inProgress @1;
completed @2;
failed @3;
}
struct Version {
major @0 :UInt16;
minor @1 :UInt16;
patch @2 :UInt16;
build @3 :UInt16;
branch @4 :Text;
}
struct MetadataEntry {
key @0 :Text;
value @1 :Text;
tags @2 :List(Text);
}
struct BackupInfo {
deviceId @0 :Text;
version @1 :UInt32;
config @2 :Text;
isEncrypted @3 :Bool;
createdAt @4 :Text; # ISO timestamp
updatedAt @5 :Text; # ISO timestamp
sunnypilotVersion @6 :Version;
backupMetadata @7 :List(MetadataEntry);
}
}
struct CustomReserved7 @0xb86e6369214c01c8 {

View File

@@ -2596,13 +2596,13 @@ struct Event {
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
customReserved0 @107 :Custom.CustomReserved0;
customReserved1 @108 :Custom.CustomReserved1;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
customReserved4 @111 :Custom.CustomReserved4;
customReserved5 @112 :Custom.CustomReserved5;
customReserved6 @113 :Custom.CustomReserved6;
selfdriveStateSP @107 :Custom.SelfdriveStateSP;
modelManagerSP @108 :Custom.ModelManagerSP;
longitudinalPlanSP @109 :Custom.LongitudinalPlanSP;
onroadEventsSP @110 :Custom.OnroadEventSP;
carParamsSP @111 :Custom.CarParamsSP;
carControlSP @112 :Custom.CarControlSP;
backupManagerSP @113 :Custom.BackupManagerSP;
customReserved7 @114 :Custom.CustomReserved7;
customReserved8 @115 :Custom.CustomReserved8;
customReserved9 @116 :Custom.CustomReserved9;

View File

@@ -0,0 +1,222 @@
#!/usr/bin/env python3
import argparse
import sys
from typing import Any, List, Tuple
DEBUG = False
def print_debug(string: str) -> None:
if DEBUG:
print(string)
def create_schema_instance(struct: Any, prop: Tuple[str, Any]) -> Any:
"""
Create a new instance of a schema type, handling different field types.
Args:
struct: The Cap'n Proto schema structure
prop: A tuple containing the field name and field metadata
Returns:
A new initialized schema instance
"""
struct_instance = struct.new_message()
field_name, field_metadata = prop
try:
field_type = field_metadata.proto.slot.type.which()
# Initialize different types of fields
if field_type in ('list', 'text', 'data'):
struct_instance.init(field_name, 1)
print_debug(f"Initialized list/text/data field: {field_name}")
elif field_type in ('struct', 'object'):
struct_instance.init(field_name)
print_debug(f"Initialized struct/object field: {field_name}")
return struct_instance
except Exception as e:
print(f"Error creating instance for {field_name}: {e}")
return None
def get_schema_fields(schema_struct: Any) -> List[Tuple[str, Any]]:
"""
Retrieve all fields from a given schema structure.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of field names and their metadata
"""
try:
# Get all fields from the schema
schema_fields = list(schema_struct.schema.fields.items())
print_debug("Discovered schema fields:")
for field_name, field_metadata in schema_fields:
print_debug(f"- {field_name}")
return schema_fields
except Exception as e:
print(f"Error retrieving schema fields: {e}")
return []
def generate_schema_instances(schema_struct: Any) -> List[Any]:
"""
Generate instances for all fields in a given schema.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of schema instances
"""
schema_fields = get_schema_fields(schema_struct)
instances = []
for field_prop in schema_fields:
try:
instance = create_schema_instance(schema_struct, field_prop)
if instance is not None:
instances.append(instance)
except Exception as e:
print(f"Skipping field due to error: {e}")
print(f"Generated {len(instances)} schema instances")
return instances
def persist_instances(instances: List[Any], filename: str) -> None:
"""
Write schema instances to a binary file.
Args:
instances: List of schema instances
filename: Output file path
"""
try:
with open(filename, 'wb') as f:
for instance in instances:
f.write(instance.to_bytes())
print(f"Successfully wrote {len(instances)} instances to {filename}")
except Exception as e:
print(f"Error persisting instances: {e}")
sys.exit(1)
def read_instances(filename: str, schema_type: Any) -> List[Any]:
"""
Read schema instances from a binary file.
Args:
filename: Input file path
schema_type: The schema type to use for reading
Returns:
A list of read schema instances
"""
try:
with open(filename, 'rb') as f:
data = f.read()
instances = list(schema_type.read_multiple_bytes(data))
print(f"Read {len(instances)} instances from {filename}")
return instances
except Exception as e:
print(f"Error reading instances: {e}")
sys.exit(1)
def compare_schemas(original_instances: List[Any], read_instances: List[Any]) -> bool:
"""
Compare original and read-back instances to detect potential breaking changes.
Args:
original_instances: List of originally generated instances
read_instances: List of instances read back from file
Returns:
Boolean indicating whether schemas appear compatible
"""
if len(original_instances) != len(read_instances):
print("❌ Schema Compatibility Warning: Instance count mismatch")
return False
compatible = True
for struct in read_instances:
try:
getattr(struct, struct.which()) # Attempting to access the field to validate readability
except Exception as e:
print(f"❌ Structural change detected: {struct.which()} is not readable.\nFull error: {e}")
compatible = False
return compatible
def main():
"""
CLI entry point for schema compatibility testing.
"""
# Setup argument parser
parser = argparse.ArgumentParser(
description='Cap\'n Proto Schema Compatibility Testing Tool',
epilog='Test schema compatibility by generating and reading back instances.'
)
# Add mutually exclusive group for generation or reading mode
mode_group = parser.add_mutually_exclusive_group(required=True)
mode_group.add_argument('-g', '--generate', action='store_true',
help='Generate schema instances')
mode_group.add_argument('-r', '--read', action='store_true',
help='Read and validate schema instances')
# Common arguments
parser.add_argument('-f', '--file',
default='schema_instances.bin',
help='Output/input binary file (default: schema_instances.bin)')
# Parse arguments
args = parser.parse_args()
# Import the schema dynamically
try:
from cereal import log
schema_type = log.Event
except ImportError:
print("Error: Unable to import schema. Ensure 'cereal' is installed.")
sys.exit(1)
# Execute based on mode
if args.generate:
print("🔧 Generating Schema Instances")
instances = generate_schema_instances(schema_type)
persist_instances(instances, args.file)
print("✅ Instance generation complete")
elif args.read:
print("🔍 Reading and Validating Schema Instances")
generated_instances = generate_schema_instances(schema_type)
read_back_instances = read_instances(args.file, schema_type)
# Compare schemas
if compare_schemas(generated_instances, read_back_instances):
print("✅ Schema Compatibility: No breaking changes detected")
sys.exit(0)
else:
print("❌ Potential Schema Breaking Changes Detected")
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -74,6 +74,15 @@ _services: dict[str, tuple] = {
"userFlag": (True, 0., 1),
"microphone": (True, 10., 10),
# sunnypilot
"modelManagerSP": (False, 1., 1),
"backupManagerSP": (False, 1., 1),
"selfdriveStateSP": (True, 100., 10),
"longitudinalPlanSP": (True, 20., 10),
"onroadEventsSP": (True, 1., 1),
"carParamsSP": (True, 0.02, 1),
"carControlSP": (True, 100., 10),
# debug
"uiDebug": (True, 0., 1),
"testJoystick": (True, 0.),

View File

@@ -1,46 +0,0 @@
import jwt
import os
import requests
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class Api:
def __init__(self, dongle_id):
self.dongle_id = dongle_id
with open(Paths.persist_root()+'/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def get_token(self, expiry_hours=1):
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours)
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
headers['User-Agent'] = "openpilot-" + get_version()
return requests.request(method, API_HOST + "/" + endpoint, timeout=timeout, headers=headers, params=params)

22
common/api/__init__.py Normal file
View File

@@ -0,0 +1,22 @@
from openpilot.common.api.comma_connect import CommaConnectApi
class Api:
def __init__(self, dongle_id):
self.service = CommaConnectApi(dongle_id)
def request(self, method, endpoint, **params):
return self.service.request(method, endpoint, **params)
def get(self, *args, **kwargs):
return self.service.get(*args, **kwargs)
def post(self, *args, **kwargs):
return self.service.post(*args, **kwargs)
def get_token(self, expiry_hours=1):
return self.service.get_token(expiry_hours)
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
return CommaConnectApi(None).api_get(endpoint, method, timeout, access_token, **params)

56
common/api/base.py Normal file
View File

@@ -0,0 +1,56 @@
import jwt
import requests
import unicodedata
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
class BaseApi:
def __init__(self, dongle_id, api_host, user_agent="openpilot-"):
self.dongle_id = dongle_id
self.api_host = api_host
self.user_agent = user_agent
with open(f'{Paths.persist_root()}/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return self.api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def _get_token(self, expiry_hours=1, **extra_payload):
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours),
**extra_payload
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def get_token(self, expiry_hours=1):
return self._get_token(expiry_hours)
def remove_non_ascii_chars(self, text):
normalized_text = unicodedata.normalize('NFD', text)
ascii_encoded_text = normalized_text.encode('ascii', 'ignore')
return ascii_encoded_text.decode()
def api_get(self, endpoint, method='GET', timeout=None, access_token=None, json=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
version = self.remove_non_ascii_chars(get_version())
headers['User-Agent'] = self.user_agent + version
return requests.request(method, f"{self.api_host}/{endpoint}", timeout=timeout, headers=headers, json=json, params=params)

View File

@@ -0,0 +1,11 @@
import os
from openpilot.common.api.base import BaseApi
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class CommaConnectApi(BaseApi):
def __init__(self, dongle_id):
super().__init__(dongle_id, API_HOST)
self.user_agent = "openpilot-"

1
common/model.h Normal file
View File

@@ -0,0 +1 @@
#define DEFAULT_MODEL "Filet o Fish (Default)"

View File

@@ -103,10 +103,12 @@ Params::~Params() {
assert(queue.empty());
}
std::vector<std::string> Params::allKeys() const {
std::vector<std::string> Params::allKeys(ParamKeyType type) const {
std::vector<std::string> ret;
for (auto &p : keys) {
ret.push_back(p.first);
if (type == ALL || (p.second & type)) {
ret.push_back(p.first);
}
}
return ret;
}

View File

@@ -16,6 +16,7 @@ enum ParamKeyType {
CLEAR_ON_OFFROAD_TRANSITION = 0x10,
DONT_LOG = 0x20,
DEVELOPMENT_ONLY = 0x40,
BACKUP = 0x80,
ALL = 0xFFFFFFFF
};
@@ -27,7 +28,7 @@ public:
Params(const Params&) = delete;
Params& operator=(const Params&) = delete;
std::vector<std::string> allKeys() const;
std::vector<std::string> allKeys(ParamKeyType type = ALL) const;
bool checkKey(const std::string &key);
ParamKeyType getKeyType(const std::string &key);
inline std::string getParamPath(const std::string &key = {}) {

View File

@@ -27,35 +27,35 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"CurrentBootlog", PERSISTENT},
{"CurrentRoute", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisableLogging", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisablePowerDown", PERSISTENT},
{"DisableUpdates", PERSISTENT},
{"DisengageOnAccelerator", PERSISTENT},
{"DisablePowerDown", PERSISTENT | BACKUP},
{"DisableUpdates", PERSISTENT | BACKUP},
{"DisengageOnAccelerator", PERSISTENT | BACKUP},
{"DongleId", PERSISTENT},
{"DoReboot", CLEAR_ON_MANAGER_START},
{"DoShutdown", CLEAR_ON_MANAGER_START},
{"DoUninstall", CLEAR_ON_MANAGER_START},
{"ExperimentalLongitudinalEnabled", PERSISTENT | DEVELOPMENT_ONLY},
{"ExperimentalMode", PERSISTENT},
{"ExperimentalModeConfirmed", PERSISTENT},
{"ExperimentalLongitudinalEnabled", PERSISTENT | DEVELOPMENT_ONLY | BACKUP},
{"ExperimentalMode", PERSISTENT | BACKUP},
{"ExperimentalModeConfirmed", PERSISTENT | BACKUP},
{"FirmwareQueryDone", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ForcePowerDown", PERSISTENT},
{"GitBranch", PERSISTENT},
{"GitCommit", PERSISTENT},
{"GitCommitDate", PERSISTENT},
{"GitDiff", PERSISTENT},
{"GithubSshKeys", PERSISTENT},
{"GithubUsername", PERSISTENT},
{"GithubSshKeys", PERSISTENT | BACKUP},
{"GithubUsername", PERSISTENT | BACKUP},
{"GitRemote", PERSISTENT},
{"GsmApn", PERSISTENT},
{"GsmMetered", PERSISTENT},
{"GsmRoaming", PERSISTENT},
{"GsmApn", PERSISTENT | BACKUP},
{"GsmMetered", PERSISTENT | BACKUP},
{"GsmRoaming", PERSISTENT | BACKUP},
{"HardwareSerial", PERSISTENT},
{"HasAcceptedTerms", PERSISTENT},
{"InstallDate", PERSISTENT},
{"IsDriverViewEnabled", CLEAR_ON_MANAGER_START},
{"IsEngaged", PERSISTENT},
{"IsLdwEnabled", PERSISTENT},
{"IsMetric", PERSISTENT},
{"IsLdwEnabled", PERSISTENT | BACKUP},
{"IsMetric", PERSISTENT | BACKUP},
{"IsOffroad", CLEAR_ON_MANAGER_START},
{"IsOnroad", PERSISTENT},
{"IsRhdDetected", PERSISTENT},
@@ -63,7 +63,7 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"IsTakingSnapshot", CLEAR_ON_MANAGER_START},
{"IsTestedBranch", CLEAR_ON_MANAGER_START},
{"JoystickDebugMode", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LanguageSetting", PERSISTENT},
{"LanguageSetting", PERSISTENT | BACKUP},
{"LastAthenaPingTime", CLEAR_ON_MANAGER_START},
{"LastGPSPosition", PERSISTENT},
{"LastManagerExitReason", CLEAR_ON_MANAGER_START},
@@ -75,7 +75,7 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"LiveTorqueParameters", PERSISTENT | DONT_LOG},
{"LocationFilterInitialState", PERSISTENT},
{"LongitudinalManeuverMode", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LongitudinalPersonality", PERSISTENT},
{"LongitudinalPersonality", PERSISTENT | BACKUP},
{"NetworkMetered", PERSISTENT},
{"ObdMultiplexingChanged", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ObdMultiplexingEnabled", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
@@ -91,17 +91,17 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"Offroad_TemperatureTooHigh", CLEAR_ON_MANAGER_START},
{"Offroad_UnofficialHardware", CLEAR_ON_MANAGER_START},
{"Offroad_UpdateFailed", CLEAR_ON_MANAGER_START},
{"OpenpilotEnabledToggle", PERSISTENT},
{"OpenpilotEnabledToggle", PERSISTENT | BACKUP},
{"PandaHeartbeatLost", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSomResetTriggered", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSignatures", CLEAR_ON_MANAGER_START},
{"PrimeType", PERSISTENT},
{"RecordFront", PERSISTENT},
{"RecordFront", PERSISTENT | BACKUP},
{"RecordFrontLock", PERSISTENT}, // for the internal fleet
{"SecOCKey", PERSISTENT | DONT_LOG},
{"SecOCKey", PERSISTENT | DONT_LOG}, // Candidate for | BACKUP
{"RouteCount", PERSISTENT},
{"SnoozeUpdate", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"SshEnabled", PERSISTENT},
{"SshEnabled", PERSISTENT | BACKUP},
{"TermsVersion", PERSISTENT},
{"TrainingVersion", PERSISTENT},
{"UbloxAvailable", PERSISTENT},
@@ -117,4 +117,55 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"UpdaterTargetBranch", CLEAR_ON_MANAGER_START},
{"UpdaterLastFetchTime", PERSISTENT},
{"Version", PERSISTENT},
// --- sunnypilot params --- //
{"ApiCache_DriveStats", PERSISTENT},
{"AutoLaneChangeBsmDelay", PERSISTENT},
{"AutoLaneChangeTimer", PERSISTENT},
{"CarParamsSP", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CarParamsSPCache", CLEAR_ON_MANAGER_START},
{"CarParamsSPPersistent", PERSISTENT},
{"CarPlatformBundle", PERSISTENT},
{"EnableGithubRunner", PERSISTENT | BACKUP},
{"MaxTimeOffroad", PERSISTENT | BACKUP},
{"ModelRunnerTypeCache", CLEAR_ON_ONROAD_TRANSITION},
{"OffroadMode", CLEAR_ON_MANAGER_START},
{"OffroadMode_Status", CLEAR_ON_MANAGER_START},
{"QuietMode", PERSISTENT | BACKUP},
// MADS params
{"Mads", PERSISTENT | BACKUP},
{"MadsMainCruiseAllowed", PERSISTENT | BACKUP},
{"MadsPauseLateralOnBrake", PERSISTENT | BACKUP},
{"MadsUnifiedEngagementMode", PERSISTENT | BACKUP},
// Model Manager params
{"ModelManager_ActiveBundle", PERSISTENT},
{"ModelManager_DownloadIndex", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ModelManager_LastSyncTime", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"ModelManager_ModelsCache", PERSISTENT | BACKUP},
// Neural Network Lateral Control
{"NeuralNetworkLateralControl", PERSISTENT | BACKUP},
// sunnylink params
{"EnableSunnylinkUploader", PERSISTENT | BACKUP},
{"LastSunnylinkPingTime", CLEAR_ON_MANAGER_START},
{"SunnylinkCache_Roles", PERSISTENT},
{"SunnylinkCache_Users", PERSISTENT},
{"SunnylinkDongleId", PERSISTENT},
{"SunnylinkdPid", PERSISTENT},
{"SunnylinkEnabled", PERSISTENT},
// Backup Manager params
{"BackupManager_CreateBackup", PERSISTENT},
{"BackupManager_RestoreVersion", PERSISTENT},
// sunnypilot car specific params
{"HyundaiRadarTracks", PERSISTENT},
{"HyundaiRadarTracksConfirmed", PERSISTENT},
{"HyundaiRadarTracksPersistent", PERSISTENT},
{"HyundaiRadarTracksToggle", PERSISTENT},
{"DynamicExperimentalControl", PERSISTENT},
};

View File

@@ -11,6 +11,7 @@ cdef extern from "common/params.h":
CLEAR_ON_ONROAD_TRANSITION
CLEAR_ON_OFFROAD_TRANSITION
DEVELOPMENT_ONLY
BACKUP
ALL
cdef cppclass c_Params "Params":
@@ -25,7 +26,7 @@ cdef extern from "common/params.h":
bool checkKey(string) nogil
string getParamPath(string) nogil
void clearAll(ParamKeyType)
vector[string] allKeys()
vector[string] allKeys(ParamKeyType)
def ensure_bytes(v):
@@ -119,5 +120,5 @@ cdef class Params:
cdef string key_bytes = ensure_bytes(key)
return self.p.getParamPath(key_bytes).decode("utf-8")
def all_keys(self):
return self.p.allKeys()
def all_keys(self, type=ParamKeyType.ALL):
return self.p.allKeys(type)

View File

@@ -18,6 +18,7 @@ collect_ignore = [
collect_ignore_glob = [
"selfdrive/debug/*.py",
"selfdrive/modeld/*.py",
"sunnypilot/modeld*/*.py",
]

1
openpilot/sunnypilot Symbolic link
View File

@@ -0,0 +1 @@
../sunnypilot

2
panda

Submodule panda updated: 8a583aaa8c...4fab184d5f

View File

@@ -163,6 +163,7 @@ testpaths = [
"tools/replay",
"tools/cabana",
"cereal/messaging/tests",
"sunnypilot",
]
[tool.codespell]

View File

@@ -74,6 +74,7 @@ find . -name '__pycache__' -delete
rm -rf .sconsign.dblite Jenkinsfile release/
rm selfdrive/modeld/models/driving_vision.onnx
rm selfdrive/modeld/models/driving_policy.onnx
rm sunnypilot/modeld*/models/supercombo.onnx
find third_party/ -name '*x86*' -exec rm -r {} +
find third_party/ -name '*Darwin*' -exec rm -r {} +

View File

@@ -0,0 +1,262 @@
#!/usr/bin/env bash
set -e
# Default values
DEFAULT_REPO_URL="https://github.com/sunnypilot"
START_AT_BOOT=false
RESTORE_MODE=false
# Parse command line arguments
while [[ $# -gt 0 ]]; do
case $1 in
--start-at-boot)
START_AT_BOOT=true
shift
;;
--token)
GITHUB_TOKEN="$2"
shift 2
;;
--repo)
REPO_URL="$2"
shift 2
;;
--restore)
RESTORE_MODE=true
shift
;;
*)
if [ -z "$GITHUB_TOKEN" ]; then
GITHUB_TOKEN="$1"
elif [ -z "$REPO_URL" ]; then
REPO_URL="$1"
fi
shift
;;
esac
done
# Determine BASE_DIR based on mount point
if mountpoint -q /data/media; then
BASE_DIR="/data/media/0/github"
else
BASE_DIR="/data/github"
fi
# Constants
RUNNER_USER="github-runner"
USER_GROUPS="comma,gpu,gpio,sudo"
RUNNER_DIR="${BASE_DIR}/runner"
BUILDS_DIR="${BASE_DIR}/builds"
LOGS_DIR="${BASE_DIR}/logs"
CACHE_DIR="${BASE_DIR}/cache"
OPENPILOT_DIR="${BASE_DIR}/openpilot"
# Basic utility functions (no dependencies)
remount_rw() {
sudo mount -o remount,rw /
}
remount_ro() {
sync || true # Try to sync but continue even if it fails
sudo mount -o remount,ro / # Always try to remount as read-only
}
# Always ensure we try to remount as read-only on exit
trap remount_ro EXIT
setup_runner_user() {
sudo useradd --comment 'GitHub Runner' --create-home --home-dir ${BASE_DIR} ${RUNNER_USER} --shell /bin/bash -G ${USER_GROUPS} || sudo usermod -aG ${USER_GROUPS} ${RUNNER_USER}
}
create_sudoers_entry() {
sudo grep -qxF "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" /etc/sudoers || echo "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers
}
set_directory_permissions() {
sudo chown -R ${RUNNER_USER}:comma "$BASE_DIR"
sudo chmod -R g+rwx "$BASE_DIR"
sudo find "$BASE_DIR" -type d -exec chmod g+s {} +
}
setup_directories() {
echo "Creating necessary directories..."
sudo mkdir -p "$RUNNER_DIR" "$BUILDS_DIR" "$LOGS_DIR" "$CACHE_DIR" "$OPENPILOT_DIR"
mkdir -p "/data/openpilot"
sudo chown -R comma:comma "/data/openpilot"
sync
}
wipe_bash_logout() {
export BASE_DIR
sudo -u ${RUNNER_USER} bash -c "touch ${BASE_DIR}/.bash_logout"
sudo -u ${RUNNER_USER} bash -c "truncate -s 0 '${BASE_DIR}/.bash_logout'"
}
# System configuration functions (depends on basic utility functions)
setup_system_configs() {
echo "Setting up system configurations..."
remount_rw
setup_runner_user
create_sudoers_entry
remount_ro
set_directory_permissions
wipe_bash_logout
}
# Runner setup functions
install_runner() {
echo "Downloading and setting up runner..."
cd "$RUNNER_DIR"
curl -o actions-runner-linux-arm64-2.322.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.322.0/actions-runner-linux-arm64-2.322.0.tar.gz
sudo -u ${RUNNER_USER} tar -xzf ./actions-runner-linux-arm64-2.322.0.tar.gz
sudo rm ./actions-runner-linux-arm64-2.322.0.tar.gz
sudo chmod +x ./config.sh
}
configure_runner() {
remount_rw
echo "Configuring runner..."
cd "$RUNNER_DIR"
sudo -u ${RUNNER_USER} ./config.sh --url "$REPO_URL" --token "$GITHUB_TOKEN" --name $(hostname) --runnergroup "tici-tizi" --labels "tici" --work "$BUILDS_DIR" --unattended
remount_ro
}
create_service_template() {
echo "Creating service template..."
cat <<EOL > "$RUNNER_DIR/bin/actions.runner.service.template"
[Unit]
Description={{Description}}
After=network-online.target nss-lookup.target time-sync.target
Wants=network-online.target nss-lookup.target time-sync.target
StartLimitInterval=5
StartLimitBurst=10
[Service]
Type=simple
User=root
ExecStart=/usr/bin/unshare -m -- /bin/bash -c 'mount --bind ${OPENPILOT_DIR} /data/openpilot && setpriv --reuid={{User}} --regid={{User}} --init-groups env HOME=${BASE_DIR} USER={{User}} LOGNAME={{User}} MAIL=/var/mail/{{User}} {{RunnerRoot}}/runsvc.sh'
WorkingDirectory={{RunnerRoot}}
KillMode=process
KillSignal=SIGTERM
TimeoutStopSec=5min
Restart=always
RestartSec=120
[Install]
WantedBy=multi-user.target
EOL
}
install_service() {
remount_rw
echo "Installing systemd service..."
cd "$RUNNER_DIR"
sudo ./svc.sh install $RUNNER_USER
if [ "$START_AT_BOOT" = false ]; then
local service_name
if [ -f "${RUNNER_DIR}/.service" ]; then
service_name=$(cat "${RUNNER_DIR}/.service")
else
service_name="actions.runner.sunnypilot.$(uname -n)"
fi
sudo systemctl disable "${service_name}"
fi
remount_ro
}
check_restore_prerequisites() {
local needs_restore=false
local can_restore=false
local service_name=""
# Check if base runner directory exists
if [ ! -d "${RUNNER_DIR}" ]; then
echo "ERROR: Runner directory ${RUNNER_DIR} does not exist"
echo "This directory is required for restore operations"
exit 1
fi
# First check if we have the required files for restoration
if [ -f "${RUNNER_DIR}/.credentials" ] && [ -f "${RUNNER_DIR}/.service" ]; then
can_restore=true
service_name=$(cat "${RUNNER_DIR}/.service")
echo "Found required runner configuration files"
else
echo "Missing required runner configuration files"
echo "Required: .credentials and .service files in ${RUNNER_DIR}"
exit 1
fi
# Then check if restoration is needed (if either service or user is missing)
if ! systemctl list-unit-files "${service_name}" &>/dev/null; then
echo "Service ${service_name} not found in systemd"
needs_restore=true
fi
if ! id "${RUNNER_USER}" &>/dev/null; then
echo "User ${RUNNER_USER} does not exist"
needs_restore=true
fi
# Only proceed if we can restore AND need to restore
if [ "$can_restore" = true ] && [ "$needs_restore" = true ]; then
echo "Restoration is needed and possible"
return 0
else
if [ "$needs_restore" = false ]; then
echo "System is already properly configured (user and service exist)"
fi
exit 0
fi
}
perform_restore() {
echo "Starting runner restoration..."
setup_directories
setup_system_configs
install_service
echo "Runner restoration completed successfully"
}
perform_install() {
echo "Starting fresh installation..."
setup_directories
setup_system_configs
install_runner
set_directory_permissions
create_service_template
configure_runner
install_service
echo "Installation completed successfully"
}
main() {
if [ "$RESTORE_MODE" = true ]; then
echo "Running in restore mode - will only restore system configurations..."
check_restore_prerequisites
perform_restore
else
# Check required arguments for normal installation
if [ -z "$GITHUB_TOKEN" ]; then
echo "Usage: $0 [--start-at-boot] [--token <github_token>] [--repo <repository_url>] [--restore]"
echo "Required argument (except for --restore): github_token"
echo "Optional arguments:"
echo " --start-at-boot Enable auto-start at boot (default: false)"
echo " --repo Repository URL (default: ${DEFAULT_REPO_URL})"
echo " --restore Restore existing runner configuration"
exit 1
fi
# Set repository URL if not provided
REPO_URL="${REPO_URL:-$DEFAULT_REPO_URL}"
perform_install
fi
echo "Starting runner service..."
cd "$RUNNER_DIR"
sudo ./svc.sh start
}
main

78
release/ci/publish.sh Executable file
View File

@@ -0,0 +1,78 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR
# Take parameters as arguments
SOURCE_DIR=$1
OUTPUT_DIR=$2
DEV_BRANCH=$3
VERSION=$4
GIT_ORIGIN=$5
EXTRA_VERSION_IDENTIFIER=$6
# Check parameters
if [ -z "$SOURCE_DIR" ] || [ -z "$OUTPUT_DIR" ]; then
echo "Error: No source or output directory provided."
exit 1
fi
if [ -z "$DEV_BRANCH" ] || [ -z "$VERSION" ]; then
echo "Error: No dev branch or version provided."
exit 1
fi
if [ -z "$GIT_ORIGIN" ]; then
echo "Error: No GIT_ORIGIN provided"
exit 1
fi
# "Tagging"
echo "#define COMMA_VERSION \"$VERSION\"" > ${OUTPUT_DIR}/common/version.h
## set git identity
#source $DIR/identity.sh
#export GIT_SSH_COMMAND="ssh -i /data/gitkey"
echo "[-] Setting up repo T=$SECONDS"
cd $OUTPUT_DIR
git init
# set git username/password
#source /data/identity.sh
git rm -rf $OUTPUT_DIR/.git || true # Doing cleanup, but it might fail if the .git doesn't exist or not allowed to delete
git remote remove origin || true # ensure cleanup
git remote add origin $GIT_ORIGIN
#git push origin -d $DEV_BRANCH || true # Ensuring we delete the remote branch if it exists as we are wiping it out
git fetch origin $DEV_BRANCH || (git checkout -b $DEV_BRANCH && git commit --allow-empty -m "sunnypilot v$VERSION release" && git push -u origin $DEV_BRANCH)
echo "[-] committing version $VERSION T=$SECONDS"
git add -f .
git commit -a -m "sunnypilot v$VERSION release"
git branch --set-upstream-to=origin/$DEV_BRANCH
# include source commit hash and build date in commit
GIT_HASH=$(git --git-dir=$SOURCE_DIR/.git rev-parse HEAD)
DATETIME=$(date '+%Y-%m-%dT%H:%M:%S')
SP_VERSION=$(cat $SOURCE_DIR/common/version.h | awk -F\" '{print $2}')
# Add built files to git
git add -f .
if [ "$EXTRA_VERSION_IDENTIFIER" = "-release" ] || [ "$EXTRA_VERSION_IDENTIFIER" = "-staging" ]; then
export VERSION=${VERSION%"$EXTRA_VERSION_IDENTIFIER"}
git commit --amend -m "sunnypilot v$VERSION"
else
git commit --amend -m "sunnypilot v$VERSION
version: sunnypilot v$SP_VERSION release
date: $DATETIME
master commit: $GIT_HASH
"
fi
git branch -m $DEV_BRANCH
# Push!
echo "[-] pushing T=$SECONDS"
git push -f origin $DEV_BRANCH

54
release/ci/squash_and_merge.py Executable file
View File

@@ -0,0 +1,54 @@
#!/usr/bin/env python3
import subprocess
import sys
import argparse
def run_git_command(command, check=True):
"""
Runs a git command and returns the trimmed stdout output.
Exits the script if the command fails.
"""
print(f"Running: {' '.join(command)}")
result = subprocess.run(command, capture_output=True, text=True)
if check and result.returncode != 0:
print(result.stdout.strip())
print(result.stderr.strip())
sys.exit(result.returncode)
return result.stdout.strip()
def main():
parser = argparse.ArgumentParser(description="Merge multiple branches with squash merges.")
parser.add_argument("--base", required=True, help="The base branch name from which the target branch will be created.")
parser.add_argument("--target", required=True, help="The target branch name to merge into.")
parser.add_argument("--title", required=False, help="Title for the commit")
parser.add_argument("branches", nargs="+", help="List of branch names to merge into the target branch.")
args = parser.parse_args()
# Checkout the base branch to ensure a common starting point.
run_git_command(["git", "checkout", args.base])
# Check if the target branch exists. If not, create it from the base branch.
branch_list = run_git_command(["git", "branch"], check=False)
branch_names = [line.strip("* ").strip() for line in branch_list.splitlines()]
if args.target in branch_names:
run_git_command(["git", "checkout", args.target])
else:
run_git_command(["git", "checkout", "-b", args.target])
# Iterate over each branch, merging it with a squash merge.
for branch in args.branches:
print(f"Merging branch '{branch}' with a squash merge.")
# Merge the branch without creating a merge commit.
run_git_command(["git", "merge", "--squash", branch])
# Commit the squashed changes with an appropriate message.
commit_message = args.title or f"Squashed merge of branch '{branch}'"
run_git_command(["git", "commit", "-m", commit_message])
print(f"All branches have been merged with squashed commits into '{args.target}'.")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,206 @@
#!/usr/bin/env python3
import subprocess
import sys
import os
import argparse
import json
from datetime import datetime
TRUST_FORK_LABEL = "trust-fork-pr"
def setup_argument_parser():
parser = argparse.ArgumentParser(description='Process and squash GitHub PRs')
parser.add_argument('--pr-data', type=str, help='PR data in JSON format')
parser.add_argument('--source-branch', type=str, default='master-new',
help='Source branch for merging')
parser.add_argument('--target-branch', type=str, default='master-dev-c3-new-test',
help='Target branch for merging')
parser.add_argument('--squash-script-path', type=str, required=True,
help='Path to the squash_and_merge.py script')
return parser
def validate_squash_script(script_path):
if not os.path.isfile(script_path):
raise FileNotFoundError(f"Squash script not found at: {script_path}")
if not os.access(script_path, os.X_OK):
raise PermissionError(f"Squash script is not executable: {script_path}")
def sort_prs_by_creation(pr_data):
"""Sort PRs by creation date"""
nodes = (pr_data.get('data', {}).get('search', {}).get('nodes', []))
return sorted(
nodes,
key=lambda x: datetime.fromisoformat(x.get('createdAt', '').replace('Z', '+00:00'))
)
def add_pr_comments(pr_number, comments: list[str]):
"""Adds or updates a comment with multiple comments to a PR using gh cli"""
comment = "\n___\n".join(comments)
_add_pr_comment(pr_number, comment)
def _add_pr_comment(pr_number, comment):
"""Add or update a comment to a PR using gh cli"""
title = "## Squash and Merge"
try:
full_comment = f"{title}\n\n{comment}"
subprocess.run(
['gh', 'pr', 'comment', '--edit-last', '--create-if-none', f"#{pr_number}", '--body', full_comment],
check=True,
capture_output=True,
text=True
)
except subprocess.CalledProcessError as e:
print(f"Failed to add/update comment on PR #{pr_number}: {e.stderr}")
except json.JSONDecodeError:
print(f"Failed to parse comments data for PR #{pr_number}")
def validate_pr(pr):
"""Validate a PR and return (is_valid, skip_reason)"""
pr_number = pr.get('number', 'UNKNOWN')
branch = pr.get('headRefName', '')
if not branch:
return False, f"missing branch name for PR #{pr_number}"
# Check if checks have passed
commits = pr.get('commits', {}).get('nodes', [])
if not commits:
return False, "no commit data found"
status = commits[0].get('commit', {}).get('statusCheckRollup', {})
if not status or status.get('state') != 'SUCCESS':
return False, "not all checks have passed"
# Check for merge conflicts
merge_status = subprocess.run(['gh', 'pr', 'view', str(pr_number), '--json', 'mergeable,mergeStateStatus'],
capture_output=True, text=True)
merge_data = json.loads(merge_status.stdout)
if not merge_data.get('mergeable'):
return False, "merge conflicts detected"
# if (mergeStateStatus := merge_data.get('mergeStateStatus')) == "BEHIND":
# return False, f"branch is `{mergeStateStatus}`"
return True, None
def process_pr(pr_data, source_branch, target_branch, squash_script_path):
try:
nodes = sort_prs_by_creation(pr_data)
if not nodes:
print("No PRs to squash")
return 0
print(f"Deleting target branch {target_branch}")
subprocess.run(['git', 'branch', '-D', target_branch], check=False)
subprocess.run(['git', 'branch', target_branch, f'origin/{source_branch}'], check=True)
success_count = 0
for pr in nodes:
pr_comments = []
try:
pr_number = pr.get('number', 'UNKNOWN')
branch = pr.get('headRefName', '')
title = pr.get('title', '')
head_repository = pr.get('headRepository', {})
pr_labels = pr.get('labels', {}).get('nodes', [])
is_fork = head_repository.get('isFork', False)
trust_fork = any(label.get('name') == TRUST_FORK_LABEL for label in pr_labels)
is_valid, skip_reason = validate_pr(pr)
origin = "origin" if not head_repository.get('isFork', False) else head_repository.get('nameWithOwner', 'origin')
if is_fork and trust_fork:
print(f"Removing label `{TRUST_FORK_LABEL}` from PR #{pr_number} as it is being processed")
subprocess.run(['gh', 'pr', 'edit', str(pr_number), '--remove-label', TRUST_FORK_LABEL], check=True)
pr_comments.append(f" This PR is from a fork. The `{TRUST_FORK_LABEL}` label was removed as it is being processed right now.")
print(f"Adding remote {origin} for PR #{pr_number}")
subprocess.run(['git', 'remote', 'add', origin, head_repository.get('url')], check=False)
if is_fork and not trust_fork:
pr_comments.append(
f"⚠️ This PR is from a fork. Please add the `{TRUST_FORK_LABEL}` label to include it in the squash." +
"\n**Note**: The label is removed after the squash is done and must be added again for the next execution for security reasons."
)
continue
if not is_valid:
print(f"Warning: {skip_reason} for PR #{pr_number}, skipping")
pr_comments.append(f"⚠️ This PR was skipped in the automated `{target_branch}` squash because **{skip_reason}**.")
continue
# Fetch PR branch
subprocess.run(['git', 'fetch', origin, branch], check=True)
# Delete branch if it exists (ignore errors if it doesn't)
subprocess.run(['git', 'branch', '-D', branch], check=False)
# Create new branch pointing to origin's branch
subprocess.run(['git', 'branch', branch, f'{origin}/{branch}'], check=True)
# Run squash script
result = subprocess.run([
squash_script_path,
'--target', target_branch,
'--base', source_branch,
'--title', f"{title} (PR-{pr_number})",
branch,
], capture_output=True, text=True)
print(result.stdout)
if result.returncode == 0:
print(f"Successfully processed PR #{pr_number}")
success_count += 1
continue
print(f"Error processing PR #{pr_number}:")
print(f"Command failed with exit code {result.returncode}")
output = result.stdout
print(f"Error output: {output}")
pr_comments.append(f"⚠️ Error during automated `{target_branch}` squash:\n```\n{output}\n```")
subprocess.run(['git', 'reset', '--hard'], check=True)
continue
except Exception as e:
print(f"Unexpected error processing PR #{pr_number}: {str(e)}")
pr_comments.append(f"⚠️ Unexpected error during automated `{target_branch}` squash:\n```\n{str(e)}\n```")
subprocess.run(['git', 'reset', '--hard'], check=True)
continue
finally:
if pr_comments:
add_pr_comments(pr_number, pr_comments) # This "commits" all the comments generated on this run before leaving loop on continue.
return success_count
except Exception as e:
import traceback
print(f"Error in process_pr: {str(e)}")
print("Full traceback:")
print(traceback.format_exc())
return 0
def main():
parser = setup_argument_parser()
try:
args = parser.parse_args()
validate_squash_script(args.squash_script_path)
pr_data_json = json.loads(args.pr_data)
# Process the PRs
success_count = process_pr(pr_data_json, args.source_branch, args.target_branch, args.squash_script_path)
print(f"Successfully processed {success_count} PRs")
except Exception as e:
print(f"Fatal error: {str(e)}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# Determine BASE_DIR based on mount point
if mountpoint -q /data/media; then
GITHUB_BASE_DIR="/data/media/0/github"
else
GITHUB_BASE_DIR="/data/github"
fi
# Define directories and user
BIN_DIR="$GITHUB_BASE_DIR/bin"
BUILDS_DIR="$GITHUB_BASE_DIR/builds"
OPENPILOT_DIR="$GITHUB_BASE_DIR/openpilot"
LOGS_DIR="$GITHUB_BASE_DIR/logs"
CACHE_DIR="$GITHUB_BASE_DIR/cache"
RUNNER_USERNAME="github-runner"
# Define the systemd service name
SERVICE_NAME="github-runner"
USER_GROUPS="comma,gpu,gpio,sudo"
# Function to stop and disable the systemd service
stop_and_uninstall_service() {
cd $GITHUB_BASE_DIR/runner
sudo ./svc.sh stop
sudo ./svc.sh uninstall
}
# Function to remove the systemd service file
remove_runner() {
cd $GITHUB_BASE_DIR/runner
sudo rm .runner
sudo su -c './config.sh remove' github-runner
}
# Function to delete the Github Runner directories
delete_directories() {
sudo rm -rf "$BIN_DIR/github-runner"
sudo rm -rf "$GITHUB_BASE_DIR" "$BIN_DIR" "$BUILDS_DIR" "$LOGS_DIR" "$CACHE_DIR" "$OPENPILOT_DIR"
}
# Function to remove the Github Runner user
delete_user() {
for group in ${USER_GROUPS//,/ }
do
sudo gpasswd -d ${RUNNER_USERNAME} ${group}
done
sudo userdel -r ${RUNNER_USERNAME}
}
# Function to remove sudoers entry
remove_sudoers_entry() {
sudo sed -i.bak "/${RUNNER_USERNAME} ALL=(ALL) NOPASSWD: ALL/d" /etc/sudoers
}
# Make filesystem writable
sudo mount -o remount rw /
# Ensure filesystem is remounted as read-only on script exit
trap "sudo mount -o remount ro /" EXIT
# Call functions
stop_and_uninstall_service
remove_runner
delete_directories
delete_user
remove_sudoers_entry
# End of uninstall script

View File

@@ -23,6 +23,7 @@ blacklist = [
whitelist: list[str] = [
]
if __name__ == "__main__":
for f in Path(ROOT).rglob("**/*"):
if not (f.is_file() or f.is_symlink()):

View File

@@ -13,7 +13,7 @@ cd $ROOT
FAILED=0
IGNORED_FILES="uv\.lock|docs\/CARS.md"
IGNORED_FILES="uv\.lock|docs\/CARS.md|LICENSE\.md"
IGNORED_DIRS="^third_party.*|^msgq.*|^msgq_repo.*|^opendbc.*|^opendbc_repo.*|^cereal.*|^panda.*|^rednose.*|^rednose_repo.*|^tinygrad.*|^tinygrad_repo.*|^teleoprtc.*|^teleoprtc_repo.*"
function run() {
@@ -56,7 +56,7 @@ function run_tests() {
if [[ -z "$FAST" ]]; then
run "mypy" mypy $PYTHON_FILES
run "codespell" codespell $ALL_FILES
run "codespell" codespell $ALL_FILES --ignore-words=$ROOT/.codespellignore
fi
return $FAILED

27
scripts/manage-powersave.py Executable file
View File

@@ -0,0 +1,27 @@
#!/usr/bin/env python3
import argparse
import multiprocessing
from openpilot.system.hardware import HARDWARE
def main():
parser = argparse.ArgumentParser(description='Control power saving mode')
parser.add_argument('--enable', action='store_true', help='Enable power saving mode')
parser.add_argument('--disable', action='store_true', help='Disable power saving mode')
args = parser.parse_args()
if args.enable and args.disable:
parser.error("Cannot specify both --enable and --disable")
elif not (args.enable or args.disable):
parser.error("Must specify either --enable or --disable")
print(f"Number of CPU cores available before: [{multiprocessing.cpu_count()}]")
HARDWARE.set_power_save(args.enable)
state = "enabled" if args.enable else "disabled"
print(f"Power save mode set to: [{state}]")
print(f"Number of CPU cores available now: [{multiprocessing.cpu_count()}]")
if __name__ == "__main__":
main()

View File

@@ -1,11 +1,12 @@
#!/usr/bin/env python3
import json
import os
import time
import threading
import cereal.messaging as messaging
from cereal import car, log
from cereal import car, log, custom
from openpilot.common.params import Params
from openpilot.common.realtime import config_realtime_process, Priority, Ratekeeper
@@ -21,6 +22,10 @@ from opendbc.safety import ALTERNATIVE_EXPERIENCE
from openpilot.selfdrive.pandad import can_capnp_to_list, can_list_to_can_capnp
from openpilot.selfdrive.car.cruise import VCruiseHelper
from openpilot.selfdrive.car.car_specific import MockCarState
from openpilot.selfdrive.car.helpers import convert_carControlSP, convert_to_capnp
from openpilot.sunnypilot.mads.helpers import set_alternative_experience, set_car_specific_params
from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfaces
REPLAY = "REPLAY" in os.environ
@@ -63,11 +68,13 @@ class Car:
CI: CarInterfaceBase
RI: RadarInterfaceBase
CP: car.CarParams
CP_SP: structs.CarParamsSP
CP_SP_capnp: custom.CarParamsSP
def __init__(self, CI=None, RI=None) -> None:
self.can_sock = messaging.sub_sock('can', timeout=20)
self.sm = messaging.SubMaster(['pandaStates', 'carControl', 'onroadEvents'])
self.pm = messaging.PubMaster(['sendcan', 'carState', 'carParams', 'carOutput', 'liveTracks'])
self.sm = messaging.SubMaster(['pandaStates', 'carControl', 'onroadEvents'] + ['carControlSP'])
self.pm = messaging.PubMaster(['sendcan', 'carState', 'carParams', 'carOutput', 'liveTracks'] + ['carParamsSP'])
self.can_rcv_cum_timeout_counter = 0
@@ -98,14 +105,18 @@ class Car:
with car.CarParams.from_bytes(cached_params_raw) as _cached_params:
cached_params = _cached_params
self.CI = get_car(*self.can_callbacks, obd_callback(self.params), experimental_long_allowed, num_pandas, cached_params)
self.RI = interfaces[self.CI.CP.carFingerprint].RadarInterface(self.CI.CP)
fixed_fingerprint = json.loads(self.params.get("CarPlatformBundle", encoding='utf-8') or "{}").get("platform", None)
self.CI = get_car(*self.can_callbacks, obd_callback(self.params), experimental_long_allowed, num_pandas, cached_params, fixed_fingerprint)
sunnypilot_interfaces.setup_interfaces(self.CI.CP, self.CI.CP_SP, self.params)
self.RI = interfaces[self.CI.CP.carFingerprint].RadarInterface(self.CI.CP, self.CI.CP_SP)
self.CP = self.CI.CP
self.CP_SP = self.CI.CP_SP
# continue onto next fingerprinting step in pandad
self.params.put_bool("FirmwareQueryDone", True)
else:
self.CI, self.CP = CI, CI.CP
self.CI, self.CP, self.CP_SP = CI, CI.CP, CI.CP_SP
self.RI = RI
# set alternative experiences from parameters
@@ -114,6 +125,13 @@ class Car:
if not disengage_on_accelerator:
self.CP.alternativeExperience |= ALTERNATIVE_EXPERIENCE.DISABLE_DISENGAGE_ON_GAS
# mads
set_alternative_experience(self.CP, self.params)
set_car_specific_params(self.CP, self.CP_SP, self.params)
# Dynamic Experimental Control
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
openpilot_enabled_toggle = self.params.get_bool("OpenpilotEnabledToggle")
controller_available = self.CI.CC is not None and openpilot_enabled_toggle and not self.CP.dashcamOnly
@@ -156,6 +174,14 @@ class Car:
self.params.put_nonblocking("CarParamsCache", cp_bytes)
self.params.put_nonblocking("CarParamsPersistent", cp_bytes)
# Write CarParamsSP for controls
# convert to pycapnp representation for caching and logging
self.CP_SP_capnp = convert_to_capnp(self.CP_SP)
cp_sp_bytes = self.CP_SP_capnp.to_bytes()
self.params.put("CarParamsSP", cp_sp_bytes)
self.params.put_nonblocking("CarParamsSPCache", cp_sp_bytes)
self.params.put_nonblocking("CarParamsSPPersistent", cp_sp_bytes)
self.mock_carstate = MockCarState()
self.v_cruise_helper = VCruiseHelper(self.CP)
@@ -165,6 +191,9 @@ class Car:
# card is driven by can recv, expected at 100Hz
self.rk = Ratekeeper(100, print_delay_threshold=None)
# log fingerprint in sentry
sunnypilot_interfaces.log_fingerprint(self.CP)
def state_update(self) -> tuple[car.CarState, structs.RadarDataT | None]:
"""carState update loop, driven by can"""
@@ -193,7 +222,7 @@ class Car:
self.v_cruise_helper.update_v_cruise(CS, self.sm['carControl'].enabled, self.is_metric)
if self.sm['carControl'].enabled and not self.CC_prev.enabled:
# Use CarState w/ buttons from the step selfdrived enables on
self.v_cruise_helper.initialize_v_cruise(self.CS_prev, self.experimental_mode)
self.v_cruise_helper.initialize_v_cruise(self.CS_prev, self.experimental_mode, self.dynamic_experimental_control)
# TODO: mirror the carState.cruiseState struct?
CS.vCruise = float(self.v_cruise_helper.v_cruise_kph)
@@ -231,20 +260,28 @@ class Car:
tracks_msg.liveTracks = RD
self.pm.send('liveTracks', tracks_msg)
def controls_update(self, CS: car.CarState, CC: car.CarControl):
# carParamsSP - logged every 50 seconds (> 1 per segment)
if self.sm.frame % int(50. / DT_CTRL) == 0:
cp_sp_send = messaging.new_message('carParamsSP')
cp_sp_send.valid = True
cp_sp_send.carParamsSP = self.CP_SP_capnp
self.pm.send('carParamsSP', cp_sp_send)
def controls_update(self, CS: car.CarState, CC: car.CarControl, CC_SP: custom.CarControlSP):
"""control update loop, driven by carControl"""
if not self.initialized_prev:
# Initialize CarInterface, once controls are ready
# TODO: this can make us miss at least a few cycles when doing an ECU knockout
self.CI.init(self.CP, *self.can_callbacks)
self.CI.init(self.CP, self.CP_SP, *self.can_callbacks)
sunnypilot_interfaces.init_interfaces(self.CP, self.CP_SP, self.params, *self.can_callbacks)
# signal pandad to switch to car safety mode
self.params.put_bool_nonblocking("ControlsReady", True)
if self.sm.all_alive(['carControl']):
# send car controls over can
now_nanos = self.can_log_mono_time if REPLAY else int(time.monotonic() * 1e9)
self.last_actuators_output, can_sends = self.CI.apply(CC, now_nanos)
self.last_actuators_output, can_sends = self.CI.apply(CC, convert_carControlSP(CC_SP), now_nanos)
self.pm.send('sendcan', can_list_to_can_capnp(can_sends, msgtype='sendcan', valid=CS.canValid))
self.CC_prev = CC
@@ -257,7 +294,7 @@ class Car:
initialized = (not any(e.name == EventName.selfdriveInitializing for e in self.sm['onroadEvents']) and
self.sm.seen['onroadEvents'])
if not self.CP.passive and initialized:
self.controls_update(CS, self.sm['carControl'])
self.controls_update(CS, self.sm['carControl'], self.sm['carControlSP'])
self.initialized_prev = initialized
self.CS_prev = CS
@@ -266,6 +303,10 @@ class Car:
while not evt.is_set():
self.is_metric = self.params.get_bool("IsMetric")
self.experimental_mode = self.params.get_bool("ExperimentalMode") and self.CP.openpilotLongitudinalControl
# sunnypilot
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
time.sleep(0.1)
def card_thread(self):

View File

@@ -123,12 +123,13 @@ class VCruiseHelper:
self.button_timers[b.type.raw] = 1 if b.pressed else 0
self.button_change_states[b.type.raw] = {"standstill": CS.cruiseState.standstill, "enabled": enabled}
def initialize_v_cruise(self, CS, experimental_mode: bool) -> None:
def initialize_v_cruise(self, CS, experimental_mode: bool, dynamic_experimental_control: bool) -> None:
# initializing is handled by the PCM
if self.CP.pcmCruise:
return
initial = V_CRUISE_INITIAL_EXPERIMENTAL_MODE if experimental_mode else V_CRUISE_INITIAL
initial_experimental_mode = experimental_mode and not dynamic_experimental_control
initial = V_CRUISE_INITIAL_EXPERIMENTAL_MODE if initial_experimental_mode else V_CRUISE_INITIAL
if any(b.type in (ButtonType.accelCruise, ButtonType.resumeCruise) for b in CS.buttonEvents) and self.v_cruise_initialized:
self.v_cruise_kph = self.v_cruise_kph_last

59
selfdrive/car/helpers.py Normal file
View File

@@ -0,0 +1,59 @@
import capnp
from typing import Any
from cereal import custom
from opendbc.car import structs
_FIELDS = '__dataclass_fields__' # copy of dataclasses._FIELDS
def is_dataclass(obj):
"""Similar to dataclasses.is_dataclass without instance type check checking"""
return hasattr(obj, _FIELDS)
def _asdictref_inner(obj) -> dict[str, Any] | Any:
if is_dataclass(obj):
ret = {}
for field in getattr(obj, _FIELDS): # similar to dataclasses.fields()
ret[field] = _asdictref_inner(getattr(obj, field))
return ret
elif isinstance(obj, (tuple, list)):
return type(obj)(_asdictref_inner(v) for v in obj)
else:
return obj
def asdictref(obj) -> dict[str, Any]:
"""
Similar to dataclasses.asdict without recursive type checking and copy.deepcopy
Note that the resulting dict will contain references to the original struct as a result
"""
if not is_dataclass(obj):
raise TypeError("asdictref() should be called on dataclass instances")
return _asdictref_inner(obj)
def convert_to_capnp(struct: structs.CarParamsSP) -> capnp.lib.capnp._DynamicStructBuilder:
struct_dict = asdictref(struct)
if isinstance(struct, structs.CarParamsSP):
struct_capnp = custom.CarParamsSP.new_message(**struct_dict)
else:
raise ValueError(f"Unsupported struct type: {type(struct)}")
return struct_capnp
def convert_carControlSP(struct: capnp.lib.capnp._DynamicStructReader) -> structs.CarControlSP:
# TODO: recursively handle any car struct as needed
def remove_deprecated(s: dict) -> dict:
return {k: v for k, v in s.items() if not k.endswith('DEPRECATED')}
struct_dict = struct.to_dict()
struct_dataclass = structs.CarControlSP(**remove_deprecated({k: v for k, v in struct_dict.items() if not isinstance(k, dict)}))
struct_dataclass.mads = structs.ModularAssistiveDrivingSystem(**remove_deprecated(struct_dict.get('mads', {})))
return struct_dataclass

View File

@@ -4,7 +4,7 @@ import hypothesis.strategies as st
from hypothesis import Phase, given, settings
from parameterized import parameterized
from cereal import car
from cereal import car, custom
from opendbc.car import DT_CTRL
from opendbc.car.car_helpers import interfaces
from opendbc.car.structs import CarParams
@@ -12,12 +12,15 @@ from opendbc.car.tests.test_car_interfaces import get_fuzzy_car_interface_args
from opendbc.car.fw_versions import FW_VERSIONS, FW_QUERY_CONFIGS
from opendbc.car.mock.values import CAR as MOCK
from opendbc.car.values import PLATFORMS
from openpilot.selfdrive.car.helpers import convert_carControlSP
from openpilot.selfdrive.controls.lib.latcontrol_angle import LatControlAngle
from openpilot.selfdrive.controls.lib.latcontrol_pid import LatControlPID
from openpilot.selfdrive.controls.lib.latcontrol_torque import LatControlTorque
from openpilot.selfdrive.controls.lib.longcontrol import LongControl
from openpilot.selfdrive.test.fuzzy_generation import FuzzyGenerator
from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfaces
ALL_ECUS = {ecu for ecus in FW_VERSIONS.values() for ecu in ecus.keys()}
ALL_ECUS |= {ecu for config in FW_QUERY_CONFIGS.values() for ecu in config.extra_ecus}
@@ -40,9 +43,13 @@ class TestCarInterfaces:
car_params = CarInterface.get_params(car_name, args['fingerprints'], args['car_fw'],
experimental_long=args['experimental_long'], docs=False)
car_params_sp = CarInterface.get_params_sp(car_params, car_name, args['fingerprints'], args['car_fw'],
experimental_long=args['experimental_long'], docs=False)
sunnypilot_interfaces.setup_interfaces(car_params, car_params_sp)
car_params = car_params.as_reader()
car_interface = CarInterface(car_params)
car_interface = CarInterface(car_params, car_params_sp)
assert car_params
assert car_params_sp
assert car_interface
assert car_params.mass > 1
@@ -69,13 +76,16 @@ class TestCarInterfaces:
assert not math.isnan(tune.torque.friction) and tune.torque.friction > 0
cc_msg = FuzzyGenerator.get_random_msg(data.draw, car.CarControl, real_floats=True)
cc_sp_msg = FuzzyGenerator.get_random_msg(data.draw, custom.CarControlSP, real_floats=True)
# Run car interface
now_nanos = 0
CC = car.CarControl.new_message(**cc_msg)
CC = CC.as_reader()
CC_SP = custom.CarControlSP.new_message(**cc_sp_msg)
CC_SP = convert_carControlSP(CC_SP.as_reader())
for _ in range(10):
car_interface.update([])
car_interface.apply(CC, now_nanos)
car_interface.apply(CC, CC_SP, now_nanos)
now_nanos += DT_CTRL * 1e9 # 10 ms
CC = car.CarControl.new_message(**cc_msg)
@@ -85,7 +95,7 @@ class TestCarInterfaces:
CC = CC.as_reader()
for _ in range(10):
car_interface.update([])
car_interface.apply(CC, now_nanos)
car_interface.apply(CC, CC_SP, now_nanos)
now_nanos += DT_CTRL * 1e9 # 10ms
# Test controller initialization
@@ -93,8 +103,8 @@ class TestCarInterfaces:
# hypothesis also slows down significantly with just one more message draw
LongControl(car_params)
if car_params.steerControlType == CarParams.SteerControlType.angle:
LatControlAngle(car_params, car_interface)
LatControlAngle(car_params, car_params_sp, car_interface)
elif car_params.lateralTuning.which() == 'pid':
LatControlPID(car_params, car_interface)
LatControlPID(car_params, car_params_sp, car_interface)
elif car_params.lateralTuning.which() == 'torque':
LatControlTorque(car_params, car_interface)
LatControlTorque(car_params, car_params_sp, car_interface)

View File

@@ -57,16 +57,16 @@ class TestVCruiseHelper:
for _ in range(2):
self.v_cruise_helper.update_v_cruise(car.CarState(cruiseState={"available": False}), enabled=False, is_metric=False)
def enable(self, v_ego, experimental_mode):
def enable(self, v_ego, experimental_mode, dynamic_experimental_control):
# Simulates user pressing set with a current speed
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), experimental_mode)
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), experimental_mode, dynamic_experimental_control)
def test_adjust_speed(self):
"""
Asserts speed changes on falling edges of buttons.
"""
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
for btn in (ButtonType.accelCruise, ButtonType.decelCruise):
for pressed in (True, False):
@@ -90,7 +90,7 @@ class TestVCruiseHelper:
CS.buttonEvents = [ButtonEvent(type=ButtonType.decelCruise, pressed=pressed)]
self.v_cruise_helper.update_v_cruise(CS, enabled=enabled, is_metric=False)
if pressed:
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
# Expected diff on enabling. Speed should not change on falling edge of pressed
assert not pressed == self.v_cruise_helper.v_cruise_kph == self.v_cruise_helper.v_cruise_kph_last
@@ -100,7 +100,7 @@ class TestVCruiseHelper:
Asserts we don't increment set speed if user presses resume/accel to exit cruise standstill.
"""
self.enable(0, False)
self.enable(0, False, False)
for standstill in (True, False):
for pressed in (True, False):
@@ -120,7 +120,7 @@ class TestVCruiseHelper:
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
# first decrement speed, then perform gas pressed logic
expected_v_cruise_kph = self.v_cruise_helper.v_cruise_kph - IMPERIAL_INCREMENT
@@ -142,10 +142,11 @@ class TestVCruiseHelper:
"""
for experimental_mode in (True, False):
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
for dynamic_experimental_control in (True, False):
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego), experimental_mode)
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego), experimental_mode, dynamic_experimental_control)
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized

View File

@@ -1,4 +1,5 @@
import time
import copy
import os
import pytest
import random
@@ -153,7 +154,9 @@ class TestCarModelBase(unittest.TestCase):
cls.CarInterface = interfaces[cls.platform]
cls.CP = cls.CarInterface.get_params(cls.platform, cls.fingerprint, car_fw, experimental_long, docs=False)
cls.CP_SP = cls.CarInterface.get_params_sp(cls.CP, cls.platform, cls.fingerprint, car_fw, experimental_long, docs=False)
assert cls.CP
assert cls.CP_SP
assert cls.CP.carFingerprint == cls.platform
os.environ["COMMA_CACHE"] = DEFAULT_DOWNLOAD_CACHE_ROOT
@@ -163,12 +166,15 @@ class TestCarModelBase(unittest.TestCase):
del cls.can_msgs
def setUp(self):
self.CI = self.CarInterface(self.CP.copy())
self.CI = self.CarInterface(self.CP.copy(), copy.deepcopy(self.CP_SP))
assert self.CI
# TODO: check safetyModel is in release panda build
self.safety = libsafety_py.libsafety
safety_param_sp = self.CP_SP.safetyParam
self.safety.set_current_safety_param_sp(safety_param_sp)
cfg = self.CP.safetyConfigs[-1]
set_status = self.safety.set_safety_hooks(cfg.safetyModel.raw, cfg.safetyParam)
self.assertEqual(0, set_status, f"failed to set safetyModel {cfg}")
@@ -195,10 +201,11 @@ class TestCarModelBase(unittest.TestCase):
can_invalid_cnt = 0
can_valid = False
CC = structs.CarControl().as_reader()
CC_SP = structs.CarControlSP()
for i, msg in enumerate(self.can_msgs):
CS = self.CI.update(msg)
self.CI.apply(CC, msg[0])
self.CI.apply(CC, CC_SP, msg[0])
if CS.canValid:
can_valid = True
@@ -210,7 +217,7 @@ class TestCarModelBase(unittest.TestCase):
self.assertEqual(can_invalid_cnt, 0)
def test_radar_interface(self):
RI = self.CarInterface.RadarInterface(self.CP)
RI = self.CarInterface.RadarInterface(self.CP, self.CP_SP)
assert RI
# Since OBD port is multiplexed to bus 1 (commonly radar bus) while fingerprinting,
@@ -270,13 +277,13 @@ class TestCarModelBase(unittest.TestCase):
if self.CP.notCar:
self.skipTest("Skipping test for notCar")
def test_car_controller(car_control):
def test_car_controller(car_control, car_control_sp):
now_nanos = 0
msgs_sent = 0
CI = self.CarInterface(self.CP)
CI = self.CarInterface(self.CP, self.CP_SP)
for _ in range(round(10.0 / DT_CTRL)): # make sure we hit the slowest messages
CI.update([])
_, sendcan = CI.apply(car_control, now_nanos)
_, sendcan = CI.apply(car_control, car_control_sp, now_nanos)
now_nanos += DT_CTRL * 1e9
msgs_sent += len(sendcan)
@@ -289,17 +296,18 @@ class TestCarModelBase(unittest.TestCase):
# Make sure we can send all messages while inactive
CC = structs.CarControl()
test_car_controller(CC.as_reader())
CC_SP = structs.CarControlSP()
test_car_controller(CC.as_reader(), CC_SP)
# Test cancel + general messages (controls_allowed=False & cruise_engaged=True)
self.safety.set_cruise_engaged_prev(True)
CC = structs.CarControl(cruiseControl=structs.CarControl.CruiseControl(cancel=True))
test_car_controller(CC.as_reader())
test_car_controller(CC.as_reader(), CC_SP)
# Test resume + general messages (controls_allowed=True & cruise_engaged=True)
self.safety.set_controls_allowed(True)
CC = structs.CarControl(cruiseControl=structs.CarControl.CruiseControl(resume=True))
test_car_controller(CC.as_reader())
test_car_controller(CC.as_reader(), CC_SP)
# Skip stdout/stderr capture with pytest, causes elevated memory usage
@pytest.mark.nocapture

View File

@@ -2,7 +2,7 @@
import math
from typing import SupportsFloat
from cereal import car, log
from cereal import car, log, custom
import cereal.messaging as messaging
from openpilot.common.conversions import Conversions as CV
from openpilot.common.params import Params
@@ -33,12 +33,17 @@ class Controls:
self.CP = messaging.log_from_bytes(self.params.get("CarParams", block=True), car.CarParams)
cloudlog.info("controlsd got CarParams")
self.CI = interfaces[self.CP.carFingerprint](self.CP)
cloudlog.info("controlsd is waiting for CarParamsSP")
self.CP_SP = messaging.log_from_bytes(self.params.get("CarParamsSP", block=True), custom.CarParamsSP)
cloudlog.info("controlsd got CarParamsSP")
self.CI = interfaces[self.CP.carFingerprint](self.CP, self.CP_SP)
self.sm = messaging.SubMaster(['liveParameters', 'liveTorqueParameters', 'modelV2', 'selfdriveState',
'liveCalibration', 'livePose', 'longitudinalPlan', 'carState', 'carOutput',
'driverMonitoringState', 'onroadEvents', 'driverAssistance'], poll='selfdriveState')
self.pm = messaging.PubMaster(['carControl', 'controlsState'])
'driverMonitoringState', 'onroadEvents', 'driverAssistance'] + ['selfdriveStateSP'],
poll='selfdriveState')
self.pm = messaging.PubMaster(['carControl', 'controlsState'] + ['carControlSP'])
self.steer_limited_by_controls = False
self.curvature = 0.0
@@ -51,11 +56,11 @@ class Controls:
self.VM = VehicleModel(self.CP)
self.LaC: LatControl
if self.CP.steerControlType == car.CarParams.SteerControlType.angle:
self.LaC = LatControlAngle(self.CP, self.CI)
self.LaC = LatControlAngle(self.CP, self.CP_SP, self.CI)
elif self.CP.lateralTuning.which() == 'pid':
self.LaC = LatControlPID(self.CP, self.CI)
self.LaC = LatControlPID(self.CP, self.CP_SP, self.CI)
elif self.CP.lateralTuning.which() == 'torque':
self.LaC = LatControlTorque(self.CP, self.CI)
self.LaC = LatControlTorque(self.CP, self.CP_SP, self.CI)
def update(self):
self.sm.update(15)
@@ -84,6 +89,8 @@ class Controls:
self.LaC.update_live_torque_params(torque_params.latAccelFactorFiltered, torque_params.latAccelOffsetFiltered,
torque_params.frictionCoefficientFiltered)
self.LaC.extension.update_model_v2(self.sm['modelV2'])
long_plan = self.sm['longitudinalPlan']
model_v2 = self.sm['modelV2']
@@ -92,7 +99,14 @@ class Controls:
# Check which actuators can be enabled
standstill = abs(CS.vEgo) <= max(self.CP.minSteerSpeed, 0.3) or CS.standstill
CC.latActive = self.sm['selfdriveState'].active and not CS.steerFaultTemporary and not CS.steerFaultPermanent and \
ss_sp = self.sm['selfdriveStateSP']
if ss_sp.mads.available:
_lat_active = ss_sp.mads.active
else:
_lat_active = self.sm['selfdriveState'].active
CC.latActive = _lat_active and not CS.steerFaultTemporary and not CS.steerFaultPermanent and \
(not standstill or self.CP.steerAtStandstill)
CC.longActive = CC.enabled and not any(e.overrideLongitudinal for e in self.sm['onroadEvents']) and self.CP.openpilotLongitudinalControl
@@ -134,9 +148,12 @@ class Controls:
cloudlog.error(f"actuators.{p} not finite {actuators.to_dict()}")
setattr(actuators, p, 0.0)
return CC, lac_log
CC_SP = custom.CarControlSP.new_message()
CC_SP.mads = ss_sp.mads
def publish(self, CC, lac_log):
return CC, CC_SP, lac_log
def publish(self, CC, CC_SP, lac_log):
CS = self.sm['carState']
# Orientation and angle rates can be useful for carcontroller
@@ -209,12 +226,18 @@ class Controls:
cc_send.carControl = CC
self.pm.send('carControl', cc_send)
# carControlSP
cc_sp_send = messaging.new_message('carControlSP')
cc_sp_send.valid = CS.canValid
cc_sp_send.carControlSP = CC_SP
self.pm.send('carControlSP', cc_sp_send)
def run(self):
rk = Ratekeeper(100, print_delay_threshold=None)
while True:
self.update()
CC, lac_log = self.state_control()
self.publish(CC, lac_log)
CC, CC_SP, lac_log = self.state_control()
self.publish(CC, CC_SP, lac_log)
rk.monitor_time()

View File

@@ -1,6 +1,7 @@
from cereal import log
from openpilot.common.conversions import Conversions as CV
from openpilot.common.realtime import DT_MDL
from openpilot.sunnypilot.selfdrive.controls.lib.auto_lane_change import AutoLaneChangeController
LaneChangeState = log.LaneChangeState
LaneChangeDirection = log.LaneChangeDirection
@@ -39,8 +40,10 @@ class DesireHelper:
self.keep_pulse_timer = 0.0
self.prev_one_blinker = False
self.desire = log.Desire.none
self.alc = AutoLaneChangeController(self)
def update(self, carstate, lateral_active, lane_change_prob):
self.alc.update_params()
v_ego = carstate.vEgo
one_blinker = carstate.leftBlinker != carstate.rightBlinker
below_lane_change_speed = v_ego < LANE_CHANGE_SPEED_MIN
@@ -67,10 +70,12 @@ class DesireHelper:
blindspot_detected = ((carstate.leftBlindspot and self.lane_change_direction == LaneChangeDirection.left) or
(carstate.rightBlindspot and self.lane_change_direction == LaneChangeDirection.right))
self.alc.update_lane_change(blindspot_detected, carstate.brakePressed)
if not one_blinker or below_lane_change_speed:
self.lane_change_state = LaneChangeState.off
self.lane_change_direction = LaneChangeDirection.none
elif torque_applied and not blindspot_detected:
elif (torque_applied or self.alc.auto_lane_change_allowed) and not blindspot_detected:
self.lane_change_state = LaneChangeState.laneChangeStarting
# LaneChangeState.laneChangeStarting
@@ -112,3 +117,5 @@ class DesireHelper:
self.keep_pulse_timer = 0.0
elif self.desire in (log.Desire.keepLeft, log.Desire.keepRight):
self.desire = log.Desire.none
self.alc.update_state()

View File

@@ -5,7 +5,7 @@ from openpilot.common.realtime import DT_CTRL
class LatControl(ABC):
def __init__(self, CP, CI):
def __init__(self, CP, CP_SP, CI):
self.sat_count_rate = 1.0 * DT_CTRL
self.sat_limit = CP.steerLimitTimer
self.sat_count = 0.

View File

@@ -7,8 +7,8 @@ STEER_ANGLE_SATURATION_THRESHOLD = 2.5 # Degrees
class LatControlAngle(LatControl):
def __init__(self, CP, CI):
super().__init__(CP, CI)
def __init__(self, CP, CP_SP, CI):
super().__init__(CP, CP_SP, CI)
self.sat_check_min_speed = 5.
def update(self, active, CS, VM, params, steer_limited_by_controls, desired_curvature, calibrated_pose, curvature_limited):

View File

@@ -6,8 +6,8 @@ from openpilot.common.pid import PIDController
class LatControlPID(LatControl):
def __init__(self, CP, CI):
super().__init__(CP, CI)
def __init__(self, CP, CP_SP, CI):
super().__init__(CP, CP_SP, CI)
self.pid = PIDController((CP.lateralTuning.pid.kpBP, CP.lateralTuning.pid.kpV),
(CP.lateralTuning.pid.kiBP, CP.lateralTuning.pid.kiV),
k_f=CP.lateralTuning.pid.kf, pos_limit=self.steer_max, neg_limit=-self.steer_max)

View File

@@ -7,6 +7,8 @@ from opendbc.car.vehicle_model import ACCELERATION_DUE_TO_GRAVITY
from openpilot.selfdrive.controls.lib.latcontrol import LatControl
from openpilot.common.pid import PIDController
from openpilot.sunnypilot.selfdrive.controls.lib.latcontrol_torque_ext import LatControlTorqueExt
# At higher speeds (25+mph) we can assume:
# Lateral acceleration achieved by a specific car correlates to
# torque applied to the steering rack. It does not correlate to
@@ -23,8 +25,8 @@ LOW_SPEED_Y = [15, 13, 10, 5]
class LatControlTorque(LatControl):
def __init__(self, CP, CI):
super().__init__(CP, CI)
def __init__(self, CP, CP_SP, CI):
super().__init__(CP, CP_SP, CI)
self.torque_params = CP.lateralTuning.torque.as_builder()
self.pid = PIDController(self.torque_params.kp, self.torque_params.ki,
k_f=self.torque_params.kf, pos_limit=self.steer_max, neg_limit=-self.steer_max)
@@ -32,6 +34,8 @@ class LatControlTorque(LatControl):
self.use_steering_angle = self.torque_params.useSteeringAngle
self.steering_angle_deadzone_deg = self.torque_params.steeringAngleDeadzoneDeg
self.extension = LatControlTorqueExt(self, CP, CP_SP)
def update_live_torque_params(self, latAccelFactor, latAccelOffset, friction):
self.torque_params.latAccelFactor = latAccelFactor
self.torque_params.latAccelOffset = latAccelOffset
@@ -73,6 +77,11 @@ class LatControlTorque(LatControl):
desired_lateral_accel - actual_lateral_accel, lateral_accel_deadzone, friction_compensation=True,
gravity_adjusted=True)
# Lateral acceleration torque controller extension updates
# Overrides stock ff and pid_log.error
ff, pid_log = self.extension.update(CS, VM, params, ff, pid_log, setpoint, measurement, calibrated_pose, roll_compensation,
desired_lateral_accel, actual_lateral_accel, lateral_accel_deadzone, gravity_adjusted_lateral_accel)
freeze_integrator = steer_limited_by_controls or CS.steeringPressed or CS.vEgo < 5
output_torque = self.pid.update(pid_log.error,
feedforward=ff,

View File

@@ -15,6 +15,8 @@ from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N, get_speed_
from openpilot.selfdrive.car.cruise import V_CRUISE_MAX, V_CRUISE_UNSET
from openpilot.common.swaglog import cloudlog
from openpilot.sunnypilot.selfdrive.controls.lib.longitudinal_planner import LongitudinalPlannerSP
LON_MPC_STEP = 0.2 # first step is 0.2s
A_CRUISE_MAX_VALS = [1.6, 1.2, 0.8, 0.6]
A_CRUISE_MAX_BP = [0., 10.0, 25., 40.]
@@ -65,10 +67,11 @@ def get_accel_from_plan(speeds, accels, action_t=DT_MDL, vEgoStopping=0.05):
return a_target, should_stop
class LongitudinalPlanner:
class LongitudinalPlanner(LongitudinalPlannerSP):
def __init__(self, CP, init_v=0.0, init_a=0.0, dt=DT_MDL):
self.CP = CP
self.mpc = LongitudinalMpc(dt=dt)
LongitudinalPlannerSP.__init__(self, self.CP, self.mpc)
self.fcw = False
self.dt = dt
self.allow_throttle = True
@@ -106,7 +109,10 @@ class LongitudinalPlanner:
return x, v, a, j, throttle_prob
def update(self, sm):
LongitudinalPlannerSP.update(self, sm)
self.mpc.mode = 'blended' if sm['selfdriveState'].experimentalMode else 'acc'
if dec_mpc_mode := self.get_mpc_mode():
self.mpc.mode = dec_mpc_mode
if len(sm['carControl'].orientationNED) == 3:
accel_coast = get_coast_accel(sm['carControl'].orientationNED[1])
@@ -208,3 +214,5 @@ class LongitudinalPlanner:
longitudinalPlan.allowThrottle = bool(self.allow_throttle)
pm.send('longitudinalPlan', plan_send)
self.publish_longitudinal_plan_sp(sm, pm)

View File

@@ -6,11 +6,13 @@ from opendbc.car.honda.values import CAR as HONDA
from opendbc.car.toyota.values import CAR as TOYOTA
from opendbc.car.nissan.values import CAR as NISSAN
from opendbc.car.vehicle_model import VehicleModel
from openpilot.selfdrive.car.helpers import convert_to_capnp
from openpilot.selfdrive.controls.lib.latcontrol_pid import LatControlPID
from openpilot.selfdrive.controls.lib.latcontrol_torque import LatControlTorque
from openpilot.selfdrive.controls.lib.latcontrol_angle import LatControlAngle
from openpilot.selfdrive.locationd.helpers import Pose
from openpilot.common.mock.generators import generate_livePose
from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfaces
class TestLatControl:
@@ -19,10 +21,13 @@ class TestLatControl:
def test_saturation(self, car_name, controller):
CarInterface = interfaces[car_name]
CP = CarInterface.get_non_essential_params(car_name)
CI = CarInterface(CP)
CP_SP = CarInterface.get_non_essential_params_sp(CP, car_name)
CI = CarInterface(CP, CP_SP)
sunnypilot_interfaces.setup_interfaces(CP, CP_SP)
CP_SP = convert_to_capnp(CP_SP)
VM = VehicleModel(CP)
controller = controller(CP.as_reader(), CI)
controller = controller(CP.as_reader(), CP_SP.as_reader(), CI)
CS = car.CarState.new_message()
CS.vEgo = 30

View File

@@ -18,7 +18,7 @@ def main():
ldw = LaneDepartureWarning()
longitudinal_planner = LongitudinalPlanner(CP)
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance'])
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance', 'longitudinalPlanSP'])
sm = messaging.SubMaster(['carControl', 'carState', 'controlsState', 'liveParameters', 'radarState', 'modelV2', 'selfdriveState'],
poll='modelV2')

View File

@@ -5,13 +5,17 @@ from collections import deque
from typing import Any
import capnp
from cereal import messaging, log, car
from cereal import messaging, log, car, custom
from openpilot.common.filter_simple import FirstOrderFilter
from openpilot.common.params import Params
from openpilot.common.realtime import DT_MDL, Priority, config_realtime_process
from openpilot.common.swaglog import cloudlog
from openpilot.common.simple_kalman import KF1D
from opendbc.car import structs
from opendbc.car.hyundai.values import HyundaiFlags
from opendbc.sunnypilot.car.hyundai.values import HyundaiFlagsSP
# Default lead acceleration decay set to 50% at 1s
_LEAD_ACCEL_TAU = 1.5
@@ -157,7 +161,7 @@ def get_RadarState_from_vision(lead_msg: capnp._DynamicStructReader, v_ego: floa
def get_lead(v_ego: float, ready: bool, tracks: dict[int, Track], lead_msg: capnp._DynamicStructReader,
model_v_ego: float, low_speed_override: bool = True) -> dict[str, Any]:
model_v_ego: float, CP: structs.CarParams, CP_SP: structs.CarParamsSP, low_speed_override: bool = True) -> dict[str, Any]:
# Determine leads, this is where the essential logic happens
if len(tracks) > 0 and ready and lead_msg.prob > .5:
track = match_vision_to_track(v_ego, lead_msg, tracks)
@@ -167,6 +171,7 @@ def get_lead(v_ego: float, ready: bool, tracks: dict[int, Track], lead_msg: capn
lead_dict = {'status': False}
if track is not None:
lead_dict = track.get_RadarState(lead_msg.prob)
lead_dict = get_custom_yrel(CP, CP_SP, lead_dict, lead_msg)
elif (track is None) and ready and (lead_msg.prob > .5):
lead_dict = get_RadarState_from_vision(lead_msg, v_ego, model_v_ego)
@@ -182,8 +187,20 @@ def get_lead(v_ego: float, ready: bool, tracks: dict[int, Track], lead_msg: capn
return lead_dict
def get_custom_yrel(CP: structs.CarParams, CP_SP: structs.CarParamsSP, lead_dict: dict[str, Any],
lead_msg: capnp._DynamicStructReader) -> dict[str, Any]:
if CP.brand == "hyundai" and (CP_SP.flags & HyundaiFlagsSP.ENHANCED_SCC or
CP.flags & (HyundaiFlags.CANFD_CAMERA_SCC | HyundaiFlags.CAMERA_SCC)):
lead_dict['yRel'] = float(-lead_msg.y[0])
return lead_dict
class RadarD:
def __init__(self, delay: float = 0.0):
def __init__(self, CP: structs.CarParams, CP_SP: structs.CarParams, delay: float = 0.0):
self.CP = CP
self.CP_SP = CP_SP
self.current_time = 0.0
self.tracks: dict[int, Track] = {}
@@ -239,8 +256,8 @@ class RadarD:
model_v_ego = self.v_ego
leads_v3 = sm['modelV2'].leadsV3
if len(leads_v3) > 1:
self.radar_state.leadOne = get_lead(self.v_ego, self.ready, self.tracks, leads_v3[0], model_v_ego, low_speed_override=True)
self.radar_state.leadTwo = get_lead(self.v_ego, self.ready, self.tracks, leads_v3[1], model_v_ego, low_speed_override=False)
self.radar_state.leadOne = get_lead(self.v_ego, self.ready, self.tracks, leads_v3[0], model_v_ego, self.CP, self.CP_SP, low_speed_override=True)
self.radar_state.leadTwo = get_lead(self.v_ego, self.ready, self.tracks, leads_v3[1], model_v_ego, self.CP, self.CP_SP, low_speed_override=False)
def publish(self, pm: messaging.PubMaster):
assert self.radar_state is not None
@@ -260,11 +277,15 @@ def main() -> None:
CP = messaging.log_from_bytes(Params().get("CarParams", block=True), car.CarParams)
cloudlog.info("radard got CarParams")
cloudlog.info("radard is waiting for CarParamsSP")
CP_SP = messaging.log_from_bytes(Params().get("CarParamsSP", block=True), custom.CarParamsSP)
cloudlog.info("radard got CarParamsSP")
# *** setup messaging
sm = messaging.SubMaster(['modelV2', 'carState', 'liveTracks'], poll='modelV2')
pm = messaging.PubMaster(['radarState'])
RD = RadarD(CP.radarDelay)
RD = RadarD(CP, CP_SP, CP.radarDelay)
while 1:
sm.update()

View File

@@ -3,14 +3,25 @@ import time
from cereal import car, log, messaging
from openpilot.common.params import Params
from openpilot.system.manager.process_config import managed_processes
from openpilot.system.manager.process_config import managed_processes, is_snpe_model, is_tinygrad_model, is_stock_model
from openpilot.system.hardware import HARDWARE
if __name__ == "__main__":
CP = car.CarParams(notCar=True, wheelbase=1, steerRatio=10)
Params().put("CarParams", CP.to_bytes())
params = Params()
params.put("CarParams", CP.to_bytes())
procs = ['camerad', 'ui', 'modeld', 'calibrationd', 'plannerd', 'dmonitoringmodeld', 'dmonitoringd']
if use_snpe_modeld := is_snpe_model(False, params, CP):
print("Using SNPE modeld")
if use_tinygrad_modeld := is_tinygrad_model(False, params, CP):
print("Using TinyGrad modeld")
if use_stock_modeld := is_stock_model(False, params, CP):
print("Using stock modeld")
HARDWARE.set_power_save(False)
procs = ['camerad', 'ui', 'calibrationd', 'plannerd', 'dmonitoringmodeld', 'dmonitoringd']
procs += ["modeld_snpe" if use_snpe_modeld else "modeld_tinygrad" if use_tinygrad_modeld else "modeld"]
for p in procs:
managed_processes[p].start()

View File

@@ -90,10 +90,17 @@ def fill_model_msg(base_msg: capnp._DynamicStructBuilder, extended_msg: capnp._D
# temporal pose
temporal_pose = modelV2.temporalPose
temporal_pose.trans = net_output_data['sim_pose'][0,:ModelConstants.POSE_WIDTH//2].tolist()
temporal_pose.transStd = net_output_data['sim_pose_stds'][0,:ModelConstants.POSE_WIDTH//2].tolist()
temporal_pose.rot = net_output_data['sim_pose'][0,ModelConstants.POSE_WIDTH//2:].tolist()
temporal_pose.rotStd = net_output_data['sim_pose_stds'][0,ModelConstants.POSE_WIDTH//2:].tolist()
if 'sim_pose' in net_output_data:
temporal_pose.trans = net_output_data['sim_pose'][0,:ModelConstants.POSE_WIDTH//2].tolist()
temporal_pose.transStd = net_output_data['sim_pose_stds'][0,:ModelConstants.POSE_WIDTH//2].tolist()
temporal_pose.rot = net_output_data['sim_pose'][0,ModelConstants.POSE_WIDTH//2:].tolist()
temporal_pose.rotStd = net_output_data['sim_pose_stds'][0,ModelConstants.POSE_WIDTH//2:].tolist()
else:
temporal_pose.trans = net_output_data['plan'][0,0,Plan.VELOCITY].tolist()
temporal_pose.transStd = net_output_data['plan_stds'][0,0,Plan.VELOCITY].tolist()
temporal_pose.rot = net_output_data['plan'][0,0,Plan.ORIENTATION_RATE].tolist()
temporal_pose.rotStd = net_output_data['plan_stds'][0,0,Plan.ORIENTATION_RATE].tolist()
# poly path
fill_xyz_poly(driving_model_data.path, ModelConstants.POLY_PATH_DEGREE, *net_output_data['plan'][0,:,Plan.POSITION].T)

View File

@@ -10,7 +10,8 @@ def dmonitoringd_thread():
params = Params()
pm = messaging.PubMaster(['driverMonitoringState'])
sm = messaging.SubMaster(['driverStateV2', 'liveCalibration', 'carState', 'selfdriveState', 'modelV2'], poll='driverStateV2')
sm = messaging.SubMaster(['driverStateV2', 'liveCalibration', 'carState', 'selfdriveState', 'modelV2',
'selfdriveStateSP'], poll='driverStateV2')
DM = DriverMonitoring(rhd_saved=params.get_bool("IsRhdDetected"), always_on=params.get_bool("AlwaysOnDM"))

View File

@@ -403,13 +403,13 @@ class DriverMonitoring:
driver_state=sm['driverStateV2'],
cal_rpy=sm['liveCalibration'].rpyCalib,
car_speed=sm['carState'].vEgo,
op_engaged=sm['selfdriveState'].enabled
op_engaged=sm['selfdriveState'].enabled or sm['selfdriveStateSP'].mads.enabled
)
# Update distraction events
self._update_events(
driver_engaged=sm['carState'].steeringPressed or sm['carState'].gasPressed,
op_engaged=sm['selfdriveState'].enabled,
op_engaged=sm['selfdriveState'].enabled or sm['selfdriveStateSP'].mads.enabled,
standstill=sm['carState'].standstill,
wrong_gear=sm['carState'].gearShifter in [car.CarState.GearShifter.reverse, car.CarState.GearShifter.park],
car_speed=sm['carState'].vEgo

View File

@@ -62,8 +62,8 @@ void Panda::set_safety_model(cereal::CarParams::SafetyModel safety_model, uint16
handle->control_write(0xdc, (uint16_t)safety_model, safety_param);
}
void Panda::set_alternative_experience(uint16_t alternative_experience) {
handle->control_write(0xdf, alternative_experience, 0);
void Panda::set_alternative_experience(uint16_t alternative_experience, uint16_t safety_param_sp) {
handle->control_write(0xdf, alternative_experience, safety_param_sp);
}
std::string Panda::serial_read(int port_number) {
@@ -156,8 +156,8 @@ void Panda::enable_deepsleep() {
handle->control_write(0xfb, 0, 0);
}
void Panda::send_heartbeat(bool engaged) {
handle->control_write(0xf3, engaged, 0);
void Panda::send_heartbeat(bool engaged, bool engaged_mads) {
handle->control_write(0xf3, engaged, engaged_mads);
}
void Panda::set_can_speed_kbps(uint16_t bus, uint16_t speed) {

View File

@@ -63,7 +63,7 @@ public:
// Panda functionality
cereal::PandaState::PandaType get_hw_type();
void set_safety_model(cereal::CarParams::SafetyModel safety_model, uint16_t safety_param=0U);
void set_alternative_experience(uint16_t alternative_experience);
void set_alternative_experience(uint16_t alternative_experience, uint16_t safety_param_sp=0U);
std::string serial_read(int port_number = 0);
void set_uart_baud(int uart, int rate);
void set_fan_speed(uint16_t fan_speed);
@@ -77,7 +77,7 @@ public:
std::optional<std::string> get_serial();
void set_power_saving(bool power_saving);
void enable_deepsleep();
void send_heartbeat(bool engaged);
void send_heartbeat(bool engaged, bool engaged_mads);
void set_can_speed_kbps(uint16_t bus, uint16_t speed);
void set_can_fd_auto(uint16_t bus, bool enabled);
void set_data_speed_kbps(uint16_t bus, uint16_t speed);

View File

@@ -10,7 +10,8 @@ void PandaSafety::configureSafetyMode() {
auto car_params = fetchCarParams();
if (!car_params.empty()) {
LOGW("got %lu bytes CarParams", car_params.size());
LOGW("got %lu bytes CarParams", car_params[0].size());
LOGW("got %lu bytes CarParamsSP", car_params[1].size());
setSafetyMode(car_params);
safety_configured_ = true;
}
@@ -43,7 +44,8 @@ void PandaSafety::updateMultiplexingMode() {
}
}
std::string PandaSafety::fetchCarParams() {
// TODO-SP: Use structs instead of vector
std::vector<std::string> PandaSafety::fetchCarParams() {
if (!params_.getBool("FirmwareQueryDone")) {
return {};
}
@@ -56,16 +58,23 @@ std::string PandaSafety::fetchCarParams() {
if (!params_.getBool("ControlsReady")) {
return {};
}
return params_.get("CarParams");
return {params_.get("CarParams"), params_.get("CarParamsSP")};
}
void PandaSafety::setSafetyMode(const std::string &params_string) {
// TODO-SP: Use structs instead of vector
void PandaSafety::setSafetyMode(const std::vector<std::string> &params_string) {
AlignedBuffer aligned_buf;
capnp::FlatArrayMessageReader cmsg(aligned_buf.align(params_string.data(), params_string.size()));
AlignedBuffer aligned_buf_sp;
capnp::FlatArrayMessageReader cmsg(aligned_buf.align(params_string[0].data(), params_string[0].size()));
cereal::CarParams::Reader car_params = cmsg.getRoot<cereal::CarParams>();
capnp::FlatArrayMessageReader cmsg_sp(aligned_buf_sp.align(params_string[1].data(), params_string[1].size()));
cereal::CarParamsSP::Reader car_params_sp = cmsg_sp.getRoot<cereal::CarParamsSP>();
auto safety_configs = car_params.getSafetyConfigs();
uint16_t alternative_experience = car_params.getAlternativeExperience();
uint16_t safety_param_sp = car_params_sp.getSafetyParam();
for (int i = 0; i < pandas_.size(); ++i) {
// Default to SILENT safety model if not specified
@@ -76,8 +85,13 @@ void PandaSafety::setSafetyMode(const std::string &params_string) {
safety_param = safety_configs[i].getSafetyParam();
}
LOGW("Panda %d: setting safety model: %d, param: %d, alternative experience: %d", i, (int)safety_model, safety_param, alternative_experience);
pandas_[i]->set_alternative_experience(alternative_experience);
LOGW("Panda %d: setting safety model: %d, param: %d, alternative experience: %d, param_sp: %d", i, (int)safety_model, safety_param, alternative_experience, safety_param_sp);
pandas_[i]->set_alternative_experience(alternative_experience, safety_param_sp);
pandas_[i]->set_safety_model(safety_model, safety_param);
}
}
bool PandaSafety::getOffroadMode() {
auto offroad_mode = params_.getBool("OffroadMode");
return offroad_mode;
}

View File

@@ -41,6 +41,8 @@
#define CUTOFF_IL 400
#define SATURATE_IL 1000
#define ALT_EXP_DISENGAGE_LATERAL_ON_BRAKE 2048
ExitHandler do_exit;
bool check_all_connected(const std::vector<Panda *> &pandas) {
@@ -53,6 +55,18 @@ bool check_all_connected(const std::vector<Panda *> &pandas) {
return true;
}
bool process_mads_heartbeat(SubMaster *sm) {
const int &alt_exp = (*sm)["carParams"].getCarParams().getAlternativeExperience();
const bool disengage_lateral_on_brake = (alt_exp & ALT_EXP_DISENGAGE_LATERAL_ON_BRAKE) != 0;
const auto &mads = (*sm)["selfdriveStateSP"].getSelfdriveStateSP().getMads();
const bool heartbeat_type = disengage_lateral_on_brake ? mads.getActive() : mads.getEnabled();
const bool engaged = sm->allAliveAndValid({"selfdriveStateSP"}) && heartbeat_type;
return engaged;
}
Panda *connect(std::string serial="", uint32_t index=0) {
std::unique_ptr<Panda> panda;
try {
@@ -188,7 +202,7 @@ void fill_panda_can_state(cereal::PandaState::PandaCanState::Builder &cs, const
cs.setCanCoreResetCnt(can_health.can_core_reset_cnt);
}
std::optional<bool> send_panda_states(PubMaster *pm, const std::vector<Panda *> &pandas, bool spoofing_started) {
std::optional<bool> send_panda_states(PubMaster *pm, const std::vector<Panda *> &pandas, bool spoofing_started, bool always_offroad) {
bool ignition_local = false;
const uint32_t pandas_cnt = pandas.size();
@@ -236,7 +250,7 @@ std::optional<bool> send_panda_states(PubMaster *pm, const std::vector<Panda *>
health.ignition_line_pkt = 0;
}
ignition_local |= ((health.ignition_line_pkt != 0) || (health.ignition_can_pkt != 0));
ignition_local |= ((health.ignition_line_pkt != 0) || (health.ignition_can_pkt != 0)) && !always_offroad;
pandaStates.push_back(health);
}
@@ -323,14 +337,14 @@ void send_peripheral_state(Panda *panda, PubMaster *pm) {
pm->send("peripheralState", msg);
}
void process_panda_state(std::vector<Panda *> &pandas, PubMaster *pm, bool engaged, bool spoofing_started) {
void process_panda_state(std::vector<Panda *> &pandas, PubMaster *pm, bool engaged, bool engaged_mads, bool spoofing_started, bool always_offroad) {
std::vector<std::string> connected_serials;
for (Panda *p : pandas) {
connected_serials.push_back(p->hw_serial());
}
{
auto ignition_opt = send_panda_states(pm, pandas, spoofing_started);
auto ignition_opt = send_panda_states(pm, pandas, spoofing_started, always_offroad);
if (!ignition_opt) {
LOGE("Failed to get ignition_opt");
return;
@@ -360,7 +374,7 @@ void process_panda_state(std::vector<Panda *> &pandas, PubMaster *pm, bool engag
}
for (const auto &panda : pandas) {
panda->send_heartbeat(engaged);
panda->send_heartbeat(engaged, engaged_mads);
}
}
}
@@ -424,11 +438,13 @@ void pandad_run(std::vector<Panda *> &pandas) {
std::thread send_thread(can_send_thread, pandas, fake_send);
RateKeeper rk("pandad", 100);
SubMaster sm({"selfdriveState"});
SubMaster sm({"selfdriveState", "selfdriveStateSP", "carParams"});
PubMaster pm({"can", "pandaStates", "peripheralState"});
PandaSafety panda_safety(pandas);
Panda *peripheral_panda = pandas[0];
bool engaged = false;
bool engaged_mads = false;
bool always_offroad = false;
// Main loop: receive CAN data and process states
while (!do_exit && check_all_connected(pandas)) {
@@ -443,7 +459,9 @@ void pandad_run(std::vector<Panda *> &pandas) {
if (rk.frame() % 10 == 0) {
sm.update(0);
engaged = sm.allAliveAndValid({"selfdriveState"}) && sm["selfdriveState"].getSelfdriveState().getEnabled();
process_panda_state(pandas, &pm, engaged, spoofing_started);
engaged_mads = process_mads_heartbeat(&sm);
always_offroad = panda_safety.getOffroadMode();
process_panda_state(pandas, &pm, engaged, engaged_mads, spoofing_started, always_offroad);
panda_safety.configureSafetyMode();
}

View File

@@ -12,11 +12,12 @@ class PandaSafety {
public:
PandaSafety(const std::vector<Panda *> &pandas) : pandas_(pandas) {}
void configureSafetyMode();
bool getOffroadMode();
private:
void updateMultiplexingMode();
std::string fetchCarParams();
void setSafetyMode(const std::string &params_string);
std::vector<std::string> fetchCarParams();
void setSafetyMode(const std::vector<std::string> &params_string);
bool initialized_ = false;
bool log_once_ = false;

View File

@@ -6,7 +6,8 @@ from dataclasses import dataclass
from openpilot.common.basedir import BASEDIR
from openpilot.common.params import Params
from openpilot.selfdrive.selfdrived.events import Alert, EmptyAlert
from openpilot.selfdrive.selfdrived.events import Alert
from openpilot.sunnypilot.selfdrive.selfdrived.events_base import EmptyAlert
with open(os.path.join(BASEDIR, "selfdrive/selfdrived/alerts_offroad.json")) as f:

View File

@@ -4,12 +4,12 @@
"severity": 0
},
"Offroad_ConnectivityNeededPrompt": {
"text": "Immediately connect to the internet to check for updates. If you do not connect to the internet, openpilot won't engage in %1",
"text": "Immediately connect to the internet to check for updates. If you do not connect to the internet, sunnypilot won't engage in %1",
"severity": 0,
"_comment": "Set extra field to number of days"
},
"Offroad_ConnectivityNeeded": {
"text": "Connect to internet to check for updates. openpilot won't automatically start until it connects to internet to check for updates.",
"text": "Connect to internet to check for updates. sunnypilot won't automatically start until it connects to internet to check for updates.",
"severity": 1
},
"Offroad_UpdateFailed": {
@@ -38,11 +38,15 @@
"severity": 1
},
"Offroad_CarUnrecognized": {
"text": "openpilot was unable to identify your car. Your car is either unsupported or its ECUs are not recognized. Please submit a pull request to add the firmware versions to the proper vehicle. Need help? Join discord.comma.ai.",
"text": "sunnypilot was unable to identify your car. Your car is either unsupported or its ECUs are not recognized. Please submit a pull request to add the firmware versions to the proper vehicle. Need help? Join discord.comma.ai.",
"severity": 0
},
"Offroad_Recalibration": {
"text": "openpilot detected a change in the device's mounting position. Ensure the device is fully seated in the mount and the mount is firmly secured to the windshield.",
"text": "sunnypilot detected a change in the device's mounting position. Ensure the device is fully seated in the mount and the mount is firmly secured to the windshield.",
"severity": 0
},
"OffroadMode_Status": {
"text": "sunnypilot is now in Always Offroad mode. sunnypilot won't start until Always Offroad mode is disabled. Go to \"Settings\" -> \"Device\" to exit Always Offroad mode.",
"severity": 1
}
}

View File

@@ -1,9 +1,6 @@
#!/usr/bin/env python3
import bisect
import math
import os
from enum import IntEnum
from collections.abc import Callable
from cereal import log, car
import cereal.messaging as messaging
@@ -12,6 +9,11 @@ from openpilot.common.git import get_short_branch
from openpilot.common.realtime import DT_CTRL
from openpilot.selfdrive.locationd.calibrationd import MIN_SPEED_FILTER
from openpilot.sunnypilot.selfdrive.selfdrived.events_base import EventsBase, Priority, ET, Alert, \
NoEntryAlert, SoftDisableAlert, UserSoftDisableAlert, ImmediateDisableAlert, EngagementAlert, NormalPermanentAlert, \
StartupAlert, AlertCallbackType, AlertCallbackTypeSP, wrong_car_mode_alert
AlertSize = log.SelfdriveState.AlertSize
AlertStatus = log.SelfdriveState.AlertStatus
VisualAlert = car.CarControl.HUDControl.VisualAlert
@@ -19,183 +21,23 @@ AudibleAlert = car.CarControl.HUDControl.AudibleAlert
EventName = log.OnroadEvent.EventName
# Alert priorities
class Priority(IntEnum):
LOWEST = 0
LOWER = 1
LOW = 2
MID = 3
HIGH = 4
HIGHEST = 5
# Event types
class ET:
ENABLE = 'enable'
PRE_ENABLE = 'preEnable'
OVERRIDE_LATERAL = 'overrideLateral'
OVERRIDE_LONGITUDINAL = 'overrideLongitudinal'
NO_ENTRY = 'noEntry'
WARNING = 'warning'
USER_DISABLE = 'userDisable'
SOFT_DISABLE = 'softDisable'
IMMEDIATE_DISABLE = 'immediateDisable'
PERMANENT = 'permanent'
# get event name from enum
EVENT_NAME = {v: k for k, v in EventName.schema.enumerants.items()}
class Events:
class Events(EventsBase):
def __init__(self):
self.events: list[int] = []
self.static_events: list[int] = []
super().__init__()
self.event_counters = dict.fromkeys(EVENTS.keys(), 0)
@property
def names(self) -> list[int]:
return self.events
def get_events_mapping(self) -> dict[int, dict[str, Alert | AlertCallbackType | AlertCallbackTypeSP]]:
return EVENTS
def __len__(self) -> int:
return len(self.events)
def get_event_name(self, event: int):
return EVENT_NAME[event]
def add(self, event_name: int, static: bool=False) -> None:
if static:
bisect.insort(self.static_events, event_name)
bisect.insort(self.events, event_name)
def clear(self) -> None:
self.event_counters = {k: (v + 1 if k in self.events else 0) for k, v in self.event_counters.items()}
self.events = self.static_events.copy()
def contains(self, event_type: str) -> bool:
return any(event_type in EVENTS.get(e, {}) for e in self.events)
def create_alerts(self, event_types: list[str], callback_args=None):
if callback_args is None:
callback_args = []
ret = []
for e in self.events:
types = EVENTS[e].keys()
for et in event_types:
if et in types:
alert = EVENTS[e][et]
if not isinstance(alert, Alert):
alert = alert(*callback_args)
if DT_CTRL * (self.event_counters[e] + 1) >= alert.creation_delay:
alert.alert_type = f"{EVENT_NAME[e]}/{et}"
alert.event_type = et
ret.append(alert)
return ret
def add_from_msg(self, events):
for e in events:
bisect.insort(self.events, e.name.raw)
def to_msg(self):
ret = []
for event_name in self.events:
event = log.OnroadEvent.new_message()
event.name = event_name
for event_type in EVENTS.get(event_name, {}):
setattr(event, event_type, True)
ret.append(event)
return ret
class Alert:
def __init__(self,
alert_text_1: str,
alert_text_2: str,
alert_status: log.SelfdriveState.AlertStatus,
alert_size: log.SelfdriveState.AlertSize,
priority: Priority,
visual_alert: car.CarControl.HUDControl.VisualAlert,
audible_alert: car.CarControl.HUDControl.AudibleAlert,
duration: float,
creation_delay: float = 0.):
self.alert_text_1 = alert_text_1
self.alert_text_2 = alert_text_2
self.alert_status = alert_status
self.alert_size = alert_size
self.priority = priority
self.visual_alert = visual_alert
self.audible_alert = audible_alert
self.duration = int(duration / DT_CTRL)
self.creation_delay = creation_delay
self.alert_type = ""
self.event_type: str | None = None
def __str__(self) -> str:
return f"{self.alert_text_1}/{self.alert_text_2} {self.priority} {self.visual_alert} {self.audible_alert}"
def __gt__(self, alert2) -> bool:
if not isinstance(alert2, Alert):
return False
return self.priority > alert2.priority
EmptyAlert = Alert("" , "", AlertStatus.normal, AlertSize.none, Priority.LOWEST,
VisualAlert.none, AudibleAlert.none, 0)
class NoEntryAlert(Alert):
def __init__(self, alert_text_2: str,
alert_text_1: str = "openpilot Unavailable",
visual_alert: car.CarControl.HUDControl.VisualAlert=VisualAlert.none):
super().__init__(alert_text_1, alert_text_2, AlertStatus.normal,
AlertSize.mid, Priority.LOW, visual_alert,
AudibleAlert.refuse, 3.)
class SoftDisableAlert(Alert):
def __init__(self, alert_text_2: str):
super().__init__("TAKE CONTROL IMMEDIATELY", alert_text_2,
AlertStatus.userPrompt, AlertSize.full,
Priority.MID, VisualAlert.steerRequired,
AudibleAlert.warningSoft, 2.),
# less harsh version of SoftDisable, where the condition is user-triggered
class UserSoftDisableAlert(SoftDisableAlert):
def __init__(self, alert_text_2: str):
super().__init__(alert_text_2),
self.alert_text_1 = "openpilot will disengage"
class ImmediateDisableAlert(Alert):
def __init__(self, alert_text_2: str):
super().__init__("TAKE CONTROL IMMEDIATELY", alert_text_2,
AlertStatus.critical, AlertSize.full,
Priority.HIGHEST, VisualAlert.steerRequired,
AudibleAlert.warningImmediate, 4.),
class EngagementAlert(Alert):
def __init__(self, audible_alert: car.CarControl.HUDControl.AudibleAlert):
super().__init__("", "",
AlertStatus.normal, AlertSize.none,
Priority.MID, VisualAlert.none,
audible_alert, .2),
class NormalPermanentAlert(Alert):
def __init__(self, alert_text_1: str, alert_text_2: str = "", duration: float = 0.2, priority: Priority = Priority.LOWER, creation_delay: float = 0.):
super().__init__(alert_text_1, alert_text_2,
AlertStatus.normal, AlertSize.mid if len(alert_text_2) else AlertSize.small,
priority, VisualAlert.none, AudibleAlert.none, duration, creation_delay=creation_delay),
class StartupAlert(Alert):
def __init__(self, alert_text_1: str, alert_text_2: str = "Always keep hands on wheel and eyes on road", alert_status=AlertStatus.normal):
super().__init__(alert_text_1, alert_text_2,
alert_status, AlertSize.mid,
Priority.LOWER, VisualAlert.none, AudibleAlert.none, 5.),
def get_event_msg_type(self):
return log.OnroadEvent
# ********** helper functions **********
@@ -207,8 +49,6 @@ def get_display_speed(speed_ms: float, metric: bool) -> str:
# ********** alert callback functions **********
AlertCallbackType = Callable[[car.CarParams, car.CarState, messaging.SubMaster, bool, int, log.ControlsState], Alert]
def soft_disable_alert(alert_text_2: str) -> AlertCallbackType:
def func(CP: car.CarParams, CS: car.CarState, sm: messaging.SubMaster, metric: bool, soft_disable_time: int, personality) -> Alert:
@@ -330,13 +170,6 @@ def modeld_lagging_alert(CP: car.CarParams, CS: car.CarState, sm: messaging.SubM
return NormalPermanentAlert("Driving Model Lagging", f"{sm['modelV2'].frameDropPerc:.1f}% frames dropped")
def wrong_car_mode_alert(CP: car.CarParams, CS: car.CarState, sm: messaging.SubMaster, metric: bool, soft_disable_time: int, personality) -> Alert:
text = "Enable Adaptive Cruise to Engage"
if CP.brand == "honda":
text = "Enable Main Switch to Engage"
return NoEntryAlert(text)
def joystick_alert(CP: car.CarParams, CS: car.CarState, sm: messaging.SubMaster, metric: bool, soft_disable_time: int, personality) -> Alert:
gb = sm['carControl'].actuators.accel / 4.
steer = sm['carControl'].actuators.torque
@@ -360,7 +193,7 @@ def personality_changed_alert(CP: car.CarParams, CS: car.CarState, sm: messaging
EVENTS: dict[int, dict[str, Alert | AlertCallbackType]] = {
EVENTS: dict[int, dict[str, Alert | AlertCallbackType | AlertCallbackTypeSP]] = {
# ********** events with no alerts **********
EventName.stockFcw: {},
@@ -995,7 +828,7 @@ if __name__ == '__main__':
for i, alerts in EVENTS.items():
for et, alert in alerts.items():
if callable(alert):
alert = alert(CP, CS, sm, False, 1, log.LongitudinalPersonality.standard)
alert = alert(CP, CS, sm, False, 1, log.LongitudinalPersonality.standard) # type: ignore[call-arg]
alerts_by_type[et][alert.priority].append(event_names[i])
all_alerts: dict[str, list[tuple[Priority, list[str]]]] = {}

View File

@@ -5,7 +5,7 @@ import threading
import cereal.messaging as messaging
from cereal import car, log
from cereal import car, log, custom
from msgq.visionipc import VisionIpcClient, VisionStreamType
from opendbc.safety import ALTERNATIVE_EXPERIENCE
@@ -23,6 +23,11 @@ from openpilot.selfdrive.selfdrived.alertmanager import AlertManager, set_offroa
from openpilot.system.hardware import HARDWARE
from openpilot.system.version import get_build_metadata
from openpilot.sunnypilot.mads.mads import ModularAssistiveDrivingSystem
from openpilot.sunnypilot.selfdrive.car.car_specific import CarSpecificEventsSP
from openpilot.sunnypilot.selfdrive.car.cruise_helpers import CruiseHelper
from openpilot.sunnypilot.selfdrive.selfdrived.events import EventsSP
REPLAY = "REPLAY" in os.environ
SIMULATION = "SIMULATION" in os.environ
TESTING_CLOSET = "TESTING_CLOSET" in os.environ
@@ -40,8 +45,8 @@ SafetyModel = car.CarParams.SafetyModel
IGNORED_SAFETY_MODES = (SafetyModel.silent, SafetyModel.noOutput)
class SelfdriveD:
def __init__(self, CP=None):
class SelfdriveD(CruiseHelper):
def __init__(self, CP=None, CP_SP=None):
self.params = Params()
# Ensure the current branch is cached, otherwise the first cycle lags
@@ -54,11 +59,18 @@ class SelfdriveD:
else:
self.CP = CP
if CP_SP is None:
cloudlog.info("selfdrived is waiting for CarParamsSP")
self.CP_SP = messaging.log_from_bytes(self.params.get("CarParamsSP", block=True), custom.CarParamsSP)
cloudlog.info("selfdrived got CarParamsSP")
else:
self.CP_SP = CP_SP
self.car_events = CarSpecificEvents(self.CP)
self.disengage_on_accelerator = not (self.CP.alternativeExperience & ALTERNATIVE_EXPERIENCE.DISABLE_DISENGAGE_ON_GAS)
# Setup sockets
self.pm = messaging.PubMaster(['selfdriveState', 'onroadEvents'])
self.pm = messaging.PubMaster(['selfdriveState', 'onroadEvents'] + ['selfdriveStateSP', 'onroadEventsSP'])
self.gps_location_service = get_gps_location_service(self.params)
self.gps_packets = [self.gps_location_service]
@@ -136,10 +148,20 @@ class SelfdriveD:
elif self.CP.passive:
self.events.add(EventName.dashcamMode, static=True)
self.events_sp = EventsSP()
self.events_sp_prev = []
self.mads = ModularAssistiveDrivingSystem(self)
self.car_events_sp = CarSpecificEventsSP(self.CP, self.params)
CruiseHelper.__init__(self, self.CP, self)
def update_events(self, CS):
"""Compute onroadEvents from carState"""
self.events.clear()
self.events_sp.clear()
if self.sm['controlsState'].lateralControlState.which() == 'debugState':
self.events.add(EventName.joystickDebug)
@@ -176,6 +198,9 @@ class SelfdriveD:
car_events = self.car_events.update(CS, self.CS_prev, self.sm['carControl']).to_msg()
self.events.add_from_msg(car_events)
car_events_sp = self.car_events_sp.update().to_msg()
self.events_sp.add_from_msg(car_events_sp)
if self.CP.notCar:
# wait for everything to init first
if self.sm.frame > int(5. / DT_CTRL) and self.initialized:
@@ -365,12 +390,20 @@ class SelfdriveD:
if self.sm['modelV2'].frameDropPerc > 20:
self.events.add(EventName.modeldLagging)
# mute canBusMissing event if in Park, as it sometimes may trigger a false alarm with MADS in Paused state
if CS.gearShifter == car.CarState.GearShifter.park and self.mads.enabled:
self.events.remove(EventName.canBusMissing)
CruiseHelper.update(self, CS)
# decrement personality on distance button press
if self.CP.openpilotLongitudinalControl:
if any(not be.pressed and be.type == ButtonType.gapAdjustCruise for be in CS.buttonEvents):
self.personality = (self.personality - 1) % 3
self.params.put_nonblocking('LongitudinalPersonality', str(self.personality))
self.events.add(EventName.personalityChanged)
if not self.experimental_mode_switched:
self.personality = (self.personality - 1) % 3
self.params.put_nonblocking('LongitudinalPersonality', str(self.personality))
self.events.add(EventName.personalityChanged)
self.experimental_mode_switched = False
def data_sample(self):
car_state = messaging.recv_one(self.car_state_sock)
@@ -427,9 +460,15 @@ class SelfdriveD:
clear_event_types.add(ET.NO_ENTRY)
pers = LONGITUDINAL_PERSONALITY_MAP[self.personality]
alerts = self.events.create_alerts(self.state_machine.current_alert_types, [self.CP, CS, self.sm, self.is_metric,
self.state_machine.soft_disable_timer, pers])
self.AM.add_many(self.sm.frame, alerts)
callback_args = [self.CP, CS, self.sm, self.is_metric,
self.state_machine.soft_disable_timer, pers]
callback_args_sp = [self.CP, CS, self.sm, self.is_metric,
self.state_machine.soft_disable_timer, pers, self.experimental_mode, self.dynamic_experimental_control]
alerts = self.events.create_alerts(self.state_machine.current_alert_types, callback_args)
alerts_sp = self.events_sp.create_alerts(self.state_machine.current_alert_types, callback_args_sp)
self.AM.add_many(self.sm.frame, alerts + alerts_sp)
self.AM.process_alerts(self.sm.frame, clear_event_types)
def publish_selfdriveState(self, CS):
@@ -462,11 +501,33 @@ class SelfdriveD:
self.pm.send('onroadEvents', ce_send)
self.events_prev = self.events.names.copy()
# selfdriveStateSP
ss_sp_msg = messaging.new_message('selfdriveStateSP')
ss_sp_msg.valid = True
ss_sp = ss_sp_msg.selfdriveStateSP
mads = ss_sp.mads
mads.state = self.mads.state_machine.state
mads.enabled = self.mads.enabled
mads.active = self.mads.active
mads.available = self.mads.enabled_toggle
self.pm.send('selfdriveStateSP', ss_sp_msg)
# onroadEventsSP - logged every second or on change
if (self.sm.frame % int(1. / DT_CTRL) == 0) or (self.events_sp.names != self.events_sp_prev):
ce_send_sp = messaging.new_message('onroadEventsSP')
ce_send_sp.valid = True
ce_send_sp.onroadEventsSP.events = self.events_sp.to_msg()
self.pm.send('onroadEventsSP', ce_send_sp)
self.events_sp_prev = self.events_sp.names.copy()
def step(self):
CS = self.data_sample()
self.update_events(CS)
if not self.CP.passive and self.initialized:
self.enabled, self.active = self.state_machine.update(self.events)
if not self.CP.notCar:
self.mads.update(CS)
self.update_alerts(CS)
self.publish_selfdriveState(CS)
@@ -484,6 +545,10 @@ class SelfdriveD:
self.is_metric = self.params.get_bool("IsMetric")
self.experimental_mode = self.params.get_bool("ExperimentalMode") and self.CP.openpilotLongitudinalControl
self.personality = self.read_personality_param()
self.mads.read_params()
self.car_events_sp.read_params()
CruiseHelper.read_params(self)
time.sleep(0.1)
def run(self):

View File

@@ -1,8 +1,10 @@
import random
from openpilot.selfdrive.selfdrived.events import Alert, EmptyAlert, EVENTS
from openpilot.selfdrive.selfdrived.events import Alert, EVENTS
from openpilot.selfdrive.selfdrived.alertmanager import AlertManager
from openpilot.sunnypilot.selfdrive.selfdrived.events_base import EmptyAlert
class TestAlertManager:

View File

@@ -41,7 +41,7 @@ class TestAlerts:
events = log.OnroadEvent.EventName.schema.enumerants
for name, e in events.items():
if not name.endswith("DEPRECATED"):
if not name.endswith("DEPRECATED") and not name.startswith("eventReserved"):
fail_msg = f"{name} @{e} not in EVENTS"
assert e in EVENTS.keys(), fail_msg

View File

@@ -22,7 +22,7 @@ from openpilot.common.params import Params
from openpilot.common.prefix import OpenpilotPrefix
from openpilot.common.timeout import Timeout
from openpilot.common.realtime import DT_CTRL
from openpilot.selfdrive.car.card import can_comm_callbacks
from openpilot.selfdrive.car.card import can_comm_callbacks, convert_to_capnp
from openpilot.system.manager.process_config import managed_processes
from openpilot.selfdrive.test.process_replay.vision_meta import meta_from_camera_state, available_streams
from openpilot.selfdrive.test.process_replay.migration import migrate_all
@@ -343,6 +343,7 @@ def get_car_params_callback(rc, pm, msgs, fingerprint):
if fingerprint:
CarInterface = interfaces[fingerprint]
CP = CarInterface.get_non_essential_params(fingerprint)
CP_SP = CarInterface.get_non_essential_params_sp(CP, fingerprint)
else:
can = DummySocket()
sendcan = DummySocket()
@@ -363,12 +364,14 @@ def get_car_params_callback(rc, pm, msgs, fingerprint):
with car.CarParams.from_bytes(cached_params_raw) as _cached_params:
cached_params = _cached_params
CP = get_car(*can_callbacks, lambda obd: None, Params().get_bool("ExperimentalLongitudinalEnabled"), cached_params=cached_params).CP
_CI = get_car(*can_callbacks, lambda obd: None, Params().get_bool("ExperimentalLongitudinalEnabled"), cached_params=cached_params)
CP, CP_SP = _CI.CP, _CI.CP_SP
if not params.get_bool("DisengageOnAccelerator"):
CP.alternativeExperience |= ALTERNATIVE_EXPERIENCE.DISABLE_DISENGAGE_ON_GAS
params.put("CarParams", CP.to_bytes())
params.put("CarParamsSP", convert_to_capnp(CP_SP).to_bytes())
def selfdrived_rcv_callback(msg, cfg, frame):

View File

@@ -16,6 +16,8 @@ from openpilot.selfdrive.test.process_replay.process_replay import CONFIGS, PROC
from openpilot.tools.lib.filereader import FileReader
from openpilot.tools.lib.logreader import LogReader, save_log
IS_AZURE_TOKEN_DEFINED = os.getenv("AZURE_TOKEN")
source_segments = [
("BODY", "937ccb7243511b65|2022-05-24--16-03-09--1"), # COMMA.COMMA_BODY
("HYUNDAI", "02c45f73a2e5c6e9|2021-01-01--19-08-22--1"), # HYUNDAI.HYUNDAI_SONATA
@@ -69,6 +71,28 @@ REF_COMMIT_FN = os.path.join(PROC_REPLAY_DIR, "ref_commit")
EXCLUDED_PROCS = {"modeld", "dmonitoringmodeld"}
def preserve_only_specified_files_from_ref_commit(*commits_to_keep):
"""Keep only files in fakedata that contain any of the specified commit hashes."""
removed = 0
for f in os.listdir(FAKEDATA):
if not any(commit in f for commit in commits_to_keep):
os.remove(os.path.join(FAKEDATA, f))
removed += 1
if removed > 0:
print(f"Removed {removed} old files from {FAKEDATA}")
def handle_output_file(cur_log_fn, local):
"""Handle the output file based on whether we're using remote or local storage."""
assert os.path.exists(cur_log_fn), f"Cannot find log to upload: {cur_log_fn}"
if local:
os.system(f"git add '{os.path.realpath(cur_log_fn)}'")
else:
upload_file(cur_log_fn, os.path.basename(cur_log_fn))
os.remove(cur_log_fn)
def run_test_process(data):
segment, cfg, args, cur_log_fn, ref_log_path, lr_dat = data
res = None
@@ -79,10 +103,9 @@ def run_test_process(data):
save_log(cur_log_fn, log_msgs)
if args.update_refs or args.upload_only:
print(f'Uploading: {os.path.basename(cur_log_fn)}')
assert os.path.exists(cur_log_fn), f"Cannot find log to upload: {cur_log_fn}"
upload_file(cur_log_fn, os.path.basename(cur_log_fn))
os.remove(cur_log_fn)
print(f'Processing: {os.path.basename(cur_log_fn)}')
handle_output_file(cur_log_fn, args.local)
return (segment, cfg.proc_name, res)
@@ -121,6 +144,27 @@ def test_process(cfg, lr, segment, ref_log_path, new_log_path, ignore_fields=Non
return str(e), log_msgs
def finalize_git_updates(cur_commit, ref_commit_fn):
"""Finalize git updates and create commit."""
try:
# Add all new files first
os.system(f"git add {os.path.realpath(ref_commit_fn)}")
os.system(f"git add {os.path.realpath(FAKEDATA)}/*.zst")
# Clean up old files - keep only new ref files since they're becoming the reference
preserve_only_specified_files_from_ref_commit(cur_commit)
# Add the deletions to git
os.system(f"git add -u {os.path.realpath(FAKEDATA)}")
# Create the commit
commit_msg = f"test_processes: update ref logs to {cur_commit[:7]}"
os.system(f'git commit -m "{commit_msg}"')
print("Successfully committed reference log updates")
except Exception as e:
print(f"Failed to commit changes: {e}")
if __name__ == "__main__":
all_cars = {car for car, _ in segments}
all_procs = {cfg.proc_name for cfg in CONFIGS if cfg.proc_name not in EXCLUDED_PROCS}
@@ -144,6 +188,8 @@ if __name__ == "__main__":
help="Updates reference logs using current commit")
parser.add_argument("--upload-only", action="store_true",
help="Skips testing processes and uploads logs from previous test run")
parser.add_argument("--local", action="store_true",
help="Use local git/ storage instead of remote (Azure for Comma)")
parser.add_argument("-j", "--jobs", type=int, default=max(cpu_count - 2, 1),
help="Max amount of parallel jobs")
args = parser.parse_args()
@@ -170,6 +216,16 @@ if __name__ == "__main__":
if not cur_commit:
raise Exception("Couldn't get current commit")
# Could be set as default in args, but wanted to be more explicit on the flow.
if upload and not args.local and not IS_AZURE_TOKEN_DEFINED:
print("***** Warning: local/git run was used by default since AZURE_TOKEN was NOT found on the env variables! *****")
args.local = True
# Clean up old files before starting
if upload and args.local:
print("***** Cleaning up old fakedata for local/git tracked refs *****")
preserve_only_specified_files_from_ref_commit(cur_commit, ref_commit)
print(f"***** testing against commit {ref_commit} *****")
# check to make sure all car brands are tested
@@ -236,4 +292,8 @@ if __name__ == "__main__":
f.write(cur_commit)
print(f"\n\nUpdated reference logs for commit: {cur_commit}")
# Only do git operations if we're in local mode
if upload and args.local:
finalize_git_updates(cur_commit, REF_COMMIT_FN)
sys.exit(int(failed))

View File

@@ -12,26 +12,33 @@ if arch == "Darwin":
del base_libs[base_libs.index('OpenCL')]
qt_env['FRAMEWORKS'] += ['OpenCL']
sp_widgets_src = []
sp_qt_src = []
sp_qt_util = []
if not GetOption('stock_ui'):
SConscript(['sunnypilot/SConscript'])
Import('sp_widgets_src', 'sp_qt_src', 'sp_qt_util')
# FIXME: remove this once we're on 5.15 (24.04)
qt_env['CXXFLAGS'] += ["-Wno-deprecated-declarations"]
qt_util = qt_env.Library("qt_util", ["#selfdrive/ui/qt/api.cc", "#selfdrive/ui/qt/util.cc"], LIBS=base_libs)
qt_util = qt_env.Library("qt_util", ["#selfdrive/ui/qt/api.cc", "#selfdrive/ui/qt/util.cc"] + sp_qt_util, LIBS=base_libs)
widgets_src = ["qt/widgets/input.cc", "qt/widgets/wifi.cc", "qt/prime_state.cc",
"qt/widgets/ssh_keys.cc", "qt/widgets/toggle.cc", "qt/widgets/controls.cc",
"qt/widgets/offroad_alerts.cc", "qt/widgets/prime.cc", "qt/widgets/keyboard.cc",
"qt/widgets/scrollview.cc", "qt/widgets/cameraview.cc", "#third_party/qrcode/QrCode.cc",
"qt/request_repeater.cc", "qt/qt_window.cc", "qt/network/networking.cc", "qt/network/wifi_manager.cc"]
"qt/request_repeater.cc", "qt/qt_window.cc", "qt/network/networking.cc", "qt/network/wifi_manager.cc"] + sp_widgets_src
widgets = qt_env.Library("qt_widgets", widgets_src, LIBS=base_libs)
Export('widgets')
qt_libs = [widgets, qt_util] + base_libs
qt_src = ["main.cc", "ui.cc", "qt/sidebar.cc", "qt/body.cc",
"qt/window.cc", "qt/home.cc", "qt/offroad/settings.cc",
"qt/window.cc", "qt/home.cc", "qt/offroad/settings.cc", "qt/offroad/offroad_home.cc",
"qt/offroad/software_settings.cc", "qt/offroad/developer_panel.cc", "qt/offroad/onboarding.cc",
"qt/offroad/driverview.cc", "qt/offroad/experimental_mode.cc", "qt/offroad/firehose.cc",
"qt/onroad/onroad_home.cc", "qt/onroad/annotated_camera.cc", "qt/onroad/model.cc",
"qt/onroad/buttons.cc", "qt/onroad/alerts.cc", "qt/onroad/driver_monitoring.cc", "qt/onroad/hud.cc"]
"qt/onroad/buttons.cc", "qt/onroad/alerts.cc", "qt/onroad/driver_monitoring.cc", "qt/onroad/hud.cc"] + sp_qt_src
# build translation files
with open(File("translations/languages.json").abspath) as f:

View File

@@ -4,10 +4,16 @@
#include <QTranslator>
#include "system/hardware/hw.h"
#include "selfdrive/ui/qt/qt_window.h"
#include "selfdrive/ui/qt/util.h"
#include "selfdrive/ui/qt/window.h"
#ifdef SUNNYPILOT
#include "selfdrive/ui/sunnypilot/qt/window.h"
#define MainWindow MainWindowSP
#else
#include "selfdrive/ui/qt/qt_window.h"
#endif
int main(int argc, char *argv[]) {
setpriority(PRIO_PROCESS, 0, -20);

View File

@@ -79,31 +79,36 @@ bool HttpRequest::timeout() const {
return reply && reply->error() == QNetworkReply::OperationCanceledError;
}
void HttpRequest::sendRequest(const QString &requestURL, const HttpRequest::Method method) {
if (active()) {
qDebug() << "HttpRequest is active";
return;
}
QNetworkRequest HttpRequest::prepareRequest(const QString &requestURL) {
QNetworkRequest request;
QString token;
if (create_jwt) {
token = CommaApi::create_jwt();
token = GetJwtToken();
} else {
QString token_json = QString::fromStdString(util::read_file(util::getenv("HOME") + "/.comma/auth.json"));
QJsonDocument json_d = QJsonDocument::fromJson(token_json.toUtf8());
token = json_d["access_token"].toString();
}
QNetworkRequest request;
request.setUrl(QUrl(requestURL));
request.setRawHeader("User-Agent", getUserAgent().toUtf8());
request.setRawHeader("User-Agent", GetUserAgent().toUtf8());
if (!token.isEmpty()) {
request.setRawHeader(QByteArray("Authorization"), ("JWT " + token).toUtf8());
}
return request;
}
if (method == HttpRequest::Method::GET) {
void HttpRequest::sendRequest(const QString &requestURL, const Method method) {
if (active()) {
qDebug() << "HttpRequest is active";
return;
}
QNetworkRequest request = prepareRequest(requestURL);
if (method == Method::GET) {
reply = nam()->get(request);
} else if (method == HttpRequest::Method::DELETE) {
} else if (method == Method::DELETE) {
reply = nam()->deleteResource(request);
}

View File

@@ -5,6 +5,7 @@
#include <QString>
#include <QTimer>
#include "util.h"
#include "common/util.h"
namespace CommaApi {
@@ -23,10 +24,11 @@ class HttpRequest : public QObject {
Q_OBJECT
public:
enum class Method {GET, DELETE};
enum class Method {GET, DELETE, POST, PUT};
explicit HttpRequest(QObject* parent, bool create_jwt = true, int timeout = 20000);
void sendRequest(const QString &requestURL, const Method method = Method::GET);
virtual void sendRequest(const QString &requestURL, Method method);
void sendRequest(const QString &requestURL) { sendRequest(requestURL, Method::GET);}
bool active() const;
bool timeout() const;
@@ -35,13 +37,14 @@ signals:
protected:
QNetworkReply *reply = nullptr;
private:
static QNetworkAccessManager *nam();
QTimer *networkTimer = nullptr;
bool create_jwt;
virtual QNetworkRequest prepareRequest(const QString& requestURL);
[[nodiscard]] virtual QString GetJwtToken() const { return CommaApi::create_jwt(); }
[[nodiscard]] virtual QString GetUserAgent() const { return getUserAgent(); }
private slots:
protected slots:
void requestTimeout();
void requestFinished();
};

View File

@@ -5,7 +5,13 @@
#include <QPushButton>
#include "common/util.h"
#ifdef SUNNYPILOT
#include "selfdrive/ui/sunnypilot/ui.h"
#define UIState UIStateSP
#else
#include "selfdrive/ui/ui.h"
#endif
class RecordButton : public QPushButton {
Q_OBJECT

Some files were not shown because too many files have changed in this diff Show More