Compare commits

...

27 Commits

Author SHA1 Message Date
rav4kumar cb8ae85ed5 ff 2025-03-31 07:16:55 -07:00
rav4kumar cc310b0f9f oops 2025-03-31 06:31:46 -07:00
rav4kumar 2bf209d3c6 lint 2025-03-31 06:24:35 -07:00
rav4kumar 7aff82909a support custom personality 2025-03-31 06:23:24 -07:00
Discountchubbs 63ba5e864f car: abstract sunnypilot interfaces (#721)
* Method abstracting radar tracks in interfaces.py This makes the code more maintainable for future implementations to this file

* Method abstracting radar tracks in interfaces.py This makes the code more maintainable for future implementations to this file

* Add type annotation to params in initialize_car_interface_sp

* rename

* just internal

---------

Co-authored-by: DevTekVE <devtekve@gmail.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-30 02:45:15 +00:00
DevTekVE 6b3f75bbf0 CI: refactor Squash and Merge with simplified branch merging (#726)
* Refactor squash and merge script for improved simplicity

Simplified the squash_and_merge.py script by replacing redundant utility functions and consolidating logic. Enhanced usability by aligning command-line arguments and leveraging streamlined git operations to improve maintainability and reliability.

* Fix argument names in squash PR script

Renamed CLI arguments from '--base' and '--source' to '--target' and '--base' to align with expected input format. This ensures the script runs correctly with proper argument mapping.

* Fix incorrect base branch argument in squash script

    Updated the `--base` argument to use `source_branch` instead of `branch` to ensure the squash script processes the correct base branch. Also adjusted the command to include `branch` as a separate argument for clarity and correctness.

* Reset to a clean state after squash error.

Add a `git reset --hard` command to ensure the repository returns to a clean state after encountering errors during the squash and merge process. This prevents lingering changes from affecting subsequent operations.

* Improve error handling in squash_and_merge_prs.py

Capture and display both stdout and stderr in error cases to provide more informative feedback. Adjust the PR comment to include available output for better debugging.

* Refactor PR squash process to enhance error handling.

Modify subprocess handling to use `result.returncode` for error checks instead of relying on exceptions. Consolidate error output retrieval and logging for better clarity, while maintaining the workflow for resetting changes on failure.

* Fix incorrect return in PR processing loop

Replaced `return` with `continue` to ensure all PRs in the loop are processed before exiting. This prevents premature termination of the function and ensures accurate success count reporting.

* Simplify subprocess output handling in squash_and_merge.py

Replaced labeled print statements with direct output of stdout and stderr. This change ensures cleaner logs and remains consistent with the function's purpose of output handling during subprocess execution.

* Update subprocess.run calls to use capture_output parameter

Replaced `stdout` and `stderr` with the `capture_output` parameter for cleaner and more concise subprocess handling. Also removed extraneous whitespace for improved code readability.

* testing moving the squash script given that it's called iteratively and switching branch might miss it

* format

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-29 21:54:49 +00:00
DevTekVE 4268d7a19c Events: Refactor OnroadEventSP structure and add upstream cereal validation (#722)
* Refactor OnroadEventSP structure to contain list of events

A restructuring of the OnroadEventSP structure has been undertaken to accommodate a list of 'Event' substructures. The change is reflected in different files where OnroadEventSP is used. This update allows for more efficient management of multiple events by grouping them together under the revised OnroadEventSP structure.

* Rename `OnroadEventSP` to `OnroadEventsSP` across codebase.

Updated all references to `OnroadEventSP` to ensure consistency with the renamed struct `OnroadEventsSP`. This change improves code clarity and aligns naming conventions across modules.

* Add optional debug logging to schema validation script

Introduced a `DEBUG` flag and a `print_debug` function to streamline debug output management. This replaces direct `print` calls with conditional logging to control verbosity during execution.

Refactor structural validation logic in cereal test

Simplify the iteration over read_instances to streamline structural validation. Removed redundant comparisons and improved error handling to detect unreadable fields more effectively. Updated error messages for better clarity during debugging.

Update build command to include 'cereal' target in CI

Modified the scons build command in selfdrive_tests workflow to explicitly build the 'cereal' target. This ensures necessary components are included during the CI process, improving reliability and consistency.

Added workflow for cereal validation artifacts generation and validation against upstream

This commit encompasses significant changes to .github/workflows/selfdrive_tests.yaml, including the addition of two new jobs. One is responsible for 'Generating cereal validation artifacts' and the other for 'Validating cereal with Upstream'. This includes generating cereal schemas, building openpilot, and running validation schema instances against master. Furthermore, a new Python script (validate_sp_cereal_upstream.py) was also added to perform cereal schema instance generation and validation. These changes aim to enhance the testing process, ensuring schema compatibility and integration quality.

* Relocate cereal validation to a dedicated GitHub workflow

This commit introduces a distinct GitHub workflow for cereal validation named 'cereal_validation.yaml'. This workflow includes two jobs: one for generating cereal validation artifacts and another for validating cereal with the upstream project. Previously, these operations were included as separate jobs in 'selfdrive_tests.yaml'. However, the decoupling in this commit allows for a better organization of GitHub workflows within the project. Additionally, this separation allows these workflows to be individually configured and run, providing a greater degree of flexibility in managing our continuous integration activities.

* Rename workflow to "cereal validation" for clarity.

Updated the workflow name in the GitHub Actions configuration to better reflect its purpose. This change improves maintainability and ensures clearer identification of the workflow's function.

* Add LFS configuration and GitLab SSH setup to workflow

Integrate GitLab LFS handling by configuring LFS URLs and enabling SSH setup. This includes adding public GitLab keys and updating the workflow to support secure connections for LFS operations. Ensures proper handling of large files and seamless integration with GitLab.

* rename

* format

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-29 17:34:31 -04:00
DevTekVE 245605dc55 ci: Add LFS configuration and GitLab SSH to squash & merge script (#725)
Integrate GitLab LFS handling by configuring LFS URLs and enabling SSH setup. This includes adding public GitLab keys and updating the workflow to support secure connections for LFS operations. Ensures proper handling of large files and seamless integration with GitLab.
2025-03-29 12:42:54 +01:00
DevTekVE e15974345b modeld: Enforce float type for lag_adjusted_curvature (#714)
Convert safe_desired_curvature to float before returning.

This ensures the returned value is explicitly a float, avoiding an issue when serializing it on capnp as it does not recognize numpy.float

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-26 08:26:21 +01:00
Jason Wen 65853f0111 NNLC: Fix exact match consistency (#716)
Move the exact_match calculation after updating model_path and max_similarity to ensure consistency. This prevents potential discrepancies when rechecking NN paths and enhances code maintainability.
2025-03-26 01:08:22 -04:00
royjr 5ce03e6ecf ui: display fingerprint as fallback in platform selector (#709)
Update platform_selector.cc
2025-03-25 08:23:21 +01:00
Jason Wen 4055efdf4b ci: add commit SHA to build notifications (#712)
This change includes the current commit SHA in the workflow outputs of the sunnypilot-build-prebuilt.yaml file. It provides better traceability for builds, ensuring each workflow run is linked to the exact commit it was triggered from.
2025-03-25 00:25:04 -04:00
royjr da79b6d494 ui: new icon for Lateral panel (#707)
* Create icon_lateral.svg

* Update settings.cc

* Delete icon_lateral.svg

* Create icon_lateral.png

* Update settings.cc

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-25 03:05:20 +00:00
Jason Wen 45c8f5c9d5 ci: Update PR title formatting for Squash and Merge (#711)
Changed the PR reference format from (`#123`) to (PR-123) in squash commit messages
to prevent GitHub from automatically adding reference comments to PRs when
`master-dev-c3-new` is force pushed.
2025-03-24 22:56:19 -04:00
Kumar 34bbdf4d7f Controls: Automatic lane change (#653)
* init alc controller

* only for sunny

* rebase fix

* ui

* add ui preview

* Update common/params_keys.h

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/SConscript

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.h

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* Update selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.cc

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>

* review sugg

* code review

* renaming

* move around

* style

* fix types and params exceptions handling

* take out magic numbers

* more

* rename

* shorter

* make sure reset happens at the end of every DH loop

* split into multiple updates

* just 3 seconds

* use default states

* oops

* more readable

* oops

* some space and lines

* run in DH loop directly

* adjust ui preview

* nudgeless should process immediately

* check option instead

* more explicit

* even more explicit

* tests

* brake pedal release should not allow auto lane change (caught by test)

* unnecessary

* no continuous auto lane change

* Revert "unnecessary"

This reverts commit 93d135b54a.

* more tests

* less

* less less

* update again

* more cleanup

* better

* AutoLaneChangeState -> AutoLaneChangeMode

* update

* lint

* unused

* test all states

* license

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-24 22:26:48 -04:00
Jason Wen cc1b233277 ui: vertical space for ListWidget (#706) 2025-03-24 00:32:31 -04:00
Kumar 029a601674 Device: Quiet Mode (#654)
* init quiet mode

* only for sunny

* static

* toggle

* let's back this up

* review sugg

* oh okay

* review

* fix: ensure boolean conversion for QuietDrive parameter

* Refactor return statement to use boolean conversion for clarity in quiet mode logic

* Update selfdrive/ui/sunnypilot/quietmode.py

* rename

* sunny

* Revert "sunny"

This reverts commit 6ac4cf4b8d.

* sunny

* Revert "sunny"

This reverts commit c2bffddc05.

* sunny

* ui: support dynamic state updates for `PushButtonSP`

* test btn

* override mouse release events

* Revert "test btn"

This reverts commit cd9c9dde9a.

* Reapply "test btn"

This reverts commit 9b36b2e085.

* abstract param flipping

* Revert "Reapply "test btn""

This reverts commit 8104a262b0.

* use new button state for PushButtonSP

* Quiet Drive -> Quiet Mode

* driver camera btn moved

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-23 23:15:11 -04:00
Jason Wen 982674b4a7 ui: support dynamic state updates for PushButtonSP (#708)
* ui: support dynamic state updates for `PushButtonSP`

* test btn

* override mouse release events

* Revert "test btn"

This reverts commit cd9c9dde9a.

* Reapply "test btn"

This reverts commit 9b36b2e085.

* abstract param flipping

* Revert "Reapply "test btn""

This reverts commit 8104a262b0.

* update device panel handling for the new param
2025-03-23 22:54:40 -04:00
royjr 96d73fe2b3 NNLC: bump max similarity for higher accuracy (#704)
Update helpers.py

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-23 20:04:12 +00:00
Jason Wen a307a08591 ui: fix offset with OptionControlSP for macOS (#705)
ui: fix offset with `OptionControlSP`
2025-03-23 15:50:21 -04:00
Jason Wen e6177c49aa athenad: avoid infinite loop waiting for comma registration (#701)
Athenad: avoid infinite loop waiting for comma registration
2025-03-22 22:48:58 -04:00
Jason Wen ed39aaddbc ui: use allKeys to reset sunnypilot settings (#699)
* ui: exclude certain params from deletion during settings reset

* clear all and exclude directly and only reboot after confirmation

* cleaner

* handle this instead

* send it

* 1-liner

* nuke nuke

* space

* less
2025-03-22 22:41:42 -04:00
Tim Wilson bab2d776b6 NNLC: use torque substitutes as fuzzy fingerprints (#700)
* NNLC: use torque substitutes as fuzzy fingerprints

* substitutes are not exact matches

* update nnlc helpers

for extra nnlc model path checks, take the first one that passes.

* Update sunnypilot/selfdrive/controls/lib/nnlc/helpers.py

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-22 23:49:05 +00:00
Jason Wen 002a37cdc4 ui: show branch selector in all branches (#698) 2025-03-22 16:20:19 -04:00
DevTekVE 43e43849ad sunnylink: Settings backup & restore (#681)
* Add AES encryption and utility methods

This commit introduces a new AESCipher class that can be used for AES encryption with support for both AES-128 and AES-256. It also adds a set of utility functions, including methods for RSA to AES key derivation, file decryption and compression, and encryption and decompression. These changes provide fundamental cryptographic functionalities for data security within the system.

* Revised backup management system with new structures

This update introduces important revisions to the backup management system. A new struct named 'BackupManagerSP' has been integrated into the 'custom.capnp' file, replacing 'CustomReserved6'. This new struct includes several variables that help keep track of backup and restore status, progress and related messages.

Additional code modifications were made in 'base.py' and 'api.py' to include a 'json' parameter in the 'api_get' method. Lastly, 'manager.py' has been updated with functions to manage device configuration backups to and from sunnylink.

The changes promote better handling and management of data backup and restoration processes. The update is specifically designed to ensure that the backup information is well integrated into the system, with properly tracked status and progress.

* Refactor `allKeys` to filter by parameter key type

Updated the `allKeys` method to support filtering by `ParamKeyType`, allowing more specific key retrieval. Added a default value for backward compatibility and updated related bindings and keys to reflect this change.

* Improve Backup and Restore mechanisms

The commit refactors and improves several aspects of the backup and restore mechanisms in the `BackupManagerSP` class.

These improvements include removing redundant status tracking variables and replacing them with unified ones, updating the messaging system to handle all changes correctly, and including an enumeration `OperationType` to keep track of the type of operation currently being processed.

This commit also applies stricter conditions for restore operations, such that it only restores parameters that are currently marked as backupable, and skips those that are not. This is a preventive measure against potential issues when restoring parameters that are no longer relevant or could conflict with current versions.

Also, the encryption and decryption methods were updated to use AES-256 for more security. These changes have increased the robustness and reliability of the backup and restoration processes.

* copyright

* Add backup_manager process to offroad sunnypilot tasks

The backup_manager process is introduced to handle backups during offroad mode when SunnyLink is ready. This ensures proper backup management functionality integrated into the system.

* Simplify backup endpoint construction in restore method

Replaced conditional expression with a more concise and readable `or` operation for constructing the backup endpoint. This makes the code cleaner and easier to maintain while preserving functionality.

* Added support for backing up and restoring sunnypilot settings

An update to the sunnypilot functionality now provides two new features that allow users to backup and restore their sunnypilot settings. The changes include the addition of UI controls for initiating backup and restore operations, and the creation of a system-wide state management function for tracking these operations. This enhancement significantly improves the user experience by providing a safety net for user settings in case of software failures, bugs, or unintended changes.

* Refactor type hints to use PEP 604 syntax for clarity

Replaced `Optional` and `Dict` type hints with `|` and `dict` syntax for improved readability and compliance with Python 3.10+. Updated related imports and adjusted list comprehension for cleaner code.

* Update import path for hardware module in utils.py

Replaced the import path for the `Paths` module to align with the new directory structure under `openpilot`. This ensures compatibility with recent project reorganization and avoids import errors.

* Improve RSA key handling and fix backup status comparison

Added explicit RSA key type checks to handle invalid key formats. Enhanced type safety in `manager.py` by ensuring the backup status comparison returns a boolean. These changes improve robustness and error handling in backups.

* format

* more

* Improve backup and restore flow with progress tracking and fixes

Added proper progress tracking and cleanup logic during restore operations. Enhanced restore experience by resetting progress after completion and introducing confirmation dialogs for errors and completion. Updated API compatibility with a version query parameter for backups.

* Enable backup button only when restore process is complete

Previously, the backup button could be enabled during a restore operation, which might cause unintended behavior. This update introduces a check to ensure the backup button remains disabled while a restore process is active. This improves user experience and prevents potential conflicts.

* Fix restore button state handling during restore process

Ensure the restore button is disabled consistently when a restore is in progress. This prevents user interaction issues and aligns the button state with the restore operation status.

* "Refactor restore logic and improve button state handling"

Replaced `is_restoring` with `restore_request_pending` for clarity and better state management. Adjusted button behavior to immediately disable upon user action, ensuring improved UX and preventing repeat inputs. Refined restore completion flow for better consistency and reliability.

* Refine restore process logic for SunnyLink settings.

Introduced `restore_request_started` to improve handling of restore states and ensure accurate UI updates during the process. Adjusted case handling to enhance clarity and maintain proper behavior when restoring settings, especially during ongoing or completed requests.

* revert

* move around

* fix enabled states for different statuses

* add prompt to notify backup is complete

* same states as restore

* disable buttons if sunnylink is off

* can use the same texts

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-03-22 15:58:04 +00:00
Jason Wen efb44aeecd ui: Fix NNLC toggle not staying in persistent state (#697)
The `showEvent` method in `NeuralNetworkLateralControl` was removed as it duplicated functionality now handled in `LateralPanel`. The unused `refresh` method was also removed to streamline the code and improve maintainability.
2025-03-22 11:18:32 -04:00
Jason Wen c70d3db1e6 Params: support filtering by ParamKeyType for allKeys (#696) 2025-03-22 10:29:24 -04:00
60 changed files with 1814 additions and 488 deletions
+77
View File
@@ -0,0 +1,77 @@
name: cereal validation
on:
push:
branches:
- master
- master-new
pull_request:
paths:
- 'cereal/**'
workflow_dispatch:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: cereal-validation-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/master-new') && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
PYTHONWARNINGS: error
BASE_IMAGE: openpilot-base
BUILD: selfdrive/test/docker_build.sh base
RUN: docker run --shm-size 2G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e CI=1 -e PYTHONWARNINGS=error -e FILEREADER_CACHE=1 -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v $GITHUB_WORKSPACE/.ci_cache/scons_cache:/tmp/scons_cache -v $GITHUB_WORKSPACE/.ci_cache/comma_download_cache:/tmp/comma_download_cache -v $GITHUB_WORKSPACE/.ci_cache/openpilot_cache:/tmp/openpilot_cache $BASE_IMAGE /bin/bash -c
jobs:
generate_cereal_artifact:
name: Generate cereal validation artifacts
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc) cereal"
- name: Generate the log file
run: |
${{ env.RUN }} "cereal/messaging/tests/validate_sp_cereal_upstream.py -g -f schema_instances.bin" && \
ls -la
ls -la cereal/messaging/tests
- name: 'Prepare artifact'
run: |
mkdir -p "cereal/messaging/tests/cereal_validations"
cp cereal/messaging/tests/validate_sp_cereal_upstream.py "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py"
cp schema_instances.bin "cereal/messaging/tests/cereal_validations/schema_instances.bin"
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: cereal_validations
path: cereal/messaging/tests/cereal_validations
validate_cereal_with_upstream:
name: Validate cereal with Upstream
runs-on: ubuntu-24.04
needs: generate_cereal_artifact
steps:
- uses: actions/checkout@v4
with:
repository: 'commaai/openpilot'
submodules: true
ref: "refs/heads/master"
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc) cereal"
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: cereal_validations
path: cereal/messaging/tests/cereal_validations
- name: 'Run the validation'
run: |
chmod +x cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py
${{ env.RUN }} "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py -r -f cereal/messaging/tests/cereal_validations/schema_instances.bin"
@@ -38,6 +38,7 @@ jobs:
new_branch: ${{ steps.set-env.outputs.new_branch }}
version: ${{ steps.set-env.outputs.version }}
extra_version_identifier: ${{ steps.set-env.outputs.extra_version_identifier }}
commit_sha: ${{ steps.set-env.outputs.commit_sha }}
steps:
- uses: actions/checkout@v4
with:
@@ -94,6 +95,7 @@ jobs:
echo "new_branch=$NEW_BRANCH" >> $GITHUB_OUTPUT
[[ ! -z "$EXTRA_VERSION_IDENTIFIER" ]] && echo "extra_version_identifier=$EXTRA_VERSION_IDENTIFIER" >> $GITHUB_OUTPUT
[[ ! -z "$VERSION" ]] && echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "commit_sha=${{ github.sha }}" >> $GITHUB_OUTPUT
# Set up common environment
source /etc/profile;
@@ -4,6 +4,8 @@ env:
DEFAULT_SOURCE_BRANCH: "master-new"
DEFAULT_TARGET_BRANCH: "nightly"
PR_LABEL: "dev-c3"
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
on:
workflow_dispatch:
@@ -35,6 +37,15 @@ jobs:
git config --global user.name 'github-actions[bot]'
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Set up Python
uses: actions/setup-python@v5
with:
@@ -52,10 +63,10 @@ jobs:
echo "Source branch ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} does not exist!"
exit 1
fi
# Make sure we have the latest source branch
git fetch origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
# Check if target branch exists
if ! git ls-remote --heads origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} | grep -q "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"; then
echo "Target branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} does not exist, creating it from ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
@@ -99,20 +110,29 @@ jobs:
}
}
}' -F label="is:pr is:open label:${PR_LABEL} sort:created-asc")
echo "PR_LIST=${PR_LIST}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Process PRs
run: |
cp ${{ github.workspace }}/release/ci/squash_and_merge.py /tmp/squash_and_merge.py && \
chmod +x /tmp/squash_and_merge.py && \
python3 ${{ github.workspace }}/release/ci/squash_and_merge_prs.py \
--pr-data '${{ steps.get-prs.outputs.PR_LIST }}' \
--target-branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} \
--squash-script-path '${{ github.workspace }}/release/ci/squash_and_merge.py'
--squash-script-path '/tmp/squash_and_merge.py'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Update LFS Config
run: |
echo '[lfs]' > .lfsconfig
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
echo ' locksverify = false' >> .lfsconfig
- name: Push changes if there are diffs
id: push-changes # Add an id so we can reference this step
run: |
+66 -13
View File
@@ -10,6 +10,13 @@ $Cxx.namespace("cereal");
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
enum LongitudinalPersonalitySP {
aggressive @0;
standard @1;
relaxed @2;
overtake @3;
}
struct ModularAssistiveDrivingSystem {
state @0 :ModularAssistiveDrivingSystemState;
enabled @1 :Bool;
@@ -27,6 +34,7 @@ struct ModularAssistiveDrivingSystem {
struct SelfdriveStateSP @0x81c2f05a394cf4af {
mads @0 :ModularAssistiveDrivingSystem;
personality @1 :LongitudinalPersonalitySP;
}
struct ModelManagerSP @0xaedffd8f31e7b55d {
@@ -89,6 +97,8 @@ struct ModelManagerSP @0xaedffd8f31e7b55d {
struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
dec @0 :DynamicExperimentalControl;
personalityDEPRECATED @1 :LongitudinalPersonalitySP;
struct DynamicExperimentalControl {
state @0 :DynamicExperimentalControlState;
enabled @1 :Bool;
@@ -102,19 +112,23 @@ struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
}
struct OnroadEventSP @0xda96579883444c35 {
name @0 :EventName;
events @0 :List(Event);
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
struct Event {
name @0 :EventName;
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
}
enum EventName {
lkasEnable @0;
@@ -156,7 +170,46 @@ struct CarControlSP @0xa5cd762cd951a455 {
mads @0 :ModularAssistiveDrivingSystem;
}
struct CustomReserved6 @0xf98d843bfd7004a3 {
struct BackupManagerSP @0xf98d843bfd7004a3 {
backupStatus @0 :Status;
restoreStatus @1 :Status;
backupProgress @2 :Float32;
restoreProgress @3 :Float32;
lastError @4 :Text;
currentBackup @5 :BackupInfo;
backupHistory @6 :List(BackupInfo);
enum Status {
idle @0;
inProgress @1;
completed @2;
failed @3;
}
struct Version {
major @0 :UInt16;
minor @1 :UInt16;
patch @2 :UInt16;
build @3 :UInt16;
branch @4 :Text;
}
struct MetadataEntry {
key @0 :Text;
value @1 :Text;
tags @2 :List(Text);
}
struct BackupInfo {
deviceId @0 :Text;
version @1 :UInt32;
config @2 :Text;
isEncrypted @3 :Bool;
createdAt @4 :Text; # ISO timestamp
updatedAt @5 :Text; # ISO timestamp
sunnypilotVersion @6 :Version;
backupMetadata @7 :List(MetadataEntry);
}
}
struct CustomReserved7 @0xb86e6369214c01c8 {
+2 -2
View File
@@ -2579,10 +2579,10 @@ struct Event {
selfdriveStateSP @107 :Custom.SelfdriveStateSP;
modelManagerSP @108 :Custom.ModelManagerSP;
longitudinalPlanSP @109 :Custom.LongitudinalPlanSP;
onroadEventsSP @110 :List(Custom.OnroadEventSP);
onroadEventsSP @110 :Custom.OnroadEventSP;
carParamsSP @111 :Custom.CarParamsSP;
carControlSP @112 :Custom.CarControlSP;
customReserved6 @113 :Custom.CustomReserved6;
backupManagerSP @113 :Custom.BackupManagerSP;
customReserved7 @114 :Custom.CustomReserved7;
customReserved8 @115 :Custom.CustomReserved8;
customReserved9 @116 :Custom.CustomReserved9;
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
import argparse
import sys
from typing import Any, List, Tuple
DEBUG = False
def print_debug(string: str) -> None:
if DEBUG:
print(string)
def create_schema_instance(struct: Any, prop: Tuple[str, Any]) -> Any:
"""
Create a new instance of a schema type, handling different field types.
Args:
struct: The Cap'n Proto schema structure
prop: A tuple containing the field name and field metadata
Returns:
A new initialized schema instance
"""
struct_instance = struct.new_message()
field_name, field_metadata = prop
try:
field_type = field_metadata.proto.slot.type.which()
# Initialize different types of fields
if field_type in ('list', 'text', 'data'):
struct_instance.init(field_name, 1)
print_debug(f"Initialized list/text/data field: {field_name}")
elif field_type in ('struct', 'object'):
struct_instance.init(field_name)
print_debug(f"Initialized struct/object field: {field_name}")
return struct_instance
except Exception as e:
print(f"Error creating instance for {field_name}: {e}")
return None
def get_schema_fields(schema_struct: Any) -> List[Tuple[str, Any]]:
"""
Retrieve all fields from a given schema structure.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of field names and their metadata
"""
try:
# Get all fields from the schema
schema_fields = list(schema_struct.schema.fields.items())
print_debug("Discovered schema fields:")
for field_name, field_metadata in schema_fields:
print_debug(f"- {field_name}")
return schema_fields
except Exception as e:
print(f"Error retrieving schema fields: {e}")
return []
def generate_schema_instances(schema_struct: Any) -> List[Any]:
"""
Generate instances for all fields in a given schema.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of schema instances
"""
schema_fields = get_schema_fields(schema_struct)
instances = []
for field_prop in schema_fields:
try:
instance = create_schema_instance(schema_struct, field_prop)
if instance is not None:
instances.append(instance)
except Exception as e:
print(f"Skipping field due to error: {e}")
print(f"Generated {len(instances)} schema instances")
return instances
def persist_instances(instances: List[Any], filename: str) -> None:
"""
Write schema instances to a binary file.
Args:
instances: List of schema instances
filename: Output file path
"""
try:
with open(filename, 'wb') as f:
for instance in instances:
f.write(instance.to_bytes())
print(f"Successfully wrote {len(instances)} instances to {filename}")
except Exception as e:
print(f"Error persisting instances: {e}")
sys.exit(1)
def read_instances(filename: str, schema_type: Any) -> List[Any]:
"""
Read schema instances from a binary file.
Args:
filename: Input file path
schema_type: The schema type to use for reading
Returns:
A list of read schema instances
"""
try:
with open(filename, 'rb') as f:
data = f.read()
instances = list(schema_type.read_multiple_bytes(data))
print(f"Read {len(instances)} instances from {filename}")
return instances
except Exception as e:
print(f"Error reading instances: {e}")
sys.exit(1)
def compare_schemas(original_instances: List[Any], read_instances: List[Any]) -> bool:
"""
Compare original and read-back instances to detect potential breaking changes.
Args:
original_instances: List of originally generated instances
read_instances: List of instances read back from file
Returns:
Boolean indicating whether schemas appear compatible
"""
if len(original_instances) != len(read_instances):
print("❌ Schema Compatibility Warning: Instance count mismatch")
return False
compatible = True
for struct in read_instances:
try:
getattr(struct, struct.which()) # Attempting to access the field to validate readability
except Exception as e:
print(f"❌ Structural change detected: {struct.which()} is not readable.\nFull error: {e}")
compatible = False
return compatible
def main():
"""
CLI entry point for schema compatibility testing.
"""
# Setup argument parser
parser = argparse.ArgumentParser(
description='Cap\'n Proto Schema Compatibility Testing Tool',
epilog='Test schema compatibility by generating and reading back instances.'
)
# Add mutually exclusive group for generation or reading mode
mode_group = parser.add_mutually_exclusive_group(required=True)
mode_group.add_argument('-g', '--generate', action='store_true',
help='Generate schema instances')
mode_group.add_argument('-r', '--read', action='store_true',
help='Read and validate schema instances')
# Common arguments
parser.add_argument('-f', '--file',
default='schema_instances.bin',
help='Output/input binary file (default: schema_instances.bin)')
# Parse arguments
args = parser.parse_args()
# Import the schema dynamically
try:
from cereal import log
schema_type = log.Event
except ImportError:
print("Error: Unable to import schema. Ensure 'cereal' is installed.")
sys.exit(1)
# Execute based on mode
if args.generate:
print("🔧 Generating Schema Instances")
instances = generate_schema_instances(schema_type)
persist_instances(instances, args.file)
print("✅ Instance generation complete")
elif args.read:
print("🔍 Reading and Validating Schema Instances")
generated_instances = generate_schema_instances(schema_type)
read_back_instances = read_instances(args.file, schema_type)
# Compare schemas
if compare_schemas(generated_instances, read_back_instances):
print("✅ Schema Compatibility: No breaking changes detected")
sys.exit(0)
else:
print("❌ Potential Schema Breaking Changes Detected")
sys.exit(1)
if __name__ == "__main__":
main()
+1
View File
@@ -76,6 +76,7 @@ _services: dict[str, tuple] = {
# sunnypilot
"modelManagerSP": (False, 1., 1),
"backupManagerSP": (False, 1., 1),
"selfdriveStateSP": (True, 100., 10),
"longitudinalPlanSP": (True, 20., 10),
"onroadEventsSP": (True, 1., 1),
+2 -2
View File
@@ -45,7 +45,7 @@ class BaseApi:
ascii_encoded_text = normalized_text.encode('ascii', 'ignore')
return ascii_encoded_text.decode()
def api_get(self, endpoint, method='GET', timeout=None, access_token=None, **params):
def api_get(self, endpoint, method='GET', timeout=None, access_token=None, json=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
@@ -53,4 +53,4 @@ class BaseApi:
version = self.remove_non_ascii_chars(get_version())
headers['User-Agent'] = self.user_agent + version
return requests.request(method, f"{self.api_host}/{endpoint}", timeout=timeout, headers=headers, params=params)
return requests.request(method, f"{self.api_host}/{endpoint}", timeout=timeout, headers=headers, json=json, params=params)
+4 -2
View File
@@ -103,10 +103,12 @@ Params::~Params() {
assert(queue.empty());
}
std::vector<std::string> Params::allKeys() const {
std::vector<std::string> Params::allKeys(ParamKeyType type) const {
std::vector<std::string> ret;
for (auto &p : keys) {
ret.push_back(p.first);
if (type == ALL || (p.second & type)) {
ret.push_back(p.first);
}
}
return ret;
}
+1 -1
View File
@@ -28,7 +28,7 @@ public:
Params(const Params&) = delete;
Params& operator=(const Params&) = delete;
std::vector<std::string> allKeys() const;
std::vector<std::string> allKeys(ParamKeyType type = ALL) const;
bool checkKey(const std::string &key);
ParamKeyType getKeyType(const std::string &key);
inline std::string getParamPath(const std::string &key = {}) {
+7
View File
@@ -120,6 +120,8 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
// --- sunnypilot params --- //
{"ApiCache_DriveStats", PERSISTENT},
{"AutoLaneChangeBsmDelay", PERSISTENT},
{"AutoLaneChangeTimer", PERSISTENT},
{"CarParamsSP", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CarParamsSPCache", CLEAR_ON_MANAGER_START},
{"CarParamsSPPersistent", PERSISTENT},
@@ -128,6 +130,7 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"ModelRunnerTypeCache", CLEAR_ON_ONROAD_TRANSITION},
{"OffroadMode", CLEAR_ON_MANAGER_START},
{"OffroadMode_Status", CLEAR_ON_MANAGER_START},
{"QuietMode", PERSISTENT | BACKUP},
// MADS params
{"Mads", PERSISTENT | BACKUP},
@@ -153,6 +156,10 @@ inline static std::unordered_map<std::string, uint32_t> keys = {
{"SunnylinkdPid", PERSISTENT},
{"SunnylinkEnabled", PERSISTENT},
// Backup Manager params
{"BackupManager_CreateBackup", PERSISTENT},
{"BackupManager_RestoreVersion", PERSISTENT},
// sunnypilot car specific params
{"HyundaiRadarTracks", PERSISTENT},
{"HyundaiRadarTracksConfirmed", PERSISTENT},
+4 -3
View File
@@ -11,6 +11,7 @@ cdef extern from "common/params.h":
CLEAR_ON_ONROAD_TRANSITION
CLEAR_ON_OFFROAD_TRANSITION
DEVELOPMENT_ONLY
BACKUP
ALL
cdef cppclass c_Params "Params":
@@ -25,7 +26,7 @@ cdef extern from "common/params.h":
bool checkKey(string) nogil
string getParamPath(string) nogil
void clearAll(ParamKeyType)
vector[string] allKeys()
vector[string] allKeys(ParamKeyType)
def ensure_bytes(v):
@@ -119,5 +120,5 @@ cdef class Params:
cdef string key_bytes = ensure_bytes(key)
return self.p.getParamPath(key_bytes).decode("utf-8")
def all_keys(self):
return self.p.allKeys()
def all_keys(self, type=ParamKeyType.ALL):
return self.p.allKeys(type)
+37 -344
View File
@@ -1,360 +1,53 @@
#!/usr/bin/env python3
import argparse
import subprocess
import sys
import shutil
import signal
import contextlib
import tempfile
import os
import argparse
def run_command(command: str) -> tuple[int, str, str]:
"""Run a shell command and return exit code, stdout, and stderr."""
process = subprocess.Popen(
command,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True
)
stdout, stderr = process.communicate()
return process.returncode, stdout.strip(), stderr.strip()
def is_gh_available() -> bool:
"""Check if GitHub CLI is available."""
return shutil.which('gh') is not None
def get_current_branch() -> str | None:
"""Get the name of the current git branch."""
code, output, error = run_command("git rev-parse --abbrev-ref HEAD")
if code != 0:
print(f"Error getting current branch: {error}")
return None
return output
def backup_branch(branch_name: str) -> bool:
"""Create a backup of the current branch."""
backup_name = f"{branch_name}-backup-$(date +%Y%m%d_%H%M%S)"
code, _, error = run_command(f"git branch {backup_name}")
if code != 0:
print(f"Error creating backup branch: {error}")
return False
print(f"Created backup branch: {backup_name}")
return True
def get_commit_messages(source_branch: str, target_branch: str) -> list[str] | None:
"""Get all commit messages between source and target branches."""
code, output, error = run_command(f"git log {target_branch}..{source_branch} --format=%B")
if code != 0:
print(f"Error getting commit messages: {error}")
return None
return [msg.strip() for msg in output.splitlines() if msg and not msg.startswith('Merge')]
def get_pr_info(branch_name: str) -> str | None:
"""Get PR title using GitHub CLI."""
if not is_gh_available():
print("Warning: GitHub CLI not found. Install it to auto-fetch PR titles:")
print(" https://cli.github.com/")
return None
# Try to get PR info using gh cli
code, output, error = run_command(f"gh pr view --json title --jq .title {branch_name}")
if code != 0:
print(f"No open PR found for branch '{branch_name}'")
return None
return output
def create_squash_message(pr_title: str | None, commit_messages: list[str], source_branch: str) -> str:
"""Create a squash commit message from PR title and commit messages."""
parts = []
# Add PR title if provided
if pr_title:
parts.append(pr_title)
else:
parts.append(f"Squashed changes from {source_branch}")
parts.append("") # Empty line after title
# Add original commits section
if commit_messages:
parts.append("Original commits:")
parts.append("") # Empty line before list
parts.extend(f"* {msg}" for msg in commit_messages)
return '\n'.join(parts)
def prompt_for_title() -> str:
"""Prompt user for a commit title."""
return input("Enter commit title (or press Enter to use default): ").strip()
@contextlib.contextmanager
def workspace_manager(original_branch: str):
"""Context manager to handle workspace state and cleanup."""
stash_created = False
stash_restored = False
temp_branch: str | None = None
def cleanup_handler(signum=None, frame=None):
"""Clean up workspace state."""
nonlocal temp_branch, stash_created, stash_restored
try:
if signum and stash_restored:
# If we're handling Ctrl+C but stash was already restored,
# just clean up branches and exit
current = get_current_branch()
if current and current != original_branch:
run_command(f"git checkout {original_branch}")
if temp_branch:
run_command(f"git branch -D {temp_branch}")
print("\nOperation interrupted, but changes were already restored.")
sys.exit(3)
# First, switch back to original branch
current = get_current_branch()
if current and current != original_branch:
run_command(f"git checkout {original_branch}")
# Then clean up temp branch
if temp_branch:
run_command(f"git branch -D {temp_branch}")
# Finally, restore stash if needed - AFTER switching branches
if stash_created and not stash_restored:
print("Restoring your uncommitted changes...")
code, stash_list, _ = run_command("git stash list")
if code == 0 and "Automatic stash by squash script" in stash_list:
run_command("git stash pop")
stash_restored = True
stash_created = False
if signum:
print("\nOperation interrupted. Cleaned up and restored original state.")
sys.exit(4)
except Exception as e:
print(f"Error during cleanup: {e}")
if signum:
sys.exit(5)
try:
# Set up signal handlers
signal.signal(signal.SIGINT, cleanup_handler)
signal.signal(signal.SIGTERM, cleanup_handler)
# Check for changes (including untracked files)
code, output, _ = run_command("git status --porcelain")
if output:
print("Stashing uncommitted changes...")
run_command("git stash push -u -m 'Automatic stash by squash script'")
stash_created = True
yield lambda x: setattr(x, 'temp_branch', temp_branch)
except Exception as e:
print(f"\nError occurred: {str(e)}")
cleanup_handler()
raise
finally:
cleanup_handler()
def create_commit_with_message(message: str) -> bool:
"""Create a commit with the given message using a temporary file."""
try:
with tempfile.NamedTemporaryFile(mode='w', delete=False) as f:
f.write(message)
temp_path = f.name
# Use the temporary file for the commit message
code, _, error = run_command(f"git commit -F {temp_path}")
os.unlink(temp_path) # Clean up the temp file
if code != 0:
print(f"Error creating commit: {error}")
return False
return True
except Exception as e:
print(f"Error handling commit message: {e}")
if os.path.exists(temp_path):
os.unlink(temp_path)
return False
def squash_and_merge(source_branch: str, target_branch: str, manual_title: str | None, backup: bool = False, push: bool = False) -> bool:
def run_git_command(command, check=True):
"""
Squash the source branch and merge into target branch.
Runs a git command and returns the trimmed stdout output.
Exits the script if the command fails.
"""
# Get original branch right away
original_branch = get_current_branch()
if not original_branch:
return False
class State:
temp_branch: str | None = None
state = State()
with workspace_manager(original_branch) as set_temp_branch:
# Validate source branch exists
code, _, error = run_command(f"git rev-parse --verify {source_branch}")
if code != 0:
print(f"Error: Source branch {source_branch} not found")
return False
if source_branch == target_branch:
print(f"Error: Source and target branches cannot be the same ({source_branch})")
return False
# Ensure target branch exists
code, _, error = run_command(f"git rev-parse --verify {target_branch}")
if code != 0:
print(f"Error: Target branch {target_branch} not found")
return False
# Find merge base
code, merge_base, error = run_command(f"git merge-base {target_branch} {source_branch}")
if code != 0:
print(f"Error finding merge base: {error}")
return False
# Create backup unless explicitly skipped
if backup and not backup_branch(source_branch):
return False
# Get commit messages
commit_messages = get_commit_messages(source_branch, target_branch)
if commit_messages is None:
return False
# Get title (priority: manual title > PR title > prompt user)
title = manual_title
if not title:
title = get_pr_info(source_branch)
if not title:
title = prompt_for_title()
try:
# Create and switch to temporary branch
temp_branch = f"temp-squash-{source_branch}"
state.temp_branch = temp_branch
set_temp_branch(state)
print(f"\nCreating temporary branch {temp_branch}...")
code, _, error = run_command(f"git checkout -b {temp_branch} {source_branch}")
if code != 0:
print(f"Error creating temp branch: {error}")
return False
print("Preparing squash by resetting temporary branch to merge base...")
code, _, error = run_command(f"git reset --soft {merge_base}")
if code != 0:
print(f"Error resetting for squash: {error}")
return False
# Create commit with message
print("Creating squash commit...")
squash_message = create_squash_message(title, commit_messages, source_branch)
if not create_commit_with_message(squash_message):
return False
# Switch to target and try merge
print(f"\nSwitching to target branch {target_branch}...")
code, _, error = run_command(f"git checkout {target_branch}")
if code != 0:
print(f"Error checking out target branch: {error}")
return False
print(f"Attempting to merge changes from {temp_branch}...")
code, _, error = run_command(f"git rebase {temp_branch}")
if code != 0:
print(f"\nMerge failed with error: {error}")
print("\nThe squash was successful, and your changes are preserved in the temporary branch.")
print("To complete the merge manually, follow these steps:")
print(f"\n1. Your squashed changes are in branch: '{temp_branch}'")
print(f"2. The target branch is: '{target_branch}'")
print("\nTo resolve the conflicts:")
print(f" git checkout {target_branch}")
print(f" git merge {temp_branch}")
print(" # resolve conflicts in your editor")
print(" git add <resolved-files>")
print(" git commit")
print(f" git push origin {target_branch} # when ready to push")
print("\nTo clean up after successful merge:")
print(f" git branch -D {temp_branch}")
# Make sure to abort the merge
print("\nAborting current merge attempt...")
run_command("git merge --abort")
# Return to original branch, but keep temp branch
print(f"Returning to {original_branch}...")
run_command(f"git checkout {original_branch}")
return False
# Clean up temp branch on success
run_command(f"git branch -D {temp_branch}")
# Push if requested
if push:
code, _, error = run_command(f"git push origin {target_branch}")
if code != 0:
print(f"Error pushing to {target_branch}: {error}")
return False
print(f"Successfully pushed to {target_branch}")
else:
print(f"Changes squashed and merged into {target_branch} locally")
print(f"To push the changes: git push origin {target_branch}")
# Return to original branch
code, _, error = run_command(f"git checkout {original_branch}")
if code != 0:
print(f"Warning: Failed to return to original branch: {error}")
return False
return True
except Exception as e:
print(f"Error during squash process: {e}")
return False
print(f"Running: {' '.join(command)}")
result = subprocess.run(command, capture_output=True, text=True)
if check and result.returncode != 0:
print(result.stdout.strip())
print(result.stderr.strip())
sys.exit(result.returncode)
return result.stdout.strip()
def main():
parser = argparse.ArgumentParser(
description='Squash branch and merge into target branch'
)
parser.add_argument('--target', '-t', required=True,
help='Target branch to merge changes into')
parser.add_argument('--source', '-s',
help='Source branch to squash (default: current branch)')
parser.add_argument('--title', '-m',
help='Optional manual title (overrides PR title)')
parser.add_argument('--backup', action='store_true',
help='Creates a backup branch for the source branch')
parser.add_argument('--push', action='store_true',
help='Push changes to remote after squashing')
parser = argparse.ArgumentParser(description="Merge multiple branches with squash merges.")
parser.add_argument("--base", required=True, help="The base branch name from which the target branch will be created.")
parser.add_argument("--target", required=True, help="The target branch name to merge into.")
parser.add_argument("--title", required=False, help="Title for the commit")
args, unknown = parser.parse_known_args()
parser.add_argument("branches", nargs="+", help="List of branch names to merge into the target branch.")
args = parser.parse_args()
# Determine source branch early
source_branch = args.source
if not source_branch:
source_branch = get_current_branch()
if not source_branch:
sys.exit(1)
# Checkout the base branch to ensure a common starting point.
run_git_command(["git", "checkout", args.base])
if not squash_and_merge(source_branch, args.target, args.title, args.backup, args.push):
sys.exit(2)
# Check if the target branch exists. If not, create it from the base branch.
branch_list = run_git_command(["git", "branch"], check=False)
branch_names = [line.strip("* ").strip() for line in branch_list.splitlines()]
if args.target in branch_names:
run_git_command(["git", "checkout", args.target])
else:
run_git_command(["git", "checkout", "-b", args.target])
# Iterate over each branch, merging it with a squash merge.
for branch in args.branches:
print(f"Merging branch '{branch}' with a squash merge.")
# Merge the branch without creating a merge commit.
run_git_command(["git", "merge", "--squash", branch])
# Commit the squashed changes with an appropriate message.
commit_message = args.title or f"Squashed merge of branch '{branch}'"
run_git_command(["git", "commit", "-m", commit_message])
print(f"All branches have been merged with squashed commits into '{args.target}'.")
if __name__ == "__main__":
+16 -13
View File
@@ -80,7 +80,6 @@ def add_pr_comment(pr_number, comment):
print(f"Failed to parse comments data for PR #{pr_number}")
def validate_pr(pr):
"""Validate a PR and return (is_valid, skip_reason)"""
pr_number = pr.get('number', 'UNKNOWN')
@@ -143,26 +142,30 @@ def process_pr(pr_data, source_branch, target_branch, squash_script_path):
subprocess.run(['git', 'branch', branch, f'origin/{branch}'], check=True)
# Run squash script
subprocess.run([
result = subprocess.run([
squash_script_path,
'--target', target_branch,
'--source', branch,
'--title', f"{title} (#{pr_number})",
], check=True)
'--base', source_branch,
'--title', f"{title} (PR-{pr_number})",
branch,
], capture_output=True, text=True)
print(f"Successfully processed PR #{pr_number}")
success_count += 1
print(result.stdout)
if result.returncode == 0:
print(f"Successfully processed PR #{pr_number}")
success_count += 1
continue
except subprocess.CalledProcessError as e:
print(f"Error processing PR #{pr_number}:")
print(f"Command failed with exit code {e.returncode}")
error_output = getattr(e, 'stderr', 'No error output available')
print(f"Error output: {error_output}")
add_pr_comment(pr_number,
f"⚠️ Error during automated `{target_branch}` squash:\n```\n{error_output}\n```")
print(f"Command failed with exit code {result.returncode}")
output = result.stdout
print(f"Error output: {output}")
add_pr_comment(pr_number, f"⚠️ Error during automated `{target_branch}` squash:\n```\n{output}\n```")
subprocess.run(['git', 'reset', '--hard'], check=True)
continue
except Exception as e:
print(f"Unexpected error processing PR #{pr_number}: {str(e)}")
subprocess.run(['git', 'reset', '--hard'], check=True)
continue
return success_count
+2 -2
View File
@@ -108,7 +108,7 @@ class Car:
fixed_fingerprint = json.loads(self.params.get("CarPlatformBundle", encoding='utf-8') or "{}").get("platform", None)
self.CI = get_car(*self.can_callbacks, obd_callback(self.params), experimental_long_allowed, num_pandas, cached_params, fixed_fingerprint)
sunnypilot_interfaces.setup_car_interface_sp(self.CI.CP, self.CI.CP_SP, self.params)
sunnypilot_interfaces.setup_interfaces(self.CI.CP, self.CI.CP_SP, self.params)
self.RI = interfaces[self.CI.CP.carFingerprint].RadarInterface(self.CI.CP, self.CI.CP_SP)
self.CP = self.CI.CP
self.CP_SP = self.CI.CP_SP
@@ -274,7 +274,7 @@ class Car:
# Initialize CarInterface, once controls are ready
# TODO: this can make us miss at least a few cycles when doing an ECU knockout
self.CI.init(self.CP, self.CP_SP, *self.can_callbacks)
sunnypilot_interfaces.initialize_car_interface_sp(self.CP, self.CP_SP, self.params, *self.can_callbacks)
sunnypilot_interfaces.init_interfaces(self.CP, self.CP_SP, self.params, *self.can_callbacks)
# signal pandad to switch to car safety mode
self.params.put_bool_nonblocking("ControlsReady", True)
+1 -1
View File
@@ -45,7 +45,7 @@ class TestCarInterfaces:
experimental_long=args['experimental_long'], docs=False)
car_params_sp = CarInterface.get_params_sp(car_params, car_name, args['fingerprints'], args['car_fw'],
experimental_long=args['experimental_long'], docs=False)
sunnypilot_interfaces.setup_car_interface_sp(car_params, car_params_sp)
sunnypilot_interfaces.setup_interfaces(car_params, car_params_sp)
car_params = car_params.as_reader()
car_interface = CarInterface(car_params, car_params_sp)
assert car_params
+8 -1
View File
@@ -1,6 +1,7 @@
from cereal import log
from openpilot.common.conversions import Conversions as CV
from openpilot.common.realtime import DT_MDL
from openpilot.sunnypilot.selfdrive.controls.lib.auto_lane_change import AutoLaneChangeController
LaneChangeState = log.LaneChangeState
LaneChangeDirection = log.LaneChangeDirection
@@ -39,8 +40,10 @@ class DesireHelper:
self.keep_pulse_timer = 0.0
self.prev_one_blinker = False
self.desire = log.Desire.none
self.alc = AutoLaneChangeController(self)
def update(self, carstate, lateral_active, lane_change_prob):
self.alc.update_params()
v_ego = carstate.vEgo
one_blinker = carstate.leftBlinker != carstate.rightBlinker
below_lane_change_speed = v_ego < LANE_CHANGE_SPEED_MIN
@@ -67,10 +70,12 @@ class DesireHelper:
blindspot_detected = ((carstate.leftBlindspot and self.lane_change_direction == LaneChangeDirection.left) or
(carstate.rightBlindspot and self.lane_change_direction == LaneChangeDirection.right))
self.alc.update_lane_change(blindspot_detected, carstate.brakePressed)
if not one_blinker or below_lane_change_speed:
self.lane_change_state = LaneChangeState.off
self.lane_change_direction = LaneChangeDirection.none
elif torque_applied and not blindspot_detected:
elif (torque_applied or self.alc.auto_lane_change_allowed) and not blindspot_detected:
self.lane_change_state = LaneChangeState.laneChangeStarting
# LaneChangeState.laneChangeStarting
@@ -112,3 +117,5 @@ class DesireHelper:
self.keep_pulse_timer = 0.0
elif self.desire in (log.Desire.keepLeft, log.Desire.keepRight):
self.desire = log.Desire.none
self.alc.update_state()
@@ -2,7 +2,7 @@
import os
import time
import numpy as np
from cereal import log
from cereal import custom
from opendbc.car.interfaces import ACCEL_MIN, ACCEL_MAX
from openpilot.common.realtime import DT_MDL
from openpilot.common.swaglog import cloudlog
@@ -58,23 +58,23 @@ STOP_DISTANCE = 6.0
CRUISE_MIN_ACCEL = -1.2
CRUISE_MAX_ACCEL = 1.6
def get_jerk_factor(personality=log.LongitudinalPersonality.standard):
if personality==log.LongitudinalPersonality.relaxed:
def get_jerk_factor(personality=custom.LongitudinalPersonalitySP.standard):
if personality==custom.LongitudinalPersonalitySP.relaxed:
return 1.0
elif personality==log.LongitudinalPersonality.standard:
elif personality==custom.LongitudinalPersonalitySP.standard:
return 1.0
elif personality==log.LongitudinalPersonality.aggressive:
elif personality==custom.LongitudinalPersonalitySP.aggressive:
return 0.5
else:
raise NotImplementedError("Longitudinal personality not supported")
def get_T_FOLLOW(personality=log.LongitudinalPersonality.standard):
if personality==log.LongitudinalPersonality.relaxed:
def get_T_FOLLOW(personality=custom.LongitudinalPersonalitySP.standard):
if personality==custom.LongitudinalPersonalitySP.relaxed:
return 1.75
elif personality==log.LongitudinalPersonality.standard:
elif personality==custom.LongitudinalPersonalitySP.standard:
return 1.45
elif personality==log.LongitudinalPersonality.aggressive:
elif personality==custom.LongitudinalPersonalitySP.aggressive:
return 1.25
else:
raise NotImplementedError("Longitudinal personality not supported")
@@ -274,7 +274,7 @@ class LongitudinalMpc:
for i in range(N):
self.solver.cost_set(i, 'Zl', Zl)
def set_weights(self, prev_accel_constraint=True, personality=log.LongitudinalPersonality.standard):
def set_weights(self, prev_accel_constraint=True, personality=custom.LongitudinalPersonalitySP.standard):
jerk_factor = get_jerk_factor(personality)
if self.mode == 'acc':
a_change_cost = A_CHANGE_COST if prev_accel_constraint else 0
@@ -327,7 +327,7 @@ class LongitudinalMpc:
lead_xv = self.extrapolate_lead(x_lead, v_lead, a_lead, a_lead_tau)
return lead_xv
def update(self, radarstate, v_cruise, x, v, a, j, personality=log.LongitudinalPersonality.standard):
def update(self, radarstate, v_cruise, x, v, a, j, personality=custom.LongitudinalPersonalitySP.standard):
t_follow = get_T_FOLLOW(personality)
v_ego = self.x0[1]
self.status = radarstate.leadOne.status or radarstate.leadTwo.status
@@ -1,7 +1,6 @@
#!/usr/bin/env python3
import math
import numpy as np
import cereal.messaging as messaging
from opendbc.car.interfaces import ACCEL_MIN, ACCEL_MAX
from openpilot.common.conversions import Conversions as CV
@@ -163,9 +162,11 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
if force_slow_decel:
v_cruise = 0.0
self.mpc.set_weights(prev_accel_constraint, personality=sm['selfdriveState'].personality)
self.mpc.set_weights(prev_accel_constraint, personality=sm['selfdriveStateSP'].personality)
self.mpc.set_cur_state(self.v_desired_filter.x, self.a_desired)
self.mpc.update(sm['radarState'], v_cruise, x, v, a, j, personality=sm['selfdriveState'].personality)
self.mpc.update(sm['radarState'], v_cruise, x, v, a, j, personality=sm['selfdriveStateSP'].personality)
self.v_desired_trajectory = np.interp(CONTROL_N_T_IDX, T_IDXS_MPC, self.mpc.v_solution)
self.a_desired_trajectory = np.interp(CONTROL_N_T_IDX, T_IDXS_MPC, self.mpc.a_solution)
@@ -193,7 +194,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
def publish(self, sm, pm):
plan_send = messaging.new_message('longitudinalPlan')
plan_send.valid = sm.all_checks(service_list=['carState', 'controlsState', 'selfdriveState'])
plan_send.valid = sm.all_checks(service_list=['carState', 'controlsState', 'selfdriveState', 'selfdriveStateSP'])
longitudinalPlan = plan_send.longitudinalPlan
longitudinalPlan.modelMonoTime = sm.logMonoTime['modelV2']
@@ -23,7 +23,7 @@ class TestLatControl:
CP = CarInterface.get_non_essential_params(car_name)
CP_SP = CarInterface.get_non_essential_params_sp(CP, car_name)
CI = CarInterface(CP, CP_SP)
sunnypilot_interfaces.setup_car_interface_sp(CP, CP_SP)
sunnypilot_interfaces.setup_interfaces(CP, CP_SP)
CP_SP = convert_to_capnp(CP_SP)
VM = VehicleModel(CP)
+1 -1
View File
@@ -19,7 +19,7 @@ def main():
ldw = LaneDepartureWarning()
longitudinal_planner = LongitudinalPlanner(CP)
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance', 'longitudinalPlanSP'])
sm = messaging.SubMaster(['carControl', 'carState', 'controlsState', 'liveParameters', 'radarState', 'modelV2', 'selfdriveState'],
sm = messaging.SubMaster(['carControl', 'carState', 'controlsState', 'liveParameters', 'radarState', 'modelV2', 'selfdriveState', 'selfdriveStateSP'],
poll='modelV2')
while True:
+5 -5
View File
@@ -32,7 +32,7 @@ REPLAY = "REPLAY" in os.environ
SIMULATION = "SIMULATION" in os.environ
TESTING_CLOSET = "TESTING_CLOSET" in os.environ
IGNORE_PROCESSES = {"loggerd", "encoderd", "statsd"}
LONGITUDINAL_PERSONALITY_MAP = {v: k for k, v in log.LongitudinalPersonality.schema.enumerants.items()}
LONGITUDINAL_PERSONALITY_MAP = {v: k for k, v in custom.LongitudinalPersonalitySP.schema.enumerants.items()}
ThermalStatus = log.DeviceState.ThermalStatus
State = log.SelfdriveState.OpenpilotState
@@ -475,7 +475,6 @@ class SelfdriveD(CruiseHelper):
ss.state = self.state_machine.state
ss.engageable = not self.events.contains(ET.NO_ENTRY)
ss.experimentalMode = self.experimental_mode
ss.personality = self.personality
ss.alertText1 = self.AM.current_alert.alert_text_1
ss.alertText2 = self.AM.current_alert.alert_text_2
@@ -504,14 +503,15 @@ class SelfdriveD(CruiseHelper):
mads.enabled = self.mads.enabled
mads.active = self.mads.active
mads.available = self.mads.enabled_toggle
ss_sp.personality = self.personality
self.pm.send('selfdriveStateSP', ss_sp_msg)
# onroadEventsSP - logged every second or on change
if (self.sm.frame % int(1. / DT_CTRL) == 0) or (self.events_sp.names != self.events_sp_prev):
ce_send_sp = messaging.new_message('onroadEventsSP', len(self.events_sp))
ce_send_sp = messaging.new_message('onroadEventsSP')
ce_send_sp.valid = True
ce_send_sp.onroadEventsSP = self.events_sp.to_msg()
ce_send_sp.onroadEventsSP.events = self.events_sp.to_msg()
self.pm.send('onroadEventsSP', ce_send_sp)
self.events_sp_prev = self.events_sp.names.copy()
@@ -532,7 +532,7 @@ class SelfdriveD(CruiseHelper):
try:
return int(self.params.get('LongitudinalPersonality'))
except (ValueError, TypeError):
return log.LongitudinalPersonality.standard
return custom.LongitudinalPersonalitySP.standard
def params_thread(self, evt):
while not evt.is_set():
@@ -25,6 +25,7 @@ class Plant:
Plant.car_state = messaging.pub_sock('carState')
Plant.plan = messaging.sub_sock('longitudinalPlan')
Plant.messaging_initialized = True
Plant.selfdrive_state_sp = messaging.pub_sock('selfdriveStateSP')
self.v_lead_prev = 0.0
@@ -63,6 +64,7 @@ class Plant:
radar = messaging.new_message('radarState')
control = messaging.new_message('controlsState')
ss = messaging.new_message('selfdriveState')
ss_sp = messaging.new_message('selfdriveStateSP')
car_state = messaging.new_message('carState')
lp = messaging.new_message('liveParameters')
car_control = messaging.new_message('carControl')
@@ -118,7 +120,7 @@ class Plant:
control.controlsState.longControlState = LongCtrlState.pid if self.enabled else LongCtrlState.off
ss.selfdriveState.experimentalMode = self.e2e
ss.selfdriveState.personality = self.personality
ss_sp.selfdriveStateSP.personality = self.personality
control.controlsState.forceDecel = self.force_decel
car_state.carState.vEgo = float(self.speed)
car_state.carState.standstill = bool(self.speed < 0.01)
@@ -131,6 +133,7 @@ class Plant:
'carControl': car_control.carControl,
'controlsState': control.controlsState,
'selfdriveState': ss.selfdriveState,
'selfdriveStateSP': ss_sp.selfdriveStateSP,
'liveParameters': lp.liveParameters,
'modelV2': model.modelV2}
self.planner.update(sm)
+1 -3
View File
@@ -70,9 +70,7 @@ SoftwarePanel::SoftwarePanel(QWidget* parent) : ListWidget(parent) {
checkForUpdates();
}
});
if (!params.getBool("IsTestedBranch")) {
addItem(targetBranchBtn);
}
addItem(targetBranchBtn);
// uninstall button
auto uninstallBtn = new ButtonControl(tr("Uninstall %1").arg(getBrand()), tr("UNINSTALL"));
+8 -2
View File
@@ -13,6 +13,8 @@ from openpilot.common.swaglog import cloudlog
from openpilot.system import micd
from openpilot.selfdrive.ui.sunnypilot.quiet_mode import QuietMode
SAMPLE_RATE = 48000
SAMPLE_BUFFER = 4096 # (approx 100ms)
MAX_VOLUME = 1.0
@@ -50,8 +52,10 @@ def check_selfdrive_timeout_alert(sm):
return False
class Soundd:
class Soundd(QuietMode):
def __init__(self):
super().__init__()
self.load_sounds()
self.current_alert = AudibleAlert.none
@@ -81,7 +85,7 @@ class Soundd:
ret = np.zeros(frames, dtype=np.float32)
if self.current_alert != AudibleAlert.none:
if self.should_play_sound(self.current_alert):
num_loops = sound_list[self.current_alert][1]
sound_data = self.loaded_sounds[self.current_alert]
written_frames = 0
@@ -144,6 +148,8 @@ class Soundd:
while True:
sm.update(0)
self.load_param()
if sm.updated['microphone'] and self.current_alert == AudibleAlert.none: # only update volume filter when not playing alert
self.spl_filter_weighted.update(sm["microphone"].soundPressureWeightedDb)
self.current_volume = self.calculate_volume(float(self.spl_filter_weighted.x))
+1
View File
@@ -35,6 +35,7 @@ qt_src = [
]
lateral_panel_qt_src = [
"sunnypilot/qt/offroad/settings/lateral/lane_change_settings.cc",
"sunnypilot/qt/offroad/settings/lateral/mads_settings.cc",
"sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.cc",
]
@@ -16,23 +16,26 @@ DevicePanelSP::DevicePanelSP(SettingsWindowSP *parent) : DevicePanel(parent) {
device_grid_layout->setHorizontalSpacing(5);
device_grid_layout->setVerticalSpacing(25);
std::vector<std::pair<QString, QString>> device_btns = {
{"dcamBtn", tr("Driver Camera Preview")},
{"retrainingBtn", tr("Training Guide")},
{"regulatoryBtn", tr("Regulatory")},
{"translateBtn", tr("Language")},
{"resetParams", tr("Reset Settings")},
std::vector<std::tuple<QString, QString, QString>> device_btns = {
{"quietModeBtn", tr("Quiet Mode"), "QuietMode"},
{"dcamBtn", tr("Driver Camera Preview"), ""},
{"retrainingBtn", tr("Training Guide"), ""},
{"regulatoryBtn", tr("Regulatory"), ""},
{"translateBtn", tr("Language"), ""},
{"resetParams", tr("Reset Settings"), ""},
};
int row = 0, col = 0;
for (int i = 0; i < device_btns.size(); i++) {
if (device_btns[i].first == "regulatoryBtn" && !Hardware::TICI()) {
for (const auto &[id, text, param] : device_btns) {
if (id == "regulatoryBtn" && !Hardware::TICI()) {
continue;
}
auto *btn = new PushButtonSP(device_btns[i].second, 720, this);
auto *btn = new PushButtonSP(text, 720, this, param);
btn->setObjectName(id);
device_grid_layout->addWidget(btn, row, col);
buttons[device_btns[i].first] = btn;
buttons[id] = btn;
col++;
if (col > 1) {
@@ -43,6 +46,8 @@ DevicePanelSP::DevicePanelSP(SettingsWindowSP *parent) : DevicePanel(parent) {
connect(buttons["dcamBtn"], &PushButtonSP::clicked, [=]() { emit showDriverView(); });
connect(buttons["quietModeBtn"], &PushButtonSP::clicked, buttons["quietModeBtn"], &PushButtonSP::updateButton);
connect(buttons["retrainingBtn"], &PushButtonSP::clicked, [=]() {
if (ConfirmationDialog::confirm(tr("Are you sure you want to review the training guide?"), tr("Review"), this)) {
emit reviewTrainingGuide();
@@ -67,14 +72,7 @@ DevicePanelSP::DevicePanelSP(SettingsWindowSP *parent) : DevicePanel(parent) {
}
});
connect(buttons["resetParams"], &PushButtonSP::clicked, [=]() {
if (ConfirmationDialog::confirm(tr("Are you sure you want to reset all sunnypilot settings to default ? This cannot be undone."), tr("Reset"), this)) {
int rm = std::system("sudo rm -rf /data/params/d/*");
if (rm == 0) {
Hardware::reboot();
}
}
});
connect(buttons["resetParams"], &PushButtonSP::clicked, this, &DevicePanelSP::resetSettings);
addItem(device_grid_layout);
@@ -141,6 +139,19 @@ void DevicePanelSP::setOffroadMode() {
updateState();
}
void DevicePanelSP::resetSettings() {
if (ConfirmationDialog::confirm(tr("Are you sure you want to reset all sunnypilot settings to default? Once the settings are reset, there is no going back."), tr("Reset"), this)) {
if (ConfirmationDialog::confirm(tr("The reset cannot be undone. You have been warned."), tr("Confirm"), this)) {
const std::vector<std::string> keys = params.allKeys();
for (const auto& key : keys) {
params.remove(key);
}
Hardware::reboot();
}
}
}
void DevicePanelSP::showEvent(QShowEvent *event) {
updateState();
}
@@ -18,6 +18,7 @@ public:
void showEvent(QShowEvent *event) override;
void setOffroadMode();
void updateState();
void resetSettings();
private:
std::map<QString, PushButtonSP*> buttons;
@@ -0,0 +1,111 @@
/**
* Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
*
* This file is part of sunnypilot and is licensed under the MIT License.
* See the LICENSE.md file in the root directory for more details.
*
* Created by kumar on March 10, 2025
*/
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.h"
#include "selfdrive/ui/sunnypilot/qt/widgets/scrollview.h"
#include <map>
#include <string>
#include <tuple>
#include <vector>
LaneChangeSettings::LaneChangeSettings(QWidget* parent) : QWidget(parent) {
QVBoxLayout* main_layout = new QVBoxLayout(this);
main_layout->setContentsMargins(50, 20, 50, 20);
main_layout->setSpacing(20);
// Back button
PanelBackButton* back = new PanelBackButton(tr("Back"));
connect(back, &QPushButton::clicked, [=]() { emit backPress(); });
main_layout->addWidget(back, 0, Qt::AlignLeft);
ListWidgetSP *list = new ListWidgetSP(this, false);
// param, title, desc, icon
std::vector<std::tuple<QString, QString, QString, QString>> toggle_defs{
{
"AutoLaneChangeBsmDelay",
tr("Auto Lane Change: Delay with Blind Spot"),
tr("Toggle to enable a delay timer for seamless lane changes when blind spot monitoring (BSM) detects a obstructing vehicle, ensuring safe maneuvering."),
"../assets/offroad/icon_blank.png",
},
};
// Controls: Auto Lane Change Timer
autoLaneChangeTimer = new AutoLaneChangeTimer();
autoLaneChangeTimer->setUpdateOtherToggles(true);
autoLaneChangeTimer->showDescription();
connect(autoLaneChangeTimer, &OptionControlSP::updateLabels, autoLaneChangeTimer, &AutoLaneChangeTimer::refresh);
connect(autoLaneChangeTimer, &AutoLaneChangeTimer::updateOtherToggles, this, &LaneChangeSettings::updateToggles);
list->addItem(autoLaneChangeTimer);
for (auto &[param, title, desc, icon] : toggle_defs) {
auto toggle = new ParamControlSP(param, title, desc, icon, this);
list->addItem(toggle);
toggles[param.toStdString()] = toggle;
}
main_layout->addWidget(new ScrollViewSP(list, this));
}
void LaneChangeSettings::showEvent(QShowEvent *event) {
updateToggles();
}
void LaneChangeSettings::updateToggles() {
if (!isVisible()) {
return;
}
auto auto_lane_change_bsm_delay_toggle = toggles["AutoLaneChangeBsmDelay"];
auto autoLaneChangeTimer_param = std::atoi(params.get("AutoLaneChangeTimer").c_str());
auto cp_bytes = params.get("CarParamsPersistent");
if (!cp_bytes.empty()) {
AlignedBuffer aligned_buf;
capnp::FlatArrayMessageReader cmsg(aligned_buf.align(cp_bytes.data(), cp_bytes.size()));
cereal::CarParams::Reader CP = cmsg.getRoot<cereal::CarParams>();
if (!CP.getEnableBsm()) {
params.remove("AutoLaneChangeBsmDelay");
}
auto_lane_change_bsm_delay_toggle->setEnabled(CP.getEnableBsm() && (autoLaneChangeTimer_param > 0));
auto_lane_change_bsm_delay_toggle->refresh();
} else {
auto_lane_change_bsm_delay_toggle->setEnabled(false);
}
}
// Auto Lane Change Timer (ALCT)
AutoLaneChangeTimer::AutoLaneChangeTimer() : OptionControlSP(
"AutoLaneChangeTimer",
tr("Auto Lane Change by Blinker"),
tr("Set a timer to delay the auto lane change operation when the blinker is used. "
"No nudge on the steering wheel is required to auto lane change if a timer is set. Default is Nudge.\n"
"Please use caution when using this feature. Only use the blinker when traffic and road conditions permit."),
"../assets/offroad/icon_blank.png",
{-1, 5}) {
refresh();
}
void AutoLaneChangeTimer::refresh() {
QString option = QString::fromStdString(params.get("AutoLaneChangeTimer"));
const QString second = tr("s");
static const QMap<QString, QString> options = {
{"-1", tr("Off")},
{"0", tr("Nudge")},
{"1", tr("Nudgeless")},
{"2", "0.5 " + second},
{"3", "1 " + second},
{"4", "2 " + second},
{"5", "3 " + second},
};
setLabel(options.value(option, tr("Nudge")));
}
@@ -0,0 +1,51 @@
/**
* Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
*
* This file is part of sunnypilot and is licensed under the MIT License.
* See the LICENSE.md file in the root directory for more details.
*/
#pragma once
#include <map>
#include <string>
#include "selfdrive/ui/sunnypilot/ui.h"
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/settings.h"
#include "selfdrive/ui/sunnypilot/qt/widgets/controls.h"
class AutoLaneChangeTimer : public OptionControlSP {
Q_OBJECT
public:
AutoLaneChangeTimer();
void refresh();
signals:
void toggleUpdated();
private:
Params params;
};
class LaneChangeSettings : public QWidget {
Q_OBJECT
public:
explicit LaneChangeSettings(QWidget* parent = nullptr);
void showEvent(QShowEvent *event) override;
signals:
void backPress();
public slots:
void updateToggles();
private:
Params params;
std::map<std::string, ParamControlSP*> toggles;
AutoLaneChangeTimer *autoLaneChangeTimer;
};
@@ -13,10 +13,6 @@ NeuralNetworkLateralControl::NeuralNetworkLateralControl() :
updateToggle();
}
void NeuralNetworkLateralControl::showEvent(QShowEvent *event) {
updateToggle();
}
void NeuralNetworkLateralControl::updateToggle() {
QString statusInitText = "<font color='yellow'>" + STATUS_CHECK_COMPATIBILITY + "</font>";
QString notLoadedText = "<font color='yellow'>" + STATUS_NOT_LOADED + "</font>";
@@ -18,7 +18,6 @@ class NeuralNetworkLateralControl : public ParamControl {
public:
NeuralNetworkLateralControl();
void showEvent(QShowEvent *event) override;
public slots:
void updateToggle();
@@ -26,8 +25,6 @@ public slots:
private:
Params params;
void refresh();
// Status messages
const QString STATUS_NOT_AVAILABLE = tr("NNLC is currently not available on this platform.");
const QString STATUS_CHECK_COMPATIBILITY = tr("Start the car to check car compatibility");
@@ -42,6 +42,29 @@ LateralPanel::LateralPanel(SettingsWindowSP *parent) : QFrame(parent) {
});
list->addItem(madsSettingsButton);
list->addItem(vertical_space());
list->addItem(horizontal_line());
list->addItem(vertical_space());
// Lane Change Settings
laneChangeSettingsButton = new PushButtonSP(tr("Customize Lane Change"));
laneChangeSettingsButton->setObjectName("lane_change_btn");
connect(laneChangeSettingsButton, &QPushButton::clicked, [=]() {
sunnypilotScroller->setLastScrollPosition();
main_layout->setCurrentWidget(laneChangeWidget);
});
laneChangeWidget = new LaneChangeSettings(this);
connect(laneChangeWidget, &LaneChangeSettings::backPress, [=]() {
sunnypilotScroller->restoreScrollPosition();
main_layout->setCurrentWidget(sunnypilotScreen);
});
list->addItem(laneChangeSettingsButton);
list->addItem(vertical_space(0));
list->addItem(horizontal_line());
// Neural Network Lateral Control
nnlcToggle = new NeuralNetworkLateralControl();
list->addItem(nnlcToggle);
@@ -65,6 +88,7 @@ LateralPanel::LateralPanel(SettingsWindowSP *parent) : QFrame(parent) {
main_layout->addWidget(sunnypilotScreen);
main_layout->addWidget(madsWidget);
main_layout->addWidget(laneChangeWidget);
setStyleSheet(R"(
#back_btn {
@@ -85,6 +109,7 @@ LateralPanel::LateralPanel(SettingsWindowSP *parent) : QFrame(parent) {
}
void LateralPanel::showEvent(QShowEvent *event) {
nnlcToggle->updateToggle();
updateToggles(offroad);
}
@@ -13,6 +13,7 @@
#include "selfdrive/ui/sunnypilot/ui.h"
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/mads_settings.h"
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/neural_network_lateral_control.h"
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/lateral/lane_change_settings.h"
#include "selfdrive/ui/qt/util.h"
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/settings.h"
#include "selfdrive/ui/sunnypilot/qt/widgets/scrollview.h"
@@ -38,5 +39,7 @@ private:
ParamControl *madsToggle;
PushButtonSP *madsSettingsButton;
MadsSettings *madsWidget = nullptr;
PushButtonSP *laneChangeSettingsButton;
LaneChangeSettings *laneChangeWidget = nullptr;
NeuralNetworkLateralControl *nnlcToggle = nullptr;
};
@@ -78,7 +78,7 @@ SettingsWindowSP::SettingsWindowSP(QWidget *parent) : SettingsWindow(parent) {
PanelInfo(" " + tr("sunnylink"), new SunnylinkPanel(this), "../assets/offroad/icon_wifi_strength_full.svg"),
PanelInfo(" " + tr("Toggles"), toggles, "../../sunnypilot/selfdrive/assets/offroad/icon_toggle.png"),
PanelInfo(" " + tr("Software"), new SoftwarePanelSP(this), "../../sunnypilot/selfdrive/assets/offroad/icon_software.png"),
PanelInfo(" " + tr("Steering"), new LateralPanel(this), "../assets/images/button_home.png"),
PanelInfo(" " + tr("Steering"), new LateralPanel(this), "../../sunnypilot/selfdrive/assets/offroad/icon_lateral.png"),
PanelInfo(" " + tr("Trips"), new TripsPanel(this), "../../sunnypilot/selfdrive/assets/offroad/icon_trips.png"),
PanelInfo(" " + tr("Vehicle"), new VehiclePanel(this), "../../sunnypilot/selfdrive/assets/offroad/icon_vehicle.png"),
PanelInfo(" " + tr("Firehose"), new FirehosePanel(this), "../../sunnypilot/selfdrive/assets/offroad/icon_firehose.svg"),
@@ -7,8 +7,10 @@
#include "selfdrive/ui/sunnypilot/qt/offroad/settings/sunnylink_panel.h"
#include "common/watchdog.h"
#include "selfdrive/ui/sunnypilot/qt/util.h"
#include "selfdrive/ui/sunnypilot/qt/widgets/controls.h"
#include <QtConcurrent>
SunnylinkPanel::SunnylinkPanel(QWidget *parent) : QFrame(parent) {
main_layout = new QStackedLayout(this);
@@ -75,7 +77,6 @@ SunnylinkPanel::SunnylinkPanel(QWidget *parent) : QFrame(parent) {
description = "<font color='SeaGreen'>"+ tr("🎉Welcome back! We're excited to see you've enabled sunnylink again! 🚀")+ "</font>";
} else {
description = "<font color='orange'>"+ tr("👋Not going to lie, it's sad to see you disabled sunnylink 😢, but we'll be here when you're ready to come back 🎉.")+ "</font>";
}
sunnylinkEnabledBtn->showDescription();
sunnylinkEnabledBtn->setDescription(description);
@@ -83,7 +84,38 @@ SunnylinkPanel::SunnylinkPanel(QWidget *parent) : QFrame(parent) {
updatePanel();
});
// Backup Settings
backupSettings = new PushButtonSP(tr("Backup Settings"), 730, this);
backupSettings->setObjectName("backup_btn");
connect(backupSettings, &QPushButton::clicked, [=]() {
backupSettings->setEnabled(false);
if (ConfirmationDialog::confirm(tr("Are you sure you want to backup sunnypilot settings?"), tr("Back Up"), this)) {
params.putBool("BackupManager_CreateBackup", true);
backup_request_pending = true;
}
});
// Restore Settings
restoreSettings = new PushButtonSP(tr("Restore Settings"), 730, this);
restoreSettings->setObjectName("restore_btn");
connect(restoreSettings, &QPushButton::clicked, [=]() {
restoreSettings->setEnabled(false);
if (ConfirmationDialog::confirm(tr("Are you sure you want to restore the last backed up sunnypilot settings?"), tr("Restore"), this)) {
params.put("BackupManager_RestoreVersion", "latest");
restore_request_pending = true;
}
});
// Settings Restore and Settings Backup in the same horizontal space
auto settings_layout = new QHBoxLayout;
settings_layout->setContentsMargins(0, 0, 0, 30);
settings_layout->addWidget(backupSettings);
settings_layout->addSpacing(10);
settings_layout->addWidget(restoreSettings);
settings_layout->setAlignment(Qt::AlignLeft);
list->addItem(settings_layout);
QObject::connect(uiState(), &UIState::offroadTransition, this, &SunnylinkPanel::updatePanel);
QObject::connect(uiStateSP(), &UIStateSP::uiUpdate, this, &SunnylinkPanel::updatePanel);
sunnylinkScroller = new ScrollViewSP(list, this);
vlayout->addWidget(sunnylinkScroller);
@@ -95,6 +127,76 @@ SunnylinkPanel::SunnylinkPanel(QWidget *parent) : QFrame(parent) {
}
}
void SunnylinkPanel::updateBackupManagerState() {
const SubMaster &sm = *(uiStateSP()->sm);
backup_manager = sm["backupManagerSP"].getBackupManagerSP();
}
void SunnylinkPanel::handleBackupProgress() {
auto backup_status = backup_manager.getBackupStatus();
auto restore_status = backup_manager.getRestoreStatus();
auto backup_progress = backup_manager.getBackupProgress();
auto restore_progress = backup_manager.getRestoreProgress();
switch (backup_status) {
case cereal::BackupManagerSP::Status::IN_PROGRESS:
backup_request_pending = false;
backup_request_started = true;
backupSettings->setEnabled(false);
backupSettings->setText(QString(tr("Backup in progress %1%").arg(backup_progress)));
break;
case cereal::BackupManagerSP::Status::FAILED:
backup_request_pending = false;
backup_request_started = false;
backupSettings->setEnabled(!is_onroad);
backupSettings->setText(tr("Backup Failed"));
break;
case cereal::BackupManagerSP::Status::COMPLETED:
backup_request_pending = false;
break;
default:
if (!backup_request_pending && backup_request_started) {
backup_request_started = false;
ConfirmationDialog::alert(tr("Settings backup completed."), this);
} else {
backupSettings->setEnabled(!is_onroad && !backup_request_pending && is_sunnylink_enabled);
}
backupSettings->setText(tr("Backup Settings"));
break;
}
switch (restore_status) {
case cereal::BackupManagerSP::Status::IN_PROGRESS:
restore_request_pending = false;
restore_request_started = true;
restoreSettings->setEnabled(false);
restoreSettings->setText(QString(tr("Restore in progress %1%").arg(restore_progress)));
break;
case cereal::BackupManagerSP::Status::FAILED:
restore_request_pending = false;
restore_request_started = false;
restoreSettings->setEnabled(!is_onroad);
restoreSettings->setText(tr("Restore Failed"));
ConfirmationDialog::alert(tr("Unable to restore the settings, try again later."), this);
break;
case cereal::BackupManagerSP::Status::COMPLETED:
restore_request_pending = false;
break;
default:
if (!restore_request_pending && restore_request_started) {
restore_request_started = false;
if (ConfirmationDialog::alert(tr("Settings restored. Confirm to restart the interface."), this)) {
qApp->exit(18);
watchdog_kick(0);
}
} else {
restoreSettings->setEnabled(!is_onroad && !restore_request_pending && is_sunnylink_enabled);
}
restoreSettings->setText(tr("Restore Settings"));
break;
}
}
void SunnylinkPanel::paramsRefresh(const QString &param_name, const QString &param_value) {
// We do it on paramsRefresh because the toggleEvent happens before the value is updated
if (param_name == "SunnylinkEnabled" && param_value == "1") {
@@ -128,7 +230,7 @@ void SunnylinkPanel::stopSunnylink() const {
void SunnylinkPanel::showEvent(QShowEvent *event) {
updatePanel();
if (is_sunnylink_enabled) {
startSunnylink();
startSunnylink();
}
}
@@ -137,6 +239,8 @@ void SunnylinkPanel::updatePanel() {
return;
}
updateBackupManagerState();
handleBackupProgress();
const auto sunnylinkDongleId = getSunnylinkDongleId().value_or(tr("N/A"));
sunnylinkEnabledBtn->setEnabled(!is_onroad);
@@ -152,13 +256,12 @@ void SunnylinkPanel::updatePanel() {
sunnylinkEnabledBtn->setValue(tr("Device ID") + " " + sunnylinkDongleId);
sponsorBtn->setEnabled(!is_onroad && is_sunnylink_enabled);
sponsorBtn->setText(is_sub ? tr("THANKS ♥")/* + " ♥️"*/ : tr("SPONSOR"));
sponsorBtn->setText(is_sub ? tr("THANKS ♥") : tr("SPONSOR"));
sponsorBtn->setValue(is_sub ? tr(role_name.toStdString().c_str()) : tr("Not Sponsor"), role_color);
pairSponsorBtn->setEnabled(!is_onroad && is_sunnylink_enabled);
pairSponsorBtn->setValue(is_paired ? tr("Paired") : tr("Not Paired"));
if (!is_sunnylink_enabled) {
sunnylinkEnabledBtn->setValue("");
sponsorBtn->setValue("");
@@ -19,7 +19,9 @@ class SunnylinkPanel : public QFrame {
public:
explicit SunnylinkPanel(QWidget *parent = nullptr);
void showEvent(QShowEvent *event) override;
void paramsRefresh(const QString&param_name, const QString&param_value);
void paramsRefresh(const QString &param_name, const QString &param_value);
void updateBackupManagerState();
void handleBackupProgress();
public slots:
void updatePanel();
@@ -31,17 +33,23 @@ private:
ScrollViewSP *sunnylinkScroller = nullptr;
SunnylinkSponsorPopup *status_popup;
SunnylinkSponsorPopup *pair_popup;
ButtonControlSP* sponsorBtn;
ButtonControlSP* pairSponsorBtn;
SunnylinkClient* sunnylink_client;
ButtonControlSP *sponsorBtn;
ButtonControlSP *pairSponsorBtn;
SunnylinkClient *sunnylink_client;
cereal::BackupManagerSP::Reader backup_manager;
ParamControl *sunnylinkEnabledBtn;
bool is_onroad = false;
bool is_backup = false;
bool is_restore = false;
bool is_sunnylink_enabled = false;
bool backup_request_pending = false;
bool backup_request_started = false;
bool restore_request_pending = false;
bool restore_request_started = false;
ParamWatcher *param_watcher;
QString sunnylinkBtnDescription;
PushButtonSP *restoreSettings;
PushButtonSP *backupSettings;
void stopSunnylink() const;
void startSunnylink() const;
};
@@ -48,8 +48,19 @@ PlatformSelector::PlatformSelector() : ButtonControl(tr("Vehicle"), "", "") {
void PlatformSelector::refresh(bool _offroad) {
QString name = getPlatformBundle("name").toString();
setValue(name);
setText(name.isEmpty() ? tr("SEARCH") : tr("REMOVE"));
if (!name.isEmpty()) {
setValue(name);
setText(tr("REMOVE"));
} else {
setText(tr("SEARCH"));
auto cp_bytes = params.get("CarParamsPersistent");
if (!cp_bytes.empty()) {
AlignedBuffer aligned_buf;
capnp::FlatArrayMessageReader cmsg(aligned_buf.align(cp_bytes.data(), cp_bytes.size()));
cereal::CarParams::Reader CP = cmsg.getRoot<cereal::CarParams>();
setValue(QString::fromStdString(CP.getCarFingerprint().cStr()));
}
}
setEnabled(true);
offroad = _offroad;
@@ -22,6 +22,13 @@ QFrame *horizontal_line(QWidget *parent) {
return line;
}
QFrame *vertical_space(int height, QWidget *parent) {
QFrame *v_space = new QFrame(parent);
v_space->setFrameShape(QFrame::StyledPanel);
v_space->setFixedHeight(height);
return v_space;
}
// AbstractControlSP
AbstractControlSP::AbstractControlSP(const QString &title, const QString &desc, const QString &icon, QWidget *parent)
+58 -13
View File
@@ -18,6 +18,7 @@
#include "selfdrive/ui/sunnypilot/qt/widgets/toggle.h"
QFrame *horizontal_line(QWidget *parent = nullptr);
QFrame *vertical_space(int height = 10, QWidget *parent = nullptr);
inline void ReplaceWidget(QWidget *old_widget, QWidget *new_widget) {
if (old_widget && old_widget->parentWidget() && old_widget->parentWidget()->layout()) {
@@ -520,7 +521,11 @@ protected:
}
// Draw the rectangle
#ifdef __APPLE__
QRect rect(0, !_title.isEmpty() ? (h - 16) : 20, w, h);
#else
QRect rect(0, !_title.isEmpty() ? (h - 24) : 20, w, h);
#endif
p.setBrush(QColor(button_enabled ? "#b24a4a4a" : "#121212")); // Background color
p.setPen(QPen(Qt::NoPen));
p.drawRoundedRect(rect, 20, 20);
@@ -550,8 +555,8 @@ class PushButtonSP : public QPushButton {
Q_OBJECT
public:
PushButtonSP(const QString &text, const int minimum_button_width = 800, QWidget *parent = nullptr) : QPushButton(text, parent) {
const QString buttonStyle = R"(
PushButtonSP(const QString &text, const int minimum_button_width = 800, QWidget *parent = nullptr, const QString &param = "") : QPushButton(text, parent) {
buttonStyle = R"(
QPushButton {
border-radius: 20px;
font-size: 50px;
@@ -560,21 +565,61 @@ public:
padding: 0 25px 0 25px;
color: #FFFFFF;
}
QPushButton:enabled {
background-color: #393939;
}
QPushButton:pressed {
background-color: #4A4A4A;
}
QPushButton:disabled {
background-color: #121212;
color: #5C5C5C;
}
)";
setStyleSheet(buttonStyle);
if (!param.isEmpty()) {
key = param.toStdString();
refresh();
} else {
updateStyle(false);
}
setFixedWidth(minimum_button_width);
}
void refresh() {
if (!key.empty()) {
bool state = params.getBool(key);
if (state != is_enabled) {
is_enabled = state;
}
updateStyle(is_enabled);
}
}
void updateButton() {
if (!key.empty()) {
params.putBool(key, !is_enabled);
refresh();
}
}
protected:
// Override mouse release event to handle style updates smoothly
void mouseReleaseEvent(QMouseEvent *event) override {
if (!key.empty()) {
bool next_state = !params.getBool(key);
updateStyle(next_state);
}
QPushButton::mouseReleaseEvent(event);
}
private:
std::string key = "";
Params params;
bool is_enabled;
QString buttonStyle;
QString btn_enabled_off_style = "QPushButton:enabled { background-color: #393939; }";
QString btn_enabled_on_style = "QPushButton:enabled { background-color: #1e79e8; }";
QString btn_pressed_style = "QPushButton:pressed { background-color: #4A4A4A; }";
QString btn_disabled_stype = "QPushButton:disabled { background-color: #121212; color: #5C5C5C; }";
void updateStyle(bool enabled) {
QString enabled_style = enabled ? btn_enabled_on_style : btn_enabled_off_style;
setStyleSheet(buttonStyle + enabled_style + btn_pressed_style + btn_disabled_stype);
}
};
class PanelBackButton : public QPushButton {
+39
View File
@@ -0,0 +1,39 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from cereal import car
from openpilot.common.params import Params
AudibleAlert = car.CarControl.HUDControl.AudibleAlert
ALERTS_ALWAYS_PLAY = {
AudibleAlert.warningSoft,
AudibleAlert.warningImmediate,
AudibleAlert.promptDistracted,
AudibleAlert.promptRepeat,
}
class QuietMode:
def __init__(self):
self.params = Params()
self.enabled: bool = self.params.get_bool("QuietMode")
self._frame = 0
def load_param(self) -> None:
self._frame += 1
if self._frame % 50 == 0: # 2.5 seconds
self.enabled = self.params.get_bool("QuietMode")
def should_play_sound(self, current_alert: int) -> bool:
"""
Check if a sound should be played based on the Quiet Mode setting
and the current alert.
"""
if not self.enabled:
return bool(current_alert != AudibleAlert.none)
return current_alert in ALERTS_ALWAYS_PLAY
+1 -1
View File
@@ -18,7 +18,7 @@ UIStateSP::UIStateSP(QObject *parent) : UIState(parent) {
"modelV2", "controlsState", "liveCalibration", "radarState", "deviceState",
"pandaStates", "carParams", "driverMonitoringState", "carState", "driverStateV2",
"wideRoadCameraState", "managerState", "selfdriveState", "longitudinalPlan",
"modelManagerSP", "selfdriveStateSP", "longitudinalPlanSP",
"modelManagerSP", "selfdriveStateSP", "longitudinalPlanSP", "backupManagerSP"
});
// update timer
+8 -1
View File
@@ -151,7 +151,7 @@ def setup_keyboard_uppercase(click, pm: PubMaster, scroll=None):
def setup_driver_camera(click, pm: PubMaster, scroll=None):
setup_settings_device(click, pm)
click(950, 620)
click(1720, 620)
DATA['deviceState'].deviceState.started = False
setup_onroad(click, pm)
DATA['deviceState'].deviceState.started = True
@@ -233,6 +233,12 @@ def setup_settings_steering_mads(click, pm: PubMaster, scroll=None):
click(970, 250)
time.sleep(UI_DELAY)
def setup_settings_steering_alc(click, pm: PubMaster, scroll=None):
setup_settings_device(click, pm)
click(278, 852)
click(970, 534)
time.sleep(UI_DELAY)
def setup_settings_trips(click, pm: PubMaster, scroll=None):
setup_settings_device(click, pm)
click(278, 962)
@@ -284,6 +290,7 @@ CASES.update({
"settings_sunnylink_sponsor_button": setup_settings_sunnylink_sponsor_button,
"settings_steering": setup_settings_steering,
"settings_steering_mads": setup_settings_steering_mads,
"settings_steering_alc": setup_settings_steering_alc,
"settings_trips": setup_settings_trips,
"settings_vehicle": setup_settings_vehicle,
})
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f993debd55727b9ad0c726b8a0d5ec76b146ee3f296b30eba8e53d61b33b24fd
size 21750
+13 -5
View File
@@ -26,7 +26,7 @@ def log_fingerprint(CP: structs.CarParams) -> None:
sentry.capture_fingerprint(CP.carFingerprint, CP.brand)
def initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params = None,
def _initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params = None,
enabled: bool = False) -> None:
if params is None:
params = Params()
@@ -47,7 +47,7 @@ def initialize_neural_network_lateral_control(CP: structs.CarParams, CP_SP: stru
CP_SP.neuralNetworkLateralControl.fuzzyFingerprint = not exact_match
def setup_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params = None):
def _initialize_radar_tracks(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params = None) -> None:
if params is None:
params = Params()
@@ -60,11 +60,14 @@ def setup_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, pa
if params.get_bool("HyundaiRadarTracks"):
CP.radarUnavailable = False
initialize_neural_network_lateral_control(CP, CP_SP, params)
def setup_interfaces(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params = None):
_initialize_neural_network_lateral_control(CP, CP_SP, params)
_initialize_radar_tracks(CP, CP_SP, params)
def initialize_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params, can_recv: CanRecvCallable,
can_send: CanSendCallable):
def _enable_radar_tracks(CP: structs.CarParams, CP_SP: structs.CarParamsSP, can_recv: CanRecvCallable,
params: Params) -> None:
if CP.brand == 'hyundai':
if CP_SP.flags & HyundaiFlagsSP.ENABLE_RADAR_TRACKS:
can_recv()
@@ -79,3 +82,8 @@ def initialize_car_interface_sp(CP: structs.CarParams, CP_SP: structs.CarParamsS
if not radar_tracks_persistent:
params.put_bool_nonblocking("HyundaiRadarTracks", not radar_unavailable)
params.put_bool_nonblocking("HyundaiRadarTracksPersistent", True)
def init_interfaces(CP: structs.CarParams, CP_SP: structs.CarParamsSP, params: Params,
can_recv: CanRecvCallable, can_send: CanSendCallable):
_enable_radar_tracks(CP, CP_SP, can_recv, params)
@@ -0,0 +1,115 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from cereal import log
from openpilot.common.params import Params
from openpilot.common.realtime import DT_MDL
class AutoLaneChangeMode:
OFF = -1
NUDGE = 0 # default
NUDGELESS = 1
HALF_SECOND = 2
ONE_SECOND = 3
TWO_SECONDS = 4
THREE_SECONDS = 5
AUTO_LANE_CHANGE_TIMER = {
AutoLaneChangeMode.OFF: 0.0, # Off
AutoLaneChangeMode.NUDGE: 0.0, # Nudge
AutoLaneChangeMode.NUDGELESS: 0.05, # Nudgeless
AutoLaneChangeMode.HALF_SECOND: 0.5, # 0.5-second delay
AutoLaneChangeMode.ONE_SECOND: 1.0, # 1-second delay
AutoLaneChangeMode.TWO_SECONDS: 2.0, # 2-second delay
AutoLaneChangeMode.THREE_SECONDS: 3.0, # 3-second delay
}
ONE_SECOND_DELAY = -1
class AutoLaneChangeController:
def __init__(self, desire_helper):
self.DH = desire_helper
self.params = Params()
self.lane_change_wait_timer = 0.0
self.param_read_counter = 0
self.lane_change_delay = 0.0
self.lane_change_set_timer = AutoLaneChangeMode.NUDGE
self.lane_change_bsm_delay = False
self.prev_brake_pressed = False
self.auto_lane_change_allowed = False
self.prev_lane_change = False
self.read_params()
def reset(self) -> None:
# Auto reset if parent state indicates we should
if self.DH.lane_change_state == log.LaneChangeState.off and \
self.DH.lane_change_direction == log.LaneChangeDirection.none:
self.lane_change_wait_timer = 0.0
self.prev_brake_pressed = False
self.prev_lane_change = False
def read_params(self) -> None:
self.lane_change_bsm_delay = self.params.get_bool("AutoLaneChangeBsmDelay")
try:
self.lane_change_set_timer = int(self.params.get("AutoLaneChangeTimer", encoding="utf8"))
except (ValueError, TypeError):
self.lane_change_set_timer = AutoLaneChangeMode.NUDGE
def update_params(self) -> None:
if self.param_read_counter % 50 == 0:
self.read_params()
self.param_read_counter += 1
def update_lane_change_timers(self, blindspot_detected: bool) -> None:
self.lane_change_delay = AUTO_LANE_CHANGE_TIMER.get(self.lane_change_set_timer,
AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.NUDGE])
self.lane_change_wait_timer += DT_MDL
if self.lane_change_bsm_delay and blindspot_detected and self.lane_change_delay > 0:
if self.lane_change_delay == AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.NUDGELESS]:
self.lane_change_wait_timer = ONE_SECOND_DELAY
else:
self.lane_change_wait_timer = self.lane_change_delay + ONE_SECOND_DELAY
def update_allowed(self) -> bool:
# Auto lane change allowed if:
# 1. A valid delay is set (non-zero)
# 2. Brake wasn't previously pressed
# 3. We've waited long enough
if self.lane_change_set_timer in (AutoLaneChangeMode.OFF, AutoLaneChangeMode.NUDGE):
return False
if self.prev_brake_pressed:
return False
if self.prev_lane_change:
return False
return bool(self.lane_change_wait_timer > self.lane_change_delay)
def update_lane_change(self, blindspot_detected: bool, brake_pressed: bool) -> None:
if brake_pressed and not self.prev_brake_pressed:
self.prev_brake_pressed = brake_pressed
self.update_lane_change_timers(blindspot_detected)
self.auto_lane_change_allowed = self.update_allowed()
def update_state(self):
if self.DH.lane_change_state == log.LaneChangeState.laneChangeStarting:
self.prev_lane_change = True
self.reset()
@@ -24,4 +24,4 @@ def get_lag_adjusted_curvature(steer_delay, v_ego, psis, curvatures):
current_curvature_desired - max_curvature_rate * DT_MDL,
current_curvature_desired + max_curvature_rate * DT_MDL)
return safe_desired_curvature
return float(safe_desired_curvature)
@@ -5,12 +5,14 @@ This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import os
import tomllib
from difflib import SequenceMatcher
from opendbc.car import structs
from openpilot.common.basedir import BASEDIR
TORQUE_NN_MODEL_PATH = os.path.join(BASEDIR, "sunnypilot", "neural_network_data", "neural_network_lateral_control")
TORQUE_NN_MODEL_SUBSTITUTE_PATH = os.path.join(BASEDIR, "opendbc", "car", "torque_data/substitute.toml")
MOCK_MODEL_PATH = os.path.join(TORQUE_NN_MODEL_PATH, "MOCK.json")
@@ -41,17 +43,26 @@ def get_nn_model_path(CP: structs.CarParams) -> tuple[str, str, bool]:
nn_candidate = car_fingerprint
model_path, max_similarity = check_nn_path(nn_candidate)
exact_match = max_similarity >= 0.99
if car_fingerprint not in model_path or 0.0 <= max_similarity < 0.8:
if car_fingerprint not in model_path or 0.0 <= max_similarity < 0.9:
nn_candidate = car_fingerprint
model_path, max_similarity = check_nn_path(nn_candidate)
if 0.0 <= max_similarity < 0.8:
model_path = MOCK_MODEL_PATH
exact_match = max_similarity >= 0.99
if 0.0 <= max_similarity < 0.9:
with open(TORQUE_NN_MODEL_SUBSTITUTE_PATH, 'rb') as f:
sub = tomllib.load(f)
sub_candidate = sub.get(car_fingerprint, car_fingerprint)
for candidate in [car_fingerprint, sub_candidate]:
model_path, max_similarity = check_nn_path(candidate)
exact_match = False
if CP.steerControlType == structs.CarParams.SteerControlType.angle:
model_path = MOCK_MODEL_PATH
model_name = os.path.splitext(os.path.basename(model_path))[0]
exact_match = max_similarity >= 0.99
return model_path, model_name, exact_match
@@ -24,7 +24,7 @@ class TestNNLCFingerprintBase:
CP_SP = CarInterface.get_non_essential_params_sp(CP, car_name)
CI = CarInterface(CP, CP_SP)
sunnypilot_interfaces.setup_car_interface_sp(CP, CP_SP, Params())
sunnypilot_interfaces.setup_interfaces(CP, CP_SP, Params())
return CI
@@ -1,4 +1,3 @@
from numpy.ma.testutils import assert_equal
from parameterized import parameterized
from opendbc.car.car_helpers import interfaces
@@ -13,8 +12,8 @@ from openpilot.sunnypilot.selfdrive.car import interfaces as sunnypilot_interfac
class TestNNTorqueModel:
@parameterized.expand([(HONDA.HONDA_CIVIC, True), (TOYOTA.TOYOTA_RAV4, True), (HYUNDAI.HYUNDAI_SANTA_CRUZ_1ST_GEN, False)])
def test_load_model(self, car_name, should_load_model):
@parameterized.expand([HONDA.HONDA_CIVIC, TOYOTA.TOYOTA_RAV4, HYUNDAI.HYUNDAI_SANTA_CRUZ_1ST_GEN])
def test_load_model(self, car_name):
params = Params()
params.put_bool("NeuralNetworkLateralControl", True)
@@ -23,10 +22,10 @@ class TestNNTorqueModel:
CP_SP = CarInterface.get_non_essential_params_sp(CP, car_name)
CI = CarInterface(CP, CP_SP)
sunnypilot_interfaces.setup_car_interface_sp(CP, CP_SP, params)
sunnypilot_interfaces.setup_interfaces(CP, CP_SP, params)
CP_SP = convert_to_capnp(CP_SP)
controller = LatControlTorque(CP.as_reader(), CP_SP.as_reader(), CI)
assert_equal(should_load_model, controller.extension.has_nn_model)
assert controller.extension.has_nn_model
@@ -0,0 +1,198 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from parameterized import parameterized
from openpilot.common.realtime import DT_MDL
from openpilot.selfdrive.controls.lib.desire_helper import DesireHelper, LaneChangeState, LaneChangeDirection
from openpilot.sunnypilot.selfdrive.controls.lib.auto_lane_change import AutoLaneChangeController, AutoLaneChangeMode, \
AUTO_LANE_CHANGE_TIMER, ONE_SECOND_DELAY
AUTO_LANE_CHANGE_TIMER_COMBOS = [
(AutoLaneChangeMode.NUDGELESS, AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.NUDGELESS]),
(AutoLaneChangeMode.HALF_SECOND, AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.HALF_SECOND]),
(AutoLaneChangeMode.ONE_SECOND, AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.ONE_SECOND]),
(AutoLaneChangeMode.TWO_SECONDS, AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.TWO_SECONDS]),
(AutoLaneChangeMode.THREE_SECONDS, AUTO_LANE_CHANGE_TIMER[AutoLaneChangeMode.THREE_SECONDS])
]
class TestAutoLaneChangeController:
def setup_method(self):
self.DH = DesireHelper()
self.alc = AutoLaneChangeController(self.DH)
def _reset_states(self):
self.alc.lane_change_bsm_delay = False
self.alc.lane_change_set_timer = AutoLaneChangeMode.NUDGE
self.lane_change_wait_timer = 0.0
self.prev_brake_pressed = False
self.prev_lane_change = False
def test_reset(self):
"""Test that reset correctly sets timers back to default."""
# Set some non-default values
self.alc.lane_change_wait_timer = 2.0
self.alc.prev_brake_pressed = True
# Set the DesireHelper to trigger a reset
self.DH.lane_change_state = LaneChangeState.off
self.DH.lane_change_direction = LaneChangeDirection.none
# Call reset
self.alc.reset()
# Check values were reset
assert self.alc.lane_change_wait_timer == 0.0
assert not self.alc.prev_brake_pressed
@parameterized.expand([(AutoLaneChangeMode.OFF, ), (AutoLaneChangeMode.NUDGE, )])
def test_off_and_nudge_mode(self, timer_state):
"""Test the default OFF and NUDGE mode behavior."""
self._reset_states()
# Setup mode
self.alc.lane_change_bsm_delay = False # BSM delay off
self.alc.lane_change_set_timer = timer_state
# Update controller
num_updates = int(5.0 / DT_MDL)
for _ in range(num_updates): # Run for 5 seconds
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# Mode should not allow lane change immediately
assert not self.alc.auto_lane_change_allowed
def test_nudgeless_mode(self):
"""Test the NUDGELESS mode behavior."""
self._reset_states()
# Setup NUDGELESS mode
self.alc.lane_change_bsm_delay = False # BSM delay off
self.alc.lane_change_set_timer = AutoLaneChangeMode.NUDGELESS
# Update controller once to read params
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# Update multiple times to exceed the timer threshold
for _ in range(1): # Should exceed 0.1s with multiple DT_MDL updates
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# Now lane change should be allowed
assert self.alc.lane_change_wait_timer > self.alc.lane_change_delay
assert self.alc.auto_lane_change_allowed
@parameterized.expand(AUTO_LANE_CHANGE_TIMER_COMBOS)
def test_timers(self, timer_state, timer_delay):
self._reset_states()
self.alc.lane_change_bsm_delay = False # BSM delay off
self.alc.lane_change_set_timer = timer_state
# Update controller once
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# The timer should still be below the threshold after one update
assert not self.alc.auto_lane_change_allowed
# Update enough times to exceed the threshold (seconds / DT_MDL)
num_updates = int(timer_delay / DT_MDL) + 1 # Add one extra updates to ensure we exceed the threshold
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# Now lane change should be allowed
assert self.alc.lane_change_wait_timer > self.alc.lane_change_delay
assert self.alc.auto_lane_change_allowed
@parameterized.expand(AUTO_LANE_CHANGE_TIMER_COMBOS)
def test_brake_pressed_disables_auto_lane_change(self, timer_state, timer_delay):
"""Test that pressing the brake disables auto lane change."""
self._reset_states()
# Setup auto lane change mode
self.alc.lane_change_bsm_delay = False
self.alc.lane_change_set_timer = timer_state
num_updates = int(timer_delay / DT_MDL) + 1 # Add one extra updates to ensure we exceed the threshold
# Update with brake pressed for 1 second
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=True)
# Even though it is an auto lane change mode, lane change should be disallowed due to brake pressed prior initiating lane change
assert not self.alc.auto_lane_change_allowed
# Check that prev_brake_pressed is saved
assert self.alc.prev_brake_pressed
# Even releasing brake shouldn't allow auto lane change
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
assert not self.alc.auto_lane_change_allowed
@parameterized.expand(AUTO_LANE_CHANGE_TIMER_COMBOS)
def test_blindspot_detected_with_bsm_delay(self, timer_state, timer_delay):
"""Test behavior when blindspot is detected with BSM delay enabled."""
# Blindspot detected - should prevent auto lane change
self._reset_states()
self.alc.lane_change_bsm_delay = True # BSM delay on
self.alc.lane_change_set_timer = timer_state
# Update with blindspot detected - this should prevent auto lane change
self.alc.update_lane_change(blindspot_detected=True, brake_pressed=False)
assert not self.alc.auto_lane_change_allowed
# Keep updating with blindspot detected - should still prevent auto lane change
num_updates = int(timer_delay / DT_MDL) + 1 # Add one extra updates to ensure we exceed the threshold
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=True, brake_pressed=False)
assert not self.alc.auto_lane_change_allowed
@parameterized.expand(AUTO_LANE_CHANGE_TIMER_COMBOS)
def test_blindspot_detected_then_undetected_with_bsm_delay(self, timer_state, timer_delay):
"""Test behavior when blindspot is detected then undetected with BSM delay enabled."""
# Blindspot clears - should allow auto lane change after sufficient time
self._reset_states()
self.alc.lane_change_bsm_delay = True
self.alc.lane_change_set_timer = timer_state
# First update with blindspot detected to set the negative timer
self.alc.update_lane_change(blindspot_detected=True, brake_pressed=False)
assert not self.alc.auto_lane_change_allowed
# Now update with blindspot cleared - should start incrementing timer from negative value
num_updates = int((timer_delay + abs(ONE_SECOND_DELAY)) / DT_MDL) + 1
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# After sufficient updates with no blindspot, auto lane change should be allowed
assert self.alc.auto_lane_change_allowed
@parameterized.expand(AUTO_LANE_CHANGE_TIMER_COMBOS)
def test_disallow_continuous_auto_lane_change(self, timer_state, timer_delay):
self._reset_states()
self.alc.lane_change_bsm_delay = False # BSM delay off
self.alc.lane_change_set_timer = timer_state
num_updates = int(timer_delay / DT_MDL) + 1 # Add one extra updates to ensure we exceed the threshold
# Update enough times to exceed the threshold (seconds / DT_MDL)
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
# Now lane change should be allowed
assert self.alc.lane_change_wait_timer > self.alc.lane_change_delay
assert self.alc.auto_lane_change_allowed
# Simulate lane change is initiated
self.DH.lane_change_state = LaneChangeState.laneChangeStarting
self.alc.update_state()
# Simulate lane change is completed, and one_blinker stays on
self.DH.lane_change_state = LaneChangeState.preLaneChange
self.alc.update_state()
# Update enough times to exceed the threshold (seconds / DT_MDL)
for _ in range(num_updates):
self.alc.update_lane_change(blindspot_detected=False, brake_pressed=False)
assert not self.alc.auto_lane_change_allowed
+1 -1
View File
@@ -26,7 +26,7 @@ class EventsSP(EventsBase):
return EVENT_NAME_SP[event]
def get_event_msg_type(self):
return custom.OnroadEventSP
return custom.OnroadEventSP.Event
EVENTS_SP: dict[int, dict[str, Alert | AlertCallbackType]] = {
+2 -2
View File
@@ -24,11 +24,11 @@ class SunnylinkApi(BaseApi):
self.spinner = None
self.params = Params()
def api_get(self, endpoint, method='GET', timeout=10, access_token=None, **kwargs):
def api_get(self, endpoint, method='GET', timeout=10, access_token=None, json=None, **kwargs):
if not self.params.get_bool("SunnylinkEnabled"):
return None
return super().api_get(endpoint, method, timeout, access_token, **kwargs)
return super().api_get(endpoint, method, timeout, access_token, json, **kwargs)
def resume_queued(self, timeout=10, **kwargs):
sunnylinkId, commaId = self._resolve_dongle_ids()
+34
View File
@@ -0,0 +1,34 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from Crypto.Cipher import AES
class AESCipher:
def __init__(self, key: bytes, iv: bytes):
if len(key) not in (16, 32):
raise ValueError("Key must be 16 bytes (AES-128) or 32 bytes (AES-256).")
if len(iv) != 16:
raise ValueError("IV must be 16 bytes.")
self.key = key
self.iv = iv
def encrypt(self, data: bytes) -> bytes:
block_size = 16
padding_length = block_size - (len(data) % block_size)
padding = bytes([padding_length]) * padding_length
padded_data = data + padding
cipher = AES.new(self.key, AES.MODE_CBC, self.iv)
return cipher.encrypt(padded_data)
def decrypt(self, encrypted_data: bytes) -> bytes:
cipher = AES.new(self.key, AES.MODE_CBC, self.iv)
decrypted_data = cipher.decrypt(encrypted_data)
padding_length = decrypted_data[-1]
return decrypted_data[:-padding_length]
+268
View File
@@ -0,0 +1,268 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import base64
import json
import time
from enum import Enum
from typing import Any
from openpilot.common.git import get_branch
from openpilot.common.params import Params, ParamKeyType
from openpilot.common.realtime import Ratekeeper
from openpilot.common.swaglog import cloudlog
from openpilot.system.version import get_version
from cereal import messaging, custom
from sunnypilot.sunnylink.api import SunnylinkApi
from sunnypilot.sunnylink.backups.utils import decrypt_compressed_data, encrypt_compress_data, SnakeCaseEncoder
class OperationType(Enum):
BACKUP = "backup"
RESTORE = "restore"
class BackupManagerSP:
"""Manages device configuration backups to/from sunnylink"""
def __init__(self):
self.params = Params()
self.device_id = self.params.get("SunnylinkDongleId", encoding="utf8")
self.api = SunnylinkApi(self.device_id)
self.pm = messaging.PubMaster(["backupManagerSP"])
# Status tracking
self.backup_status = custom.BackupManagerSP.Status.idle
self.restore_status = custom.BackupManagerSP.Status.idle
# Unified progress & operation type (only one operation runs at a time)
self.progress = 0.0
self.operation: OperationType | None = None
self.last_error = ""
def _report_status(self) -> None:
"""Reports current backup manager state through the messaging system."""
msg = messaging.new_message('backupManagerSP', valid=True)
backup_state = msg.backupManagerSP
backup_state.backupStatus = self.backup_status
backup_state.restoreStatus = self.restore_status
# Both progress fields use the unified progress value
backup_state.backupProgress = self.progress
backup_state.restoreProgress = self.progress
backup_state.lastError = self.last_error
# Optionally, add a field for operation type if supported:
# backup_state.operationType = self.operation.value if self.operation else "none"
self.pm.send('backupManagerSP', msg)
def _update_progress(self, progress: float, op_type: OperationType) -> None:
"""Updates the unified progress and operation type, then reports status."""
self.progress = progress
self.operation = op_type
self._report_status()
def _collect_config_data(self) -> dict[str, Any]:
"""Collects configuration data to be backed up."""
config_data = {}
params_to_backup = [k.decode('utf-8') for k in self.params.all_keys(ParamKeyType.BACKUP)]
for param in params_to_backup:
value = self.params.get(param)
if value is not None:
config_data[param] = base64.b64encode(value).decode('utf-8')
return config_data
def _get_metadata_value(self, metadata_list, key, default_value=None):
return next((entry.get("value") for entry in metadata_list if entry.get("key") == key), default_value)
async def create_backup(self) -> bool:
"""Creates and uploads a new backup to sunnylink."""
try:
self.backup_status = custom.BackupManagerSP.Status.inProgress
self._update_progress(0.0, OperationType.BACKUP)
# Collect configuration data
config_data = self._collect_config_data()
self._update_progress(25.0, OperationType.BACKUP)
# Serialize and encrypt config data
config_json = json.dumps(config_data)
encrypted_config = encrypt_compress_data(config_json, use_aes_256=True)
self._update_progress(50.0, OperationType.BACKUP)
backup_info = custom.BackupManagerSP.BackupInfo()
backup_info.deviceId = self.device_id
backup_info.config = encrypted_config
backup_info.isEncrypted = True
backup_info.createdAt = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime())
backup_info.updatedAt = backup_info.createdAt
backup_info.sunnypilotVersion = self._get_current_version()
backup_info.backupMetadata = [
custom.BackupManagerSP.MetadataEntry(key="creator", value="BackupManagerSP"),
custom.BackupManagerSP.MetadataEntry(key="all_values_encoded", value="True"),
custom.BackupManagerSP.MetadataEntry(key="AES", value="256")
]
payload = json.loads(json.dumps(backup_info.to_dict(), cls=SnakeCaseEncoder))
self._update_progress(75.0, OperationType.BACKUP)
# Upload to sunnylink
result = self.api.api_get(
f"backup/{self.device_id}",
method='PUT',
access_token=self.api.get_token(),
json=payload
)
if result:
self.backup_status = custom.BackupManagerSP.Status.completed
self._update_progress(100.0, OperationType.BACKUP)
else:
self.backup_status = custom.BackupManagerSP.Status.failed
self.last_error = "Failed to upload backup"
self._report_status()
return bool(self.backup_status == custom.BackupManagerSP.Status.completed)
except Exception as e:
cloudlog.exception(f"Error creating backup: {str(e)}")
self.backup_status = custom.BackupManagerSP.Status.failed
self.last_error = str(e)
self._report_status()
return False
async def restore_backup(self, version: int | None = None) -> bool:
"""Restores a backup from sunnylink."""
try:
self.restore_status = custom.BackupManagerSP.Status.inProgress
self._update_progress(0.0, OperationType.RESTORE)
# Get backup data from API for the specified version
endpoint = f"backup/{self.device_id}" + f"/{version or ''}" + "?api-version=1"
backup_data = self.api.api_get(endpoint, access_token=self.api.get_token())
if not backup_data:
raise Exception(f"No backup found for device {self.device_id}")
self._update_progress(25.0, OperationType.RESTORE)
data = backup_data.json()
backup_metadata = data.get("backup_metadata", [])
encrypted_config = data.get("config", "")
if not encrypted_config:
raise Exception("Empty backup configuration")
self._update_progress(50.0, OperationType.RESTORE)
# Decrypt config and load data
use_aes_256 = self._get_metadata_value(backup_metadata, "AES", "128") == "256"
config_json = decrypt_compressed_data(encrypted_config, use_aes_256)
if not config_json:
raise Exception("Failed to decrypt backup configuration")
config_data = json.loads(config_json)
self._update_progress(75.0, OperationType.RESTORE)
# Apply configuration
all_values_encoded = self._get_metadata_value(backup_metadata, "all_values_encoded", "false")
self._apply_config(config_data, str(all_values_encoded).lower() == "true")
self.restore_status = custom.BackupManagerSP.Status.completed
self._update_progress(100.0, OperationType.RESTORE)
return True
except Exception as e:
cloudlog.exception(f"Error restoring backup: {str(e)}")
self.restore_status = custom.BackupManagerSP.Status.failed
self.last_error = str(e)
self._report_status()
return False
def _apply_config(self, config_data: dict[str, str], all_values_encoded: bool = False) -> None:
"""Applies configuration data from a backup, but only for parameters marked as backupable."""
# Get the current list of parameters that can be backed up
backupable_params = [k.decode('utf-8') for k in self.params.all_keys(ParamKeyType.BACKUP)]
# Count for logging/reporting
restored_count = 0
skipped_count = 0
for param, encoded_value in config_data.items():
try:
# Only restore parameters that are currently marked as backupable
if param in backupable_params:
value = base64.b64decode(encoded_value) if all_values_encoded else encoded_value
self.params.put(param, value)
restored_count += 1
else:
skipped_count += 1
cloudlog.info(f"Skipped restoring param {param}: not marked for backup in current version")
except Exception as e:
cloudlog.error(f"Failed to restore param {param}: {str(e)}")
cloudlog.info(f"Restore complete: {restored_count} params restored, {skipped_count} params skipped")
def _get_current_version(self) -> custom.BackupManagerSP.Version:
"""Gets current sunnypilot version information."""
version_obj = custom.BackupManagerSP.Version()
version_parts = get_version().split('.')
version_obj.major = int(version_parts[0]) if len(version_parts) > 0 else 0
version_obj.minor = int(version_parts[1]) if len(version_parts) > 1 else 0
version_obj.patch = int(version_parts[2]) if len(version_parts) > 2 else 0
version_obj.build = int(version_parts[3]) if len(version_parts) > 3 else 0
version_obj.branch = get_branch()
return version_obj
async def main_thread(self) -> None:
"""Main thread for backup management."""
rk = Ratekeeper(1, print_delay_threshold=None)
reset_progress = False
while True:
try:
if reset_progress:
self.progress = 100.0
self.operation = None
self.restore_status = custom.BackupManagerSP.Status.idle
self.backup_status = custom.BackupManagerSP.Status.idle
# Check for backup command
if self.params.get_bool("BackupManager_CreateBackup"):
try:
await self.create_backup()
reset_progress = True
finally:
self.params.remove("BackupManager_CreateBackup")
# Check for restore command
restore_version = self.params.get("BackupManager_RestoreVersion", encoding="utf8")
if restore_version:
try:
version = int(restore_version) if restore_version.isdigit() else None
await self.restore_backup(version)
reset_progress = True
finally:
self.params.remove("BackupManager_RestoreVersion")
self._report_status()
rk.keep_time()
except Exception as e:
cloudlog.exception(f"Error in backup manager main thread: {str(e)}")
self.last_error = str(e)
self._report_status()
rk.keep_time()
def main():
import asyncio
asyncio.run(BackupManagerSP().main_thread())
if __name__ == "__main__":
main()
+170
View File
@@ -0,0 +1,170 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import base64
import hashlib
import zlib
import re
import json
from pathlib import Path
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from sunnypilot.sunnylink.backups.AESCipher import AESCipher
from openpilot.system.hardware.hw import Paths
class KeyDerivation:
@staticmethod
def _load_key(file_path: str) -> bytes:
with open(file_path, 'rb') as f:
return f.read()
@staticmethod
def derive_aes_key_iv_from_rsa(key_path: str, use_aes_256: bool) -> tuple[bytes, bytes]:
rsa_key_pem: bytes = KeyDerivation._load_key(key_path)
key_plain = rsa_key_pem.decode(errors="ignore")
if "private" in key_plain.lower():
private_key = serialization.load_pem_private_key(rsa_key_pem, password=None, backend=default_backend())
if not isinstance(private_key, rsa.RSAPrivateKey):
raise ValueError("Invalid RSA key format: Unable to determine if key is public or private.")
der_data = private_key.private_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption()
)
elif "public" in key_plain.lower():
public_key = serialization.load_pem_public_key(rsa_key_pem, backend=default_backend())
if not isinstance(public_key, rsa.RSAPublicKey):
raise ValueError("Invalid RSA key format: Unable to determine if key is public or private.")
der_data = public_key.public_bytes(encoding=serialization.Encoding.DER, format=serialization.PublicFormat.PKCS1)
else:
raise ValueError("Unknown key format: Unable to determine if key is public or private.")
sha256_hash = hashlib.sha256(der_data).digest()
aes_key = sha256_hash[:32] if use_aes_256 else sha256_hash[:16]
aes_iv = sha256_hash[16:32]
return aes_key, aes_iv
def qUncompress(data):
"""
Decompress data using zlib.
Args:
data (bytes): Compressed data
Returns:
bytes: Decompressed data
"""
data_stripped_4 = data[4:]
return zlib.decompress(data_stripped_4)
def qCompress(data):
"""
Compress data using zlib.
Args:
data (bytes): Data to compress
Returns:
bytes: Compressed data
"""
compressed_data = zlib.compress(data, level=9)
return b"ZLIB" + compressed_data
def decrypt_compressed_data(encrypted_base64, use_aes_256=False):
"""
Decrypt and decompress data from base64 string.
Args:
encrypted_base64 (str): Base64 encoded encrypted data
key_path (str, optional): Path to RSA public key
Returns:
str: Decrypted and decompressed string
"""
key_path = Path(f"{Paths.persist_root()}/comma/id_rsa") if use_aes_256 else Path(f"{Paths.persist_root()}/comma/id_rsa.pub")
try:
# Decode base64
encrypted_data = base64.b64decode(encrypted_base64)
# Decrypt
key, iv = KeyDerivation.derive_aes_key_iv_from_rsa(str(key_path), use_aes_256)
cipher = AESCipher(key, iv)
decrypted_data = cipher.decrypt(encrypted_data)
# Decompress
decompressed_data = qUncompress(decrypted_data)
# Decode UTF-8
result = decompressed_data.decode('utf-8')
return result
except Exception as e:
print(f"Decryption and decompression failed: {e}")
return ""
def encrypt_compress_data(text, use_aes_256=True):
"""
Compress and encrypt string data to base64.
Args:
text (str): Text to compress and encrypt
key_path (str, optional): Path to RSA public key
Returns:
str: Base64 encoded encrypted data
"""
key_path = Path(f"{Paths.persist_root()}/comma/id_rsa") if use_aes_256 else Path(f"{Paths.persist_root()}/comma/id_rsa.pub")
try:
# Encode to UTF-8
text_bytes = text.encode('utf-8')
# Compress
compressed_data = qCompress(text_bytes)
# Encrypt
key, iv = KeyDerivation.derive_aes_key_iv_from_rsa(str(key_path), use_aes_256)
cipher = AESCipher(key, iv)
encrypted_data = cipher.encrypt(compressed_data)
# Encode to base64
result = base64.b64encode(encrypted_data).decode('utf-8')
return result
except Exception as e:
print(f"Compression and encryption failed: {e}")
return ""
def camel_to_snake(name):
"""Convert camelCase to snake_case."""
name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()
def transform_dict(obj):
"""Recursively transform dictionary keys from camelCase to snake_case."""
if isinstance(obj, dict):
return {camel_to_snake(k): transform_dict(v) for k, v in obj.items()}
elif isinstance(obj, list):
return [transform_dict(item) for item in obj]
return obj
class SnakeCaseEncoder(json.JSONEncoder):
def encode(self, obj):
transformed_obj = transform_dict(obj)
return super().encode(transformed_obj)
+1
View File
@@ -91,6 +91,7 @@ def register(show_spinner=False) -> str | None:
if time.monotonic() - start_time > 60 and show_spinner:
spinner.update(f"registering device - serial: {serial}, IMEI: ({imei1}, {imei2})")
return UNREGISTERED_DONGLE_ID # hotfix to prevent an infinite wait for registration
if show_spinner:
spinner.close()
+3
View File
@@ -43,6 +43,8 @@ def manager_init() -> None:
]
sunnypilot_default_params: list[tuple[str, str | bytes]] = [
("AutoLaneChangeTimer", "0"),
("AutoLaneChangeBsmDelay", "0"),
("DynamicExperimentalControl", "0"),
("Mads", "1"),
("MadsMainCruiseAllowed", "1"),
@@ -51,6 +53,7 @@ def manager_init() -> None:
("ModelManager_LastSyncTime", "0"),
("ModelManager_ModelsCache", ""),
("NeuralNetworkLateralControl", "0"),
("QuietMode", "0"),
]
if params.get_bool("RecordFrontLock"):
+4
View File
@@ -148,9 +148,13 @@ procs = [
# sunnypilot
procs += [
# Models
PythonProcess("models_manager", "sunnypilot.models.manager", only_offroad),
NativeProcess("modeld_snpe", "sunnypilot/modeld", ["./modeld"], and_(only_onroad, is_snpe_model)),
NativeProcess("modeld_tinygrad", "sunnypilot/modeld_v2", ["./modeld"], and_(only_onroad, is_tinygrad_model)),
# Backup
PythonProcess("backup_manager", "sunnypilot.sunnylink.backups.manager", and_(only_offroad, sunnylink_ready_shim)),
]
if os.path.exists("./github_runner.sh"):