Compare commits

..

13 Commits

Author SHA1 Message Date
DevTekVE
40d9e092b6 Refactor type annotations and return types in model_runner
Removed specific type hints and return annotations for `self.inputs` and `run_model` methods to enhance flexibility and maintain consistency. These changes streamline the code and improve compatibility with varying input/output types during model inference.
2024-12-29 18:04:05 +01:00
DevTekVE
6318aa52e3 Refactor type hints for improved code consistency.
Replaced specific type hints like `dict[str, np.ndarray]` with generic `dict` in several method signatures to simplify annotations. This improves overall readability and maintains functionality, while aligning with existing code practices.
2024-12-29 17:59:50 +01:00
DevTekVE
7852fa66b1 Clean 2024-12-29 17:51:30 +01:00
DevTekVE
8d9c1e2035 Add type annotations to inputs and prepare_inputs method
This commit adds explicit type annotations for the `inputs` dictionary and the return value of the `prepare_inputs` method in `model_runner.py`. These changes improve code readability and ensure type consistency, enhancing maintainability and reducing potential errors.
2024-12-29 17:50:58 +01:00
DevTekVE
c39f722f7b Rename TinyGradRunner to TinygradRunner for consistency.
The class name and references were updated to maintain naming consistency across the codebase. This aligns with naming conventions and improves code clarity. No functional changes were introduced.
2024-12-29 17:48:00 +01:00
DevTekVE
550c08ac4c Refactor prepare_inputs to use explicit CLMem type.
This update replaces the generic `any` type with the more explicit `CLMem` type for better type safety and clarity. It ensures consistency across the `prepare_inputs` method implementations in derived classes, improving code readability and robustness.
2024-12-29 17:44:23 +01:00
DevTekVE
8c838af5fa Refactor model inference to use internal state for inputs
Simplified the `run_model` method by removing the requirement to pass inputs as arguments, and instead leveraging an internal `inputs` state. Adjusted `prepare_inputs` methods across model runners to populate this internal state. This refactor improves code clarity and reduces redundancy in managing input data.
2024-12-29 17:40:45 +01:00
DevTekVE
2521d60b1b Fix model_runner output to ensure tensor conversion to NumPy.
Updated the `run_model` method to explicitly convert tensor outputs to NumPy arrays using `.numpy()`. This ensures compatibility with downstream processes relying on NumPy array inputs.
2024-12-29 17:31:27 +01:00
DevTekVE
206398d3b5 Remove unused imports and redundant pass statements.
This change cleans up the code by removing unused imports and redundant `pass` statements in abstract methods. It improves code readability and adheres to cleaner coding practices.
2024-12-29 17:22:13 +01:00
DevTekVE
a9e99615cd Remove redundant tensor assignment in model runner.
The `assign` operation was unnecessary as new tensors are realized for updated inputs. This simplifies the code and avoids redundant updates, improving clarity and maintainability.
2024-12-29 17:20:15 +01:00
DevTekVE
ec44b78ad8 Refactor model runner initialization logic.
Removed the `create_model_runner` factory function and replaced it with direct initialization of `TinyGradRunner` or `ONNXRunner`. Simplified the `__init__` methods by standardizing paths as constants within `model_runner.py` for cleaner and more maintainable code.
2024-12-29 17:14:08 +01:00
DevTekVE
dcd3e09294 Refactor formatting in model_runner.py for readability.
Consolidated multiline function declarations and calls into single lines where appropriate to improve code readability and maintainability. No changes were made to the functionality.
2024-12-29 17:01:26 +01:00
DevTekVE
839a7a58e0 Refactoring model handling in modeld.py with ModelRunner abstraction
A significant refactoring of `modeld.py` was performed to enhance the handling of model logic. A new abstraction called `ModelRunner` has been introduced which encapsulates the model-running logic. This refactor simplifies the `modeld.py` script and provides easier management across different hardware configurations. Using this segregation, varying processing methods for models can be handled distinctly ensuring cleaner and more maintainable code. An instance of the appropriate model runner is now created during initialization based on whether a TICI hardware or a different type is used.
2024-12-29 16:59:10 +01:00
539 changed files with 5343 additions and 26186 deletions

View File

@@ -1,3 +0,0 @@
Wen
REGIST
PullRequest

15
.github/PULL_REQUEST_TEMPLATE/bugfix.md vendored Normal file
View File

@@ -0,0 +1,15 @@
---
name: Bug fix
about: For openpilot bug fixes
title: ''
labels: 'bugfix'
assignees: ''
---
**Description**
<!-- A description of the bug and the fix. Also link the issue if it exists. -->
**Verification**
<!-- Explain how you tested this bug fix. -->

View File

@@ -0,0 +1,19 @@
---
name: Car Bug fix
about: For vehicle/brand specific bug fixes
title: ''
labels: 'car bug fix'
assignees: ''
---
**Description**
<!-- A description of the bug and the fix. Also link the issue if it exists. -->
**Verification**
<!-- Explain how you tested this bug fix. -->
**Route**
Route: [a route with the bug fix]

View File

@@ -0,0 +1,15 @@
---
name: Car port
about: For new car ports
title: ''
labels: 'car port'
assignees: ''
---
**Checklist**
- [ ] added entry to CAR in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
- [ ] route with openpilot:
- [ ] route with stock system:
- [ ] car harness used (if comma doesn't sell it, put N/A):

View File

@@ -0,0 +1,13 @@
---
name: Fingerprint
about: For adding fingerprints to existing cars
title: ''
labels: 'fingerprint'
assignees: ''
---
**Car**
Which car (make, model, year) this fingerprint is for
**Route**
A route with the fingerprint

View File

@@ -0,0 +1,15 @@
---
name: Refactor
about: For code refactors
title: ''
labels: 'refactor'
assignees: ''
---
**Description**
<!-- A description of the refactor, including the goals it accomplishes. -->
**Verification**
<!-- Explain how you tested the refactor for regressions. -->

31
.github/PULL_REQUEST_TEMPLATE/tuning.md vendored Normal file
View File

@@ -0,0 +1,31 @@
---
name: Tuning
about: For openpilot tuning changes
title: ''
labels: 'tuning'
assignees: ''
---
**Description**
<!-- A description of what is wrong with the current tuning and how the PR addresses this. -->
**Verification**
<!-- To verify tuning, capture the following scenarios (broadly, not exactly), with current tune and this tune.
Use the PlotJuggler tuning layout to compare planned versus actual behavior.
Run ./juggle.py <route> --layout layouts/tuning.xml , screenshot the full tab of interest, and paste into this PR.
Longitudinal:
* Maintaining speed at 25, 40, 65mph
* Driving up and down hills
* Accelerating from a stop
* Decelerating to a stop
* Following large changes in set speed
* Coming to a stop behind a lead car
Lateral:
* Straight driving at ~25, ~45 and ~65mph
* Turns driving at ~25, ~45 and ~65mph
-->

30
.github/build.py vendored Normal file
View File

@@ -0,0 +1,30 @@
import pathlib
GITHUB_FOLDER = pathlib.Path(__file__).parent
PULL_REQUEST_TEMPLATES = (GITHUB_FOLDER / "PULL_REQUEST_TEMPLATE")
order = ["fingerprint", "car_bugfix", "bugfix", "car_port", "refactor"]
def create_pull_request_template():
with open(GITHUB_FOLDER / "pull_request_template.md", "w") as f:
f.write("<!-- Please copy and paste the relevant template -->\n\n")
for t in order:
template = PULL_REQUEST_TEMPLATES / f"{t}.md"
text = template.read_text()
# Remove metadata for GitHub
start = text.find("---")
end = text.find("---", start+1)
text = text[end + 4:]
# Remove comments
text = text.replace("<!-- ", "").replace("-->", "")
f.write(f"<!--- ***** Template: {template.stem.replace('_', ' ').title()} *****\n")
f.write(text)
f.write("\n\n")
f.write("-->\n\n")
create_pull_request_template()

View File

@@ -1,43 +0,0 @@
exclude-labels:
- 'no-changelog'
categories:
- title: '🚀 Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
collapse-after: 5
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧰 Maintenance'
collapse-after: 5
label: 'chore'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
change-title-escapes: '\<*_&'
replacers:
- search: '/[Ss][Uu][Nn][Nn][Yy][Pp][Ii][Ll][Oo][Tt]/g'
replace: 'sunnypilot'
- search: '/\b[Ss][Pp]\b/g'
replace: 'SP'
version-resolver:
major:
labels:
- 'major'
minor:
labels:
- 'minor'
patch:
labels:
- 'patch'
default: patch
name-template: 'v$RESOLVED_VERSION 🚀'
tag-template: 'v$RESOLVED_VERSION'
version-template: "0.$MAJOR.$MINOR.$PATCH" # The day OP becomes v1, we need to bump this
tag-prefix: "v0." # The day OP becomes v1, we need to bump this
prerelease-identifier: "staging"
template: |
## Changes
$CHANGES

View File

@@ -1,29 +0,0 @@
name: Release Drafter
on:
push:
branches:
- master-new
- master
tags:
- 'v*'
pull_request_target:
types: [opened, reopened, synchronize]
workflow_dispatch:
permissions:
contents: read
jobs:
update_release_draft:
permissions:
contents: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v6
with:
config-name: release-drafter.yml
prerelease: ${{ !startsWith(github.ref, 'refs/tags/v') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -67,7 +67,15 @@ jobs:
timeout-minutes: 1
run: |
cd $STRIPPED_DIR
${{ env.RUN }} "release/check-dirty.sh"
${{ env.RUN }} "release/check-dirty.sh && \
MAX_EXAMPLES=5 $PYTEST -m 'not slow' selfdrive/car system/manager"
- name: Static analysis
timeout-minutes: 1
run: |
cd $GITHUB_WORKSPACE
cp pyproject.toml $STRIPPED_DIR
cd $STRIPPED_DIR
${{ env.RUN }} "scripts/lint/lint.sh --skip check_added_large_files"
build:
runs-on:
@@ -344,7 +352,7 @@ jobs:
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc)"
- name: Create Test Report
timeout-minutes: ${{ ((steps.frames-cache.outputs.cache-hit == 'true') && 2 || 4) }}
timeout-minutes: ${{ ((steps.frames-cache.outputs.cache-hit == 'true') && 1 || 3) }}
run: >
${{ env.RUN }} "PYTHONWARNINGS=ignore &&
source selfdrive/test/setup_xvfb.sh &&

View File

@@ -18,18 +18,6 @@ on:
description: 'Custom name for the model'
required: false
type: string
file_name:
description: 'File name prefix for the model files'
required: false
type: string
is_20hz:
description: 'Is this a 20Hz model'
required: false
type: boolean
default: false
run-name: Build model [${{ inputs.custom_name || inputs.upstream_branch }}] from ref [${{ inputs.upstream_branch }}]
jobs:
build_model:
@@ -49,15 +37,12 @@ jobs:
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-master-new
scons-${{ runner.os }}-${{ runner.arch }}-master
scons-${{ runner.os }}-${{ runner.arch }}
- name: Setup build environment
@@ -84,8 +69,6 @@ jobs:
run: |
sudo rm -rf ${OUTPUT_DIR}
mkdir -p ${OUTPUT_DIR}
# Copy the model files
rsync -avm \
--include='*.dlc' \
--include='*.thneed' \
@@ -95,36 +78,6 @@ jobs:
--delete-excluded \
--chown=comma:comma \
./selfdrive/modeld/models/ ${OUTPUT_DIR}/
# Rename files if file_name is provided
if [ ! -z "${{ inputs.file_name }}" ]; then
mv ${OUTPUT_DIR}/supercombo.thneed ${OUTPUT_DIR}/supercombo-${{ inputs.file_name }}.thneed
mv ${OUTPUT_DIR}/supercombo_metadata.pkl ${OUTPUT_DIR}/supercombo-${{ inputs.file_name }}_metadata.pkl
fi
# Calculate SHA256 hashes
HASH=$(sha256sum ${OUTPUT_DIR}/supercombo*.thneed | cut -d' ' -f1)
METADATA_HASH=$(sha256sum ${OUTPUT_DIR}/supercombo*_metadata.pkl | cut -d' ' -f1)
# Create metadata.json
cat > ${OUTPUT_DIR}/metadata.json << EOF
{
"display_name": "${{ inputs.custom_name || inputs.upstream_branch }}",
"full_name": "${{ inputs.file_name || 'default' }}",
"is_20hz": ${{ inputs.is_20hz || false }},
"files": {
"drive_model": {
"file_name": "$(basename ${OUTPUT_DIR}/supercombo*.thneed)",
"sha256": "${HASH}"
},
"metadata": {
"file_name": "$(basename ${OUTPUT_DIR}/supercombo*_metadata.pkl)",
"sha256": "${METADATA_HASH}"
}
},
"ref": "${{ inputs.upstream_branch }}",
"build_time": "$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
}
- name: Upload Build Artifacts
uses: actions/upload-artifact@v4

View File

@@ -51,10 +51,9 @@ jobs:
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}
@@ -120,7 +119,6 @@ jobs:
if [[ "${{ runner.debug }}" == "1" ]]; then
printenv
fi
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
- name: Build Panda
run: |
@@ -189,11 +187,6 @@ jobs:
name: prebuilt
path: prebuilt.tar.gz
- name: Re-enable powersave
if: always()
run: |
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
publish:
concurrency:
group: publish-${{ github.head_ref || github.ref_name }}

View File

@@ -1,162 +0,0 @@
name: Nightly Branch Reset and PR Squash
env:
DEFAULT_SOURCE_BRANCH: "master-new"
DEFAULT_TARGET_BRANCH: "nightly"
PR_LABEL: "dev-c3"
on:
workflow_dispatch:
inputs:
source_branch:
description: 'Source branch to reset from'
required: true
default: 'master-new'
type: string
target_branch:
description: 'Target branch to reset and squash into'
required: true
default: 'master-dev-c3-new'
type: string
# schedule:
# - cron: '0 0 * * *' # Run at midnight UTC for nightly
jobs:
reset-and-squash:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches
token: ${{ secrets.GITHUB_TOKEN }}
- name: Configure Git
run: |
git config --global user.name 'github-actions[bot]'
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install PyGithub
- name: Check branches exist
run: |
# Check if source branch exists
if ! git ls-remote --heads origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} | grep -q "${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"; then
echo "Source branch ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} does not exist!"
exit 1
fi
# Make sure we have the latest source branch
git fetch origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
# Check if target branch exists
if ! git ls-remote --heads origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} | grep -q "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"; then
echo "Target branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} does not exist, creating it from ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
git checkout -b ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
git push origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
else
# Fetch target branch if it exists
git fetch origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
fi
- name: Reset target branch
run: |
echo "Resetting ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} to match ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
# Delete if exists and recreate pointing to source
git branch -D ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} || true
git branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
- name: Get PRs to squash
id: get-prs
run: |
# Use GitHub API to get PRs with specific label, ordered by creation date
PR_LIST=$(gh api graphql -f query='
query($label:String!) {
search(query: $label, type:ISSUE, first:100) {
nodes {
... on PullRequest {
number
headRefName
title
createdAt
commits(last: 1) {
nodes {
commit {
statusCheckRollup {
state
}
}
}
}
}
}
}
}' -F label="is:pr is:open label:${PR_LABEL} sort:created-asc")
echo "PR_LIST=${PR_LIST}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Process PRs
run: |
python3 ${{ github.workspace }}/release/ci/squash_and_merge_prs.py \
--pr-data '${{ steps.get-prs.outputs.PR_LIST }}' \
--target-branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} \
--squash-script-path '${{ github.workspace }}/release/ci/squash_and_merge.py'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Push changes if there are diffs
id: push-changes # Add an id so we can reference this step
run: |
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
# Fetch the latest from remote
git fetch origin $TARGET_BRANCH
# Check for diffs between local and remote
if git diff $TARGET_BRANCH origin/$TARGET_BRANCH --quiet; then
echo "No changes to push - local and remote branches are identical"
echo "has_changes=false" >> $GITHUB_OUTPUT
exit 0
fi
# If we get here, there are diffs, so push
if ! git push origin $TARGET_BRANCH --force; then
echo "Failed to push changes to $TARGET_BRANCH"
exit 1
fi
echo "Branch $TARGET_BRANCH has been reset and updated with squashed PRs"
echo "has_changes=true" >> $GITHUB_OUTPUT
- name: Trigger and wait for selfdrive tests
if: steps.push-changes.outputs.has_changes == 'true'
run: |
echo "Triggering selfdrive tests..."
gh workflow run selfdrive_tests.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
echo "Sleeping for 120s to give plenty of time for the action to start and then we wait"
sleep 120
echo "Getting latest run ID..."
RUN_ID=$(gh run list --workflow=selfdrive_tests.yaml --branch="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}" --limit=1 --json databaseId --jq '.[0].databaseId')
echo "Watching run ID: $RUN_ID"
gh run watch "$RUN_ID"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Trigger prebuilt workflow
if: success() && steps.push-changes.outputs.has_changes == 'true'
run: |
gh workflow run sunnypilot-build-prebuilt.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -86,7 +86,7 @@ jobs:
run: >-
sudo apt-get install -y imagemagick
scenes="homescreen settings_device settings_network settings_network_advanced settings_software settings_sunnylink settings_toggles settings_sunnypilot settings_sunnypilot_mads settings_trips settings_vehicle settings_developer offroad_alert update_available prime onroad onroad_disengaged onroad_override onroad_sidebar onroad_wide onroad_wide_sidebar onroad_alert_small onroad_alert_mid onroad_alert_full driver_camera body keyboard keyboard_uppercase"
scenes="homescreen settings_device settings_software settings_toggles settings_developer offroad_alert update_available prime onroad onroad_disengaged onroad_override onroad_sidebar onroad_wide onroad_wide_sidebar onroad_alert_small onroad_alert_mid onroad_alert_full driver_camera body keyboard"
A=($scenes)
DIFF=""

3
.gitignore vendored
View File

@@ -74,9 +74,6 @@ comma*.sh
selfdrive/modeld/thneed/compile
selfdrive/modeld/models/*.thneed
selfdrive/modeld/models/*.pkl
sunnypilot/modeld/thneed/compile
sunnypilot/modeld/models/*.thneed
sunnypilot/modeld/models/*.pkl
*.bz2
*.zst

2
.gitmodules vendored
View File

@@ -15,4 +15,4 @@
url = https://github.com/commaai/teleoprtc
[submodule "tinygrad"]
path = tinygrad_repo
url = https://github.com/commaai/tinygrad.git
url = https://github.com/tinygrad/tinygrad.git

View File

@@ -1,23 +1,16 @@
Version 0.9.9 (2025-03-30)
========================
* Coming soon
* Rivian support
* F-150 & Mach-E support
* Tesla Model 3 support
Version 0.9.8 (2025-01-30)
Version 0.9.8 (2024-XX-XX)
========================
* New driving model
* Trained in brand new ML simulator
* Model now gates applying positive accel in Chill mode
* New driving monitoring model
* Reduced false positives related to passengers
* Image processing pipeline moved to the ISP
* More GPU time for driving models
* Power draw reduced 0.5W, which means your device runs cooler
* Power draw reduced 0.5W, which means your device runs cooler
* Added toggle to enable driver monitoring even when openpilot is not engaged
* Enable openpilot longitudinal control for Ford Q3 vehicles
* New Toyota TSS2 longitudinal tune
* Coming soon
* New driving model with gas gating
* Training data upload mode
Version 0.9.7 (2024-06-13)
========================

View File

@@ -49,6 +49,10 @@ AddOption('--ccflags',
default='',
help='pass arbitrary flags over the command line')
AddOption('--snpe',
action='store_true',
help='use SNPE on PC')
AddOption('--external-sconscript',
action='store',
metavar='FILE',
@@ -70,12 +74,6 @@ AddOption('--minimal',
default=os.path.exists(File('#.lfsconfig').abspath), # minimal by default on release branch (where there's no LFS)
help='the minimum build to run openpilot. no tests, tools, etc.')
AddOption('--stock-ui',
action='store_true',
dest='stock_ui',
default=False,
help='Build stock openpilot UI instead of sunnypilot UI')
## Architecture name breakdown (arch)
## - larch64: linux tici aarch64
## - aarch64: linux pc aarch64
@@ -174,10 +172,6 @@ else:
if arch != "Darwin":
ldflags += ["-Wl,--as-needed", "-Wl,--no-undefined"]
if not GetOption('stock_ui'):
cflags += ["-DSUNNYPILOT"]
cxxflags += ["-DSUNNYPILOT"]
ccflags_option = GetOption('ccflags')
if ccflags_option:
ccflags += ccflags_option.split(' ')
@@ -392,8 +386,6 @@ SConscript(['third_party/SConscript'])
SConscript(['selfdrive/SConscript'])
SConscript(['sunnypilot/SConscript'])
if Dir('#tools/cabana/').exists() and GetOption('extras'):
SConscript(['tools/replay/SConscript'])
if arch != "larch64":

View File

@@ -29,50 +29,6 @@ then means breaking backwards-compatibility with all old logs of your fork. So w
[custom.capnp](custom.capnp) that we will leave empty in mainline cereal/openpilot. **If you only modify those, you can ensure your
fork will remain backwards-compatible with all versions of mainline openpilot and your fork.**
An example of compatible changes:
```diff
diff --git a/cereal/custom.capnp b/cereal/custom.capnp
index 3348e859e..3365c7b98 100644
--- a/cereal/custom.capnp
+++ b/cereal/custom.capnp
@@ -10,7 +10,11 @@ $Cxx.namespace("cereal");
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
-struct CustomReserved0 @0x81c2f05a394cf4af {
+struct SteeringInfo @0x81c2f05a394cf4af {
+ active @0 :Bool;
+ steeringAngleDeg @1 :Float32;
+ steeringRateDeg @2 :Float32;
+ steeringAccelDeg @3 :Float32;
}
struct CustomReserved1 @0xaedffd8f31e7b55d {
diff --git a/cereal/log.capnp b/cereal/log.capnp
index 1209f3fd9..b189f58b6 100644
--- a/cereal/log.capnp
+++ b/cereal/log.capnp
@@ -2558,14 +2558,14 @@ struct Event {
# DO change the name of the field
# DON'T change anything after the "@"
- customReservedRawData0 @124 :Data;
+ rawCanData @124 :Data;
customReservedRawData1 @125 :Data;
customReservedRawData2 @126 :Data;
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
- customReserved0 @107 :Custom.CustomReserved0;
+ steeringInfo @107 :Custom.SteeringInfo;
customReserved1 @108 :Custom.CustomReserved1;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
```
---
Example
---
```python

View File

@@ -7,139 +7,23 @@ $Cxx.namespace("cereal");
# These structs are guaranteed to remain reserved and empty in mainline
# cereal, so use these if you want custom events in your fork.
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
struct ModularAssistiveDrivingSystem {
state @0 :ModularAssistiveDrivingSystemState;
enabled @1 :Bool;
active @2 :Bool;
available @3 :Bool;
enum ModularAssistiveDrivingSystemState {
disabled @0;
paused @1;
enabled @2;
softDisabling @3;
overriding @4;
}
# you can rename the struct, but don't change the identifier
struct CustomReserved0 @0x81c2f05a394cf4af {
}
struct SelfdriveStateSP @0x81c2f05a394cf4af {
mads @0 :ModularAssistiveDrivingSystem;
struct CustomReserved1 @0xaedffd8f31e7b55d {
}
struct ModelManagerSP @0xaedffd8f31e7b55d {
activeBundle @0 :ModelBundle;
selectedBundle @1 :ModelBundle;
availableBundles @2 :List(ModelBundle);
struct DownloadUri {
uri @0 :Text;
sha256 @1 :Text;
}
enum Type {
drive @0;
navigation @1;
metadata @2;
}
struct Model {
fullName @0 :Text;
fileName @1 :Text;
downloadUri @2 :DownloadUri;
downloadProgress @3 :DownloadProgress;
type @4 :Type;
}
enum DownloadStatus {
notDownloading @0;
downloading @1;
downloaded @2;
cached @3;
failed @4;
}
struct DownloadProgress {
status @0 :DownloadStatus;
progress @1 :Float32;
eta @2 :UInt32;
}
enum Runner {
snpe @0;
tinygrad @1;
}
struct ModelBundle {
index @0 :UInt32;
internalName @1 :Text;
displayName @2 :Text;
models @3 :List(Model);
status @4 :DownloadStatus;
generation @5 :UInt32;
environment @6 :Text;
runner @7 :Runner;
is20hz @8 :Bool;
}
struct CustomReserved2 @0xf35cc4560bbf6ec2 {
}
struct LongitudinalPlanSP @0xf35cc4560bbf6ec2 {
dec @0 :DynamicExperimentalControl;
struct DynamicExperimentalControl {
state @0 :DynamicExperimentalControlState;
enabled @1 :Bool;
active @2 :Bool;
enum DynamicExperimentalControlState {
acc @0;
blended @1;
}
}
struct CustomReserved3 @0xda96579883444c35 {
}
struct OnroadEventSP @0xda96579883444c35 {
name @0 :EventName;
# event types
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool; # alerts presented only when enabled or soft disabling
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool; # alerts presented regardless of openpilot state
overrideLateral @10 :Bool;
overrideLongitudinal @9 :Bool;
enum EventName {
lkasEnable @0;
lkasDisable @1;
manualSteeringRequired @2;
manualLongitudinalRequired @3;
silentLkasEnable @4;
silentLkasDisable @5;
silentBrakeHold @6;
silentWrongGear @7;
silentReverseGear @8;
silentDoorOpen @9;
silentSeatbeltNotLatched @10;
silentParkBrake @11;
controlsMismatchLateral @12;
hyundaiRadarTracksConfirmed @13;
experimentalModeSwitched @14;
}
struct CustomReserved4 @0x80ae746ee2596b11 {
}
struct CarParamsSP @0x80ae746ee2596b11 {
flags @0 :UInt32; # flags for car specific quirks in sunnypilot
}
struct CarControlSP @0xa5cd762cd951a455 {
mads @0 :ModularAssistiveDrivingSystem;
struct CustomReserved5 @0xa5cd762cd951a455 {
}
struct CustomReserved6 @0xf98d843bfd7004a3 {
@@ -153,33 +37,3 @@ struct CustomReserved8 @0xf416ec09499d9d19 {
struct CustomReserved9 @0xa1680744031fdb2d {
}
struct CustomReserved10 @0xcb9fd56c7057593a {
}
struct CustomReserved11 @0xc2243c65e0340384 {
}
struct CustomReserved12 @0x9ccdc8676701b412 {
}
struct CustomReserved13 @0xcd96dafb67a082d0 {
}
struct CustomReserved14 @0xb057204d7deadf3f {
}
struct CustomReserved15 @0xbd443b539493bc68 {
}
struct CustomReserved16 @0xfc6241ed8877b611 {
}
struct CustomReserved17 @0xa30662f84033036c {
}
struct CustomReserved18 @0xc86a3d38d13eb3ef {
}
struct CustomReserved19 @0xa4f1eb3323f5f582 {
}

View File

@@ -409,7 +409,6 @@ struct GpsLocationData {
speedAccuracy @12 :Float32;
hasFix @13 :Bool;
satelliteCount @14 :Int8;
enum SensorSource {
android @0;
@@ -2554,37 +2553,21 @@ struct Event {
livestreamWideRoadEncodeData @121 :EncodeData;
livestreamDriverEncodeData @122 :EncodeData;
# *********** Custom: reserved for forks ***********
# DO change the name of the field
# DON'T change anything after the "@"
customReservedRawData0 @124 :Data;
customReservedRawData1 @125 :Data;
customReservedRawData2 @126 :Data;
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
selfdriveStateSP @107 :Custom.SelfdriveStateSP;
modelManagerSP @108 :Custom.ModelManagerSP;
longitudinalPlanSP @109 :Custom.LongitudinalPlanSP;
onroadEventsSP @110 :List(Custom.OnroadEventSP);
carParamsSP @111 :Custom.CarParamsSP;
carControlSP @112 :Custom.CarControlSP;
# *********** Custom: reserved for forks ***********
customReserved0 @107 :Custom.CustomReserved0;
customReserved1 @108 :Custom.CustomReserved1;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
customReserved4 @111 :Custom.CustomReserved4;
customReserved5 @112 :Custom.CustomReserved5;
customReserved6 @113 :Custom.CustomReserved6;
customReserved7 @114 :Custom.CustomReserved7;
customReserved8 @115 :Custom.CustomReserved8;
customReserved9 @116 :Custom.CustomReserved9;
customReserved10 @136 :Custom.CustomReserved10;
customReserved11 @137 :Custom.CustomReserved11;
customReserved12 @138 :Custom.CustomReserved12;
customReserved13 @139 :Custom.CustomReserved13;
customReserved14 @140 :Custom.CustomReserved14;
customReserved15 @141 :Custom.CustomReserved15;
customReserved16 @142 :Custom.CustomReserved16;
customReserved17 @143 :Custom.CustomReserved17;
customReserved18 @144 :Custom.CustomReserved18;
customReserved19 @145 :Custom.CustomReserved19;
# *********** legacy + deprecated ***********
model @9 :Legacy.ModelData; # TODO: rename modelV2 and mark this as deprecated

49
cereal/maptile.capnp Normal file
View File

@@ -0,0 +1,49 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
@0xa086df597ef5d7a0;
# Geometry
struct Point {
x @0: Float64;
y @1: Float64;
z @2: Float64;
}
struct PolyLine {
points @0: List(Point);
}
# Map features
struct Lane {
id @0 :Text;
leftBoundary @1 :LaneBoundary;
rightBoundary @2 :LaneBoundary;
leftAdjacentId @3 :Text;
rightAdjacentId @4 :Text;
inboundIds @5 :List(Text);
outboundIds @6 :List(Text);
struct LaneBoundary {
polyLine @0 :PolyLine;
startHeading @1 :Float32; # WRT north
}
}
# Map tiles
struct TileSummary {
version @0 :Text;
updatedAt @1 :UInt64; # Millis since epoch
level @2 :UInt8;
x @3 :UInt16;
y @4 :UInt16;
}
struct MapTile {
summary @0 :TileSummary;
lanes @1 :List(Lane);
}

View File

@@ -6,12 +6,12 @@
ExitHandler do_exit;
static std::vector<std::string> get_services(const std::string &whitelist_str, bool zmq_to_msgq) {
static std::vector<std::string> get_services(std::string whitelist_str, bool zmq_to_msgq) {
std::vector<std::string> service_list;
for (const auto& it : services) {
std::string name = it.second.name;
bool in_whitelist = whitelist_str.find(name) != std::string::npos;
if (zmq_to_msgq && !in_whitelist) {
if (name == "plusFrame" || name == "uiLayoutState" || (zmq_to_msgq && !in_whitelist)) {
continue;
}
service_list.push_back(name);

View File

@@ -74,14 +74,6 @@ _services: dict[str, tuple] = {
"userFlag": (True, 0., 1),
"microphone": (True, 10., 10),
# sunnypilot
"modelManagerSP": (False, 1., 1),
"selfdriveStateSP": (True, 100., 10),
"longitudinalPlanSP": (True, 20., 10),
"onroadEventsSP": (True, 1., 1),
"carParamsSP": (True, 0.02, 1),
"carControlSP": (True, 100., 10),
# debug
"uiDebug": (True, 0., 1),
"testJoystick": (True, 0.),

View File

@@ -1,27 +1,46 @@
from openpilot.common.api.comma_connect import CommaConnectApi
from sunnypilot.sunnylink.api import SunnylinkApi
import jwt
import os
import requests
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class Api:
def __init__(self, dongle_id, use_sunnylink=False):
if use_sunnylink:
self.service = SunnylinkApi(dongle_id)
else:
self.service = CommaConnectApi(dongle_id)
def request(self, method, endpoint, **params):
return self.service.request(method, endpoint, **params)
def __init__(self, dongle_id):
self.dongle_id = dongle_id
with open(Paths.persist_root()+'/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.service.get(*args, **kwargs)
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.service.post(*args, **kwargs)
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def get_token(self, expiry_hours=1):
return self.service.get_token(expiry_hours)
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours)
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def api_get(endpoint, method='GET', timeout=None, access_token=None, use_sunnylink=False, **params):
return SunnylinkApi(None).api_get(endpoint, method, timeout, access_token, **params) if use_sunnylink \
else CommaConnectApi(None).api_get(endpoint, method, timeout, access_token, **params)
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
headers['User-Agent'] = "openpilot-" + get_version()
return requests.request(method, API_HOST + "/" + endpoint, timeout=timeout, headers=headers, params=params)

View File

@@ -1,56 +0,0 @@
import jwt
import requests
import unicodedata
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
class BaseApi:
def __init__(self, dongle_id, api_host, user_agent="openpilot-"):
self.dongle_id = dongle_id
self.api_host = api_host
self.user_agent = user_agent
with open(f'{Paths.persist_root()}/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return self.api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def _get_token(self, expiry_hours=1, **extra_payload):
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours),
**extra_payload
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def get_token(self, expiry_hours=1):
return self._get_token(expiry_hours)
def remove_non_ascii_chars(self, text):
normalized_text = unicodedata.normalize('NFD', text)
ascii_encoded_text = normalized_text.encode('ascii', 'ignore')
return ascii_encoded_text.decode()
def api_get(self, endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
version = self.remove_non_ascii_chars(get_version())
headers['User-Agent'] = self.user_agent + version
return requests.request(method, f"{self.api_host}/{endpoint}", timeout=timeout, headers=headers, params=params)

View File

@@ -1,11 +0,0 @@
import os
from openpilot.common.api.base import BaseApi
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class CommaConnectApi(BaseApi):
def __init__(self, dongle_id):
super().__init__(dongle_id, API_HOST)
self.user_agent = "openpilot-"

View File

@@ -79,10 +79,6 @@ cl_context cl_create_context(cl_device_id device_id) {
return CL_CHECK_ERR(clCreateContext(NULL, 1, &device_id, NULL, NULL, &err));
}
void cl_release_context(cl_context context) {
clReleaseContext(context);
}
cl_program cl_program_from_file(cl_context ctx, cl_device_id device_id, const char* path, const char* args) {
return cl_program_from_source(ctx, device_id, util::read_file(path), args);
}

View File

@@ -23,7 +23,6 @@
cl_device_id cl_get_device_id(cl_device_type device_type);
cl_context cl_create_context(cl_device_id device_id);
void cl_release_context(cl_context context);
cl_program cl_program_from_source(cl_context ctx, cl_device_id device_id, const std::string& src, const char* args = nullptr);
cl_program cl_program_from_binary(cl_context ctx, cl_device_id device_id, const uint8_t* binary, size_t length, const char* args = nullptr);
cl_program cl_program_from_file(cl_context ctx, cl_device_id device_id, const char* path, const char* args);

View File

@@ -8,7 +8,6 @@ import uuid
import socket
import logging
import traceback
import numpy as np
from threading import local
from collections import OrderedDict
from contextlib import contextmanager
@@ -16,8 +15,6 @@ from contextlib import contextmanager
LOG_TIMESTAMPS = "LOG_TIMESTAMPS" in os.environ
def json_handler(obj):
if isinstance(obj, np.bool_):
return bool(obj)
# if isinstance(obj, (datetime.date, datetime.time)):
# return obj.isoformat()
return repr(obj)

View File

@@ -1 +0,0 @@
#define DEFAULT_MODEL "Notre Dame (Default)"

View File

@@ -89,7 +89,6 @@ private:
std::unordered_map<std::string, uint32_t> keys = {
{"AccessToken", CLEAR_ON_MANAGER_START | DONT_LOG},
{"AdbEnabled", PERSISTENT},
{"AlwaysOnDM", PERSISTENT},
{"ApiCache_Device", PERSISTENT},
{"AssistNowToken", PERSISTENT},
@@ -110,35 +109,36 @@ std::unordered_map<std::string, uint32_t> keys = {
{"CurrentBootlog", PERSISTENT},
{"CurrentRoute", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisableLogging", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"DisablePowerDown", PERSISTENT | BACKUP},
{"DisableUpdates", PERSISTENT | BACKUP},
{"DisengageOnAccelerator", PERSISTENT | BACKUP},
{"DisablePowerDown", PERSISTENT},
{"DisableUpdates", PERSISTENT},
{"DisengageOnAccelerator", PERSISTENT},
{"DongleId", PERSISTENT},
{"DoReboot", CLEAR_ON_MANAGER_START},
{"DoShutdown", CLEAR_ON_MANAGER_START},
{"DoUninstall", CLEAR_ON_MANAGER_START},
{"ExperimentalLongitudinalEnabled", PERSISTENT | DEVELOPMENT_ONLY | BACKUP},
{"ExperimentalMode", PERSISTENT | BACKUP},
{"ExperimentalModeConfirmed", PERSISTENT | BACKUP},
{"ExperimentalLongitudinalEnabled", PERSISTENT | DEVELOPMENT_ONLY},
{"ExperimentalMode", PERSISTENT},
{"ExperimentalModeConfirmed", PERSISTENT},
{"FirmwareQueryDone", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ForcePowerDown", PERSISTENT},
{"GitBranch", PERSISTENT},
{"GitCommit", PERSISTENT},
{"GitCommitDate", PERSISTENT},
{"GitDiff", PERSISTENT},
{"GithubSshKeys", PERSISTENT | BACKUP},
{"GithubUsername", PERSISTENT | BACKUP},
{"GithubSshKeys", PERSISTENT},
{"GithubUsername", PERSISTENT},
{"GitRemote", PERSISTENT},
{"GsmApn", PERSISTENT | BACKUP},
{"GsmMetered", PERSISTENT | BACKUP},
{"GsmRoaming", PERSISTENT | BACKUP},
{"GsmApn", PERSISTENT},
{"GsmMetered", PERSISTENT},
{"GsmRoaming", PERSISTENT},
{"HardwareSerial", PERSISTENT},
{"HasAcceptedTerms", PERSISTENT},
{"IMEI", PERSISTENT},
{"InstallDate", PERSISTENT},
{"IsDriverViewEnabled", CLEAR_ON_MANAGER_START},
{"IsEngaged", PERSISTENT},
{"IsLdwEnabled", PERSISTENT | BACKUP},
{"IsMetric", PERSISTENT | BACKUP},
{"IsLdwEnabled", PERSISTENT},
{"IsMetric", PERSISTENT},
{"IsOffroad", CLEAR_ON_MANAGER_START},
{"IsOnroad", PERSISTENT},
{"IsRhdDetected", PERSISTENT},
@@ -146,7 +146,7 @@ std::unordered_map<std::string, uint32_t> keys = {
{"IsTakingSnapshot", CLEAR_ON_MANAGER_START},
{"IsTestedBranch", CLEAR_ON_MANAGER_START},
{"JoystickDebugMode", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LanguageSetting", PERSISTENT | BACKUP},
{"LanguageSetting", PERSISTENT},
{"LastAthenaPingTime", CLEAR_ON_MANAGER_START},
{"LastGPSPosition", PERSISTENT},
{"LastManagerExitReason", CLEAR_ON_MANAGER_START},
@@ -158,7 +158,7 @@ std::unordered_map<std::string, uint32_t> keys = {
{"LiveTorqueParameters", PERSISTENT | DONT_LOG},
{"LocationFilterInitialState", PERSISTENT},
{"LongitudinalManeuverMode", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"LongitudinalPersonality", PERSISTENT | BACKUP},
{"LongitudinalPersonality", PERSISTENT},
{"NetworkMetered", PERSISTENT},
{"ObdMultiplexingChanged", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ObdMultiplexingEnabled", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
@@ -174,17 +174,17 @@ std::unordered_map<std::string, uint32_t> keys = {
{"Offroad_TemperatureTooHigh", CLEAR_ON_MANAGER_START},
{"Offroad_UnofficialHardware", CLEAR_ON_MANAGER_START},
{"Offroad_UpdateFailed", CLEAR_ON_MANAGER_START},
{"OpenpilotEnabledToggle", PERSISTENT | BACKUP},
{"OpenpilotEnabledToggle", PERSISTENT},
{"PandaHeartbeatLost", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSomResetTriggered", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"PandaSignatures", CLEAR_ON_MANAGER_START},
{"PrimeType", PERSISTENT},
{"RecordFront", PERSISTENT | BACKUP},
{"RecordFront", PERSISTENT},
{"RecordFrontLock", PERSISTENT}, // for the internal fleet
{"SecOCKey", PERSISTENT | DONT_LOG}, // Candidate for | BACKUP
{"SecOCKey", PERSISTENT | DONT_LOG},
{"RouteCount", PERSISTENT},
{"SnoozeUpdate", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"SshEnabled", PERSISTENT | BACKUP},
{"SshEnabled", PERSISTENT},
{"TermsVersion", PERSISTENT},
{"TrainingVersion", PERSISTENT},
{"UbloxAvailable", PERSISTENT},
@@ -200,44 +200,7 @@ std::unordered_map<std::string, uint32_t> keys = {
{"UpdaterTargetBranch", CLEAR_ON_MANAGER_START},
{"UpdaterLastFetchTime", PERSISTENT},
{"Version", PERSISTENT},
// --- sunnypilot params --- //
{"ApiCache_DriveStats", PERSISTENT},
{"CarParamsSP", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"CarParamsSPCache", CLEAR_ON_MANAGER_START},
{"CarParamsSPPersistent", PERSISTENT},
{"CarPlatformBundle", PERSISTENT},
{"EnableGithubRunner", PERSISTENT | BACKUP},
{"ModelRunnerTypeCache", CLEAR_ON_ONROAD_TRANSITION},
{"OffroadMode", CLEAR_ON_MANAGER_START},
{"OffroadMode_Status", CLEAR_ON_MANAGER_START},
// MADS params
{"Mads", PERSISTENT | BACKUP},
{"MadsMainCruiseAllowed", PERSISTENT | BACKUP},
{"MadsPauseLateralOnBrake", PERSISTENT | BACKUP},
{"MadsUnifiedEngagementMode", PERSISTENT | BACKUP},
// Model Manager params
{"ModelManager_ActiveBundle", PERSISTENT},
{"ModelManager_DownloadIndex", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION},
{"ModelManager_LastSyncTime", CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION},
{"ModelManager_ModelsCache", PERSISTENT | BACKUP},
// sunnylink params
{"EnableSunnylinkUploader", PERSISTENT | BACKUP},
{"LastSunnylinkPingTime", CLEAR_ON_MANAGER_START},
{"SunnylinkDongleId", PERSISTENT},
{"SunnylinkdPid", PERSISTENT},
{"SunnylinkEnabled", PERSISTENT},
// sunnypilot car specific params
{"HyundaiRadarTracks", PERSISTENT},
{"HyundaiRadarTracksConfirmed", PERSISTENT},
{"HyundaiRadarTracksPersistent", PERSISTENT},
{"HyundaiRadarTracksToggle", PERSISTENT},
{"DynamicExperimentalControl", PERSISTENT},
{"EnableGithubRunner", PERSISTENT},
};
} // namespace

View File

@@ -16,7 +16,6 @@ enum ParamKeyType {
CLEAR_ON_OFFROAD_TRANSITION = 0x10,
DONT_LOG = 0x20,
DEVELOPMENT_ONLY = 0x40,
BACKUP = 0x80,
ALL = 0xFFFFFFFF
};

View File

@@ -1,6 +1,9 @@
import numpy as np
from numbers import Number
from openpilot.common.numpy_fast import clip, interp
class PIDController:
def __init__(self, k_p, k_i, k_f=0., k_d=0., pos_limit=1e308, neg_limit=-1e308, rate=100):
self._k_p = k_p
@@ -25,15 +28,15 @@ class PIDController:
@property
def k_p(self):
return np.interp(self.speed, self._k_p[0], self._k_p[1])
return interp(self.speed, self._k_p[0], self._k_p[1])
@property
def k_i(self):
return np.interp(self.speed, self._k_i[0], self._k_i[1])
return interp(self.speed, self._k_i[0], self._k_i[1])
@property
def k_d(self):
return np.interp(self.speed, self._k_d[0], self._k_d[1])
return interp(self.speed, self._k_d[0], self._k_d[1])
@property
def error_integral(self):
@@ -61,10 +64,10 @@ class PIDController:
# Clip i to prevent exceeding control limits
control_no_i = self.p + self.d + self.f
control_no_i = np.clip(control_no_i, self.neg_limit, self.pos_limit)
self.i = np.clip(self.i, self.neg_limit - control_no_i, self.pos_limit - control_no_i)
control_no_i = clip(control_no_i, self.neg_limit, self.pos_limit)
self.i = clip(self.i, self.neg_limit - control_no_i, self.pos_limit - control_no_i)
control = self.p + self.i + self.d + self.f
self.control = np.clip(control, self.neg_limit, self.pos_limit)
self.control = clip(control, self.neg_limit, self.pos_limit)
return self.control

View File

@@ -26,9 +26,6 @@ public:
zmq_setsockopt(sock, ZMQ_LINGER, &timeout, sizeof(timeout));
zmq_connect(sock, Path::swaglog_ipc().c_str());
// workaround for https://github.com/dropbox/json11/issues/38
setlocale(LC_NUMERIC, "C");
print_level = CLOUDLOG_WARNING;
if (const char* print_lvl = getenv("LOGPRINT")) {
if (strcmp(print_lvl, "debug") == 0) {

View File

@@ -0,0 +1,21 @@
import numpy as np
from openpilot.common.numpy_fast import interp
class TestInterp:
def test_correctness_controls(self):
_A_CRUISE_MIN_BP = np.asarray([0., 5., 10., 20., 40.])
_A_CRUISE_MIN_V = np.asarray([-1.0, -.8, -.67, -.5, -.30])
v_ego_arr = [-1, -1e-12, 0, 4, 5, 6, 7, 10, 11, 15.2, 20, 21, 39,
39.999999, 40, 41]
expected = np.interp(v_ego_arr, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
actual = interp(v_ego_arr, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
np.testing.assert_equal(actual, expected)
for v_ego in v_ego_arr:
expected = np.interp(v_ego, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
actual = interp(v_ego, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
np.testing.assert_equal(actual, expected)

View File

@@ -255,29 +255,6 @@ bool ends_with(const std::string& s, const std::string& suffix) {
strcmp(s.c_str() + (s.size() - suffix.size()), suffix.c_str()) == 0;
}
std::string strip(const std::string &str) {
auto should_trim = [](unsigned char ch) {
// trim whitespace or a null character
return std::isspace(ch) || ch == '\0';
};
size_t start = 0;
while (start < str.size() && should_trim(static_cast<unsigned char>(str[start]))) {
start++;
}
if (start == str.size()) {
return "";
}
size_t end = str.size() - 1;
while (end > 0 && should_trim(static_cast<unsigned char>(str[end]))) {
end--;
}
return str.substr(start, end - start + 1);
}
std::string check_output(const std::string& command) {
char buffer[128];
std::string result;
@@ -294,17 +271,4 @@ std::string check_output(const std::string& command) {
return result;
}
bool system_time_valid() {
// Default to August 26, 2024
tm min_tm = {.tm_year = 2024 - 1900, .tm_mon = 7, .tm_mday = 26};
time_t min_date = mktime(&min_tm);
struct stat st;
if (stat("/lib/systemd/systemd", &st) == 0) {
min_date = std::max(min_date, st.st_mtime + 86400); // Add 1 day (86400 seconds)
}
return time(nullptr) > min_date;
}
} // namespace util

View File

@@ -74,7 +74,6 @@ float getenv(const char* key, float default_val);
std::string hexdump(const uint8_t* in, const size_t size);
bool starts_with(const std::string &s1, const std::string &s2);
bool ends_with(const std::string &s, const std::string &suffix);
std::string strip(const std::string &str);
// ***** random helpers *****
int random_int(int min, int max);
@@ -96,8 +95,6 @@ bool create_directories(const std::string &dir, mode_t mode);
std::string check_output(const std::string& command);
bool system_time_valid();
inline void sleep_for(const int milliseconds) {
if (milliseconds > 0) {
std::this_thread::sleep_for(std::chrono::milliseconds(milliseconds));

View File

@@ -4,13 +4,13 @@
A supported vehicle is one that just works when you install a comma device. All supported cars provide a better experience than any stock system. Supported vehicles reference the US market unless otherwise specified.
# 290 Supported Cars
# 289 Supported Cars
|Make|Model|Supported Package|ACC|No ACC accel below|No ALC below|Steering Torque|Resume from stop|<a href="##"><img width=2000></a>Hardware Needed<br>&nbsp;|Video|
|---|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|Acura|ILX 2016-19|AcuraWatch Plus|openpilot|26 mph|25 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Acura&model=ILX 2016-19">Buy Here</a></sub></details>||
|Acura|RDX 2016-18|AcuraWatch Plus|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Acura&model=RDX 2016-18">Buy Here</a></sub></details>||
|Acura|RDX 2019-21|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|3 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch A connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Acura&model=RDX 2019-21">Buy Here</a></sub></details>||
|Acura|RDX 2019-22|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|3 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Bosch A connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Acura&model=RDX 2019-22">Buy Here</a></sub></details>||
|Audi|A3 2014-19|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Audi&model=A3 2014-19">Buy Here</a></sub></details>||
|Audi|A3 Sportback e-tron 2017-18|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Audi&model=A3 Sportback e-tron 2017-18">Buy Here</a></sub></details>||
|Audi|Q2 2018|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Audi&model=Q2 2018">Buy Here</a></sub></details>||
@@ -56,7 +56,6 @@ A supported vehicle is one that just works when you install a comma device. All
|Genesis|GV60 (Performance Trim) 2022-23[<sup>5</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV60 (Performance Trim) 2022-23">Buy Here</a></sub></details>||
|Genesis|GV70 (2.5T Trim, without HDA II) 2022-23[<sup>5</sup>](#footnotes)|All|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai L connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV70 (2.5T Trim, without HDA II) 2022-23">Buy Here</a></sub></details>||
|Genesis|GV70 (3.5T Trim, without HDA II) 2022-23[<sup>5</sup>](#footnotes)|All|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai M connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV70 (3.5T Trim, without HDA II) 2022-23">Buy Here</a></sub></details>||
|Genesis|GV70 Electrified (Australia Only) 2022[<sup>5</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai Q connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV70 Electrified (Australia Only) 2022">Buy Here</a></sub></details>||
|Genesis|GV70 Electrified (with HDA II) 2023[<sup>5</sup>](#footnotes)|Highway Driving Assist II|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai Q connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV70 Electrified (with HDA II) 2023">Buy Here</a></sub></details>||
|Genesis|GV80 2023[<sup>5</sup>](#footnotes)|All|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai M connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Genesis&model=GV80 2023">Buy Here</a></sub></details>||
|GMC|Sierra 1500 2020-21|Driver Alert Package II|openpilot available[<sup>1</sup>](#footnotes)|0 mph|6 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 GM connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=GMC&model=Sierra 1500 2020-21">Buy Here</a></sub></details>|<a href="https://youtu.be/5HbNoBLzRwE" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
@@ -80,7 +79,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Honda|Odyssey 2018-20|Honda Sensing|openpilot|26 mph|0 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Honda&model=Odyssey 2018-20">Buy Here</a></sub></details>||
|Honda|Passport 2019-23|All|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Honda&model=Passport 2019-23">Buy Here</a></sub></details>||
|Honda|Pilot 2016-22|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Honda&model=Pilot 2016-22">Buy Here</a></sub></details>||
|Honda|Ridgeline 2017-25|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Honda&model=Ridgeline 2017-25">Buy Here</a></sub></details>||
|Honda|Ridgeline 2017-24|Honda Sensing|openpilot|26 mph|12 mph|[![star](assets/icon-star-empty.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Honda Nidec connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Honda&model=Ridgeline 2017-24">Buy Here</a></sub></details>||
|Hyundai|Azera 2022|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Azera 2022">Buy Here</a></sub></details>||
|Hyundai|Azera Hybrid 2019|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai C connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Azera Hybrid 2019">Buy Here</a></sub></details>||
|Hyundai|Azera Hybrid 2020|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Azera Hybrid 2020">Buy Here</a></sub></details>||
@@ -88,11 +87,11 @@ A supported vehicle is one that just works when you install a comma device. All
|Hyundai|Elantra 2017-18|Smart Cruise Control (SCC)|Stock|19 mph|32 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai B connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra 2017-18">Buy Here</a></sub></details>||
|Hyundai|Elantra 2019|Smart Cruise Control (SCC)|Stock|19 mph|32 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai G connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra 2019">Buy Here</a></sub></details>||
|Hyundai|Elantra 2021-23|Smart Cruise Control (SCC)|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra 2021-23">Buy Here</a></sub></details>|<a href="https://youtu.be/_EdYQtV52-c" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Hyundai|Elantra GT 2017-20|Smart Cruise Control (SCC)|Stock|0 mph|32 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai E connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra GT 2017-20">Buy Here</a></sub></details>||
|Hyundai|Elantra GT 2017-19|Smart Cruise Control (SCC)|Stock|0 mph|32 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai E connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra GT 2017-19">Buy Here</a></sub></details>||
|Hyundai|Elantra Hybrid 2021-23|Smart Cruise Control (SCC)|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Elantra Hybrid 2021-23">Buy Here</a></sub></details>|<a href="https://youtu.be/_EdYQtV52-c" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Hyundai|Genesis 2015-16|Smart Cruise Control (SCC)|Stock|19 mph|37 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai J connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Genesis 2015-16">Buy Here</a></sub></details>||
|Hyundai|i30 2017-19|Smart Cruise Control (SCC)|Stock|0 mph|32 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai E connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=i30 2017-19">Buy Here</a></sub></details>||
|Hyundai|Ioniq 5 (Southeast Asia and Europe only) 2022-24[<sup>5</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai Q connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Ioniq 5 (Southeast Asia and Europe only) 2022-24">Buy Here</a></sub></details>||
|Hyundai|Ioniq 5 (Non-US only) 2022-24[<sup>5</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai Q connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Ioniq 5 (Non-US only) 2022-24">Buy Here</a></sub></details>||
|Hyundai|Ioniq 5 (with HDA II) 2022-24[<sup>5</sup>](#footnotes)|Highway Driving Assist II|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai Q connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Ioniq 5 (with HDA II) 2022-24">Buy Here</a></sub></details>||
|Hyundai|Ioniq 5 (without HDA II) 2022-24[<sup>5</sup>](#footnotes)|Highway Driving Assist|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Ioniq 5 (without HDA II) 2022-24">Buy Here</a></sub></details>||
|Hyundai|Ioniq 6 (with HDA II) 2023-24[<sup>5</sup>](#footnotes)|Highway Driving Assist II|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai P connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Hyundai&model=Ioniq 6 (with HDA II) 2023-24">Buy Here</a></sub></details>||
@@ -128,7 +127,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Jeep|Grand Cherokee 2019-21|Adaptive Cruise Control (ACC)|Stock|0 mph|39 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 FCA connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Jeep&model=Grand Cherokee 2019-21">Buy Here</a></sub></details>|<a href="https://www.youtube.com/watch?v=jBe4lWnRSu4" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Kia|Carnival 2022-24[<sup>5</sup>](#footnotes)|Smart Cruise Control (SCC)|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai A connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=Carnival 2022-24">Buy Here</a></sub></details>||
|Kia|Carnival (China only) 2023[<sup>5</sup>](#footnotes)|Smart Cruise Control (SCC)|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai K connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=Carnival (China only) 2023">Buy Here</a></sub></details>||
|Kia|Ceed 2019-21|Smart Cruise Control (SCC)|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai E connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=Ceed 2019-21">Buy Here</a></sub></details>||
|Kia|Ceed 2019|Smart Cruise Control (SCC)|Stock|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai E connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=Ceed 2019">Buy Here</a></sub></details>||
|Kia|EV6 (Southeast Asia only) 2022-24[<sup>5</sup>](#footnotes)|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai P connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=EV6 (Southeast Asia only) 2022-24">Buy Here</a></sub></details>||
|Kia|EV6 (with HDA II) 2022-24[<sup>5</sup>](#footnotes)|Highway Driving Assist II|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai P connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=EV6 (with HDA II) 2022-24">Buy Here</a></sub></details>||
|Kia|EV6 (without HDA II) 2022-24[<sup>5</sup>](#footnotes)|Highway Driving Assist|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 Hyundai L connector<br>- 1 RJ45 cable (7 ft)<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Kia&model=EV6 (without HDA II) 2022-24">Buy Here</a></sub></details>||
@@ -260,7 +259,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Toyota|RAV4 Hybrid 2017-18|All|openpilot available[<sup>2</sup>](#footnotes)|19 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=RAV4 Hybrid 2017-18">Buy Here</a></sub></details>|<a href="https://youtu.be/LhT5VzJVfNI?t=26" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Toyota|RAV4 Hybrid 2019-21|All|openpilot|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=RAV4 Hybrid 2019-21">Buy Here</a></sub></details>||
|Toyota|RAV4 Hybrid 2022|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=RAV4 Hybrid 2022">Buy Here</a></sub></details>|<a href="https://youtu.be/U0nH9cnrFB0" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Toyota|RAV4 Hybrid 2023-25|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=RAV4 Hybrid 2023-25">Buy Here</a></sub></details>|<a href="https://youtu.be/4eIsEq4L4Ng" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Toyota|RAV4 Hybrid 2023-24|All|openpilot available[<sup>1</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=RAV4 Hybrid 2023-24">Buy Here</a></sub></details>||
|Toyota|Sienna 2018-20|All|openpilot available[<sup>2</sup>](#footnotes)|19 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-empty.svg)](##)|<details><summary>Parts</summary><sub>- 1 RJ45 cable (7 ft)<br>- 1 Toyota A connector<br>- 1 comma 3X<br>- 1 comma power v2<br>- 1 harness box<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Toyota&model=Sienna 2018-20">Buy Here</a></sub></details>|<a href="https://www.youtube.com/watch?v=q1UPOo4Sh68" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Volkswagen|Arteon 2018-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Arteon 2018-23">Buy Here</a></sub></details>|<a href="https://youtu.be/FAomFKPFlDA" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
|Volkswagen|Arteon eHybrid 2020-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Arteon eHybrid 2020-23">Buy Here</a></sub></details>|<a href="https://youtu.be/FAomFKPFlDA" target="_blank"><img height="18px" src="assets/icon-youtube.svg"></img></a>|
@@ -295,7 +294,7 @@ A supported vehicle is one that just works when you install a comma device. All
|Volkswagen|Teramont 2018-22|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Teramont 2018-22">Buy Here</a></sub></details>||
|Volkswagen|Teramont Cross Sport 2021-22|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Teramont Cross Sport 2021-22">Buy Here</a></sub></details>||
|Volkswagen|Teramont X 2021-22|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Teramont X 2021-22">Buy Here</a></sub></details>||
|Volkswagen|Tiguan 2018-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Tiguan 2018-23">Buy Here</a></sub></details>||
|Volkswagen|Tiguan 2018-24|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Tiguan 2018-24">Buy Here</a></sub></details>||
|Volkswagen|Tiguan eHybrid 2021-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Tiguan eHybrid 2021-23">Buy Here</a></sub></details>||
|Volkswagen|Touran 2016-23|Adaptive Cruise Control (ACC) & Lane Assist|openpilot available[<sup>1,12</sup>](#footnotes)|0 mph|0 mph|[![star](assets/icon-star-full.svg)](##)|[![star](assets/icon-star-full.svg)](##)|<details><summary>Parts</summary><sub>- 1 USB-C coupler<br>- 1 VW J533 connector<br>- 1 comma 3X<br>- 1 harness box<br>- 1 long OBD-C cable<br>- 1 mount<br>- 1 right angle OBD-C cable (1.5 ft)<br><a href="https://comma.ai/shop/comma-3x.html?make=Volkswagen&model=Touran 2016-23">Buy Here</a></sub></details>||

View File

@@ -63,12 +63,3 @@ A good pull request has all of the following:
* Connect your device to Wi-Fi regularly, so that we can pull data for training better driving models.
* Run the `nightly` branch and report issues. This branch is like `master` but it's built just like a release.
* Annotate images in the [comma10k dataset](https://github.com/commaai/comma10k).
## Contributing Training Data
### A guide for forks
In order for your fork's data to be eligible for the training set:
* **Your cereal messaging structs must be [compatible](../cereal#custom-forks)**
* **The definitions of all the stock messaging structs must not change**: Do not change how any of the fields are set, including everything from `selfdriveState.enabled` to `carState.steeringAngleDeg`. Instead, create your own structs and set them however you'd like.
* **Do not include cars that are not supported in upstream platforms**: Instead, create new opendbc platforms for cars that you'd like to support outside of upstream, even if it's just a trim-level difference.

View File

@@ -29,22 +29,6 @@ Here's an example command for connecting to your device using its tethered conne
For doing development work on device, it's recommended to use [SSH agent forwarding](https://docs.github.com/en/developers/overview/using-ssh-agent-forwarding).
## ADB
In order to use ADB on your device, you'll need to enable it in the device's settings.
* Enable ADB in your device's settings
* Connect to your device
* `adb shell` over USB
* `adb connect` over WiFi
* Here's an example command for connecting to your device using its tethered connection: `adb connect 192.168.43.1:5555`
> [!NOTE]
> The default port for ADB is 5555 on the comma 3/3X.
For more info on ADB, see the [Android Debug Bridge (ADB) documentation](https://developer.android.com/tools/adb).
### Notes
The public keys are only fetched from your GitHub account once. In order to update your device's authorized keys, you'll need to re-enter your GitHub username.

View File

@@ -7,7 +7,7 @@ export OPENBLAS_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
if [ -z "$AGNOS_VERSION" ]; then
export AGNOS_VERSION="11.6"
export AGNOS_VERSION="11.4"
fi
export STAGING_ROOT="/data/safe_staging"

2
panda

Submodule panda updated: 4ca963345a...0d4b79a3c7

View File

@@ -103,6 +103,7 @@ dev = [
"lru-dict",
"matplotlib",
"parameterized >=0.8, <0.9",
#"pprofile",
"pyautogui",
"pyopencl; platform_machine != 'aarch64'", # broken on arm64
"pytools < 2024.1.11; platform_machine != 'aarch64'", # pyopencl use a broken version
@@ -111,10 +112,13 @@ dev = [
"tabulate",
"types-requests",
"types-tabulate",
# this is only pinned since 5.15.11 is broken
"pyqt5 ==5.15.2; platform_machine == 'x86_64'", # no aarch64 wheels for macOS/linux
]
tools = [
"metadrive-simulator @ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal-0.4.2.4/metadrive_simulator-0.4.2.4-py3-none-any.whl ; (platform_machine != 'aarch64')",
"metadrive-simulator @ https://github.com/commaai/metadrive/releases/download/MetaDrive-minimal/metadrive_simulator-0.4.2.3-py3-none-any.whl ; (platform_machine != 'aarch64')",
"rerun-sdk >= 0.18",
]
@@ -159,13 +163,12 @@ testpaths = [
"tools/replay",
"tools/cabana",
"cereal/messaging/tests",
"sunnypilot",
]
[tool.codespell]
quiet-level = 3
# if you've got a short variable name that's getting flagged, add it here
ignore-words-list = "bu,ro,te,ue,alo,hda,ois,nam,nams,ned,som,parm,setts,inout,warmup,bumb,nd,sie,preints,whit,indexIn,ws,uint,grey,deque,stdio,amin,BA,LITE,atEnd,UIs,errorString,arange,FocusIn,od,tim,relA,hist,copyable,jupyter,thead,TGE,abl"
ignore-words-list = "bu,ro,te,ue,alo,hda,ois,nam,nams,ned,som,parm,setts,inout,warmup,bumb,nd,sie,preints,whit,indexIn,ws,uint,grey,deque,stdio,amin,BA,LITE,atEnd,UIs,errorString,arange,FocusIn,od,tim,relA,hist,copyable,jupyter,thead"
builtin = "clear,rare,informal,code,names,en-GB_to_en-US"
skip = "./third_party/*, ./tinygrad/*, ./tinygrad_repo/*, ./msgq/*, ./panda/*, ./opendbc/*, ./opendbc_repo/*, ./rednose/*, ./rednose_repo/*, ./teleoprtc/*, ./teleoprtc_repo/*, *.ts, uv.lock, *.onnx, ./cereal/gen/*, */c_generated_code/*"

View File

@@ -4,7 +4,6 @@ set -e
# Default values
DEFAULT_REPO_URL="https://github.com/sunnypilot"
START_AT_BOOT=false
RESTORE_MODE=false
# Parse command line arguments
while [[ $# -gt 0 ]]; do
@@ -21,10 +20,6 @@ while [[ $# -gt 0 ]]; do
REPO_URL="$2"
shift 2
;;
--restore)
RESTORE_MODE=true
shift
;;
*)
if [ -z "$GITHUB_TOKEN" ]; then
GITHUB_TOKEN="$1"
@@ -36,6 +31,19 @@ while [[ $# -gt 0 ]]; do
esac
done
# Check required arguments
if [ -z "$GITHUB_TOKEN" ]; then
echo "Usage: $0 [--start-at-boot] [--token <github_token>] [--repo <repository_url>]"
echo "Required argument: github_token"
echo "Optional arguments:"
echo " --start-at-boot Enable auto-start at boot (default: false)"
echo " --repo Repository URL (default: ${DEFAULT_REPO_URL})"
exit 1
fi
# Set repository URL if not provided
REPO_URL="${REPO_URL:-$DEFAULT_REPO_URL}"
# Determine BASE_DIR based on mount point
if mountpoint -q /data/media; then
BASE_DIR="/data/media/0/github"
@@ -52,19 +60,20 @@ LOGS_DIR="${BASE_DIR}/logs"
CACHE_DIR="${BASE_DIR}/cache"
OPENPILOT_DIR="${BASE_DIR}/openpilot"
# Basic utility functions (no dependencies)
remount_rw() {
sudo mount -o remount,rw /
create_directories() {
sudo mkdir -p "$RUNNER_DIR" "$BUILDS_DIR" "$LOGS_DIR" "$CACHE_DIR" "$OPENPILOT_DIR"
mkdir -p "/data/openpilot"
sudo chown -R comma:comma "/data/openpilot"
}
remount_ro() {
sync || true # Try to sync but continue even if it fails
sudo mount -o remount,ro / # Always try to remount as read-only
download_and_setup_runner() {
cd "$RUNNER_DIR"
curl -o actions-runner-linux-arm64-2.321.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-arm64-2.321.0.tar.gz
tar xzf ./actions-runner-linux-arm64-2.321.0.tar.gz
rm ./actions-runner-linux-arm64-2.321.0.tar.gz
chmod +x ./config.sh
}
# Always ensure we try to remount as read-only on exit
trap remount_ro EXIT
setup_runner_user() {
sudo useradd --comment 'GitHub Runner' --create-home --home-dir ${BASE_DIR} ${RUNNER_USER} --shell /bin/bash -G ${USER_GROUPS} || sudo usermod -aG ${USER_GROUPS} ${RUNNER_USER}
export BASE_DIR
@@ -75,45 +84,18 @@ create_sudoers_entry() {
sudo grep -qxF "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" /etc/sudoers || echo "${RUNNER_USER} ALL=(ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers
}
configure_runner() {
cd "$RUNNER_DIR"
sudo -u ${RUNNER_USER} ./config.sh --url "$REPO_URL" --token "$GITHUB_TOKEN" --name $(hostname) --runnergroup "tici-tizi" --labels "tici" --work "$BUILDS_DIR" --unattended
}
set_directory_permissions() {
sudo chown -R ${RUNNER_USER}:comma "$BASE_DIR"
sudo chmod g+rwx "$BASE_DIR"
sudo chmod g+s "$BASE_DIR"
}
setup_directories() {
echo "Creating necessary directories..."
sudo mkdir -p "$RUNNER_DIR" "$BUILDS_DIR" "$LOGS_DIR" "$CACHE_DIR" "$OPENPILOT_DIR"
mkdir -p "/data/openpilot"
sudo chown -R comma:comma "/data/openpilot"
}
# System configuration functions (depends on basic utility functions)
setup_system_configs() {
echo "Setting up system configurations..."
setup_runner_user
create_sudoers_entry
set_directory_permissions
}
# Runner setup functions
install_runner() {
echo "Downloading and setting up runner..."
cd "$RUNNER_DIR"
curl -o actions-runner-linux-arm64-2.321.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-arm64-2.321.0.tar.gz
tar xzf ./actions-runner-linux-arm64-2.321.0.tar.gz
rm ./actions-runner-linux-arm64-2.321.0.tar.gz
chmod +x ./config.sh
}
configure_runner() {
echo "Configuring runner..."
cd "$RUNNER_DIR"
sudo -u ${RUNNER_USER} ./config.sh --url "$REPO_URL" --token "$GITHUB_TOKEN" --name $(hostname) --runnergroup "tici-tizi" --labels "tici" --work "$BUILDS_DIR" --unattended
}
create_service_template() {
echo "Creating service template..."
modify_service_template() {
cat <<EOL > "$RUNNER_DIR/bin/actions.runner.service.template"
[Unit]
Description={{Description}}
@@ -138,116 +120,28 @@ WantedBy=multi-user.target
EOL
}
install_service() {
echo "Installing systemd service..."
cd "$RUNNER_DIR"
sudo ./svc.sh install $RUNNER_USER
# Make filesystem writable
sudo mount -o remount,rw /
if [ "$START_AT_BOOT" = false ]; then
local service_name
if [ -f "${RUNNER_DIR}/.service" ]; then
service_name=$(cat "${RUNNER_DIR}/.service")
else
service_name="actions.runner.sunnypilot.$(uname -n)"
fi
sudo systemctl disable "${service_name}"
fi
}
# Ensure filesystem is remounted as read-only on script exit
trap "sudo mount -o remount,ro /" EXIT
check_restore_prerequisites() {
local needs_restore=false
local can_restore=false
local service_name=""
# Execute installation steps
setup_runner_user
create_sudoers_entry
create_directories
download_and_setup_runner
modify_service_template
configure_runner
set_directory_permissions
# Check if base runner directory exists
if [ ! -d "${RUNNER_DIR}" ]; then
echo "ERROR: Runner directory ${RUNNER_DIR} does not exist"
echo "This directory is required for restore operations"
exit 1
fi
# Install and start service using built-in installer
cd "$RUNNER_DIR"
sudo ./svc.sh install $RUNNER_USER
# First check if we have the required files for restoration
if [ -f "${RUNNER_DIR}/.credentials" ] && [ -f "${RUNNER_DIR}/.service" ]; then
can_restore=true
service_name=$(cat "${RUNNER_DIR}/.service")
echo "Found required runner configuration files"
else
echo "Missing required runner configuration files"
echo "Required: .credentials and .service files in ${RUNNER_DIR}"
exit 1
fi
# Handle auto-start configuration
if [ "$START_AT_BOOT" = false ]; then
sudo systemctl disable actions.runner.sunnypilot.$(uname -n)
fi
# Then check if restoration is needed (if either service or user is missing)
if ! systemctl list-unit-files "${service_name}" &>/dev/null; then
echo "Service ${service_name} not found in systemd"
needs_restore=true
fi
if ! id "${RUNNER_USER}" &>/dev/null; then
echo "User ${RUNNER_USER} does not exist"
needs_restore=true
fi
# Only proceed if we can restore AND need to restore
if [ "$can_restore" = true ] && [ "$needs_restore" = true ]; then
echo "Restoration is needed and possible"
return 0
else
if [ "$needs_restore" = false ]; then
echo "System is already properly configured (user and service exist)"
fi
exit 0
fi
}
perform_restore() {
echo "Starting runner restoration..."
setup_directories
remount_rw
setup_system_configs
install_service
remount_ro
echo "Runner restoration completed successfully"
}
perform_install() {
echo "Starting fresh installation..."
setup_directories
install_runner
create_service_template
remount_rw
setup_system_configs
configure_runner
install_service
remount_ro
echo "Installation completed successfully"
}
main() {
if [ "$RESTORE_MODE" = true ]; then
echo "Running in restore mode - will only restore system configurations..."
check_restore_prerequisites
perform_restore
else
# Check required arguments for normal installation
if [ -z "$GITHUB_TOKEN" ]; then
echo "Usage: $0 [--start-at-boot] [--token <github_token>] [--repo <repository_url>] [--restore]"
echo "Required argument (except for --restore): github_token"
echo "Optional arguments:"
echo " --start-at-boot Enable auto-start at boot (default: false)"
echo " --repo Repository URL (default: ${DEFAULT_REPO_URL})"
echo " --restore Restore existing runner configuration"
exit 1
fi
# Set repository URL if not provided
REPO_URL="${REPO_URL:-$DEFAULT_REPO_URL}"
perform_install
fi
echo "Starting runner service..."
cd "$RUNNER_DIR"
sudo ./svc.sh start
}
main
sudo ./svc.sh start

View File

@@ -1,361 +0,0 @@
#!/usr/bin/env python3
import argparse
import subprocess
import sys
import shutil
import signal
import contextlib
import tempfile
import os
def run_command(command: str) -> tuple[int, str, str]:
"""Run a shell command and return exit code, stdout, and stderr."""
process = subprocess.Popen(
command,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True
)
stdout, stderr = process.communicate()
return process.returncode, stdout.strip(), stderr.strip()
def is_gh_available() -> bool:
"""Check if GitHub CLI is available."""
return shutil.which('gh') is not None
def get_current_branch() -> str | None:
"""Get the name of the current git branch."""
code, output, error = run_command("git rev-parse --abbrev-ref HEAD")
if code != 0:
print(f"Error getting current branch: {error}")
return None
return output
def backup_branch(branch_name: str) -> bool:
"""Create a backup of the current branch."""
backup_name = f"{branch_name}-backup-$(date +%Y%m%d_%H%M%S)"
code, _, error = run_command(f"git branch {backup_name}")
if code != 0:
print(f"Error creating backup branch: {error}")
return False
print(f"Created backup branch: {backup_name}")
return True
def get_commit_messages(source_branch: str, target_branch: str) -> list[str] | None:
"""Get all commit messages between source and target branches."""
code, output, error = run_command(f"git log {target_branch}..{source_branch} --format=%B")
if code != 0:
print(f"Error getting commit messages: {error}")
return None
return [msg.strip() for msg in output.splitlines() if msg and not msg.startswith('Merge')]
def get_pr_info(branch_name: str) -> str | None:
"""Get PR title using GitHub CLI."""
if not is_gh_available():
print("Warning: GitHub CLI not found. Install it to auto-fetch PR titles:")
print(" https://cli.github.com/")
return None
# Try to get PR info using gh cli
code, output, error = run_command(f"gh pr view --json title --jq .title {branch_name}")
if code != 0:
print(f"No open PR found for branch '{branch_name}'")
return None
return output
def create_squash_message(pr_title: str | None, commit_messages: list[str], source_branch: str) -> str:
"""Create a squash commit message from PR title and commit messages."""
parts = []
# Add PR title if provided
if pr_title:
parts.append(pr_title)
else:
parts.append(f"Squashed changes from {source_branch}")
parts.append("") # Empty line after title
# Add original commits section
if commit_messages:
parts.append("Original commits:")
parts.append("") # Empty line before list
parts.extend(f"* {msg}" for msg in commit_messages)
return '\n'.join(parts)
def prompt_for_title() -> str:
"""Prompt user for a commit title."""
return input("Enter commit title (or press Enter to use default): ").strip()
@contextlib.contextmanager
def workspace_manager(original_branch: str):
"""Context manager to handle workspace state and cleanup."""
stash_created = False
stash_restored = False
temp_branch: str | None = None
def cleanup_handler(signum=None, frame=None):
"""Clean up workspace state."""
nonlocal temp_branch, stash_created, stash_restored
try:
if signum and stash_restored:
# If we're handling Ctrl+C but stash was already restored,
# just clean up branches and exit
current = get_current_branch()
if current and current != original_branch:
run_command(f"git checkout {original_branch}")
if temp_branch:
run_command(f"git branch -D {temp_branch}")
print("\nOperation interrupted, but changes were already restored.")
sys.exit(3)
# First, switch back to original branch
current = get_current_branch()
if current and current != original_branch:
run_command(f"git checkout {original_branch}")
# Then clean up temp branch
if temp_branch:
run_command(f"git branch -D {temp_branch}")
# Finally, restore stash if needed - AFTER switching branches
if stash_created and not stash_restored:
print("Restoring your uncommitted changes...")
code, stash_list, _ = run_command("git stash list")
if code == 0 and "Automatic stash by squash script" in stash_list:
run_command("git stash pop")
stash_restored = True
stash_created = False
if signum:
print("\nOperation interrupted. Cleaned up and restored original state.")
sys.exit(4)
except Exception as e:
print(f"Error during cleanup: {e}")
if signum:
sys.exit(5)
try:
# Set up signal handlers
signal.signal(signal.SIGINT, cleanup_handler)
signal.signal(signal.SIGTERM, cleanup_handler)
# Check for changes (including untracked files)
code, output, _ = run_command("git status --porcelain")
if output:
print("Stashing uncommitted changes...")
run_command("git stash push -u -m 'Automatic stash by squash script'")
stash_created = True
yield lambda x: setattr(x, 'temp_branch', temp_branch)
except Exception as e:
print(f"\nError occurred: {str(e)}")
cleanup_handler()
raise
finally:
cleanup_handler()
def create_commit_with_message(message: str) -> bool:
"""Create a commit with the given message using a temporary file."""
try:
with tempfile.NamedTemporaryFile(mode='w', delete=False) as f:
f.write(message)
temp_path = f.name
# Use the temporary file for the commit message
code, _, error = run_command(f"git commit -F {temp_path}")
os.unlink(temp_path) # Clean up the temp file
if code != 0:
print(f"Error creating commit: {error}")
return False
return True
except Exception as e:
print(f"Error handling commit message: {e}")
if os.path.exists(temp_path):
os.unlink(temp_path)
return False
def squash_and_merge(source_branch: str, target_branch: str, manual_title: str | None, backup: bool = False, push: bool = False) -> bool:
"""
Squash the source branch and merge into target branch.
"""
# Get original branch right away
original_branch = get_current_branch()
if not original_branch:
return False
class State:
temp_branch: str | None = None
state = State()
with workspace_manager(original_branch) as set_temp_branch:
# Validate source branch exists
code, _, error = run_command(f"git rev-parse --verify {source_branch}")
if code != 0:
print(f"Error: Source branch {source_branch} not found")
return False
if source_branch == target_branch:
print(f"Error: Source and target branches cannot be the same ({source_branch})")
return False
# Ensure target branch exists
code, _, error = run_command(f"git rev-parse --verify {target_branch}")
if code != 0:
print(f"Error: Target branch {target_branch} not found")
return False
# Find merge base
code, merge_base, error = run_command(f"git merge-base {target_branch} {source_branch}")
if code != 0:
print(f"Error finding merge base: {error}")
return False
# Create backup unless explicitly skipped
if backup and not backup_branch(source_branch):
return False
# Get commit messages
commit_messages = get_commit_messages(source_branch, target_branch)
if commit_messages is None:
return False
# Get title (priority: manual title > PR title > prompt user)
title = manual_title
if not title:
title = get_pr_info(source_branch)
if not title:
title = prompt_for_title()
try:
# Create and switch to temporary branch
temp_branch = f"temp-squash-{source_branch}"
state.temp_branch = temp_branch
set_temp_branch(state)
print(f"\nCreating temporary branch {temp_branch}...")
code, _, error = run_command(f"git checkout -b {temp_branch} {source_branch}")
if code != 0:
print(f"Error creating temp branch: {error}")
return False
print("Preparing squash by resetting temporary branch to merge base...")
code, _, error = run_command(f"git reset --soft {merge_base}")
if code != 0:
print(f"Error resetting for squash: {error}")
return False
# Create commit with message
print("Creating squash commit...")
squash_message = create_squash_message(title, commit_messages, source_branch)
if not create_commit_with_message(squash_message):
return False
# Switch to target and try merge
print(f"\nSwitching to target branch {target_branch}...")
code, _, error = run_command(f"git checkout {target_branch}")
if code != 0:
print(f"Error checking out target branch: {error}")
return False
print(f"Attempting to merge changes from {temp_branch}...")
code, _, error = run_command(f"git rebase {temp_branch}")
if code != 0:
print(f"\nMerge failed with error: {error}")
print("\nThe squash was successful, and your changes are preserved in the temporary branch.")
print("To complete the merge manually, follow these steps:")
print(f"\n1. Your squashed changes are in branch: '{temp_branch}'")
print(f"2. The target branch is: '{target_branch}'")
print("\nTo resolve the conflicts:")
print(f" git checkout {target_branch}")
print(f" git merge {temp_branch}")
print(" # resolve conflicts in your editor")
print(" git add <resolved-files>")
print(" git commit")
print(f" git push origin {target_branch} # when ready to push")
print("\nTo clean up after successful merge:")
print(f" git branch -D {temp_branch}")
# Make sure to abort the merge
print("\nAborting current merge attempt...")
run_command("git merge --abort")
# Return to original branch, but keep temp branch
print(f"Returning to {original_branch}...")
run_command(f"git checkout {original_branch}")
return False
# Clean up temp branch on success
run_command(f"git branch -D {temp_branch}")
# Push if requested
if push:
code, _, error = run_command(f"git push origin {target_branch}")
if code != 0:
print(f"Error pushing to {target_branch}: {error}")
return False
print(f"Successfully pushed to {target_branch}")
else:
print(f"Changes squashed and merged into {target_branch} locally")
print(f"To push the changes: git push origin {target_branch}")
# Return to original branch
code, _, error = run_command(f"git checkout {original_branch}")
if code != 0:
print(f"Warning: Failed to return to original branch: {error}")
return False
return True
except Exception as e:
print(f"Error during squash process: {e}")
return False
def main():
parser = argparse.ArgumentParser(
description='Squash branch and merge into target branch'
)
parser.add_argument('--target', '-t', required=True,
help='Target branch to merge changes into')
parser.add_argument('--source', '-s',
help='Source branch to squash (default: current branch)')
parser.add_argument('--title', '-m',
help='Optional manual title (overrides PR title)')
parser.add_argument('--backup', action='store_true',
help='Creates a backup branch for the source branch')
parser.add_argument('--push', action='store_true',
help='Push changes to remote after squashing')
args, unknown = parser.parse_known_args()
# Determine source branch early
source_branch = args.source
if not source_branch:
source_branch = get_current_branch()
if not source_branch:
sys.exit(1)
if not squash_and_merge(source_branch, args.target, args.title, args.backup, args.push):
sys.exit(2)
if __name__ == "__main__":
main()

View File

@@ -1,163 +0,0 @@
#!/usr/bin/env python3
import subprocess
import sys
import os
import argparse
import json
from datetime import datetime
def setup_argument_parser():
parser = argparse.ArgumentParser(description='Process and squash GitHub PRs')
parser.add_argument('--pr-data', type=str, help='PR data in JSON format')
parser.add_argument('--source-branch', type=str, default='master-new',
help='Source branch for merging')
parser.add_argument('--target-branch', type=str, default='master-dev-c3-new-test',
help='Target branch for merging')
parser.add_argument('--squash-script-path', type=str, required=True,
help='Path to the squash_and_merge.py script')
return parser
def validate_squash_script(script_path):
if not os.path.isfile(script_path):
raise FileNotFoundError(f"Squash script not found at: {script_path}")
if not os.access(script_path, os.X_OK):
raise PermissionError(f"Squash script is not executable: {script_path}")
def sort_prs_by_creation(pr_data):
"""Sort PRs by creation date"""
nodes = (pr_data.get('data', {}).get('search', {}).get('nodes', []))
return sorted(
nodes,
key=lambda x: datetime.fromisoformat(x.get('createdAt', '').replace('Z', '+00:00'))
)
def add_pr_comment(pr_number, comment):
"""Add a comment to a PR using gh cli"""
try:
subprocess.run(
['gh', 'pr', 'comment', str(pr_number), '--body', comment],
check=True,
capture_output=True,
text=True
)
except subprocess.CalledProcessError as e:
print(f"Failed to add comment to PR #{pr_number}: {e.stderr}")
def validate_pr(pr):
"""Validate a PR and return (is_valid, skip_reason)"""
pr_number = pr.get('number', 'UNKNOWN')
branch = pr.get('headRefName', '')
if not branch:
return False, f"Missing branch name for PR #{pr_number}"
# Check if checks have passed
commits = pr.get('commits', {}).get('nodes', [])
if not commits:
return False, "No commit data found"
status = commits[0].get('commit', {}).get('statusCheckRollup', {})
if not status or status.get('state') != 'SUCCESS':
return False, "Not all checks have passed"
# Check for merge conflicts
merge_status = subprocess.run(['gh', 'pr', 'view', str(pr_number), '--json', 'mergeable,mergeStateStatus'], capture_output=True, text=True)
merge_data = json.loads(merge_status.stdout)
if not merge_data.get('mergeable'):
return False, "Merge conflicts detected"
if (mergeStateStatus := merge_data.get('mergeStateStatus')) == "BEHIND":
return False, f"Branch is `{mergeStateStatus}`"
return True, None
def process_pr(pr_data, source_branch, target_branch, squash_script_path):
try:
nodes = sort_prs_by_creation(pr_data)
if not nodes:
print("No PRs to squash")
return 0
print(f"Deleting target branch {target_branch}")
subprocess.run(['git', 'branch', '-D', target_branch], check=False)
subprocess.run(['git', 'branch', target_branch, f'origin/{source_branch}'], check=True)
success_count = 0
for pr in nodes:
pr_number = pr.get('number', 'UNKNOWN')
branch = pr.get('headRefName', '')
title = pr.get('title', '')
is_valid, skip_reason = validate_pr(pr)
if not is_valid:
print(f"Warning: {skip_reason} for PR #{pr_number}, skipping")
add_pr_comment(pr_number, f"⚠️ This PR was skipped in the automated `{target_branch}` squash because {skip_reason}.")
continue
try:
# Fetch PR branch
subprocess.run(['git', 'fetch', 'origin', branch], check=True)
# Delete branch if it exists (ignore errors if it doesn't)
subprocess.run(['git', 'branch', '-D', branch], check=False)
# Create new branch pointing to origin's branch
subprocess.run(['git', 'branch', branch, f'origin/{branch}'], check=True)
# Run squash script
subprocess.run([
squash_script_path,
'--target', target_branch,
'--source', branch,
'--title', f"{title} (#{pr_number})",
], check=True)
print(f"Successfully processed PR #{pr_number}")
success_count += 1
except subprocess.CalledProcessError as e:
print(f"Error processing PR #{pr_number}:")
print(f"Command failed with exit code {e.returncode}")
error_output = getattr(e, 'stderr', 'No error output available')
print(f"Error output: {error_output}")
add_pr_comment(pr_number, f"⚠️ Error during automated {target_branch} squash:\n```\n{error_output}\n```")
continue
except Exception as e:
print(f"Unexpected error processing PR #{pr_number}: {str(e)}")
continue
return success_count
except Exception as e:
import traceback
print(f"Error in process_pr: {str(e)}")
print("Full traceback:")
print(traceback.format_exc())
return 0
def main():
parser = setup_argument_parser()
try:
args = parser.parse_args()
validate_squash_script(args.squash_script_path)
pr_data_json = json.loads(args.pr_data)
# Process the PRs
success_count = process_pr(pr_data_json, args.source_branch, args.target_branch, args.squash_script_path)
print(f"Successfully processed {success_count} PRs")
except Exception as e:
print(f"Fatal error: {str(e)}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -6,11 +6,41 @@ from pathlib import Path
HERE = os.path.abspath(os.path.dirname(__file__))
ROOT = HERE + "/.."
# blacklisting is for two purposes:
# - minimizing release download size
# - keeping the diff readable
blacklist = [
".git/",
"panda/drivers/",
"panda/examples/",
"panda/tests/safety/",
"opendbc_repo/dbc/.*.dbc$",
"opendbc_repo/dbc/generator/",
"cereal/.*test.*",
"^common/tests/",
# particularly large text files
"uv.lock",
"third_party/catch2",
"selfdrive/car/tests/test_models.*",
"^tools/",
"^tinygrad_repo/",
"matlab.*.md",
".git/",
".github/",
".devcontainer/",
"Darwin/",
".vscode",
# common things
"LICENSE",
"Dockerfile",
".pre-commit",
# no LFS or submodules in release
".lfsconfig",
".gitattributes",
@@ -18,10 +48,153 @@ blacklist = [
".gitmodules",
]
# gets you through the blacklist
whitelist: list[str] = [
# Sunnypilot blacklist
sunnypilot_blacklist = [
"system/loggerd/sunnylink_uploader.py", # Temporarily, until we are ready to roll it out widely
".idea/",
".run/",
".*__pycache__/.*",
".*\\.pyc",
"teleoprtc/*",
"third_party/snpe/x86_64/*",
"body/board/canloader.py",
"body/board/flash_base.sh",
"body/board/flash_knee.sh",
"body/board/recover.sh",
".*/test/",
".*/tests/",
".*tinygrad_repo/tinygrad/renderer/",
"README.md",
".*internal/",
"docs/.*",
".sconsign.dblite",
"release/ci/scons_cache/",
".gitlab-ci.yml",
".clang-tidy",
".dockerignore",
".editorconfig",
".python-version",
"SECURITY.md",
"codecov.yml",
"conftest.py",
"poetry.lock",
".venv/",
]
# Merge the blacklists
blacklist += sunnypilot_blacklist
# gets you through the blacklist
whitelist = [
"tools/lib/",
"tools/bodyteleop/",
"tools/joystick/",
"tools/longitudinal_maneuvers/",
"tinygrad_repo/examples/openpilot/compile3.py",
"tinygrad_repo/extra/onnx.py",
"tinygrad_repo/extra/onnx_ops.py",
"tinygrad_repo/extra/thneed.py",
"tinygrad_repo/extra/utils.py",
"tinygrad_repo/tinygrad/codegen/kernel.py",
"tinygrad_repo/tinygrad/codegen/linearizer.py",
"tinygrad_repo/tinygrad/features/image.py",
"tinygrad_repo/tinygrad/features/search.py",
"tinygrad_repo/tinygrad/nn/*",
"tinygrad_repo/tinygrad/renderer/cstyle.py",
"tinygrad_repo/tinygrad/renderer/opencl.py",
"tinygrad_repo/tinygrad/runtime/lib.py",
"tinygrad_repo/tinygrad/runtime/ops_cpu.py",
"tinygrad_repo/tinygrad/runtime/ops_disk.py",
"tinygrad_repo/tinygrad/runtime/ops_gpu.py",
"tinygrad_repo/tinygrad/shape/*",
"tinygrad_repo/tinygrad/.*.py",
# TODO: do this automatically
"opendbc_repo/dbc/comma_body.dbc",
"opendbc_repo/dbc/chrysler_ram_hd_generated.dbc",
"opendbc_repo/dbc/chrysler_ram_dt_generated.dbc",
"opendbc_repo/dbc/chrysler_pacifica_2017_hybrid_generated.dbc",
"opendbc_repo/dbc/chrysler_pacifica_2017_hybrid_private_fusion.dbc",
"opendbc_repo/dbc/gm_global_a_powertrain_generated.dbc",
"opendbc_repo/dbc/gm_global_a_object.dbc",
"opendbc_repo/dbc/gm_global_a_chassis.dbc",
"opendbc_repo/dbc/FORD_CADS.dbc",
"opendbc_repo/dbc/ford_fusion_2018_adas.dbc",
"opendbc_repo/dbc/ford_lincoln_base_pt.dbc",
"opendbc_repo/dbc/honda_accord_2018_can_generated.dbc",
"opendbc_repo/dbc/acura_ilx_2016_can_generated.dbc",
"opendbc_repo/dbc/acura_rdx_2018_can_generated.dbc",
"opendbc_repo/dbc/acura_rdx_2020_can_generated.dbc",
"opendbc_repo/dbc/honda_civic_touring_2016_can_generated.dbc",
"opendbc_repo/dbc/honda_civic_hatchback_ex_2017_can_generated.dbc",
"opendbc_repo/dbc/honda_crv_touring_2016_can_generated.dbc",
"opendbc_repo/dbc/honda_crv_ex_2017_can_generated.dbc",
"opendbc_repo/dbc/honda_crv_ex_2017_body_generated.dbc",
"opendbc_repo/dbc/honda_crv_executive_2016_can_generated.dbc",
"opendbc_repo/dbc/honda_fit_ex_2018_can_generated.dbc",
"opendbc_repo/dbc/honda_odyssey_exl_2018_generated.dbc",
"opendbc_repo/dbc/honda_odyssey_extreme_edition_2018_china_can_generated.dbc",
"opendbc_repo/dbc/honda_insight_ex_2019_can_generated.dbc",
"opendbc_repo/dbc/acura_ilx_2016_nidec.dbc",
"opendbc_repo/dbc/honda_civic_ex_2022_can_generated.dbc",
"opendbc_repo/dbc/hyundai_canfd.dbc",
"opendbc_repo/dbc/hyundai_kia_generic.dbc",
"opendbc_repo/dbc/hyundai_kia_mando_front_radar_generated.dbc",
"opendbc_repo/dbc/mazda_2017.dbc",
"opendbc_repo/dbc/nissan_x_trail_2017_generated.dbc",
"opendbc_repo/dbc/nissan_leaf_2018_generated.dbc",
"opendbc_repo/dbc/subaru_global_2017_generated.dbc",
"opendbc_repo/dbc/subaru_global_2020_hybrid_generated.dbc",
"opendbc_repo/dbc/subaru_outback_2015_generated.dbc",
"opendbc_repo/dbc/subaru_outback_2019_generated.dbc",
"opendbc_repo/dbc/subaru_forester_2017_generated.dbc",
"opendbc_repo/dbc/toyota_tnga_k_pt_generated.dbc",
"opendbc_repo/dbc/toyota_new_mc_pt_generated.dbc",
"opendbc_repo/dbc/toyota_nodsu_pt_generated.dbc",
"opendbc_repo/dbc/toyota_adas.dbc",
"opendbc_repo/dbc/toyota_tss2_adas.dbc",
"opendbc_repo/dbc/vw_golf_mk4.dbc",
"opendbc_repo/dbc/vw_mqb_2010.dbc",
"opendbc_repo/dbc/tesla_can.dbc",
"opendbc_repo/dbc/tesla_radar_bosch_generated.dbc",
"opendbc_repo/dbc/tesla_radar_continental_generated.dbc",
"opendbc_repo/dbc/tesla_powertrain.dbc",
]
# Sunnypilot whitelist
sunnypilot_whitelist = [
"^README.md",
".*selfdrive/test/fuzzy_generation.py",
".*selfdrive/test/helpers.py",
".*selfdrive/test/__init__.py",
".*selfdrive/test/setup_device_ci.sh",
".*selfdrive/test/test_time_to_onroad.py",
".*selfdrive/test/test_onroad.py",
".*system/manager/test/test_manager.py",
".*system/manager/test/__init__.py",
".*system/qcomgpsd/tests/test_qcomgpsd.py",
".*system/updated/casync/tests/test_casync.py",
".*system/updated/tests/test_git.py",
".*system/updated/tests/test_base.py",
".*selfdrive/ui/tests/test_translations.py",
".*selfdrive/car/tests/__init__.py",
".*selfdrive/car/tests/test_car_interfaces.py",
".*selfdrive/navd/tests/test_navd.py",
".*selfdrive/navd/tests/test_map_renderer.py",
".*selfdrive/boardd/tests/test_boardd_loopback.py",
".*INTEGRATION.md",
".*HOW-TOS.md",
".*CARS.md",
".*LIMITATIONS.md",
".*CONTRIBUTING.md",
".*sunnyhaibin0850_qrcode_paypal.me.png",
"opendbc/.*.dbc",
]
# Merge the whitelists
whitelist += sunnypilot_whitelist
if __name__ == "__main__":
for f in Path(ROOT).rglob("**/*"):

39
scripts/build_small.sh Executable file
View File

@@ -0,0 +1,39 @@
#!/usr/bin/env bash
set -ex
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR
# git clone --mirror
SRC=/tmp/openpilot.git/
OUT=/tmp/smallpilot/
echo "starting size $(du -hs .git/)"
rm -rf $OUT
cd $SRC
git remote update
# copy contents
#rsync -a --exclude='.git/' $DIR $OUT
cp -r $SRC $OUT
cd $OUT
# remove all tags
git tag -l | xargs git tag -d
# remove non-master branches
BRANCHES="release2 release3 devel master-ci nightly"
for branch in $BRANCHES; do
git branch -D $branch
git branch -D ${branch}-staging || true
done
#git gc
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now
echo "new one is $(du -hs .)"

54
scripts/code_stats.py Executable file
View File

@@ -0,0 +1,54 @@
#!/usr/bin/env python3
import os
import ast
import stat
import subprocess
fouts = {x.decode('utf-8') for x in subprocess.check_output(['git', 'ls-files']).strip().split()}
pyf = []
for d in ["cereal", "common", "scripts", "selfdrive", "tools"]:
for root, _, files in os.walk(d):
for f in files:
if f.endswith(".py"):
pyf.append(os.path.join(root, f))
imps: set[str] = set()
class Analyzer(ast.NodeVisitor):
def visit_Import(self, node):
for alias in node.names:
imps.add(alias.name)
self.generic_visit(node)
def visit_ImportFrom(self, node):
imps.add(node.module)
self.generic_visit(node)
tlns = 0
carlns = 0
scriptlns = 0
testlns = 0
for f in sorted(pyf):
if f not in fouts:
continue
xbit = bool(os.stat(f)[stat.ST_MODE] & stat.S_IXUSR)
src = open(f).read()
lns = len(src.split("\n"))
tree = ast.parse(src)
Analyzer().visit(tree)
print(f"{lns:5d} {f} {xbit}")
if 'test' in f:
testlns += lns
elif f.startswith(('tools/', 'scripts/', 'selfdrive/debug')):
scriptlns += lns
elif f.startswith('selfdrive/car'):
carlns += lns
else:
tlns += lns
print(f"{tlns} lines of openpilot python")
print(f"{carlns} lines of car ports")
print(f"{scriptlns} lines of tools/scripts/debug")
print(f"{testlns} lines of tests")
#print(sorted(list(imps)))

11
scripts/count_cars.py Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env python3
from collections import Counter
from pprint import pprint
from opendbc.car.docs import get_all_car_docs
if __name__ == "__main__":
cars = get_all_car_docs()
make_count = Counter(l.make for l in cars)
print("\n", "*" * 20, len(cars), "total", "*" * 20, "\n")
pprint(make_count)

View File

@@ -0,0 +1,391 @@
#!/usr/bin/env bash
set -e
SRC=/tmp/openpilot/
SRC_CLONE=/tmp/openpilot-clone/
OUT=/tmp/openpilot-tiny/
REWRITE_IGNORE_BRANCHES=(
dashcam3
devel
master-ci
nightly
release2
release3
release3-staging
)
VALIDATE_IGNORE_FILES=(
".github/ISSUE_TEMPLATE/bug_report.md"
".github/pull_request_template.md"
)
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR
LOGS_DIR=$DIR/git-rewrite-$(date +"%Y-%m-%dT%H:%M:%S%z")
mkdir -p $LOGS_DIR
GIT_REWRITE_LOG=$LOGS_DIR/git-rewrite-log.txt
BRANCH_DIFF_LOG=$LOGS_DIR/branch-diff-log.txt
COMMIT_DIFF_LOG=$LOGS_DIR/commit-diff-log.txt
START_TIME=$(date +%s)
exec > >(while IFS= read -r line; do
CURRENT_TIME=$(date +%s)
ELAPSED_TIME=$((CURRENT_TIME - START_TIME))
echo "[${ELAPSED_TIME}s] $line"
done | tee -a "$GIT_REWRITE_LOG") 2>&1
# INSTALL git-filter-repo
if [ ! -f /tmp/git-filter-repo ]; then
echo "Installing git-filter-repo..."
curl -sSo /tmp/git-filter-repo https://raw.githubusercontent.com/newren/git-filter-repo/main/git-filter-repo
chmod +x /tmp/git-filter-repo
fi
# MIRROR openpilot
if [ ! -d $SRC ]; then
echo "Mirroring openpilot..."
git clone --mirror https://github.com/commaai/openpilot.git $SRC # 4.18 GiB (488034 objects)
cd $SRC
echo "Starting size $(du -sh .)"
git remote update
# the git-filter-repo analysis is bliss - can be found in the repo root/filter-repo/analysis
echo "Analyzing with git-filter-repo..."
/tmp/git-filter-repo --force --analyze
echo "Pushing to openpilot-archive..."
# push to archive repo - in smaller parts because the 2 GB push limit - https://docs.github.com/en/get-started/using-git/troubleshooting-the-2-gb-push-limit
ARCHIVE_REPO=git@github.com:commaai/openpilot-archive.git
git push --prune $ARCHIVE_REPO +refs/heads/master:refs/heads/master # push master first so it's the default branch (when openpilot-archive is an empty repo)
git push --prune $ARCHIVE_REPO +refs/heads/*:refs/heads/* # 956.39 MiB (110725 objects)
git push --prune $ARCHIVE_REPO +refs/tags/*:refs/tags/* # 1.75 GiB (21694 objects)
# git push --mirror $ARCHIVE_REPO || true # fails to push refs/pull/* (deny updating a hidden ref) for pull requests
# we fail and continue - more reading: https://stackoverflow.com/a/34266401/639708 and https://blog.plataformatec.com.br/2013/05/how-to-properly-mirror-a-git-repository/
fi
# REWRITE master and tags
if [ ! -d $SRC_CLONE ]; then
echo "Cloning $SRC..."
GIT_LFS_SKIP_SMUDGE=1 git clone $SRC $SRC_CLONE
cd $SRC_CLONE
echo "Checking out old history..."
git checkout tags/v0.7.1 > /dev/null 2>&1
# checkout as main, since we need master ref later
git checkout -b main
echo "Creating setup commits..."
# rm these so we don't get conflicts later
git rm -r cereal opendbc panda selfdrive/ui/ui > /dev/null
git commit -m "removed conflicting files" > /dev/null
# skip-smudge to get rid of some lfs errors that it can't find the reference of some lfs files
# we don't care about fetching/pushing lfs right now
git lfs install --skip-smudge --local
# squash initial setup commits
git cherry-pick -n -X theirs 6c33a5c..59b3d06 > /dev/null
git commit -m "switching to master" > /dev/null
# squash the two commits
git reset --soft HEAD~2
git commit -m "switching to master" -m "$(git log --reverse --format=%B 6c33a5c..59b3d06)" -m "removed conflicting files" > /dev/null
# get commits we want to cherry-pick
# will start with the next commit after #59b3d06 tools is local now
COMMITS=$(git rev-list --reverse 59b3d06..master)
# we need this for logging
TOTAL_COMMITS=$(echo $COMMITS | wc -w | xargs)
CURRENT_COMMIT_NUMBER=0
# empty this file
> commit-map.txt
echo "Rewriting master commits..."
for COMMIT in $COMMITS; do
CURRENT_COMMIT_NUMBER=$((CURRENT_COMMIT_NUMBER + 1))
# echo -ne "[$CURRENT_COMMIT_NUMBER/$TOTAL_COMMITS] Cherry-picking commit: $COMMIT"\\r
echo "[$CURRENT_COMMIT_NUMBER/$TOTAL_COMMITS] Cherry-picking commit: $COMMIT"
# set environment variables to preserve author/committer and dates
export GIT_AUTHOR_NAME=$(git show -s --format='%an' $COMMIT)
export GIT_AUTHOR_EMAIL=$(git show -s --format='%ae' $COMMIT)
export GIT_COMMITTER_NAME=$(git show -s --format='%cn' $COMMIT)
export GIT_COMMITTER_EMAIL=$(git show -s --format='%ce' $COMMIT)
export GIT_AUTHOR_DATE=$(git show -s --format='%ad' $COMMIT)
export GIT_COMMITTER_DATE=$(git show -s --format='%cd' $COMMIT)
# cherry-pick the commit
if ! GIT_OUTPUT=$(git cherry-pick -m 1 -X theirs $COMMIT 2>&1); then
# check if the failure is because of an empty commit
if [[ "$GIT_OUTPUT" == *"The previous cherry-pick is now empty"* ]]; then
echo "Empty commit detected. Skipping commit $COMMIT"
git cherry-pick --skip
# log it was empty to the mapping file
echo "$COMMIT EMPTY" >> commit-map.txt
else
# handle other errors or conflicts
echo "Cherry-pick failed. Handling error..."
echo "$GIT_OUTPUT"
exit 1
fi
else
# capture the new commit hash
NEW_COMMIT=$(git rev-parse HEAD)
# save the old and new commit hashes to the mapping file
echo "$COMMIT $NEW_COMMIT" >> commit-map.txt
# append the old commit ID to the commit message
git commit --amend -m "$(git log -1 --pretty=%B)" -m "Former-commit-id: $COMMIT" > /dev/null
fi
# prune every 3000 commits to avoid gc errors
if [ $((CURRENT_COMMIT_NUMBER % 3000)) -eq 0 ]; then
echo "Pruning repo..."
git gc
fi
done
echo "Rewriting tags..."
# remove all old tags
git tag -l | xargs git tag -d
# read each line from the tag-commit-map.txt
while IFS=' ' read -r TAG OLD_COMMIT; do
# search for the new commit in commit-map.txt corresponding to the old commit
NEW_COMMIT=$(grep "^$OLD_COMMIT " "commit-map.txt" | awk '{print $2}')
# check if this is a rebased commit
if [ -z "$NEW_COMMIT" ]; then
# if not, then just use old commit hash
NEW_COMMIT=$OLD_COMMIT
fi
echo "Rewriting tag $TAG from commit $NEW_COMMIT"
git tag -f "$TAG" "$NEW_COMMIT"
done < "$DIR/tag-commit-map.txt"
# uninstall lfs since we don't want to touch (push to) lfs right now
# git push will also push lfs, if we don't uninstall (--local so just for this repo)
git lfs uninstall --local
# force push new master
git push --force origin main:master
# force push new tags
git push --force --tags
fi
# REWRITE branches based on master
if [ ! -f "$SRC_CLONE/rewrite-branches-done" ]; then
cd $SRC_CLONE
> rewrite-branches-done
# empty file
> $BRANCH_DIFF_LOG
echo "Rewriting branches based on master..."
# will store raw diffs here, if exist
mkdir -p differences
# get a list of all branches except master and REWRITE_IGNORE_BRANCHES
BRANCHES=$(git branch -r | grep -v ' -> ' | sed 's/.*origin\///' | grep -v '^master$' | grep -v -f <(echo "${REWRITE_IGNORE_BRANCHES[*]}" | tr ' ' '\n'))
for BRANCH in $BRANCHES; do
# check if the branch is based on master history
MERGE_BASE=$(git merge-base master origin/$BRANCH) || true
if [ -n "$MERGE_BASE" ]; then
echo "Rewriting branch: $BRANCH"
# create a new branch based on the new master
NEW_MERGE_BASE=$(grep "^$MERGE_BASE " "commit-map.txt" | awk '{print $2}')
if [ -z "$NEW_MERGE_BASE" ]; then
echo "Error: could not find new merge base for branch $BRANCH" >> $BRANCH_DIFF_LOG
continue
fi
git checkout -b ${BRANCH}_new $NEW_MERGE_BASE
# get the range of commits unique to this branch
COMMITS=$(git rev-list --reverse $MERGE_BASE..origin/${BRANCH})
HAS_ERROR=0
# simple delimiter
echo "BRANCH ${BRANCH}" >> commit-map.txt
for COMMIT in $COMMITS; do
# set environment variables to preserve author/committer and dates
export GIT_AUTHOR_NAME=$(git show -s --format='%an' $COMMIT)
export GIT_AUTHOR_EMAIL=$(git show -s --format='%ae' $COMMIT)
export GIT_COMMITTER_NAME=$(git show -s --format='%cn' $COMMIT)
export GIT_COMMITTER_EMAIL=$(git show -s --format='%ce' $COMMIT)
export GIT_AUTHOR_DATE=$(git show -s --format='%ad' $COMMIT)
export GIT_COMMITTER_DATE=$(git show -s --format='%cd' $COMMIT)
# cherry-pick the commit
if ! GIT_OUTPUT=$(git cherry-pick -m 1 -X theirs $COMMIT 2>&1); then
# check if the failure is because of an empty commit
if [[ "$GIT_OUTPUT" == *"The previous cherry-pick is now empty"* ]]; then
echo "Empty commit detected. Skipping commit $COMMIT"
git cherry-pick --skip
# log it was empty to the mapping file
echo "$COMMIT EMPTY" >> commit-map.txt
else
# handle other errors or conflicts
echo "Cherry-pick of ${BRANCH} branch failed. Removing branch upstream..." >> $BRANCH_DIFF_LOG
echo "$GIT_OUTPUT" > "$LOGS_DIR/branch-${BRANCH}"
git cherry-pick --abort
git push --delete origin ${BRANCH}
HAS_ERROR=1
break
fi
else
# capture the new commit hash
NEW_COMMIT=$(git rev-parse HEAD)
# save the old and new commit hashes to the mapping file
echo "$COMMIT $NEW_COMMIT" >> commit-map.txt
# append the old commit ID to the commit message
git commit --amend -m "$(git log -1 --pretty=%B)" -m "Former-commit-id: $COMMIT" > /dev/null
fi
done
# force push the new branch
if [ $HAS_ERROR -eq 0 ]; then
# git lfs goes haywire here, so we need to install and uninstall
# git lfs install --skip-smudge --local
git lfs uninstall --local > /dev/null
git push -f origin ${BRANCH}_new:${BRANCH}
fi
# clean up local branch
git checkout master > /dev/null
git branch -D ${BRANCH}_new > /dev/null
else
echo "Deleting branch $BRANCH as it's not based on master history" >> $BRANCH_DIFF_LOG
git push --delete origin ${BRANCH}
fi
done
fi
# VALIDATE cherry-pick
if [ ! -f "$SRC_CLONE/validation-done" ]; then
cd $SRC_CLONE
> validation-done
TOTAL_COMMITS=$(grep -cve '^\s*$' commit-map.txt)
CURRENT_COMMIT_NUMBER=0
COUNT_SAME=0
COUNT_DIFF=0
# empty file
> $COMMIT_DIFF_LOG
echo "Validating commits..."
# will store raw diffs here, if exist
mkdir -p differences
# read each line from commit-map.txt
while IFS=' ' read -r OLD_COMMIT NEW_COMMIT; do
if [ "$NEW_COMMIT" == "EMPTY" ]; then
continue
fi
if [ "$OLD_COMMIT" == "BRANCH" ]; then
echo "Branch ${NEW_COMMIT} below:" >> $COMMIT_DIFF_LOG
continue
fi
CURRENT_COMMIT_NUMBER=$((CURRENT_COMMIT_NUMBER + 1))
# retrieve short hashes and dates for the old and new commits
OLD_COMMIT_SHORT=$(git rev-parse --short $OLD_COMMIT)
NEW_COMMIT_SHORT=$(git rev-parse --short $NEW_COMMIT)
OLD_DATE=$(git show -s --format='%cd' $OLD_COMMIT)
NEW_DATE=$(git show -s --format='%cd' $NEW_COMMIT)
# echo -ne "[$CURRENT_COMMIT_NUMBER/$TOTAL_COMMITS] Comparing old commit $OLD_COMMIT_SHORT ($OLD_DATE) with new commit $NEW_COMMIT_SHORT ($NEW_DATE)"\\r
echo "[$CURRENT_COMMIT_NUMBER/$TOTAL_COMMITS] Comparing old commit $OLD_COMMIT_SHORT ($OLD_DATE) with new commit $NEW_COMMIT_SHORT ($NEW_DATE)"
# generate lists of files and their hashes for the old and new commits, excluding ignored files
OLD_FILES=$(git ls-tree -r $OLD_COMMIT | grep -vE "$(IFS='|'; echo "${VALIDATE_IGNORE_FILES[*]}")")
NEW_FILES=$(git ls-tree -r $NEW_COMMIT | grep -vE "$(IFS='|'; echo "${VALIDATE_IGNORE_FILES[*]}")")
# Compare the diffs
if diff <(echo "$OLD_FILES") <(echo "$NEW_FILES") > /dev/null; then
# echo "Old commit $OLD_COMMIT_SHORT and new commit $NEW_COMMIT_SHORT are equivalent."
COUNT_SAME=$((COUNT_SAME + 1))
else
echo "[$CURRENT_COMMIT_NUMBER/$TOTAL_COMMITS] Difference found between old commit $OLD_COMMIT_SHORT and new commit $NEW_COMMIT_SHORT" >> $COMMIT_DIFF_LOG
COUNT_DIFF=$((COUNT_DIFF + 1))
set +e
diff -u <(echo "$OLD_FILES") <(echo "$NEW_FILES") > "$LOGS_DIR/commit-$CURRENT_COMMIT_NUMBER-$OLD_COMMIT_SHORT-$NEW_COMMIT_SHORT"
set -e
fi
done < "commit-map.txt"
echo "Summary:" >> $COMMIT_DIFF_LOG
echo "Equivalent commits: $COUNT_SAME" >> $COMMIT_DIFF_LOG
echo "Different commits: $COUNT_DIFF" >> $COMMIT_DIFF_LOG
fi
if [ ! -d $OUT ]; then
cp -r $SRC $OUT
cd $OUT
# remove all non-master branches
# git branch | grep -v "^ master$" | grep -v "\*" | xargs git branch -D
# echo "cleaning up refs"
# delete pull request refs since we can't alter them anyway (https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally#error-failed-to-push-some-refs)
# git for-each-ref --format='%(refname)' | grep '^refs/pull/' | xargs -I {} git update-ref -d {}
echo "importing new lfs files"
# import "almost" everything to lfs
BRANCHES=$(git for-each-ref --format='%(refname)' refs/heads/ | sed 's%refs/heads/%%g' | grep -v -f <(echo "${REWRITE_IGNORE_BRANCHES[*]}" | tr ' ' '\n') | tr '\n' ' ')
git lfs migrate import --include="*.dlc,*.onnx,*.svg,*.png,*.gif,*.ttf,*.wav,selfdrive/car/tests/test_models_segs.txt,system/hardware/tici/updater,selfdrive/ui/qt/spinner_larch64,selfdrive/ui/qt/text_larch64,third_party/**/*.a,third_party/**/*.so,third_party/**/*.so.*,third_party/**/*.dylib,third_party/acados/*/t_renderer,third_party/qt5/larch64/bin/lrelease,third_party/qt5/larch64/bin/lupdate,third_party/catch2/include/catch2/catch.hpp,*.apk,*.apkpatch,*.jar,*.pdf,*.jpg,*.mp3,*.thneed,*.tar.gz,*.npy,*.csv,*.a,*.so*,*.dylib,*.o,*.b64,selfdrive/hardware/tici/updater,selfdrive/boardd/tests/test_boardd,selfdrive/ui/qt/spinner_aarch64,installer/updater/updater,selfdrive/debug/profiling/simpleperf/**/*,selfdrive/hardware/eon/updater,selfdrive/ui/qt/text_aarch64,selfdrive/debug/profiling/pyflame/**/*,installer/installers/installer_openpilot,installer/installers/installer_dashcam,selfdrive/ui/text/text,selfdrive/ui/android/text/text,selfdrive/ui/spinner/spinner,selfdrive/visiond/visiond,selfdrive/loggerd/loggerd,selfdrive/sensord/sensord,selfdrive/sensord/gpsd,selfdrive/ui/android/spinner/spinner,selfdrive/ui/qt/spinner,selfdrive/ui/qt/text,_stringdefs.py,dfu-util-aarch64-linux,dfu-util-aarch64,dfu-util-x86_64-linux,dfu-util-x86_64,stb_image.h,clpeak3,clwaste,apk/**/*,external/**/*,phonelibs/**/*,third_party/boringssl/**/*,flask/**/*,panda/**/*,board/**/*,messaging/**/*,opendbc/**/*,tools/cabana/chartswidget.cc,third_party/nanovg/**/*,selfdrive/controls/lib/lateral_mpc/lib_mpc_export/**/*,selfdrive/ui/paint.cc,werkzeug/**/*,pyextra/**/*,third_party/android_hardware_libhardware/**/*,selfdrive/controls/lib/lead_mpc_lib/lib_mpc_export/**/*,selfdrive/locationd/laikad.py,selfdrive/locationd/test/test_laikad.py,tools/gpstest/test_laikad.py,selfdrive/locationd/laikad_helpers.py,tools/nui/**/*,jsonrpc/**/*,selfdrive/controls/lib/longitudinal_mpc/lib_mpc_export/**/*,selfdrive/controls/lib/lateral_mpc/mpc_export/**/*,selfdrive/camerad/cameras/camera_qcom.cc,selfdrive/manager.py,selfdrive/modeld/models/driving.cc,third_party/curl/**/*,selfdrive/modeld/thneed/debug/**/*,selfdrive/modeld/thneed/include/**/*,third_party/openmax/**/*,selfdrive/controls/lib/longitudinal_mpc/mpc_export/**/*,selfdrive/controls/lib/longitudinal_mpc_model/lib_mpc_export/**/*,Pipfile,Pipfile.lock,gunicorn/**/*,*.qm,jinja2/**/*,click/**/*,dbcs/**/*,websocket/**/*" $BRANCHES
echo "reflog and gc"
# this is needed after lfs import
git reflog expire --expire=now --all
git gc --prune=now --aggressive
# check the git-filter-repo analysis again - can be found in the repo root/filter-repo/analysis
echo "Analyzing with git-filter-repo..."
/tmp/git-filter-repo --force --analyze
echo "New size is $(du -sh .)"
fi
cd $OUT
# fetch all lfs files from https://github.com/commaai/openpilot.git
# some lfs files are missing on gitlab, but they can be found on github
git config lfs.url https://github.com/commaai/openpilot.git/info/lfs
git config lfs.pushurl ssh://git@github.com/commaai/openpilot.git
git lfs fetch --all || true
# also fetch all lfs files from https://gitlab.com/commaai/openpilot-lfs.git
git config lfs.url https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
git config lfs.pushurl ssh://git@gitlab.com/commaai/openpilot-lfs.git
git lfs fetch --all || true
# final push - will also push lfs
# TODO: switch to git@github.com:commaai/openpilot.git when ready
# git push --mirror git@github.com:commaai/openpilot-tiny.git
# using this instead to ignore refs/pull/* - since this is also what --mirror does - https://blog.plataformatec.com.br/2013/05/how-to-properly-mirror-a-git-repository/
git push --prune git@github.com:commaai/openpilot-tiny.git +refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/*

59
scripts/git_rewrite/rewrite.sh Executable file
View File

@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -e
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
cd $DIR
git clone --bare https://github.com/commaai/openpilot
cp -r openpilot.git openpilot_backup
cd openpilot.git
# backup old repo
git push git@github.com:commaai/openpilot-archive.git +refs/heads/master:refs/heads/master
git push git@github.com:commaai/openpilot-archive.git +refs/heads/*:refs/heads/*
git push git@github.com:commaai/openpilot-archive.git +refs/tags/*:refs/tags/*
git push --mirror git@github.com:commaai/openpilot-archive.git
# ignore all release branches
git for-each-ref --format='delete %(refname)' | grep 'dashcam3\|devel\|master-ci\|nightly\|release2\|release3\|release3-staging' | git update-ref --stdin
# re-tag old releases on master
declare -A TAGS=( ["f8cb04e4a8b032b72a909f68b808a50936184bee"]="v0.9.7" ["0b4d08fab8e35a264bc7383e878538f8083c33e5"]="v0.9.6" ["3b1e9017c560499786d8a0e46aaaeea65037acac"]="v0.9.5" ["fa310d9e2542cf497d92f007baec8fd751ffa99c"]="v0.9.4" ["8704c1ff952b5c85a44f50143bbd1a4f7b4887e2"]="v0.9.3" ["c7d3b28b93faa6c955fb24bc64031512ee985ee9"]="v0.9.2" ["89f68bf0cbf53a81b0553d3816fdbe522f941fa1"]="v0.9.1" ["58b84fb401a804967aa0dd5ee66fafa90194fd30"]="v0.9.0" ["f41dc62a12cc0f3cb8c5453c0caa0ba21e1bd01e"]="v0.8.16" ["5a7c2f90361e72e9c35e88abd2e11acdc4aba354"]="v0.8.15" ["71901c94dbbaa2f9f156a80c14cc7ea65219fc7c"]="v0.8.14" ["95da47079510afc91665263619e5939126da637c"]="v0.8.13" ["472177e2a8a1d002e56f9096326fd2dff62e54f9"]="v0.8.12" ["08078acbd0b4f7da469c7dff6159000e358974a9"]="v0.8.11" ["687925c775c375495f9827946138a724bde00b9d"]="v0.8.10" ["204e5a090735a059d69c29145a4cee49450da07e"]="v0.8.9" ["4be956f8861ecbb521ef9503a3c87b07c9d36721"]="v0.8.8" ["589f82c76627d634761a31a34b2488403556eb0b"]="v0.8.7" ["507cfc8910f74ddb8810039d68b880b426ff9ff9"]="v0.8.6" ["d47b00b45a866bef088f51d1ff31de5885ab04e9"]="v0.8.5" ["553e7d1cce314e7eb0587186b1764c3ff43bed62"]="v0.8.4" ["9896438d1511602a1ff87f7c4eb3c7172b30104a"]="v0.8.3" ["280192ed1443f112463417c2d815ea8ee2762fbd"]="v0.8.2" ["8039361567e4659eae2a084e6f39f34acadf4cac"]="v0.8.1" ["d56e04c0d960c8d3d4ab88b578dc508a2b4e07dc"]="v0.8" ["3d456e5d0fbf0c9887d0499dee812f2b029edf6d"]="v0.7.10" ["81763a18b5d0e379b749e090ecce36a91fca7c43"]="v0.7.9" ["9bc0b350fd273bbb2deb3dcaef0312944e4f6cfd"]="v0.7.8" ["ede5b632b58c55e4ff003f948efae07fe03c2280"]="v0.7.7" ["775acd11ba2e0a8c2f5a5655338718d796491b36"]="v0.7.6.1" ["302417b4cf0dcf00d45e4995b5410e543ad121d1"]="v0.7.5" ["12ff088b42221dd17d9d97decb1fc61a7cb0a861"]="v0.7.4" ["9563f7730252451fdcba9bc3d9fe36dab9c86a26"]="v0.7.3" ["8321cf283abbc2ca3fda7e0c7a069a77a492fe0c"]="v0.7.2" ["1e1de64a1e59476b7b3d3558b92149246d5c3292"]="v0.7.1" ["a2ae18d1dbd1e59c38ce22fa25ddffbd1d3084e3"]="v0.7" ["d4eb5a6eafdd4803d09e6f3963918216cca5a81f"]="v0.6.6" ["70d17cd69b80e7627dcad8fd5b6438f2309ac307"]="v0.6.5" ["58f376002e0c654fbc2de127765fa297cf694a33"]="v0.6.4" ["d5f9caa82d80cdcc7f1b7748f2cf3ccbf94f82a3"]="v0.6.3" ["095ef5f9f60fca1b269aabcc3cfd322b17b9e674"]="v0.6.2" ["cf5c4aeacb1703d0ffd35bdb5297d3494fee9a22"]="v0.6.1" ["60a20537c5f3fcc7f11946d81aebc8f90c08c117"]="v0.6" ["dd34ccfe288ebda8e2568cf550994ae890379f45"]="v0.5.13" ["3f9059fea886f1fa3b0c19a62a981d891dcc84eb"]="v0.5.12" ["2f92d577f995ff6ae1945ef6b89df3cb69b92999"]="v0.5.11" ["5a9d89ed42ddcd209d001a10d7eb828ef0e6d9de"]="v0.5.10" ["0207a970400ee28d3e366f2e8f5c551281accf02"]="v0.5.9" ["b967da5fc1f7a07e3561db072dd714d325e857b0"]="v0.5.8" ["210db686bb89f8696aa040e6e16de65424b808c9"]="v0.5.7" ["860a48765d1016ba226fb2c64aea35a45fe40e4a"]="v0.5.6" ["8f3539a27b28851153454eb737da9624cccaed2d"]="v0.5.5" ["a422246dc30bce11e970514f13f7c110f4470cc3"]="v0.5.4" ["285c52eb693265a0a530543e9ca0aeb593a2a55e"]="v0.5.3" ["0129a8a4ff8da5314e8e4d4d3336e89667ff6d54"]="v0.5.2" ["6f3d10a4c475c4c4509f0b370805419acd13912d"]="v0.5.1" ["de33bc46452b1046387ee2b3a03191b2c71135fb"]="v0.5" ["ae5cb7a0dab8b1bed9d52292f9b4e8e66a0f8ec9"]="v0.4.7" ["c6df34f55ba8c5a911b60d3f9eb20e3fa45f68c1"]="v0.4.6" ["37285038d3f91fa1b49159c4a35a8383168e644f"]="v0.4.5" ["9a9ff839a9b70cb2601d7696af743f5652395389"]="v0.4.4" ["28c0797d30175043bbfa31307b63aab4197cf996"]="v0.4.2" ["4474b9b3718653aeb0aee26422caefb90460cc0e"]="v0.4.1" ["da52d065a4c4f52d6017a537f3a80326f5af8bdc"]="v0.4.0.2" ["9d3963559ae7b15193057937ff3e72481899f40d"]="v0.3.5" ["1b8c44b5067525a5d266b6e99799d8097da76a29"]="v0.3.4" ["5cf91d0496688fed4f2a6c7021349b1fc0e057a2"]="v0.3.3" ["7fe46f1e1df5dec08a940451ba0feefd5c039165"]="v0.3.2" ["41e3a0f699f5c39cb61a15c0eb7a4aa816d47c24"]="v0.3.1" ["c5d8aec28b5230d34ae4b677c2091cc3dec7e3e8"]="v0.3.0" ["693bcb0f83478f2651db6bac9be5ca5ad60d03f3"]="v0.2.9" ["95a349abcc050712c50d4d85a1c8a804eee7f6c2"]="v0.2.8" ["c6ba5dc5391d3ca6cda479bf1923b88ce45509a0"]="v0.2.7" ["6c3afeec0fb439070b2912978b8dbb659033b1d9"]="v0.2.6" ["29c58b45882ac79595356caf98580c1d2a626011"]="v0.2.5" ["ecc565aa3fdc4c7e719aadc000e1fdc4d80d4fe0"]="v0.2.4" ["adaa4ed350acda4067fc0b455ad15b54cdf4c768"]="v0.2.3" ["a64b9aa9b8cb5863c917b6926516291a63c02fe5"]="v0.2.2" ["17d9becd3c673091b22f09aa02559a9ed9230f50"]="v0.2.1" ["449b482cc3236ccf31829830b4f6a44b2dcc06c2"]="v0.2" ["e94a30bec07e719c5a7b037ca1f4db8312702cce"]="v0.1" )
for tag in "${!TAGS[@]}"; do git tag -f "${TAGS[$tag]}" "$tag" ; done
# get master root commit
ROOT_COMMIT=$(git rev-list --max-parents=0 HEAD | tail -n 1)
# link master and devel
git replace --graft $ROOT_COMMIT v0.7.1
git-filter-repo --prune-empty never --force --commit-callback 'h=commit.original_id.decode("utf-8");m=commit.message.decode("utf-8");commit.message=str.encode(m + "\n" + "old-commit-hash: " + h)'
# delete replace refs
git for-each-ref --format='delete %(refname)' refs/replace | git update-ref --stdin
# machine validation
tail -n +2 "filter-repo/commit-map" | tr ' ' '\n' | xargs -P $(nproc) -n 2 bash -c 'H1=$(cd ../openpilot_backup && git ls-tree -r $0 | sha1sum) && H2=$(git ls-tree -r $1 | sha1sum) && echo "$H1 $H2" >> /tmp/GIT_HASHES && diff <(echo $H1) <(echo $H2) || exit 255'
# human validation
less /tmp/GIT_HASH
# cleanup
git reflog expire --expire=now --all
git gc --prune=now --aggressive
# get all lfs files
set +e
git config lfs.url https://github.com/commaai/openpilot.git/info/lfs
git lfs fetch --all
git config lfs.url https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
git lfs fetch --all
set -e
# add new files to lfs
git lfs migrate import --everything --include="*.ico,*.dlc,*.onnx,*.svg,*.png,*.gif,*.ttf,*.wav,system/hardware/tici/updater,selfdrive/ui/qt/spinner_larch64,selfdrive/ui/qt/text_larch64,third_party/**/*.a,third_party/**/*.so,third_party/**/*.so.*,third_party/**/*.dylib,third_party/acados/*/t_renderer,third_party/qt5/larch64/bin/lrelease,third_party/qt5/larch64/bin/lupdate,third_party/catch2/include/catch2/catch.hpp,*.apk,*.apkpatch,*.jar,*.pdf,*.jpg,*.mp3,*.thneed,*.tar.gz,*.npy,*.csv,*.a,*.so*,*.dylib,*.o,*.b64,selfdrive/hardware/tici/updater,selfdrive/boardd/tests/test_boardd,selfdrive/ui/qt/spinner_aarch64,installer/updater/updater,selfdrive/debug/profiling/simpleperf/**/*,selfdrive/hardware/eon/updater,selfdrive/ui/qt/text_aarch64,selfdrive/debug/profiling/pyflame/**/*,installer/installers/installer_openpilot,installer/installers/installer_dashcam,selfdrive/ui/text/text,selfdrive/ui/android/text/text,selfdrive/ui/spinner/spinner,selfdrive/visiond/visiond,selfdrive/loggerd/loggerd,selfdrive/sensord/sensord,selfdrive/sensord/gpsd,selfdrive/ui/android/spinner/spinner,selfdrive/ui/qt/spinner,selfdrive/ui/qt/text,_stringdefs.py,dfu-util-aarch64-linux,dfu-util-aarch64,dfu-util-x86_64-linux,dfu-util-x86_64,stb_image.h,clpeak3,clwaste,apk/**/*,external/**/*,phonelibs/**/*,third_party/boringssl/**/*,pyextra/**/*,panda/board/**/inc/*.h,panda/board/obj/*.elf,board/inc/*.h,third_party/nanovg/**/*,selfdrive/controls/lib/lateral_mpc/lib_mpc_export/**/*,pyextra/**/*,third_party/android_hardware_libhardware/**/*,selfdrive/controls/lib/lead_mpc_lib/lib_mpc_export/**/*,*.pro,selfdrive/controls/lib/longitudinal_mpc/lib_mpc_export/**/*,selfdrive/controls/lib/lateral_mpc/mpc_export/**/*,third_party/curl/**/*,selfdrive/modeld/thneed/debug/**/*,selfdrive/modeld/thneed/include/**/*,third_party/openmax/**/*,selfdrive/controls/lib/longitudinal_mpc/mpc_export/**/*,selfdrive/controls/lib/longitudinal_mpc_model/lib_mpc_export/**/*,Pipfile,Pipfile.lock,poetry.lock,*.qm"
# set new lfs endpoint
git config lfs.url https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
git config lfs.pushurl ssh://git@gitlab.com/commaai/openpilot-lfs.git
# push all branch+tag (scary stuff...)
git push -f --set-upstream git@github.com:commaai/openpilot.git +refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/*

View File

@@ -0,0 +1,82 @@
v0.1 e94a30bec07e719c5a7b037ca1f4db8312702cce
v0.2 449b482cc3236ccf31829830b4f6a44b2dcc06c2
v0.2.1 17d9becd3c673091b22f09aa02559a9ed9230f50
v0.2.2 a64b9aa9b8cb5863c917b6926516291a63c02fe5
v0.2.3 adaa4ed350acda4067fc0b455ad15b54cdf4c768
v0.2.4 ecc565aa3fdc4c7e719aadc000e1fdc4d80d4fe0
v0.2.5 29c58b45882ac79595356caf98580c1d2a626011
v0.2.6 6c3afeec0fb439070b2912978b8dbb659033b1d9
v0.2.7 c6ba5dc5391d3ca6cda479bf1923b88ce45509a0
v0.2.8 95a349abcc050712c50d4d85a1c8a804eee7f6c2
v0.2.9 693bcb0f83478f2651db6bac9be5ca5ad60d03f3
v0.3.0 c5d8aec28b5230d34ae4b677c2091cc3dec7e3e8
v0.3.1 41e3a0f699f5c39cb61a15c0eb7a4aa816d47c24
v0.3.2 7fe46f1e1df5dec08a940451ba0feefd5c039165
v0.3.3 5cf91d0496688fed4f2a6c7021349b1fc0e057a2
v0.3.4 1b8c44b5067525a5d266b6e99799d8097da76a29
v0.3.5 b111277f464cf66fa34b67819a83ea683e0f64df
v0.4.0.2 da52d065a4c4f52d6017a537f3a80326f5af8bdc
v0.4.1 4474b9b3718653aeb0aee26422caefb90460cc0e
v0.4.2 28c0797d30175043bbfa31307b63aab4197cf996
v0.4.4 9a9ff839a9b70cb2601d7696af743f5652395389
v0.4.5 37285038d3f91fa1b49159c4a35a8383168e644f
v0.4.6 c6df34f55ba8c5a911b60d3f9eb20e3fa45f68c1
v0.4.7 ae5cb7a0dab8b1bed9d52292f9b4e8e66a0f8ec9
v0.5 de33bc46452b1046387ee2b3a03191b2c71135fb
v0.5.1 8f22f52235c48eada586795ac57edb22688e4d08
v0.5.2 0129a8a4ff8da5314e8e4d4d3336e89667ff6d54
v0.5.3 285c52eb693265a0a530543e9ca0aeb593a2a55e
v0.5.4 a422246dc30bce11e970514f13f7c110f4470cc3
v0.5.5 8f3539a27b28851153454eb737da9624cccaed2d
v0.5.6 860a48765d1016ba226fb2c64aea35a45fe40e4a
v0.5.7 9ce3045f139ee29bf0eea5ec59dfe7df9c3d2c51
v0.5.8 2cee2e05ba0f3824fdbb8b957958800fa99071a1
v0.5.9 ad145da3bcded0fe75306df02061d07a633963c3
v0.5.10 ff4c1557d8358f158f4358788ff18ef93d2470ef
v0.5.11 d1866845df423c6855e2b365ff230cf7d89a420b
v0.5.12 f6e8ef27546e9a406724841e75f8df71cc4c2c97
v0.5.13 dd34ccfe288ebda8e2568cf550994ae890379f45
v0.6 60a20537c5f3fcc7f11946d81aebc8f90c08c117
v0.6.1 cf5c4aeacb1703d0ffd35bdb5297d3494fee9a22
v0.6.2 095ef5f9f60fca1b269aabcc3cfd322b17b9e674
v0.6.3 d5f9caa82d80cdcc7f1b7748f2cf3ccbf94f82a3
v0.6.4 58f376002e0c654fbc2de127765fa297cf694a33
v0.6.5 70d17cd69b80e7627dcad8fd5b6438f2309ac307
v0.6.6 d4eb5a6eafdd4803d09e6f3963918216cca5a81f
v0.7 a2ae18d1dbd1e59c38ce22fa25ddffbd1d3084e3
v0.7.1 1e1de64a1e59476b7b3d3558b92149246d5c3292
v0.7.2 59bd58c940673b4c4a6a86f299022614bcf42b22
v0.7.3 d7acd8b68f8131e0e714400cf124a3e228638643
v0.7.4 e93649882c5e914eec4a8b8b593dc0587e497033
v0.7.5 8abc0afe464626a461d2c7e192c912eeebeccc65
v0.7.6 69aacd9d179fe6dd3110253a099c38b34cff7899
v0.7.7 f1caed7299cdba5e45635d8377da6cc1e5fd7072
v0.7.8 2189fe8741b635d8394d55dee28959425cfd5ad0
v0.7.9 86dc54b836a973f132ed26db9f5a60b29f9b25b2
v0.7.10 47a42ff432db8a2494e922ca5e767e58020f0446
v0.7.11 f46ed718ba8d6bb4d42cd7b0f0150c406017c373
v0.8 d56e04c0d960c8d3d4ab88b578dc508a2b4e07dc
v0.8.1 cd6f26664cb8d32a13847d6648567c47c580e248
v0.8.2 7cc0999aebfe63b6bb6dd83c1dff62c3915c4820
v0.8.3 986500fe2f10870018f1fba1e5465476b8915977
v0.8.4 f0d0b82b8d6f5f450952113e234d0a5a49e80c48
v0.8.5 f5d9ddc6c2a2802a61e5ce590c6b6688bf736a69
v0.8.6 75904ed7452c6cbfb2a70cd379a899d8a75b97c2
v0.8.7 4f9e568019492126e236da85b5ca0a059f292900
v0.8.8 a949a49d5efaaf2d881143d23e9fb5ff9e28e88c
v0.8.9 a034926264cd1025c69d6ceb3fe444965f960b75
v0.8.10 59accdd814398b884167c0f41dbf46dcccf0c29c
v0.8.11 d630ec9092f039cb5e51c5dd6d92fc47b91407e4
v0.8.12 57871c99031cf597ffa0d819057ac1401e129f32
v0.8.13 e43e6e876513450d235124fcb711f1724ed9814c
v0.8.14 71901c94dbbaa2f9f156a80c14cc7ea65219fc7c
v0.8.15 5a7c2f90361e72e9c35e88abd2e11acdc4aba354
v0.8.16 f41dc62a12cc0f3cb8c5453c0caa0ba21e1bd01e
v0.9.0 58b84fb401a804967aa0dd5ee66fafa90194fd30
v0.9.1 89f68bf0cbf53a81b0553d3816fdbe522f941fa1
v0.9.2 c7d3b28b93faa6c955fb24bc64031512ee985ee9
v0.9.3 8704c1ff952b5c85a44f50143bbd1a4f7b4887e2
v0.9.4 fa310d9e2542cf497d92f007baec8fd751ffa99c
v0.9.5 3b1e9017c560499786d8a0e46aaaeea65037acac
v0.9.6 0b4d08fab8e35a264bc7383e878538f8083c33e5
v0.9.7 f8cb04e4a8b032b72a909f68b808a50936184bee

View File

@@ -57,7 +57,7 @@ function run_tests() {
if [[ -z "$FAST" ]]; then
run "mypy" mypy $PYTHON_FILES
run "codespell" codespell $ALL_FILES --ignore-words=$ROOT/.codespellignore
run "codespell" codespell $ALL_FILES
fi
return $FAILED

View File

@@ -1,27 +0,0 @@
#!/usr/bin/env python3
import argparse
import multiprocessing
from openpilot.system.hardware import HARDWARE
def main():
parser = argparse.ArgumentParser(description='Control power saving mode')
parser.add_argument('--enable', action='store_true', help='Enable power saving mode')
parser.add_argument('--disable', action='store_true', help='Disable power saving mode')
args = parser.parse_args()
if args.enable and args.disable:
parser.error("Cannot specify both --enable and --disable")
elif not (args.enable or args.disable):
parser.error("Must specify either --enable or --disable")
print(f"Number of CPU cores available before: [{multiprocessing.cpu_count()}]")
HARDWARE.set_power_save(args.enable)
state = "enabled" if args.enable else "disabled"
print(f"Power save mode set to: [{state}]")
print(f"Number of CPU cores available now: [{multiprocessing.cpu_count()}]")
if __name__ == "__main__":
main()

14
scripts/pyqt_demo.py Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env python3
from PyQt5.QtWidgets import QApplication, QLabel
from openpilot.selfdrive.ui.qt.python_helpers import set_main_window
if __name__ == "__main__":
app = QApplication([])
label = QLabel('Hello World!')
# Set full screen and rotate
set_main_window(label)
app.exec_()

Binary file not shown.

View File

@@ -42,19 +42,19 @@ class CarSpecificEvents:
self.cruise_buttons: deque = deque([], maxlen=HYUNDAI_PREV_BUTTON_SAMPLES)
def update(self, CS: car.CarState, CS_prev: car.CarState, CC: car.CarControl):
if self.CP.brand in ('body', 'mock'):
if self.CP.carName in ('body', 'mock'):
events = Events()
elif self.CP.brand in ('subaru', 'mazda'):
elif self.CP.carName in ('subaru', 'mazda'):
events = self.create_common_events(CS, CS_prev)
elif self.CP.brand == 'ford':
elif self.CP.carName == 'ford':
events = self.create_common_events(CS, CS_prev, extra_gears=[GearShifter.manumatic])
elif self.CP.brand == 'nissan':
elif self.CP.carName == 'nissan':
events = self.create_common_events(CS, CS_prev, extra_gears=[GearShifter.brake])
elif self.CP.brand == 'chrysler':
elif self.CP.carName == 'chrysler':
events = self.create_common_events(CS, CS_prev, extra_gears=[GearShifter.low])
# Low speed steer alert hysteresis logic
@@ -65,7 +65,7 @@ class CarSpecificEvents:
if self.low_speed_alert:
events.add(EventName.belowSteerSpeed)
elif self.CP.brand == 'honda':
elif self.CP.carName == 'honda':
events = self.create_common_events(CS, CS_prev, pcm_enable=False)
if self.CP.pcmCruise and CS.vEgo < self.CP.minEnableSpeed:
@@ -86,7 +86,7 @@ class CarSpecificEvents:
if self.CP.minEnableSpeed > 0 and CS.vEgo < 0.001:
events.add(EventName.manualRestart)
elif self.CP.brand == 'toyota':
elif self.CP.carName == 'toyota':
events = self.create_common_events(CS, CS_prev)
if self.CP.openpilotLongitudinalControl:
@@ -101,7 +101,7 @@ class CarSpecificEvents:
# while in standstill, send a user alert
events.add(EventName.manualRestart)
elif self.CP.brand == 'gm':
elif self.CP.carName == 'gm':
# The ECM allows enabling on falling edge of set, but only rising edge of resume
events = self.create_common_events(CS, CS_prev, extra_gears=[GearShifter.sport, GearShifter.low,
GearShifter.eco, GearShifter.manumatic],
@@ -120,9 +120,9 @@ class CarSpecificEvents:
if CS.vEgo < self.CP.minSteerSpeed:
events.add(EventName.belowSteerSpeed)
elif self.CP.brand == 'volkswagen':
elif self.CP.carName == 'volkswagen':
events = self.create_common_events(CS, CS_prev, extra_gears=[GearShifter.eco, GearShifter.sport, GearShifter.manumatic],
pcm_enable=self.CP.pcmCruise,
pcm_enable=not self.CP.openpilotLongitudinalControl,
enable_buttons=(ButtonType.setCruise, ButtonType.resumeCruise))
# Low speed steer alert hysteresis logic
@@ -143,13 +143,13 @@ class CarSpecificEvents:
# if CC.eps_timer_soft_disable_alert: # type: ignore[attr-defined]
# events.add(EventName.steerTimeLimit)
elif self.CP.brand == 'hyundai':
elif self.CP.carName == 'hyundai':
# On some newer model years, the CANCEL button acts as a pause/resume button based on the PCM state
# To avoid re-engaging when openpilot cancels, check user engagement intention via buttons
# Main button also can trigger an engagement on these cars
self.cruise_buttons.append(any(ev.type in HYUNDAI_ENABLE_BUTTONS for ev in CS.buttonEvents))
events = self.create_common_events(CS, CS_prev, extra_gears=(GearShifter.sport, GearShifter.manumatic),
pcm_enable=self.CP.pcmCruise, allow_enable=any(self.cruise_buttons), allow_button_cancel=False)
pcm_enable=self.CP.pcmCruise, allow_enable=any(self.cruise_buttons))
# low speed steer alert hysteresis logic (only for cars with steer cut off above 10 m/s)
if CS.vEgo < (self.CP.minSteerSpeed + 2.) and self.CP.minSteerSpeed > 10.:
@@ -160,12 +160,12 @@ class CarSpecificEvents:
events.add(EventName.belowSteerSpeed)
else:
raise ValueError(f"Unsupported car: {self.CP.brand}")
raise ValueError(f"Unsupported car: {self.CP.carName}")
return events
def create_common_events(self, CS: structs.CarState, CS_prev: car.CarState, extra_gears=None, pcm_enable=True,
allow_enable=True, allow_button_cancel=True, enable_buttons=(ButtonType.accelCruise, ButtonType.decelCruise)):
allow_enable=True, enable_buttons=(ButtonType.accelCruise, ButtonType.decelCruise)):
events = Events()
if CS.doorOpen:
@@ -216,8 +216,7 @@ class CarSpecificEvents:
if not self.CP.pcmCruise and (b.type in enable_buttons and not b.pressed):
events.add(EventName.buttonEnable)
# Disable on rising and falling edge of cancel for both stock and OP long
# TODO: only check the cancel button with openpilot longitudinal on all brands to match panda safety
if b.type == ButtonType.cancel and (allow_button_cancel or not self.CP.pcmCruise):
if b.type == ButtonType.cancel:
events.add(EventName.buttonCancel)
# Handle permanent and temporary steering faults

View File

@@ -1,12 +1,11 @@
#!/usr/bin/env python3
import json
import os
import time
import threading
import cereal.messaging as messaging
from cereal import car, log, custom
from cereal import car, log
from panda import ALTERNATIVE_EXPERIENCE
@@ -22,10 +21,6 @@ from opendbc.car.interfaces import CarInterfaceBase, RadarInterfaceBase
from openpilot.selfdrive.pandad import can_capnp_to_list, can_list_to_can_capnp
from openpilot.selfdrive.car.cruise import VCruiseHelper
from openpilot.selfdrive.car.car_specific import MockCarState
from openpilot.selfdrive.car.helpers import convert_carControlSP, convert_to_capnp
from openpilot.sunnypilot.mads.mads import MadsParams
from openpilot.sunnypilot.selfdrive.car import interfaces
REPLAY = "REPLAY" in os.environ
@@ -68,18 +63,15 @@ class Car:
CI: CarInterfaceBase
RI: RadarInterfaceBase
CP: car.CarParams
CP_SP: structs.CarParamsSP
CP_SP_capnp: custom.CarParamsSP
def __init__(self, CI=None, RI=None) -> None:
self.can_sock = messaging.sub_sock('can', timeout=20)
self.sm = messaging.SubMaster(['pandaStates', 'carControl', 'onroadEvents'] + ['carControlSP'])
self.pm = messaging.PubMaster(['sendcan', 'carState', 'carParams', 'carOutput', 'liveTracks'] + ['carParamsSP'])
self.sm = messaging.SubMaster(['pandaStates', 'carControl', 'onroadEvents'])
self.pm = messaging.PubMaster(['sendcan', 'carState', 'carParams', 'carOutput', 'liveTracks'])
self.can_rcv_cum_timeout_counter = 0
self.CC_prev = car.CarControl.new_message()
self.CS_prev = car.CarState.new_message()
self.initialized_prev = False
self.last_actuators_output = structs.CarControl.Actuators()
@@ -105,18 +97,14 @@ class Car:
with car.CarParams.from_bytes(cached_params_raw) as _cached_params:
cached_params = _cached_params
fixed_fingerprint = json.loads(self.params.get("CarPlatformBundle", encoding='utf-8') or "{}").get("platform", None)
self.CI = get_car(*self.can_callbacks, obd_callback(self.params), experimental_long_allowed, num_pandas, cached_params, fixed_fingerprint)
interfaces.setup_car_interface_sp(self.CI.CP, self.CI.CP_SP, self.params)
self.RI = get_radar_interface(self.CI.CP, self.CI.CP_SP)
self.CI = get_car(*self.can_callbacks, obd_callback(self.params), experimental_long_allowed, num_pandas, cached_params)
self.RI = get_radar_interface(self.CI.CP)
self.CP = self.CI.CP
self.CP_SP = self.CI.CP_SP
# continue onto next fingerprinting step in pandad
self.params.put_bool("FirmwareQueryDone", True)
else:
self.CI, self.CP, self.CP_SP = CI, CI.CP, CI.CP_SP
self.CI, self.CP = CI, CI.CP
self.RI = RI
# set alternative experiences from parameters
@@ -125,13 +113,6 @@ class Car:
if not disengage_on_accelerator:
self.CP.alternativeExperience |= ALTERNATIVE_EXPERIENCE.DISABLE_DISENGAGE_ON_GAS
# mads
MadsParams().set_alternative_experience(self.CP)
MadsParams().set_car_specific_params(self.CP, self.CP_SP)
# Dynamic Experimental Control
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
openpilot_enabled_toggle = self.params.get_bool("OpenpilotEnabledToggle")
controller_available = self.CI.CC is not None and openpilot_enabled_toggle and not self.CP.dashcamOnly
@@ -143,15 +124,6 @@ class Car:
self.CP.safetyConfigs = [safety_config]
if self.CP.secOcRequired and not self.params.get_bool("IsReleaseBranch"):
# Copy user key if available
try:
with open("/cache/params/SecOCKey") as f:
user_key = f.readline().strip()
if len(user_key) == 32:
self.params.put("SecOCKey", user_key)
except Exception:
pass
secoc_key = self.params.get("SecOCKey", encoding='utf8')
if secoc_key is not None:
saved_secoc_key = bytes.fromhex(secoc_key.strip())
@@ -174,14 +146,6 @@ class Car:
self.params.put_nonblocking("CarParamsCache", cp_bytes)
self.params.put_nonblocking("CarParamsPersistent", cp_bytes)
# Write CarParamsSP for controls
# convert to pycapnp representation for caching and logging
self.CP_SP_capnp = convert_to_capnp(self.CP_SP)
cp_sp_bytes = self.CP_SP_capnp.to_bytes()
self.params.put("CarParamsSP", cp_sp_bytes)
self.params.put_nonblocking("CarParamsSPCache", cp_sp_bytes)
self.params.put_nonblocking("CarParamsSPPersistent", cp_sp_bytes)
self.mock_carstate = MockCarState()
self.v_cruise_helper = VCruiseHelper(self.CP)
@@ -191,9 +155,6 @@ class Car:
# card is driven by can recv, expected at 100Hz
self.rk = Ratekeeper(100, print_delay_threshold=None)
# log fingerprint in sentry
interfaces.log_fingerprint(self.CP)
def state_update(self) -> tuple[car.CarState, structs.RadarDataT | None]:
"""carState update loop, driven by can"""
@@ -202,7 +163,7 @@ class Car:
# Update carState from CAN
CS = self.CI.update(can_list)
if self.CP.brand == 'mock':
if self.CP.carName == 'mock':
CS = self.mock_carstate.update(CS)
# Update radar tracks from CAN
@@ -219,12 +180,8 @@ class Car:
if can_rcv_valid and REPLAY:
self.can_log_mono_time = messaging.log_from_bytes(can_strs[0]).logMonoTime
self.v_cruise_helper.update_v_cruise(CS, self.sm['carControl'].enabled, self.is_metric)
if self.sm['carControl'].enabled and not self.CC_prev.enabled:
# Use CarState w/ buttons from the step selfdrived enables on
self.v_cruise_helper.initialize_v_cruise(self.CS_prev, self.experimental_mode, self.dynamic_experimental_control)
# TODO: mirror the carState.cruiseState struct?
self.v_cruise_helper.update_v_cruise(CS, self.sm['carControl'].enabled, self.is_metric)
CS.vCruise = float(self.v_cruise_helper.v_cruise_kph)
CS.vCruiseCluster = float(self.v_cruise_helper.v_cruise_cluster_kph)
@@ -260,28 +217,20 @@ class Car:
tracks_msg.liveTracks = RD
self.pm.send('liveTracks', tracks_msg)
# carParamsSP - logged every 50 seconds (> 1 per segment)
if self.sm.frame % int(50. / DT_CTRL) == 0:
cp_sp_send = messaging.new_message('carParamsSP')
cp_sp_send.valid = True
cp_sp_send.carParamsSP = self.CP_SP_capnp
self.pm.send('carParamsSP', cp_sp_send)
def controls_update(self, CS: car.CarState, CC: car.CarControl, CC_SP: custom.CarControlSP):
def controls_update(self, CS: car.CarState, CC: car.CarControl):
"""control update loop, driven by carControl"""
if not self.initialized_prev:
# Initialize CarInterface, once controls are ready
# TODO: this can make us miss at least a few cycles when doing an ECU knockout
self.CI.init(self.CP, self.CP_SP, *self.can_callbacks)
interfaces.initialize_car_interface_sp(self.CP, self.CP_SP, self.params, *self.can_callbacks)
self.CI.init(self.CP, *self.can_callbacks)
# signal pandad to switch to car safety mode
self.params.put_bool_nonblocking("ControlsReady", True)
if self.sm.all_alive(['carControl']):
# send car controls over can
now_nanos = self.can_log_mono_time if REPLAY else int(time.monotonic() * 1e9)
self.last_actuators_output, can_sends = self.CI.apply(CC, convert_carControlSP(CC_SP), now_nanos)
self.last_actuators_output, can_sends = self.CI.apply(CC, now_nanos)
self.pm.send('sendcan', can_list_to_can_capnp(can_sends, msgtype='sendcan', valid=CS.canValid))
self.CC_prev = CC
@@ -289,24 +238,22 @@ class Car:
def step(self):
CS, RD = self.state_update()
if self.sm['carControl'].enabled and not self.CC_prev.enabled:
self.v_cruise_helper.initialize_v_cruise(CS, self.experimental_mode)
self.state_publish(CS, RD)
initialized = (not any(e.name == EventName.selfdriveInitializing for e in self.sm['onroadEvents']) and
self.sm.seen['onroadEvents'])
if not self.CP.passive and initialized:
self.controls_update(CS, self.sm['carControl'], self.sm['carControlSP'])
self.controls_update(CS, self.sm['carControl'])
self.initialized_prev = initialized
self.CS_prev = CS
def params_thread(self, evt):
while not evt.is_set():
self.is_metric = self.params.get_bool("IsMetric")
self.experimental_mode = self.params.get_bool("ExperimentalMode") and self.CP.openpilotLongitudinalControl
# sunnypilot
self.dynamic_experimental_control = self.params.get_bool("DynamicExperimentalControl")
time.sleep(0.1)
def card_thread(self):

View File

@@ -1,8 +1,8 @@
import math
import numpy as np
from cereal import car
from openpilot.common.conversions import Conversions as CV
from openpilot.common.numpy_fast import clip
# WARNING: this value was determined based on the model's training distribution,
@@ -106,7 +106,7 @@ class VCruiseHelper:
if CS.gasPressed and button_type in (ButtonType.decelCruise, ButtonType.setCruise):
self.v_cruise_kph = max(self.v_cruise_kph, CS.vEgo * CV.MS_TO_KPH)
self.v_cruise_kph = np.clip(round(self.v_cruise_kph, 1), V_CRUISE_MIN, V_CRUISE_MAX)
self.v_cruise_kph = clip(round(self.v_cruise_kph, 1), V_CRUISE_MIN, V_CRUISE_MAX)
def update_button_timers(self, CS, enabled):
# increment timer for buttons still pressed
@@ -120,17 +120,16 @@ class VCruiseHelper:
self.button_timers[b.type.raw] = 1 if b.pressed else 0
self.button_change_states[b.type.raw] = {"standstill": CS.cruiseState.standstill, "enabled": enabled}
def initialize_v_cruise(self, CS, experimental_mode: bool, dynamic_experimental_control: bool) -> None:
def initialize_v_cruise(self, CS, experimental_mode: bool) -> None:
# initializing is handled by the PCM
if self.CP.pcmCruise:
return
initial_experimental_mode = experimental_mode and not dynamic_experimental_control
initial = V_CRUISE_INITIAL_EXPERIMENTAL_MODE if initial_experimental_mode else V_CRUISE_INITIAL
initial = V_CRUISE_INITIAL_EXPERIMENTAL_MODE if experimental_mode else V_CRUISE_INITIAL
if any(b.type in (ButtonType.accelCruise, ButtonType.resumeCruise) for b in CS.buttonEvents) and self.v_cruise_initialized:
self.v_cruise_kph = self.v_cruise_kph_last
else:
self.v_cruise_kph = int(round(np.clip(CS.vEgo * CV.MS_TO_KPH, initial, V_CRUISE_MAX)))
self.v_cruise_kph = int(round(clip(CS.vEgo * CV.MS_TO_KPH, initial, V_CRUISE_MAX)))
self.v_cruise_cluster_kph = self.v_cruise_kph

View File

@@ -1,59 +0,0 @@
import capnp
from typing import Any
from cereal import custom
from opendbc.car import structs
_FIELDS = '__dataclass_fields__' # copy of dataclasses._FIELDS
def is_dataclass(obj):
"""Similar to dataclasses.is_dataclass without instance type check checking"""
return hasattr(obj, _FIELDS)
def _asdictref_inner(obj) -> dict[str, Any] | Any:
if is_dataclass(obj):
ret = {}
for field in getattr(obj, _FIELDS): # similar to dataclasses.fields()
ret[field] = _asdictref_inner(getattr(obj, field))
return ret
elif isinstance(obj, (tuple, list)):
return type(obj)(_asdictref_inner(v) for v in obj)
else:
return obj
def asdictref(obj) -> dict[str, Any]:
"""
Similar to dataclasses.asdict without recursive type checking and copy.deepcopy
Note that the resulting dict will contain references to the original struct as a result
"""
if not is_dataclass(obj):
raise TypeError("asdictref() should be called on dataclass instances")
return _asdictref_inner(obj)
def convert_to_capnp(struct: structs.CarParamsSP) -> capnp.lib.capnp._DynamicStructBuilder:
struct_dict = asdictref(struct)
if isinstance(struct, structs.CarParamsSP):
struct_capnp = custom.CarParamsSP.new_message(**struct_dict)
else:
raise ValueError(f"Unsupported struct type: {type(struct)}")
return struct_capnp
def convert_carControlSP(struct: capnp.lib.capnp._DynamicStructReader) -> structs.CarControlSP:
# TODO: recursively handle any car struct as needed
def remove_deprecated(s: dict) -> dict:
return {k: v for k, v in s.items() if not k.endswith('DEPRECATED')}
struct_dict = struct.to_dict()
struct_dataclass = structs.CarControlSP(**remove_deprecated({k: v for k, v in struct_dict.items() if not isinstance(k, dict)}))
struct_dataclass.mads = structs.ModularAssistiveDrivingSystem(**remove_deprecated(struct_dict.get('mads', {})))
return struct_dataclass

View File

@@ -4,7 +4,7 @@ import hypothesis.strategies as st
from hypothesis import Phase, given, settings
from parameterized import parameterized
from cereal import car, custom
from cereal import car
from opendbc.car import DT_CTRL
from opendbc.car.car_helpers import interfaces
from opendbc.car.structs import CarParams
@@ -12,7 +12,6 @@ from opendbc.car.tests.test_car_interfaces import get_fuzzy_car_interface_args
from opendbc.car.fingerprints import all_known_cars
from opendbc.car.fw_versions import FW_VERSIONS, FW_QUERY_CONFIGS
from opendbc.car.mock.values import CAR as MOCK
from openpilot.selfdrive.car.helpers import convert_carControlSP
from openpilot.selfdrive.controls.lib.latcontrol_angle import LatControlAngle
from openpilot.selfdrive.controls.lib.latcontrol_pid import LatControlPID
from openpilot.selfdrive.controls.lib.latcontrol_torque import LatControlTorque
@@ -41,12 +40,9 @@ class TestCarInterfaces:
car_params = CarInterface.get_params(car_name, args['fingerprints'], args['car_fw'],
experimental_long=args['experimental_long'], docs=False)
car_params_sp = CarInterface.get_params_sp(car_params, car_name, args['fingerprints'], args['car_fw'],
experimental_long=args['experimental_long'], docs=False)
car_params = car_params.as_reader()
car_interface = CarInterface(car_params, car_params_sp, CarController, CarState)
car_interface = CarInterface(car_params, CarController, CarState)
assert car_params
assert car_params_sp
assert car_interface
assert car_params.mass > 1
@@ -73,16 +69,13 @@ class TestCarInterfaces:
assert not math.isnan(tune.torque.friction) and tune.torque.friction > 0
cc_msg = FuzzyGenerator.get_random_msg(data.draw, car.CarControl, real_floats=True)
cc_sp_msg = FuzzyGenerator.get_random_msg(data.draw, custom.CarControlSP, real_floats=True)
# Run car interface
now_nanos = 0
CC = car.CarControl.new_message(**cc_msg)
CC = CC.as_reader()
CC_SP = custom.CarControlSP.new_message(**cc_sp_msg)
CC_SP = convert_carControlSP(CC_SP.as_reader())
for _ in range(10):
car_interface.update([])
car_interface.apply(CC, CC_SP, now_nanos)
car_interface.apply(CC, now_nanos)
now_nanos += DT_CTRL * 1e9 # 10 ms
CC = car.CarControl.new_message(**cc_msg)
@@ -90,7 +83,7 @@ class TestCarInterfaces:
CC = CC.as_reader()
for _ in range(10):
car_interface.update([])
car_interface.apply(CC, CC_SP, now_nanos)
car_interface.apply(CC, now_nanos)
now_nanos += DT_CTRL * 1e9 # 10ms
# Test controller initialization

View File

@@ -57,16 +57,16 @@ class TestVCruiseHelper:
for _ in range(2):
self.v_cruise_helper.update_v_cruise(car.CarState(cruiseState={"available": False}), enabled=False, is_metric=False)
def enable(self, v_ego, experimental_mode, dynamic_experimental_control):
def enable(self, v_ego, experimental_mode):
# Simulates user pressing set with a current speed
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), experimental_mode, dynamic_experimental_control)
self.v_cruise_helper.initialize_v_cruise(car.CarState(vEgo=v_ego), experimental_mode)
def test_adjust_speed(self):
"""
Asserts speed changes on falling edges of buttons.
"""
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
for btn in (ButtonType.accelCruise, ButtonType.decelCruise):
for pressed in (True, False):
@@ -90,7 +90,7 @@ class TestVCruiseHelper:
CS.buttonEvents = [ButtonEvent(type=ButtonType.decelCruise, pressed=pressed)]
self.v_cruise_helper.update_v_cruise(CS, enabled=enabled, is_metric=False)
if pressed:
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
# Expected diff on enabling. Speed should not change on falling edge of pressed
assert not pressed == self.v_cruise_helper.v_cruise_kph == self.v_cruise_helper.v_cruise_kph_last
@@ -100,7 +100,7 @@ class TestVCruiseHelper:
Asserts we don't increment set speed if user presses resume/accel to exit cruise standstill.
"""
self.enable(0, False, False)
self.enable(0, False)
for standstill in (True, False):
for pressed in (True, False):
@@ -120,7 +120,7 @@ class TestVCruiseHelper:
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False, False)
self.enable(V_CRUISE_INITIAL * CV.KPH_TO_MS, False)
# first decrement speed, then perform gas pressed logic
expected_v_cruise_kph = self.v_cruise_helper.v_cruise_kph - IMPERIAL_INCREMENT
@@ -142,11 +142,10 @@ class TestVCruiseHelper:
"""
for experimental_mode in (True, False):
for dynamic_experimental_control in (True, False):
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
for v_ego in np.linspace(0, 100, 101):
self.reset_cruise_speed_state()
assert not self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego), experimental_mode, dynamic_experimental_control)
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized
self.enable(float(v_ego), experimental_mode)
assert V_CRUISE_INITIAL <= self.v_cruise_helper.v_cruise_kph <= V_CRUISE_MAX
assert self.v_cruise_helper.v_cruise_initialized

View File

@@ -1,5 +1,4 @@
import capnp
import copy
import os
import pytest
import random
@@ -159,9 +158,7 @@ class TestCarModelBase(unittest.TestCase):
cls.CarInterface, cls.CarController, cls.CarState, cls.RadarInterface = interfaces[cls.platform]
cls.CP = cls.CarInterface.get_params(cls.platform, cls.fingerprint, car_fw, experimental_long, docs=False)
cls.CP_SP = cls.CarInterface.get_params_sp(cls.CP, cls.platform, cls.fingerprint, car_fw, experimental_long, docs=False)
assert cls.CP
assert cls.CP_SP
assert cls.CP.carFingerprint == cls.platform
os.environ["COMMA_CACHE"] = DEFAULT_DOWNLOAD_CACHE_ROOT
@@ -171,7 +168,7 @@ class TestCarModelBase(unittest.TestCase):
del cls.can_msgs
def setUp(self):
self.CI = self.CarInterface(self.CP.copy(), copy.deepcopy(self.CP_SP), self.CarController, self.CarState)
self.CI = self.CarInterface(self.CP.copy(), self.CarController, self.CarState)
assert self.CI
Params().put_bool("OpenpilotEnabledToggle", self.openpilot_enabled)
@@ -205,11 +202,10 @@ class TestCarModelBase(unittest.TestCase):
can_invalid_cnt = 0
can_valid = False
CC = structs.CarControl().as_reader()
CC_SP = structs.CarControlSP()
for i, msg in enumerate(self.can_msgs):
CS = self.CI.update(can_capnp_to_list((msg.as_builder().to_bytes(),)))
self.CI.apply(CC, CC_SP, msg.logMonoTime)
self.CI.apply(CC, msg.logMonoTime)
if CS.canValid:
can_valid = True
@@ -221,7 +217,7 @@ class TestCarModelBase(unittest.TestCase):
self.assertEqual(can_invalid_cnt, 0)
def test_radar_interface(self):
RI = self.RadarInterface(self.CP, self.CP_SP)
RI = self.RadarInterface(self.CP)
assert RI
# Since OBD port is multiplexed to bus 1 (commonly radar bus) while fingerprinting,
@@ -278,13 +274,13 @@ class TestCarModelBase(unittest.TestCase):
if self.CP.notCar:
self.skipTest("Skipping test for notCar")
def test_car_controller(car_control, car_control_sp):
def test_car_controller(car_control):
now_nanos = 0
msgs_sent = 0
CI = self.CarInterface(self.CP, self.CP_SP, self.CarController, self.CarState)
CI = self.CarInterface(self.CP, self.CarController, self.CarState)
for _ in range(round(10.0 / DT_CTRL)): # make sure we hit the slowest messages
CI.update([])
_, sendcan = CI.apply(car_control, car_control_sp, now_nanos)
_, sendcan = CI.apply(car_control, now_nanos)
now_nanos += DT_CTRL * 1e9
msgs_sent += len(sendcan)
@@ -297,18 +293,17 @@ class TestCarModelBase(unittest.TestCase):
# Make sure we can send all messages while inactive
CC = structs.CarControl()
CC_SP = structs.CarControlSP()
test_car_controller(CC.as_reader(), CC_SP)
test_car_controller(CC.as_reader())
# Test cancel + general messages (controls_allowed=False & cruise_engaged=True)
self.safety.set_cruise_engaged_prev(True)
CC = structs.CarControl(cruiseControl=structs.CarControl.CruiseControl(cancel=True))
test_car_controller(CC.as_reader(), CC_SP)
test_car_controller(CC.as_reader())
# Test resume + general messages (controls_allowed=True & cruise_engaged=True)
self.safety.set_controls_allowed(True)
CC = structs.CarControl(cruiseControl=structs.CarControl.CruiseControl(resume=True))
test_car_controller(CC.as_reader(), CC_SP)
test_car_controller(CC.as_reader())
# Skip stdout/stderr capture with pytest, causes elevated memory usage
@pytest.mark.nocapture
@@ -367,11 +362,11 @@ class TestCarModelBase(unittest.TestCase):
if self.safety.get_vehicle_moving() != prev_panda_vehicle_moving:
self.assertEqual(not CS.standstill, self.safety.get_vehicle_moving())
if not (self.CP.brand == "honda" and not (self.CP.flags & HondaFlags.BOSCH)):
if not (self.CP.carName == "honda" and not (self.CP.flags & HondaFlags.BOSCH)):
if self.safety.get_cruise_engaged_prev() != prev_panda_cruise_engaged:
self.assertEqual(CS.cruiseState.enabled, self.safety.get_cruise_engaged_prev())
if self.CP.brand == "honda":
if self.CP.carName == "honda":
if self.safety.get_acc_main_on() != prev_panda_acc_main_on:
self.assertEqual(CS.cruiseState.available, self.safety.get_acc_main_on())
@@ -392,7 +387,7 @@ class TestCarModelBase(unittest.TestCase):
controls_allowed_prev = False
CS_prev = car.CarState.new_message()
checks = defaultdict(int)
selfdrived = SelfdriveD(CP=self.CP, CP_SP=self.CP_SP)
selfdrived = SelfdriveD(CP=self.CP)
selfdrived.initialized = True
for idx, can in enumerate(self.can_msgs):
CS = self.CI.update(can_capnp_to_list((can.as_builder().to_bytes(), ))).as_reader()
@@ -426,7 +421,7 @@ class TestCarModelBase(unittest.TestCase):
# On most pcmCruise cars, openpilot's state is always tied to the PCM's cruise state.
# On Honda Nidec, we always engage on the rising edge of the PCM cruise state, but
# openpilot brakes to zero even if the min ACC speed is non-zero (i.e. the PCM disengages).
if self.CP.brand == "honda" and not (self.CP.flags & HondaFlags.BOSCH):
if self.CP.carName == "honda" and not (self.CP.flags & HondaFlags.BOSCH):
# only the rising edges are expected to match
if CS.cruiseState.enabled and not CS_prev.cruiseState.enabled:
checks['controlsAllowed'] += not self.safety.get_controls_allowed()
@@ -448,7 +443,7 @@ class TestCarModelBase(unittest.TestCase):
if button_enable and not mismatch:
self.safety.set_controls_allowed(False)
if self.CP.brand == "honda":
if self.CP.carName == "honda":
checks['mainOn'] += CS.cruiseState.available != self.safety.get_acc_main_on()
CS_prev = CS

View File

@@ -2,7 +2,7 @@
import math
from typing import SupportsFloat
from cereal import car, log, custom
from cereal import car, log
import cereal.messaging as messaging
from openpilot.common.conversions import Conversions as CV
from openpilot.common.params import Params
@@ -19,6 +19,7 @@ from openpilot.selfdrive.controls.lib.longcontrol import LongControl
from openpilot.selfdrive.controls.lib.vehicle_model import VehicleModel
from openpilot.selfdrive.locationd.helpers import PoseCalibrator, Pose
State = log.SelfdriveState.OpenpilotState
LaneChangeState = log.LaneChangeState
LaneChangeDirection = log.LaneChangeDirection
@@ -32,17 +33,12 @@ class Controls:
self.CP = messaging.log_from_bytes(self.params.get("CarParams", block=True), car.CarParams)
cloudlog.info("controlsd got CarParams")
cloudlog.info("controlsd is waiting for CarParamsSP")
self.CP_SP = messaging.log_from_bytes(self.params.get("CarParamsSP", block=True), custom.CarParamsSP)
cloudlog.info("controlsd got CarParamsSP")
self.CI = get_car_interface(self.CP, self.CP_SP)
self.CI = get_car_interface(self.CP)
self.sm = messaging.SubMaster(['liveParameters', 'liveTorqueParameters', 'modelV2', 'selfdriveState',
'liveCalibration', 'livePose', 'longitudinalPlan', 'carState', 'carOutput',
'driverMonitoringState', 'onroadEvents', 'driverAssistance'] + ['selfdriveStateSP'],
poll='selfdriveState')
self.pm = messaging.PubMaster(['carControl', 'controlsState'] + ['carControlSP'])
'driverMonitoringState', 'onroadEvents', 'driverAssistance'], poll='selfdriveState')
self.pm = messaging.PubMaster(['carControl', 'controlsState'])
self.steer_limited = False
self.desired_curvature = 0.0
@@ -92,14 +88,7 @@ class Controls:
# Check which actuators can be enabled
standstill = abs(CS.vEgo) <= max(self.CP.minSteerSpeed, MIN_LATERAL_CONTROL_SPEED) or CS.standstill
ss_sp = self.sm['selfdriveStateSP']
if ss_sp.mads.available:
_lat_active = ss_sp.mads.active
else:
_lat_active = self.sm['selfdriveState'].active
CC.latActive = _lat_active and not CS.steerFaultTemporary and not CS.steerFaultPermanent and not standstill
CC.latActive = self.sm['selfdriveState'].active and not CS.steerFaultTemporary and not CS.steerFaultPermanent and not standstill
CC.longActive = CC.enabled and not any(e.overrideLongitudinal for e in self.sm['onroadEvents']) and self.CP.openpilotLongitudinalControl
actuators = CC.actuators
@@ -117,16 +106,15 @@ class Controls:
# accel PID loop
pid_accel_limits = self.CI.get_pid_accel_limits(self.CP, CS.vEgo, CS.vCruise * CV.KPH_TO_MS)
actuators.accel = float(self.LoC.update(CC.longActive, CS, long_plan.aTarget, long_plan.shouldStop, pid_accel_limits))
actuators.accel = self.LoC.update(CC.longActive, CS, long_plan.aTarget, long_plan.shouldStop, pid_accel_limits)
# Steering PID loop and lateral MPC
self.desired_curvature = clip_curvature(CS.vEgo, self.desired_curvature, model_v2.action.desiredCurvature)
actuators.curvature = float(self.desired_curvature)
steer, steeringAngleDeg, lac_log = self.LaC.update(CC.latActive, CS, self.VM, lp,
actuators.curvature = self.desired_curvature
actuators.steer, actuators.steeringAngleDeg, lac_log = self.LaC.update(CC.latActive, CS, self.VM, lp,
self.steer_limited, self.desired_curvature,
self.calibrated_pose) # TODO what if not available
actuators.steer = float(steer)
actuators.steeringAngleDeg = float(steeringAngleDeg)
# Ensure no NaNs/Infs
for p in ACTUATOR_FIELDS:
attr = getattr(actuators, p)
@@ -137,12 +125,9 @@ class Controls:
cloudlog.error(f"actuators.{p} not finite {actuators.to_dict()}")
setattr(actuators, p, 0.0)
CC_SP = custom.CarControlSP.new_message()
CC_SP.mads = ss_sp.mads
return CC, lac_log
return CC, CC_SP, lac_log
def publish(self, CC, CC_SP, lac_log):
def publish(self, CC, lac_log):
CS = self.sm['carState']
# Orientation and angle rates can be useful for carcontroller
@@ -194,7 +179,7 @@ class Controls:
cs.longitudinalPlanMonoTime = self.sm.logMonoTime['longitudinalPlan']
cs.lateralPlanMonoTime = self.sm.logMonoTime['modelV2']
cs.desiredCurvature = float(self.desired_curvature)
cs.desiredCurvature = self.desired_curvature
cs.longControlState = self.LoC.long_control_state
cs.upAccelCmd = float(self.LoC.pid.p)
cs.uiAccelCmd = float(self.LoC.pid.i)
@@ -218,18 +203,12 @@ class Controls:
cc_send.carControl = CC
self.pm.send('carControl', cc_send)
# carControlSP
cc_sp_send = messaging.new_message('carControlSP')
cc_sp_send.valid = CS.canValid
cc_sp_send.carControlSP = CC_SP
self.pm.send('carControlSP', cc_sp_send)
def run(self):
rk = Ratekeeper(100, print_delay_threshold=None)
while True:
self.update()
CC, CC_SP, lac_log = self.state_control()
self.publish(CC, CC_SP, lac_log)
CC, lac_log = self.state_control()
self.publish(CC, lac_log)
rk.monitor_time()
def main():

View File

@@ -1,5 +1,5 @@
import numpy as np
from cereal import log
from openpilot.common.numpy_fast import clip
from openpilot.common.realtime import DT_CTRL
MIN_SPEED = 1.0
@@ -13,10 +13,10 @@ MAX_LATERAL_JERK = 5.0
MAX_VEL_ERR = 5.0
def clip_curvature(v_ego, prev_curvature, new_curvature):
new_curvature = np.clip(new_curvature, -MAX_CURVATURE, MAX_CURVATURE)
new_curvature = clip(new_curvature, -MAX_CURVATURE, MAX_CURVATURE)
v_ego = max(MIN_SPEED, v_ego)
max_curvature_rate = MAX_LATERAL_JERK / (v_ego**2) # inexact calculation, check https://github.com/commaai/openpilot/pull/24755
safe_desired_curvature = np.clip(new_curvature,
safe_desired_curvature = clip(new_curvature,
prev_curvature - max_curvature_rate * DT_CTRL,
prev_curvature + max_curvature_rate * DT_CTRL)
@@ -26,6 +26,6 @@ def clip_curvature(v_ego, prev_curvature, new_curvature):
def get_speed_error(modelV2: log.ModelDataV2, v_ego: float) -> float:
# ToDo: Try relative error, and absolute speed
if len(modelV2.temporalPose.trans):
vel_err = np.clip(modelV2.temporalPose.trans[0] - v_ego, -MAX_VEL_ERR, MAX_VEL_ERR)
vel_err = clip(modelV2.temporalPose.trans[0] - v_ego, -MAX_VEL_ERR, MAX_VEL_ERR)
return float(vel_err)
return 0.0

View File

@@ -1,6 +1,6 @@
import numpy as np
from abc import abstractmethod, ABC
from openpilot.common.numpy_fast import clip
from openpilot.common.realtime import DT_CTRL
MIN_LATERAL_CONTROL_SPEED = 0.3 # m/s
@@ -28,5 +28,5 @@ class LatControl(ABC):
self.sat_count += self.sat_count_rate
else:
self.sat_count -= self.sat_count_rate
self.sat_count = np.clip(self.sat_count, 0.0, self.sat_limit)
self.sat_count = clip(self.sat_count, 0.0, self.sat_limit)
return self.sat_count > (self.sat_limit - 1e-3)

View File

@@ -23,7 +23,7 @@ class LatControlAngle(LatControl):
angle_steers_des += params.angleOffsetDeg
angle_control_saturated = abs(angle_steers_des - CS.steeringAngleDeg) > STEER_ANGLE_SATURATION_THRESHOLD
angle_log.saturated = bool(self._check_saturation(angle_control_saturated, CS, False))
angle_log.saturated = self._check_saturation(angle_control_saturated, CS, False)
angle_log.steeringAngleDeg = float(CS.steeringAngleDeg)
angle_log.steeringAngleDesiredDeg = angle_steers_des
return 0, float(angle_steers_des), angle_log

View File

@@ -39,10 +39,10 @@ class LatControlPID(LatControl):
output_steer = self.pid.update(error, override=CS.steeringPressed,
feedforward=steer_feedforward, speed=CS.vEgo)
pid_log.active = True
pid_log.p = float(self.pid.p)
pid_log.i = float(self.pid.i)
pid_log.f = float(self.pid.f)
pid_log.output = float(output_steer)
pid_log.saturated = bool(self._check_saturation(self.steer_max - abs(output_steer) < 1e-3, CS, steer_limited))
pid_log.p = self.pid.p
pid_log.i = self.pid.i
pid_log.f = self.pid.f
pid_log.output = output_steer
pid_log.saturated = self._check_saturation(self.steer_max - abs(output_steer) < 1e-3, CS, steer_limited)
return output_steer, angle_steers_des, pid_log

View File

@@ -1,8 +1,8 @@
import math
import numpy as np
from cereal import log
from opendbc.car.interfaces import LatControlInputs
from openpilot.common.numpy_fast import interp
from openpilot.selfdrive.controls.lib.latcontrol import LatControl
from openpilot.common.pid import PIDController
from openpilot.selfdrive.controls.lib.vehicle_model import ACCELERATION_DUE_TO_GRAVITY
@@ -51,7 +51,7 @@ class LatControlTorque(LatControl):
else:
assert calibrated_pose is not None
actual_curvature_pose = calibrated_pose.angular_velocity.yaw / CS.vEgo
actual_curvature = np.interp(CS.vEgo, [2.0, 5.0], [actual_curvature_vm, actual_curvature_pose])
actual_curvature = interp(CS.vEgo, [2.0, 5.0], [actual_curvature_vm, actual_curvature_pose])
curvature_deadzone = 0.0
desired_lateral_accel = desired_curvature * CS.vEgo ** 2
@@ -60,7 +60,7 @@ class LatControlTorque(LatControl):
actual_lateral_accel = actual_curvature * CS.vEgo ** 2
lateral_accel_deadzone = curvature_deadzone * CS.vEgo ** 2
low_speed_factor = np.interp(CS.vEgo, LOW_SPEED_X, LOW_SPEED_Y)**2
low_speed_factor = interp(CS.vEgo, LOW_SPEED_X, LOW_SPEED_Y)**2
setpoint = desired_lateral_accel + low_speed_factor * desired_curvature
measurement = actual_lateral_accel + low_speed_factor * actual_curvature
gravity_adjusted_lateral_accel = desired_lateral_accel - roll_compensation
@@ -68,7 +68,7 @@ class LatControlTorque(LatControl):
setpoint, lateral_accel_deadzone, friction_compensation=False, gravity_adjusted=False)
torque_from_measurement = self.torque_from_lateral_accel(LatControlInputs(measurement, roll_compensation, CS.vEgo, CS.aEgo), self.torque_params,
measurement, lateral_accel_deadzone, friction_compensation=False, gravity_adjusted=False)
pid_log.error = float(torque_from_setpoint - torque_from_measurement)
pid_log.error = torque_from_setpoint - torque_from_measurement
ff = self.torque_from_lateral_accel(LatControlInputs(gravity_adjusted_lateral_accel, roll_compensation, CS.vEgo, CS.aEgo), self.torque_params,
desired_lateral_accel - actual_lateral_accel, lateral_accel_deadzone, friction_compensation=True,
gravity_adjusted=True)
@@ -80,14 +80,14 @@ class LatControlTorque(LatControl):
freeze_integrator=freeze_integrator)
pid_log.active = True
pid_log.p = float(self.pid.p)
pid_log.i = float(self.pid.i)
pid_log.d = float(self.pid.d)
pid_log.f = float(self.pid.f)
pid_log.output = float(-output_torque)
pid_log.actualLateralAccel = float(actual_lateral_accel)
pid_log.desiredLateralAccel = float(desired_lateral_accel)
pid_log.saturated = bool(self._check_saturation(self.steer_max - abs(output_torque) < 1e-3, CS, steer_limited))
pid_log.p = self.pid.p
pid_log.i = self.pid.i
pid_log.d = self.pid.d
pid_log.f = self.pid.f
pid_log.output = -output_torque
pid_log.actualLateralAccel = actual_lateral_accel
pid_log.desiredLateralAccel = desired_lateral_accel
pid_log.saturated = self._check_saturation(self.steer_max - abs(output_torque) < 1e-3, CS, steer_limited)
# TODO left is positive in this convention
return -output_torque, 0.0, pid_log

View File

@@ -1,5 +1,5 @@
import numpy as np
from cereal import car
from openpilot.common.numpy_fast import clip
from openpilot.common.realtime import DT_CTRL
from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N
from openpilot.common.pid import PIDController
@@ -84,5 +84,5 @@ class LongControl:
output_accel = self.pid.update(error, speed=CS.vEgo,
feedforward=a_target)
self.last_output_accel = np.clip(output_accel, accel_limits[0], accel_limits[1])
self.last_output_accel = clip(output_accel, accel_limits[0], accel_limits[1])
return self.last_output_accel

View File

@@ -4,6 +4,7 @@ import time
import numpy as np
from cereal import log
from opendbc.car.interfaces import ACCEL_MIN
from openpilot.common.numpy_fast import clip
from openpilot.common.realtime import DT_MDL
from openpilot.common.swaglog import cloudlog
# WARNING: imports outside of constants will not trigger a rebuild
@@ -319,9 +320,9 @@ class LongitudinalMpc:
# MPC will not converge if immediate crash is expected
# Clip lead distance to what is still possible to brake for
min_x_lead = ((v_ego + v_lead)/2) * (v_ego - v_lead) / (-ACCEL_MIN * 2)
x_lead = np.clip(x_lead, min_x_lead, 1e8)
v_lead = np.clip(v_lead, 0.0, 1e8)
a_lead = np.clip(a_lead, -10., 5.)
x_lead = clip(x_lead, min_x_lead, 1e8)
v_lead = clip(v_lead, 0.0, 1e8)
a_lead = clip(a_lead, -10., 5.)
lead_xv = self.extrapolate_lead(x_lead, v_lead, a_lead, a_lead_tau)
return lead_xv

View File

@@ -1,6 +1,7 @@
#!/usr/bin/env python3
import math
import numpy as np
from openpilot.common.numpy_fast import clip, interp
import cereal.messaging as messaging
from opendbc.car.interfaces import ACCEL_MIN, ACCEL_MAX
@@ -15,8 +16,6 @@ from openpilot.selfdrive.controls.lib.drive_helpers import CONTROL_N, get_speed_
from openpilot.selfdrive.car.cruise import V_CRUISE_MAX, V_CRUISE_UNSET
from openpilot.common.swaglog import cloudlog
from openpilot.sunnypilot.selfdrive.controls.lib.longitudinal_planner import LongitudinalPlannerSP
LON_MPC_STEP = 0.2 # first step is 0.2s
A_CRUISE_MIN = -1.2
A_CRUISE_MAX_VALS = [1.6, 1.2, 0.8, 0.6]
@@ -31,7 +30,7 @@ _A_TOTAL_MAX_BP = [20., 40.]
def get_max_accel(v_ego):
return np.interp(v_ego, A_CRUISE_MAX_BP, A_CRUISE_MAX_VALS)
return interp(v_ego, A_CRUISE_MAX_BP, A_CRUISE_MAX_VALS)
def get_coast_accel(pitch):
return np.sin(pitch) * -5.65 - 0.3 # fitted from data using xx/projects/allow_throttle/compute_coast_accel.py
@@ -44,7 +43,7 @@ def limit_accel_in_turns(v_ego, angle_steers, a_target, CP):
"""
# FIXME: This function to calculate lateral accel is incorrect and should use the VehicleModel
# The lookup table for turns should also be updated if we do this
a_total_max = np.interp(v_ego, _A_TOTAL_MAX_BP, _A_TOTAL_MAX_V)
a_total_max = interp(v_ego, _A_TOTAL_MAX_BP, _A_TOTAL_MAX_V)
a_y = v_ego ** 2 * angle_steers * CV.DEG_TO_RAD / (CP.steerRatio * CP.wheelbase)
a_x_allowed = math.sqrt(max(a_total_max ** 2 - a_y ** 2, 0.))
@@ -56,9 +55,9 @@ def get_accel_from_plan(speeds, accels, action_t=DT_MDL, vEgoStopping=0.05):
v_now = speeds[0]
a_now = accels[0]
v_target = np.interp(action_t, CONTROL_N_T_IDX, speeds)
v_target = interp(action_t, CONTROL_N_T_IDX, speeds)
a_target = 2 * (v_target - v_now) / (action_t) - a_now
v_target_1sec = np.interp(action_t + 1.0, CONTROL_N_T_IDX, speeds)
v_target_1sec = interp(action_t + 1.0, CONTROL_N_T_IDX, speeds)
else:
v_target = 0.0
v_target_1sec = 0.0
@@ -68,11 +67,10 @@ def get_accel_from_plan(speeds, accels, action_t=DT_MDL, vEgoStopping=0.05):
return a_target, should_stop
class LongitudinalPlanner(LongitudinalPlannerSP):
class LongitudinalPlanner:
def __init__(self, CP, init_v=0.0, init_a=0.0, dt=DT_MDL):
self.CP = CP
self.mpc = LongitudinalMpc(dt=dt)
LongitudinalPlannerSP.__init__(self, self.CP, self.mpc)
self.fcw = False
self.dt = dt
self.allow_throttle = True
@@ -107,10 +105,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
return x, v, a, j, throttle_prob
def update(self, sm):
LongitudinalPlannerSP.update(self, sm)
self.mpc.mode = 'blended' if sm['selfdriveState'].experimentalMode else 'acc'
if dec_mpc_mode := self.get_mpc_mode():
self.mpc.mode = dec_mpc_mode
if len(sm['carControl'].orientationNED) == 3:
accel_coast = get_coast_accel(sm['carControl'].orientationNED[1])
@@ -144,7 +139,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
if reset_state:
self.v_desired_filter.x = v_ego
# Clip aEgo to cruise limits to prevent large accelerations when becoming active
self.a_desired = np.clip(sm['carState'].aEgo, accel_limits[0], accel_limits[1])
self.a_desired = clip(sm['carState'].aEgo, accel_limits[0], accel_limits[1])
# Prevent divergence, smooth in current v_ego
self.v_desired_filter.x = max(0.0, self.v_desired_filter.update(v_ego))
@@ -156,7 +151,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
if not self.allow_throttle:
clipped_accel_coast = max(accel_coast, accel_limits_turns[0])
clipped_accel_coast_interp = np.interp(v_ego, [MIN_ALLOW_THROTTLE_SPEED, MIN_ALLOW_THROTTLE_SPEED*2], [accel_limits_turns[1], clipped_accel_coast])
clipped_accel_coast_interp = interp(v_ego, [MIN_ALLOW_THROTTLE_SPEED, MIN_ALLOW_THROTTLE_SPEED*2], [accel_limits_turns[1], clipped_accel_coast])
accel_limits_turns[1] = min(accel_limits_turns[1], clipped_accel_coast_interp)
if force_slow_decel:
@@ -181,7 +176,7 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
# Interpolate 0.05 seconds and save as starting point for next iteration
a_prev = self.a_desired
self.a_desired = float(np.interp(self.dt, CONTROL_N_T_IDX, self.a_desired_trajectory))
self.a_desired = float(interp(self.dt, CONTROL_N_T_IDX, self.a_desired_trajectory))
self.v_desired_filter.x = self.v_desired_filter.x + self.dt * (self.a_desired + a_prev) / 2.0
def publish(self, sm, pm):
@@ -205,11 +200,9 @@ class LongitudinalPlanner(LongitudinalPlannerSP):
action_t = self.CP.longitudinalActuatorDelay + DT_MDL
a_target, should_stop = get_accel_from_plan(longitudinalPlan.speeds, longitudinalPlan.accels,
action_t=action_t, vEgoStopping=self.CP.vEgoStopping)
longitudinalPlan.aTarget = float(a_target)
longitudinalPlan.shouldStop = bool(should_stop)
longitudinalPlan.aTarget = a_target
longitudinalPlan.shouldStop = should_stop
longitudinalPlan.allowBrake = True
longitudinalPlan.allowThrottle = bool(self.allow_throttle)
longitudinalPlan.allowThrottle = self.allow_throttle
pm.send('longitudinalPlan', plan_send)
self.publish_longitudinal_plan_sp(sm, pm)

View File

@@ -19,8 +19,7 @@ class TestLatControl:
def test_saturation(self, car_name, controller):
CarInterface, CarController, CarState, RadarInterface = interfaces[car_name]
CP = CarInterface.get_non_essential_params(car_name)
CP_SP = CarInterface.get_non_essential_params_sp(CP, car_name)
CI = CarInterface(CP, CP_SP, CarController, CarState)
CI = CarInterface(CP, CarController, CarState)
VM = VehicleModel(CP)
controller = controller(CP.as_reader(), CI)

View File

@@ -14,11 +14,11 @@ def main():
cloudlog.info("plannerd is waiting for CarParams")
params = Params()
CP = messaging.log_from_bytes(params.get("CarParams", block=True), car.CarParams)
cloudlog.info("plannerd got CarParams: %s", CP.brand)
cloudlog.info("plannerd got CarParams: %s", CP.carName)
ldw = LaneDepartureWarning()
longitudinal_planner = LongitudinalPlanner(CP)
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance', 'longitudinalPlanSP'])
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance'])
sm = messaging.SubMaster(['carControl', 'carState', 'controlsState', 'liveParameters', 'radarState', 'modelV2', 'selfdriveState'],
poll='modelV2', ignore_avg_freq=['radarState'])

View File

@@ -1,11 +1,11 @@
#!/usr/bin/env python3
import math
import numpy as np
from collections import deque
from typing import Any
import capnp
from cereal import messaging, log, car
from openpilot.common.numpy_fast import interp
from openpilot.common.params import Params
from openpilot.common.realtime import DT_MDL, Priority, config_realtime_process
from openpilot.common.swaglog import cloudlog
@@ -44,7 +44,7 @@ class KalmanParams:
0.28144091, 0.27958406, 0.27783249, 0.27617149, 0.27458948, 0.27307714,
0.27162685, 0.27023228, 0.26888809, 0.26758976, 0.26633338, 0.26511557,
0.26393339, 0.26278425]
self.K = [[np.interp(dt, dts, K0)], [np.interp(dt, dts, K1)]]
self.K = [[interp(dt, dts, K0)], [interp(dt, dts, K1)]]
class Track:

11
selfdrive/debug/adb.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -e
PORT=5555
setprop service.adb.tcp.port $PORT
sudo systemctl start adbd
IP=$(echo $SSH_CONNECTION | awk '{ print $3}')
echo "then, connect on your computer:"
echo "adb connect $IP:$PORT"

View File

@@ -2,8 +2,8 @@
import sys
import argparse
from subprocess import check_output, CalledProcessError
from opendbc.car.uds import UdsClient, MessageTimeoutError, SESSION_TYPE, DTC_GROUP_TYPE
from panda import Panda
from panda.python.uds import UdsClient, MessageTimeoutError, SESSION_TYPE, DTC_GROUP_TYPE
parser = argparse.ArgumentParser(description="clear DTC status")
parser.add_argument("addr", type=lambda x: int(x,0), nargs="?", default=0x7DF) # default is functional (broadcast) address

View File

@@ -1,9 +1,9 @@
#!/usr/bin/env python3
import argparse
from opendbc.car import uds
from openpilot.tools.lib.live_logreader import live_logreader
from openpilot.tools.lib.logreader import LogReader, ReadMode
from panda.python import uds
def main(route: str | None, addrs: list[int]):

View File

@@ -16,8 +16,8 @@ import argparse
from typing import NamedTuple
from subprocess import check_output, CalledProcessError
from opendbc.car.uds import UdsClient, SESSION_TYPE, DATA_IDENTIFIER_TYPE
from panda.python import Panda
from panda.python.uds import UdsClient, SESSION_TYPE, DATA_IDENTIFIER_TYPE
class ConfigValues(NamedTuple):
default_config: bytes

View File

@@ -1,10 +1,10 @@
#!/usr/bin/env python3
import argparse
import numpy as np
import capnp
from collections import defaultdict
from cereal.messaging import SubMaster
from openpilot.common.numpy_fast import mean
def cputime_total(ct):
return ct.user + ct.nice + ct.system + ct.idle + ct.iowait + ct.irq + ct.softirq
@@ -49,7 +49,7 @@ if __name__ == "__main__":
if sm.updated['deviceState']:
t = sm['deviceState']
last_temp = np.mean(t.cpuTempC)
last_temp = mean(t.cpuTempC)
last_mem = t.memoryUsagePercent
if sm.updated['procLog']:
@@ -72,7 +72,7 @@ if __name__ == "__main__":
total_times = total_times_new[:]
busy_times = busy_times_new[:]
print(f"CPU {100.0 * np.mean(cores):.2f}% - RAM: {last_mem:.2f}% - Temp {last_temp:.2f}C")
print(f"CPU {100.0 * mean(cores):.2f}% - RAM: {last_mem:.2f}% - Temp {last_temp:.2f}C")
if args.cpu and prev_proclog is not None and prev_proclog_t is not None:
procs: dict[str, float] = defaultdict(float)

View File

@@ -2,8 +2,9 @@
import sys
import argparse
from subprocess import check_output, CalledProcessError
from opendbc.car.uds import UdsClient, SESSION_TYPE, DTC_REPORT_TYPE, DTC_STATUS_MASK_TYPE, get_dtc_num_as_str, get_dtc_status_names
from panda import Panda
from panda.python.uds import UdsClient, SESSION_TYPE, DTC_REPORT_TYPE, DTC_STATUS_MASK_TYPE
from panda.python.uds import get_dtc_num_as_str, get_dtc_status_names
parser = argparse.ArgumentParser(description="read DTC status")
parser.add_argument("addr", type=lambda x: int(x,0))

View File

@@ -30,7 +30,7 @@ if __name__ == '__main__':
elif event.type == 3 and event.code == 57 and event.value == -1:
fingers[current_slot] = [-1, -1]
elif event.type == 3 and event.code == 53:
fingers[current_slot][1] = event.value
fingers[current_slot][1] = h - (h - event.value)
if fingers[current_slot][0] != -1:
touch_points.append(fingers[current_slot].copy())
elif event.type == 3 and event.code == 54:

View File

@@ -3,19 +3,14 @@ import time
from cereal import car, log, messaging
from openpilot.common.params import Params
from openpilot.system.manager.process_config import managed_processes, is_snpe_model
from openpilot.system.manager.process_config import managed_processes
from openpilot.system.hardware import HARDWARE
if __name__ == "__main__":
CP = car.CarParams(notCar=True, wheelbase=1, steerRatio=10)
params = Params()
params.put("CarParams", CP.to_bytes())
if use_snpe_modeld := is_snpe_model(False, params, CP):
print("Using SNPE modeld")
HARDWARE.set_power_save(False)
Params().put("CarParams", CP.to_bytes())
procs = ['camerad', 'ui', 'calibrationd', 'plannerd', 'dmonitoringmodeld', 'dmonitoringd']
procs += ["modeld_snpe" if use_snpe_modeld else "modeld"]
procs = ['camerad', 'ui', 'modeld', 'calibrationd', 'plannerd', 'dmonitoringmodeld', 'dmonitoringd']
for p in procs:
managed_processes[p].start()

View File

@@ -3,9 +3,9 @@
import argparse
import struct
from enum import IntEnum
from opendbc.car.uds import UdsClient, MessageTimeoutError, NegativeResponseError, SESSION_TYPE,\
DATA_IDENTIFIER_TYPE, ACCESS_TYPE
from panda import Panda
from panda.python.uds import UdsClient, MessageTimeoutError, NegativeResponseError, SESSION_TYPE,\
DATA_IDENTIFIER_TYPE, ACCESS_TYPE
from datetime import date
# TODO: extend UDS library to allow custom/vendor-defined data identifiers without ignoring type checks

View File

@@ -24,16 +24,14 @@ MIN_STD_SANITY_CHECK = 1e-5 # m or rad
MAX_FILTER_REWIND_TIME = 0.8 # s
MAX_SENSOR_TIME_DIFF = 0.1 # s
YAWRATE_CROSS_ERR_CHECK_FACTOR = 30
INPUT_INVALID_LIMIT = 2.0 # 1 (camodo) / 9 (sensor) bad input[s] ignored
INPUT_INVALID_RECOVERY = 10.0 # ~10 secs to resume after exceeding allowed bad inputs by one
INPUT_INVALID_THRESHOLD = 0.5 # 0 bad inputs ignored
TIMING_INVALID_THRESHOLD = 2.5 # 2 bad timings ignored
INPUT_INVALID_DECAY = 0.9993 # ~10 secs to resume after exceeding allowed bad inputs by one (at 100hz)
TIMING_INVALID_DECAY = 0.9990 # ~2 secs to resume after exceeding allowed bad timings by one (at 100hz)
POSENET_STD_INITIAL_VALUE = 10.0
POSENET_STD_HIST_HALF = 20
def calculate_invalid_input_decay(invalid_limit, recovery_time, frequency):
return (1 - 1 / (2 * invalid_limit)) ** (1 / (recovery_time * frequency))
def init_xyz_measurement(measurement: capnp._DynamicStructBuilder, values: np.ndarray, stds: np.ndarray, valid: bool):
assert len(values) == len(stds) == 3
measurement.x, measurement.y, measurement.z = map(float, values)
@@ -271,11 +269,11 @@ def main():
filter_initialized = False
critcal_services = ["accelerometer", "gyroscope", "cameraOdometry"]
observation_timing_invalid = defaultdict(int)
observation_input_invalid = defaultdict(int)
input_invalid_limit = {s: round(INPUT_INVALID_LIMIT * (SERVICE_LIST[s].frequency / 20.)) for s in critcal_services}
input_invalid_threshold = {s: input_invalid_limit[s] - 0.5 for s in critcal_services}
input_invalid_decay = {s: calculate_invalid_input_decay(input_invalid_limit[s], INPUT_INVALID_RECOVERY, SERVICE_LIST[s].frequency) for s in critcal_services}
input_invalid_decay = {s: INPUT_INVALID_DECAY ** (100. / SERVICE_LIST[s].frequency) for s in critcal_services}
timing_invalid_decay = {s: TIMING_INVALID_DECAY ** (100. / SERVICE_LIST[s].frequency) for s in critcal_services}
initial_pose = params.get("LocationFilterInitialState")
if initial_pose is not None:
@@ -308,20 +306,19 @@ def main():
continue
if res == HandleLogResult.TIMING_INVALID:
print(f"Observation {which} ignored due to failed timing check")
observation_input_invalid[which] += 1
print(observation_input_invalid[which])
observation_timing_invalid[which] += 1
elif res == HandleLogResult.INPUT_INVALID:
print(f"Observation {which} ignored due to failed sanity check")
observation_input_invalid[which] += 1
else:
observation_input_invalid[which] *= input_invalid_decay[which]
observation_timing_invalid[which] *= timing_invalid_decay[which]
else:
filter_initialized = sm.all_checks() and sensor_all_checks(acc_msgs, gyro_msgs, sensor_valid, sensor_recv_time, sensor_alive, SIMULATION)
if sm.updated["cameraOdometry"]:
critical_service_inputs_valid = all(observation_input_invalid[s] < input_invalid_threshold[s] for s in critcal_services)
inputs_valid = sm.all_valid() and critical_service_inputs_valid
critical_service_inputs_valid = all(observation_input_invalid[s] < INPUT_INVALID_THRESHOLD for s in critcal_services)
critical_service_timing_valid = all(observation_timing_invalid[s] < TIMING_INVALID_THRESHOLD for s in critcal_services)
inputs_valid = sm.all_valid() and critical_service_inputs_valid and critical_service_timing_valid
sensors_valid = sensor_all_checks(acc_msgs, gyro_msgs, sensor_valid, sensor_recv_time, sensor_alive, SIMULATION)
msg = estimator.get_msg(sensors_valid, inputs_valid, filter_initialized)

View File

@@ -8,6 +8,7 @@ import cereal.messaging as messaging
from cereal import car, log
from openpilot.common.params import Params
from openpilot.common.realtime import config_realtime_process, DT_MDL
from openpilot.common.numpy_fast import clip
from openpilot.selfdrive.locationd.models.car_kf import CarKalman, ObservationKind, States
from openpilot.selfdrive.locationd.models.constants import GENERATED_DIR
from openpilot.selfdrive.locationd.helpers import PoseCalibrator, Pose
@@ -65,7 +66,7 @@ class ParamsLearner:
# This is done to bound the road roll estimate when localizer values are invalid
roll = 0.0
roll_std = np.radians(10.0)
self.roll = np.clip(roll, self.roll - ROLL_MAX_DELTA, self.roll + ROLL_MAX_DELTA)
self.roll = clip(roll, self.roll - ROLL_MAX_DELTA, self.roll + ROLL_MAX_DELTA)
yaw_rate_valid = msg.angularVelocityDevice.valid and self.calibrator.calib_valid
yaw_rate_valid = yaw_rate_valid and 0 < self.yaw_rate_std < 10 # rad/s
@@ -202,11 +203,11 @@ def main():
learner = ParamsLearner(CP, CP.steerRatio, 1.0, 0.0)
x = learner.kf.x
angle_offset_average = np.clip(math.degrees(x[States.ANGLE_OFFSET].item()),
angle_offset_average = clip(math.degrees(x[States.ANGLE_OFFSET].item()),
angle_offset_average - MAX_ANGLE_OFFSET_DELTA, angle_offset_average + MAX_ANGLE_OFFSET_DELTA)
angle_offset = np.clip(math.degrees(x[States.ANGLE_OFFSET].item() + x[States.ANGLE_OFFSET_FAST].item()),
angle_offset = clip(math.degrees(x[States.ANGLE_OFFSET].item() + x[States.ANGLE_OFFSET_FAST].item()),
angle_offset - MAX_ANGLE_OFFSET_DELTA, angle_offset + MAX_ANGLE_OFFSET_DELTA)
roll = np.clip(float(x[States.ROAD_ROLL].item()), roll - ROLL_MAX_DELTA, roll + ROLL_MAX_DELTA)
roll = clip(float(x[States.ROAD_ROLL].item()), roll - ROLL_MAX_DELTA, roll + ROLL_MAX_DELTA)
roll_std = float(P[States.ROAD_ROLL].item())
if learner.active and learner.speed > LOW_ACTIVE_SPEED:
# Account for the opposite signs of the yaw rates
@@ -225,9 +226,9 @@ def main():
liveParameters.sensorValid = sensors_valid
liveParameters.steerRatio = float(x[States.STEER_RATIO].item())
liveParameters.stiffnessFactor = float(x[States.STIFFNESS].item())
liveParameters.roll = float(roll)
liveParameters.angleOffsetAverageDeg = float(angle_offset_average)
liveParameters.angleOffsetDeg = float(angle_offset)
liveParameters.roll = roll
liveParameters.angleOffsetAverageDeg = angle_offset_average
liveParameters.angleOffsetDeg = angle_offset
liveParameters.valid = all((
avg_offset_valid,
total_offset_valid,

View File

@@ -0,0 +1,56 @@
import capnp
import cereal.messaging as messaging
from openpilot.common.params import Params
from openpilot.system.manager.process_config import managed_processes
class TestLocationdProc:
LLD_MSGS = ['gpsLocationExternal', 'cameraOdometry', 'carState', 'liveCalibration',
'accelerometer', 'gyroscope', 'magnetometer']
def setup_method(self):
self.pm = messaging.PubMaster(self.LLD_MSGS)
self.params = Params()
self.params.put_bool("UbloxAvailable", True)
managed_processes['locationd'].prepare()
managed_processes['locationd'].start()
def teardown_method(self):
managed_processes['locationd'].stop()
def get_msg(self, name, t):
try:
msg = messaging.new_message(name)
except capnp.lib.capnp.KjException:
msg = messaging.new_message(name, 0)
if name == "gpsLocationExternal":
msg.gpsLocationExternal.flags = 1
msg.gpsLocationExternal.hasFix = True
msg.gpsLocationExternal.verticalAccuracy = 1.0
msg.gpsLocationExternal.speedAccuracy = 1.0
msg.gpsLocationExternal.bearingAccuracyDeg = 1.0
msg.gpsLocationExternal.vNED = [0.0, 0.0, 0.0]
msg.gpsLocationExternal.latitude = float(self.lat)
msg.gpsLocationExternal.longitude = float(self.lon)
msg.gpsLocationExternal.unixTimestampMillis = t * 1e6
msg.gpsLocationExternal.altitude = float(self.alt)
#if name == "gnssMeasurements":
# msg.gnssMeasurements.measTime = t
# msg.gnssMeasurements.positionECEF.value = [self.x , self.y, self.z]
# msg.gnssMeasurements.positionECEF.std = [0,0,0]
# msg.gnssMeasurements.positionECEF.valid = True
# msg.gnssMeasurements.velocityECEF.value = []
# msg.gnssMeasurements.velocityECEF.std = [0,0,0]
# msg.gnssMeasurements.velocityECEF.valid = True
elif name == 'cameraOdometry':
msg.cameraOdometry.rot = [0.0, 0.0, 0.0]
msg.cameraOdometry.rotStd = [0.0, 0.0, 0.0]
msg.cameraOdometry.trans = [0.0, 0.0, 0.0]
msg.cameraOdometry.transStd = [0.0, 0.0, 0.0]
msg.logMonoTime = t
msg.valid = True
return msg

View File

@@ -23,10 +23,8 @@ class Scenario(Enum):
BASE = 'base'
GYRO_OFF = 'gyro_off'
GYRO_SPIKE_MIDWAY = 'gyro_spike_midway'
GYRO_CONSISTENT_SPIKES = 'gyro_consistent_spikes'
ACCEL_OFF = 'accel_off'
ACCEL_SPIKE_MIDWAY = 'accel_spike_midway'
ACCEL_CONSISTENT_SPIKES = 'accel_consistent_spikes'
SENSOR_TIMING_SPIKE_MIDWAY = 'timing_spikes'
SENSOR_TIMING_CONSISTENT_SPIKES = 'timing_consistent_spikes'
@@ -65,20 +63,18 @@ def run_scenarios(scenario, logs):
elif scenario == Scenario.GYRO_OFF:
logs = sorted([x for x in logs if x.which() != 'gyroscope'], key=lambda x: x.logMonoTime)
elif scenario == Scenario.GYRO_SPIKE_MIDWAY or scenario == Scenario.GYRO_CONSISTENT_SPIKES:
elif scenario == Scenario.GYRO_SPIKE_MIDWAY:
def gyro_spike(msg):
msg.gyroscope.gyroUncalibrated.v[0] += 3.0
count = 1 if scenario == Scenario.GYRO_SPIKE_MIDWAY else CONSISTENT_SPIKES_COUNT
logs = modify_logs_midway(logs, 'gyroscope', count, gyro_spike)
logs = modify_logs_midway(logs, 'gyroscope', 1, gyro_spike)
elif scenario == Scenario.ACCEL_OFF:
logs = sorted([x for x in logs if x.which() != 'accelerometer'], key=lambda x: x.logMonoTime)
elif scenario == Scenario.ACCEL_SPIKE_MIDWAY or scenario == Scenario.ACCEL_CONSISTENT_SPIKES:
elif scenario == Scenario.ACCEL_SPIKE_MIDWAY:
def acc_spike(msg):
msg.accelerometer.acceleration.v[0] += 100.0
count = 1 if scenario == Scenario.ACCEL_SPIKE_MIDWAY else CONSISTENT_SPIKES_COUNT
logs = modify_logs_midway(logs, 'accelerometer', count, acc_spike)
msg.accelerometer.acceleration.v[0] += 10.0
logs = modify_logs_midway(logs, 'accelerometer', 1, acc_spike)
elif scenario == Scenario.SENSOR_TIMING_SPIKE_MIDWAY or scenario == Scenario.SENSOR_TIMING_CONSISTENT_SPIKES:
def timing_spike(msg):
@@ -125,7 +121,7 @@ class TestLocationdScenarios:
assert np.allclose(replayed_data['roll'], 0.0)
assert np.all(replayed_data['sensors_flag'] == 0.0)
def test_gyro_spike(self):
def test_gyro_spikes(self):
"""
Test: a gyroscope spike in the middle of the segment
Expected Result:
@@ -136,17 +132,8 @@ class TestLocationdScenarios:
orig_data, replayed_data = run_scenarios(Scenario.GYRO_SPIKE_MIDWAY, self.logs)
assert np.allclose(orig_data['yaw_rate'], replayed_data['yaw_rate'], atol=np.radians(0.35))
assert np.allclose(orig_data['roll'], replayed_data['roll'], atol=np.radians(0.55))
assert np.all(replayed_data['inputs_flag'] == orig_data['inputs_flag'])
assert np.all(replayed_data['sensors_flag'] == orig_data['sensors_flag'])
def test_consistent_gyro_spikes(self):
"""
Test: consistent timing spikes for N gyroscope messages in the middle of the segment
Expected Result: inputsOK becomes False after N of bad measurements
"""
orig_data, replayed_data = run_scenarios(Scenario.GYRO_CONSISTENT_SPIKES, self.logs)
assert np.diff(replayed_data['inputs_flag'])[501] == -1.0
assert np.diff(replayed_data['inputs_flag'])[708] == 1.0
assert np.diff(replayed_data['inputs_flag'])[499] == -1.0
assert np.diff(replayed_data['inputs_flag'])[704] == 1.0
def test_accel_off(self):
"""
@@ -161,7 +148,7 @@ class TestLocationdScenarios:
assert np.allclose(replayed_data['roll'], 0.0)
assert np.all(replayed_data['sensors_flag'] == 0.0)
def test_accel_spike(self):
def test_accel_spikes(self):
"""
ToDo:
Test: an accelerometer spike in the middle of the segment
@@ -186,5 +173,5 @@ class TestLocationdScenarios:
Expected Result: inputsOK becomes False after N of bad measurements
"""
orig_data, replayed_data = run_scenarios(Scenario.SENSOR_TIMING_CONSISTENT_SPIKES, self.logs)
assert np.diff(replayed_data['inputs_flag'])[501] == -1.0
assert np.diff(replayed_data['inputs_flag'])[707] == 1.0
assert np.diff(replayed_data['inputs_flag'])[500] == -1.0
assert np.diff(replayed_data['inputs_flag'])[787] == 1.0

View File

@@ -71,7 +71,7 @@ class TorqueEstimator(ParameterEstimator):
self.offline_friction = 0.0
self.offline_latAccelFactor = 0.0
self.resets = 0.0
self.use_params = CP.brand in ALLOWED_CARS and CP.lateralTuning.which() == 'torque'
self.use_params = CP.carName in ALLOWED_CARS and CP.lateralTuning.which() == 'torque'
if CP.lateralTuning.which() == 'torque':
self.offline_friction = CP.lateralTuning.torque.friction

Some files were not shown because too many files have changed in this diff Show More