Compare commits

..

1 Commits

Author SHA1 Message Date
github-actions[bot] ac6b182524 sunnypilot v2026.07.16-4566
version: sunnypilot v2026.002.000 (dev)
date: 2026-07-16T04:41:37
master commit: 70f2c3ac79
2026-07-16 04:41:37 +00:00
3772 changed files with 1148657 additions and 103807 deletions
-18
View File
@@ -1,18 +0,0 @@
**/.git
.DS_Store
*.dylib
*.DSYM
*.d
*.pyc
*.pyo
.*.swp
.*.swo
.*.un~
*.tmp
*.o
*.o-*
*.os
*.os-*
venv/
.venv/
-23
View File
@@ -1,23 +0,0 @@
* text=auto
# to move existing files into LFS:
# git add --renormalize .
*.onnx filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
*.ttf filter=lfs diff=lfs merge=lfs -text
*.otf filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
selfdrive/car/tests/test_models_segs.txt filter=lfs diff=lfs merge=lfs -text
system/hardware/tici/updater_weston filter=lfs diff=lfs merge=lfs -text
system/hardware/tici/updater_magic filter=lfs diff=lfs merge=lfs -text
third_party/**/*.a filter=lfs diff=lfs merge=lfs -text
third_party/**/*.so filter=lfs diff=lfs merge=lfs -text
third_party/**/*.so.* filter=lfs diff=lfs merge=lfs -text
third_party/**/*.dylib filter=lfs diff=lfs merge=lfs -text
third_party/acados/*/t_renderer filter=lfs diff=lfs merge=lfs -text
third_party/qt5/larch64/bin/lrelease filter=lfs diff=lfs merge=lfs -text
third_party/qt5/larch64/bin/lupdate filter=lfs diff=lfs merge=lfs -text
third_party/catch2/include/catch2/catch.hpp filter=lfs diff=lfs merge=lfs -text
-11
View File
@@ -1,11 +0,0 @@
* @sunnypilot/dev-internal
/.github/ @devtekve @sunnyhaibin
/release/ci/ @devtekve @sunnyhaibin
/tinygrad_repo @devtekve @Discountchubbs
/tinygrad/ @devtekve @Discountchubbs
/selfdrive/controls/lib/longitudinal_planner.py @devtekve @Discountchubbs
/selfdrive/controls/lib/longitudinal_mpc_lib/long_mpc.py @devtekve @Discountchubbs
/selfdrive/modeld/ @devtekve @Discountchubbs
/sunnypilot/model* @devtekve @Discountchubbs
/sunnypilot/sunnylink/ @devtekve
/system/athena/ @devtekve
-47
View File
@@ -1,47 +0,0 @@
name: Bug report
description: For issues with running openpilot on your comma device
labels: ["bug"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* If the issue likely only affects your car model or make, go back and open a **car bug report** instead.
* If the issue is related to the driving or driver monitoring models, you should open a [discussion](https://github.com/commaai/openpilot/discussions/categories/model-feedback) instead.
* Ensure you're running the latest openpilot release.
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: route
attributes:
label: Provide a route where the issue occurs
description: Ensure the route is fully uploaded at https://useradmin.comma.ai. We cannot look into issues without routes, or at least a Dongle ID.
placeholder: 77611a1fac303767|2020-05-11--16-37-07
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version
description: If you're not on release, provide the commit hash
placeholder: 0.8.10
validations:
required: true
- type: textarea
attributes:
label: Additional info
-14
View File
@@ -1,14 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: Car bug report
url: https://github.com/commaai/opendbc/issues/new
about: For issues with a particular car make or model
- name: Join the Discord
url: https://discord.comma.ai
about: The community Discord is for both openpilot development and experience discussion
- name: Report driving behavior feedback
url: https://discord.com/channels/469524606043160576/1254834193066623017
about: Feedback for the driving and driver monitoring models goes in the #driving-feedback in Discord
- name: Community Wiki
url: https://github.com/commaai/openpilot/wiki
about: Check out our community wiki
-8
View File
@@ -1,8 +0,0 @@
---
name: Enhancement
about: For openpilot enhancement suggestions
title: ''
labels: 'enhancement'
assignees: ''
---
-42
View File
@@ -1,42 +0,0 @@
name: PC bug report
description: For issues with running openpilot on PC
labels: ["PC"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* Ensure you're running the latest openpilot release.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: os-version
attributes:
label: OS Version
placeholder: Ubuntu 24.04
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version or commit
placeholder: bd36f2ec8d3559909678eff2690c10a520938367
validations:
required: false
- type: textarea
attributes:
label: Additional info
-31
View File
@@ -1,31 +0,0 @@
ci:
- changed-files:
- any-glob-to-all-files: "{.github/**,**/test_*,**/test/**,Jenkinsfile}"
chore:
- changed-files:
- any-glob-to-all-files: "{.github/**}"
car:
- changed-files:
- any-glob-to-all-files: '{selfdrive/car/**,opendbc_repo}'
simulation:
- changed-files:
- any-glob-to-all-files: 'tools/sim/**'
ui:
- changed-files:
- any-glob-to-all-files: '{selfdrive/assets/**,selfdrive/ui/**,system/ui/**}'
tools:
- changed-files:
- any-glob-to-all-files: 'tools/**'
multilanguage:
- changed-files:
- any-glob-to-all-files: 'selfdrive/ui/translations/**'
autonomy:
- changed-files:
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit,sunnypilot/modeld*/models/**}"
-68
View File
@@ -1,68 +0,0 @@
<!-- Please copy and paste the relevant template -->
<!--- ***** Template: Fingerprint *****
**Car**
Which car (make, model, year) this fingerprint is for
**Route**
A route with the fingerprint
-->
<!--- ***** Template: Car Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
**Route**
Route: [a route with the bug fix]
-->
<!--- ***** Template: Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
-->
<!--- ***** Template: Car Port *****
**Checklist**
- [ ] added entry to CAR in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
- [ ] route with openpilot:
- [ ] route with stock system:
- [ ] car harness used (if comma doesn't sell it, put N/A):
-->
<!--- ***** Template: Refactor *****
**Description**
A description of the refactor, including the goals it accomplishes.
**Verification**
Explain how you tested the refactor for regressions.
-->
-43
View File
@@ -1,43 +0,0 @@
exclude-labels:
- 'no-changelog'
categories:
- title: '🚀 Features'
labels:
- 'feature'
- 'enhancement'
- title: '🐛 Bug Fixes'
collapse-after: 5
labels:
- 'fix'
- 'bugfix'
- 'bug'
- title: '🧰 Maintenance'
collapse-after: 5
label: 'chore'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
change-title-escapes: '\<*_&'
replacers:
- search: '/[Ss][Uu][Nn][Nn][Yy][Pp][Ii][Ll][Oo][Tt]/g'
replace: 'sunnypilot'
- search: '/\b[Ss][Pp]\b/g'
replace: 'SP'
version-resolver:
major:
labels:
- 'major'
minor:
labels:
- 'minor'
patch:
labels:
- 'patch'
default: patch
name-template: 'v$RESOLVED_VERSION 🚀'
tag-template: 'v$RESOLVED_VERSION'
version-template: "0.$MAJOR.$MINOR.$PATCH" # The day OP becomes v1, we need to bump this
tag-prefix: "v0." # The day OP becomes v1, we need to bump this
prerelease-identifier: "staging"
template: |
## Changes
$CHANGES
-91
View File
@@ -1,91 +0,0 @@
name: "PR review"
on:
pull_request_target:
types: [ opened, reopened, synchronize, edited ]
jobs:
labeler:
name: review
permissions:
contents: read
pull-requests: write
issues: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
submodules: false
# Label PRs
- uses: actions/labeler@v6
with:
dot: true
configuration-path: .github/labeler.yaml
# Check PR target branch
- name: check branch
uses: Vankka/pr-target-branch-action@def32ec9d93514138d6ac0132ee62e120a72aed5
if: github.repository == 'sunnypilot/sunnypilot'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
target: /^(?!master$).*/
exclude: /sunnypilot:.*/
change-to: ${{ github.base_ref }}
already-exists-action: close_this
already-exists-comment: "Your PR should be made against the `master` branch"
update-pr-labels:
name: Update fork's PR Labels
runs-on: ubuntu-latest
if: (github.event.pull_request.head.repo.fork && (contains(github.event_name, 'pull_request') && github.event.action == 'synchronize'))
env:
PR_LABEL: 'dev'
TRUST_FORK_PR_LABEL: 'trust-fork-pr'
steps:
- name: Check if PR has dev label
id: check-labels
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const prNumber = context.payload.pull_request.number;
const { data: labels } = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber
});
const hasDevC3Label = labels.some(label => label.name === process.env.PR_LABEL);
const hasTrustLabel = labels.some(label => label.name === process.env.TRUST_FORK_PR_LABEL);
console.log(`PR #${prNumber} has ${process.env.PR_LABEL} label: ${hasDevC3Label}`);
console.log(`PR #${prNumber} has ${process.env.TRUST_FORK_PR_LABEL} label: ${hasTrustLabel}`);
core.setOutput('has-dev', hasDevC3Label ? 'true' : 'false');
core.setOutput('has-trust', hasTrustLabel ? 'true' : 'false');
- name: Remove trust-fork-pr label if present
if: steps.check-labels.outputs.has-dev == 'true' && steps.check-labels.outputs.has-trust == 'true'
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const prNumber = context.payload.pull_request.number;
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
name: process.env.TRUST_FORK_PR_LABEL
});
console.log(`Removed '${process.env.TRUST_FORK_PR_LABEL}' label from PR #${prNumber} as it received new commits`);
// Add a comment to the PR
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body: `The \`${process.env.TRUST_FORK_PR_LABEL}\` label has been automatically removed because new commits were pushed to this PR. This PR will need to be re-reviewed before the label can be applied again.`
});
-35
View File
@@ -1,35 +0,0 @@
name: badges
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
env:
PYTHONPATH: ${{ github.workspace }}
jobs:
badges:
name: create badges
runs-on: ubuntu-latest
if: github.repository == 'sunnypilot/sunnypilot'
permissions:
contents: write
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Push badges
run: |
python3 selfdrive/ui/translations/create_badges.py
rm .gitattributes
git checkout --orphan badges
git rm -rf --cached .
git config user.email "badge-researcher@sunnypilot.ai"
git config user.name "Badge Researcher"
git add translation_badge.svg
git commit -m "Add/Update badges"
git push -f origin HEAD
@@ -1,304 +0,0 @@
name: Build and push all tinygrad models
on:
workflow_dispatch:
inputs:
set_min_version:
description: 'Minimum selector version required for the models (see helpers.py or readme.md)'
required: true
type: string
jobs:
setup:
runs-on: ubuntu-latest
outputs:
json_version: ${{ steps.get-json.outputs.json_version }}
recompiled_dir: ${{ steps.create-recompiled-dir.outputs.recompiled_dir }}
json_file: ${{ steps.get-json.outputs.json_file }}
model_matrix: ${{ steps.set-matrix.outputs.model_matrix }}
tinygrad_ref: ${{ steps.get-tinygrad-ref.outputs.tinygrad_ref }}
steps:
- name: Checkout sunnypilot repo
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot
path: sunnypilot
submodules: recursive
- name: Get tinygrad_repo ref
id: get-tinygrad-ref
run: |
cd sunnypilot
export PYTHONPATH=$(pwd)
ref=$(python3 sunnypilot/models/tinygrad_ref.py)
echo "tinygrad_ref=$ref" >> $GITHUB_OUTPUT
echo "tinygrad_ref is $ref"
- name: Checkout docs repo (sunnypilot-models, gh-pages)
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot-models
ref: gh-pages
path: docs
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
- name: Get next JSON version to use (from GitHub docs repo)
id: get-json
run: |
cd docs/docs
latest=$(ls driving_models_v*.json | sed -E 's/.*_v([0-9]+)\.json/\1/' | sort -n | tail -1)
next=$((latest+1))
json_file="driving_models_v${next}.json"
cp "driving_models_v${latest}.json" "$json_file"
echo "json_file=docs/docs/$json_file" >> $GITHUB_OUTPUT
echo "json_version=$((next+0))" >> $GITHUB_OUTPUT
echo "SRC_JSON_FILE=docs/docs/driving_models_v${latest}.json" >> $GITHUB_ENV
- name: Extract tinygrad models
id: set-matrix
working-directory: docs/docs
run: |
jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' "$(basename "${SRC_JSON_FILE}")" > matrix.json
echo "model_matrix=$(cat matrix.json)" >> $GITHUB_OUTPUT
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
- run: |
mkdir -p ~/.ssh
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Clone GitLab docs repo and create new recompiled dir
id: create-recompiled-dir
env:
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
run: |
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
cd gitlab_docs
git checkout main
git sparse-checkout set --no-cone models/
cd models
latest_dir=$(ls -d recompiled* 2>/dev/null | sed -E 's/recompiled([0-9]+)/\1/' | sort -n | tail -1)
if [[ -z "$latest_dir" ]]; then
next_dir=1
else
next_dir=$((latest_dir+1))
fi
recompiled_dir="${next_dir}"
mkdir -p "recompiled${recompiled_dir}"
touch "recompiled${recompiled_dir}/.gitkeep"
cd ../..
echo "recompiled_dir=$recompiled_dir" >> $GITHUB_OUTPUT
- name: Push empty recompiled dir to GitLab
run: |
cd gitlab_docs
git add models/recompiled${{ steps.create-recompiled-dir.outputs.recompiled_dir }}
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git commit -m "Add recompiled${{ steps.create-recompiled-dir.outputs.recompiled_dir }} for build-all" || echo "No changes to commit"
git push origin main
- name: Push new JSON to GitHub docs repo
run: |
cd docs
git pull origin gh-pages
git add docs/"$(basename ${{ steps.get-json.outputs.json_file }})"
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git commit -m "Add new ${{ steps.get-json.outputs.json_file }} for build-all" || echo "No changes to commit"
git push origin gh-pages
get_and_build:
needs: [setup]
strategy:
matrix:
model: ${{ fromJson(needs.setup.outputs.model_matrix) }}
fail-fast: false
uses: ./.github/workflows/build-single-tinygrad-model.yaml
with:
upstream_branch: ${{ matrix.model.ref }}
custom_name: ${{ matrix.model.display_name }}
recompiled_dir: ${{ needs.setup.outputs.recompiled_dir }}
json_version: ${{ needs.setup.outputs.json_version }}
secrets: inherit
retry_failed_models:
needs: [setup, get_and_build]
runs-on: ubuntu-latest
if: ${{ needs.setup.result != 'failure' && !cancelled() }}
outputs:
retry_matrix: ${{ steps.set-retry-matrix.outputs.retry_matrix }}
steps:
- uses: actions/download-artifact@v4
with:
pattern: model-*
path: output
- id: set-retry-matrix
run: |
echo '${{ needs.setup.outputs.model_matrix }}' > matrix.json
built=(); while IFS= read -r line; do built+=("$line"); done < <(
find output -maxdepth 1 -name 'model-*' -printf "%f\n" | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
)
jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' matrix.json > retry_matrix.json
echo "retry_matrix=$(cat retry_matrix.json)" >> $GITHUB_OUTPUT
retry_get_and_build:
needs: [setup, get_and_build, retry_failed_models]
if: ${{ needs.get_and_build.result == 'failure' || (needs.retry_failed_models.outputs.retry_matrix != '[]' && needs.retry_failed_models.outputs.retry_matrix != '') }}
strategy:
matrix:
model: ${{ fromJson(needs.retry_failed_models.outputs.retry_matrix) }}
fail-fast: false
uses: ./.github/workflows/build-single-tinygrad-model.yaml
with:
upstream_branch: ${{ matrix.model.ref }}
custom_name: ${{ matrix.model.display_name }}
recompiled_dir: ${{ needs.setup.outputs.recompiled_dir }}
json_version: ${{ needs.setup.outputs.json_version }}
artifact_suffix: -retry
secrets: inherit
publish_models:
name: Publish models sequentially
needs: [setup, get_and_build, retry_failed_models, retry_get_and_build]
if: ${{ !cancelled() && (needs.get_and_build.result != 'failure' || needs.retry_get_and_build.result == 'success' || (needs.retry_failed_models.outputs.retry_matrix != '[]' && needs.retry_failed_models.outputs.retry_matrix != '')) }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
max-parallel: 1
matrix:
model: ${{ fromJson(needs.setup.outputs.model_matrix) }}
env:
RECOMPILED_DIR: recompiled${{ needs.setup.outputs.recompiled_dir }}
JSON_FILE: ${{ needs.setup.outputs.json_file }}
ARTIFACT_NAME_INPUT: ${{ matrix.model.display_name }}
steps:
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
- name: Add GitLab.com SSH key to known_hosts
run: |
mkdir -p ~/.ssh
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Clone GitLab docs repo
env:
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
run: |
echo "Cloning GitLab"
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
cd gitlab_docs
echo "checkout models/${RECOMPILED_DIR}"
git sparse-checkout set --no-cone models/${RECOMPILED_DIR}
git checkout main
cd ..
- name: Checkout docs repo
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot-models
ref: gh-pages
path: docs
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
- name: Validate recompiled dir and JSON version
run: |
if [ ! -d "gitlab_docs/models/$RECOMPILED_DIR" ]; then
echo "Recompiled dir $RECOMPILED_DIR does not exist in GitLab repo"
exit 1
fi
if [ ! -f "$JSON_FILE" ]; then
echo "JSON file $JSON_FILE does not exist!"
exit 1
fi
- name: Download artifact name file
uses: actions/download-artifact@v4
with:
name: artifact-name-${{ env.ARTIFACT_NAME_INPUT }}
path: artifact_name
- name: Read artifact name
id: read-artifact-name
run: |
ARTIFACT_NAME=$(cat artifact_name/artifact_name.txt)
echo "artifact_name=$ARTIFACT_NAME" >> $GITHUB_OUTPUT
- name: Download model artifact
uses: actions/download-artifact@v4
with:
name: ${{ steps.read-artifact-name.outputs.artifact_name }}
path: output
- name: Remove onnx files bc not needed for recompiled dir since they already exist from single build
run: |
find output -type f -name '*.onnx' -delete
find output -type f -name 'big_*.pkl' -delete
find output -type f -name 'dmonitoring_model_tinygrad.pkl' -delete
- name: Copy model artifacts to gitlab
env:
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
run: |
ARTIFACT_DIR="gitlab_docs/models/${RECOMPILED_DIR}/${ARTIFACT_NAME}"
mkdir -p "$ARTIFACT_DIR"
for path in output/*; do
if [ "$(basename "$path")" = "artifact_name.txt" ]; then
continue
fi
name="$(basename "$path")"
if [ -d "$path" ]; then
mkdir -p "$ARTIFACT_DIR/$name"
cp -r "$path"/* "$ARTIFACT_DIR/$name/"
echo "Copied dir $name -> $ARTIFACT_DIR/$name"
else
cp "$path" "$ARTIFACT_DIR/"
echo "Copied file $name -> $ARTIFACT_DIR/"
fi
done
- name: Push recompiled dir to GitLab
env:
GITLAB_SSH_PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
run: |
cd gitlab_docs
git checkout main
git pull origin main
for d in models/"$RECOMPILED_DIR"/*/; do
git sparse-checkout add "$d"
done
git add models/"$RECOMPILED_DIR"
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git commit -m "Update $RECOMPILED_DIR with model from build-all-tinygrad-models" || echo "No changes to commit"
git push origin main
- run: |
cd docs
git pull origin gh-pages
- name: update json
run: |
ARGS=""
[ -n "${{ inputs.set_min_version }}" ] && ARGS="$ARGS --set-min-version \"${{ inputs.set_min_version }}\""
ARGS="$ARGS --sort-by-date"
ARGS="$ARGS --tinygrad-ref \"${{ needs.setup.outputs.tinygrad_ref }}\""
eval python3 docs/json_parser.py \
--json-path "$JSON_FILE" \
--recompiled-dir "gitlab_docs/models/$RECOMPILED_DIR" \
$ARGS
- name: Push updated json to GitHub
run: |
cd docs
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git checkout gh-pages
git add docs/"$(basename $JSON_FILE)"
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || echo "No changes to commit"
git push origin gh-pages
@@ -1,228 +0,0 @@
name: Build Single Tinygrad Model and Push
on:
workflow_call:
inputs:
upstream_branch:
description: 'Upstream commit to build from'
required: true
type: string
custom_name:
description: 'Custom name for the model (no date, only name)'
required: false
type: string
recompiled_dir:
description: 'Existing recompiled directory number (e.g. 3 for recompiled3)'
required: true
type: string
json_version:
description: 'driving_models version number to update (e.g. 5 for driving_models_v5.json)'
required: true
type: string
artifact_suffix:
description: 'Suffix for artifact name'
required: false
type: string
default: ''
bypass_push:
description: 'Bypass pushing to GitLab for build-all'
required: false
default: true
type: boolean
workflow_dispatch:
inputs:
upstream_branch:
description: 'Upstream commit to build from'
required: true
type: string
custom_name:
description: 'Custom name for the model (no date, only name)'
required: false
type: string
recompiled_dir:
description: 'Existing recompiled directory number (e.g. 3 for recompiled3)'
required: true
type: string
json_version:
description: 'driving_models version number to update (e.g. 5 for driving_models_v5.json)'
required: true
type: string
model_folder:
description: 'Model folder'
type: choice
default: 'None'
options:
- None
- Simple Plan Models
- Space Lab Models
- TR Models
- DTR Models
- Custom Merge Models
- FOF series models
- Other
custom_model_folder:
description: 'Custom model folder name (if "Other" selected)'
required: false
type: string
generation:
description: 'Model generation'
required: false
type: string
version:
description: 'Minimum selector version'
required: false
type: string
env:
RECOMPILED_DIR: recompiled${{ inputs.recompiled_dir }}
JSON_FILE: docs/docs/driving_models_v${{ inputs.json_version }}.json
jobs:
build_model:
uses: ./.github/workflows/sunnypilot-build-model.yaml
with:
upstream_branch: ${{ inputs.upstream_branch }}
custom_name: ${{ inputs.custom_name || inputs.upstream_branch }}
is_20hz: true
artifact_suffix: ${{ inputs.artifact_suffix }}
secrets: inherit
publish_model:
if: ${{ !inputs.bypass_push && !cancelled() }}
concurrency:
group: gitlab-push-${{ inputs.recompiled_dir }}
cancel-in-progress: false
needs: build_model
runs-on: ubuntu-latest
steps:
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
- name: Add GitLab.com SSH key to known_hosts
run: |
mkdir -p ~/.ssh
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Clone GitLab docs repo
env:
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
run: |
echo "Cloning GitLab"
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
cd gitlab_docs
echo "checkout models/${RECOMPILED_DIR}"
git sparse-checkout set --no-cone models/${RECOMPILED_DIR}
git checkout main
cd ..
- name: Checkout docs repo
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot-models
ref: gh-pages
path: docs
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
- name: Validate recompiled dir and JSON version
run: |
if [ ! -d "gitlab_docs/models/$RECOMPILED_DIR" ]; then
echo "Recompiled dir $RECOMPILED_DIR does not exist in GitLab repo"
exit 1
fi
if [ ! -f "$JSON_FILE" ]; then
echo "JSON file $JSON_FILE does not exist!"
exit 1
fi
- name: Download artifact name file
uses: actions/download-artifact@v4
with:
name: artifact-name-${{ inputs.custom_name || inputs.upstream_branch }}
path: artifact_name
- name: Read artifact name
id: read-artifact-name
run: |
ARTIFACT_NAME=$(cat artifact_name/artifact_name.txt)
echo "artifact_name=$ARTIFACT_NAME" >> $GITHUB_OUTPUT
- name: Download and extract model artifact
uses: actions/download-artifact@v4
with:
name: ${{ steps.read-artifact-name.outputs.artifact_name }}
path: output
- name: Remove unwanted files
run: |
find output -type f -name 'dmonitoring_model_tinygrad.pkl' -delete
find output -type f -name 'dmonitoring_model.onnx' -delete
- name: Copy model artifact(s) to GitLab recompiled dir
env:
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
run: |
ARTIFACT_DIR="gitlab_docs/models/${RECOMPILED_DIR}/${ARTIFACT_NAME}"
mkdir -p "$ARTIFACT_DIR"
for path in output/*; do
if [ "$(basename "$path")" = "artifact_name.txt" ]; then
continue
fi
name="$(basename "$path")"
if [ -d "$path" ]; then
mkdir -p "$ARTIFACT_DIR/$name"
cp -r "$path"/* "$ARTIFACT_DIR/$name/"
echo "Copied dir $name -> $ARTIFACT_DIR/$name"
else
cp "$path" "$ARTIFACT_DIR/"
echo "Copied file $name -> $ARTIFACT_DIR/"
fi
done
- name: Push recompiled dir to GitLab
env:
GITLAB_SSH_PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
run: |
cd gitlab_docs
git checkout main
git pull origin main
for d in models/"$RECOMPILED_DIR"/*/; do
git sparse-checkout add "$d"
done
git add models/"$RECOMPILED_DIR"
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git commit -m "Create/Update $RECOMPILED_DIR with new/updated model from build-single-tinygrad-model" || echo "No changes to commit"
git push origin main
- run: |
cd docs
git pull origin gh-pages
- name: Run json_parser.py to update JSON
run: |
FOLDER="${{ inputs.model_folder }}"
if [ "$FOLDER" = "Other" ]; then
FOLDER="${{ inputs.custom_model_folder }}"
fi
ARGS=""
if [ "$FOLDER" != "None" ] && [ -n "$FOLDER" ]; then
ARGS="$ARGS --model-folder \"$FOLDER\""
fi
[ -n "${{ inputs.generation }}" ] && ARGS="$ARGS --generation \"${{ inputs.generation }}\""
[ -n "${{ inputs.version }}" ] && ARGS="$ARGS --version \"${{ inputs.version }}\""
eval python3 docs/json_parser.py \
--json-path "$JSON_FILE" \
--recompiled-dir "gitlab_docs/models/$RECOMPILED_DIR" \
--sort-by-date \
$ARGS
- name: Push updated JSON to GitHub docs repo
run: |
cd docs
git config --global user.name "GitHub Action"
git config --global user.email "action@github.com"
git checkout gh-pages
git add docs/"$(basename $JSON_FILE)"
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || echo "No changes to commit"
git push origin gh-pages
-78
View File
@@ -1,78 +0,0 @@
name: cereal validation
on:
push:
branches:
- master
pull_request:
paths:
- 'cereal/**'
workflow_dispatch:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: cereal-validation-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
CI: 1
jobs:
generate_cereal_artifact:
name: Generate cereal validation artifacts
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Build openpilot
run: scons -j$(nproc) cereal
- name: Generate the log file
run: |
export PYTHONPATH=${{ github.workspace }}
python3 cereal/messaging/tests/validate_sp_cereal_upstream.py -g -f schema_instances.bin
- name: 'Prepare artifact'
run: |
mkdir -p "cereal/messaging/tests/cereal_validations"
cp cereal/messaging/tests/validate_sp_cereal_upstream.py "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py"
cp schema_instances.bin "cereal/messaging/tests/cereal_validations/schema_instances.bin"
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: cereal_validations
path: cereal/messaging/tests/cereal_validations
validate_cereal_with_upstream:
name: Validate cereal with Upstream
runs-on: ubuntu-24.04
needs: generate_cereal_artifact
steps:
- name: Checkout sunnypilot
uses: actions/checkout@v6
- name: Checkout upstream openpilot
uses: actions/checkout@v6
with:
repository: 'commaai/openpilot'
path: openpilot
submodules: true
ref: "refs/heads/master"
- run: ./tools/op.sh setup
- name: Build openpilot
working-directory: openpilot
run: scons -j$(nproc) cereal
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: cereal_validations
path: openpilot/cereal/messaging/tests/cereal_validations
- name: 'Run the validation'
run: |
export PYTHONPATH=${{ github.workspace }}/openpilot
chmod +x openpilot/cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py
python3 openpilot/cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py -r -f openpilot/cereal/messaging/tests/cereal_validations/schema_instances.bin
-45
View File
@@ -1,45 +0,0 @@
name: diff report
on:
pull_request_target:
types: [opened, synchronize, reopened]
jobs:
comment:
name: comment
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
pull-requests: write
actions: read
steps:
- name: Wait for process replay
id: wait
continue-on-error: true
uses: lewagon/wait-on-check-action@v1.3.4
with:
ref: ${{ github.event.pull_request.head.sha }}
check-name: process replay
repo-token: ${{ secrets.GITHUB_TOKEN }}
allowed-conclusions: success,failure
wait-interval: 20
- name: Download diff
if: steps.wait.outcome == 'success'
uses: dawidd6/action-download-artifact@v6
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
workflow: tests.yaml
workflow_conclusion: ''
pr: ${{ github.event.number }}
name: diff_report_${{ github.event.number }}
path: .
allow_forks: true
- name: Comment on PR
if: steps.wait.outcome == 'success'
uses: thollander/actions-comment-pull-request@v2
with:
filePath: diff_report.txt
comment_tag: diff_report
pr_number: ${{ github.event.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-65
View File
@@ -1,65 +0,0 @@
name: docs
on:
push:
branches:
- master
pull_request:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: docs-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
jobs:
docs:
name: build docs
runs-on: ubuntu-24.04
steps:
- uses: commaai/timeout@v1
- uses: actions/checkout@v6
with:
submodules: true
# Build
- name: Build docs
run: |
# TODO: can we install just the "docs" dependency group without the normal deps?
pip install mkdocs
mkdocs build
# Push to docs.comma.ai
- uses: actions/checkout@v6
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
with:
path: openpilot-docs
ssh-key: ${{ secrets.OPENPILOT_DOCS_KEY }}
repository: sunnypilot/sunnypilot-docs
- name: Push
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
run: |
set -x
source release/identity.sh
cd openpilot-docs
git checkout --orphan tmp
git rm -rf .
# copy over docs
cp -r ../docs_site/ docs/
# GitHub pages config
touch docs/.nojekyll
echo -n docs.comma.ai > docs/CNAME
git add -f .
git commit -m "build docs"
# docs live in different repo to not bloat openpilot's full clone size
git push -f origin tmp:gh-pages
-96
View File
@@ -1,96 +0,0 @@
name: jenkins scan
on:
issue_comment:
types: [created, edited]
jobs:
cleanup-branches:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Delete stale Jenkins branches
uses: actions/github-script@v8
with:
script: |
const cutoff = Date.now() - 24 * 60 * 60 * 1000;
const prefixes = ['tmp-jenkins', '__jenkins'];
for await (const response of github.paginate.iterator(github.rest.repos.listBranches, {
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 100,
})) {
for (const branch of response.data) {
if (!prefixes.some(p => branch.name.startsWith(p))) continue;
const { data: commit } = await github.rest.repos.getCommit({
owner: context.repo.owner,
repo: context.repo.repo,
ref: branch.commit.sha,
});
const commitDate = new Date(commit.commit.committer.date).getTime();
if (commitDate < cutoff) {
console.log(`Deleting branch: ${branch.name} (last commit: ${commit.commit.committer.date})`);
await github.rest.git.deleteRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: `heads/${branch.name}`,
});
}
}
}
scan-comments:
runs-on: ubuntu-latest
if: ${{ github.event.issue.pull_request }}
permissions:
contents: write
issues: write
steps:
- name: Check for trigger phrase
id: check_comment
uses: actions/github-script@v8
with:
script: |
const triggerPhrase = "trigger-jenkins";
const comment = context.payload.comment.body;
const commenter = context.payload.comment.user.login;
const { data: permissions } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: commenter
});
const hasWriteAccess = permissions.permission === 'write' || permissions.permission === 'admin';
return (hasWriteAccess && comment.includes(triggerPhrase));
result-encoding: json
- name: Checkout repository
if: steps.check_comment.outputs.result == 'true'
uses: actions/checkout@v6
with:
ref: refs/pull/${{ github.event.issue.number }}/head
- name: Push to tmp-jenkins branch
if: steps.check_comment.outputs.result == 'true'
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b tmp-jenkins-${{ github.event.issue.number }}
GIT_LFS_SKIP_PUSH=1 git push -f origin tmp-jenkins-${{ github.event.issue.number }}
- name: Delete trigger comment
if: steps.check_comment.outputs.result == 'true' && always()
uses: actions/github-script@v8
with:
script: |
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
});
-72
View File
@@ -1,72 +0,0 @@
name: Sync comma's LFS
env:
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
on:
schedule:
- cron: '0 0 * * *' # Runs at 00:00 UTC every day
push:
branches:
- 'master'
pull_request:
branches:
- 'master'
workflow_dispatch: # enables manual triggering
inputs:
upstream_branch:
default: 'master'
type: string
jobs:
sync:
runs-on: ubuntu-latest
# Skip if PR is in draft mode
if: (github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false)) && !github.event.pull_request.head.repo.fork
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
repository: 'commaai/openpilot'
ref: ${{ inputs.upstream_branch }}
- name: LFS Fetch
run: |
git lfs fetch
- name: Set up Git
run: |
git config --global user.name 'GitHub Action'
git config --global user.email 'action@github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Ensure branch
run: |
if git symbolic-ref -q HEAD >/dev/null; then
echo "Already on a branch, proceeding with push"
else
echo "Detached HEAD state detected, creating temporary branch"
git checkout -b temp_branch
fi
- name: Update LFS Config
run: |
echo '[lfs]' > .lfsconfig
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
echo ' locksverify = false' >> .lfsconfig
- name: Push LFS
id: sync-and-commit
run: |
git lfs ls-files -l
git lfs push --all origin
-42
View File
@@ -1,42 +0,0 @@
name: "model review"
on:
pull_request:
types: [opened, reopened, synchronize]
paths:
- 'selfdrive/modeld/models/*.onnx'
workflow_dispatch:
jobs:
comment:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
steps:
- name: Checkout
uses: actions/checkout@v6
with:
submodules: true
- name: Checkout master
uses: actions/checkout@v6
with:
ref: master
path: base
- run: git lfs pull
- run: cd base && git lfs pull
- name: scripts/reporter.py
id: report
run: |
echo "content<<EOF" >> $GITHUB_OUTPUT
echo "## Model Review" >> $GITHUB_OUTPUT
PYTHONPATH=${{ github.workspace }} MASTER_PATH=${{ github.workspace }}/base python scripts/reporter.py >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Post model report comment
uses: marocchino/sticky-pull-request-comment@baa7203ed60924babbe5dcd0ac8eae3b66ec5e16
with:
header: model-review
message: ${{ steps.report.outputs.content }}
@@ -1,105 +0,0 @@
name: 'Post to Discourse'
description: 'Posts a message to a Discourse topic (existing or new)'
inputs:
discourse-url:
description: 'Discourse instance URL (e.g., https://discourse.example.com)'
required: true
api-key:
description: 'Discourse API key'
required: true
api-username:
description: 'Discourse API username'
required: true
topic-id:
description: 'Discourse topic ID to post to (use this OR category-id + title)'
required: false
category-id:
description: 'Category ID for new topic (required if topic-id not provided)'
required: false
title:
description: 'Title for new topic (required if topic-id not provided)'
required: false
message:
description: 'Message content (markdown supported)'
required: true
outputs:
post-number:
description: 'The post number in the topic'
value: ${{ steps.post.outputs.post_number }}
post-url:
description: 'Direct URL to the post'
value: ${{ steps.post.outputs.post_url }}
topic-id:
description: 'The topic ID (useful when creating a new topic)'
value: ${{ steps.post.outputs.topic_id }}
runs:
using: "composite"
steps:
- name: Post to Discourse
id: post
shell: bash
run: |
# Validate inputs
if [ -z "${{ inputs.topic-id }}" ] && ([ -z "${{ inputs.category-id }}" ] || [ -z "${{ inputs.title }}" ]); then
echo "❌ Error: Must provide either topic-id OR both category-id and title"
exit 1
fi
if [ -n "${{ inputs.topic-id }}" ] && ([ -n "${{ inputs.category-id }}" ] || [ -n "${{ inputs.title }}" ]); then
echo "⚠️ Warning: Both topic-id and category-id/title provided. Will post to existing topic."
fi
# Determine if creating new topic or posting to existing
if [ -n "${{ inputs.topic-id }}" ]; then
echo "📝 Posting to existing topic ID: ${{ inputs.topic-id }}"
# Create JSON payload for posting to existing topic
PAYLOAD=$(jq -n \
--arg content '${{ inputs.message }}' \
--arg topic_id "${{ inputs.topic-id }}" \
'{topic_id: $topic_id, raw: $content}')
else
echo "✨ Creating new topic: ${{ inputs.title }}"
# Create JSON payload for new topic
PAYLOAD=$(jq -n \
--arg content '${{ inputs.message }}' \
--arg title "${{ inputs.title }}" \
--arg category "${{ inputs.category-id }}" \
'{title: $title, category: ($category | tonumber), raw: $content}')
fi
# Post to Discourse
RESPONSE=$(curl -s -w "\n%{http_code}" \
-X POST "${{ inputs.discourse-url }}/posts.json" \
-H "Content-Type: application/json" \
-H "Api-Key: ${{ inputs.api-key }}" \
-H "Api-Username: ${{ inputs.api-username }}" \
-d "$PAYLOAD")
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
BODY=$(echo "$RESPONSE" | sed '$d')
if [ "$HTTP_CODE" -ge 200 ] && [ "$HTTP_CODE" -lt 300 ]; then
echo "✅ Successfully posted to Discourse!"
POST_NUMBER=$(echo "$BODY" | jq -r '.post_number // "unknown"')
TOPIC_ID=$(echo "$BODY" | jq -r '.topic_id // "${{ inputs.topic-id }}"')
POST_URL="${{ inputs.discourse-url }}/t/${TOPIC_ID}/${POST_NUMBER}"
echo "post_number=${POST_NUMBER}" >> $GITHUB_OUTPUT
echo "post_url=${POST_URL}" >> $GITHUB_OUTPUT
echo "topic_id=${TOPIC_ID}" >> $GITHUB_OUTPUT
echo "Topic ID: ${TOPIC_ID}"
echo "Post number: ${POST_NUMBER}"
echo "URL: ${POST_URL}"
else
echo "❌ Failed to post to Discourse"
echo "HTTP Code: ${HTTP_CODE}"
echo "Response: ${BODY}"
exit 1
fi
-39
View File
@@ -1,39 +0,0 @@
name: prebuilt
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
env:
DOCKER_LOGIN: docker login ghcr.io -u ${{ github.actor }} -p ${{ secrets.GITHUB_TOKEN }}
BUILD: release/ci/docker_build_sp.sh
jobs:
build_prebuilt:
name: build prebuilt
runs-on: ubuntu-latest
if: github.repository == 'sunnypilot/sunnypilot'
env:
PUSH_IMAGE: true
permissions:
checks: read
contents: read
packages: write
steps:
- name: Wait for green check mark
if: ${{ github.event_name != 'workflow_dispatch' }}
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
with:
ref: master
wait-interval: 30
running-workflow-name: 'build prebuilt'
repo-token: ${{ secrets.GITHUB_TOKEN }}
check-regexp: ^((?!.*(build master-ci|create badges).*).)*$
- uses: actions/checkout@v6
with:
submodules: true
- run: git lfs pull
- name: Build and Push docker image
run: |
$DOCKER_LOGIN
eval "$BUILD"
-28
View File
@@ -1,28 +0,0 @@
name: Release Drafter
on:
push:
branches:
- master
tags:
- 'v*'
pull_request_target:
types: [opened, reopened, synchronize]
workflow_dispatch:
permissions:
contents: read
jobs:
update_release_draft:
permissions:
contents: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v6
with:
config-name: release-drafter.yml
prerelease: ${{ !startsWith(github.ref, 'refs/tags/v') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-31
View File
@@ -1,31 +0,0 @@
name: release
on:
schedule:
- cron: '0 9 * * *'
workflow_dispatch:
jobs:
build___nightly:
name: build __nightly
runs-on: ubuntu-latest
if: github.repository == 'sunnypilot/sunnypilot'
permissions:
checks: read
contents: write
steps:
- name: Wait for green check mark
if: ${{ github.event_name == 'schedule' }}
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
with:
ref: master
wait-interval: 30
running-workflow-name: 'build __nightly'
repo-token: ${{ secrets.GITHUB_TOKEN }}
check-regexp: ^((?!.*(build prebuilt|create badges).*).)*$
- uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 0
- run: ./tools/op.sh setup
- name: Push __nightly
run: BRANCH=__nightly release/build_stripped.sh
-77
View File
@@ -1,77 +0,0 @@
name: repo maintenance
on:
schedule:
- cron: "0 14 * * 1" # every Monday at 2am UTC (6am PST)
workflow_dispatch:
env:
PYTHONPATH: ${{ github.workspace }}
jobs:
package_updates:
name: package_updates
runs-on: ubuntu-latest
if: github.repository == 'sunnypilot/sunnypilot'
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: uv lock
run: uv lock --upgrade
- name: uv pip tree
id: pip_tree
run: |
echo 'PIP_TREE<<EOF' >> $GITHUB_OUTPUT
uv pip tree >> $GITHUB_OUTPUT
echo 'EOF' >> $GITHUB_OUTPUT
- name: venv size
id: venv_size
run: |
echo 'VENV_SIZE<<EOF' >> $GITHUB_OUTPUT
echo "Total: $(du -sh .venv | cut -f1)" >> $GITHUB_OUTPUT
echo "" >> $GITHUB_OUTPUT
echo "Top 10 by size:" >> $GITHUB_OUTPUT
du -sh .venv/lib/python*/site-packages/* 2>/dev/null \
| grep -v '\.dist-info' \
| grep -v '__pycache__' \
| sort -rh \
| head -10 \
| while IFS=$'\t' read size path; do echo "$size ${path##*/}"; done >> $GITHUB_OUTPUT
echo 'EOF' >> $GITHUB_OUTPUT
- name: bump submodules
run: |
git config submodule.msgq.update none
git config submodule.rednose_repo.update none
git config submodule.teleoprtc_repo.update none
git config submodule.tinygrad.update none
git submodule update --remote
git add .
- name: update car docs
run: |
python selfdrive/car/docs.py
git add docs/CARS.md
- name: Create Pull Request
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0
with:
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
token: ${{ github.repository == 'commaai/openpilot' && secrets.ACTIONS_CREATE_PR_PAT || secrets.GITHUB_TOKEN }}
commit-message: Update Python packages
title: '[bot] Update Python packages'
branch: auto-package-updates
base: master
delete-branch: true
body: |
Automatic PR from repo-maintenance -> package_updates
```
$ du -sh .venv && du -sh .venv/lib/python*/site-packages/* | sort -rh | head -10
${{ steps.venv_size.outputs.VENV_SIZE }}
```
```
$ uv pip tree
${{ steps.pip_tree.outputs.PIP_TREE }}
```
labels: bot
-52
View File
@@ -1,52 +0,0 @@
name: stale
on:
schedule:
- cron: '30 1 * * *'
workflow_dispatch:
env:
DAYS_BEFORE_PR_CLOSE: 7
DAYS_BEFORE_PR_STALE: 24
DAYS_BEFORE_PR_STALE_DRAFT: 30
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v10
with:
exempt-all-milestones: true
# pull request config
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
stale-pr-label: stale
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'sunnypilot/sunnypilot' }} # only delete branches on the main repo
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE }}
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}
exempt-draft-pr: false
# issue config
days-before-issue-stale: -1 # ignore issues for now
# same as above, but give draft PRs more time
stale_drafts:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v10
with:
exempt-all-milestones: true
# pull request config
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE_DRAFT }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
stale-pr-label: stale
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'commaai/openpilot' }} # only delete branches on the main repo
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE_DRAFT }}
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}
exempt-draft-pr: true
# issue config
days-before-issue-stale: -1 # ignore issues for now
@@ -1,229 +0,0 @@
name: Build Model from Upstream
env:
BUILD_DIR: "/data/openpilot"
OUTPUT_DIR: ${{ github.workspace }}/output
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
UPSTREAM_REPO: "commaai/openpilot"
TINYGRAD_PATH: ${{ github.workspace }}/tinygrad_repo
MODELS_DIR: ${{ github.workspace }}/selfdrive/modeld/models
on:
workflow_call:
inputs:
upstream_branch:
description: 'Upstream branch to build from'
required: true
default: 'master'
type: string
custom_name:
description: 'Custom name for the model (no date, only name)'
required: false
type: string
is_20hz:
description: 'Is this a 20Hz model'
required: false
type: boolean
default: true
artifact_suffix:
description: 'Suffix for artifact name'
required: false
type: string
default: ''
workflow_dispatch:
inputs:
upstream_branch:
description: 'Upstream branch to build from'
required: true
default: 'master'
type: string
custom_name:
description: 'Custom name for the model (no date, only name)'
required: false
type: string
is_20hz:
description: 'Is this a 20Hz model'
required: false
type: boolean
default: true
run-name: Build model [${{ inputs.custom_name || inputs.upstream_branch }}] from ref [${{ inputs.upstream_branch }}]
jobs:
get_model:
runs-on: ubuntu-latest
env:
REF: ${{ inputs.upstream_branch }}
outputs:
model_date: ${{ steps.commit-date.outputs.model_date }}
steps:
# Note: To allow dynamic models from both openpilot and sunnypilot (merges/mashups), we try commaai as default,
# and fallback to sunnypilot if the ref checkout fails.
- name: Checkout commaai/openpilot
id: checkout_upstream
continue-on-error: true
uses: actions/checkout@v4
with:
repository: commaai/openpilot
ref: ${{ inputs.upstream_branch }}
submodules: recursive
path: openpilot
- name: Fallback to sunnypilot/sunnypilot
if: steps.checkout_upstream.outcome == 'failure'
uses: actions/checkout@v4
with:
repository: sunnypilot/sunnypilot
ref: ${{ inputs.upstream_branch }}
submodules: recursive
path: openpilot
- name: Get commit date
id: commit-date
run: |
cd ${{ github.workspace }}/openpilot
commit_date=$(git log -1 --format=%cd --date=format:'%B %d, %Y')
echo "model_date=${commit_date}" >> $GITHUB_OUTPUT
cat $GITHUB_OUTPUT
- run: |
cd ${{ github.workspace }}/openpilot
git lfs pull
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: models-${{ env.REF }}${{ inputs.artifact_suffix }}
path: ${{ github.workspace }}/openpilot/selfdrive/modeld/models/*.onnx
build_model:
runs-on: [self-hosted, tici]
needs: get_model
env:
MODEL_NAME: ${{ inputs.custom_name || inputs.upstream_branch }} (${{ needs.get_model.outputs.model_date }})
REF: ${{ inputs.upstream_branch }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- run: git lfs pull
- name: Cache SCons
uses: actions/cache@v4
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}-model
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}
- name: Set environment variables
id: set-env
run: |
# Set up common environment
source /etc/profile;
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
printenv >> $GITHUB_ENV
if [[ "${{ runner.debug }}" == "1" ]]; then
cat $GITHUB_OUTPUT
fi
- name: Setup build environment
run: |
mkdir -p "${BUILD_DIR}/"
sudo find $BUILD_DIR/ -mindepth 1 -delete
echo "Starting build stage..."
echo "BUILD_DIR: ${BUILD_DIR}"
echo "CI_DIR: ${CI_DIR}"
echo "VERSION: ${{ steps.set-env.outputs.version }}"
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
echo "-------"
if [[ "${{ runner.debug }}" == "1" ]]; then
printenv
fi
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
rm -rf ${{ env.MODELS_DIR }}/*.onnx
- name: Download model artifacts
uses: actions/download-artifact@v4
with:
name: models-${{ env.REF }}${{ inputs.artifact_suffix }}
path: ${{ github.workspace }}/selfdrive/modeld/models
- run: |
rm -f ${{ github.workspace }}/selfdrive/modeld/models/{dmonitoring_model,big_driving_policy,big_driving_vision}.onnx
- name: Build Model
run: |
source /etc/profile
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
export PYTHONPATH="${PYTHONPATH}:${{ env.TINYGRAD_PATH }}"
# Loop through all .onnx files
find "${{ env.MODELS_DIR }}" -maxdepth 1 -name '*.onnx' | while IFS= read -r onnx_file; do
base_name=$(basename "$onnx_file" .onnx)
output_file="${{ env.MODELS_DIR }}/${base_name}_tinygrad.pkl"
echo "Compiling: $onnx_file -> $output_file"
QCOM=1 python3 "${{ env.TINYGRAD_PATH }}/examples/openpilot/compile3.py" "$onnx_file" "$output_file"
DEV=QCOM FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 python3 "${{ env.MODELS_DIR }}/../get_model_metadata.py" "$onnx_file" || true
done
- name: Validate Model Outputs
run: |
source /etc/profile
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
--validate-only \
--model-dir "${{ env.MODELS_DIR }}"
- name: Prepare Output
run: |
sudo rm -rf ${{ env.OUTPUT_DIR }}
mkdir -p ${{ env.OUTPUT_DIR }}
# Copy the model files
rsync -avm \
--include='*.dlc' \
--include='*.pkl' \
--include='*.onnx' \
--exclude='*' \
--delete-excluded \
--chown=comma:comma \
${{ env.MODELS_DIR }}/ ${{ env.OUTPUT_DIR }}/
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
--model-dir "${{ env.MODELS_DIR }}" \
--output-dir "${{ env.OUTPUT_DIR }}" \
--custom-name "${{ env.MODEL_NAME }}" \
--upstream-branch "${{ inputs.upstream_branch }}" \
${{ inputs.is_20hz && '--is-20hz' || '' }}
- name: Write artifact name to file
run: echo "model-${{ env.MODEL_NAME }}${{ inputs.artifact_suffix }}-${{ github.run_number }}" > ${{ env.OUTPUT_DIR }}/artifact_name.txt
- name: Upload Build Artifacts
id: upload-artifact
uses: actions/upload-artifact@v4
with:
name: model-${{ env.MODEL_NAME }}${{ inputs.artifact_suffix }}-${{ github.run_number }}
path: ${{ env.OUTPUT_DIR }}
- name: Upload artifact name file
uses: actions/upload-artifact@v4
with:
name: artifact-name-${{ inputs.custom_name || inputs.upstream_branch }}
path: ${{ env.OUTPUT_DIR }}/artifact_name.txt
- name: Re-enable powersave
if: always()
run: |
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
@@ -1,369 +0,0 @@
name: sunnypilot prebuilt action
env:
BUILD_DIR: "/data/openpilot"
OUTPUT_DIR: ${{ github.workspace }}/output
CI_DIR: ${{ github.workspace }}/release/ci
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
PUBLIC_REPO_URL: "https://github.com/sunnypilot/sunnypilot"
# Branch configurations
STAGING_SOURCE_BRANCH: 'master'
# Runtime configuration
SOURCE_BRANCH: "${{ github.head_ref || github.ref_name }}"
on:
push:
branches: [ master, master-dev ]
tags: [ 'release/*' ]
pull_request_target:
types: [ labeled ]
workflow_dispatch:
inputs:
wait_for_tests:
description: 'Wait for tests to finish'
required: false
type: boolean
default: false
jobs:
prepare_strategy:
runs-on: ubuntu-24.04
if: (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
outputs:
environment: ${{ steps.strategy.outputs.environment }}
new_branch: ${{ steps.strategy.outputs.new_branch }}
extra_version_identifier: ${{ steps.strategy.outputs.extra_version_identifier }}
version: ${{ steps.strategy.outputs.version }}
cancel_publish_in_progress: ${{ steps.strategy.outputs.cancel_publish_in_progress }}
publish_concurrency_group: ${{ steps.strategy.outputs.publish_concurrency_group }}
is_stable_branch: ${{ steps.strategy.outputs.is_stable_branch }}
build: ${{ steps.strategy.outputs.build }}
steps:
- uses: actions/checkout@v4
- name: Extract deploy strategy
id: strategy
run: |
echo '::group::Strategy Extraction'
BRANCH="${{ github.head_ref || github.ref_name }}"
echo "Current branch: $BRANCH"
STRATEGY_JSON='${{ vars.DEPLOY_STRATEGY }}'
CONFIG=$(echo "$STRATEGY_JSON" | jq -r --arg branch "$BRANCH" '
.configs[] | select(.branch == $branch)
')
BUILD="$(date '+%Y.%m.%d')-${{ github.run_number }}"
if [[ -z "$CONFIG" || "$CONFIG" == "null" ]]; then
echo "No exact strategy match found. Falling back to feature/fork logic."
IS_FORK="${{ github.event.pull_request.head.repo.fork && 'true' || 'false' }}"
FORK_SUFFIX=$( [[ "$IS_FORK" == "true" ]] && echo "-fork" || echo "" )
NEW_BRANCH="${BRANCH}${FORK_SUFFIX}-prebuilt"
echo "new_branch=$NEW_BRANCH" >> $GITHUB_OUTPUT
echo "version=$BUILD" >> $GITHUB_OUTPUT
echo "cancel_publish_in_progress=true" >> $GITHUB_OUTPUT
echo "publish_concurrency_group=publish-${BRANCH}" >> $GITHUB_OUTPUT
echo "environment=feature-branch" >> $GITHUB_OUTPUT
echo "extra_version_identifier=feature-branch" >> $GITHUB_OUTPUT
else
echo "Matched config: $CONFIG"
environment=$(echo "$CONFIG" | jq -r '.environment')
echo "environment=$environment" >> $GITHUB_OUTPUT
echo "new_branch=$(echo "$CONFIG" | jq -r '.target_branch')" >> $GITHUB_OUTPUT
cancel="$(echo "$CONFIG" | jq -r '.cancel_publish_in_progress')";
echo "cancel_publish_in_progress=$( [ "$cancel" = "null" ] && echo "true" || echo $cancel)" >> $GITHUB_OUTPUT
echo "publish_concurrency_group=publish-${BRANCH}$( [ "$cancel" = "null" ] || [ "$cancel" = "true" ] || echo "${{ github.sha }}" )" >> $GITHUB_OUTPUT
is_stable_branch="$(echo "$CONFIG" | jq -r '.stable_branch // false')";
echo "is_stable_branch=$is_stable_branch" >> $GITHUB_OUTPUT
stable_version=$(cat sunnypilot/common/version.h | grep SUNNYPILOT_VERSION | sed -e 's/[^0-9|.]//g');
echo "version=$([ "$is_stable_branch" = "true" ] && echo "$stable_version" || echo "$BUILD")" >> $GITHUB_OUTPUT
echo "extra_version_identifier=${environment}" >> $GITHUB_OUTPUT
fi
echo "build=$BUILD" >> $GITHUB_OUTPUT
cat $GITHUB_OUTPUT
validate_tests:
runs-on: ubuntu-24.04
needs: [ prepare_strategy ]
if: ${{
((github.event_name == 'workflow_dispatch' && inputs.wait_for_tests) ||
(github.event_name == 'push' && needs.prepare_strategy.outputs.is_stable_branch == 'true') ||
contains(github.event_name, 'pull_request') && (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
}}
steps:
- uses: actions/checkout@v4
- name: Wait for Tests
uses: ./.github/workflows/wait-for-action # Path to where you place the action
with:
workflow: tests.yaml # The workflow file to monitor
github-token: ${{ secrets.GITHUB_TOKEN }}
should-wait-for-start: ${{ github.event_name == 'push' && 'true' || 'false' }}
build:
needs: [ validate_tests, prepare_strategy ]
concurrency:
group: build-${{ github.head_ref || github.ref_name }}
cancel-in-progress: false
runs-on: [self-hosted, tici]
outputs:
new_branch: ${{ needs.prepare_strategy.outputs.new_branch }}
version: ${{ needs.prepare_strategy.outputs.version }}
extra_version_identifier: ${{ needs.prepare_strategy.outputs.extra_version_identifier }}
commit_sha: ${{ github.sha }}
if: ${{
(always() && !cancelled() && !failure()) &&
needs.prepare_strategy.result == 'success' &&
(needs.validate_tests.result == 'success' || needs.validate_tests.result == 'skipped') &&
(!contains(github.event_name, 'pull_request') ||
(github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
}}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
ref: ${{ env.SOURCE_BRANCH }}
repository: ${{ github.event.pull_request.head.repo.fork && github.event.pull_request.head.repo.full_name || github.repository }}
- run: git lfs pull
- name: Cache SCons
uses: actions/cache@v4
with:
path: ${{env.SCONS_CACHE_DIR}}
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ env.SOURCE_BRANCH }}-${{ github.sha }}
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
restore-keys: |
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.SOURCE_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.STAGING_SOURCE_BRANCH }}
scons-${{ runner.os }}-${{ runner.arch }}
- name: Set environment variables
id: set-env
run: |
echo "new_branch=${{ needs.prepare_strategy.outputs.new_branch }}" >> $GITHUB_OUTPUT
echo "version=${{ needs.prepare_strategy.outputs.version }}" >> $GITHUB_OUTPUT
echo "extra_version_identifier=${{ needs.prepare_strategy.outputs.extra_version_identifier }}" >> $GITHUB_OUTPUT
echo "commit_sha=${{ github.sha }}" >> $GITHUB_OUTPUT
# Set up common environment
source /etc/profile;
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
printenv >> $GITHUB_ENV
if [[ "${{ runner.debug }}" == "1" ]]; then
cat $GITHUB_OUTPUT
fi
- name: Setup build environment
run: |
mkdir -p "${BUILD_DIR}/"
sudo find $BUILD_DIR/ -mindepth 1 -delete
echo "Starting build stage..."
echo "BUILD_DIR: ${BUILD_DIR}"
echo "CI_DIR: ${CI_DIR}"
echo "VERSION: ${{ steps.set-env.outputs.version }}"
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
echo "-------"
if [[ "${{ runner.debug }}" == "1" ]]; then
printenv
fi
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
- name: Build Main Project
run: |
export PYTHONPATH="$BUILD_DIR"
./release/release_files.py | sort | uniq | rsync -rRl${RUNNER_DEBUG:+v} --files-from=- . $BUILD_DIR/
cd $BUILD_DIR
sed -i '/from .board.jungle import PandaJungle, PandaJungleDFU/s/^/#/' panda/__init__.py
echo "Building sunnypilot's modeld_v2..."
scons -j$(nproc) cache_dir=${{env.SCONS_CACHE_DIR}} --minimal sunnypilot/modeld_v2
echo "Building sunnypilot's locationd..."
scons -j2 cache_dir=${{env.SCONS_CACHE_DIR}} --minimal sunnypilot/selfdrive/locationd
echo "Building openpilot's locationd..."
scons -j1 cache_dir=${{env.SCONS_CACHE_DIR}} --minimal selfdrive/locationd
echo "Building rest of sunnypilot"
scons -j$(nproc) cache_dir=${{env.SCONS_CACHE_DIR}} --minimal
touch ${BUILD_DIR}/prebuilt
if [[ "${{ runner.debug }}" == "1" ]]; then
ls -la ${BUILD_DIR}
fi
- name: Prepare Output
run: |
sudo rm -rf ${OUTPUT_DIR}
mkdir -p ${OUTPUT_DIR}
rsync -am${RUNNER_DEBUG:+v} \
--exclude='.sconsign.dblite' \
--exclude='*.a' \
--exclude='*.o' \
--exclude='*.os' \
--exclude='*.pyc' \
--exclude='moc_*' \
--exclude='__pycache__' \
--exclude='Jenkinsfile' \
--exclude='**/release/' \
--exclude='**/.github/' \
--exclude='**/selfdrive/ui/replay/' \
--exclude='**/__pycache__/' \
--exclude='${{env.SCONS_CACHE_DIR}}' \
--exclude='**/.git/' \
--exclude='**/SConstruct' \
--exclude='**/SConscript' \
--exclude='**/.venv/' \
--exclude='selfdrive/modeld/models/driving_vision.onnx' \
--exclude='selfdrive/modeld/models/driving_policy.onnx' \
--exclude='third_party/*x86*' \
--exclude='third_party/*Darwin*' \
--delete-excluded \
--chown=comma:comma \
${BUILD_DIR}/ ${OUTPUT_DIR}/
- name: 'Tar.gz files'
run: |
tar czf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }} .
ls -la prebuilt.tar.gz
- name: 'Upload Artifact'
uses: actions/upload-artifact@v4
with:
name: prebuilt
path: prebuilt.tar.gz
- name: Re-enable powersave
if: always()
run: |
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
publish:
concurrency:
# We do a bit of a hack here to avoid canceling the publishing job if a new commit comes in while we're publishing by adding the sha to the group name.
# This means that if multiple commits come in while we're publishing, they will be queued up and publish one after the other.
# Otherwise, if a job is waiting to be published due to environment wait time, it would be canceled by a new commit and restart the wait time.
group: ${{ needs.prepare_strategy.outputs.publish_concurrency_group }}
cancel-in-progress: ${{ needs.prepare_strategy.outputs.cancel_publish_in_progress == 'true' }}
if: ${{ (always() && !cancelled() && !failure()) && needs.build.result == 'success' && needs.prepare_strategy.result == 'success' && (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt')) }}
needs: [ build, prepare_strategy ]
runs-on: ubuntu-24.04
environment: ${{ needs.prepare_strategy.outputs.environment }}
steps:
- uses: actions/checkout@v4
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: prebuilt
- name: Untar prebuilt
run: |
mkdir -p ${{ env.OUTPUT_DIR }}
tar xzf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }}
- name: Configure Git
run: |
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "github-actions[bot]"
- name: Publish to Public Repository
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo '${{ toJSON(needs.build.outputs) }}'
ls -la ${{ env.OUTPUT_DIR }}
${{ env.CI_DIR }}/publish.sh \
"${{ github.workspace }}" \
"${{ env.OUTPUT_DIR }}" \
"${{ needs.build.outputs.new_branch }}" \
"${{ needs.build.outputs.version }}" \
"https://x-access-token:${{github.token}}@github.com/sunnypilot/sunnypilot.git" \
"${{ needs.build.outputs.extra_version_identifier }}"
echo ""
echo "---- ️ To update the list of branches that auto deploy prebuilts -----"
echo ""
echo "1. Go to: ${{ github.server_url }}/${{ github.repository }}/settings/variables/actions/AUTO_DEPLOY_PREBUILT_BRANCHES"
echo "2. Current value: ${{ vars.AUTO_DEPLOY_PREBUILT_BRANCHES }}"
echo "3. Update as needed (JSON array with no spaces)"
- name: Tag ${{ needs.prepare_strategy.outputs.environment }}
if: ${{ needs.prepare_strategy.outputs.is_stable_branch == 'true' && (github.event_name != 'push' || !startsWith(github.ref, 'refs/tags/')) }}
run: |
TAG="${{ needs.prepare_strategy.outputs.environment }}/${{ needs.prepare_strategy.outputs.version }}/${{ needs.prepare_strategy.outputs.build }}"
git tag -f -a ${TAG} -m "${{ needs.prepare_strategy.outputs.environment }} @ ${{ needs.prepare_strategy.outputs.version }} of build ${{ needs.build.outputs.build }}."
git push -f origin ${TAG}
notify:
needs:
- prepare_strategy
- build
- publish
runs-on: ubuntu-24.04
if: ${{ (always() && !cancelled() && !failure())
&& needs.publish.result == 'success'
&& (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
&& (fromJSON(vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES_V2)[github.head_ref || github.ref_name] != null) }}
steps:
- uses: actions/checkout@v4
- name: Prepare notification message
id: message
run: |
TEMPLATE='${{ vars.DISCOURSE_GENERAL_UPDATE_NOTICE }}'
export VERSION="${{ needs.prepare_strategy.outputs.version }}"
export branch_name="${{ env.SOURCE_BRANCH }}"
export new_branch="${{ needs.prepare_strategy.outputs.new_branch }}"
export commit_sha="${{ github.sha }}"
export commit_short_sha="${{ github.sha }}"
export commit_short_sha="${commit_short_sha:0:7}"
export extra_version_identifier="${{ needs.prepare_strategy.outputs.extra_version_identifier || github.run_number }}"
export PUBLIC_REPO_URL="${{ env.PUBLIC_REPO_URL }}"
MESSAGE=$(cat << 'EOF' | envsubst
${{ vars.DISCOURSE_GENERAL_UPDATE_NOTICE }}
EOF
)
{
echo 'content<<EOFMARKER'
echo "$MESSAGE"
echo 'EOFMARKER'
} >> $GITHUB_OUTPUT
shell: bash
- name: Post to Discourse
uses: ./.github/workflows/post-to-discourse
with:
discourse-url: ${{ vars.DISCOURSE_URL }}
api-key: ${{ secrets.DISCOURSE_API_KEY }}
api-username: "system"
topic-id: ${{ fromJSON(vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES_V2)[github.head_ref || github.ref_name].topic_id }}
message: ${{ steps.message.outputs.content }}
manage-pr-labels:
name: Remove prebuilt label
runs-on: ubuntu-latest
if: (always() && contains(github.event_name, 'pull_request') && (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
env:
LABEL: prebuilt
steps:
- name: Remove trust-fork-pr label if present
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const prNumber = context.payload.pull_request.number;
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
name: process.env.LABEL
});
console.log(`Removed '${process.env.LABEL}' label from PR #${prNumber}`);
@@ -1,243 +0,0 @@
name: Build dev
env:
DEFAULT_SOURCE_BRANCH: "master"
DEFAULT_TARGET_BRANCH: "master-dev"
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
on:
push:
branches:
- master
pull_request_target:
types: [ labeled ]
branches:
- 'master'
workflow_dispatch:
inputs:
source_branch:
description: 'Source branch to reset from'
required: true
default: 'master'
type: string
target_branch:
description: 'Target branch to reset and squash into'
required: true
default: 'master-dev'
type: string
cancel_in_progress:
description: 'Cancel any in-progress runs of this workflow'
required: false
default: true
type: boolean
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: ${{ inputs.cancel_in_progress || github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
jobs:
reset-and-squash:
runs-on: ubuntu-latest
if: (
(github.event_name == 'workflow_dispatch')
|| (github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch))
|| (contains(github.event_name, 'pull_request') && ((github.event.action == 'labeled' && (github.event.label.name == vars.PREBUILT_PR_LABEL || github.event.label.name == 'trust-fork-pr') && contains(github.event.pull_request.labels.*.name, vars.PREBUILT_PR_LABEL))))
)
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches
token: ${{ secrets.GITHUB_TOKEN }}
persist-credentials: false
- name: Wait for Tests
uses: ./.github/workflows/wait-for-action # Path to where you place the action
if: (
(github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch))
|| (contains(github.event_name, 'pull_request') && ((github.event.action == 'labeled' && (github.event.label.name == vars.PREBUILT_PR_LABEL || github.event.label.name == 'trust-fork-pr') && contains(github.event.pull_request.labels.*.name, vars.PREBUILT_PR_LABEL))))
)
with:
workflow: tests.yaml # The workflow file to monitor
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Configure Git
run: |
git config --global user.name 'github-actions[bot]'
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
- name: Set up SSH
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Add GitLab public keys
run: |
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install PyGithub
- name: Check branches exist
run: |
# Check if source branch exists
if ! git ls-remote --heads origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} | grep -q "${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"; then
echo "Source branch ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} does not exist!"
exit 1
fi
# Make sure we have the latest source branch
git fetch origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
# Check if target branch exists
if ! git ls-remote --heads origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} | grep -q "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"; then
echo "Target branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} does not exist, creating it from ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
git checkout -b ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
git push origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
else
# Fetch target branch if it exists
git fetch origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
fi
- name: Reset target branch
run: |
echo "Resetting ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} to match ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
# Delete if exists and recreate pointing to source
git branch -D ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} || true
git branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
- name: Get PRs to squash
id: get-prs
run: |
# Use GitHub API to get PRs with specific label, ordered by creation date
PR_LIST=$(gh api graphql -f query='
query($search_query:String!) {
search(query: $search_query, type:ISSUE, first:40) {
nodes {
... on PullRequest {
number
headRefName
title
createdAt
labels(last:10) {
nodes {
name
}
}
headRepository {
name
nameWithOwner
url
isFork
}
commits(last: 1) {
nodes {
commit {
statusCheckRollup {
state
}
}
}
}
}
}
}
}' -F search_query="repo:${{ github.repository }} is:pr is:open label:${{ vars.PREBUILT_PR_LABEL }},${{ vars.PREBUILT_PR_LABEL }}-c3 draft:false sort:created-asc")
PR_LIST=${PR_LIST//\'/}
echo "PR_LIST=${PR_LIST}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Process PRs
run: |
cp ${{ github.workspace }}/release/ci/squash_and_merge.py /tmp/squash_and_merge.py && \
chmod +x /tmp/squash_and_merge.py && \
python3 ${{ github.workspace }}/release/ci/squash_and_merge_prs.py \
--pr-data '${{ steps.get-prs.outputs.PR_LIST }}' \
--target-branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} \
--squash-script-path '/tmp/squash_and_merge.py'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Update LFS Config
run: |
echo '[lfs]' > .lfsconfig
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
echo ' locksverify = false' >> .lfsconfig
- name: Restore workflows from source
run: |
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
SOURCE_BRANCH="${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
# Ensure we are on the target branch
git checkout $TARGET_BRANCH
echo "Restoring .github/workflows from $SOURCE_BRANCH"
git checkout origin/$SOURCE_BRANCH -- .github/workflows
if ! git diff --cached --quiet; then
echo "Workflows differ. Committing restoration."
git commit -m "chore: restore .github/workflows from $SOURCE_BRANCH"
else
echo "Workflows match $SOURCE_BRANCH."
fi
- uses: actions/create-github-app-token@v2
id: ci-token
with:
app-id: ${{ secrets.CI_GITHUB_ACTIONS_TOKEN_APP_ID }}
private-key: ${{ secrets.CI_GITHUB_ACTIONS_TOKEN_APP_PRIVATE_KEY }}
- name: Push changes if there are diffs
id: push-changes
run: |
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
# Use the App Token to set the remote URL with authentication
git remote set-url origin "https://x-access-token:${{ steps.ci-token.outputs.token }}@github.com/${{ github.repository }}.git"
# Fetch the latest from remote
git fetch origin $TARGET_BRANCH
# Check for diffs between local and remote
if git diff $TARGET_BRANCH origin/$TARGET_BRANCH --quiet; then
echo "No changes to push - local and remote branches are identical"
echo "has_changes=false" >> $GITHUB_OUTPUT
exit 0
fi
# Push with the authenticated origin
if ! git push origin $TARGET_BRANCH --force; then
echo "Failed to push changes to $TARGET_BRANCH"
exit 1
fi
echo "Branch $TARGET_BRANCH has been reset and updated with squashed PRs"
echo "has_changes=true" >> $GITHUB_OUTPUT
- name: Trigger and wait for selfdrive tests
if: steps.push-changes.outputs.has_changes == 'true'
run: |
echo "Triggering selfdrive tests..."
gh workflow run tests.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
echo "Sleeping for 120s to give plenty of time for the action to start and then we wait"
sleep 120
echo "Getting latest run ID..."
RUN_ID=$(gh run list --workflow=tests.yaml --branch="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}" --limit=1 --json databaseId --jq '.[0].databaseId')
echo "Watching run ID: $RUN_ID"
gh run watch "$RUN_ID"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-78
View File
@@ -1,78 +0,0 @@
name: Debug Discourse Posting
on:
push:
jobs:
test-discourse-post:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Post test message to Discourse
uses: ./.github/workflows/post-to-discourse
with:
discourse-url: ${{ vars.DISCOURSE_URL }}
api-key: ${{ secrets.DISCOURSE_API_KEY }}
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
message: |
## 🧪 Test Post from GitHub Actions
**This is a test post to verify Discourse integration**
- **Workflow**: ${{ github.workflow }}
- **Run Number**: #${{ github.run_number }}
- **Branch**: `${{ github.ref_name }}`
- **Commit**: ${{ github.sha }}
- **Actor**: @${{ github.actor }}
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
---
### Fake Build Info (for testing)
- **Version**: 0.9.8-test
- **Build**: #42
- **Branch**: release-test
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
*This is an automated test message. Drive safe! 🚗💨*
- name: Create topic on Discourse
uses: ./.github/workflows/post-to-discourse
with:
discourse-url: ${{ vars.DISCOURSE_URL }}
api-key: ${{ secrets.DISCOURSE_API_KEY }}
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
#topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
category-id: 4
title: "This is a test of a new topic instead of a reply"
message: |
## 🧪 Test Post from GitHub Actions
**This is a test post to verify Discourse integration**
- **Workflow**: ${{ github.workflow }}
- **Run Number**: #${{ github.run_number }}
- **Branch**: `${{ github.ref_name }}`
- **Commit**: ${{ github.sha }}
- **Actor**: @${{ github.actor }}
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
---
### Fake Build Info (for testing)
- **Version**: 0.9.8-test
- **Build**: #42
- **Branch**: release-test
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
*This is an automated test message. Drive safe! 🚗💨*
- name: Display results
if: always()
run: |
echo "::notice::Discourse post test completed"
echo "Check your Discourse topic to verify the post appeared correctly"
-248
View File
@@ -1,248 +0,0 @@
name: tests
on:
push:
branches:
- master
pull_request:
workflow_dispatch:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
CI: 1
PYTHONPATH: ${{ github.workspace }}
PYTEST: pytest --continue-on-collection-errors --durations=0 -n logical
jobs:
build_release:
name: build release
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
env:
STRIPPED_DIR: /tmp/releasepilot
PYTHONPATH: /tmp/releasepilot
steps:
- uses: actions/checkout@v6
with:
submodules: true
- name: Getting LFS files
uses: nick-fields/retry@7152eba30c6575329ac0576536151aca5a72780e
with:
timeout_minutes: 2
max_attempts: 3
command: git lfs pull
- name: Build devel
timeout-minutes: 1
run: TARGET_DIR=$STRIPPED_DIR release/build_stripped.sh
- run: ./tools/op.sh setup
- name: Build openpilot and run checks
timeout-minutes: 30
working-directory: ${{ env.STRIPPED_DIR }}
run: python3 system/manager/build.py
- name: Run tests
timeout-minutes: 1
working-directory: ${{ env.STRIPPED_DIR }}
run: release/check-dirty.sh
- name: Check submodules
if: github.repository == 'sunnypilot/sunnypilot'
timeout-minutes: 3
run: |
if [ "${{ github.ref }}" != "refs/heads/master" ]; then
git fetch origin master:refs/remotes/origin/master
SUBMODULE_PATHS=$(git diff origin/master HEAD --name-only | grep -E '^[^/]+$' | while read path; do
if git ls-files --stage "$path" | grep -q "^160000"; then
echo "$path"
fi
done | tr '\n' ' ')
if [ -n "$SUBMODULE_PATHS" ]; then
echo "Changed submodule paths: $SUBMODULE_PATHS"
export SUBMODULE_PATHS="$SUBMODULE_PATHS"
export CHECK_PR_REFS=true
fi
fi
release/check-submodules.sh
build_mac:
name: build macOS
runs-on: ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-macos-8x14' || 'macos-latest' }}
steps:
- uses: actions/checkout@v6
with:
submodules: true
- name: Remove Homebrew from environment
run: |
FILTERED=$(echo "$PATH" | tr ':' '\n' | grep -v '/opt/homebrew' | tr '\n' ':')
echo "PATH=${FILTERED}/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" >> $GITHUB_ENV
- run: ./tools/op.sh setup
- name: Building openpilot
run: scons
static_analysis:
name: static analysis
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Static analysis
timeout-minutes: 1
run: scripts/lint/lint.sh
unit_tests:
name: unit tests
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Build openpilot
run: scons -j$(nproc)
- name: Run unit tests
timeout-minutes: ${{ contains(runner.name, 'nsc') && 2 || 999 }}
run: |
source selfdrive/test/setup_xvfb.sh
# Pre-compile Python bytecode so each pytest worker doesn't need to
$PYTEST --collect-only -m 'not slow' -qq
MAX_EXAMPLES=1 $PYTEST -m 'not slow'
process_replay:
name: process replay
if: false # disable process_replay for forks
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Build openpilot
run: scons -j$(nproc)
- name: Run replay
timeout-minutes: ${{ contains(runner.name, 'nsc') && 2 || 20 }}
continue-on-error: ${{ github.ref == 'refs/heads/master' }}
run: selfdrive/test/process_replay/test_processes.py -j$(nproc)
- name: Print diff
id: print-diff
if: always()
run: cat selfdrive/test/process_replay/diff.txt
- name: Print diff report
if: always()
run: cat selfdrive/test/process_replay/diff_report.txt
- uses: actions/upload-artifact@v6
if: always()
continue-on-error: true
with:
name: process_replay_diff.txt
path: selfdrive/test/process_replay/diff.txt
- name: Upload diff report
uses: actions/upload-artifact@v6
if: always() && github.event_name == 'pull_request'
continue-on-error: true
with:
name: diff_report_${{ github.event.number }}
path: selfdrive/test/process_replay/diff_report.txt
- name: Checkout ci-artifacts
if: github.repository == 'commaai/openpilot' && github.ref == 'refs/heads/master'
uses: actions/checkout@v4
with:
repository: commaai/ci-artifacts
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
path: ${{ github.workspace }}/ci-artifacts
- name: Push refs
if: github.repository == 'commaai/openpilot' && github.ref == 'refs/heads/master'
working-directory: ${{ github.workspace }}/ci-artifacts
run: |
git config user.name "GitHub Actions Bot"
git config user.email "<>"
git fetch origin process-replay || true
git checkout process-replay 2>/dev/null || git checkout --orphan process-replay
cp ${{ github.workspace }}/selfdrive/test/process_replay/fakedata/*.zst .
echo "${{ github.sha }}" > ref_commit
git add .
git commit -m "process-replay refs for ${{ github.repository }}@${{ github.sha }}" || echo "No changes to commit"
git push origin process-replay --force
- name: Run regen
if: false
timeout-minutes: 4
env:
ONNXCPU: 1
run: $PYTEST selfdrive/test/process_replay/test_regen.py
simulator_driving:
name: simulator driving
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
if: false # FIXME: Started to timeout recently
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Build openpilot
run: scons -j$(nproc)
- name: Driving test
timeout-minutes: 2
run: |
source selfdrive/test/setup_xvfb.sh
pytest -s tools/sim/tests/test_metadrive_bridge.py
create_ui_report:
name: Create UI Report
runs-on: ${{
(github.repository == 'commaai/openpilot') &&
((github.event_name != 'pull_request') ||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
&& fromJSON('["namespace-profile-amd64-8x16"]')
|| fromJSON('["ubuntu-24.04"]') }}
steps:
- uses: actions/checkout@v6
with:
submodules: true
- run: ./tools/op.sh setup
- name: Build openpilot
run: scons -j$(nproc)
- name: Create UI Report
run: |
source selfdrive/test/setup_xvfb.sh
python3 selfdrive/ui/tests/diff/replay.py
python3 selfdrive/ui/tests/diff/replay.py --big
- name: Upload UI Report
uses: actions/upload-artifact@v6
with:
name: ui-report-${{ inputs.run_number || '1' }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
path: selfdrive/ui/tests/diff/report
-175
View File
@@ -1,175 +0,0 @@
name: "ui preview"
on:
push:
branches:
- master
pull_request_target:
types: [assigned, opened, synchronize, reopened, edited]
branches:
- 'master'
paths:
- 'selfdrive/assets/**'
- 'selfdrive/ui/**'
- 'system/ui/**'
workflow_dispatch:
env:
UI_JOB_NAME: "Create UI Report"
REPORT_NAME: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
SHA: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.sha || github.event.pull_request.head.sha }}
BRANCH_NAME: "openpilot/pr-${{ github.event.number }}-ui-preview"
REPORT_FILES_BRANCH_NAME: "mici-raylib-ui-reports"
# variant:video_prefix:master_branch
VARIANTS: "mici:mici_ui_replay:openpilot_master_ui_mici_raylib big:tizi_ui_replay:openpilot_master_ui_big_raylib"
jobs:
preview:
if: github.repository == 'sunnypilot/sunnypilot'
name: preview
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
pull-requests: write
actions: read
steps:
- uses: actions/checkout@v6
with:
submodules: true
- name: Waiting for ui generation to end
uses: lewagon/wait-on-check-action@v1.3.4
with:
ref: ${{ env.SHA }}
check-name: ${{ env.UI_JOB_NAME }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
allowed-conclusions: success
wait-interval: 20
- name: Getting workflow run ID
id: get_run_id
run: |
echo "run_id=$(curl https://api.github.com/repos/${{ github.repository }}/commits/${{ env.SHA }}/check-runs | jq -r '.check_runs[] | select(.name == "${{ env.UI_JOB_NAME }}") | .html_url | capture("(?<number>[0-9]+)") | .number')" >> $GITHUB_OUTPUT
- name: Getting proposed ui
uses: dawidd6/action-download-artifact@v6
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
run_id: ${{ steps.get_run_id.outputs.run_id }}
search_artifacts: true
name: ui-report-1-${{ env.REPORT_NAME }}
path: ${{ github.workspace }}/pr_ui
- name: Getting mici master ui
uses: actions/checkout@v6
with:
repository: sunnypilot/ci-artifacts
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
path: ${{ github.workspace }}/master_mici
ref: openpilot_master_ui_mici_raylib
- name: Getting big master ui
uses: actions/checkout@v6
with:
repository: sunnypilot/ci-artifacts
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
path: ${{ github.workspace }}/master_big
ref: openpilot_master_ui_big_raylib
- name: Saving new master ui
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
run: |
for variant in $VARIANTS; do
IFS=':' read -r name video branch <<< "$variant"
master_dir="${{ github.workspace }}/master_${name}"
cd "$master_dir"
git checkout --orphan=new_branch
git rm -rf *
git branch -D "$branch"
git branch -m "$branch"
git config user.name "GitHub Actions Bot"
git config user.email "<>"
cp "${{ github.workspace }}/pr_ui/${video}.mp4" .
git add .
git commit -m "${name} video for commit ${{ env.SHA }}"
git push origin "$branch" --force
done
- name: Setup FFmpeg
uses: AnimMouse/setup-ffmpeg@ae28d57dabbb148eff63170b6bf7f2b60062cbae
- name: Finding diffs
if: github.event_name == 'pull_request_target'
id: find_diff
run: |
export PYTHONPATH=${{ github.workspace }}
baseurl="https://github.com/sunnypilot/ci-artifacts/raw/refs/heads/${{ env.BRANCH_NAME }}"
COMMENT=""
for variant in $VARIANTS; do
IFS=':' read -r name video _ <<< "$variant"
diff_name="${name}_diff"
mv "${{ github.workspace }}/pr_ui/${video}.mp4" "${{ github.workspace }}/pr_ui/${video}_proposed.mp4"
cp "${{ github.workspace }}/master_${name}/${video}.mp4" "${{ github.workspace }}/pr_ui/${video}_master.mp4"
diff_exit_code=0
python3 ${{ github.workspace }}/selfdrive/ui/tests/diff/diff.py \
"${{ github.workspace }}/pr_ui/${video}_master.mp4" \
"${{ github.workspace }}/pr_ui/${video}_proposed.mp4" \
"${diff_name}.html" --basedir "$baseurl" --no-open || diff_exit_code=$?
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.html" "${{ github.workspace }}/pr_ui/"
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.mp4" "${{ github.workspace }}/pr_ui/"
REPORT_URL="https://sunnypilot.github.io/ci-artifacts/${diff_name}_pr_${{ github.event.number }}.html"
if [ $diff_exit_code -eq 0 ]; then
COMMENT+="**${name}**: Videos are identical! [View Diff Report]($REPORT_URL)"$'\n'
else
COMMENT+="**${name}**: ⚠️ <strong>Videos differ!</strong> [View Diff Report]($REPORT_URL)"$'\n'
fi
done
{
echo "COMMENT<<EOF"
echo "$COMMENT"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Saving proposed ui
if: github.event_name == 'pull_request_target'
working-directory: ${{ github.workspace }}/master_mici
run: |
git config user.name "GitHub Actions Bot"
git config user.email "<>"
git checkout --orphan=${{ env.BRANCH_NAME }}
git rm -rf *
mv ${{ github.workspace }}/pr_ui/* .
git add .
git commit -m "ui videos for PR #${{ github.event.number }}"
git push origin ${{ env.BRANCH_NAME }} --force
# Append diff reports to report files branch
git fetch origin ${{ env.REPORT_FILES_BRANCH_NAME }}
git checkout ${{ env.REPORT_FILES_BRANCH_NAME }}
for variant in $VARIANTS; do
IFS=':' read -r name _ _ <<< "$variant"
diff_name="${name}_diff"
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.html" "${diff_name}_pr_${{ github.event.number }}.html"
git add "${diff_name}_pr_${{ github.event.number }}.html"
done
git commit -m "ui diff reports for PR #${{ github.event.number }}" || echo "No changes to commit"
git push origin ${{ env.REPORT_FILES_BRANCH_NAME }}
- name: Comment on PR
if: github.event_name == 'pull_request_target'
uses: thollander/actions-comment-pull-request@v2
with:
message: |
<!-- _(run_id_ui_preview **${{ github.run_id }}**)_ -->
## UI Preview
${{ steps.find_diff.outputs.COMMENT }}
comment_tag: run_id_ui_preview
pr_number: ${{ github.event.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -1,52 +0,0 @@
name: 'Wait for Tests'
description: 'Action to wait for workflow tests to start and complete'
inputs:
workflow:
description: 'The workflow file name to monitor'
required: true
default: 'tests.yaml'
branch:
description: 'The branch to monitor (defaults to current branch)'
required: false
default: ''
github-token:
description: 'GitHub token for API access'
required: true
wait-time:
description: 'Initial sleep time in seconds before monitoring starts'
required: false
default: '30'
should-wait-for-start:
description: 'Whether to wait for tests to start'
required: false
default: false
runs:
using: 'composite'
steps:
- name: Wait for tests to start
if: inputs.should-wait-for-start == 'true'
shell: bash
run: |
echo "Sleeping for ${{ inputs.wait-time }} seconds to give some time for the action to start and then we'll wait"
sleep ${{ inputs.wait-time }}
- name: Wait for tests to finish
shell: bash
run: |
BRANCH="${{ inputs.branch || github.head_ref || github.ref_name }}"
echo "Looking for workflow runs of ${{ inputs.workflow }} on branch $BRANCH"
RUN_ID=$(gh run list --workflow=${{ inputs.workflow }} --branch="$BRANCH" --limit=1 --json databaseId --jq '.[0].databaseId')
echo "Watching run ID: $RUN_ID"
gh run watch "$RUN_ID"
CONCLUSION=$(gh run view "$RUN_ID" --json conclusion --jq '.conclusion')
echo "Run concluded with: $CONCLUSION"
if [[ "$CONCLUSION" != "success" ]]; then
echo "❌ Workflow run failed with conclusion: $CONCLUSION"
exit 1
fi
env:
GITHUB_TOKEN: ${{ inputs.github-token }}
+2
View File
@@ -44,8 +44,10 @@ bin/
config.json
compile_commands.json
compare_runtime*.html
selfdrive/modeld/models/tg_input_devices.json
# build artifacts
docs_site/
selfdrive/pandad/pandad
cereal/services.h
cereal/gen
-21
View File
@@ -1,21 +0,0 @@
[submodule "panda"]
path = panda
url = https://github.com/sunnyhaibin/panda.git
[submodule "opendbc"]
path = opendbc_repo
url = https://github.com/sunnypilot/opendbc.git
[submodule "msgq"]
path = msgq_repo
url = https://github.com/commaai/msgq.git
[submodule "rednose_repo"]
path = rednose_repo
url = https://github.com/commaai/rednose.git
[submodule "teleoprtc_repo"]
path = teleoprtc_repo
url = https://github.com/commaai/teleoprtc
[submodule "tinygrad"]
path = tinygrad_repo
url = https://github.com/sunnypilot/tinygrad.git
[submodule "sunnypilot/neural_network_data"]
path = sunnypilot/neural_network_data
url = https://github.com/sunnypilot/neural-network-data.git
-25
View File
@@ -1,25 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CLionExternalBuildManager">
<target id="a62f99e8-5ec4-434c-8122-49efed5af108" name="uv Scons Build Debug" defaultType="TOOL">
<configuration id="b93ec964-16e5-4962-a12e-3ed360ce8f02" name="uv Scons Build Debug">
<build type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Build Debug" />
</build>
<clean type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Clean" />
</clean>
</configuration>
</target>
<target id="edd8ad9d-183b-467c-a355-0d9a0ecab026" name="uv Scons Build Release" defaultType="TOOL">
<configuration id="09523339-5ce3-4223-ab9e-904f38ad7752" name="uv Scons Build Release">
<build type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Build Release" />
</build>
<clean type="TOOL">
<tool actionId="Tool_External Tools_uv Scons Clean" />
</clean>
</configuration>
</target>
</component>
</project>
-23
View File
@@ -1,23 +0,0 @@
<toolSet name="External Tools">
<tool name="uv Scons Build Debug" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -u -j$(nproc) --ccflags=\&quot;-fno-inline\&quot;&quot;" />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
<tool name="uv Scons Clean" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -c&quot; " />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
<tool name="uv Scons Build Release" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
<exec>
<option name="COMMAND" value="bash" />
<option name="PARAMETERS" value="-c &quot;source .venv/bin/activate &amp;&amp; scons -u -j$(nproc)&quot; " />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
</exec>
</tool>
</toolSet>
-4
View File
@@ -1,4 +0,0 @@
[lfs]
url = https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git
locksverify = false
-4
View File
@@ -1,4 +0,0 @@
[lfs]
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
locksverify = false
-10
View File
@@ -1,10 +0,0 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build Debug" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Debug" CONFIG_NAME="uv Scons Build Debug" RUN_PATH="ui">
<envs>
<env name="QT_DBL_CLICK_DIST" value="150" />
</envs>
<method v="2">
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
</method>
</configuration>
</component>
-10
View File
@@ -1,10 +0,0 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build Release" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Release" CONFIG_NAME="uv Scons Build Release" RUN_PATH="ui">
<envs>
<env name="QT_DBL_CLICK_DIST" value="150" />
</envs>
<method v="2">
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
</method>
</configuration>
</component>
-26
View File
@@ -1,26 +0,0 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build_BIG_UI" type="PythonConfigurationType" factoryName="Python">
<module name="sunnypilot" />
<option name="ENV_FILES" value="" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<envs>
<env name="BIG" value="1" />
</envs>
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$/" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<option name="SCRIPT_NAME" value="$ProjectFileDir$/selfdrive/ui/ui.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2">
<option name="ToolBeforeRunTask" enabled="true" actionId="Tool_External Tools_uv Scons Build Debug" />
</method>
</configuration>
</component>
-23
View File
@@ -1,23 +0,0 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build_SMALL_UI" type="PythonConfigurationType" factoryName="Python">
<module name="sunnypilot" />
<option name="ENV_FILES" value="" />
<option name="INTERPRETER_OPTIONS" value="" />
<option name="PARENT_ENVS" value="true" />
<option name="SDK_HOME" value="" />
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$/" />
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<option name="SCRIPT_NAME" value="$ProjectFileDir$/selfdrive/ui/ui.py" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="false" />
<option name="MODULE_MODE" value="false" />
<option name="REDIRECT_INPUT" value="false" />
<option name="INPUT_FILE" value="" />
<method v="2">
<option name="ToolBeforeRunTask" enabled="true" actionId="Tool_External Tools_uv Scons Build Debug" />
</method>
</configuration>
</component>
-1
View File
@@ -21,7 +21,6 @@
"common/**",
"selfdrive/**",
"system/**",
"third_party/**",
"tools/**",
]
}
+118 -1
View File
@@ -1,4 +1,36 @@
sunnypilot Version 2026.001.000 (2026-03-xx)
sunnypilot Version 2026.002.000 (2026-06-28)
========================
* What's Changed (sunnypilot/sunnypilot)
* ui: update gates for certain toggles by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1830
* release: ignore upstream IsReleaseBranch by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1831
* manager: disable DEVELOPMENT_ONLY reset by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1833
* sunnylink: fix max time offroad values by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1835
* ui: show default model name by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1837
* sunnylink: add CarParams fallback for brand-specific capabilities by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1839
* sunnylink SDUI: tweak DisableUpdate param for clarity by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1842
* Revert "DM: Lancia Delta HF Integrale model" by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1849
* modeld_v2: safe model validation by @Discountchubbs in https://github.com/sunnypilot/sunnypilot/pull/1855
* Revert "deprecate `carState.brake`" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/sunnypilot/pull/1860
* sunnylink: deprecate legacy params metadata by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1862
* ui: reset Enforce Torque Control and NNLC if both are enabled by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1863
* What's Changed (sunnypilot/opendbc)
* Rivian: suppress ACM hold-the-wheel warning during MADS-only lateral by @lukasloetkolben in https://github.com/sunnypilot/opendbc/pull/465
* Sync: `commaai/opendbc:master``sunnypilot/opendbc:master` by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/479
* safety: add option to ignore frequency check for RX checks by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/480
* Revert "deprecate carState.brake" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/opendbc/pull/481
* New Contributors (sunnypilot/sunnypilot)
* @mvl-boston made their first contribution in https://github.com/sunnypilot/sunnypilot/pull/1860
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2026.001.007...v2026.002.000
************************
* Synced with commaai's openpilot (v0.11.1)
* master commit 69e2c321e49760e52f7983eaa0a5f77cb95de637 (June 02, 2026)
* New driver monitoring model
* Improved image processing pipeline for driver camera
* Improved thermal policy for comma four
* Acura MDX 2022-24 support thanks to mvl-boston!
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
sunnypilot Version 2026.001.000 (2026-05-06)
========================
* What's Changed (sunnypilot/sunnypilot)
* Complete rewrite of the user interface from Qt C++ to Raylib Python
@@ -66,6 +98,64 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
* Pause Lateral Control with Blinker: Post-Blinker Delay by @CHaucke89
* SCC-V: Use p97 for predicted lateral accel by @yasu-oh
* Controls: Support for Torque Lateral Control v0 Tune by @sunnyhaibin
* [TIZI/TICI] ui: ensure null checks for `CarParams` and `CarParamsSP` by @sunnyhaibin
* [TIZI/TICI] ui: use `vCruiseCluster` and `vEgoCluster` for SLA `preActive` by @sunnyhaibin
* Fix display of values when using use_float_scaling by @CHaucke89
* models: fix default & index "0" by @nayan8teen
* [TIZI/TICI] visuals: Improved speed limit by @angaz
* ICBM: ensure button timers update on disable to clear stale presses by @jamesmikesell
* [TIZI/TICI] ui: simplify Smart Cruise Control text rendering by @sunnyhaibin
* controlsd: fix steer_limited_by_safety not updating under MADS by @zephleggett
* soundd: trigger timeout warning during MADS lateral-only by @zephleggett
* pandad: flasher for Rivian long upgrade module by @lukasloetkolben
* modeld_v2: tinygrad transformation warp by @Discountchubbs
* tools: block `manage_sunnylinkd` in sim startup script by @sunnyhaibin
* [MICI] ui: need superclass `_render` in `HudRendererSP` by @sunnyhaibin
* [TIZI/TICI] ui: Speed Limit Assist active status by @sunnyhaibin
* ui: reimplement "Screen Off" option to Onroad Brightness by @sunnyhaibin
* ui: don't hide steering wheel when blindspot disabled by @royjr
* ui: Speed Limit Assist `preActive` improvements by @sunnyhaibin
* ui: consolidate Speed Limit Assist `preActive` status rendering by @sunnyhaibin
* [MICI] ui: Speed Limit Assist `preActive` status by @sunnyhaibin
* sunnypilot modeld: remove thneed modeld by @Discountchubbs
* modeld_v2: decouple planplus scaling from accel by @Discountchubbs
* sunnylink: Handle exceptions in `getParamsAllKeysV1` to log crashes by @devtekve
* [TIZI/TICI] ui: Developer UI cleanup by @sunnyhaibin
* [TIZI/TICI] ui: dynamic alert size by @nayan8teen
* i18n(fr): Add French translations by @didlawowo
* Toyota: Stop and Go Hack (Alpha) by @sunnyhaibin
* ui: `AlertFadeAnimator` for longitudinal-related statuses by @sunnyhaibin
* pandad: gate unsupported pandas before flashing by @sunnyhaibin
* Rivian: Flash xnor's Longitudinal Upgrade Kit prior supported panda check by @lukasloetkolben
* [TIZI/TICI] ui: add back gate steering arc behind toggle by @sunnyhaibin
* ui: gate Onroad Brightness Delay on readiness by @sunnyhaibin
* ui: add new timer options for Onroad Brightness Delay by @sunnyhaibin
* [TIZI/TICI] ui: branch switcher is always available by @sunnyhaibin
* pandad: always prioritize internal panda by @sunnyhaibin
* sunnylinkd: fetch compressed params schema by @sunnyhaibin
* sunnypilot locationd: remove unused car_ekf filter by @sunnyhaibin
* modeld_v2: update deprecated temporalPose ref by @sunnyhaibin
* NNLC: restore pre-v1 PID gains in torque extension by @mmmorks
* MADS safety: enable heartbeat and lateral controls mismatch checks by @sunnyhaibin
* [MICI] ui: models panel enhancements by @nayan8teen
* [TIZI/TICI] ui: fix unintended selection while scrolling in TreeOptionDialog by @TheSecurityDev
* tools: script for video concatenation by @Discountchubbs
* tools: profile memory usage by @Discountchubbs
* [TIZI/TICI] ui: remove per-frame param sync by @sunnyhaibin
* [MICI] ui: always offroad by @nayan8teen
* controls: always default Torque Lateral Control to v0 Tune by @sunnyhaibin
* Revert "controls: always default Torque Lateral Control to v0 Tune" by @sunnyhaibin
* Reapply "controls: always default Torque Lateral Control to v0 Tune" (#1806) by @sunnyhaibin
* [MICI] ui: add sunnylink info & connectivity check by @nayan8teen
* sunnylink: Remove unused API endpoint by @devtekve
* DM: wheel touch enforcement in MADS by @sunnyhaibin
* torque: show static override values in Dev UI & gate `useParams` on custom torque tune by @sunnyhaibin
* MADS: suppress espActive event when long is not engaged by @sunnyhaibin
* sunnylink: SDUI by @sunnyhaibin
* [MICI] ui: align upstream changes with sunnypilot settings buttons by @nayan8teen
* ui: fix cellular toggles by @AmyJeanes
* sunnylink: switch athena domain by @devtekve
* Platform List: dynamically migrate CarPlatformBundle by @sunnyhaibin
* What's Changed (sunnypilot/opendbc)
* Honda: DBC for Accord 9th Generation by @mvl-boston
* FCA: update tire stiffness values for `RAM_HD` by @dparring
@@ -84,12 +174,25 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
* Honda: add missing `GasInterceptor` messages to Taiwan Odyssey DBC by @mvl-boston
* GM: remove `CHEVROLET_EQUINOX_NON_ACC_3RD_GEN` from `dashcamOnly` by @sunnyhaibin
* GM: remove `CHEVROLET_BOLT_NON_ACC_2ND_GEN` from `dashcamOnly` by @sunnyhaibin
* Hyundai Longitudinal: deprecate ramp update for dynamic tune by @Discountchubbs
* Rivian: long upgrade messages on bus 1 by @lukasloetkolben
* Toyota: Stop and Go Hack (Alpha) by @sunnyhaibin
* Toyota: gate Smart DSU behind Alpha Longitudinal by @sunnyhaibin
* Toyota: Gas Interceptor always set `standstill_req` by @sunnyhaibin
* MADS safety: dedicated `controls_allowed_lateral` by @sunnyhaibin
* Platform List: include community supported platforms by @sunnyhaibin
* New Contributors (sunnypilot/sunnypilot)
* @TheSecurityDev made their first contribution in "ui: fix sidebar scroll in UI screenshots"
* @zikeji made their first contribution in "sunnylink: block remote modification of SSH key parameters"
* @Candy0707 made their first contribution in "[TIZI/TICI] ui: Fix misaligned turn signals and blindspot indicators with sidebar"
* @CHaucke89 made their first contribution in "Pause Lateral Control with Blinker: Post-Blinker Delay"
* @yasu-oh made their first contribution in "SCC-V: Use p97 for predicted lateral accel"
* @angaz made their first contribution in "[TIZI/TICI] visuals: Improved speed limit"
* @jamesmikesell made their first contribution in "ICBM: ensure button timers update on disable to clear stale presses"
* @zephleggett made their first contribution in "controlsd: fix steer_limited_by_safety not updating under MADS"
* @lukasloetkolben made their first contribution in "pandad: flasher for Rivian long upgrade module"
* @didlawowo made their first contribution in "i18n(fr): Add French translations"
* @mmmorks made their first contribution in "NNLC: restore pre-v1 PID gains in torque extension"
* New Contributors (sunnypilot/opendbc)
* @AmyJeanes made their first contribution in "Tesla: Fix stock LKAS being blocked when MADS is enabled"
* @mvl-boston made their first contribution in "Honda: Update Clarity brake to renamed DBC message name"
@@ -99,6 +202,20 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
* @royjr made their first contribution in "HKG: add KIA_FORTE_2019_NON_SCC fingerprint"
* @ssysm made their first contribution in "Tesla: remove `TESLA_MODEL_X` from `dashcamOnly`"
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2025.002.000...v2026.001.000
************************
* Synced with commaai's openpilot (v0.11.1)
* master commit c001f3c9b490a80e69539f0af6022f6e07ceb721 (April 16, 2026)
* New driver monitoring model
* Improved image processing pipeline for driver camera
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
* New driving model #36798
* Fully trained using a learned simulator
* Improved longitudinal performance in Experimental mode
* Reduce comma four standby power usage by 77% to 52 mW
* Kia K7 2017 support thanks to royjr!
* Lexus LS 2018 support thanks to Hacheoy!
* Improved inter-process communication memory efficiency
* comma four support
sunnypilot Version 2025.002.000 (2025-11-06)
========================
-38
View File
@@ -1,38 +0,0 @@
FROM ubuntu:24.04
ENV PYTHONUNBUFFERED=1
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends sudo tzdata locales && \
rm -rf /var/lib/apt/lists/*
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute
ARG USER=batman
ARG USER_UID=1001
RUN useradd -m -s /bin/bash -u $USER_UID $USER
RUN usermod -aG sudo $USER
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER $USER
ENV OPENPILOT_PATH=/home/$USER/openpilot
RUN mkdir -p ${OPENPILOT_PATH}
WORKDIR ${OPENPILOT_PATH}
COPY --chown=$USER . ${OPENPILOT_PATH}/
ENV UV_BIN="/home/$USER/.local/bin/"
ENV VIRTUAL_ENV=${OPENPILOT_PATH}/.venv
ENV PATH="$UV_BIN:$VIRTUAL_ENV/bin:$PATH"
RUN tools/setup_dependencies.sh && \
sudo rm -rf /var/lib/apt/lists/*
USER root
RUN git config --global --add safe.directory '*'
Vendored
-260
View File
@@ -1,260 +0,0 @@
def retryWithDelay(int maxRetries, int delay, Closure body) {
for (int i = 0; i < maxRetries; i++) {
try {
return body()
} catch (Exception e) {
sleep(delay)
}
}
throw Exception("Failed after ${maxRetries} retries")
}
def device(String ip, String step_label, String cmd) {
withCredentials([file(credentialsId: 'id_rsa', variable: 'key_file')]) {
def ssh_cmd = """
ssh -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec /usr/bin/bash <<'END'
set -e
export TERM=xterm-256color
shopt -s huponexit # kill all child processes when the shell exits
export CI=1
export PYTHONWARNINGS=error
#export LOGPRINT=debug # this has gotten too spammy...
export TEST_DIR=${env.TEST_DIR}
export SOURCE_DIR=${env.SOURCE_DIR}
export GIT_BRANCH=${env.GIT_BRANCH}
export GIT_COMMIT=${env.GIT_COMMIT}
export CI_ARTIFACTS_TOKEN=${env.CI_ARTIFACTS_TOKEN}
export GITHUB_COMMENTS_TOKEN=${env.GITHUB_COMMENTS_TOKEN}
export AZURE_TOKEN='${env.AZURE_TOKEN}'
# only use 1 thread for tici tests since most require HIL
export PYTEST_ADDOPTS="-n0 -s"
export GIT_SSH_COMMAND="ssh -i /data/gitkey"
source ~/.bash_profile
if [ -f /TICI ]; then
source /etc/profile
rm -rf /tmp/tmp*
rm -rf ~/.commacache
rm -rf /dev/shm/*
rm -rf /dev/tmp/tmp*
if ! systemctl is-active --quiet systemd-resolved; then
echo "restarting resolved"
sudo systemctl start systemd-resolved
sleep 3
fi
# restart aux USB
if [ -e /sys/bus/usb/drivers/hub/3-0:1.0 ]; then
echo "restarting aux usb"
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/unbind
sleep 0.5
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/bind
fi
fi
if [ -f /data/openpilot/launch_env.sh ]; then
source /data/openpilot/launch_env.sh
fi
ln -snf ${env.TEST_DIR} /data/pythonpath
cd ${env.TEST_DIR} || true
time ${cmd}
END"""
sh script: ssh_cmd, label: step_label
}
}
def deviceStage(String stageName, String deviceType, List extra_env, def steps) {
stage(stageName) {
if (currentBuild.result != null) {
return
}
if (isReplay()) {
error("REPLAYING TESTS IS NOT ALLOWED. FIX THEM INSTEAD.")
}
def extra = extra_env.collect { "export ${it}" }.join('\n');
def branch = env.BRANCH_NAME ?: 'master';
def gitDiff = sh returnStdout: true, script: 'curl -s -H "Authorization: Bearer ${GITHUB_COMMENTS_TOKEN}" https://api.github.com/repos/commaai/openpilot/compare/master...${GIT_BRANCH} | jq .files[].filename || echo "/"', label: 'Getting changes'
lock(resource: "", label: deviceType, inversePrecedence: true, variable: 'device_ip', quantity: 1, resourceSelectStrategy: 'random') {
docker.image('ghcr.io/commaai/alpine-ssh').inside('--user=root') {
timeout(time: 35, unit: 'MINUTES') {
retry (3) {
def date = sh(script: 'date', returnStdout: true).trim();
device(device_ip, "set time", "date -s '" + date + "'")
device(device_ip, "git checkout", extra + "\n" + readFile("selfdrive/test/setup_device_ci.sh"))
}
steps.each { item ->
def name = item[0]
def cmd = item[1]
def args = item[2]
def diffPaths = args.diffPaths ?: []
def cmdTimeout = args.timeout ?: 9999
if (branch != "master" && !branch.contains("__jenkins_loop_") && diffPaths && !hasPathChanged(gitDiff, diffPaths)) {
println "Skipping ${name}: no changes in ${diffPaths}."
return
} else {
timeout(time: cmdTimeout, unit: 'SECONDS') {
device(device_ip, name, cmd)
}
}
}
}
}
}
}
}
def hasPathChanged(String gitDiff, List<String> paths) {
for (path in paths) {
if (gitDiff.contains(path)) {
return true
}
}
return false
}
def isReplay() {
def replayClass = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
return currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replayClass) }
}
def setupCredentials() {
withCredentials([
string(credentialsId: 'azure_token', variable: 'AZURE_TOKEN'),
]) {
env.AZURE_TOKEN = "${AZURE_TOKEN}"
}
withCredentials([
string(credentialsId: 'ci_artifacts_pat', variable: 'CI_ARTIFACTS_TOKEN'),
]) {
env.CI_ARTIFACTS_TOKEN = "${CI_ARTIFACTS_TOKEN}"
}
withCredentials([
string(credentialsId: 'post_comments_github_pat', variable: 'GITHUB_COMMENTS_TOKEN'),
]) {
env.GITHUB_COMMENTS_TOKEN = "${GITHUB_COMMENTS_TOKEN}"
}
}
def step(String name, String cmd, Map args = [:]) {
return [name, cmd, args]
}
node {
env.CI = "1"
env.PYTHONWARNINGS = "error"
env.TEST_DIR = "/data/openpilot"
env.SOURCE_DIR = "/data/openpilot_source/"
setupCredentials()
env.GIT_BRANCH = checkout(scm).GIT_BRANCH
env.GIT_COMMIT = checkout(scm).GIT_COMMIT
def excludeBranches = ['__nightly', 'devel', 'devel-staging', 'release3', 'release3-staging',
'release-tici', 'release-tizi', 'release-tizi-staging', 'release-mici-staging', 'testing-closet*', 'hotfix-*']
def excludeRegex = excludeBranches.join('|').replaceAll('\\*', '.*')
if (env.BRANCH_NAME != 'master' && !env.BRANCH_NAME.contains('__jenkins_loop_')) {
properties([
disableConcurrentBuilds(abortPrevious: true)
])
}
try {
if (env.BRANCH_NAME == 'devel-staging') {
deviceStage("build release-tizi-staging", "tizi-needs-can", [], [
step("build release-tizi-staging", "RELEASE_BRANCH=release-tizi-staging $SOURCE_DIR/release/build_release.sh && git push -f origin release-tizi-staging:release-mici-staging"),
])
}
if (env.BRANCH_NAME == '__nightly') {
parallel (
'nightly': {
deviceStage("build nightly", "tizi-needs-can", [], [
step("build nightly", "RELEASE_BRANCH=nightly $SOURCE_DIR/release/build_release.sh"),
])
},
'nightly-dev': {
deviceStage("build nightly-dev", "tizi-needs-can", [], [
step("build nightly-dev", "PANDA_DEBUG_BUILD=1 RELEASE_BRANCH=nightly-dev $SOURCE_DIR/release/build_release.sh"),
])
},
)
}
if (!env.BRANCH_NAME.matches(excludeRegex)) {
parallel (
'onroad tests': {
deviceStage("onroad", "tizi-needs-can", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("check dirty", "release/check-dirty.sh"),
step("onroad tests", "pytest selfdrive/test/test_onroad.py -s", [timeout: 60]),
])
},
'HW + Unit Tests': {
deviceStage("tizi-hardware", "tizi-common", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test power draw", "pytest -s system/hardware/tici/tests/test_power_draw.py"),
step("test encoder", "LD_LIBRARY_PATH=/usr/local/lib pytest system/loggerd/tests/test_encoder.py", [diffPaths: ["system/loggerd/"]]),
step("test manager", "pytest system/manager/test/test_manager.py"),
])
},
'camerad OX03C10': {
deviceStage("OX03C10", "tizi-ox03c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 90]),
])
},
'camerad OS04C10': {
deviceStage("OS04C10", "tici-os04c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 90]),
])
},
'sensord': {
deviceStage("LSM + MMC", "tizi-lsmc", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test sensord", "pytest system/sensord/tests/test_sensord.py"),
])
},
'replay': {
deviceStage("model-replay", "tizi-replay", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
step("model replay", "selfdrive/test/process_replay/model_replay.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
])
},
'tizi': {
deviceStage("tizi", "tizi", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("test pandad loopback", "pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
step("test pandad spi", "pytest selfdrive/pandad/tests/test_pandad_spi.py"),
step("test amp", "pytest system/hardware/tici/tests/test_amplifier.py"),
step("test qcomgpsd", "pytest system/qcomgpsd/tests/test_qcomgpsd.py", [diffPaths: ["system/qcomgpsd/"]]),
])
},
)
}
} catch (Exception e) {
currentBuild.result = 'FAILED'
throw e
}
}
+7 -1
View File
@@ -1,7 +1,13 @@
Version 0.11.1 (2026-04-22)
Version 0.11.2 (2026-06-15)
========================
Version 0.11.1 (2026-05-18)
========================
* New driver monitoring model
* Improved image processing pipeline for driver camera
* Improved thermal policy for comma four
* Acura MDX 2022-24 support thanks to mvl-boston!
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
Version 0.11.0 (2026-03-17)
-269
View File
@@ -1,269 +0,0 @@
import os
import subprocess
import sys
import sysconfig
import platform
import shlex
import importlib
import numpy as np
import SCons.Errors
from SCons.Defaults import _stripixes
SCons.Warnings.warningAsException(True)
Decider('MD5-timestamp')
SetOption('num_jobs', max(1, int(os.cpu_count()/2)))
AddOption('--ccflags', action='store', type='string', default='', help='pass arbitrary flags over the command line')
AddOption('--verbose', action='store_true', default=False, help='show full build commands')
AddOption('--minimal',
action='store_false',
dest='extras',
default=os.path.exists(File('#.gitattributes').abspath), # minimal by default on release branch (where there's no LFS)
help='the minimum build to run openpilot. no tests, tools, etc.')
# Detect platform
arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
if platform.system() == "Darwin":
arch = "Darwin"
elif arch == "aarch64" and os.path.isfile('/TICI'):
arch = "larch64"
assert arch in [
"larch64", # linux tici arm64
"aarch64", # linux pc arm64
"x86_64", # linux pc x64
"Darwin", # macOS arm64 (x86 not supported)
]
pkg_names = ['bzip2', 'capnproto', 'eigen', 'ffmpeg', 'libjpeg', 'libyuv', 'ncurses', 'zeromq', 'zstd']
pkgs = [importlib.import_module(name) for name in pkg_names]
# ***** enforce a whitelist of system libraries *****
# this prevents silently relying on a 3rd party package,
# e.g. apt-installed libusb. all libraries should either
# be distributed with all Linux distros and macOS, or
# vendored in commaai/dependencies.
allowed_system_libs = {
"EGL", "GLESv2", "GL",
"Qt5Charts", "Qt5Core", "Qt5Gui", "Qt5Widgets",
"dl", "drm", "gbm", "m", "pthread",
}
def _resolve_lib(env, name):
for d in env.Flatten(env.get('LIBPATH', [])):
p = Dir(str(d)).abspath
for ext in ('.a', '.so', '.dylib'):
f = File(os.path.join(p, f'lib{name}{ext}'))
if f.exists() or f.has_builder():
return name
if name in allowed_system_libs:
return name
raise SCons.Errors.UserError(f"Unexpected non-vendored library '{name}'")
def _libflags(target, source, env, for_signature):
libs = []
lp = env.subst('$LIBLITERALPREFIX')
for lib in env.Flatten(env.get('LIBS', [])):
if isinstance(lib, str):
if os.sep in lib or lib.startswith('#'):
libs.append(File(lib))
elif lib.startswith('-') or (lp and lib.startswith(lp)):
libs.append(lib)
else:
libs.append(_resolve_lib(env, lib))
else:
libs.append(lib)
return _stripixes(env['LIBLINKPREFIX'], libs, env['LIBLINKSUFFIX'],
env['LIBPREFIXES'], env['LIBSUFFIXES'], env, env['LIBLITERALPREFIX'])
env = Environment(
ENV={
"PATH": os.environ['PATH'],
"PYTHONPATH": Dir("#").abspath + ':' + Dir(f"#third_party/acados").abspath,
"ACADOS_SOURCE_DIR": Dir("#third_party/acados").abspath,
"ACADOS_PYTHON_INTERFACE_PATH": Dir("#third_party/acados/acados_template").abspath,
"TERA_PATH": Dir("#").abspath + f"/third_party/acados/{arch}/t_renderer"
},
CCFLAGS=[
"-g",
"-fPIC",
"-O2",
"-Wunused",
"-Werror",
"-Wshadow" if arch in ("Darwin", "larch64") else "-Wshadow=local",
"-Wno-unknown-warning-option",
"-Wno-inconsistent-missing-override",
"-Wno-c99-designator",
"-Wno-reorder-init-list",
"-Wno-vla-cxx-extension",
],
CFLAGS=["-std=gnu11"],
CXXFLAGS=["-std=c++1z"],
CPPPATH=[
"#",
"#msgq",
"#third_party",
"#third_party/json11",
"#third_party/linux/include",
"#third_party/acados/include",
"#third_party/acados/include/blasfeo/include",
"#third_party/acados/include/hpipm/include",
"#third_party/catch2/include",
[x.INCLUDE_DIR for x in pkgs],
],
LIBPATH=[
"#common",
"#msgq_repo",
"#third_party",
"#selfdrive/pandad",
"#rednose/helpers",
f"#third_party/acados/{arch}/lib",
[x.LIB_DIR for x in pkgs],
],
RPATH=[],
CYTHONCFILESUFFIX=".cpp",
COMPILATIONDB_USE_ABSPATH=True,
REDNOSE_ROOT="#",
tools=["default", "cython", "compilation_db", "rednose_filter"],
toolpath=["#site_scons/site_tools", "#rednose_repo/site_scons/site_tools"],
)
if arch != "larch64":
env['_LIBFLAGS'] = _libflags
# Arch-specific flags and paths
if arch == "larch64":
env["CC"] = "clang"
env["CXX"] = "clang++"
env.Append(LIBPATH=[
"/usr/lib/aarch64-linux-gnu",
])
arch_flags = ["-D__TICI__", "-mcpu=cortex-a57", "-DQCOM2"]
env.Append(CCFLAGS=arch_flags)
env.Append(CXXFLAGS=arch_flags)
elif arch == "Darwin":
env.Append(LIBPATH=[
"/System/Library/Frameworks/OpenGL.framework/Libraries",
])
env.Append(CCFLAGS=["-DGL_SILENCE_DEPRECATION"])
env.Append(CXXFLAGS=["-DGL_SILENCE_DEPRECATION"])
_extra_cc = shlex.split(GetOption('ccflags') or '')
if _extra_cc:
env.Append(CCFLAGS=_extra_cc)
# no --as-needed on mac linker
if arch != "Darwin":
env.Append(LINKFLAGS=["-Wl,--as-needed", "-Wl,--no-undefined"])
# Shorter build output: show brief descriptions instead of full commands.
# Full command lines are still printed on failure by scons.
if not GetOption('verbose'):
for action, short in (
("CC", "CC"),
("CXX", "CXX"),
("LINK", "LINK"),
("SHCC", "CC"),
("SHCXX", "CXX"),
("SHLINK", "LINK"),
("AR", "AR"),
("RANLIB", "RANLIB"),
("AS", "AS"),
):
env[f"{action}COMSTR"] = f" [{short}] $TARGET"
# progress output
node_interval = 5
node_count = 0
def progress_function(node):
global node_count
node_count += node_interval
sys.stderr.write("progress: %d\n" % node_count)
if os.environ.get('SCONS_PROGRESS'):
Progress(progress_function, interval=node_interval)
# ********** Cython build environment **********
envCython = env.Clone()
envCython["CPPPATH"] += [sysconfig.get_paths()['include'], np.get_include()]
envCython["CCFLAGS"] += ["-Wno-#warnings", "-Wno-cpp", "-Wno-shadow", "-Wno-deprecated-declarations"]
envCython["CCFLAGS"].remove("-Werror")
envCython["LIBS"] = []
if arch == "Darwin":
envCython["LINKFLAGS"] = env["LINKFLAGS"] + ["-bundle", "-undefined", "dynamic_lookup"]
else:
envCython["LINKFLAGS"] = ["-pthread", "-shared"]
np_version = SCons.Script.Value(np.__version__)
Export('envCython', 'np_version')
Export('env', 'arch')
# Setup cache dir
default_cache_dir = '/data/scons_cache' if arch == "larch64" else '/tmp/scons_cache'
cache_dir = ARGUMENTS.get('cache_dir', default_cache_dir)
CacheDir(cache_dir)
Clean(["."], cache_dir)
# ********** start building stuff **********
# Build common module
SConscript(['common/SConscript'])
Import('_common')
common = [_common, 'json11', 'zmq']
Export('common')
# Build messaging (cereal + msgq + socketmaster + their dependencies)
# Enable swaglog include in submodules
env_swaglog = env.Clone()
env_swaglog['CXXFLAGS'].append('-DSWAGLOG="\\"common/swaglog.h\\""')
SConscript(['msgq_repo/SConscript'], exports={'env': env_swaglog})
SConscript(['cereal/SConscript'])
Import('socketmaster', 'msgq')
messaging = [socketmaster, msgq, 'capnp', 'kj',]
Export('messaging')
# Build other submodules
SConscript(['panda/SConscript'])
# Build rednose library
SConscript(['rednose/SConscript'])
# Build system services
SConscript([
'system/loggerd/SConscript',
])
if arch == "larch64":
SConscript(['system/camerad/SConscript'])
# Build openpilot
SConscript(['third_party/SConscript'])
# Build selfdrive
SConscript([
'selfdrive/pandad/SConscript',
'selfdrive/controls/lib/lateral_mpc_lib/SConscript',
'selfdrive/controls/lib/longitudinal_mpc_lib/SConscript',
'selfdrive/locationd/SConscript',
'selfdrive/modeld/SConscript',
'selfdrive/ui/SConscript',
])
SConscript(['sunnypilot/SConscript'])
# Build tools
if arch != "larch64":
SConscript([
'tools/replay/SConscript',
'tools/cabana/SConscript',
'tools/jotpluggler/SConscript',
])
env.CompilationDatabase('compile_commands.json')
-20
View File
@@ -1,20 +0,0 @@
Import('env', 'common', 'msgq')
cereal_dir = Dir('.')
gen_dir = Dir('gen')
# Build cereal
schema_files = ['log.capnp', 'car.capnp', 'deprecated.capnp', 'custom.capnp']
env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files],
schema_files,
f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/")
cereal = env.Library('cereal', [f'gen/cpp/{s}.c++' for s in schema_files])
# Build messaging
services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET')
env.Program('messaging/bridge', ['messaging/bridge.cc', 'messaging/msgq_to_zmq.cc', 'messaging/bridge_zmq.cc'], LIBS=[msgq, common, 'pthread'])
socketmaster = env.Library('socketmaster', ['messaging/socketmaster.cc'])
Export('cereal', 'socketmaster')
-12
View File
@@ -760,18 +760,6 @@ struct LateralLQRState @0x9024e2d790c82ade {
steeringAngleDesiredDeg @6 :Float32;
}
struct LateralCurvatureState @0xad9d8095c06f7c61 {
active @0 :Bool;
actualCurvature @1 :Float32;
desiredCurvature @2 :Float32;
error @3 :Float32;
p @4 :Float32;
i @5 :Float32;
f @6 :Float32;
output @7 :Float32;
saturated @8 :Bool;
}
struct LateralPlannerSolution @0x84caeca5a6b4acfe {
x @0 :List(Float32);
y @1 :List(Float32);
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+113 -29
View File
@@ -273,11 +273,7 @@ struct GPSNMEAData {
nmea @2 :Text;
}
# android sensor_event_t
struct SensorEventData {
version @0 :Int32;
sensor @1 :Int32;
type @2 :Int32;
timestamp @3 :Int64;
union {
@@ -296,7 +292,10 @@ struct SensorEventData {
struct SensorVec {
v @0 :List(Float32);
status @1 :Int8;
deprecated :group {
status @1 :Int8;
}
}
enum SensorSource {
@@ -314,7 +313,11 @@ struct SensorEventData {
mmc5603nj @11;
}
# formerly based on android sensor_event_t
deprecated :group {
version @0 :Int32;
sensor @1 :Int32;
type @2 :Int32;
uncalibrated @10 :Bool;
}
}
@@ -457,10 +460,10 @@ struct DeviceState @0xa4d8b5af2aa492eb {
}
enum ThermalStatus {
green @0;
yellow @1;
red @2;
danger @3;
ok @0;
warmDEPRECATED @1;
overheated @2;
critical @3;
}
enum NetworkType {
@@ -815,7 +818,7 @@ struct ControlsState @0x97ff69c53601abf1 {
debugState @59 :LateralDebugState;
torqueState @60 :LateralTorqueState;
curvatureStateDEPRECATED @65 :Deprecated.LateralCurvatureState;
curvatureState @65 :LateralCurvatureState;
lqrStateDEPRECATED @55 :Deprecated.LateralLQRState;
indiStateDEPRECATED @52 :Deprecated.LateralINDIState;
}
@@ -864,6 +867,18 @@ struct ControlsState @0x97ff69c53601abf1 {
saturated @3 :Bool;
}
struct LateralCurvatureState @0xad9d8095c06f7c61 {
active @0 :Bool;
actualCurvature @1 :Float32;
desiredCurvature @2 :Float32;
error @3 :Float32;
p @4 :Float32;
i @5 :Float32;
f @6 :Float32;
output @7 :Float32;
saturated @8 :Bool;
}
deprecated :group {
vEgo @0 :Float32;
vEgoRaw @32 :Float32;
@@ -2054,16 +2069,15 @@ struct DriverStateV2 {
facePosition @2 :List(Float32);
facePositionStd @3 :List(Float32);
faceProb @4 :Float32;
eyesVisibleProb @14 :Float32;
eyesClosedProb @15 :Float32;
leftEyeProb @5 :Float32;
rightEyeProb @6 :Float32;
leftBlinkProb @7 :Float32;
rightBlinkProb @8 :Float32;
sunglassesProb @9 :Float32;
phoneProb @13 :Float32;
sleepProb @14 :Float32;
deprecated :group {
leftEyeProb @5 :Float32;
rightEyeProb @6 :Float32;
leftBlinkProb @7 :Float32;
rightBlinkProb @8 :Float32;
sunglassesProb @9 :Float32;
notReadyProb @12 :List(Float32);
occludedProb @10 :Float32;
readyProb @11 :List(Float32);
@@ -2076,7 +2090,7 @@ struct DriverStateV2 {
}
}
struct DriverMonitoringState @0xb83cda094a1da284 {
struct DriverMonitoringStateDEPRECATED @0xb83cda094a1da284 {
events @18 :List(OnroadEvent);
faceDetected @1 :Bool;
isDistracted @2 :Bool;
@@ -2104,6 +2118,75 @@ struct DriverMonitoringState @0xb83cda094a1da284 {
}
}
struct DriverMonitoringState {
lockout @0 :Bool;
alertCountLockoutPercent @1 :Int8;
alertTimeLockoutPercent @2 :Int8;
alwaysOn @3 :Bool;
alwaysOnLockout @4 :Bool;
alertLevel @5 :AlertLevel;
activePolicy @6 :MonitoringPolicy;
isRHD @7 :Bool;
rhdCalibration @8 :CalibrationState;
visionPolicyState @9 :VisionPolicyState;
wheeltouchPolicyState @10 :WheeltouchPolicyState;
enum AlertLevel {
# ordinal must match the name to prevent bugs
# comparing against the raw ordinal value
none @0;
one @1;
two @2;
three @3;
}
enum MonitoringPolicy {
wheeltouch @0;
vision @1;
}
struct VisionPolicyState {
awarenessPercent @0 :Int8;
awarenessStep @1 :Float32;
isDistracted @2 :Bool;
distractedTypes @3 :DistractedTypes;
faceDetected @4 :Bool;
pose @5 :Pose;
wheeltouchFallbackPercent @6 :Int8;
uncertainOffroadAlertPercent @7 :Int8;
struct DistractedTypes {
pose @0: Bool;
eye @1: Bool;
phone @2: Bool;
}
struct Pose {
pitch @0 :Float32;
yaw @1 :Float32;
pitchCalib @2 :CalibrationState;
yawCalib @3 :CalibrationState;
calibrated @4 :Bool;
uncertainty @5 :Float32;
}
}
struct WheeltouchPolicyState {
awarenessPercent @0 :Int8;
awarenessStep @1 :Float32;
driverInteracting @2 :Bool;
}
struct CalibrationState {
calibratedPercent @0 :Int8;
offset @1 :Float32;
}
}
struct Boot {
wallTimeNanos @0 :UInt64;
pstore @4 :Map(Text, Data);
@@ -2176,6 +2259,7 @@ struct LiveDelayData {
lateralDelayEstimateStd @5 :Float32;
points @4 :List(Float32);
calPerc @6 :Int8;
version @7 :Int32;
enum Status {
unestimated @0;
@@ -2229,7 +2313,8 @@ struct Sentinel {
}
struct UIDebug {
drawTimeMillis @0 :Float32;
cpuTimeMillis @0 :Float32;
frameTimeMillis @1 :Float32;
}
struct ManagerState {
@@ -2377,7 +2462,6 @@ struct Event {
boot @60 :Boot;
# ********** openpilot daemon msgs **********
gpsNMEA @3 :GPSNMEAData;
can @5 :List(CanData);
controlsState @7 :ControlsState;
selfdriveState @130 :SelfdriveState;
@@ -2402,7 +2486,6 @@ struct Event {
qcomGnss @31 :QcomGnss;
gpsLocationExternal @48 :GpsLocationData;
gpsLocation @21 :GpsLocationData;
gnssMeasurements @91 :GnssMeasurements;
liveParameters @61 :LiveParametersData;
liveTorqueParameters @94 :LiveTorqueParametersData;
liveDelay @146 : LiveDelayData;
@@ -2410,7 +2493,7 @@ struct Event {
thumbnail @66: Thumbnail;
onroadEvents @134: List(OnroadEvent);
carParams @69: Car.CarParams;
driverMonitoringState @71: DriverMonitoringState;
driverMonitoringState @151 :DriverMonitoringState;
livePose @129 :LivePose;
modelV2 @75 :ModelDataV2;
drivingModelData @128 :DrivingModelData;
@@ -2436,7 +2519,6 @@ struct Event {
# systems stuff
androidLog @20 :AndroidLogEntry;
managerState @78 :ManagerState;
uploaderState @79 :UploaderState;
procLog @33 :ProcLog;
clocks @35 :Clocks;
deviceState @6 :DeviceState;
@@ -2446,12 +2528,6 @@ struct Event {
# touch frame
touch @135 :List(Touch);
# navigation
navInstruction @82 :NavInstruction;
navRoute @83 :NavRoute;
navThumbnail @84: Thumbnail;
mapRenderState @105: MapRenderState;
# UI services
uiDebug @102 :UIDebug;
@@ -2553,5 +2629,13 @@ struct Event {
gyroscope2DEPRECATED @100 :SensorEventData;
accelerometer2DEPRECATED @101 :SensorEventData;
temperatureSensor2DEPRECATED @123 :SensorEventData;
driverMonitoringStateDEPRECATED @71 :DriverMonitoringStateDEPRECATED;
gpsNMEADEPRECATED @3 :GPSNMEAData;
uploaderStateDEPRECATED @79 :UploaderState;
navInstructionDEPRECATED @82 :NavInstruction;
navRouteDEPRECATED @83 :NavRoute;
navThumbnailDEPRECATED @84 :Thumbnail;
gnssMeasurementsDEPRECATED @91 :GnssMeasurements;
mapRenderStateDEPRECATED @105: MapRenderState;
}
}
+6 -6
View File
@@ -259,11 +259,11 @@ class PubMaster:
self.sock[s].send(dat)
def wait_for_readers_to_update(self, s: str, timeout: int, dt: float = 0.05) -> bool:
for _ in range(int(timeout*(1./dt))):
if self.sock[s].all_readers_updated():
return True
time.sleep(dt)
return False
try:
self.sock[s].wait_for_readers(timeout=timeout, interval=dt)
return True
except TimeoutError:
return False
def all_readers_updated(self, s: str) -> bool:
return self.sock[s].all_readers_updated() # type: ignore
return self.sock[s].all_readers_updated()
BIN
View File
Binary file not shown.
+1 -1
View File
@@ -30,7 +30,7 @@ def zmq_sleep(t=1):
# TODO: this should take any capnp struct and returrn a msg with random populated data
def random_carstate():
fields = ["vEgo", "aEgo", "brake", "steeringAngleDeg"]
fields = ["vEgo", "aEgo", "steeringTorque", "steeringAngleDeg"]
msg = messaging.new_message("carState")
cs = msg.carState
for f in fields:
@@ -1,222 +1,165 @@
#!/usr/bin/env python3
"""Validate sunnypilot routes are parseable by stock commaai/openpilot.
Cap'n Proto is wire-compatible across renames, type relocations, and
additive fields. The only breaking change is a union variant that
upstream doesn't recognize — an unknown discriminant makes the entire
union unreadable.
This script checks: for every struct with a union that exists in both
schemas, does sunnypilot introduce union variants upstream doesn't have?
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from typing import Any, List, Tuple
from typing import Any
DEBUG = False
NO_DISCRIMINANT = 0xFFFF
def print_debug(string: str) -> None:
if DEBUG:
print(string)
def hex_id(value: int) -> str:
return f"0x{value:016x}"
def create_schema_instance(struct: Any, prop: Tuple[str, Any]) -> Any:
"""
Create a new instance of a schema type, handling different field types.
Args:
struct: The Cap'n Proto schema structure
prop: A tuple containing the field name and field metadata
Returns:
A new initialized schema instance
"""
struct_instance = struct.new_message()
field_name, field_metadata = prop
try:
field_type = field_metadata.proto.slot.type.which()
# Initialize different types of fields
if field_type in ('list', 'text', 'data'):
struct_instance.init(field_name, 1)
print_debug(f"Initialized list/text/data field: {field_name}")
elif field_type in ('struct', 'object'):
struct_instance.init(field_name)
print_debug(f"Initialized struct/object field: {field_name}")
return struct_instance
except Exception as e:
print(f"Error creating instance for {field_name}: {e}")
return None
def encode_struct(schema: Any) -> dict:
node = schema.node
fields = []
for name, field in schema.fields.items():
proto = field.proto
ordinal = proto.ordinal.explicit if proto.ordinal.which() == "explicit" else None
discriminant = proto.discriminantValue if proto.discriminantValue != NO_DISCRIMINANT else None
fields.append({"name": name, "ordinal": ordinal, "discriminant": discriminant})
return {
"typeId": hex_id(node.id),
"displayName": node.displayName,
"hasUnion": node.struct.discriminantCount > 0,
"fields": fields,
}
def get_schema_fields(schema_struct: Any) -> List[Tuple[str, Any]]:
"""
Retrieve all fields from a given schema structure.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of field names and their metadata
"""
try:
# Get all fields from the schema
schema_fields = list(schema_struct.schema.fields.items())
print_debug("Discovered schema fields:")
for field_name, field_metadata in schema_fields:
print_debug(f"- {field_name}")
return schema_fields
except Exception as e:
print(f"Error retrieving schema fields: {e}")
return []
def _child_struct_schema(field: Any) -> Any:
proto = field.proto
if proto.which() == "group":
return field.schema
type_node = proto.slot.type
which = type_node.which()
if which == "struct":
return field.schema
if which == "list":
container = field.schema
element_type = type_node.list.elementType
while element_type.which() == "list":
container = container.elementType
element_type = element_type.list.elementType
if element_type.which() == "struct":
return container.elementType
return None
def generate_schema_instances(schema_struct: Any) -> List[Any]:
"""
Generate instances for all fields in a given schema.
Args:
schema_struct: The Cap'n Proto schema structure
Returns:
A list of schema instances
"""
schema_fields = get_schema_fields(schema_struct)
instances = []
for field_prop in schema_fields:
try:
instance = create_schema_instance(schema_struct, field_prop)
if instance is not None:
instances.append(instance)
except Exception as e:
print(f"Skipping field due to error: {e}")
print(f"Generated {len(instances)} schema instances")
return instances
def collect_schema(root: Any) -> dict[str, dict]:
structs: dict[str, dict] = {}
stack = [root]
while stack:
schema = stack.pop()
type_id = hex_id(schema.node.id)
if type_id in structs:
continue
structs[type_id] = encode_struct(schema)
for _name, field in schema.fields.items():
try:
child = _child_struct_schema(field)
except Exception:
child = None
if child is not None:
stack.append(child)
return structs
def persist_instances(instances: List[Any], filename: str) -> None:
"""
Write schema instances to a binary file.
Args:
instances: List of schema instances
filename: Output file path
"""
try:
with open(filename, 'wb') as f:
for instance in instances:
f.write(instance.to_bytes())
print(f"Successfully wrote {len(instances)} instances to {filename}")
except Exception as e:
print(f"Error persisting instances: {e}")
sys.exit(1)
def load_log(cereal_dir: str, extra_imports: list[str] | None = None) -> Any:
import capnp
cereal_dir = os.path.abspath(cereal_dir)
capnp.remove_import_hook()
imports = [cereal_dir] + [os.path.abspath(p) for p in (extra_imports or [])]
return capnp.load(os.path.join(cereal_dir, "log.capnp"), imports=imports)
def read_instances(filename: str, schema_type: Any) -> List[Any]:
"""
Read schema instances from a binary file.
Args:
filename: Input file path
schema_type: The schema type to use for reading
Returns:
A list of read schema instances
"""
try:
with open(filename, 'rb') as f:
data = f.read()
instances = list(schema_type.read_multiple_bytes(data))
print(f"Read {len(instances)} instances from {filename}")
return instances
except Exception as e:
print(f"Error reading instances: {e}")
sys.exit(1)
def dump_schema(cereal_dir: str, path: str, extra_imports: list[str] | None = None) -> None:
log = load_log(cereal_dir, extra_imports)
payload = {
"root": hex_id(log.Event.schema.node.id),
"structs": collect_schema(log.Event.schema),
}
with open(path, "w", encoding="utf-8") as handle:
json.dump(payload, handle, indent=2, sort_keys=True)
print(f"wrote schema dump with {len(payload['structs'])} structs to {path}")
def compare_schemas(original_instances: List[Any], read_instances: List[Any]) -> bool:
"""
Compare original and read-back instances to detect potential breaking changes.
def compare(sunnypilot_dump: dict, upstream_dump: dict) -> list[str]:
violations: list[str] = []
sunnypilot_structs = sunnypilot_dump["structs"]
upstream_structs = upstream_dump["structs"]
Args:
original_instances: List of originally generated instances
read_instances: List of instances read back from file
for type_id, sp_struct in sunnypilot_structs.items():
if not sp_struct["hasUnion"]:
continue
up_struct = upstream_structs.get(type_id)
if up_struct is None:
continue
Returns:
Boolean indicating whether schemas appear compatible
"""
if len(original_instances) != len(read_instances):
print("❌ Schema Compatibility Warning: Instance count mismatch")
return False
up_ordinals = {f["ordinal"] for f in up_struct["fields"] if f.get("discriminant") is not None}
display = sp_struct["displayName"]
compatible = True
for struct in read_instances:
try:
getattr(struct, struct.which()) # Attempting to access the field to validate readability
except Exception as e:
print(f"❌ Structural change detected: {struct.which()} is not readable.\nFull error: {e}")
compatible = False
for field in sp_struct["fields"]:
if field.get("discriminant") is None:
continue
if field["ordinal"] not in up_ordinals:
violations.append(
f"{display} @{field['ordinal']} '{field['name']}': "
f"union variant not present upstream (discriminant={field['discriminant']})"
)
return compatible
return violations
def main():
"""
CLI entry point for schema compatibility testing.
"""
# Setup argument parser
parser = argparse.ArgumentParser(
description='Cap\'n Proto Schema Compatibility Testing Tool',
epilog='Test schema compatibility by generating and reading back instances.'
)
def run_read(cereal_dir: str, peer_path: str, extra_imports: list[str] | None = None) -> int:
log = load_log(cereal_dir, extra_imports)
with open(peer_path, "r", encoding="utf-8") as f:
peer_dump = json.load(f)
local_dump = {
"root": hex_id(log.Event.schema.node.id),
"structs": collect_schema(log.Event.schema),
}
violations = compare(sunnypilot_dump=peer_dump, upstream_dump=local_dump)
# Add mutually exclusive group for generation or reading mode
mode_group = parser.add_mutually_exclusive_group(required=True)
mode_group.add_argument('-g', '--generate', action='store_true',
help='Generate schema instances')
mode_group.add_argument('-r', '--read', action='store_true',
help='Read and validate schema instances')
if not violations:
print("cereal compat OK: upstream can parse sunnypilot routes.")
return 0
# Common arguments
parser.add_argument('-f', '--file',
default='schema_instances.bin',
help='Output/input binary file (default: schema_instances.bin)')
print(f"cereal compat FAIL ({len(violations)} leaked union variant(s)):")
for v in violations:
print(f" {v}")
return 1
# Parse arguments
def main() -> int:
parser = argparse.ArgumentParser(description="sunnypilot cereal upstream compat check")
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("-g", "--generate", action="store_true", help="dump local schema to JSON")
mode.add_argument("-r", "--read", action="store_true", help="validate against peer schema")
parser.add_argument("-f", "--file", default="schema.json", help="JSON file path")
parser.add_argument("--cereal-dir", required=True, help="path to cereal directory")
parser.add_argument("-I", "--import-path", action="append", default=[], help="extra capnp import paths")
args = parser.parse_args()
# Import the schema dynamically
try:
from cereal import log
schema_type = log.Event
except ImportError:
print("Error: Unable to import schema. Ensure 'cereal' is installed.")
sys.exit(1)
# Execute based on mode
if args.generate:
print("🔧 Generating Schema Instances")
instances = generate_schema_instances(schema_type)
persist_instances(instances, args.file)
print("✅ Instance generation complete")
elif args.read:
print("🔍 Reading and Validating Schema Instances")
generated_instances = generate_schema_instances(schema_type)
read_back_instances = read_instances(args.file, schema_type)
# Compare schemas
if compare_schemas(generated_instances, read_back_instances):
print("✅ Schema Compatibility: No breaking changes detected")
sys.exit(0)
else:
print("❌ Potential Schema Breaking Changes Detected")
sys.exit(1)
dump_schema(args.cereal_dir, args.file, args.import_path)
return 0
return run_read(args.cereal_dir, args.file, args.import_path)
if __name__ == "__main__":
main()
sys.exit(main())
+90
View File
@@ -0,0 +1,90 @@
/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT services.py */
#ifndef __SERVICES_H
#define __SERVICES_H
#include <map>
#include <string>
struct service { std::string name; bool should_log; float frequency; int decimation; size_t queue_size; };
static std::map<std::string, service> services = {
{ "gyroscope", {"gyroscope", true, 104.000000, 104, 256000}},
{ "accelerometer", {"accelerometer", true, 104.000000, 104, 256000}},
{ "temperatureSensor", {"temperatureSensor", true, 2.000000, 200, 256000}},
{ "deviceState", {"deviceState", true, 2.000000, 1, 256000}},
{ "touch", {"touch", true, 20.000000, 1, 256000}},
{ "can", {"can", true, 100.000000, 2053, 10485760}},
{ "controlsState", {"controlsState", true, 100.000000, 10, 2097152}},
{ "selfdriveState", {"selfdriveState", true, 100.000000, 10, 256000}},
{ "pandaStates", {"pandaStates", true, 10.000000, 1, 256000}},
{ "peripheralState", {"peripheralState", true, 2.000000, 1, 256000}},
{ "radarState", {"radarState", true, 20.000000, 5, 256000}},
{ "roadEncodeIdx", {"roadEncodeIdx", false, 20.000000, 1, 256000}},
{ "liveTracks", {"liveTracks", true, 20.000000, -1, 256000}},
{ "sendcan", {"sendcan", true, 100.000000, 139, 2097152}},
{ "logMessage", {"logMessage", true, 0.000000, -1, 10485760}},
{ "errorLogMessage", {"errorLogMessage", true, 0.000000, 1, 10485760}},
{ "liveCalibration", {"liveCalibration", true, 4.000000, 4, 256000}},
{ "liveTorqueParameters", {"liveTorqueParameters", true, 4.000000, 1, 256000}},
{ "liveDelay", {"liveDelay", true, 4.000000, 1, 256000}},
{ "androidLog", {"androidLog", true, 0.000000, -1, 256000}},
{ "carState", {"carState", true, 100.000000, 10, 256000}},
{ "carControl", {"carControl", true, 100.000000, 10, 256000}},
{ "carOutput", {"carOutput", true, 100.000000, 10, 256000}},
{ "longitudinalPlan", {"longitudinalPlan", true, 20.000000, 10, 256000}},
{ "lateralManeuverPlan", {"lateralManeuverPlan", true, 20.000000, -1, 256000}},
{ "driverAssistance", {"driverAssistance", true, 20.000000, 20, 256000}},
{ "procLog", {"procLog", true, 0.500000, 15, 10485760}},
{ "gpsLocationExternal", {"gpsLocationExternal", true, 10.000000, 10, 256000}},
{ "gpsLocation", {"gpsLocation", true, 1.000000, 1, 256000}},
{ "ubloxGnss", {"ubloxGnss", true, 10.000000, -1, 256000}},
{ "qcomGnss", {"qcomGnss", true, 2.000000, -1, 256000}},
{ "clocks", {"clocks", true, 0.100000, 1, 256000}},
{ "ubloxRaw", {"ubloxRaw", true, 20.000000, -1, 256000}},
{ "livePose", {"livePose", true, 20.000000, 4, 256000}},
{ "liveParameters", {"liveParameters", true, 20.000000, 5, 256000}},
{ "cameraOdometry", {"cameraOdometry", true, 20.000000, 10, 256000}},
{ "thumbnail", {"thumbnail", true, 0.016667, 1, 256000}},
{ "onroadEvents", {"onroadEvents", true, 1.000000, 1, 256000}},
{ "carParams", {"carParams", true, 0.020000, 1, 256000}},
{ "roadCameraState", {"roadCameraState", true, 20.000000, 20, 256000}},
{ "driverCameraState", {"driverCameraState", true, 20.000000, 20, 256000}},
{ "driverEncodeIdx", {"driverEncodeIdx", false, 20.000000, 1, 256000}},
{ "driverStateV2", {"driverStateV2", true, 20.000000, 10, 256000}},
{ "driverMonitoringState", {"driverMonitoringState", true, 20.000000, 10, 256000}},
{ "wideRoadEncodeIdx", {"wideRoadEncodeIdx", false, 20.000000, 1, 256000}},
{ "wideRoadCameraState", {"wideRoadCameraState", true, 20.000000, 20, 256000}},
{ "drivingModelData", {"drivingModelData", true, 20.000000, 10, 256000}},
{ "modelV2", {"modelV2", true, 20.000000, -1, 10485760}},
{ "managerState", {"managerState", true, 2.000000, 1, 256000}},
{ "qRoadEncodeIdx", {"qRoadEncodeIdx", false, 20.000000, -1, 256000}},
{ "userBookmark", {"userBookmark", true, 0.000000, 1, 256000}},
{ "soundPressure", {"soundPressure", true, 10.000000, 10, 256000}},
{ "rawAudioData", {"rawAudioData", false, 20.000000, -1, 256000}},
{ "bookmarkButton", {"bookmarkButton", true, 0.000000, 1, 256000}},
{ "audioFeedback", {"audioFeedback", true, 0.000000, 1, 256000}},
{ "roadEncodeData", {"roadEncodeData", false, 20.000000, -1, 10485760}},
{ "driverEncodeData", {"driverEncodeData", false, 20.000000, -1, 10485760}},
{ "wideRoadEncodeData", {"wideRoadEncodeData", false, 20.000000, -1, 10485760}},
{ "qRoadEncodeData", {"qRoadEncodeData", false, 20.000000, -1, 10485760}},
{ "modelManagerSP", {"modelManagerSP", false, 1.000000, 1, 10485760}},
{ "backupManagerSP", {"backupManagerSP", false, 1.000000, 1, 10485760}},
{ "selfdriveStateSP", {"selfdriveStateSP", true, 100.000000, 10, 256000}},
{ "longitudinalPlanSP", {"longitudinalPlanSP", true, 20.000000, 10, 256000}},
{ "onroadEventsSP", {"onroadEventsSP", true, 1.000000, 1, 256000}},
{ "carParamsSP", {"carParamsSP", true, 0.020000, 1, 256000}},
{ "carControlSP", {"carControlSP", true, 100.000000, 10, 256000}},
{ "carStateSP", {"carStateSP", true, 100.000000, 10, 256000}},
{ "liveMapDataSP", {"liveMapDataSP", true, 1.000000, 1, 256000}},
{ "modelDataV2SP", {"modelDataV2SP", true, 20.000000, -1, 10485760}},
{ "liveLocationKalman", {"liveLocationKalman", true, 20.000000, -1, 256000}},
{ "uiDebug", {"uiDebug", true, 0.000000, 1, 256000}},
{ "testJoystick", {"testJoystick", true, 0.000000, -1, 256000}},
{ "alertDebug", {"alertDebug", true, 20.000000, 5, 256000}},
{ "livestreamWideRoadEncodeIdx", {"livestreamWideRoadEncodeIdx", false, 20.000000, -1, 256000}},
{ "livestreamRoadEncodeIdx", {"livestreamRoadEncodeIdx", false, 20.000000, -1, 256000}},
{ "livestreamDriverEncodeIdx", {"livestreamDriverEncodeIdx", false, 20.000000, -1, 256000}},
{ "livestreamWideRoadEncodeData", {"livestreamWideRoadEncodeData", false, 20.000000, -1, 2097152}},
{ "livestreamRoadEncodeData", {"livestreamRoadEncodeData", false, 20.000000, -1, 2097152}},
{ "livestreamDriverEncodeData", {"livestreamDriverEncodeData", false, 20.000000, -1, 2097152}},
{ "customReservedRawData0", {"customReservedRawData0", true, 0.000000, -1, 256000}},
};
#endif
-10
View File
@@ -24,10 +24,7 @@ _services: dict[str, tuple] = {
# note: the "EncodeIdx" packets will still be in the log
"gyroscope": (True, 104., 104),
"accelerometer": (True, 104., 104),
"magnetometer": (True, 25.),
"lightSensor": (True, 100., 100),
"temperatureSensor": (True, 2., 200),
"gpsNMEA": (True, 9.),
"deviceState": (True, 2., 1),
"touch": (True, 20., 1),
"can": (True, 100., 2053, QueueSize.BIG), # decimation gives ~3 msgs in a full segment
@@ -56,7 +53,6 @@ _services: dict[str, tuple] = {
"gpsLocation": (True, 1., 1),
"ubloxGnss": (True, 10.),
"qcomGnss": (True, 2.),
"gnssMeasurements": (True, 10., 10),
"clocks": (True, 0.1, 1),
"ubloxRaw": (True, 20.),
"livePose": (True, 20., 4),
@@ -75,10 +71,6 @@ _services: dict[str, tuple] = {
"drivingModelData": (True, 20., 10),
"modelV2": (True, 20., None, QueueSize.BIG),
"managerState": (True, 2., 1),
"uploaderState": (True, 0., 1),
"navInstruction": (True, 1., 10),
"navRoute": (True, 0.),
"navThumbnail": (True, 0.),
"qRoadEncodeIdx": (False, 20.),
"userBookmark": (True, 0., 1),
"soundPressure": (True, 10., 10),
@@ -114,8 +106,6 @@ _services: dict[str, tuple] = {
"livestreamRoadEncodeData": (False, 20., None, QueueSize.MEDIUM),
"livestreamDriverEncodeData": (False, 20., None, QueueSize.MEDIUM),
"customReservedRawData0": (True, 0.),
"customReservedRawData1": (True, 0.),
"customReservedRawData2": (True, 0.),
}
SERVICE_LIST = {name: Service(*vals) for
idx, (name, vals) in enumerate(_services.items())}
-23
View File
@@ -1,23 +0,0 @@
Import('env', 'envCython')
common_libs = [
'params.cc',
'swaglog.cc',
'util.cc',
'ratekeeper.cc',
]
_common = env.Library('common', common_libs, LIBS="json11")
Export('_common')
if GetOption('extras'):
env.Program('tests/test_common',
['tests/test_runner.cc', 'tests/test_params.cc', 'tests/test_util.cc', 'tests/test_swaglog.cc'],
LIBS=[_common, 'json11', 'zmq', 'pthread'])
# Cython bindings
params_python = envCython.Program('params_pyx.so', 'params_pyx.pyx', LIBS=envCython['LIBS'] + [_common, 'zmq', 'json11'])
common_python = [params_python]
Export('common_python')
+1 -1
View File
@@ -3,7 +3,7 @@ import os
import requests
import unicodedata
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.common.hardware.hw import Paths
from openpilot.system.version import get_version
# name: jwt signature algorithm
+3
View File
@@ -0,0 +1,3 @@
from openpilot.common.esim.base import LPABase, LPAError, LPAProfileNotFoundError, Profile
__all__ = ["LPABase", "LPAError", "LPAProfileNotFoundError", "Profile"]
+55
View File
@@ -0,0 +1,55 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
class LPAError(RuntimeError):
pass
class LPAProfileNotFoundError(LPAError):
pass
@dataclass
class Profile:
iccid: str
nickname: str
enabled: bool
provider: str
@property
def is_comma(self) -> bool:
return self.provider == 'Webbing' and self.iccid.startswith('8985235')
class LPABase(ABC):
@abstractmethod
def list_profiles(self) -> list[Profile]:
pass
@abstractmethod
def get_active_profile(self) -> Profile | None:
pass
@abstractmethod
def delete_profile(self, iccid: str) -> None:
pass
@abstractmethod
def download_profile(self, qr: str, nickname: str | None = None) -> None:
pass
@abstractmethod
def nickname_profile(self, iccid: str, nickname: str) -> None:
pass
@abstractmethod
def switch_profile(self, iccid: str) -> None:
pass
def process_notifications(self) -> None:
pass
@abstractmethod
def is_euicc(self) -> bool:
pass
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
import argparse
from openpilot.common.hardware import HARDWARE
from openpilot.common.esim.base import LPABase, Profile
def sorted_profiles(lpa: LPABase) -> list[Profile]:
return sorted(lpa.list_profiles(), key=lambda p: p.iccid)
def resolve_iccid(lpa: LPABase, ref: str) -> str:
# ref is either a 1-based index into the sorted list, or a literal iccid
if ref.isdigit():
profiles = sorted_profiles(lpa)
idx = int(ref) - 1
if not 0 <= idx < len(profiles):
raise SystemExit(f'no profile at index {ref} (have {len(profiles)})')
return profiles[idx].iccid
return ref
def print_profiles(lpa: LPABase) -> None:
profiles = sorted_profiles(lpa)
print(f'\n{len(profiles)} profile{"s" if len(profiles) != 1 else ""}:')
for i, p in enumerate(profiles, start=1):
print(f'{i}. {p.iccid} (nickname: {p.nickname or "<none provided>"}) (provider: {p.provider}) - {"enabled" if p.enabled else "disabled"}')
if __name__ == '__main__':
parser = argparse.ArgumentParser(prog='esim.py', description='manage eSIM profiles on your comma device', epilog='comma.ai')
sub = parser.add_subparsers(dest='cmd')
sub.add_parser('list', help='list profiles')
p_switch = sub.add_parser('switch', help='switch to profile')
p_switch.add_argument('profile', help='iccid or 1-based index from `list`')
p_delete = sub.add_parser('delete', help='delete profile (warning: this cannot be undone)')
p_delete.add_argument('profile', help='iccid or 1-based index from `list`')
p_download = sub.add_parser('download', help='download a profile using QR code (format: LPA:1$rsp.truphone.com$QRF-SPEEDTEST)')
p_download.add_argument('qr')
p_download.add_argument('name')
p_nickname = sub.add_parser('nickname', help='update the nickname for a profile')
p_nickname.add_argument('profile', help='iccid or 1-based index from `list`')
p_nickname.add_argument('name')
args = parser.parse_args()
lpa = HARDWARE.get_sim_lpa()
if args.cmd == 'switch':
lpa.switch_profile(resolve_iccid(lpa, args.profile))
elif args.cmd == 'delete':
iccid = resolve_iccid(lpa, args.profile)
confirm = input(f'are you sure you want to delete profile {iccid}? (y/N) ')
if confirm == 'y':
lpa.delete_profile(iccid)
else:
print('cancelled')
exit(0)
elif args.cmd == 'download':
lpa.download_profile(args.qr, args.name)
elif args.cmd == 'nickname':
lpa.nickname_profile(resolve_iccid(lpa, args.profile), args.name)
else:
if args.cmd is None:
parser.print_help()
print_profiles(lpa)
+133
View File
@@ -0,0 +1,133 @@
# GSMA Certificate Issuer (CI) bundle for eSIM RSP
# Source: https://euicc-manual.osmocom.org/docs/pki/ci/bundle.pem
issuer=
countryName = CH
organizationName = OISTE Foundation
commonName = OISTE GSMA CI G1
notBefore=2024-01-16 23:17:39Z
notAfter=2059-01-07 23:17:38Z
-----BEGIN CERTIFICATE-----
MIIB9zCCAZ2gAwIBAgIUSpBSCCDYPOEG/IFHUCKpZ2pIAQMwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCQ0gxGTAXBgNVBAoMEE9JU1RFIEZvdW5kYXRpb24xGTAXBgNV
BAMMEE9JU1RFIEdTTUEgQ0kgRzEwIBcNMjQwMTE2MjMxNzM5WhgPMjA1OTAxMDcy
MzE3MzhaMEMxCzAJBgNVBAYTAkNIMRkwFwYDVQQKDBBPSVNURSBGb3VuZGF0aW9u
MRkwFwYDVQQDDBBPSVNURSBHU01BIENJIEcxMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEvZ3s3PFC4NgrCcCMmHJ6DJ66uzAHuLcvjJnOn+TtBNThS7YHLDyHCa2v
7D+zTP+XTtgqgcLoB56Gha9EQQQ4xKNtMGswDwYDVR0TAQH/BAUwAwEB/zAQBgNV
HREECTAHiAVghXQFDjAXBgNVHSABAf8EDTALMAkGB2eBEgECAQAwHQYDVR0OBBYE
FEwnlnrSDBSzkelgHkHmBK1XwCIvMA4GA1UdDwEB/wQEAwIBBjAKBggqhkjOPQQD
AgNIADBFAiBVcywTj017jKpAQ+gwy4MqK2hQvzve6lkvQkgSP6ykHwIhAI0KFwCD
jnPbmcJsG41hUrWNlf+IcrMvFuYii0DasBNi
-----END CERTIFICATE-----
issuer=
organizationName = GSM Association
commonName = GSM Association - RSP2 Root CI1
notBefore=2017-02-22 00:00:00Z
notAfter=2052-02-21 23:59:59Z
-----BEGIN CERTIFICATE-----
MIICSTCCAe+gAwIBAgIQbmhWeneg7nyF7hg5Y9+qejAKBggqhkjOPQQDAjBEMRgw
FgYDVQQKEw9HU00gQXNzb2NpYXRpb24xKDAmBgNVBAMTH0dTTSBBc3NvY2lhdGlv
biAtIFJTUDIgUm9vdCBDSTEwIBcNMTcwMjIyMDAwMDAwWhgPMjA1MjAyMjEyMzU5
NTlaMEQxGDAWBgNVBAoTD0dTTSBBc3NvY2lhdGlvbjEoMCYGA1UEAxMfR1NNIEFz
c29jaWF0aW9uIC0gUlNQMiBSb290IENJMTBZMBMGByqGSM49AgEGCCqGSM49AwEH
A0IABJ1qutL0HCMX52GJ6/jeibsAqZfULWj/X10p/Min6seZN+hf5llovbCNuB2n
unLz+O8UD0SUCBUVo8e6n9X1TuajgcAwgb0wDgYDVR0PAQH/BAQDAgEGMA8GA1Ud
EwEB/wQFMAMBAf8wEwYDVR0RBAwwCogIKwYBBAGC6WAwFwYDVR0gAQH/BA0wCzAJ
BgdngRIBAgEAME0GA1UdHwRGMEQwQqBAoD6GPGh0dHA6Ly9nc21hLWNybC5zeW1h
dXRoLmNvbS9vZmZsaW5lY2EvZ3NtYS1yc3AyLXJvb3QtY2kxLmNybDAdBgNVHQ4E
FgQUgTcPUSXQsdQI1MOyMubSXnlb6/swCgYIKoZIzj0EAwIDSAAwRQIgIJdYsOMF
WziPK7l8nh5mu0qiRiVf25oa9ullG/OIASwCIQDqCmDrYf+GziHXBOiwJwnBaeBO
aFsiLzIEOaUuZwdNUw==
-----END CERTIFICATE-----
issuer=
countryName = US
organizationName = Entrust, Inc.
organizationalUnitName = See www.entrust.net/legal-terms
organizationalUnitName = (c) 2016 Entrust, Inc. - for authorized use only
commonName = Entrust eSIM Certification Authority
notBefore=2016-11-16 16:04:02Z
notAfter=2051-10-16 16:34:02Z
-----BEGIN CERTIFICATE-----
MIIC6DCCAo2gAwIBAgIRAIy4GT7M5nHsAAAAAFgsinowCgYIKoZIzj0EAwIwgbkx
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1FbnRydXN0LCBJbmMuMSgwJgYDVQQLEx9T
ZWUgd3d3LmVudHJ1c3QubmV0L2xlZ2FsLXRlcm1zMTkwNwYDVQQLEzAoYykgMjAx
NiBFbnRydXN0LCBJbmMuIC0gZm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxLTArBgNV
BAMTJEVudHJ1c3QgZVNJTSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAgFw0xNjEx
MTYxNjA0MDJaGA8yMDUxMTAxNjE2MzQwMlowgbkxCzAJBgNVBAYTAlVTMRYwFAYD
VQQKEw1FbnRydXN0LCBJbmMuMSgwJgYDVQQLEx9TZWUgd3d3LmVudHJ1c3QubmV0
L2xlZ2FsLXRlcm1zMTkwNwYDVQQLEzAoYykgMjAxNiBFbnRydXN0LCBJbmMuIC0g
Zm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxLTArBgNVBAMTJEVudHJ1c3QgZVNJTSBD
ZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IA
BAdzwGHeQ1Wb2f4DmHTByR5/IWL3JugQ1U3908a++bHdlt+TTA7K4c5cYZ+51Yz/
hg/bacxguPDh9uQUK6Wg3a6jcjBwMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/
BAQDAgEGMBcGA1UdIAEB/wQNMAswCQYHZ4ESAQIBADAVBgNVHREEDjAMiApghkgB
hvpsFAoAMB0GA1UdDgQWBBQWcEt/NR42B/GMS3AAXDoAPf1BSjAKBggqhkjOPQQD
AgNJADBGAiEAspjXMvaBZyAg86Z0AAtT0yBRAi1EyaAfNz9kDJeAE04CIQC3efj8
ATL7/tDBOhANy3cK8PS/1NIlu9vqMLCZsZvJ0Q==
-----END CERTIFICATE-----
issuer=
countryName = FR
organizationName = OBERTHUR TECHNOLOGIES
organizationalUnitName = TELECOM
commonName = MC4 OT ROOT CI v1
notBefore=2016-11-15 00:00:01Z
notAfter=2046-11-08 23:59:59Z
-----BEGIN CERTIFICATE-----
MIICOjCCAeGgAwIBAgIBATAKBggqhkjOPQQDAjBbMQswCQYDVQQGEwJGUjEeMBwG
A1UEChMVT0JFUlRIVVIgVEVDSE5PTE9HSUVTMRAwDgYDVQQLEwdURUxFQ09NMRow
GAYDVQQDExFNQzQgT1QgUk9PVCBDSSB2MTAeFw0xNjExMTUwMDAwMDFaFw00NjEx
MDgyMzU5NTlaMFsxCzAJBgNVBAYTAkZSMR4wHAYDVQQKExVPQkVSVEhVUiBURUNI
Tk9MT0dJRVMxEDAOBgNVBAsTB1RFTEVDT00xGjAYBgNVBAMTEU1DNCBPVCBST09U
IENJIHYxMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEHb/Gajt3OZxuaDSklBQE
D4lOd6PGPLSvtfkM952ubdyy45tJwAeA0eEii0CLrFT6tcfXkW+H/5mQyMRXaAUk
T6OBlTCBkjAfBgNVHSMEGDAWgBTNbmC3LXoGPLyEYluR6A/jBAbhPjAdBgNVHQ4E
FgQUzW5gty16Bjy8hGJbkegP4wQG4T4wDgYDVR0PAQH/BAQDAgAGMBcGA1UdIAEB
/wQNMAswCQYHZ4ESAQIBADAWBgNVHREEDzANiAsrBgEEAYHvb7OITTAPBgNVHRMB
Af8EBTADAQH/MAoGCCqGSM49BAMCA0cAMEQCIEw4Nc7f2fDtoH+6ON/bknfDQxmT
ikThXjhpLtSrSKN2AiAxHxgC87L0FDnH8dJNlkdGX9c0JIx6oLheIplfS6k+jg==
-----END CERTIFICATE-----
issuer=
commonName = SubMan V4.2 CI Google Pixel
organizationName = Giesecke and Devrient GmbH
organizationalUnitName = Mobile Security
countryName = DE
notBefore=2017-05-10 00:00:00Z
notAfter=2027-05-10 00:00:00Z
-----BEGIN CERTIFICATE-----
MIICaTCCAg6gAwIBAgICASwwCgYIKoZIzj0EAwIwczElMCMGA1UEAxMcIFN1Yk1h
biBWNC4yIENJIEdvb2dsZSBQaXhlbDEjMCEGA1UEChMaR2llc2Vja2UgYW5kIERl
dnJpZW50IEdtYkgxGDAWBgNVBAsTD01vYmlsZSBTZWN1cml0eTELMAkGA1UEBhMC
REUwHhcNMTcwNTEwMDAwMDAwWhcNMjcwNTEwMDAwMDAwWjBzMSUwIwYDVQQDExwg
U3ViTWFuIFY0LjIgQ0kgR29vZ2xlIFBpeGVsMSMwIQYDVQQKExpHaWVzZWNrZSBh
bmQgRGV2cmllbnQgR21iSDEYMBYGA1UECxMPTW9iaWxlIFNlY3VyaXR5MQswCQYD
VQQGEwJERTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABHNorfaJsGzqWNawyAhl
IAv9QL2/+b9RsUoso06t/dKX1MRr5CUJ51acvv5TAFhQKIml+dwLbFnV5aO+8W6Z
wxajgZEwgY4wHwYDVR0jBBgwFoAUtg8LiX/WMLiM/tYWH46oCMU4KsMwHQYDVR0O
BBYEFLYPC4l/1jC4jP7WFh+OqAjFOCrDMA4GA1UdDwEB/wQEAwIBBjAXBgNVHSAB
Af8EDTALMAkGB2eBEgECAQAwDwYDVR0TAQH/BAUwAwEB/zASBgNVHREECzAJiAcr
BgEEAdwPMAoGCCqGSM49BAMCA0kAMEYCIQDpoZcuAQrjATW8U+AWqMUJ0dY6nWW1
R1QmFzVZ1yMXSwIhALCvRqkCtgiavdeFeSgsSNbY5Fhd+QoCltuSh1U4TE7A
-----END CERTIFICATE-----
issuer=
countryName = DE
commonName = SubMan V4.2 CI
organizationName = Giesecke and Devrient
organizationalUnitName = Mobile Security
notBefore=2016-08-12 13:51:48Z
notAfter=2026-08-12 13:51:48Z
-----BEGIN CERTIFICATE-----
MIICUjCCAfigAwIBAgIDQgAAMAoGCCqGSM49BAMCMGAxCzAJBgNVBAYTAkRFMRcw
FQYDVQQDEw5TdWJNYW4gVjQuMiBDSTEeMBwGA1UEChMVR2llc2Vja2UgYW5kIERl
dnJpZW50MRgwFgYDVQQLEw9Nb2JpbGUgU2VjdXJpdHkwHhcNMTYwODEyMTM1MTQ4
WhcNMjYwODEyMTM1MTQ4WjBgMQswCQYDVQQGEwJERTEXMBUGA1UEAxMOU3ViTWFu
IFY0LjIgQ0kxHjAcBgNVBAoTFUdpZXNlY2tlIGFuZCBEZXZyaWVudDEYMBYGA1UE
CxMPTW9iaWxlIFNlY3VyaXR5MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYIgl
VQr9wbXOlwPp8qMg5Df08Cli9Mc+lpr3Lwa9PlVA3QWlLeX4GfD4H3phLBqVIa17
yHttmtheTxi0KoEqhKOBoDCBnTAdBgNVHQ4EFgQU6lOt7zMpuVCa/XVf1Ei4LcG8
7P8wDgYDVR0PAQH/BAQDAgEGMBcGA1UdIAEB/wQNMAswCQYHZ4ESAQIBADAPBgNV
HRMBAf8EBTADAQH/MBIGA1UdEQQLMAmIBysGAQQB3A8wLgYDVR0fBCcwJTAjoCGg
H4YdaHR0cDovL2dpLWRlLmNvbS90ZXN0LmNybC5wZW0wCgYIKoZIzj0EAwIDSAAw
RQIhAMMx2L/VHDiOW+Fl/OuFmhCdizYM17Yn9zAVieKO2T0iAiANWtCMmY+DzkqK
yHxBFX0U2tBd682zP4DpgRt8j3Ylew==
-----END CERTIFICATE-----
+791
View File
@@ -0,0 +1,791 @@
# SGP.22 v2.3: https://www.gsma.com/solutions-and-impact/technologies/esim/wp-content/uploads/2021/07/SGP.22-v2.3.pdf
import atexit
import base64
import fcntl
import hashlib
import os
import requests
import serial
import subprocess
import sys
import termios
import time
from collections.abc import Callable, Generator
from contextlib import contextmanager
from typing import Any
from pathlib import Path
from openpilot.common.time_helpers import system_time_valid
from openpilot.common.esim.base import LPABase, LPAError, LPAProfileNotFoundError, Profile
GSMA_CI_BUNDLE = str(Path(__file__).parent / "gsma_ci_bundle.pem")
DEFAULT_DEVICE = "/dev/modem_at0"
DEFAULT_BAUD = 9600
DEFAULT_TIMEOUT = 5.0
# https://euicc-manual.osmocom.org/docs/lpa/applet-id/
ISDR_AID = "A0000005591010FFFFFFFF8900000100"
ES10X_MSS = 120
HTTP_TIMEOUT = 30
OPEN_ISDR_RETRIES = 10
OPEN_ISDR_RETRY_DELAY_S = 0.25
OPEN_ISDR_RESET_ATTEMPT = 5
SEND_APDU_RETRIES = 3
LOCK_FILE = '/dev/shm/modem.lock'
DEBUG = os.environ.get("DEBUG") == "1"
# TLV Tags
TAG_ICCID = 0x5A
TAG_STATUS = 0x80
TAG_EUICC_INFO = 0xBF20
TAG_PREPARE_DOWNLOAD = 0xBF21
TAG_BPP_COMMAND = 0xBF23
TAG_PROFILE_METADATA = 0xBF25
TAG_INSTALL_RESULT_DATA = 0xBF27
TAG_LIST_NOTIFICATION = 0xBF28
TAG_SET_NICKNAME = 0xBF29
TAG_RETRIEVE_NOTIFICATION = 0xBF2B
TAG_PROFILE_INFO_LIST = 0xBF2D
TAG_EUICC_CHALLENGE = 0xBF2E
TAG_NOTIFICATION_METADATA = 0xBF2F
TAG_NOTIFICATION_SENT = 0xBF30
TAG_ENABLE_PROFILE = 0xBF31
TAG_DELETE_PROFILE = 0xBF33
TAG_BPP = 0xBF36
TAG_PROFILE_INSTALL_RESULT = 0xBF37
TAG_AUTH_SERVER = 0xBF38
TAG_CANCEL_SESSION = 0xBF41
TAG_OK = 0xA0
PROFILE_OK = 0x00
PROFILE_NOT_IN_DISABLED_STATE = 0x02
PROFILE_CAT_BUSY = 0x05
PROFILE_ERROR_CODES = {
0x01: "iccidOrAidNotFound", PROFILE_NOT_IN_DISABLED_STATE: "profileNotInDisabledState",
0x03: "disallowedByPolicy", 0x04: "wrongProfileReenabling",
PROFILE_CAT_BUSY: "catBusy", 0x06: "undefinedError",
}
AUTH_SERVER_ERROR_CODES = {
0x01: "eUICCVerificationFailed", 0x02: "eUICCCertificateExpired",
0x03: "eUICCCertificateRevoked", 0x05: "invalidServerSignature",
0x06: "euiccCiPKUnknown", 0x0A: "matchingIdRefused",
0x10: "insufficientMemory",
}
BPP_COMMAND_NAMES = {
0: "initialiseSecureChannel", 1: "configureISDP", 2: "storeMetadata",
3: "storeMetadata2", 4: "replaceSessionKeys", 5: "loadProfileElements",
}
BPP_ERROR_REASONS = {
1: "incorrectInputValues", 2: "invalidSignature", 3: "invalidTransactionId",
4: "unsupportedCrtValues", 5: "unsupportedRemoteOperationType",
6: "unsupportedProfileClass", 7: "scp03tStructureError", 8: "scp03tSecurityError",
9: "iccidAlreadyExistsOnEuicc", 10: "insufficientMemoryForProfile",
11: "installInterrupted", 12: "peProcessingError", 13: "dataMismatch",
14: "invalidNAA",
}
BPP_ERROR_MESSAGES = {
9: "This eSIM profile is already installed on this device.",
10: "Not enough memory on the eUICC to install this profile.",
12: "Profile installation failed. The QR code may have already been used.",
}
# SGP.22 §5.2.6 SM-DP+ reason/subject codes mapped to user-friendly messages
ES9P_ERROR_MESSAGES: dict[tuple[str, str], str] = {
('3.8', '8.2.6'): "This eSIM profile is already installed on another device. Please use a new QR code.",
('3.8', '8.2.1'): "This eSIM profile has expired. Please request a new QR code.",
('3.8', '8.1'): "The SM-DP+ server refused this request.",
('3.1', '8.2.6'): "This eSIM profile has been revoked by the carrier.",
('3.9', '8.2.6'): "This eSIM profile download has already been completed.",
('2.1', '8.8'): "The device is not compatible with this eSIM profile.",
('1.2', '8.1'): "The SM-DP+ server is temporarily unavailable. Try again later.",
}
NOTIFICATION_OPERATIONS = {0x80: "install", 0x40: "enable", 0x20: "disable", 0x10: "delete"}
STATE_LABELS = {0: "disabled", 1: "enabled", 255: "unknown"}
ICON_LABELS = {0: "jpeg", 1: "png", 255: "unknown"}
CLASS_LABELS = {0: "test", 1: "provisioning", 2: "operational", 255: "unknown"}
# TLV tag -> (field_name, decoder)
FieldMap = dict[int, tuple[str, Callable[[bytes], Any]]]
def b64e(data: bytes) -> str:
return base64.b64encode(data).decode("ascii")
def base64_trim(s: str) -> str:
return "".join(c for c in s if c not in "\n\r \t")
def b64d(s: str) -> bytes:
return base64.b64decode(base64_trim(s))
class AtClient:
def __init__(self, device: str, baud: int, timeout: float) -> None:
self.channel: str | None = None
self._device = device
self._baud = baud
self._timeout = timeout
self._serial: serial.Serial | None = None
def send_raw(self, data: bytes) -> None:
self._ensure_serial()
self._serial.reset_input_buffer()
self._serial.write(data)
self._serial.flush()
def close(self) -> None:
try:
if self.channel:
try:
self.query(f"AT+CCHC={self.channel}")
except (RuntimeError, TimeoutError):
pass
self.channel = None
finally:
if self._serial:
self._serial.close()
def _send(self, cmd: str) -> None:
if DEBUG:
print(f"SER >> {cmd}", file=sys.stderr)
self._serial.write((cmd + "\r").encode("ascii"))
def _expect(self) -> list[str]:
lines: list[str] = []
while True:
raw = self._serial.readline()
if not raw:
raise TimeoutError("AT command timed out")
line = raw.decode(errors="ignore").strip()
if not line:
continue
if DEBUG:
print(f"SER << {line}", file=sys.stderr)
if line == "OK":
return lines
if line == "ERROR" or line.startswith("+CME ERROR"):
raise RuntimeError(f"AT command failed: {line}")
lines.append(line)
def _ensure_serial(self, reconnect: bool = False) -> None:
if reconnect:
self.channel = None
try:
if self._serial:
self._serial.close()
except Exception:
pass
self._serial = None
if self._serial is None:
self._serial = serial.Serial(self._device, baudrate=self._baud, timeout=self._timeout)
def query(self, cmd: str) -> list[str]:
self._ensure_serial()
try:
self._send(cmd)
return self._expect()
except serial.SerialException:
self._ensure_serial(reconnect=True)
self._send(cmd)
return self._expect()
def _open_isdr_once(self) -> None:
if self.channel:
try:
self.query(f"AT+CCHC={self.channel}")
except RuntimeError:
pass
self.channel = None
# drain any unsolicited responses before opening
if self._serial:
try:
self._serial.reset_input_buffer()
except (OSError, serial.SerialException, termios.error):
self._ensure_serial(reconnect=True)
for line in self.query(f'AT+CCHO="{ISDR_AID}"'):
if line.startswith("+CCHO:") and (ch := line.split(":", 1)[1].strip()):
self.channel = ch
return
raise RuntimeError("Failed to open ISD-R application")
def _reset_modem(self) -> None:
if self._serial:
try:
self._serial.close()
except Exception:
pass
self._serial = None
subprocess.run(['/usr/comma/lte/lte.sh', 'start'], capture_output=True)
def open_isdr(self) -> None:
for attempt in range(OPEN_ISDR_RETRIES):
try:
self._open_isdr_once()
return
except (RuntimeError, TimeoutError, termios.error, serial.SerialException):
time.sleep(OPEN_ISDR_RETRY_DELAY_S)
if attempt == OPEN_ISDR_RESET_ATTEMPT:
self._reset_modem()
raise RuntimeError("Failed to open ISD-R after retries")
def send_apdu(self, apdu: bytes) -> tuple[bytes, int, int]:
for attempt in range(SEND_APDU_RETRIES):
try:
if not self.channel:
self.open_isdr()
hex_payload = apdu.hex().upper()
for line in self.query(f'AT+CGLA={self.channel},{len(hex_payload)},"{hex_payload}"'):
if line.startswith("+CGLA:"):
parts = line.split(":", 1)[1].split(",", 1)
if len(parts) == 2:
data = bytes.fromhex(parts[1].strip().strip('"'))
if len(data) >= 2:
return data[:-2], data[-2], data[-1]
raise RuntimeError("Missing +CGLA response")
except (RuntimeError, ValueError):
self.channel = None
if attempt == SEND_APDU_RETRIES - 1:
raise
raise RuntimeError("send_apdu failed")
# --- TLV utilities ---
def iter_tlv(data: bytes, with_positions: bool = False) -> Generator:
idx, length = 0, len(data)
while idx < length:
start_pos = idx
tag = data[idx]
idx += 1
if tag & 0x1F == 0x1F: # Multi-byte tag
tag_value = tag
while idx < length:
next_byte = data[idx]
idx += 1
tag_value = (tag_value << 8) | next_byte
if not (next_byte & 0x80):
break
else:
tag_value = tag
if idx >= length:
break
size = data[idx]
idx += 1
if size & 0x80: # Multi-byte length
num_bytes = size & 0x7F
if idx + num_bytes > length:
break
size = int.from_bytes(data[idx : idx + num_bytes], "big")
idx += num_bytes
if idx + size > length:
break
value = data[idx : idx + size]
idx += size
yield (tag_value, value, start_pos, idx) if with_positions else (tag_value, value)
def find_tag(data: bytes, target: int) -> bytes | None:
return next((v for t, v in iter_tlv(data) if t == target), None)
def require_tag(data: bytes, target: int, label: str = "") -> bytes:
v = find_tag(data, target)
if v is None:
raise RuntimeError(f"Missing {label or f'tag 0x{target:X}'}")
return v
def tbcd_to_string(raw: bytes) -> str:
return "".join(str(n) for b in raw for n in (b & 0x0F, b >> 4) if n <= 9)
def string_to_tbcd(s: str) -> bytes:
digits = [int(c) for c in s if c.isdigit()]
return bytes(digits[i] | ((digits[i + 1] if i + 1 < len(digits) else 0xF) << 4) for i in range(0, len(digits), 2))
def encode_tlv(tag: int, value: bytes) -> bytes:
tag_bytes = bytes([(tag >> 8) & 0xFF, tag & 0xFF]) if tag > 255 else bytes([tag])
vlen = len(value)
if vlen <= 127:
return tag_bytes + bytes([vlen]) + value
length_bytes = vlen.to_bytes((vlen.bit_length() + 7) // 8, "big")
return tag_bytes + bytes([0x80 | len(length_bytes)]) + length_bytes + value
def int_bytes(n: int) -> bytes:
"""Encode a positive integer as minimal big-endian bytes (at least 1 byte)."""
return n.to_bytes((n.bit_length() + 7) // 8 or 1, "big")
PROFILE: FieldMap = {
TAG_ICCID: ("iccid", tbcd_to_string),
0x4F: ("isdpAid", lambda v: v.hex().upper()),
0x9F70: ("profileState", lambda v: STATE_LABELS.get(v[0], "unknown")),
0x90: ("profileNickname", lambda v: v.decode("utf-8", errors="ignore") or None),
0x91: ("serviceProviderName", lambda v: v.decode("utf-8", errors="ignore") or None),
0x92: ("profileName", lambda v: v.decode("utf-8", errors="ignore") or None),
0x93: ("iconType", lambda v: ICON_LABELS.get(v[0], "unknown")),
0x94: ("icon", b64e),
0x95: ("profileClass", lambda v: CLASS_LABELS.get(v[0], "unknown")),
}
def decode_struct(data: bytes, field_map: FieldMap) -> dict[str, Any]:
"""Parse TLV data using a {tag: (field_name, decoder)} map into a dict."""
result: dict[str, Any] = {name: None for name, _ in field_map.values()}
for tag, value in iter_tlv(data):
if (field := field_map.get(tag)):
result[field[0]] = field[1](value)
return result
# --- ES10x command transport ---
def es10x_command(client: AtClient, data: bytes) -> bytes:
response = bytearray()
sequence = 0
offset = 0
while offset < len(data):
chunk = data[offset : offset + ES10X_MSS]
offset += len(chunk)
is_last = offset == len(data)
apdu = bytes([0x80, 0xE2, 0x91 if is_last else 0x11, sequence & 0xFF, len(chunk)]) + chunk
segment, sw1, sw2 = client.send_apdu(apdu)
response.extend(segment)
while True:
if sw1 == 0x61: # More data available
segment, sw1, sw2 = client.send_apdu(bytes([0x80, 0xC0, 0x00, 0x00, sw2 or 0]))
response.extend(segment)
continue
if (sw1 & 0xF0) == 0x90:
break
raise RuntimeError(f"APDU failed with SW={sw1:02X}{sw2:02X}")
sequence += 1
return bytes(response)
# --- Profile operations ---
NOTIFICATION: FieldMap = {
TAG_STATUS: ("seqNumber", lambda v: int.from_bytes(v, "big")),
0x81: ("profileManagementOperation",
lambda v: NOTIFICATION_OPERATIONS.get(next((m for m in NOTIFICATION_OPERATIONS if len(v) >= 2 and v[1] & m), 0), "unknown")),
0x0C: ("notificationAddress", lambda v: v.decode("utf-8", errors="ignore")),
TAG_ICCID: ("iccid", tbcd_to_string),
}
def decode_profiles(blob: bytes) -> list[dict]:
root = require_tag(blob, TAG_PROFILE_INFO_LIST, "ProfileInfoList")
list_ok = find_tag(root, TAG_OK)
if list_ok is None:
return []
return [decode_struct(value, PROFILE) for tag, value in iter_tlv(list_ok) if tag == 0xE3]
def list_profiles(client: AtClient) -> list[dict]:
return decode_profiles(es10x_command(client, TAG_PROFILE_INFO_LIST.to_bytes(2, "big") + b"\x00"))
def set_profile_nickname(client: AtClient, iccid: str, nickname: str) -> None:
nickname_bytes = nickname.encode("utf-8")
if len(nickname_bytes) > 64:
raise ValueError("Profile nickname must be 64 bytes or less")
content = encode_tlv(TAG_ICCID, string_to_tbcd(iccid)) + encode_tlv(0x90, nickname_bytes)
response = es10x_command(client, encode_tlv(TAG_SET_NICKNAME, content))
code = require_tag(require_tag(response, TAG_SET_NICKNAME, "SetNicknameResponse"), TAG_STATUS, "SetNickname status")[0]
if code == 0x01:
raise LPAError(f"profile {iccid} not found")
if code != 0x00:
raise RuntimeError(f"SetNickname failed with status 0x{code:02X}")
# --- ES9P HTTP ---
def es9p_request(smdp_address: str, endpoint: str, payload: dict, error_prefix: str = "Request", session: requests.Session | None = None) -> dict:
url = f"https://{smdp_address}/gsma/rsp2/es9plus/{endpoint}"
headers = {"User-Agent": "gsma-rsp-lpad", "X-Admin-Protocol": "gsma/rsp/v2.3.0", "Content-Type": "application/json"}
http = session or requests
resp = http.post(url, json=payload, headers=headers, timeout=HTTP_TIMEOUT, verify=GSMA_CI_BUNDLE)
resp.raise_for_status()
if not resp.content:
return {}
data = resp.json()
if "header" in data and "functionExecutionStatus" in data["header"]:
status = data["header"]["functionExecutionStatus"]
if status.get("status") == "Failed":
sd = status.get("statusCodeData", {})
reason = sd.get("reasonCode", "unknown")
subject = sd.get("subjectCode", "unknown")
msg = ES9P_ERROR_MESSAGES.get((reason, subject),
f"{error_prefix} failed: {reason}/{subject} - {sd.get('message', 'unknown')}")
raise RuntimeError(msg)
return data
# --- Notifications ---
def list_notifications(client: AtClient) -> list[dict]:
response = es10x_command(client, encode_tlv(TAG_LIST_NOTIFICATION, b""))
root = require_tag(response, TAG_LIST_NOTIFICATION, "ListNotificationResponse")
metadata_list = find_tag(root, TAG_OK)
if metadata_list is None:
return []
return [decode_struct(value, NOTIFICATION) for tag, value in iter_tlv(metadata_list) if tag == TAG_NOTIFICATION_METADATA]
def process_notifications(client: AtClient) -> None:
for notification in list_notifications(client):
seq_number, smdp_address = notification["seqNumber"], notification["notificationAddress"]
try:
request = encode_tlv(TAG_RETRIEVE_NOTIFICATION, encode_tlv(TAG_OK, encode_tlv(TAG_STATUS, int_bytes(seq_number))))
response = es10x_command(client, request)
content = require_tag(require_tag(response, TAG_RETRIEVE_NOTIFICATION, "RetrieveNotificationsListResponse"),
TAG_OK, "RetrieveNotificationsListResponse")
pending_notif = next((v for t, v in iter_tlv(content) if t in (TAG_PROFILE_INSTALL_RESULT, 0x30)), None)
if pending_notif is None:
raise RuntimeError("Missing PendingNotification")
es9p_request(smdp_address, "handleNotification", {"pendingNotification": b64e(pending_notif)}, "HandleNotification")
response = es10x_command(client, encode_tlv(TAG_NOTIFICATION_SENT, encode_tlv(TAG_STATUS, int_bytes(seq_number))))
root = require_tag(response, TAG_NOTIFICATION_SENT, "NotificationSentResponse")
if int.from_bytes(require_tag(root, TAG_STATUS, "RemoveNotificationFromList status"), "big") != 0:
raise RuntimeError("RemoveNotificationFromList failed")
except Exception as e:
print(f"notification {seq_number} failed: {e}", file=sys.stderr)
# --- Authentication & Download ---
def get_challenge_and_info(client: AtClient) -> tuple[bytes, bytes]:
challenge_resp = es10x_command(client, encode_tlv(TAG_EUICC_CHALLENGE, b""))
challenge = require_tag(require_tag(challenge_resp, TAG_EUICC_CHALLENGE, "GetEuiccDataResponse"),
TAG_STATUS, "challenge in response")
info_resp = es10x_command(client, encode_tlv(TAG_EUICC_INFO, b""))
require_tag(info_resp, TAG_EUICC_INFO, "GetEuiccInfo1Response")
return challenge, info_resp
def authenticate_server(client: AtClient, b64_signed1: str, b64_sig1: str, b64_pk_id: str, b64_cert: str, matching_id: str) -> str:
tac = bytes([0x35, 0x29, 0x06, 0x11])
device_info = encode_tlv(TAG_STATUS, tac) + encode_tlv(0xA1, b"")
ctx_inner = encode_tlv(TAG_STATUS, matching_id.encode("utf-8")) + encode_tlv(0xA1, device_info)
content = b64d(b64_signed1) + b64d(b64_sig1) + b64d(b64_pk_id) + b64d(b64_cert) + encode_tlv(0xA0, ctx_inner)
response = es10x_command(client, encode_tlv(TAG_AUTH_SERVER, content))
root = require_tag(response, TAG_AUTH_SERVER, "AuthenticateServerResponse")
error_tag = find_tag(root, 0xA1)
if error_tag is not None:
code = int.from_bytes(error_tag, "big") if error_tag else 0
raise RuntimeError(f"AuthenticateServer rejected by eUICC: {AUTH_SERVER_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
return b64e(response)
def prepare_download(client: AtClient, b64_signed2: str, b64_sig2: str, b64_cert: str, cc: str | None = None) -> str:
smdp_signed2 = b64d(b64_signed2)
smdp_signature2 = b64d(b64_sig2)
smdp_certificate = b64d(b64_cert)
smdp_signed2_root = find_tag(smdp_signed2, 0x30)
if smdp_signed2_root is None:
raise RuntimeError("Invalid smdpSigned2")
transaction_id = find_tag(smdp_signed2_root, TAG_STATUS)
cc_required_flag = find_tag(smdp_signed2_root, 0x01)
if transaction_id is None or cc_required_flag is None:
raise RuntimeError("Invalid smdpSigned2")
content = smdp_signed2 + smdp_signature2
if int.from_bytes(cc_required_flag, "big") != 0:
if not cc:
raise RuntimeError("Confirmation code required but not provided")
content += encode_tlv(0x04, hashlib.sha256(hashlib.sha256(cc.encode("utf-8")).digest() + transaction_id).digest())
content += smdp_certificate
response = es10x_command(client, encode_tlv(TAG_PREPARE_DOWNLOAD, content))
require_tag(response, TAG_PREPARE_DOWNLOAD, "PrepareDownloadResponse")
return b64e(response)
def _parse_tlv_header_len(data: bytes) -> int:
tag_len = 2 if data[0] & 0x1F == 0x1F else 1
length_byte = data[tag_len]
return tag_len + (1 + (length_byte & 0x7F) if length_byte & 0x80 else 1)
def _split_bpp(bpp: bytes) -> list[bytes]:
"""Split a BoundProfilePackage into APDU chunks per SGP.22 §5.7.6."""
root_value = None
for tag, value, start, end in iter_tlv(bpp, with_positions=True):
if tag == TAG_BPP:
root_value = value
val_start = start + _parse_tlv_header_len(bpp[start:end])
break
if root_value is None:
raise RuntimeError("Invalid BoundProfilePackage")
chunks: list[bytes] = []
for tag, value, start, end in iter_tlv(root_value, with_positions=True):
if tag == TAG_BPP_COMMAND:
chunks.append(bpp[0 : val_start + end])
elif tag in (0xA0, 0xA2):
chunks.append(bpp[val_start + start : val_start + end])
elif tag in (0xA1, 0xA3):
hdr_len = _parse_tlv_header_len(root_value[start:end])
chunks.append(bpp[val_start + start : val_start + start + hdr_len])
for _, _, cs, ce in iter_tlv(value, with_positions=True):
chunks.append(value[cs:ce])
return chunks
def _parse_install_result(response: bytes) -> dict[str, Any] | None:
"""Parse a ProfileInstallResult from an APDU response, or None if not present."""
root = find_tag(response, TAG_PROFILE_INSTALL_RESULT)
if not root:
return None
result_data = find_tag(root, TAG_INSTALL_RESULT_DATA)
if not result_data:
return None
result: dict[str, Any] = {"seqNumber": 0, "success": False, "bppCommandId": None, "errorReason": None}
notif_meta = find_tag(result_data, TAG_NOTIFICATION_METADATA)
if notif_meta:
seq_num = find_tag(notif_meta, TAG_STATUS)
if seq_num:
result["seqNumber"] = int.from_bytes(seq_num, "big")
final_result = find_tag(result_data, 0xA2)
if final_result:
for tag, value in iter_tlv(final_result):
if tag == 0xA0:
result["success"] = True
elif tag == 0xA1:
bpp_cmd = find_tag(value, TAG_STATUS)
if bpp_cmd:
result["bppCommandId"] = int.from_bytes(bpp_cmd, "big")
err = find_tag(value, 0x81)
if err:
result["errorReason"] = int.from_bytes(err, "big")
return result
def load_bpp(client: AtClient, b64_bpp: str) -> dict:
bpp = b64d(b64_bpp)
result = None
for chunk in _split_bpp(bpp):
response = es10x_command(client, chunk)
if response and (parsed := _parse_install_result(response)):
result = parsed
break
if result is None:
raise RuntimeError("Profile installation failed: no result from eUICC")
if not result["success"] and result["errorReason"] is not None:
msg = BPP_ERROR_MESSAGES.get(result["errorReason"])
if not msg:
cmd_name = BPP_COMMAND_NAMES.get(result["bppCommandId"], f"unknown({result['bppCommandId']})")
err_name = BPP_ERROR_REASONS.get(result["errorReason"], f"unknown({result['errorReason']})")
msg = f"Profile installation failed at {cmd_name}: {err_name}"
raise RuntimeError(msg)
if not result["success"]:
raise RuntimeError("Profile installation failed: no result from eUICC")
return result
def parse_metadata(b64_metadata: str) -> dict:
root = find_tag(b64d(b64_metadata), TAG_PROFILE_METADATA)
if root is None:
raise RuntimeError("Invalid profileMetadata")
return decode_struct(root, PROFILE)
def cancel_session(client: AtClient, transaction_id: bytes, reason: int = 127) -> str:
content = encode_tlv(0x80, transaction_id) + encode_tlv(0x81, bytes([reason]))
response = es10x_command(client, encode_tlv(TAG_CANCEL_SESSION, content))
return b64e(response)
def parse_lpa_activation_code(activation_code: str) -> tuple[str, str]:
"""Parse 'LPA:1$smdp.example.com$MATCHING-ID' into (smdp_address, matching_id)."""
if not activation_code.startswith("LPA:"):
raise ValueError("Invalid activation code format")
parts = activation_code[4:].split("$")
if len(parts) != 3:
raise ValueError("Invalid activation code format")
return parts[1], parts[2]
def _b64_field(data: dict, key: str) -> str:
return base64_trim(data[key])
def _cancel_session_safe(client: AtClient, smdp: str, tx_id: str, session: requests.Session) -> None:
b64_cancel = ""
try:
b64_cancel = cancel_session(client, b64d(tx_id))
except Exception:
pass
try:
es9p_request(smdp, "cancelSession", {"transactionId": tx_id, "cancelSessionResponse": b64_cancel}, "CancelSession", session=session)
except Exception:
pass
def download_profile(client: AtClient, activation_code: str) -> str:
"""Download and install an eSIM profile. Returns the ICCID of the installed profile."""
if not system_time_valid():
raise RuntimeError("System time is not set; TLS certificate validation requires a valid clock")
smdp, matching_id = parse_lpa_activation_code(activation_code)
challenge, euicc_info = get_challenge_and_info(client)
session = requests.Session()
tx_id = None
try:
# step 1: initiate authentication
auth = es9p_request(smdp, "initiateAuthentication", {
"smdpAddress": smdp, "euiccChallenge": b64e(challenge),
"euiccInfo1": b64e(euicc_info), "matchingId": matching_id,
}, "Authentication", session=session)
tx_id = _b64_field(auth, "transactionId")
# step 2: authenticate server
b64_auth = authenticate_server(client,
_b64_field(auth, "serverSigned1"), _b64_field(auth, "serverSignature1"),
_b64_field(auth, "euiccCiPKIdToBeUsed"), _b64_field(auth, "serverCertificate"),
matching_id)
# step 3: authenticate client + get metadata
cli = es9p_request(smdp, "authenticateClient", {
"transactionId": tx_id, "authenticateServerResponse": b64_auth,
}, "Authentication", session=session)
iccid = parse_metadata(_b64_field(cli, "profileMetadata"))["iccid"]
# step 4: prepare download
b64_prep = prepare_download(client,
_b64_field(cli, "smdpSigned2"), _b64_field(cli, "smdpSignature2"),
_b64_field(cli, "smdpCertificate"))
# step 5: get and install bound profile package
bpp = es9p_request(smdp, "getBoundProfilePackage", {
"transactionId": tx_id, "prepareDownloadResponse": b64_prep,
}, "GetBoundProfilePackage", session=session)
load_bpp(client, _b64_field(bpp, "boundProfilePackage"))
return iccid
except Exception:
if tx_id:
_cancel_session_safe(client, smdp, tx_id, session)
raise
finally:
session.close()
class TiciLPA(LPABase):
def __init__(self):
if hasattr(self, '_client'):
return
self._client = AtClient(DEFAULT_DEVICE, DEFAULT_BAUD, DEFAULT_TIMEOUT)
atexit.register(self._client.close)
@contextmanager
def _acquire_lock(self):
fd = os.open(LOCK_FILE, os.O_CREAT | os.O_RDWR)
try:
fcntl.flock(fd, fcntl.LOCK_EX)
yield
finally:
fcntl.flock(fd, fcntl.LOCK_UN)
os.close(fd)
@contextmanager
def _acquire_channel(self):
with self._acquire_lock():
try:
self._client.open_isdr()
yield
finally:
if self._client.channel:
try:
self._client.query(f"AT+CCHC={self._client.channel}")
except (RuntimeError, TimeoutError):
pass
self._client.channel = None
def list_profiles(self) -> list[Profile]:
with self._acquire_channel():
return [
Profile(
iccid=p.get("iccid", ""),
nickname=p.get("profileNickname") or "",
enabled=p.get("profileState") == "enabled",
provider=p.get("serviceProviderName") or "",
)
for p in list_profiles(self._client)
]
def get_active_profile(self) -> Profile | None:
return None
def process_notifications(self) -> None:
if not system_time_valid():
raise RuntimeError("System time is not set; TLS certificate validation requires a valid clock")
with self._acquire_channel():
process_notifications(self._client)
def delete_profile(self, iccid: str) -> None:
profile = next((p for p in self.list_profiles() if p.iccid == iccid), None)
if profile is None:
raise LPAProfileNotFoundError(f"profile not found: {iccid}")
if profile.is_comma:
raise LPAError("refusing to delete a comma profile")
with self._acquire_channel():
request = encode_tlv(TAG_DELETE_PROFILE, encode_tlv(TAG_ICCID, string_to_tbcd(iccid)))
response = es10x_command(self._client, request)
code = require_tag(require_tag(response, TAG_DELETE_PROFILE, "DeleteProfileResponse"), TAG_STATUS, "DeleteProfile status")[0]
if code != PROFILE_OK:
raise LPAError(f"DeleteProfile failed: {PROFILE_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
def download_profile(self, qr: str, nickname: str | None = None) -> None:
with self._acquire_channel():
iccid = download_profile(self._client, qr)
if nickname and iccid:
set_profile_nickname(self._client, iccid, nickname)
def nickname_profile(self, iccid: str, nickname: str) -> None:
with self._acquire_channel():
set_profile_nickname(self._client, iccid, nickname)
def _enable_profile(self, iccid: str) -> int:
inner = encode_tlv(TAG_OK, encode_tlv(TAG_ICCID, string_to_tbcd(iccid)))
inner += b'\x01\x01\x01' # refreshFlag=1
response = es10x_command(self._client, encode_tlv(TAG_ENABLE_PROFILE, inner))
return require_tag(require_tag(response, TAG_ENABLE_PROFILE, "EnableProfileResponse"), TAG_STATUS, "EnableProfile status")[0]
def switch_profile(self, iccid: str) -> None:
with self._acquire_channel():
code = self._enable_profile(iccid)
if code == PROFILE_CAT_BUSY: # stale eUICC transaction, reset and retry
self._client._reset_modem()
self._client.open_isdr()
code = self._enable_profile(iccid)
if code not in (PROFILE_OK, PROFILE_NOT_IN_DISABLED_STATE):
raise LPAError(f"EnableProfile failed: {PROFILE_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
def is_euicc(self) -> bool:
# +CCHO:<n> -> ISD-R applet present, eUICC. Any error -> non-eUICC.
with self._acquire_lock():
try:
lines = self._client.query(f'AT+CCHO="{ISDR_AID}"')
except RuntimeError:
return False
for line in lines:
if line.startswith("+CCHO:") and (ch := line.split(":", 1)[1].strip()):
try:
self._client.query(f"AT+CCHC={ch}")
except (RuntimeError, TimeoutError):
pass
self._client.channel = None
return True
return False
Regular → Executable
+20 -2
View File
@@ -1,3 +1,5 @@
#!/usr/bin/env python3
import sys
import math
import os
from pathlib import Path
@@ -10,10 +12,13 @@ def get_chunk_name(name, idx, num_chunks):
def get_manifest_path(name):
return f"{name}.chunkmanifest"
def get_chunk_paths(path, file_size):
num_chunks = math.ceil(file_size / CHUNK_SIZE)
def _chunk_paths(path, num_chunks):
return [get_manifest_path(path)] + [get_chunk_name(path, i, num_chunks) for i in range(num_chunks)]
def get_chunk_targets(path, file_size):
num_chunks = math.ceil(file_size / CHUNK_SIZE)
return _chunk_paths(path, num_chunks)
def chunk_file(path, targets):
manifest_path, *chunk_paths = targets
with open(path, 'rb') as f:
@@ -26,6 +31,13 @@ def chunk_file(path, targets):
Path(manifest_path).write_text(str(len(chunk_paths)))
os.remove(path)
def get_existing_chunks(path):
if os.path.isfile(path):
return [path]
if os.path.isfile(manifest := get_manifest_path(path)):
num_chunks = int(Path(manifest).read_text().strip())
return _chunk_paths(path, num_chunks)
raise FileNotFoundError(path)
def read_file_chunked(path):
manifest_path = get_manifest_path(path)
@@ -35,3 +47,9 @@ def read_file_chunked(path):
if os.path.isfile(path):
return Path(path).read_bytes()
raise FileNotFoundError(path)
if __name__ == "__main__":
path = sys.argv[1]
chunk_paths = get_chunk_targets(path, os.path.getsize(path))
chunk_file(path, chunk_paths)
+16
View File
@@ -0,0 +1,16 @@
import os
from typing import cast
from openpilot.common.hardware.base import HardwareBase
from openpilot.common.hardware.tici.hardware import Tici
from openpilot.common.hardware.pc.hardware import Pc
TICI = os.path.isfile('/TICI')
AGNOS = os.path.isfile('/AGNOS')
PC = not TICI
if TICI:
HARDWARE = cast(HardwareBase, Tici())
else:
HARDWARE = cast(HardwareBase, Pc())
@@ -12,8 +12,6 @@ class HardwareNone {
public:
static std::string get_name() { return ""; }
static cereal::InitData::DeviceType get_device_type() { return cereal::InitData::DeviceType::UNKNOWN; }
static int get_voltage() { return 0; }
static int get_current() { return 0; }
static std::string get_serial() { return "cccccc"; }
@@ -24,6 +22,4 @@ public:
static void set_ir_power(int percentage) {}
static bool PC() { return false; }
static bool TICI() { return false; }
static bool AGNOS() { return false; }
};
+161
View File
@@ -0,0 +1,161 @@
import os
from abc import abstractmethod, ABC
from dataclasses import dataclass, fields
from cereal import log
from openpilot.common.esim.base import LPABase
NetworkType = log.DeviceState.NetworkType
NetworkStrength = log.DeviceState.NetworkStrength
@dataclass
class ThermalZone:
# a zone from /sys/class/thermal/thermal_zone*
name: str # a.k.a type
scale: float = 1000. # scale to get degrees in C
zone_number = -1
def read(self) -> float:
if self.zone_number < 0:
for n in os.listdir("/sys/devices/virtual/thermal"):
if not n.startswith("thermal_zone"):
continue
with open(os.path.join("/sys/devices/virtual/thermal", n, "type")) as f:
if f.read().strip() == self.name:
self.zone_number = int(n.removeprefix("thermal_zone"))
break
try:
with open(f"/sys/devices/virtual/thermal/thermal_zone{self.zone_number}/temp") as f:
return int(f.read()) / self.scale
except FileNotFoundError:
return 0
@dataclass
class ThermalConfig:
cpu: list[ThermalZone] | None = None
gpu: list[ThermalZone] | None = None
dsp: ThermalZone | None = None
pmic: list[ThermalZone] | None = None
memory: ThermalZone | None = None
intake: ThermalZone | None = None
exhaust: ThermalZone | None = None
gnss: ThermalZone | None = None
bottomSoc: ThermalZone | None = None
def get_msg(self):
ret = {}
for f in fields(ThermalConfig):
v = getattr(self, f.name)
if v is not None:
if isinstance(v, list):
ret[f.name + "TempC"] = [x.read() for x in v]
else:
ret[f.name + "TempC"] = v.read()
return ret
class HardwareBase(ABC):
@staticmethod
def get_cmdline() -> dict[str, str]:
with open('/proc/cmdline') as f:
cmdline = f.read()
return {kv[0]: kv[1] for kv in [s.split('=') for s in cmdline.split(' ')] if len(kv) == 2}
@staticmethod
def read_param_file(path, parser, default=0):
try:
with open(path) as f:
return parser(f.read())
except Exception:
return default
def booted(self) -> bool:
return True
def reboot(self, reason=None):
print("REBOOT!")
def uninstall(self):
print("uninstall")
def get_os_version(self):
return None
@abstractmethod
def get_device_type(self):
pass
def get_imei(self) -> str:
return ""
def get_serial(self):
return ""
def get_network_info(self):
return None
def get_network_type(self):
return NetworkType.none
def get_sim_info(self):
return {
'sim_id': '',
'mcc_mnc': None,
'network_type': ["Unknown"],
'sim_state': ["ABSENT"],
'data_connected': False
}
def get_sim_lpa(self) -> LPABase:
raise NotImplementedError("SIM LPA not available")
def get_network_strength(self, network_type):
return NetworkStrength.unknown
def get_network_metered(self, network_type) -> bool:
return network_type not in (NetworkType.none, NetworkType.wifi, NetworkType.ethernet)
def get_current_power_draw(self):
return 0
def get_som_power_draw(self):
return 0
def shutdown(self):
print("SHUTDOWN!")
def get_thermal_config(self):
return ThermalConfig()
def set_display_power(self, on: bool):
pass
def set_screen_brightness(self, percentage):
pass
def get_screen_brightness(self):
return 0
def set_power_save(self, powersave_enabled):
pass
def get_gpu_usage_percent(self):
return 0
def get_modem_temperatures(self):
return []
def initialize_hardware(self):
pass
def reset_internal_panda(self):
pass
def recover_internal_panda(self):
pass
def get_modem_data_usage(self):
return -1, -1
def set_ir_power(self, percent: int):
pass
@@ -2,14 +2,14 @@
#include <string>
#include "system/hardware/base.h"
#include "common/hardware/base.h"
#include "common/util.h"
#if __TICI__
#include "system/hardware/tici/hardware.h"
#include "common/hardware/tici/hardware.h"
#define Hardware HardwareTici
#else
#include "system/hardware/pc/hardware.h"
#include "common/hardware/pc/hardware.h"
#define Hardware HardwarePC
#endif
@@ -2,7 +2,7 @@ import os
import platform
from pathlib import Path
from openpilot.system.hardware import PC
from openpilot.common.hardware import PC
DEFAULT_DOWNLOAD_CACHE_ROOT = "/tmp/comma_download_cache"
+12
View File
@@ -0,0 +1,12 @@
#pragma once
#include <string>
#include "common/hardware/base.h"
class HardwarePC : public HardwareNone {
public:
static std::string get_name() { return "pc"; }
static cereal::InitData::DeviceType get_device_type() { return cereal::InitData::DeviceType::PC; }
static bool PC() { return true; }
};
+10
View File
@@ -0,0 +1,10 @@
from cereal import log
from openpilot.common.hardware.base import HardwareBase
class Pc(HardwareBase):
def get_device_type(self):
return "pc"
def get_network_type(self):
# some stuff is gated on wifi, so just assume for now
return log.DeviceState.NetworkType.wifi
+84
View File
@@ -0,0 +1,84 @@
[
{
"name": "xbl",
"url": "https://commadist.azureedge.net/agnosupdate/xbl-e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb.img.xz",
"hash": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
"hash_raw": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
"size": 3282256,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "bea7f1a24428c3ededf672fa4fc78baf180cfbd8aafb77c974655b38517283e3"
},
{
"name": "xbl_config",
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85.img.xz",
"hash": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
"hash_raw": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
"size": 98124,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "fb18cde08a98a168961ecd357e92474823046752b94e112f59fe51a6acd7197d"
},
{
"name": "abl",
"url": "https://commadist.azureedge.net/agnosupdate/abl-b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c.img.xz",
"hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
"hash_raw": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
"size": 274432,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c"
},
{
"name": "aop",
"url": "https://commadist.azureedge.net/agnosupdate/aop-78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1.img.xz",
"hash": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
"hash_raw": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
"size": 184364,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "6c9135446bd3fc075fcee59b887a12e49029ab1f98ed8d6d1e32c73569d47de3"
},
{
"name": "devcfg",
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc.img.xz",
"hash": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
"hash_raw": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
"size": 40336,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "2a67971602012c1b43544964709da13c322786b456a8e78568b117e8b1540ce3"
},
{
"name": "boot",
"url": "https://commadist.azureedge.net/agnosupdate/boot-8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8.img.xz",
"hash": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
"hash_raw": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
"size": 17487872,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "edca8bee1531e66953d107eeceeed2dc7b3ca46417e49d55508f94e58bf95db8"
},
{
"name": "system",
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img.xz",
"hash": "78acfe16a7b62a3a91fc7a81f40a693e4468cec1c69df7d0b1e550aacc646113",
"hash_raw": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
"size": 4718592000,
"sparse": true,
"full_check": false,
"has_ab": true,
"ondevice_hash": "743142c5a898f27b2a1029cca42c8a5d5d1fc0096414422b850fe84c8d0b8342",
"alt": {
"hash": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img",
"size": 4718592000
}
}
]
@@ -10,10 +10,7 @@ from collections.abc import Generator
import requests
import openpilot.system.updated.casync.casync as casync
SPARSE_CHUNK_FMT = struct.Struct('H2xI4x')
CAIBX_URL = "https://commadist.azureedge.net/agnosupdate/"
AGNOS_MANIFEST_FILE = "system/hardware/tici/agnos.json"
@@ -187,50 +184,6 @@ def extract_compressed_image(target_slot_number: int, partition: dict, cloudlog)
os.sync()
def extract_casync_image(target_slot_number: int, partition: dict, cloudlog):
path = get_partition_path(target_slot_number, partition)
seed_path = path[:-1] + ('b' if path[-1] == 'a' else 'a')
target = casync.parse_caibx(partition['casync_caibx'])
sources: list[tuple[str, casync.ChunkReader, casync.ChunkDict]] = []
# First source is the current partition.
try:
raw_hash = get_raw_hash(seed_path, partition['size'])
caibx_url = f"{CAIBX_URL}{partition['name']}-{raw_hash}.caibx"
try:
cloudlog.info(f"casync fetching {caibx_url}")
sources += [('seed', casync.FileChunkReader(seed_path), casync.build_chunk_dict(casync.parse_caibx(caibx_url)))]
except requests.RequestException:
cloudlog.error(f"casync failed to load {caibx_url}")
except Exception:
cloudlog.exception("casync failed to hash seed partition")
# Second source is the target partition, this allows for resuming
sources += [('target', casync.FileChunkReader(path), casync.build_chunk_dict(target))]
# Finally we add the remote source to download any missing chunks
sources += [('remote', casync.RemoteChunkReader(partition['casync_store']), casync.build_chunk_dict(target))]
last_p = 0
def progress(cur):
nonlocal last_p
p = int(cur / partition['size'] * 100)
if p != last_p:
last_p = p
print(f"Installing {partition['name']}: {p}", flush=True)
stats = casync.extract(target, sources, path, progress)
cloudlog.error(f'casync done {json.dumps(stats)}')
os.sync()
if not verify_partition(target_slot_number, partition, force_full_check=True):
raise Exception(f"Raw hash mismatch '{partition['hash_raw'].lower()}'")
def flash_partition(target_slot_number: int, partition: dict, cloudlog, standalone=False):
cloudlog.info(f"Downloading and writing {partition['name']}")
@@ -245,10 +198,7 @@ def flash_partition(target_slot_number: int, partition: dict, cloudlog, standalo
path = get_partition_path(target_slot_number, partition)
if ('casync_caibx' in partition) and not standalone:
extract_casync_image(target_slot_number, partition, cloudlog)
else:
extract_compressed_image(target_slot_number, partition, cloudlog)
extract_compressed_image(target_slot_number, partition, cloudlog)
# Write hash after successful flash
if not full_check:
@@ -130,47 +130,47 @@
},
{
"name": "xbl",
"url": "https://commadist.azureedge.net/agnosupdate/xbl-dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6.img.xz",
"hash": "dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6",
"hash_raw": "dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6",
"url": "https://commadist.azureedge.net/agnosupdate/xbl-e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb.img.xz",
"hash": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
"hash_raw": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
"size": 3282256,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "d47a08914d2376557b03f1231b7233508222c04b57d781f9daf77c63eab92c2e"
"ondevice_hash": "bea7f1a24428c3ededf672fa4fc78baf180cfbd8aafb77c974655b38517283e3"
},
{
"name": "xbl_config",
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9.img.xz",
"hash": "1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9",
"hash_raw": "1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9",
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85.img.xz",
"hash": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
"hash_raw": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
"size": 98124,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "e7d04d9f040c9c040cdf013335d0b6d6e9346311458baeb2461b193e954f5f1c"
"ondevice_hash": "fb18cde08a98a168961ecd357e92474823046752b94e112f59fe51a6acd7197d"
},
{
"name": "abl",
"url": "https://commadist.azureedge.net/agnosupdate/abl-556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee.img.xz",
"hash": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee",
"hash_raw": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee",
"url": "https://commadist.azureedge.net/agnosupdate/abl-b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c.img.xz",
"hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
"hash_raw": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
"size": 274432,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee"
"ondevice_hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c"
},
{
"name": "aop",
"url": "https://commadist.azureedge.net/agnosupdate/aop-4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788.img.xz",
"hash": "4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788",
"hash_raw": "4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788",
"url": "https://commadist.azureedge.net/agnosupdate/aop-78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1.img.xz",
"hash": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
"hash_raw": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
"size": 184364,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "3aa0a79149ec57f4bc8c38f7bbdf4f6630dd659e49a111ce6258d2d06a07c8e5"
"ondevice_hash": "6c9135446bd3fc075fcee59b887a12e49029ab1f98ed8d6d1e32c73569d47de3"
},
{
"name": "bluetooth",
@@ -207,14 +207,14 @@
},
{
"name": "devcfg",
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585.img.xz",
"hash": "2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585",
"hash_raw": "2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585",
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc.img.xz",
"hash": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
"hash_raw": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
"size": 40336,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "3d7bb33588491a2a40091a7e1cf6cb65e6dd503f69b640aba484d723f1ad47e8"
"ondevice_hash": "2a67971602012c1b43544964709da13c322786b456a8e78568b117e8b1540ce3"
},
{
"name": "devinfo",
@@ -339,51 +339,51 @@
},
{
"name": "boot",
"url": "https://commadist.azureedge.net/agnosupdate/boot-d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51.img.xz",
"hash": "d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51",
"hash_raw": "d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51",
"size": 17500160,
"url": "https://commadist.azureedge.net/agnosupdate/boot-8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8.img.xz",
"hash": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
"hash_raw": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
"size": 17487872,
"sparse": false,
"full_check": true,
"has_ab": true,
"ondevice_hash": "2454108de1161289bc4a75449ad6421f1772b13b3e5cba68a84fca7530557699"
"ondevice_hash": "edca8bee1531e66953d107eeceeed2dc7b3ca46417e49d55508f94e58bf95db8"
},
{
"name": "system",
"url": "https://commadist.azureedge.net/agnosupdate/system-dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5.img.xz",
"hash": "5f319030ad05942267b77f1a4686c4ca24cc09b2c2a4688e57342ffc9720fd49",
"hash_raw": "dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5",
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img.xz",
"hash": "78acfe16a7b62a3a91fc7a81f40a693e4468cec1c69df7d0b1e550aacc646113",
"hash_raw": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
"size": 4718592000,
"sparse": true,
"full_check": false,
"has_ab": true,
"ondevice_hash": "c12f1b7d790a418aea17424accf4cd59c575e5745cad82bdc9452f384483648c",
"ondevice_hash": "743142c5a898f27b2a1029cca42c8a5d5d1fc0096414422b850fe84c8d0b8342",
"alt": {
"hash": "dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5",
"url": "https://commadist.azureedge.net/agnosupdate/system-dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5.img",
"hash": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img",
"size": 4718592000
}
},
{
"name": "userdata_90",
"url": "https://commadist.azureedge.net/agnosupdate/userdata_90-a7b25ea29255f4fd3a2da99e037f40b4ca10bd4afd57dd96563353b8dfb0f634.img.xz",
"hash": "7ea9d7d4685ec36bbfdf06afe0b51650d567416c3092fef96bd97158ed322742",
"hash_raw": "a7b25ea29255f4fd3a2da99e037f40b4ca10bd4afd57dd96563353b8dfb0f634",
"url": "https://commadist.azureedge.net/agnosupdate/userdata_90-14a3fc6e9bd148b9deebf6ae9df2f1b3b759e629b337e41b6895864cdd51f630.img.xz",
"hash": "52160dd01b30b3dc572226e8d549a034b03bc328e80f1f4cd6a857b6dd447687",
"hash_raw": "14a3fc6e9bd148b9deebf6ae9df2f1b3b759e629b337e41b6895864cdd51f630",
"size": 96636764160,
"sparse": true,
"full_check": true,
"has_ab": false,
"ondevice_hash": "79ed653c1679d84b13ee23083a511b0e668454e4af9b0db99a3279072ed041c1"
"ondevice_hash": "3bbc052c7793087946b0cd668c1778f930084f6b00896aeebd193dfce53fa518"
},
{
"name": "userdata_89",
"url": "https://commadist.azureedge.net/agnosupdate/userdata_89-8e428632c967aa609cac184bff938a90240e53ffd3b4fca40bc94c33c81202ba.img.xz",
"hash": "7104cdb0384e4ecb1ebfa6136a2330251bc8aa829b9ec48c4b740f656252d382",
"hash_raw": "8e428632c967aa609cac184bff938a90240e53ffd3b4fca40bc94c33c81202ba",
"url": "https://commadist.azureedge.net/agnosupdate/userdata_89-425c69d021f4ee2f767963bf7b991d1a492485fa465c5f5d001e1cf7de3d62a0.img.xz",
"hash": "02a8c5512754d7781d930d242be3fad01fbce652297c69dfe8722dc92b18dc09",
"hash_raw": "425c69d021f4ee2f767963bf7b991d1a492485fa465c5f5d001e1cf7de3d62a0",
"size": 95563022336,
"sparse": true,
"full_check": true,
"has_ab": false,
"ondevice_hash": "fbede3b0831dbc4a4edd336e5f547f4978902b9421fb1484e86c416192c59165"
"ondevice_hash": "3667d500f91b08a4671d7c09623eb6ae8fc905d9273893b50689e214005c7fa6"
}
]
+87
View File
@@ -0,0 +1,87 @@
#pragma once
#include <cassert>
#include <fstream>
#include <map>
#include <string>
#include <algorithm> // for std::clamp
#include "common/util.h"
#include "common/hardware/base.h"
class HardwareTici : public HardwareNone {
public:
static std::string get_name() {
static const std::string name = []() {
std::string model = util::read_file("/sys/firmware/devicetree/base/model");
return util::strip(model.substr(std::string("comma ").size()));
}();
return name;
}
static cereal::InitData::DeviceType get_device_type() {
static const std::map<std::string, cereal::InitData::DeviceType> device_map = {
{"tizi", cereal::InitData::DeviceType::TIZI},
{"mici", cereal::InitData::DeviceType::MICI}
};
static const auto it = device_map.find(get_name());
assert(it != device_map.end());
return it->second;
}
static std::string get_serial() {
static std::string serial("");
if (serial.empty()) {
std::ifstream stream("/proc/cmdline");
std::string cmdline;
std::getline(stream, cmdline);
auto start = cmdline.find("serialno=");
if (start == std::string::npos) {
serial = "cccccc";
} else {
auto end = cmdline.find(" ", start + 9);
serial = cmdline.substr(start + 9, end - start - 9);
}
}
return serial;
}
static void set_ir_power(int percent) {
auto device = get_device_type();
if (device == cereal::InitData::DeviceType::TIZI) {
return;
}
int value = util::map_val(std::clamp(percent, 0, 100), 0, 100, 0, 300);
std::ofstream("/sys/class/leds/led:switch_2/brightness") << 0 << "\n";
std::ofstream("/sys/class/leds/led:torch_2/brightness") << value << "\n";
std::ofstream("/sys/class/leds/led:switch_2/brightness") << value << "\n";
}
static std::map<std::string, std::string> get_init_logs() {
std::map<std::string, std::string> ret = {
{"/BUILD", util::read_file("/BUILD")},
{"lsblk", util::check_output("lsblk -o NAME,SIZE,STATE,VENDOR,MODEL,REV,SERIAL")},
{"SOM ID", util::read_file("/sys/devices/platform/vendor/vendor:gpio-som-id/som_id")},
};
std::string bs = util::check_output("abctl --boot_slot");
ret["boot slot"] = bs.substr(0, bs.find_first_of("\n"));
std::string temp = util::read_file("/dev/disk/by-partlabel/ssd");
temp.erase(temp.find_last_not_of(std::string("\0\r\n", 3))+1);
ret["boot temp"] = temp;
// TODO: log something from system and boot
for (std::string part : {"xbl", "abl", "aop", "devcfg", "xbl_config"}) {
for (std::string slot : {"a", "b"}) {
std::string partition = part + "_" + slot;
std::string hash = util::check_output("sha256sum /dev/disk/by-partlabel/" + partition);
ret[partition] = hash.substr(0, hash.find_first_of(" "));
}
}
return ret;
}
};
+419
View File
@@ -0,0 +1,419 @@
import configparser
import json
import os
import socket
import subprocess
import time
from functools import cached_property, lru_cache
from pathlib import Path
from cereal import log
from openpilot.common.utils import sudo_read, sudo_write
from openpilot.common.gpio import gpio_set, gpio_init, get_irqs_for_action
from openpilot.common.esim.base import LPABase
from openpilot.common.hardware.base import HardwareBase, ThermalConfig, ThermalZone
from openpilot.common.esim.lpa import TiciLPA
from openpilot.common.hardware.tici.pins import GPIO
from openpilot.common.hardware.tici.amplifier import Amplifier
MODEM_STATE_PATH = "/dev/shm/modem"
NetworkType = log.DeviceState.NetworkType
NetworkStrength = log.DeviceState.NetworkStrength
def affine_irq(val, action):
irqs = get_irqs_for_action(action)
if len(irqs) == 0:
print(f"No IRQs found for '{action}'")
return
for i in irqs:
sudo_write(str(val), f"/proc/irq/{i}/smp_affinity_list")
@lru_cache
def get_device_type():
# lru_cache and cache can cause memory leaks when used in classes
with open("/sys/firmware/devicetree/base/model") as f:
model = f.read().strip('\x00')
return model.split('comma ')[-1]
def wpa_supplicant_cmd(cmd: str, timeout: float = 0.2) -> dict[str, str]:
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
sock.settimeout(timeout)
sock.bind(f"\0openpilot-wpa-{os.getpid()}-{time.monotonic_ns()}")
sock.connect("/run/wpa_supplicant/wlan0")
sock.send(cmd.encode())
while True:
out = sock.recv(8192).decode("utf-8", "replace")
if out.startswith("<"):
continue
if out.startswith("FAIL"):
return {}
return dict(l.split("=", 1) for l in out.splitlines() if "=" in l)
def get_default_route_iface():
with open("/proc/net/route") as f:
routes = [(int(route[6]), route[0]) for line in f.readlines()[1:] if (route := line.split())[1] == "00000000" and int(route[3], 16) & 0x1]
return min(routes)[1] if routes else None
class Tici(HardwareBase):
@cached_property
def amplifier(self):
if self.get_device_type() == "mici":
return None
return Amplifier()
def get_modem_state(self) -> dict:
try:
with open(MODEM_STATE_PATH) as f:
return json.load(f)
except (FileNotFoundError, json.JSONDecodeError):
return {}
def get_os_version(self):
with open("/VERSION") as f:
return f.read().strip()
def get_device_type(self):
return get_device_type()
def reboot(self, reason=None):
subprocess.check_output(["sudo", "reboot"])
def uninstall(self):
Path("/data/__system_reset__").touch()
os.sync()
self.reboot()
def get_serial(self):
return self.get_cmdline()['androidboot.serialno']
def get_voltage(self):
with open("/sys/class/hwmon/hwmon1/in1_input") as f:
return int(f.read())
def get_current(self):
with open("/sys/class/hwmon/hwmon1/curr1_input") as f:
return int(f.read())
def set_ir_power(self, percent: int):
if self.get_device_type() == "tizi":
return
value = int((percent / 100) * 300)
with open("/sys/class/leds/led:switch_2/brightness", "w") as f:
f.write("0\n")
with open("/sys/class/leds/led:torch_2/brightness", "w") as f:
f.write(f"{value}\n")
with open("/sys/class/leds/led:switch_2/brightness", "w") as f:
f.write(f"{value}\n")
def get_network_type(self):
try:
if (iface := get_default_route_iface()):
if iface.startswith('wlan'):
return NetworkType.wifi
if iface.startswith('eth'):
return NetworkType.ethernet
except Exception:
pass
ms = self.get_modem_state()
if ms.get('connected'):
nt = ms.get('network_type', '')
if nt == 'nr':
return NetworkType.cell5G
elif nt == 'lte':
return NetworkType.cell4G
elif nt in ('utran', 'umts'):
return NetworkType.cell3G
elif nt == 'gsm':
return NetworkType.cell2G
return NetworkType.none
def get_sim_info(self):
ms = self.get_modem_state()
sim_id = ms.get('iccid', '')
return {
'sim_id': sim_id,
'mcc_mnc': ms.get('mcc_mnc') or None,
'network_type': ["Unknown"],
'sim_state': ["ABSENT"] if not sim_id else ["READY"],
'data_connected': ms.get('connected', False),
}
def get_sim_lpa(self) -> LPABase:
return TiciLPA()
def get_imei(self):
return self.get_modem_state().get('imei', '')
def get_network_info(self):
if self.get_device_type() == "mici":
return None
ms = self.get_modem_state()
return {
'technology': ms.get('network_type', '').upper() if ms.get('network_type') else '',
'operator': ms.get('operator', ''),
'band': ms.get('band', ''),
'channel': ms.get('channel', 0),
'extra': ms.get('extra', ''),
'state': ms.get('state', 'UNKNOWN'),
}
def parse_strength(self, percentage):
if percentage < 25:
return NetworkStrength.poor
elif percentage < 50:
return NetworkStrength.moderate
elif percentage < 75:
return NetworkStrength.good
else:
return NetworkStrength.great
def get_network_strength(self, network_type):
network_strength = NetworkStrength.unknown
try:
if network_type == NetworkType.none:
pass
elif network_type == NetworkType.ethernet:
network_strength = NetworkStrength.great
elif network_type == NetworkType.wifi:
rssi = wpa_supplicant_cmd("SIGNAL_POLL").get("RSSI")
if rssi is not None:
dbm = int(rssi)
if -100 < dbm <= 0:
network_strength = self.parse_strength(120 + max(-100, min(-20, dbm)))
else: # Cellular
network_strength = self.parse_strength(self.get_modem_state().get('signal_quality', 0))
except Exception:
pass
return network_strength
def get_network_metered(self, network_type) -> bool:
if network_type in (NetworkType.cell2G, NetworkType.cell3G, NetworkType.cell4G, NetworkType.cell5G):
from openpilot.common.params import Params
return Params().get_bool("GsmMetered")
try:
if network_type == NetworkType.wifi:
ssid = wpa_supplicant_cmd("STATUS").get("ssid", "")
if ssid:
# wpa_supplicant escapes non-printable bytes as \xNN; NM keyfile stores ASCII SSIDs as a literal and others as a byte;byte; list
ssid_bytes = ssid.encode().decode('unicode_escape').encode('latin-1')
ssid_keyfile_list = ';'.join(str(b) for b in ssid_bytes) + ';'
nm_dirs = ("/run/NetworkManager/system-connections", "/data/etc/NetworkManager/system-connections")
for fpath in (p for d in nm_dirs for p in Path(d).glob("*.nmconnection")):
raw = sudo_read(str(fpath))
if not raw:
continue
cp = configparser.ConfigParser(interpolation=None)
try:
cp.read_string(raw)
keyfile_ssid = cp.get("wifi", "ssid", fallback="")
if keyfile_ssid != ssid and keyfile_ssid != ssid_keyfile_list:
continue
metered = cp.getint("connection", "metered", fallback=0)
except (configparser.Error, ValueError):
continue
if metered == 1: # NM_METERED_YES
return True
if metered == 2: # NM_METERED_NO
return False
break
except Exception:
pass
return super().get_network_metered(network_type)
def get_modem_temperatures(self):
return self.get_modem_state().get('temperatures', [])
def get_current_power_draw(self):
return (self.read_param_file("/sys/class/hwmon/hwmon1/power1_input", int) / 1e6)
def get_som_power_draw(self):
return (self.read_param_file("/sys/class/power_supply/bms/voltage_now", int) * self.read_param_file("/sys/class/power_supply/bms/current_now", int) / 1e12)
def shutdown(self):
os.system("sudo poweroff")
def get_thermal_config(self):
intake, exhaust, gnss, bottomSoc = None, None, None, None
if self.get_device_type() == "mici":
gnss = ThermalZone("gnss")
intake = ThermalZone("intake")
exhaust = ThermalZone("exhaust")
bottomSoc = ThermalZone("bottom_soc")
return ThermalConfig(cpu=[ThermalZone(f"cpu{i}-silver-usr") for i in range(4)] +
[ThermalZone(f"cpu{i}-gold-usr") for i in range(4)],
gpu=[ThermalZone("gpu0-usr"), ThermalZone("gpu1-usr")],
dsp=ThermalZone("compute-hvx-usr"),
memory=ThermalZone("ddr-usr"),
pmic=[ThermalZone("pm8998_tz"), ThermalZone("pm8005_tz")],
intake=intake,
exhaust=exhaust,
gnss=gnss,
bottomSoc=bottomSoc)
def set_display_power(self, on):
try:
with open("/sys/class/backlight/panel0-backlight/bl_power", "w") as f:
f.write("0" if on else "4")
except Exception:
pass
def set_screen_brightness(self, percentage):
try:
with open("/sys/class/backlight/panel0-backlight/max_brightness") as f:
max_brightness = float(f.read().strip())
val = int(percentage * (max_brightness / 100.))
with open("/sys/class/backlight/panel0-backlight/brightness", "w") as f:
f.write(str(val))
except Exception:
pass
def get_screen_brightness(self):
try:
with open("/sys/class/backlight/panel0-backlight/max_brightness") as f:
max_brightness = float(f.read().strip())
with open("/sys/class/backlight/panel0-backlight/brightness") as f:
return int(float(f.read()) / (max_brightness / 100.))
except Exception:
return 0
def set_power_save(self, powersave_enabled):
# amplifier, 100mW at idle
if self.amplifier is not None:
self.amplifier.set_global_shutdown(amp_disabled=powersave_enabled)
if not powersave_enabled:
self.amplifier.initialize_configuration()
# *** CPU config ***
# offline big cluster
for i in range(4, 8):
val = '0' if powersave_enabled else '1'
sudo_write(val, f'/sys/devices/system/cpu/cpu{i}/online')
for n in ('0', '4'):
if powersave_enabled and n == '4':
continue
gov = 'ondemand' if powersave_enabled else 'performance'
sudo_write(gov, f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_governor')
if not powersave_enabled:
# cap max core freq to 1689 Mhz
sudo_write('1689600', f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_max_freq')
# *** IRQ config ***
# GPU, modeld core
affine_irq(7, "kgsl-3d0")
# camerad core
camera_irqs = ("a5", "cci", "cpas_camnoc", "cpas-cdm", "csid", "ife", "csid-lite", "ife-lite")
for n in camera_irqs:
affine_irq(6, n)
def get_gpu_usage_percent(self):
try:
with open('/sys/class/kgsl/kgsl-3d0/gpubusy') as f:
used, total = f.read().strip().split()
return 100.0 * int(used) / int(total)
except Exception:
return 0
def initialize_hardware(self):
if self.amplifier is not None:
self.amplifier.initialize_configuration()
# Allow hardwared to write engagement status to kmsg
os.system("sudo chmod a+w /dev/kmsg")
# Ensure fan gpio is enabled so fan runs until shutdown, also turned on at boot by the ABL
gpio_init(GPIO.SOM_ST_IO, True)
gpio_set(GPIO.SOM_ST_IO, 1)
# *** IRQ config ***
# mask off big cluster from default affinity
sudo_write("f", "/proc/irq/default_smp_affinity")
# move these off the default core
affine_irq(1, "msm_vidc") # encoders
affine_irq(1, "i2c_geni") # sensors
# *** GPU config ***
# https://github.com/commaai/agnos-kernel-sdm845/blob/master/arch/arm64/boot/dts/qcom/sdm845-gpu.dtsi#L216
affine_irq(5, "fts_ts") # touch
affine_irq(5, "msm_drm") # display
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/min_pwrlevel")
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/max_pwrlevel")
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_bus_on")
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_clk_on")
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_rail_on")
sudo_write("1000", "/sys/class/kgsl/kgsl-3d0/idle_timer")
sudo_write("performance", "/sys/class/kgsl/kgsl-3d0/devfreq/governor")
sudo_write("710", "/sys/class/kgsl/kgsl-3d0/max_clock_mhz")
# setup governors
sudo_write("performance", "/sys/class/devfreq/soc:qcom,cpubw/governor")
sudo_write("performance", "/sys/class/devfreq/soc:qcom,memlat-cpu0/governor")
sudo_write("performance", "/sys/class/devfreq/soc:qcom,memlat-cpu4/governor")
# *** VIDC (encoder) config ***
sudo_write("N", "/sys/kernel/debug/msm_vidc/clock_scaling")
sudo_write("Y", "/sys/kernel/debug/msm_vidc/disable_thermal_mitigation")
# pandad core
affine_irq(3, "spi_geni") # SPI
try:
pid = subprocess.check_output(["pgrep", "-f", "spi0"], encoding='utf8').strip()
subprocess.call(["sudo", "chrt", "-f", "-p", "1", pid])
subprocess.call(["sudo", "taskset", "-pc", "3", pid])
except subprocess.CalledProcessError as e:
print(str(e))
def get_modem_data_usage(self):
ms = self.get_modem_state()
return ms.get('tx_bytes', -1), ms.get('rx_bytes', -1)
def reset_internal_panda(self):
gpio_init(GPIO.STM_RST_N, True)
gpio_init(GPIO.STM_BOOT0, True)
gpio_set(GPIO.STM_RST_N, 1)
gpio_set(GPIO.STM_BOOT0, 0)
time.sleep(0.01)
gpio_set(GPIO.STM_RST_N, 0)
def recover_internal_panda(self):
gpio_init(GPIO.STM_RST_N, True)
gpio_init(GPIO.STM_BOOT0, True)
gpio_set(GPIO.STM_RST_N, 1)
gpio_set(GPIO.STM_BOOT0, 1)
time.sleep(0.01)
gpio_set(GPIO.STM_RST_N, 0)
time.sleep(0.01)
gpio_set(GPIO.STM_BOOT0, 0)
def booted(self):
# this normally boots within 8s, but on rare occasions takes 30+s
encoder_state = sudo_read("/sys/kernel/debug/msm_vidc/core0/info")
if "Core state: 0" in encoder_state and (time.monotonic() < 60*2):
return False
return True
if __name__ == "__main__":
t = Tici()
t.initialize_hardware()
t.set_power_save(False)
print(t.get_sim_info())
+587
View File
@@ -0,0 +1,587 @@
#!/usr/bin/env python3
import fcntl
import json
import logging
import os
import serial
import signal
import subprocess
import tempfile
import time
from ipaddress import IPv4Address, AddressValueError
from enum import Enum
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s.%(msecs)03d %(levelname)-7s modem: %(message)s",
datefmt="%H:%M:%S",
)
AT_PORT = "/dev/modem_at0"
PPP_PORT = "/dev/modem_at1"
STATE_PATH = "/dev/shm/modem"
AT_LOCK = "/dev/shm/modem.lock" # shared with LPA
AT_INIT = [
"ATE0", # disable command echo
"ATV1", # verbose result codes (CONNECT/BUSY/NO CARRIER, not numeric)
"AT+CMEE=1", # numeric +CME ERROR codes on failures (per 3GPP 27.007)
"ATX4", # extended result codes: busy + dial tone detection, line speed in CONNECT
"AT&C1", # DCD pin follows carrier state (V.250 default)
"AT+CREG=2", # registration URCs include location info
"AT+CGREG=2", # GPRS registration URCs include location info
]
CREG = {0: "not_registered", 1: "home", 2: "searching", 3: "denied", 4: "unknown", 5: "roaming"}
# 3GPP TS 27.007 +COPS <AcT> -> network type
NETWORK_TYPE = {0: "gsm", 1: "gsm", 3: "gsm", 8: "gsm",
2: "utran", 4: "utran", 5: "utran", 6: "utran",
7: "lte", 9: "lte", 10: "lte",
11: "nr", 12: "nr", 13: "nr"}
DIAL_CID = 1
WEBBING_ICCID_PREFIX = "8985235"
PPPD_CMD = [
"sudo", "pppd", PPP_PORT, "460800", "noauth", "nodetach", "noipdefault", "usepeerdns",
"nodefaultroute", "connect",
"/usr/sbin/chat -v ABORT 'NO CARRIER' ABORT 'NO DIALTONE' ABORT 'BUSY' " +
f"ABORT 'NO ANSWER' ABORT 'ERROR' TIMEOUT 5 '' AT OK ATD*99***{DIAL_CID}# CONNECT ''",
"lcp-echo-interval", "30", "lcp-echo-failure", "4", "mtu", "1500", "mru", "1500",
"novj", "novjccomp", "ipcp-accept-local", "ipcp-accept-remote", "nomagic",
"user", '""', "password", '""',
]
INITIAL_STATE = {
"seconds_since_boot": 0,
"state": "INITIALIZING",
"connected": False, "ip_address": "",
"iccid": "", "mcc_mnc": "", "imei": "", "modem_version": "",
"signal_strength": 0, "signal_quality": 0,
"network_type": "unknown", "operator": "", "band": "", "channel": 0,
"registration": "unknown", "temperatures": [], "extra": "",
"tx_bytes": 0, "rx_bytes": 0,
}
class State(Enum):
INITIALIZING = "INITIALIZING"
SEARCHING = "SEARCHING"
CONNECTING = "CONNECTING"
CONNECTED = "CONNECTED"
DISCONNECTING = "DISCONNECTING"
STATE_WAIT = 1.0 # seconds to wait after each state handler returns
class PPPSession:
"""Owns pppd lifecycle, fail tracking, and PPP routing."""
MAX_FAILS = 3
def __init__(self):
self._proc: subprocess.Popen | None = None
self._fails = 0
self._peer = ""
def start(self):
self._proc = subprocess.Popen(PPPD_CMD, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
self._peer = ""
logging.info(f"PPP dialing CID {DIAL_CID}")
def kill(self):
subprocess.run(["sudo", "killall", "-9", "pppd"], capture_output=True)
self._peer = ""
@staticmethod
def reset_data_port():
"""Drop DTR on PPP_PORT so the modem terminates any stuck PPP session."""
try:
with serial.Serial(PPP_PORT, 460800, timeout=1) as s:
s.dtr = False
time.sleep(0.2)
s.dtr = True
except Exception as e:
logging.warning(f"data port reset failed: {e}")
def has_exited(self) -> bool:
return self._proc is not None and self._proc.poll() is not None
def reset_fail_counter(self):
self._fails = 0
def record_fail(self) -> bool:
"""Bump fail counter; return True if at the give-up limit."""
self._fails += 1
return self._fails >= self.MAX_FAILS
@property
def fails(self) -> int:
return self._fails
def maybe_install_routes(self, ip: str, peer: str) -> bool:
"""Install routes if peer changed; kill the session on failure so the state machine reconnects."""
if not peer or peer == self._peer:
return False
try:
IPv4Address(ip)
IPv4Address(peer)
except AddressValueError:
logging.warning(f"refusing route install with non-IPv4 ip={ip!r} peer={peer!r}")
self.kill()
return False
self.cleanup_routes()
cmds = [
["sudo", "ip", "route", "add", "default", "via", peer, "dev", "ppp0", "metric", "1000"],
["sudo", "ip", "route", "add", "default", "via", peer, "dev", "ppp0", "table", "1000"],
["sudo", "ip", "rule", "add", "from", ip, "table", "1000"],
]
for cmd in cmds:
r = subprocess.run(cmd, capture_output=True, text=True)
if r.returncode != 0:
logging.warning(f"route install failed ({' '.join(cmd[1:])}): {r.stderr.strip()}")
self.cleanup_routes()
self.kill()
return False
logging.info(f"route set up for {ip} via {peer}")
self._peer = peer
return True
def maybe_install_dns(self, dns_servers: list[str]) -> bool:
"""Register DNS servers with systemd-resolved; kill the session on failure to force a retry."""
if not dns_servers:
return False
for cmd in (["sudo", "resolvectl", "dns", "ppp0", *dns_servers],
["sudo", "resolvectl", "default-route", "ppp0", "yes"]):
r = subprocess.run(cmd, capture_output=True, text=True)
if r.returncode != 0:
logging.warning(f"resolvectl failed ({' '.join(cmd[1:])}): {r.stderr.strip()}")
self.kill()
return False
logging.info(f"resolvectl: ppp0 DNS = {dns_servers}")
return True
@staticmethod
def cleanup_routes():
subprocess.run(["sudo", "ip", "route", "del", "default", "dev", "ppp0"], capture_output=True)
subprocess.run(["sudo", "ip", "route", "flush", "table", "1000"], capture_output=True)
# rules don't have a flush; delete until none remain
while subprocess.run(["sudo", "ip", "rule", "del", "table", "1000"], capture_output=True).returncode == 0:
pass
subprocess.run(["sudo", "resolvectl", "revert", "ppp0"], capture_output=True)
class Modem:
def __init__(self):
self._ppp = PPPSession()
self._sim_change = False
self._apn = "" # blank = network-provided via PCO
self._roaming_allowed = True
self.running = True
self.S = INITIAL_STATE.copy()
@staticmethod
def _read_param(key):
try:
with open(f"/data/params/d/{key}") as f:
return f.read().strip()
except FileNotFoundError:
return ""
@staticmethod
def _parse_reg(v: str) -> str:
try:
return CREG.get(int(v.split(",")[1].strip('"')), "unknown")
except (ValueError, IndexError):
return "unknown"
@staticmethod
def _has_modem_manager() -> bool:
return os.path.isfile("/lib/systemd/system/ModemManager.service")
def _is_roaming_allowed(self) -> bool:
if self.S["iccid"].startswith(WEBBING_ICCID_PREFIX):
return True
return self._read_param("GsmRoaming") == "1"
def _publish_state(self, **kwargs):
self.S.update(kwargs)
self.S["seconds_since_boot"] = time.monotonic()
with tempfile.NamedTemporaryFile(mode="w", dir="/dev/shm", delete=False) as f:
json.dump(self.S, f, indent=2)
os.chmod(f.name, 0o644)
os.replace(f.name, STATE_PATH)
def _at(self, cmd):
"""Send AT command, return response lines. [] on error or if LPA holds port."""
fd = os.open(AT_LOCK, os.O_CREAT | os.O_RDWR, 0o666)
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
os.close(fd)
return []
try:
with serial.Serial(AT_PORT, 9600, timeout=5) as ser:
ser.reset_input_buffer()
ser.write((cmd + "\r").encode())
lines = []
while True:
raw = ser.readline()
if not raw:
raise TimeoutError("AT timeout")
line = raw.decode(errors="ignore").strip()
if not line:
continue
if line == "OK":
break
if line == "ERROR" or line.startswith("+CME ERROR"):
raise RuntimeError(line)
lines.append(line)
return lines
except (RuntimeError, TimeoutError, OSError) as e:
logging.info(f"AT {cmd} failed: {e}")
return []
finally:
fcntl.flock(fd, fcntl.LOCK_UN)
os.close(fd)
def _atv(self, cmd, pfx):
for line in self._at(cmd):
if pfx in line and ":" in line:
return line.split(":", 1)[1].strip()
return None
def _init_at_channel(self) -> bool:
"""Run AT_INIT and confirm ATE0 took effect. Returns False if echo is still on."""
for c in AT_INIT:
self._at(c)
r = self._at("AT+CGMI")
return bool(r) and not r[0].startswith("AT")
def _configure_modem(self, modem_version: str):
if not modem_version.startswith("EG25"):
return
cmds = [
# clear initial EPS bearer APN (some carriers reject the default)
'AT+CGDCONT=0,"IP",""',
# SIM hot swap
'AT+QSIMDET=1,0',
'AT+QSIMSTAT=1',
# configure modem as data-centric
'AT+QNVW=5280,0,"0102000000000000"',
'AT+QNVFW="/nv/item_files/ims/IMS_enable",00',
'AT+QNVFW="/nv/item_files/modem/mmode/ue_usage_setting",01',
]
for c in cmds:
self._at(c)
def _do_initializing(self):
if not os.path.exists(AT_PORT):
return State.INITIALIZING
logging.info("port found, initializing")
self._ppp.kill()
self._ppp.cleanup_routes()
if not self._init_at_channel():
logging.warning("AT echo still on, retrying")
return State.INITIALIZING
identity = self._read_identity()
if not identity["iccid"] or not identity["imei"]:
logging.warning(f"identity read incomplete: {identity}, retrying")
return State.INITIALIZING
self._configure_modem(identity["modem_version"])
self.S.update(identity)
self._apn = self._read_param("GsmApn")
self._roaming_allowed = self._is_roaming_allowed()
# blank APN lets the carrier supply one via PCO
self._at(f'AT+CGDCONT={DIAL_CID},"IP","{self._apn}"')
logging.info(f"APN '{self._apn or '(network-provided)'}' written to CID {DIAL_CID}, roaming={'on' if self._roaming_allowed else 'off'}")
self._sim_change = False # clear since we just re-read identity with the new SIM
self._publish_state(**identity)
return State.SEARCHING
def _read_identity(self):
def first_line(cmd):
r = self._at(cmd)
return r[0].strip() if r else ""
imei = first_line("AT+CGSN")
if not (imei.isdigit() and 14 <= len(imei) <= 17): # 3GPP TS 23.003
imei = ""
iccid = (self._atv("AT+QCCID", "+QCCID:") or "").rstrip("F")
if not iccid.isdigit():
iccid = ""
imsi = first_line("AT+CIMI")
mcc_mnc = imsi[:6] if imsi.isdigit() and len(imsi) >= 6 else ""
modem_version = first_line("AT+GMR")
logging.info(f"imei={imei} iccid={iccid} mcc_mnc={mcc_mnc} ver={modem_version}")
return {"imei": imei, "iccid": iccid, "mcc_mnc": mcc_mnc, "modem_version": modem_version}
def _do_searching(self):
new_roaming = self._is_roaming_allowed()
if new_roaming != self._roaming_allowed:
logging.info(f"roaming changed: {self._roaming_allowed} -> {new_roaming}")
self._roaming_allowed = new_roaming
v = self._atv("AT+CREG?", "+CREG:")
if not v:
return self._searching_idle()
reg = self._parse_reg(v)
greg = self._parse_reg(self._atv("AT+CGREG?", "+CGREG:") or "")
logging.debug(f"creg={reg} cgreg={greg} roaming_allowed={self._roaming_allowed}")
if reg == "roaming" and not self._roaming_allowed:
self._publish_state(registration=reg)
return State.SEARCHING
if reg in ("home", "roaming") and greg in ("home", "roaming"):
self._publish_state(registration=reg)
return State.CONNECTING
if reg != self.S.get("registration"):
self._publish_state(registration=reg)
return self._searching_idle()
def _searching_idle(self):
if self._sim_change or not os.path.exists(AT_PORT):
logging.info(f"-> reconnecting (sim_change={self._sim_change} port={os.path.exists(AT_PORT)})")
return State.DISCONNECTING
return State.SEARCHING
def _do_connecting(self):
logging.info("starting pppd")
self._ppp.reset_fail_counter()
self._sim_change = False
self._ppp.start()
return State.CONNECTED
def _handle_pppd_exit(self):
if self._sim_change or not os.path.exists(AT_PORT):
return State.DISCONNECTING
give_up = self._ppp.record_fail()
if give_up:
logging.warning(f"PPP fail {self._ppp.fails}/{self._ppp.MAX_FAILS}, reconnecting")
return State.DISCONNECTING
logging.warning(f"PPP fail {self._ppp.fails}/{self._ppp.MAX_FAILS}, retrying")
self._ppp.reset_data_port()
if not os.path.exists(AT_PORT):
return State.DISCONNECTING
self._ppp.start()
return State.CONNECTED
def _params_changed(self) -> bool:
new_apn = self._read_param("GsmApn")
if new_apn != self._apn:
logging.info(f"GsmApn changed: '{self._apn}' -> '{new_apn}'")
return True
new_roaming = self._is_roaming_allowed()
if new_roaming != self._roaming_allowed:
logging.info(f"roaming changed: {self._roaming_allowed} -> {new_roaming}")
return True
return False
def _check_iccid(self, state):
if state in (State.INITIALIZING, State.DISCONNECTING) or not self.S["iccid"]:
return
iccid = (self._atv("AT+QCCID", "+QCCID:") or "").rstrip("F")
if iccid and iccid != self.S["iccid"]:
logging.warning(f"iccid changed: {self.S['iccid']} -> {iccid}")
self._sim_change = True
def _do_connected(self):
if self._ppp.has_exited():
return self._handle_pppd_exit()
if self._sim_change or not os.path.exists(AT_PORT) or self._params_changed():
return State.DISCONNECTING
self._poll()
return State.CONNECTED
def _do_disconnecting(self):
logging.warning("reconnecting")
self._publish_state(**INITIAL_STATE)
self._ppp.kill()
self._ppp.cleanup_routes()
self._ppp.reset_data_port()
self._sim_change = False
return State.INITIALIZING
def _poll_signal(self) -> dict:
v = self._atv("AT+CSQ", "+CSQ:")
if not v:
return {}
try:
rssi = int(v.split(",")[0])
if rssi == 99:
return {}
return {"signal_strength": rssi, "signal_quality": min(100, int(rssi / 31 * 100))}
except (ValueError, IndexError):
return {}
def _poll_operator(self) -> dict:
v = self._atv("AT+COPS?", "+COPS:")
if not v:
return {}
p = v.split(",")
out: dict = {}
try:
if len(p) >= 3:
out["operator"] = p[2].strip('"')
if len(p) >= 4:
out["network_type"] = NETWORK_TYPE.get(int(p[3]), "unknown")
except (ValueError, IndexError):
pass
return out
def _poll_band(self) -> dict:
v = self._atv("AT+QNWINFO", "+QNWINFO:")
if not v:
return {}
info = v.replace('"', '').split(",")
try:
if len(info) >= 4:
return {"band": info[2], "channel": int(info[3])}
except ValueError:
pass
return {}
def _poll_extra(self) -> dict:
v = self._atv('AT+QENG="servingcell"', "+QENG:")
return {"extra": v.replace('"', '')} if v else {}
def _poll_temps(self) -> dict:
v = self._atv("AT+QTEMP", "+QTEMP:")
if not v:
return {}
try:
return {"temperatures": [t for t in (int(x) for x in v.split(",") if x.strip()) if t != 255]}
except (ValueError, IndexError):
return {}
def _poll_iface(self) -> dict:
try:
r = subprocess.run(["ip", "-4", "addr", "show", "ppp0"], capture_output=True, text=True, timeout=2)
ip, peer = "", ""
for line in r.stdout.splitlines():
# `inet 10.x.x.x peer 10.64.64.64/32 ...`
parts = line.strip().split()
if "inet" in parts:
i = parts.index("inet")
ip = parts[i + 1].split("/")[0]
if "peer" in parts:
peer = parts[parts.index("peer") + 1].split("/")[0]
break
if ip:
if self._ppp.maybe_install_routes(ip, peer):
self._ppp.maybe_install_dns(self._read_cellular_dns())
return {"ip_address": ip, "connected": True}
if self.S["connected"]:
return {"connected": False, "ip_address": ""}
except Exception:
pass
return {}
def _read_cellular_dns(self) -> list[str]:
v = self._atv(f"AT+CGCONTRDP={DIAL_CID}", "+CGCONTRDP:")
if not v:
return []
# +CGCONTRDP: <cid>,<bearer_id>,<apn>,<local_addr>,<gw_addr>,<dns_prim>,<dns_sec>,...
fields = [f.strip().strip('"') for f in v.split(",")]
dns_servers = []
for d in fields[5:7]:
try:
dns_servers.append(str(IPv4Address(d)))
except (AddressValueError, ValueError):
pass
if not dns_servers:
logging.warning(f"no cellular DNS servers reported by modem: {v!r}")
return dns_servers
def _poll_byte_counters(self) -> dict:
try:
with open("/sys/class/net/ppp0/statistics/tx_bytes") as f:
tx = int(f.read().strip())
with open("/sys/class/net/ppp0/statistics/rx_bytes") as f:
rx = int(f.read().strip())
except Exception:
return {}
return {"tx_bytes": tx, "rx_bytes": rx}
def _poll(self):
s: dict = {}
for fn in (self._poll_signal, self._poll_operator, self._poll_band,
self._poll_extra, self._poll_temps, self._poll_iface,
self._poll_byte_counters):
s.update(fn())
if s:
self._publish_state(**s)
def run(self):
logging.info("starting")
self._publish_state(state=State.INITIALIZING.value)
if self._has_modem_manager():
subprocess.run(["sudo", "systemctl", "mask", "--runtime", "ModemManager"], capture_output=True)
subprocess.run(["sudo", "systemctl", "stop", "ModemManager"], capture_output=True)
self._ppp.kill()
state = State.INITIALIZING
handlers = {
State.INITIALIZING: self._do_initializing,
State.SEARCHING: self._do_searching,
State.CONNECTING: self._do_connecting,
State.CONNECTED: self._do_connected,
State.DISCONNECTING: self._do_disconnecting,
}
while self.running:
try:
self._check_iccid(state)
prev = state
state = handlers[state]()
if state != prev:
self._publish_state(state=state.value)
logging.info(f"{prev.value} -> {state.value}")
except Exception:
logging.exception(f"error in {state.value}")
state = State.DISCONNECTING
time.sleep(STATE_WAIT)
def stop(self):
self.running = False
self._ppp.kill()
self._ppp.cleanup_routes()
try:
os.remove(STATE_PATH)
except FileNotFoundError:
pass
if self._has_modem_manager():
subprocess.run(["sudo", "systemctl", "unmask", "--runtime", "ModemManager"], capture_output=True)
subprocess.run(["sudo", "systemctl", "start", "ModemManager"], capture_output=True)
def main():
m = Modem()
def _sig(*_):
m.running = False
signal.signal(signal.SIGINT, _sig)
signal.signal(signal.SIGTERM, _sig)
m.run()
m.stop()
if __name__ == "__main__":
main()
+1
View File
@@ -0,0 +1 @@
@@ -4,8 +4,8 @@ import random
import subprocess
from panda import Panda
from openpilot.system.hardware import TICI, HARDWARE
from openpilot.system.hardware.tici.amplifier import Amplifier
from openpilot.common.hardware import TICI, HARDWARE
from openpilot.common.hardware.tici.amplifier import Amplifier
class TestAmplifier:
BIN
View File
Binary file not shown.
-1
View File
@@ -1 +0,0 @@
#define DEFAULT_MODEL "OP Model 7 (Default)"
+8 -1
View File
@@ -1,8 +1,13 @@
import re
import sys
import pytest
import inspect
def _to_safe_name(s):
return re.sub(r"[^a-zA-Z0-9_]+", "_", str(s)).strip("_")
class parameterized:
@staticmethod
def expand(cases):
@@ -34,7 +39,9 @@ def parameterized_class(attrs, input_list=None):
def decorator(cls):
globs = sys._getframe(1).f_globals
for i, params in enumerate(params_list):
name = f"{cls.__name__}_{i}"
# append sanitized string param values so pytest -k can filter by them
suffix = "_".join(filter(None, (_to_safe_name(v) for v in params.values() if isinstance(v, str))))
name = f"{cls.__name__}_{i}" + (f"_{suffix}" if suffix else "")
new_cls = type(name, (cls,), dict(params))
new_cls.__module__ = cls.__module__
new_cls.__test__ = True # override inherited False so pytest collects this subclass
+1 -1
View File
@@ -12,7 +12,7 @@
#include "common/queue.h"
#include "common/swaglog.h"
#include "common/util.h"
#include "system/hardware/hw.h"
#include "common/hardware/hw.h"
namespace {

Some files were not shown because too many files have changed in this diff Show More