mirror of
https://github.com/sunnypilot/sunnypilot.git
synced 2026-07-17 17:12:05 +08:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ac6b182524 |
@@ -1,18 +0,0 @@
|
||||
**/.git
|
||||
.DS_Store
|
||||
*.dylib
|
||||
*.DSYM
|
||||
*.d
|
||||
*.pyc
|
||||
*.pyo
|
||||
.*.swp
|
||||
.*.swo
|
||||
.*.un~
|
||||
*.tmp
|
||||
*.o
|
||||
*.o-*
|
||||
*.os
|
||||
*.os-*
|
||||
|
||||
venv/
|
||||
.venv/
|
||||
@@ -1,23 +0,0 @@
|
||||
* text=auto
|
||||
|
||||
# to move existing files into LFS:
|
||||
# git add --renormalize .
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.svg filter=lfs diff=lfs merge=lfs -text
|
||||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
*.gif filter=lfs diff=lfs merge=lfs -text
|
||||
*.ttf filter=lfs diff=lfs merge=lfs -text
|
||||
*.otf filter=lfs diff=lfs merge=lfs -text
|
||||
*.wav filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
selfdrive/car/tests/test_models_segs.txt filter=lfs diff=lfs merge=lfs -text
|
||||
system/hardware/tici/updater_weston filter=lfs diff=lfs merge=lfs -text
|
||||
system/hardware/tici/updater_magic filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/**/*.a filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/**/*.so filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/**/*.so.* filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/**/*.dylib filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/acados/*/t_renderer filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/qt5/larch64/bin/lrelease filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/qt5/larch64/bin/lupdate filter=lfs diff=lfs merge=lfs -text
|
||||
third_party/catch2/include/catch2/catch.hpp filter=lfs diff=lfs merge=lfs -text
|
||||
@@ -1,11 +0,0 @@
|
||||
* @sunnypilot/dev-internal
|
||||
/.github/ @devtekve @sunnyhaibin
|
||||
/release/ci/ @devtekve @sunnyhaibin
|
||||
/tinygrad_repo @devtekve @Discountchubbs
|
||||
/tinygrad/ @devtekve @Discountchubbs
|
||||
/selfdrive/controls/lib/longitudinal_planner.py @devtekve @Discountchubbs
|
||||
/selfdrive/controls/lib/longitudinal_mpc_lib/long_mpc.py @devtekve @Discountchubbs
|
||||
/selfdrive/modeld/ @devtekve @Discountchubbs
|
||||
/sunnypilot/model* @devtekve @Discountchubbs
|
||||
/sunnypilot/sunnylink/ @devtekve
|
||||
/system/athena/ @devtekve
|
||||
@@ -1,47 +0,0 @@
|
||||
name: Bug report
|
||||
description: For issues with running openpilot on your comma device
|
||||
labels: ["bug"]
|
||||
body:
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Before creating a **bug report**, please check the following:
|
||||
* If the issue likely only affects your car model or make, go back and open a **car bug report** instead.
|
||||
* If the issue is related to the driving or driver monitoring models, you should open a [discussion](https://github.com/commaai/openpilot/discussions/categories/model-feedback) instead.
|
||||
* Ensure you're running the latest openpilot release.
|
||||
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
|
||||
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
|
||||
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
|
||||
|
||||
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: Also include a description of how to reproduce the bug
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: route
|
||||
attributes:
|
||||
label: Provide a route where the issue occurs
|
||||
description: Ensure the route is fully uploaded at https://useradmin.comma.ai. We cannot look into issues without routes, or at least a Dongle ID.
|
||||
placeholder: 77611a1fac303767|2020-05-11--16-37-07
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: openpilot version
|
||||
description: If you're not on release, provide the commit hash
|
||||
placeholder: 0.8.10
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional info
|
||||
|
||||
@@ -1,14 +0,0 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Car bug report
|
||||
url: https://github.com/commaai/opendbc/issues/new
|
||||
about: For issues with a particular car make or model
|
||||
- name: Join the Discord
|
||||
url: https://discord.comma.ai
|
||||
about: The community Discord is for both openpilot development and experience discussion
|
||||
- name: Report driving behavior feedback
|
||||
url: https://discord.com/channels/469524606043160576/1254834193066623017
|
||||
about: Feedback for the driving and driver monitoring models goes in the #driving-feedback in Discord
|
||||
- name: Community Wiki
|
||||
url: https://github.com/commaai/openpilot/wiki
|
||||
about: Check out our community wiki
|
||||
@@ -1,8 +0,0 @@
|
||||
---
|
||||
name: Enhancement
|
||||
about: For openpilot enhancement suggestions
|
||||
title: ''
|
||||
labels: 'enhancement'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
name: PC bug report
|
||||
description: For issues with running openpilot on PC
|
||||
labels: ["PC"]
|
||||
body:
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Before creating a **bug report**, please check the following:
|
||||
* Ensure you're running the latest openpilot release.
|
||||
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
|
||||
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
|
||||
|
||||
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: Also include a description of how to reproduce the bug
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: os-version
|
||||
attributes:
|
||||
label: OS Version
|
||||
placeholder: Ubuntu 24.04
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: openpilot version or commit
|
||||
placeholder: bd36f2ec8d3559909678eff2690c10a520938367
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional info
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
ci:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: "{.github/**,**/test_*,**/test/**,Jenkinsfile}"
|
||||
|
||||
chore:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: "{.github/**}"
|
||||
|
||||
car:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: '{selfdrive/car/**,opendbc_repo}'
|
||||
|
||||
simulation:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: 'tools/sim/**'
|
||||
|
||||
ui:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: '{selfdrive/assets/**,selfdrive/ui/**,system/ui/**}'
|
||||
|
||||
tools:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: 'tools/**'
|
||||
|
||||
multilanguage:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: 'selfdrive/ui/translations/**'
|
||||
|
||||
autonomy:
|
||||
- changed-files:
|
||||
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit,sunnypilot/modeld*/models/**}"
|
||||
@@ -1,68 +0,0 @@
|
||||
<!-- Please copy and paste the relevant template -->
|
||||
|
||||
<!--- ***** Template: Fingerprint *****
|
||||
|
||||
**Car**
|
||||
Which car (make, model, year) this fingerprint is for
|
||||
|
||||
**Route**
|
||||
A route with the fingerprint
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Car Bugfix *****
|
||||
|
||||
**Description**
|
||||
|
||||
A description of the bug and the fix. Also link the issue if it exists.
|
||||
|
||||
**Verification**
|
||||
|
||||
Explain how you tested this bug fix.
|
||||
|
||||
**Route**
|
||||
|
||||
Route: [a route with the bug fix]
|
||||
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Bugfix *****
|
||||
|
||||
**Description**
|
||||
|
||||
A description of the bug and the fix. Also link the issue if it exists.
|
||||
|
||||
**Verification**
|
||||
|
||||
Explain how you tested this bug fix.
|
||||
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Car Port *****
|
||||
|
||||
**Checklist**
|
||||
|
||||
- [ ] added entry to CAR in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
|
||||
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
|
||||
- [ ] route with openpilot:
|
||||
- [ ] route with stock system:
|
||||
- [ ] car harness used (if comma doesn't sell it, put N/A):
|
||||
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Refactor *****
|
||||
|
||||
**Description**
|
||||
|
||||
A description of the refactor, including the goals it accomplishes.
|
||||
|
||||
**Verification**
|
||||
|
||||
Explain how you tested the refactor for regressions.
|
||||
|
||||
|
||||
-->
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
exclude-labels:
|
||||
- 'no-changelog'
|
||||
categories:
|
||||
- title: '🚀 Features'
|
||||
labels:
|
||||
- 'feature'
|
||||
- 'enhancement'
|
||||
- title: '🐛 Bug Fixes'
|
||||
collapse-after: 5
|
||||
labels:
|
||||
- 'fix'
|
||||
- 'bugfix'
|
||||
- 'bug'
|
||||
- title: '🧰 Maintenance'
|
||||
collapse-after: 5
|
||||
label: 'chore'
|
||||
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
|
||||
change-title-escapes: '\<*_&'
|
||||
replacers:
|
||||
- search: '/[Ss][Uu][Nn][Nn][Yy][Pp][Ii][Ll][Oo][Tt]/g'
|
||||
replace: 'sunnypilot'
|
||||
- search: '/\b[Ss][Pp]\b/g'
|
||||
replace: 'SP'
|
||||
version-resolver:
|
||||
major:
|
||||
labels:
|
||||
- 'major'
|
||||
minor:
|
||||
labels:
|
||||
- 'minor'
|
||||
patch:
|
||||
labels:
|
||||
- 'patch'
|
||||
default: patch
|
||||
name-template: 'v$RESOLVED_VERSION 🚀'
|
||||
tag-template: 'v$RESOLVED_VERSION'
|
||||
version-template: "0.$MAJOR.$MINOR.$PATCH" # The day OP becomes v1, we need to bump this
|
||||
tag-prefix: "v0." # The day OP becomes v1, we need to bump this
|
||||
prerelease-identifier: "staging"
|
||||
template: |
|
||||
## Changes
|
||||
|
||||
$CHANGES
|
||||
@@ -1,91 +0,0 @@
|
||||
name: "PR review"
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [ opened, reopened, synchronize, edited ]
|
||||
|
||||
jobs:
|
||||
labeler:
|
||||
name: review
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
issues: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
# Label PRs
|
||||
- uses: actions/labeler@v6
|
||||
with:
|
||||
dot: true
|
||||
configuration-path: .github/labeler.yaml
|
||||
|
||||
# Check PR target branch
|
||||
- name: check branch
|
||||
uses: Vankka/pr-target-branch-action@def32ec9d93514138d6ac0132ee62e120a72aed5
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
with:
|
||||
target: /^(?!master$).*/
|
||||
exclude: /sunnypilot:.*/
|
||||
change-to: ${{ github.base_ref }}
|
||||
already-exists-action: close_this
|
||||
already-exists-comment: "Your PR should be made against the `master` branch"
|
||||
|
||||
update-pr-labels:
|
||||
name: Update fork's PR Labels
|
||||
runs-on: ubuntu-latest
|
||||
if: (github.event.pull_request.head.repo.fork && (contains(github.event_name, 'pull_request') && github.event.action == 'synchronize'))
|
||||
env:
|
||||
PR_LABEL: 'dev'
|
||||
TRUST_FORK_PR_LABEL: 'trust-fork-pr'
|
||||
steps:
|
||||
- name: Check if PR has dev label
|
||||
id: check-labels
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
script: |
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
const { data: labels } = await github.rest.issues.listLabelsOnIssue({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber
|
||||
});
|
||||
|
||||
const hasDevC3Label = labels.some(label => label.name === process.env.PR_LABEL);
|
||||
const hasTrustLabel = labels.some(label => label.name === process.env.TRUST_FORK_PR_LABEL);
|
||||
|
||||
console.log(`PR #${prNumber} has ${process.env.PR_LABEL} label: ${hasDevC3Label}`);
|
||||
console.log(`PR #${prNumber} has ${process.env.TRUST_FORK_PR_LABEL} label: ${hasTrustLabel}`);
|
||||
|
||||
core.setOutput('has-dev', hasDevC3Label ? 'true' : 'false');
|
||||
core.setOutput('has-trust', hasTrustLabel ? 'true' : 'false');
|
||||
|
||||
- name: Remove trust-fork-pr label if present
|
||||
if: steps.check-labels.outputs.has-dev == 'true' && steps.check-labels.outputs.has-trust == 'true'
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
script: |
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
|
||||
await github.rest.issues.removeLabel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber,
|
||||
name: process.env.TRUST_FORK_PR_LABEL
|
||||
});
|
||||
|
||||
console.log(`Removed '${process.env.TRUST_FORK_PR_LABEL}' label from PR #${prNumber} as it received new commits`);
|
||||
|
||||
// Add a comment to the PR
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber,
|
||||
body: `The \`${process.env.TRUST_FORK_PR_LABEL}\` label has been automatically removed because new commits were pushed to this PR. This PR will need to be re-reviewed before the label can be applied again.`
|
||||
});
|
||||
@@ -1,35 +0,0 @@
|
||||
name: badges
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 * * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
PYTHONPATH: ${{ github.workspace }}
|
||||
|
||||
jobs:
|
||||
badges:
|
||||
name: create badges
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Push badges
|
||||
run: |
|
||||
python3 selfdrive/ui/translations/create_badges.py
|
||||
|
||||
rm .gitattributes
|
||||
|
||||
git checkout --orphan badges
|
||||
git rm -rf --cached .
|
||||
git config user.email "badge-researcher@sunnypilot.ai"
|
||||
git config user.name "Badge Researcher"
|
||||
|
||||
git add translation_badge.svg
|
||||
git commit -m "Add/Update badges"
|
||||
git push -f origin HEAD
|
||||
@@ -1,304 +0,0 @@
|
||||
name: Build and push all tinygrad models
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
set_min_version:
|
||||
description: 'Minimum selector version required for the models (see helpers.py or readme.md)'
|
||||
required: true
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
json_version: ${{ steps.get-json.outputs.json_version }}
|
||||
recompiled_dir: ${{ steps.create-recompiled-dir.outputs.recompiled_dir }}
|
||||
json_file: ${{ steps.get-json.outputs.json_file }}
|
||||
model_matrix: ${{ steps.set-matrix.outputs.model_matrix }}
|
||||
tinygrad_ref: ${{ steps.get-tinygrad-ref.outputs.tinygrad_ref }}
|
||||
steps:
|
||||
- name: Checkout sunnypilot repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: sunnypilot/sunnypilot
|
||||
path: sunnypilot
|
||||
submodules: recursive
|
||||
|
||||
- name: Get tinygrad_repo ref
|
||||
id: get-tinygrad-ref
|
||||
run: |
|
||||
cd sunnypilot
|
||||
export PYTHONPATH=$(pwd)
|
||||
ref=$(python3 sunnypilot/models/tinygrad_ref.py)
|
||||
echo "tinygrad_ref=$ref" >> $GITHUB_OUTPUT
|
||||
echo "tinygrad_ref is $ref"
|
||||
|
||||
- name: Checkout docs repo (sunnypilot-models, gh-pages)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: sunnypilot/sunnypilot-models
|
||||
ref: gh-pages
|
||||
path: docs
|
||||
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
|
||||
|
||||
- name: Get next JSON version to use (from GitHub docs repo)
|
||||
id: get-json
|
||||
run: |
|
||||
cd docs/docs
|
||||
latest=$(ls driving_models_v*.json | sed -E 's/.*_v([0-9]+)\.json/\1/' | sort -n | tail -1)
|
||||
next=$((latest+1))
|
||||
json_file="driving_models_v${next}.json"
|
||||
cp "driving_models_v${latest}.json" "$json_file"
|
||||
echo "json_file=docs/docs/$json_file" >> $GITHUB_OUTPUT
|
||||
echo "json_version=$((next+0))" >> $GITHUB_OUTPUT
|
||||
echo "SRC_JSON_FILE=docs/docs/driving_models_v${latest}.json" >> $GITHUB_ENV
|
||||
|
||||
- name: Extract tinygrad models
|
||||
id: set-matrix
|
||||
working-directory: docs/docs
|
||||
run: |
|
||||
jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' "$(basename "${SRC_JSON_FILE}")" > matrix.json
|
||||
echo "model_matrix=$(cat matrix.json)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Set up SSH
|
||||
uses: webfactory/ssh-agent@v0.9.0
|
||||
with:
|
||||
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
|
||||
- run: |
|
||||
mkdir -p ~/.ssh
|
||||
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Clone GitLab docs repo and create new recompiled dir
|
||||
id: create-recompiled-dir
|
||||
env:
|
||||
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
|
||||
run: |
|
||||
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
|
||||
cd gitlab_docs
|
||||
git checkout main
|
||||
git sparse-checkout set --no-cone models/
|
||||
cd models
|
||||
latest_dir=$(ls -d recompiled* 2>/dev/null | sed -E 's/recompiled([0-9]+)/\1/' | sort -n | tail -1)
|
||||
if [[ -z "$latest_dir" ]]; then
|
||||
next_dir=1
|
||||
else
|
||||
next_dir=$((latest_dir+1))
|
||||
fi
|
||||
recompiled_dir="${next_dir}"
|
||||
mkdir -p "recompiled${recompiled_dir}"
|
||||
touch "recompiled${recompiled_dir}/.gitkeep"
|
||||
cd ../..
|
||||
echo "recompiled_dir=$recompiled_dir" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Push empty recompiled dir to GitLab
|
||||
run: |
|
||||
cd gitlab_docs
|
||||
git add models/recompiled${{ steps.create-recompiled-dir.outputs.recompiled_dir }}
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git commit -m "Add recompiled${{ steps.create-recompiled-dir.outputs.recompiled_dir }} for build-all" || echo "No changes to commit"
|
||||
git push origin main
|
||||
|
||||
- name: Push new JSON to GitHub docs repo
|
||||
run: |
|
||||
cd docs
|
||||
git pull origin gh-pages
|
||||
git add docs/"$(basename ${{ steps.get-json.outputs.json_file }})"
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git commit -m "Add new ${{ steps.get-json.outputs.json_file }} for build-all" || echo "No changes to commit"
|
||||
git push origin gh-pages
|
||||
|
||||
get_and_build:
|
||||
needs: [setup]
|
||||
strategy:
|
||||
matrix:
|
||||
model: ${{ fromJson(needs.setup.outputs.model_matrix) }}
|
||||
fail-fast: false
|
||||
uses: ./.github/workflows/build-single-tinygrad-model.yaml
|
||||
with:
|
||||
upstream_branch: ${{ matrix.model.ref }}
|
||||
custom_name: ${{ matrix.model.display_name }}
|
||||
recompiled_dir: ${{ needs.setup.outputs.recompiled_dir }}
|
||||
json_version: ${{ needs.setup.outputs.json_version }}
|
||||
secrets: inherit
|
||||
|
||||
retry_failed_models:
|
||||
needs: [setup, get_and_build]
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ needs.setup.result != 'failure' && !cancelled() }}
|
||||
outputs:
|
||||
retry_matrix: ${{ steps.set-retry-matrix.outputs.retry_matrix }}
|
||||
steps:
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
pattern: model-*
|
||||
path: output
|
||||
|
||||
- id: set-retry-matrix
|
||||
run: |
|
||||
echo '${{ needs.setup.outputs.model_matrix }}' > matrix.json
|
||||
built=(); while IFS= read -r line; do built+=("$line"); done < <(
|
||||
find output -maxdepth 1 -name 'model-*' -printf "%f\n" | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
|
||||
)
|
||||
jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
|
||||
'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' matrix.json > retry_matrix.json
|
||||
echo "retry_matrix=$(cat retry_matrix.json)" >> $GITHUB_OUTPUT
|
||||
|
||||
retry_get_and_build:
|
||||
needs: [setup, get_and_build, retry_failed_models]
|
||||
if: ${{ needs.get_and_build.result == 'failure' || (needs.retry_failed_models.outputs.retry_matrix != '[]' && needs.retry_failed_models.outputs.retry_matrix != '') }}
|
||||
strategy:
|
||||
matrix:
|
||||
model: ${{ fromJson(needs.retry_failed_models.outputs.retry_matrix) }}
|
||||
fail-fast: false
|
||||
uses: ./.github/workflows/build-single-tinygrad-model.yaml
|
||||
with:
|
||||
upstream_branch: ${{ matrix.model.ref }}
|
||||
custom_name: ${{ matrix.model.display_name }}
|
||||
recompiled_dir: ${{ needs.setup.outputs.recompiled_dir }}
|
||||
json_version: ${{ needs.setup.outputs.json_version }}
|
||||
artifact_suffix: -retry
|
||||
secrets: inherit
|
||||
|
||||
publish_models:
|
||||
name: Publish models sequentially
|
||||
needs: [setup, get_and_build, retry_failed_models, retry_get_and_build]
|
||||
if: ${{ !cancelled() && (needs.get_and_build.result != 'failure' || needs.retry_get_and_build.result == 'success' || (needs.retry_failed_models.outputs.retry_matrix != '[]' && needs.retry_failed_models.outputs.retry_matrix != '')) }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
model: ${{ fromJson(needs.setup.outputs.model_matrix) }}
|
||||
env:
|
||||
RECOMPILED_DIR: recompiled${{ needs.setup.outputs.recompiled_dir }}
|
||||
JSON_FILE: ${{ needs.setup.outputs.json_file }}
|
||||
ARTIFACT_NAME_INPUT: ${{ matrix.model.display_name }}
|
||||
steps:
|
||||
- name: Set up SSH
|
||||
uses: webfactory/ssh-agent@v0.9.0
|
||||
with:
|
||||
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
|
||||
|
||||
- name: Add GitLab.com SSH key to known_hosts
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Clone GitLab docs repo
|
||||
env:
|
||||
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
|
||||
run: |
|
||||
echo "Cloning GitLab"
|
||||
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
|
||||
cd gitlab_docs
|
||||
echo "checkout models/${RECOMPILED_DIR}"
|
||||
git sparse-checkout set --no-cone models/${RECOMPILED_DIR}
|
||||
git checkout main
|
||||
cd ..
|
||||
|
||||
- name: Checkout docs repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: sunnypilot/sunnypilot-models
|
||||
ref: gh-pages
|
||||
path: docs
|
||||
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
|
||||
|
||||
- name: Validate recompiled dir and JSON version
|
||||
run: |
|
||||
if [ ! -d "gitlab_docs/models/$RECOMPILED_DIR" ]; then
|
||||
echo "Recompiled dir $RECOMPILED_DIR does not exist in GitLab repo"
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$JSON_FILE" ]; then
|
||||
echo "JSON file $JSON_FILE does not exist!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Download artifact name file
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: artifact-name-${{ env.ARTIFACT_NAME_INPUT }}
|
||||
path: artifact_name
|
||||
|
||||
- name: Read artifact name
|
||||
id: read-artifact-name
|
||||
run: |
|
||||
ARTIFACT_NAME=$(cat artifact_name/artifact_name.txt)
|
||||
echo "artifact_name=$ARTIFACT_NAME" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Download model artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: ${{ steps.read-artifact-name.outputs.artifact_name }}
|
||||
path: output
|
||||
|
||||
- name: Remove onnx files bc not needed for recompiled dir since they already exist from single build
|
||||
run: |
|
||||
find output -type f -name '*.onnx' -delete
|
||||
find output -type f -name 'big_*.pkl' -delete
|
||||
find output -type f -name 'dmonitoring_model_tinygrad.pkl' -delete
|
||||
|
||||
- name: Copy model artifacts to gitlab
|
||||
env:
|
||||
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
|
||||
run: |
|
||||
ARTIFACT_DIR="gitlab_docs/models/${RECOMPILED_DIR}/${ARTIFACT_NAME}"
|
||||
mkdir -p "$ARTIFACT_DIR"
|
||||
for path in output/*; do
|
||||
if [ "$(basename "$path")" = "artifact_name.txt" ]; then
|
||||
continue
|
||||
fi
|
||||
name="$(basename "$path")"
|
||||
if [ -d "$path" ]; then
|
||||
mkdir -p "$ARTIFACT_DIR/$name"
|
||||
cp -r "$path"/* "$ARTIFACT_DIR/$name/"
|
||||
echo "Copied dir $name -> $ARTIFACT_DIR/$name"
|
||||
else
|
||||
cp "$path" "$ARTIFACT_DIR/"
|
||||
echo "Copied file $name -> $ARTIFACT_DIR/"
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Push recompiled dir to GitLab
|
||||
env:
|
||||
GITLAB_SSH_PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
|
||||
run: |
|
||||
cd gitlab_docs
|
||||
git checkout main
|
||||
git pull origin main
|
||||
for d in models/"$RECOMPILED_DIR"/*/; do
|
||||
git sparse-checkout add "$d"
|
||||
done
|
||||
git add models/"$RECOMPILED_DIR"
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git commit -m "Update $RECOMPILED_DIR with model from build-all-tinygrad-models" || echo "No changes to commit"
|
||||
git push origin main
|
||||
- run: |
|
||||
cd docs
|
||||
git pull origin gh-pages
|
||||
|
||||
- name: update json
|
||||
run: |
|
||||
ARGS=""
|
||||
[ -n "${{ inputs.set_min_version }}" ] && ARGS="$ARGS --set-min-version \"${{ inputs.set_min_version }}\""
|
||||
ARGS="$ARGS --sort-by-date"
|
||||
ARGS="$ARGS --tinygrad-ref \"${{ needs.setup.outputs.tinygrad_ref }}\""
|
||||
eval python3 docs/json_parser.py \
|
||||
--json-path "$JSON_FILE" \
|
||||
--recompiled-dir "gitlab_docs/models/$RECOMPILED_DIR" \
|
||||
$ARGS
|
||||
|
||||
- name: Push updated json to GitHub
|
||||
run: |
|
||||
cd docs
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git checkout gh-pages
|
||||
git add docs/"$(basename $JSON_FILE)"
|
||||
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || echo "No changes to commit"
|
||||
git push origin gh-pages
|
||||
@@ -1,228 +0,0 @@
|
||||
name: Build Single Tinygrad Model and Push
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
upstream_branch:
|
||||
description: 'Upstream commit to build from'
|
||||
required: true
|
||||
type: string
|
||||
custom_name:
|
||||
description: 'Custom name for the model (no date, only name)'
|
||||
required: false
|
||||
type: string
|
||||
recompiled_dir:
|
||||
description: 'Existing recompiled directory number (e.g. 3 for recompiled3)'
|
||||
required: true
|
||||
type: string
|
||||
json_version:
|
||||
description: 'driving_models version number to update (e.g. 5 for driving_models_v5.json)'
|
||||
required: true
|
||||
type: string
|
||||
artifact_suffix:
|
||||
description: 'Suffix for artifact name'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
bypass_push:
|
||||
description: 'Bypass pushing to GitLab for build-all'
|
||||
required: false
|
||||
default: true
|
||||
type: boolean
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
upstream_branch:
|
||||
description: 'Upstream commit to build from'
|
||||
required: true
|
||||
type: string
|
||||
custom_name:
|
||||
description: 'Custom name for the model (no date, only name)'
|
||||
required: false
|
||||
type: string
|
||||
recompiled_dir:
|
||||
description: 'Existing recompiled directory number (e.g. 3 for recompiled3)'
|
||||
required: true
|
||||
type: string
|
||||
json_version:
|
||||
description: 'driving_models version number to update (e.g. 5 for driving_models_v5.json)'
|
||||
required: true
|
||||
type: string
|
||||
model_folder:
|
||||
description: 'Model folder'
|
||||
type: choice
|
||||
default: 'None'
|
||||
options:
|
||||
- None
|
||||
- Simple Plan Models
|
||||
- Space Lab Models
|
||||
- TR Models
|
||||
- DTR Models
|
||||
- Custom Merge Models
|
||||
- FOF series models
|
||||
- Other
|
||||
custom_model_folder:
|
||||
description: 'Custom model folder name (if "Other" selected)'
|
||||
required: false
|
||||
type: string
|
||||
generation:
|
||||
description: 'Model generation'
|
||||
required: false
|
||||
type: string
|
||||
version:
|
||||
description: 'Minimum selector version'
|
||||
required: false
|
||||
type: string
|
||||
env:
|
||||
RECOMPILED_DIR: recompiled${{ inputs.recompiled_dir }}
|
||||
JSON_FILE: docs/docs/driving_models_v${{ inputs.json_version }}.json
|
||||
|
||||
jobs:
|
||||
build_model:
|
||||
uses: ./.github/workflows/sunnypilot-build-model.yaml
|
||||
with:
|
||||
upstream_branch: ${{ inputs.upstream_branch }}
|
||||
custom_name: ${{ inputs.custom_name || inputs.upstream_branch }}
|
||||
is_20hz: true
|
||||
artifact_suffix: ${{ inputs.artifact_suffix }}
|
||||
secrets: inherit
|
||||
|
||||
publish_model:
|
||||
if: ${{ !inputs.bypass_push && !cancelled() }}
|
||||
concurrency:
|
||||
group: gitlab-push-${{ inputs.recompiled_dir }}
|
||||
cancel-in-progress: false
|
||||
needs: build_model
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Set up SSH
|
||||
uses: webfactory/ssh-agent@v0.9.0
|
||||
with:
|
||||
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
|
||||
|
||||
- name: Add GitLab.com SSH key to known_hosts
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Clone GitLab docs repo
|
||||
env:
|
||||
GIT_SSH_COMMAND: 'ssh -o UserKnownHostsFile=~/.ssh/known_hosts'
|
||||
run: |
|
||||
echo "Cloning GitLab"
|
||||
git clone --depth 1 --filter=tree:0 --sparse git@gitlab.com:sunnypilot/public/${{ vars.MODELS_GITLAB }} gitlab_docs
|
||||
cd gitlab_docs
|
||||
echo "checkout models/${RECOMPILED_DIR}"
|
||||
git sparse-checkout set --no-cone models/${RECOMPILED_DIR}
|
||||
git checkout main
|
||||
cd ..
|
||||
|
||||
- name: Checkout docs repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: sunnypilot/sunnypilot-models
|
||||
ref: gh-pages
|
||||
path: docs
|
||||
ssh-key: ${{ secrets.CI_SUNNYPILOT_DOCS_PRIVATE_KEY }}
|
||||
|
||||
- name: Validate recompiled dir and JSON version
|
||||
run: |
|
||||
if [ ! -d "gitlab_docs/models/$RECOMPILED_DIR" ]; then
|
||||
echo "Recompiled dir $RECOMPILED_DIR does not exist in GitLab repo"
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$JSON_FILE" ]; then
|
||||
echo "JSON file $JSON_FILE does not exist!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Download artifact name file
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: artifact-name-${{ inputs.custom_name || inputs.upstream_branch }}
|
||||
path: artifact_name
|
||||
|
||||
- name: Read artifact name
|
||||
id: read-artifact-name
|
||||
run: |
|
||||
ARTIFACT_NAME=$(cat artifact_name/artifact_name.txt)
|
||||
echo "artifact_name=$ARTIFACT_NAME" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Download and extract model artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: ${{ steps.read-artifact-name.outputs.artifact_name }}
|
||||
path: output
|
||||
|
||||
- name: Remove unwanted files
|
||||
run: |
|
||||
find output -type f -name 'dmonitoring_model_tinygrad.pkl' -delete
|
||||
find output -type f -name 'dmonitoring_model.onnx' -delete
|
||||
|
||||
- name: Copy model artifact(s) to GitLab recompiled dir
|
||||
env:
|
||||
ARTIFACT_NAME: ${{ steps.read-artifact-name.outputs.artifact_name }}
|
||||
run: |
|
||||
ARTIFACT_DIR="gitlab_docs/models/${RECOMPILED_DIR}/${ARTIFACT_NAME}"
|
||||
mkdir -p "$ARTIFACT_DIR"
|
||||
for path in output/*; do
|
||||
if [ "$(basename "$path")" = "artifact_name.txt" ]; then
|
||||
continue
|
||||
fi
|
||||
name="$(basename "$path")"
|
||||
if [ -d "$path" ]; then
|
||||
mkdir -p "$ARTIFACT_DIR/$name"
|
||||
cp -r "$path"/* "$ARTIFACT_DIR/$name/"
|
||||
echo "Copied dir $name -> $ARTIFACT_DIR/$name"
|
||||
else
|
||||
cp "$path" "$ARTIFACT_DIR/"
|
||||
echo "Copied file $name -> $ARTIFACT_DIR/"
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Push recompiled dir to GitLab
|
||||
env:
|
||||
GITLAB_SSH_PRIVATE_KEY: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}
|
||||
run: |
|
||||
cd gitlab_docs
|
||||
git checkout main
|
||||
git pull origin main
|
||||
for d in models/"$RECOMPILED_DIR"/*/; do
|
||||
git sparse-checkout add "$d"
|
||||
done
|
||||
git add models/"$RECOMPILED_DIR"
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git commit -m "Create/Update $RECOMPILED_DIR with new/updated model from build-single-tinygrad-model" || echo "No changes to commit"
|
||||
git push origin main
|
||||
|
||||
- run: |
|
||||
cd docs
|
||||
git pull origin gh-pages
|
||||
|
||||
- name: Run json_parser.py to update JSON
|
||||
run: |
|
||||
FOLDER="${{ inputs.model_folder }}"
|
||||
if [ "$FOLDER" = "Other" ]; then
|
||||
FOLDER="${{ inputs.custom_model_folder }}"
|
||||
fi
|
||||
ARGS=""
|
||||
if [ "$FOLDER" != "None" ] && [ -n "$FOLDER" ]; then
|
||||
ARGS="$ARGS --model-folder \"$FOLDER\""
|
||||
fi
|
||||
[ -n "${{ inputs.generation }}" ] && ARGS="$ARGS --generation \"${{ inputs.generation }}\""
|
||||
[ -n "${{ inputs.version }}" ] && ARGS="$ARGS --version \"${{ inputs.version }}\""
|
||||
eval python3 docs/json_parser.py \
|
||||
--json-path "$JSON_FILE" \
|
||||
--recompiled-dir "gitlab_docs/models/$RECOMPILED_DIR" \
|
||||
--sort-by-date \
|
||||
$ARGS
|
||||
|
||||
- name: Push updated JSON to GitHub docs repo
|
||||
run: |
|
||||
cd docs
|
||||
git config --global user.name "GitHub Action"
|
||||
git config --global user.email "action@github.com"
|
||||
git checkout gh-pages
|
||||
git add docs/"$(basename $JSON_FILE)"
|
||||
git commit -m "Update $(basename $JSON_FILE) after recompiling model" || echo "No changes to commit"
|
||||
git push origin gh-pages
|
||||
@@ -1,78 +0,0 @@
|
||||
name: cereal validation
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
paths:
|
||||
- 'cereal/**'
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run_number:
|
||||
default: '1'
|
||||
required: true
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
group: cereal-validation-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
CI: 1
|
||||
|
||||
jobs:
|
||||
generate_cereal_artifact:
|
||||
name: Generate cereal validation artifacts
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
run: scons -j$(nproc) cereal
|
||||
- name: Generate the log file
|
||||
run: |
|
||||
export PYTHONPATH=${{ github.workspace }}
|
||||
python3 cereal/messaging/tests/validate_sp_cereal_upstream.py -g -f schema_instances.bin
|
||||
- name: 'Prepare artifact'
|
||||
run: |
|
||||
mkdir -p "cereal/messaging/tests/cereal_validations"
|
||||
cp cereal/messaging/tests/validate_sp_cereal_upstream.py "cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py"
|
||||
cp schema_instances.bin "cereal/messaging/tests/cereal_validations/schema_instances.bin"
|
||||
- name: 'Upload Artifact'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: cereal_validations
|
||||
path: cereal/messaging/tests/cereal_validations
|
||||
|
||||
validate_cereal_with_upstream:
|
||||
name: Validate cereal with Upstream
|
||||
runs-on: ubuntu-24.04
|
||||
needs: generate_cereal_artifact
|
||||
steps:
|
||||
- name: Checkout sunnypilot
|
||||
uses: actions/checkout@v6
|
||||
- name: Checkout upstream openpilot
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: 'commaai/openpilot'
|
||||
path: openpilot
|
||||
submodules: true
|
||||
ref: "refs/heads/master"
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
working-directory: openpilot
|
||||
run: scons -j$(nproc) cereal
|
||||
- name: Download build artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: cereal_validations
|
||||
path: openpilot/cereal/messaging/tests/cereal_validations
|
||||
- name: 'Run the validation'
|
||||
run: |
|
||||
export PYTHONPATH=${{ github.workspace }}/openpilot
|
||||
chmod +x openpilot/cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py
|
||||
python3 openpilot/cereal/messaging/tests/cereal_validations/validate_sp_cereal_upstream.py -r -f openpilot/cereal/messaging/tests/cereal_validations/schema_instances.bin
|
||||
@@ -1,45 +0,0 @@
|
||||
name: diff report
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened, synchronize, reopened]
|
||||
|
||||
jobs:
|
||||
comment:
|
||||
name: comment
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
actions: read
|
||||
steps:
|
||||
- name: Wait for process replay
|
||||
id: wait
|
||||
continue-on-error: true
|
||||
uses: lewagon/wait-on-check-action@v1.3.4
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
check-name: process replay
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
allowed-conclusions: success,failure
|
||||
wait-interval: 20
|
||||
- name: Download diff
|
||||
if: steps.wait.outcome == 'success'
|
||||
uses: dawidd6/action-download-artifact@v6
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
workflow: tests.yaml
|
||||
workflow_conclusion: ''
|
||||
pr: ${{ github.event.number }}
|
||||
name: diff_report_${{ github.event.number }}
|
||||
path: .
|
||||
allow_forks: true
|
||||
- name: Comment on PR
|
||||
if: steps.wait.outcome == 'success'
|
||||
uses: thollander/actions-comment-pull-request@v2
|
||||
with:
|
||||
filePath: diff_report.txt
|
||||
comment_tag: diff_report
|
||||
pr_number: ${{ github.event.number }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -1,65 +0,0 @@
|
||||
name: docs
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run_number:
|
||||
default: '1'
|
||||
required: true
|
||||
type: string
|
||||
concurrency:
|
||||
group: docs-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
docs:
|
||||
name: build docs
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: commaai/timeout@v1
|
||||
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
# Build
|
||||
- name: Build docs
|
||||
run: |
|
||||
# TODO: can we install just the "docs" dependency group without the normal deps?
|
||||
pip install mkdocs
|
||||
mkdocs build
|
||||
|
||||
# Push to docs.comma.ai
|
||||
- uses: actions/checkout@v6
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
|
||||
with:
|
||||
path: openpilot-docs
|
||||
ssh-key: ${{ secrets.OPENPILOT_DOCS_KEY }}
|
||||
repository: sunnypilot/sunnypilot-docs
|
||||
- name: Push
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'sunnypilot/sunnypilot'
|
||||
run: |
|
||||
set -x
|
||||
|
||||
source release/identity.sh
|
||||
|
||||
cd openpilot-docs
|
||||
git checkout --orphan tmp
|
||||
git rm -rf .
|
||||
|
||||
# copy over docs
|
||||
cp -r ../docs_site/ docs/
|
||||
|
||||
# GitHub pages config
|
||||
touch docs/.nojekyll
|
||||
echo -n docs.comma.ai > docs/CNAME
|
||||
|
||||
git add -f .
|
||||
git commit -m "build docs"
|
||||
|
||||
# docs live in different repo to not bloat openpilot's full clone size
|
||||
git push -f origin tmp:gh-pages
|
||||
@@ -1,96 +0,0 @@
|
||||
name: jenkins scan
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created, edited]
|
||||
|
||||
jobs:
|
||||
cleanup-branches:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- name: Delete stale Jenkins branches
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: |
|
||||
const cutoff = Date.now() - 24 * 60 * 60 * 1000;
|
||||
const prefixes = ['tmp-jenkins', '__jenkins'];
|
||||
|
||||
for await (const response of github.paginate.iterator(github.rest.repos.listBranches, {
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
per_page: 100,
|
||||
})) {
|
||||
for (const branch of response.data) {
|
||||
if (!prefixes.some(p => branch.name.startsWith(p))) continue;
|
||||
|
||||
const { data: commit } = await github.rest.repos.getCommit({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
ref: branch.commit.sha,
|
||||
});
|
||||
|
||||
const commitDate = new Date(commit.commit.committer.date).getTime();
|
||||
if (commitDate < cutoff) {
|
||||
console.log(`Deleting branch: ${branch.name} (last commit: ${commit.commit.committer.date})`);
|
||||
await github.rest.git.deleteRef({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
ref: `heads/${branch.name}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
scan-comments:
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ github.event.issue.pull_request }}
|
||||
permissions:
|
||||
contents: write
|
||||
issues: write
|
||||
steps:
|
||||
- name: Check for trigger phrase
|
||||
id: check_comment
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: |
|
||||
const triggerPhrase = "trigger-jenkins";
|
||||
const comment = context.payload.comment.body;
|
||||
const commenter = context.payload.comment.user.login;
|
||||
|
||||
const { data: permissions } = await github.rest.repos.getCollaboratorPermissionLevel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
username: commenter
|
||||
});
|
||||
|
||||
const hasWriteAccess = permissions.permission === 'write' || permissions.permission === 'admin';
|
||||
|
||||
return (hasWriteAccess && comment.includes(triggerPhrase));
|
||||
result-encoding: json
|
||||
|
||||
- name: Checkout repository
|
||||
if: steps.check_comment.outputs.result == 'true'
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: refs/pull/${{ github.event.issue.number }}/head
|
||||
|
||||
- name: Push to tmp-jenkins branch
|
||||
if: steps.check_comment.outputs.result == 'true'
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -b tmp-jenkins-${{ github.event.issue.number }}
|
||||
GIT_LFS_SKIP_PUSH=1 git push -f origin tmp-jenkins-${{ github.event.issue.number }}
|
||||
|
||||
- name: Delete trigger comment
|
||||
if: steps.check_comment.outputs.result == 'true' && always()
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: |
|
||||
await github.rest.issues.deleteComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
comment_id: context.payload.comment.id,
|
||||
});
|
||||
@@ -1,72 +0,0 @@
|
||||
name: Sync comma's LFS
|
||||
|
||||
env:
|
||||
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
|
||||
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 0 * * *' # Runs at 00:00 UTC every day
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
workflow_dispatch: # enables manual triggering
|
||||
inputs:
|
||||
upstream_branch:
|
||||
default: 'master'
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
runs-on: ubuntu-latest
|
||||
# Skip if PR is in draft mode
|
||||
if: (github.event_name != 'pull_request' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false)) && !github.event.pull_request.head.repo.fork
|
||||
steps:
|
||||
- name: Checkout Repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: 'commaai/openpilot'
|
||||
ref: ${{ inputs.upstream_branch }}
|
||||
|
||||
- name: LFS Fetch
|
||||
run: |
|
||||
git lfs fetch
|
||||
|
||||
- name: Set up Git
|
||||
run: |
|
||||
git config --global user.name 'GitHub Action'
|
||||
git config --global user.email 'action@github.com'
|
||||
|
||||
- name: Set up SSH
|
||||
uses: webfactory/ssh-agent@v0.9.0
|
||||
with:
|
||||
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
|
||||
|
||||
- name: Add GitLab public keys
|
||||
run: |
|
||||
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Ensure branch
|
||||
run: |
|
||||
if git symbolic-ref -q HEAD >/dev/null; then
|
||||
echo "Already on a branch, proceeding with push"
|
||||
else
|
||||
echo "Detached HEAD state detected, creating temporary branch"
|
||||
git checkout -b temp_branch
|
||||
fi
|
||||
|
||||
- name: Update LFS Config
|
||||
run: |
|
||||
echo '[lfs]' > .lfsconfig
|
||||
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
|
||||
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
|
||||
echo ' locksverify = false' >> .lfsconfig
|
||||
|
||||
- name: Push LFS
|
||||
id: sync-and-commit
|
||||
run: |
|
||||
git lfs ls-files -l
|
||||
git lfs push --all origin
|
||||
@@ -1,42 +0,0 @@
|
||||
name: "model review"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, reopened, synchronize]
|
||||
paths:
|
||||
- 'selfdrive/modeld/models/*.onnx'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
comment:
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'commaai/openpilot'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- name: Checkout master
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: master
|
||||
path: base
|
||||
- run: git lfs pull
|
||||
- run: cd base && git lfs pull
|
||||
|
||||
- name: scripts/reporter.py
|
||||
id: report
|
||||
run: |
|
||||
echo "content<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "## Model Review" >> $GITHUB_OUTPUT
|
||||
PYTHONPATH=${{ github.workspace }} MASTER_PATH=${{ github.workspace }}/base python scripts/reporter.py >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Post model report comment
|
||||
uses: marocchino/sticky-pull-request-comment@baa7203ed60924babbe5dcd0ac8eae3b66ec5e16
|
||||
with:
|
||||
header: model-review
|
||||
message: ${{ steps.report.outputs.content }}
|
||||
@@ -1,105 +0,0 @@
|
||||
name: 'Post to Discourse'
|
||||
description: 'Posts a message to a Discourse topic (existing or new)'
|
||||
|
||||
inputs:
|
||||
discourse-url:
|
||||
description: 'Discourse instance URL (e.g., https://discourse.example.com)'
|
||||
required: true
|
||||
api-key:
|
||||
description: 'Discourse API key'
|
||||
required: true
|
||||
api-username:
|
||||
description: 'Discourse API username'
|
||||
required: true
|
||||
topic-id:
|
||||
description: 'Discourse topic ID to post to (use this OR category-id + title)'
|
||||
required: false
|
||||
category-id:
|
||||
description: 'Category ID for new topic (required if topic-id not provided)'
|
||||
required: false
|
||||
title:
|
||||
description: 'Title for new topic (required if topic-id not provided)'
|
||||
required: false
|
||||
message:
|
||||
description: 'Message content (markdown supported)'
|
||||
required: true
|
||||
|
||||
outputs:
|
||||
post-number:
|
||||
description: 'The post number in the topic'
|
||||
value: ${{ steps.post.outputs.post_number }}
|
||||
post-url:
|
||||
description: 'Direct URL to the post'
|
||||
value: ${{ steps.post.outputs.post_url }}
|
||||
topic-id:
|
||||
description: 'The topic ID (useful when creating a new topic)'
|
||||
value: ${{ steps.post.outputs.topic_id }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Post to Discourse
|
||||
id: post
|
||||
shell: bash
|
||||
run: |
|
||||
# Validate inputs
|
||||
if [ -z "${{ inputs.topic-id }}" ] && ([ -z "${{ inputs.category-id }}" ] || [ -z "${{ inputs.title }}" ]); then
|
||||
echo "❌ Error: Must provide either topic-id OR both category-id and title"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -n "${{ inputs.topic-id }}" ] && ([ -n "${{ inputs.category-id }}" ] || [ -n "${{ inputs.title }}" ]); then
|
||||
echo "⚠️ Warning: Both topic-id and category-id/title provided. Will post to existing topic."
|
||||
fi
|
||||
|
||||
# Determine if creating new topic or posting to existing
|
||||
if [ -n "${{ inputs.topic-id }}" ]; then
|
||||
echo "📝 Posting to existing topic ID: ${{ inputs.topic-id }}"
|
||||
|
||||
# Create JSON payload for posting to existing topic
|
||||
PAYLOAD=$(jq -n \
|
||||
--arg content '${{ inputs.message }}' \
|
||||
--arg topic_id "${{ inputs.topic-id }}" \
|
||||
'{topic_id: $topic_id, raw: $content}')
|
||||
else
|
||||
echo "✨ Creating new topic: ${{ inputs.title }}"
|
||||
|
||||
# Create JSON payload for new topic
|
||||
PAYLOAD=$(jq -n \
|
||||
--arg content '${{ inputs.message }}' \
|
||||
--arg title "${{ inputs.title }}" \
|
||||
--arg category "${{ inputs.category-id }}" \
|
||||
'{title: $title, category: ($category | tonumber), raw: $content}')
|
||||
fi
|
||||
|
||||
# Post to Discourse
|
||||
RESPONSE=$(curl -s -w "\n%{http_code}" \
|
||||
-X POST "${{ inputs.discourse-url }}/posts.json" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Api-Key: ${{ inputs.api-key }}" \
|
||||
-H "Api-Username: ${{ inputs.api-username }}" \
|
||||
-d "$PAYLOAD")
|
||||
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
|
||||
BODY=$(echo "$RESPONSE" | sed '$d')
|
||||
|
||||
if [ "$HTTP_CODE" -ge 200 ] && [ "$HTTP_CODE" -lt 300 ]; then
|
||||
echo "✅ Successfully posted to Discourse!"
|
||||
|
||||
POST_NUMBER=$(echo "$BODY" | jq -r '.post_number // "unknown"')
|
||||
TOPIC_ID=$(echo "$BODY" | jq -r '.topic_id // "${{ inputs.topic-id }}"')
|
||||
POST_URL="${{ inputs.discourse-url }}/t/${TOPIC_ID}/${POST_NUMBER}"
|
||||
|
||||
echo "post_number=${POST_NUMBER}" >> $GITHUB_OUTPUT
|
||||
echo "post_url=${POST_URL}" >> $GITHUB_OUTPUT
|
||||
echo "topic_id=${TOPIC_ID}" >> $GITHUB_OUTPUT
|
||||
|
||||
echo "Topic ID: ${TOPIC_ID}"
|
||||
echo "Post number: ${POST_NUMBER}"
|
||||
echo "URL: ${POST_URL}"
|
||||
else
|
||||
echo "❌ Failed to post to Discourse"
|
||||
echo "HTTP Code: ${HTTP_CODE}"
|
||||
echo "Response: ${BODY}"
|
||||
exit 1
|
||||
fi
|
||||
@@ -1,39 +0,0 @@
|
||||
name: prebuilt
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 * * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
DOCKER_LOGIN: docker login ghcr.io -u ${{ github.actor }} -p ${{ secrets.GITHUB_TOKEN }}
|
||||
BUILD: release/ci/docker_build_sp.sh
|
||||
|
||||
jobs:
|
||||
build_prebuilt:
|
||||
name: build prebuilt
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
env:
|
||||
PUSH_IMAGE: true
|
||||
permissions:
|
||||
checks: read
|
||||
contents: read
|
||||
packages: write
|
||||
steps:
|
||||
- name: Wait for green check mark
|
||||
if: ${{ github.event_name != 'workflow_dispatch' }}
|
||||
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
|
||||
with:
|
||||
ref: master
|
||||
wait-interval: 30
|
||||
running-workflow-name: 'build prebuilt'
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
check-regexp: ^((?!.*(build master-ci|create badges).*).)*$
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: git lfs pull
|
||||
- name: Build and Push docker image
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
eval "$BUILD"
|
||||
@@ -1,28 +0,0 @@
|
||||
name: Release Drafter
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
tags:
|
||||
- 'v*'
|
||||
pull_request_target:
|
||||
types: [opened, reopened, synchronize]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
update_release_draft:
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: release-drafter/release-drafter@v6
|
||||
with:
|
||||
config-name: release-drafter.yml
|
||||
prerelease: ${{ !startsWith(github.ref, 'refs/tags/v') }}
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -1,31 +0,0 @@
|
||||
name: release
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 9 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build___nightly:
|
||||
name: build __nightly
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
permissions:
|
||||
checks: read
|
||||
contents: write
|
||||
steps:
|
||||
- name: Wait for green check mark
|
||||
if: ${{ github.event_name == 'schedule' }}
|
||||
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
|
||||
with:
|
||||
ref: master
|
||||
wait-interval: 30
|
||||
running-workflow-name: 'build __nightly'
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
check-regexp: ^((?!.*(build prebuilt|create badges).*).)*$
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
fetch-depth: 0
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Push __nightly
|
||||
run: BRANCH=__nightly release/build_stripped.sh
|
||||
@@ -1,77 +0,0 @@
|
||||
name: repo maintenance
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 14 * * 1" # every Monday at 2am UTC (6am PST)
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
PYTHONPATH: ${{ github.workspace }}
|
||||
|
||||
jobs:
|
||||
package_updates:
|
||||
name: package_updates
|
||||
runs-on: ubuntu-latest
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: uv lock
|
||||
run: uv lock --upgrade
|
||||
- name: uv pip tree
|
||||
id: pip_tree
|
||||
run: |
|
||||
echo 'PIP_TREE<<EOF' >> $GITHUB_OUTPUT
|
||||
uv pip tree >> $GITHUB_OUTPUT
|
||||
echo 'EOF' >> $GITHUB_OUTPUT
|
||||
- name: venv size
|
||||
id: venv_size
|
||||
run: |
|
||||
echo 'VENV_SIZE<<EOF' >> $GITHUB_OUTPUT
|
||||
echo "Total: $(du -sh .venv | cut -f1)" >> $GITHUB_OUTPUT
|
||||
echo "" >> $GITHUB_OUTPUT
|
||||
echo "Top 10 by size:" >> $GITHUB_OUTPUT
|
||||
du -sh .venv/lib/python*/site-packages/* 2>/dev/null \
|
||||
| grep -v '\.dist-info' \
|
||||
| grep -v '__pycache__' \
|
||||
| sort -rh \
|
||||
| head -10 \
|
||||
| while IFS=$'\t' read size path; do echo "$size ${path##*/}"; done >> $GITHUB_OUTPUT
|
||||
echo 'EOF' >> $GITHUB_OUTPUT
|
||||
- name: bump submodules
|
||||
run: |
|
||||
git config submodule.msgq.update none
|
||||
git config submodule.rednose_repo.update none
|
||||
git config submodule.teleoprtc_repo.update none
|
||||
git config submodule.tinygrad.update none
|
||||
git submodule update --remote
|
||||
git add .
|
||||
- name: update car docs
|
||||
run: |
|
||||
python selfdrive/car/docs.py
|
||||
git add docs/CARS.md
|
||||
- name: Create Pull Request
|
||||
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0
|
||||
with:
|
||||
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
||||
token: ${{ github.repository == 'commaai/openpilot' && secrets.ACTIONS_CREATE_PR_PAT || secrets.GITHUB_TOKEN }}
|
||||
commit-message: Update Python packages
|
||||
title: '[bot] Update Python packages'
|
||||
branch: auto-package-updates
|
||||
base: master
|
||||
delete-branch: true
|
||||
body: |
|
||||
Automatic PR from repo-maintenance -> package_updates
|
||||
|
||||
```
|
||||
$ du -sh .venv && du -sh .venv/lib/python*/site-packages/* | sort -rh | head -10
|
||||
${{ steps.venv_size.outputs.VENV_SIZE }}
|
||||
```
|
||||
|
||||
```
|
||||
$ uv pip tree
|
||||
${{ steps.pip_tree.outputs.PIP_TREE }}
|
||||
```
|
||||
labels: bot
|
||||
@@ -1,52 +0,0 @@
|
||||
name: stale
|
||||
on:
|
||||
schedule:
|
||||
- cron: '30 1 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
DAYS_BEFORE_PR_CLOSE: 7
|
||||
DAYS_BEFORE_PR_STALE: 24
|
||||
DAYS_BEFORE_PR_STALE_DRAFT: 30
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@v10
|
||||
with:
|
||||
exempt-all-milestones: true
|
||||
|
||||
# pull request config
|
||||
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
|
||||
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
|
||||
stale-pr-label: stale
|
||||
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'sunnypilot/sunnypilot' }} # only delete branches on the main repo
|
||||
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
|
||||
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE }}
|
||||
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}
|
||||
exempt-draft-pr: false
|
||||
|
||||
# issue config
|
||||
days-before-issue-stale: -1 # ignore issues for now
|
||||
|
||||
# same as above, but give draft PRs more time
|
||||
stale_drafts:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@v10
|
||||
with:
|
||||
exempt-all-milestones: true
|
||||
|
||||
# pull request config
|
||||
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE_DRAFT }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
|
||||
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
|
||||
stale-pr-label: stale
|
||||
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'commaai/openpilot' }} # only delete branches on the main repo
|
||||
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
|
||||
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE_DRAFT }}
|
||||
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}
|
||||
exempt-draft-pr: true
|
||||
|
||||
# issue config
|
||||
days-before-issue-stale: -1 # ignore issues for now
|
||||
@@ -1,229 +0,0 @@
|
||||
name: Build Model from Upstream
|
||||
|
||||
env:
|
||||
BUILD_DIR: "/data/openpilot"
|
||||
OUTPUT_DIR: ${{ github.workspace }}/output
|
||||
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
|
||||
UPSTREAM_REPO: "commaai/openpilot"
|
||||
TINYGRAD_PATH: ${{ github.workspace }}/tinygrad_repo
|
||||
MODELS_DIR: ${{ github.workspace }}/selfdrive/modeld/models
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
upstream_branch:
|
||||
description: 'Upstream branch to build from'
|
||||
required: true
|
||||
default: 'master'
|
||||
type: string
|
||||
custom_name:
|
||||
description: 'Custom name for the model (no date, only name)'
|
||||
required: false
|
||||
type: string
|
||||
is_20hz:
|
||||
description: 'Is this a 20Hz model'
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
artifact_suffix:
|
||||
description: 'Suffix for artifact name'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
upstream_branch:
|
||||
description: 'Upstream branch to build from'
|
||||
required: true
|
||||
default: 'master'
|
||||
type: string
|
||||
custom_name:
|
||||
description: 'Custom name for the model (no date, only name)'
|
||||
required: false
|
||||
type: string
|
||||
is_20hz:
|
||||
description: 'Is this a 20Hz model'
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
|
||||
run-name: Build model [${{ inputs.custom_name || inputs.upstream_branch }}] from ref [${{ inputs.upstream_branch }}]
|
||||
|
||||
jobs:
|
||||
get_model:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
REF: ${{ inputs.upstream_branch }}
|
||||
outputs:
|
||||
model_date: ${{ steps.commit-date.outputs.model_date }}
|
||||
steps:
|
||||
# Note: To allow dynamic models from both openpilot and sunnypilot (merges/mashups), we try commaai as default,
|
||||
# and fallback to sunnypilot if the ref checkout fails.
|
||||
- name: Checkout commaai/openpilot
|
||||
id: checkout_upstream
|
||||
continue-on-error: true
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: commaai/openpilot
|
||||
ref: ${{ inputs.upstream_branch }}
|
||||
submodules: recursive
|
||||
path: openpilot
|
||||
|
||||
- name: Fallback to sunnypilot/sunnypilot
|
||||
if: steps.checkout_upstream.outcome == 'failure'
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: sunnypilot/sunnypilot
|
||||
ref: ${{ inputs.upstream_branch }}
|
||||
submodules: recursive
|
||||
path: openpilot
|
||||
- name: Get commit date
|
||||
id: commit-date
|
||||
run: |
|
||||
cd ${{ github.workspace }}/openpilot
|
||||
commit_date=$(git log -1 --format=%cd --date=format:'%B %d, %Y')
|
||||
echo "model_date=${commit_date}" >> $GITHUB_OUTPUT
|
||||
cat $GITHUB_OUTPUT
|
||||
- run: |
|
||||
cd ${{ github.workspace }}/openpilot
|
||||
git lfs pull
|
||||
- name: 'Upload Artifact'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: models-${{ env.REF }}${{ inputs.artifact_suffix }}
|
||||
path: ${{ github.workspace }}/openpilot/selfdrive/modeld/models/*.onnx
|
||||
|
||||
build_model:
|
||||
runs-on: [self-hosted, tici]
|
||||
needs: get_model
|
||||
env:
|
||||
MODEL_NAME: ${{ inputs.custom_name || inputs.upstream_branch }} (${{ needs.get_model.outputs.model_date }})
|
||||
REF: ${{ inputs.upstream_branch }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- run: git lfs pull
|
||||
- name: Cache SCons
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{env.SCONS_CACHE_DIR}}
|
||||
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
|
||||
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
|
||||
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
|
||||
restore-keys: |
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}-model
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}-model
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_NEW_BRANCH }}
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.MASTER_BRANCH }}
|
||||
scons-${{ runner.os }}-${{ runner.arch }}
|
||||
|
||||
- name: Set environment variables
|
||||
id: set-env
|
||||
run: |
|
||||
# Set up common environment
|
||||
source /etc/profile;
|
||||
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
|
||||
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
|
||||
printenv >> $GITHUB_ENV
|
||||
if [[ "${{ runner.debug }}" == "1" ]]; then
|
||||
cat $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Setup build environment
|
||||
run: |
|
||||
mkdir -p "${BUILD_DIR}/"
|
||||
sudo find $BUILD_DIR/ -mindepth 1 -delete
|
||||
echo "Starting build stage..."
|
||||
echo "BUILD_DIR: ${BUILD_DIR}"
|
||||
echo "CI_DIR: ${CI_DIR}"
|
||||
echo "VERSION: ${{ steps.set-env.outputs.version }}"
|
||||
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
|
||||
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
|
||||
echo "-------"
|
||||
if [[ "${{ runner.debug }}" == "1" ]]; then
|
||||
printenv
|
||||
fi
|
||||
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
|
||||
rm -rf ${{ env.MODELS_DIR }}/*.onnx
|
||||
|
||||
- name: Download model artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: models-${{ env.REF }}${{ inputs.artifact_suffix }}
|
||||
path: ${{ github.workspace }}/selfdrive/modeld/models
|
||||
- run: |
|
||||
rm -f ${{ github.workspace }}/selfdrive/modeld/models/{dmonitoring_model,big_driving_policy,big_driving_vision}.onnx
|
||||
|
||||
- name: Build Model
|
||||
run: |
|
||||
source /etc/profile
|
||||
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
|
||||
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
|
||||
export PYTHONPATH="${PYTHONPATH}:${{ env.TINYGRAD_PATH }}"
|
||||
|
||||
# Loop through all .onnx files
|
||||
find "${{ env.MODELS_DIR }}" -maxdepth 1 -name '*.onnx' | while IFS= read -r onnx_file; do
|
||||
base_name=$(basename "$onnx_file" .onnx)
|
||||
output_file="${{ env.MODELS_DIR }}/${base_name}_tinygrad.pkl"
|
||||
|
||||
echo "Compiling: $onnx_file -> $output_file"
|
||||
QCOM=1 python3 "${{ env.TINYGRAD_PATH }}/examples/openpilot/compile3.py" "$onnx_file" "$output_file"
|
||||
DEV=QCOM FLOAT16=1 NOLOCALS=1 JIT_BATCH_SIZE=0 python3 "${{ env.MODELS_DIR }}/../get_model_metadata.py" "$onnx_file" || true
|
||||
done
|
||||
|
||||
- name: Validate Model Outputs
|
||||
run: |
|
||||
source /etc/profile
|
||||
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
|
||||
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
|
||||
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
|
||||
--validate-only \
|
||||
--model-dir "${{ env.MODELS_DIR }}"
|
||||
|
||||
- name: Prepare Output
|
||||
run: |
|
||||
sudo rm -rf ${{ env.OUTPUT_DIR }}
|
||||
mkdir -p ${{ env.OUTPUT_DIR }}
|
||||
|
||||
# Copy the model files
|
||||
rsync -avm \
|
||||
--include='*.dlc' \
|
||||
--include='*.pkl' \
|
||||
--include='*.onnx' \
|
||||
--exclude='*' \
|
||||
--delete-excluded \
|
||||
--chown=comma:comma \
|
||||
${{ env.MODELS_DIR }}/ ${{ env.OUTPUT_DIR }}/
|
||||
|
||||
python3 "${{ github.workspace }}/release/ci/model_generator.py" \
|
||||
--model-dir "${{ env.MODELS_DIR }}" \
|
||||
--output-dir "${{ env.OUTPUT_DIR }}" \
|
||||
--custom-name "${{ env.MODEL_NAME }}" \
|
||||
--upstream-branch "${{ inputs.upstream_branch }}" \
|
||||
${{ inputs.is_20hz && '--is-20hz' || '' }}
|
||||
|
||||
- name: Write artifact name to file
|
||||
run: echo "model-${{ env.MODEL_NAME }}${{ inputs.artifact_suffix }}-${{ github.run_number }}" > ${{ env.OUTPUT_DIR }}/artifact_name.txt
|
||||
|
||||
- name: Upload Build Artifacts
|
||||
id: upload-artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: model-${{ env.MODEL_NAME }}${{ inputs.artifact_suffix }}-${{ github.run_number }}
|
||||
path: ${{ env.OUTPUT_DIR }}
|
||||
|
||||
- name: Upload artifact name file
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: artifact-name-${{ inputs.custom_name || inputs.upstream_branch }}
|
||||
path: ${{ env.OUTPUT_DIR }}/artifact_name.txt
|
||||
|
||||
- name: Re-enable powersave
|
||||
if: always()
|
||||
run: |
|
||||
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
|
||||
@@ -1,369 +0,0 @@
|
||||
name: sunnypilot prebuilt action
|
||||
|
||||
env:
|
||||
BUILD_DIR: "/data/openpilot"
|
||||
OUTPUT_DIR: ${{ github.workspace }}/output
|
||||
CI_DIR: ${{ github.workspace }}/release/ci
|
||||
SCONS_CACHE_DIR: ${{ github.workspace }}/release/ci/scons_cache
|
||||
PUBLIC_REPO_URL: "https://github.com/sunnypilot/sunnypilot"
|
||||
|
||||
# Branch configurations
|
||||
STAGING_SOURCE_BRANCH: 'master'
|
||||
|
||||
# Runtime configuration
|
||||
SOURCE_BRANCH: "${{ github.head_ref || github.ref_name }}"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ master, master-dev ]
|
||||
tags: [ 'release/*' ]
|
||||
pull_request_target:
|
||||
types: [ labeled ]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
wait_for_tests:
|
||||
description: 'Wait for tests to finish'
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
prepare_strategy:
|
||||
runs-on: ubuntu-24.04
|
||||
if: (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
|
||||
outputs:
|
||||
environment: ${{ steps.strategy.outputs.environment }}
|
||||
new_branch: ${{ steps.strategy.outputs.new_branch }}
|
||||
extra_version_identifier: ${{ steps.strategy.outputs.extra_version_identifier }}
|
||||
version: ${{ steps.strategy.outputs.version }}
|
||||
cancel_publish_in_progress: ${{ steps.strategy.outputs.cancel_publish_in_progress }}
|
||||
publish_concurrency_group: ${{ steps.strategy.outputs.publish_concurrency_group }}
|
||||
is_stable_branch: ${{ steps.strategy.outputs.is_stable_branch }}
|
||||
build: ${{ steps.strategy.outputs.build }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Extract deploy strategy
|
||||
id: strategy
|
||||
run: |
|
||||
echo '::group::Strategy Extraction'
|
||||
BRANCH="${{ github.head_ref || github.ref_name }}"
|
||||
echo "Current branch: $BRANCH"
|
||||
|
||||
STRATEGY_JSON='${{ vars.DEPLOY_STRATEGY }}'
|
||||
CONFIG=$(echo "$STRATEGY_JSON" | jq -r --arg branch "$BRANCH" '
|
||||
.configs[] | select(.branch == $branch)
|
||||
')
|
||||
|
||||
BUILD="$(date '+%Y.%m.%d')-${{ github.run_number }}"
|
||||
if [[ -z "$CONFIG" || "$CONFIG" == "null" ]]; then
|
||||
echo "No exact strategy match found. Falling back to feature/fork logic."
|
||||
IS_FORK="${{ github.event.pull_request.head.repo.fork && 'true' || 'false' }}"
|
||||
FORK_SUFFIX=$( [[ "$IS_FORK" == "true" ]] && echo "-fork" || echo "" )
|
||||
NEW_BRANCH="${BRANCH}${FORK_SUFFIX}-prebuilt"
|
||||
|
||||
echo "new_branch=$NEW_BRANCH" >> $GITHUB_OUTPUT
|
||||
echo "version=$BUILD" >> $GITHUB_OUTPUT
|
||||
echo "cancel_publish_in_progress=true" >> $GITHUB_OUTPUT
|
||||
echo "publish_concurrency_group=publish-${BRANCH}" >> $GITHUB_OUTPUT
|
||||
echo "environment=feature-branch" >> $GITHUB_OUTPUT
|
||||
echo "extra_version_identifier=feature-branch" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Matched config: $CONFIG"
|
||||
environment=$(echo "$CONFIG" | jq -r '.environment')
|
||||
echo "environment=$environment" >> $GITHUB_OUTPUT
|
||||
echo "new_branch=$(echo "$CONFIG" | jq -r '.target_branch')" >> $GITHUB_OUTPUT
|
||||
cancel="$(echo "$CONFIG" | jq -r '.cancel_publish_in_progress')";
|
||||
echo "cancel_publish_in_progress=$( [ "$cancel" = "null" ] && echo "true" || echo $cancel)" >> $GITHUB_OUTPUT
|
||||
echo "publish_concurrency_group=publish-${BRANCH}$( [ "$cancel" = "null" ] || [ "$cancel" = "true" ] || echo "${{ github.sha }}" )" >> $GITHUB_OUTPUT
|
||||
|
||||
is_stable_branch="$(echo "$CONFIG" | jq -r '.stable_branch // false')";
|
||||
echo "is_stable_branch=$is_stable_branch" >> $GITHUB_OUTPUT
|
||||
|
||||
stable_version=$(cat sunnypilot/common/version.h | grep SUNNYPILOT_VERSION | sed -e 's/[^0-9|.]//g');
|
||||
echo "version=$([ "$is_stable_branch" = "true" ] && echo "$stable_version" || echo "$BUILD")" >> $GITHUB_OUTPUT
|
||||
echo "extra_version_identifier=${environment}" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
echo "build=$BUILD" >> $GITHUB_OUTPUT
|
||||
cat $GITHUB_OUTPUT
|
||||
|
||||
validate_tests:
|
||||
runs-on: ubuntu-24.04
|
||||
needs: [ prepare_strategy ]
|
||||
if: ${{
|
||||
((github.event_name == 'workflow_dispatch' && inputs.wait_for_tests) ||
|
||||
(github.event_name == 'push' && needs.prepare_strategy.outputs.is_stable_branch == 'true') ||
|
||||
contains(github.event_name, 'pull_request') && (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
|
||||
}}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Wait for Tests
|
||||
uses: ./.github/workflows/wait-for-action # Path to where you place the action
|
||||
with:
|
||||
workflow: tests.yaml # The workflow file to monitor
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
should-wait-for-start: ${{ github.event_name == 'push' && 'true' || 'false' }}
|
||||
|
||||
build:
|
||||
needs: [ validate_tests, prepare_strategy ]
|
||||
concurrency:
|
||||
group: build-${{ github.head_ref || github.ref_name }}
|
||||
cancel-in-progress: false
|
||||
runs-on: [self-hosted, tici]
|
||||
outputs:
|
||||
new_branch: ${{ needs.prepare_strategy.outputs.new_branch }}
|
||||
version: ${{ needs.prepare_strategy.outputs.version }}
|
||||
extra_version_identifier: ${{ needs.prepare_strategy.outputs.extra_version_identifier }}
|
||||
commit_sha: ${{ github.sha }}
|
||||
if: ${{
|
||||
(always() && !cancelled() && !failure()) &&
|
||||
needs.prepare_strategy.result == 'success' &&
|
||||
(needs.validate_tests.result == 'success' || needs.validate_tests.result == 'skipped') &&
|
||||
(!contains(github.event_name, 'pull_request') ||
|
||||
(github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
|
||||
}}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
ref: ${{ env.SOURCE_BRANCH }}
|
||||
repository: ${{ github.event.pull_request.head.repo.fork && github.event.pull_request.head.repo.full_name || github.repository }}
|
||||
- run: git lfs pull
|
||||
|
||||
- name: Cache SCons
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{env.SCONS_CACHE_DIR}}
|
||||
key: scons-${{ runner.os }}-${{ runner.arch }}-${{ env.SOURCE_BRANCH }}-${{ github.sha }}
|
||||
# Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
|
||||
# for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
|
||||
restore-keys: |
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.SOURCE_BRANCH }}
|
||||
scons-${{ runner.os }}-${{ runner.arch }}-${{ env.STAGING_SOURCE_BRANCH }}
|
||||
scons-${{ runner.os }}-${{ runner.arch }}
|
||||
|
||||
- name: Set environment variables
|
||||
id: set-env
|
||||
run: |
|
||||
echo "new_branch=${{ needs.prepare_strategy.outputs.new_branch }}" >> $GITHUB_OUTPUT
|
||||
echo "version=${{ needs.prepare_strategy.outputs.version }}" >> $GITHUB_OUTPUT
|
||||
echo "extra_version_identifier=${{ needs.prepare_strategy.outputs.extra_version_identifier }}" >> $GITHUB_OUTPUT
|
||||
echo "commit_sha=${{ github.sha }}" >> $GITHUB_OUTPUT
|
||||
|
||||
# Set up common environment
|
||||
source /etc/profile;
|
||||
export UV_PROJECT_ENVIRONMENT=${HOME}/venv
|
||||
export VIRTUAL_ENV=$UV_PROJECT_ENVIRONMENT
|
||||
printenv >> $GITHUB_ENV
|
||||
if [[ "${{ runner.debug }}" == "1" ]]; then
|
||||
cat $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Setup build environment
|
||||
run: |
|
||||
mkdir -p "${BUILD_DIR}/"
|
||||
sudo find $BUILD_DIR/ -mindepth 1 -delete
|
||||
echo "Starting build stage..."
|
||||
echo "BUILD_DIR: ${BUILD_DIR}"
|
||||
echo "CI_DIR: ${CI_DIR}"
|
||||
echo "VERSION: ${{ steps.set-env.outputs.version }}"
|
||||
echo "UV_PROJECT_ENVIRONMENT: ${UV_PROJECT_ENVIRONMENT}"
|
||||
echo "VIRTUAL_ENV: ${VIRTUAL_ENV}"
|
||||
echo "-------"
|
||||
if [[ "${{ runner.debug }}" == "1" ]]; then
|
||||
printenv
|
||||
fi
|
||||
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --disable
|
||||
|
||||
- name: Build Main Project
|
||||
run: |
|
||||
export PYTHONPATH="$BUILD_DIR"
|
||||
./release/release_files.py | sort | uniq | rsync -rRl${RUNNER_DEBUG:+v} --files-from=- . $BUILD_DIR/
|
||||
cd $BUILD_DIR
|
||||
sed -i '/from .board.jungle import PandaJungle, PandaJungleDFU/s/^/#/' panda/__init__.py
|
||||
echo "Building sunnypilot's modeld_v2..."
|
||||
scons -j$(nproc) cache_dir=${{env.SCONS_CACHE_DIR}} --minimal sunnypilot/modeld_v2
|
||||
echo "Building sunnypilot's locationd..."
|
||||
scons -j2 cache_dir=${{env.SCONS_CACHE_DIR}} --minimal sunnypilot/selfdrive/locationd
|
||||
echo "Building openpilot's locationd..."
|
||||
scons -j1 cache_dir=${{env.SCONS_CACHE_DIR}} --minimal selfdrive/locationd
|
||||
echo "Building rest of sunnypilot"
|
||||
scons -j$(nproc) cache_dir=${{env.SCONS_CACHE_DIR}} --minimal
|
||||
touch ${BUILD_DIR}/prebuilt
|
||||
if [[ "${{ runner.debug }}" == "1" ]]; then
|
||||
ls -la ${BUILD_DIR}
|
||||
fi
|
||||
|
||||
- name: Prepare Output
|
||||
run: |
|
||||
sudo rm -rf ${OUTPUT_DIR}
|
||||
mkdir -p ${OUTPUT_DIR}
|
||||
rsync -am${RUNNER_DEBUG:+v} \
|
||||
--exclude='.sconsign.dblite' \
|
||||
--exclude='*.a' \
|
||||
--exclude='*.o' \
|
||||
--exclude='*.os' \
|
||||
--exclude='*.pyc' \
|
||||
--exclude='moc_*' \
|
||||
--exclude='__pycache__' \
|
||||
--exclude='Jenkinsfile' \
|
||||
--exclude='**/release/' \
|
||||
--exclude='**/.github/' \
|
||||
--exclude='**/selfdrive/ui/replay/' \
|
||||
--exclude='**/__pycache__/' \
|
||||
--exclude='${{env.SCONS_CACHE_DIR}}' \
|
||||
--exclude='**/.git/' \
|
||||
--exclude='**/SConstruct' \
|
||||
--exclude='**/SConscript' \
|
||||
--exclude='**/.venv/' \
|
||||
--exclude='selfdrive/modeld/models/driving_vision.onnx' \
|
||||
--exclude='selfdrive/modeld/models/driving_policy.onnx' \
|
||||
--exclude='third_party/*x86*' \
|
||||
--exclude='third_party/*Darwin*' \
|
||||
--delete-excluded \
|
||||
--chown=comma:comma \
|
||||
${BUILD_DIR}/ ${OUTPUT_DIR}/
|
||||
|
||||
- name: 'Tar.gz files'
|
||||
run: |
|
||||
tar czf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }} .
|
||||
ls -la prebuilt.tar.gz
|
||||
|
||||
- name: 'Upload Artifact'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: prebuilt
|
||||
path: prebuilt.tar.gz
|
||||
|
||||
- name: Re-enable powersave
|
||||
if: always()
|
||||
run: |
|
||||
PYTHONPATH=$PYTHONPATH:${{ github.workspace }}/ ${{ github.workspace }}/scripts/manage-powersave.py --enable
|
||||
|
||||
|
||||
publish:
|
||||
concurrency:
|
||||
# We do a bit of a hack here to avoid canceling the publishing job if a new commit comes in while we're publishing by adding the sha to the group name.
|
||||
# This means that if multiple commits come in while we're publishing, they will be queued up and publish one after the other.
|
||||
# Otherwise, if a job is waiting to be published due to environment wait time, it would be canceled by a new commit and restart the wait time.
|
||||
group: ${{ needs.prepare_strategy.outputs.publish_concurrency_group }}
|
||||
cancel-in-progress: ${{ needs.prepare_strategy.outputs.cancel_publish_in_progress == 'true' }}
|
||||
if: ${{ (always() && !cancelled() && !failure()) && needs.build.result == 'success' && needs.prepare_strategy.result == 'success' && (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt')) }}
|
||||
needs: [ build, prepare_strategy ]
|
||||
runs-on: ubuntu-24.04
|
||||
environment: ${{ needs.prepare_strategy.outputs.environment }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Download build artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: prebuilt
|
||||
|
||||
- name: Untar prebuilt
|
||||
run: |
|
||||
mkdir -p ${{ env.OUTPUT_DIR }}
|
||||
tar xzf prebuilt.tar.gz -C ${{ env.OUTPUT_DIR }}
|
||||
|
||||
- name: Configure Git
|
||||
run: |
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
git config --global user.name "github-actions[bot]"
|
||||
|
||||
- name: Publish to Public Repository
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
echo '${{ toJSON(needs.build.outputs) }}'
|
||||
ls -la ${{ env.OUTPUT_DIR }}
|
||||
|
||||
${{ env.CI_DIR }}/publish.sh \
|
||||
"${{ github.workspace }}" \
|
||||
"${{ env.OUTPUT_DIR }}" \
|
||||
"${{ needs.build.outputs.new_branch }}" \
|
||||
"${{ needs.build.outputs.version }}" \
|
||||
"https://x-access-token:${{github.token}}@github.com/sunnypilot/sunnypilot.git" \
|
||||
"${{ needs.build.outputs.extra_version_identifier }}"
|
||||
|
||||
echo ""
|
||||
echo "---- ℹ️ To update the list of branches that auto deploy prebuilts -----"
|
||||
echo ""
|
||||
echo "1. Go to: ${{ github.server_url }}/${{ github.repository }}/settings/variables/actions/AUTO_DEPLOY_PREBUILT_BRANCHES"
|
||||
echo "2. Current value: ${{ vars.AUTO_DEPLOY_PREBUILT_BRANCHES }}"
|
||||
echo "3. Update as needed (JSON array with no spaces)"
|
||||
|
||||
- name: Tag ${{ needs.prepare_strategy.outputs.environment }}
|
||||
if: ${{ needs.prepare_strategy.outputs.is_stable_branch == 'true' && (github.event_name != 'push' || !startsWith(github.ref, 'refs/tags/')) }}
|
||||
run: |
|
||||
TAG="${{ needs.prepare_strategy.outputs.environment }}/${{ needs.prepare_strategy.outputs.version }}/${{ needs.prepare_strategy.outputs.build }}"
|
||||
git tag -f -a ${TAG} -m "${{ needs.prepare_strategy.outputs.environment }} @ ${{ needs.prepare_strategy.outputs.version }} of build ${{ needs.build.outputs.build }}."
|
||||
git push -f origin ${TAG}
|
||||
|
||||
notify:
|
||||
needs:
|
||||
- prepare_strategy
|
||||
- build
|
||||
- publish
|
||||
runs-on: ubuntu-24.04
|
||||
if: ${{ (always() && !cancelled() && !failure())
|
||||
&& needs.publish.result == 'success'
|
||||
&& (!contains(github.event_name, 'pull_request') || (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
|
||||
&& (fromJSON(vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES_V2)[github.head_ref || github.ref_name] != null) }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Prepare notification message
|
||||
id: message
|
||||
run: |
|
||||
TEMPLATE='${{ vars.DISCOURSE_GENERAL_UPDATE_NOTICE }}'
|
||||
export VERSION="${{ needs.prepare_strategy.outputs.version }}"
|
||||
export branch_name="${{ env.SOURCE_BRANCH }}"
|
||||
export new_branch="${{ needs.prepare_strategy.outputs.new_branch }}"
|
||||
export commit_sha="${{ github.sha }}"
|
||||
export commit_short_sha="${{ github.sha }}"
|
||||
export commit_short_sha="${commit_short_sha:0:7}"
|
||||
export extra_version_identifier="${{ needs.prepare_strategy.outputs.extra_version_identifier || github.run_number }}"
|
||||
export PUBLIC_REPO_URL="${{ env.PUBLIC_REPO_URL }}"
|
||||
|
||||
MESSAGE=$(cat << 'EOF' | envsubst
|
||||
${{ vars.DISCOURSE_GENERAL_UPDATE_NOTICE }}
|
||||
EOF
|
||||
)
|
||||
|
||||
{
|
||||
echo 'content<<EOFMARKER'
|
||||
echo "$MESSAGE"
|
||||
echo 'EOFMARKER'
|
||||
} >> $GITHUB_OUTPUT
|
||||
shell: bash
|
||||
|
||||
- name: Post to Discourse
|
||||
uses: ./.github/workflows/post-to-discourse
|
||||
with:
|
||||
discourse-url: ${{ vars.DISCOURSE_URL }}
|
||||
api-key: ${{ secrets.DISCOURSE_API_KEY }}
|
||||
api-username: "system"
|
||||
topic-id: ${{ fromJSON(vars.DEV_FEEDBACK_NOTIFICATION_BRANCHES_V2)[github.head_ref || github.ref_name].topic_id }}
|
||||
message: ${{ steps.message.outputs.content }}
|
||||
|
||||
manage-pr-labels:
|
||||
name: Remove prebuilt label
|
||||
runs-on: ubuntu-latest
|
||||
if: (always() && contains(github.event_name, 'pull_request') && (github.event.action == 'labeled' && github.event.label.name == 'prebuilt'))
|
||||
env:
|
||||
LABEL: prebuilt
|
||||
steps:
|
||||
- name: Remove trust-fork-pr label if present
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
script: |
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
|
||||
await github.rest.issues.removeLabel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber,
|
||||
name: process.env.LABEL
|
||||
});
|
||||
|
||||
console.log(`Removed '${process.env.LABEL}' label from PR #${prNumber}`);
|
||||
@@ -1,243 +0,0 @@
|
||||
name: Build dev
|
||||
|
||||
env:
|
||||
DEFAULT_SOURCE_BRANCH: "master"
|
||||
DEFAULT_TARGET_BRANCH: "master-dev"
|
||||
LFS_URL: 'https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs'
|
||||
LFS_PUSH_URL: 'ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request_target:
|
||||
types: [ labeled ]
|
||||
branches:
|
||||
- 'master'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
source_branch:
|
||||
description: 'Source branch to reset from'
|
||||
required: true
|
||||
default: 'master'
|
||||
type: string
|
||||
target_branch:
|
||||
description: 'Target branch to reset and squash into'
|
||||
required: true
|
||||
default: 'master-dev'
|
||||
type: string
|
||||
cancel_in_progress:
|
||||
description: 'Cancel any in-progress runs of this workflow'
|
||||
required: false
|
||||
default: true
|
||||
type: boolean
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: ${{ inputs.cancel_in_progress || github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
|
||||
|
||||
jobs:
|
||||
reset-and-squash:
|
||||
runs-on: ubuntu-latest
|
||||
if: (
|
||||
(github.event_name == 'workflow_dispatch')
|
||||
|| (github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch))
|
||||
|| (contains(github.event_name, 'pull_request') && ((github.event.action == 'labeled' && (github.event.label.name == vars.PREBUILT_PR_LABEL || github.event.label.name == 'trust-fork-pr') && contains(github.event.pull_request.labels.*.name, vars.PREBUILT_PR_LABEL))))
|
||||
)
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0 # Fetch all history for all branches
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Wait for Tests
|
||||
uses: ./.github/workflows/wait-for-action # Path to where you place the action
|
||||
if: (
|
||||
(github.event_name == 'push' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch))
|
||||
|| (contains(github.event_name, 'pull_request') && ((github.event.action == 'labeled' && (github.event.label.name == vars.PREBUILT_PR_LABEL || github.event.label.name == 'trust-fork-pr') && contains(github.event.pull_request.labels.*.name, vars.PREBUILT_PR_LABEL))))
|
||||
)
|
||||
with:
|
||||
workflow: tests.yaml # The workflow file to monitor
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Configure Git
|
||||
run: |
|
||||
git config --global user.name 'github-actions[bot]'
|
||||
git config --global user.email 'github-actions[bot]@users.noreply.github.com'
|
||||
|
||||
- name: Set up SSH
|
||||
uses: webfactory/ssh-agent@v0.9.0
|
||||
with:
|
||||
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
|
||||
|
||||
- name: Add GitLab public keys
|
||||
run: |
|
||||
ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.10'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install PyGithub
|
||||
|
||||
- name: Check branches exist
|
||||
run: |
|
||||
# Check if source branch exists
|
||||
if ! git ls-remote --heads origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} | grep -q "${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"; then
|
||||
echo "Source branch ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }} does not exist!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Make sure we have the latest source branch
|
||||
git fetch origin ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
|
||||
|
||||
# Check if target branch exists
|
||||
if ! git ls-remote --heads origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} | grep -q "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"; then
|
||||
echo "Target branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} does not exist, creating it from ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
|
||||
git checkout -b ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
|
||||
git push origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
|
||||
else
|
||||
# Fetch target branch if it exists
|
||||
git fetch origin ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}
|
||||
fi
|
||||
|
||||
- name: Reset target branch
|
||||
run: |
|
||||
echo "Resetting ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} to match ${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
|
||||
# Delete if exists and recreate pointing to source
|
||||
git branch -D ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} || true
|
||||
git branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} origin/${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}
|
||||
|
||||
- name: Get PRs to squash
|
||||
id: get-prs
|
||||
run: |
|
||||
# Use GitHub API to get PRs with specific label, ordered by creation date
|
||||
PR_LIST=$(gh api graphql -f query='
|
||||
query($search_query:String!) {
|
||||
search(query: $search_query, type:ISSUE, first:40) {
|
||||
nodes {
|
||||
... on PullRequest {
|
||||
number
|
||||
headRefName
|
||||
title
|
||||
createdAt
|
||||
labels(last:10) {
|
||||
nodes {
|
||||
name
|
||||
}
|
||||
}
|
||||
headRepository {
|
||||
name
|
||||
nameWithOwner
|
||||
url
|
||||
isFork
|
||||
}
|
||||
commits(last: 1) {
|
||||
nodes {
|
||||
commit {
|
||||
statusCheckRollup {
|
||||
state
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}' -F search_query="repo:${{ github.repository }} is:pr is:open label:${{ vars.PREBUILT_PR_LABEL }},${{ vars.PREBUILT_PR_LABEL }}-c3 draft:false sort:created-asc")
|
||||
|
||||
PR_LIST=${PR_LIST//\'/}
|
||||
echo "PR_LIST=${PR_LIST}" >> $GITHUB_OUTPUT
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Process PRs
|
||||
run: |
|
||||
cp ${{ github.workspace }}/release/ci/squash_and_merge.py /tmp/squash_and_merge.py && \
|
||||
chmod +x /tmp/squash_and_merge.py && \
|
||||
python3 ${{ github.workspace }}/release/ci/squash_and_merge_prs.py \
|
||||
--pr-data '${{ steps.get-prs.outputs.PR_LIST }}' \
|
||||
--target-branch ${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }} \
|
||||
--squash-script-path '/tmp/squash_and_merge.py'
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Update LFS Config
|
||||
run: |
|
||||
echo '[lfs]' > .lfsconfig
|
||||
echo ' url = ${{ env.LFS_URL }}' >> .lfsconfig
|
||||
echo ' pushurl = ${{ env.LFS_PUSH_URL }}' >> .lfsconfig
|
||||
echo ' locksverify = false' >> .lfsconfig
|
||||
|
||||
- name: Restore workflows from source
|
||||
run: |
|
||||
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
|
||||
SOURCE_BRANCH="${{ inputs.source_branch || env.DEFAULT_SOURCE_BRANCH }}"
|
||||
|
||||
# Ensure we are on the target branch
|
||||
git checkout $TARGET_BRANCH
|
||||
|
||||
echo "Restoring .github/workflows from $SOURCE_BRANCH"
|
||||
git checkout origin/$SOURCE_BRANCH -- .github/workflows
|
||||
|
||||
if ! git diff --cached --quiet; then
|
||||
echo "Workflows differ. Committing restoration."
|
||||
git commit -m "chore: restore .github/workflows from $SOURCE_BRANCH"
|
||||
else
|
||||
echo "Workflows match $SOURCE_BRANCH."
|
||||
fi
|
||||
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: ci-token
|
||||
with:
|
||||
app-id: ${{ secrets.CI_GITHUB_ACTIONS_TOKEN_APP_ID }}
|
||||
private-key: ${{ secrets.CI_GITHUB_ACTIONS_TOKEN_APP_PRIVATE_KEY }}
|
||||
|
||||
- name: Push changes if there are diffs
|
||||
id: push-changes
|
||||
run: |
|
||||
TARGET_BRANCH="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
|
||||
|
||||
# Use the App Token to set the remote URL with authentication
|
||||
git remote set-url origin "https://x-access-token:${{ steps.ci-token.outputs.token }}@github.com/${{ github.repository }}.git"
|
||||
|
||||
# Fetch the latest from remote
|
||||
git fetch origin $TARGET_BRANCH
|
||||
|
||||
# Check for diffs between local and remote
|
||||
if git diff $TARGET_BRANCH origin/$TARGET_BRANCH --quiet; then
|
||||
echo "No changes to push - local and remote branches are identical"
|
||||
echo "has_changes=false" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Push with the authenticated origin
|
||||
if ! git push origin $TARGET_BRANCH --force; then
|
||||
echo "Failed to push changes to $TARGET_BRANCH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Branch $TARGET_BRANCH has been reset and updated with squashed PRs"
|
||||
echo "has_changes=true" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Trigger and wait for selfdrive tests
|
||||
if: steps.push-changes.outputs.has_changes == 'true'
|
||||
run: |
|
||||
echo "Triggering selfdrive tests..."
|
||||
gh workflow run tests.yaml --ref "${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}"
|
||||
|
||||
echo "Sleeping for 120s to give plenty of time for the action to start and then we wait"
|
||||
sleep 120
|
||||
|
||||
echo "Getting latest run ID..."
|
||||
RUN_ID=$(gh run list --workflow=tests.yaml --branch="${{ inputs.target_branch || env.DEFAULT_TARGET_BRANCH }}" --limit=1 --json databaseId --jq '.[0].databaseId')
|
||||
|
||||
echo "Watching run ID: $RUN_ID"
|
||||
gh run watch "$RUN_ID"
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -1,78 +0,0 @@
|
||||
name: Debug Discourse Posting
|
||||
|
||||
on:
|
||||
push:
|
||||
|
||||
jobs:
|
||||
test-discourse-post:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Post test message to Discourse
|
||||
uses: ./.github/workflows/post-to-discourse
|
||||
with:
|
||||
discourse-url: ${{ vars.DISCOURSE_URL }}
|
||||
api-key: ${{ secrets.DISCOURSE_API_KEY }}
|
||||
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
|
||||
topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
|
||||
message: |
|
||||
## 🧪 Test Post from GitHub Actions
|
||||
|
||||
**This is a test post to verify Discourse integration**
|
||||
|
||||
- **Workflow**: ${{ github.workflow }}
|
||||
- **Run Number**: #${{ github.run_number }}
|
||||
- **Branch**: `${{ github.ref_name }}`
|
||||
- **Commit**: ${{ github.sha }}
|
||||
- **Actor**: @${{ github.actor }}
|
||||
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
|
||||
|
||||
---
|
||||
|
||||
### Fake Build Info (for testing)
|
||||
- **Version**: 0.9.8-test
|
||||
- **Build**: #42
|
||||
- **Branch**: release-test
|
||||
|
||||
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
|
||||
|
||||
*This is an automated test message. Drive safe! 🚗💨*
|
||||
|
||||
|
||||
- name: Create topic on Discourse
|
||||
uses: ./.github/workflows/post-to-discourse
|
||||
with:
|
||||
discourse-url: ${{ vars.DISCOURSE_URL }}
|
||||
api-key: ${{ secrets.DISCOURSE_API_KEY }}
|
||||
api-username: ${{ secrets.DISCOURSE_API_USERNAME }}
|
||||
#topic-id: ${{ vars.DISCOURSE_UPDATES_TOPIC_ID }}
|
||||
category-id: 4
|
||||
title: "This is a test of a new topic instead of a reply"
|
||||
message: |
|
||||
## 🧪 Test Post from GitHub Actions
|
||||
|
||||
**This is a test post to verify Discourse integration**
|
||||
|
||||
- **Workflow**: ${{ github.workflow }}
|
||||
- **Run Number**: #${{ github.run_number }}
|
||||
- **Branch**: `${{ github.ref_name }}`
|
||||
- **Commit**: ${{ github.sha }}
|
||||
- **Actor**: @${{ github.actor }}
|
||||
- **Timestamp**: ${{ github.event.head_commit.timestamp }}
|
||||
|
||||
---
|
||||
|
||||
### Fake Build Info (for testing)
|
||||
- **Version**: 0.9.8-test
|
||||
- **Build**: #42
|
||||
- **Branch**: release-test
|
||||
|
||||
[View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
|
||||
|
||||
*This is an automated test message. Drive safe! 🚗💨*
|
||||
- name: Display results
|
||||
if: always()
|
||||
run: |
|
||||
echo "::notice::Discourse post test completed"
|
||||
echo "Check your Discourse topic to verify the post appeared correctly"
|
||||
@@ -1,248 +0,0 @@
|
||||
name: tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run_number:
|
||||
default: '1'
|
||||
required: true
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
group: tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
CI: 1
|
||||
PYTHONPATH: ${{ github.workspace }}
|
||||
PYTEST: pytest --continue-on-collection-errors --durations=0 -n logical
|
||||
|
||||
jobs:
|
||||
build_release:
|
||||
name: build release
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
env:
|
||||
STRIPPED_DIR: /tmp/releasepilot
|
||||
PYTHONPATH: /tmp/releasepilot
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- name: Getting LFS files
|
||||
uses: nick-fields/retry@7152eba30c6575329ac0576536151aca5a72780e
|
||||
with:
|
||||
timeout_minutes: 2
|
||||
max_attempts: 3
|
||||
command: git lfs pull
|
||||
- name: Build devel
|
||||
timeout-minutes: 1
|
||||
run: TARGET_DIR=$STRIPPED_DIR release/build_stripped.sh
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot and run checks
|
||||
timeout-minutes: 30
|
||||
working-directory: ${{ env.STRIPPED_DIR }}
|
||||
run: python3 system/manager/build.py
|
||||
- name: Run tests
|
||||
timeout-minutes: 1
|
||||
working-directory: ${{ env.STRIPPED_DIR }}
|
||||
run: release/check-dirty.sh
|
||||
- name: Check submodules
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
timeout-minutes: 3
|
||||
run: |
|
||||
if [ "${{ github.ref }}" != "refs/heads/master" ]; then
|
||||
git fetch origin master:refs/remotes/origin/master
|
||||
|
||||
SUBMODULE_PATHS=$(git diff origin/master HEAD --name-only | grep -E '^[^/]+$' | while read path; do
|
||||
if git ls-files --stage "$path" | grep -q "^160000"; then
|
||||
echo "$path"
|
||||
fi
|
||||
done | tr '\n' ' ')
|
||||
|
||||
if [ -n "$SUBMODULE_PATHS" ]; then
|
||||
echo "Changed submodule paths: $SUBMODULE_PATHS"
|
||||
export SUBMODULE_PATHS="$SUBMODULE_PATHS"
|
||||
export CHECK_PR_REFS=true
|
||||
fi
|
||||
fi
|
||||
release/check-submodules.sh
|
||||
|
||||
build_mac:
|
||||
name: build macOS
|
||||
runs-on: ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-macos-8x14' || 'macos-latest' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- name: Remove Homebrew from environment
|
||||
run: |
|
||||
FILTERED=$(echo "$PATH" | tr ':' '\n' | grep -v '/opt/homebrew' | tr '\n' ':')
|
||||
echo "PATH=${FILTERED}/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin" >> $GITHUB_ENV
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Building openpilot
|
||||
run: scons
|
||||
|
||||
static_analysis:
|
||||
name: static analysis
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Static analysis
|
||||
timeout-minutes: 1
|
||||
run: scripts/lint/lint.sh
|
||||
|
||||
unit_tests:
|
||||
name: unit tests
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
run: scons -j$(nproc)
|
||||
- name: Run unit tests
|
||||
timeout-minutes: ${{ contains(runner.name, 'nsc') && 2 || 999 }}
|
||||
run: |
|
||||
source selfdrive/test/setup_xvfb.sh
|
||||
# Pre-compile Python bytecode so each pytest worker doesn't need to
|
||||
$PYTEST --collect-only -m 'not slow' -qq
|
||||
MAX_EXAMPLES=1 $PYTEST -m 'not slow'
|
||||
|
||||
process_replay:
|
||||
name: process replay
|
||||
if: false # disable process_replay for forks
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
run: scons -j$(nproc)
|
||||
- name: Run replay
|
||||
timeout-minutes: ${{ contains(runner.name, 'nsc') && 2 || 20 }}
|
||||
continue-on-error: ${{ github.ref == 'refs/heads/master' }}
|
||||
run: selfdrive/test/process_replay/test_processes.py -j$(nproc)
|
||||
- name: Print diff
|
||||
id: print-diff
|
||||
if: always()
|
||||
run: cat selfdrive/test/process_replay/diff.txt
|
||||
- name: Print diff report
|
||||
if: always()
|
||||
run: cat selfdrive/test/process_replay/diff_report.txt
|
||||
- uses: actions/upload-artifact@v6
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: process_replay_diff.txt
|
||||
path: selfdrive/test/process_replay/diff.txt
|
||||
- name: Upload diff report
|
||||
uses: actions/upload-artifact@v6
|
||||
if: always() && github.event_name == 'pull_request'
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: diff_report_${{ github.event.number }}
|
||||
path: selfdrive/test/process_replay/diff_report.txt
|
||||
- name: Checkout ci-artifacts
|
||||
if: github.repository == 'commaai/openpilot' && github.ref == 'refs/heads/master'
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: commaai/ci-artifacts
|
||||
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
|
||||
path: ${{ github.workspace }}/ci-artifacts
|
||||
- name: Push refs
|
||||
if: github.repository == 'commaai/openpilot' && github.ref == 'refs/heads/master'
|
||||
working-directory: ${{ github.workspace }}/ci-artifacts
|
||||
run: |
|
||||
git config user.name "GitHub Actions Bot"
|
||||
git config user.email "<>"
|
||||
git fetch origin process-replay || true
|
||||
git checkout process-replay 2>/dev/null || git checkout --orphan process-replay
|
||||
cp ${{ github.workspace }}/selfdrive/test/process_replay/fakedata/*.zst .
|
||||
echo "${{ github.sha }}" > ref_commit
|
||||
git add .
|
||||
git commit -m "process-replay refs for ${{ github.repository }}@${{ github.sha }}" || echo "No changes to commit"
|
||||
git push origin process-replay --force
|
||||
- name: Run regen
|
||||
if: false
|
||||
timeout-minutes: 4
|
||||
env:
|
||||
ONNXCPU: 1
|
||||
run: $PYTEST selfdrive/test/process_replay/test_regen.py
|
||||
|
||||
simulator_driving:
|
||||
name: simulator driving
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
if: false # FIXME: Started to timeout recently
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
run: scons -j$(nproc)
|
||||
- name: Driving test
|
||||
timeout-minutes: 2
|
||||
run: |
|
||||
source selfdrive/test/setup_xvfb.sh
|
||||
pytest -s tools/sim/tests/test_metadrive_bridge.py
|
||||
|
||||
create_ui_report:
|
||||
name: Create UI Report
|
||||
runs-on: ${{
|
||||
(github.repository == 'commaai/openpilot') &&
|
||||
((github.event_name != 'pull_request') ||
|
||||
(github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
|
||||
&& fromJSON('["namespace-profile-amd64-8x16"]')
|
||||
|| fromJSON('["ubuntu-24.04"]') }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
- run: ./tools/op.sh setup
|
||||
- name: Build openpilot
|
||||
run: scons -j$(nproc)
|
||||
- name: Create UI Report
|
||||
run: |
|
||||
source selfdrive/test/setup_xvfb.sh
|
||||
python3 selfdrive/ui/tests/diff/replay.py
|
||||
python3 selfdrive/ui/tests/diff/replay.py --big
|
||||
- name: Upload UI Report
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: ui-report-${{ inputs.run_number || '1' }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
|
||||
path: selfdrive/ui/tests/diff/report
|
||||
@@ -1,175 +0,0 @@
|
||||
name: "ui preview"
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request_target:
|
||||
types: [assigned, opened, synchronize, reopened, edited]
|
||||
branches:
|
||||
- 'master'
|
||||
paths:
|
||||
- 'selfdrive/assets/**'
|
||||
- 'selfdrive/ui/**'
|
||||
- 'system/ui/**'
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
UI_JOB_NAME: "Create UI Report"
|
||||
REPORT_NAME: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
|
||||
SHA: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.sha || github.event.pull_request.head.sha }}
|
||||
BRANCH_NAME: "openpilot/pr-${{ github.event.number }}-ui-preview"
|
||||
REPORT_FILES_BRANCH_NAME: "mici-raylib-ui-reports"
|
||||
|
||||
# variant:video_prefix:master_branch
|
||||
VARIANTS: "mici:mici_ui_replay:openpilot_master_ui_mici_raylib big:tizi_ui_replay:openpilot_master_ui_big_raylib"
|
||||
|
||||
jobs:
|
||||
preview:
|
||||
if: github.repository == 'sunnypilot/sunnypilot'
|
||||
name: preview
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
actions: read
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Waiting for ui generation to end
|
||||
uses: lewagon/wait-on-check-action@v1.3.4
|
||||
with:
|
||||
ref: ${{ env.SHA }}
|
||||
check-name: ${{ env.UI_JOB_NAME }}
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
allowed-conclusions: success
|
||||
wait-interval: 20
|
||||
|
||||
- name: Getting workflow run ID
|
||||
id: get_run_id
|
||||
run: |
|
||||
echo "run_id=$(curl https://api.github.com/repos/${{ github.repository }}/commits/${{ env.SHA }}/check-runs | jq -r '.check_runs[] | select(.name == "${{ env.UI_JOB_NAME }}") | .html_url | capture("(?<number>[0-9]+)") | .number')" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Getting proposed ui
|
||||
uses: dawidd6/action-download-artifact@v6
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
run_id: ${{ steps.get_run_id.outputs.run_id }}
|
||||
search_artifacts: true
|
||||
name: ui-report-1-${{ env.REPORT_NAME }}
|
||||
path: ${{ github.workspace }}/pr_ui
|
||||
|
||||
- name: Getting mici master ui
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: sunnypilot/ci-artifacts
|
||||
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
|
||||
path: ${{ github.workspace }}/master_mici
|
||||
ref: openpilot_master_ui_mici_raylib
|
||||
|
||||
- name: Getting big master ui
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
repository: sunnypilot/ci-artifacts
|
||||
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
|
||||
path: ${{ github.workspace }}/master_big
|
||||
ref: openpilot_master_ui_big_raylib
|
||||
|
||||
- name: Saving new master ui
|
||||
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
|
||||
run: |
|
||||
for variant in $VARIANTS; do
|
||||
IFS=':' read -r name video branch <<< "$variant"
|
||||
master_dir="${{ github.workspace }}/master_${name}"
|
||||
cd "$master_dir"
|
||||
git checkout --orphan=new_branch
|
||||
git rm -rf *
|
||||
git branch -D "$branch"
|
||||
git branch -m "$branch"
|
||||
git config user.name "GitHub Actions Bot"
|
||||
git config user.email "<>"
|
||||
cp "${{ github.workspace }}/pr_ui/${video}.mp4" .
|
||||
git add .
|
||||
git commit -m "${name} video for commit ${{ env.SHA }}"
|
||||
git push origin "$branch" --force
|
||||
done
|
||||
|
||||
- name: Setup FFmpeg
|
||||
uses: AnimMouse/setup-ffmpeg@ae28d57dabbb148eff63170b6bf7f2b60062cbae
|
||||
|
||||
- name: Finding diffs
|
||||
if: github.event_name == 'pull_request_target'
|
||||
id: find_diff
|
||||
run: |
|
||||
export PYTHONPATH=${{ github.workspace }}
|
||||
baseurl="https://github.com/sunnypilot/ci-artifacts/raw/refs/heads/${{ env.BRANCH_NAME }}"
|
||||
|
||||
COMMENT=""
|
||||
for variant in $VARIANTS; do
|
||||
IFS=':' read -r name video _ <<< "$variant"
|
||||
diff_name="${name}_diff"
|
||||
|
||||
mv "${{ github.workspace }}/pr_ui/${video}.mp4" "${{ github.workspace }}/pr_ui/${video}_proposed.mp4"
|
||||
cp "${{ github.workspace }}/master_${name}/${video}.mp4" "${{ github.workspace }}/pr_ui/${video}_master.mp4"
|
||||
|
||||
diff_exit_code=0
|
||||
python3 ${{ github.workspace }}/selfdrive/ui/tests/diff/diff.py \
|
||||
"${{ github.workspace }}/pr_ui/${video}_master.mp4" \
|
||||
"${{ github.workspace }}/pr_ui/${video}_proposed.mp4" \
|
||||
"${diff_name}.html" --basedir "$baseurl" --no-open || diff_exit_code=$?
|
||||
|
||||
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.html" "${{ github.workspace }}/pr_ui/"
|
||||
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.mp4" "${{ github.workspace }}/pr_ui/"
|
||||
|
||||
REPORT_URL="https://sunnypilot.github.io/ci-artifacts/${diff_name}_pr_${{ github.event.number }}.html"
|
||||
if [ $diff_exit_code -eq 0 ]; then
|
||||
COMMENT+="**${name}**: Videos are identical! [View Diff Report]($REPORT_URL)"$'\n'
|
||||
else
|
||||
COMMENT+="**${name}**: ⚠️ <strong>Videos differ!</strong> [View Diff Report]($REPORT_URL)"$'\n'
|
||||
fi
|
||||
done
|
||||
|
||||
{
|
||||
echo "COMMENT<<EOF"
|
||||
echo "$COMMENT"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Saving proposed ui
|
||||
if: github.event_name == 'pull_request_target'
|
||||
working-directory: ${{ github.workspace }}/master_mici
|
||||
run: |
|
||||
git config user.name "GitHub Actions Bot"
|
||||
git config user.email "<>"
|
||||
git checkout --orphan=${{ env.BRANCH_NAME }}
|
||||
git rm -rf *
|
||||
mv ${{ github.workspace }}/pr_ui/* .
|
||||
git add .
|
||||
git commit -m "ui videos for PR #${{ github.event.number }}"
|
||||
git push origin ${{ env.BRANCH_NAME }} --force
|
||||
|
||||
# Append diff reports to report files branch
|
||||
git fetch origin ${{ env.REPORT_FILES_BRANCH_NAME }}
|
||||
git checkout ${{ env.REPORT_FILES_BRANCH_NAME }}
|
||||
for variant in $VARIANTS; do
|
||||
IFS=':' read -r name _ _ <<< "$variant"
|
||||
diff_name="${name}_diff"
|
||||
cp "${{ github.workspace }}/selfdrive/ui/tests/diff/report/${diff_name}.html" "${diff_name}_pr_${{ github.event.number }}.html"
|
||||
git add "${diff_name}_pr_${{ github.event.number }}.html"
|
||||
done
|
||||
git commit -m "ui diff reports for PR #${{ github.event.number }}" || echo "No changes to commit"
|
||||
git push origin ${{ env.REPORT_FILES_BRANCH_NAME }}
|
||||
|
||||
- name: Comment on PR
|
||||
if: github.event_name == 'pull_request_target'
|
||||
uses: thollander/actions-comment-pull-request@v2
|
||||
with:
|
||||
message: |
|
||||
<!-- _(run_id_ui_preview **${{ github.run_id }}**)_ -->
|
||||
## UI Preview
|
||||
${{ steps.find_diff.outputs.COMMENT }}
|
||||
comment_tag: run_id_ui_preview
|
||||
pr_number: ${{ github.event.number }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -1,52 +0,0 @@
|
||||
name: 'Wait for Tests'
|
||||
description: 'Action to wait for workflow tests to start and complete'
|
||||
inputs:
|
||||
workflow:
|
||||
description: 'The workflow file name to monitor'
|
||||
required: true
|
||||
default: 'tests.yaml'
|
||||
branch:
|
||||
description: 'The branch to monitor (defaults to current branch)'
|
||||
required: false
|
||||
default: ''
|
||||
github-token:
|
||||
description: 'GitHub token for API access'
|
||||
required: true
|
||||
wait-time:
|
||||
description: 'Initial sleep time in seconds before monitoring starts'
|
||||
required: false
|
||||
default: '30'
|
||||
should-wait-for-start:
|
||||
description: 'Whether to wait for tests to start'
|
||||
required: false
|
||||
default: false
|
||||
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Wait for tests to start
|
||||
if: inputs.should-wait-for-start == 'true'
|
||||
shell: bash
|
||||
run: |
|
||||
echo "Sleeping for ${{ inputs.wait-time }} seconds to give some time for the action to start and then we'll wait"
|
||||
sleep ${{ inputs.wait-time }}
|
||||
|
||||
- name: Wait for tests to finish
|
||||
shell: bash
|
||||
run: |
|
||||
BRANCH="${{ inputs.branch || github.head_ref || github.ref_name }}"
|
||||
|
||||
echo "Looking for workflow runs of ${{ inputs.workflow }} on branch $BRANCH"
|
||||
RUN_ID=$(gh run list --workflow=${{ inputs.workflow }} --branch="$BRANCH" --limit=1 --json databaseId --jq '.[0].databaseId')
|
||||
echo "Watching run ID: $RUN_ID"
|
||||
gh run watch "$RUN_ID"
|
||||
|
||||
CONCLUSION=$(gh run view "$RUN_ID" --json conclusion --jq '.conclusion')
|
||||
echo "Run concluded with: $CONCLUSION"
|
||||
|
||||
if [[ "$CONCLUSION" != "success" ]]; then
|
||||
echo "❌ Workflow run failed with conclusion: $CONCLUSION"
|
||||
exit 1
|
||||
fi
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ inputs.github-token }}
|
||||
@@ -44,8 +44,10 @@ bin/
|
||||
config.json
|
||||
compile_commands.json
|
||||
compare_runtime*.html
|
||||
selfdrive/modeld/models/tg_input_devices.json
|
||||
|
||||
# build artifacts
|
||||
docs_site/
|
||||
selfdrive/pandad/pandad
|
||||
cereal/services.h
|
||||
cereal/gen
|
||||
|
||||
-21
@@ -1,21 +0,0 @@
|
||||
[submodule "panda"]
|
||||
path = panda
|
||||
url = https://github.com/sunnyhaibin/panda.git
|
||||
[submodule "opendbc"]
|
||||
path = opendbc_repo
|
||||
url = https://github.com/sunnypilot/opendbc.git
|
||||
[submodule "msgq"]
|
||||
path = msgq_repo
|
||||
url = https://github.com/commaai/msgq.git
|
||||
[submodule "rednose_repo"]
|
||||
path = rednose_repo
|
||||
url = https://github.com/commaai/rednose.git
|
||||
[submodule "teleoprtc_repo"]
|
||||
path = teleoprtc_repo
|
||||
url = https://github.com/commaai/teleoprtc
|
||||
[submodule "tinygrad"]
|
||||
path = tinygrad_repo
|
||||
url = https://github.com/sunnypilot/tinygrad.git
|
||||
[submodule "sunnypilot/neural_network_data"]
|
||||
path = sunnypilot/neural_network_data
|
||||
url = https://github.com/sunnypilot/neural-network-data.git
|
||||
Generated
-25
@@ -1,25 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project version="4">
|
||||
<component name="CLionExternalBuildManager">
|
||||
<target id="a62f99e8-5ec4-434c-8122-49efed5af108" name="uv Scons Build Debug" defaultType="TOOL">
|
||||
<configuration id="b93ec964-16e5-4962-a12e-3ed360ce8f02" name="uv Scons Build Debug">
|
||||
<build type="TOOL">
|
||||
<tool actionId="Tool_External Tools_uv Scons Build Debug" />
|
||||
</build>
|
||||
<clean type="TOOL">
|
||||
<tool actionId="Tool_External Tools_uv Scons Clean" />
|
||||
</clean>
|
||||
</configuration>
|
||||
</target>
|
||||
<target id="edd8ad9d-183b-467c-a355-0d9a0ecab026" name="uv Scons Build Release" defaultType="TOOL">
|
||||
<configuration id="09523339-5ce3-4223-ab9e-904f38ad7752" name="uv Scons Build Release">
|
||||
<build type="TOOL">
|
||||
<tool actionId="Tool_External Tools_uv Scons Build Release" />
|
||||
</build>
|
||||
<clean type="TOOL">
|
||||
<tool actionId="Tool_External Tools_uv Scons Clean" />
|
||||
</clean>
|
||||
</configuration>
|
||||
</target>
|
||||
</component>
|
||||
</project>
|
||||
Generated
-23
@@ -1,23 +0,0 @@
|
||||
<toolSet name="External Tools">
|
||||
<tool name="uv Scons Build Debug" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
|
||||
<exec>
|
||||
<option name="COMMAND" value="bash" />
|
||||
<option name="PARAMETERS" value="-c "source .venv/bin/activate && scons -u -j$(nproc) --ccflags=\"-fno-inline\""" />
|
||||
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
|
||||
</exec>
|
||||
</tool>
|
||||
<tool name="uv Scons Clean" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
|
||||
<exec>
|
||||
<option name="COMMAND" value="bash" />
|
||||
<option name="PARAMETERS" value="-c "source .venv/bin/activate && scons -c" " />
|
||||
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
|
||||
</exec>
|
||||
</tool>
|
||||
<tool name="uv Scons Build Release" showInMainMenu="false" showInEditor="false" showInProject="false" showInSearchPopup="false" disabled="false" useConsole="true" showConsoleOnStdOut="false" showConsoleOnStdErr="false" synchronizeAfterRun="true">
|
||||
<exec>
|
||||
<option name="COMMAND" value="bash" />
|
||||
<option name="PARAMETERS" value="-c "source .venv/bin/activate && scons -u -j$(nproc)" " />
|
||||
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$" />
|
||||
</exec>
|
||||
</tool>
|
||||
</toolSet>
|
||||
@@ -1,4 +0,0 @@
|
||||
[lfs]
|
||||
url = https://gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git/info/lfs
|
||||
pushurl = ssh://git@gitlab.com/sunnypilot/public/sunnypilot-new-lfs.git
|
||||
locksverify = false
|
||||
@@ -1,4 +0,0 @@
|
||||
[lfs]
|
||||
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
|
||||
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
|
||||
locksverify = false
|
||||
@@ -1,10 +0,0 @@
|
||||
<component name="ProjectRunConfigurationManager">
|
||||
<configuration default="false" name="Build Debug" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Debug" CONFIG_NAME="uv Scons Build Debug" RUN_PATH="ui">
|
||||
<envs>
|
||||
<env name="QT_DBL_CLICK_DIST" value="150" />
|
||||
</envs>
|
||||
<method v="2">
|
||||
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
|
||||
</method>
|
||||
</configuration>
|
||||
</component>
|
||||
@@ -1,10 +0,0 @@
|
||||
<component name="ProjectRunConfigurationManager">
|
||||
<configuration default="false" name="Build Release" type="CLionExternalRunConfiguration" factoryName="Application" REDIRECT_INPUT="false" ELEVATE="false" USE_EXTERNAL_CONSOLE="false" EMULATE_TERMINAL="false" WORKING_DIR="file://$ProjectFileDir$/selfdrive/ui" PASS_PARENT_ENVS_2="true" PROJECT_NAME="sunnypilot" TARGET_NAME="uv Scons Build Release" CONFIG_NAME="uv Scons Build Release" RUN_PATH="ui">
|
||||
<envs>
|
||||
<env name="QT_DBL_CLICK_DIST" value="150" />
|
||||
</envs>
|
||||
<method v="2">
|
||||
<option name="CLION.EXTERNAL.BUILD" enabled="true" />
|
||||
</method>
|
||||
</configuration>
|
||||
</component>
|
||||
@@ -1,26 +0,0 @@
|
||||
<component name="ProjectRunConfigurationManager">
|
||||
<configuration default="false" name="Build_BIG_UI" type="PythonConfigurationType" factoryName="Python">
|
||||
<module name="sunnypilot" />
|
||||
<option name="ENV_FILES" value="" />
|
||||
<option name="INTERPRETER_OPTIONS" value="" />
|
||||
<option name="PARENT_ENVS" value="true" />
|
||||
<envs>
|
||||
<env name="BIG" value="1" />
|
||||
</envs>
|
||||
<option name="SDK_HOME" value="" />
|
||||
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$/" />
|
||||
<option name="IS_MODULE_SDK" value="true" />
|
||||
<option name="ADD_CONTENT_ROOTS" value="true" />
|
||||
<option name="ADD_SOURCE_ROOTS" value="true" />
|
||||
<option name="SCRIPT_NAME" value="$ProjectFileDir$/selfdrive/ui/ui.py" />
|
||||
<option name="PARAMETERS" value="" />
|
||||
<option name="SHOW_COMMAND_LINE" value="false" />
|
||||
<option name="EMULATE_TERMINAL" value="false" />
|
||||
<option name="MODULE_MODE" value="false" />
|
||||
<option name="REDIRECT_INPUT" value="false" />
|
||||
<option name="INPUT_FILE" value="" />
|
||||
<method v="2">
|
||||
<option name="ToolBeforeRunTask" enabled="true" actionId="Tool_External Tools_uv Scons Build Debug" />
|
||||
</method>
|
||||
</configuration>
|
||||
</component>
|
||||
@@ -1,23 +0,0 @@
|
||||
<component name="ProjectRunConfigurationManager">
|
||||
<configuration default="false" name="Build_SMALL_UI" type="PythonConfigurationType" factoryName="Python">
|
||||
<module name="sunnypilot" />
|
||||
<option name="ENV_FILES" value="" />
|
||||
<option name="INTERPRETER_OPTIONS" value="" />
|
||||
<option name="PARENT_ENVS" value="true" />
|
||||
<option name="SDK_HOME" value="" />
|
||||
<option name="WORKING_DIRECTORY" value="$ProjectFileDir$/" />
|
||||
<option name="IS_MODULE_SDK" value="true" />
|
||||
<option name="ADD_CONTENT_ROOTS" value="true" />
|
||||
<option name="ADD_SOURCE_ROOTS" value="true" />
|
||||
<option name="SCRIPT_NAME" value="$ProjectFileDir$/selfdrive/ui/ui.py" />
|
||||
<option name="PARAMETERS" value="" />
|
||||
<option name="SHOW_COMMAND_LINE" value="false" />
|
||||
<option name="EMULATE_TERMINAL" value="false" />
|
||||
<option name="MODULE_MODE" value="false" />
|
||||
<option name="REDIRECT_INPUT" value="false" />
|
||||
<option name="INPUT_FILE" value="" />
|
||||
<method v="2">
|
||||
<option name="ToolBeforeRunTask" enabled="true" actionId="Tool_External Tools_uv Scons Build Debug" />
|
||||
</method>
|
||||
</configuration>
|
||||
</component>
|
||||
Vendored
-1
@@ -21,7 +21,6 @@
|
||||
"common/**",
|
||||
"selfdrive/**",
|
||||
"system/**",
|
||||
"third_party/**",
|
||||
"tools/**",
|
||||
]
|
||||
}
|
||||
|
||||
+118
-1
@@ -1,4 +1,36 @@
|
||||
sunnypilot Version 2026.001.000 (2026-03-xx)
|
||||
sunnypilot Version 2026.002.000 (2026-06-28)
|
||||
========================
|
||||
* What's Changed (sunnypilot/sunnypilot)
|
||||
* ui: update gates for certain toggles by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1830
|
||||
* release: ignore upstream IsReleaseBranch by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1831
|
||||
* manager: disable DEVELOPMENT_ONLY reset by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1833
|
||||
* sunnylink: fix max time offroad values by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1835
|
||||
* ui: show default model name by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1837
|
||||
* sunnylink: add CarParams fallback for brand-specific capabilities by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1839
|
||||
* sunnylink SDUI: tweak DisableUpdate param for clarity by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1842
|
||||
* Revert "DM: Lancia Delta HF Integrale model" by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1849
|
||||
* modeld_v2: safe model validation by @Discountchubbs in https://github.com/sunnypilot/sunnypilot/pull/1855
|
||||
* Revert "deprecate `carState.brake`" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/sunnypilot/pull/1860
|
||||
* sunnylink: deprecate legacy params metadata by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1862
|
||||
* ui: reset Enforce Torque Control and NNLC if both are enabled by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1863
|
||||
* What's Changed (sunnypilot/opendbc)
|
||||
* Rivian: suppress ACM hold-the-wheel warning during MADS-only lateral by @lukasloetkolben in https://github.com/sunnypilot/opendbc/pull/465
|
||||
* Sync: `commaai/opendbc:master` → `sunnypilot/opendbc:master` by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/479
|
||||
* safety: add option to ignore frequency check for RX checks by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/480
|
||||
* Revert "deprecate carState.brake" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/opendbc/pull/481
|
||||
* New Contributors (sunnypilot/sunnypilot)
|
||||
* @mvl-boston made their first contribution in https://github.com/sunnypilot/sunnypilot/pull/1860
|
||||
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2026.001.007...v2026.002.000
|
||||
************************
|
||||
* Synced with commaai's openpilot (v0.11.1)
|
||||
* master commit 69e2c321e49760e52f7983eaa0a5f77cb95de637 (June 02, 2026)
|
||||
* New driver monitoring model
|
||||
* Improved image processing pipeline for driver camera
|
||||
* Improved thermal policy for comma four
|
||||
* Acura MDX 2022-24 support thanks to mvl-boston!
|
||||
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
|
||||
|
||||
sunnypilot Version 2026.001.000 (2026-05-06)
|
||||
========================
|
||||
* What's Changed (sunnypilot/sunnypilot)
|
||||
* Complete rewrite of the user interface from Qt C++ to Raylib Python
|
||||
@@ -66,6 +98,64 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
|
||||
* Pause Lateral Control with Blinker: Post-Blinker Delay by @CHaucke89
|
||||
* SCC-V: Use p97 for predicted lateral accel by @yasu-oh
|
||||
* Controls: Support for Torque Lateral Control v0 Tune by @sunnyhaibin
|
||||
* [TIZI/TICI] ui: ensure null checks for `CarParams` and `CarParamsSP` by @sunnyhaibin
|
||||
* [TIZI/TICI] ui: use `vCruiseCluster` and `vEgoCluster` for SLA `preActive` by @sunnyhaibin
|
||||
* Fix display of values when using use_float_scaling by @CHaucke89
|
||||
* models: fix default & index "0" by @nayan8teen
|
||||
* [TIZI/TICI] visuals: Improved speed limit by @angaz
|
||||
* ICBM: ensure button timers update on disable to clear stale presses by @jamesmikesell
|
||||
* [TIZI/TICI] ui: simplify Smart Cruise Control text rendering by @sunnyhaibin
|
||||
* controlsd: fix steer_limited_by_safety not updating under MADS by @zephleggett
|
||||
* soundd: trigger timeout warning during MADS lateral-only by @zephleggett
|
||||
* pandad: flasher for Rivian long upgrade module by @lukasloetkolben
|
||||
* modeld_v2: tinygrad transformation warp by @Discountchubbs
|
||||
* tools: block `manage_sunnylinkd` in sim startup script by @sunnyhaibin
|
||||
* [MICI] ui: need superclass `_render` in `HudRendererSP` by @sunnyhaibin
|
||||
* [TIZI/TICI] ui: Speed Limit Assist active status by @sunnyhaibin
|
||||
* ui: reimplement "Screen Off" option to Onroad Brightness by @sunnyhaibin
|
||||
* ui: don't hide steering wheel when blindspot disabled by @royjr
|
||||
* ui: Speed Limit Assist `preActive` improvements by @sunnyhaibin
|
||||
* ui: consolidate Speed Limit Assist `preActive` status rendering by @sunnyhaibin
|
||||
* [MICI] ui: Speed Limit Assist `preActive` status by @sunnyhaibin
|
||||
* sunnypilot modeld: remove thneed modeld by @Discountchubbs
|
||||
* modeld_v2: decouple planplus scaling from accel by @Discountchubbs
|
||||
* sunnylink: Handle exceptions in `getParamsAllKeysV1` to log crashes by @devtekve
|
||||
* [TIZI/TICI] ui: Developer UI cleanup by @sunnyhaibin
|
||||
* [TIZI/TICI] ui: dynamic alert size by @nayan8teen
|
||||
* i18n(fr): Add French translations by @didlawowo
|
||||
* Toyota: Stop and Go Hack (Alpha) by @sunnyhaibin
|
||||
* ui: `AlertFadeAnimator` for longitudinal-related statuses by @sunnyhaibin
|
||||
* pandad: gate unsupported pandas before flashing by @sunnyhaibin
|
||||
* Rivian: Flash xnor's Longitudinal Upgrade Kit prior supported panda check by @lukasloetkolben
|
||||
* [TIZI/TICI] ui: add back gate steering arc behind toggle by @sunnyhaibin
|
||||
* ui: gate Onroad Brightness Delay on readiness by @sunnyhaibin
|
||||
* ui: add new timer options for Onroad Brightness Delay by @sunnyhaibin
|
||||
* [TIZI/TICI] ui: branch switcher is always available by @sunnyhaibin
|
||||
* pandad: always prioritize internal panda by @sunnyhaibin
|
||||
* sunnylinkd: fetch compressed params schema by @sunnyhaibin
|
||||
* sunnypilot locationd: remove unused car_ekf filter by @sunnyhaibin
|
||||
* modeld_v2: update deprecated temporalPose ref by @sunnyhaibin
|
||||
* NNLC: restore pre-v1 PID gains in torque extension by @mmmorks
|
||||
* MADS safety: enable heartbeat and lateral controls mismatch checks by @sunnyhaibin
|
||||
* [MICI] ui: models panel enhancements by @nayan8teen
|
||||
* [TIZI/TICI] ui: fix unintended selection while scrolling in TreeOptionDialog by @TheSecurityDev
|
||||
* tools: script for video concatenation by @Discountchubbs
|
||||
* tools: profile memory usage by @Discountchubbs
|
||||
* [TIZI/TICI] ui: remove per-frame param sync by @sunnyhaibin
|
||||
* [MICI] ui: always offroad by @nayan8teen
|
||||
* controls: always default Torque Lateral Control to v0 Tune by @sunnyhaibin
|
||||
* Revert "controls: always default Torque Lateral Control to v0 Tune" by @sunnyhaibin
|
||||
* Reapply "controls: always default Torque Lateral Control to v0 Tune" (#1806) by @sunnyhaibin
|
||||
* [MICI] ui: add sunnylink info & connectivity check by @nayan8teen
|
||||
* sunnylink: Remove unused API endpoint by @devtekve
|
||||
* DM: wheel touch enforcement in MADS by @sunnyhaibin
|
||||
* torque: show static override values in Dev UI & gate `useParams` on custom torque tune by @sunnyhaibin
|
||||
* MADS: suppress espActive event when long is not engaged by @sunnyhaibin
|
||||
* sunnylink: SDUI by @sunnyhaibin
|
||||
* [MICI] ui: align upstream changes with sunnypilot settings buttons by @nayan8teen
|
||||
* ui: fix cellular toggles by @AmyJeanes
|
||||
* sunnylink: switch athena domain by @devtekve
|
||||
* Platform List: dynamically migrate CarPlatformBundle by @sunnyhaibin
|
||||
* What's Changed (sunnypilot/opendbc)
|
||||
* Honda: DBC for Accord 9th Generation by @mvl-boston
|
||||
* FCA: update tire stiffness values for `RAM_HD` by @dparring
|
||||
@@ -84,12 +174,25 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
|
||||
* Honda: add missing `GasInterceptor` messages to Taiwan Odyssey DBC by @mvl-boston
|
||||
* GM: remove `CHEVROLET_EQUINOX_NON_ACC_3RD_GEN` from `dashcamOnly` by @sunnyhaibin
|
||||
* GM: remove `CHEVROLET_BOLT_NON_ACC_2ND_GEN` from `dashcamOnly` by @sunnyhaibin
|
||||
* Hyundai Longitudinal: deprecate ramp update for dynamic tune by @Discountchubbs
|
||||
* Rivian: long upgrade messages on bus 1 by @lukasloetkolben
|
||||
* Toyota: Stop and Go Hack (Alpha) by @sunnyhaibin
|
||||
* Toyota: gate Smart DSU behind Alpha Longitudinal by @sunnyhaibin
|
||||
* Toyota: Gas Interceptor always set `standstill_req` by @sunnyhaibin
|
||||
* MADS safety: dedicated `controls_allowed_lateral` by @sunnyhaibin
|
||||
* Platform List: include community supported platforms by @sunnyhaibin
|
||||
* New Contributors (sunnypilot/sunnypilot)
|
||||
* @TheSecurityDev made their first contribution in "ui: fix sidebar scroll in UI screenshots"
|
||||
* @zikeji made their first contribution in "sunnylink: block remote modification of SSH key parameters"
|
||||
* @Candy0707 made their first contribution in "[TIZI/TICI] ui: Fix misaligned turn signals and blindspot indicators with sidebar"
|
||||
* @CHaucke89 made their first contribution in "Pause Lateral Control with Blinker: Post-Blinker Delay"
|
||||
* @yasu-oh made their first contribution in "SCC-V: Use p97 for predicted lateral accel"
|
||||
* @angaz made their first contribution in "[TIZI/TICI] visuals: Improved speed limit"
|
||||
* @jamesmikesell made their first contribution in "ICBM: ensure button timers update on disable to clear stale presses"
|
||||
* @zephleggett made their first contribution in "controlsd: fix steer_limited_by_safety not updating under MADS"
|
||||
* @lukasloetkolben made their first contribution in "pandad: flasher for Rivian long upgrade module"
|
||||
* @didlawowo made their first contribution in "i18n(fr): Add French translations"
|
||||
* @mmmorks made their first contribution in "NNLC: restore pre-v1 PID gains in torque extension"
|
||||
* New Contributors (sunnypilot/opendbc)
|
||||
* @AmyJeanes made their first contribution in "Tesla: Fix stock LKAS being blocked when MADS is enabled"
|
||||
* @mvl-boston made their first contribution in "Honda: Update Clarity brake to renamed DBC message name"
|
||||
@@ -99,6 +202,20 @@ sunnypilot Version 2026.001.000 (2026-03-xx)
|
||||
* @royjr made their first contribution in "HKG: add KIA_FORTE_2019_NON_SCC fingerprint"
|
||||
* @ssysm made their first contribution in "Tesla: remove `TESLA_MODEL_X` from `dashcamOnly`"
|
||||
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2025.002.000...v2026.001.000
|
||||
************************
|
||||
* Synced with commaai's openpilot (v0.11.1)
|
||||
* master commit c001f3c9b490a80e69539f0af6022f6e07ceb721 (April 16, 2026)
|
||||
* New driver monitoring model
|
||||
* Improved image processing pipeline for driver camera
|
||||
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
|
||||
* New driving model #36798
|
||||
* Fully trained using a learned simulator
|
||||
* Improved longitudinal performance in Experimental mode
|
||||
* Reduce comma four standby power usage by 77% to 52 mW
|
||||
* Kia K7 2017 support thanks to royjr!
|
||||
* Lexus LS 2018 support thanks to Hacheoy!
|
||||
* Improved inter-process communication memory efficiency
|
||||
* comma four support
|
||||
|
||||
sunnypilot Version 2025.002.000 (2025-11-06)
|
||||
========================
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
FROM ubuntu:24.04
|
||||
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends sudo tzdata locales && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
|
||||
ENV LANG=en_US.UTF-8
|
||||
ENV LANGUAGE=en_US:en
|
||||
ENV LC_ALL=en_US.UTF-8
|
||||
|
||||
ENV NVIDIA_VISIBLE_DEVICES=all
|
||||
ENV NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute
|
||||
|
||||
ARG USER=batman
|
||||
ARG USER_UID=1001
|
||||
RUN useradd -m -s /bin/bash -u $USER_UID $USER
|
||||
RUN usermod -aG sudo $USER
|
||||
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
|
||||
USER $USER
|
||||
|
||||
ENV OPENPILOT_PATH=/home/$USER/openpilot
|
||||
RUN mkdir -p ${OPENPILOT_PATH}
|
||||
WORKDIR ${OPENPILOT_PATH}
|
||||
|
||||
COPY --chown=$USER . ${OPENPILOT_PATH}/
|
||||
|
||||
ENV UV_BIN="/home/$USER/.local/bin/"
|
||||
ENV VIRTUAL_ENV=${OPENPILOT_PATH}/.venv
|
||||
ENV PATH="$UV_BIN:$VIRTUAL_ENV/bin:$PATH"
|
||||
RUN tools/setup_dependencies.sh && \
|
||||
sudo rm -rf /var/lib/apt/lists/*
|
||||
|
||||
USER root
|
||||
RUN git config --global --add safe.directory '*'
|
||||
Vendored
-260
@@ -1,260 +0,0 @@
|
||||
def retryWithDelay(int maxRetries, int delay, Closure body) {
|
||||
for (int i = 0; i < maxRetries; i++) {
|
||||
try {
|
||||
return body()
|
||||
} catch (Exception e) {
|
||||
sleep(delay)
|
||||
}
|
||||
}
|
||||
throw Exception("Failed after ${maxRetries} retries")
|
||||
}
|
||||
|
||||
def device(String ip, String step_label, String cmd) {
|
||||
withCredentials([file(credentialsId: 'id_rsa', variable: 'key_file')]) {
|
||||
def ssh_cmd = """
|
||||
ssh -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec /usr/bin/bash <<'END'
|
||||
|
||||
set -e
|
||||
|
||||
export TERM=xterm-256color
|
||||
|
||||
shopt -s huponexit # kill all child processes when the shell exits
|
||||
|
||||
export CI=1
|
||||
export PYTHONWARNINGS=error
|
||||
#export LOGPRINT=debug # this has gotten too spammy...
|
||||
export TEST_DIR=${env.TEST_DIR}
|
||||
export SOURCE_DIR=${env.SOURCE_DIR}
|
||||
export GIT_BRANCH=${env.GIT_BRANCH}
|
||||
export GIT_COMMIT=${env.GIT_COMMIT}
|
||||
export CI_ARTIFACTS_TOKEN=${env.CI_ARTIFACTS_TOKEN}
|
||||
export GITHUB_COMMENTS_TOKEN=${env.GITHUB_COMMENTS_TOKEN}
|
||||
export AZURE_TOKEN='${env.AZURE_TOKEN}'
|
||||
# only use 1 thread for tici tests since most require HIL
|
||||
export PYTEST_ADDOPTS="-n0 -s"
|
||||
|
||||
|
||||
export GIT_SSH_COMMAND="ssh -i /data/gitkey"
|
||||
|
||||
source ~/.bash_profile
|
||||
if [ -f /TICI ]; then
|
||||
source /etc/profile
|
||||
|
||||
rm -rf /tmp/tmp*
|
||||
rm -rf ~/.commacache
|
||||
rm -rf /dev/shm/*
|
||||
rm -rf /dev/tmp/tmp*
|
||||
|
||||
if ! systemctl is-active --quiet systemd-resolved; then
|
||||
echo "restarting resolved"
|
||||
sudo systemctl start systemd-resolved
|
||||
sleep 3
|
||||
fi
|
||||
|
||||
# restart aux USB
|
||||
if [ -e /sys/bus/usb/drivers/hub/3-0:1.0 ]; then
|
||||
echo "restarting aux usb"
|
||||
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/unbind
|
||||
sleep 0.5
|
||||
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/bind
|
||||
fi
|
||||
fi
|
||||
if [ -f /data/openpilot/launch_env.sh ]; then
|
||||
source /data/openpilot/launch_env.sh
|
||||
fi
|
||||
|
||||
ln -snf ${env.TEST_DIR} /data/pythonpath
|
||||
|
||||
cd ${env.TEST_DIR} || true
|
||||
time ${cmd}
|
||||
END"""
|
||||
|
||||
sh script: ssh_cmd, label: step_label
|
||||
}
|
||||
}
|
||||
|
||||
def deviceStage(String stageName, String deviceType, List extra_env, def steps) {
|
||||
stage(stageName) {
|
||||
if (currentBuild.result != null) {
|
||||
return
|
||||
}
|
||||
|
||||
if (isReplay()) {
|
||||
error("REPLAYING TESTS IS NOT ALLOWED. FIX THEM INSTEAD.")
|
||||
}
|
||||
|
||||
def extra = extra_env.collect { "export ${it}" }.join('\n');
|
||||
def branch = env.BRANCH_NAME ?: 'master';
|
||||
def gitDiff = sh returnStdout: true, script: 'curl -s -H "Authorization: Bearer ${GITHUB_COMMENTS_TOKEN}" https://api.github.com/repos/commaai/openpilot/compare/master...${GIT_BRANCH} | jq .files[].filename || echo "/"', label: 'Getting changes'
|
||||
|
||||
lock(resource: "", label: deviceType, inversePrecedence: true, variable: 'device_ip', quantity: 1, resourceSelectStrategy: 'random') {
|
||||
docker.image('ghcr.io/commaai/alpine-ssh').inside('--user=root') {
|
||||
timeout(time: 35, unit: 'MINUTES') {
|
||||
retry (3) {
|
||||
def date = sh(script: 'date', returnStdout: true).trim();
|
||||
device(device_ip, "set time", "date -s '" + date + "'")
|
||||
device(device_ip, "git checkout", extra + "\n" + readFile("selfdrive/test/setup_device_ci.sh"))
|
||||
}
|
||||
steps.each { item ->
|
||||
def name = item[0]
|
||||
def cmd = item[1]
|
||||
|
||||
def args = item[2]
|
||||
def diffPaths = args.diffPaths ?: []
|
||||
def cmdTimeout = args.timeout ?: 9999
|
||||
|
||||
if (branch != "master" && !branch.contains("__jenkins_loop_") && diffPaths && !hasPathChanged(gitDiff, diffPaths)) {
|
||||
println "Skipping ${name}: no changes in ${diffPaths}."
|
||||
return
|
||||
} else {
|
||||
timeout(time: cmdTimeout, unit: 'SECONDS') {
|
||||
device(device_ip, name, cmd)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def hasPathChanged(String gitDiff, List<String> paths) {
|
||||
for (path in paths) {
|
||||
if (gitDiff.contains(path)) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
def isReplay() {
|
||||
def replayClass = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
|
||||
return currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replayClass) }
|
||||
}
|
||||
|
||||
def setupCredentials() {
|
||||
withCredentials([
|
||||
string(credentialsId: 'azure_token', variable: 'AZURE_TOKEN'),
|
||||
]) {
|
||||
env.AZURE_TOKEN = "${AZURE_TOKEN}"
|
||||
}
|
||||
|
||||
withCredentials([
|
||||
string(credentialsId: 'ci_artifacts_pat', variable: 'CI_ARTIFACTS_TOKEN'),
|
||||
]) {
|
||||
env.CI_ARTIFACTS_TOKEN = "${CI_ARTIFACTS_TOKEN}"
|
||||
}
|
||||
|
||||
withCredentials([
|
||||
string(credentialsId: 'post_comments_github_pat', variable: 'GITHUB_COMMENTS_TOKEN'),
|
||||
]) {
|
||||
env.GITHUB_COMMENTS_TOKEN = "${GITHUB_COMMENTS_TOKEN}"
|
||||
}
|
||||
}
|
||||
|
||||
def step(String name, String cmd, Map args = [:]) {
|
||||
return [name, cmd, args]
|
||||
}
|
||||
|
||||
node {
|
||||
env.CI = "1"
|
||||
env.PYTHONWARNINGS = "error"
|
||||
env.TEST_DIR = "/data/openpilot"
|
||||
env.SOURCE_DIR = "/data/openpilot_source/"
|
||||
setupCredentials()
|
||||
|
||||
env.GIT_BRANCH = checkout(scm).GIT_BRANCH
|
||||
env.GIT_COMMIT = checkout(scm).GIT_COMMIT
|
||||
|
||||
def excludeBranches = ['__nightly', 'devel', 'devel-staging', 'release3', 'release3-staging',
|
||||
'release-tici', 'release-tizi', 'release-tizi-staging', 'release-mici-staging', 'testing-closet*', 'hotfix-*']
|
||||
def excludeRegex = excludeBranches.join('|').replaceAll('\\*', '.*')
|
||||
|
||||
if (env.BRANCH_NAME != 'master' && !env.BRANCH_NAME.contains('__jenkins_loop_')) {
|
||||
properties([
|
||||
disableConcurrentBuilds(abortPrevious: true)
|
||||
])
|
||||
}
|
||||
|
||||
try {
|
||||
if (env.BRANCH_NAME == 'devel-staging') {
|
||||
deviceStage("build release-tizi-staging", "tizi-needs-can", [], [
|
||||
step("build release-tizi-staging", "RELEASE_BRANCH=release-tizi-staging $SOURCE_DIR/release/build_release.sh && git push -f origin release-tizi-staging:release-mici-staging"),
|
||||
])
|
||||
}
|
||||
|
||||
if (env.BRANCH_NAME == '__nightly') {
|
||||
parallel (
|
||||
'nightly': {
|
||||
deviceStage("build nightly", "tizi-needs-can", [], [
|
||||
step("build nightly", "RELEASE_BRANCH=nightly $SOURCE_DIR/release/build_release.sh"),
|
||||
])
|
||||
},
|
||||
'nightly-dev': {
|
||||
deviceStage("build nightly-dev", "tizi-needs-can", [], [
|
||||
step("build nightly-dev", "PANDA_DEBUG_BUILD=1 RELEASE_BRANCH=nightly-dev $SOURCE_DIR/release/build_release.sh"),
|
||||
])
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
if (!env.BRANCH_NAME.matches(excludeRegex)) {
|
||||
parallel (
|
||||
'onroad tests': {
|
||||
deviceStage("onroad", "tizi-needs-can", ["UNSAFE=1"], [
|
||||
step("build openpilot", "cd system/manager && ./build.py"),
|
||||
step("check dirty", "release/check-dirty.sh"),
|
||||
step("onroad tests", "pytest selfdrive/test/test_onroad.py -s", [timeout: 60]),
|
||||
])
|
||||
},
|
||||
'HW + Unit Tests': {
|
||||
deviceStage("tizi-hardware", "tizi-common", ["UNSAFE=1"], [
|
||||
step("build", "cd system/manager && ./build.py"),
|
||||
step("test power draw", "pytest -s system/hardware/tici/tests/test_power_draw.py"),
|
||||
step("test encoder", "LD_LIBRARY_PATH=/usr/local/lib pytest system/loggerd/tests/test_encoder.py", [diffPaths: ["system/loggerd/"]]),
|
||||
step("test manager", "pytest system/manager/test/test_manager.py"),
|
||||
])
|
||||
},
|
||||
'camerad OX03C10': {
|
||||
deviceStage("OX03C10", "tizi-ox03c10", ["UNSAFE=1"], [
|
||||
step("build", "cd system/manager && ./build.py"),
|
||||
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py"),
|
||||
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 90]),
|
||||
])
|
||||
},
|
||||
'camerad OS04C10': {
|
||||
deviceStage("OS04C10", "tici-os04c10", ["UNSAFE=1"], [
|
||||
step("build", "cd system/manager && ./build.py"),
|
||||
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py"),
|
||||
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 90]),
|
||||
])
|
||||
},
|
||||
'sensord': {
|
||||
deviceStage("LSM + MMC", "tizi-lsmc", ["UNSAFE=1"], [
|
||||
step("build", "cd system/manager && ./build.py"),
|
||||
step("test sensord", "pytest system/sensord/tests/test_sensord.py"),
|
||||
])
|
||||
},
|
||||
'replay': {
|
||||
deviceStage("model-replay", "tizi-replay", ["UNSAFE=1"], [
|
||||
step("build", "cd system/manager && ./build.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
|
||||
step("model replay", "selfdrive/test/process_replay/model_replay.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
|
||||
])
|
||||
},
|
||||
'tizi': {
|
||||
deviceStage("tizi", "tizi", ["UNSAFE=1"], [
|
||||
step("build openpilot", "cd system/manager && ./build.py"),
|
||||
step("test pandad loopback", "pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
|
||||
step("test pandad spi", "pytest selfdrive/pandad/tests/test_pandad_spi.py"),
|
||||
step("test amp", "pytest system/hardware/tici/tests/test_amplifier.py"),
|
||||
step("test qcomgpsd", "pytest system/qcomgpsd/tests/test_qcomgpsd.py", [diffPaths: ["system/qcomgpsd/"]]),
|
||||
])
|
||||
},
|
||||
|
||||
)
|
||||
}
|
||||
} catch (Exception e) {
|
||||
currentBuild.result = 'FAILED'
|
||||
throw e
|
||||
}
|
||||
}
|
||||
+7
-1
@@ -1,7 +1,13 @@
|
||||
Version 0.11.1 (2026-04-22)
|
||||
Version 0.11.2 (2026-06-15)
|
||||
========================
|
||||
|
||||
|
||||
Version 0.11.1 (2026-05-18)
|
||||
========================
|
||||
* New driver monitoring model
|
||||
* Improved image processing pipeline for driver camera
|
||||
* Improved thermal policy for comma four
|
||||
* Acura MDX 2022-24 support thanks to mvl-boston!
|
||||
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
|
||||
|
||||
Version 0.11.0 (2026-03-17)
|
||||
|
||||
-269
@@ -1,269 +0,0 @@
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import sysconfig
|
||||
import platform
|
||||
import shlex
|
||||
import importlib
|
||||
import numpy as np
|
||||
|
||||
import SCons.Errors
|
||||
from SCons.Defaults import _stripixes
|
||||
|
||||
SCons.Warnings.warningAsException(True)
|
||||
|
||||
Decider('MD5-timestamp')
|
||||
|
||||
SetOption('num_jobs', max(1, int(os.cpu_count()/2)))
|
||||
|
||||
AddOption('--ccflags', action='store', type='string', default='', help='pass arbitrary flags over the command line')
|
||||
AddOption('--verbose', action='store_true', default=False, help='show full build commands')
|
||||
AddOption('--minimal',
|
||||
action='store_false',
|
||||
dest='extras',
|
||||
default=os.path.exists(File('#.gitattributes').abspath), # minimal by default on release branch (where there's no LFS)
|
||||
help='the minimum build to run openpilot. no tests, tools, etc.')
|
||||
|
||||
# Detect platform
|
||||
arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
|
||||
if platform.system() == "Darwin":
|
||||
arch = "Darwin"
|
||||
elif arch == "aarch64" and os.path.isfile('/TICI'):
|
||||
arch = "larch64"
|
||||
assert arch in [
|
||||
"larch64", # linux tici arm64
|
||||
"aarch64", # linux pc arm64
|
||||
"x86_64", # linux pc x64
|
||||
"Darwin", # macOS arm64 (x86 not supported)
|
||||
]
|
||||
|
||||
pkg_names = ['bzip2', 'capnproto', 'eigen', 'ffmpeg', 'libjpeg', 'libyuv', 'ncurses', 'zeromq', 'zstd']
|
||||
pkgs = [importlib.import_module(name) for name in pkg_names]
|
||||
|
||||
|
||||
# ***** enforce a whitelist of system libraries *****
|
||||
# this prevents silently relying on a 3rd party package,
|
||||
# e.g. apt-installed libusb. all libraries should either
|
||||
# be distributed with all Linux distros and macOS, or
|
||||
# vendored in commaai/dependencies.
|
||||
allowed_system_libs = {
|
||||
"EGL", "GLESv2", "GL",
|
||||
"Qt5Charts", "Qt5Core", "Qt5Gui", "Qt5Widgets",
|
||||
"dl", "drm", "gbm", "m", "pthread",
|
||||
}
|
||||
|
||||
def _resolve_lib(env, name):
|
||||
for d in env.Flatten(env.get('LIBPATH', [])):
|
||||
p = Dir(str(d)).abspath
|
||||
for ext in ('.a', '.so', '.dylib'):
|
||||
f = File(os.path.join(p, f'lib{name}{ext}'))
|
||||
if f.exists() or f.has_builder():
|
||||
return name
|
||||
if name in allowed_system_libs:
|
||||
return name
|
||||
raise SCons.Errors.UserError(f"Unexpected non-vendored library '{name}'")
|
||||
|
||||
def _libflags(target, source, env, for_signature):
|
||||
libs = []
|
||||
lp = env.subst('$LIBLITERALPREFIX')
|
||||
for lib in env.Flatten(env.get('LIBS', [])):
|
||||
if isinstance(lib, str):
|
||||
if os.sep in lib or lib.startswith('#'):
|
||||
libs.append(File(lib))
|
||||
elif lib.startswith('-') or (lp and lib.startswith(lp)):
|
||||
libs.append(lib)
|
||||
else:
|
||||
libs.append(_resolve_lib(env, lib))
|
||||
else:
|
||||
libs.append(lib)
|
||||
return _stripixes(env['LIBLINKPREFIX'], libs, env['LIBLINKSUFFIX'],
|
||||
env['LIBPREFIXES'], env['LIBSUFFIXES'], env, env['LIBLITERALPREFIX'])
|
||||
|
||||
env = Environment(
|
||||
ENV={
|
||||
"PATH": os.environ['PATH'],
|
||||
"PYTHONPATH": Dir("#").abspath + ':' + Dir(f"#third_party/acados").abspath,
|
||||
"ACADOS_SOURCE_DIR": Dir("#third_party/acados").abspath,
|
||||
"ACADOS_PYTHON_INTERFACE_PATH": Dir("#third_party/acados/acados_template").abspath,
|
||||
"TERA_PATH": Dir("#").abspath + f"/third_party/acados/{arch}/t_renderer"
|
||||
},
|
||||
CCFLAGS=[
|
||||
"-g",
|
||||
"-fPIC",
|
||||
"-O2",
|
||||
"-Wunused",
|
||||
"-Werror",
|
||||
"-Wshadow" if arch in ("Darwin", "larch64") else "-Wshadow=local",
|
||||
"-Wno-unknown-warning-option",
|
||||
"-Wno-inconsistent-missing-override",
|
||||
"-Wno-c99-designator",
|
||||
"-Wno-reorder-init-list",
|
||||
"-Wno-vla-cxx-extension",
|
||||
],
|
||||
CFLAGS=["-std=gnu11"],
|
||||
CXXFLAGS=["-std=c++1z"],
|
||||
CPPPATH=[
|
||||
"#",
|
||||
"#msgq",
|
||||
"#third_party",
|
||||
"#third_party/json11",
|
||||
"#third_party/linux/include",
|
||||
"#third_party/acados/include",
|
||||
"#third_party/acados/include/blasfeo/include",
|
||||
"#third_party/acados/include/hpipm/include",
|
||||
"#third_party/catch2/include",
|
||||
[x.INCLUDE_DIR for x in pkgs],
|
||||
],
|
||||
LIBPATH=[
|
||||
"#common",
|
||||
"#msgq_repo",
|
||||
"#third_party",
|
||||
"#selfdrive/pandad",
|
||||
"#rednose/helpers",
|
||||
f"#third_party/acados/{arch}/lib",
|
||||
[x.LIB_DIR for x in pkgs],
|
||||
],
|
||||
RPATH=[],
|
||||
CYTHONCFILESUFFIX=".cpp",
|
||||
COMPILATIONDB_USE_ABSPATH=True,
|
||||
REDNOSE_ROOT="#",
|
||||
tools=["default", "cython", "compilation_db", "rednose_filter"],
|
||||
toolpath=["#site_scons/site_tools", "#rednose_repo/site_scons/site_tools"],
|
||||
)
|
||||
if arch != "larch64":
|
||||
env['_LIBFLAGS'] = _libflags
|
||||
|
||||
# Arch-specific flags and paths
|
||||
if arch == "larch64":
|
||||
env["CC"] = "clang"
|
||||
env["CXX"] = "clang++"
|
||||
env.Append(LIBPATH=[
|
||||
"/usr/lib/aarch64-linux-gnu",
|
||||
])
|
||||
arch_flags = ["-D__TICI__", "-mcpu=cortex-a57", "-DQCOM2"]
|
||||
env.Append(CCFLAGS=arch_flags)
|
||||
env.Append(CXXFLAGS=arch_flags)
|
||||
elif arch == "Darwin":
|
||||
env.Append(LIBPATH=[
|
||||
"/System/Library/Frameworks/OpenGL.framework/Libraries",
|
||||
])
|
||||
env.Append(CCFLAGS=["-DGL_SILENCE_DEPRECATION"])
|
||||
env.Append(CXXFLAGS=["-DGL_SILENCE_DEPRECATION"])
|
||||
|
||||
_extra_cc = shlex.split(GetOption('ccflags') or '')
|
||||
if _extra_cc:
|
||||
env.Append(CCFLAGS=_extra_cc)
|
||||
|
||||
# no --as-needed on mac linker
|
||||
if arch != "Darwin":
|
||||
env.Append(LINKFLAGS=["-Wl,--as-needed", "-Wl,--no-undefined"])
|
||||
|
||||
# Shorter build output: show brief descriptions instead of full commands.
|
||||
# Full command lines are still printed on failure by scons.
|
||||
if not GetOption('verbose'):
|
||||
for action, short in (
|
||||
("CC", "CC"),
|
||||
("CXX", "CXX"),
|
||||
("LINK", "LINK"),
|
||||
("SHCC", "CC"),
|
||||
("SHCXX", "CXX"),
|
||||
("SHLINK", "LINK"),
|
||||
("AR", "AR"),
|
||||
("RANLIB", "RANLIB"),
|
||||
("AS", "AS"),
|
||||
):
|
||||
env[f"{action}COMSTR"] = f" [{short}] $TARGET"
|
||||
|
||||
# progress output
|
||||
node_interval = 5
|
||||
node_count = 0
|
||||
def progress_function(node):
|
||||
global node_count
|
||||
node_count += node_interval
|
||||
sys.stderr.write("progress: %d\n" % node_count)
|
||||
if os.environ.get('SCONS_PROGRESS'):
|
||||
Progress(progress_function, interval=node_interval)
|
||||
|
||||
# ********** Cython build environment **********
|
||||
envCython = env.Clone()
|
||||
envCython["CPPPATH"] += [sysconfig.get_paths()['include'], np.get_include()]
|
||||
envCython["CCFLAGS"] += ["-Wno-#warnings", "-Wno-cpp", "-Wno-shadow", "-Wno-deprecated-declarations"]
|
||||
envCython["CCFLAGS"].remove("-Werror")
|
||||
|
||||
envCython["LIBS"] = []
|
||||
if arch == "Darwin":
|
||||
envCython["LINKFLAGS"] = env["LINKFLAGS"] + ["-bundle", "-undefined", "dynamic_lookup"]
|
||||
else:
|
||||
envCython["LINKFLAGS"] = ["-pthread", "-shared"]
|
||||
|
||||
np_version = SCons.Script.Value(np.__version__)
|
||||
Export('envCython', 'np_version')
|
||||
|
||||
Export('env', 'arch')
|
||||
|
||||
# Setup cache dir
|
||||
default_cache_dir = '/data/scons_cache' if arch == "larch64" else '/tmp/scons_cache'
|
||||
cache_dir = ARGUMENTS.get('cache_dir', default_cache_dir)
|
||||
CacheDir(cache_dir)
|
||||
Clean(["."], cache_dir)
|
||||
|
||||
# ********** start building stuff **********
|
||||
|
||||
# Build common module
|
||||
SConscript(['common/SConscript'])
|
||||
Import('_common')
|
||||
common = [_common, 'json11', 'zmq']
|
||||
Export('common')
|
||||
|
||||
# Build messaging (cereal + msgq + socketmaster + their dependencies)
|
||||
# Enable swaglog include in submodules
|
||||
env_swaglog = env.Clone()
|
||||
env_swaglog['CXXFLAGS'].append('-DSWAGLOG="\\"common/swaglog.h\\""')
|
||||
SConscript(['msgq_repo/SConscript'], exports={'env': env_swaglog})
|
||||
|
||||
SConscript(['cereal/SConscript'])
|
||||
|
||||
Import('socketmaster', 'msgq')
|
||||
messaging = [socketmaster, msgq, 'capnp', 'kj',]
|
||||
Export('messaging')
|
||||
|
||||
|
||||
# Build other submodules
|
||||
SConscript(['panda/SConscript'])
|
||||
|
||||
# Build rednose library
|
||||
SConscript(['rednose/SConscript'])
|
||||
|
||||
# Build system services
|
||||
SConscript([
|
||||
'system/loggerd/SConscript',
|
||||
])
|
||||
|
||||
if arch == "larch64":
|
||||
SConscript(['system/camerad/SConscript'])
|
||||
|
||||
# Build openpilot
|
||||
SConscript(['third_party/SConscript'])
|
||||
|
||||
# Build selfdrive
|
||||
SConscript([
|
||||
'selfdrive/pandad/SConscript',
|
||||
'selfdrive/controls/lib/lateral_mpc_lib/SConscript',
|
||||
'selfdrive/controls/lib/longitudinal_mpc_lib/SConscript',
|
||||
'selfdrive/locationd/SConscript',
|
||||
'selfdrive/modeld/SConscript',
|
||||
'selfdrive/ui/SConscript',
|
||||
])
|
||||
|
||||
SConscript(['sunnypilot/SConscript'])
|
||||
|
||||
# Build tools
|
||||
if arch != "larch64":
|
||||
SConscript([
|
||||
'tools/replay/SConscript',
|
||||
'tools/cabana/SConscript',
|
||||
'tools/jotpluggler/SConscript',
|
||||
])
|
||||
|
||||
|
||||
env.CompilationDatabase('compile_commands.json')
|
||||
@@ -1,20 +0,0 @@
|
||||
Import('env', 'common', 'msgq')
|
||||
|
||||
cereal_dir = Dir('.')
|
||||
gen_dir = Dir('gen')
|
||||
|
||||
# Build cereal
|
||||
schema_files = ['log.capnp', 'car.capnp', 'deprecated.capnp', 'custom.capnp']
|
||||
env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files],
|
||||
schema_files,
|
||||
f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/")
|
||||
|
||||
cereal = env.Library('cereal', [f'gen/cpp/{s}.c++' for s in schema_files])
|
||||
|
||||
# Build messaging
|
||||
services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET')
|
||||
env.Program('messaging/bridge', ['messaging/bridge.cc', 'messaging/msgq_to_zmq.cc', 'messaging/bridge_zmq.cc'], LIBS=[msgq, common, 'pthread'])
|
||||
|
||||
socketmaster = env.Library('socketmaster', ['messaging/socketmaster.cc'])
|
||||
|
||||
Export('cereal', 'socketmaster')
|
||||
@@ -760,18 +760,6 @@ struct LateralLQRState @0x9024e2d790c82ade {
|
||||
steeringAngleDesiredDeg @6 :Float32;
|
||||
}
|
||||
|
||||
struct LateralCurvatureState @0xad9d8095c06f7c61 {
|
||||
active @0 :Bool;
|
||||
actualCurvature @1 :Float32;
|
||||
desiredCurvature @2 :Float32;
|
||||
error @3 :Float32;
|
||||
p @4 :Float32;
|
||||
i @5 :Float32;
|
||||
f @6 :Float32;
|
||||
output @7 :Float32;
|
||||
saturated @8 :Bool;
|
||||
}
|
||||
|
||||
struct LateralPlannerSolution @0x84caeca5a6b4acfe {
|
||||
x @0 :List(Float32);
|
||||
y @1 :List(Float32);
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+113
-29
@@ -273,11 +273,7 @@ struct GPSNMEAData {
|
||||
nmea @2 :Text;
|
||||
}
|
||||
|
||||
# android sensor_event_t
|
||||
struct SensorEventData {
|
||||
version @0 :Int32;
|
||||
sensor @1 :Int32;
|
||||
type @2 :Int32;
|
||||
timestamp @3 :Int64;
|
||||
|
||||
union {
|
||||
@@ -296,7 +292,10 @@ struct SensorEventData {
|
||||
|
||||
struct SensorVec {
|
||||
v @0 :List(Float32);
|
||||
status @1 :Int8;
|
||||
|
||||
deprecated :group {
|
||||
status @1 :Int8;
|
||||
}
|
||||
}
|
||||
|
||||
enum SensorSource {
|
||||
@@ -314,7 +313,11 @@ struct SensorEventData {
|
||||
mmc5603nj @11;
|
||||
}
|
||||
|
||||
# formerly based on android sensor_event_t
|
||||
deprecated :group {
|
||||
version @0 :Int32;
|
||||
sensor @1 :Int32;
|
||||
type @2 :Int32;
|
||||
uncalibrated @10 :Bool;
|
||||
}
|
||||
}
|
||||
@@ -457,10 +460,10 @@ struct DeviceState @0xa4d8b5af2aa492eb {
|
||||
}
|
||||
|
||||
enum ThermalStatus {
|
||||
green @0;
|
||||
yellow @1;
|
||||
red @2;
|
||||
danger @3;
|
||||
ok @0;
|
||||
warmDEPRECATED @1;
|
||||
overheated @2;
|
||||
critical @3;
|
||||
}
|
||||
|
||||
enum NetworkType {
|
||||
@@ -815,7 +818,7 @@ struct ControlsState @0x97ff69c53601abf1 {
|
||||
debugState @59 :LateralDebugState;
|
||||
torqueState @60 :LateralTorqueState;
|
||||
|
||||
curvatureStateDEPRECATED @65 :Deprecated.LateralCurvatureState;
|
||||
curvatureState @65 :LateralCurvatureState;
|
||||
lqrStateDEPRECATED @55 :Deprecated.LateralLQRState;
|
||||
indiStateDEPRECATED @52 :Deprecated.LateralINDIState;
|
||||
}
|
||||
@@ -864,6 +867,18 @@ struct ControlsState @0x97ff69c53601abf1 {
|
||||
saturated @3 :Bool;
|
||||
}
|
||||
|
||||
struct LateralCurvatureState @0xad9d8095c06f7c61 {
|
||||
active @0 :Bool;
|
||||
actualCurvature @1 :Float32;
|
||||
desiredCurvature @2 :Float32;
|
||||
error @3 :Float32;
|
||||
p @4 :Float32;
|
||||
i @5 :Float32;
|
||||
f @6 :Float32;
|
||||
output @7 :Float32;
|
||||
saturated @8 :Bool;
|
||||
}
|
||||
|
||||
deprecated :group {
|
||||
vEgo @0 :Float32;
|
||||
vEgoRaw @32 :Float32;
|
||||
@@ -2054,16 +2069,15 @@ struct DriverStateV2 {
|
||||
facePosition @2 :List(Float32);
|
||||
facePositionStd @3 :List(Float32);
|
||||
faceProb @4 :Float32;
|
||||
eyesVisibleProb @14 :Float32;
|
||||
eyesClosedProb @15 :Float32;
|
||||
leftEyeProb @5 :Float32;
|
||||
rightEyeProb @6 :Float32;
|
||||
leftBlinkProb @7 :Float32;
|
||||
rightBlinkProb @8 :Float32;
|
||||
sunglassesProb @9 :Float32;
|
||||
phoneProb @13 :Float32;
|
||||
sleepProb @14 :Float32;
|
||||
|
||||
deprecated :group {
|
||||
leftEyeProb @5 :Float32;
|
||||
rightEyeProb @6 :Float32;
|
||||
leftBlinkProb @7 :Float32;
|
||||
rightBlinkProb @8 :Float32;
|
||||
sunglassesProb @9 :Float32;
|
||||
notReadyProb @12 :List(Float32);
|
||||
occludedProb @10 :Float32;
|
||||
readyProb @11 :List(Float32);
|
||||
@@ -2076,7 +2090,7 @@ struct DriverStateV2 {
|
||||
}
|
||||
}
|
||||
|
||||
struct DriverMonitoringState @0xb83cda094a1da284 {
|
||||
struct DriverMonitoringStateDEPRECATED @0xb83cda094a1da284 {
|
||||
events @18 :List(OnroadEvent);
|
||||
faceDetected @1 :Bool;
|
||||
isDistracted @2 :Bool;
|
||||
@@ -2104,6 +2118,75 @@ struct DriverMonitoringState @0xb83cda094a1da284 {
|
||||
}
|
||||
}
|
||||
|
||||
struct DriverMonitoringState {
|
||||
lockout @0 :Bool;
|
||||
alertCountLockoutPercent @1 :Int8;
|
||||
alertTimeLockoutPercent @2 :Int8;
|
||||
|
||||
alwaysOn @3 :Bool;
|
||||
alwaysOnLockout @4 :Bool;
|
||||
|
||||
alertLevel @5 :AlertLevel;
|
||||
activePolicy @6 :MonitoringPolicy;
|
||||
isRHD @7 :Bool;
|
||||
rhdCalibration @8 :CalibrationState;
|
||||
|
||||
visionPolicyState @9 :VisionPolicyState;
|
||||
wheeltouchPolicyState @10 :WheeltouchPolicyState;
|
||||
|
||||
enum AlertLevel {
|
||||
# ordinal must match the name to prevent bugs
|
||||
# comparing against the raw ordinal value
|
||||
none @0;
|
||||
one @1;
|
||||
two @2;
|
||||
three @3;
|
||||
}
|
||||
|
||||
enum MonitoringPolicy {
|
||||
wheeltouch @0;
|
||||
vision @1;
|
||||
}
|
||||
|
||||
struct VisionPolicyState {
|
||||
awarenessPercent @0 :Int8;
|
||||
awarenessStep @1 :Float32;
|
||||
isDistracted @2 :Bool;
|
||||
distractedTypes @3 :DistractedTypes;
|
||||
|
||||
faceDetected @4 :Bool;
|
||||
pose @5 :Pose;
|
||||
wheeltouchFallbackPercent @6 :Int8;
|
||||
uncertainOffroadAlertPercent @7 :Int8;
|
||||
|
||||
struct DistractedTypes {
|
||||
pose @0: Bool;
|
||||
eye @1: Bool;
|
||||
phone @2: Bool;
|
||||
}
|
||||
|
||||
struct Pose {
|
||||
pitch @0 :Float32;
|
||||
yaw @1 :Float32;
|
||||
pitchCalib @2 :CalibrationState;
|
||||
yawCalib @3 :CalibrationState;
|
||||
calibrated @4 :Bool;
|
||||
uncertainty @5 :Float32;
|
||||
}
|
||||
}
|
||||
|
||||
struct WheeltouchPolicyState {
|
||||
awarenessPercent @0 :Int8;
|
||||
awarenessStep @1 :Float32;
|
||||
driverInteracting @2 :Bool;
|
||||
}
|
||||
|
||||
struct CalibrationState {
|
||||
calibratedPercent @0 :Int8;
|
||||
offset @1 :Float32;
|
||||
}
|
||||
}
|
||||
|
||||
struct Boot {
|
||||
wallTimeNanos @0 :UInt64;
|
||||
pstore @4 :Map(Text, Data);
|
||||
@@ -2176,6 +2259,7 @@ struct LiveDelayData {
|
||||
lateralDelayEstimateStd @5 :Float32;
|
||||
points @4 :List(Float32);
|
||||
calPerc @6 :Int8;
|
||||
version @7 :Int32;
|
||||
|
||||
enum Status {
|
||||
unestimated @0;
|
||||
@@ -2229,7 +2313,8 @@ struct Sentinel {
|
||||
}
|
||||
|
||||
struct UIDebug {
|
||||
drawTimeMillis @0 :Float32;
|
||||
cpuTimeMillis @0 :Float32;
|
||||
frameTimeMillis @1 :Float32;
|
||||
}
|
||||
|
||||
struct ManagerState {
|
||||
@@ -2377,7 +2462,6 @@ struct Event {
|
||||
boot @60 :Boot;
|
||||
|
||||
# ********** openpilot daemon msgs **********
|
||||
gpsNMEA @3 :GPSNMEAData;
|
||||
can @5 :List(CanData);
|
||||
controlsState @7 :ControlsState;
|
||||
selfdriveState @130 :SelfdriveState;
|
||||
@@ -2402,7 +2486,6 @@ struct Event {
|
||||
qcomGnss @31 :QcomGnss;
|
||||
gpsLocationExternal @48 :GpsLocationData;
|
||||
gpsLocation @21 :GpsLocationData;
|
||||
gnssMeasurements @91 :GnssMeasurements;
|
||||
liveParameters @61 :LiveParametersData;
|
||||
liveTorqueParameters @94 :LiveTorqueParametersData;
|
||||
liveDelay @146 : LiveDelayData;
|
||||
@@ -2410,7 +2493,7 @@ struct Event {
|
||||
thumbnail @66: Thumbnail;
|
||||
onroadEvents @134: List(OnroadEvent);
|
||||
carParams @69: Car.CarParams;
|
||||
driverMonitoringState @71: DriverMonitoringState;
|
||||
driverMonitoringState @151 :DriverMonitoringState;
|
||||
livePose @129 :LivePose;
|
||||
modelV2 @75 :ModelDataV2;
|
||||
drivingModelData @128 :DrivingModelData;
|
||||
@@ -2436,7 +2519,6 @@ struct Event {
|
||||
# systems stuff
|
||||
androidLog @20 :AndroidLogEntry;
|
||||
managerState @78 :ManagerState;
|
||||
uploaderState @79 :UploaderState;
|
||||
procLog @33 :ProcLog;
|
||||
clocks @35 :Clocks;
|
||||
deviceState @6 :DeviceState;
|
||||
@@ -2446,12 +2528,6 @@ struct Event {
|
||||
# touch frame
|
||||
touch @135 :List(Touch);
|
||||
|
||||
# navigation
|
||||
navInstruction @82 :NavInstruction;
|
||||
navRoute @83 :NavRoute;
|
||||
navThumbnail @84: Thumbnail;
|
||||
mapRenderState @105: MapRenderState;
|
||||
|
||||
# UI services
|
||||
uiDebug @102 :UIDebug;
|
||||
|
||||
@@ -2553,5 +2629,13 @@ struct Event {
|
||||
gyroscope2DEPRECATED @100 :SensorEventData;
|
||||
accelerometer2DEPRECATED @101 :SensorEventData;
|
||||
temperatureSensor2DEPRECATED @123 :SensorEventData;
|
||||
driverMonitoringStateDEPRECATED @71 :DriverMonitoringStateDEPRECATED;
|
||||
gpsNMEADEPRECATED @3 :GPSNMEAData;
|
||||
uploaderStateDEPRECATED @79 :UploaderState;
|
||||
navInstructionDEPRECATED @82 :NavInstruction;
|
||||
navRouteDEPRECATED @83 :NavRoute;
|
||||
navThumbnailDEPRECATED @84 :Thumbnail;
|
||||
gnssMeasurementsDEPRECATED @91 :GnssMeasurements;
|
||||
mapRenderStateDEPRECATED @105: MapRenderState;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -259,11 +259,11 @@ class PubMaster:
|
||||
self.sock[s].send(dat)
|
||||
|
||||
def wait_for_readers_to_update(self, s: str, timeout: int, dt: float = 0.05) -> bool:
|
||||
for _ in range(int(timeout*(1./dt))):
|
||||
if self.sock[s].all_readers_updated():
|
||||
return True
|
||||
time.sleep(dt)
|
||||
return False
|
||||
try:
|
||||
self.sock[s].wait_for_readers(timeout=timeout, interval=dt)
|
||||
return True
|
||||
except TimeoutError:
|
||||
return False
|
||||
|
||||
def all_readers_updated(self, s: str) -> bool:
|
||||
return self.sock[s].all_readers_updated() # type: ignore
|
||||
return self.sock[s].all_readers_updated()
|
||||
|
||||
Executable
BIN
Binary file not shown.
@@ -30,7 +30,7 @@ def zmq_sleep(t=1):
|
||||
|
||||
# TODO: this should take any capnp struct and returrn a msg with random populated data
|
||||
def random_carstate():
|
||||
fields = ["vEgo", "aEgo", "brake", "steeringAngleDeg"]
|
||||
fields = ["vEgo", "aEgo", "steeringTorque", "steeringAngleDeg"]
|
||||
msg = messaging.new_message("carState")
|
||||
cs = msg.carState
|
||||
for f in fields:
|
||||
|
||||
@@ -1,222 +1,165 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate sunnypilot routes are parseable by stock commaai/openpilot.
|
||||
|
||||
Cap'n Proto is wire-compatible across renames, type relocations, and
|
||||
additive fields. The only breaking change is a union variant that
|
||||
upstream doesn't recognize — an unknown discriminant makes the entire
|
||||
union unreadable.
|
||||
|
||||
This script checks: for every struct with a union that exists in both
|
||||
schemas, does sunnypilot introduce union variants upstream doesn't have?
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from typing import Any, List, Tuple
|
||||
from typing import Any
|
||||
|
||||
DEBUG = False
|
||||
NO_DISCRIMINANT = 0xFFFF
|
||||
|
||||
|
||||
def print_debug(string: str) -> None:
|
||||
if DEBUG:
|
||||
print(string)
|
||||
def hex_id(value: int) -> str:
|
||||
return f"0x{value:016x}"
|
||||
|
||||
|
||||
def create_schema_instance(struct: Any, prop: Tuple[str, Any]) -> Any:
|
||||
"""
|
||||
Create a new instance of a schema type, handling different field types.
|
||||
|
||||
Args:
|
||||
struct: The Cap'n Proto schema structure
|
||||
prop: A tuple containing the field name and field metadata
|
||||
|
||||
Returns:
|
||||
A new initialized schema instance
|
||||
"""
|
||||
struct_instance = struct.new_message()
|
||||
field_name, field_metadata = prop
|
||||
|
||||
try:
|
||||
field_type = field_metadata.proto.slot.type.which()
|
||||
|
||||
# Initialize different types of fields
|
||||
if field_type in ('list', 'text', 'data'):
|
||||
struct_instance.init(field_name, 1)
|
||||
print_debug(f"Initialized list/text/data field: {field_name}")
|
||||
elif field_type in ('struct', 'object'):
|
||||
struct_instance.init(field_name)
|
||||
print_debug(f"Initialized struct/object field: {field_name}")
|
||||
|
||||
return struct_instance
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error creating instance for {field_name}: {e}")
|
||||
return None
|
||||
def encode_struct(schema: Any) -> dict:
|
||||
node = schema.node
|
||||
fields = []
|
||||
for name, field in schema.fields.items():
|
||||
proto = field.proto
|
||||
ordinal = proto.ordinal.explicit if proto.ordinal.which() == "explicit" else None
|
||||
discriminant = proto.discriminantValue if proto.discriminantValue != NO_DISCRIMINANT else None
|
||||
fields.append({"name": name, "ordinal": ordinal, "discriminant": discriminant})
|
||||
return {
|
||||
"typeId": hex_id(node.id),
|
||||
"displayName": node.displayName,
|
||||
"hasUnion": node.struct.discriminantCount > 0,
|
||||
"fields": fields,
|
||||
}
|
||||
|
||||
|
||||
def get_schema_fields(schema_struct: Any) -> List[Tuple[str, Any]]:
|
||||
"""
|
||||
Retrieve all fields from a given schema structure.
|
||||
|
||||
Args:
|
||||
schema_struct: The Cap'n Proto schema structure
|
||||
|
||||
Returns:
|
||||
A list of field names and their metadata
|
||||
"""
|
||||
try:
|
||||
# Get all fields from the schema
|
||||
schema_fields = list(schema_struct.schema.fields.items())
|
||||
|
||||
print_debug("Discovered schema fields:")
|
||||
for field_name, field_metadata in schema_fields:
|
||||
print_debug(f"- {field_name}")
|
||||
|
||||
return schema_fields
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error retrieving schema fields: {e}")
|
||||
return []
|
||||
def _child_struct_schema(field: Any) -> Any:
|
||||
proto = field.proto
|
||||
if proto.which() == "group":
|
||||
return field.schema
|
||||
type_node = proto.slot.type
|
||||
which = type_node.which()
|
||||
if which == "struct":
|
||||
return field.schema
|
||||
if which == "list":
|
||||
container = field.schema
|
||||
element_type = type_node.list.elementType
|
||||
while element_type.which() == "list":
|
||||
container = container.elementType
|
||||
element_type = element_type.list.elementType
|
||||
if element_type.which() == "struct":
|
||||
return container.elementType
|
||||
return None
|
||||
|
||||
|
||||
def generate_schema_instances(schema_struct: Any) -> List[Any]:
|
||||
"""
|
||||
Generate instances for all fields in a given schema.
|
||||
|
||||
Args:
|
||||
schema_struct: The Cap'n Proto schema structure
|
||||
|
||||
Returns:
|
||||
A list of schema instances
|
||||
"""
|
||||
schema_fields = get_schema_fields(schema_struct)
|
||||
instances = []
|
||||
|
||||
for field_prop in schema_fields:
|
||||
try:
|
||||
instance = create_schema_instance(schema_struct, field_prop)
|
||||
if instance is not None:
|
||||
instances.append(instance)
|
||||
except Exception as e:
|
||||
print(f"Skipping field due to error: {e}")
|
||||
|
||||
print(f"Generated {len(instances)} schema instances")
|
||||
return instances
|
||||
def collect_schema(root: Any) -> dict[str, dict]:
|
||||
structs: dict[str, dict] = {}
|
||||
stack = [root]
|
||||
while stack:
|
||||
schema = stack.pop()
|
||||
type_id = hex_id(schema.node.id)
|
||||
if type_id in structs:
|
||||
continue
|
||||
structs[type_id] = encode_struct(schema)
|
||||
for _name, field in schema.fields.items():
|
||||
try:
|
||||
child = _child_struct_schema(field)
|
||||
except Exception:
|
||||
child = None
|
||||
if child is not None:
|
||||
stack.append(child)
|
||||
return structs
|
||||
|
||||
|
||||
def persist_instances(instances: List[Any], filename: str) -> None:
|
||||
"""
|
||||
Write schema instances to a binary file.
|
||||
|
||||
Args:
|
||||
instances: List of schema instances
|
||||
filename: Output file path
|
||||
"""
|
||||
try:
|
||||
with open(filename, 'wb') as f:
|
||||
for instance in instances:
|
||||
f.write(instance.to_bytes())
|
||||
|
||||
print(f"Successfully wrote {len(instances)} instances to {filename}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error persisting instances: {e}")
|
||||
sys.exit(1)
|
||||
def load_log(cereal_dir: str, extra_imports: list[str] | None = None) -> Any:
|
||||
import capnp
|
||||
cereal_dir = os.path.abspath(cereal_dir)
|
||||
capnp.remove_import_hook()
|
||||
imports = [cereal_dir] + [os.path.abspath(p) for p in (extra_imports or [])]
|
||||
return capnp.load(os.path.join(cereal_dir, "log.capnp"), imports=imports)
|
||||
|
||||
|
||||
def read_instances(filename: str, schema_type: Any) -> List[Any]:
|
||||
"""
|
||||
Read schema instances from a binary file.
|
||||
|
||||
Args:
|
||||
filename: Input file path
|
||||
schema_type: The schema type to use for reading
|
||||
|
||||
Returns:
|
||||
A list of read schema instances
|
||||
"""
|
||||
try:
|
||||
with open(filename, 'rb') as f:
|
||||
data = f.read()
|
||||
|
||||
instances = list(schema_type.read_multiple_bytes(data))
|
||||
|
||||
print(f"Read {len(instances)} instances from {filename}")
|
||||
return instances
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error reading instances: {e}")
|
||||
sys.exit(1)
|
||||
def dump_schema(cereal_dir: str, path: str, extra_imports: list[str] | None = None) -> None:
|
||||
log = load_log(cereal_dir, extra_imports)
|
||||
payload = {
|
||||
"root": hex_id(log.Event.schema.node.id),
|
||||
"structs": collect_schema(log.Event.schema),
|
||||
}
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
json.dump(payload, handle, indent=2, sort_keys=True)
|
||||
print(f"wrote schema dump with {len(payload['structs'])} structs to {path}")
|
||||
|
||||
|
||||
def compare_schemas(original_instances: List[Any], read_instances: List[Any]) -> bool:
|
||||
"""
|
||||
Compare original and read-back instances to detect potential breaking changes.
|
||||
def compare(sunnypilot_dump: dict, upstream_dump: dict) -> list[str]:
|
||||
violations: list[str] = []
|
||||
sunnypilot_structs = sunnypilot_dump["structs"]
|
||||
upstream_structs = upstream_dump["structs"]
|
||||
|
||||
Args:
|
||||
original_instances: List of originally generated instances
|
||||
read_instances: List of instances read back from file
|
||||
for type_id, sp_struct in sunnypilot_structs.items():
|
||||
if not sp_struct["hasUnion"]:
|
||||
continue
|
||||
up_struct = upstream_structs.get(type_id)
|
||||
if up_struct is None:
|
||||
continue
|
||||
|
||||
Returns:
|
||||
Boolean indicating whether schemas appear compatible
|
||||
"""
|
||||
if len(original_instances) != len(read_instances):
|
||||
print("❌ Schema Compatibility Warning: Instance count mismatch")
|
||||
return False
|
||||
up_ordinals = {f["ordinal"] for f in up_struct["fields"] if f.get("discriminant") is not None}
|
||||
display = sp_struct["displayName"]
|
||||
|
||||
compatible = True
|
||||
for struct in read_instances:
|
||||
try:
|
||||
getattr(struct, struct.which()) # Attempting to access the field to validate readability
|
||||
except Exception as e:
|
||||
print(f"❌ Structural change detected: {struct.which()} is not readable.\nFull error: {e}")
|
||||
compatible = False
|
||||
for field in sp_struct["fields"]:
|
||||
if field.get("discriminant") is None:
|
||||
continue
|
||||
if field["ordinal"] not in up_ordinals:
|
||||
violations.append(
|
||||
f"{display} @{field['ordinal']} '{field['name']}': "
|
||||
f"union variant not present upstream (discriminant={field['discriminant']})"
|
||||
)
|
||||
|
||||
return compatible
|
||||
return violations
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
CLI entry point for schema compatibility testing.
|
||||
"""
|
||||
# Setup argument parser
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Cap\'n Proto Schema Compatibility Testing Tool',
|
||||
epilog='Test schema compatibility by generating and reading back instances.'
|
||||
)
|
||||
def run_read(cereal_dir: str, peer_path: str, extra_imports: list[str] | None = None) -> int:
|
||||
log = load_log(cereal_dir, extra_imports)
|
||||
with open(peer_path, "r", encoding="utf-8") as f:
|
||||
peer_dump = json.load(f)
|
||||
local_dump = {
|
||||
"root": hex_id(log.Event.schema.node.id),
|
||||
"structs": collect_schema(log.Event.schema),
|
||||
}
|
||||
violations = compare(sunnypilot_dump=peer_dump, upstream_dump=local_dump)
|
||||
|
||||
# Add mutually exclusive group for generation or reading mode
|
||||
mode_group = parser.add_mutually_exclusive_group(required=True)
|
||||
mode_group.add_argument('-g', '--generate', action='store_true',
|
||||
help='Generate schema instances')
|
||||
mode_group.add_argument('-r', '--read', action='store_true',
|
||||
help='Read and validate schema instances')
|
||||
if not violations:
|
||||
print("cereal compat OK: upstream can parse sunnypilot routes.")
|
||||
return 0
|
||||
|
||||
# Common arguments
|
||||
parser.add_argument('-f', '--file',
|
||||
default='schema_instances.bin',
|
||||
help='Output/input binary file (default: schema_instances.bin)')
|
||||
print(f"cereal compat FAIL ({len(violations)} leaked union variant(s)):")
|
||||
for v in violations:
|
||||
print(f" {v}")
|
||||
return 1
|
||||
|
||||
# Parse arguments
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="sunnypilot cereal upstream compat check")
|
||||
mode = parser.add_mutually_exclusive_group(required=True)
|
||||
mode.add_argument("-g", "--generate", action="store_true", help="dump local schema to JSON")
|
||||
mode.add_argument("-r", "--read", action="store_true", help="validate against peer schema")
|
||||
parser.add_argument("-f", "--file", default="schema.json", help="JSON file path")
|
||||
parser.add_argument("--cereal-dir", required=True, help="path to cereal directory")
|
||||
parser.add_argument("-I", "--import-path", action="append", default=[], help="extra capnp import paths")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Import the schema dynamically
|
||||
try:
|
||||
from cereal import log
|
||||
schema_type = log.Event
|
||||
except ImportError:
|
||||
print("Error: Unable to import schema. Ensure 'cereal' is installed.")
|
||||
sys.exit(1)
|
||||
|
||||
# Execute based on mode
|
||||
if args.generate:
|
||||
print("🔧 Generating Schema Instances")
|
||||
instances = generate_schema_instances(schema_type)
|
||||
persist_instances(instances, args.file)
|
||||
print("✅ Instance generation complete")
|
||||
|
||||
elif args.read:
|
||||
print("🔍 Reading and Validating Schema Instances")
|
||||
generated_instances = generate_schema_instances(schema_type)
|
||||
read_back_instances = read_instances(args.file, schema_type)
|
||||
|
||||
# Compare schemas
|
||||
if compare_schemas(generated_instances, read_back_instances):
|
||||
print("✅ Schema Compatibility: No breaking changes detected")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("❌ Potential Schema Breaking Changes Detected")
|
||||
sys.exit(1)
|
||||
dump_schema(args.cereal_dir, args.file, args.import_path)
|
||||
return 0
|
||||
return run_read(args.cereal_dir, args.file, args.import_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
sys.exit(main())
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT services.py */
|
||||
#ifndef __SERVICES_H
|
||||
#define __SERVICES_H
|
||||
#include <map>
|
||||
#include <string>
|
||||
struct service { std::string name; bool should_log; float frequency; int decimation; size_t queue_size; };
|
||||
static std::map<std::string, service> services = {
|
||||
{ "gyroscope", {"gyroscope", true, 104.000000, 104, 256000}},
|
||||
{ "accelerometer", {"accelerometer", true, 104.000000, 104, 256000}},
|
||||
{ "temperatureSensor", {"temperatureSensor", true, 2.000000, 200, 256000}},
|
||||
{ "deviceState", {"deviceState", true, 2.000000, 1, 256000}},
|
||||
{ "touch", {"touch", true, 20.000000, 1, 256000}},
|
||||
{ "can", {"can", true, 100.000000, 2053, 10485760}},
|
||||
{ "controlsState", {"controlsState", true, 100.000000, 10, 2097152}},
|
||||
{ "selfdriveState", {"selfdriveState", true, 100.000000, 10, 256000}},
|
||||
{ "pandaStates", {"pandaStates", true, 10.000000, 1, 256000}},
|
||||
{ "peripheralState", {"peripheralState", true, 2.000000, 1, 256000}},
|
||||
{ "radarState", {"radarState", true, 20.000000, 5, 256000}},
|
||||
{ "roadEncodeIdx", {"roadEncodeIdx", false, 20.000000, 1, 256000}},
|
||||
{ "liveTracks", {"liveTracks", true, 20.000000, -1, 256000}},
|
||||
{ "sendcan", {"sendcan", true, 100.000000, 139, 2097152}},
|
||||
{ "logMessage", {"logMessage", true, 0.000000, -1, 10485760}},
|
||||
{ "errorLogMessage", {"errorLogMessage", true, 0.000000, 1, 10485760}},
|
||||
{ "liveCalibration", {"liveCalibration", true, 4.000000, 4, 256000}},
|
||||
{ "liveTorqueParameters", {"liveTorqueParameters", true, 4.000000, 1, 256000}},
|
||||
{ "liveDelay", {"liveDelay", true, 4.000000, 1, 256000}},
|
||||
{ "androidLog", {"androidLog", true, 0.000000, -1, 256000}},
|
||||
{ "carState", {"carState", true, 100.000000, 10, 256000}},
|
||||
{ "carControl", {"carControl", true, 100.000000, 10, 256000}},
|
||||
{ "carOutput", {"carOutput", true, 100.000000, 10, 256000}},
|
||||
{ "longitudinalPlan", {"longitudinalPlan", true, 20.000000, 10, 256000}},
|
||||
{ "lateralManeuverPlan", {"lateralManeuverPlan", true, 20.000000, -1, 256000}},
|
||||
{ "driverAssistance", {"driverAssistance", true, 20.000000, 20, 256000}},
|
||||
{ "procLog", {"procLog", true, 0.500000, 15, 10485760}},
|
||||
{ "gpsLocationExternal", {"gpsLocationExternal", true, 10.000000, 10, 256000}},
|
||||
{ "gpsLocation", {"gpsLocation", true, 1.000000, 1, 256000}},
|
||||
{ "ubloxGnss", {"ubloxGnss", true, 10.000000, -1, 256000}},
|
||||
{ "qcomGnss", {"qcomGnss", true, 2.000000, -1, 256000}},
|
||||
{ "clocks", {"clocks", true, 0.100000, 1, 256000}},
|
||||
{ "ubloxRaw", {"ubloxRaw", true, 20.000000, -1, 256000}},
|
||||
{ "livePose", {"livePose", true, 20.000000, 4, 256000}},
|
||||
{ "liveParameters", {"liveParameters", true, 20.000000, 5, 256000}},
|
||||
{ "cameraOdometry", {"cameraOdometry", true, 20.000000, 10, 256000}},
|
||||
{ "thumbnail", {"thumbnail", true, 0.016667, 1, 256000}},
|
||||
{ "onroadEvents", {"onroadEvents", true, 1.000000, 1, 256000}},
|
||||
{ "carParams", {"carParams", true, 0.020000, 1, 256000}},
|
||||
{ "roadCameraState", {"roadCameraState", true, 20.000000, 20, 256000}},
|
||||
{ "driverCameraState", {"driverCameraState", true, 20.000000, 20, 256000}},
|
||||
{ "driverEncodeIdx", {"driverEncodeIdx", false, 20.000000, 1, 256000}},
|
||||
{ "driverStateV2", {"driverStateV2", true, 20.000000, 10, 256000}},
|
||||
{ "driverMonitoringState", {"driverMonitoringState", true, 20.000000, 10, 256000}},
|
||||
{ "wideRoadEncodeIdx", {"wideRoadEncodeIdx", false, 20.000000, 1, 256000}},
|
||||
{ "wideRoadCameraState", {"wideRoadCameraState", true, 20.000000, 20, 256000}},
|
||||
{ "drivingModelData", {"drivingModelData", true, 20.000000, 10, 256000}},
|
||||
{ "modelV2", {"modelV2", true, 20.000000, -1, 10485760}},
|
||||
{ "managerState", {"managerState", true, 2.000000, 1, 256000}},
|
||||
{ "qRoadEncodeIdx", {"qRoadEncodeIdx", false, 20.000000, -1, 256000}},
|
||||
{ "userBookmark", {"userBookmark", true, 0.000000, 1, 256000}},
|
||||
{ "soundPressure", {"soundPressure", true, 10.000000, 10, 256000}},
|
||||
{ "rawAudioData", {"rawAudioData", false, 20.000000, -1, 256000}},
|
||||
{ "bookmarkButton", {"bookmarkButton", true, 0.000000, 1, 256000}},
|
||||
{ "audioFeedback", {"audioFeedback", true, 0.000000, 1, 256000}},
|
||||
{ "roadEncodeData", {"roadEncodeData", false, 20.000000, -1, 10485760}},
|
||||
{ "driverEncodeData", {"driverEncodeData", false, 20.000000, -1, 10485760}},
|
||||
{ "wideRoadEncodeData", {"wideRoadEncodeData", false, 20.000000, -1, 10485760}},
|
||||
{ "qRoadEncodeData", {"qRoadEncodeData", false, 20.000000, -1, 10485760}},
|
||||
{ "modelManagerSP", {"modelManagerSP", false, 1.000000, 1, 10485760}},
|
||||
{ "backupManagerSP", {"backupManagerSP", false, 1.000000, 1, 10485760}},
|
||||
{ "selfdriveStateSP", {"selfdriveStateSP", true, 100.000000, 10, 256000}},
|
||||
{ "longitudinalPlanSP", {"longitudinalPlanSP", true, 20.000000, 10, 256000}},
|
||||
{ "onroadEventsSP", {"onroadEventsSP", true, 1.000000, 1, 256000}},
|
||||
{ "carParamsSP", {"carParamsSP", true, 0.020000, 1, 256000}},
|
||||
{ "carControlSP", {"carControlSP", true, 100.000000, 10, 256000}},
|
||||
{ "carStateSP", {"carStateSP", true, 100.000000, 10, 256000}},
|
||||
{ "liveMapDataSP", {"liveMapDataSP", true, 1.000000, 1, 256000}},
|
||||
{ "modelDataV2SP", {"modelDataV2SP", true, 20.000000, -1, 10485760}},
|
||||
{ "liveLocationKalman", {"liveLocationKalman", true, 20.000000, -1, 256000}},
|
||||
{ "uiDebug", {"uiDebug", true, 0.000000, 1, 256000}},
|
||||
{ "testJoystick", {"testJoystick", true, 0.000000, -1, 256000}},
|
||||
{ "alertDebug", {"alertDebug", true, 20.000000, 5, 256000}},
|
||||
{ "livestreamWideRoadEncodeIdx", {"livestreamWideRoadEncodeIdx", false, 20.000000, -1, 256000}},
|
||||
{ "livestreamRoadEncodeIdx", {"livestreamRoadEncodeIdx", false, 20.000000, -1, 256000}},
|
||||
{ "livestreamDriverEncodeIdx", {"livestreamDriverEncodeIdx", false, 20.000000, -1, 256000}},
|
||||
{ "livestreamWideRoadEncodeData", {"livestreamWideRoadEncodeData", false, 20.000000, -1, 2097152}},
|
||||
{ "livestreamRoadEncodeData", {"livestreamRoadEncodeData", false, 20.000000, -1, 2097152}},
|
||||
{ "livestreamDriverEncodeData", {"livestreamDriverEncodeData", false, 20.000000, -1, 2097152}},
|
||||
{ "customReservedRawData0", {"customReservedRawData0", true, 0.000000, -1, 256000}},
|
||||
};
|
||||
#endif
|
||||
|
||||
@@ -24,10 +24,7 @@ _services: dict[str, tuple] = {
|
||||
# note: the "EncodeIdx" packets will still be in the log
|
||||
"gyroscope": (True, 104., 104),
|
||||
"accelerometer": (True, 104., 104),
|
||||
"magnetometer": (True, 25.),
|
||||
"lightSensor": (True, 100., 100),
|
||||
"temperatureSensor": (True, 2., 200),
|
||||
"gpsNMEA": (True, 9.),
|
||||
"deviceState": (True, 2., 1),
|
||||
"touch": (True, 20., 1),
|
||||
"can": (True, 100., 2053, QueueSize.BIG), # decimation gives ~3 msgs in a full segment
|
||||
@@ -56,7 +53,6 @@ _services: dict[str, tuple] = {
|
||||
"gpsLocation": (True, 1., 1),
|
||||
"ubloxGnss": (True, 10.),
|
||||
"qcomGnss": (True, 2.),
|
||||
"gnssMeasurements": (True, 10., 10),
|
||||
"clocks": (True, 0.1, 1),
|
||||
"ubloxRaw": (True, 20.),
|
||||
"livePose": (True, 20., 4),
|
||||
@@ -75,10 +71,6 @@ _services: dict[str, tuple] = {
|
||||
"drivingModelData": (True, 20., 10),
|
||||
"modelV2": (True, 20., None, QueueSize.BIG),
|
||||
"managerState": (True, 2., 1),
|
||||
"uploaderState": (True, 0., 1),
|
||||
"navInstruction": (True, 1., 10),
|
||||
"navRoute": (True, 0.),
|
||||
"navThumbnail": (True, 0.),
|
||||
"qRoadEncodeIdx": (False, 20.),
|
||||
"userBookmark": (True, 0., 1),
|
||||
"soundPressure": (True, 10., 10),
|
||||
@@ -114,8 +106,6 @@ _services: dict[str, tuple] = {
|
||||
"livestreamRoadEncodeData": (False, 20., None, QueueSize.MEDIUM),
|
||||
"livestreamDriverEncodeData": (False, 20., None, QueueSize.MEDIUM),
|
||||
"customReservedRawData0": (True, 0.),
|
||||
"customReservedRawData1": (True, 0.),
|
||||
"customReservedRawData2": (True, 0.),
|
||||
}
|
||||
SERVICE_LIST = {name: Service(*vals) for
|
||||
idx, (name, vals) in enumerate(_services.items())}
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
Import('env', 'envCython')
|
||||
|
||||
common_libs = [
|
||||
'params.cc',
|
||||
'swaglog.cc',
|
||||
'util.cc',
|
||||
'ratekeeper.cc',
|
||||
]
|
||||
|
||||
_common = env.Library('common', common_libs, LIBS="json11")
|
||||
Export('_common')
|
||||
|
||||
if GetOption('extras'):
|
||||
env.Program('tests/test_common',
|
||||
['tests/test_runner.cc', 'tests/test_params.cc', 'tests/test_util.cc', 'tests/test_swaglog.cc'],
|
||||
LIBS=[_common, 'json11', 'zmq', 'pthread'])
|
||||
|
||||
# Cython bindings
|
||||
params_python = envCython.Program('params_pyx.so', 'params_pyx.pyx', LIBS=envCython['LIBS'] + [_common, 'zmq', 'json11'])
|
||||
|
||||
common_python = [params_python]
|
||||
|
||||
Export('common_python')
|
||||
+1
-1
@@ -3,7 +3,7 @@ import os
|
||||
import requests
|
||||
import unicodedata
|
||||
from datetime import datetime, timedelta, UTC
|
||||
from openpilot.system.hardware.hw import Paths
|
||||
from openpilot.common.hardware.hw import Paths
|
||||
from openpilot.system.version import get_version
|
||||
|
||||
# name: jwt signature algorithm
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
from openpilot.common.esim.base import LPABase, LPAError, LPAProfileNotFoundError, Profile
|
||||
|
||||
__all__ = ["LPABase", "LPAError", "LPAProfileNotFoundError", "Profile"]
|
||||
@@ -0,0 +1,55 @@
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
class LPAError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class LPAProfileNotFoundError(LPAError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class Profile:
|
||||
iccid: str
|
||||
nickname: str
|
||||
enabled: bool
|
||||
provider: str
|
||||
|
||||
@property
|
||||
def is_comma(self) -> bool:
|
||||
return self.provider == 'Webbing' and self.iccid.startswith('8985235')
|
||||
|
||||
|
||||
class LPABase(ABC):
|
||||
@abstractmethod
|
||||
def list_profiles(self) -> list[Profile]:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_active_profile(self) -> Profile | None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def delete_profile(self, iccid: str) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def download_profile(self, qr: str, nickname: str | None = None) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def nickname_profile(self, iccid: str, nickname: str) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def switch_profile(self, iccid: str) -> None:
|
||||
pass
|
||||
|
||||
def process_notifications(self) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def is_euicc(self) -> bool:
|
||||
pass
|
||||
Executable
+70
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
from openpilot.common.hardware import HARDWARE
|
||||
from openpilot.common.esim.base import LPABase, Profile
|
||||
|
||||
|
||||
def sorted_profiles(lpa: LPABase) -> list[Profile]:
|
||||
return sorted(lpa.list_profiles(), key=lambda p: p.iccid)
|
||||
|
||||
|
||||
def resolve_iccid(lpa: LPABase, ref: str) -> str:
|
||||
# ref is either a 1-based index into the sorted list, or a literal iccid
|
||||
if ref.isdigit():
|
||||
profiles = sorted_profiles(lpa)
|
||||
idx = int(ref) - 1
|
||||
if not 0 <= idx < len(profiles):
|
||||
raise SystemExit(f'no profile at index {ref} (have {len(profiles)})')
|
||||
return profiles[idx].iccid
|
||||
return ref
|
||||
|
||||
|
||||
def print_profiles(lpa: LPABase) -> None:
|
||||
profiles = sorted_profiles(lpa)
|
||||
print(f'\n{len(profiles)} profile{"s" if len(profiles) != 1 else ""}:')
|
||||
for i, p in enumerate(profiles, start=1):
|
||||
print(f'{i}. {p.iccid} (nickname: {p.nickname or "<none provided>"}) (provider: {p.provider}) - {"enabled" if p.enabled else "disabled"}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(prog='esim.py', description='manage eSIM profiles on your comma device', epilog='comma.ai')
|
||||
sub = parser.add_subparsers(dest='cmd')
|
||||
|
||||
sub.add_parser('list', help='list profiles')
|
||||
|
||||
p_switch = sub.add_parser('switch', help='switch to profile')
|
||||
p_switch.add_argument('profile', help='iccid or 1-based index from `list`')
|
||||
|
||||
p_delete = sub.add_parser('delete', help='delete profile (warning: this cannot be undone)')
|
||||
p_delete.add_argument('profile', help='iccid or 1-based index from `list`')
|
||||
|
||||
p_download = sub.add_parser('download', help='download a profile using QR code (format: LPA:1$rsp.truphone.com$QRF-SPEEDTEST)')
|
||||
p_download.add_argument('qr')
|
||||
p_download.add_argument('name')
|
||||
|
||||
p_nickname = sub.add_parser('nickname', help='update the nickname for a profile')
|
||||
p_nickname.add_argument('profile', help='iccid or 1-based index from `list`')
|
||||
p_nickname.add_argument('name')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
lpa = HARDWARE.get_sim_lpa()
|
||||
if args.cmd == 'switch':
|
||||
lpa.switch_profile(resolve_iccid(lpa, args.profile))
|
||||
elif args.cmd == 'delete':
|
||||
iccid = resolve_iccid(lpa, args.profile)
|
||||
confirm = input(f'are you sure you want to delete profile {iccid}? (y/N) ')
|
||||
if confirm == 'y':
|
||||
lpa.delete_profile(iccid)
|
||||
else:
|
||||
print('cancelled')
|
||||
exit(0)
|
||||
elif args.cmd == 'download':
|
||||
lpa.download_profile(args.qr, args.name)
|
||||
elif args.cmd == 'nickname':
|
||||
lpa.nickname_profile(resolve_iccid(lpa, args.profile), args.name)
|
||||
else:
|
||||
if args.cmd is None:
|
||||
parser.print_help()
|
||||
print_profiles(lpa)
|
||||
@@ -0,0 +1,133 @@
|
||||
# GSMA Certificate Issuer (CI) bundle for eSIM RSP
|
||||
# Source: https://euicc-manual.osmocom.org/docs/pki/ci/bundle.pem
|
||||
|
||||
issuer=
|
||||
countryName = CH
|
||||
organizationName = OISTE Foundation
|
||||
commonName = OISTE GSMA CI G1
|
||||
notBefore=2024-01-16 23:17:39Z
|
||||
notAfter=2059-01-07 23:17:38Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIIB9zCCAZ2gAwIBAgIUSpBSCCDYPOEG/IFHUCKpZ2pIAQMwCgYIKoZIzj0EAwIw
|
||||
QzELMAkGA1UEBhMCQ0gxGTAXBgNVBAoMEE9JU1RFIEZvdW5kYXRpb24xGTAXBgNV
|
||||
BAMMEE9JU1RFIEdTTUEgQ0kgRzEwIBcNMjQwMTE2MjMxNzM5WhgPMjA1OTAxMDcy
|
||||
MzE3MzhaMEMxCzAJBgNVBAYTAkNIMRkwFwYDVQQKDBBPSVNURSBGb3VuZGF0aW9u
|
||||
MRkwFwYDVQQDDBBPSVNURSBHU01BIENJIEcxMFkwEwYHKoZIzj0CAQYIKoZIzj0D
|
||||
AQcDQgAEvZ3s3PFC4NgrCcCMmHJ6DJ66uzAHuLcvjJnOn+TtBNThS7YHLDyHCa2v
|
||||
7D+zTP+XTtgqgcLoB56Gha9EQQQ4xKNtMGswDwYDVR0TAQH/BAUwAwEB/zAQBgNV
|
||||
HREECTAHiAVghXQFDjAXBgNVHSABAf8EDTALMAkGB2eBEgECAQAwHQYDVR0OBBYE
|
||||
FEwnlnrSDBSzkelgHkHmBK1XwCIvMA4GA1UdDwEB/wQEAwIBBjAKBggqhkjOPQQD
|
||||
AgNIADBFAiBVcywTj017jKpAQ+gwy4MqK2hQvzve6lkvQkgSP6ykHwIhAI0KFwCD
|
||||
jnPbmcJsG41hUrWNlf+IcrMvFuYii0DasBNi
|
||||
-----END CERTIFICATE-----
|
||||
issuer=
|
||||
organizationName = GSM Association
|
||||
commonName = GSM Association - RSP2 Root CI1
|
||||
notBefore=2017-02-22 00:00:00Z
|
||||
notAfter=2052-02-21 23:59:59Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICSTCCAe+gAwIBAgIQbmhWeneg7nyF7hg5Y9+qejAKBggqhkjOPQQDAjBEMRgw
|
||||
FgYDVQQKEw9HU00gQXNzb2NpYXRpb24xKDAmBgNVBAMTH0dTTSBBc3NvY2lhdGlv
|
||||
biAtIFJTUDIgUm9vdCBDSTEwIBcNMTcwMjIyMDAwMDAwWhgPMjA1MjAyMjEyMzU5
|
||||
NTlaMEQxGDAWBgNVBAoTD0dTTSBBc3NvY2lhdGlvbjEoMCYGA1UEAxMfR1NNIEFz
|
||||
c29jaWF0aW9uIC0gUlNQMiBSb290IENJMTBZMBMGByqGSM49AgEGCCqGSM49AwEH
|
||||
A0IABJ1qutL0HCMX52GJ6/jeibsAqZfULWj/X10p/Min6seZN+hf5llovbCNuB2n
|
||||
unLz+O8UD0SUCBUVo8e6n9X1TuajgcAwgb0wDgYDVR0PAQH/BAQDAgEGMA8GA1Ud
|
||||
EwEB/wQFMAMBAf8wEwYDVR0RBAwwCogIKwYBBAGC6WAwFwYDVR0gAQH/BA0wCzAJ
|
||||
BgdngRIBAgEAME0GA1UdHwRGMEQwQqBAoD6GPGh0dHA6Ly9nc21hLWNybC5zeW1h
|
||||
dXRoLmNvbS9vZmZsaW5lY2EvZ3NtYS1yc3AyLXJvb3QtY2kxLmNybDAdBgNVHQ4E
|
||||
FgQUgTcPUSXQsdQI1MOyMubSXnlb6/swCgYIKoZIzj0EAwIDSAAwRQIgIJdYsOMF
|
||||
WziPK7l8nh5mu0qiRiVf25oa9ullG/OIASwCIQDqCmDrYf+GziHXBOiwJwnBaeBO
|
||||
aFsiLzIEOaUuZwdNUw==
|
||||
-----END CERTIFICATE-----
|
||||
issuer=
|
||||
countryName = US
|
||||
organizationName = Entrust, Inc.
|
||||
organizationalUnitName = See www.entrust.net/legal-terms
|
||||
organizationalUnitName = (c) 2016 Entrust, Inc. - for authorized use only
|
||||
commonName = Entrust eSIM Certification Authority
|
||||
notBefore=2016-11-16 16:04:02Z
|
||||
notAfter=2051-10-16 16:34:02Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIIC6DCCAo2gAwIBAgIRAIy4GT7M5nHsAAAAAFgsinowCgYIKoZIzj0EAwIwgbkx
|
||||
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1FbnRydXN0LCBJbmMuMSgwJgYDVQQLEx9T
|
||||
ZWUgd3d3LmVudHJ1c3QubmV0L2xlZ2FsLXRlcm1zMTkwNwYDVQQLEzAoYykgMjAx
|
||||
NiBFbnRydXN0LCBJbmMuIC0gZm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxLTArBgNV
|
||||
BAMTJEVudHJ1c3QgZVNJTSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAgFw0xNjEx
|
||||
MTYxNjA0MDJaGA8yMDUxMTAxNjE2MzQwMlowgbkxCzAJBgNVBAYTAlVTMRYwFAYD
|
||||
VQQKEw1FbnRydXN0LCBJbmMuMSgwJgYDVQQLEx9TZWUgd3d3LmVudHJ1c3QubmV0
|
||||
L2xlZ2FsLXRlcm1zMTkwNwYDVQQLEzAoYykgMjAxNiBFbnRydXN0LCBJbmMuIC0g
|
||||
Zm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxLTArBgNVBAMTJEVudHJ1c3QgZVNJTSBD
|
||||
ZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IA
|
||||
BAdzwGHeQ1Wb2f4DmHTByR5/IWL3JugQ1U3908a++bHdlt+TTA7K4c5cYZ+51Yz/
|
||||
hg/bacxguPDh9uQUK6Wg3a6jcjBwMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/
|
||||
BAQDAgEGMBcGA1UdIAEB/wQNMAswCQYHZ4ESAQIBADAVBgNVHREEDjAMiApghkgB
|
||||
hvpsFAoAMB0GA1UdDgQWBBQWcEt/NR42B/GMS3AAXDoAPf1BSjAKBggqhkjOPQQD
|
||||
AgNJADBGAiEAspjXMvaBZyAg86Z0AAtT0yBRAi1EyaAfNz9kDJeAE04CIQC3efj8
|
||||
ATL7/tDBOhANy3cK8PS/1NIlu9vqMLCZsZvJ0Q==
|
||||
-----END CERTIFICATE-----
|
||||
issuer=
|
||||
countryName = FR
|
||||
organizationName = OBERTHUR TECHNOLOGIES
|
||||
organizationalUnitName = TELECOM
|
||||
commonName = MC4 OT ROOT CI v1
|
||||
notBefore=2016-11-15 00:00:01Z
|
||||
notAfter=2046-11-08 23:59:59Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICOjCCAeGgAwIBAgIBATAKBggqhkjOPQQDAjBbMQswCQYDVQQGEwJGUjEeMBwG
|
||||
A1UEChMVT0JFUlRIVVIgVEVDSE5PTE9HSUVTMRAwDgYDVQQLEwdURUxFQ09NMRow
|
||||
GAYDVQQDExFNQzQgT1QgUk9PVCBDSSB2MTAeFw0xNjExMTUwMDAwMDFaFw00NjEx
|
||||
MDgyMzU5NTlaMFsxCzAJBgNVBAYTAkZSMR4wHAYDVQQKExVPQkVSVEhVUiBURUNI
|
||||
Tk9MT0dJRVMxEDAOBgNVBAsTB1RFTEVDT00xGjAYBgNVBAMTEU1DNCBPVCBST09U
|
||||
IENJIHYxMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEHb/Gajt3OZxuaDSklBQE
|
||||
D4lOd6PGPLSvtfkM952ubdyy45tJwAeA0eEii0CLrFT6tcfXkW+H/5mQyMRXaAUk
|
||||
T6OBlTCBkjAfBgNVHSMEGDAWgBTNbmC3LXoGPLyEYluR6A/jBAbhPjAdBgNVHQ4E
|
||||
FgQUzW5gty16Bjy8hGJbkegP4wQG4T4wDgYDVR0PAQH/BAQDAgAGMBcGA1UdIAEB
|
||||
/wQNMAswCQYHZ4ESAQIBADAWBgNVHREEDzANiAsrBgEEAYHvb7OITTAPBgNVHRMB
|
||||
Af8EBTADAQH/MAoGCCqGSM49BAMCA0cAMEQCIEw4Nc7f2fDtoH+6ON/bknfDQxmT
|
||||
ikThXjhpLtSrSKN2AiAxHxgC87L0FDnH8dJNlkdGX9c0JIx6oLheIplfS6k+jg==
|
||||
-----END CERTIFICATE-----
|
||||
issuer=
|
||||
commonName = SubMan V4.2 CI Google Pixel
|
||||
organizationName = Giesecke and Devrient GmbH
|
||||
organizationalUnitName = Mobile Security
|
||||
countryName = DE
|
||||
notBefore=2017-05-10 00:00:00Z
|
||||
notAfter=2027-05-10 00:00:00Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICaTCCAg6gAwIBAgICASwwCgYIKoZIzj0EAwIwczElMCMGA1UEAxMcIFN1Yk1h
|
||||
biBWNC4yIENJIEdvb2dsZSBQaXhlbDEjMCEGA1UEChMaR2llc2Vja2UgYW5kIERl
|
||||
dnJpZW50IEdtYkgxGDAWBgNVBAsTD01vYmlsZSBTZWN1cml0eTELMAkGA1UEBhMC
|
||||
REUwHhcNMTcwNTEwMDAwMDAwWhcNMjcwNTEwMDAwMDAwWjBzMSUwIwYDVQQDExwg
|
||||
U3ViTWFuIFY0LjIgQ0kgR29vZ2xlIFBpeGVsMSMwIQYDVQQKExpHaWVzZWNrZSBh
|
||||
bmQgRGV2cmllbnQgR21iSDEYMBYGA1UECxMPTW9iaWxlIFNlY3VyaXR5MQswCQYD
|
||||
VQQGEwJERTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABHNorfaJsGzqWNawyAhl
|
||||
IAv9QL2/+b9RsUoso06t/dKX1MRr5CUJ51acvv5TAFhQKIml+dwLbFnV5aO+8W6Z
|
||||
wxajgZEwgY4wHwYDVR0jBBgwFoAUtg8LiX/WMLiM/tYWH46oCMU4KsMwHQYDVR0O
|
||||
BBYEFLYPC4l/1jC4jP7WFh+OqAjFOCrDMA4GA1UdDwEB/wQEAwIBBjAXBgNVHSAB
|
||||
Af8EDTALMAkGB2eBEgECAQAwDwYDVR0TAQH/BAUwAwEB/zASBgNVHREECzAJiAcr
|
||||
BgEEAdwPMAoGCCqGSM49BAMCA0kAMEYCIQDpoZcuAQrjATW8U+AWqMUJ0dY6nWW1
|
||||
R1QmFzVZ1yMXSwIhALCvRqkCtgiavdeFeSgsSNbY5Fhd+QoCltuSh1U4TE7A
|
||||
-----END CERTIFICATE-----
|
||||
issuer=
|
||||
countryName = DE
|
||||
commonName = SubMan V4.2 CI
|
||||
organizationName = Giesecke and Devrient
|
||||
organizationalUnitName = Mobile Security
|
||||
notBefore=2016-08-12 13:51:48Z
|
||||
notAfter=2026-08-12 13:51:48Z
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICUjCCAfigAwIBAgIDQgAAMAoGCCqGSM49BAMCMGAxCzAJBgNVBAYTAkRFMRcw
|
||||
FQYDVQQDEw5TdWJNYW4gVjQuMiBDSTEeMBwGA1UEChMVR2llc2Vja2UgYW5kIERl
|
||||
dnJpZW50MRgwFgYDVQQLEw9Nb2JpbGUgU2VjdXJpdHkwHhcNMTYwODEyMTM1MTQ4
|
||||
WhcNMjYwODEyMTM1MTQ4WjBgMQswCQYDVQQGEwJERTEXMBUGA1UEAxMOU3ViTWFu
|
||||
IFY0LjIgQ0kxHjAcBgNVBAoTFUdpZXNlY2tlIGFuZCBEZXZyaWVudDEYMBYGA1UE
|
||||
CxMPTW9iaWxlIFNlY3VyaXR5MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEYIgl
|
||||
VQr9wbXOlwPp8qMg5Df08Cli9Mc+lpr3Lwa9PlVA3QWlLeX4GfD4H3phLBqVIa17
|
||||
yHttmtheTxi0KoEqhKOBoDCBnTAdBgNVHQ4EFgQU6lOt7zMpuVCa/XVf1Ei4LcG8
|
||||
7P8wDgYDVR0PAQH/BAQDAgEGMBcGA1UdIAEB/wQNMAswCQYHZ4ESAQIBADAPBgNV
|
||||
HRMBAf8EBTADAQH/MBIGA1UdEQQLMAmIBysGAQQB3A8wLgYDVR0fBCcwJTAjoCGg
|
||||
H4YdaHR0cDovL2dpLWRlLmNvbS90ZXN0LmNybC5wZW0wCgYIKoZIzj0EAwIDSAAw
|
||||
RQIhAMMx2L/VHDiOW+Fl/OuFmhCdizYM17Yn9zAVieKO2T0iAiANWtCMmY+DzkqK
|
||||
yHxBFX0U2tBd682zP4DpgRt8j3Ylew==
|
||||
-----END CERTIFICATE-----
|
||||
@@ -0,0 +1,791 @@
|
||||
# SGP.22 v2.3: https://www.gsma.com/solutions-and-impact/technologies/esim/wp-content/uploads/2021/07/SGP.22-v2.3.pdf
|
||||
|
||||
import atexit
|
||||
import base64
|
||||
import fcntl
|
||||
import hashlib
|
||||
import os
|
||||
import requests
|
||||
import serial
|
||||
import subprocess
|
||||
import sys
|
||||
import termios
|
||||
import time
|
||||
|
||||
from collections.abc import Callable, Generator
|
||||
from contextlib import contextmanager
|
||||
from typing import Any
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from openpilot.common.time_helpers import system_time_valid
|
||||
from openpilot.common.esim.base import LPABase, LPAError, LPAProfileNotFoundError, Profile
|
||||
|
||||
GSMA_CI_BUNDLE = str(Path(__file__).parent / "gsma_ci_bundle.pem")
|
||||
|
||||
DEFAULT_DEVICE = "/dev/modem_at0"
|
||||
DEFAULT_BAUD = 9600
|
||||
DEFAULT_TIMEOUT = 5.0
|
||||
# https://euicc-manual.osmocom.org/docs/lpa/applet-id/
|
||||
ISDR_AID = "A0000005591010FFFFFFFF8900000100"
|
||||
ES10X_MSS = 120
|
||||
HTTP_TIMEOUT = 30
|
||||
OPEN_ISDR_RETRIES = 10
|
||||
OPEN_ISDR_RETRY_DELAY_S = 0.25
|
||||
OPEN_ISDR_RESET_ATTEMPT = 5
|
||||
SEND_APDU_RETRIES = 3
|
||||
LOCK_FILE = '/dev/shm/modem.lock'
|
||||
DEBUG = os.environ.get("DEBUG") == "1"
|
||||
|
||||
|
||||
# TLV Tags
|
||||
TAG_ICCID = 0x5A
|
||||
TAG_STATUS = 0x80
|
||||
TAG_EUICC_INFO = 0xBF20
|
||||
TAG_PREPARE_DOWNLOAD = 0xBF21
|
||||
TAG_BPP_COMMAND = 0xBF23
|
||||
TAG_PROFILE_METADATA = 0xBF25
|
||||
TAG_INSTALL_RESULT_DATA = 0xBF27
|
||||
TAG_LIST_NOTIFICATION = 0xBF28
|
||||
TAG_SET_NICKNAME = 0xBF29
|
||||
TAG_RETRIEVE_NOTIFICATION = 0xBF2B
|
||||
TAG_PROFILE_INFO_LIST = 0xBF2D
|
||||
TAG_EUICC_CHALLENGE = 0xBF2E
|
||||
TAG_NOTIFICATION_METADATA = 0xBF2F
|
||||
TAG_NOTIFICATION_SENT = 0xBF30
|
||||
TAG_ENABLE_PROFILE = 0xBF31
|
||||
TAG_DELETE_PROFILE = 0xBF33
|
||||
TAG_BPP = 0xBF36
|
||||
TAG_PROFILE_INSTALL_RESULT = 0xBF37
|
||||
TAG_AUTH_SERVER = 0xBF38
|
||||
TAG_CANCEL_SESSION = 0xBF41
|
||||
TAG_OK = 0xA0
|
||||
|
||||
PROFILE_OK = 0x00
|
||||
PROFILE_NOT_IN_DISABLED_STATE = 0x02
|
||||
PROFILE_CAT_BUSY = 0x05
|
||||
|
||||
PROFILE_ERROR_CODES = {
|
||||
0x01: "iccidOrAidNotFound", PROFILE_NOT_IN_DISABLED_STATE: "profileNotInDisabledState",
|
||||
0x03: "disallowedByPolicy", 0x04: "wrongProfileReenabling",
|
||||
PROFILE_CAT_BUSY: "catBusy", 0x06: "undefinedError",
|
||||
}
|
||||
AUTH_SERVER_ERROR_CODES = {
|
||||
0x01: "eUICCVerificationFailed", 0x02: "eUICCCertificateExpired",
|
||||
0x03: "eUICCCertificateRevoked", 0x05: "invalidServerSignature",
|
||||
0x06: "euiccCiPKUnknown", 0x0A: "matchingIdRefused",
|
||||
0x10: "insufficientMemory",
|
||||
}
|
||||
BPP_COMMAND_NAMES = {
|
||||
0: "initialiseSecureChannel", 1: "configureISDP", 2: "storeMetadata",
|
||||
3: "storeMetadata2", 4: "replaceSessionKeys", 5: "loadProfileElements",
|
||||
}
|
||||
BPP_ERROR_REASONS = {
|
||||
1: "incorrectInputValues", 2: "invalidSignature", 3: "invalidTransactionId",
|
||||
4: "unsupportedCrtValues", 5: "unsupportedRemoteOperationType",
|
||||
6: "unsupportedProfileClass", 7: "scp03tStructureError", 8: "scp03tSecurityError",
|
||||
9: "iccidAlreadyExistsOnEuicc", 10: "insufficientMemoryForProfile",
|
||||
11: "installInterrupted", 12: "peProcessingError", 13: "dataMismatch",
|
||||
14: "invalidNAA",
|
||||
}
|
||||
BPP_ERROR_MESSAGES = {
|
||||
9: "This eSIM profile is already installed on this device.",
|
||||
10: "Not enough memory on the eUICC to install this profile.",
|
||||
12: "Profile installation failed. The QR code may have already been used.",
|
||||
}
|
||||
|
||||
# SGP.22 §5.2.6 SM-DP+ reason/subject codes mapped to user-friendly messages
|
||||
ES9P_ERROR_MESSAGES: dict[tuple[str, str], str] = {
|
||||
('3.8', '8.2.6'): "This eSIM profile is already installed on another device. Please use a new QR code.",
|
||||
('3.8', '8.2.1'): "This eSIM profile has expired. Please request a new QR code.",
|
||||
('3.8', '8.1'): "The SM-DP+ server refused this request.",
|
||||
('3.1', '8.2.6'): "This eSIM profile has been revoked by the carrier.",
|
||||
('3.9', '8.2.6'): "This eSIM profile download has already been completed.",
|
||||
('2.1', '8.8'): "The device is not compatible with this eSIM profile.",
|
||||
('1.2', '8.1'): "The SM-DP+ server is temporarily unavailable. Try again later.",
|
||||
}
|
||||
|
||||
NOTIFICATION_OPERATIONS = {0x80: "install", 0x40: "enable", 0x20: "disable", 0x10: "delete"}
|
||||
|
||||
STATE_LABELS = {0: "disabled", 1: "enabled", 255: "unknown"}
|
||||
ICON_LABELS = {0: "jpeg", 1: "png", 255: "unknown"}
|
||||
CLASS_LABELS = {0: "test", 1: "provisioning", 2: "operational", 255: "unknown"}
|
||||
|
||||
# TLV tag -> (field_name, decoder)
|
||||
FieldMap = dict[int, tuple[str, Callable[[bytes], Any]]]
|
||||
|
||||
|
||||
def b64e(data: bytes) -> str:
|
||||
return base64.b64encode(data).decode("ascii")
|
||||
|
||||
|
||||
def base64_trim(s: str) -> str:
|
||||
return "".join(c for c in s if c not in "\n\r \t")
|
||||
|
||||
|
||||
def b64d(s: str) -> bytes:
|
||||
return base64.b64decode(base64_trim(s))
|
||||
|
||||
|
||||
class AtClient:
|
||||
def __init__(self, device: str, baud: int, timeout: float) -> None:
|
||||
self.channel: str | None = None
|
||||
self._device = device
|
||||
self._baud = baud
|
||||
self._timeout = timeout
|
||||
self._serial: serial.Serial | None = None
|
||||
|
||||
def send_raw(self, data: bytes) -> None:
|
||||
self._ensure_serial()
|
||||
self._serial.reset_input_buffer()
|
||||
self._serial.write(data)
|
||||
self._serial.flush()
|
||||
|
||||
def close(self) -> None:
|
||||
try:
|
||||
if self.channel:
|
||||
try:
|
||||
self.query(f"AT+CCHC={self.channel}")
|
||||
except (RuntimeError, TimeoutError):
|
||||
pass
|
||||
self.channel = None
|
||||
finally:
|
||||
if self._serial:
|
||||
self._serial.close()
|
||||
|
||||
def _send(self, cmd: str) -> None:
|
||||
if DEBUG:
|
||||
print(f"SER >> {cmd}", file=sys.stderr)
|
||||
self._serial.write((cmd + "\r").encode("ascii"))
|
||||
|
||||
def _expect(self) -> list[str]:
|
||||
lines: list[str] = []
|
||||
while True:
|
||||
raw = self._serial.readline()
|
||||
if not raw:
|
||||
raise TimeoutError("AT command timed out")
|
||||
line = raw.decode(errors="ignore").strip()
|
||||
if not line:
|
||||
continue
|
||||
if DEBUG:
|
||||
print(f"SER << {line}", file=sys.stderr)
|
||||
if line == "OK":
|
||||
return lines
|
||||
if line == "ERROR" or line.startswith("+CME ERROR"):
|
||||
raise RuntimeError(f"AT command failed: {line}")
|
||||
lines.append(line)
|
||||
|
||||
def _ensure_serial(self, reconnect: bool = False) -> None:
|
||||
if reconnect:
|
||||
self.channel = None
|
||||
try:
|
||||
if self._serial:
|
||||
self._serial.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._serial = None
|
||||
if self._serial is None:
|
||||
self._serial = serial.Serial(self._device, baudrate=self._baud, timeout=self._timeout)
|
||||
|
||||
def query(self, cmd: str) -> list[str]:
|
||||
self._ensure_serial()
|
||||
try:
|
||||
self._send(cmd)
|
||||
return self._expect()
|
||||
except serial.SerialException:
|
||||
self._ensure_serial(reconnect=True)
|
||||
self._send(cmd)
|
||||
return self._expect()
|
||||
|
||||
def _open_isdr_once(self) -> None:
|
||||
if self.channel:
|
||||
try:
|
||||
self.query(f"AT+CCHC={self.channel}")
|
||||
except RuntimeError:
|
||||
pass
|
||||
self.channel = None
|
||||
# drain any unsolicited responses before opening
|
||||
if self._serial:
|
||||
try:
|
||||
self._serial.reset_input_buffer()
|
||||
except (OSError, serial.SerialException, termios.error):
|
||||
self._ensure_serial(reconnect=True)
|
||||
for line in self.query(f'AT+CCHO="{ISDR_AID}"'):
|
||||
if line.startswith("+CCHO:") and (ch := line.split(":", 1)[1].strip()):
|
||||
self.channel = ch
|
||||
return
|
||||
raise RuntimeError("Failed to open ISD-R application")
|
||||
|
||||
def _reset_modem(self) -> None:
|
||||
if self._serial:
|
||||
try:
|
||||
self._serial.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._serial = None
|
||||
subprocess.run(['/usr/comma/lte/lte.sh', 'start'], capture_output=True)
|
||||
|
||||
def open_isdr(self) -> None:
|
||||
for attempt in range(OPEN_ISDR_RETRIES):
|
||||
try:
|
||||
self._open_isdr_once()
|
||||
return
|
||||
except (RuntimeError, TimeoutError, termios.error, serial.SerialException):
|
||||
time.sleep(OPEN_ISDR_RETRY_DELAY_S)
|
||||
if attempt == OPEN_ISDR_RESET_ATTEMPT:
|
||||
self._reset_modem()
|
||||
raise RuntimeError("Failed to open ISD-R after retries")
|
||||
|
||||
def send_apdu(self, apdu: bytes) -> tuple[bytes, int, int]:
|
||||
for attempt in range(SEND_APDU_RETRIES):
|
||||
try:
|
||||
if not self.channel:
|
||||
self.open_isdr()
|
||||
hex_payload = apdu.hex().upper()
|
||||
for line in self.query(f'AT+CGLA={self.channel},{len(hex_payload)},"{hex_payload}"'):
|
||||
if line.startswith("+CGLA:"):
|
||||
parts = line.split(":", 1)[1].split(",", 1)
|
||||
if len(parts) == 2:
|
||||
data = bytes.fromhex(parts[1].strip().strip('"'))
|
||||
if len(data) >= 2:
|
||||
return data[:-2], data[-2], data[-1]
|
||||
raise RuntimeError("Missing +CGLA response")
|
||||
except (RuntimeError, ValueError):
|
||||
self.channel = None
|
||||
if attempt == SEND_APDU_RETRIES - 1:
|
||||
raise
|
||||
raise RuntimeError("send_apdu failed")
|
||||
|
||||
|
||||
# --- TLV utilities ---
|
||||
|
||||
def iter_tlv(data: bytes, with_positions: bool = False) -> Generator:
|
||||
idx, length = 0, len(data)
|
||||
while idx < length:
|
||||
start_pos = idx
|
||||
tag = data[idx]
|
||||
idx += 1
|
||||
if tag & 0x1F == 0x1F: # Multi-byte tag
|
||||
tag_value = tag
|
||||
while idx < length:
|
||||
next_byte = data[idx]
|
||||
idx += 1
|
||||
tag_value = (tag_value << 8) | next_byte
|
||||
if not (next_byte & 0x80):
|
||||
break
|
||||
else:
|
||||
tag_value = tag
|
||||
if idx >= length:
|
||||
break
|
||||
size = data[idx]
|
||||
idx += 1
|
||||
if size & 0x80: # Multi-byte length
|
||||
num_bytes = size & 0x7F
|
||||
if idx + num_bytes > length:
|
||||
break
|
||||
size = int.from_bytes(data[idx : idx + num_bytes], "big")
|
||||
idx += num_bytes
|
||||
if idx + size > length:
|
||||
break
|
||||
value = data[idx : idx + size]
|
||||
idx += size
|
||||
yield (tag_value, value, start_pos, idx) if with_positions else (tag_value, value)
|
||||
|
||||
|
||||
def find_tag(data: bytes, target: int) -> bytes | None:
|
||||
return next((v for t, v in iter_tlv(data) if t == target), None)
|
||||
|
||||
|
||||
def require_tag(data: bytes, target: int, label: str = "") -> bytes:
|
||||
v = find_tag(data, target)
|
||||
if v is None:
|
||||
raise RuntimeError(f"Missing {label or f'tag 0x{target:X}'}")
|
||||
return v
|
||||
|
||||
|
||||
def tbcd_to_string(raw: bytes) -> str:
|
||||
return "".join(str(n) for b in raw for n in (b & 0x0F, b >> 4) if n <= 9)
|
||||
|
||||
|
||||
def string_to_tbcd(s: str) -> bytes:
|
||||
digits = [int(c) for c in s if c.isdigit()]
|
||||
return bytes(digits[i] | ((digits[i + 1] if i + 1 < len(digits) else 0xF) << 4) for i in range(0, len(digits), 2))
|
||||
|
||||
|
||||
def encode_tlv(tag: int, value: bytes) -> bytes:
|
||||
tag_bytes = bytes([(tag >> 8) & 0xFF, tag & 0xFF]) if tag > 255 else bytes([tag])
|
||||
vlen = len(value)
|
||||
if vlen <= 127:
|
||||
return tag_bytes + bytes([vlen]) + value
|
||||
length_bytes = vlen.to_bytes((vlen.bit_length() + 7) // 8, "big")
|
||||
return tag_bytes + bytes([0x80 | len(length_bytes)]) + length_bytes + value
|
||||
|
||||
|
||||
def int_bytes(n: int) -> bytes:
|
||||
"""Encode a positive integer as minimal big-endian bytes (at least 1 byte)."""
|
||||
return n.to_bytes((n.bit_length() + 7) // 8 or 1, "big")
|
||||
|
||||
|
||||
PROFILE: FieldMap = {
|
||||
TAG_ICCID: ("iccid", tbcd_to_string),
|
||||
0x4F: ("isdpAid", lambda v: v.hex().upper()),
|
||||
0x9F70: ("profileState", lambda v: STATE_LABELS.get(v[0], "unknown")),
|
||||
0x90: ("profileNickname", lambda v: v.decode("utf-8", errors="ignore") or None),
|
||||
0x91: ("serviceProviderName", lambda v: v.decode("utf-8", errors="ignore") or None),
|
||||
0x92: ("profileName", lambda v: v.decode("utf-8", errors="ignore") or None),
|
||||
0x93: ("iconType", lambda v: ICON_LABELS.get(v[0], "unknown")),
|
||||
0x94: ("icon", b64e),
|
||||
0x95: ("profileClass", lambda v: CLASS_LABELS.get(v[0], "unknown")),
|
||||
}
|
||||
|
||||
|
||||
def decode_struct(data: bytes, field_map: FieldMap) -> dict[str, Any]:
|
||||
"""Parse TLV data using a {tag: (field_name, decoder)} map into a dict."""
|
||||
result: dict[str, Any] = {name: None for name, _ in field_map.values()}
|
||||
for tag, value in iter_tlv(data):
|
||||
if (field := field_map.get(tag)):
|
||||
result[field[0]] = field[1](value)
|
||||
return result
|
||||
|
||||
|
||||
# --- ES10x command transport ---
|
||||
|
||||
def es10x_command(client: AtClient, data: bytes) -> bytes:
|
||||
response = bytearray()
|
||||
sequence = 0
|
||||
offset = 0
|
||||
while offset < len(data):
|
||||
chunk = data[offset : offset + ES10X_MSS]
|
||||
offset += len(chunk)
|
||||
is_last = offset == len(data)
|
||||
apdu = bytes([0x80, 0xE2, 0x91 if is_last else 0x11, sequence & 0xFF, len(chunk)]) + chunk
|
||||
segment, sw1, sw2 = client.send_apdu(apdu)
|
||||
response.extend(segment)
|
||||
while True:
|
||||
if sw1 == 0x61: # More data available
|
||||
segment, sw1, sw2 = client.send_apdu(bytes([0x80, 0xC0, 0x00, 0x00, sw2 or 0]))
|
||||
response.extend(segment)
|
||||
continue
|
||||
if (sw1 & 0xF0) == 0x90:
|
||||
break
|
||||
raise RuntimeError(f"APDU failed with SW={sw1:02X}{sw2:02X}")
|
||||
sequence += 1
|
||||
return bytes(response)
|
||||
|
||||
|
||||
# --- Profile operations ---
|
||||
|
||||
NOTIFICATION: FieldMap = {
|
||||
TAG_STATUS: ("seqNumber", lambda v: int.from_bytes(v, "big")),
|
||||
0x81: ("profileManagementOperation",
|
||||
lambda v: NOTIFICATION_OPERATIONS.get(next((m for m in NOTIFICATION_OPERATIONS if len(v) >= 2 and v[1] & m), 0), "unknown")),
|
||||
0x0C: ("notificationAddress", lambda v: v.decode("utf-8", errors="ignore")),
|
||||
TAG_ICCID: ("iccid", tbcd_to_string),
|
||||
}
|
||||
|
||||
|
||||
def decode_profiles(blob: bytes) -> list[dict]:
|
||||
root = require_tag(blob, TAG_PROFILE_INFO_LIST, "ProfileInfoList")
|
||||
list_ok = find_tag(root, TAG_OK)
|
||||
if list_ok is None:
|
||||
return []
|
||||
return [decode_struct(value, PROFILE) for tag, value in iter_tlv(list_ok) if tag == 0xE3]
|
||||
|
||||
|
||||
def list_profiles(client: AtClient) -> list[dict]:
|
||||
return decode_profiles(es10x_command(client, TAG_PROFILE_INFO_LIST.to_bytes(2, "big") + b"\x00"))
|
||||
|
||||
|
||||
def set_profile_nickname(client: AtClient, iccid: str, nickname: str) -> None:
|
||||
nickname_bytes = nickname.encode("utf-8")
|
||||
if len(nickname_bytes) > 64:
|
||||
raise ValueError("Profile nickname must be 64 bytes or less")
|
||||
content = encode_tlv(TAG_ICCID, string_to_tbcd(iccid)) + encode_tlv(0x90, nickname_bytes)
|
||||
response = es10x_command(client, encode_tlv(TAG_SET_NICKNAME, content))
|
||||
code = require_tag(require_tag(response, TAG_SET_NICKNAME, "SetNicknameResponse"), TAG_STATUS, "SetNickname status")[0]
|
||||
if code == 0x01:
|
||||
raise LPAError(f"profile {iccid} not found")
|
||||
if code != 0x00:
|
||||
raise RuntimeError(f"SetNickname failed with status 0x{code:02X}")
|
||||
|
||||
|
||||
# --- ES9P HTTP ---
|
||||
|
||||
def es9p_request(smdp_address: str, endpoint: str, payload: dict, error_prefix: str = "Request", session: requests.Session | None = None) -> dict:
|
||||
url = f"https://{smdp_address}/gsma/rsp2/es9plus/{endpoint}"
|
||||
headers = {"User-Agent": "gsma-rsp-lpad", "X-Admin-Protocol": "gsma/rsp/v2.3.0", "Content-Type": "application/json"}
|
||||
http = session or requests
|
||||
resp = http.post(url, json=payload, headers=headers, timeout=HTTP_TIMEOUT, verify=GSMA_CI_BUNDLE)
|
||||
resp.raise_for_status()
|
||||
if not resp.content:
|
||||
return {}
|
||||
data = resp.json()
|
||||
if "header" in data and "functionExecutionStatus" in data["header"]:
|
||||
status = data["header"]["functionExecutionStatus"]
|
||||
if status.get("status") == "Failed":
|
||||
sd = status.get("statusCodeData", {})
|
||||
reason = sd.get("reasonCode", "unknown")
|
||||
subject = sd.get("subjectCode", "unknown")
|
||||
msg = ES9P_ERROR_MESSAGES.get((reason, subject),
|
||||
f"{error_prefix} failed: {reason}/{subject} - {sd.get('message', 'unknown')}")
|
||||
raise RuntimeError(msg)
|
||||
return data
|
||||
|
||||
|
||||
# --- Notifications ---
|
||||
|
||||
def list_notifications(client: AtClient) -> list[dict]:
|
||||
response = es10x_command(client, encode_tlv(TAG_LIST_NOTIFICATION, b""))
|
||||
root = require_tag(response, TAG_LIST_NOTIFICATION, "ListNotificationResponse")
|
||||
metadata_list = find_tag(root, TAG_OK)
|
||||
if metadata_list is None:
|
||||
return []
|
||||
return [decode_struct(value, NOTIFICATION) for tag, value in iter_tlv(metadata_list) if tag == TAG_NOTIFICATION_METADATA]
|
||||
|
||||
|
||||
def process_notifications(client: AtClient) -> None:
|
||||
for notification in list_notifications(client):
|
||||
seq_number, smdp_address = notification["seqNumber"], notification["notificationAddress"]
|
||||
try:
|
||||
request = encode_tlv(TAG_RETRIEVE_NOTIFICATION, encode_tlv(TAG_OK, encode_tlv(TAG_STATUS, int_bytes(seq_number))))
|
||||
response = es10x_command(client, request)
|
||||
content = require_tag(require_tag(response, TAG_RETRIEVE_NOTIFICATION, "RetrieveNotificationsListResponse"),
|
||||
TAG_OK, "RetrieveNotificationsListResponse")
|
||||
pending_notif = next((v for t, v in iter_tlv(content) if t in (TAG_PROFILE_INSTALL_RESULT, 0x30)), None)
|
||||
if pending_notif is None:
|
||||
raise RuntimeError("Missing PendingNotification")
|
||||
|
||||
es9p_request(smdp_address, "handleNotification", {"pendingNotification": b64e(pending_notif)}, "HandleNotification")
|
||||
|
||||
response = es10x_command(client, encode_tlv(TAG_NOTIFICATION_SENT, encode_tlv(TAG_STATUS, int_bytes(seq_number))))
|
||||
root = require_tag(response, TAG_NOTIFICATION_SENT, "NotificationSentResponse")
|
||||
if int.from_bytes(require_tag(root, TAG_STATUS, "RemoveNotificationFromList status"), "big") != 0:
|
||||
raise RuntimeError("RemoveNotificationFromList failed")
|
||||
except Exception as e:
|
||||
print(f"notification {seq_number} failed: {e}", file=sys.stderr)
|
||||
|
||||
|
||||
# --- Authentication & Download ---
|
||||
|
||||
def get_challenge_and_info(client: AtClient) -> tuple[bytes, bytes]:
|
||||
challenge_resp = es10x_command(client, encode_tlv(TAG_EUICC_CHALLENGE, b""))
|
||||
challenge = require_tag(require_tag(challenge_resp, TAG_EUICC_CHALLENGE, "GetEuiccDataResponse"),
|
||||
TAG_STATUS, "challenge in response")
|
||||
info_resp = es10x_command(client, encode_tlv(TAG_EUICC_INFO, b""))
|
||||
require_tag(info_resp, TAG_EUICC_INFO, "GetEuiccInfo1Response")
|
||||
return challenge, info_resp
|
||||
|
||||
|
||||
def authenticate_server(client: AtClient, b64_signed1: str, b64_sig1: str, b64_pk_id: str, b64_cert: str, matching_id: str) -> str:
|
||||
tac = bytes([0x35, 0x29, 0x06, 0x11])
|
||||
device_info = encode_tlv(TAG_STATUS, tac) + encode_tlv(0xA1, b"")
|
||||
ctx_inner = encode_tlv(TAG_STATUS, matching_id.encode("utf-8")) + encode_tlv(0xA1, device_info)
|
||||
content = b64d(b64_signed1) + b64d(b64_sig1) + b64d(b64_pk_id) + b64d(b64_cert) + encode_tlv(0xA0, ctx_inner)
|
||||
response = es10x_command(client, encode_tlv(TAG_AUTH_SERVER, content))
|
||||
root = require_tag(response, TAG_AUTH_SERVER, "AuthenticateServerResponse")
|
||||
error_tag = find_tag(root, 0xA1)
|
||||
if error_tag is not None:
|
||||
code = int.from_bytes(error_tag, "big") if error_tag else 0
|
||||
raise RuntimeError(f"AuthenticateServer rejected by eUICC: {AUTH_SERVER_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
|
||||
return b64e(response)
|
||||
|
||||
|
||||
def prepare_download(client: AtClient, b64_signed2: str, b64_sig2: str, b64_cert: str, cc: str | None = None) -> str:
|
||||
smdp_signed2 = b64d(b64_signed2)
|
||||
smdp_signature2 = b64d(b64_sig2)
|
||||
smdp_certificate = b64d(b64_cert)
|
||||
smdp_signed2_root = find_tag(smdp_signed2, 0x30)
|
||||
if smdp_signed2_root is None:
|
||||
raise RuntimeError("Invalid smdpSigned2")
|
||||
transaction_id = find_tag(smdp_signed2_root, TAG_STATUS)
|
||||
cc_required_flag = find_tag(smdp_signed2_root, 0x01)
|
||||
if transaction_id is None or cc_required_flag is None:
|
||||
raise RuntimeError("Invalid smdpSigned2")
|
||||
content = smdp_signed2 + smdp_signature2
|
||||
if int.from_bytes(cc_required_flag, "big") != 0:
|
||||
if not cc:
|
||||
raise RuntimeError("Confirmation code required but not provided")
|
||||
content += encode_tlv(0x04, hashlib.sha256(hashlib.sha256(cc.encode("utf-8")).digest() + transaction_id).digest())
|
||||
content += smdp_certificate
|
||||
response = es10x_command(client, encode_tlv(TAG_PREPARE_DOWNLOAD, content))
|
||||
require_tag(response, TAG_PREPARE_DOWNLOAD, "PrepareDownloadResponse")
|
||||
return b64e(response)
|
||||
|
||||
|
||||
def _parse_tlv_header_len(data: bytes) -> int:
|
||||
tag_len = 2 if data[0] & 0x1F == 0x1F else 1
|
||||
length_byte = data[tag_len]
|
||||
return tag_len + (1 + (length_byte & 0x7F) if length_byte & 0x80 else 1)
|
||||
|
||||
|
||||
def _split_bpp(bpp: bytes) -> list[bytes]:
|
||||
"""Split a BoundProfilePackage into APDU chunks per SGP.22 §5.7.6."""
|
||||
root_value = None
|
||||
for tag, value, start, end in iter_tlv(bpp, with_positions=True):
|
||||
if tag == TAG_BPP:
|
||||
root_value = value
|
||||
val_start = start + _parse_tlv_header_len(bpp[start:end])
|
||||
break
|
||||
if root_value is None:
|
||||
raise RuntimeError("Invalid BoundProfilePackage")
|
||||
|
||||
chunks: list[bytes] = []
|
||||
for tag, value, start, end in iter_tlv(root_value, with_positions=True):
|
||||
if tag == TAG_BPP_COMMAND:
|
||||
chunks.append(bpp[0 : val_start + end])
|
||||
elif tag in (0xA0, 0xA2):
|
||||
chunks.append(bpp[val_start + start : val_start + end])
|
||||
elif tag in (0xA1, 0xA3):
|
||||
hdr_len = _parse_tlv_header_len(root_value[start:end])
|
||||
chunks.append(bpp[val_start + start : val_start + start + hdr_len])
|
||||
for _, _, cs, ce in iter_tlv(value, with_positions=True):
|
||||
chunks.append(value[cs:ce])
|
||||
return chunks
|
||||
|
||||
|
||||
def _parse_install_result(response: bytes) -> dict[str, Any] | None:
|
||||
"""Parse a ProfileInstallResult from an APDU response, or None if not present."""
|
||||
root = find_tag(response, TAG_PROFILE_INSTALL_RESULT)
|
||||
if not root:
|
||||
return None
|
||||
result_data = find_tag(root, TAG_INSTALL_RESULT_DATA)
|
||||
if not result_data:
|
||||
return None
|
||||
result: dict[str, Any] = {"seqNumber": 0, "success": False, "bppCommandId": None, "errorReason": None}
|
||||
notif_meta = find_tag(result_data, TAG_NOTIFICATION_METADATA)
|
||||
if notif_meta:
|
||||
seq_num = find_tag(notif_meta, TAG_STATUS)
|
||||
if seq_num:
|
||||
result["seqNumber"] = int.from_bytes(seq_num, "big")
|
||||
final_result = find_tag(result_data, 0xA2)
|
||||
if final_result:
|
||||
for tag, value in iter_tlv(final_result):
|
||||
if tag == 0xA0:
|
||||
result["success"] = True
|
||||
elif tag == 0xA1:
|
||||
bpp_cmd = find_tag(value, TAG_STATUS)
|
||||
if bpp_cmd:
|
||||
result["bppCommandId"] = int.from_bytes(bpp_cmd, "big")
|
||||
err = find_tag(value, 0x81)
|
||||
if err:
|
||||
result["errorReason"] = int.from_bytes(err, "big")
|
||||
return result
|
||||
|
||||
|
||||
def load_bpp(client: AtClient, b64_bpp: str) -> dict:
|
||||
bpp = b64d(b64_bpp)
|
||||
result = None
|
||||
for chunk in _split_bpp(bpp):
|
||||
response = es10x_command(client, chunk)
|
||||
if response and (parsed := _parse_install_result(response)):
|
||||
result = parsed
|
||||
break
|
||||
|
||||
if result is None:
|
||||
raise RuntimeError("Profile installation failed: no result from eUICC")
|
||||
if not result["success"] and result["errorReason"] is not None:
|
||||
msg = BPP_ERROR_MESSAGES.get(result["errorReason"])
|
||||
if not msg:
|
||||
cmd_name = BPP_COMMAND_NAMES.get(result["bppCommandId"], f"unknown({result['bppCommandId']})")
|
||||
err_name = BPP_ERROR_REASONS.get(result["errorReason"], f"unknown({result['errorReason']})")
|
||||
msg = f"Profile installation failed at {cmd_name}: {err_name}"
|
||||
raise RuntimeError(msg)
|
||||
if not result["success"]:
|
||||
raise RuntimeError("Profile installation failed: no result from eUICC")
|
||||
return result
|
||||
|
||||
|
||||
def parse_metadata(b64_metadata: str) -> dict:
|
||||
root = find_tag(b64d(b64_metadata), TAG_PROFILE_METADATA)
|
||||
if root is None:
|
||||
raise RuntimeError("Invalid profileMetadata")
|
||||
return decode_struct(root, PROFILE)
|
||||
|
||||
|
||||
def cancel_session(client: AtClient, transaction_id: bytes, reason: int = 127) -> str:
|
||||
content = encode_tlv(0x80, transaction_id) + encode_tlv(0x81, bytes([reason]))
|
||||
response = es10x_command(client, encode_tlv(TAG_CANCEL_SESSION, content))
|
||||
return b64e(response)
|
||||
|
||||
|
||||
def parse_lpa_activation_code(activation_code: str) -> tuple[str, str]:
|
||||
"""Parse 'LPA:1$smdp.example.com$MATCHING-ID' into (smdp_address, matching_id)."""
|
||||
if not activation_code.startswith("LPA:"):
|
||||
raise ValueError("Invalid activation code format")
|
||||
parts = activation_code[4:].split("$")
|
||||
if len(parts) != 3:
|
||||
raise ValueError("Invalid activation code format")
|
||||
return parts[1], parts[2]
|
||||
|
||||
|
||||
def _b64_field(data: dict, key: str) -> str:
|
||||
return base64_trim(data[key])
|
||||
|
||||
|
||||
def _cancel_session_safe(client: AtClient, smdp: str, tx_id: str, session: requests.Session) -> None:
|
||||
b64_cancel = ""
|
||||
try:
|
||||
b64_cancel = cancel_session(client, b64d(tx_id))
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
es9p_request(smdp, "cancelSession", {"transactionId": tx_id, "cancelSessionResponse": b64_cancel}, "CancelSession", session=session)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def download_profile(client: AtClient, activation_code: str) -> str:
|
||||
"""Download and install an eSIM profile. Returns the ICCID of the installed profile."""
|
||||
if not system_time_valid():
|
||||
raise RuntimeError("System time is not set; TLS certificate validation requires a valid clock")
|
||||
smdp, matching_id = parse_lpa_activation_code(activation_code)
|
||||
challenge, euicc_info = get_challenge_and_info(client)
|
||||
session = requests.Session()
|
||||
tx_id = None
|
||||
|
||||
try:
|
||||
# step 1: initiate authentication
|
||||
auth = es9p_request(smdp, "initiateAuthentication", {
|
||||
"smdpAddress": smdp, "euiccChallenge": b64e(challenge),
|
||||
"euiccInfo1": b64e(euicc_info), "matchingId": matching_id,
|
||||
}, "Authentication", session=session)
|
||||
tx_id = _b64_field(auth, "transactionId")
|
||||
|
||||
# step 2: authenticate server
|
||||
b64_auth = authenticate_server(client,
|
||||
_b64_field(auth, "serverSigned1"), _b64_field(auth, "serverSignature1"),
|
||||
_b64_field(auth, "euiccCiPKIdToBeUsed"), _b64_field(auth, "serverCertificate"),
|
||||
matching_id)
|
||||
|
||||
# step 3: authenticate client + get metadata
|
||||
cli = es9p_request(smdp, "authenticateClient", {
|
||||
"transactionId": tx_id, "authenticateServerResponse": b64_auth,
|
||||
}, "Authentication", session=session)
|
||||
iccid = parse_metadata(_b64_field(cli, "profileMetadata"))["iccid"]
|
||||
|
||||
# step 4: prepare download
|
||||
b64_prep = prepare_download(client,
|
||||
_b64_field(cli, "smdpSigned2"), _b64_field(cli, "smdpSignature2"),
|
||||
_b64_field(cli, "smdpCertificate"))
|
||||
|
||||
# step 5: get and install bound profile package
|
||||
bpp = es9p_request(smdp, "getBoundProfilePackage", {
|
||||
"transactionId": tx_id, "prepareDownloadResponse": b64_prep,
|
||||
}, "GetBoundProfilePackage", session=session)
|
||||
load_bpp(client, _b64_field(bpp, "boundProfilePackage"))
|
||||
return iccid
|
||||
except Exception:
|
||||
if tx_id:
|
||||
_cancel_session_safe(client, smdp, tx_id, session)
|
||||
raise
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
class TiciLPA(LPABase):
|
||||
def __init__(self):
|
||||
if hasattr(self, '_client'):
|
||||
return
|
||||
self._client = AtClient(DEFAULT_DEVICE, DEFAULT_BAUD, DEFAULT_TIMEOUT)
|
||||
atexit.register(self._client.close)
|
||||
|
||||
@contextmanager
|
||||
def _acquire_lock(self):
|
||||
fd = os.open(LOCK_FILE, os.O_CREAT | os.O_RDWR)
|
||||
try:
|
||||
fcntl.flock(fd, fcntl.LOCK_EX)
|
||||
yield
|
||||
finally:
|
||||
fcntl.flock(fd, fcntl.LOCK_UN)
|
||||
os.close(fd)
|
||||
|
||||
@contextmanager
|
||||
def _acquire_channel(self):
|
||||
with self._acquire_lock():
|
||||
try:
|
||||
self._client.open_isdr()
|
||||
yield
|
||||
finally:
|
||||
if self._client.channel:
|
||||
try:
|
||||
self._client.query(f"AT+CCHC={self._client.channel}")
|
||||
except (RuntimeError, TimeoutError):
|
||||
pass
|
||||
self._client.channel = None
|
||||
|
||||
def list_profiles(self) -> list[Profile]:
|
||||
with self._acquire_channel():
|
||||
return [
|
||||
Profile(
|
||||
iccid=p.get("iccid", ""),
|
||||
nickname=p.get("profileNickname") or "",
|
||||
enabled=p.get("profileState") == "enabled",
|
||||
provider=p.get("serviceProviderName") or "",
|
||||
)
|
||||
for p in list_profiles(self._client)
|
||||
]
|
||||
|
||||
def get_active_profile(self) -> Profile | None:
|
||||
return None
|
||||
|
||||
def process_notifications(self) -> None:
|
||||
if not system_time_valid():
|
||||
raise RuntimeError("System time is not set; TLS certificate validation requires a valid clock")
|
||||
with self._acquire_channel():
|
||||
process_notifications(self._client)
|
||||
|
||||
def delete_profile(self, iccid: str) -> None:
|
||||
profile = next((p for p in self.list_profiles() if p.iccid == iccid), None)
|
||||
if profile is None:
|
||||
raise LPAProfileNotFoundError(f"profile not found: {iccid}")
|
||||
if profile.is_comma:
|
||||
raise LPAError("refusing to delete a comma profile")
|
||||
with self._acquire_channel():
|
||||
request = encode_tlv(TAG_DELETE_PROFILE, encode_tlv(TAG_ICCID, string_to_tbcd(iccid)))
|
||||
response = es10x_command(self._client, request)
|
||||
code = require_tag(require_tag(response, TAG_DELETE_PROFILE, "DeleteProfileResponse"), TAG_STATUS, "DeleteProfile status")[0]
|
||||
if code != PROFILE_OK:
|
||||
raise LPAError(f"DeleteProfile failed: {PROFILE_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
|
||||
|
||||
def download_profile(self, qr: str, nickname: str | None = None) -> None:
|
||||
with self._acquire_channel():
|
||||
iccid = download_profile(self._client, qr)
|
||||
if nickname and iccid:
|
||||
set_profile_nickname(self._client, iccid, nickname)
|
||||
|
||||
def nickname_profile(self, iccid: str, nickname: str) -> None:
|
||||
with self._acquire_channel():
|
||||
set_profile_nickname(self._client, iccid, nickname)
|
||||
|
||||
def _enable_profile(self, iccid: str) -> int:
|
||||
inner = encode_tlv(TAG_OK, encode_tlv(TAG_ICCID, string_to_tbcd(iccid)))
|
||||
inner += b'\x01\x01\x01' # refreshFlag=1
|
||||
response = es10x_command(self._client, encode_tlv(TAG_ENABLE_PROFILE, inner))
|
||||
return require_tag(require_tag(response, TAG_ENABLE_PROFILE, "EnableProfileResponse"), TAG_STATUS, "EnableProfile status")[0]
|
||||
|
||||
def switch_profile(self, iccid: str) -> None:
|
||||
with self._acquire_channel():
|
||||
code = self._enable_profile(iccid)
|
||||
if code == PROFILE_CAT_BUSY: # stale eUICC transaction, reset and retry
|
||||
self._client._reset_modem()
|
||||
self._client.open_isdr()
|
||||
code = self._enable_profile(iccid)
|
||||
if code not in (PROFILE_OK, PROFILE_NOT_IN_DISABLED_STATE):
|
||||
raise LPAError(f"EnableProfile failed: {PROFILE_ERROR_CODES.get(code, 'unknown')} (0x{code:02X})")
|
||||
|
||||
def is_euicc(self) -> bool:
|
||||
# +CCHO:<n> -> ISD-R applet present, eUICC. Any error -> non-eUICC.
|
||||
with self._acquire_lock():
|
||||
try:
|
||||
lines = self._client.query(f'AT+CCHO="{ISDR_AID}"')
|
||||
except RuntimeError:
|
||||
return False
|
||||
for line in lines:
|
||||
if line.startswith("+CCHO:") and (ch := line.split(":", 1)[1].strip()):
|
||||
try:
|
||||
self._client.query(f"AT+CCHC={ch}")
|
||||
except (RuntimeError, TimeoutError):
|
||||
pass
|
||||
self._client.channel = None
|
||||
return True
|
||||
return False
|
||||
Regular → Executable
+20
-2
@@ -1,3 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -10,10 +12,13 @@ def get_chunk_name(name, idx, num_chunks):
|
||||
def get_manifest_path(name):
|
||||
return f"{name}.chunkmanifest"
|
||||
|
||||
def get_chunk_paths(path, file_size):
|
||||
num_chunks = math.ceil(file_size / CHUNK_SIZE)
|
||||
def _chunk_paths(path, num_chunks):
|
||||
return [get_manifest_path(path)] + [get_chunk_name(path, i, num_chunks) for i in range(num_chunks)]
|
||||
|
||||
def get_chunk_targets(path, file_size):
|
||||
num_chunks = math.ceil(file_size / CHUNK_SIZE)
|
||||
return _chunk_paths(path, num_chunks)
|
||||
|
||||
def chunk_file(path, targets):
|
||||
manifest_path, *chunk_paths = targets
|
||||
with open(path, 'rb') as f:
|
||||
@@ -26,6 +31,13 @@ def chunk_file(path, targets):
|
||||
Path(manifest_path).write_text(str(len(chunk_paths)))
|
||||
os.remove(path)
|
||||
|
||||
def get_existing_chunks(path):
|
||||
if os.path.isfile(path):
|
||||
return [path]
|
||||
if os.path.isfile(manifest := get_manifest_path(path)):
|
||||
num_chunks = int(Path(manifest).read_text().strip())
|
||||
return _chunk_paths(path, num_chunks)
|
||||
raise FileNotFoundError(path)
|
||||
|
||||
def read_file_chunked(path):
|
||||
manifest_path = get_manifest_path(path)
|
||||
@@ -35,3 +47,9 @@ def read_file_chunked(path):
|
||||
if os.path.isfile(path):
|
||||
return Path(path).read_bytes()
|
||||
raise FileNotFoundError(path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
path = sys.argv[1]
|
||||
chunk_paths = get_chunk_targets(path, os.path.getsize(path))
|
||||
chunk_file(path, chunk_paths)
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
import os
|
||||
from typing import cast
|
||||
|
||||
from openpilot.common.hardware.base import HardwareBase
|
||||
from openpilot.common.hardware.tici.hardware import Tici
|
||||
from openpilot.common.hardware.pc.hardware import Pc
|
||||
|
||||
TICI = os.path.isfile('/TICI')
|
||||
AGNOS = os.path.isfile('/AGNOS')
|
||||
PC = not TICI
|
||||
|
||||
|
||||
if TICI:
|
||||
HARDWARE = cast(HardwareBase, Tici())
|
||||
else:
|
||||
HARDWARE = cast(HardwareBase, Pc())
|
||||
@@ -12,8 +12,6 @@ class HardwareNone {
|
||||
public:
|
||||
static std::string get_name() { return ""; }
|
||||
static cereal::InitData::DeviceType get_device_type() { return cereal::InitData::DeviceType::UNKNOWN; }
|
||||
static int get_voltage() { return 0; }
|
||||
static int get_current() { return 0; }
|
||||
|
||||
static std::string get_serial() { return "cccccc"; }
|
||||
|
||||
@@ -24,6 +22,4 @@ public:
|
||||
static void set_ir_power(int percentage) {}
|
||||
|
||||
static bool PC() { return false; }
|
||||
static bool TICI() { return false; }
|
||||
static bool AGNOS() { return false; }
|
||||
};
|
||||
@@ -0,0 +1,161 @@
|
||||
import os
|
||||
from abc import abstractmethod, ABC
|
||||
from dataclasses import dataclass, fields
|
||||
|
||||
from cereal import log
|
||||
from openpilot.common.esim.base import LPABase
|
||||
|
||||
NetworkType = log.DeviceState.NetworkType
|
||||
NetworkStrength = log.DeviceState.NetworkStrength
|
||||
|
||||
@dataclass
|
||||
class ThermalZone:
|
||||
# a zone from /sys/class/thermal/thermal_zone*
|
||||
name: str # a.k.a type
|
||||
scale: float = 1000. # scale to get degrees in C
|
||||
zone_number = -1
|
||||
|
||||
def read(self) -> float:
|
||||
if self.zone_number < 0:
|
||||
for n in os.listdir("/sys/devices/virtual/thermal"):
|
||||
if not n.startswith("thermal_zone"):
|
||||
continue
|
||||
with open(os.path.join("/sys/devices/virtual/thermal", n, "type")) as f:
|
||||
if f.read().strip() == self.name:
|
||||
self.zone_number = int(n.removeprefix("thermal_zone"))
|
||||
break
|
||||
|
||||
try:
|
||||
with open(f"/sys/devices/virtual/thermal/thermal_zone{self.zone_number}/temp") as f:
|
||||
return int(f.read()) / self.scale
|
||||
except FileNotFoundError:
|
||||
return 0
|
||||
|
||||
@dataclass
|
||||
class ThermalConfig:
|
||||
cpu: list[ThermalZone] | None = None
|
||||
gpu: list[ThermalZone] | None = None
|
||||
dsp: ThermalZone | None = None
|
||||
pmic: list[ThermalZone] | None = None
|
||||
memory: ThermalZone | None = None
|
||||
intake: ThermalZone | None = None
|
||||
exhaust: ThermalZone | None = None
|
||||
gnss: ThermalZone | None = None
|
||||
bottomSoc: ThermalZone | None = None
|
||||
|
||||
def get_msg(self):
|
||||
ret = {}
|
||||
for f in fields(ThermalConfig):
|
||||
v = getattr(self, f.name)
|
||||
if v is not None:
|
||||
if isinstance(v, list):
|
||||
ret[f.name + "TempC"] = [x.read() for x in v]
|
||||
else:
|
||||
ret[f.name + "TempC"] = v.read()
|
||||
return ret
|
||||
|
||||
class HardwareBase(ABC):
|
||||
@staticmethod
|
||||
def get_cmdline() -> dict[str, str]:
|
||||
with open('/proc/cmdline') as f:
|
||||
cmdline = f.read()
|
||||
return {kv[0]: kv[1] for kv in [s.split('=') for s in cmdline.split(' ')] if len(kv) == 2}
|
||||
|
||||
@staticmethod
|
||||
def read_param_file(path, parser, default=0):
|
||||
try:
|
||||
with open(path) as f:
|
||||
return parser(f.read())
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
def booted(self) -> bool:
|
||||
return True
|
||||
|
||||
def reboot(self, reason=None):
|
||||
print("REBOOT!")
|
||||
|
||||
def uninstall(self):
|
||||
print("uninstall")
|
||||
|
||||
def get_os_version(self):
|
||||
return None
|
||||
|
||||
@abstractmethod
|
||||
def get_device_type(self):
|
||||
pass
|
||||
|
||||
def get_imei(self) -> str:
|
||||
return ""
|
||||
|
||||
def get_serial(self):
|
||||
return ""
|
||||
|
||||
def get_network_info(self):
|
||||
return None
|
||||
|
||||
def get_network_type(self):
|
||||
return NetworkType.none
|
||||
|
||||
def get_sim_info(self):
|
||||
return {
|
||||
'sim_id': '',
|
||||
'mcc_mnc': None,
|
||||
'network_type': ["Unknown"],
|
||||
'sim_state': ["ABSENT"],
|
||||
'data_connected': False
|
||||
}
|
||||
|
||||
def get_sim_lpa(self) -> LPABase:
|
||||
raise NotImplementedError("SIM LPA not available")
|
||||
|
||||
def get_network_strength(self, network_type):
|
||||
return NetworkStrength.unknown
|
||||
|
||||
def get_network_metered(self, network_type) -> bool:
|
||||
return network_type not in (NetworkType.none, NetworkType.wifi, NetworkType.ethernet)
|
||||
|
||||
def get_current_power_draw(self):
|
||||
return 0
|
||||
|
||||
def get_som_power_draw(self):
|
||||
return 0
|
||||
|
||||
def shutdown(self):
|
||||
print("SHUTDOWN!")
|
||||
|
||||
def get_thermal_config(self):
|
||||
return ThermalConfig()
|
||||
|
||||
def set_display_power(self, on: bool):
|
||||
pass
|
||||
|
||||
def set_screen_brightness(self, percentage):
|
||||
pass
|
||||
|
||||
def get_screen_brightness(self):
|
||||
return 0
|
||||
|
||||
def set_power_save(self, powersave_enabled):
|
||||
pass
|
||||
|
||||
def get_gpu_usage_percent(self):
|
||||
return 0
|
||||
|
||||
def get_modem_temperatures(self):
|
||||
return []
|
||||
|
||||
def initialize_hardware(self):
|
||||
pass
|
||||
|
||||
def reset_internal_panda(self):
|
||||
pass
|
||||
|
||||
def recover_internal_panda(self):
|
||||
pass
|
||||
|
||||
def get_modem_data_usage(self):
|
||||
return -1, -1
|
||||
|
||||
def set_ir_power(self, percent: int):
|
||||
pass
|
||||
@@ -2,14 +2,14 @@
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "system/hardware/base.h"
|
||||
#include "common/hardware/base.h"
|
||||
#include "common/util.h"
|
||||
|
||||
#if __TICI__
|
||||
#include "system/hardware/tici/hardware.h"
|
||||
#include "common/hardware/tici/hardware.h"
|
||||
#define Hardware HardwareTici
|
||||
#else
|
||||
#include "system/hardware/pc/hardware.h"
|
||||
#include "common/hardware/pc/hardware.h"
|
||||
#define Hardware HardwarePC
|
||||
#endif
|
||||
|
||||
@@ -2,7 +2,7 @@ import os
|
||||
import platform
|
||||
from pathlib import Path
|
||||
|
||||
from openpilot.system.hardware import PC
|
||||
from openpilot.common.hardware import PC
|
||||
|
||||
DEFAULT_DOWNLOAD_CACHE_ROOT = "/tmp/comma_download_cache"
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "common/hardware/base.h"
|
||||
|
||||
class HardwarePC : public HardwareNone {
|
||||
public:
|
||||
static std::string get_name() { return "pc"; }
|
||||
static cereal::InitData::DeviceType get_device_type() { return cereal::InitData::DeviceType::PC; }
|
||||
static bool PC() { return true; }
|
||||
};
|
||||
@@ -0,0 +1,10 @@
|
||||
from cereal import log
|
||||
from openpilot.common.hardware.base import HardwareBase
|
||||
|
||||
class Pc(HardwareBase):
|
||||
def get_device_type(self):
|
||||
return "pc"
|
||||
|
||||
def get_network_type(self):
|
||||
# some stuff is gated on wifi, so just assume for now
|
||||
return log.DeviceState.NetworkType.wifi
|
||||
@@ -0,0 +1,84 @@
|
||||
[
|
||||
{
|
||||
"name": "xbl",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl-e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb.img.xz",
|
||||
"hash": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
|
||||
"hash_raw": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
|
||||
"size": 3282256,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "bea7f1a24428c3ededf672fa4fc78baf180cfbd8aafb77c974655b38517283e3"
|
||||
},
|
||||
{
|
||||
"name": "xbl_config",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85.img.xz",
|
||||
"hash": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
|
||||
"hash_raw": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
|
||||
"size": 98124,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "fb18cde08a98a168961ecd357e92474823046752b94e112f59fe51a6acd7197d"
|
||||
},
|
||||
{
|
||||
"name": "abl",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/abl-b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c.img.xz",
|
||||
"hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
|
||||
"hash_raw": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
|
||||
"size": 274432,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c"
|
||||
},
|
||||
{
|
||||
"name": "aop",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/aop-78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1.img.xz",
|
||||
"hash": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
|
||||
"hash_raw": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
|
||||
"size": 184364,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "6c9135446bd3fc075fcee59b887a12e49029ab1f98ed8d6d1e32c73569d47de3"
|
||||
},
|
||||
{
|
||||
"name": "devcfg",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc.img.xz",
|
||||
"hash": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
|
||||
"hash_raw": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
|
||||
"size": 40336,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "2a67971602012c1b43544964709da13c322786b456a8e78568b117e8b1540ce3"
|
||||
},
|
||||
{
|
||||
"name": "boot",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/boot-8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8.img.xz",
|
||||
"hash": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
|
||||
"hash_raw": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
|
||||
"size": 17487872,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "edca8bee1531e66953d107eeceeed2dc7b3ca46417e49d55508f94e58bf95db8"
|
||||
},
|
||||
{
|
||||
"name": "system",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img.xz",
|
||||
"hash": "78acfe16a7b62a3a91fc7a81f40a693e4468cec1c69df7d0b1e550aacc646113",
|
||||
"hash_raw": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
|
||||
"size": 4718592000,
|
||||
"sparse": true,
|
||||
"full_check": false,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "743142c5a898f27b2a1029cca42c8a5d5d1fc0096414422b850fe84c8d0b8342",
|
||||
"alt": {
|
||||
"hash": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img",
|
||||
"size": 4718592000
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -10,10 +10,7 @@ from collections.abc import Generator
|
||||
|
||||
import requests
|
||||
|
||||
import openpilot.system.updated.casync.casync as casync
|
||||
|
||||
SPARSE_CHUNK_FMT = struct.Struct('H2xI4x')
|
||||
CAIBX_URL = "https://commadist.azureedge.net/agnosupdate/"
|
||||
|
||||
AGNOS_MANIFEST_FILE = "system/hardware/tici/agnos.json"
|
||||
|
||||
@@ -187,50 +184,6 @@ def extract_compressed_image(target_slot_number: int, partition: dict, cloudlog)
|
||||
os.sync()
|
||||
|
||||
|
||||
def extract_casync_image(target_slot_number: int, partition: dict, cloudlog):
|
||||
path = get_partition_path(target_slot_number, partition)
|
||||
seed_path = path[:-1] + ('b' if path[-1] == 'a' else 'a')
|
||||
|
||||
target = casync.parse_caibx(partition['casync_caibx'])
|
||||
|
||||
sources: list[tuple[str, casync.ChunkReader, casync.ChunkDict]] = []
|
||||
|
||||
# First source is the current partition.
|
||||
try:
|
||||
raw_hash = get_raw_hash(seed_path, partition['size'])
|
||||
caibx_url = f"{CAIBX_URL}{partition['name']}-{raw_hash}.caibx"
|
||||
|
||||
try:
|
||||
cloudlog.info(f"casync fetching {caibx_url}")
|
||||
sources += [('seed', casync.FileChunkReader(seed_path), casync.build_chunk_dict(casync.parse_caibx(caibx_url)))]
|
||||
except requests.RequestException:
|
||||
cloudlog.error(f"casync failed to load {caibx_url}")
|
||||
except Exception:
|
||||
cloudlog.exception("casync failed to hash seed partition")
|
||||
|
||||
# Second source is the target partition, this allows for resuming
|
||||
sources += [('target', casync.FileChunkReader(path), casync.build_chunk_dict(target))]
|
||||
|
||||
# Finally we add the remote source to download any missing chunks
|
||||
sources += [('remote', casync.RemoteChunkReader(partition['casync_store']), casync.build_chunk_dict(target))]
|
||||
|
||||
last_p = 0
|
||||
|
||||
def progress(cur):
|
||||
nonlocal last_p
|
||||
p = int(cur / partition['size'] * 100)
|
||||
if p != last_p:
|
||||
last_p = p
|
||||
print(f"Installing {partition['name']}: {p}", flush=True)
|
||||
|
||||
stats = casync.extract(target, sources, path, progress)
|
||||
cloudlog.error(f'casync done {json.dumps(stats)}')
|
||||
|
||||
os.sync()
|
||||
if not verify_partition(target_slot_number, partition, force_full_check=True):
|
||||
raise Exception(f"Raw hash mismatch '{partition['hash_raw'].lower()}'")
|
||||
|
||||
|
||||
def flash_partition(target_slot_number: int, partition: dict, cloudlog, standalone=False):
|
||||
cloudlog.info(f"Downloading and writing {partition['name']}")
|
||||
|
||||
@@ -245,10 +198,7 @@ def flash_partition(target_slot_number: int, partition: dict, cloudlog, standalo
|
||||
|
||||
path = get_partition_path(target_slot_number, partition)
|
||||
|
||||
if ('casync_caibx' in partition) and not standalone:
|
||||
extract_casync_image(target_slot_number, partition, cloudlog)
|
||||
else:
|
||||
extract_compressed_image(target_slot_number, partition, cloudlog)
|
||||
extract_compressed_image(target_slot_number, partition, cloudlog)
|
||||
|
||||
# Write hash after successful flash
|
||||
if not full_check:
|
||||
+39
-39
@@ -130,47 +130,47 @@
|
||||
},
|
||||
{
|
||||
"name": "xbl",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl-dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6.img.xz",
|
||||
"hash": "dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6",
|
||||
"hash_raw": "dd45c0febdf0e022dab82ed0219370a86e8e6c0dfabfe29f3dab7eb1174d6bc6",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl-e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb.img.xz",
|
||||
"hash": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
|
||||
"hash_raw": "e8acf2a9cc7f0ce84cb803bfea9477f765c0d7b4daf26048e59651b9e6a7bfbb",
|
||||
"size": 3282256,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "d47a08914d2376557b03f1231b7233508222c04b57d781f9daf77c63eab92c2e"
|
||||
"ondevice_hash": "bea7f1a24428c3ededf672fa4fc78baf180cfbd8aafb77c974655b38517283e3"
|
||||
},
|
||||
{
|
||||
"name": "xbl_config",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9.img.xz",
|
||||
"hash": "1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9",
|
||||
"hash_raw": "1074ae051df159ba6dba988d8f6ba2cfc304ed1466cce0db531df6f7b1e44aa9",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/xbl_config-758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85.img.xz",
|
||||
"hash": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
|
||||
"hash_raw": "758552ecf92b5569677197783bf0ccb73d7f961685308e45d3276ac9dd974f85",
|
||||
"size": 98124,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "e7d04d9f040c9c040cdf013335d0b6d6e9346311458baeb2461b193e954f5f1c"
|
||||
"ondevice_hash": "fb18cde08a98a168961ecd357e92474823046752b94e112f59fe51a6acd7197d"
|
||||
},
|
||||
{
|
||||
"name": "abl",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/abl-556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee.img.xz",
|
||||
"hash": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee",
|
||||
"hash_raw": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/abl-b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c.img.xz",
|
||||
"hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
|
||||
"hash_raw": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c",
|
||||
"size": 274432,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "556bbb4ed1c671402b217bd2f3c07edce4f88b0bbd64e92241b82e396aa9ebee"
|
||||
"ondevice_hash": "b6fba807b9bcd66a31f2afb0eba5163ec239693ad32e2e4200f6c356adfe098c"
|
||||
},
|
||||
{
|
||||
"name": "aop",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/aop-4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788.img.xz",
|
||||
"hash": "4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788",
|
||||
"hash_raw": "4d925c9248672e4a69a236991983375008c44997a854ee7846d1b5fd7c787788",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/aop-78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1.img.xz",
|
||||
"hash": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
|
||||
"hash_raw": "78b2287ca219a0811b3004c523fa0f4749e4d1fd92be3aba61699305b7943ad1",
|
||||
"size": 184364,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "3aa0a79149ec57f4bc8c38f7bbdf4f6630dd659e49a111ce6258d2d06a07c8e5"
|
||||
"ondevice_hash": "6c9135446bd3fc075fcee59b887a12e49029ab1f98ed8d6d1e32c73569d47de3"
|
||||
},
|
||||
{
|
||||
"name": "bluetooth",
|
||||
@@ -207,14 +207,14 @@
|
||||
},
|
||||
{
|
||||
"name": "devcfg",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585.img.xz",
|
||||
"hash": "2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585",
|
||||
"hash_raw": "2f374581243910db92f62bb13bd66ec8e3d56d434997ba007ded06d2d6cc8585",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/devcfg-f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc.img.xz",
|
||||
"hash": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
|
||||
"hash_raw": "f71df3a86958c093ba3969254c4db025187eef9385427f1ade946742939b43cc",
|
||||
"size": 40336,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "3d7bb33588491a2a40091a7e1cf6cb65e6dd503f69b640aba484d723f1ad47e8"
|
||||
"ondevice_hash": "2a67971602012c1b43544964709da13c322786b456a8e78568b117e8b1540ce3"
|
||||
},
|
||||
{
|
||||
"name": "devinfo",
|
||||
@@ -339,51 +339,51 @@
|
||||
},
|
||||
{
|
||||
"name": "boot",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/boot-d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51.img.xz",
|
||||
"hash": "d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51",
|
||||
"hash_raw": "d726315cf98a43e1090e5b49297404cf3d084cfbd42ad8bb7d8afb68136b9f51",
|
||||
"size": 17500160,
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/boot-8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8.img.xz",
|
||||
"hash": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
|
||||
"hash_raw": "8806802b195a5b1396a3ae8dd92a8b7711dc522f6aceafd820e871bae5c8a6d8",
|
||||
"size": 17487872,
|
||||
"sparse": false,
|
||||
"full_check": true,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "2454108de1161289bc4a75449ad6421f1772b13b3e5cba68a84fca7530557699"
|
||||
"ondevice_hash": "edca8bee1531e66953d107eeceeed2dc7b3ca46417e49d55508f94e58bf95db8"
|
||||
},
|
||||
{
|
||||
"name": "system",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5.img.xz",
|
||||
"hash": "5f319030ad05942267b77f1a4686c4ca24cc09b2c2a4688e57342ffc9720fd49",
|
||||
"hash_raw": "dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img.xz",
|
||||
"hash": "78acfe16a7b62a3a91fc7a81f40a693e4468cec1c69df7d0b1e550aacc646113",
|
||||
"hash_raw": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
|
||||
"size": 4718592000,
|
||||
"sparse": true,
|
||||
"full_check": false,
|
||||
"has_ab": true,
|
||||
"ondevice_hash": "c12f1b7d790a418aea17424accf4cd59c575e5745cad82bdc9452f384483648c",
|
||||
"ondevice_hash": "743142c5a898f27b2a1029cca42c8a5d5d1fc0096414422b850fe84c8d0b8342",
|
||||
"alt": {
|
||||
"hash": "dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-dcdea6bd675d0276a63c25151727829620794baf42ada2e5e19a3f77b3f583a5.img",
|
||||
"hash": "ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/system-ef0d879302cb29e72110e9c8d3f947c830fd7d37c8192744fc9dbea1af78501f.img",
|
||||
"size": 4718592000
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "userdata_90",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/userdata_90-a7b25ea29255f4fd3a2da99e037f40b4ca10bd4afd57dd96563353b8dfb0f634.img.xz",
|
||||
"hash": "7ea9d7d4685ec36bbfdf06afe0b51650d567416c3092fef96bd97158ed322742",
|
||||
"hash_raw": "a7b25ea29255f4fd3a2da99e037f40b4ca10bd4afd57dd96563353b8dfb0f634",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/userdata_90-14a3fc6e9bd148b9deebf6ae9df2f1b3b759e629b337e41b6895864cdd51f630.img.xz",
|
||||
"hash": "52160dd01b30b3dc572226e8d549a034b03bc328e80f1f4cd6a857b6dd447687",
|
||||
"hash_raw": "14a3fc6e9bd148b9deebf6ae9df2f1b3b759e629b337e41b6895864cdd51f630",
|
||||
"size": 96636764160,
|
||||
"sparse": true,
|
||||
"full_check": true,
|
||||
"has_ab": false,
|
||||
"ondevice_hash": "79ed653c1679d84b13ee23083a511b0e668454e4af9b0db99a3279072ed041c1"
|
||||
"ondevice_hash": "3bbc052c7793087946b0cd668c1778f930084f6b00896aeebd193dfce53fa518"
|
||||
},
|
||||
{
|
||||
"name": "userdata_89",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/userdata_89-8e428632c967aa609cac184bff938a90240e53ffd3b4fca40bc94c33c81202ba.img.xz",
|
||||
"hash": "7104cdb0384e4ecb1ebfa6136a2330251bc8aa829b9ec48c4b740f656252d382",
|
||||
"hash_raw": "8e428632c967aa609cac184bff938a90240e53ffd3b4fca40bc94c33c81202ba",
|
||||
"url": "https://commadist.azureedge.net/agnosupdate/userdata_89-425c69d021f4ee2f767963bf7b991d1a492485fa465c5f5d001e1cf7de3d62a0.img.xz",
|
||||
"hash": "02a8c5512754d7781d930d242be3fad01fbce652297c69dfe8722dc92b18dc09",
|
||||
"hash_raw": "425c69d021f4ee2f767963bf7b991d1a492485fa465c5f5d001e1cf7de3d62a0",
|
||||
"size": 95563022336,
|
||||
"sparse": true,
|
||||
"full_check": true,
|
||||
"has_ab": false,
|
||||
"ondevice_hash": "fbede3b0831dbc4a4edd336e5f547f4978902b9421fb1484e86c416192c59165"
|
||||
"ondevice_hash": "3667d500f91b08a4671d7c09623eb6ae8fc905d9273893b50689e214005c7fa6"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,87 @@
|
||||
#pragma once
|
||||
|
||||
#include <cassert>
|
||||
#include <fstream>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <algorithm> // for std::clamp
|
||||
|
||||
#include "common/util.h"
|
||||
#include "common/hardware/base.h"
|
||||
|
||||
class HardwareTici : public HardwareNone {
|
||||
public:
|
||||
static std::string get_name() {
|
||||
static const std::string name = []() {
|
||||
std::string model = util::read_file("/sys/firmware/devicetree/base/model");
|
||||
return util::strip(model.substr(std::string("comma ").size()));
|
||||
}();
|
||||
return name;
|
||||
}
|
||||
|
||||
static cereal::InitData::DeviceType get_device_type() {
|
||||
static const std::map<std::string, cereal::InitData::DeviceType> device_map = {
|
||||
{"tizi", cereal::InitData::DeviceType::TIZI},
|
||||
{"mici", cereal::InitData::DeviceType::MICI}
|
||||
};
|
||||
static const auto it = device_map.find(get_name());
|
||||
assert(it != device_map.end());
|
||||
return it->second;
|
||||
}
|
||||
|
||||
static std::string get_serial() {
|
||||
static std::string serial("");
|
||||
if (serial.empty()) {
|
||||
std::ifstream stream("/proc/cmdline");
|
||||
std::string cmdline;
|
||||
std::getline(stream, cmdline);
|
||||
|
||||
auto start = cmdline.find("serialno=");
|
||||
if (start == std::string::npos) {
|
||||
serial = "cccccc";
|
||||
} else {
|
||||
auto end = cmdline.find(" ", start + 9);
|
||||
serial = cmdline.substr(start + 9, end - start - 9);
|
||||
}
|
||||
}
|
||||
return serial;
|
||||
}
|
||||
|
||||
static void set_ir_power(int percent) {
|
||||
auto device = get_device_type();
|
||||
if (device == cereal::InitData::DeviceType::TIZI) {
|
||||
return;
|
||||
}
|
||||
|
||||
int value = util::map_val(std::clamp(percent, 0, 100), 0, 100, 0, 300);
|
||||
std::ofstream("/sys/class/leds/led:switch_2/brightness") << 0 << "\n";
|
||||
std::ofstream("/sys/class/leds/led:torch_2/brightness") << value << "\n";
|
||||
std::ofstream("/sys/class/leds/led:switch_2/brightness") << value << "\n";
|
||||
}
|
||||
|
||||
static std::map<std::string, std::string> get_init_logs() {
|
||||
std::map<std::string, std::string> ret = {
|
||||
{"/BUILD", util::read_file("/BUILD")},
|
||||
{"lsblk", util::check_output("lsblk -o NAME,SIZE,STATE,VENDOR,MODEL,REV,SERIAL")},
|
||||
{"SOM ID", util::read_file("/sys/devices/platform/vendor/vendor:gpio-som-id/som_id")},
|
||||
};
|
||||
|
||||
std::string bs = util::check_output("abctl --boot_slot");
|
||||
ret["boot slot"] = bs.substr(0, bs.find_first_of("\n"));
|
||||
|
||||
std::string temp = util::read_file("/dev/disk/by-partlabel/ssd");
|
||||
temp.erase(temp.find_last_not_of(std::string("\0\r\n", 3))+1);
|
||||
ret["boot temp"] = temp;
|
||||
|
||||
// TODO: log something from system and boot
|
||||
for (std::string part : {"xbl", "abl", "aop", "devcfg", "xbl_config"}) {
|
||||
for (std::string slot : {"a", "b"}) {
|
||||
std::string partition = part + "_" + slot;
|
||||
std::string hash = util::check_output("sha256sum /dev/disk/by-partlabel/" + partition);
|
||||
ret[partition] = hash.substr(0, hash.find_first_of(" "));
|
||||
}
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,419 @@
|
||||
import configparser
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import time
|
||||
from functools import cached_property, lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
from cereal import log
|
||||
from openpilot.common.utils import sudo_read, sudo_write
|
||||
from openpilot.common.gpio import gpio_set, gpio_init, get_irqs_for_action
|
||||
from openpilot.common.esim.base import LPABase
|
||||
from openpilot.common.hardware.base import HardwareBase, ThermalConfig, ThermalZone
|
||||
from openpilot.common.esim.lpa import TiciLPA
|
||||
from openpilot.common.hardware.tici.pins import GPIO
|
||||
from openpilot.common.hardware.tici.amplifier import Amplifier
|
||||
|
||||
MODEM_STATE_PATH = "/dev/shm/modem"
|
||||
|
||||
NetworkType = log.DeviceState.NetworkType
|
||||
NetworkStrength = log.DeviceState.NetworkStrength
|
||||
|
||||
|
||||
def affine_irq(val, action):
|
||||
irqs = get_irqs_for_action(action)
|
||||
if len(irqs) == 0:
|
||||
print(f"No IRQs found for '{action}'")
|
||||
return
|
||||
|
||||
for i in irqs:
|
||||
sudo_write(str(val), f"/proc/irq/{i}/smp_affinity_list")
|
||||
|
||||
@lru_cache
|
||||
def get_device_type():
|
||||
# lru_cache and cache can cause memory leaks when used in classes
|
||||
with open("/sys/firmware/devicetree/base/model") as f:
|
||||
model = f.read().strip('\x00')
|
||||
return model.split('comma ')[-1]
|
||||
|
||||
def wpa_supplicant_cmd(cmd: str, timeout: float = 0.2) -> dict[str, str]:
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
|
||||
sock.settimeout(timeout)
|
||||
sock.bind(f"\0openpilot-wpa-{os.getpid()}-{time.monotonic_ns()}")
|
||||
sock.connect("/run/wpa_supplicant/wlan0")
|
||||
sock.send(cmd.encode())
|
||||
|
||||
while True:
|
||||
out = sock.recv(8192).decode("utf-8", "replace")
|
||||
if out.startswith("<"):
|
||||
continue
|
||||
if out.startswith("FAIL"):
|
||||
return {}
|
||||
return dict(l.split("=", 1) for l in out.splitlines() if "=" in l)
|
||||
|
||||
def get_default_route_iface():
|
||||
with open("/proc/net/route") as f:
|
||||
routes = [(int(route[6]), route[0]) for line in f.readlines()[1:] if (route := line.split())[1] == "00000000" and int(route[3], 16) & 0x1]
|
||||
return min(routes)[1] if routes else None
|
||||
|
||||
class Tici(HardwareBase):
|
||||
@cached_property
|
||||
def amplifier(self):
|
||||
if self.get_device_type() == "mici":
|
||||
return None
|
||||
return Amplifier()
|
||||
|
||||
def get_modem_state(self) -> dict:
|
||||
try:
|
||||
with open(MODEM_STATE_PATH) as f:
|
||||
return json.load(f)
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return {}
|
||||
|
||||
def get_os_version(self):
|
||||
with open("/VERSION") as f:
|
||||
return f.read().strip()
|
||||
|
||||
def get_device_type(self):
|
||||
return get_device_type()
|
||||
|
||||
def reboot(self, reason=None):
|
||||
subprocess.check_output(["sudo", "reboot"])
|
||||
|
||||
def uninstall(self):
|
||||
Path("/data/__system_reset__").touch()
|
||||
os.sync()
|
||||
self.reboot()
|
||||
|
||||
def get_serial(self):
|
||||
return self.get_cmdline()['androidboot.serialno']
|
||||
|
||||
def get_voltage(self):
|
||||
with open("/sys/class/hwmon/hwmon1/in1_input") as f:
|
||||
return int(f.read())
|
||||
|
||||
def get_current(self):
|
||||
with open("/sys/class/hwmon/hwmon1/curr1_input") as f:
|
||||
return int(f.read())
|
||||
|
||||
def set_ir_power(self, percent: int):
|
||||
if self.get_device_type() == "tizi":
|
||||
return
|
||||
|
||||
value = int((percent / 100) * 300)
|
||||
with open("/sys/class/leds/led:switch_2/brightness", "w") as f:
|
||||
f.write("0\n")
|
||||
with open("/sys/class/leds/led:torch_2/brightness", "w") as f:
|
||||
f.write(f"{value}\n")
|
||||
with open("/sys/class/leds/led:switch_2/brightness", "w") as f:
|
||||
f.write(f"{value}\n")
|
||||
|
||||
def get_network_type(self):
|
||||
try:
|
||||
if (iface := get_default_route_iface()):
|
||||
if iface.startswith('wlan'):
|
||||
return NetworkType.wifi
|
||||
if iface.startswith('eth'):
|
||||
return NetworkType.ethernet
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
ms = self.get_modem_state()
|
||||
if ms.get('connected'):
|
||||
nt = ms.get('network_type', '')
|
||||
if nt == 'nr':
|
||||
return NetworkType.cell5G
|
||||
elif nt == 'lte':
|
||||
return NetworkType.cell4G
|
||||
elif nt in ('utran', 'umts'):
|
||||
return NetworkType.cell3G
|
||||
elif nt == 'gsm':
|
||||
return NetworkType.cell2G
|
||||
return NetworkType.none
|
||||
|
||||
def get_sim_info(self):
|
||||
ms = self.get_modem_state()
|
||||
sim_id = ms.get('iccid', '')
|
||||
return {
|
||||
'sim_id': sim_id,
|
||||
'mcc_mnc': ms.get('mcc_mnc') or None,
|
||||
'network_type': ["Unknown"],
|
||||
'sim_state': ["ABSENT"] if not sim_id else ["READY"],
|
||||
'data_connected': ms.get('connected', False),
|
||||
}
|
||||
|
||||
def get_sim_lpa(self) -> LPABase:
|
||||
return TiciLPA()
|
||||
|
||||
def get_imei(self):
|
||||
return self.get_modem_state().get('imei', '')
|
||||
|
||||
def get_network_info(self):
|
||||
if self.get_device_type() == "mici":
|
||||
return None
|
||||
|
||||
ms = self.get_modem_state()
|
||||
return {
|
||||
'technology': ms.get('network_type', '').upper() if ms.get('network_type') else '',
|
||||
'operator': ms.get('operator', ''),
|
||||
'band': ms.get('band', ''),
|
||||
'channel': ms.get('channel', 0),
|
||||
'extra': ms.get('extra', ''),
|
||||
'state': ms.get('state', 'UNKNOWN'),
|
||||
}
|
||||
|
||||
def parse_strength(self, percentage):
|
||||
if percentage < 25:
|
||||
return NetworkStrength.poor
|
||||
elif percentage < 50:
|
||||
return NetworkStrength.moderate
|
||||
elif percentage < 75:
|
||||
return NetworkStrength.good
|
||||
else:
|
||||
return NetworkStrength.great
|
||||
|
||||
def get_network_strength(self, network_type):
|
||||
network_strength = NetworkStrength.unknown
|
||||
|
||||
try:
|
||||
if network_type == NetworkType.none:
|
||||
pass
|
||||
elif network_type == NetworkType.ethernet:
|
||||
network_strength = NetworkStrength.great
|
||||
elif network_type == NetworkType.wifi:
|
||||
rssi = wpa_supplicant_cmd("SIGNAL_POLL").get("RSSI")
|
||||
if rssi is not None:
|
||||
dbm = int(rssi)
|
||||
if -100 < dbm <= 0:
|
||||
network_strength = self.parse_strength(120 + max(-100, min(-20, dbm)))
|
||||
else: # Cellular
|
||||
network_strength = self.parse_strength(self.get_modem_state().get('signal_quality', 0))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return network_strength
|
||||
|
||||
def get_network_metered(self, network_type) -> bool:
|
||||
if network_type in (NetworkType.cell2G, NetworkType.cell3G, NetworkType.cell4G, NetworkType.cell5G):
|
||||
from openpilot.common.params import Params
|
||||
return Params().get_bool("GsmMetered")
|
||||
try:
|
||||
if network_type == NetworkType.wifi:
|
||||
ssid = wpa_supplicant_cmd("STATUS").get("ssid", "")
|
||||
if ssid:
|
||||
# wpa_supplicant escapes non-printable bytes as \xNN; NM keyfile stores ASCII SSIDs as a literal and others as a byte;byte; list
|
||||
ssid_bytes = ssid.encode().decode('unicode_escape').encode('latin-1')
|
||||
ssid_keyfile_list = ';'.join(str(b) for b in ssid_bytes) + ';'
|
||||
|
||||
nm_dirs = ("/run/NetworkManager/system-connections", "/data/etc/NetworkManager/system-connections")
|
||||
for fpath in (p for d in nm_dirs for p in Path(d).glob("*.nmconnection")):
|
||||
raw = sudo_read(str(fpath))
|
||||
if not raw:
|
||||
continue
|
||||
cp = configparser.ConfigParser(interpolation=None)
|
||||
try:
|
||||
cp.read_string(raw)
|
||||
keyfile_ssid = cp.get("wifi", "ssid", fallback="")
|
||||
if keyfile_ssid != ssid and keyfile_ssid != ssid_keyfile_list:
|
||||
continue
|
||||
metered = cp.getint("connection", "metered", fallback=0)
|
||||
except (configparser.Error, ValueError):
|
||||
continue
|
||||
if metered == 1: # NM_METERED_YES
|
||||
return True
|
||||
if metered == 2: # NM_METERED_NO
|
||||
return False
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return super().get_network_metered(network_type)
|
||||
|
||||
def get_modem_temperatures(self):
|
||||
return self.get_modem_state().get('temperatures', [])
|
||||
|
||||
def get_current_power_draw(self):
|
||||
return (self.read_param_file("/sys/class/hwmon/hwmon1/power1_input", int) / 1e6)
|
||||
|
||||
def get_som_power_draw(self):
|
||||
return (self.read_param_file("/sys/class/power_supply/bms/voltage_now", int) * self.read_param_file("/sys/class/power_supply/bms/current_now", int) / 1e12)
|
||||
|
||||
def shutdown(self):
|
||||
os.system("sudo poweroff")
|
||||
|
||||
def get_thermal_config(self):
|
||||
intake, exhaust, gnss, bottomSoc = None, None, None, None
|
||||
if self.get_device_type() == "mici":
|
||||
gnss = ThermalZone("gnss")
|
||||
intake = ThermalZone("intake")
|
||||
exhaust = ThermalZone("exhaust")
|
||||
bottomSoc = ThermalZone("bottom_soc")
|
||||
return ThermalConfig(cpu=[ThermalZone(f"cpu{i}-silver-usr") for i in range(4)] +
|
||||
[ThermalZone(f"cpu{i}-gold-usr") for i in range(4)],
|
||||
gpu=[ThermalZone("gpu0-usr"), ThermalZone("gpu1-usr")],
|
||||
dsp=ThermalZone("compute-hvx-usr"),
|
||||
memory=ThermalZone("ddr-usr"),
|
||||
pmic=[ThermalZone("pm8998_tz"), ThermalZone("pm8005_tz")],
|
||||
intake=intake,
|
||||
exhaust=exhaust,
|
||||
gnss=gnss,
|
||||
bottomSoc=bottomSoc)
|
||||
|
||||
def set_display_power(self, on):
|
||||
try:
|
||||
with open("/sys/class/backlight/panel0-backlight/bl_power", "w") as f:
|
||||
f.write("0" if on else "4")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def set_screen_brightness(self, percentage):
|
||||
try:
|
||||
with open("/sys/class/backlight/panel0-backlight/max_brightness") as f:
|
||||
max_brightness = float(f.read().strip())
|
||||
|
||||
val = int(percentage * (max_brightness / 100.))
|
||||
with open("/sys/class/backlight/panel0-backlight/brightness", "w") as f:
|
||||
f.write(str(val))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def get_screen_brightness(self):
|
||||
try:
|
||||
with open("/sys/class/backlight/panel0-backlight/max_brightness") as f:
|
||||
max_brightness = float(f.read().strip())
|
||||
|
||||
with open("/sys/class/backlight/panel0-backlight/brightness") as f:
|
||||
return int(float(f.read()) / (max_brightness / 100.))
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
def set_power_save(self, powersave_enabled):
|
||||
# amplifier, 100mW at idle
|
||||
if self.amplifier is not None:
|
||||
self.amplifier.set_global_shutdown(amp_disabled=powersave_enabled)
|
||||
if not powersave_enabled:
|
||||
self.amplifier.initialize_configuration()
|
||||
|
||||
# *** CPU config ***
|
||||
|
||||
# offline big cluster
|
||||
for i in range(4, 8):
|
||||
val = '0' if powersave_enabled else '1'
|
||||
sudo_write(val, f'/sys/devices/system/cpu/cpu{i}/online')
|
||||
|
||||
for n in ('0', '4'):
|
||||
if powersave_enabled and n == '4':
|
||||
continue
|
||||
gov = 'ondemand' if powersave_enabled else 'performance'
|
||||
sudo_write(gov, f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_governor')
|
||||
if not powersave_enabled:
|
||||
# cap max core freq to 1689 Mhz
|
||||
sudo_write('1689600', f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_max_freq')
|
||||
|
||||
# *** IRQ config ***
|
||||
|
||||
# GPU, modeld core
|
||||
affine_irq(7, "kgsl-3d0")
|
||||
|
||||
# camerad core
|
||||
camera_irqs = ("a5", "cci", "cpas_camnoc", "cpas-cdm", "csid", "ife", "csid-lite", "ife-lite")
|
||||
for n in camera_irqs:
|
||||
affine_irq(6, n)
|
||||
|
||||
def get_gpu_usage_percent(self):
|
||||
try:
|
||||
with open('/sys/class/kgsl/kgsl-3d0/gpubusy') as f:
|
||||
used, total = f.read().strip().split()
|
||||
return 100.0 * int(used) / int(total)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
def initialize_hardware(self):
|
||||
if self.amplifier is not None:
|
||||
self.amplifier.initialize_configuration()
|
||||
|
||||
# Allow hardwared to write engagement status to kmsg
|
||||
os.system("sudo chmod a+w /dev/kmsg")
|
||||
|
||||
# Ensure fan gpio is enabled so fan runs until shutdown, also turned on at boot by the ABL
|
||||
gpio_init(GPIO.SOM_ST_IO, True)
|
||||
gpio_set(GPIO.SOM_ST_IO, 1)
|
||||
|
||||
# *** IRQ config ***
|
||||
|
||||
# mask off big cluster from default affinity
|
||||
sudo_write("f", "/proc/irq/default_smp_affinity")
|
||||
|
||||
# move these off the default core
|
||||
affine_irq(1, "msm_vidc") # encoders
|
||||
affine_irq(1, "i2c_geni") # sensors
|
||||
|
||||
# *** GPU config ***
|
||||
# https://github.com/commaai/agnos-kernel-sdm845/blob/master/arch/arm64/boot/dts/qcom/sdm845-gpu.dtsi#L216
|
||||
affine_irq(5, "fts_ts") # touch
|
||||
affine_irq(5, "msm_drm") # display
|
||||
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/min_pwrlevel")
|
||||
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/max_pwrlevel")
|
||||
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_bus_on")
|
||||
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_clk_on")
|
||||
sudo_write("1", "/sys/class/kgsl/kgsl-3d0/force_rail_on")
|
||||
sudo_write("1000", "/sys/class/kgsl/kgsl-3d0/idle_timer")
|
||||
sudo_write("performance", "/sys/class/kgsl/kgsl-3d0/devfreq/governor")
|
||||
sudo_write("710", "/sys/class/kgsl/kgsl-3d0/max_clock_mhz")
|
||||
|
||||
# setup governors
|
||||
sudo_write("performance", "/sys/class/devfreq/soc:qcom,cpubw/governor")
|
||||
sudo_write("performance", "/sys/class/devfreq/soc:qcom,memlat-cpu0/governor")
|
||||
sudo_write("performance", "/sys/class/devfreq/soc:qcom,memlat-cpu4/governor")
|
||||
|
||||
# *** VIDC (encoder) config ***
|
||||
sudo_write("N", "/sys/kernel/debug/msm_vidc/clock_scaling")
|
||||
sudo_write("Y", "/sys/kernel/debug/msm_vidc/disable_thermal_mitigation")
|
||||
|
||||
# pandad core
|
||||
affine_irq(3, "spi_geni") # SPI
|
||||
try:
|
||||
pid = subprocess.check_output(["pgrep", "-f", "spi0"], encoding='utf8').strip()
|
||||
subprocess.call(["sudo", "chrt", "-f", "-p", "1", pid])
|
||||
subprocess.call(["sudo", "taskset", "-pc", "3", pid])
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(str(e))
|
||||
|
||||
def get_modem_data_usage(self):
|
||||
ms = self.get_modem_state()
|
||||
return ms.get('tx_bytes', -1), ms.get('rx_bytes', -1)
|
||||
|
||||
def reset_internal_panda(self):
|
||||
gpio_init(GPIO.STM_RST_N, True)
|
||||
gpio_init(GPIO.STM_BOOT0, True)
|
||||
|
||||
gpio_set(GPIO.STM_RST_N, 1)
|
||||
gpio_set(GPIO.STM_BOOT0, 0)
|
||||
time.sleep(0.01)
|
||||
gpio_set(GPIO.STM_RST_N, 0)
|
||||
|
||||
def recover_internal_panda(self):
|
||||
gpio_init(GPIO.STM_RST_N, True)
|
||||
gpio_init(GPIO.STM_BOOT0, True)
|
||||
|
||||
gpio_set(GPIO.STM_RST_N, 1)
|
||||
gpio_set(GPIO.STM_BOOT0, 1)
|
||||
time.sleep(0.01)
|
||||
gpio_set(GPIO.STM_RST_N, 0)
|
||||
time.sleep(0.01)
|
||||
gpio_set(GPIO.STM_BOOT0, 0)
|
||||
|
||||
def booted(self):
|
||||
# this normally boots within 8s, but on rare occasions takes 30+s
|
||||
encoder_state = sudo_read("/sys/kernel/debug/msm_vidc/core0/info")
|
||||
if "Core state: 0" in encoder_state and (time.monotonic() < 60*2):
|
||||
return False
|
||||
return True
|
||||
|
||||
if __name__ == "__main__":
|
||||
t = Tici()
|
||||
t.initialize_hardware()
|
||||
t.set_power_save(False)
|
||||
print(t.get_sim_info())
|
||||
Executable
+587
@@ -0,0 +1,587 @@
|
||||
#!/usr/bin/env python3
|
||||
import fcntl
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import serial
|
||||
import signal
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
from ipaddress import IPv4Address, AddressValueError
|
||||
|
||||
from enum import Enum
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s.%(msecs)03d %(levelname)-7s modem: %(message)s",
|
||||
datefmt="%H:%M:%S",
|
||||
)
|
||||
|
||||
AT_PORT = "/dev/modem_at0"
|
||||
PPP_PORT = "/dev/modem_at1"
|
||||
STATE_PATH = "/dev/shm/modem"
|
||||
AT_LOCK = "/dev/shm/modem.lock" # shared with LPA
|
||||
AT_INIT = [
|
||||
"ATE0", # disable command echo
|
||||
"ATV1", # verbose result codes (CONNECT/BUSY/NO CARRIER, not numeric)
|
||||
"AT+CMEE=1", # numeric +CME ERROR codes on failures (per 3GPP 27.007)
|
||||
"ATX4", # extended result codes: busy + dial tone detection, line speed in CONNECT
|
||||
"AT&C1", # DCD pin follows carrier state (V.250 default)
|
||||
"AT+CREG=2", # registration URCs include location info
|
||||
"AT+CGREG=2", # GPRS registration URCs include location info
|
||||
]
|
||||
CREG = {0: "not_registered", 1: "home", 2: "searching", 3: "denied", 4: "unknown", 5: "roaming"}
|
||||
# 3GPP TS 27.007 +COPS <AcT> -> network type
|
||||
NETWORK_TYPE = {0: "gsm", 1: "gsm", 3: "gsm", 8: "gsm",
|
||||
2: "utran", 4: "utran", 5: "utran", 6: "utran",
|
||||
7: "lte", 9: "lte", 10: "lte",
|
||||
11: "nr", 12: "nr", 13: "nr"}
|
||||
|
||||
DIAL_CID = 1
|
||||
WEBBING_ICCID_PREFIX = "8985235"
|
||||
|
||||
PPPD_CMD = [
|
||||
"sudo", "pppd", PPP_PORT, "460800", "noauth", "nodetach", "noipdefault", "usepeerdns",
|
||||
"nodefaultroute", "connect",
|
||||
"/usr/sbin/chat -v ABORT 'NO CARRIER' ABORT 'NO DIALTONE' ABORT 'BUSY' " +
|
||||
f"ABORT 'NO ANSWER' ABORT 'ERROR' TIMEOUT 5 '' AT OK ATD*99***{DIAL_CID}# CONNECT ''",
|
||||
"lcp-echo-interval", "30", "lcp-echo-failure", "4", "mtu", "1500", "mru", "1500",
|
||||
"novj", "novjccomp", "ipcp-accept-local", "ipcp-accept-remote", "nomagic",
|
||||
"user", '""', "password", '""',
|
||||
]
|
||||
INITIAL_STATE = {
|
||||
"seconds_since_boot": 0,
|
||||
"state": "INITIALIZING",
|
||||
"connected": False, "ip_address": "",
|
||||
"iccid": "", "mcc_mnc": "", "imei": "", "modem_version": "",
|
||||
"signal_strength": 0, "signal_quality": 0,
|
||||
"network_type": "unknown", "operator": "", "band": "", "channel": 0,
|
||||
"registration": "unknown", "temperatures": [], "extra": "",
|
||||
"tx_bytes": 0, "rx_bytes": 0,
|
||||
}
|
||||
|
||||
|
||||
class State(Enum):
|
||||
INITIALIZING = "INITIALIZING"
|
||||
SEARCHING = "SEARCHING"
|
||||
CONNECTING = "CONNECTING"
|
||||
CONNECTED = "CONNECTED"
|
||||
DISCONNECTING = "DISCONNECTING"
|
||||
|
||||
|
||||
STATE_WAIT = 1.0 # seconds to wait after each state handler returns
|
||||
|
||||
|
||||
class PPPSession:
|
||||
"""Owns pppd lifecycle, fail tracking, and PPP routing."""
|
||||
MAX_FAILS = 3
|
||||
|
||||
def __init__(self):
|
||||
self._proc: subprocess.Popen | None = None
|
||||
self._fails = 0
|
||||
self._peer = ""
|
||||
|
||||
def start(self):
|
||||
self._proc = subprocess.Popen(PPPD_CMD, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
self._peer = ""
|
||||
logging.info(f"PPP dialing CID {DIAL_CID}")
|
||||
|
||||
def kill(self):
|
||||
subprocess.run(["sudo", "killall", "-9", "pppd"], capture_output=True)
|
||||
self._peer = ""
|
||||
|
||||
@staticmethod
|
||||
def reset_data_port():
|
||||
"""Drop DTR on PPP_PORT so the modem terminates any stuck PPP session."""
|
||||
try:
|
||||
with serial.Serial(PPP_PORT, 460800, timeout=1) as s:
|
||||
s.dtr = False
|
||||
time.sleep(0.2)
|
||||
s.dtr = True
|
||||
except Exception as e:
|
||||
logging.warning(f"data port reset failed: {e}")
|
||||
|
||||
def has_exited(self) -> bool:
|
||||
return self._proc is not None and self._proc.poll() is not None
|
||||
|
||||
def reset_fail_counter(self):
|
||||
self._fails = 0
|
||||
|
||||
def record_fail(self) -> bool:
|
||||
"""Bump fail counter; return True if at the give-up limit."""
|
||||
self._fails += 1
|
||||
return self._fails >= self.MAX_FAILS
|
||||
|
||||
@property
|
||||
def fails(self) -> int:
|
||||
return self._fails
|
||||
|
||||
def maybe_install_routes(self, ip: str, peer: str) -> bool:
|
||||
"""Install routes if peer changed; kill the session on failure so the state machine reconnects."""
|
||||
if not peer or peer == self._peer:
|
||||
return False
|
||||
try:
|
||||
IPv4Address(ip)
|
||||
IPv4Address(peer)
|
||||
except AddressValueError:
|
||||
logging.warning(f"refusing route install with non-IPv4 ip={ip!r} peer={peer!r}")
|
||||
self.kill()
|
||||
return False
|
||||
self.cleanup_routes()
|
||||
cmds = [
|
||||
["sudo", "ip", "route", "add", "default", "via", peer, "dev", "ppp0", "metric", "1000"],
|
||||
["sudo", "ip", "route", "add", "default", "via", peer, "dev", "ppp0", "table", "1000"],
|
||||
["sudo", "ip", "rule", "add", "from", ip, "table", "1000"],
|
||||
]
|
||||
for cmd in cmds:
|
||||
r = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if r.returncode != 0:
|
||||
logging.warning(f"route install failed ({' '.join(cmd[1:])}): {r.stderr.strip()}")
|
||||
self.cleanup_routes()
|
||||
self.kill()
|
||||
return False
|
||||
logging.info(f"route set up for {ip} via {peer}")
|
||||
self._peer = peer
|
||||
return True
|
||||
|
||||
def maybe_install_dns(self, dns_servers: list[str]) -> bool:
|
||||
"""Register DNS servers with systemd-resolved; kill the session on failure to force a retry."""
|
||||
if not dns_servers:
|
||||
return False
|
||||
for cmd in (["sudo", "resolvectl", "dns", "ppp0", *dns_servers],
|
||||
["sudo", "resolvectl", "default-route", "ppp0", "yes"]):
|
||||
r = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if r.returncode != 0:
|
||||
logging.warning(f"resolvectl failed ({' '.join(cmd[1:])}): {r.stderr.strip()}")
|
||||
self.kill()
|
||||
return False
|
||||
logging.info(f"resolvectl: ppp0 DNS = {dns_servers}")
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def cleanup_routes():
|
||||
subprocess.run(["sudo", "ip", "route", "del", "default", "dev", "ppp0"], capture_output=True)
|
||||
subprocess.run(["sudo", "ip", "route", "flush", "table", "1000"], capture_output=True)
|
||||
# rules don't have a flush; delete until none remain
|
||||
while subprocess.run(["sudo", "ip", "rule", "del", "table", "1000"], capture_output=True).returncode == 0:
|
||||
pass
|
||||
subprocess.run(["sudo", "resolvectl", "revert", "ppp0"], capture_output=True)
|
||||
|
||||
|
||||
class Modem:
|
||||
def __init__(self):
|
||||
self._ppp = PPPSession()
|
||||
self._sim_change = False
|
||||
self._apn = "" # blank = network-provided via PCO
|
||||
self._roaming_allowed = True
|
||||
self.running = True
|
||||
self.S = INITIAL_STATE.copy()
|
||||
|
||||
@staticmethod
|
||||
def _read_param(key):
|
||||
try:
|
||||
with open(f"/data/params/d/{key}") as f:
|
||||
return f.read().strip()
|
||||
except FileNotFoundError:
|
||||
return ""
|
||||
|
||||
@staticmethod
|
||||
def _parse_reg(v: str) -> str:
|
||||
try:
|
||||
return CREG.get(int(v.split(",")[1].strip('"')), "unknown")
|
||||
except (ValueError, IndexError):
|
||||
return "unknown"
|
||||
|
||||
@staticmethod
|
||||
def _has_modem_manager() -> bool:
|
||||
return os.path.isfile("/lib/systemd/system/ModemManager.service")
|
||||
|
||||
def _is_roaming_allowed(self) -> bool:
|
||||
if self.S["iccid"].startswith(WEBBING_ICCID_PREFIX):
|
||||
return True
|
||||
return self._read_param("GsmRoaming") == "1"
|
||||
|
||||
def _publish_state(self, **kwargs):
|
||||
self.S.update(kwargs)
|
||||
self.S["seconds_since_boot"] = time.monotonic()
|
||||
with tempfile.NamedTemporaryFile(mode="w", dir="/dev/shm", delete=False) as f:
|
||||
json.dump(self.S, f, indent=2)
|
||||
os.chmod(f.name, 0o644)
|
||||
os.replace(f.name, STATE_PATH)
|
||||
|
||||
def _at(self, cmd):
|
||||
"""Send AT command, return response lines. [] on error or if LPA holds port."""
|
||||
fd = os.open(AT_LOCK, os.O_CREAT | os.O_RDWR, 0o666)
|
||||
try:
|
||||
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except OSError:
|
||||
os.close(fd)
|
||||
return []
|
||||
try:
|
||||
with serial.Serial(AT_PORT, 9600, timeout=5) as ser:
|
||||
ser.reset_input_buffer()
|
||||
ser.write((cmd + "\r").encode())
|
||||
lines = []
|
||||
while True:
|
||||
raw = ser.readline()
|
||||
if not raw:
|
||||
raise TimeoutError("AT timeout")
|
||||
line = raw.decode(errors="ignore").strip()
|
||||
if not line:
|
||||
continue
|
||||
if line == "OK":
|
||||
break
|
||||
if line == "ERROR" or line.startswith("+CME ERROR"):
|
||||
raise RuntimeError(line)
|
||||
lines.append(line)
|
||||
return lines
|
||||
except (RuntimeError, TimeoutError, OSError) as e:
|
||||
logging.info(f"AT {cmd} failed: {e}")
|
||||
return []
|
||||
finally:
|
||||
fcntl.flock(fd, fcntl.LOCK_UN)
|
||||
os.close(fd)
|
||||
|
||||
def _atv(self, cmd, pfx):
|
||||
for line in self._at(cmd):
|
||||
if pfx in line and ":" in line:
|
||||
return line.split(":", 1)[1].strip()
|
||||
return None
|
||||
|
||||
def _init_at_channel(self) -> bool:
|
||||
"""Run AT_INIT and confirm ATE0 took effect. Returns False if echo is still on."""
|
||||
for c in AT_INIT:
|
||||
self._at(c)
|
||||
r = self._at("AT+CGMI")
|
||||
return bool(r) and not r[0].startswith("AT")
|
||||
|
||||
def _configure_modem(self, modem_version: str):
|
||||
if not modem_version.startswith("EG25"):
|
||||
return
|
||||
cmds = [
|
||||
# clear initial EPS bearer APN (some carriers reject the default)
|
||||
'AT+CGDCONT=0,"IP",""',
|
||||
|
||||
# SIM hot swap
|
||||
'AT+QSIMDET=1,0',
|
||||
'AT+QSIMSTAT=1',
|
||||
|
||||
# configure modem as data-centric
|
||||
'AT+QNVW=5280,0,"0102000000000000"',
|
||||
'AT+QNVFW="/nv/item_files/ims/IMS_enable",00',
|
||||
'AT+QNVFW="/nv/item_files/modem/mmode/ue_usage_setting",01',
|
||||
]
|
||||
for c in cmds:
|
||||
self._at(c)
|
||||
|
||||
def _do_initializing(self):
|
||||
if not os.path.exists(AT_PORT):
|
||||
return State.INITIALIZING
|
||||
logging.info("port found, initializing")
|
||||
self._ppp.kill()
|
||||
self._ppp.cleanup_routes()
|
||||
|
||||
if not self._init_at_channel():
|
||||
logging.warning("AT echo still on, retrying")
|
||||
return State.INITIALIZING
|
||||
|
||||
identity = self._read_identity()
|
||||
if not identity["iccid"] or not identity["imei"]:
|
||||
logging.warning(f"identity read incomplete: {identity}, retrying")
|
||||
return State.INITIALIZING
|
||||
|
||||
self._configure_modem(identity["modem_version"])
|
||||
|
||||
self.S.update(identity)
|
||||
self._apn = self._read_param("GsmApn")
|
||||
self._roaming_allowed = self._is_roaming_allowed()
|
||||
# blank APN lets the carrier supply one via PCO
|
||||
self._at(f'AT+CGDCONT={DIAL_CID},"IP","{self._apn}"')
|
||||
logging.info(f"APN '{self._apn or '(network-provided)'}' written to CID {DIAL_CID}, roaming={'on' if self._roaming_allowed else 'off'}")
|
||||
|
||||
self._sim_change = False # clear since we just re-read identity with the new SIM
|
||||
self._publish_state(**identity)
|
||||
return State.SEARCHING
|
||||
|
||||
def _read_identity(self):
|
||||
def first_line(cmd):
|
||||
r = self._at(cmd)
|
||||
return r[0].strip() if r else ""
|
||||
|
||||
imei = first_line("AT+CGSN")
|
||||
if not (imei.isdigit() and 14 <= len(imei) <= 17): # 3GPP TS 23.003
|
||||
imei = ""
|
||||
|
||||
iccid = (self._atv("AT+QCCID", "+QCCID:") or "").rstrip("F")
|
||||
if not iccid.isdigit():
|
||||
iccid = ""
|
||||
|
||||
imsi = first_line("AT+CIMI")
|
||||
mcc_mnc = imsi[:6] if imsi.isdigit() and len(imsi) >= 6 else ""
|
||||
|
||||
modem_version = first_line("AT+GMR")
|
||||
|
||||
logging.info(f"imei={imei} iccid={iccid} mcc_mnc={mcc_mnc} ver={modem_version}")
|
||||
return {"imei": imei, "iccid": iccid, "mcc_mnc": mcc_mnc, "modem_version": modem_version}
|
||||
|
||||
def _do_searching(self):
|
||||
new_roaming = self._is_roaming_allowed()
|
||||
if new_roaming != self._roaming_allowed:
|
||||
logging.info(f"roaming changed: {self._roaming_allowed} -> {new_roaming}")
|
||||
self._roaming_allowed = new_roaming
|
||||
|
||||
v = self._atv("AT+CREG?", "+CREG:")
|
||||
if not v:
|
||||
return self._searching_idle()
|
||||
|
||||
reg = self._parse_reg(v)
|
||||
greg = self._parse_reg(self._atv("AT+CGREG?", "+CGREG:") or "")
|
||||
logging.debug(f"creg={reg} cgreg={greg} roaming_allowed={self._roaming_allowed}")
|
||||
|
||||
if reg == "roaming" and not self._roaming_allowed:
|
||||
self._publish_state(registration=reg)
|
||||
return State.SEARCHING
|
||||
|
||||
if reg in ("home", "roaming") and greg in ("home", "roaming"):
|
||||
self._publish_state(registration=reg)
|
||||
return State.CONNECTING
|
||||
|
||||
if reg != self.S.get("registration"):
|
||||
self._publish_state(registration=reg)
|
||||
return self._searching_idle()
|
||||
|
||||
def _searching_idle(self):
|
||||
if self._sim_change or not os.path.exists(AT_PORT):
|
||||
logging.info(f"-> reconnecting (sim_change={self._sim_change} port={os.path.exists(AT_PORT)})")
|
||||
return State.DISCONNECTING
|
||||
return State.SEARCHING
|
||||
|
||||
def _do_connecting(self):
|
||||
logging.info("starting pppd")
|
||||
self._ppp.reset_fail_counter()
|
||||
self._sim_change = False
|
||||
self._ppp.start()
|
||||
return State.CONNECTED
|
||||
|
||||
def _handle_pppd_exit(self):
|
||||
if self._sim_change or not os.path.exists(AT_PORT):
|
||||
return State.DISCONNECTING
|
||||
give_up = self._ppp.record_fail()
|
||||
if give_up:
|
||||
logging.warning(f"PPP fail {self._ppp.fails}/{self._ppp.MAX_FAILS}, reconnecting")
|
||||
return State.DISCONNECTING
|
||||
logging.warning(f"PPP fail {self._ppp.fails}/{self._ppp.MAX_FAILS}, retrying")
|
||||
self._ppp.reset_data_port()
|
||||
if not os.path.exists(AT_PORT):
|
||||
return State.DISCONNECTING
|
||||
self._ppp.start()
|
||||
return State.CONNECTED
|
||||
|
||||
def _params_changed(self) -> bool:
|
||||
new_apn = self._read_param("GsmApn")
|
||||
if new_apn != self._apn:
|
||||
logging.info(f"GsmApn changed: '{self._apn}' -> '{new_apn}'")
|
||||
return True
|
||||
new_roaming = self._is_roaming_allowed()
|
||||
if new_roaming != self._roaming_allowed:
|
||||
logging.info(f"roaming changed: {self._roaming_allowed} -> {new_roaming}")
|
||||
return True
|
||||
return False
|
||||
|
||||
def _check_iccid(self, state):
|
||||
if state in (State.INITIALIZING, State.DISCONNECTING) or not self.S["iccid"]:
|
||||
return
|
||||
iccid = (self._atv("AT+QCCID", "+QCCID:") or "").rstrip("F")
|
||||
if iccid and iccid != self.S["iccid"]:
|
||||
logging.warning(f"iccid changed: {self.S['iccid']} -> {iccid}")
|
||||
self._sim_change = True
|
||||
|
||||
def _do_connected(self):
|
||||
if self._ppp.has_exited():
|
||||
return self._handle_pppd_exit()
|
||||
|
||||
if self._sim_change or not os.path.exists(AT_PORT) or self._params_changed():
|
||||
return State.DISCONNECTING
|
||||
|
||||
self._poll()
|
||||
return State.CONNECTED
|
||||
|
||||
def _do_disconnecting(self):
|
||||
logging.warning("reconnecting")
|
||||
self._publish_state(**INITIAL_STATE)
|
||||
self._ppp.kill()
|
||||
self._ppp.cleanup_routes()
|
||||
self._ppp.reset_data_port()
|
||||
self._sim_change = False
|
||||
return State.INITIALIZING
|
||||
|
||||
def _poll_signal(self) -> dict:
|
||||
v = self._atv("AT+CSQ", "+CSQ:")
|
||||
if not v:
|
||||
return {}
|
||||
try:
|
||||
rssi = int(v.split(",")[0])
|
||||
if rssi == 99:
|
||||
return {}
|
||||
return {"signal_strength": rssi, "signal_quality": min(100, int(rssi / 31 * 100))}
|
||||
except (ValueError, IndexError):
|
||||
return {}
|
||||
|
||||
def _poll_operator(self) -> dict:
|
||||
v = self._atv("AT+COPS?", "+COPS:")
|
||||
if not v:
|
||||
return {}
|
||||
p = v.split(",")
|
||||
out: dict = {}
|
||||
try:
|
||||
if len(p) >= 3:
|
||||
out["operator"] = p[2].strip('"')
|
||||
if len(p) >= 4:
|
||||
out["network_type"] = NETWORK_TYPE.get(int(p[3]), "unknown")
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
return out
|
||||
|
||||
def _poll_band(self) -> dict:
|
||||
v = self._atv("AT+QNWINFO", "+QNWINFO:")
|
||||
if not v:
|
||||
return {}
|
||||
info = v.replace('"', '').split(",")
|
||||
try:
|
||||
if len(info) >= 4:
|
||||
return {"band": info[2], "channel": int(info[3])}
|
||||
except ValueError:
|
||||
pass
|
||||
return {}
|
||||
|
||||
def _poll_extra(self) -> dict:
|
||||
v = self._atv('AT+QENG="servingcell"', "+QENG:")
|
||||
return {"extra": v.replace('"', '')} if v else {}
|
||||
|
||||
def _poll_temps(self) -> dict:
|
||||
v = self._atv("AT+QTEMP", "+QTEMP:")
|
||||
if not v:
|
||||
return {}
|
||||
try:
|
||||
return {"temperatures": [t for t in (int(x) for x in v.split(",") if x.strip()) if t != 255]}
|
||||
except (ValueError, IndexError):
|
||||
return {}
|
||||
|
||||
def _poll_iface(self) -> dict:
|
||||
try:
|
||||
r = subprocess.run(["ip", "-4", "addr", "show", "ppp0"], capture_output=True, text=True, timeout=2)
|
||||
ip, peer = "", ""
|
||||
for line in r.stdout.splitlines():
|
||||
# `inet 10.x.x.x peer 10.64.64.64/32 ...`
|
||||
parts = line.strip().split()
|
||||
if "inet" in parts:
|
||||
i = parts.index("inet")
|
||||
ip = parts[i + 1].split("/")[0]
|
||||
if "peer" in parts:
|
||||
peer = parts[parts.index("peer") + 1].split("/")[0]
|
||||
break
|
||||
if ip:
|
||||
if self._ppp.maybe_install_routes(ip, peer):
|
||||
self._ppp.maybe_install_dns(self._read_cellular_dns())
|
||||
return {"ip_address": ip, "connected": True}
|
||||
if self.S["connected"]:
|
||||
return {"connected": False, "ip_address": ""}
|
||||
except Exception:
|
||||
pass
|
||||
return {}
|
||||
|
||||
def _read_cellular_dns(self) -> list[str]:
|
||||
v = self._atv(f"AT+CGCONTRDP={DIAL_CID}", "+CGCONTRDP:")
|
||||
if not v:
|
||||
return []
|
||||
# +CGCONTRDP: <cid>,<bearer_id>,<apn>,<local_addr>,<gw_addr>,<dns_prim>,<dns_sec>,...
|
||||
fields = [f.strip().strip('"') for f in v.split(",")]
|
||||
dns_servers = []
|
||||
for d in fields[5:7]:
|
||||
try:
|
||||
dns_servers.append(str(IPv4Address(d)))
|
||||
except (AddressValueError, ValueError):
|
||||
pass
|
||||
if not dns_servers:
|
||||
logging.warning(f"no cellular DNS servers reported by modem: {v!r}")
|
||||
return dns_servers
|
||||
|
||||
def _poll_byte_counters(self) -> dict:
|
||||
try:
|
||||
with open("/sys/class/net/ppp0/statistics/tx_bytes") as f:
|
||||
tx = int(f.read().strip())
|
||||
with open("/sys/class/net/ppp0/statistics/rx_bytes") as f:
|
||||
rx = int(f.read().strip())
|
||||
except Exception:
|
||||
return {}
|
||||
return {"tx_bytes": tx, "rx_bytes": rx}
|
||||
|
||||
def _poll(self):
|
||||
s: dict = {}
|
||||
for fn in (self._poll_signal, self._poll_operator, self._poll_band,
|
||||
self._poll_extra, self._poll_temps, self._poll_iface,
|
||||
self._poll_byte_counters):
|
||||
s.update(fn())
|
||||
if s:
|
||||
self._publish_state(**s)
|
||||
|
||||
def run(self):
|
||||
logging.info("starting")
|
||||
self._publish_state(state=State.INITIALIZING.value)
|
||||
if self._has_modem_manager():
|
||||
subprocess.run(["sudo", "systemctl", "mask", "--runtime", "ModemManager"], capture_output=True)
|
||||
subprocess.run(["sudo", "systemctl", "stop", "ModemManager"], capture_output=True)
|
||||
self._ppp.kill()
|
||||
|
||||
state = State.INITIALIZING
|
||||
|
||||
handlers = {
|
||||
State.INITIALIZING: self._do_initializing,
|
||||
State.SEARCHING: self._do_searching,
|
||||
State.CONNECTING: self._do_connecting,
|
||||
State.CONNECTED: self._do_connected,
|
||||
State.DISCONNECTING: self._do_disconnecting,
|
||||
}
|
||||
|
||||
while self.running:
|
||||
try:
|
||||
self._check_iccid(state)
|
||||
prev = state
|
||||
state = handlers[state]()
|
||||
if state != prev:
|
||||
self._publish_state(state=state.value)
|
||||
logging.info(f"{prev.value} -> {state.value}")
|
||||
except Exception:
|
||||
logging.exception(f"error in {state.value}")
|
||||
state = State.DISCONNECTING
|
||||
time.sleep(STATE_WAIT)
|
||||
|
||||
def stop(self):
|
||||
self.running = False
|
||||
self._ppp.kill()
|
||||
self._ppp.cleanup_routes()
|
||||
try:
|
||||
os.remove(STATE_PATH)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
if self._has_modem_manager():
|
||||
subprocess.run(["sudo", "systemctl", "unmask", "--runtime", "ModemManager"], capture_output=True)
|
||||
subprocess.run(["sudo", "systemctl", "start", "ModemManager"], capture_output=True)
|
||||
|
||||
|
||||
def main():
|
||||
m = Modem()
|
||||
|
||||
def _sig(*_):
|
||||
m.running = False
|
||||
|
||||
signal.signal(signal.SIGINT, _sig)
|
||||
signal.signal(signal.SIGTERM, _sig)
|
||||
m.run()
|
||||
m.stop()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
|
||||
+2
-2
@@ -4,8 +4,8 @@ import random
|
||||
import subprocess
|
||||
|
||||
from panda import Panda
|
||||
from openpilot.system.hardware import TICI, HARDWARE
|
||||
from openpilot.system.hardware.tici.amplifier import Amplifier
|
||||
from openpilot.common.hardware import TICI, HARDWARE
|
||||
from openpilot.common.hardware.tici.amplifier import Amplifier
|
||||
|
||||
|
||||
class TestAmplifier:
|
||||
Executable
BIN
Binary file not shown.
@@ -1 +0,0 @@
|
||||
#define DEFAULT_MODEL "OP Model 7 (Default)"
|
||||
@@ -1,8 +1,13 @@
|
||||
import re
|
||||
import sys
|
||||
import pytest
|
||||
import inspect
|
||||
|
||||
|
||||
def _to_safe_name(s):
|
||||
return re.sub(r"[^a-zA-Z0-9_]+", "_", str(s)).strip("_")
|
||||
|
||||
|
||||
class parameterized:
|
||||
@staticmethod
|
||||
def expand(cases):
|
||||
@@ -34,7 +39,9 @@ def parameterized_class(attrs, input_list=None):
|
||||
def decorator(cls):
|
||||
globs = sys._getframe(1).f_globals
|
||||
for i, params in enumerate(params_list):
|
||||
name = f"{cls.__name__}_{i}"
|
||||
# append sanitized string param values so pytest -k can filter by them
|
||||
suffix = "_".join(filter(None, (_to_safe_name(v) for v in params.values() if isinstance(v, str))))
|
||||
name = f"{cls.__name__}_{i}" + (f"_{suffix}" if suffix else "")
|
||||
new_cls = type(name, (cls,), dict(params))
|
||||
new_cls.__module__ = cls.__module__
|
||||
new_cls.__test__ = True # override inherited False so pytest collects this subclass
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@
|
||||
#include "common/queue.h"
|
||||
#include "common/swaglog.h"
|
||||
#include "common/util.h"
|
||||
#include "system/hardware/hw.h"
|
||||
#include "common/hardware/hw.h"
|
||||
|
||||
namespace {
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user