Compare commits

..

126 Commits

Author SHA1 Message Date
Vehicle Researcher 28c0797d30 openpilot v0.4.2 release 2018-02-06 12:43:45 -08:00
Vehicle Researcher a701aa7292 Merge opendbc subtree 2018-02-06 12:39:39 -08:00
Vehicle Researcher 7ae657880d Squashed 'opendbc/' changes from 3ca654f..81d9871
81d9871 Lexus: added first draft of RX dbc file
cb352da fix toyota steering lka message length
d294f64 fix gas pedal message length rav4h
2d035de acura rdx remove double defined message
500a734 fix acura rdx dbc, import was missing

git-subtree-dir: opendbc
git-subtree-split: 81d9871dbe00ccfc5fe4418ea3577726fffa1def
2018-02-06 12:39:39 -08:00
Vehicle Researcher 433f934783 Merge panda subtree 2018-02-06 12:39:38 -08:00
Vehicle Researcher db67cffb4d Squashed 'panda/' changes from b42db6d..98f29a4
98f29a4 v1.0.7 has working grey panda

git-subtree-dir: panda
git-subtree-split: 98f29a4a97af58aa4707bdaba948ac366ab2e6a0
2018-02-06 12:39:38 -08:00
vanillagorillaa 0dc6778548 Acura RDX 2018 AcuraWatch Plus Port (#162)
* Adding Acura RDX alpha support
2018-02-05 18:38:40 -08:00
ErichMoraga c86b52698b Update fingerprints.py
Taiwanese Prius Prime fingerprint was identical to the regular prime (107 msgs.), but should really have 110.  I have the correct one in there now.
2018-02-05 14:16:58 -08:00
George Hotz e5e5aa7ded openpilot v0.4.1 hotfix 2018-02-02 19:30:44 -08:00
George Hotz 4474b9b371 openpilot v0.4.1 tweaks 2018-01-30 21:34:01 -08:00
Vehicle Researcher 26e966852c openpilot v0.4.1 release 2018-01-30 12:58:14 -08:00
Vehicle Researcher 852166f5cc Squashed 'opendbc/' changes from 8753665..3ca654f
3ca654f move acura rdx to generator folder
1c7c490 move pilot touring to generated
89856dd fix bugs in some counter/checksum definitions
451a2b4 honda wheelspeed in kph to match toyota
290dd14 rename generic honda and toyota dbcs to include year
ccb8a14 readded gas interceptor to hondas so it does not break regression tests
66d256e readme explanation of preprocessor
6c8d9f7 regenerate new steer torque eps factor
beba6d5 move generated files to root folder
2b5b0aa Toyota: different factor for STEER_TORQUE_EPS
e41b174 fix typo
682f557 cleanup
1faf67c fix crv steering_control message
cce6c79 add odometer message to civic and odyssey
74cc834 fix PCM_SPEED factor
3f2baa5 fix honda dometer scaling
f6fd6b9 fix comments and values
f0ba5e2 whitespace consistency
8c77a5d dbc file preprocessor
10f8c6e consistent can message names for supported Hondas
af07d6a Topyota Corolla: added dbc file, seems the same as Rav4
a632fde Cleanup duplicate message names

git-subtree-dir: opendbc
git-subtree-split: 3ca654f88545d05bf23cb8ff945f06a28c32902e
2018-01-30 12:54:13 -08:00
Vehicle Researcher 75ac92b90f Merge opendbc subtree 2018-01-30 12:54:13 -08:00
Vehicle Researcher 96f8e5158e Squashed 'panda/' changes from 2573d86..b42db6d
b42db6d Merge pull request #82 from commaai/uart_dma
fd68f86 smallr
be99ffc ok that doesn't hurt i think
a9f6bf0 this
8b7e849 working now
7fa4808 froze up, maybe thats the fix
1465aa4 ok, it's fixed
915cd84 ugh, ok, need that
fd05376 comment out debug
37c5263 big fifo
497f069 dma is all critical, no interrupts
7c34afe minor change
743d244 high baud rate works
5d2a4ba v1.0.6
fbf1390 Toyota Safety: fix in input param
6c01d09 Toyota: less torque error allowance to meet Corolla acceptable behavior
07c01b2 Toyota safety: using input param
4410a59 add safety param support
fc81fc1 uart dma in progress
65fb2b2 grey panda query, 1.0.5
f415c9a grey panda detection
b68957e add pandadebug support
b5e4962 leave msgs around in isotp
0acce2d add recvaddr support
3fc38f4 set bootmode with power
d4c052a make that work
21f8195 fix panda serial write
af74aa9 from python import

git-subtree-dir: panda
git-subtree-split: b42db6dc082fb13ef5ac63ed197a63e179651ef6
2018-01-30 12:54:12 -08:00
Vehicle Researcher b68f3f7781 Merge panda subtree 2018-01-30 12:54:12 -08:00
George Hotz 36bef17224 Merge pull request #155 from slesinski/odyssey-port
Honda Odyssey 2018 EX-L with Honda Sensing
2018-01-18 21:49:14 -08:00
Ted Slesinski 715771bcff Unnecessary characters in comma's dongle id 2018-01-19 00:47:47 -05:00
Ted Slesinski 8e264baaa2 SteerRatio should be 14.35 2018-01-19 00:37:32 -05:00
Ted Slesinski 8ff147de6d Added the extra carstate reads 2018-01-19 00:34:02 -05:00
Ted Slesinski 71057c586f Tuning specific to odyssey 2018-01-19 00:34:02 -05:00
Ted Slesinski fac22f4038 Gear shifter values like acura 2018-01-19 00:32:51 -05:00
Ted Slesinski 40094779d7 Car_gas 130 missing canbus, using 17c 2018-01-19 00:32:51 -05:00
Ted Slesinski 2552aa45c3 BRAKE_HOLD signal not ready yet 2018-01-19 00:32:51 -05:00
Ted Slesinski c2972a80f9 Park Brake not yet in DBC 2018-01-19 00:32:51 -05:00
Ted Slesinski 791a440709 Wrong value for MAIN_ON 2018-01-19 00:32:51 -05:00
Ted Slesinski 13145dd9a7 Packet size off on a fingerprint id 2018-01-19 00:32:51 -05:00
Ted Slesinski e9c477dcf2 Add hondacan changes 2018-01-19 00:30:33 -05:00
Ted Slesinski 8eebd19884 Add carcontroller changes 2018-01-19 00:30:33 -05:00
Ted Slesinski a54b35a4d6 Add odyssey car params to interface 2018-01-19 00:30:33 -05:00
Ted Slesinski 8a6f7a5e78 Add carstate changes 2018-01-19 00:30:33 -05:00
Ted Slesinski 6752959d4a Add odyssey fingerprint 2018-01-19 00:30:33 -05:00
Vehicle Researcher da52d065a4 openpilot v0.4.0.2 release 2018-01-18 15:46:12 -08:00
Vehicle Researcher 255e45ca58 Squashed 'opendbc/' changes from 48968cc..8753665
8753665 Fix Chevy Volt actuator signal parsing (#66)
89963c9 Toyota: added auto high beam
b07a84d Add Honda Clarity Hybrid (#65)
ca352b3 add hyundai 2015 (#63)

git-subtree-dir: opendbc
git-subtree-split: 87536651c21daddba932f5d8668b52a0ece9082f
2018-01-18 15:42:13 -08:00
Vehicle Researcher b731b7cf12 Merge opendbc subtree 2018-01-18 15:42:13 -08:00
Vehicle Researcher 5f014635e1 Squashed 'panda/' changes from c371fe6..2573d86
2573d86 docs say max is 4, respect the docs
864cd8f failing on some devices
54bcc67 Merge pull request #75 from gregjhogan/j2534-vs-2017-upgrade
1664270 Merge pull request #74 from gregjhogan/j2534-disconnect-fix
a7e3a8f bump panda version for serial bug
aa0cfad fix UART hang
09ab8f6 add sendaddr support to isotp
40a1883 fix up baud rate
65997ff add PandaSerial and location panda (aka pigeon) test
57d633c upgraded to VS 2017
35cc32a fixed pointer exception on disconnect

git-subtree-dir: panda
git-subtree-split: 2573d861e605a2dcf456a6421b31e83fdd9ca606
2018-01-18 15:42:12 -08:00
Vehicle Researcher ef3d8314d3 Merge panda subtree 2018-01-18 15:42:12 -08:00
Vehicle Researcher 7ef3fd567f openpilot v0.4.0.1 tweaks 2018-01-16 23:26:39 -08:00
heatnation b773e27ad1 Update fingerprints.py 2018-01-15 18:22:15 -08:00
Vehicle Researcher a77c0a1098 openpilot v0.4.0.1 release 2017-12-23 17:15:27 -08:00
Vehicle Researcher c7b5fb9116 Merge commit '57b9ddf20ef28d7a9b4fd57c08630281ef103422' as 'pyextra' 2017-12-23 17:10:45 -08:00
Vehicle Researcher 57b9ddf20e Squashed 'pyextra/' content from commit 4eda4dd
git-subtree-dir: pyextra
git-subtree-split: 4eda4dd765c2bc719da9064774de6b2c14c322d1
2017-12-23 17:10:45 -08:00
Vehicle Researcher ba97d0e838 Squashed 'opendbc/' content from commit 48968cc
git-subtree-dir: opendbc
git-subtree-split: 48968cc97e20bbbff79a4380b69a6a5a858c9d1c
2017-12-23 17:10:44 -08:00
Vehicle Researcher b8085e2c42 Merge commit 'ba97d0e83837b4c893edc609001dde7c41b8e24b' as 'opendbc' 2017-12-23 17:10:44 -08:00
Vehicle Researcher c251b312d8 Squashed 'panda/' content from commit c371fe6
git-subtree-dir: panda
git-subtree-split: c371fe688dbad4c53635905d3471a01c185e811d
2017-12-23 17:10:42 -08:00
Vehicle Researcher 14fb17e22f Merge commit 'c251b312d87d26b5ed347b267f6f1570793f9b91' as 'panda' 2017-12-23 17:10:42 -08:00
Vehicle Researcher a34c87ab46 remove submodules 2017-12-23 17:06:56 -08:00
Vehicle Researcher 1ad9cc8c67 openpilot v0.3.9 tweaks 2017-12-06 12:48:00 -08:00
Vehicle Researcher 5627d0d7fd openpilot v0.3.9 release 2017-11-22 04:30:24 -08:00
Vehicle Researcher 2cfdbefde8 update releases 2017-11-04 05:31:15 -07:00
Vehicle Researcher 7dabcdace8 openpilot v0.3.8.2 tweaks 2017-11-03 18:21:38 -07:00
Vehicle Researcher 187a70f760 openpilot v0.3.8.2 release 2017-10-31 02:27:39 -07:00
Vehicle Researcher 48303589e9 openpilot v0.3.7 tweaks 2017-10-03 23:46:23 -07:00
Vehicle Researcher 8385b27cad openpilot v0.3.7 tweaks 2017-10-03 00:35:46 -07:00
George Hotz abd75aedd7 Bump panda version to fix old board updating 2017-09-30 23:30:38 -07:00
Vehicle Researcher daf54ad54d openpilot v0.3.7 release 2017-09-30 19:05:03 -07:00
espes 94fe677f91 Fix rear view mirror setting 2017-08-28 22:24:11 -07:00
Vehicle Researcher 3de85098e5 openpilot v0.3.6.1 tweaks 2017-08-16 05:21:06 -07:00
Vehicle Researcher 5524dc8773 openpilot v0.3.6.1 release 2017-08-15 03:15:00 -07:00
Vehicle Researcher 19dd5f3e32 openpilot v0.3.6 tweaks 2017-08-10 20:38:56 -07:00
Vehicle Researcher 99cb610b12 openpilot v0.3.6 release 2017-08-09 17:41:38 -07:00
Vehicle Researcher 9d3963559a openpilot v0.3.5 release 2017-07-30 17:59:37 -07:00
Vehicle Researcher 1b8c44b506 openpilot v0.3.4 tweaks 2017-07-28 20:51:27 -07:00
Vehicle Researcher 6f46f988d9 openpilot v0.3.4 release 2017-07-28 03:23:57 -07:00
George Hotz 68485aa4e4 Merge pull request #116 from commaai/revert-114-new_panda_code
Revert "Pulled in new panda firmware and updated boardd to support the changes."
2017-07-17 23:17:13 -07:00
George Hotz 1581fdc198 Revert "Pulled in new panda firmware and updated boardd to support the changes." 2017-07-17 23:16:57 -07:00
George Hotz 317ae0fb37 Merge pull request #114 from diamondman/new_panda_code
Pulled in new panda firmware and updated boardd to support the changes.
2017-07-11 22:24:38 -07:00
Jessy Diamond Exum 5bf4196aed Removed unnecessary TODO 2017-07-11 22:01:04 -07:00
Jessy Diamond Exum 38aa03e0f7 Moved boardd loopback config to envvar. 2017-07-11 21:59:03 -07:00
Jessy Diamond Exum 9a9dc3ab23 Pulled in new panda firmware and updated boardd to support the changes. 2017-07-11 21:45:16 -07:00
George Hotz e4aa959e2c Merge pull request #113 from pjlao307/update-alert-text
Update alert text
2017-07-07 10:25:59 -07:00
Joey Lao 2aa9a56f40 Update steering controls saturated message 2017-07-06 22:36:55 -07:00
pjlao307 721ed4ec0e Merge pull request #2 from commaai/release
Release
2017-07-03 11:15:42 -07:00
George Hotz 70be4ceab1 Merge pull request #109 from commaai/devel
openpilot 0.3.3
2017-06-30 18:23:10 -07:00
Vehicle Researcher 5cf91d0496 openpilot v0.3.3 release 2017-06-28 13:57:09 -07:00
George Hotz 6fee0bdb2d Merge pull request #102 from energee/devel-crv
Bounty: 2016 Honda CR-V Touring
2017-06-22 11:52:38 -07:00
Ted Slesinski e40c161125 Addresses brake error review comment 2017-06-20 02:51:58 -04:00
Ted Slesinski 97eb55cc55 Fixes global saturation change 2017-06-20 02:47:47 -04:00
Ted Slesinski 65134be0d1 Adds correct value of 8 to gearshifter check array 2017-06-20 02:46:59 -04:00
Ted Slesinski dbf71a23aa Init crv variable 2017-06-20 02:26:50 -04:00
George Hotz 7fec3db1d6 Merge pull request #103 from pjlao307/pjlao-update-ui
Update UI to make text more readable in all conditions as requested by community
2017-06-19 20:37:37 -07:00
Joey Lao 26b573c1d0 Make lead car text a little brighter 2017-06-12 16:39:10 -07:00
Vehicle Researcher 5ec1e7307e Revert last commit since KPH can get 3 digits long 2017-06-12 09:30:47 -07:00
Vehicle Researcher 8bc36b7f21 Adjust position of left speed to align with label (based on 2 digit speeds) 2017-06-12 09:03:49 -07:00
Vehicle Researcher 3b909eb693 More code cleanup 2017-06-12 08:41:11 -07:00
Vehicle Researcher 57e39c4472 Use color param in ui_draw_rounded_rect instead of hard coding 2017-06-12 08:27:57 -07:00
Vehicle Researcher ff7672339c Add background to radar text. Code cleanup. 2017-06-12 08:19:47 -07:00
John Jones 6e824a2c22 missed one update in latcontrol 2017-06-11 21:33:44 -04:00
Vehicle Researcher 32fa49e093 Clean up code 2017-06-11 15:18:39 -07:00
Vehicle Researcher 2250eac58f Remove commented line 2017-06-11 15:16:46 -07:00
Vehicle Researcher 97be6b3a0e Update UI to make speed text more readable in all conditions as requested by community 2017-06-11 15:12:50 -07:00
John Jones 942655c947 adding steering change from video and removing integer div and adding ki/kp changes 2017-06-11 13:20:19 -04:00
Ted Slesinski 615db3f7fd Reverting steering to original value 2017-06-08 13:15:30 -04:00
Ted Slesinski 0bb75c5389 Syntax typo 2017-06-08 02:33:31 -04:00
Ted Slesinski 7b5ee81d2d Adds support for 2016 Honda CR-V Touring 2017-06-08 01:37:39 -04:00
Vehicle Researcher 7fe46f1e1d openpilot v0.3.2 release 2017-05-22 22:26:12 -07:00
George Hotz 50c0d1c9da Merge pull request #82 from energee/devel
DBC file for 2017 Honda CR-V
2017-05-22 09:00:44 -07:00
Ted Slesinski 6dbf544d06 Adds 2017 CR-V dbc file 2017-05-21 19:46:34 -04:00
Vehicle Researcher 41e3a0f699 openpilot v0.3.1 release 2017-05-17 00:40:33 -07:00
Vehicle Researcher c5d8aec28b openpilot v0.3.0 release 2017-05-12 16:46:20 -07:00
Vehicle Researcher 4653a9aef0 openpilot v0.3.0-devel release 2017-05-11 12:41:17 -07:00
George Hotz ab3492bb90 Merge pull request #93 from heidecjj/master
Bug fixes for testing suite
2017-05-02 23:34:32 -07:00
George Hotz ed7cbb3866 Merge pull request #90 from heidecjj/patch-1
Update maneuver.py
2017-05-02 23:30:36 -07:00
Josh Heidecker a30626cfe3 Fixed relative distance errors in testing 2017-05-02 22:01:03 -04:00
heidecjj d2c087b3e2 Update maneuver.py
self.speed_lead_breakpoints was reading from "speed_lead_values" instead of "speed_lead_breakpoints"
2017-04-20 17:09:15 -04:00
George Hotz 0a747f991d Merge pull request #65 from jeankalud/master
Fixed little endian (and scaling/offset)
2017-03-01 16:09:46 -08:00
jeankalud f5a1e86d85 Add files via upload
Removed unused debug strings
2017-03-01 18:13:37 -05:00
Vehicle Researcher 693bcb0f83 openpilot v0.2.9 release 2017-03-01 10:54:12 -08:00
jeankalud 0d0daed86e Add files via upload
Fixed little endian handling
Fixed offset / scaling handling
2017-03-01 11:02:46 -05:00
Vehicle Researcher 95a349abcc openpilot v0.2.8 release 2017-02-28 00:03:11 -08:00
Vehicle Researcher c6ba5dc539 openpilot v0.2.7 release 2017-02-08 15:01:17 -08:00
Vehicle Researcher 6c3afeec0f openpilot v0.2.6 release 2017-01-31 01:33:55 -08:00
Vehicle Researcher 29c58b4588 openpilot v0.2.5 release 2017-01-30 14:10:01 -08:00
Vehicle Researcher ecc565aa3f openpilot v0.2.4 release 2017-01-29 16:10:13 -08:00
George Hotz db61810f98 Merge pull request #49 from mullakhmetov/master
Added context manager to file reading
2017-01-27 01:17:23 -08:00
Artur Mullakhmetov 48f203ad5b Added context manager to file reading 2017-01-27 11:33:27 +03:00
George Hotz 6ab4ac2dfb Merge pull request #47 from mullakhmetov/crossplatform-gettime
Refactor cross-platform libc usage
2017-01-25 09:19:22 -08:00
Artur Mullakhmetov 9cb3c7b6e6 Refactor cross-platform libc usage 2017-01-25 16:16:00 +03:00
Vehicle Researcher adaa4ed350 openpilot v0.2.3 release 2017-01-11 14:30:55 -08:00
Vehicle Researcher a64b9aa9b8 openpilot v0.2.2 release 2017-01-10 11:10:04 -08:00
George Hotz 0138eca61d Merge pull request #40 from TheMutley/master
DBC modified to match the Vector format
2017-01-09 20:47:52 -08:00
Mutley 139a40de29 DBC modified to match the Vector format 2017-01-05 16:55:54 -05:00
Vehicle Researcher 17d9becd3c openpilot v0.2.1 release 2016-12-14 21:29:12 -08:00
Vehicle Researcher 449b482cc3 openpilot v0.2 release 2016-12-12 20:18:19 -08:00
espes a7e099c946 Update README.md 2016-11-30 14:31:28 -08:00
espes 610462be5a Merge pull request #2 from autti/ford-fusion
Ford Fusion 2017 dbc can file. refs #1.
2016-11-30 11:57:31 -08:00
Ariel Nuñez 207d32668f Added ford fusion dbc can file. refs #1.
Obtained from:

https://bitbucket.org/DataspeedInc/dbw_mkz_ros/downloads
2016-11-30 14:52:47 -05:00
Vehicle Researcher e94a30bec0 openpilot release 2016-11-29 18:34:21 -08:00
2061 changed files with 151551 additions and 220922 deletions
-19
View File
@@ -1,19 +0,0 @@
---
Checks: '
bugprone-*,
-bugprone-integer-division,
-bugprone-narrowing-conversions,
performance-*,
clang-analyzer-*,
misc-*,
-misc-unused-parameters,
modernize-*,
-modernize-avoid-c-arrays,
-modernize-deprecated-headers,
-modernize-use-auto,
-modernize-use-using,
-modernize-use-nullptr,
-modernize-use-trailing-return-type,
'
CheckOptions:
...
-39
View File
@@ -1,39 +0,0 @@
**/.git
.DS_Store
*.dylib
*.DSYM
*.d
*.pyc
*.pyo
.*.swp
.*.swo
.*.un~
*.tmp
*.o
*.o-*
*.os
*.os-*
*.so
*.a
venv/
.venv/
notebooks
phone
massivemap
neos
installer
chffr/app2
chffr/backend/env
selfdrive/nav
selfdrive/baseui
selfdrive/test/simulator2
**/cache_data
xx/plus
xx/community
xx/projects
!xx/projects/eon_testing_master
!xx/projects/map3d
xx/ops
xx/junk
-11
View File
@@ -1,11 +0,0 @@
root = true
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.{py,pyx,pxd}]
charset = utf-8
indent_style = space
indent_size = 2
-21
View File
@@ -1,21 +0,0 @@
* text=auto
# to move existing files into LFS:
# git add --renormalize .
*.onnx filter=lfs diff=lfs merge=lfs -text
*.svg filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
*.ttf filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
selfdrive/car/tests/test_models_segs.txt filter=lfs diff=lfs merge=lfs -text
system/hardware/tici/updater filter=lfs diff=lfs merge=lfs -text
third_party/**/*.a filter=lfs diff=lfs merge=lfs -text
third_party/**/*.so filter=lfs diff=lfs merge=lfs -text
third_party/**/*.so.* filter=lfs diff=lfs merge=lfs -text
third_party/**/*.dylib filter=lfs diff=lfs merge=lfs -text
third_party/acados/*/t_renderer filter=lfs diff=lfs merge=lfs -text
third_party/qt5/larch64/bin/lrelease filter=lfs diff=lfs merge=lfs -text
third_party/qt5/larch64/bin/lupdate filter=lfs diff=lfs merge=lfs -text
third_party/catch2/include/catch2/catch.hpp filter=lfs diff=lfs merge=lfs -text
-47
View File
@@ -1,47 +0,0 @@
name: Bug report
description: For issues with running openpilot on your comma device
labels: ["bug"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* If the issue likely only affects your car model or make, go back and open a **car bug report** instead.
* If the issue is related to the driving or driver monitoring models, you should open a [discussion](https://github.com/commaai/openpilot/discussions/categories/model-feedback) instead.
* Ensure you're running the latest openpilot release.
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: route
attributes:
label: Provide a route where the issue occurs
description: Ensure the route is fully uploaded at https://useradmin.comma.ai. We cannot look into issues without routes, or at least a Dongle ID.
placeholder: 77611a1fac303767|2020-05-11--16-37-07
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version
description: If you're not on release, provide the commit hash
placeholder: 0.8.10
validations:
required: true
- type: textarea
attributes:
label: Additional info
-14
View File
@@ -1,14 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: Car bug report
url: https://github.com/commaai/opendbc/issues/new
about: For issues with a particular car make or model
- name: Join the Discord
url: https://discord.comma.ai
about: The community Discord is for both openpilot development and experience discussion
- name: Report driving behavior feedback
url: https://discord.com/channels/469524606043160576/1254834193066623017
about: Feedback for the driving and driver monitoring models goes in the #driving-feedback in Discord
- name: Community Wiki
url: https://github.com/commaai/openpilot/wiki
about: Check out our community wiki
-8
View File
@@ -1,8 +0,0 @@
---
name: Enhancement
about: For openpilot enhancement suggestions
title: ''
labels: 'enhancement'
assignees: ''
---
-42
View File
@@ -1,42 +0,0 @@
name: PC bug report
description: For issues with running openpilot on PC
labels: ["PC"]
body:
- type: markdown
attributes:
value: >
Before creating a **bug report**, please check the following:
* Ensure you're running the latest openpilot release.
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
- type: textarea
attributes:
label: Describe the bug
description: Also include a description of how to reproduce the bug
validations:
required: true
- type: input
id: os-version
attributes:
label: OS Version
placeholder: Ubuntu 24.04
validations:
required: true
- type: input
id: version
attributes:
label: openpilot version or commit
placeholder: bd36f2ec8d3559909678eff2690c10a520938367
validations:
required: false
- type: textarea
attributes:
label: Additional info
-27
View File
@@ -1,27 +0,0 @@
CI / testing:
- changed-files:
- any-glob-to-all-files: "{.github/**,**/test_*,Jenkinsfile}"
car:
- changed-files:
- any-glob-to-all-files: '{selfdrive/car/**,opendbc_repo}'
simulation:
- changed-files:
- any-glob-to-all-files: 'tools/sim/**'
ui:
- changed-files:
- any-glob-to-all-files: '{selfdrive/ui/**,system/ui/**}'
tools:
- changed-files:
- any-glob-to-all-files: 'tools/**'
multilanguage:
- changed-files:
- any-glob-to-all-files: 'selfdrive/ui/translations/**'
autonomy:
- changed-files:
- any-glob-to-all-files: "{selfdrive/modeld/models/**,selfdrive/test/process_replay/model_replay_ref_commit}"
-68
View File
@@ -1,68 +0,0 @@
<!-- Please copy and paste the relevant template -->
<!--- ***** Template: Fingerprint *****
**Car**
Which car (make, model, year) this fingerprint is for
**Route**
A route with the fingerprint
-->
<!--- ***** Template: Car Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
**Route**
Route: [a route with the bug fix]
-->
<!--- ***** Template: Bugfix *****
**Description**
A description of the bug and the fix. Also link the issue if it exists.
**Verification**
Explain how you tested this bug fix.
-->
<!--- ***** Template: Car Port *****
**Checklist**
- [ ] added entry to CAR in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
- [ ] route with openpilot:
- [ ] route with stock system:
- [ ] car harness used (if comma doesn't sell it, put N/A):
-->
<!--- ***** Template: Refactor *****
**Description**
A description of the refactor, including the goals it accomplishes.
**Verification**
Explain how you tested the refactor for regressions.
-->
-58
View File
@@ -1,58 +0,0 @@
name: 'automatically cache based on current runner'
inputs:
path:
description: 'path to cache'
required: true
key:
description: 'key'
required: true
restore-keys:
description: 'restore-keys'
required: true
save:
description: 'whether to save the cache'
default: 'true'
required: false
outputs:
cache-hit:
description: 'cache hit occurred'
value: ${{ (contains(runner.name, 'nsc') && steps.ns-cache.outputs.cache-hit) ||
(!contains(runner.name, 'nsc') && inputs.save != 'false' && steps.gha-cache.outputs.cache-hit) ||
(!contains(runner.name, 'nsc') && inputs.save == 'false' && steps.gha-cache-ro.outputs.cache-hit) }}
runs:
using: "composite"
steps:
- name: setup namespace cache
id: ns-cache
if: ${{ contains(runner.name, 'nsc') }}
uses: namespacelabs/nscloud-cache-action@v1
with:
path: ${{ inputs.path }}
- name: setup github cache
id: gha-cache
if: ${{ !contains(runner.name, 'nsc') && inputs.save != 'false' }}
uses: 'actions/cache@v4'
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: ${{ inputs.restore-keys }}
- name: setup github cache
id: gha-cache-ro
if: ${{ !contains(runner.name, 'nsc') && inputs.save == 'false' }}
uses: 'actions/cache/restore@v4'
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: ${{ inputs.restore-keys }}
# make the directory manually in case we didn't get a hit, so it doesn't fail on future steps
- id: scons-cache-setup
shell: bash
run: |
mkdir -p ${{ inputs.path }}
sudo chmod -R 777 ${{ inputs.path }}
sudo chown -R $USER ${{ inputs.path }}
-52
View File
@@ -1,52 +0,0 @@
name: "PR review"
on:
pull_request_target:
types: [opened, reopened, synchronize, edited]
jobs:
labeler:
name: review
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: false
# Label PRs
- uses: actions/labeler@v5.0.0
with:
dot: true
configuration-path: .github/labeler.yaml
# Check PR target branch
- name: check branch
uses: Vankka/pr-target-branch-action@def32ec9d93514138d6ac0132ee62e120a72aed5
if: github.repository == 'commaai/openpilot'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
target: /^(?!master$).*/
exclude: /commaai:.*/
change-to: ${{ github.base_ref }}
already-exists-action: close_this
already-exists-comment: "Your PR should be made against the `master` branch"
# Welcome comment
- name: "First timers PR"
uses: actions/first-interaction@v1
if: github.event.pull_request.head.repo.full_name != 'commaai/openpilot'
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
pr-message: |
<!-- _(run_id **${{ github.run_id }}**)_ -->
Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:
* Convert your PR to a draft unless it's ready to review
* Read the [contributing docs](https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md)
* Before marking as "ready for review", ensure:
* the goal is clearly stated in the description
* all the tests are passing
* the change is [something we merge](https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md#what-gets-merged)
* include a route or your device' dongle ID if relevant
-37
View File
@@ -1,37 +0,0 @@
name: badges
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
env:
BASE_IMAGE: openpilot-base
DOCKER_REGISTRY: ghcr.io/commaai
RUN: docker run --shm-size 2G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v $GITHUB_WORKSPACE/.ci_cache/scons_cache:/tmp/scons_cache -v $GITHUB_WORKSPACE/.ci_cache/comma_download_cache:/tmp/comma_download_cache -v $GITHUB_WORKSPACE/.ci_cache/openpilot_cache:/tmp/openpilot_cache $DOCKER_REGISTRY/$BASE_IMAGE:latest /bin/bash -c
jobs:
badges:
name: create badges
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Push badges
run: |
${{ env.RUN }} "scons -j$(nproc) && python3 selfdrive/ui/translations/create_badges.py"
rm .gitattributes
git checkout --orphan badges
git rm -rf --cached .
git config user.email "badge-researcher@comma.ai"
git config user.name "Badge Researcher"
git add translation_badge.svg
git commit -m "Add/Update badges"
git push -f origin HEAD
-101
View File
@@ -1,101 +0,0 @@
name: weekly CI test report
on:
schedule:
- cron: '37 9 * * 1' # 9:37AM UTC -> 2:37AM PST every monday
workflow_dispatch:
inputs:
ci_runs:
description: 'The amount of runs to trigger in CI test report'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
CI_RUNS: ${{ github.event.inputs.ci_runs || '50' }}
jobs:
setup:
if: github.repository == 'commaai/openpilot'
runs-on: ubuntu-latest
outputs:
ci_runs: ${{ steps.ci_runs_setup.outputs.matrix }}
steps:
- id: ci_runs_setup
name: CI_RUNS=${{ env.CI_RUNS }}
run: |
matrix=$(python3 -c "import json; print(json.dumps({ 'run_number' : list(range(${{ env.CI_RUNS }})) }))")
echo "matrix=$matrix" >> $GITHUB_OUTPUT
ci_matrix_run:
needs: [ setup ]
strategy:
fail-fast: false
matrix: ${{fromJSON(needs.setup.outputs.ci_runs)}}
uses: commaai/openpilot/.github/workflows/ci_weekly_run.yaml@master
with:
run_number: ${{ matrix.run_number }}
report:
needs: [ci_matrix_run]
runs-on: ubuntu-latest
if: always()
steps:
- name: Get job results
uses: actions/github-script@v7
id: get-job-results
with:
script: |
const jobs = await github
.paginate("GET /repos/{owner}/{repo}/actions/runs/{run_id}/attempts/{attempt}/jobs", {
owner: "commaai",
repo: "${{ github.event.repository.name }}",
run_id: "${{ github.run_id }}",
attempt: "${{ github.run_attempt }}",
})
var report = {}
jobs.slice(1, jobs.length-1).forEach(job => {
if (job.conclusion === "skipped") return;
const jobName = job.name.split(" / ")[2];
const runRegex = /\((.*?)\)/;
const run = job.name.match(runRegex)[1];
report[jobName] = report[jobName] || { successes: [], failures: [], canceled: [] };
switch (job.conclusion) {
case "success":
report[jobName].successes.push({ "run_number": run, "link": job.html_url}); break;
case "failure":
report[jobName].failures.push({ "run_number": run, "link": job.html_url }); break;
case "canceled":
report[jobName].canceled.push({ "run_number": run, "link": job.html_url }); break;
}
});
return JSON.stringify({"jobs": report});
- name: Add job results to summary
env:
JOB_RESULTS: ${{ fromJSON(steps.get-job-results.outputs.result) }}
run: |
cat <<EOF >> template.html
<table>
<thead>
<tr>
<th></th>
<th>Job</th>
<th>✅ Passing</th>
<th>❌ Failure Details</th>
</tr>
</thead>
<tbody>
{% for key in jobs.keys() %}<tr>
<td>{% for i in range(5) %}{% if i+1 <= (5 * jobs[key]["successes"]|length // ${{ env.CI_RUNS }}) %}🟩{% else %}🟥{% endif %}{% endfor%}</td>
<td>{{ key }}</td>
<td>{{ 100 * jobs[key]["successes"]|length // ${{ env.CI_RUNS }} }}%</td>
<td>{% if jobs[key]["failures"]|length > 0 %}<details>{% for failure in jobs[key]["failures"] %}<a href="{{ failure['link'] }}">Log for run #{{ failure['run_number'] }}</a><br>{% endfor %}</details>{% else %}{% endif %}</td>
</td>
</tr>{% endfor %}
</table>
EOF
pip install jinja2-cli
echo $JOB_RESULTS | jinja2 template.html > report.html
echo "# CI Test Report - ${{ env.CI_RUNS }} Runs" >> $GITHUB_STEP_SUMMARY
cat report.html >> $GITHUB_STEP_SUMMARY
-17
View File
@@ -1,17 +0,0 @@
name: weekly CI test run
on:
workflow_call:
inputs:
run_number:
required: true
type: string
concurrency:
group: ci-run-${{ inputs.run_number }}-${{ github.ref }}
cancel-in-progress: true
jobs:
selfdrive_tests:
uses: commaai/openpilot/.github/workflows/selfdrive_tests.yaml@master
with:
run_number: ${{ inputs.run_number }}
@@ -1,21 +0,0 @@
name: 'compile openpilot'
runs:
using: "composite"
steps:
- shell: bash
name: Build openpilot with all flags
run: |
${{ env.RUN }} "scons -j$(nproc)"
${{ env.RUN }} "release/check-dirty.sh"
- shell: bash
name: Cleanup scons cache and rebuild
run: |
${{ env.RUN }} "rm -rf /tmp/scons_cache/* && \
scons -j$(nproc) --cache-populate"
- name: Save scons cache
uses: actions/cache/save@v4
if: github.ref == 'refs/heads/master'
with:
path: .ci_cache/scons_cache
key: scons-${{ runner.arch }}-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
-65
View File
@@ -1,65 +0,0 @@
name: docs
on:
push:
branches:
- master
pull_request:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: docs-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
jobs:
docs:
name: build docs
runs-on: ubuntu-24.04
steps:
- uses: commaai/timeout@v1
- uses: actions/checkout@v4
with:
submodules: true
# Build
- name: Build docs
run: |
# TODO: can we install just the "docs" dependency group without the normal deps?
pip install mkdocs
mkdocs build
# Push to docs.comma.ai
- uses: actions/checkout@v4
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
with:
path: openpilot-docs
ssh-key: ${{ secrets.OPENPILOT_DOCS_KEY }}
repository: commaai/openpilot-docs
- name: Push
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
run: |
set -x
source release/identity.sh
cd openpilot-docs
git checkout --orphan tmp
git rm -rf .
# copy over docs
cp -r ../docs_site/ docs/
# GitHub pages config
touch docs/.nojekyll
echo -n docs.comma.ai > docs/CNAME
git add -f .
git commit -m "build docs"
# docs live in different repo to not bloat openpilot's full clone size
git push -f origin tmp:gh-pages
-45
View File
@@ -1,45 +0,0 @@
name: jenkins scan
on:
issue_comment:
types: [created, edited]
jobs:
# TODO: gc old branches in a separate job in this workflow
scan-comments:
runs-on: ubuntu-latest
if: ${{ github.event.issue.pull_request }}
steps:
- name: Check for trigger phrase
id: check_comment
uses: actions/github-script@v7
with:
script: |
const triggerPhrase = "trigger-jenkins";
const comment = context.payload.comment.body;
const commenter = context.payload.comment.user.login;
const { data: permissions } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: commenter
});
const hasWriteAccess = permissions.permission === 'write' || permissions.permission === 'admin';
return (hasWriteAccess && comment.includes(triggerPhrase));
result-encoding: json
- name: Checkout repository
if: steps.check_comment.outputs.result == 'true'
uses: actions/checkout@v4
with:
ref: refs/pull/${{ github.event.issue.number }}/head
- name: Push to tmp-jenkins branch
if: steps.check_comment.outputs.result == 'true'
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b tmp-jenkins-${{ github.event.issue.number }}
GIT_LFS_SKIP_PUSH=1 git push -f origin tmp-jenkins-${{ github.event.issue.number }}
-42
View File
@@ -1,42 +0,0 @@
name: "model review"
on:
pull_request:
types: [opened, reopened, synchronize]
paths:
- 'selfdrive/modeld/models/*.onnx'
workflow_dispatch:
jobs:
comment:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Checkout master
uses: actions/checkout@v4
with:
ref: master
path: base
- run: git lfs pull
- run: cd base && git lfs pull
- run: pip install onnx
- name: scripts/reporter.py
id: report
run: |
echo "content<<EOF" >> $GITHUB_OUTPUT
echo "## Model Review" >> $GITHUB_OUTPUT
MASTER_PATH=${{ github.workspace }}/base python scripts/reporter.py >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Post model report comment
uses: marocchino/sticky-pull-request-comment@baa7203ed60924babbe5dcd0ac8eae3b66ec5e16
with:
header: model-review
message: ${{ steps.report.outputs.content }}
-39
View File
@@ -1,39 +0,0 @@
name: prebuilt
on:
schedule:
- cron: '0 * * * *'
workflow_dispatch:
env:
DOCKER_LOGIN: docker login ghcr.io -u ${{ github.actor }} -p ${{ secrets.GITHUB_TOKEN }}
BUILD: selfdrive/test/docker_build.sh prebuilt
jobs:
build_prebuilt:
name: build prebuilt
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
env:
PUSH_IMAGE: true
permissions:
checks: read
contents: read
packages: write
steps:
- name: Wait for green check mark
if: ${{ github.event_name != 'workflow_dispatch' }}
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
with:
ref: master
wait-interval: 30
running-workflow-name: 'build prebuilt'
repo-token: ${{ secrets.GITHUB_TOKEN }}
check-regexp: ^((?!.*(build master-ci).*).)*$
- uses: actions/checkout@v4
with:
submodules: true
- run: git lfs pull
- name: Build and Push docker image
run: |
$DOCKER_LOGIN
eval "$BUILD"
-54
View File
@@ -1,54 +0,0 @@
name: release
on:
schedule:
- cron: '0 9 * * *'
workflow_dispatch:
jobs:
build_masterci:
name: build master-ci
env:
TARGET_DIR: /tmp/openpilot
ImageOS: ubuntu20
container:
image: ghcr.io/commaai/openpilot-base:latest
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
permissions:
checks: read
contents: write
steps:
- name: Install wait-on-check-action dependencies
run: |
sudo apt-get update
sudo apt-get install -y libyaml-dev
- name: Wait for green check mark
if: ${{ github.event_name != 'workflow_dispatch' }}
uses: lewagon/wait-on-check-action@ccfb013c15c8afb7bf2b7c028fb74dc5a068cccc
with:
ref: master
wait-interval: 30
running-workflow-name: 'build master-ci'
repo-token: ${{ secrets.GITHUB_TOKEN }}
check-regexp: ^((?!.*(build prebuilt).*).)*$
- uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 0
- name: Pull LFS
run: |
git config --global --add safe.directory '*'
git lfs pull
- name: Build master-ci
run: |
release/build_devel.sh
- name: Run tests
run: |
export PYTHONPATH=$TARGET_DIR
cd $TARGET_DIR
scons -j$(nproc)
pytest -n logical selfdrive/car/tests/test_car_interfaces.py
- name: Push master-ci
run: |
unset TARGET_DIR
BRANCH=__nightly release/build_devel.sh
-71
View File
@@ -1,71 +0,0 @@
name: repo maintenance
on:
schedule:
- cron: "0 14 * * 1" # every Monday at 2am UTC (6am PST)
workflow_dispatch:
env:
BASE_IMAGE: openpilot-base
BUILD: selfdrive/test/docker_build.sh base
RUN: docker run --shm-size 2G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e CI=1 -e PYTHONWARNINGS=error -e FILEREADER_CACHE=1 -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v $GITHUB_WORKSPACE/.ci_cache/scons_cache:/tmp/scons_cache -v $GITHUB_WORKSPACE/.ci_cache/comma_download_cache:/tmp/comma_download_cache -v $GITHUB_WORKSPACE/.ci_cache/openpilot_cache:/tmp/openpilot_cache $BASE_IMAGE /bin/bash -c
jobs:
update_translations:
runs-on: ubuntu-latest
if: github.repository == 'commaai/openpilot'
steps:
- uses: actions/checkout@v4
- uses: ./.github/workflows/setup-with-retry
- name: Update translations
run: |
${{ env.RUN }} "python3 selfdrive/ui/update_translations.py --vanish"
- name: Create Pull Request
uses: peter-evans/create-pull-request@9153d834b60caba6d51c9b9510b087acf9f33f83
with:
author: Vehicle Researcher <user@comma.ai>
commit-message: "Update translations"
title: "[bot] Update translations"
body: "Automatic PR from repo-maintenance -> update_translations"
branch: "update-translations"
base: "master"
delete-branch: true
labels: bot
package_updates:
name: package_updates
runs-on: ubuntu-latest
container:
image: ghcr.io/commaai/openpilot-base:latest
if: github.repository == 'commaai/openpilot'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: uv lock
run: |
python3 -m ensurepip --upgrade
pip3 install uv
uv lock --upgrade
- name: bump submodules
run: |
git config --global --add safe.directory '*'
git submodule update --remote
git add .
- name: update car docs
run: |
scons -j$(nproc) --minimal opendbc_repo
PYTHONPATH=. python selfdrive/car/docs.py
git add docs/CARS.md
- name: Create Pull Request
uses: peter-evans/create-pull-request@9153d834b60caba6d51c9b9510b087acf9f33f83
with:
author: Vehicle Researcher <user@comma.ai>
token: ${{ secrets.ACTIONS_CREATE_PR_PAT }}
commit-message: Update Python packages
title: '[bot] Update Python packages'
branch: auto-package-updates
base: master
delete-branch: true
body: 'Automatic PR from repo-maintenance -> package_updates'
labels: bot
-352
View File
@@ -1,352 +0,0 @@
name: selfdrive
on:
push:
branches:
- master
pull_request:
workflow_dispatch:
workflow_call:
inputs:
run_number:
default: '1'
required: true
type: string
concurrency:
group: selfdrive-tests-ci-run-${{ inputs.run_number }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.run_id || github.head_ref || github.ref }}-${{ github.workflow }}-${{ github.event_name }}
cancel-in-progress: true
env:
PYTHONWARNINGS: error
BASE_IMAGE: openpilot-base
AZURE_TOKEN: ${{ secrets.AZURE_COMMADATACI_OPENPILOTCI_TOKEN }}
DOCKER_LOGIN: docker login ghcr.io -u ${{ github.actor }} -p ${{ secrets.GITHUB_TOKEN }}
BUILD: selfdrive/test/docker_build.sh base
RUN: docker run --shm-size 2G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e CI=1 -e PYTHONWARNINGS=error -e FILEREADER_CACHE=1 -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v $GITHUB_WORKSPACE/.ci_cache/scons_cache:/tmp/scons_cache -v $GITHUB_WORKSPACE/.ci_cache/comma_download_cache:/tmp/comma_download_cache -v $GITHUB_WORKSPACE/.ci_cache/openpilot_cache:/tmp/openpilot_cache $BASE_IMAGE /bin/bash -c
PYTEST: pytest --continue-on-collection-errors --cov --cov-report=xml --cov-append --durations=0 --durations-min=5 --hypothesis-seed 0 -n logical
jobs:
# build_release:
# name: build release
# runs-on:
# - ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-profile-amd64-8x16' || 'ubuntu-24.04' }}
- ${{ ((github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))) && 'namespace-experiments:docker.builds.local-cache=separate' || 'ubuntu-24.04' }}
# env:
# STRIPPED_DIR: /tmp/releasepilot
# steps:
# - uses: actions/checkout@v4
# with:
# submodules: true
# - name: Getting LFS files
# uses: nick-fields/retry@7152eba30c6575329ac0576536151aca5a72780e
# with:
# timeout_minutes: 2
# max_attempts: 3
# command: git lfs pull
# - name: Build devel
# timeout-minutes: 1
# run: TARGET_DIR=$STRIPPED_DIR release/build_devel.sh
# - uses: ./.github/workflows/setup-with-retry
# - name: Build openpilot and run checks
# timeout-minutes: ${{ ((steps.restore-scons-cache.outputs.cache-hit == 'true') && 10 || 30) }} # allow more time when we missed the scons cache
# run: |
# cd $STRIPPED_DIR
# ${{ env.RUN }} "python3 system/manager/build.py"
# - name: Run tests
# timeout-minutes: 1
# run: |
# cd $STRIPPED_DIR
# ${{ env.RUN }} "release/check-dirty.sh"
# - name: Check submodules
# if: github.repository == 'commaai/openpilot'
# timeout-minutes: 3
# run: release/check-submodules.sh
build:
runs-on:
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: Setup docker push
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'commaai/openpilot'
run: |
echo "PUSH_IMAGE=true" >> "$GITHUB_ENV"
$DOCKER_LOGIN
- uses: ./.github/workflows/setup-with-retry
- uses: ./.github/workflows/compile-openpilot
timeout-minutes: 30
build_mac:
name: build macOS
runs-on: 'macos-latest'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- run: echo "CACHE_COMMIT_DATE=$(git log -1 --pretty='format:%cd' --date=format:'%Y-%m-%d-%H:%M')" >> $GITHUB_ENV
- name: Homebrew cache
uses: ./.github/workflows/auto-cache
with:
path: ~/Library/Caches/Homebrew
key: brew-macos-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
restore-keys: |
brew-macos-${{ env.CACHE_COMMIT_DATE }}
brew-macos
- name: Install dependencies
run: ./tools/mac_setup.sh
env:
# package install has DeprecationWarnings
PYTHONWARNINGS: default
- run: git lfs pull
- name: Getting scons cache
uses: ./.github/workflows/auto-cache
with:
path: /tmp/scons_cache
key: scons-${{ runner.arch }}-macos-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
restore-keys: |
scons-${{ runner.arch }}-macos-${{ env.CACHE_COMMIT_DATE }}
scons-${{ runner.arch }}-macos
- name: Building openpilot
run: . .venv/bin/activate && scons -j$(nproc)
static_analysis:
name: static analysis
runs-on:
- 'ubuntu-24.04'
env:
PYTHONWARNINGS: default
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Static analysis
timeout-minutes: 1
run: ${{ env.RUN }} "scripts/lint/lint.sh"
unit_tests:
name: unit tests
runs-on:
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc)"
- name: Run unit tests
timeout-minutes: ${{ contains(runner.name, 'nsc') && 1 || 20 }}
run: |
${{ env.RUN }} "$PYTEST --collect-only -m 'not slow' &> /dev/null && \
MAX_EXAMPLES=1 $PYTEST -m 'not slow' && \
./selfdrive/ui/tests/create_test_translations.sh && \
QT_QPA_PLATFORM=offscreen ./selfdrive/ui/tests/test_translations && \
chmod -R 777 /tmp/comma_download_cache"
- name: "Upload coverage to Codecov"
uses: codecov/codecov-action@v4
with:
name: ${{ github.job }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
process_replay:
name: process replay
runs-on:
- 'ubuntu-24.04'
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Cache test routes
id: dependency-cache
uses: actions/cache@v4
with:
path: .ci_cache/comma_download_cache
key: proc-replay-${{ hashFiles('selfdrive/test/process_replay/ref_commit', 'selfdrive/test/process_replay/test_processes.py') }}
- name: Build openpilot
run: |
${{ env.RUN }} "scons -j$(nproc)"
- name: Run replay
timeout-minutes: ${{ contains(runner.name, 'nsc') && (steps.dependency-cache.outputs.cache-hit == 'true') && 1 || 20 }}
run: |
${{ env.RUN }} "coverage run selfdrive/test/process_replay/test_processes.py -j$(nproc) && \
chmod -R 777 /tmp/comma_download_cache && \
coverage combine && \
coverage xml"
- name: Print diff
id: print-diff
if: always()
run: cat selfdrive/test/process_replay/diff.txt
- uses: actions/upload-artifact@v4
if: always()
continue-on-error: true
with:
name: process_replay_diff.txt
path: selfdrive/test/process_replay/diff.txt
- name: Upload reference logs
if: ${{ failure() && steps.print-diff.outcome == 'success' && github.repository == 'commaai/openpilot' && env.AZURE_TOKEN != '' }}
run: |
${{ env.RUN }} "unset PYTHONWARNINGS && AZURE_TOKEN='$AZURE_TOKEN' python3 selfdrive/test/process_replay/test_processes.py -j$(nproc) --upload-only"
- name: Run regen
if: false
timeout-minutes: 4
run: |
${{ env.RUN }} "ONNXCPU=1 $PYTEST selfdrive/test/process_replay/test_regen.py && \
chmod -R 777 /tmp/comma_download_cache"
- name: "Upload coverage to Codecov"
uses: codecov/codecov-action@v4
with:
name: ${{ github.job }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
test_cars:
name: cars
runs-on:
- 'ubuntu-24.04'
strategy:
fail-fast: false
matrix:
job: [0, 1, 2, 3]
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Cache test routes
id: routes-cache
uses: actions/cache@v4
with:
path: .ci_cache/comma_download_cache
key: car_models-${{ hashFiles('selfdrive/car/tests/test_models.py', 'opendbc/car/tests/routes.py') }}-${{ matrix.job }}
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc)"
- name: Test car models
timeout-minutes: ${{ contains(runner.name, 'nsc') && (steps.routes-cache.outputs.cache-hit == 'true') && 1 || 6 }}
run: |
${{ env.RUN }} "MAX_EXAMPLES=1 $PYTEST selfdrive/car/tests/test_models.py && \
chmod -R 777 /tmp/comma_download_cache"
env:
NUM_JOBS: 4
JOB_ID: ${{ matrix.job }}
- name: "Upload coverage to Codecov"
uses: codecov/codecov-action@v4
with:
name: ${{ github.job }}-${{ matrix.job }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
car_docs_diff:
name: PR comments
runs-on: ubuntu-latest
#if: github.event_name == 'pull_request'
if: false # TODO: run this in opendbc?
steps:
- uses: actions/checkout@v4
with:
submodules: true
ref: ${{ github.event.pull_request.base.ref }}
- run: git lfs pull
- uses: ./.github/workflows/setup-with-retry
- name: Get base car info
run: |
${{ env.RUN }} "scons -j$(nproc) && python3 selfdrive/debug/dump_car_docs.py --path /tmp/openpilot_cache/base_car_docs"
sudo chown -R $USER:$USER ${{ github.workspace }}
- uses: actions/checkout@v4
with:
submodules: true
path: current
- run: cd current && git lfs pull
- name: Save car docs diff
id: save_diff
run: |
cd current
${{ env.RUN }} "scons -j$(nproc)"
output=$(${{ env.RUN }} "python3 selfdrive/debug/print_docs_diff.py --path /tmp/openpilot_cache/base_car_docs")
output="${output//$'\n'/'%0A'}"
echo "::set-output name=diff::$output"
- name: Find comment
if: ${{ env.AZURE_TOKEN != '' }}
uses: peter-evans/find-comment@3eae4d37986fb5a8592848f6a574fdf654e61f9e
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
body-includes: This PR makes changes to
- name: Update comment
if: ${{ steps.save_diff.outputs.diff != '' && env.AZURE_TOKEN != '' }}
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: "${{ steps.save_diff.outputs.diff }}"
edit-mode: replace
- name: Delete comment
if: ${{ steps.fc.outputs.comment-id != '' && steps.save_diff.outputs.diff == '' && env.AZURE_TOKEN != '' }}
uses: actions/github-script@v7
with:
script: |
github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: ${{ steps.fc.outputs.comment-id }}
})
simulator_driving:
name: simulator driving
runs-on:
- 'ubuntu-24.04'
if: (github.repository == 'commaai/openpilot') && ((github.event_name != 'pull_request') || (github.event.pull_request.head.repo.full_name == 'commaai/openpilot'))
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: Build openpilot
run: |
${{ env.RUN }} "scons -j$(nproc)"
- name: Driving test
timeout-minutes: 1
run: |
${{ env.RUN }} "source selfdrive/test/setup_xvfb.sh && \
source selfdrive/test/setup_vsound.sh && \
CI=1 pytest -s tools/sim/tests/test_metadrive_bridge.py"
create_ui_report:
# This job name needs to be the same as UI_JOB_NAME in ui_preview.yaml
name: Create UI Report
runs-on:
- 'ubuntu-24.04'
if: false # FIXME: FrameReader is broken on CI runners
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: ./.github/workflows/setup-with-retry
- name: caching frames
id: frames-cache
uses: actions/cache@v4
with:
path: .ci_cache/comma_download_cache
key: ui_screenshots_test_${{ hashFiles('selfdrive/ui/tests/test_ui/run.py') }}
- name: Build openpilot
run: ${{ env.RUN }} "scons -j$(nproc)"
- name: Create Test Report
timeout-minutes: ${{ ((steps.frames-cache.outputs.cache-hit == 'true') && 1 || 3) }}
run: >
${{ env.RUN }} "PYTHONWARNINGS=ignore &&
source selfdrive/test/setup_xvfb.sh &&
CACHE_ROOT=/tmp/comma_download_cache python3 selfdrive/ui/tests/test_ui/run.py &&
chmod -R 777 /tmp/comma_download_cache"
- name: Upload Test Report
uses: actions/upload-artifact@v4
with:
name: report-${{ inputs.run_number || '1' }}-${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
path: selfdrive/ui/tests/test_ui/report_1/screenshots
@@ -1,37 +0,0 @@
name: 'openpilot env setup, with retry on failure'
inputs:
docker_hub_pat:
description: 'Auth token for Docker Hub, required for BuildJet jobs'
required: false
default: ''
sleep_time:
description: 'Time to sleep between retries'
required: false
default: 30
runs:
using: "composite"
steps:
- id: setup1
uses: ./.github/workflows/setup
continue-on-error: true
with:
is_retried: true
- if: steps.setup1.outcome == 'failure'
shell: bash
run: sleep ${{ inputs.sleep_time }}
- id: setup2
if: steps.setup1.outcome == 'failure'
uses: ./.github/workflows/setup
continue-on-error: true
with:
is_retried: true
- if: steps.setup2.outcome == 'failure'
shell: bash
run: sleep ${{ inputs.sleep_time }}
- id: setup3
if: steps.setup2.outcome == 'failure'
uses: ./.github/workflows/setup
with:
is_retried: true
-56
View File
@@ -1,56 +0,0 @@
name: 'openpilot env setup'
inputs:
is_retried:
description: 'A mock param that asserts that we use the setup-with-retry instead of this action directly'
required: false
default: 'false'
runs:
using: "composite"
steps:
# assert that this action is retried using the setup-with-retry
- shell: bash
if: ${{ inputs.is_retried == 'false' }}
run: |
echo "You should not run this action directly. Use setup-with-retry instead"
exit 1
- shell: bash
name: No retries!
run: |
if [ "${{ github.run_attempt }}" -gt 1 ]; then
echo -e "\033[0;31m##################################################"
echo -e "\033[0;31m Retries not allowed! Fix the flaky test! "
echo -e "\033[0;31m##################################################\033[0m"
exit 1
fi
# do this after checkout to ensure our custom LFS config is used to pull from GitLab
- shell: bash
run: git lfs pull
# build cache
- id: date
shell: bash
run: echo "CACHE_COMMIT_DATE=$(git log -1 --pretty='format:%cd' --date=format:'%Y-%m-%d-%H:%M')" >> $GITHUB_ENV
- shell: bash
run: echo "$CACHE_COMMIT_DATE"
- id: scons-cache
uses: ./.github/workflows/auto-cache
with:
path: .ci_cache/scons_cache
key: scons-${{ runner.arch }}-${{ env.CACHE_COMMIT_DATE }}-${{ github.sha }}
restore-keys: |
scons-${{ runner.arch }}-${{ env.CACHE_COMMIT_DATE }}
scons-${{ runner.arch }}
# as suggested here: https://github.com/moby/moby/issues/32816#issuecomment-910030001
- id: normalize-file-permissions
shell: bash
name: Normalize file permissions to ensure a consistent docker build cache
run: |
find . -type f -executable -not -perm 755 -exec chmod 755 {} \;
find . -type f -not -executable -not -perm 644 -exec chmod 644 {} \;
# build our docker image
- shell: bash
run: eval ${{ env.BUILD }}
-29
View File
@@ -1,29 +0,0 @@
name: stale
on:
schedule:
- cron: '30 1 * * *'
workflow_dispatch:
env:
DAYS_BEFORE_PR_CLOSE: 2
DAYS_BEFORE_PR_STALE: 9
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
with:
exempt-all-milestones: true
# pull request config
stale-pr-message: 'This PR has had no activity for ${{ env.DAYS_BEFORE_PR_STALE }} days. It will be automatically closed in ${{ env.DAYS_BEFORE_PR_CLOSE }} days if there is no activity.'
close-pr-message: 'This PR has been automatically closed due to inactivity. Feel free to re-open once activity resumes.'
stale-pr-label: stale
delete-branch: ${{ github.event.pull_request.head.repo.full_name == 'commaai/openpilot' }} # only delete branches on the main repo
exempt-pr-labels: "ignore stale,needs testing" # if wip or it needs testing from the community, don't mark as stale
days-before-pr-stale: ${{ env.DAYS_BEFORE_PR_STALE }}
days-before-pr-close: ${{ env.DAYS_BEFORE_PR_CLOSE }}
# issue config
days-before-issue-stale: -1 # ignore issues for now
-174
View File
@@ -1,174 +0,0 @@
name: "ui preview"
on:
push:
branches:
- master
pull_request_target:
types: [assigned, opened, synchronize, reopened, edited]
branches:
- 'master'
paths:
- 'selfdrive/ui/**'
workflow_dispatch:
env:
UI_JOB_NAME: "Create UI Report"
REPORT_NAME: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && 'master' || github.event.number }}
SHA: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && github.sha || github.event.pull_request.head.sha }}
BRANCH_NAME: "openpilot/pr-${{ github.event.number }}"
jobs:
preview:
#if: github.repository == 'commaai/openpilot'
if: false # FIXME: FrameReader is broken on CI runners
name: preview
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
pull-requests: write
actions: read
steps:
- name: Waiting for ui generation to start
run: sleep 30
- name: Waiting for ui generation to end
uses: lewagon/wait-on-check-action@v1.3.4
with:
ref: ${{ env.SHA }}
check-name: ${{ env.UI_JOB_NAME }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
allowed-conclusions: success
wait-interval: 20
- name: Getting workflow run ID
id: get_run_id
run: |
echo "run_id=$(curl https://api.github.com/repos/${{ github.repository }}/commits/${{ env.SHA }}/check-runs | jq -r '.check_runs[] | select(.name == "${{ env.UI_JOB_NAME }}") | .html_url | capture("(?<number>[0-9]+)") | .number')" >> $GITHUB_OUTPUT
- name: Getting proposed ui
id: download-artifact
uses: dawidd6/action-download-artifact@v6
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
run_id: ${{ steps.get_run_id.outputs.run_id }}
search_artifacts: true
name: report-1-${{ env.REPORT_NAME }}
path: ${{ github.workspace }}/pr_ui
- name: Getting master ui
uses: actions/checkout@v4
with:
repository: commaai/ci-artifacts
ssh-key: ${{ secrets.CI_ARTIFACTS_DEPLOY_KEY }}
path: ${{ github.workspace }}/master_ui
ref: openpilot_master_ui
- name: Saving new master ui
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
working-directory: ${{ github.workspace }}/master_ui
run: |
git checkout --orphan=new_master_ui
git rm -rf *
git branch -D openpilot_master_ui
git branch -m openpilot_master_ui
git config user.name "GitHub Actions Bot"
git config user.email "<>"
mv ${{ github.workspace }}/pr_ui/*.png .
git add .
git commit -m "screenshots for commit ${{ env.SHA }}"
git push origin openpilot_master_ui --force
- name: Finding diff
if: github.event_name == 'pull_request_target'
id: find_diff
run: >-
sudo apt-get update && sudo apt-get install -y imagemagick
scenes=$(find ${{ github.workspace }}/pr_ui/*.png -type f -printf "%f\n" | cut -d '.' -f 1 | grep -v 'pair_device')
A=($scenes)
DIFF=""
TABLE="<details><summary>All Screenshots</summary>"
TABLE="${TABLE}<table>"
for ((i=0; i<${#A[*]}; i=i+1));
do
# Check if the master file exists
if [ ! -f "${{ github.workspace }}/master_ui/${A[$i]}.png" ]; then
# This is a new file in PR UI that doesn't exist in master
DIFF="${DIFF}<details open>"
DIFF="${DIFF}<summary>${A[$i]} : \$\${\\color{cyan}\\text{NEW}}\$\$</summary>"
DIFF="${DIFF}<table>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}</table>"
DIFF="${DIFF}</details>"
elif ! compare -fuzz 2% -highlight-color DeepSkyBlue1 -lowlight-color Black -compose Src ${{ github.workspace }}/master_ui/${A[$i]}.png ${{ github.workspace }}/pr_ui/${A[$i]}.png ${{ github.workspace }}/pr_ui/${A[$i]}_diff.png; then
convert ${{ github.workspace }}/pr_ui/${A[$i]}_diff.png -transparent black mask.png
composite mask.png ${{ github.workspace }}/master_ui/${A[$i]}.png composite_diff.png
convert -delay 100 ${{ github.workspace }}/master_ui/${A[$i]}.png composite_diff.png -loop 0 ${{ github.workspace }}/pr_ui/${A[$i]}_diff.gif
mv ${{ github.workspace }}/master_ui/${A[$i]}.png ${{ github.workspace }}/pr_ui/${A[$i]}_master_ref.png
DIFF="${DIFF}<details open>"
DIFF="${DIFF}<summary>${A[$i]} : \$\${\\color{red}\\text{DIFFERENT}}\$\$</summary>"
DIFF="${DIFF}<table>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> master <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_master_ref.png\"> </td>"
DIFF="${DIFF} <td> proposed <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}<tr>"
DIFF="${DIFF} <td> diff <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.png\"> </td>"
DIFF="${DIFF} <td> composite diff <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}_diff.gif\"> </td>"
DIFF="${DIFF}</tr>"
DIFF="${DIFF}</table>"
DIFF="${DIFF}</details>"
else
rm -f ${{ github.workspace }}/pr_ui/${A[$i]}_diff.png
fi
INDEX=$(($i % 2))
if [[ $INDEX -eq 0 ]]; then
TABLE="${TABLE}<tr>"
fi
TABLE="${TABLE} <td> <img src=\"https://raw.githubusercontent.com/commaai/ci-artifacts/${{ env.BRANCH_NAME }}/${A[$i]}.png\"> </td>"
if [[ $INDEX -eq 1 || $(($i + 1)) -eq ${#A[*]} ]]; then
TABLE="${TABLE}</tr>"
fi
done
TABLE="${TABLE}</table></details>"
echo "DIFF=$DIFF$TABLE" >> "$GITHUB_OUTPUT"
- name: Saving proposed ui
if: github.event_name == 'pull_request_target'
working-directory: ${{ github.workspace }}/master_ui
run: |
git config user.name "GitHub Actions Bot"
git config user.email "<>"
git checkout --orphan=${{ env.BRANCH_NAME }}
git rm -rf *
mv ${{ github.workspace }}/pr_ui/* .
git add .
git commit -m "screenshots for PR #${{ github.event.number }}"
git push origin ${{ env.BRANCH_NAME }} --force
- name: Comment Screenshots on PR
if: github.event_name == 'pull_request_target'
uses: thollander/actions-comment-pull-request@v2
with:
message: |
<!-- _(run_id_screenshots **${{ github.run_id }}**)_ -->
## UI Preview
${{ steps.find_diff.outputs.DIFF }}
comment_tag: run_id_screenshots
pr_number: ${{ github.event.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+6 -80
View File
@@ -1,22 +1,9 @@
venv/
.venv/
.ci_cache
.env
.clang-format
.DS_Store
.tags
.ipynb_checkpoints
.idea
.overlay_init
.overlay_consistent
.sconsign.dblite
model2.png
a.out
.hypothesis
/docs_site/
*.dylib
*.DSYM
*.d
*.pyc
@@ -24,81 +11,20 @@ a.out
.*.swp
.*.swo
.*.un~
*.tmp
*.o
*.o-*
*.os
*.os-*
*.so
*.a
*.clb
*.class
*.pyxbldc
*.vcd
*.qm
*_pyx.cpp
config.json
clcache
compile_commands.json
compare_runtime*.html
persist
selfdrive/pandad/pandad
cereal/services.h
cereal/gen
cereal/messaging/bridge
selfdrive/mapd/default_speeds_by_region.json
system/proclogd/proclogd
selfdrive/ui/translations/tmp
selfdrive/test/longitudinal_maneuvers/out
selfdrive/car/tests/cars_dump
system/camerad/camerad
system/camerad/test/ae_gray_test
board/obj/
selfdrive/boardd/boardd
selfdrive/logcatd/logcatd
selfdrive/proclogd/proclogd
selfdrive/ui/ui
/src/
notebooks
hyperthneed
provisioning
.coverage*
coverage.xml
htmlcov
pandaextra
.mypy_cache/
flycheck_*
cppcheck_report.txt
comma*.sh
selfdrive/modeld/thneed/compile
selfdrive/modeld/models/*.thneed
selfdrive/modeld/models/*.pkl
*.bz2
*.zst
build/
!**/.gitkeep
poetry.toml
Pipfile
### VisualStudioCode ###
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
!.vscode/*.code-snippets
# Local History for Visual Studio Code
.history/
# Built Visual Studio Code Extensions
*.vsix
### VisualStudioCode Patch ###
# Ignore all local history of files
.history
.ionide
-18
View File
@@ -1,18 +0,0 @@
[submodule "panda"]
path = panda
url = ../../commaai/panda.git
[submodule "opendbc"]
path = opendbc_repo
url = ../../commaai/opendbc.git
[submodule "msgq"]
path = msgq_repo
url = ../../commaai/msgq.git
[submodule "rednose_repo"]
path = rednose_repo
url = ../../commaai/rednose.git
[submodule "teleoprtc_repo"]
path = teleoprtc_repo
url = ../../commaai/teleoprtc
[submodule "tinygrad"]
path = tinygrad_repo
url = https://github.com/tinygrad/tinygrad.git
-4
View File
@@ -1,4 +0,0 @@
[lfs]
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
locksverify = false
+12
View File
@@ -0,0 +1,12 @@
sudo: required
services:
- docker
install:
- docker build -t tmppilot -f Dockerfile.openpilot .
script:
- docker run --rm
-v "$(pwd)"/selfdrive/test/tests/plant/out:/tmp/openpilot/selfdrive/test/tests/plant/out
tmppilot /bin/sh -c 'cd /tmp/openpilot/selfdrive/test/tests/plant && OPTEST=1 ./test_longitudinal.py'
-8
View File
@@ -1,8 +0,0 @@
{
"recommendations": [
"ms-python.python",
"ms-vscode.cpptools",
"elagil.pre-commit-helper",
"charliermarsh.ruff",
]
}
-46
View File
@@ -1,46 +0,0 @@
{
"version": "0.2.0",
"inputs": [
{
"id": "python_process",
"type": "pickString",
"description": "Select the process to debug",
"options": [
"selfdrive/controls/controlsd.py",
"system/timed/timed.py",
"tools/sim/run_bridge.py"
]
},
{
"id": "cpp_process",
"type": "pickString",
"description": "Select the process to debug",
"options": [
"selfdrive/ui/ui"
]
},
{
"id": "args",
"description": "Arguments to pass to the process",
"type": "promptString"
}
],
"configurations": [
{
"name": "Python: openpilot Process",
"type": "debugpy",
"request": "launch",
"program": "${input:python_process}",
"console": "integratedTerminal",
"justMyCode": true,
"args": "${input:args}"
},
{
"name": "C++: openpilot Process",
"type": "cppdbg",
"request": "launch",
"program": "${workspaceFolder}/${input:cpp_process}",
"cwd": "${workspaceFolder}",
}
]
}
-27
View File
@@ -1,27 +0,0 @@
{
"editor.tabSize": 2,
"editor.insertSpaces": true,
"editor.renderWhitespace": "trailing",
"files.trimTrailingWhitespace": true,
"search.exclude": {
"**/.git": true,
"**/.venv": true,
"**/__pycache__": true
},
"files.exclude": {
"**/.git": true,
"**/.venv": true,
"**/__pycache__": true
},
"python.analysis.exclude": [
"**/.git",
"**/.venv",
"**/__pycache__",
// exclude directories that should be using the symlinked version
"common/**",
"selfdrive/**",
"system/**",
"third_party/**",
"tools/**",
]
}
+14 -8
View File
@@ -1,13 +1,19 @@
FROM ghcr.io/commaai/openpilot-base:latest
FROM ubuntu:16.04
ENV PYTHONUNBUFFERED 1
ENV PYTHONUNBUFFERED=1
RUN apt-get update && apt-get install -y build-essential clang vim screen wget bzip2 git libglib2.0-0 python-pip capnproto libcapnp-dev libzmq5-dev libffi-dev libusb-1.0-0
RUN pip install numpy==1.11.2 scipy==0.18.1 matplotlib
ENV OPENPILOT_PATH=/home/batman/openpilot
ENV PYTHONPATH=${OPENPILOT_PATH}:${PYTHONPATH}
COPY requirements_openpilot.txt /tmp/
RUN pip install -r /tmp/requirements_openpilot.txt
RUN mkdir -p ${OPENPILOT_PATH}
WORKDIR ${OPENPILOT_PATH}
ENV PYTHONPATH /tmp/openpilot:$PYTHONPATH
COPY . ${OPENPILOT_PATH}/
COPY ./common /tmp/openpilot/common
COPY ./cereal /tmp/openpilot/cereal
COPY ./opendbc /tmp/openpilot/opendbc
COPY ./selfdrive /tmp/openpilot/selfdrive
COPY ./phonelibs /tmp/openpilot/phonelibs
COPY ./pyextra /tmp/openpilot/pyextra
RUN scons --cache-readonly -j$(nproc)
RUN mkdir -p /tmp/openpilot/selfdrive/test/out
-81
View File
@@ -1,81 +0,0 @@
FROM ubuntu:24.04
ENV PYTHONUNBUFFERED=1
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends sudo tzdata locales ssh pulseaudio xvfb x11-xserver-utils gnome-screenshot python3-tk python3-dev && \
rm -rf /var/lib/apt/lists/*
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8
COPY tools/install_ubuntu_dependencies.sh /tmp/tools/
RUN /tmp/tools/install_ubuntu_dependencies.sh && \
rm -rf /var/lib/apt/lists/* /tmp/* && \
cd /usr/lib/gcc/arm-none-eabi/* && \
rm -rf arm/ thumb/nofp thumb/v6* thumb/v8* thumb/v7+fp thumb/v7-r+fp.sp
# Add OpenCL
RUN apt-get update && apt-get install -y --no-install-recommends \
apt-utils \
alien \
unzip \
tar \
curl \
xz-utils \
dbus \
gcc-arm-none-eabi \
tmux \
vim \
libx11-6 \
wget \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir -p /tmp/opencl-driver-intel && \
cd /tmp/opencl-driver-intel && \
wget https://github.com/intel/llvm/releases/download/2024-WW14/oclcpuexp-2024.17.3.0.09_rel.tar.gz && \
wget https://github.com/oneapi-src/oneTBB/releases/download/v2021.12.0/oneapi-tbb-2021.12.0-lin.tgz && \
mkdir -p /opt/intel/oclcpuexp_2024.17.3.0.09_rel && \
cd /opt/intel/oclcpuexp_2024.17.3.0.09_rel && \
tar -zxvf /tmp/opencl-driver-intel/oclcpuexp-2024.17.3.0.09_rel.tar.gz && \
mkdir -p /etc/OpenCL/vendors && \
echo /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64/libintelocl.so > /etc/OpenCL/vendors/intel_expcpu.icd && \
cd /opt/intel && \
tar -zxvf /tmp/opencl-driver-intel/oneapi-tbb-2021.12.0-lin.tgz && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbb.so /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbbmalloc.so /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbb.so.12 /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
ln -s /opt/intel/oneapi-tbb-2021.12.0/lib/intel64/gcc4.8/libtbbmalloc.so.2 /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 && \
mkdir -p /etc/ld.so.conf.d && \
echo /opt/intel/oclcpuexp_2024.17.3.0.09_rel/x64 > /etc/ld.so.conf.d/libintelopenclexp.conf && \
ldconfig -f /etc/ld.so.conf.d/libintelopenclexp.conf && \
cd / && \
rm -rf /tmp/opencl-driver-intel
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=graphics,utility,compute
ENV QTWEBENGINE_DISABLE_SANDBOX=1
RUN dbus-uuidgen > /etc/machine-id
ARG USER=batman
ARG USER_UID=1001
RUN useradd -m -s /bin/bash -u $USER_UID $USER
RUN usermod -aG sudo $USER
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER $USER
COPY --chown=$USER pyproject.toml uv.lock /home/$USER
COPY --chown=$USER tools/install_python_dependencies.sh /home/$USER/tools/
ENV VIRTUAL_ENV=/home/$USER/.venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN cd /home/$USER && \
tools/install_python_dependencies.sh && \
rm -rf tools/ pyproject.toml uv.lock .cache
USER root
RUN sudo git config --global --add safe.directory /tmp/openpilot
Vendored
-283
View File
@@ -1,283 +0,0 @@
def retryWithDelay(int maxRetries, int delay, Closure body) {
for (int i = 0; i < maxRetries; i++) {
try {
return body()
} catch (Exception e) {
sleep(delay)
}
}
throw Exception("Failed after ${maxRetries} retries")
}
def device(String ip, String step_label, String cmd) {
withCredentials([file(credentialsId: 'id_rsa', variable: 'key_file')]) {
def ssh_cmd = """
ssh -o ConnectTimeout=5 -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o BatchMode=yes -o StrictHostKeyChecking=no -i ${key_file} 'comma@${ip}' exec /usr/bin/bash <<'END'
set -e
export TERM=xterm-256color
shopt -s huponexit # kill all child processes when the shell exits
export CI=1
export PYTHONWARNINGS=error
export LOGPRINT=debug
export TEST_DIR=${env.TEST_DIR}
export SOURCE_DIR=${env.SOURCE_DIR}
export GIT_BRANCH=${env.GIT_BRANCH}
export GIT_COMMIT=${env.GIT_COMMIT}
export CI_ARTIFACTS_TOKEN=${env.CI_ARTIFACTS_TOKEN}
export GITHUB_COMMENTS_TOKEN=${env.GITHUB_COMMENTS_TOKEN}
export AZURE_TOKEN='${env.AZURE_TOKEN}'
# only use 1 thread for tici tests since most require HIL
export PYTEST_ADDOPTS="-n0 -s"
export GIT_SSH_COMMAND="ssh -i /data/gitkey"
source ~/.bash_profile
if [ -f /TICI ]; then
source /etc/profile
rm -rf /tmp/tmp*
rm -rf ~/.commacache
rm -rf /dev/shm/*
rm -rf /dev/tmp/tmp*
if ! systemctl is-active --quiet systemd-resolved; then
echo "restarting resolved"
sudo systemctl start systemd-resolved
sleep 3
fi
# restart aux USB
if [ -e /sys/bus/usb/drivers/hub/3-0:1.0 ]; then
echo "restarting aux usb"
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/unbind
sleep 0.5
echo "3-0:1.0" | sudo tee /sys/bus/usb/drivers/hub/bind
fi
fi
if [ -f /data/openpilot/launch_env.sh ]; then
source /data/openpilot/launch_env.sh
fi
ln -snf ${env.TEST_DIR} /data/pythonpath
cd ${env.TEST_DIR} || true
time ${cmd}
END"""
sh script: ssh_cmd, label: step_label
}
}
def deviceStage(String stageName, String deviceType, List extra_env, def steps) {
stage(stageName) {
if (currentBuild.result != null) {
return
}
if (isReplay()) {
error("REPLAYING TESTS IS NOT ALLOWED. FIX THEM INSTEAD.")
}
def extra = extra_env.collect { "export ${it}" }.join('\n');
def branch = env.BRANCH_NAME ?: 'master';
def gitDiff = sh returnStdout: true, script: 'curl -s -H "Authorization: Bearer ${GITHUB_COMMENTS_TOKEN}" https://api.github.com/repos/commaai/openpilot/compare/master...${GIT_BRANCH} | jq .files[].filename || echo "/"', label: 'Getting changes'
lock(resource: "", label: deviceType, inversePrecedence: true, variable: 'device_ip', quantity: 1, resourceSelectStrategy: 'random') {
docker.image('ghcr.io/commaai/alpine-ssh').inside('--user=root') {
timeout(time: 35, unit: 'MINUTES') {
retry (3) {
def date = sh(script: 'date', returnStdout: true).trim();
device(device_ip, "set time", "date -s '" + date + "'")
device(device_ip, "git checkout", extra + "\n" + readFile("selfdrive/test/setup_device_ci.sh"))
}
steps.each { item ->
def name = item[0]
def cmd = item[1]
def args = item[2]
def diffPaths = args.diffPaths ?: []
def cmdTimeout = args.timeout ?: 9999
if (branch != "master" && !branch.contains("__jenkins_loop_") && diffPaths && !hasPathChanged(gitDiff, diffPaths)) {
println "Skipping ${name}: no changes in ${diffPaths}."
return
} else {
timeout(time: cmdTimeout, unit: 'SECONDS') {
device(device_ip, name, cmd)
}
}
}
}
}
}
}
}
def hasPathChanged(String gitDiff, List<String> paths) {
for (path in paths) {
if (gitDiff.contains(path)) {
return true
}
}
return false
}
def isReplay() {
def replayClass = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
return currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replayClass) }
}
def setupCredentials() {
withCredentials([
string(credentialsId: 'azure_token', variable: 'AZURE_TOKEN'),
]) {
env.AZURE_TOKEN = "${AZURE_TOKEN}"
}
withCredentials([
string(credentialsId: 'ci_artifacts_pat', variable: 'CI_ARTIFACTS_TOKEN'),
]) {
env.CI_ARTIFACTS_TOKEN = "${CI_ARTIFACTS_TOKEN}"
}
withCredentials([
string(credentialsId: 'post_comments_github_pat', variable: 'GITHUB_COMMENTS_TOKEN'),
]) {
env.GITHUB_COMMENTS_TOKEN = "${GITHUB_COMMENTS_TOKEN}"
}
}
def step(String name, String cmd, Map args = [:]) {
return [name, cmd, args]
}
node {
env.CI = "1"
env.PYTHONWARNINGS = "error"
env.TEST_DIR = "/data/openpilot"
env.SOURCE_DIR = "/data/openpilot_source/"
setupCredentials()
env.GIT_BRANCH = checkout(scm).GIT_BRANCH
env.GIT_COMMIT = checkout(scm).GIT_COMMIT
def excludeBranches = ['__nightly', 'devel', 'devel-staging', 'release3', 'release3-staging',
'testing-closet*', 'hotfix-*']
def excludeRegex = excludeBranches.join('|').replaceAll('\\*', '.*')
if (env.BRANCH_NAME != 'master' && !env.BRANCH_NAME.contains('__jenkins_loop_')) {
properties([
disableConcurrentBuilds(abortPrevious: true)
])
}
try {
if (env.BRANCH_NAME == 'devel-staging') {
deviceStage("build release3-staging", "tici-needs-can", [], [
step("build release3-staging", "RELEASE_BRANCH=release3-staging $SOURCE_DIR/release/build_release.sh"),
])
}
if (env.BRANCH_NAME == '__nightly') {
parallel (
'nightly': {
deviceStage("build nightly", "tici-needs-can", [], [
step("build nightly", "RELEASE_BRANCH=nightly $SOURCE_DIR/release/build_release.sh"),
])
},
'nightly-dev': {
deviceStage("build nightly-dev", "tici-needs-can", [], [
step("build nightly-dev", "PANDA_DEBUG_BUILD=1 RELEASE_BRANCH=nightly-dev $SOURCE_DIR/release/build_release.sh"),
])
},
)
}
if (!env.BRANCH_NAME.matches(excludeRegex)) {
parallel (
// tici tests
'onroad tests': {
deviceStage("onroad", "tici-needs-can", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("check dirty", "release/check-dirty.sh"),
step("onroad tests", "pytest selfdrive/test/test_onroad.py -s", [timeout: 60]),
])
},
'HW + Unit Tests': {
deviceStage("tici-hardware", "tici-common", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py", [diffPaths: ["panda", "selfdrive/pandad/"]]),
step("test power draw", "pytest -s system/hardware/tici/tests/test_power_draw.py"),
step("test encoder", "LD_LIBRARY_PATH=/usr/local/lib pytest system/loggerd/tests/test_encoder.py", [diffPaths: ["system/loggerd/"]]),
step("test pigeond", "pytest system/ubloxd/tests/test_pigeond.py", [diffPaths: ["system/ubloxd/"]]),
step("test manager", "pytest system/manager/test/test_manager.py"),
])
},
'loopback': {
deviceStage("loopback", "tici-loopback", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("test pandad loopback", "pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
])
},
'camerad AR0231': {
deviceStage("AR0231", "tici-ar0231", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 60]),
step("test exposure", "pytest system/camerad/test/test_exposure.py"),
])
},
'camerad OX03C10': {
deviceStage("OX03C10", "tici-ox03c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 60]),
step("test exposure", "pytest system/camerad/test/test_exposure.py"),
])
},
'camerad OS04C10': {
deviceStage("OS04C10", "tici-os04c10", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test camerad", "pytest system/camerad/test/test_camerad.py", [timeout: 60]),
step("test exposure", "pytest system/camerad/test/test_exposure.py"),
])
},
'sensord': {
deviceStage("LSM + MMC", "tici-lsmc", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test sensord", "pytest system/sensord/tests/test_sensord.py"),
])
deviceStage("BMX + LSM", "tici-bmx-lsm", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py"),
step("test sensord", "pytest system/sensord/tests/test_sensord.py"),
])
},
'replay': {
deviceStage("model-replay", "tici-replay", ["UNSAFE=1"], [
step("build", "cd system/manager && ./build.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
step("model replay", "selfdrive/test/process_replay/model_replay.py", [diffPaths: ["selfdrive/modeld/", "tinygrad_repo", "selfdrive/test/process_replay/model_replay.py"]]),
])
},
'tizi': {
deviceStage("tizi", "tizi", ["UNSAFE=1"], [
step("build openpilot", "cd system/manager && ./build.py"),
step("test pandad loopback", "SINGLE_PANDA=1 pytest selfdrive/pandad/tests/test_pandad_loopback.py"),
step("test pandad spi", "pytest selfdrive/pandad/tests/test_pandad_spi.py"),
step("test pandad", "pytest selfdrive/pandad/tests/test_pandad.py", [diffPaths: ["panda", "selfdrive/pandad/"]]),
step("test amp", "pytest system/hardware/tici/tests/test_amplifier.py"),
// TODO: enable once new AGNOS is available
// step("test esim", "pytest system/hardware/tici/tests/test_esim.py"),
step("test qcomgpsd", "pytest system/qcomgpsd/tests/test_qcomgpsd.py", [diffPaths: ["system/qcomgpsd/"]]),
])
},
)
}
} catch (Exception e) {
currentBuild.result = 'FAILED'
throw e
}
}
+7
View File
@@ -0,0 +1,7 @@
Copyright (c) 2016, Comma.ai, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+9
View File
@@ -0,0 +1,9 @@
code_dir := $(shell pwd)
# TODO: Add a global build system
.PHONY: all
all:
cd selfdrive && PYTHONPATH=$(code_dir) PREPAREONLY=1 ./manager.py
+126 -35
View File
@@ -1,62 +1,153 @@
# OPGM: openpilot for Unsupported GM Cars
Welcome to openpilot
======
OPGM is a fork of [openpilot](https://github.com/commaai/openpilot/), the open source driver assistance system by comma.ai. This fork enables support for GM vehicles that are not officially supported upstream, while staying as close as possible to stock openpilot behavior.
[openpilot](http://github.com/commaai/openpilot) is an open source driving agent.
> OPGM strives to be **"stock openpilot for unsupported GM cars."**
Currently it performs the functions of Adaptive Cruise Control (ACC) and Lane Keeping Assist System (LKAS) for Hondas, Acuras and Toyotas. It's about on par with Tesla Autopilot at launch, and better than [all other manufacturers](http://www.thedrive.com/tech/5707/the-war-for-autonomous-driving-part-iii-us-vs-germany-vs-japan).
To achieve this, OPGM follows these development guidelines:
The openpilot codebase has been written to be concise and enable rapid prototyping. We look forward to your contributions - improving real vehicle automation has never been easier.
1. **Non-GM code is never modified** by OPGM. Shared/common code is modified only when absolutely necessary.
2. **Upstream-supported GM vehicles** are only patched to enable additional functionality (e.g., stop-and-go support, lower min engagement speed).
3. **Control schemes already supported by upstream openpilot** must not be modified. Alternate control schemes, such as `pedal long` or `CC long`, are exempt from this rule.
Here are [some](https://www.youtube.com/watch?v=9OwTJFuDI7g) [videos](https://www.youtube.com/watch?v=64Wvt5pYQmE) [of](https://www.youtube.com/watch?v=6IW7Nejsr3A) [it](https://www.youtube.com/watch?v=-VN1YcC83nA) [running](https://www.youtube.com/watch?v=EQJZvVeihZk). And a really cool [tutorial](https://www.youtube.com/watch?v=PwOnsT2UW5o).
For general openpilot documentation, setup, and developer guidance, refer to the [official openpilot repo](https://github.com/commaai/openpilot/).
Hardware
------
---
Right now openpilot supports the [NEO research platform](http://github.com/commaai/neo) and the [EON Dashcam DevKit](https://shop.comma.ai/products/eon-dashcam-devkit). We'd like to support other platforms as well.
## Features
Install openpilot on a neo device by entering ``https://openpilot.comma.ai`` during NEOS setup.
- Support for GM Global A vehicles with LKAS but no ACC
- `pedal long`: Longitudinal control using a pedal interceptor
- Enables full regenerative braking on Bolt EV/EUV (even those with ACC)
- `CC long`: Non-ACC cruise control adjustment, aka [“redneck ACC”](https://www.youtube.com/watch?v=41wZ1EAmf94)
- ⚠️ Cannot reduce speed below stock cruise control limits (usually 24mph)
Supported Cars
------
---
- Acura ILX 2016 with AcuraWatch Plus
- Due to use of the cruise control for gas, it can only be enabled above 25 mph
## Supported Vehicles
- Honda Civic 2016-2017 with Honda Sensing
- Due to limitations in steering firmware, steering is disabled below 12 mph
- Note that the hatchback model is not supported
Support is focused on GM Global A vehicles with factory LKAS. For the latest list, see [CARS.md](https://github.com/opgm/opendbc/blob/master/docs/CARS.md)
- Honda CR-V Touring 2015-2016
- Can only be enabled above 25 mph
If your vehicle isnt listed, but has LKAS and a front-facing camera, it might still be supported or supportable. See the “contributing” section below.
- Honda Odyssey 2018 with Honda Sensing (alpha!)
- Can only be enabled above 25 mph
---
- Acura RDX 2018 with AcuraWatch Plus (alpha!)
- Can only be enabled above 25 mph
## Installation
- Toyota RAV-4 2016+ non-hybrid with TSS-P
- By default it uses stock Toyota ACC for longitudinal control
- openpilot longitudinal control available after unplugging the [Driving Support ECU](https://community.comma.ai/wiki/index.php/Toyota#Rav4_.28for_openpilot.29) and can be enabled above 20 mph
Hardware installation is the same as for any vehicle with the standard GM harness in upstream openpilot.
- Toyota Prius 2017 (alpha!)
- By default it uses stock Toyota ACC for longitudinal control
- openpilot longitudinal control available after unplugging the [Driving Support ECU](https://community.comma.ai/wiki/index.php/Toyota#Prius_.28for_openpilot.29)
- Lateral control needs improvements
For software installation, type `opgm/release` into the custom software field, or `opgm/nightly` if you want to live on the bleeding edge.
- Toyota RAV-4 2017 hybrid (alpha!)
- By default it uses stock Toyota ACC for longitudinal control
- openpilot longitudinal control available after unplugging the [Driving Support ECU](https://community.comma.ai/wiki/index.php/Toyota#Rav4_.28for_openpilot.29) and can do stop and go
> **Note:** For EVs without ACC, a [pedal interceptor](https://www.etsy.com/listing/952895642/openpilot-comma-pedal-non-customizable?variation0=3013902165) is **strongly recommended** for a better experience.
- Toyota Corolla 2017 (alpha!)
- By default it uses stock Toyota ACC for longitudinal control
- openpilot longitudinal control available after unplugging the [Driving Support ECU](https://community.comma.ai/wiki/index.php/Toyota#Corolla_.28for_openpilot.29) and can be enabled above 20 mph
---
- Lexus RX 2017 hybrid (alpha!)
- By default it uses stock Lexus ACC for longitudinal control
- openpilot longitudinal control available after unplugging the [Driving Support ECU](https://community.comma.ai/wiki/index.php/Toyota#Lexus_RX_hybrid)
## Contributing
In Progress Cars
------
- All TSS-P Toyota with Steering Assist.
- 'Full Speed Range Dynamic Radar Cruise Control' is required to enable stop-and-go. Only the Prius, Camry and C-HR have this option.
- Even though the Tundra, Sequoia and the Land Cruiser have TSS-P, they don't have Steering Assist and are not supported.
- All LSS-P Lexus with Steering Assist or Lane Keep Assist.
- 'All-Speed Range Dynamic Radar Cruise Control' is required to enable stop-and-go. Only the GS, GSH, GS, F, RX, RXH, LX, NX, NXH, LC, LCH, LS, LSH have this option.
- Even though the LX have TSS-P, it does not have Steering Assist and is not supported.
All development occurs on the `master` branch. Contributions are welcome: feel free to open a PR.
Community WIP Cars
------
Bug reports and vehicle porting discussions happen in the OPGM channel on the [openpilot community Discord](https://discord.gg/KGWEdwSnCU).
- [Chevy Volt 2016-2018 Premier with Driver Confidence II](https://github.com/commaai/openpilot/pull/104)
---
- [Classic Tesla Model S (pre-AP)](https://github.com/commaai/openpilot/pull/145)
## Known Issues
- [Honda Pilot 2017 with Honda Sensing](https://github.com/commaai/openpilot/pull/161)
- Curve hugging and lane line drift are inherited from upstream openpilot.
- Upstream-supported GM cars (e.g. ACC Volt/Bolt) may fail to fingerprint automatically in OPGM.
Directory structure
------
---
- cereal -- The messaging spec used for all logs on the phone
- common -- Library like functionality we've developed here
- opendbc -- Files showing how to interpret data from cars
- panda -- Code used to communicate on CAN and LIN
- phonelibs -- Libraries used on the phone
- selfdrive -- Code needed to drive the car
- assets -- Fonts for ui
- boardd -- Daemon to talk to the board
- car -- Code that talks to the car and implements CarInterface
- common -- Shared C/C++ code for the daemons
- controls -- Python controls (PID loops etc) for the car
- debug -- Tools to help you debug and do car ports
- logcatd -- Android logcat as a service
- loggerd -- Logger and uploader of car data
- proclogd -- Logs information from proc
- sensord -- IMU / GPS interface code
- test/plant -- Car simulator running code through virtual maneuvers
- ui -- The UI
- visiond -- embedded vision pipeline
## License
To understand how the services interact, see `selfdrive/service_list.yaml`
Licensed under the MIT License. See [LICENSE](LICENSE) for details.
Testing on PC
------
There is rudimentary infrastructure to run a basic simulation and generate a report of openpilot's behavior in different scenarios.
```bash
# Requires working docker
./run_docker_tests.sh
```
The results are written to `selfdrive/test/plant/out/index.html`
More extensive testing infrastructure and simulation environments are coming soon.
Adding Car Support
------
comma.ai offers [bounties](http://comma.ai/bounties.html) for adding additional car support.
CR-V Touring support came in through this program. Chevy Volt is close. Accord is close as well.
User Data / chffr Account / Crash Reporting
------
By default openpilot creates an account and includes a client for chffr, our dashcam app. We use your data to train better models and improve openpilot for everyone.
It's open source software, so you are free to disable it if you wish.
It logs the road facing camera, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs.
It does not log the user facing camera or the microphone.
By using it, you agree to [our privacy policy](https://beta.comma.ai/privacy.html). You understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma.ai. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma.ai for the use of this data.
Contributing
------
We welcome both pull requests and issues on
[github](http://github.com/commaai/openpilot). See the TODO file for a list of
good places to start.
Want to get paid to work on openpilot? [comma.ai is hiring](http://comma.ai/positions.html)
Licensing
------
openpilot is released under the MIT license.
Any user of this software shall indemnify and hold harmless Comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys fees and costs) which arise out of, relate to or result from any use of this software by user.
**THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT.
YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS.
NO WARRANTY EXPRESSED OR IMPLIED.**
+51 -932
View File
File diff suppressed because it is too large Load Diff
+73
View File
@@ -0,0 +1,73 @@
openpilot Safety
======
openpilot is an Adaptive Cruise Control (ACC) and Lane Keeping Assist (LKA) system.
Like other ACC and LKA systems, openpilot requires the driver to be alert and to
pay attention at all times. We repeat, **driver alertness is necessary, but not
sufficient, for openpilot to be used safely**.
Even with an attentive driver, we must make further efforts for the system to be
safe. We have designed openpilot with two other safety considerations.
1. The driver must always be capable to immediately retake manual control of the vehicle,
by stepping on either pedal or by pressing the cancel button.
2. The vehicle must not alter its trajectory too quickly for the driver to safely
react. This means that while the system is engaged, the actuators are constrained
to operate within reasonable limits.
Following are details of the car specific safety implementations:
Honda/Acura
------
- While the system is engaged, gas, brake and steer limits are subject to the same limits used by
the stock system.
- Without an interceptor, the gas is controlled by the Powertrain Control Module (PCM).
The PCM limits acceleration to what is reasonable for a cruise control system. With an
interceptor, the gas is clipped to 60%.
- The brake is controlled by the 0x1FA CAN message. This message allows full
braking, although the board and the software clip it to 1/4th of the max.
This is around .3g of braking.
- Steering is controlled by the 0xE4 CAN message. The Electronic Power Steering (EPS)
controller in the car limits the torque to a very small amount, so regardless of the
message, the controller cannot jerk the wheel.
- Brake and gas pedal pressed signals are contained in the 0x17C CAN message. A rising edge of
either signal triggers a disengagement, which is enforced by the board and in software. The
green led on the board signifies if the board is allowing control messages.
- Honda CAN uses both a counter and a checksum to ensure integrity and prevent
replay of the same message.
Toyota/Lexus
------
- While the system is engaged, gas, brake and steer limits are subject to the same limits used by
the stock system.
- With the stock Driving Support Unit (DSU) enabled, the acceleration is controlled
by the stock system and is subject to the stock adaptive cruise control limits. Without the
stock DSU connected, the acceleration command is controlled by the 0x343 CAN message and its
value is limited by the board and the software to between .3g of deceleration and .15g of
acceleration. The acceleration command is ignored by the Engine Control Module (ECM) while the
cruise control system is disengaged.
- Steering torque is controlled through the 0x2E4 CAN message and it's limited by the board and in
software to a value of -1500 and 1500. In addition, the vehicle EPS unit will not respond to
commands outside these limits. A steering torque rate limit is enforced by the board and in
software so that the commanded steering torque must rise from 0 to max value no faster than
1.5s. Commanded steering torque is limited by the board and in software to be no more than 350
units above the actual EPS generated motor torque to ensure limited differences between
commanded and actual torques.
- Brake and gas pedal pressed signals are contained in the 0x224 and 0x1D2 CAN messages,
respectively. A rising edge of either signal triggers a disengagement, which is enforced by the
board and in software. Additionally, the cruise control system disengages on the rising edge of
the brake pedal pressed signal.
- The cruise control system state is contained in the 0x1D2 message. No control messages are
allowed if the cruise control system is not active. This is enforced by the software and the
board. The green led on the board signifies if the board is allowing control messages.
-370
View File
@@ -1,370 +0,0 @@
import os
import subprocess
import sys
import sysconfig
import platform
import numpy as np
import SCons.Errors
SCons.Warnings.warningAsException(True)
# pending upstream fix - https://github.com/SCons/scons/issues/4461
#SetOption('warn', 'all')
TICI = os.path.isfile('/TICI')
AGNOS = TICI
Decider('MD5-timestamp')
SetOption('num_jobs', int(os.cpu_count()/2))
AddOption('--kaitai',
action='store_true',
help='Regenerate kaitai struct parsers')
AddOption('--asan',
action='store_true',
help='turn on ASAN')
AddOption('--ubsan',
action='store_true',
help='turn on UBSan')
AddOption('--coverage',
action='store_true',
help='build with test coverage options')
AddOption('--clazy',
action='store_true',
help='build with clazy')
AddOption('--compile_db',
action='store_true',
help='build clang compilation database')
AddOption('--ccflags',
action='store',
type='string',
default='',
help='pass arbitrary flags over the command line')
AddOption('--external-sconscript',
action='store',
metavar='FILE',
dest='external_sconscript',
help='add an external SConscript to the build')
AddOption('--mutation',
action='store_true',
help='generate mutation-ready code')
AddOption('--minimal',
action='store_false',
dest='extras',
default=os.path.exists(File('#.lfsconfig').abspath), # minimal by default on release branch (where there's no LFS)
help='the minimum build to run openpilot. no tests, tools, etc.')
## Architecture name breakdown (arch)
## - larch64: linux tici aarch64
## - aarch64: linux pc aarch64
## - x86_64: linux pc x64
## - Darwin: mac x64 or arm64
real_arch = arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
if platform.system() == "Darwin":
arch = "Darwin"
brew_prefix = subprocess.check_output(['brew', '--prefix'], encoding='utf8').strip()
elif arch == "aarch64" and AGNOS:
arch = "larch64"
assert arch in ["larch64", "aarch64", "x86_64", "Darwin"]
lenv = {
"PATH": os.environ['PATH'],
"LD_LIBRARY_PATH": [Dir(f"#third_party/acados/{arch}/lib").abspath],
"PYTHONPATH": Dir("#").abspath + ':' + Dir(f"#third_party/acados").abspath,
"ACADOS_SOURCE_DIR": Dir("#third_party/acados").abspath,
"ACADOS_PYTHON_INTERFACE_PATH": Dir("#third_party/acados/acados_template").abspath,
"TERA_PATH": Dir("#").abspath + f"/third_party/acados/{arch}/t_renderer"
}
rpath = lenv["LD_LIBRARY_PATH"].copy()
if arch == "larch64":
cpppath = [
"#third_party/opencl/include",
]
libpath = [
"/usr/local/lib",
"/system/vendor/lib64",
f"#third_party/acados/{arch}/lib",
]
libpath += [
"#third_party/libyuv/larch64/lib",
"/usr/lib/aarch64-linux-gnu"
]
cflags = ["-DQCOM2", "-mcpu=cortex-a57"]
cxxflags = ["-DQCOM2", "-mcpu=cortex-a57"]
rpath += ["/usr/local/lib"]
else:
cflags = []
cxxflags = []
cpppath = []
rpath += []
# MacOS
if arch == "Darwin":
libpath = [
f"#third_party/libyuv/{arch}/lib",
f"#third_party/acados/{arch}/lib",
f"{brew_prefix}/lib",
f"{brew_prefix}/opt/openssl@3.0/lib",
"/System/Library/Frameworks/OpenGL.framework/Libraries",
]
cflags += ["-DGL_SILENCE_DEPRECATION"]
cxxflags += ["-DGL_SILENCE_DEPRECATION"]
cpppath += [
f"{brew_prefix}/include",
f"{brew_prefix}/opt/openssl@3.0/include",
]
lenv["DYLD_LIBRARY_PATH"] = lenv["LD_LIBRARY_PATH"]
# Linux
else:
libpath = [
f"#third_party/acados/{arch}/lib",
f"#third_party/libyuv/{arch}/lib",
"/usr/lib",
"/usr/local/lib",
]
if GetOption('asan'):
ccflags = ["-fsanitize=address", "-fno-omit-frame-pointer"]
ldflags = ["-fsanitize=address"]
elif GetOption('ubsan'):
ccflags = ["-fsanitize=undefined"]
ldflags = ["-fsanitize=undefined"]
else:
ccflags = []
ldflags = []
# no --as-needed on mac linker
if arch != "Darwin":
ldflags += ["-Wl,--as-needed", "-Wl,--no-undefined"]
ccflags_option = GetOption('ccflags')
if ccflags_option:
ccflags += ccflags_option.split(' ')
env = Environment(
ENV=lenv,
CCFLAGS=[
"-g",
"-fPIC",
"-O2",
"-Wunused",
"-Werror",
"-Wshadow",
"-Wno-unknown-warning-option",
"-Wno-inconsistent-missing-override",
"-Wno-c99-designator",
"-Wno-reorder-init-list",
"-Wno-vla-cxx-extension",
] + cflags + ccflags,
CPPPATH=cpppath + [
"#",
"#third_party/acados/include",
"#third_party/acados/include/blasfeo/include",
"#third_party/acados/include/hpipm/include",
"#third_party/catch2/include",
"#third_party/libyuv/include",
"#third_party/json11",
"#third_party/linux/include",
"#third_party",
"#msgq",
],
CC='clang',
CXX='clang++',
LINKFLAGS=ldflags,
RPATH=rpath,
CFLAGS=["-std=gnu11"] + cflags,
CXXFLAGS=["-std=c++1z"] + cxxflags,
LIBPATH=libpath + [
"#msgq_repo",
"#third_party",
"#selfdrive/pandad",
"#common",
"#rednose/helpers",
],
CYTHONCFILESUFFIX=".cpp",
COMPILATIONDB_USE_ABSPATH=True,
REDNOSE_ROOT="#",
tools=["default", "cython", "compilation_db", "rednose_filter"],
toolpath=["#site_scons/site_tools", "#rednose_repo/site_scons/site_tools"],
)
if arch == "Darwin":
# RPATH is not supported on macOS, instead use the linker flags
darwin_rpath_link_flags = [f"-Wl,-rpath,{path}" for path in env["RPATH"]]
env["LINKFLAGS"] += darwin_rpath_link_flags
if GetOption('compile_db'):
env.CompilationDatabase('compile_commands.json')
# Setup cache dir
cache_dir = '/data/scons_cache' if AGNOS else '/tmp/scons_cache'
CacheDir(cache_dir)
Clean(["."], cache_dir)
node_interval = 5
node_count = 0
def progress_function(node):
global node_count
node_count += node_interval
sys.stderr.write("progress: %d\n" % node_count)
if os.environ.get('SCONS_PROGRESS'):
Progress(progress_function, interval=node_interval)
# Cython build environment
py_include = sysconfig.get_paths()['include']
envCython = env.Clone()
envCython["CPPPATH"] += [py_include, np.get_include()]
envCython["CCFLAGS"] += ["-Wno-#warnings", "-Wno-shadow", "-Wno-deprecated-declarations"]
envCython["CCFLAGS"].remove("-Werror")
envCython["LIBS"] = []
if arch == "Darwin":
envCython["LINKFLAGS"] = ["-bundle", "-undefined", "dynamic_lookup"] + darwin_rpath_link_flags
else:
envCython["LINKFLAGS"] = ["-pthread", "-shared"]
np_version = SCons.Script.Value(np.__version__)
Export('envCython', 'np_version')
# Qt build environment
qt_env = env.Clone()
qt_modules = ["Widgets", "Gui", "Core", "Network", "Concurrent", "DBus", "Xml"]
qt_libs = []
if arch == "Darwin":
qt_env['QTDIR'] = f"{brew_prefix}/opt/qt@5"
qt_dirs = [
os.path.join(qt_env['QTDIR'], "include"),
]
qt_dirs += [f"{qt_env['QTDIR']}/include/Qt{m}" for m in qt_modules]
qt_env["LINKFLAGS"] += ["-F" + os.path.join(qt_env['QTDIR'], "lib")]
qt_env["FRAMEWORKS"] += [f"Qt{m}" for m in qt_modules] + ["OpenGL"]
qt_env.AppendENVPath('PATH', os.path.join(qt_env['QTDIR'], "bin"))
else:
qt_install_prefix = subprocess.check_output(['qmake', '-query', 'QT_INSTALL_PREFIX'], encoding='utf8').strip()
qt_install_headers = subprocess.check_output(['qmake', '-query', 'QT_INSTALL_HEADERS'], encoding='utf8').strip()
qt_env['QTDIR'] = qt_install_prefix
qt_dirs = [
f"{qt_install_headers}",
]
qt_gui_path = os.path.join(qt_install_headers, "QtGui")
qt_gui_dirs = [d for d in os.listdir(qt_gui_path) if os.path.isdir(os.path.join(qt_gui_path, d))]
qt_dirs += [f"{qt_install_headers}/QtGui/{qt_gui_dirs[0]}/QtGui", ] if qt_gui_dirs else []
qt_dirs += [f"{qt_install_headers}/Qt{m}" for m in qt_modules]
qt_libs = [f"Qt5{m}" for m in qt_modules]
if arch == "larch64":
qt_libs += ["GLESv2", "wayland-client"]
qt_env.PrependENVPath('PATH', Dir("#third_party/qt5/larch64/bin/").abspath)
elif arch != "Darwin":
qt_libs += ["GL"]
qt_env['QT3DIR'] = qt_env['QTDIR']
qt_env.Tool('qt3')
qt_env['CPPPATH'] += qt_dirs + ["#third_party/qrcode"]
qt_flags = [
"-D_REENTRANT",
"-DQT_NO_DEBUG",
"-DQT_WIDGETS_LIB",
"-DQT_GUI_LIB",
"-DQT_CORE_LIB",
"-DQT_MESSAGELOGCONTEXT",
]
qt_env['CXXFLAGS'] += qt_flags
qt_env['LIBPATH'] += ['#selfdrive/ui', ]
qt_env['LIBS'] = qt_libs
if GetOption("clazy"):
checks = [
"level0",
"level1",
"no-range-loop",
"no-non-pod-global-static",
]
qt_env['CXX'] = 'clazy'
qt_env['ENV']['CLAZY_IGNORE_DIRS'] = qt_dirs[0]
qt_env['ENV']['CLAZY_CHECKS'] = ','.join(checks)
Export('env', 'qt_env', 'arch', 'real_arch')
# Build common module
SConscript(['common/SConscript'])
Import('_common', '_gpucommon')
common = [_common, 'json11', 'zmq']
gpucommon = [_gpucommon]
Export('common', 'gpucommon')
# Build messaging (cereal + msgq + socketmaster + their dependencies)
# Enable swaglog include in submodules
env_swaglog = env.Clone()
env_swaglog['CXXFLAGS'].append('-DSWAGLOG="\\"common/swaglog.h\\""')
SConscript(['msgq_repo/SConscript'], exports={'env': env_swaglog})
SConscript(['opendbc_repo/SConscript'], exports={'env': env_swaglog})
SConscript(['cereal/SConscript'])
Import('socketmaster', 'msgq')
messaging = [socketmaster, msgq, 'capnp', 'kj',]
Export('messaging')
# Build other submodules
SConscript(['panda/SConscript'])
# Build rednose library
SConscript(['rednose/SConscript'])
# Build system services
SConscript([
'system/proclogd/SConscript',
'system/ubloxd/SConscript',
'system/loggerd/SConscript',
])
if arch != "Darwin":
SConscript([
'system/sensord/SConscript',
'system/logcatd/SConscript',
])
if arch == "larch64":
SConscript(['system/camerad/SConscript'])
# Build openpilot
SConscript(['third_party/SConscript'])
SConscript(['selfdrive/SConscript'])
if Dir('#tools/cabana/').exists() and GetOption('extras'):
SConscript(['tools/replay/SConscript'])
if arch != "larch64":
SConscript(['tools/cabana/SConscript'])
external_sconscript = GetOption('external_sconscript')
if external_sconscript:
SConscript([external_sconscript])
-5
View File
@@ -1,5 +0,0 @@
# Security Policy
## Reporting a Vulnerability
Suspected vulnerabilities can be reported to both `adeeb@comma.ai` and `security@comma.ai`.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1 +1,2 @@
src/*
out/*
Binary file not shown.
BIN
View File
Binary file not shown.
Vendored Executable
+122
View File
@@ -0,0 +1,122 @@
#!/usr/bin/env python2.7
import os
import sys
import glob
import shutil
import urllib2
import hashlib
import subprocess
EXTERNAL_PATH = os.path.dirname(os.path.abspath(__file__))
if os.path.exists("/init.qcom.rc"):
# android
APKPATCH = os.path.join(EXTERNAL_PATH, 'tools/apkpatch_android')
SIGNAPK = os.path.join(EXTERNAL_PATH, 'tools/signapk_android')
else:
APKPATCH = os.path.join(EXTERNAL_PATH, 'tools/apkpatch')
SIGNAPK = os.path.join(EXTERNAL_PATH, 'tools/signapk')
APKS = {
'com.waze': {
'src': 'https://apkcache.s3.amazonaws.com/com.waze_1021278.apk',
'src_sha256': 'f00957e93e2389f9e30502ac54994b98ac769314b0963c263d4e8baa625ab0c2',
'patch': 'com.waze.apkpatch',
'out_sha256': '9ec8b0ea3c78c666342865b1bfb66e368a3f5c911df2ad12835206ec8b19f444'
},
'com.spotify.music': {
'src': 'https://apkcache.s3.amazonaws.com/com.spotify.music_24382006.apk',
'src_sha256': '0610fea68ee7ba5f8e4e0732ad429d729dd6cbb8bc21222c4c99db6cb09fbff4',
'patch': 'com.spotify.music.apkpatch',
'out_sha256': '5a3d6f478c7e40403a98ccc8906d7e0ae12b06543b41f5df52149dd09c647c11'
},
}
def sha256_path(path):
with open(path, 'rb') as f:
return hashlib.sha256(f.read()).hexdigest()
def remove(path):
try:
os.remove(path)
except OSError:
pass
def process(download, patch):
# clean up any junk apks
for out_apk in glob.glob(os.path.join(EXTERNAL_PATH, 'out/*.apk')):
app = os.path.basename(out_apk)[:-4]
if app not in APKS:
print "remove junk", out_apk
remove(out_apk)
complete = True
for k,v in APKS.iteritems():
apk_path = os.path.join(EXTERNAL_PATH, 'out', k+'.apk')
print "checking", apk_path
if os.path.exists(apk_path) and sha256_path(apk_path) == v['out_sha256']:
# nothing to do
continue
complete = False
remove(apk_path)
src_path = os.path.join(EXTERNAL_PATH, 'src', v['src_sha256'])
if not os.path.exists(src_path) or sha256_path(src_path) != v['src_sha256']:
if not download:
continue
print "downloading", v['src'], "to", src_path
# download it
resp = urllib2.urlopen(v['src'])
data = resp.read()
with open(src_path, 'wb') as src_f:
src_f.write(data)
if sha256_path(src_path) != v['src_sha256']:
print "download was corrupted..."
continue
if not patch:
continue
# ignoring lots of TOCTTOU here...
apk_temp = "/tmp/"+k+".patched"
remove(apk_temp)
apk_temp2 = "/tmp/"+k+".signed"
remove(apk_temp2)
try:
print "patching", v['patch']
subprocess.check_call([APKPATCH, 'apply', src_path, apk_temp, os.path.join(EXTERNAL_PATH, v['patch'])])
print "signing", apk_temp
subprocess.check_call([SIGNAPK,
os.path.join(EXTERNAL_PATH, 'tools/certificate.pem'), os.path.join(EXTERNAL_PATH, 'tools/key.pk8'),
apk_temp, apk_temp2])
out_sha256 = sha256_path(apk_temp2) if os.path.exists(apk_temp2) else None
if out_sha256 == v['out_sha256']:
print "done", apk_path
shutil.move(apk_temp2, apk_path)
else:
print "patch was corrupted", apk_temp2, out_sha256
finally:
remove(apk_temp)
remove(apk_temp2)
return complete
if __name__ == "__main__":
ret = True
if len(sys.argv) == 2 and sys.argv[1] == "download":
ret = process(True, False)
elif len(sys.argv) == 2 and sys.argv[1] == "patch":
ret = process(False, True)
else:
ret = process(True, True)
sys.exit(0 if ret else 1)
Binary file not shown.
+7
View File
@@ -0,0 +1,7 @@
#!/system/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
export LD_LIBRARY_PATH=/system/lib64
export CLASSPATH="$DIR"/ApkPatch.android.jar
exec app_process "$DIR" ApkPatch "$@"
+17
View File
@@ -0,0 +1,17 @@
-----BEGIN CERTIFICATE-----
MIICtTCCAh4CCQDm79UqF+Dc5zANBgkqhkiG9w0BAQUFADCBnjELMAkGA1UEBhMC
SUQxEzARBgNVBAgTCkphd2EgQmFyYXQxEDAOBgNVBAcTB0JhbmR1bmcxEjAQBgNV
BAoTCUxvbmRhdGlnYTETMBEGA1UECxMKQW5kcm9pZERldjEaMBgGA1UEAxMRTG9y
ZW5zaXVzIFcuIEwuIFQxIzAhBgkqhkiG9w0BCQEWFGxvcmVuekBsb25kYXRpZ2Eu
bmV0MB4XDTEwMDUwNTA5MjEzOFoXDTEzMDEyODA5MjEzOFowgZ4xCzAJBgNVBAYT
AklEMRMwEQYDVQQIEwpKYXdhIEJhcmF0MRAwDgYDVQQHEwdCYW5kdW5nMRIwEAYD
VQQKEwlMb25kYXRpZ2ExEzARBgNVBAsTCkFuZHJvaWREZXYxGjAYBgNVBAMTEUxv
cmVuc2l1cyBXLiBMLiBUMSMwIQYJKoZIhvcNAQkBFhRsb3JlbnpAbG9uZGF0aWdh
Lm5ldDCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAy2oWtbdVXMHGiS6cA3qi
3VfZt5Vz9jTlux+TEcGx5h18ZKwclyo+z2B0L/p5bYdnrTdFEiD7IxvX+h3lu0JV
B9rdXZdyrzXNOw5YFrsn2k7hKvB8KEBaga1gZEwodlc6N14H3FbZdZkIA9V716Pu
e5CWBZ2VqU03lUJmKnpH8c8CAwEAATANBgkqhkiG9w0BAQUFAAOBgQBpNgXh8dw9
uMjZxzLUXovV5ptHd61jAcZlQlffqPsz6/2QNfIShVdGH9jkm0IudfKkbvvOKive
a77t9c4sDh2Sat2L/rx6BfTuS1+y9wFr1Ee8Rrr7wGHhRkx2qqGrXGVWqXn8aE3E
P6e7BTPF0ibS+tG8cdDPEisqGFxw36nTNQ==
-----END CERTIFICATE-----
Vendored Executable
BIN
View File
Binary file not shown.
Binary file not shown.
+7
View File
@@ -0,0 +1,7 @@
#!/system/bin/sh
DIR="$(cd "$(dirname "$0")" && pwd)"
export LD_LIBRARY_PATH=/system/lib64
export CLASSPATH="$DIR"/signapk.android.jar
exec app_process "$DIR" com.android.signapk.SignApk "$@"
+1
View File
@@ -0,0 +1 @@
gen
+52
View File
@@ -0,0 +1,52 @@
PWD := $(shell pwd)
SRCS := log.capnp car.capnp
GENS := gen/cpp/car.capnp.c++ gen/cpp/log.capnp.c++
UNAME_M ?= $(shell uname -m)
# only generate C++ for docker tests
ifneq ($(OPTEST),1)
GENS += gen/c/car.capnp.c gen/c/log.capnp.c gen/c/c++.capnp.h gen/c/java.capnp.h
# Dont build java on the phone...
ifeq ($(UNAME_M),x86_64)
GENS += gen/java/Car.java gen/java/Log.java
endif
endif
ifeq ($(UNAME_M),aarch64)
CAPNPC=PATH=$(PWD)/../phonelibs/capnp-cpp/aarch64/bin/:$$PATH capnpc
else
CAPNPC=capnpc
endif
.PHONY: all
all: $(GENS)
.PHONY: clean
clean:
rm -rf gen
gen/c/%.capnp.c: %.capnp
@echo "[ CAPNPC C ] $@"
mkdir -p gen/c/
$(CAPNPC) '$<' -o c:gen/c/
gen/cpp/%.capnp.c++: %.capnp
@echo "[ CAPNPC C++ ] $@"
mkdir -p gen/cpp/
$(CAPNPC) '$<' -o c++:gen/cpp/
gen/java/Car.java gen/java/Log.java: $(SRCS)
@echo "[ CAPNPC java ] $@"
mkdir -p gen/java/
$(CAPNPC) $^ -o java:gen/java
# c-capnproto needs some empty headers
gen/c/c++.capnp.h gen/c/java.capnp.h:
mkdir -p gen/c/
touch '$@'
-95
View File
@@ -1,95 +0,0 @@
# What is cereal?
cereal is the messaging system for openpilot. It uses [msgq](https://github.com/commaai/msgq) as a pub/sub backend, and [Cap'n proto](https://capnproto.org/capnp-tool.html) for serialization of the structs.
## Messaging Spec
You'll find the message types in [log.capnp](log.capnp). It uses [Cap'n proto](https://capnproto.org/capnp-tool.html) and defines one struct called `Event`.
All `Events` have a `logMonoTime` and a `valid`. Then a big union defines the packet type.
### Best Practices
- **All fields must describe quantities in SI units**, unless otherwise specified in the field name.
- In the context of the message they are in, field names should be completely unambiguous.
- All values should be easy to plot and be human-readable with minimal parsing.
### Maintaining backwards-compatibility
When making changes to the messaging spec you want to maintain backwards-compatibility, such that old logs can
be parsed with a new version of cereal. Adding structs and adding members to structs is generally safe, most other
things are not. Read more details [here](https://capnproto.org/language.html).
### Custom forks
Forks of [openpilot](https://github.com/commaai/openpilot) might want to add things to the messaging
spec, however this could conflict with future changes made in mainline cereal/openpilot. Rebasing against mainline openpilot
then means breaking backwards-compatibility with all old logs of your fork. So we added reserved events in
[custom.capnp](custom.capnp) that we will leave empty in mainline cereal/openpilot. **If you only modify those, you can ensure your
fork will remain backwards-compatible with all versions of mainline openpilot and your fork.**
An example of compatible changes:
```diff
diff --git a/cereal/custom.capnp b/cereal/custom.capnp
index 3348e859e..3365c7b98 100644
--- a/cereal/custom.capnp
+++ b/cereal/custom.capnp
@@ -10,7 +10,11 @@ $Cxx.namespace("cereal");
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
-struct CustomReserved0 @0x81c2f05a394cf4af {
+struct SteeringInfo @0x81c2f05a394cf4af {
+ active @0 :Bool;
+ steeringAngleDeg @1 :Float32;
+ steeringRateDeg @2 :Float32;
+ steeringAccelDeg @3 :Float32;
}
struct CustomReserved1 @0xaedffd8f31e7b55d {
diff --git a/cereal/log.capnp b/cereal/log.capnp
index 1209f3fd9..b189f58b6 100644
--- a/cereal/log.capnp
+++ b/cereal/log.capnp
@@ -2558,14 +2558,14 @@ struct Event {
# DO change the name of the field
# DON'T change anything after the "@"
- customReservedRawData0 @124 :Data;
+ rawCanData @124 :Data;
customReservedRawData1 @125 :Data;
customReservedRawData2 @126 :Data;
# DO change the name of the field and struct
# DON'T change the ID (e.g. @107)
# DON'T change which struct it points to
- customReserved0 @107 :Custom.CustomReserved0;
+ steeringInfo @107 :Custom.SteeringInfo;
customReserved1 @108 :Custom.CustomReserved1;
customReserved2 @109 :Custom.CustomReserved2;
customReserved3 @110 :Custom.CustomReserved3;
```
---
Example
---
```python
import cereal.messaging as messaging
# in subscriber
sm = messaging.SubMaster(['sensorEvents'])
while 1:
sm.update()
print(sm['sensorEvents'])
```
```python
# in publisher
pm = messaging.PubMaster(['sensorEvents'])
dat = messaging.new_message('sensorEvents', size=1)
dat.sensorEvents[0] = {"gyro": {"v": [0.1, -0.1, 0.1]}}
pm.send('sensorEvents', dat)
```
-20
View File
@@ -1,20 +0,0 @@
Import('env', 'common', 'msgq')
cereal_dir = Dir('.')
gen_dir = Dir('gen')
# Build cereal
schema_files = ['log.capnp', 'car.capnp', 'legacy.capnp', 'custom.capnp']
env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files],
schema_files,
f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/")
cereal = env.Library('cereal', [f'gen/cpp/{s}.c++' for s in schema_files])
# Build messaging
services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET')
env.Program('messaging/bridge', ['messaging/bridge.cc', 'messaging/msgq_to_zmq.cc'], LIBS=[msgq, common, 'pthread'])
socketmaster = env.Library('socketmaster', ['messaging/socketmaster.cc'])
Export('cereal', 'socketmaster')
+13 -4
View File
@@ -1,11 +1,20 @@
import os
import capnp
from importlib.resources import as_file, files
CEREAL_PATH = os.path.dirname(os.path.abspath(__file__))
capnp.remove_import_hook()
with as_file(files("cereal")) as fspath:
CEREAL_PATH = fspath.as_posix()
if os.getenv("NEWCAPNP"):
import tempfile
import pyximport
importers = pyximport.install(build_dir=os.path.join(tempfile.gettempdir(), ".pyxbld"))
try:
import cereal.gen.cython.log_capnp_cython as log
import cereal.gen.cython.car_capnp_cython as car
finally:
pyximport.uninstall(*importers)
del importers
else:
log = capnp.load(os.path.join(CEREAL_PATH, "log.capnp"))
car = capnp.load(os.path.join(CEREAL_PATH, "car.capnp"))
custom = capnp.load(os.path.join(CEREAL_PATH, "custom.capnp"))
-1
View File
@@ -1 +0,0 @@
../opendbc_repo/opendbc/car/car.capnp
+321
View File
@@ -0,0 +1,321 @@
using Cxx = import "c++.capnp";
$Cxx.namespace("cereal");
using Java = import "java.capnp";
$Java.package("ai.comma.openpilot.cereal");
$Java.outerClassname("Car");
@0x8e2af1e708af8b8d;
# ******* events causing controls state machine transition *******
struct CarEvent @0x9b1657f34caf3ad3 {
name @0 :EventName;
enable @1 :Bool;
noEntry @2 :Bool;
warning @3 :Bool;
userDisable @4 :Bool;
softDisable @5 :Bool;
immediateDisable @6 :Bool;
preEnable @7 :Bool;
permanent @8 :Bool;
enum EventName @0xbaa8c5d505f727de {
# TODO: copy from error list
commIssue @0;
steerUnavailable @1;
brakeUnavailable @2;
gasUnavailable @3;
wrongGear @4;
doorOpen @5;
seatbeltNotLatched @6;
espDisabled @7;
wrongCarMode @8;
steerTempUnavailable @9;
reverseGear @10;
buttonCancel @11;
buttonEnable @12;
pedalPressed @13;
cruiseDisabled @14;
radarCommIssue @15;
dataNeeded @16;
speedTooLow @17;
outOfSpace @18;
overheat @19;
calibrationInProgress @20;
calibrationInvalid @21;
controlsMismatch @22;
pcmEnable @23;
pcmDisable @24;
noTarget @25;
radarFault @26;
modelCommIssue @27;
brakeHold @28;
parkBrake @29;
manualRestart @30;
lowSpeedLockout @31;
}
}
# ******* main car state @ 100hz *******
# all speeds in m/s
struct CarState {
errorsDEPRECATED @0 :List(CarEvent.EventName);
events @13 :List(CarEvent);
# car speed
vEgo @1 :Float32; # best estimate of speed
aEgo @16 :Float32; # best estimate of acceleration
vEgoRaw @17 :Float32; # unfiltered speed from CAN sensors
yawRate @22 :Float32; # best estimate of yaw rate
standstill @18 :Bool;
wheelSpeeds @2 :WheelSpeeds;
# gas pedal, 0.0-1.0
gas @3 :Float32; # this is user + computer
gasPressed @4 :Bool; # this is user pedal only
# brake pedal, 0.0-1.0
brake @5 :Float32; # this is user pedal only
brakePressed @6 :Bool; # this is user pedal only
brakeLights @19 :Bool;
# steering wheel
steeringAngle @7 :Float32; # deg
steeringRate @15 :Float32; # deg/s
steeringTorque @8 :Float32; # TODO: standardize units
steeringPressed @9 :Bool; # if the user is using the steering wheel
# cruise state
cruiseState @10 :CruiseState;
# gear
gearShifter @14 :GearShifter;
# button presses
buttonEvents @11 :List(ButtonEvent);
leftBlinker @20 :Bool;
rightBlinker @21 :Bool;
genericToggle @23 :Bool;
# lock info
doorOpen @24 :Bool;
seatbeltUnlatched @25 :Bool;
# which packets this state came from
canMonoTimes @12: List(UInt64);
struct WheelSpeeds {
# optional wheel speeds
fl @0 :Float32;
fr @1 :Float32;
rl @2 :Float32;
rr @3 :Float32;
}
struct CruiseState {
enabled @0 :Bool;
speed @1 :Float32;
available @2 :Bool;
speedOffset @3 :Float32;
standstill @4 :Bool;
}
enum GearShifter {
unknown @0;
park @1;
drive @2;
neutral @3;
reverse @4;
sport @5;
low @6;
brake @7;
}
# send on change
struct ButtonEvent {
pressed @0 :Bool;
type @1 :Type;
enum Type {
unknown @0;
leftBlinker @1;
rightBlinker @2;
accelCruise @3;
decelCruise @4;
cancel @5;
altButton1 @6;
altButton2 @7;
altButton3 @8;
}
}
}
# ******* radar state @ 20hz *******
struct RadarState {
errors @0 :List(Error);
points @1 :List(RadarPoint);
# which packets this state came from
canMonoTimes @2 :List(UInt64);
enum Error {
commIssue @0;
fault @1;
}
# similar to LiveTracks
# is one timestamp valid for all? I think so
struct RadarPoint {
trackId @0 :UInt64; # no trackId reuse
# these 3 are the minimum required
dRel @1 :Float32; # m from the front bumper of the car
yRel @2 :Float32; # m
vRel @3 :Float32; # m/s
# these are optional and valid if they are not NaN
aRel @4 :Float32; # m/s^2
yvRel @5 :Float32; # m/s
# some radars flag measurements VS estimates
measured @6 :Bool;
}
}
# ******* car controls @ 100hz *******
struct CarControl {
# must be true for any actuator commands to work
enabled @0 :Bool;
gasDEPRECATED @1 :Float32;
brakeDEPRECATED @2 :Float32;
steeringTorqueDEPRECATED @3 :Float32;
actuators @6 :Actuators;
cruiseControl @4 :CruiseControl;
hudControl @5 :HUDControl;
struct Actuators {
# range from 0.0 - 1.0
gas @0: Float32;
brake @1: Float32;
# range from -1.0 - 1.0
steer @2: Float32;
}
struct CruiseControl {
cancel @0: Bool;
override @1: Bool;
speedOverride @2: Float32;
accelOverride @3: Float32;
}
struct HUDControl {
speedVisible @0: Bool;
setSpeed @1: Float32;
lanesVisible @2: Bool;
leadVisible @3: Bool;
visualAlert @4: VisualAlert;
audibleAlert @5: AudibleAlert;
enum VisualAlert {
# these are the choices from the Honda
# map as good as you can for your car
none @0;
fcw @1;
steerRequired @2;
brakePressed @3;
wrongGear @4;
seatbeltUnbuckled @5;
speedTooHigh @6;
}
enum AudibleAlert {
# these are the choices from the Honda
# map as good as you can for your car
none @0;
beepSingle @1;
beepTriple @2;
beepRepeated @3;
chimeSingle @4;
chimeDouble @5;
chimeRepeated @6;
chimeContinuous @7;
}
}
}
# ****** car param ******
struct CarParams {
carName @0 :Text;
radarNameDEPRECATED @1 :Text;
carFingerprint @2 :Text;
enableSteer @3 :Bool;
enableGas @4 :Bool;
enableBrake @5 :Bool;
enableCruise @6 :Bool;
enableCamera @26 :Bool;
enableDsu @27 :Bool; # driving support unit
enableApgs @28 :Bool; # advanced parking guidance system
minEnableSpeed @17 :Float32;
safetyModel @18 :Int16;
safetyParam @41 :Int16;
steerMaxBP @19 :List(Float32);
steerMaxV @20 :List(Float32);
gasMaxBP @21 :List(Float32);
gasMaxV @22 :List(Float32);
brakeMaxBP @23 :List(Float32);
brakeMaxV @24 :List(Float32);
longPidDeadzoneBP @32 :List(Float32);
longPidDeadzoneV @33 :List(Float32);
enum SafetyModels {
# does NOT match board setting
noOutput @0;
honda @1;
toyota @2;
elm327 @3;
}
# things about the car in the manual
mass @7 :Float32; # [kg] running weight
wheelbase @8 :Float32; # [m] distance from rear to front axle
centerToFront @9 :Float32; # [m] GC distance to front axle
steerRatio @10 :Float32; # [] ratio between front wheels and steering wheel angles
steerRatioRear @11 :Float32; # [] rear steering ratio wrt front steering (usually 0)
# things we can derive
rotationalInertia @12 :Float32; # [kg*m2] body rotational inertia
tireStiffnessFront @13 :Float32; # [N/rad] front tire coeff of stiff
tireStiffnessRear @14 :Float32; # [N/rad] rear tire coeff of stiff
# Kp and Ki for the lateral control
steerKp @15 :Float32;
steerKi @16 :Float32;
steerKf @25 :Float32;
# Kp and Ki for the longitudinal control
longitudinalKpBP @36 :List(Float32);
longitudinalKpV @37 :List(Float32);
longitudinalKiBP @38 :List(Float32);
longitudinalKiV @39 :List(Float32);
steerLimitAlert @29 :Bool;
vEgoStopping @30 :Float32; # Speed at which the car goes into stopping state
directAccelControl @31 :Bool; # Does the car have direct accel control or just gas/brake
stoppingControl @34 :Bool; # Does the car allows full control even at lows speeds when stopping
startAccel @35 :Float32; # Required acceleraton to overcome creep braking
steerRateCost @40 :Float32; # Lateral MPC cost on steering rate
}
-71
View File
@@ -1,71 +0,0 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
@0xb526ba661d550a59;
# custom.capnp: a home for empty structs reserved for custom forks
# These structs are guaranteed to remain reserved and empty in mainline
# cereal, so use these if you want custom events in your fork.
# DO rename the structs
# DON'T change the identifier (e.g. @0x81c2f05a394cf4af)
struct CustomReserved0 @0x81c2f05a394cf4af {
}
struct CustomReserved1 @0xaedffd8f31e7b55d {
}
struct CustomReserved2 @0xf35cc4560bbf6ec2 {
}
struct CustomReserved3 @0xda96579883444c35 {
}
struct CustomReserved4 @0x80ae746ee2596b11 {
}
struct CustomReserved5 @0xa5cd762cd951a455 {
}
struct CustomReserved6 @0xf98d843bfd7004a3 {
}
struct CustomReserved7 @0xb86e6369214c01c8 {
}
struct CustomReserved8 @0xf416ec09499d9d19 {
}
struct CustomReserved9 @0xa1680744031fdb2d {
}
struct CustomReserved10 @0xcb9fd56c7057593a {
}
struct CustomReserved11 @0xc2243c65e0340384 {
}
struct CustomReserved12 @0x9ccdc8676701b412 {
}
struct CustomReserved13 @0xcd96dafb67a082d0 {
}
struct CustomReserved14 @0xb057204d7deadf3f {
}
struct CustomReserved15 @0xbd443b539493bc68 {
}
struct CustomReserved16 @0xfc6241ed8877b611 {
}
struct CustomReserved17 @0xa30662f84033036c {
}
struct CustomReserved18 @0xc86a3d38d13eb3ef {
}
struct CustomReserved19 @0xa4f1eb3323f5f582 {
}
+28
View File
@@ -0,0 +1,28 @@
# Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
# Licensed under the MIT License:
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
@0xc5f1af96651f70ea;
annotation package @0x9ee4c8f803b3b596 (file) : Text;
# Name of the package, such as "org.example.foo", in which the generated code will reside.
annotation outerClassname @0x9b066bb4881f7cd3 (file) : Text;
# Name of the outer class that will wrap the generated code.
-574
View File
@@ -1,574 +0,0 @@
using Cxx = import "./include/c++.capnp";
$Cxx.namespace("cereal");
@0x80ef1ec4889c2a63;
# legacy.capnp: a home for deprecated structs
struct LogRotate @0x9811e1f38f62f2d1 {
segmentNum @0 :Int32;
path @1 :Text;
}
struct LiveUI @0xc08240f996aefced {
rearViewCam @0 :Bool;
alertText1 @1 :Text;
alertText2 @2 :Text;
awarenessStatus @3 :Float32;
}
struct UiLayoutState @0x88dcce08ad29dda0 {
activeApp @0 :App;
sidebarCollapsed @1 :Bool;
mapEnabled @2 :Bool;
mockEngaged @3 :Bool;
enum App @0x9917470acf94d285 {
home @0;
music @1;
nav @2;
settings @3;
none @4;
}
}
struct OrbslamCorrection @0x8afd33dc9b35e1aa {
correctionMonoTime @0 :UInt64;
prePositionECEF @1 :List(Float64);
postPositionECEF @2 :List(Float64);
prePoseQuatECEF @3 :List(Float32);
postPoseQuatECEF @4 :List(Float32);
numInliers @5 :UInt32;
}
struct EthernetPacket @0xa99a9d5b33cf5859 {
pkt @0 :Data;
ts @1 :Float32;
}
struct CellInfo @0xcff7566681c277ce {
timestamp @0 :UInt64;
repr @1 :Text; # android toString() for now
}
struct WifiScan @0xd4df5a192382ba0b {
bssid @0 :Text;
ssid @1 :Text;
capabilities @2 :Text;
frequency @3 :Int32;
level @4 :Int32;
timestamp @5 :Int64;
centerFreq0 @6 :Int32;
centerFreq1 @7 :Int32;
channelWidth @8 :ChannelWidth;
operatorFriendlyName @9 :Text;
venueName @10 :Text;
is80211mcResponder @11 :Bool;
passpoint @12 :Bool;
distanceCm @13 :Int32;
distanceSdCm @14 :Int32;
enum ChannelWidth @0xcb6a279f015f6b51 {
w20Mhz @0;
w40Mhz @1;
w80Mhz @2;
w160Mhz @3;
w80Plus80Mhz @4;
}
}
struct LiveEventData @0x94b7baa90c5c321e {
name @0 :Text;
value @1 :Int32;
}
struct ModelData @0xb8aad62cffef28a9 {
frameId @0 :UInt32;
frameAge @12 :UInt32;
frameDropPerc @13 :Float32;
timestampEof @9 :UInt64;
modelExecutionTime @14 :Float32;
gpuExecutionTime @16 :Float32;
rawPred @15 :Data;
path @1 :PathData;
leftLane @2 :PathData;
rightLane @3 :PathData;
lead @4 :LeadData;
freePath @6 :List(Float32);
settings @5 :ModelSettings;
leadFuture @7 :LeadData;
speed @8 :List(Float32);
meta @10 :MetaData;
longitudinal @11 :LongitudinalData;
struct PathData @0x8817eeea389e9f08 {
points @0 :List(Float32);
prob @1 :Float32;
std @2 :Float32;
stds @3 :List(Float32);
poly @4 :List(Float32);
validLen @5 :Float32;
}
struct LeadData @0xd1c9bef96d26fa91 {
dist @0 :Float32;
prob @1 :Float32;
std @2 :Float32;
relVel @3 :Float32;
relVelStd @4 :Float32;
relY @5 :Float32;
relYStd @6 :Float32;
relA @7 :Float32;
relAStd @8 :Float32;
}
struct ModelSettings @0xa26e3710efd3e914 {
bigBoxX @0 :UInt16;
bigBoxY @1 :UInt16;
bigBoxWidth @2 :UInt16;
bigBoxHeight @3 :UInt16;
boxProjection @4 :List(Float32);
yuvCorrection @5 :List(Float32);
inputTransform @6 :List(Float32);
}
struct MetaData @0x9744f25fb60f2bf8 {
engagedProb @0 :Float32;
desirePrediction @1 :List(Float32);
brakeDisengageProb @2 :Float32;
gasDisengageProb @3 :Float32;
steerOverrideProb @4 :Float32;
desireState @5 :List(Float32);
}
struct LongitudinalData @0xf98f999c6a071122 {
distances @2 :List(Float32);
speeds @0 :List(Float32);
accelerations @1 :List(Float32);
}
}
struct ECEFPoint @0xc25bbbd524983447 {
x @0 :Float64;
y @1 :Float64;
z @2 :Float64;
}
struct ECEFPointDEPRECATED @0xe10e21168db0c7f7 {
x @0 :Float32;
y @1 :Float32;
z @2 :Float32;
}
struct GPSPlannerPoints @0xab54c59699f8f9f3 {
curPosDEPRECATED @0 :ECEFPointDEPRECATED;
pointsDEPRECATED @1 :List(ECEFPointDEPRECATED);
curPos @6 :ECEFPoint;
points @7 :List(ECEFPoint);
valid @2 :Bool;
trackName @3 :Text;
speedLimit @4 :Float32;
accelTarget @5 :Float32;
}
struct GPSPlannerPlan @0xf5ad1d90cdc1dd6b {
valid @0 :Bool;
poly @1 :List(Float32);
trackName @2 :Text;
speed @3 :Float32;
acceleration @4 :Float32;
pointsDEPRECATED @5 :List(ECEFPointDEPRECATED);
points @6 :List(ECEFPoint);
xLookahead @7 :Float32;
}
struct UiNavigationEvent @0x90c8426c3eaddd3b {
type @0: Type;
status @1: Status;
distanceTo @2: Float32;
endRoadPointDEPRECATED @3: ECEFPointDEPRECATED;
endRoadPoint @4: ECEFPoint;
enum Type @0xe8db07dcf8fcea05 {
none @0;
laneChangeLeft @1;
laneChangeRight @2;
mergeLeft @3;
mergeRight @4;
turnLeft @5;
turnRight @6;
}
enum Status @0xb9aa88c75ef99a1f {
none @0;
passive @1;
approaching @2;
active @3;
}
}
struct LiveLocationData @0xb99b2bc7a57e8128 {
status @0 :UInt8;
# 3D fix
lat @1 :Float64;
lon @2 :Float64;
alt @3 :Float32; # m
# speed
speed @4 :Float32; # m/s
# NED velocity components
vNED @5 :List(Float32);
# roll, pitch, heading (x,y,z)
roll @6 :Float32; # WRT to center of earth?
pitch @7 :Float32; # WRT to center of earth?
heading @8 :Float32; # WRT to north?
# what are these?
wanderAngle @9 :Float32;
trackAngle @10 :Float32;
# car frame -- https://upload.wikimedia.org/wikipedia/commons/f/f5/RPY_angles_of_cars.png
# gyro, in car frame, deg/s
gyro @11 :List(Float32);
# accel, in car frame, m/s^2
accel @12 :List(Float32);
accuracy @13 :Accuracy;
source @14 :SensorSource;
# if we are fixing a location in the past
fixMonoTime @15 :UInt64;
gpsWeek @16 :Int32;
timeOfWeek @17 :Float64;
positionECEF @18 :List(Float64);
poseQuatECEF @19 :List(Float32);
pitchCalibration @20 :Float32;
yawCalibration @21 :Float32;
imuFrame @22 :List(Float32);
struct Accuracy @0x943dc4625473b03f {
pNEDError @0 :List(Float32);
vNEDError @1 :List(Float32);
rollError @2 :Float32;
pitchError @3 :Float32;
headingError @4 :Float32;
ellipsoidSemiMajorError @5 :Float32;
ellipsoidSemiMinorError @6 :Float32;
ellipsoidOrientationError @7 :Float32;
}
enum SensorSource @0xc871d3cc252af657 {
applanix @0;
kalman @1;
orbslam @2;
timing @3;
dummy @4;
}
}
struct OrbOdometry @0xd7700859ed1f5b76 {
# timing first
startMonoTime @0 :UInt64;
endMonoTime @1 :UInt64;
# fundamental matrix and error
f @2: List(Float64);
err @3: Float64;
# number of inlier points
inliers @4: Int32;
# for debug only
# indexed by endMonoTime features
# value is startMonoTime feature match
# -1 if no match
matches @5: List(Int16);
}
struct OrbFeatures @0xcd60164a8a0159ef {
timestampEof @0 :UInt64;
# transposed arrays of normalized image coordinates
# len(xs) == len(ys) == len(descriptors) * 32
xs @1 :List(Float32);
ys @2 :List(Float32);
descriptors @3 :Data;
octaves @4 :List(Int8);
# match index to last OrbFeatures
# -1 if no match
timestampLastEof @5 :UInt64;
matches @6: List(Int16);
}
struct OrbFeaturesSummary @0xd500d30c5803fa4f {
timestampEof @0 :UInt64;
timestampLastEof @1 :UInt64;
featureCount @2 :UInt16;
matchCount @3 :UInt16;
computeNs @4 :UInt64;
}
struct OrbKeyFrame @0xc8233c0345e27e24 {
# this is a globally unique id for the KeyFrame
id @0: UInt64;
# this is the location of the KeyFrame
pos @1: ECEFPoint;
# these are the features in the world
# len(dpos) == len(descriptors) * 32
dpos @2 :List(ECEFPoint);
descriptors @3 :Data;
}
struct KalmanOdometry @0x92e21bb7ea38793a {
trans @0 :List(Float32); # m/s in device frame
rot @1 :List(Float32); # rad/s in device frame
transStd @2 :List(Float32); # std m/s in device frame
rotStd @3 :List(Float32); # std rad/s in device frame
}
struct OrbObservation @0x9b326d4e436afec7 {
observationMonoTime @0 :UInt64;
normalizedCoordinates @1 :List(Float32);
locationECEF @2 :List(Float64);
matchDistance @3: UInt32;
}
struct CalibrationFeatures @0x8fdfadb254ea867a {
frameId @0 :UInt32;
p0 @1 :List(Float32);
p1 @2 :List(Float32);
status @3 :List(Int8);
}
struct NavStatus @0xbd8822120928120c {
isNavigating @0 :Bool;
currentAddress @1 :Address;
struct Address @0xce7cd672cacc7814 {
title @0 :Text;
lat @1 :Float64;
lng @2 :Float64;
house @3 :Text;
address @4 :Text;
street @5 :Text;
city @6 :Text;
state @7 :Text;
country @8 :Text;
}
}
struct NavUpdate @0xdb98be6565516acb {
isNavigating @0 :Bool;
curSegment @1 :Int32;
segments @2 :List(Segment);
struct LatLng @0x9eaef9187cadbb9b {
lat @0 :Float64;
lng @1 :Float64;
}
struct Segment @0xa5b39b4fc4d7da3f {
from @0 :LatLng;
to @1 :LatLng;
updateTime @2 :Int32;
distance @3 :Int32;
crossTime @4 :Int32;
exitNo @5 :Int32;
instruction @6 :Instruction;
parts @7 :List(LatLng);
enum Instruction @0xc5417a637451246f {
turnLeft @0;
turnRight @1;
keepLeft @2;
keepRight @3;
straight @4;
roundaboutExitNumber @5;
roundaboutExit @6;
roundaboutTurnLeft @7;
unkn8 @8;
roundaboutStraight @9;
unkn10 @10;
roundaboutTurnRight @11;
unkn12 @12;
roundaboutUturn @13;
unkn14 @14;
arrive @15;
exitLeft @16;
exitRight @17;
unkn18 @18;
uturn @19;
# ...
}
}
}
struct TrafficEvent @0xacfa74a094e62626 {
type @0 :Type;
distance @1 :Float32;
action @2 :Action;
resuming @3 :Bool;
enum Type @0xd85d75253435bf4b {
stopSign @0;
lightRed @1;
lightYellow @2;
lightGreen @3;
stopLight @4;
}
enum Action @0xa6f6ce72165ccb49 {
none @0;
yield @1;
stop @2;
resumeReady @3;
}
}
struct AndroidGnss @0xdfdf30d03fc485bd {
union {
measurements @0 :Measurements;
navigationMessage @1 :NavigationMessage;
}
struct Measurements @0xa20710d4f428d6cd {
clock @0 :Clock;
measurements @1 :List(Measurement);
struct Clock @0xa0e27b453a38f450 {
timeNanos @0 :Int64;
hardwareClockDiscontinuityCount @1 :Int32;
hasTimeUncertaintyNanos @2 :Bool;
timeUncertaintyNanos @3 :Float64;
hasLeapSecond @4 :Bool;
leapSecond @5 :Int32;
hasFullBiasNanos @6 :Bool;
fullBiasNanos @7 :Int64;
hasBiasNanos @8 :Bool;
biasNanos @9 :Float64;
hasBiasUncertaintyNanos @10 :Bool;
biasUncertaintyNanos @11 :Float64;
hasDriftNanosPerSecond @12 :Bool;
driftNanosPerSecond @13 :Float64;
hasDriftUncertaintyNanosPerSecond @14 :Bool;
driftUncertaintyNanosPerSecond @15 :Float64;
}
struct Measurement @0xd949bf717d77614d {
svId @0 :Int32;
constellation @1 :Constellation;
timeOffsetNanos @2 :Float64;
state @3 :Int32;
receivedSvTimeNanos @4 :Int64;
receivedSvTimeUncertaintyNanos @5 :Int64;
cn0DbHz @6 :Float64;
pseudorangeRateMetersPerSecond @7 :Float64;
pseudorangeRateUncertaintyMetersPerSecond @8 :Float64;
accumulatedDeltaRangeState @9 :Int32;
accumulatedDeltaRangeMeters @10 :Float64;
accumulatedDeltaRangeUncertaintyMeters @11 :Float64;
hasCarrierFrequencyHz @12 :Bool;
carrierFrequencyHz @13 :Float32;
hasCarrierCycles @14 :Bool;
carrierCycles @15 :Int64;
hasCarrierPhase @16 :Bool;
carrierPhase @17 :Float64;
hasCarrierPhaseUncertainty @18 :Bool;
carrierPhaseUncertainty @19 :Float64;
hasSnrInDb @20 :Bool;
snrInDb @21 :Float64;
multipathIndicator @22 :MultipathIndicator;
enum Constellation @0x9ef1f3ff0deb5ffb {
unknown @0;
gps @1;
sbas @2;
glonass @3;
qzss @4;
beidou @5;
galileo @6;
}
enum State @0xcbb9490adce12d72 {
unknown @0;
codeLock @1;
bitSync @2;
subframeSync @3;
towDecoded @4;
msecAmbiguous @5;
symbolSync @6;
gloStringSync @7;
gloTodDecoded @8;
bdsD2BitSync @9;
bdsD2SubframeSync @10;
galE1bcCodeLock @11;
galE1c2ndCodeLock @12;
galE1bPageSync @13;
sbasSync @14;
}
enum MultipathIndicator @0xc04e7b6231d4caa8 {
unknown @0;
detected @1;
notDetected @2;
}
}
}
struct NavigationMessage @0xe2517b083095fd4e {
type @0 :Int32;
svId @1 :Int32;
messageId @2 :Int32;
submessageId @3 :Int32;
data @4 :Data;
status @5 :Status;
enum Status @0xec1ff7996b35366f {
unknown @0;
parityPassed @1;
parityRebuilt @2;
}
}
}
struct LidarPts @0xe3d6685d4e9d8f7a {
r @0 :List(UInt16); # uint16 m*500.0
theta @1 :List(UInt16); # uint16 deg*100.0
reflect @2 :List(UInt8); # uint8 0-255
# For storing out of file.
idx @3 :UInt64;
# For storing in file
pkt @4 :Data;
}
+721 -1959
View File
File diff suppressed because it is too large Load Diff
-257
View File
@@ -1,257 +0,0 @@
# must be built with scons
from msgq.ipc_pyx import Context, Poller, SubSocket, PubSocket, SocketEventHandle, toggle_fake_events, \
set_fake_prefix, get_fake_prefix, delete_fake_prefix, wait_for_one_event
from msgq.ipc_pyx import MultiplePublishersError, IpcError
from msgq import fake_event_handle, pub_sock, sub_sock, drain_sock_raw
import msgq
import os
import capnp
import time
from typing import Optional, List, Union, Dict
from cereal import log
from cereal.services import SERVICE_LIST
from openpilot.common.util import MovingAverage
NO_TRAVERSAL_LIMIT = 2**64-1
def reset_context():
msgq.context = Context()
def log_from_bytes(dat: bytes, struct: capnp.lib.capnp._StructModule = log.Event) -> capnp.lib.capnp._DynamicStructReader:
with struct.from_bytes(dat, traversal_limit_in_words=NO_TRAVERSAL_LIMIT) as msg:
return msg
def new_message(service: Optional[str], size: Optional[int] = None, **kwargs) -> capnp.lib.capnp._DynamicStructBuilder:
args = {
'valid': False,
'logMonoTime': int(time.monotonic() * 1e9),
**kwargs
}
dat = log.Event.new_message(**args)
if service is not None:
if size is None:
dat.init(service)
else:
dat.init(service, size)
return dat
def drain_sock(sock: SubSocket, wait_for_one: bool = False) -> List[capnp.lib.capnp._DynamicStructReader]:
"""Receive all message currently available on the queue"""
msgs = drain_sock_raw(sock, wait_for_one=wait_for_one)
return [log_from_bytes(m) for m in msgs]
# TODO: print when we drop packets?
def recv_sock(sock: SubSocket, wait: bool = False) -> Optional[capnp.lib.capnp._DynamicStructReader]:
"""Same as drain sock, but only returns latest message. Consider using conflate instead."""
dat = None
while 1:
if wait and dat is None:
recv = sock.receive()
else:
recv = sock.receive(non_blocking=True)
if recv is None: # Timeout hit
break
dat = recv
if dat is not None:
dat = log_from_bytes(dat)
return dat
def recv_one(sock: SubSocket) -> Optional[capnp.lib.capnp._DynamicStructReader]:
dat = sock.receive()
if dat is not None:
dat = log_from_bytes(dat)
return dat
def recv_one_or_none(sock: SubSocket) -> Optional[capnp.lib.capnp._DynamicStructReader]:
dat = sock.receive(non_blocking=True)
if dat is not None:
dat = log_from_bytes(dat)
return dat
def recv_one_retry(sock: SubSocket) -> capnp.lib.capnp._DynamicStructReader:
"""Keep receiving until we get a message"""
while True:
dat = sock.receive()
if dat is not None:
return log_from_bytes(dat)
class FrequencyTracker:
def __init__(self, service_freq: float, update_freq: float, is_poll: bool):
freq = max(min(service_freq, update_freq), 1.)
if is_poll:
min_freq = max_freq = freq
else:
max_freq = min(freq, update_freq)
if service_freq >= 2 * update_freq:
min_freq = update_freq
elif update_freq >= 2* service_freq:
min_freq = freq
else:
min_freq = min(freq, freq / 2.)
self.min_freq = min_freq * 0.8
self.max_freq = max_freq * 1.2
self.avg_dt = MovingAverage(int(10 * freq))
self.recent_avg_dt = MovingAverage(int(freq))
self.prev_time = 0.0
def record_recv_time(self, cur_time: float) -> None:
# TODO: Handle case where cur_time is less than prev_time
if self.prev_time > 1e-5:
dt = cur_time - self.prev_time
self.avg_dt.add_value(dt)
self.recent_avg_dt.add_value(dt)
self.prev_time = cur_time
@property
def valid(self) -> bool:
if self.avg_dt.count == 0:
return False
avg_freq = 1.0 / self.avg_dt.get_average()
if self.min_freq <= avg_freq <= self.max_freq:
return True
avg_freq_recent = 1.0 / self.recent_avg_dt.get_average()
return self.min_freq <= avg_freq_recent <= self.max_freq
class SubMaster:
def __init__(self, services: List[str], poll: Optional[str] = None,
ignore_alive: Optional[List[str]] = None, ignore_avg_freq: Optional[List[str]] = None,
ignore_valid: Optional[List[str]] = None, addr: str = "127.0.0.1", frequency: Optional[float] = None):
self.frame = -1
self.services = services
self.seen = {s: False for s in services}
self.updated = {s: False for s in services}
self.recv_time = {s: 0. for s in services}
self.recv_frame = {s: 0 for s in services}
self.sock = {}
self.data = {}
self.logMonoTime = {s: 0 for s in services}
# zero-frequency / on-demand services are always alive and presumed valid; all others must pass checks
on_demand = {s: SERVICE_LIST[s].frequency <= 1e-5 for s in services}
self.static_freq_services = set(s for s in services if not on_demand[s])
self.alive = {s: on_demand[s] for s in services}
self.freq_ok = {s: on_demand[s] for s in services}
self.valid = {s: on_demand[s] for s in services}
self.freq_tracker: Dict[str, FrequencyTracker] = {}
self.poller = Poller()
polled_services = set([poll, ] if poll is not None else services)
self.non_polled_services = set(services) - polled_services
self.ignore_average_freq = [] if ignore_avg_freq is None else ignore_avg_freq
self.ignore_alive = [] if ignore_alive is None else ignore_alive
self.ignore_valid = [] if ignore_valid is None else ignore_valid
self.simulation = bool(int(os.getenv("SIMULATION", "0")))
# if freq and poll aren't specified, assume the max to be conservative
assert frequency is None or poll is None, "Do not specify 'frequency' - frequency of the polled service will be used."
self.update_freq = frequency or max([SERVICE_LIST[s].frequency for s in polled_services])
for s in services:
p = self.poller if s not in self.non_polled_services else None
self.sock[s] = sub_sock(s, poller=p, addr=addr, conflate=True)
try:
data = new_message(s)
except capnp.lib.capnp.KjException:
data = new_message(s, 0) # lists
self.data[s] = getattr(data.as_reader(), s)
self.freq_tracker[s] = FrequencyTracker(SERVICE_LIST[s].frequency, self.update_freq, s == poll)
def __getitem__(self, s: str) -> capnp.lib.capnp._DynamicStructReader:
return self.data[s]
def _check_avg_freq(self, s: str) -> bool:
return SERVICE_LIST[s].frequency > 0.99 and (s not in self.ignore_average_freq) and (s not in self.ignore_alive)
def update(self, timeout: int = 100) -> None:
msgs = []
for sock in self.poller.poll(timeout):
msgs.append(recv_one_or_none(sock))
# non-blocking receive for non-polled sockets
for s in self.non_polled_services:
msgs.append(recv_one_or_none(self.sock[s]))
self.update_msgs(time.monotonic(), msgs)
def update_msgs(self, cur_time: float, msgs: List[capnp.lib.capnp._DynamicStructReader]) -> None:
self.frame += 1
self.updated = dict.fromkeys(self.services, False)
for msg in msgs:
if msg is None:
continue
s = msg.which()
self.seen[s] = True
self.updated[s] = True
self.freq_tracker[s].record_recv_time(cur_time)
self.recv_time[s] = cur_time
self.recv_frame[s] = self.frame
self.data[s] = getattr(msg, s)
self.logMonoTime[s] = msg.logMonoTime
self.valid[s] = msg.valid
for s in self.static_freq_services:
# alive if delay is within 10x the expected frequency; checks relaxed in simulator
self.alive[s] = (cur_time - self.recv_time[s]) < (10. / SERVICE_LIST[s].frequency) or (self.seen[s] and self.simulation)
self.freq_ok[s] = self.freq_tracker[s].valid or self.simulation
def all_alive(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.alive[s] for s in (service_list or self.services) if s not in self.ignore_alive)
def all_freq_ok(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.freq_ok[s] for s in (service_list or self.services) if self._check_avg_freq(s))
def all_valid(self, service_list: Optional[List[str]] = None) -> bool:
return all(self.valid[s] for s in (service_list or self.services) if s not in self.ignore_valid)
def all_checks(self, service_list: Optional[List[str]] = None) -> bool:
return self.all_alive(service_list) and self.all_freq_ok(service_list) and self.all_valid(service_list)
class PubMaster:
def __init__(self, services: List[str]):
self.sock = {}
for s in services:
self.sock[s] = pub_sock(s)
def send(self, s: str, dat: Union[bytes, capnp.lib.capnp._DynamicStructBuilder]) -> None:
if not isinstance(dat, bytes):
dat = dat.to_bytes()
self.sock[s].send(dat)
def wait_for_readers_to_update(self, s: str, timeout: int, dt: float = 0.05) -> bool:
for _ in range(int(timeout*(1./dt))):
if self.sock[s].all_readers_updated():
return True
time.sleep(dt)
return False
def all_readers_updated(self, s: str) -> bool:
return self.sock[s].all_readers_updated() # type: ignore
-71
View File
@@ -1,71 +0,0 @@
#include <cassert>
#include "cereal/messaging/msgq_to_zmq.h"
#include "cereal/services.h"
#include "common/util.h"
ExitHandler do_exit;
static std::vector<std::string> get_services(const std::string &whitelist_str, bool zmq_to_msgq) {
std::vector<std::string> service_list;
for (const auto& it : services) {
std::string name = it.second.name;
bool in_whitelist = whitelist_str.find(name) != std::string::npos;
if (zmq_to_msgq && !in_whitelist) {
continue;
}
service_list.push_back(name);
}
return service_list;
}
void msgq_to_zmq(const std::vector<std::string> &endpoints, const std::string &ip) {
MsgqToZmq bridge;
bridge.run(endpoints, ip);
}
void zmq_to_msgq(const std::vector<std::string> &endpoints, const std::string &ip) {
auto poller = std::make_unique<ZMQPoller>();
auto pub_context = std::make_unique<MSGQContext>();
auto sub_context = std::make_unique<ZMQContext>();
std::map<SubSocket *, PubSocket *> sub2pub;
for (auto endpoint : endpoints) {
auto pub_sock = new MSGQPubSocket();
auto sub_sock = new ZMQSubSocket();
pub_sock->connect(pub_context.get(), endpoint);
sub_sock->connect(sub_context.get(), endpoint, ip, false);
poller->registerSocket(sub_sock);
sub2pub[sub_sock] = pub_sock;
}
while (!do_exit) {
for (auto sub_sock : poller->poll(100)) {
std::unique_ptr<Message> msg(sub_sock->receive(true));
if (msg) {
sub2pub[sub_sock]->sendMessage(msg.get());
}
}
}
// Clean up allocated sockets
for (auto &[sub_sock, pub_sock] : sub2pub) {
delete sub_sock;
delete pub_sock;
}
}
int main(int argc, char **argv) {
bool is_zmq_to_msgq = argc > 2;
std::string ip = is_zmq_to_msgq ? argv[1] : "127.0.0.1";
std::string whitelist_str = is_zmq_to_msgq ? std::string(argv[2]) : "";
std::vector<std::string> endpoints = get_services(whitelist_str, is_zmq_to_msgq);
if (is_zmq_to_msgq) {
zmq_to_msgq(endpoints, ip);
} else {
msgq_to_zmq(endpoints, ip);
}
return 0;
}
-102
View File
@@ -1,102 +0,0 @@
#pragma once
#include <cstddef>
#include <map>
#include <string>
#include <vector>
#include <utility>
#include <capnp/serialize.h>
#include "cereal/gen/cpp/log.capnp.h"
#include "common/timing.h"
#include "msgq/ipc.h"
class SubMaster {
public:
SubMaster(const std::vector<const char *> &service_list, const std::vector<const char *> &poll = {},
const char *address = nullptr, const std::vector<const char *> &ignore_alive = {});
void update(int timeout = 1000);
void update_msgs(uint64_t current_time, const std::vector<std::pair<std::string, cereal::Event::Reader>> &messages);
inline bool allAlive(const std::vector<const char *> &service_list = {}) { return all_(service_list, false, true); }
inline bool allValid(const std::vector<const char *> &service_list = {}) { return all_(service_list, true, false); }
inline bool allAliveAndValid(const std::vector<const char *> &service_list = {}) { return all_(service_list, true, true); }
void drain();
~SubMaster();
uint64_t frame = 0;
bool updated(const char *name) const;
bool alive(const char *name) const;
bool valid(const char *name) const;
uint64_t rcv_frame(const char *name) const;
uint64_t rcv_time(const char *name) const;
cereal::Event::Reader &operator[](const char *name) const;
private:
bool all_(const std::vector<const char *> &service_list, bool valid, bool alive);
Poller *poller_ = nullptr;
struct SubMessage;
std::map<SubSocket *, SubMessage *> messages_;
std::map<std::string, SubMessage *> services_;
};
class MessageBuilder : public capnp::MallocMessageBuilder {
public:
MessageBuilder() = default;
cereal::Event::Builder initEvent(bool valid = true) {
cereal::Event::Builder event = initRoot<cereal::Event>();
event.setLogMonoTime(nanos_since_boot());
event.setValid(valid);
return event;
}
kj::ArrayPtr<capnp::byte> toBytes() {
heapArray_ = capnp::messageToFlatArray(*this);
return heapArray_.asBytes();
}
size_t getSerializedSize() {
return capnp::computeSerializedSizeInWords(*this) * sizeof(capnp::word);
}
int serializeToBuffer(unsigned char *buffer, size_t buffer_size) {
size_t serialized_size = getSerializedSize();
if (serialized_size > buffer_size) { return -1; }
kj::ArrayOutputStream out(kj::ArrayPtr<capnp::byte>(buffer, buffer_size));
capnp::writeMessage(out, *this);
return serialized_size;
}
private:
kj::Array<capnp::word> heapArray_;
};
class PubMaster {
public:
PubMaster(const std::vector<const char *> &service_list);
inline int send(const char *name, capnp::byte *data, size_t size) { return sockets_.at(name)->send((char *)data, size); }
int send(const char *name, MessageBuilder &msg);
~PubMaster();
private:
std::map<std::string, PubSocket *> sockets_;
};
class AlignedBuffer {
public:
kj::ArrayPtr<const capnp::word> align(const char *data, const size_t size) {
words_size = size / sizeof(capnp::word) + 1;
if (aligned_buf.size() < words_size) {
aligned_buf = kj::heapArray<capnp::word>(words_size < 512 ? 512 : words_size);
}
memcpy(aligned_buf.begin(), data, size);
return aligned_buf.slice(0, words_size);
}
inline kj::ArrayPtr<const capnp::word> align(Message *m) {
return align(m->getData(), m->getSize());
}
private:
kj::Array<capnp::word> aligned_buf;
size_t words_size;
};
-144
View File
@@ -1,144 +0,0 @@
#include "cereal/messaging/msgq_to_zmq.h"
#include <cassert>
#include "common/util.h"
extern ExitHandler do_exit;
// Max messages to process per socket per poll
constexpr int MAX_MESSAGES_PER_SOCKET = 50;
static std::string recv_zmq_msg(void *sock) {
zmq_msg_t msg;
zmq_msg_init(&msg);
std::string ret;
if (zmq_msg_recv(&msg, sock, 0) > 0) {
ret.assign((char *)zmq_msg_data(&msg), zmq_msg_size(&msg));
}
zmq_msg_close(&msg);
return ret;
}
void MsgqToZmq::run(const std::vector<std::string> &endpoints, const std::string &ip) {
zmq_context = std::make_unique<ZMQContext>();
msgq_context = std::make_unique<MSGQContext>();
// Create ZMQPubSockets for each endpoint
for (const auto &endpoint : endpoints) {
auto &socket_pair = socket_pairs.emplace_back();
socket_pair.endpoint = endpoint;
socket_pair.pub_sock = std::make_unique<ZMQPubSocket>();
int ret = socket_pair.pub_sock->connect(zmq_context.get(), endpoint);
if (ret != 0) {
printf("Failed to create ZMQ publisher for [%s]: %s\n", endpoint.c_str(), zmq_strerror(zmq_errno()));
return;
}
}
// Start ZMQ monitoring thread to monitor socket events
std::thread thread(&MsgqToZmq::zmqMonitorThread, this);
// Main loop for processing messages
while (!do_exit) {
{
std::unique_lock lk(mutex);
cv.wait(lk, [this]() { return do_exit || !sub2pub.empty(); });
if (do_exit) break;
for (auto sub_sock : msgq_poller->poll(100)) {
// Process messages for each socket
ZMQPubSocket *pub_sock = sub2pub.at(sub_sock);
for (int i = 0; i < MAX_MESSAGES_PER_SOCKET; ++i) {
auto msg = std::unique_ptr<Message>(sub_sock->receive(true));
if (!msg) break;
while (pub_sock->sendMessage(msg.get()) == -1) {
if (errno != EINTR) break;
}
}
}
}
util::sleep_for(1); // Give zmqMonitorThread a chance to acquire the mutex
}
thread.join();
}
void MsgqToZmq::zmqMonitorThread() {
std::vector<zmq_pollitem_t> pollitems;
// Set up ZMQ monitor for each pub socket
for (int i = 0; i < socket_pairs.size(); ++i) {
std::string addr = "inproc://op-bridge-monitor-" + std::to_string(i);
zmq_socket_monitor(socket_pairs[i].pub_sock->sock, addr.c_str(), ZMQ_EVENT_ACCEPTED | ZMQ_EVENT_DISCONNECTED);
void *monitor_socket = zmq_socket(zmq_context->getRawContext(), ZMQ_PAIR);
zmq_connect(monitor_socket, addr.c_str());
pollitems.emplace_back(zmq_pollitem_t{.socket = monitor_socket, .events = ZMQ_POLLIN});
}
while (!do_exit) {
int ret = zmq_poll(pollitems.data(), pollitems.size(), 1000);
if (ret < 0) {
if (errno == EINTR) {
// Due to frequent EINTR signals from msgq, introduce a brief delay (200 ms)
// to reduce CPU usage during retry attempts.
util::sleep_for(200);
}
continue;
}
for (int i = 0; i < pollitems.size(); ++i) {
if (pollitems[i].revents & ZMQ_POLLIN) {
// First frame in message contains event number and value
std::string frame = recv_zmq_msg(pollitems[i].socket);
if (frame.empty()) continue;
uint16_t event_type = *(uint16_t *)(frame.data());
// Second frame in message contains event address
frame = recv_zmq_msg(pollitems[i].socket);
if (frame.empty()) continue;
std::unique_lock lk(mutex);
auto &pair = socket_pairs[i];
if (event_type & ZMQ_EVENT_ACCEPTED) {
printf("socket [%s] connected\n", pair.endpoint.c_str());
if (++pair.connected_clients == 1) {
// Create new MSGQ subscriber socket and map to ZMQ publisher
pair.sub_sock = std::make_unique<MSGQSubSocket>();
pair.sub_sock->connect(msgq_context.get(), pair.endpoint, "127.0.0.1");
sub2pub[pair.sub_sock.get()] = pair.pub_sock.get();
registerSockets();
}
} else if (event_type & ZMQ_EVENT_DISCONNECTED) {
printf("socket [%s] disconnected\n", pair.endpoint.c_str());
if (pair.connected_clients == 0 || --pair.connected_clients == 0) {
// Remove MSGQ subscriber socket from mapping and reset it
sub2pub.erase(pair.sub_sock.get());
pair.sub_sock.reset(nullptr);
registerSockets();
}
}
cv.notify_one();
}
}
}
// Clean up monitor sockets
for (int i = 0; i < pollitems.size(); ++i) {
zmq_socket_monitor(socket_pairs[i].pub_sock->sock, nullptr, 0);
zmq_close(pollitems[i].socket);
}
cv.notify_one();
}
void MsgqToZmq::registerSockets() {
msgq_poller = std::make_unique<MSGQPoller>();
for (const auto &socket_pair : socket_pairs) {
if (socket_pair.sub_sock) {
msgq_poller->registerSocket(socket_pair.sub_sock.get());
}
}
}
-37
View File
@@ -1,37 +0,0 @@
#pragma once
#include <condition_variable>
#include <map>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
#define private public
#include "msgq/impl_msgq.h"
#include "msgq/impl_zmq.h"
class MsgqToZmq {
public:
MsgqToZmq() {}
void run(const std::vector<std::string> &endpoints, const std::string &ip);
protected:
void registerSockets();
void zmqMonitorThread();
struct SocketPair {
std::string endpoint;
std::unique_ptr<ZMQPubSocket> pub_sock;
std::unique_ptr<MSGQSubSocket> sub_sock;
int connected_clients = 0;
};
std::unique_ptr<MSGQContext> msgq_context;
std::unique_ptr<ZMQContext> zmq_context;
std::mutex mutex;
std::condition_variable cv;
std::unique_ptr<MSGQPoller> msgq_poller;
std::map<SubSocket *, ZMQPubSocket *> sub2pub;
std::vector<SocketPair> socket_pairs;
};
-203
View File
@@ -1,203 +0,0 @@
#include <assert.h>
#include <stdlib.h>
#include <string>
#include <mutex>
#include "cereal/services.h"
#include "cereal/messaging/messaging.h"
const bool SIMULATION = (getenv("SIMULATION") != nullptr) && (std::string(getenv("SIMULATION")) == "1");
static inline bool inList(const std::vector<const char *> &list, const char *value) {
for (auto &v : list) {
if (strcmp(value, v) == 0) return true;
}
return false;
}
class MessageContext {
public:
MessageContext() : ctx_(nullptr) {}
~MessageContext() { delete ctx_; }
inline Context *context() {
std::call_once(init_flag, [=]() { ctx_ = Context::create(); });
return ctx_;
}
private:
Context *ctx_;
std::once_flag init_flag;
};
MessageContext message_context;
struct SubMaster::SubMessage {
std::string name;
SubSocket *socket = nullptr;
int freq = 0;
bool updated = false, alive = false, valid = false, ignore_alive;
uint64_t rcv_time = 0, rcv_frame = 0;
void *allocated_msg_reader = nullptr;
bool is_polled = false;
capnp::FlatArrayMessageReader *msg_reader = nullptr;
AlignedBuffer aligned_buf;
cereal::Event::Reader event;
};
SubMaster::SubMaster(const std::vector<const char *> &service_list, const std::vector<const char *> &poll,
const char *address, const std::vector<const char *> &ignore_alive) {
poller_ = Poller::create();
for (auto name : service_list) {
assert(services.count(std::string(name)) > 0);
service serv = services.at(std::string(name));
SubSocket *socket = SubSocket::create(message_context.context(), name, address ? address : "127.0.0.1", true);
assert(socket != 0);
bool is_polled = inList(poll, name) || poll.empty();
if (is_polled) poller_->registerSocket(socket);
SubMessage *m = new SubMessage{
.name = name,
.socket = socket,
.freq = serv.frequency,
.ignore_alive = inList(ignore_alive, name),
.allocated_msg_reader = malloc(sizeof(capnp::FlatArrayMessageReader)),
.is_polled = is_polled};
m->msg_reader = new (m->allocated_msg_reader) capnp::FlatArrayMessageReader({});
messages_[socket] = m;
services_[name] = m;
}
}
void SubMaster::update(int timeout) {
for (auto &kv : messages_) kv.second->updated = false;
auto sockets = poller_->poll(timeout);
// add non-polled sockets for non-blocking receive
for (auto &kv : messages_) {
SubMessage *m = kv.second;
SubSocket *s = kv.first;
if (!m->is_polled) sockets.push_back(s);
}
uint64_t current_time = nanos_since_boot();
std::vector<std::pair<std::string, cereal::Event::Reader>> messages;
for (auto s : sockets) {
Message *msg = s->receive(true);
if (msg == nullptr) continue;
SubMessage *m = messages_.at(s);
m->msg_reader->~FlatArrayMessageReader();
capnp::ReaderOptions options;
options.traversalLimitInWords = kj::maxValue; // Don't limit
m->msg_reader = new (m->allocated_msg_reader) capnp::FlatArrayMessageReader(m->aligned_buf.align(msg), options);
delete msg;
messages.push_back({m->name, m->msg_reader->getRoot<cereal::Event>()});
}
update_msgs(current_time, messages);
}
void SubMaster::update_msgs(uint64_t current_time, const std::vector<std::pair<std::string, cereal::Event::Reader>> &messages){
if (++frame == UINT64_MAX) frame = 1;
for (auto &kv : messages) {
auto m_find = services_.find(kv.first);
if (m_find == services_.end()){
continue;
}
SubMessage *m = m_find->second;
m->event = kv.second;
m->updated = true;
m->rcv_time = current_time;
m->rcv_frame = frame;
m->valid = m->event.getValid();
if (SIMULATION) m->alive = true;
}
if (!SIMULATION) {
for (auto &kv : messages_) {
SubMessage *m = kv.second;
m->alive = (m->freq <= (1e-5) || ((current_time - m->rcv_time) * (1e-9)) < (10.0 / m->freq));
}
}
}
bool SubMaster::all_(const std::vector<const char *> &service_list, bool valid, bool alive) {
int found = 0;
for (auto &kv : messages_) {
SubMessage *m = kv.second;
if (service_list.size() == 0 || inList(service_list, m->name.c_str())) {
found += (!valid || m->valid) && (!alive || (m->alive || m->ignore_alive));
}
}
return service_list.size() == 0 ? found == messages_.size() : found == service_list.size();
}
void SubMaster::drain() {
while (true) {
auto polls = poller_->poll(0);
if (polls.size() == 0)
break;
for (auto sock : polls) {
Message *msg = sock->receive(true);
delete msg;
}
}
}
bool SubMaster::updated(const char *name) const {
return services_.at(name)->updated;
}
bool SubMaster::alive(const char *name) const {
return services_.at(name)->alive;
}
bool SubMaster::valid(const char *name) const {
return services_.at(name)->valid;
}
uint64_t SubMaster::rcv_frame(const char *name) const {
return services_.at(name)->rcv_frame;
}
uint64_t SubMaster::rcv_time(const char *name) const {
return services_.at(name)->rcv_time;
}
cereal::Event::Reader &SubMaster::operator[](const char *name) const {
return services_.at(name)->event;
}
SubMaster::~SubMaster() {
delete poller_;
for (auto &kv : messages_) {
SubMessage *m = kv.second;
m->msg_reader->~FlatArrayMessageReader();
free(m->allocated_msg_reader);
delete m->socket;
delete m;
}
}
PubMaster::PubMaster(const std::vector<const char *> &service_list) {
for (auto name : service_list) {
assert(services.count(name) > 0);
PubSocket *socket = PubSocket::create(message_context.context(), name);
assert(socket);
sockets_[name] = socket;
}
}
int PubMaster::send(const char *name, MessageBuilder &msg) {
auto bytes = msg.toBytes();
return send(name, bytes.begin(), bytes.size());
}
PubMaster::~PubMaster() {
for (auto s : sockets_) delete s.second;
}
-185
View File
@@ -1,185 +0,0 @@
import os
import capnp
import multiprocessing
import numbers
import random
import threading
import time
from parameterized import parameterized
import pytest
from cereal import log, car
import cereal.messaging as messaging
from cereal.services import SERVICE_LIST
events = [evt for evt in log.Event.schema.union_fields if evt in SERVICE_LIST.keys()]
def random_sock():
return random.choice(events)
def random_socks(num_socks=10):
return list({random_sock() for _ in range(num_socks)})
def random_bytes(length=1000):
return bytes([random.randrange(0xFF) for _ in range(length)])
def zmq_sleep(t=1):
if "ZMQ" in os.environ:
time.sleep(t)
# TODO: this should take any capnp struct and returrn a msg with random populated data
def random_carstate():
fields = ["vEgo", "aEgo", "gas", "steeringAngleDeg"]
msg = messaging.new_message("carState")
cs = msg.carState
for f in fields:
setattr(cs, f, random.random() * 10)
return msg
# TODO: this should compare any capnp structs
def assert_carstate(cs1, cs2):
for f in car.CarState.schema.non_union_fields:
# TODO: check all types
val1, val2 = getattr(cs1, f), getattr(cs2, f)
if isinstance(val1, numbers.Number):
assert val1 == val2, f"{f}: sent '{val1}' vs recvd '{val2}'"
def delayed_send(delay, sock, dat):
def send_func():
sock.send(dat)
threading.Timer(delay, send_func).start()
class TestMessaging:
def setUp(self):
# TODO: ZMQ tests are too slow; all sleeps will need to be
# replaced with logic to block on the necessary condition
if "ZMQ" in os.environ:
pytest.skip()
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep()
@parameterized.expand(events)
def test_new_message(self, evt):
try:
msg = messaging.new_message(evt)
except capnp.lib.capnp.KjException:
msg = messaging.new_message(evt, random.randrange(200))
assert (time.monotonic() - msg.logMonoTime) < 0.1
assert not msg.valid
assert evt == msg.which()
@parameterized.expand(events)
def test_pub_sock(self, evt):
messaging.pub_sock(evt)
@parameterized.expand(events)
def test_sub_sock(self, evt):
messaging.sub_sock(evt)
@parameterized.expand([
(messaging.drain_sock, capnp._DynamicStructReader),
(messaging.drain_sock_raw, bytes),
])
def test_drain_sock(self, func, expected_type):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=1000)
zmq_sleep()
# no wait and no msgs in queue
msgs = func(sub_sock)
assert isinstance(msgs, list)
assert len(msgs) == 0
# no wait but msgs are queued up
num_msgs = random.randrange(3, 10)
for _ in range(num_msgs):
pub_sock.send(messaging.new_message(sock).to_bytes())
time.sleep(0.1)
msgs = func(sub_sock)
assert isinstance(msgs, list)
assert all(isinstance(msg, expected_type) for msg in msgs)
assert len(msgs) == num_msgs
def test_recv_sock(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=100)
zmq_sleep()
# no wait and no msg in queue, socket should timeout
recvd = messaging.recv_sock(sub_sock)
assert recvd is None
# no wait and one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
time.sleep(0.01)
recvd = messaging.recv_sock(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
# https://github.com/python/mypy/issues/13038
assert_carstate(msg.carState, recvd.carState)
def test_recv_one(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=1000)
zmq_sleep()
# no msg in queue, socket should timeout
recvd = messaging.recv_one(sub_sock)
assert recvd is None
# one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
recvd = messaging.recv_one(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)
@pytest.mark.xfail(condition="ZMQ" in os.environ, reason='ZMQ detected')
def test_recv_one_or_none(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock)
zmq_sleep()
# no msg in queue, socket shouldn't block
recvd = messaging.recv_one_or_none(sub_sock)
assert recvd is None
# one msg in queue
msg = random_carstate()
pub_sock.send(msg.to_bytes())
recvd = messaging.recv_one_or_none(sub_sock)
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)
def test_recv_one_retry(self):
sock = "carState"
sock_timeout = 0.1
pub_sock = messaging.pub_sock(sock)
sub_sock = messaging.sub_sock(sock, timeout=round(sock_timeout*1000))
zmq_sleep()
# this test doesn't work with ZMQ since multiprocessing interrupts it
if "ZMQ" not in os.environ:
# wait 5 socket timeouts and make sure it's still retrying
p = multiprocessing.Process(target=messaging.recv_one_retry, args=(sub_sock,))
p.start()
time.sleep(sock_timeout*5)
assert p.is_alive()
p.terminate()
# wait 5 socket timeouts before sending
msg = random_carstate()
delayed_send(sock_timeout*5, pub_sock, msg.to_bytes())
start_time = time.monotonic()
recvd = messaging.recv_one_retry(sub_sock)
assert (time.monotonic() - start_time) >= sock_timeout*5
assert isinstance(recvd, capnp._DynamicStructReader)
assert_carstate(msg.carState, recvd.carState)
@@ -1,160 +0,0 @@
import random
import time
from typing import Sized, cast
import cereal.messaging as messaging
from cereal.messaging.tests.test_messaging import events, random_sock, random_socks, \
random_bytes, random_carstate, assert_carstate, \
zmq_sleep
from cereal.services import SERVICE_LIST
class TestSubMaster:
def setup_method(self):
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep(3)
def test_init(self):
sm = messaging.SubMaster(events)
for p in [sm.updated, sm.recv_time, sm.recv_frame, sm.alive,
sm.sock, sm.data, sm.logMonoTime, sm.valid]:
assert len(cast(Sized, p)) == len(events)
def test_init_state(self):
socks = random_socks()
sm = messaging.SubMaster(socks)
assert sm.frame == -1
assert not any(sm.updated.values())
assert not any(sm.seen.values())
on_demand = {s: SERVICE_LIST[s].frequency <= 1e-5 for s in sm.services}
assert all(sm.alive[s] == sm.valid[s] == sm.freq_ok[s] == on_demand[s] for s in sm.services)
assert all(t == 0. for t in sm.recv_time.values())
assert all(f == 0 for f in sm.recv_frame.values())
assert all(t == 0 for t in sm.logMonoTime.values())
for p in [sm.updated, sm.recv_time, sm.recv_frame, sm.alive,
sm.sock, sm.data, sm.logMonoTime, sm.valid]:
assert len(cast(Sized, p)) == len(socks)
def test_getitem(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
zmq_sleep()
msg = random_carstate()
pub_sock.send(msg.to_bytes())
sm.update(1000)
assert_carstate(msg.carState, sm[sock])
# TODO: break this test up to individually test SubMaster.update and SubMaster.update_msgs
def test_update(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
zmq_sleep()
for i in range(10):
msg = messaging.new_message(sock)
pub_sock.send(msg.to_bytes())
sm.update(1000)
assert sm.frame == i
assert all(sm.updated.values())
def test_update_timeout(self):
sock = random_sock()
sm = messaging.SubMaster([sock,])
timeout = random.randrange(1000, 3000)
start_time = time.monotonic()
sm.update(timeout)
t = time.monotonic() - start_time
assert t >= timeout/1000.
assert t < 3
assert not any(sm.updated.values())
def test_avg_frequency_checks(self):
for poll in (True, False):
sm = messaging.SubMaster(["modelV2", "carParams", "carState", "cameraOdometry", "liveCalibration"],
poll=("modelV2" if poll else None),
frequency=(20. if not poll else None))
checks = {
"carState": (20, 20),
"modelV2": (20, 20 if poll else 10),
"cameraOdometry": (20, 10),
"liveCalibration": (4, 4),
"carParams": (None, None),
"userFlag": (None, None),
}
for service, (max_freq, min_freq) in checks.items():
if max_freq is not None:
assert sm._check_avg_freq(service)
assert sm.freq_tracker[service].max_freq == max_freq*1.2
assert sm.freq_tracker[service].min_freq == min_freq*0.8
else:
assert not sm._check_avg_freq(service)
def test_alive(self):
pass
def test_ignore_alive(self):
pass
def test_valid(self):
pass
# SubMaster should always conflate
def test_conflate(self):
sock = "carState"
pub_sock = messaging.pub_sock(sock)
sm = messaging.SubMaster([sock,])
n = 10
for i in range(n+1):
msg = messaging.new_message(sock)
msg.carState.vEgo = i
pub_sock.send(msg.to_bytes())
time.sleep(0.01)
sm.update(1000)
assert sm[sock].vEgo == n
class TestPubMaster:
def setup_method(self):
# ZMQ pub socket takes too long to die
# sleep to prevent multiple publishers error between tests
zmq_sleep(3)
def test_init(self):
messaging.PubMaster(events)
def test_send(self):
socks = random_socks()
pm = messaging.PubMaster(socks)
sub_socks = {s: messaging.sub_sock(s, conflate=True, timeout=1000) for s in socks}
zmq_sleep()
# PubMaster accepts either a capnp msg builder or bytes
for capnp in [True, False]:
for i in range(100):
sock = socks[i % len(socks)]
if capnp:
try:
msg = messaging.new_message(sock)
except Exception:
msg = messaging.new_message(sock, random.randrange(50))
else:
msg = random_bytes()
pm.send(sock, msg)
recvd = sub_socks[sock].receive()
if capnp:
msg.clear_write_flag()
msg = msg.to_bytes()
assert msg == recvd, i
-21
View File
@@ -1,21 +0,0 @@
import os
import tempfile
from typing import Dict
from parameterized import parameterized
import cereal.services as services
from cereal.services import SERVICE_LIST
class TestServices:
@parameterized.expand(SERVICE_LIST.keys())
def test_services(self, s):
service = SERVICE_LIST[s]
assert service.frequency <= 104
assert service.decimation != 0
def test_generated_header(self):
with tempfile.NamedTemporaryFile(suffix=".h") as f:
ret = os.system(f"python3 {services.__file__} > {f.name} && clang++ {f.name} -std=c++11")
assert ret == 0, "generated services header is not valid C"
-123
View File
@@ -1,123 +0,0 @@
#!/usr/bin/env python3
from typing import Optional
class Service:
def __init__(self, should_log: bool, frequency: float, decimation: Optional[int] = None):
self.should_log = should_log
self.frequency = frequency
self.decimation = decimation
_services: dict[str, tuple] = {
# service: (should_log, frequency, qlog decimation (optional))
# note: the "EncodeIdx" packets will still be in the log
"gyroscope": (True, 104., 104),
"gyroscope2": (True, 100., 100),
"accelerometer": (True, 104., 104),
"accelerometer2": (True, 100., 100),
"magnetometer": (True, 25.),
"lightSensor": (True, 100., 100),
"temperatureSensor": (True, 2., 200),
"temperatureSensor2": (True, 2., 200),
"gpsNMEA": (True, 9.),
"deviceState": (True, 2., 1),
"touch": (True, 20., 1),
"can": (True, 100., 2053), # decimation gives ~3 msgs in a full segment
"controlsState": (True, 100., 10),
"selfdriveState": (True, 100., 10),
"pandaStates": (True, 10., 1),
"peripheralState": (True, 2., 1),
"radarState": (True, 20., 5),
"roadEncodeIdx": (False, 20., 1),
"liveTracks": (True, 20.),
"sendcan": (True, 100., 139),
"logMessage": (True, 0.),
"errorLogMessage": (True, 0., 1),
"liveCalibration": (True, 4., 4),
"liveTorqueParameters": (True, 4., 1),
"liveDelay": (True, 4., 1),
"androidLog": (True, 0.),
"carState": (True, 100., 10),
"carControl": (True, 100., 10),
"carOutput": (True, 100., 10),
"longitudinalPlan": (True, 20., 10),
"driverAssistance": (True, 20., 20),
"procLog": (True, 0.5, 15),
"gpsLocationExternal": (True, 10., 10),
"gpsLocation": (True, 1., 1),
"ubloxGnss": (True, 10.),
"qcomGnss": (True, 2.),
"gnssMeasurements": (True, 10., 10),
"clocks": (True, 0.1, 1),
"ubloxRaw": (True, 20.),
"livePose": (True, 20., 4),
"liveParameters": (True, 20., 5),
"cameraOdometry": (True, 20., 10),
"thumbnail": (True, 1 / 60., 1),
"onroadEvents": (True, 1., 1),
"carParams": (True, 0.02, 1),
"roadCameraState": (True, 20., 20),
"driverCameraState": (True, 20., 20),
"driverEncodeIdx": (False, 20., 1),
"driverStateV2": (True, 20., 10),
"driverMonitoringState": (True, 20., 10),
"wideRoadEncodeIdx": (False, 20., 1),
"wideRoadCameraState": (True, 20., 20),
"drivingModelData": (True, 20., 10),
"modelV2": (True, 20.),
"managerState": (True, 2., 1),
"uploaderState": (True, 0., 1),
"navInstruction": (True, 1., 10),
"navRoute": (True, 0.),
"navThumbnail": (True, 0.),
"qRoadEncodeIdx": (False, 20.),
"userFlag": (True, 0., 1),
"microphone": (True, 10., 10),
# debug
"uiDebug": (True, 0., 1),
"testJoystick": (True, 0.),
"alertDebug": (True, 20., 5),
"roadEncodeData": (False, 20.),
"driverEncodeData": (False, 20.),
"wideRoadEncodeData": (False, 20.),
"qRoadEncodeData": (False, 20.),
"livestreamWideRoadEncodeIdx": (False, 20.),
"livestreamRoadEncodeIdx": (False, 20.),
"livestreamDriverEncodeIdx": (False, 20.),
"livestreamWideRoadEncodeData": (False, 20.),
"livestreamRoadEncodeData": (False, 20.),
"livestreamDriverEncodeData": (False, 20.),
"customReservedRawData0": (True, 0.),
"customReservedRawData1": (True, 0.),
"customReservedRawData2": (True, 0.),
}
SERVICE_LIST = {name: Service(*vals) for
idx, (name, vals) in enumerate(_services.items())}
def build_header():
h = ""
h += "/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT services.py */\n"
h += "#ifndef __SERVICES_H\n"
h += "#define __SERVICES_H\n"
h += "#include <map>\n"
h += "#include <string>\n"
h += "struct service { std::string name; bool should_log; int frequency; int decimation; };\n"
h += "static std::map<std::string, service> services = {\n"
for k, v in SERVICE_LIST.items():
should_log = "true" if v.should_log else "false"
decimation = -1 if v.decimation is None else v.decimation
h += ' { "%s", {"%s", %s, %d, %d}},\n' % \
(k, k, should_log, v.frequency, decimation)
h += "};\n"
h += "#endif\n"
return h
if __name__ == "__main__":
print(build_header())
-13
View File
@@ -1,13 +0,0 @@
comment: false
coverage:
status:
project:
default:
informational: true
patch: off
ignore:
- "**/test_*.py"
- "selfdrive/test/**"
- "system/version.py" # codecov changes depending on if we are in a branch or not
- "tools"
-1
View File
@@ -1 +0,0 @@
*.cpp
-39
View File
@@ -1,39 +0,0 @@
Import('env', 'envCython', 'arch')
common_libs = [
'params.cc',
'swaglog.cc',
'util.cc',
'i2c.cc',
'watchdog.cc',
'ratekeeper.cc'
]
if arch != "Darwin":
common_libs.append('gpio.cc')
_common = env.Library('common', common_libs, LIBS="json11")
files = [
'clutil.cc',
]
_gpucommon = env.Library('gpucommon', files)
Export('_common', '_gpucommon')
if GetOption('extras'):
env.Program('tests/test_common',
['tests/test_runner.cc', 'tests/test_params.cc', 'tests/test_util.cc', 'tests/test_swaglog.cc'],
LIBS=[_common, 'json11', 'zmq', 'pthread'])
# Cython bindings
params_python = envCython.Program('params_pyx.so', 'params_pyx.pyx', LIBS=envCython['LIBS'] + [_common, 'zmq', 'json11'])
SConscript([
'transformations/SConscript',
])
Import('transformations_python')
common_python = [params_python, transformations_python]
Export('common_python')
-46
View File
@@ -1,46 +0,0 @@
import jwt
import os
import requests
from datetime import datetime, timedelta, UTC
from openpilot.system.hardware.hw import Paths
from openpilot.system.version import get_version
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
class Api:
def __init__(self, dongle_id):
self.dongle_id = dongle_id
with open(Paths.persist_root()+'/comma/id_rsa') as f:
self.private_key = f.read()
def get(self, *args, **kwargs):
return self.request('GET', *args, **kwargs)
def post(self, *args, **kwargs):
return self.request('POST', *args, **kwargs)
def request(self, method, endpoint, timeout=None, access_token=None, **params):
return api_get(endpoint, method=method, timeout=timeout, access_token=access_token, **params)
def get_token(self, expiry_hours=1):
now = datetime.now(UTC).replace(tzinfo=None)
payload = {
'identity': self.dongle_id,
'nbf': now,
'iat': now,
'exp': now + timedelta(hours=expiry_hours)
}
token = jwt.encode(payload, self.private_key, algorithm='RS256')
if isinstance(token, bytes):
token = token.decode('utf8')
return token
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT " + access_token
headers['User-Agent'] = "openpilot-" + get_version()
return requests.request(method, API_HOST + "/" + endpoint, timeout=timeout, headers=headers, params=params)
+15
View File
@@ -0,0 +1,15 @@
import requests
from selfdrive.version import version
def api_get(endpoint, method='GET', timeout=None, access_token=None, **params):
backend = "https://api.commadotai.com/"
headers = {}
if access_token is not None:
headers['Authorization'] = "JWT "+access_token
headers['User-Agent'] = "openpilot-" + version
return requests.request(method, backend+endpoint, timeout=timeout, headers = headers, params=params)
+2 -2
View File
@@ -1,4 +1,4 @@
import os
BASEDIR = os.path.abspath(os.path.join(os.path.dirname(os.path.realpath(__file__)), "../"))
-98
View File
@@ -1,98 +0,0 @@
#include "common/clutil.h"
#include <cassert>
#include <iostream>
#include <memory>
#include "common/util.h"
#include "common/swaglog.h"
namespace { // helper functions
template <typename Func, typename Id, typename Name>
std::string get_info(Func get_info_func, Id id, Name param_name) {
size_t size = 0;
CL_CHECK(get_info_func(id, param_name, 0, NULL, &size));
std::string info(size, '\0');
CL_CHECK(get_info_func(id, param_name, size, info.data(), NULL));
return info;
}
inline std::string get_platform_info(cl_platform_id id, cl_platform_info name) { return get_info(&clGetPlatformInfo, id, name); }
inline std::string get_device_info(cl_device_id id, cl_device_info name) { return get_info(&clGetDeviceInfo, id, name); }
void cl_print_info(cl_platform_id platform, cl_device_id device) {
size_t work_group_size = 0;
cl_device_type device_type = 0;
clGetDeviceInfo(device, CL_DEVICE_MAX_WORK_GROUP_SIZE, sizeof(work_group_size), &work_group_size, NULL);
clGetDeviceInfo(device, CL_DEVICE_TYPE, sizeof(device_type), &device_type, NULL);
const char *type_str = "Other...";
switch (device_type) {
case CL_DEVICE_TYPE_CPU: type_str ="CL_DEVICE_TYPE_CPU"; break;
case CL_DEVICE_TYPE_GPU: type_str = "CL_DEVICE_TYPE_GPU"; break;
case CL_DEVICE_TYPE_ACCELERATOR: type_str = "CL_DEVICE_TYPE_ACCELERATOR"; break;
}
LOGD("vendor: %s", get_platform_info(platform, CL_PLATFORM_VENDOR).c_str());
LOGD("platform version: %s", get_platform_info(platform, CL_PLATFORM_VERSION).c_str());
LOGD("profile: %s", get_platform_info(platform, CL_PLATFORM_PROFILE).c_str());
LOGD("extensions: %s", get_platform_info(platform, CL_PLATFORM_EXTENSIONS).c_str());
LOGD("name: %s", get_device_info(device, CL_DEVICE_NAME).c_str());
LOGD("device version: %s", get_device_info(device, CL_DEVICE_VERSION).c_str());
LOGD("max work group size: %zu", work_group_size);
LOGD("type = %d, %s", (int)device_type, type_str);
}
void cl_print_build_errors(cl_program program, cl_device_id device) {
cl_build_status status;
clGetProgramBuildInfo(program, device, CL_PROGRAM_BUILD_STATUS, sizeof(status), &status, NULL);
size_t log_size;
clGetProgramBuildInfo(program, device, CL_PROGRAM_BUILD_LOG, 0, NULL, &log_size);
std::string log(log_size, '\0');
clGetProgramBuildInfo(program, device, CL_PROGRAM_BUILD_LOG, log_size, &log[0], NULL);
LOGE("build failed; status=%d, log: %s", status, log.c_str());
}
} // namespace
cl_device_id cl_get_device_id(cl_device_type device_type) {
cl_uint num_platforms = 0;
CL_CHECK(clGetPlatformIDs(0, NULL, &num_platforms));
std::unique_ptr<cl_platform_id[]> platform_ids = std::make_unique<cl_platform_id[]>(num_platforms);
CL_CHECK(clGetPlatformIDs(num_platforms, &platform_ids[0], NULL));
for (size_t i = 0; i < num_platforms; ++i) {
LOGD("platform[%zu] CL_PLATFORM_NAME: %s", i, get_platform_info(platform_ids[i], CL_PLATFORM_NAME).c_str());
// Get first device
if (cl_device_id device_id = NULL; clGetDeviceIDs(platform_ids[i], device_type, 1, &device_id, NULL) == 0 && device_id) {
cl_print_info(platform_ids[i], device_id);
return device_id;
}
}
LOGE("No valid openCL platform found");
assert(0);
return nullptr;
}
cl_context cl_create_context(cl_device_id device_id) {
return CL_CHECK_ERR(clCreateContext(NULL, 1, &device_id, NULL, NULL, &err));
}
void cl_release_context(cl_context context) {
clReleaseContext(context);
}
cl_program cl_program_from_file(cl_context ctx, cl_device_id device_id, const char* path, const char* args) {
return cl_program_from_source(ctx, device_id, util::read_file(path), args);
}
cl_program cl_program_from_source(cl_context ctx, cl_device_id device_id, const std::string& src, const char* args) {
const char *csrc = src.c_str();
cl_program prg = CL_CHECK_ERR(clCreateProgramWithSource(ctx, 1, &csrc, NULL, &err));
if (int err = clBuildProgram(prg, 1, &device_id, args, NULL, NULL); err != 0) {
cl_print_build_errors(prg, device_id);
assert(0);
}
return prg;
}
-28
View File
@@ -1,28 +0,0 @@
#pragma once
#ifdef __APPLE__
#include <OpenCL/cl.h>
#else
#include <CL/cl.h>
#endif
#include <string>
#define CL_CHECK(_expr) \
do { \
assert(CL_SUCCESS == (_expr)); \
} while (0)
#define CL_CHECK_ERR(_expr) \
({ \
cl_int err = CL_INVALID_VALUE; \
__typeof__(_expr) _ret = _expr; \
assert(_ret&& err == CL_SUCCESS); \
_ret; \
})
cl_device_id cl_get_device_id(cl_device_type device_type);
cl_context cl_create_context(cl_device_id device_id);
void cl_release_context(cl_context context);
cl_program cl_program_from_source(cl_context ctx, cl_device_id device_id, const std::string& src, const char* args = nullptr);
cl_program cl_program_from_file(cl_context ctx, cl_device_id device_id, const char* path, const char* args);
-19
View File
@@ -1,19 +0,0 @@
import numpy as np
class Conversions:
# Speed
MPH_TO_KPH = 1.609344
KPH_TO_MPH = 1. / MPH_TO_KPH
MS_TO_KPH = 3.6
KPH_TO_MS = 1. / MS_TO_KPH
MS_TO_MPH = MS_TO_KPH * KPH_TO_MPH
MPH_TO_MS = MPH_TO_KPH * KPH_TO_MS
MS_TO_KNOTS = 1.9438
KNOTS_TO_MS = 1. / MS_TO_KNOTS
# Angle
DEG_TO_RAD = np.pi / 180.
RAD_TO_DEG = 1. / DEG_TO_RAD
# Mass
LB_TO_KG = 0.453592
Executable
+220
View File
@@ -0,0 +1,220 @@
import re
import os
import struct
import bitstring
import sys
import numbers
from collections import namedtuple
def int_or_float(s):
# return number, trying to maintain int format
try:
return int(s)
except ValueError:
return float(s)
DBCSignal = namedtuple(
"DBCSignal", ["name", "start_bit", "size", "is_little_endian", "is_signed",
"factor", "offset", "tmin", "tmax", "units"])
class dbc(object):
def __init__(self, fn):
self.name, _ = os.path.splitext(os.path.basename(fn))
with open(fn) as f:
self.txt = f.read().split("\n")
self._warned_addresses = set()
# regexps from https://github.com/ebroecker/canmatrix/blob/master/canmatrix/importdbc.py
bo_regexp = re.compile("^BO\_ (\w+) (\w+) *: (\w+) (\w+)")
sg_regexp = re.compile("^SG\_ (\w+) : (\d+)\|(\d+)@(\d+)([\+|\-]) \(([0-9.+\-eE]+),([0-9.+\-eE]+)\) \[([0-9.+\-eE]+)\|([0-9.+\-eE]+)\] \"(.*)\" (.*)")
sgm_regexp = re.compile("^SG\_ (\w+) (\w+) *: (\d+)\|(\d+)@(\d+)([\+|\-]) \(([0-9.+\-eE]+),([0-9.+\-eE]+)\) \[([0-9.+\-eE]+)\|([0-9.+\-eE]+)\] \"(.*)\" (.*)")
# A dictionary which maps message ids to tuples ((name, size), signals).
# name is the ASCII name of the message.
# size is the size of the message in bytes.
# signals is a list signals contained in the message.
# signals is a list of DBCSignal in order of increasing start_bit.
self.msgs = {}
# lookup to bit reverse each byte
self.bits_index = [(i & ~0b111) + ((-i-1) & 0b111) for i in xrange(64)]
for l in self.txt:
l = l.strip()
if l.startswith("BO_ "):
# new group
dat = bo_regexp.match(l)
if dat is None:
print "bad BO", l
name = dat.group(2)
size = int(dat.group(3))
ids = int(dat.group(1), 0) # could be hex
if ids in self.msgs:
sys.exit("Duplicate address detected %d %s" % (ids, self.name))
self.msgs[ids] = ((name, size), [])
if l.startswith("SG_ "):
# new signal
dat = sg_regexp.match(l)
go = 0
if dat is None:
dat = sgm_regexp.match(l)
go = 1
if dat is None:
print "bad SG", l
sgname = dat.group(1)
start_bit = int(dat.group(go+2))
signal_size = int(dat.group(go+3))
is_little_endian = int(dat.group(go+4))==1
is_signed = dat.group(go+5)=='-'
factor = int_or_float(dat.group(go+6))
offset = int_or_float(dat.group(go+7))
tmin = int_or_float(dat.group(go+8))
tmax = int_or_float(dat.group(go+9))
units = dat.group(go+10)
self.msgs[ids][1].append(
DBCSignal(sgname, start_bit, signal_size, is_little_endian,
is_signed, factor, offset, tmin, tmax, units))
for msg in self.msgs.viewvalues():
msg[1].sort(key=lambda x: x.start_bit)
self.msg_name_to_address = {}
for address, m in self.msgs.items():
name = m[0][0]
self.msg_name_to_address[name] = address
def lookup_msg_id(self, msg_id):
if not isinstance(msg_id, numbers.Number):
msg_id = self.msg_name_to_address[msg_id]
return msg_id
def encode(self, msg_id, dd):
"""Encode a CAN message using the dbc.
Inputs:
msg_id: The message ID.
dd: A dictionary mapping signal name to signal data.
"""
msg_id = self.lookup_msg_id(msg_id)
# TODO: Stop using bitstring, which is super slow.
msg_def = self.msgs[msg_id]
size = msg_def[0][1]
bsf = bitstring.Bits(hex="00"*size)
for s in msg_def[1]:
ival = dd.get(s.name)
if ival is not None:
ival = (ival / s.factor) - s.offset
ival = int(round(ival))
# should pack this
if s.is_little_endian:
ss = s.start_bit
else:
ss = self.bits_index[s.start_bit]
if s.is_signed:
tbs = bitstring.Bits(int=ival, length=s.size)
else:
tbs = bitstring.Bits(uint=ival, length=s.size)
lpad = bitstring.Bits(bin="0b"+"0"*ss)
rpad = bitstring.Bits(bin="0b"+"0"*(8*size-(ss+s.size)))
tbs = lpad+tbs+rpad
bsf |= tbs
return bsf.tobytes()
def decode(self, x, arr=None, debug=False):
"""Decode a CAN message using the dbc.
Inputs:
x: A collection with elements (address, time, data), where address is
the CAN address, time is the bus time, and data is the CAN data as a
hex string.
arr: Optional list of signals which should be decoded and returned.
debug: True to print debugging statements.
Returns:
A tuple (name, data), where name is the name of the CAN message and data
is the decoded result. If arr is None, data is a dict of properties.
Otherwise data is a list of the same length as arr.
Returns (None, None) if the message could not be decoded.
"""
if arr is None:
out = {}
else:
out = [None]*len(arr)
msg = self.msgs.get(x[0])
if msg is None:
if x[0] not in self._warned_addresses:
#print("WARNING: Unknown message address {}".format(x[0]))
self._warned_addresses.add(x[0])
return None, None
name = msg[0][0]
if debug:
print name
blen = 8*len(x[2])
st = x[2].rjust(8, '\x00')
le, be = None, None
for s in msg[1]:
if arr is not None and s[0] not in arr:
continue
# big or little endian?
# see http://vi-firmware.openxcplatform.com/en/master/config/bit-numbering.html
if s[3] is False:
ss = self.bits_index[s[1]]
if be is None:
be = struct.unpack(">Q", st)[0]
x2_int = be
data_bit_pos = (blen - (ss + s[2]))
else:
if le is None:
le = struct.unpack("<Q", st)[0]
x2_int = le
ss = s[1]
data_bit_pos = ss
if data_bit_pos < 0:
continue
ival = (x2_int >> data_bit_pos) & ((1 << (s[2])) - 1)
if s[4] and (ival & (1<<(s[2]-1))): # signed
ival -= (1<<s[2])
# control the offset
ival = (ival * s[5]) + s[6]
#if debug:
# print "%40s %2d %2d %7.2f %s" % (s[0], s[1], s[2], ival, s[-1])
if arr is None:
out[s[0]] = ival
else:
out[arr.index(s[0])] = ival
return name, out
def get_signals(self, msg):
msg = self.lookup_msg_id(msg)
return [sgs.name for sgs in self.msgs[msg][1]]
if __name__ == "__main__":
from opendbc import DBC_PATH
dbc_test = dbc(os.path.join(DBC_PATH, sys.argv[1]))
print dbc_test.get_signals(0xe4)
-9
View File
@@ -1,9 +0,0 @@
# remove all keys that end in DEPRECATED
def strip_deprecated_keys(d):
for k in list(d.keys()):
if isinstance(k, str):
if k.endswith('DEPRECATED'):
d.pop(k)
elif isinstance(d[k], dict):
strip_deprecated_keys(d[k])
return d
-58
View File
@@ -1,58 +0,0 @@
import io
import os
import tempfile
import contextlib
import zstandard as zstd
LOG_COMPRESSION_LEVEL = 10 # little benefit up to level 15. level ~17 is a small step change
class CallbackReader:
"""Wraps a file, but overrides the read method to also
call a callback function with the number of bytes read so far."""
def __init__(self, f, callback, *args):
self.f = f
self.callback = callback
self.cb_args = args
self.total_read = 0
def __getattr__(self, attr):
return getattr(self.f, attr)
def read(self, *args, **kwargs):
chunk = self.f.read(*args, **kwargs)
self.total_read += len(chunk)
self.callback(*self.cb_args, self.total_read)
return chunk
@contextlib.contextmanager
def atomic_write_in_dir(path: str, mode: str = 'w', buffering: int = -1, encoding: str = None, newline: str = None,
overwrite: bool = False):
"""Write to a file atomically using a temporary file in the same directory as the destination file."""
dir_name = os.path.dirname(path)
if not overwrite and os.path.exists(path):
raise FileExistsError(f"File '{path}' already exists. To overwrite it, set 'overwrite' to True.")
with tempfile.NamedTemporaryFile(mode=mode, buffering=buffering, encoding=encoding, newline=newline, dir=dir_name, delete=False) as tmp_file:
yield tmp_file
tmp_file_name = tmp_file.name
os.replace(tmp_file_name, path)
def get_upload_stream(filepath: str, should_compress: bool) -> tuple[io.BufferedIOBase, int]:
if not should_compress:
file_size = os.path.getsize(filepath)
file_stream = open(filepath, "rb")
return file_stream, file_size
# Compress the file on the fly
compressed_stream = io.BytesIO()
compressor = zstd.ZstdCompressor(level=LOG_COMPRESSION_LEVEL)
with open(filepath, "rb") as f:
compressor.copy_stream(f, compressed_stream)
compressed_size = compressed_stream.tell()
compressed_stream.seek(0)
return compressed_stream, compressed_size
-17
View File
@@ -1,17 +0,0 @@
class FirstOrderFilter:
def __init__(self, x0, rc, dt, initialized=True):
self.x = x0
self.dt = dt
self.update_alpha(rc)
self.initialized = initialized
def update_alpha(self, rc):
self.alpha = self.dt / (rc + self.dt)
def update(self, x):
if self.initialized:
self.x = (1. - self.alpha) * self.x + self.alpha * x
else:
self.initialized = True
self.x = x
return self.x
+104
View File
@@ -0,0 +1,104 @@
class HONDA:
CIVIC = "HONDA CIVIC 2016 TOURING"
ACURA_ILX = "ACURA ILX 2016 ACURAWATCH PLUS"
CRV = "HONDA CR-V 2016 TOURING"
ODYSSEY = "HONDA ODYSSEY 2018 EX-L"
ACURA_RDX = "ACURA RDX 2018 ACURAWATCH PLUS"
class TOYOTA:
PRIUS = "TOYOTA PRIUS 2017"
RAV4H = "TOYOTA RAV4 2017 HYBRID"
RAV4 = "TOYOTA RAV4 2017"
COROLLA = "TOYOTA COROLLA 2017"
LEXUS_RXH = "LEXUS RX HYBRID 2017"
_FINGERPRINTS = {
HONDA.ACURA_ILX: {
1024L: 5, 513L: 5, 1027L: 5, 1029L: 8, 929L: 4, 1057L: 5, 777L: 8, 1034L: 5, 1036L: 8, 398L: 3, 399L: 7, 145L: 8, 660L: 8, 985L: 3, 923L: 2, 542L: 7, 773L: 7, 800L: 8, 432L: 7, 419L: 8, 420L: 8, 1030L: 5, 422L: 8, 808L: 8, 428L: 8, 304L: 8, 819L: 7, 821L: 5, 57L: 3, 316L: 8, 545L: 4, 464L: 8, 1108L: 8, 597L: 8, 342L: 6, 983L: 8, 344L: 8, 804L: 8, 1039L: 8, 476L: 4, 892L: 8, 490L: 8, 1064L: 7, 882L: 2, 884L: 7, 887L: 8, 888L: 8, 380L: 8, 1365L: 5,
# sent messages
0xe4: 5, 0x1fa: 8, 0x200: 3, 0x30c: 8, 0x33d: 5,
},
HONDA.ACURA_RDX: {
57L: 3, 145L: 8, 229L: 4, 308L: 5, 316L: 8, 342L: 6, 344L: 8, 380L: 8, 392L: 6, 398L: 3, 399L: 6, 404L: 4, 420L: 8, 422L: 8, 426L: 8, 432L: 7, 464L: 8, 474L: 5, 476L: 4, 487L: 4, 490L: 8, 506L: 8, 542L: 7, 545L: 4, 597L: 8, 660L: 8, 773L: 7, 777L: 8, 780L: 8, 800L: 8, 804L: 8, 808L: 8, 819L: 7, 821L: 5, 829L: 5, 882L: 2, 884L: 7, 887L: 8, 888L: 8, 892L: 8, 923L: 2, 929L: 4, 963L: 8, 965L: 8, 966L: 8, 967L: 8, 983L: 8, 985L: 3, 1024L: 5, 1027L: 5, 1029L: 8, 1033L: 5, 1034L: 5, 1036L: 8, 1039L: 8, 1057L: 5, 1064L: 7, 1108L: 8, 1365L: 5, 1424L: 5, 1729L: 1
},
HONDA.CIVIC: {
1024L: 5, 513L: 5, 1027L: 5, 1029L: 8, 777L: 8, 1036L: 8, 1039L: 8, 1424L: 5, 401L: 8, 148L: 8, 662L: 4, 985L: 3, 795L: 8, 773L: 7, 800L: 8, 545L: 6, 420L: 8, 806L: 8, 808L: 8, 1322L: 5, 427L: 3, 428L: 8, 304L: 8, 432L: 7, 57L: 3, 450L: 8, 929L: 8, 330L: 8, 1302L: 8, 464L: 8, 1361L: 5, 1108L: 8, 597L: 8, 470L: 2, 344L: 8, 804L: 8, 399L: 7, 476L: 7, 1633L: 8, 487L: 4, 892L: 8, 490L: 8, 493L: 5, 884L: 8, 891L: 8, 380L: 8, 1365L: 5,
# sent messages
0xe4: 5, 0x1fa: 8, 0x200: 3, 0x30c: 8, 0x33d: 5, 0x35e: 8, 0x39f: 8,
},
HONDA.CRV: {
57L: 3, 145L: 8, 316L: 8, 340L: 8, 342L: 6, 344L: 8, 380L: 8, 398L: 3, 399L: 6, 401L: 8, 420L: 8, 422L: 8, 426L: 8, 432L: 7, 464L: 8, 474L: 5, 476L: 4, 487L: 4, 490L: 8, 493L: 3, 507L: 1, 542L: 7, 545L: 4, 597L: 8, 660L: 8, 661L: 4, 773L: 7, 777L: 8, 800L: 8, 804L: 8, 808L: 8, 882L: 2, 884L: 7, 888L: 8, 891L: 8, 892L: 8, 923L: 2, 929L: 8, 983L: 8, 985L: 3, 1024L: 5, 1027L: 5, 1029L: 8, 1033L: 5, 1036L: 8, 1039L: 8, 1057L: 5, 1064L: 7, 1108L: 8, 1125L: 8, 1296L: 8, 1365L: 5, 1424L: 5, 1600L: 5, 1601L: 8,
# sent messages
0x194: 4, 0x1fa: 8, 0x30c: 8, 0x33d: 5,
},
HONDA.ODYSSEY: {
57L: 3, 148L: 8, 228L: 5, 229L: 4, 316L: 8, 342L: 6, 344L: 8, 380L: 8, 399L: 7, 411L: 5, 419L: 8, 420L: 8, 427L: 3, 432L: 7, 450L: 8, 463L: 8, 464L: 8, 476L: 4, 490L: 8, 506L: 8, 542L: 7, 545L: 6, 597L: 8, 662L: 4, 773L: 7, 777L: 8, 780L: 8, 795L: 8, 800L: 8, 804L: 8, 806L: 8, 808L: 8, 817L: 4, 819L: 7, 821L: 5, 825L: 4, 829L: 5, 837L: 5, 856L: 7, 862L: 8, 871L: 8, 881L: 8, 882L: 4, 884L: 8, 891L: 8, 892L: 8, 905L: 8, 923L: 2, 927L: 8, 929L: 8, 963L: 8, 965L: 8, 966L: 8, 967L: 8, 983L: 8, 985L: 3, 1029L: 8, 1036L: 8, 1052L: 8, 1064L: 7, 1088L: 8, 1089L: 8, 1092L: 1, 1108L: 8, 1110L: 8, 1125L: 8, 1296L: 8, 1302L: 8, 1600L: 5, 1601L: 8, 1612L: 5, 1613L: 5, 1614L: 5, 1615L: 8, 1616L: 5, 1619L: 5, 1623L: 5, 1668L: 5
},
TOYOTA.RAV4: {
36L: 8, 37L: 8, 170L: 8, 180L: 8, 186L: 4, 426L: 6, 452L: 8, 464L: 8, 466L: 8, 467L: 8, 547L: 8, 548L: 8, 552L: 4, 562L: 4, 608L: 8, 610L: 5, 643L: 7, 705L: 8, 725L: 2, 740L: 5, 800L: 8, 835L: 8, 836L: 8, 849L: 4, 869L: 7, 870L: 7, 871L: 2, 896L: 8, 897L: 8, 900L: 6, 902L: 6, 905L: 8, 911L: 8, 916L: 3, 918L: 7, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 951L: 8, 955L: 4, 956L: 8, 979L: 2, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1005L: 2, 1008L: 2, 1014L: 8, 1017L: 8, 1041L: 8, 1042L: 8, 1043L: 8, 1044L: 8, 1056L: 8, 1059L: 1, 1114L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1176L: 8, 1177L: 8, 1178L: 8, 1179L: 8, 1180L: 8, 1181L: 8, 1190L: 8, 1191L: 8, 1192L: 8, 1196L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1263L: 8, 1264L: 8, 1279L: 8, 1408L: 8, 1409L: 8, 1410L: 8, 1552L: 8, 1553L: 8, 1554L: 8, 1555L: 8, 1556L: 8, 1557L: 8, 1561L: 8, 1562L: 8, 1568L: 8, 1569L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1584L: 8, 1589L: 8, 1592L: 8, 1593L: 8, 1595L: 8, 1596L: 8, 1597L: 8, 1600L: 8, 1656L: 8, 1664L: 8, 1728L: 8, 1745L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8
},
TOYOTA.RAV4H: {
36L: 8, 37L: 8, 170L: 8, 180L: 8, 186L: 4, 426L: 6, 452L: 8, 464L: 8, 466L: 8, 467L: 8, 547L: 8, 548L: 8, 552L: 4, 562L: 4, 608L: 8, 610L: 5, 643L: 7, 705L: 8, 725L: 2, 740L: 5, 800L: 8, 835L: 8, 836L: 8, 849L: 4, 869L: 7, 870L: 7, 871L: 2, 896L: 8, 897L: 8, 900L: 6, 902L: 6, 905L: 8, 911L: 8, 916L: 3, 918L: 7, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 951L: 8, 955L: 8, 956L: 8, 979L: 2, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1008L: 2, 1014L: 8, 1017L: 8, 1041L: 8, 1042L: 8, 1043L: 8, 1044L: 8, 1056L: 8, 1059L: 1, 1114L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1176L: 8, 1177L: 8, 1178L: 8, 1179L: 8, 1180L: 8, 1181L: 8, 1190L: 8, 1191L: 8, 1192L: 8, 1196L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1263L: 8, 1279L: 8, 1408L: 8, 1409L: 8, 1410L: 8, 1552L: 8, 1553L: 8, 1554L: 8, 1555L: 8, 1556L: 8, 1557L: 8, 1561L: 8, 1562L: 8, 1568L: 8, 1569L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1584L: 8, 1589L: 8, 1592L: 8, 1593L: 8, 1595L: 8, 1596L: 8, 1597L: 8, 1600L: 8, 1656L: 8, 1664L: 8, 1728L: 8, 1745L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8, 581L: 5, 296: 8, 560L: 7, 713L: 8, 550L: 8, 608L: 8, 37L: 8, 36L: 8, 950L: 8, 1198L: 8, 1197L: 8, 1199L: 8, 1212L: 8, 953L: 3, 1264L: 8, 1184L: 8, 1005L: 2, 1185L: 8, 1232L: 8, 1186L: 8
},
TOYOTA.PRIUS: [{
36L: 8, 37L: 8, 166L: 8, 170L: 8, 180L: 8, 295L: 8, 296L: 8, 426L: 6, 452L: 8, 466L: 8, 467L: 8, 550L: 8, 552L: 4, 560L: 7, 562L: 6, 581L: 5, 608L: 8, 610L: 8, 614L: 8, 643L: 7, 658L: 8, 713L: 8, 740L: 5, 742L: 8, 743L: 8, 800L: 8, 810L: 2, 814L: 8, 829L: 2, 830L: 7, 835L: 8, 836L: 8, 863L: 8, 869L: 7, 870L: 7, 871L: 2, 898L: 8, 900L: 6, 902L: 6, 905L: 8, 918L: 8, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 950L: 8, 951L: 8, 953L: 8, 955L: 8, 956L: 8, 971L: 7, 975L: 5, 993L: 8, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1014L: 8, 1017L: 8, 1020L: 8, 1041L: 8, 1042L: 8, 1044L: 8, 1056L: 8, 1057L: 8, 1059L: 1, 1071L: 8, 1077L: 8, 1082L: 8, 1083L: 8, 1084L: 8, 1085L: 8, 1086L: 8, 1114L: 8, 1132L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1175L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1279L: 8, 1552L: 8, 1553L: 8, 1556L: 8, 1557L: 8, 1568L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1595L: 8, 1777L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8
},
# Prius Prime
{
36L: 8, 37L: 8, 166L: 8, 170L: 8, 180L: 8, 295L: 8, 296L: 8, 426L: 6, 452L: 8, 466L: 8, 467L: 8, 550L: 8, 552L: 4, 560L: 7, 562L: 6, 581L: 5, 608L: 8, 610L: 8, 614L: 8, 643L: 7, 658L: 8, 713L: 8, 740L: 5, 742L: 8, 743L: 8, 800L: 8, 810L: 2, 814L: 8, 824L: 2, 829L: 2, 830L: 7, 835L: 8, 836L: 8, 863L: 8, 869L: 7, 870L: 7, 871L: 2,898L: 8, 900L: 6, 902L: 6, 905L: 8, 913L: 8, 918L: 8, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 950L: 8, 951L: 8, 953L: 8, 955L: 8, 956L: 8, 971L: 7, 974L: 8, 975L: 5, 993L: 8, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1014L: 8, 1017L: 8, 1020L: 8, 1041L: 8, 1042L: 8, 1044L: 8, 1056L: 8, 1057L: 8, 1059L: 1, 1071L: 8, 1076L: 8, 1077L: 8, 1082L: 8, 1083L: 8, 1084L: 8, 1085L: 8, 1086L: 8, 1114L: 8, 1132L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1164L: 8, 1165L: 8, 1166L: 8, 1167L: 8, 1175L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1279L: 8, 1552L: 8, 1553L: 8, 1556L: 8, 1557L: 8, 1568L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1595L: 8, 1777L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8
},
# Taiwanese Prius Prime
{
36L: 8, 37L: 8, 166L: 8, 170L: 8, 180L: 8, 295L: 8, 296L: 8, 426L: 6, 452L: 8, 466L: 8, 467L: 8, 550L: 8, 552L: 4, 560L: 7, 562L: 6, 581L: 5, 608L: 8, 610L: 8, 614L: 8, 643L: 7, 658L: 8, 713L: 8, 740L: 5, 742L: 8, 743L: 8, 800L: 8, 810L: 2, 814L: 8, 824L: 2, 829L: 2, 830L: 7, 835L: 8, 836L: 8, 845L: 5, 863L: 8, 869L: 7, 870L: 7, 871L: 2,898L: 8, 900L: 6, 902L: 6, 905L: 8, 913L: 8, 918L: 8, 921L: 8, 933L: 8, 944L: 8, 945L: 8, 950L: 8, 951L: 8, 953L: 8, 955L: 8, 956L: 8, 971L: 7, 974L: 8, 975L: 5, 993L: 8, 998L: 5, 999L: 7, 1000L: 8, 1001L: 8, 1005L: 2, 1014L: 8, 1017L: 8, 1020L: 8, 1041L: 8, 1042L: 8, 1044L: 8, 1056L: 8, 1057L: 8, 1059L: 1, 1071L: 8, 1076L: 8, 1077L: 8, 1082L: 8, 1083L: 8, 1084L: 8, 1085L: 8, 1086L: 8, 1114L: 8, 1132L: 8, 1161L: 8, 1162L: 8, 1163L: 8, 1164L: 8, 1165L: 8, 1166L: 8, 1167L: 8, 1175L: 8, 1227L: 8, 1228L: 8, 1235L: 8, 1237L: 8, 1264L: 8, 1279L: 8, 1552L: 8, 1553L: 8, 1556L: 8, 1557L: 8, 1568L: 8, 1570L: 8, 1571L: 8, 1572L: 8, 1595L: 8, 1777L: 8, 1779L: 8, 1904L: 8, 1912L: 8, 1990L: 8, 1998L: 8
}
],
TOYOTA.COROLLA: {
36: 8, 37: 8, 170: 8, 180: 8, 186: 4, 426: 6, 452: 8, 464: 8, 466: 8, 467: 8, 547: 8, 548: 8, 552: 4, 608: 8, 610: 5, 643: 7, 705: 8, 740: 5, 800: 8, 835: 8, 836: 8, 849: 4, 869: 7, 870: 7, 871: 2, 896: 8, 897: 8, 900: 6, 902: 6, 905: 8, 911: 8, 916: 2, 921: 8, 933: 8, 944: 8, 945: 8, 951: 8, 955: 4, 956: 8, 979: 2, 992: 8, 998: 5, 999: 7, 1000: 8, 1001: 8, 1017: 8, 1041: 8, 1042: 8, 1043: 8, 1044: 8, 1056: 8, 1059: 1, 1114: 8, 1161: 8, 1162: 8, 1163: 8, 1196: 8, 1227: 8, 1235: 8, 1279: 8, 1552: 8, 1553: 8, 1556: 8, 1557: 8, 1561: 8, 1562: 8, 1568: 8, 1569: 8, 1570: 8, 1571: 8, 1572: 8, 1584: 8, 1589: 8, 1592: 8, 1596: 8, 1597: 8, 1600: 8, 1664: 8, 1728: 8, 1779: 8, 1904: 8, 1912: 8, 1990: 8, 1998: 8
},
TOYOTA.LEXUS_RXH: {
36: 8, 37: 8, 166: 8, 170: 8, 180: 8, 295: 8, 296: 8, 426: 6, 452: 8, 466: 8, 467: 8, 550: 8, 552: 4, 560: 7, 562: 6, 581: 5, 608: 8, 610: 5, 643: 7, 658: 8, 713: 8, 740: 5, 742: 8, 743: 8, 800: 8, 810: 2, 812: 3, 814: 8, 830: 7, 835: 8, 836: 8, 845: 5, 863: 8, 869: 7, 870: 7, 871: 2, 898: 8, 900: 6, 902: 6, 905: 8, 913: 8, 918: 8, 921: 8, 933: 8, 944: 8, 945: 8, 950: 8, 951: 8, 953: 8, 955: 8, 956: 8, 971: 7, 975: 6, 993: 8, 998: 5, 999: 7, 1000: 8, 1001: 8, 1005: 2, 1014: 8, 1017: 8, 1020: 8, 1041: 8, 1042: 8, 1044: 8, 1056: 8, 1059: 1, 1063: 8, 1071: 8, 1077: 8, 1082: 8, 1114: 8, 1161: 8, 1162: 8, 1163: 8, 1164: 8, 1165: 8, 1166: 8, 1167: 8, 1227: 8, 1228: 8, 1235: 8, 1237: 8, 1264: 8, 1279: 8, 1552: 8, 1553: 8, 1556: 8, 1557: 8, 1568: 8, 1570: 8, 1571: 8, 1572: 8, 1575: 8, 1595: 8, 1777: 8, 1779: 8, 1808: 8, 1810: 8, 1816: 8, 1818: 8, 1840: 8, 1848: 8, 1904: 8, 1912: 8, 1940: 8, 1941: 8, 1948: 8, 1949: 8, 1952: 8, 1956: 8, 1960: 8, 1964: 8, 1986: 8, 1990: 8, 1994: 8, 1998: 8, 2004: 8, 2012: 8
},
}
# support additional internal only fingerprints
try:
from common.fingerprints_internal import add_additional_fingerprints
add_additional_fingerprints(_FINGERPRINTS)
except ImportError:
pass
def is_valid_for_fingerprint(msg, car_fingerprint):
adr = msg.address
return msg.src != 0 or (adr in car_fingerprint and car_fingerprint[adr] == len(msg.dat))
def eliminate_incompatible_cars(msg, candidate_cars):
"""Removes cars that could not have sent msg.
Inputs:
msg: A cereal/log CanData message from the car.
candidate_cars: A list of cars to consider.
Returns:
A list containing the subset of candidate_cars that could have sent msg.
"""
compatible_cars = []
for car_name in candidate_cars:
car_fingerprints = _FINGERPRINTS[car_name]
if not isinstance(car_fingerprints, list):
car_fingerprints = [car_fingerprints]
for fingerprint in car_fingerprints:
if is_valid_for_fingerprint(msg, fingerprint):
compatible_cars.append(car_name)
break
return compatible_cars
def all_known_cars():
"""Returns a list of all known car strings."""
return _FINGERPRINTS.keys()
-42
View File
@@ -1,42 +0,0 @@
from functools import cache
import subprocess
from openpilot.common.run import run_cmd, run_cmd_default
@cache
def get_commit(cwd: str = None, branch: str = "HEAD") -> str:
return run_cmd_default(["git", "rev-parse", branch], cwd=cwd)
@cache
def get_commit_date(cwd: str = None, commit: str = "HEAD") -> str:
return run_cmd_default(["git", "show", "--no-patch", "--format='%ct %ci'", commit], cwd=cwd)
@cache
def get_short_branch(cwd: str = None) -> str:
return run_cmd_default(["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=cwd)
@cache
def get_branch(cwd: str = None) -> str:
return run_cmd_default(["git", "rev-parse", "--abbrev-ref", "--symbolic-full-name", "@{u}"], cwd=cwd)
@cache
def get_origin(cwd: str = None) -> str:
try:
local_branch = run_cmd(["git", "name-rev", "--name-only", "HEAD"], cwd=cwd)
tracking_remote = run_cmd(["git", "config", "branch." + local_branch + ".remote"], cwd=cwd)
return run_cmd(["git", "config", "remote." + tracking_remote + ".url"], cwd=cwd)
except subprocess.CalledProcessError: # Not on a branch, fallback
return run_cmd_default(["git", "config", "--get", "remote.origin.url"], cwd=cwd)
@cache
def get_normalized_origin(cwd: str = None) -> str:
return get_origin(cwd) \
.replace("git@", "", 1) \
.replace(".git", "", 1) \
.replace("https://", "", 1) \
.replace(":", "/", 1)
-84
View File
@@ -1,84 +0,0 @@
#include "common/gpio.h"
#include <string>
#ifdef __APPLE__
int gpio_init(int pin_nr, bool output) {
return 0;
}
int gpio_set(int pin_nr, bool high) {
return 0;
}
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr) {
return 0;
}
#else
#include <fcntl.h>
#include <unistd.h>
#include <cstring>
#include <linux/gpio.h>
#include <sys/ioctl.h>
#include "common/util.h"
#include "common/swaglog.h"
int gpio_init(int pin_nr, bool output) {
char pin_dir_path[50];
int pin_dir_path_len = snprintf(pin_dir_path, sizeof(pin_dir_path),
"/sys/class/gpio/gpio%d/direction", pin_nr);
if (pin_dir_path_len <= 0) {
return -1;
}
const char *value = output ? "out" : "in";
return util::write_file(pin_dir_path, (void*)value, strlen(value));
}
int gpio_set(int pin_nr, bool high) {
char pin_val_path[50];
int pin_val_path_len = snprintf(pin_val_path, sizeof(pin_val_path),
"/sys/class/gpio/gpio%d/value", pin_nr);
if (pin_val_path_len <= 0) {
return -1;
}
return util::write_file(pin_val_path, (void*)(high ? "1" : "0"), 1);
}
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr) {
// Assumed that all interrupt pins are unexported and rights are given to
// read from gpiochip0.
std::string gpiochip_path = "/dev/gpiochip" + std::to_string(gpiochiop_id);
int fd = open(gpiochip_path.c_str(), O_RDONLY);
if (fd < 0) {
LOGE("Error opening gpiochip0 fd");
return -1;
}
// Setup event
struct gpioevent_request rq;
rq.lineoffset = pin_nr;
rq.handleflags = GPIOHANDLE_REQUEST_INPUT;
/* Requesting both edges as the data ready pulse from the lsm6ds sensor is
very short(75us) and is mostly detected as falling edge instead of rising.
So if it is detected as rising the following falling edge is skipped. */
rq.eventflags = GPIOEVENT_REQUEST_BOTH_EDGES;
strncpy(rq.consumer_label, consumer_label, std::size(rq.consumer_label) - 1);
int ret = util::safe_ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, &rq);
if (ret == -1) {
LOGE("Unable to get line event from ioctl : %s", strerror(errno));
close(fd);
return -1;
}
close(fd);
return rq.fd;
}
#endif
-33
View File
@@ -1,33 +0,0 @@
#pragma once
// Pin definitions
#ifdef QCOM2
#define GPIO_HUB_RST_N 30
#define GPIO_UBLOX_RST_N 32
#define GPIO_UBLOX_SAFEBOOT_N 33
#define GPIO_GNSS_PWR_EN 34 /* SCHEMATIC LABEL: GPIO_UBLOX_PWR_EN */
#define GPIO_STM_RST_N 124
#define GPIO_STM_BOOT0 134
#define GPIO_BMX_ACCEL_INT 21
#define GPIO_BMX_GYRO_INT 23
#define GPIO_BMX_MAGN_INT 87
#define GPIO_LSM_INT 84
#define GPIOCHIP_INT 0
#else
#define GPIO_HUB_RST_N 0
#define GPIO_UBLOX_RST_N 0
#define GPIO_UBLOX_SAFEBOOT_N 0
#define GPIO_GNSS_PWR_EN 0 /* SCHEMATIC LABEL: GPIO_UBLOX_PWR_EN */
#define GPIO_STM_RST_N 0
#define GPIO_STM_BOOT0 0
#define GPIO_BMX_ACCEL_INT 0
#define GPIO_BMX_GYRO_INT 0
#define GPIO_BMX_MAGN_INT 0
#define GPIO_LSM_INT 0
#define GPIOCHIP_INT 0
#endif
int gpio_init(int pin_nr, bool output);
int gpio_set(int pin_nr, bool high);
int gpiochip_get_ro_value_fd(const char* consumer_label, int gpiochiop_id, int pin_nr);
-54
View File
@@ -1,54 +0,0 @@
import os
from functools import cache
def gpio_init(pin: int, output: bool) -> None:
try:
with open(f"/sys/class/gpio/gpio{pin}/direction", 'wb') as f:
f.write(b"out" if output else b"in")
except Exception as e:
print(f"Failed to set gpio {pin} direction: {e}")
def gpio_set(pin: int, high: bool) -> None:
try:
with open(f"/sys/class/gpio/gpio{pin}/value", 'wb') as f:
f.write(b"1" if high else b"0")
except Exception as e:
print(f"Failed to set gpio {pin} value: {e}")
def gpio_read(pin: int) -> bool | None:
val = None
try:
with open(f"/sys/class/gpio/gpio{pin}/value", 'rb') as f:
val = bool(int(f.read().strip()))
except Exception as e:
print(f"Failed to set gpio {pin} value: {e}")
return val
def gpio_export(pin: int) -> None:
if os.path.isdir(f"/sys/class/gpio/gpio{pin}"):
return
try:
with open("/sys/class/gpio/export", 'w') as f:
f.write(str(pin))
except Exception:
print(f"Failed to export gpio {pin}")
@cache
def get_irq_action(irq: int) -> list[str]:
try:
with open(f"/sys/kernel/irq/{irq}/actions") as f:
actions = f.read().strip().split(',')
return actions
except FileNotFoundError:
return []
def get_irqs_for_action(action: str) -> list[str]:
ret = []
with open("/proc/interrupts") as f:
for l in f.readlines():
irq = l.split(':')[0].strip()
if irq.isdigit() and action in get_irq_action(irq):
ret.append(irq)
return ret
-8
View File
@@ -1,8 +0,0 @@
from openpilot.common.params import Params
def get_gps_location_service(params: Params) -> str:
if params.get_bool("UbloxAvailable"):
return "gpsLocationExternal"
else:
return "gpsLocation"

Some files were not shown because too many files have changed in this diff Show More