mirror of
https://github.com/sunnypilot/sunnypilot.git
synced 2026-08-23 02:23:47 +08:00
sunnypilot v2026.003.000 release
date: 2026-08-19T09:43:43
master commit: ba29a38507
This commit is contained in:
github-actions[bot]
@@ -0,0 +1,15 @@
|
||||
brew "git-lfs"
|
||||
brew "capnp"
|
||||
brew "coreutils"
|
||||
brew "eigen"
|
||||
brew "ffmpeg"
|
||||
brew "glfw"
|
||||
brew "libusb"
|
||||
brew "libtool"
|
||||
brew "llvm"
|
||||
brew "openssl@3.0"
|
||||
brew "qt@5"
|
||||
brew "zeromq"
|
||||
cask "gcc-arm-embedded"
|
||||
brew "portaudio"
|
||||
brew "gcc@13"
|
||||
@@ -0,0 +1,20 @@
|
||||
## CTF
|
||||
Welcome to the first part of the comma CTF!
|
||||
|
||||
* all the flags are contained in this route: `0c7f0c7f0c7f0c7f|2021-10-13--13-00-00`
|
||||
* there's 2 flags in each segment, with roughly increasing difficulty
|
||||
* everything you'll need to find the flags is in the openpilot repo
|
||||
* grep is also your friend
|
||||
* first, [setup](https://github.com/commaai/openpilot/tree/master/tools) your PC
|
||||
* read the docs & checkout out the tools in openpilot/tools/
|
||||
* tip: once you get the replay and UI up, start by familiarizing yourself with seeking in replay
|
||||
|
||||
getting started
|
||||
```bash
|
||||
# start the route replay
|
||||
cd openpilot/tools/replay
|
||||
./replay '0c7f0c7f0c7f0c7f|2021-10-13--13-00-00' --dcam --ecam
|
||||
|
||||
# start the UI in another terminal
|
||||
openpilot/selfdrive/ui/ui
|
||||
```
|
||||
@@ -0,0 +1,64 @@
|
||||
# openpilot tools
|
||||
|
||||
## System Requirements
|
||||
|
||||
openpilot is developed and tested on **Ubuntu 24.04**, which is the primary development target aside from the [supported embedded hardware](https://github.com/commaai/openpilot#running-on-a-dedicated-device-in-a-car).
|
||||
|
||||
Most of openpilot should work natively on macOS. On Windows you can use WSL for a nearly native Ubuntu experience. Running natively on any other system is not currently recommended and will likely require modifications.
|
||||
|
||||
## Native setup on Ubuntu 24.04 and macOS
|
||||
|
||||
Follow these instructions for a fully managed setup experience. If you'd like to manage the dependencies yourself, just read the setup scripts in this directory.
|
||||
|
||||
**1. Clone openpilot**
|
||||
``` bash
|
||||
git clone https://github.com/commaai/openpilot.git
|
||||
```
|
||||
|
||||
**2. Run the setup script**
|
||||
``` bash
|
||||
cd openpilot
|
||||
tools/op.sh setup
|
||||
```
|
||||
|
||||
**3. Activate a Python shell**
|
||||
Activate a shell with the Python dependencies installed:
|
||||
``` bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**4. Build openpilot**
|
||||
``` bash
|
||||
scons -u
|
||||
```
|
||||
|
||||
## WSL on Windows
|
||||
|
||||
[Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) should provide a similar experience to native Ubuntu. [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/compare-versions) specifically has been reported by several users to be a seamless experience.
|
||||
|
||||
Follow [these instructions](https://docs.microsoft.com/en-us/windows/wsl/install) to setup the WSL and install the `Ubuntu-24.04` distribution. Once your Ubuntu WSL environment is setup, follow the Linux setup instructions to finish setting up your environment. See [these instructions](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps) for running GUI apps.
|
||||
|
||||
**NOTE**: If you are running WSL 2 and experiencing performance issues with the UI or simulator, you may need to explicitly enable hardware acceleration by setting `GALLIUM_DRIVER=d3d12` before commands. Add `export GALLIUM_DRIVER=d3d12` to your `~/.bashrc` file to make it automatic for future sessions.
|
||||
|
||||
## CTF
|
||||
Learn about the openpilot ecosystem and tools by playing our [CTF](/tools/CTF.md).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
├── car_porting/ # Tools for porting new cars
|
||||
├── release/ # Scripts for building openpilot releases
|
||||
└── scripts/ # Miscellaneous scripts
|
||||
```
|
||||
|
||||
Development tools such as cabana, plotjuggler, and replay live in [openpilot/tools/](/openpilot/tools/):
|
||||
|
||||
```
|
||||
├── cabana/ # View and plot CAN messages from drives or in realtime
|
||||
├── camerastream/ # Cameras stream over the network
|
||||
├── joystick/ # Control your car with a joystick
|
||||
├── lib/ # Libraries to support the tools and reading openpilot logs
|
||||
├── plotjuggler/ # A tool to plot openpilot logs
|
||||
├── replay/ # Replay drives and mock openpilot services
|
||||
└── sim/ # Run openpilot in a simulator
|
||||
```
|
||||
@@ -0,0 +1,129 @@
|
||||
# tools/car_porting
|
||||
|
||||
Check out [this blog post](https://blog.comma.ai/how-to-write-a-car-port-for-openpilot/) for a high-level overview of porting a car.
|
||||
|
||||
## Useful car porting utilities
|
||||
|
||||
Testing car ports in your car is very time-consuming. Check out these utilities to do basic checks on your work before running it in your car.
|
||||
|
||||
### [Cabana](/openpilot/tools/cabana/README.md)
|
||||
|
||||
View your car's CAN signals through DBC files, which openpilot uses to parse and create messages that talk to the car.
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> openpilot/tools/cabana/cabana '1bbe6bf2d62f58a8|2022-07-14--17-11-43'
|
||||
```
|
||||
|
||||
### [tools/car_porting/auto_fingerprint.py](/tools/car_porting/auto_fingerprint.py)
|
||||
|
||||
Given a route and platform, automatically inserts FW fingerprints from the platform into the correct place in fingerprints.py
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> python3 tools/car_porting/auto_fingerprint.py '1bbe6bf2d62f58a8|2022-07-14--17-11-43' 'OUTBACK'
|
||||
Attempting to add fw version for: OUTBACK
|
||||
```
|
||||
|
||||
### [openpilot/selfdrive/car/tests/test_car_interfaces.py](/openpilot/selfdrive/car/tests/test_car_interfaces.py)
|
||||
|
||||
Finds common bugs for car interfaces, without even requiring a route.
|
||||
|
||||
|
||||
#### Example: Typo in signal name
|
||||
```bash
|
||||
> tools/test_runner.py openpilot/selfdrive/car/tests/test_car_interfaces.py -k subaru # replace with the brand you are working on
|
||||
|
||||
=====================================================================
|
||||
FAILED openpilot/selfdrive/car/tests/test_car_interfaces.py::TestCarInterfaces::test_car_interfaces_165_SUBARU_LEGACY_7TH_GEN - KeyError: 'CruiseControlOOPS'
|
||||
|
||||
```
|
||||
|
||||
### [tools/car_porting/test_car_model.py](/tools/car_porting/test_car_model.py)
|
||||
|
||||
Given a route, runs most of the car interface to check for common errors like missing signals, blocked panda messages, and safety mismatches.
|
||||
|
||||
#### Example: panda safety mismatch for gasPressed
|
||||
```bash
|
||||
> python3 tools/car_porting/test_car_model.py '4822a427b188122a|2023-08-14--16-22-21'
|
||||
|
||||
=====================================================================
|
||||
FAIL: test_panda_safety_carstate (__main__.CarModelTestCase.test_panda_safety_carstate)
|
||||
Assert that panda safety matches openpilot's carState
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/home/batman/openpilot/opendbc_repo/opendbc/car/tests/test_models.py", line 440, in test_panda_safety_carstate
|
||||
self.assertFalse(failed_checks, f"panda safety doesn't agree with CarState: {failed_checks}")
|
||||
AssertionError: {'gasPressed': 116} is not false : panda safety doesn't agree with CarState: {'gasPressed': 116}
|
||||
```
|
||||
|
||||
## Jupyter notebooks
|
||||
|
||||
To use these notebooks, install Jupyter within your [openpilot virtual environment](/tools/README.md).
|
||||
|
||||
```bash
|
||||
uv pip install jupyter ipykernel
|
||||
```
|
||||
|
||||
Launching:
|
||||
|
||||
```bash
|
||||
jupyter notebook
|
||||
```
|
||||
|
||||
### [examples/subaru_steer_temp_fault.ipynb](/tools/car_porting/examples/subaru_steer_temp_fault.ipynb)
|
||||
|
||||
An example of searching through a database of segments for a specific condition, and plotting the results.
|
||||
|
||||
|
||||
|
||||
*a plot of the steer_warning vs steering angle, where we can see it is clearly caused by a large steering angle change*
|
||||
|
||||
### [examples/subaru_long_accel.ipynb](/tools/car_porting/examples/subaru_long_accel.ipynb)
|
||||
|
||||
An example of plotting the response of an actuator when it is active.
|
||||
|
||||
|
||||
|
||||
*a plot of the brake_pressure vs acceleration, where we can see it is a fairly linear response.*
|
||||
|
||||
### [examples/ford_vin_fingerprint.ipynb](/tools/car_porting/examples/ford_vin_fingerprint.ipynb)
|
||||
|
||||
In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.
|
||||
|
||||
```
|
||||
vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 00000000000XXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8F98NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True
|
||||
vin: 1FM5K7LC0MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False
|
||||
vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True
|
||||
vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 1FM5K8GC7NGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC8MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E31PRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
```
|
||||
|
||||
### [examples/find_segments_with_message.ipynb](/tools/car_porting/examples/find_segments_with_message.ipynb)
|
||||
|
||||
Searches for segments where a set of given CAN message IDs are present. In the example, we search for all messages
|
||||
used for CAN-based ignition detection.
|
||||
|
||||
```
|
||||
Match found: 46b21f1c5f7aa885/2024-01-23--15-19-34/20/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: a63a23c3e628f288/2023-11-05--18-36-20/8/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: ce31b7a998781ba8/2024-01-19--07-05-29/23/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2023-12-25--19-31-00/4/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2024-01-10--14-33-57/2/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: ae679616266f4096/2023-12-05--15-43-46/4/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2023-11-18--17-49-42/3/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2024-01-03--21-57-09/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-10--11-53-15/17/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-03--17-31-17/29/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-11-27--23-29-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
```
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
from opendbc.car.debug.format_fingerprints import format_brand_fw_versions
|
||||
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.fw_versions import MODEL_TO_BRAND, match_fw_to_car
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Auto fingerprint from a route")
|
||||
parser.add_argument("route", help="The route name to use")
|
||||
parser.add_argument("platform", help="The platform, or leave empty to auto-determine using fuzzy", default=None, nargs="?")
|
||||
args = parser.parse_args()
|
||||
|
||||
lr = LogReader(args.route, ReadMode.QLOG)
|
||||
CP = lr.first("carParams")
|
||||
assert CP is not None, "No carParams in route"
|
||||
|
||||
carPlatform = MIGRATION.get(CP.carFingerprint, CP.carFingerprint)
|
||||
|
||||
if args.platform is not None:
|
||||
platform = args.platform
|
||||
elif carPlatform != "MOCK":
|
||||
platform = carPlatform
|
||||
else:
|
||||
_, matches = match_fw_to_car(CP.carFw, CP.carVin, log=False)
|
||||
assert len(matches) == 1, f"Unable to auto-determine platform, matches: {matches}"
|
||||
platform = list(matches)[0]
|
||||
|
||||
print("Attempting to add fw version for:", platform)
|
||||
|
||||
fw_versions: dict[str, dict[tuple, list[bytes]]] = defaultdict(lambda: defaultdict(list))
|
||||
brand = MODEL_TO_BRAND[platform]
|
||||
|
||||
for fw in CP.carFw:
|
||||
if fw.brand == brand and not fw.logging:
|
||||
addr = fw.address
|
||||
subAddr = None if fw.subAddress == 0 else fw.subAddress
|
||||
key = (fw.ecu.raw, addr, subAddr)
|
||||
|
||||
fw_versions[platform][key].append(fw.fwVersion)
|
||||
|
||||
format_brand_fw_versions(brand, fw_versions)
|
||||
@@ -0,0 +1,231 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 85,
|
||||
"id": "facb8edc-9924-491a-a4dd-fe6135b0c6c4",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import all cars from opendbc\n",
|
||||
"\n",
|
||||
"from opendbc.car.values import PLATFORMS as TEST_PLATFORMS\n",
|
||||
"\n",
|
||||
"# Example: add additional platforms/segments to test outside of commaCarSegments\n",
|
||||
"\n",
|
||||
"EXTRA_SEGMENTS = {\n",
|
||||
" # \"81dd9e9fe256c397/0000001f--97c42cf98d\", # Volkswagen ID.4 test route, new car port, not in public dataset\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 86,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Searching 221 platforms\n",
|
||||
"No segments available for DODGE_DURANGO\n",
|
||||
"No segments available for FORD_RANGER_MK2\n",
|
||||
"No segments available for HOLDEN_ASTRA\n",
|
||||
"No segments available for CADILLAC_ATS\n",
|
||||
"No segments available for CHEVROLET_MALIBU\n",
|
||||
"No segments available for CADILLAC_XT4\n",
|
||||
"No segments available for CHEVROLET_VOLT_2019\n",
|
||||
"No segments available for CHEVROLET_TRAVERSE\n",
|
||||
"No segments available for GMC_YUKON\n",
|
||||
"No segments available for HONDA_ODYSSEY_CHN\n",
|
||||
"No segments available for HYUNDAI_KONA_2022\n",
|
||||
"No segments available for HYUNDAI_NEXO_1ST_GEN\n",
|
||||
"No segments available for GENESIS_GV70_ELECTRIFIED_1ST_GEN\n",
|
||||
"No segments available for GENESIS_G80_2ND_GEN_FL\n",
|
||||
"No segments available for RIVIAN_R1_GEN1\n",
|
||||
"No segments available for SUBARU_FORESTER_HYBRID\n",
|
||||
"No segments available for TESLA_MODEL_3\n",
|
||||
"No segments available for TESLA_MODEL_Y\n",
|
||||
"No segments available for TOYOTA_RAV4_PRIME\n",
|
||||
"No segments available for TOYOTA_SIENNA_4TH_GEN\n",
|
||||
"No segments available for LEXUS_LC_TSS2\n",
|
||||
"No segments available for VOLKSWAGEN_CADDY_MK3\n",
|
||||
"No segments available for VOLKSWAGEN_CRAFTER_MK2\n",
|
||||
"No segments available for VOLKSWAGEN_JETTA_MK6\n",
|
||||
"Searching 577 segments\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 3 # Increase this to search more segments\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_PLATFORMS)} platforms\")\n",
|
||||
"\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"No segments available for {platform}\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"TEST_SEGMENTS.extend(EXTRA_SEGMENTS)\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_SEGMENTS)} segments\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "27a243c33de44498b2b946190df44b23",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"segments searched: 0%| | 0/577 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Match found: 0f53b336851e1384/2023-11-20--09-44-03/12/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7620ad20d3cefc64/2023-10-28--08-14-40/3/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 00d247a9bb1f9196/2023-11-06--13-33-17/9/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 120a432f63cb0de2/2023-10-30--20-01-34/1/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: b70b56b76a6217f2/2023-12-19--08-30-22/35/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 97e388680a6716ed/2024-01-17--10-15-13/9/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2137b01aa0ca63f9/2024-01-06--22-06-14/70/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 8fc6a1b72c8b1357/2023-11-06--07-50-05/8/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7e705eb5c27a49cc/2024-01-18--16-51-20/3/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2024-01-20--14-46-24/12/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2023-11-30--12-01-09/2/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9cad19e0efce3650/2024-01-26--10-24-52/27/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9db428338427dec2/2023-11-05--18-40-09/21/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: d50ada8ee55a5e74/2023-12-11--13-38-09/0/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 900dfa83b4addfe6/2023-12-30--19-20-08/28/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 20acda0eb23d7f23/2024-01-19--17-33-26/41/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 1cc3b46843cad2ca/2024-01-10--20-20-54/24/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2d9b6425552c52c1/2023-12-07--10-31-46/22/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-04--13-13-56/16/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2024-01-08--07-58-12/65/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-05--15-43-46/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-09--21-41-11/4/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 440a155809ba2b6d/2023-12-30--08-51-53/2/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-06--10-11-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/19/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-10--14-13-14/0/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/4/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-12-10--19-02-33/3/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 0b91b433b9332780/2023-12-28--14-02-49/4/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-11-09--07-28-12/1/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/59/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/82/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/106/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from openpilot.tools.lib.logreader import comma_car_segments_source\n",
|
||||
"from tqdm.notebook import tnrange\n",
|
||||
"\n",
|
||||
"# Example search for CAN ignition messages\n",
|
||||
"# Be careful when filtering by bus, account for odd harness arrangements on Honda/HKG\n",
|
||||
"\n",
|
||||
"BUSES_TO_SEARCH = [0, 1, 2]\n",
|
||||
"\n",
|
||||
"# Support for external Red Panda\n",
|
||||
"EXTERNAL_PANDA_BUSES = [bus + 4 for bus in BUSES_TO_SEARCH]\n",
|
||||
"\n",
|
||||
"MESSAGES_TO_FIND = {\n",
|
||||
" 0x1F1: \"GM CAN Ign\",\n",
|
||||
" 0x152: \"Rivian CAN Ign\",\n",
|
||||
" 0x221: \"Tesla 3/Y CAN Ign\",\n",
|
||||
" 0x9E: \"Mazda CAN Ign\",\n",
|
||||
" 0x3C0: \"VW CAN Ign\",\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"progress_bar = tnrange(len(TEST_SEGMENTS), desc=\"segments searched\")\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" progress_bar.update()\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_packets = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" matched_messages = set()\n",
|
||||
"\n",
|
||||
" for packet in can_packets:\n",
|
||||
" for msg in packet.can:\n",
|
||||
" if msg.address in MESSAGES_TO_FIND and msg.src in (BUSES_TO_SEARCH + EXTERNAL_PANDA_BUSES):\n",
|
||||
" # print(msg)\n",
|
||||
" matched_messages.add(msg.address)\n",
|
||||
"\n",
|
||||
" if len(matched_messages) > 0:\n",
|
||||
" message_names = [MESSAGES_TO_FIND[message] for message in matched_messages]\n",
|
||||
" print(f\"Match found: {segment:<45} {CP.carFingerprint:<38} {message_names}\")\n",
|
||||
"\n",
|
||||
" progress_bar.update()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"jupyter": {
|
||||
"is_executing": true
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Got 9 Ford cars from opendbc\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"\"\"\"In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.\"\"\"\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader, comma_car_segments_source\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.ford.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"\n",
|
||||
"platforms = [c.value for c in CAR]\n",
|
||||
"print(f\"Got {len(platforms)} Ford cars from opendbc\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Adapted from https://github.com/commaai/openpilot/issues/31052#issuecomment-1902690083\n",
|
||||
"\n",
|
||||
"MODEL_YEAR_CODES = {'M': 2021, 'N': 2022, 'P': 2023, 'R': 2024, 'S': 2025}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"F150_CODES = ['F1C', 'F1E', 'W1C', 'W1E', 'X1C', 'X1E', 'W1R', 'W1P', 'W1S', 'W1T']\n",
|
||||
"LIGHTNING_CODES = ['L', 'V']\n",
|
||||
"MACHE_CODES = ['K1R', 'K1S', 'K2S', 'K3R', 'K3S', 'K4S']\n",
|
||||
"\n",
|
||||
"FORD_VIN_START = ['1FT', '3FM', '5LM']\n",
|
||||
"\n",
|
||||
"def ford_vin_fingerprint(vin): # Check if it's a Ford vehicle and determine the model\n",
|
||||
" vin_positions_567 = vin[4:7]\n",
|
||||
"\n",
|
||||
" if vin.startswith('1FT'):\n",
|
||||
" if vin_positions_567 in F150_CODES:\n",
|
||||
" if vin[7] in LIGHTNING_CODES:\n",
|
||||
" return \"FORD F-150 LIGHTNING 1ST GEN\"\n",
|
||||
" else:\n",
|
||||
" return \"FORD F-150 14TH GEN\"\n",
|
||||
" elif vin.startswith('3FM'):\n",
|
||||
" if vin_positions_567 in MACHE_CODES:\n",
|
||||
" return \"FORD MUSTANG MACH-E 1ST GEN\"\n",
|
||||
" elif vin.startswith('5LM'):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" return \"mock\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 287 segments for platform FORD_BRONCO_SPORT_MK1, sampling 5 segments\n",
|
||||
"Got 137 segments for platform FORD_ESCAPE_MK4, sampling 5 segments\n",
|
||||
"Got 1041 segments for platform FORD_EXPLORER_MK6, sampling 5 segments\n",
|
||||
"Got 5 segments for platform FORD_F_150_MK14, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_F_150_LIGHTNING_MK1, sampling 3 segments\n",
|
||||
"Got 56 segments for platform FORD_FOCUS_MK4, sampling 5 segments\n",
|
||||
"Got 637 segments for platform FORD_MAVERICK_MK1, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_MUSTANG_MACH_E_MK1, sampling 3 segments\n",
|
||||
"Skipping platform: FORD_RANGER_MK2, no data available\n",
|
||||
"Segment collection finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5\n",
|
||||
"\n",
|
||||
"VINS_TO_CHECK = set()\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in platforms:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" segments = random.sample(all_segments, NUM_SEGMENTS)\n",
|
||||
"\n",
|
||||
" for segment in segments:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if \"FORD\" not in CP.carFingerprint:\n",
|
||||
" print(segment, CP.carFingerprint)\n",
|
||||
" VINS_TO_CHECK.add((CP.carVin, CP.carFingerprint))\n",
|
||||
"\n",
|
||||
"print(\"Segment collection finished\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 00000000000XXXXXX real platform: FORD F-150 14TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True\n",
|
||||
"vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True\n",
|
||||
"vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E33NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC1LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E3XPRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for vin, real_fingerprint in VINS_TO_CHECK:\n",
|
||||
" determined_fingerprint = ford_vin_fingerprint(vin)\n",
|
||||
" print(f\"vin: {vin} real platform: {real_fingerprint: <30} \" +\n",
|
||||
" f\"determined platform: {determined_fingerprint: <30} correct: {real_fingerprint == determined_fingerprint}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"id": "228a6736-de31-4255-9d72-a6ff391b968d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Found 6 qualifying vehicles:\n",
|
||||
" KIA_EV6\n",
|
||||
" HYUNDAI_KONA_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_5\n",
|
||||
" KIA_NIRO_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_6\n",
|
||||
" GENESIS_GV60_EV_1ST_GEN\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from opendbc.car.hyundai.values import CAR, HyundaiFlags\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) & set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD electric vehicles only\n",
|
||||
"#TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) - set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD hybrid and ICE vehicles only\n",
|
||||
"\n",
|
||||
"print(f\"Found {len(TEST_PLATFORMS)} qualifying vehicles:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" print(f\" {platform}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 1300 segments for platform KIA_EV6, sampling 5 segments\n",
|
||||
"Got 9 segments for platform HYUNDAI_KONA_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 1570 segments for platform HYUNDAI_IONIQ_5, sampling 5 segments\n",
|
||||
"Got 34 segments for platform KIA_NIRO_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 974 segments for platform HYUNDAI_IONIQ_6, sampling 5 segments\n",
|
||||
"Got 157 segments for platform GENESIS_GV60_EV_1ST_GEN, sampling 5 segments\n",
|
||||
"Collected 30 segments for analysis\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.hyundai.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5 # TODO: Increase this to search more segments\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" assert(platform in database)\n",
|
||||
" #if platform not in database:\n",
|
||||
" # print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" # continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"print(f\"Collected {len(TEST_SEGMENTS)} segments for analysis\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Analyzing segment ff2bd20623fcaeaa/2023-11-26--16-27-04/5/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 3f1a6480f940cf9a/2024-01-10--23-06-11/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b0a9998109ed0053/2023-12-15--11-10-18/12/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 6e14aa2ed85025df/2023-11-15--13-18-12/24/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment a43f21df3a1ca12d/2024-01-25--08-56-22/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--13-49-24/11/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-11-26--12-31-18/17/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--11-51-44/3/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-08-27--09-32-14/13/s for HYUNDAI KONA 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2024-01-25--15-07-04/24/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 223780ed74116bc2/2023-11-16--09-44-56/15/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment ba9951252624f37d/2024-01-20--22-33-23/118/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 8379b28e51ceb3b1/2023-11-09--23-21-58/92/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26fac43e27cd6091/2023-11-06--12-23-21/9/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=1.0\n",
|
||||
" ACCELERATOR gear=0.0\n",
|
||||
"Analyzing segment 5edb897a0ec7a477/2024-01-13--20-41-36/101/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 66cf8ea23b7c2789/2023-12-04--13-48-53/5/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-10--20-31-37/2/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-03--21-08-30/14/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-11-07--19-52-23/0/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-07-12--19-25-18/6/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-18--07-59-26/11/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 0ad7facc77922c3e/2023-12-21--17-47-25/18/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26968f888e7330d3/2024-01-02--11-18-37/8/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-27--21-03-24/33/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment df7fdd56970d90fe/2024-01-07--01-04-39/26/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--14-45-44/0/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--20-57-09/8/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--12-52-38/5/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-19--19-57-52/47/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--13-01-23/1/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analysis finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.hyundai.values import DBC\n",
|
||||
"from opendbc.car.hyundai.hyundaicanfd import CanBus\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import comma_car_segments_source\n",
|
||||
"\n",
|
||||
"message_names = [\"GEAR_SHIFTER\", \"ACCELERATOR\", \"GEAR\", \"GEAR_ALT\", \"GEAR_ALT_2\"]\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" parser_messages = []\n",
|
||||
" for name in message_names:\n",
|
||||
" parser_messages.append((name, 0))\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], parser_messages, CanBus(CP).ECAN)\n",
|
||||
"\n",
|
||||
" parsed_message_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" parsed_message_history.append(copy.copy(cp.vl))\n",
|
||||
"\n",
|
||||
" print(f\"Analyzing segment {segment:<44} for {CP.carFingerprint}\")\n",
|
||||
" for name in message_names:\n",
|
||||
" if parsed_message_history[0][name][\"CHECKSUM\"] != 0: # Message is present for this segment\n",
|
||||
" gear_prev = parsed_message_history[0][name][\"GEAR\"]\n",
|
||||
" print(f\" {name:<15} gear={gear_prev}\")\n",
|
||||
" for i, parsed_messages in enumerate(parsed_message_history):\n",
|
||||
" gear = parsed_messages[name][\"GEAR\"]\n",
|
||||
" if gear != gear_prev:\n",
|
||||
" print(\" *** Signal transition found! ***\")\n",
|
||||
" examples.append(i)\n",
|
||||
" gear_prev = gear\n",
|
||||
"\n",
|
||||
"print(\"Analysis finished\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.subaru.values import CAR, SubaruFlags\n",
|
||||
"from opendbc.car.subaru.fingerprints import FW_VERSIONS\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR) - CAR.with_flags(SubaruFlags.PREGLOBAL)\n",
|
||||
"\n",
|
||||
"Ecu = structs.CarParams.Ecu\n",
|
||||
"\n",
|
||||
"FW_BY_ECU = {platform: {ecu: versions for (ecu, addr, sub_addr), versions in fw_versions.items()} for platform, fw_versions in FW_VERSIONS.items()}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PLATFORM_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 0: {\n",
|
||||
" b'\\xa5': [CAR.SUBARU_ASCENT, CAR.SUBARU_ASCENT_2023],\n",
|
||||
" b'\\xa2': [CAR.SUBARU_IMPREZA, CAR.SUBARU_IMPREZA_2020, CAR.SUBARU_CROSSTREK_HYBRID],\n",
|
||||
" b'\\xa1': [CAR.SUBARU_OUTBACK, CAR.SUBARU_LEGACY, CAR.SUBARU_OUTBACK_2023],\n",
|
||||
" b'\\xa3': [CAR.SUBARU_FORESTER, CAR.SUBARU_FORESTER_HYBRID, CAR.SUBARU_FORESTER_2022],\n",
|
||||
" b'z': [CAR.SUBARU_IMPREZA],\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"YEAR_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 2: {\n",
|
||||
" b'\\x18': 2018,\n",
|
||||
" b'\\x19': 2019,\n",
|
||||
" b'\\x20': 2020,\n",
|
||||
" b'\\x21': 2021,\n",
|
||||
" b'\\x22': 2022,\n",
|
||||
" b'\\x23': 2023,\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_codes(platforms, codes):\n",
|
||||
" results = []\n",
|
||||
" for platform in platforms:\n",
|
||||
" for ecu in codes:\n",
|
||||
" for i in codes[ecu]:\n",
|
||||
" if isinstance(i, tuple):\n",
|
||||
" j = slice(i[0], i[1])\n",
|
||||
" else:\n",
|
||||
" j = slice(i, i+1)\n",
|
||||
" for version in FW_BY_ECU[platform][ecu]:\n",
|
||||
" code = version[j]\n",
|
||||
" if code not in codes[ecu][i]:\n",
|
||||
" print(f\"{platform} {code.hex()} not in {codes[ecu][i].keys()}\")\n",
|
||||
" else:\n",
|
||||
" results.append((platform, codes[ecu][i][code]))\n",
|
||||
" return results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 2e not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 3f not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2021 years=[2020]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2018 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2021 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_year_code(platform, year):\n",
|
||||
" car_docs = CAR(platform).config.car_docs\n",
|
||||
" if isinstance(car_docs, list):\n",
|
||||
" car_docs = car_docs[0]\n",
|
||||
" years = [int(y) for y in car_docs.year_list]\n",
|
||||
" correct_year = year in years\n",
|
||||
" print(f\"{correct_year=!s: <6} {platform=: <32} {year=: <5} {years=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, YEAR_CODES)\n",
|
||||
"for platform, year in codes:\n",
|
||||
" test_year_code(platform, year)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_HYBRID platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT_2023 platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_platform_code(platform, platforms):\n",
|
||||
" platforms = [str(p) for p in platforms]\n",
|
||||
" in_possible_platforms = platform in platforms\n",
|
||||
" print(f\"{in_possible_platforms=!s: <6} {platform=: <32} {platforms=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, PLATFORM_CODES)\n",
|
||||
"for platform, possible_platforms in codes:\n",
|
||||
" test_platform_code(platform, possible_platforms)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"segments = [\n",
|
||||
" \"d9df6f87e8feff94|2023-03-28--17-41-10/1:12\"\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we plot the relationship between Cruise_Brake and Acceleration for stock eyesight.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"ES_Distance\", 20),\n",
|
||||
" (\"ES_Brake\", 20),\n",
|
||||
" (\"ES_Status\", 20),\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, 1)\n",
|
||||
"\n",
|
||||
" es_distance_history = []\n",
|
||||
" es_status_history = []\n",
|
||||
" es_brake_history = []\n",
|
||||
" acceleration_history = []\n",
|
||||
"\n",
|
||||
" last_acc = 0\n",
|
||||
"\n",
|
||||
" for msg in lr:\n",
|
||||
" if msg.which() == \"can\":\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" es_distance_history.append(copy.copy(cp.vl[\"ES_Distance\"]))\n",
|
||||
" es_brake_history.append(copy.copy(cp.vl[\"ES_Brake\"]))\n",
|
||||
" es_status_history.append(copy.copy(cp.vl[\"ES_Status\"]))\n",
|
||||
"\n",
|
||||
" acceleration_history.append(last_acc)\n",
|
||||
"\n",
|
||||
" if msg.which() == \"carState\":\n",
|
||||
" last_acc = msg.carState.aEgo"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def process(history, func):\n",
|
||||
" return np.array([func(h) for h in history])\n",
|
||||
"\n",
|
||||
"cruise_activated = process(es_status_history, lambda es_status: es_status[\"Cruise_Activated\"])\n",
|
||||
"cruise_throttle = process(es_distance_history, lambda es_distance: es_distance[\"Cruise_Throttle\"])\n",
|
||||
"cruise_rpm = process(es_status_history, lambda es_status: es_status[\"Cruise_RPM\"])\n",
|
||||
"cruise_brake = process(es_brake_history, lambda es_brake: es_brake[\"Brake_Pressure\"])\n",
|
||||
"acceleration = process(acceleration_history, lambda acc: acc)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"valid_brake = (cruise_activated==1) & (cruise_brake>0) # only when cruise is activated and eyesight is braking\n",
|
||||
"\n",
|
||||
"ax = plt.figure().add_subplot()\n",
|
||||
"\n",
|
||||
"ax.set_title(\"Brake_Pressure vs Acceleration\")\n",
|
||||
"ax.set_xlabel(\"Brake_Pessure\")\n",
|
||||
"ax.set_ylabel(\"Acceleration\")\n",
|
||||
"ax.scatter(cruise_brake[valid_brake], -acceleration[valid_brake])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# An example of searching through a database of segments for a specific condition, and plotting the results.\n",
|
||||
"\n",
|
||||
"segments = [\n",
|
||||
" \"c3d1ccb52f5f9d65|2023-07-22--01-23-20/6:10\",\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import CanBus, DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we search for positive transitions of Steer_Warning, which indicate that the EPS\n",
|
||||
"has stopped responding to our messages. This analysis would allow you to find the cause of these\n",
|
||||
"steer warnings and potentially work around them.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"Steering_Torque\", 50)\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, CanBus.main)\n",
|
||||
"\n",
|
||||
" steering_torque_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" steering_torque_history.append(copy.copy(cp.vl[\"Steering_Torque\"]))\n",
|
||||
"\n",
|
||||
" steer_warning_last = False\n",
|
||||
" for i, steering_torque_msg in enumerate(steering_torque_history):\n",
|
||||
" steer_warning = steering_torque_msg[\"Steer_Warning\"]\n",
|
||||
"\n",
|
||||
" steer_angle = steering_torque_msg[\"Steering_Angle\"]\n",
|
||||
"\n",
|
||||
" if steer_warning and not steer_warning_last: # positive transition of \"Steer_Warning\"\n",
|
||||
" examples.append(i)\n",
|
||||
"\n",
|
||||
" steer_warning_last = steer_warning\n",
|
||||
"\n",
|
||||
" FRAME_DELTA = 100 # plot this many frames around the positive transition\n",
|
||||
"\n",
|
||||
" for example in examples:\n",
|
||||
" fig, axs = plt.subplots(2)\n",
|
||||
"\n",
|
||||
" min_frame = int(example-FRAME_DELTA/2)\n",
|
||||
" max_frame = int(example+FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" steering_angle_history = [msg[\"Steering_Angle\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
" steering_warning_history = [msg[\"Steer_Warning\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
"\n",
|
||||
" xs = np.arange(-FRAME_DELTA/2, FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" axs[0].plot(xs, steering_angle_history)\n",
|
||||
" axs[0].set_ylabel(\"Steering Angle (deg)\")\n",
|
||||
" axs[1].plot(xs, steering_warning_history)\n",
|
||||
" axs[1].set_ylabel(\"Steer Warning\")\n",
|
||||
"\n",
|
||||
" plt.show()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
+154
@@ -0,0 +1,154 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import os
|
||||
import time
|
||||
import argparse
|
||||
import signal
|
||||
from collections import defaultdict
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
def sigint_handler(signal, frame):
|
||||
exit(0)
|
||||
signal.signal(signal.SIGINT, sigint_handler)
|
||||
|
||||
class SteeringAccuracyTool:
|
||||
all_groups = {"germany": (45, "45 - up m/s // 162 - up km/h // 101 - up mph"),
|
||||
"veryfast": (35, "35 - 45 m/s // 126 - 162 km/h // 78 - 101 mph"),
|
||||
"fast": (25, "25 - 35 m/s // 90 - 126 km/h // 56 - 78 mph"),
|
||||
"medium": (15, "15 - 25 m/s // 54 - 90 km/h // 34 - 56 mph"),
|
||||
"slow": (5, " 5 - 15 m/s // 18 - 54 km/h // 11 - 34 mph"),
|
||||
"crawl": (0, " 0 - 5 m/s // 0 - 18 km/h // 0 - 11 mph")}
|
||||
|
||||
def __init__(self, args):
|
||||
self.msg_cnt = 0
|
||||
self.cnt = 0
|
||||
self.total_error = 0
|
||||
|
||||
if args.group == "all":
|
||||
self.display_groups = self.all_groups.keys()
|
||||
elif args.group in self.all_groups.keys():
|
||||
self.display_groups = [args.group]
|
||||
else:
|
||||
raise ValueError("invalid speed group, see help")
|
||||
|
||||
self.speed_group_stats = {}
|
||||
for group in self.all_groups:
|
||||
self.speed_group_stats[group] = defaultdict(lambda: {'err': 0, "cnt": 0, "=": 0, "+": 0, "-": 0, "steer": 0, "limited": 0, "saturated": 0, "dpp": 0})
|
||||
|
||||
def update(self, sm):
|
||||
self.msg_cnt += 1
|
||||
|
||||
lateralControlState = sm['controlsState'].lateralControlState
|
||||
control_type = list(lateralControlState.to_dict().keys())[0]
|
||||
control_state = lateralControlState.__getattr__(control_type)
|
||||
|
||||
v_ego = sm['carState'].vEgo
|
||||
active = sm['controlsState'].active
|
||||
steer = sm['carOutput'].actuatorsOutput.torque
|
||||
standstill = sm['carState'].standstill
|
||||
steer_limited_by_safety = abs(sm['carControl'].actuators.torque - sm['carControl'].actuatorsOutput.torque) > 1e-2
|
||||
overriding = sm['carState'].steeringPressed
|
||||
changing_lanes = sm['modelV2'].meta.laneChangeState != 0
|
||||
model_points = sm['modelV2'].position.y
|
||||
# must be engaged, not at standstill, not overriding steering, and not changing lanes
|
||||
if active and not standstill and not overriding and not changing_lanes:
|
||||
self.cnt += 1
|
||||
|
||||
# wait 5 seconds after engage / standstill / override / lane change
|
||||
if self.cnt >= 500:
|
||||
actual_angle = control_state.steeringAngleDeg
|
||||
desired_angle = control_state.steeringAngleDesiredDeg
|
||||
|
||||
# calculate error before rounding, then round for stats grouping
|
||||
angle_error = abs(desired_angle - actual_angle)
|
||||
actual_angle = round(actual_angle, 1)
|
||||
desired_angle = round(desired_angle, 1)
|
||||
angle_error = round(angle_error, 2)
|
||||
angle_abs = int(abs(round(desired_angle, 0)))
|
||||
|
||||
for group, group_props in self.all_groups.items():
|
||||
if v_ego > group_props[0]:
|
||||
# collect stats
|
||||
self.speed_group_stats[group][angle_abs]["cnt"] += 1
|
||||
self.speed_group_stats[group][angle_abs]["err"] += angle_error
|
||||
self.speed_group_stats[group][angle_abs]["steer"] += abs(steer)
|
||||
if len(model_points):
|
||||
self.speed_group_stats[group][angle_abs]["dpp"] += abs(model_points[0])
|
||||
if steer_limited_by_safety:
|
||||
self.speed_group_stats[group][angle_abs]["limited"] += 1
|
||||
if control_state.saturated:
|
||||
self.speed_group_stats[group][angle_abs]["saturated"] += 1
|
||||
if actual_angle == desired_angle:
|
||||
self.speed_group_stats[group][angle_abs]["="] += 1
|
||||
else:
|
||||
if desired_angle == 0.:
|
||||
overshoot = True
|
||||
else:
|
||||
overshoot = desired_angle < actual_angle if desired_angle > 0. else desired_angle > actual_angle
|
||||
self.speed_group_stats[group][angle_abs]["+" if overshoot else "-"] += 1
|
||||
break
|
||||
else:
|
||||
self.cnt = 0
|
||||
|
||||
if self.msg_cnt % 100 == 0:
|
||||
print(chr(27) + "[2J")
|
||||
if self.cnt != 0:
|
||||
print("COLLECTING ...\n")
|
||||
else:
|
||||
print("DISABLED (not active, standstill, steering override, or lane change)\n")
|
||||
for group in self.display_groups:
|
||||
if len(self.speed_group_stats[group]) > 0:
|
||||
print(f"speed group: {group:10s} {self.all_groups[group][1]:>96s}")
|
||||
print(f" {'-'*118}")
|
||||
for k in sorted(self.speed_group_stats[group].keys()):
|
||||
v = self.speed_group_stats[group][k]
|
||||
print(f' {k:#2}° | actuator:{int(v["steer"] / v["cnt"] * 100):#3}% ' +
|
||||
f'| error: {round(v["err"] / v["cnt"], 2):2.2f}° | -:{int(v["-"] / v["cnt"] * 100):#3}% ' +
|
||||
f'| =:{int(v["="] / v["cnt"] * 100):#3}% | +:{int(v["+"] / v["cnt"] * 100):#3}% | lim:{v["limited"]:#5} ' +
|
||||
f'| sat:{v["saturated"]:#5} | path dev: {round(v["dpp"] / v["cnt"], 2):2.2f}m | total: {v["cnt"]:#5}')
|
||||
print("")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
parser = argparse.ArgumentParser(description='Steering accuracy measurement tool')
|
||||
parser.add_argument('--route', help="route name")
|
||||
parser.add_argument('--addr', default='127.0.0.1', help="IP address for optional ZMQ listener, default to msgq")
|
||||
parser.add_argument('--group', default='all', help="speed group to display, [crawl|slow|medium|fast|veryfast|germany|all], default to all")
|
||||
args = parser.parse_args()
|
||||
|
||||
tool = SteeringAccuracyTool(args)
|
||||
|
||||
if args.route is not None:
|
||||
print(f"loading {args.route}...")
|
||||
lr = LogReader(args.route, sort_by_time=True)
|
||||
|
||||
sm = {}
|
||||
for msg in lr:
|
||||
if msg.which() == 'carState':
|
||||
sm['carState'] = msg.carState
|
||||
elif msg.which() == 'carControl':
|
||||
sm['carControl'] = msg.carControl
|
||||
elif msg.which() == 'controlsState':
|
||||
sm['controlsState'] = msg.controlsState
|
||||
elif msg.which() == 'modelV2':
|
||||
sm['modelV2'] = msg.modelV2
|
||||
|
||||
if msg.which() == 'carControl' and 'carState' in sm and 'controlsState' in sm and 'modelV2' in sm:
|
||||
tool.update(sm)
|
||||
|
||||
else:
|
||||
if args.addr != "127.0.0.1":
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
|
||||
carControl = messaging.sub_sock('carControl', addr=args.addr, conflate=True)
|
||||
sm = messaging.SubMaster(['carState', 'carControl', 'carOutput', 'controlsState', 'modelV2'], addr=args.addr)
|
||||
time.sleep(1) # Make sure all submaster data is available before going further
|
||||
|
||||
print("waiting for messages...")
|
||||
while messaging.recv_one(carControl):
|
||||
sm.update()
|
||||
tool.update(sm)
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import sys
|
||||
import unittest
|
||||
from opendbc.car.tests.routes import CarTestRoute
|
||||
from opendbc.car.tests.test_models import TestCarModelBase
|
||||
from openpilot.tools.lib.route import SegmentRange
|
||||
|
||||
|
||||
def create_test_models_suite(routes: list[CarTestRoute]) -> unittest.TestSuite:
|
||||
test_suite = unittest.TestSuite()
|
||||
for test_route in routes:
|
||||
# create new test case and discover tests
|
||||
test_case_args = {"platform": test_route.car_model, "test_route": test_route}
|
||||
CarModelTestCase = type("CarModelTestCase", (TestCarModelBase,), test_case_args)
|
||||
test_suite.addTest(unittest.TestLoader().loadTestsFromTestCase(CarModelTestCase))
|
||||
return test_suite
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Test any route against common issues with a new car port. " +
|
||||
"Uses opendbc_repo/opendbc/car/tests/test_models.py")
|
||||
parser.add_argument("route_or_segment_name", help="Specify route to run tests on")
|
||||
parser.add_argument("--car", help="Specify car model for test route")
|
||||
args = parser.parse_args()
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
|
||||
sr = SegmentRange(args.route_or_segment_name)
|
||||
|
||||
test_routes = [CarTestRoute(sr.route_name, args.car, segment=seg_idx) for seg_idx in sr.seg_idxs]
|
||||
test_suite = create_test_models_suite(test_routes)
|
||||
|
||||
unittest.TextTestRunner().run(test_suite)
|
||||
Executable
+518
@@ -0,0 +1,518 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [[ ! "${BASH_SOURCE[0]}" = "${0}" ]]; then
|
||||
echo "Invalid invocation! This script must not be sourced."
|
||||
echo "Run 'op.sh' directly or check your .bashrc for a valid alias"
|
||||
return 0
|
||||
fi
|
||||
|
||||
set -e
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
UNDERLINE='\033[4m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
SHELL_NAME="$(basename ${SHELL})"
|
||||
RC_FILE="${HOME}/.$(basename ${SHELL})rc"
|
||||
if [ "$(uname)" == "Darwin" ] && [ $SHELL == "/bin/bash" ]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
|
||||
function retry() {
|
||||
local attempts=$1
|
||||
shift
|
||||
for i in $(seq 1 "$attempts"); do
|
||||
if "$@"; then
|
||||
return 0
|
||||
fi
|
||||
if [ "$i" -lt "$attempts" ]; then
|
||||
echo " Attempt $i/$attempts failed, retrying in 5s..."
|
||||
sleep 5
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
function op_run_command() {
|
||||
CMD="$*"
|
||||
|
||||
echo -e "${BOLD}Running command →${NC} $CMD │"
|
||||
for ((i=0; i<$((19 + ${#CMD})); i++)); do
|
||||
echo -n "─"
|
||||
done
|
||||
echo -e "┘\n"
|
||||
|
||||
if [[ -z "$DRY" ]]; then
|
||||
"$@"
|
||||
fi
|
||||
}
|
||||
|
||||
# be default, assume openpilot dir is in current directory
|
||||
OPENPILOT_ROOT=$(pwd)
|
||||
function op_get_openpilot_dir() {
|
||||
# First try traversing up the directory tree
|
||||
while [[ "$OPENPILOT_ROOT" != '/' ]];
|
||||
do
|
||||
if find "$OPENPILOT_ROOT/launch_openpilot.sh" -maxdepth 1 -mindepth 1 &> /dev/null; then
|
||||
return 0
|
||||
fi
|
||||
OPENPILOT_ROOT="$(readlink -f "$OPENPILOT_ROOT/"..)"
|
||||
done
|
||||
|
||||
# Fallback to hardcoded directories if not found
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
|
||||
for dir in "$(readlink -f "$SCRIPT_DIR/../..")" "$HOME/openpilot" "/data/openpilot"; do
|
||||
if [[ -f "$dir/launch_openpilot.sh" ]]; then
|
||||
OPENPILOT_ROOT="$dir"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
function op_install_post_commit() {
|
||||
op_get_openpilot_dir
|
||||
if [[ ! -d $OPENPILOT_ROOT/.git/hooks/post-commit.d ]]; then
|
||||
mkdir $OPENPILOT_ROOT/.git/hooks/post-commit.d
|
||||
mv $OPENPILOT_ROOT/.git/hooks/post-commit $OPENPILOT_ROOT/.git/hooks/post-commit.d 2>/dev/null || true
|
||||
fi
|
||||
cd $OPENPILOT_ROOT/.git/hooks
|
||||
ln -sf ../../scripts/post-commit post-commit
|
||||
}
|
||||
|
||||
function op_check_openpilot_dir() {
|
||||
echo "Checking for openpilot directory..."
|
||||
if [[ -f "$OPENPILOT_ROOT/launch_openpilot.sh" ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] openpilot found."
|
||||
return 0
|
||||
fi
|
||||
echo -e " ↳ [${RED}✗${NC}] openpilot directory not found! Make sure that you are"
|
||||
echo " inside the openpilot directory or specify one with the"
|
||||
echo " --dir option!"
|
||||
return 1
|
||||
}
|
||||
|
||||
function op_check_git() {
|
||||
echo "Checking for git..."
|
||||
if ! command -v "git" > /dev/null 2>&1; then
|
||||
echo -e " ↳ [${RED}✗${NC}] git not found on your system!"
|
||||
return 1
|
||||
else
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git found."
|
||||
fi
|
||||
|
||||
echo "Checking for git lfs files..."
|
||||
if [[ $(file -b $OPENPILOT_ROOT/openpilot/selfdrive/modeld/models/dmonitoring_model.onnx) == "data" ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git lfs files found."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] git lfs files not found! Run 'git lfs pull'"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Checking for git submodules..."
|
||||
for name in $(git config --file .gitmodules --get-regexp path | awk '{ print $2 }' | tr '\n' ' '); do
|
||||
if [[ -z $(ls $OPENPILOT_ROOT/$name) ]]; then
|
||||
echo -e " ↳ [${RED}✗${NC}] git submodule $name not found! Run 'git submodule update --init --recursive'"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git submodules found."
|
||||
}
|
||||
|
||||
function op_check_os() {
|
||||
echo "Checking for compatible os version..."
|
||||
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
|
||||
if [ -f "/etc/os-release" ]; then
|
||||
source /etc/os-release
|
||||
case "$VERSION_CODENAME" in
|
||||
"jammy" | "kinetic" | "noble" | "focal")
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Ubuntu $VERSION_CODENAME detected."
|
||||
;;
|
||||
* )
|
||||
echo -e " ↳ [${RED}✗${NC}] Incompatible Ubuntu version $VERSION_CODENAME detected!"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] No /etc/os-release on your system. Make sure you're running on Ubuntu, or similar!"
|
||||
return 1
|
||||
fi
|
||||
|
||||
elif [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] macOS detected."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] OS type $OSTYPE not supported!"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
function op_check_venv() {
|
||||
echo "Checking for venv..."
|
||||
if [[ -f $OPENPILOT_ROOT/.venv/bin/activate ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] venv detected."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] Can't activate venv in $OPENPILOT_ROOT. Assuming global env!"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_before_cmd() {
|
||||
if [[ ! -z "$NO_VERIFY" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
op_get_openpilot_dir
|
||||
cd $OPENPILOT_ROOT
|
||||
|
||||
result="$((op_check_openpilot_dir ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_git ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_os ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_venv ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
|
||||
op_activate_venv
|
||||
|
||||
if [[ -z $VERBOSE ]]; then
|
||||
echo -e "${BOLD}Checking system →${NC} [${GREEN}✔${NC}]"
|
||||
else
|
||||
echo -e "$result"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_setup() {
|
||||
echo "Installing op system-wide..."
|
||||
OP_SH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )/op.sh"
|
||||
CMD=$(cat <<EOF
|
||||
alias op='$OP_SH "\$@"'
|
||||
_op_completions() { [ "\$COMP_CWORD" -eq 1 ] && COMPREPLY=(\$(compgen -W "\$(awk '/shift 1; op_/{print \$1}' $OP_SH)" -- "\${COMP_WORDS[1]}")); }
|
||||
[ -n "\$BASH_VERSION" ] && complete -F _op_completions -o default op
|
||||
EOF
|
||||
)
|
||||
grep -q "alias op=" "$RC_FILE" 2>/dev/null || printf '\n%s\n' "$CMD" >> "$RC_FILE"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] op installed successfully. Open a new shell to use it."
|
||||
|
||||
op_get_openpilot_dir
|
||||
cd $OPENPILOT_ROOT
|
||||
|
||||
op_check_openpilot_dir
|
||||
op_check_os
|
||||
|
||||
# Submodules must be present before uv sync: pyproject path sources
|
||||
# (pandacan, opendbc, msgq, ...) live in the submodule checkouts.
|
||||
echo "Getting git submodules..."
|
||||
st="$(date +%s)"
|
||||
if ! retry 3 git submodule update --jobs 4 --init --recursive; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Getting git submodules failed!"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Submodules installed successfully in $((et - st)) seconds."
|
||||
|
||||
echo "Installing dependencies..."
|
||||
st="$(date +%s)"
|
||||
SETUP_SCRIPT="tools/setup_dependencies.sh"
|
||||
if ! $OPENPILOT_ROOT/$SETUP_SCRIPT; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Dependencies installation failed!"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Dependencies installed successfully in $((et - st)) seconds."
|
||||
|
||||
op_activate_venv
|
||||
|
||||
echo "Pulling git lfs files..."
|
||||
st="$(date +%s)"
|
||||
if ! retry 3 git lfs pull; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Pulling git lfs files failed!"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Files pulled successfully in $((et - st)) seconds."
|
||||
|
||||
op_check
|
||||
}
|
||||
|
||||
function op_auth() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/tools/lib/auth.py "$@"
|
||||
}
|
||||
|
||||
function op_activate_venv() {
|
||||
# bash 3.2 can't handle this without the 'set +e'
|
||||
set +e
|
||||
source $OPENPILOT_ROOT/.venv/bin/activate &> /dev/null || true
|
||||
set -e
|
||||
|
||||
# persist venv on PATH across GitHub Actions steps
|
||||
if [ -n "$GITHUB_PATH" ]; then
|
||||
echo "$OPENPILOT_ROOT/.venv/bin" >> "$GITHUB_PATH"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_venv() {
|
||||
op_before_cmd
|
||||
|
||||
if [[ ! -f $OPENPILOT_ROOT/.venv/bin/activate ]]; then
|
||||
echo -e "No venv found in $OPENPILOT_ROOT"
|
||||
return 1
|
||||
fi
|
||||
|
||||
case $SHELL_NAME in
|
||||
"zsh")
|
||||
ZSHRC_DIR=$(mktemp -d 2>/dev/null || mktemp -d -t 'tmp_zsh')
|
||||
echo "source $RC_FILE; source $OPENPILOT_ROOT/.venv/bin/activate" >> $ZSHRC_DIR/.zshrc
|
||||
ZDOTDIR=$ZSHRC_DIR zsh ;;
|
||||
*)
|
||||
bash --rcfile <(echo "source $RC_FILE; source $OPENPILOT_ROOT/.venv/bin/activate") ;;
|
||||
esac
|
||||
}
|
||||
|
||||
function op_adb() {
|
||||
op_before_cmd
|
||||
op_run_command tools/scripts/adb_ssh.sh "$@"
|
||||
}
|
||||
|
||||
function op_ssh() {
|
||||
op_before_cmd
|
||||
op_run_command tools/scripts/ssh.py "$@"
|
||||
}
|
||||
|
||||
function op_script() {
|
||||
op_before_cmd
|
||||
|
||||
case $1 in
|
||||
som-debug ) op_run_command panda/scripts/som_debug.sh "${@:2}" ;;
|
||||
* )
|
||||
echo -e "Unknown script '$1'. Available scripts:"
|
||||
echo -e " ${BOLD}som-debug${NC} SOM serial debug console via panda"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
function op_check() {
|
||||
VERBOSE=1
|
||||
op_before_cmd
|
||||
unset VERBOSE
|
||||
}
|
||||
|
||||
function op_esim() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/common/esim/esim.py "$@"
|
||||
}
|
||||
|
||||
function op_build() {
|
||||
CDIR=$(pwd)
|
||||
op_before_cmd
|
||||
cd "$CDIR"
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
# needed on AGNOS to not run out of memory
|
||||
op_run_command openpilot/system/manager/build.py
|
||||
else
|
||||
op_run_command scons -u "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_juggle() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/tools/plotjuggler/juggle.py "$@"
|
||||
}
|
||||
|
||||
function op_lint() {
|
||||
op_before_cmd
|
||||
op_run_command scripts/lint/lint.sh "$@"
|
||||
}
|
||||
|
||||
function op_test() {
|
||||
op_before_cmd
|
||||
op_run_command tools/test_runner.py "$@"
|
||||
}
|
||||
|
||||
function op_replay() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/tools/replay/replay "$@"
|
||||
}
|
||||
|
||||
function op_cabana() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/tools/cabana/cabana "$@"
|
||||
}
|
||||
|
||||
function op_sim() {
|
||||
op_before_cmd
|
||||
op_run_command exec openpilot/tools/sim/run_bridge.py &
|
||||
op_run_command exec openpilot/tools/sim/launch_openpilot.sh
|
||||
}
|
||||
|
||||
function op_clip() {
|
||||
op_before_cmd
|
||||
op_run_command openpilot/tools/clip/run.py "$@"
|
||||
}
|
||||
|
||||
function op_check_agnos_update() {
|
||||
if [[ ! -f "/AGNOS" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
local choice current_version target_version
|
||||
current_version="$(< /VERSION)"
|
||||
target_version="$(unset AGNOS_VERSION; source "$OPENPILOT_ROOT/launch_env.sh"; echo "$AGNOS_VERSION")"
|
||||
|
||||
if [[ "$current_version" == "$target_version" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo -e "${BOLD}AGNOS update available:${NC} $current_version → $target_version"
|
||||
if read -r -p "Install it now? [y/N] " choice && [[ "$choice" =~ ^[Yy]$ ]]; then
|
||||
op_run_command "$OPENPILOT_ROOT/openpilot/common/hardware/comma/agnos.py" --swap \
|
||||
"$OPENPILOT_ROOT/openpilot/common/hardware/comma/agnos.json"
|
||||
|
||||
if read -r -p "Reboot now to apply the update? [y/N] " choice && [[ "$choice" =~ ^[Yy]$ ]]; then
|
||||
op_run_command sudo reboot
|
||||
else
|
||||
echo "Reboot before starting openpilot to apply the AGNOS update."
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
function op_switch() {
|
||||
REMOTE="origin"
|
||||
if [ "$#" -gt 1 ]; then
|
||||
REMOTE="$1"
|
||||
shift
|
||||
fi
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo -e "${BOLD}${UNDERLINE}Usage:${NC} op switch [REMOTE] <BRANCH>"
|
||||
return 1
|
||||
fi
|
||||
BRANCH="$1"
|
||||
|
||||
git config --replace-all remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
|
||||
git submodule deinit --all --force
|
||||
git fetch "$REMOTE" "$BRANCH"
|
||||
git checkout -f FETCH_HEAD
|
||||
git checkout -B "$BRANCH" --track "$REMOTE"/"$BRANCH"
|
||||
git submodule deinit --all --force
|
||||
git reset --hard "${REMOTE}/${BRANCH}"
|
||||
git clean -df
|
||||
git submodule update --init --recursive
|
||||
git submodule foreach git reset --hard
|
||||
git submodule foreach git clean -df
|
||||
|
||||
# remove openpilot update flag if present
|
||||
rm -f .overlay_init
|
||||
|
||||
op_check_agnos_update
|
||||
}
|
||||
|
||||
function op_start() {
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
op_before_cmd
|
||||
op_check_agnos_update
|
||||
op_run_command sudo systemctl restart comma $@
|
||||
fi
|
||||
}
|
||||
|
||||
function op_stop() {
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
op_before_cmd
|
||||
op_run_command sudo systemctl stop comma $@
|
||||
fi
|
||||
}
|
||||
|
||||
function op_default() {
|
||||
echo "An openpilot helper"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Description:${NC}"
|
||||
echo " op is your entry point for all things related to openpilot development."
|
||||
echo " op is only a wrapper for existing scripts, tools, and commands."
|
||||
echo " op will always show you what it will run on your system."
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Usage:${NC} op [OPTIONS] <COMMAND>"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [System]:${NC}"
|
||||
echo -e " ${BOLD}auth${NC} Authenticate yourself for API use"
|
||||
echo -e " ${BOLD}check${NC} Check the development environment (git, os) to start using openpilot"
|
||||
echo -e " ${BOLD}esim${NC} Manage eSIM profiles on your comma device"
|
||||
echo -e " ${BOLD}venv${NC} Activate the python virtual environment"
|
||||
echo -e " ${BOLD}setup${NC} Install the 'op' tool and openpilot dependencies"
|
||||
echo -e " ${BOLD}build${NC} Run the openpilot build system in the current working directory"
|
||||
echo -e " ${BOLD}switch${NC} Switch to a different git branch with a clean slate (nukes any changes)"
|
||||
echo -e " ${BOLD}start${NC} Starts (or restarts) openpilot"
|
||||
echo -e " ${BOLD}stop${NC} Stops openpilot"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [Tooling]:${NC}"
|
||||
echo -e " ${BOLD}juggle${NC} Run PlotJuggler"
|
||||
echo -e " ${BOLD}replay${NC} Run Replay"
|
||||
echo -e " ${BOLD}cabana${NC} Run Cabana"
|
||||
echo -e " ${BOLD}clip${NC} Run clip (linux only)"
|
||||
echo -e " ${BOLD}adb${NC} Run adb shell"
|
||||
echo -e " ${BOLD}ssh${NC} comma prime SSH helper"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [Scripts]:${NC}"
|
||||
echo -e " ${BOLD}script${NC} Run a script (e.g. op script som-debug)"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [Testing]:${NC}"
|
||||
echo -e " ${BOLD}sim${NC} Run openpilot in a simulator"
|
||||
echo -e " ${BOLD}lint${NC} Run the linter"
|
||||
echo -e " ${BOLD}post-commit${NC} Install the linter as a post-commit hook"
|
||||
echo -e " ${BOLD}test${NC} Run all unit tests"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Options:${NC}"
|
||||
echo -e " ${BOLD}-d, --dir${NC}"
|
||||
echo " Specify the openpilot directory you want to use"
|
||||
echo -e " ${BOLD}--dry${NC}"
|
||||
echo " Don't actually run anything, just print what would be run"
|
||||
echo -e " ${BOLD}-n, --no-verify${NC}"
|
||||
echo " Skip environment check before running commands"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Examples:${NC}"
|
||||
echo " op setup"
|
||||
echo " Run the setup script to install"
|
||||
echo " openpilot's dependencies."
|
||||
echo ""
|
||||
echo " op build -j4"
|
||||
echo " Compile openpilot using 4 cores"
|
||||
echo ""
|
||||
echo " op juggle --demo"
|
||||
echo " Run PlotJuggler on the demo route"
|
||||
}
|
||||
|
||||
|
||||
function _op() {
|
||||
# parse Options
|
||||
case $1 in
|
||||
-d | --dir ) shift 1; OPENPILOT_ROOT="$1"; shift 1 ;;
|
||||
--dry ) shift 1; DRY="1" ;;
|
||||
-n | --no-verify ) shift 1; NO_VERIFY="1" ;;
|
||||
esac
|
||||
|
||||
# parse Commands
|
||||
case $1 in
|
||||
auth ) shift 1; op_auth "$@" ;;
|
||||
venv ) shift 1; op_venv "$@" ;;
|
||||
check ) shift 1; op_check "$@" ;;
|
||||
esim ) shift 1; op_esim "$@" ;;
|
||||
setup ) shift 1; op_setup "$@" ;;
|
||||
build ) shift 1; op_build "$@" ;;
|
||||
juggle ) shift 1; op_juggle "$@" ;;
|
||||
cabana ) shift 1; op_cabana "$@" ;;
|
||||
lint ) shift 1; op_lint "$@" ;;
|
||||
test ) shift 1; op_test "$@" ;;
|
||||
replay ) shift 1; op_replay "$@" ;;
|
||||
clip ) shift 1; op_clip "$@" ;;
|
||||
sim ) shift 1; op_sim "$@" ;;
|
||||
switch ) shift 1; op_switch "$@" ;;
|
||||
start ) shift 1; op_start "$@" ;;
|
||||
stop ) shift 1; op_stop "$@" ;;
|
||||
restart ) shift 1; op_restart "$@" ;;
|
||||
post-commit ) shift 1; op_install_post_commit "$@" ;;
|
||||
adb ) shift 1; op_adb "$@" ;;
|
||||
ssh ) shift 1; op_ssh "$@" ;;
|
||||
script ) shift 1; op_script "$@" ;;
|
||||
* ) op_default "$@" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
_op "$@"
|
||||
@@ -0,0 +1,35 @@
|
||||
# openpilot releases
|
||||
|
||||
```
|
||||
## release checklist
|
||||
|
||||
### Go to staging
|
||||
- [ ] make a GitHub issue to track release with this checklist
|
||||
- [ ] create release master branch
|
||||
- [ ] create a branch from upstream master named `zerotentwo` for release `v0.10.2`
|
||||
- [ ] revert risky commits (double check with autonomy team)
|
||||
- [ ] push the new branch
|
||||
- [ ] push to staging:
|
||||
- [ ] make sure you are on the newly created release master branch (`zerotentwo`)
|
||||
- [ ] run `BRANCH=devel-staging tools/release/build_stripped.sh`. Jenkins will then automatically build staging on device, run `test_onroad` and update the staging branch
|
||||
- [ ] bump version on master: `openpilot/common/version.h` and `RELEASES.md`
|
||||
- [ ] post on Discord, tag `@release crew`
|
||||
|
||||
### Go to release
|
||||
- [ ] before going to release, test the following:
|
||||
- [ ] update from previous release -> new release
|
||||
- [ ] update from new release -> previous release
|
||||
- [ ] fresh install with `openpilot-test.comma.ai`
|
||||
- [ ] drive on fresh install
|
||||
- [ ] no submodules or LFS
|
||||
- [ ] check sentry, MTBF, etc.
|
||||
- [ ] stress test passes in production
|
||||
- [ ] publish the blog post
|
||||
- [ ] `git reset --hard origin/release-mici-staging`
|
||||
- [ ] tag the release: `git tag v0.X.X <commit-hash> && git push origin v0.X.X`
|
||||
- [ ] create GitHub release
|
||||
- [ ] final test install on `openpilot.comma.ai`
|
||||
- [ ] update factory provisioning
|
||||
- [ ] close out milestone and issue
|
||||
- [ ] post on Discord, X, etc.
|
||||
```
|
||||
Executable
+111
@@ -0,0 +1,111 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
set -x
|
||||
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
|
||||
cd $DIR
|
||||
|
||||
BUILD_DIR=/data/openpilot
|
||||
SOURCE_DIR="$(git rev-parse --show-toplevel)"
|
||||
|
||||
export PYTHONPATH="$BUILD_DIR:$BUILD_DIR/msgq_repo:$BUILD_DIR/opendbc_repo:$BUILD_DIR/rednose_repo:$BUILD_DIR/teleoprtc_repo:$BUILD_DIR/tinygrad_repo"
|
||||
|
||||
if [ -z "$RELEASE_BRANCH" ]; then
|
||||
echo "RELEASE_BRANCH is not set"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BUILD_BRANCH=release-mici-staging
|
||||
|
||||
|
||||
# set git identity
|
||||
source $DIR/identity.sh
|
||||
|
||||
echo "[-] Setting up repo T=$SECONDS"
|
||||
if ! git -C "$SOURCE_DIR" worktree remove --force "$BUILD_DIR" 2>/dev/null; then
|
||||
rm -rf $BUILD_DIR
|
||||
fi
|
||||
git -C "$SOURCE_DIR" worktree prune
|
||||
git -C "$SOURCE_DIR" worktree add --detach --no-checkout "$BUILD_DIR"
|
||||
cd $BUILD_DIR
|
||||
git update-ref -d "refs/heads/$BUILD_BRANCH"
|
||||
git symbolic-ref HEAD "refs/heads/$BUILD_BRANCH"
|
||||
git read-tree --empty
|
||||
|
||||
# do the files copy
|
||||
echo "[-] copying files T=$SECONDS"
|
||||
cd $SOURCE_DIR
|
||||
./tools/release/release_files.py | xargs -0 cp -pR --parents -t "$BUILD_DIR" --
|
||||
|
||||
# in the directory
|
||||
cd $BUILD_DIR
|
||||
|
||||
# use the full CPU available for speeding up the build.
|
||||
# openpilot resets the CPU frequencies when test_onroad.py runs below.
|
||||
for policy in /sys/devices/system/cpu/cpufreq/policy*; do
|
||||
[ -d "$policy" ] || continue
|
||||
hardware_max="$(cat "$policy/cpuinfo_max_freq")"
|
||||
echo "$hardware_max" | sudo tee "$policy/scaling_max_freq" >/dev/null
|
||||
done
|
||||
|
||||
scons
|
||||
if [ -n "$INCLUDE_BIG_MODEL" ]; then
|
||||
test -f openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl.chunkmanifest
|
||||
fi
|
||||
|
||||
if [ -z "$PANDA_DEBUG_BUILD" ]; then
|
||||
# release panda fw
|
||||
CERT=/data/pandaextra/certs/release RELEASE=1 scons panda/
|
||||
else
|
||||
# build with ALLOW_DEBUG=1 to enable features like experimental longitudinal
|
||||
scons panda/
|
||||
fi
|
||||
|
||||
# Ensure no submodules in release
|
||||
if test "$(git submodule--helper list | wc -l)" -gt "0"; then
|
||||
echo "submodules found:"
|
||||
git submodule--helper list
|
||||
exit 1
|
||||
fi
|
||||
git submodule status
|
||||
|
||||
# Cleanup
|
||||
find . -name '*.a' -delete
|
||||
find . -name '*.o' -delete
|
||||
find . -name '*.os' -delete
|
||||
find . -name '*.pyc' -delete
|
||||
find . -name '__pycache__' -delete
|
||||
rm -rf .sconsign.dblite Jenkinsfile tools/release/
|
||||
rm -f openpilot/selfdrive/modeld/models/*.onnx*
|
||||
rm -f openpilot/sunnypilot/modeld*/models/*.onnx*
|
||||
|
||||
find openpilot/third_party/ -name '*x86*' -exec rm -r {} +
|
||||
find openpilot/third_party/ -name '*Darwin*' -exec rm -r {} +
|
||||
|
||||
|
||||
# Restore third_party
|
||||
git checkout openpilot/third_party/
|
||||
|
||||
# Mark as prebuilt release
|
||||
touch prebuilt
|
||||
|
||||
VERSION=$(cat openpilot/sunnypilot/common/version.h | awk -F[\"-] '{print $2}')
|
||||
# Add built files to git
|
||||
# writing larger objects is faster than compressing them on-device
|
||||
git -c core.compression=0 add -f .
|
||||
git -c core.compression=0 -c gc.auto=0 commit -m "openpilot v$VERSION"
|
||||
|
||||
# Run tests
|
||||
cd $BUILD_DIR
|
||||
RELEASE=1 ./openpilot/selfdrive/test/test_onroad.py
|
||||
#tools/test_runner.py openpilot/selfdrive/car/tests/test_car_interfaces.py
|
||||
|
||||
echo "[-] pushing release T=$SECONDS"
|
||||
REFS=()
|
||||
for branch in ${RELEASE_BRANCH//,/ }; do
|
||||
REFS+=("$BUILD_BRANCH:$branch")
|
||||
done
|
||||
# uploading the larger pack is faster than spending CPU to optimize it
|
||||
git -c pack.window=0 -c pack.depth=0 -c pack.compression=0 push -f origin "${REFS[@]}"
|
||||
|
||||
echo "[-] done T=$SECONDS"
|
||||
Executable
+85
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
|
||||
|
||||
SOURCE_DIR="$(git -C $DIR rev-parse --show-toplevel)"
|
||||
if [ -z "$TARGET_DIR" ]; then
|
||||
TARGET_DIR="$(mktemp -d)"
|
||||
fi
|
||||
|
||||
# set git identity
|
||||
source $DIR/identity.sh
|
||||
|
||||
echo "[-] Setting up target repo T=$SECONDS"
|
||||
|
||||
rm -rf $TARGET_DIR
|
||||
mkdir -p $TARGET_DIR
|
||||
cd $TARGET_DIR
|
||||
cp -r $SOURCE_DIR/.git $TARGET_DIR
|
||||
|
||||
echo "[-] setting up stripped branch sync T=$SECONDS"
|
||||
cd $TARGET_DIR
|
||||
|
||||
# tmp branch
|
||||
git checkout --orphan tmp
|
||||
|
||||
# remove everything except .git
|
||||
echo "[-] erasing old sunnypilot T=$SECONDS"
|
||||
git submodule deinit -f --all
|
||||
git rm -rf --cached .
|
||||
find . -maxdepth 1 -not -path './.git' -not -name '.' -not -name '..' -exec rm -rf '{}' \;
|
||||
|
||||
# do the files copy
|
||||
echo "[-] copying files T=$SECONDS"
|
||||
cd $SOURCE_DIR
|
||||
./tools/release/release_files.py | xargs -0 cp -pR --parents -t "$TARGET_DIR" --
|
||||
|
||||
# in the directory
|
||||
cd $TARGET_DIR
|
||||
rm -rf .git/modules/
|
||||
|
||||
find openpilot/selfdrive/modeld/models -name '*.onnx' -size +95M -exec ./openpilot/common/file_chunker.py {} \;
|
||||
|
||||
# include source commit hash and build date in commit
|
||||
GIT_HASH=$(git --git-dir=$SOURCE_DIR/.git rev-parse HEAD)
|
||||
GIT_COMMIT_DATE=$(git --git-dir=$SOURCE_DIR/.git show --no-patch --format='%ct %ci' HEAD)
|
||||
DATETIME=$(date '+%Y-%m-%dT%H:%M:%S')
|
||||
VERSION=$(cat $SOURCE_DIR/openpilot/sunnypilot/common/version.h | awk -F\" '{print $2}')
|
||||
|
||||
echo -n "$GIT_HASH" > git_src_commit
|
||||
echo -n "$GIT_COMMIT_DATE" > git_src_commit_date
|
||||
|
||||
echo "[-] committing version $VERSION T=$SECONDS"
|
||||
# writing larger objects is faster than compressing them on-device
|
||||
git -c core.compression=0 add -f .
|
||||
git status
|
||||
git -c core.compression=0 commit -a -m "sunnypilot v$VERSION release
|
||||
|
||||
date: $DATETIME
|
||||
master commit: $GIT_HASH
|
||||
"
|
||||
|
||||
# should be no submodules or LFS files
|
||||
git submodule status
|
||||
if [ ! -z "$(git lfs ls-files)" ]; then
|
||||
echo "LFS files detected!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ensure files are within GitHub's limit
|
||||
BIG_FILES="$(find . -type f -not -path './.git/*' -size +95M)"
|
||||
if [ ! -z "$BIG_FILES" ]; then
|
||||
printf '\n\n\n'
|
||||
echo "Found files exceeding GitHub's 100MB limit:"
|
||||
echo "$BIG_FILES"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -z "$BRANCH" ]; then
|
||||
echo "[-] Pushing to $BRANCH T=$SECONDS"
|
||||
# uploading the larger pack is faster than spending CPU to optimize it
|
||||
git -c pack.window=0 -c pack.depth=0 -c pack.compression=0 push -f origin tmp:$BRANCH
|
||||
fi
|
||||
|
||||
echo "[-] done T=$SECONDS, ready at $TARGET_DIR"
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
|
||||
cd $DIR
|
||||
|
||||
if [ ! -z "$(git status --porcelain)" ]; then
|
||||
echo "Dirty working tree after build:"
|
||||
git status --porcelain
|
||||
exit 1
|
||||
fi
|
||||
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
has_submodule_changes() {
|
||||
local submodule_path="$1"
|
||||
if [ -n "$SUBMODULE_PATHS" ]; then
|
||||
echo "$SUBMODULE_PATHS" | grep -q "$submodule_path"
|
||||
return $?
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
while read hash submodule ref; do
|
||||
if [ -z "$hash" ] || [ -z "$submodule" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
hash=$(echo "$hash" | sed 's/^[+-]//')
|
||||
|
||||
if [ "$submodule" = "tinygrad_repo" ]; then
|
||||
echo "Skipping $submodule"
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ "$CHECK_PR_REFS" = "true" ] && has_submodule_changes "$submodule"; then
|
||||
echo "Checking $submodule (non-master): verifying hash $hash exists"
|
||||
git -C $submodule fetch --depth 100 origin
|
||||
if git -C $submodule cat-file -e $hash 2>/dev/null; then
|
||||
echo "$submodule ok (hash exists)"
|
||||
else
|
||||
echo "$submodule: $hash does not exist in the repository"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
git -C $submodule fetch --depth 100 origin master
|
||||
git -C $submodule branch -r --contains $hash | grep "origin/master"
|
||||
if [ "$?" -eq 0 ]; then
|
||||
echo "$submodule ok"
|
||||
else
|
||||
echo "$submodule: $hash is not on master"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
done <<< $(git submodule status --recursive)
|
||||
@@ -0,0 +1,4 @@
|
||||
export GIT_COMMITTER_NAME="github-actions[bot]"
|
||||
export GIT_COMMITTER_EMAIL="github-actions[bot]@users.noreply.github.com"
|
||||
export GIT_AUTHOR_NAME="github-actions[bot]"
|
||||
export GIT_AUTHOR_EMAIL="github-actions[bot]@users.noreply.github.com"
|
||||
Executable
+53
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import importlib
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import zipapp
|
||||
from argparse import ArgumentParser
|
||||
from pathlib import Path
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
|
||||
|
||||
DIRS = ['openpilot']
|
||||
EXTS = ['.png', '.py', '.ttf', '.capnp', '.json', '.fnt', '.mo', '.po']
|
||||
EXCLUDE = ['openpilot/selfdrive/assets/training']
|
||||
INTERPRETER = '/usr/bin/env python3'
|
||||
|
||||
|
||||
def copy(src, dest):
|
||||
if any(src.endswith(ext) for ext in EXTS) and not any(exc in src for exc in EXCLUDE):
|
||||
shutil.copy2(src, dest, follow_symlinks=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = ArgumentParser(prog='pack.py', description="package script into a portable executable", epilog='comma.ai')
|
||||
parser.add_argument('-e', '--entrypoint', help="function to call in module, default is 'main'", default='main')
|
||||
parser.add_argument('-o', '--output', help='output file')
|
||||
parser.add_argument('module', help="the module to target, e.g. 'openpilot.system.ui.spinner'")
|
||||
args = parser.parse_args()
|
||||
|
||||
print('WARNING: copying all files! make sure to run scons and git tree is clean')
|
||||
|
||||
if not args.output:
|
||||
args.output = args.module
|
||||
|
||||
try:
|
||||
mod = importlib.import_module(args.module)
|
||||
except ModuleNotFoundError:
|
||||
print(f'{args.module} not found, typo?')
|
||||
sys.exit(1)
|
||||
|
||||
if not hasattr(mod, args.entrypoint):
|
||||
print(f'{args.module} does not have a {args.entrypoint}() function, typo?')
|
||||
sys.exit(1)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
for directory in DIRS:
|
||||
shutil.copytree(BASEDIR + '/' + directory, tmp + '/' + directory, symlinks=False, dirs_exist_ok=True, copy_function=copy)
|
||||
entry = f'{args.module}:{args.entrypoint}'
|
||||
zipapp.create_archive(tmp, target=args.output, interpreter=INTERPRETER, main=entry)
|
||||
|
||||
print(f'created executable {Path(args.output).resolve()}')
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HERE = os.path.abspath(os.path.dirname(__file__))
|
||||
ROOT = os.path.abspath(os.path.join(HERE, "../.."))
|
||||
|
||||
blacklist = [
|
||||
".git/",
|
||||
".venv/",
|
||||
".github/workflows/",
|
||||
|
||||
"matlab.*.md",
|
||||
|
||||
# no LFS or submodules in release
|
||||
".lfsconfig",
|
||||
".gitattributes",
|
||||
".git$",
|
||||
".gitmodules",
|
||||
".run/",
|
||||
".idea/",
|
||||
]
|
||||
|
||||
# gets you through the blacklist
|
||||
whitelist: list[str] = [
|
||||
]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
tracked_files = subprocess.check_output(["git", "ls-files", "-z", "--recurse-submodules"], cwd=ROOT).split(b"\0")
|
||||
for tracked_file in tracked_files:
|
||||
if not tracked_file:
|
||||
continue
|
||||
|
||||
rf = os.fsdecode(tracked_file)
|
||||
if not os.getenv("INCLUDE_BIG_MODEL") and rf.startswith("openpilot/selfdrive/modeld/models/big_driving_supercombo.onnx"):
|
||||
continue
|
||||
blacklisted = any(re.search(p, rf) for p in blacklist)
|
||||
whitelisted = any(re.search(p, rf) for p in whitelist)
|
||||
if blacklisted and not whitelisted:
|
||||
continue
|
||||
|
||||
sys.stdout.buffer.write(tracked_file + b"\0")
|
||||
Executable
+42
@@ -0,0 +1,42 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Forward all openpilot service ports
|
||||
while IFS=' ' read -r name port; do
|
||||
adb forward "tcp:${port}" "tcp:${port}" > /dev/null
|
||||
done < <(python3 - <<'PY'
|
||||
from openpilot.cereal.services import SERVICE_LIST
|
||||
|
||||
FNV_PRIME = 0x100000001b3
|
||||
FNV_OFFSET_BASIS = 0xcbf29ce484222325
|
||||
START_PORT = 8023
|
||||
MAX_PORT = 65535
|
||||
PORT_RANGE = MAX_PORT - START_PORT
|
||||
MASK = 0xffffffffffffffff
|
||||
|
||||
def fnv1a(endpoint: str) -> int:
|
||||
h = FNV_OFFSET_BASIS
|
||||
for b in endpoint.encode():
|
||||
h ^= b
|
||||
h = (h * FNV_PRIME) & MASK
|
||||
return h
|
||||
|
||||
ports = set()
|
||||
for name in SERVICE_LIST.keys():
|
||||
port = START_PORT + fnv1a(name) % PORT_RANGE
|
||||
ports.add((name, port))
|
||||
|
||||
for name, port in sorted(ports):
|
||||
print(f"{name} {port}")
|
||||
PY
|
||||
)
|
||||
|
||||
# Forward SSH port, finding a free local port if 2222 is taken.
|
||||
SSH_PORT=2222
|
||||
while ss -tln | grep -q ":${SSH_PORT} "; do
|
||||
SSH_PORT=$((SSH_PORT + 1))
|
||||
done
|
||||
adb forward tcp:${SSH_PORT} tcp:22
|
||||
|
||||
# SSH!
|
||||
ssh comma@localhost -p ${SSH_PORT} "$@"
|
||||
Executable
+109
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import binascii
|
||||
import time
|
||||
from collections import defaultdict
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from tools.scripts.car.can_table import can_table
|
||||
from openpilot.tools.lib.logreader import LogIterable, LogReader
|
||||
|
||||
RED = '\033[91m'
|
||||
CLEAR = '\033[0m'
|
||||
|
||||
def update(msgs, bus, dat, low_to_high, high_to_low, quiet=False):
|
||||
for x in msgs:
|
||||
if x.which() != 'can':
|
||||
continue
|
||||
|
||||
for y in x.can:
|
||||
if y.src == bus:
|
||||
dat[y.address] = y.dat
|
||||
|
||||
i = int.from_bytes(y.dat, byteorder='big')
|
||||
l_h = low_to_high[y.address]
|
||||
h_l = high_to_low[y.address]
|
||||
|
||||
change = None
|
||||
if (i | l_h) != l_h:
|
||||
low_to_high[y.address] = i | l_h
|
||||
change = "+"
|
||||
|
||||
if (~i | h_l) != h_l:
|
||||
high_to_low[y.address] = ~i | h_l
|
||||
change = "-"
|
||||
|
||||
if change and not quiet:
|
||||
print(f"{time.monotonic():.2f}\t{hex(y.address)} ({y.address})\t{change}{binascii.hexlify(y.dat)}")
|
||||
|
||||
|
||||
def can_printer(bus=0, init_msgs=None, new_msgs=None, table=False):
|
||||
logcan = messaging.sub_sock('can', timeout=10)
|
||||
|
||||
dat = defaultdict(int)
|
||||
low_to_high = defaultdict(int)
|
||||
high_to_low = defaultdict(int)
|
||||
|
||||
if init_msgs is not None:
|
||||
update(init_msgs, bus, dat, low_to_high, high_to_low, quiet=True)
|
||||
|
||||
low_to_high_init = low_to_high.copy()
|
||||
high_to_low_init = high_to_low.copy()
|
||||
|
||||
if new_msgs is not None:
|
||||
update(new_msgs, bus, dat, low_to_high, high_to_low)
|
||||
else:
|
||||
# Live mode
|
||||
print(f"Waiting for messages on bus {bus}")
|
||||
try:
|
||||
while 1:
|
||||
can_recv = messaging.drain_sock(logcan)
|
||||
update(can_recv, bus, dat, low_to_high, high_to_low)
|
||||
time.sleep(0.02)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
print("\n\n")
|
||||
tables = ""
|
||||
for addr in sorted(dat.keys()):
|
||||
init = low_to_high_init[addr] & high_to_low_init[addr]
|
||||
now = low_to_high[addr] & high_to_low[addr]
|
||||
d = now & ~init
|
||||
if d == 0:
|
||||
continue
|
||||
b = d.to_bytes(len(dat[addr]), byteorder='big')
|
||||
|
||||
byts = ''.join([(c if c == '0' else f'{RED}{c}{CLEAR}') for c in str(binascii.hexlify(b))[2:-1]])
|
||||
header = f"{hex(addr).ljust(6)}({str(addr).ljust(4)})"
|
||||
print(header, byts)
|
||||
tables += f"{header}\n"
|
||||
tables += can_table(b) + "\n\n"
|
||||
|
||||
if table:
|
||||
print(tables)
|
||||
|
||||
if __name__ == "__main__":
|
||||
desc = """Collects messages and prints when a new bit transition is observed.
|
||||
This is very useful to find signals based on user triggered actions, such as blinkers and seatbelt.
|
||||
Leave the script running until no new transitions are seen, then perform the action."""
|
||||
parser = argparse.ArgumentParser(description=desc,
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument("--bus", type=int, help="CAN bus to print out", default=0)
|
||||
parser.add_argument("--table", action="store_true", help="Print a cabana-like table")
|
||||
parser.add_argument("init", type=str, nargs='?', help="Route or segment to initialize with. Use empty quotes to compare against all zeros.")
|
||||
parser.add_argument("comp", type=str, nargs='?', help="Route or segment to compare against init")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
init_lr: LogIterable | None = None
|
||||
new_lr: LogIterable | None = None
|
||||
|
||||
if args.init:
|
||||
if args.init == '':
|
||||
init_lr = []
|
||||
else:
|
||||
init_lr = LogReader(args.init)
|
||||
if args.comp:
|
||||
new_lr = LogReader(args.comp)
|
||||
|
||||
can_printer(args.bus, init_msgs=init_lr, new_msgs=new_lr, table=args.table)
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import binascii
|
||||
import time
|
||||
from collections import defaultdict
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
|
||||
|
||||
def can_printer(bus, max_msg, addr, ascii_decode):
|
||||
logcan = messaging.sub_sock('can', addr=addr)
|
||||
|
||||
start = time.monotonic()
|
||||
lp = time.monotonic()
|
||||
msgs = defaultdict(list)
|
||||
while 1:
|
||||
can_recv = messaging.drain_sock(logcan, wait_for_one=True)
|
||||
for x in can_recv:
|
||||
for y in x.can:
|
||||
if y.src == bus:
|
||||
msgs[y.address].append(y.dat)
|
||||
|
||||
if time.monotonic() - lp > 0.1:
|
||||
dd = chr(27) + "[2J"
|
||||
dd += f"{time.monotonic() - start:5.2f}\n"
|
||||
for _addr in sorted(msgs.keys()):
|
||||
a = f"\"{msgs[_addr][-1].decode('ascii', 'backslashreplace')}\"" if ascii_decode else ""
|
||||
x = binascii.hexlify(msgs[_addr][-1]).decode('ascii')
|
||||
freq = len(msgs[_addr]) / (time.monotonic() - start)
|
||||
if max_msg is None or _addr < max_msg:
|
||||
dd += f"{_addr:04X}({_addr:4d})({len(msgs[_addr]):6d})({freq:3}dHz) {x.ljust(20)} {a}\n"
|
||||
print(dd)
|
||||
lp = time.monotonic()
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="simple CAN data viewer",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--bus", type=int, help="CAN bus to print out", default=0)
|
||||
parser.add_argument("--max_msg", type=int, help="max addr")
|
||||
parser.add_argument("--ascii", action='store_true', help="decode as ascii")
|
||||
parser.add_argument("--addr", default="127.0.0.1")
|
||||
|
||||
args = parser.parse_args()
|
||||
can_printer(args.bus, args.max_msg, args.addr, args.ascii)
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import pandas as pd
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
|
||||
|
||||
def can_table(dat):
|
||||
rows = []
|
||||
for b in dat:
|
||||
r = list(bin(b).lstrip('0b').zfill(8))
|
||||
r += [hex(b)]
|
||||
rows.append(r)
|
||||
|
||||
df = pd.DataFrame(data=rows)
|
||||
df.columns = [str(n) for n in range(7, -1, -1)] + [' ']
|
||||
table = df.to_markdown(tablefmt='grid')
|
||||
return table
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Cabana-like table of bits for your terminal",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument("addr", type=str, nargs=1)
|
||||
parser.add_argument("bus", type=int, default=0, nargs='?')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
addr = int(args.addr[0], 0)
|
||||
can = messaging.sub_sock('can', conflate=False, timeout=None)
|
||||
|
||||
print(f"waiting for {hex(addr)} ({addr}) on bus {args.bus}...")
|
||||
|
||||
latest = None
|
||||
while True:
|
||||
for msg in messaging.drain_sock(can, wait_for_one=True):
|
||||
for m in msg.can:
|
||||
if m.address == addr and m.src == args.bus:
|
||||
latest = m
|
||||
|
||||
if latest is None:
|
||||
continue
|
||||
|
||||
table = can_table(latest.dat)
|
||||
print(f"\n\n{hex(addr)} ({addr}) on bus {args.bus}\n{table}")
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import argparse
|
||||
from subprocess import check_output, CalledProcessError
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.uds import UdsClient, MessageTimeoutError, SESSION_TYPE, DTC_GROUP_TYPE
|
||||
from opendbc.car.structs import CarParams
|
||||
from panda import Panda
|
||||
|
||||
parser = argparse.ArgumentParser(description="clear DTC status")
|
||||
parser.add_argument("addr", type=lambda x: int(x,0), nargs="?", default=0x7DF) # default is functional (broadcast) address
|
||||
parser.add_argument("--bus", type=int, default=0)
|
||||
parser.add_argument('--debug', action='store_true')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
try:
|
||||
check_output(["pidof", "pandad"])
|
||||
print("pandad is running, please kill openpilot before running this script! (aborted)")
|
||||
sys.exit(1)
|
||||
except CalledProcessError as e:
|
||||
if e.returncode != 1: # 1 == no process found (pandad not running)
|
||||
raise e
|
||||
|
||||
panda = Panda()
|
||||
panda.set_safety_mode(CarParams.SafetyModel.elm327)
|
||||
uds_client = UdsClient(panda, args.addr, bus=args.bus)
|
||||
print("extended diagnostic session ...")
|
||||
try:
|
||||
uds_client.diagnostic_session_control(SESSION_TYPE.EXTENDED_DIAGNOSTIC)
|
||||
except MessageTimeoutError:
|
||||
# functional address isn't properly handled so a timeout occurs
|
||||
if args.addr != 0x7DF:
|
||||
raise
|
||||
print("clear diagnostic info ...")
|
||||
try:
|
||||
uds_client.clear_diagnostic_information(DTC_GROUP_TYPE.ALL)
|
||||
except MessageTimeoutError:
|
||||
# functional address isn't properly handled so a timeout occurs
|
||||
if args.addr != 0x7DF:
|
||||
pass
|
||||
print("")
|
||||
print("you may need to power cycle your vehicle now")
|
||||
Executable
+15
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from opendbc.car.disable_ecu import disable_ecu
|
||||
from openpilot.selfdrive.car.card import can_comm_callbacks
|
||||
|
||||
if __name__ == "__main__":
|
||||
sendcan = messaging.pub_sock('sendcan')
|
||||
logcan = messaging.sub_sock('can')
|
||||
can_callbacks = can_comm_callbacks(logcan, sendcan)
|
||||
time.sleep(1)
|
||||
|
||||
# honda bosch radar disable
|
||||
disabled = disable_ecu(*can_callbacks, bus=1, addr=0x18DAB0F1, com_cont_req=b'\x28\x83\x03', timeout=0.5)
|
||||
print(f"disabled: {disabled}")
|
||||
Executable
+44
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import time
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.ecu_addrs import get_all_ecu_addrs
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.selfdrive.car.card import can_comm_callbacks, obd_callback
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='Get addresses of all ECUs')
|
||||
parser.add_argument('--debug', action='store_true')
|
||||
parser.add_argument('--bus', type=int, default=1)
|
||||
parser.add_argument('--no-obd', action='store_true')
|
||||
parser.add_argument('--timeout', type=float, default=1.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
logcan = messaging.sub_sock('can')
|
||||
sendcan = messaging.pub_sock('sendcan')
|
||||
can_callbacks = can_comm_callbacks(logcan, sendcan)
|
||||
|
||||
# Set up params for pandad
|
||||
params = Params()
|
||||
params.remove("FirmwareQueryDone")
|
||||
params.put_bool("IsOffroad", True, block=True)
|
||||
time.sleep(0.2) # thread is 10 Hz
|
||||
params.put_bool("IsOffroad", False, block=True)
|
||||
|
||||
obd_callback(params)(not args.no_obd)
|
||||
|
||||
print("Getting ECU addresses ...")
|
||||
ecu_addrs = get_all_ecu_addrs(*can_callbacks, args.bus, args.timeout)
|
||||
|
||||
print()
|
||||
print("Found ECUs on rx addresses:")
|
||||
for addr, subaddr, _ in ecu_addrs:
|
||||
msg = f" {hex(addr)}"
|
||||
if subaddr is not None:
|
||||
msg += f" (sub-address: {hex(subaddr)})"
|
||||
print(msg)
|
||||
Executable
+72
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
import argparse
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from opendbc.car.structs import car
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.fw_versions import get_fw_versions, match_fw_to_car
|
||||
from opendbc.car.vin import get_vin
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.selfdrive.car.card import can_comm_callbacks, obd_callback
|
||||
from typing import Any
|
||||
|
||||
Ecu = car.CarParams.Ecu
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='Get firmware version of ECUs')
|
||||
parser.add_argument('--scan', action='store_true')
|
||||
parser.add_argument('--debug', action='store_true')
|
||||
parser.add_argument('--brand', help='Only query addresses/with requests for this brand')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
logcan = messaging.sub_sock('can')
|
||||
pandaStates_sock = messaging.sub_sock('pandaStates')
|
||||
sendcan = messaging.pub_sock('sendcan')
|
||||
can_callbacks = can_comm_callbacks(logcan, sendcan)
|
||||
|
||||
# Set up params for pandad
|
||||
params = Params()
|
||||
params.remove("FirmwareQueryDone")
|
||||
params.put_bool("IsOffroad", True, block=True)
|
||||
time.sleep(0.2) # thread is 10 Hz
|
||||
params.put_bool("IsOffroad", False, block=True)
|
||||
set_obd_multiplexing = obd_callback(params)
|
||||
|
||||
extra: Any = None
|
||||
if args.scan:
|
||||
extra = {}
|
||||
# Honda
|
||||
for i in range(256):
|
||||
extra[(Ecu.unknown, 0x18da00f1 + (i << 8), None)] = []
|
||||
extra[(Ecu.unknown, 0x700 + i, None)] = []
|
||||
extra[(Ecu.unknown, 0x750, i)] = []
|
||||
extra = {"any": {"debug": extra}}
|
||||
|
||||
t = time.monotonic()
|
||||
print("Getting vin...")
|
||||
set_obd_multiplexing(True)
|
||||
vin_rx_addr, vin_rx_bus, vin = get_vin(*can_callbacks, (0, 1))
|
||||
print(f'RX: {hex(vin_rx_addr)}, BUS: {vin_rx_bus}, VIN: {vin}')
|
||||
print(f"Getting VIN took {time.monotonic() - t:.3f} s")
|
||||
print()
|
||||
|
||||
t = time.monotonic()
|
||||
fw_vers = get_fw_versions(*can_callbacks, set_obd_multiplexing, query_brand=args.brand, extra=extra, progress=True)
|
||||
_, candidates = match_fw_to_car(fw_vers, vin)
|
||||
|
||||
print()
|
||||
print("Found FW versions")
|
||||
print("{")
|
||||
padding = max([len(fw.brand) for fw in fw_vers] or [0])
|
||||
for version in fw_vers:
|
||||
subaddr = None if version.subAddress == 0 else hex(version.subAddress)
|
||||
print(f" Brand: {version.brand:{padding}}, bus: {version.bus}, OBD: {version.obdMultiplexing} - " +
|
||||
f"(Ecu.{version.ecu}, {hex(version.address)}, {subaddr}): [{version.fwVersion!r}]")
|
||||
print("}")
|
||||
|
||||
print()
|
||||
print("Possible matches:", candidates)
|
||||
print(f"Getting fw took {time.monotonic() - t:.3f} s")
|
||||
+142
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Some Hyundai radars can be reconfigured to output (debug) radar points on bus 1.
|
||||
Reconfiguration is done over UDS by reading/writing to 0x0142 using the Read/Write Data By Identifier
|
||||
endpoints (0x22 & 0x2E). This script checks your radar firmware version against a list of known
|
||||
firmware versions. If you want to try on a new radar make sure to note the default config value
|
||||
in case it's different from the other radars and you need to revert the changes.
|
||||
|
||||
After changing the config the car should not show any faults when openpilot is not running.
|
||||
These config changes are persistent across car reboots. You need to run this script again
|
||||
to go back to the default values.
|
||||
|
||||
USE AT YOUR OWN RISK! Safety features, like AEB and FCW, might be affected by these changes."""
|
||||
|
||||
import sys
|
||||
import argparse
|
||||
from typing import NamedTuple
|
||||
from subprocess import check_output, CalledProcessError
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.uds import UdsClient, SESSION_TYPE, DATA_IDENTIFIER_TYPE
|
||||
from opendbc.car.structs import CarParams
|
||||
from panda.python import Panda
|
||||
|
||||
class ConfigValues(NamedTuple):
|
||||
default_config: bytes
|
||||
tracks_enabled: bytes
|
||||
|
||||
# If your radar supports changing data identifier 0x0142 as well make a PR to
|
||||
# this file to add your firmware version. Make sure to post a drive as proof!
|
||||
# NOTE: these firmware versions do not match what openpilot uses
|
||||
# because this script uses a different diagnostic session type
|
||||
SUPPORTED_FW_VERSIONS = {
|
||||
# 2020 SONATA
|
||||
b"DN8_ SCC FHCUP 1.00 1.00 99110-L0000\x19\x08)\x15T ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
b"DN8_ SCC F-CUP 1.00 1.00 99110-L0000\x19\x08)\x15T ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2021 SONATA HYBRID
|
||||
b"DNhe SCC FHCUP 1.00 1.00 99110-L5000\x19\x04&\x13' ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
b"DNhe SCC FHCUP 1.00 1.02 99110-L5000 \x01#\x15# ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2020 PALISADE
|
||||
b"LX2_ SCC FHCUP 1.00 1.04 99110-S8100\x19\x05\x02\x16V ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2022 PALISADE
|
||||
b"LX2_ SCC FHCUP 1.00 1.00 99110-S8110!\x04\x05\x17\x01 ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2020 SANTA FE
|
||||
b"TM__ SCC F-CUP 1.00 1.03 99110-S2000\x19\x050\x13' ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2020 GENESIS G70
|
||||
b'IK__ SCC F-CUP 1.00 1.02 96400-G9100\x18\x07\x06\x17\x12 ': ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2019 SANTA FE
|
||||
b"TM__ SCC F-CUP 1.00 1.00 99110-S1210\x19\x01%\x168 ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
b"TM__ SCC F-CUP 1.00 1.02 99110-S2000\x18\x07\x08\x18W ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2021 K5 HEV
|
||||
b"DLhe SCC FHCUP 1.00 1.02 99110-L7000 \x01 \x102 ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
# 2022 Niro EV
|
||||
b"DEev SCC F-CUP 1.00 1.00 99110-Q4600\x01\x42 ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
b"DEev SCC F-CUP 1.00 1.00 99110-Q4600 \x07\x03\t% ": ConfigValues(
|
||||
default_config=b"\x00\x00\x00\x01\x00\x00",
|
||||
tracks_enabled=b"\x00\x00\x00\x01\x00\x01"),
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='configure radar to output points (or reset to default)')
|
||||
parser.add_argument('--default', action="store_true", default=False, help='reset to default configuration (default: false)')
|
||||
parser.add_argument('--debug', action="store_true", default=False, help='enable debug output (default: false)')
|
||||
parser.add_argument('--bus', type=int, default=0, help='can bus to use (default: 0)')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
try:
|
||||
check_output(["pidof", "pandad"])
|
||||
print("pandad is running, please kill openpilot before running this script! (aborted)")
|
||||
sys.exit(1)
|
||||
except CalledProcessError as e:
|
||||
if e.returncode != 1: # 1 == no process found (pandad not running)
|
||||
raise e
|
||||
|
||||
confirm = input("power on the vehicle keeping the engine off (press start button twice) then type OK to continue: ").upper().strip()
|
||||
if confirm != "OK":
|
||||
print("\nyou didn't type 'OK! (aborted)")
|
||||
sys.exit(0)
|
||||
|
||||
panda = Panda()
|
||||
panda.set_safety_mode(CarParams.SafetyModel.elm327)
|
||||
uds_client = UdsClient(panda, 0x7D0, bus=args.bus)
|
||||
|
||||
print("\n[START DIAGNOSTIC SESSION]")
|
||||
session_type : SESSION_TYPE = 0x07
|
||||
uds_client.diagnostic_session_control(session_type)
|
||||
|
||||
print("[HARDWARE/SOFTWARE VERSION]")
|
||||
fw_version_data_id : DATA_IDENTIFIER_TYPE = 0xf100
|
||||
fw_version = uds_client.read_data_by_identifier(fw_version_data_id)
|
||||
print(fw_version)
|
||||
if fw_version not in SUPPORTED_FW_VERSIONS.keys():
|
||||
print("radar not supported! (aborted)")
|
||||
sys.exit(1)
|
||||
|
||||
print("[GET CONFIGURATION]")
|
||||
config_data_id : DATA_IDENTIFIER_TYPE = 0x0142
|
||||
current_config = uds_client.read_data_by_identifier(config_data_id)
|
||||
config_values = SUPPORTED_FW_VERSIONS[fw_version]
|
||||
new_config = config_values.default_config if args.default else config_values.tracks_enabled
|
||||
print(f"current config: 0x{current_config.hex()}")
|
||||
if current_config != new_config:
|
||||
print("[CHANGE CONFIGURATION]")
|
||||
print(f"new config: 0x{new_config.hex()}")
|
||||
uds_client.write_data_by_identifier(config_data_id, new_config)
|
||||
if not args.default and current_config != SUPPORTED_FW_VERSIONS[fw_version].default_config:
|
||||
print("\ncurrent config does not match expected default! (aborted)")
|
||||
sys.exit(1)
|
||||
|
||||
print("[DONE]")
|
||||
print("\nrestart your vehicle and ensure there are no faults")
|
||||
if not args.default:
|
||||
print("you can run this script again with --default to go back to the original (factory) settings")
|
||||
else:
|
||||
print("[DONE]")
|
||||
print("\ncurrent config is already the desired configuration")
|
||||
sys.exit(0)
|
||||
Executable
+131
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import numpy as np
|
||||
import matplotlib.pyplot as plt
|
||||
from functools import partial
|
||||
from tqdm import tqdm
|
||||
from typing import NamedTuple
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
from openpilot.selfdrive.locationd.models.pose_kf import EARTH_G
|
||||
|
||||
RLOG_MIN_LAT_ACTIVE = 50
|
||||
RLOG_MIN_STEERING_UNPRESSED = 50
|
||||
RLOG_MIN_REQUESTING_MAX = 25 # sample many times after reaching max torque
|
||||
|
||||
QLOG_DECIMATION = 10
|
||||
|
||||
|
||||
class Event(NamedTuple):
|
||||
lateral_accel: float
|
||||
speed: float
|
||||
roll: float
|
||||
timestamp: float # relative to start of route (s)
|
||||
|
||||
|
||||
def find_events(lr: LogReader, extrapolate: bool = False, qlog: bool = False) -> list[Event]:
|
||||
min_lat_active = RLOG_MIN_LAT_ACTIVE // QLOG_DECIMATION if qlog else RLOG_MIN_LAT_ACTIVE
|
||||
min_steering_unpressed = RLOG_MIN_STEERING_UNPRESSED // QLOG_DECIMATION if qlog else RLOG_MIN_STEERING_UNPRESSED
|
||||
min_requesting_max = RLOG_MIN_REQUESTING_MAX // QLOG_DECIMATION if qlog else RLOG_MIN_REQUESTING_MAX
|
||||
|
||||
# if we test with driver torque safety, max torque can be slightly noisy
|
||||
steer_threshold = 0.7 if extrapolate else 0.95
|
||||
|
||||
events = []
|
||||
|
||||
# state tracking
|
||||
steering_unpressed = 0 # frames
|
||||
requesting_max = 0 # frames
|
||||
lat_active = 0 # frames
|
||||
|
||||
# current state
|
||||
curvature = 0
|
||||
v_ego = 0
|
||||
roll = 0
|
||||
out_torque = 0
|
||||
|
||||
start_ts = 0
|
||||
for msg in lr:
|
||||
if msg.which() == 'carControl':
|
||||
if start_ts == 0:
|
||||
start_ts = msg.logMonoTime
|
||||
|
||||
lat_active = lat_active + 1 if msg.carControl.latActive else 0
|
||||
|
||||
elif msg.which() == 'carOutput':
|
||||
out_torque = msg.carOutput.actuatorsOutput.torque
|
||||
requesting_max = requesting_max + 1 if abs(out_torque) > steer_threshold else 0
|
||||
|
||||
elif msg.which() == 'carState':
|
||||
steering_unpressed = steering_unpressed + 1 if not msg.carState.steeringPressed else 0
|
||||
v_ego = msg.carState.vEgo
|
||||
|
||||
elif msg.which() == 'controlsState':
|
||||
curvature = msg.controlsState.curvature
|
||||
|
||||
elif msg.which() == 'vehicleParameters':
|
||||
roll = msg.vehicleParameters.roll
|
||||
|
||||
if lat_active > min_lat_active and steering_unpressed > min_steering_unpressed and requesting_max > min_requesting_max:
|
||||
# TODO: record max lat accel at the end of the event, need to use the past lat accel as overriding can happen before we detect it
|
||||
requesting_max = 0
|
||||
|
||||
factor = 1 / abs(out_torque)
|
||||
current_lateral_accel = (curvature * v_ego ** 2 * factor) - roll * EARTH_G
|
||||
events.append(Event(current_lateral_accel, v_ego, roll, round((msg.logMonoTime - start_ts) * 1e-9, 2)))
|
||||
print(events[-1])
|
||||
|
||||
return events
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description="Find max lateral acceleration events",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("route", nargs='+')
|
||||
parser.add_argument("-e", "--extrapolate", action="store_true", help="Extrapolates max lateral acceleration events linearly. " +
|
||||
"This option can be far less accurate.")
|
||||
args = parser.parse_args()
|
||||
|
||||
events = []
|
||||
for route in tqdm(args.route):
|
||||
try:
|
||||
lr = LogReader(route, sort_by_time=True)
|
||||
except Exception:
|
||||
print(f'Skipping {route}')
|
||||
continue
|
||||
|
||||
qlog = route.endswith('/q')
|
||||
if qlog:
|
||||
print('WARNING: Treating route as qlog!')
|
||||
|
||||
print('Finding events...')
|
||||
events += lr.run_across_segments(8, partial(find_events, extrapolate=args.extrapolate, qlog=qlog), disable_tqdm=True)
|
||||
|
||||
print()
|
||||
print(f'Found {len(events)} events')
|
||||
|
||||
perc_left_accel = -np.percentile([-ev.lateral_accel for ev in events if ev.lateral_accel < 0] or [0], 90)
|
||||
perc_right_accel = np.percentile([ev.lateral_accel for ev in events if ev.lateral_accel > 0] or [0], 90)
|
||||
|
||||
CP = lr.first('carParams')
|
||||
|
||||
plt.ion()
|
||||
plt.clf()
|
||||
plt.suptitle(f'{CP.carFingerprint} - Max lateral acceleration events')
|
||||
plt.title(', '.join(args.route))
|
||||
plt.scatter([ev.speed for ev in events], [ev.lateral_accel for ev in events], label='max lateral accel events')
|
||||
|
||||
plt.plot([0, 35], [3, 3], c='r', label='ISO 11270 - 3 m/s^2')
|
||||
plt.plot([0, 35], [-3, -3], c='r')
|
||||
|
||||
plt.plot([0, 35], [perc_left_accel, perc_left_accel], c='g', linestyle='--', label='90th percentile left lateral accel')
|
||||
plt.plot([0, 35], [perc_right_accel, perc_right_accel], c='#ff7f0e', linestyle='--', label='90th percentile right lateral accel')
|
||||
plt.text(0.4, float(perc_left_accel + 0.4), f'{perc_left_accel:.2f} m/s^2', verticalalignment='center', fontsize=12)
|
||||
plt.text(0.4, float(perc_right_accel - 0.4), f'{perc_right_accel:.2f} m/s^2', verticalalignment='center', fontsize=12)
|
||||
|
||||
plt.xlim(0, 35)
|
||||
plt.ylim(-5, 5)
|
||||
plt.xlabel('speed (m/s)')
|
||||
plt.ylabel('lateral acceleration (m/s^2)')
|
||||
plt.legend()
|
||||
plt.show(block=True)
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import os
|
||||
import argparse
|
||||
import struct
|
||||
from collections import deque
|
||||
from statistics import mean
|
||||
|
||||
from openpilot.cereal import log
|
||||
import openpilot.cereal.messaging as messaging
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
parser = argparse.ArgumentParser(description='Sniff a communication socket')
|
||||
parser.add_argument('--addr', default='127.0.0.1')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.addr != "127.0.0.1":
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
|
||||
poller = messaging.Poller()
|
||||
messaging.sub_sock('can', poller, addr=args.addr)
|
||||
|
||||
active = 0
|
||||
start_t = 0
|
||||
start_v = 0
|
||||
max_v = 0
|
||||
max_t = 0
|
||||
window = deque(maxlen=10)
|
||||
avg = 0
|
||||
while 1:
|
||||
polld = poller.poll(1000)
|
||||
for sock in polld:
|
||||
msg = sock.receive()
|
||||
with log.Event.from_bytes(msg) as log_evt:
|
||||
evt = log_evt
|
||||
|
||||
for item in evt.can:
|
||||
if item.address == 0xe4 and item.src == 128:
|
||||
torque_req = struct.unpack('!h', item.dat[0:2])[0]
|
||||
# print(torque_req)
|
||||
active = abs(torque_req) > 0
|
||||
if abs(torque_req) < 100:
|
||||
if max_v > 5:
|
||||
print(f'{start_v} -> {max_v} = {round(max_v - start_v, 2)} over {round(max_t - start_t, 2)}s')
|
||||
start_t = evt.logMonoTime / 1e9
|
||||
start_v = avg
|
||||
max_t = 0
|
||||
max_v = 0
|
||||
if item.address == 0x1ab and item.src == 0:
|
||||
motor_torque = ((item.dat[0] & 0x3) << 8) + item.dat[1]
|
||||
window.append(motor_torque)
|
||||
avg = mean(window)
|
||||
#print(f'{evt.logMonoTime}: {avg}')
|
||||
if active and avg > max_v + 0.5:
|
||||
max_v = avg
|
||||
max_t = evt.logMonoTime / 1e9
|
||||
Executable
+39
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import argparse
|
||||
from subprocess import check_output, CalledProcessError
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.uds import UdsClient, SESSION_TYPE, DTC_REPORT_TYPE, DTC_STATUS_MASK_TYPE, get_dtc_num_as_str, get_dtc_status_names
|
||||
from opendbc.car.structs import CarParams
|
||||
from panda import Panda
|
||||
|
||||
parser = argparse.ArgumentParser(description="read DTC status")
|
||||
parser.add_argument("addr", type=lambda x: int(x,0))
|
||||
parser.add_argument("--bus", type=int, default=0)
|
||||
parser.add_argument('--debug', action='store_true')
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
try:
|
||||
check_output(["pidof", "pandad"])
|
||||
print("pandad is running, please kill openpilot before running this script! (aborted)")
|
||||
sys.exit(1)
|
||||
except CalledProcessError as e:
|
||||
if e.returncode != 1: # 1 == no process found (pandad not running)
|
||||
raise e
|
||||
|
||||
panda = Panda()
|
||||
panda.set_safety_mode(CarParams.SafetyModel.elm327)
|
||||
uds_client = UdsClient(panda, args.addr, bus=args.bus)
|
||||
print("extended diagnostic session ...")
|
||||
uds_client.diagnostic_session_control(SESSION_TYPE.EXTENDED_DIAGNOSTIC)
|
||||
print("read diagnostic codes ...")
|
||||
data = uds_client.read_dtc_information(DTC_REPORT_TYPE.DTC_BY_STATUS_MASK, DTC_STATUS_MASK_TYPE.ALL)
|
||||
print("status availability:", " ".join(get_dtc_status_names(data[0])))
|
||||
print("DTC status:")
|
||||
for i in range(1, len(data), 4):
|
||||
dtc_num = get_dtc_num_as_str(data[i:i+3])
|
||||
dtc_status = " ".join(get_dtc_status_names(data[i+3]))
|
||||
print(dtc_num, dtc_status)
|
||||
Executable
+62
@@ -0,0 +1,62 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import numpy as np
|
||||
import matplotlib.pyplot as plt
|
||||
from sklearn import linear_model
|
||||
from opendbc.car.toyota.values import STEER_THRESHOLD
|
||||
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
MIN_SAMPLES = 30 * 100
|
||||
|
||||
|
||||
def to_signed(n, bits):
|
||||
if n >= (1 << max((bits - 1), 0)):
|
||||
n = n - (1 << max(bits, 0))
|
||||
return n
|
||||
|
||||
|
||||
def get_eps_factor(lr, plot=False):
|
||||
engaged = False
|
||||
steering_pressed = False
|
||||
torque_cmd, eps_torque = None, None
|
||||
cmds, eps = [], []
|
||||
|
||||
for msg in lr:
|
||||
if msg.which() != 'can':
|
||||
continue
|
||||
|
||||
for m in msg.can:
|
||||
if m.address == 0x2e4 and m.src == 128:
|
||||
engaged = bool(m.dat[0] & 1)
|
||||
torque_cmd = to_signed((m.dat[1] << 8) | m.dat[2], 16)
|
||||
elif m.address == 0x260 and m.src == 0:
|
||||
eps_torque = to_signed((m.dat[5] << 8) | m.dat[6], 16)
|
||||
steering_pressed = abs(to_signed((m.dat[1] << 8) | m.dat[2], 16)) > STEER_THRESHOLD
|
||||
|
||||
if engaged and torque_cmd is not None and eps_torque is not None and not steering_pressed:
|
||||
cmds.append(torque_cmd)
|
||||
eps.append(eps_torque)
|
||||
else:
|
||||
if len(cmds) > MIN_SAMPLES:
|
||||
break
|
||||
cmds, eps = [], []
|
||||
|
||||
if len(cmds) < MIN_SAMPLES:
|
||||
raise Exception("too few samples found in route")
|
||||
|
||||
lm = linear_model.LinearRegression(fit_intercept=False)
|
||||
lm.fit(np.array(cmds).reshape(-1, 1), eps)
|
||||
scale_factor = 1. / lm.coef_[0]
|
||||
|
||||
if plot:
|
||||
plt.plot(np.array(eps) * scale_factor)
|
||||
plt.plot(cmds)
|
||||
plt.show()
|
||||
return scale_factor
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
lr = LogReader(sys.argv[1])
|
||||
n = get_eps_factor(lr, plot="--plot" in sys.argv)
|
||||
print("EPS torque factor: ", n)
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import time
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.vin import get_vin
|
||||
from openpilot.selfdrive.car.card import can_comm_callbacks
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='Get VIN of the car')
|
||||
parser.add_argument('--debug', action='store_true')
|
||||
parser.add_argument('--bus', type=int, default=1)
|
||||
parser.add_argument('--timeout', type=float, default=0.1)
|
||||
parser.add_argument('--retry', type=int, default=5)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
sendcan = messaging.pub_sock('sendcan')
|
||||
logcan = messaging.sub_sock('can')
|
||||
can_callbacks = can_comm_callbacks(logcan, sendcan)
|
||||
time.sleep(1)
|
||||
|
||||
vin_rx_addr, vin_rx_bus, vin = get_vin(*can_callbacks, (args.bus,), args.timeout, args.retry)
|
||||
print(f'RX: {hex(vin_rx_addr)}, BUS: {vin_rx_bus}, VIN: {vin}')
|
||||
Executable
+164
@@ -0,0 +1,164 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
import struct
|
||||
from enum import IntEnum
|
||||
from opendbc.car.carlog import carlog
|
||||
from opendbc.car.uds import UdsClient, MessageTimeoutError, NegativeResponseError, SESSION_TYPE,\
|
||||
DATA_IDENTIFIER_TYPE, ACCESS_TYPE
|
||||
from opendbc.car.structs import CarParams
|
||||
from panda import Panda
|
||||
from datetime import date
|
||||
|
||||
# TODO: extend UDS library to allow custom/vendor-defined data identifiers without ignoring type checks
|
||||
class VOLKSWAGEN_DATA_IDENTIFIER_TYPE(IntEnum):
|
||||
CODING = 0x0600
|
||||
|
||||
# TODO: extend UDS library security_access() to take an access level offset per ISO 14229-1:2020 10.4 and remove this
|
||||
class ACCESS_TYPE_LEVEL_1(IntEnum):
|
||||
REQUEST_SEED = ACCESS_TYPE.REQUEST_SEED + 2
|
||||
SEND_KEY = ACCESS_TYPE.SEND_KEY + 2
|
||||
|
||||
MQB_EPS_CAN_ADDR = 0x712
|
||||
RX_OFFSET = 0x6a
|
||||
|
||||
if __name__ == "__main__":
|
||||
desc_text = "Shows Volkswagen EPS software and coding info, and enables or disables Heading Control Assist " + \
|
||||
"(Lane Assist). Useful for enabling HCA on cars without factory Lane Assist that want to use " + \
|
||||
"openpilot integrated at the CAN gateway (J533)."
|
||||
epilog_text = "This tool is meant to run directly on a vehicle-installed comma three, with the " + \
|
||||
"openpilot/tmux processes stopped. It should also work on a separate PC with a USB-attached comma " + \
|
||||
"panda. Vehicle ignition must be on. Recommend engine not be running when making changes. Must " + \
|
||||
"turn ignition off and on again for any changes to take effect."
|
||||
parser = argparse.ArgumentParser(description=desc_text, epilog=epilog_text)
|
||||
parser.add_argument("--debug", action="store_true", help="enable ISO-TP/UDS stack debugging output")
|
||||
parser.add_argument("action", choices={"show", "enable", "disable"}, help="show or modify current EPS HCA config")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
carlog.setLevel('DEBUG')
|
||||
|
||||
panda = Panda()
|
||||
panda.set_safety_mode(CarParams.SafetyModel.elm327)
|
||||
uds_client = UdsClient(panda, MQB_EPS_CAN_ADDR, MQB_EPS_CAN_ADDR + RX_OFFSET, 1, timeout=0.2)
|
||||
|
||||
try:
|
||||
uds_client.diagnostic_session_control(SESSION_TYPE.EXTENDED_DIAGNOSTIC)
|
||||
except MessageTimeoutError:
|
||||
print("Timeout opening session with EPS")
|
||||
quit()
|
||||
|
||||
odx_file, current_coding = None, None
|
||||
try:
|
||||
hw_pn = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.VEHICLE_MANUFACTURER_ECU_HARDWARE_NUMBER).decode("utf-8")
|
||||
sw_pn = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.VEHICLE_MANUFACTURER_SPARE_PART_NUMBER).decode("utf-8")
|
||||
sw_ver = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.VEHICLE_MANUFACTURER_ECU_SOFTWARE_VERSION_NUMBER).decode("utf-8")
|
||||
component = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.SYSTEM_NAME_OR_ENGINE_TYPE).decode("utf-8")
|
||||
odx_file = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.ODX_FILE).decode("utf-8").rstrip('\x00')
|
||||
current_coding = uds_client.read_data_by_identifier(VOLKSWAGEN_DATA_IDENTIFIER_TYPE.CODING)
|
||||
coding_text = current_coding.hex()
|
||||
|
||||
print("\nEPS diagnostic data\n")
|
||||
print(f" Part No HW: {hw_pn}")
|
||||
print(f" Part No SW: {sw_pn}")
|
||||
print(f" SW Version: {sw_ver}")
|
||||
print(f" Component: {component}")
|
||||
print(f" Coding: {coding_text}")
|
||||
print(f" ASAM Dataset: {odx_file}")
|
||||
except NegativeResponseError:
|
||||
print("Error fetching data from EPS")
|
||||
quit()
|
||||
except MessageTimeoutError:
|
||||
print("Timeout fetching data from EPS")
|
||||
quit()
|
||||
|
||||
coding_variant, current_coding_array, coding_byte, coding_bit = None, None, 0, 0
|
||||
coding_length = len(current_coding)
|
||||
|
||||
# EPS_MQB_ZFLS
|
||||
if odx_file in ("EV_SteerAssisMQB", "EV_SteerAssisMNB"):
|
||||
coding_variant = "ZFLS"
|
||||
coding_byte = 0
|
||||
coding_bit = 4
|
||||
|
||||
# MQB_PP_APA, MQB_VWBS_GEN2
|
||||
elif odx_file in ("EV_SteerAssisVWBSMQBA", "EV_SteerAssisVWBSMQBGen2"):
|
||||
coding_variant = "APA"
|
||||
coding_byte = 3
|
||||
coding_bit = 0
|
||||
|
||||
else:
|
||||
print("Configuration changes not yet supported on this EPS!")
|
||||
quit()
|
||||
|
||||
current_coding_array = struct.unpack(f"!{coding_length}B", current_coding)
|
||||
hca_enabled = (current_coding_array[coding_byte] & (1 << coding_bit) != 0)
|
||||
hca_text = ("DISABLED", "ENABLED")[hca_enabled]
|
||||
print(f" Lane Assist: {hca_text}")
|
||||
|
||||
try:
|
||||
params = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.APPLICATION_DATA_IDENTIFICATION).decode("utf-8")
|
||||
param_version_system_params = params[1:3]
|
||||
param_vehicle_type = params[3:5]
|
||||
param_index_char_curve = params[5:7]
|
||||
param_version_char_values = params[7:9]
|
||||
param_version_memory_map = params[9:11]
|
||||
print("\nEPS parameterization (per-vehicle calibration) data\n")
|
||||
print(f" Version of system parameters: {param_version_system_params}")
|
||||
print(f" Vehicle type: {param_vehicle_type}")
|
||||
print(f" Index of characteristic curve: {param_index_char_curve}")
|
||||
print(f" Version of characteristic values: {param_version_char_values}")
|
||||
print(f" Version of memory map: {param_version_memory_map}")
|
||||
except (NegativeResponseError, MessageTimeoutError):
|
||||
print("Error fetching parameterization data from EPS!")
|
||||
quit()
|
||||
|
||||
if args.action in ["enable", "disable"]:
|
||||
print("\nAttempting configuration update")
|
||||
|
||||
assert(coding_variant in ("ZFLS", "APA"))
|
||||
# ZFLS EPS config coding length can be anywhere from 1 to 4 bytes, but the
|
||||
# bit we care about is always in the same place in the first byte
|
||||
if args.action == "enable":
|
||||
new_byte = current_coding_array[coding_byte] | (1 << coding_bit)
|
||||
else:
|
||||
new_byte = current_coding_array[coding_byte] & ~(1 << coding_bit)
|
||||
new_coding = current_coding[0:coding_byte] + new_byte.to_bytes(1, "little") + current_coding[coding_byte+1:]
|
||||
|
||||
try:
|
||||
seed = uds_client.security_access(ACCESS_TYPE_LEVEL_1.REQUEST_SEED)
|
||||
key = struct.unpack("!I", seed)[0] + 28183 # yeah, it's like that
|
||||
uds_client.security_access(ACCESS_TYPE_LEVEL_1.SEND_KEY, struct.pack("!I", key))
|
||||
except (NegativeResponseError, MessageTimeoutError):
|
||||
print("Security access failed!")
|
||||
print("Open the hood and retry (disables the \"diagnostic firewall\" on newer vehicles)")
|
||||
quit()
|
||||
|
||||
try:
|
||||
# Programming date and tester number must be written before making
|
||||
# a change, or write to CODING will fail with request sequence error
|
||||
# Encoding on tester is unclear, it contains the workshop code in the
|
||||
# last two bytes, but not the VZ/importer or tester serial number
|
||||
# Can't seem to read it back, but we can read the calibration tester,
|
||||
# so fib a little and say that same tester did the programming
|
||||
current_date = date.today()
|
||||
formatted_date = current_date.strftime('%y-%m-%d')
|
||||
year, month, day = (int(part) for part in formatted_date.split('-'))
|
||||
prog_date = bytes([year, month, day])
|
||||
uds_client.write_data_by_identifier(DATA_IDENTIFIER_TYPE.PROGRAMMING_DATE, prog_date)
|
||||
tester_num = uds_client.read_data_by_identifier(DATA_IDENTIFIER_TYPE.CALIBRATION_REPAIR_SHOP_CODE_OR_CALIBRATION_EQUIPMENT_SERIAL_NUMBER)
|
||||
uds_client.write_data_by_identifier(DATA_IDENTIFIER_TYPE.REPAIR_SHOP_CODE_OR_TESTER_SERIAL_NUMBER, tester_num)
|
||||
uds_client.write_data_by_identifier(VOLKSWAGEN_DATA_IDENTIFIER_TYPE.CODING, new_coding)
|
||||
except (NegativeResponseError, MessageTimeoutError):
|
||||
print("Writing new configuration failed!")
|
||||
print("Make sure the comma processes are stopped: tmux kill-session -t comma")
|
||||
quit()
|
||||
|
||||
try:
|
||||
# Read back result just to make 100% sure everything worked
|
||||
current_coding_text = uds_client.read_data_by_identifier(VOLKSWAGEN_DATA_IDENTIFIER_TYPE.CODING).hex()
|
||||
print(f" New coding: {current_coding_text}")
|
||||
except (NegativeResponseError, MessageTimeoutError):
|
||||
print("Reading back updated coding failed!")
|
||||
quit()
|
||||
print("EPS configuration successfully updated")
|
||||
Executable
+77
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import math
|
||||
import datetime
|
||||
from collections import Counter
|
||||
from pprint import pprint
|
||||
from typing import cast
|
||||
|
||||
from openpilot.cereal.services import SERVICE_LIST
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
from openpilot.selfdrive.test.process_replay.migration import migrate_all
|
||||
|
||||
if __name__ == "__main__":
|
||||
cnt_events: Counter = Counter()
|
||||
|
||||
cams = [s for s in SERVICE_LIST if s.endswith('CameraState')]
|
||||
cnt_cameras = dict.fromkeys(cams, 0)
|
||||
|
||||
events: list[tuple[float, set[str]]] = []
|
||||
alerts: list[tuple[float, str]] = []
|
||||
start_time = math.inf
|
||||
end_time = -math.inf
|
||||
ignition_off = None
|
||||
for msg in migrate_all(LogReader(sys.argv[1], ReadMode.QLOG)):
|
||||
t = (msg.logMonoTime - start_time) / 1e9
|
||||
end_time = max(end_time, msg.logMonoTime)
|
||||
start_time = min(start_time, msg.logMonoTime)
|
||||
|
||||
if msg.which() == 'onroadEvents':
|
||||
for e in msg.onroadEvents:
|
||||
cnt_events[e.name] += 1
|
||||
|
||||
ae = {str(e.name) for e in msg.onroadEvents if e.name not in ('pedalPressed', 'steerOverride', 'gasPressedOverride')}
|
||||
if len(events) == 0 or ae != events[-1][1]:
|
||||
events.append((t, ae))
|
||||
|
||||
elif msg.which() == 'selfdriveState':
|
||||
at = msg.selfdriveState.alertType
|
||||
if "/override" not in at or "lanechange" in at.lower():
|
||||
if len(alerts) == 0 or alerts[-1][1] != at:
|
||||
alerts.append((t, at))
|
||||
elif msg.which() == 'pandaStates':
|
||||
if ignition_off is None:
|
||||
ign = any(ps.ignitionLine or ps.ignitionCan for ps in msg.pandaStates)
|
||||
if not ign:
|
||||
ignition_off = msg.logMonoTime
|
||||
break
|
||||
elif msg.which() in cams:
|
||||
cnt_cameras[msg.which()] += 1
|
||||
|
||||
duration = (end_time - start_time) / 1e9
|
||||
|
||||
print("Events")
|
||||
pprint(cnt_events)
|
||||
|
||||
print("\n")
|
||||
print("Events")
|
||||
for t, evt in events:
|
||||
print(f"{t:8.2f} {evt}")
|
||||
|
||||
print("\n")
|
||||
print("Cameras")
|
||||
for k, v in cnt_cameras.items():
|
||||
s = SERVICE_LIST[k]
|
||||
expected_frames = int(s.frequency * duration / cast(float, s.decimation))
|
||||
print(" ", k.ljust(20), f"{v}, {v/expected_frames:.1%} of expected")
|
||||
|
||||
print("\n")
|
||||
print("Alerts")
|
||||
for t, a in alerts:
|
||||
print(f"{t:8.2f} {a}")
|
||||
|
||||
print("\n")
|
||||
if ignition_off is not None:
|
||||
ignition_off = round((ignition_off - start_time) / 1e9, 2)
|
||||
print("Ignition off at", ignition_off)
|
||||
print("Route duration", datetime.timedelta(seconds=duration))
|
||||
Executable
+128
@@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
import random
|
||||
|
||||
from openpilot.cereal import log
|
||||
from opendbc.car.structs import car
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from opendbc.car.honda.interface import CarInterface
|
||||
from openpilot.common.realtime import DT_CTRL
|
||||
from openpilot.selfdrive.selfdrived.events import ET, Events
|
||||
from openpilot.selfdrive.selfdrived.alertmanager import AlertManager
|
||||
from openpilot.system.manager.process_config import managed_processes
|
||||
|
||||
EventName = log.OnroadEvent.EventName
|
||||
|
||||
def randperc() -> float:
|
||||
return 100. * random.random()
|
||||
|
||||
def cycle_alerts(duration=200, is_metric=False):
|
||||
# all alerts
|
||||
#alerts = list(EVENTS.keys())
|
||||
|
||||
# this plays each type of audible alert
|
||||
alerts = [
|
||||
(EventName.buttonEnable, ET.ENABLE),
|
||||
(EventName.buttonCancel, ET.USER_DISABLE),
|
||||
(EventName.wrongGear, ET.NO_ENTRY),
|
||||
|
||||
(EventName.locationdTemporaryError, ET.SOFT_DISABLE),
|
||||
(EventName.paramsdTemporaryError, ET.SOFT_DISABLE),
|
||||
(EventName.accFaulted, ET.IMMEDIATE_DISABLE),
|
||||
|
||||
# DM sequence
|
||||
(EventName.driverDistracted1, ET.WARNING),
|
||||
(EventName.driverDistracted2, ET.WARNING),
|
||||
(EventName.driverDistracted3, ET.WARNING),
|
||||
]
|
||||
|
||||
# debug alerts
|
||||
alerts = [
|
||||
#(EventName.highCpuUsage, ET.NO_ENTRY),
|
||||
#(EventName.lowMemory, ET.PERMANENT),
|
||||
#(EventName.overheat, ET.PERMANENT),
|
||||
#(EventName.outOfSpace, ET.PERMANENT),
|
||||
#(EventName.modeldLagging, ET.PERMANENT),
|
||||
#(EventName.processNotRunning, ET.NO_ENTRY),
|
||||
#(EventName.commIssue, ET.NO_ENTRY),
|
||||
#(EventName.calibrationInvalid, ET.PERMANENT),
|
||||
(EventName.cameraMalfunction, ET.PERMANENT),
|
||||
(EventName.cameraFrameRate, ET.PERMANENT),
|
||||
]
|
||||
|
||||
cameras = ['narrowRoadCameraState', 'wideRoadCameraState', 'cabinCameraState']
|
||||
|
||||
CS = car.CarState.new_message()
|
||||
CP = CarInterface.get_non_essential_params("HONDA_CIVIC")
|
||||
sm = messaging.SubMaster(['deviceState', 'pandaStates', 'narrowRoadCameraState', 'modelV2', 'extrinsicsCalibration',
|
||||
'driverMonitoringState', 'longitudinalPlan', 'deviceMotion',
|
||||
'managerState'] + cameras)
|
||||
|
||||
pm = messaging.PubMaster(['selfdriveState', 'pandaStates', 'deviceState'])
|
||||
|
||||
events = Events()
|
||||
AM = AlertManager()
|
||||
|
||||
frame = 0
|
||||
while True:
|
||||
for alert, et in alerts:
|
||||
events.clear()
|
||||
events.add(alert)
|
||||
|
||||
sm['deviceState'].freeSpacePercent = randperc()
|
||||
sm['deviceState'].memoryUsagePercent = int(randperc())
|
||||
sm['deviceState'].cpuTempC = [randperc() for _ in range(3)]
|
||||
sm['deviceState'].gpuTempC = [randperc() for _ in range(3)]
|
||||
sm['deviceState'].cpuUsagePercent = [int(randperc()) for _ in range(8)]
|
||||
sm['modelV2'].frameDropPerc = randperc()
|
||||
|
||||
if random.random() > 0.25:
|
||||
sm['modelV2'].velocity.x = [random.random(), ]
|
||||
if random.random() > 0.25:
|
||||
CS.vEgo = random.random()
|
||||
|
||||
procs = [p.get_process_state_msg() for p in managed_processes.values()]
|
||||
random.shuffle(procs)
|
||||
for i in range(random.randint(0, 10)):
|
||||
procs[i].shouldBeRunning = True
|
||||
sm['managerState'].processes = procs
|
||||
|
||||
sm['extrinsicsCalibration'].rpyCalib = [-1 * random.random() for _ in range(random.randint(0, 3))]
|
||||
|
||||
for s in sm.data.keys():
|
||||
prob = 0.3 if s in cameras else 0.08
|
||||
sm.alive[s] = random.random() > prob
|
||||
sm.valid[s] = random.random() > prob
|
||||
sm.freq_ok[s] = random.random() > prob
|
||||
|
||||
a = events.create_alerts([et, ], [CP, CS, sm, is_metric, 0])
|
||||
AM.add_many(frame, a)
|
||||
alert = AM.process_alerts(frame, [])
|
||||
print(alert)
|
||||
for _ in range(duration):
|
||||
dat = messaging.new_message('selfdriveState')
|
||||
dat.selfdriveState.enabled = False
|
||||
|
||||
if alert:
|
||||
dat.selfdriveState.alertText1 = alert.alert_text_1
|
||||
dat.selfdriveState.alertText2 = alert.alert_text_2
|
||||
dat.selfdriveState.alertSize = alert.alert_size
|
||||
dat.selfdriveState.alertStatus = alert.alert_status
|
||||
dat.selfdriveState.alertType = alert.alert_type
|
||||
dat.selfdriveState.alertSound = alert.audible_alert
|
||||
pm.send('selfdriveState', dat)
|
||||
|
||||
dat = messaging.new_message('deviceState')
|
||||
dat.deviceState.started = True
|
||||
pm.send('deviceState', dat)
|
||||
|
||||
dat = messaging.new_message('pandaStates', 1)
|
||||
dat.pandaStates[0].ignitionLine = True
|
||||
dat.pandaStates[0].pandaType = log.PandaState.PandaType.uno
|
||||
pm.send('pandaStates', dat)
|
||||
|
||||
frame += 1
|
||||
time.sleep(DT_CTRL)
|
||||
|
||||
if __name__ == '__main__':
|
||||
cycle_alerts()
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
from opendbc.car import uds
|
||||
from openpilot.tools.lib.live_logreader import live_logreader
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
|
||||
def main(route: str | None, addrs: list[int], rxoffset: int | None):
|
||||
"""
|
||||
TODO:
|
||||
- highlight TX vs RX clearly
|
||||
- disambiguate sendcan and can (useful to know if something sent on sendcan made it to the bus on can->128)
|
||||
- print as fixed width table, easier to read
|
||||
"""
|
||||
|
||||
if route is None:
|
||||
lr = live_logreader()
|
||||
else:
|
||||
lr = LogReader(route, default_mode=ReadMode.RLOG, sort_by_time=True)
|
||||
|
||||
start_mono_time = None
|
||||
prev_mono_time = 0
|
||||
|
||||
# include rx addresses
|
||||
addrs = addrs + [uds.get_rx_addr_for_tx_addr(addr, rxoffset) for addr in addrs]
|
||||
|
||||
for msg in lr:
|
||||
if msg.which() == 'can':
|
||||
if start_mono_time is None:
|
||||
start_mono_time = msg.logMonoTime
|
||||
|
||||
if msg.which() in ("can", 'sendcan'):
|
||||
for can in getattr(msg, msg.which()):
|
||||
if can.address in addrs or not len(addrs):
|
||||
if msg.logMonoTime != prev_mono_time:
|
||||
print()
|
||||
prev_mono_time = msg.logMonoTime
|
||||
print(f"{msg.which():>7}: rxaddr={can.address}, bus={str(can.src) + ',':<4} {round((msg.logMonoTime - start_mono_time) * 1e-6)} ms, " +
|
||||
f"0x{can.dat.hex()}, {can.dat}, {len(can.dat)=}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='View back and forth ISO-TP communication between various ECUs given an address')
|
||||
parser.add_argument('route', nargs='?', help='Route name, live if not specified')
|
||||
parser.add_argument('--addrs', nargs='*', default=[], help='List of tx address to view (0x7e0 for engine)')
|
||||
parser.add_argument('--rxoffset', default='0x8')
|
||||
args = parser.parse_args()
|
||||
|
||||
addrs = [int(addr, base=16) if addr.startswith('0x') else int(addr) for addr in args.addrs]
|
||||
rxoffset = int(args.rxoffset, base=16) if args.rxoffset else None
|
||||
main(args.route, addrs, rxoffset)
|
||||
Executable
+93
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import shlex
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
from watchdog.observers import Observer
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
|
||||
|
||||
def build_rsync_cmd(args) -> list[str]:
|
||||
ssh = [
|
||||
"ssh",
|
||||
"-o", "ControlMaster=auto",
|
||||
"-o", f"ControlPath=/tmp/devsync-{args.ip}.ctl",
|
||||
"-o", "ControlPersist=10m",
|
||||
"-o", "StrictHostKeyChecking=accept-new",
|
||||
]
|
||||
if args.identity:
|
||||
ssh += ["-i", args.identity]
|
||||
|
||||
return [
|
||||
"rsync", "-az",
|
||||
"--files-from=-", "--from0",
|
||||
"-e", " ".join(shlex.quote(p) for p in ssh),
|
||||
"--out-format=%n",
|
||||
BASEDIR + "/", f"comma@{args.ip}:{args.remote}/",
|
||||
]
|
||||
|
||||
|
||||
def git_tracked_files() -> bytes:
|
||||
return subprocess.check_output(
|
||||
["git", "-C", BASEDIR, "ls-files", "--recurse-submodules", "-z"]
|
||||
)
|
||||
|
||||
|
||||
class Handler(FileSystemEventHandler):
|
||||
def __init__(self, sync_fn):
|
||||
self.dirty = threading.Event()
|
||||
self.sync_fn = sync_fn
|
||||
|
||||
def on_any_event(self, event):
|
||||
if not event.is_directory:
|
||||
self.dirty.set()
|
||||
|
||||
def run(self):
|
||||
while True:
|
||||
time.sleep(1)
|
||||
if self.dirty.is_set():
|
||||
self.dirty.clear()
|
||||
self.sync_fn()
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser()
|
||||
p.add_argument("ip", help="device IP / hostname")
|
||||
p.add_argument("--remote", default="/data/openpilot", help="remote path on device")
|
||||
p.add_argument("-i", "--identity", default=None, help="ssh identity file")
|
||||
args = p.parse_args()
|
||||
|
||||
print(f"[devsync] watching {BASEDIR}")
|
||||
print(f"[devsync] target comma@{args.ip}:{args.remote}")
|
||||
|
||||
def run_sync():
|
||||
file_list = git_tracked_files()
|
||||
cmd = build_rsync_cmd(args)
|
||||
t0 = time.monotonic()
|
||||
r = subprocess.run(cmd, input=file_list, capture_output=True)
|
||||
dt = time.monotonic() - t0
|
||||
if r.returncode:
|
||||
print(f"[devsync] ERR rc={r.returncode} in {dt:.2f}s")
|
||||
return
|
||||
files = [ln for ln in r.stdout.decode().splitlines() if ln.strip()]
|
||||
msg = f"{len(files)} files: {', '.join(files)}" if files else "no changes"
|
||||
print(f"[devsync] {dt:.2f}s · {msg}")
|
||||
|
||||
run_sync()
|
||||
|
||||
handler = Handler(run_sync)
|
||||
obs = Observer()
|
||||
obs.schedule(handler, BASEDIR, recursive=True)
|
||||
obs.start()
|
||||
handler.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except KeyboardInterrupt:
|
||||
print("\n[devsync] stopping")
|
||||
Executable
+74
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import argparse
|
||||
import json
|
||||
import codecs
|
||||
|
||||
from openpilot.cereal import log
|
||||
from openpilot.cereal.services import SERVICE_LIST
|
||||
from openpilot.tools.lib.live_logreader import raw_live_logreader
|
||||
|
||||
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
|
||||
def hexdump(msg):
|
||||
m = str.upper(msg.hex())
|
||||
m = [m[i:i+2] for i in range(0,len(m),2)]
|
||||
m = [m[i:i+16] for i in range(0,len(m),16)]
|
||||
for row,dump in enumerate(m):
|
||||
addr = '%08X:' % (row*16)
|
||||
raw = ' '.join(dump[:8]) + ' ' + ' '.join(dump[8:])
|
||||
space = ' ' * (48 - len(raw))
|
||||
asci = ''.join(chr(int(x,16)) if 0x20 <= int(x,16) <= 0x7E else '.' for x in dump)
|
||||
print(f'{addr} {raw} {space} {asci}')
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
parser = argparse.ArgumentParser(description='Dump communication sockets. See openpilot/cereal/services.py for a complete list of available sockets.')
|
||||
parser.add_argument('--pipe', action='store_true')
|
||||
parser.add_argument('--raw', action='store_true')
|
||||
parser.add_argument('--json', action='store_true')
|
||||
parser.add_argument('--dump-json', action='store_true')
|
||||
parser.add_argument('--no-print', action='store_true')
|
||||
parser.add_argument('--addr', default='127.0.0.1')
|
||||
parser.add_argument('--values', help='values to monitor (instead of entire event)')
|
||||
parser.add_argument("socket", type=str, nargs='*', default=list(SERVICE_LIST.keys()), help="socket names to dump. defaults to all services defined in cereal")
|
||||
args = parser.parse_args()
|
||||
|
||||
lr = raw_live_logreader(args.socket, args.addr)
|
||||
|
||||
values = None
|
||||
if args.values:
|
||||
values = [s.strip().split(".") for s in args.values.split(",")]
|
||||
|
||||
for msg in lr:
|
||||
with log.Event.from_bytes(msg) as evt:
|
||||
if not args.no_print:
|
||||
if args.pipe:
|
||||
sys.stdout.write(str(msg))
|
||||
sys.stdout.flush()
|
||||
elif args.raw:
|
||||
hexdump(msg)
|
||||
elif args.json:
|
||||
print(json.loads(msg))
|
||||
elif args.dump_json:
|
||||
print(json.dumps(evt.to_dict()))
|
||||
elif values:
|
||||
print(f"logMonotime = {evt.logMonoTime}")
|
||||
for value in values:
|
||||
if hasattr(evt, value[0]):
|
||||
item = evt
|
||||
for key in value:
|
||||
item = getattr(item, key)
|
||||
print(f"{'.'.join(value)} = {item}")
|
||||
print("")
|
||||
else:
|
||||
try:
|
||||
print(evt)
|
||||
except UnicodeDecodeError:
|
||||
w = evt.which()
|
||||
s = f"( logMonoTime {evt.logMonoTime} \n {w} = "
|
||||
s += str(evt.__getattr__(w))
|
||||
s += f"\n valid = {evt.valid} )"
|
||||
print(s)
|
||||
Executable
+77
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import sys
|
||||
import wave
|
||||
import argparse
|
||||
import numpy as np
|
||||
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
|
||||
def extract_audio(route_or_segment_name, output_file=None, play=False):
|
||||
lr = LogReader(route_or_segment_name, default_mode=ReadMode.AUTO_INTERACTIVE)
|
||||
audio_messages = list(lr.filter("rawAudioData"))
|
||||
if not audio_messages:
|
||||
print("No rawAudioData messages found in logs")
|
||||
return
|
||||
sample_rate = audio_messages[0].sampleRate
|
||||
|
||||
audio_chunks = []
|
||||
total_frames = 0
|
||||
for msg in audio_messages:
|
||||
audio_array = np.frombuffer(msg.data, dtype=np.int16)
|
||||
audio_chunks.append(audio_array)
|
||||
total_frames += len(audio_array)
|
||||
full_audio = np.concatenate(audio_chunks)
|
||||
|
||||
print(f"Found {total_frames} frames from {len(audio_messages)} audio messages at {sample_rate} Hz")
|
||||
|
||||
if output_file:
|
||||
if write_wav_file(output_file, full_audio, sample_rate):
|
||||
print(f"Audio written to {output_file}")
|
||||
else:
|
||||
print("Audio extraction canceled.")
|
||||
if play:
|
||||
play_audio(full_audio, sample_rate)
|
||||
|
||||
|
||||
def write_wav_file(filename, audio_data, sample_rate):
|
||||
if os.path.exists(filename):
|
||||
if input(f"File '{filename}' exists. Overwrite? (y/N): ").lower() not in ['y', 'yes']:
|
||||
return False
|
||||
|
||||
with wave.open(filename, 'wb') as wav_file:
|
||||
wav_file.setnchannels(1) # Mono
|
||||
wav_file.setsampwidth(2) # 16-bit
|
||||
wav_file.setframerate(sample_rate)
|
||||
wav_file.writeframes(audio_data.tobytes())
|
||||
return True
|
||||
|
||||
|
||||
def play_audio(audio_data, sample_rate):
|
||||
try:
|
||||
import sounddevice as sd
|
||||
|
||||
print("Playing audio... Press Ctrl+C to stop")
|
||||
sd.play(audio_data, sample_rate)
|
||||
sd.wait()
|
||||
except KeyboardInterrupt:
|
||||
print("\nPlayback stopped")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Extract audio data from openpilot logs")
|
||||
parser.add_argument("-o", "--output", help="Output WAV file path")
|
||||
parser.add_argument("--play", action="store_true", help="Play audio with sounddevice")
|
||||
parser.add_argument("route_or_segment_name", nargs='?', help="The route or segment name")
|
||||
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
args = parser.parse_args()
|
||||
|
||||
output_file = args.output
|
||||
if not args.output and not args.play:
|
||||
output_file = "extracted_audio.wav"
|
||||
|
||||
extract_audio(args.route_or_segment_name.strip(), output_file, args.play)
|
||||
Executable
+79
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
LEVELS = {
|
||||
"DEBUG": 10,
|
||||
"INFO": 20,
|
||||
"WARNING": 30,
|
||||
"ERROR": 40,
|
||||
"CRITICAL": 50,
|
||||
}
|
||||
|
||||
OPERATING_SYSTEM_LOG_SOURCE = {
|
||||
0: "MAIN",
|
||||
1: "RADIO",
|
||||
2: "EVENTS",
|
||||
3: "SYSTEM",
|
||||
4: "CRASH",
|
||||
5: "KERNEL",
|
||||
}
|
||||
|
||||
|
||||
def print_logmessage(t, msg, min_level):
|
||||
try:
|
||||
log = json.loads(msg)
|
||||
if log['levelnum'] >= min_level:
|
||||
print(f"[{t / 1e9:.6f}] {log['filename']}:{log.get('lineno', '')} - {log.get('funcname', '')}: {log['msg']}")
|
||||
if 'exc_info' in log:
|
||||
print(log['exc_info'])
|
||||
except json.decoder.JSONDecodeError:
|
||||
print(f"[{t / 1e9:.6f}] decode error: {msg}")
|
||||
|
||||
|
||||
def print_operating_system_log(t, msg):
|
||||
source = msg.tag or OPERATING_SYSTEM_LOG_SOURCE.get(msg.id, "SYSTEM")
|
||||
try:
|
||||
m = json.loads(msg.message)['MESSAGE']
|
||||
except Exception:
|
||||
m = msg.message
|
||||
|
||||
print(f"[{t / 1e9:.6f}] {source} {msg.pid} {msg.tag} - {m}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--absolute', action='store_true')
|
||||
parser.add_argument('--level', default='DEBUG')
|
||||
parser.add_argument('--addr', default='127.0.0.1')
|
||||
parser.add_argument("route", type=str, nargs='*', help="route name + segment number for offline usage")
|
||||
args = parser.parse_args()
|
||||
|
||||
min_level = LEVELS[args.level]
|
||||
|
||||
if args.route:
|
||||
st = None if not args.absolute else 0
|
||||
for route in args.route:
|
||||
lr = LogReader(route, sort_by_time=True)
|
||||
for m in lr:
|
||||
if st is None:
|
||||
st = m.logMonoTime
|
||||
if m.which() == 'logMessage':
|
||||
print_logmessage(m.logMonoTime-st, m.logMessage, min_level)
|
||||
elif m.which() == 'errorLogMessage':
|
||||
print_logmessage(m.logMonoTime-st, m.errorLogMessage, min_level)
|
||||
elif m.which() == 'operatingSystemLog':
|
||||
print_operating_system_log(m.logMonoTime-st, m.operatingSystemLog)
|
||||
else:
|
||||
sm = messaging.SubMaster(['logMessage', 'operatingSystemLog'], addr=args.addr)
|
||||
while True:
|
||||
sm.update()
|
||||
|
||||
if sm.updated['logMessage']:
|
||||
print_logmessage(sm.logMonoTime['logMessage'], sm['logMessage'], min_level)
|
||||
|
||||
if sm.updated['operatingSystemLog']:
|
||||
print_operating_system_log(sm.logMonoTime['operatingSystemLog'], sm['operatingSystemLog'])
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import sys
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
|
||||
def get_fingerprint(lr):
|
||||
# TODO: make this a nice tool for car ports. should also work with qlogs for FW
|
||||
|
||||
fw = None
|
||||
vin = None
|
||||
msgs = {}
|
||||
for msg in lr:
|
||||
if msg.which() == 'carParams':
|
||||
fw = msg.carParams.carFw
|
||||
vin = msg.carParams.carVin
|
||||
elif msg.which() == 'can':
|
||||
for c in msg.can:
|
||||
# read also msgs sent by EON on CAN bus 0x80 and filter out the
|
||||
# addr with more than 11 bits
|
||||
if c.src % 0x80 == 0 and c.address < 0x800 and c.address not in (0x7df, 0x7e0, 0x7e8):
|
||||
msgs[c.address] = len(c.dat)
|
||||
|
||||
# show CAN fingerprint
|
||||
fingerprint = ', '.join(f"{v[0]}: {v[1]}" for v in sorted(msgs.items()))
|
||||
print(f"\nfound {len(msgs)} messages. CAN fingerprint:\n")
|
||||
print(fingerprint)
|
||||
|
||||
# TODO: also print the fw fingerprint merged with the existing ones
|
||||
# show FW fingerprint
|
||||
if fw:
|
||||
print("\nFW fingerprint:\n")
|
||||
for f in fw:
|
||||
print(f" (Ecu.{f.ecu}, {hex(f.address)}, {None if f.subAddress == 0 else f.subAddress}): [")
|
||||
print(f" {f.fwVersion},")
|
||||
print(" ],")
|
||||
print()
|
||||
print(f"VIN: {vin}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: ./fingerprint_from_route.py <route>")
|
||||
sys.exit(1)
|
||||
|
||||
lr = LogReader(sys.argv[1], ReadMode.QLOG)
|
||||
get_fingerprint(lr)
|
||||
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
import random
|
||||
from collections import defaultdict
|
||||
|
||||
from tqdm import tqdm
|
||||
from opendbc.car.fw_versions import match_fw_to_car_fuzzy
|
||||
from opendbc.car.toyota.values import FW_VERSIONS as TOYOTA_FW_VERSIONS
|
||||
from opendbc.car.honda.values import FW_VERSIONS as HONDA_FW_VERSIONS
|
||||
from opendbc.car.hyundai.values import FW_VERSIONS as HYUNDAI_FW_VERSIONS
|
||||
from opendbc.car.volkswagen.values import FW_VERSIONS as VW_FW_VERSIONS
|
||||
|
||||
|
||||
FWS = {}
|
||||
FWS.update(TOYOTA_FW_VERSIONS)
|
||||
FWS.update(HONDA_FW_VERSIONS)
|
||||
FWS.update(HYUNDAI_FW_VERSIONS)
|
||||
FWS.update(VW_FW_VERSIONS)
|
||||
|
||||
if __name__ == "__main__":
|
||||
total = 0
|
||||
match = 0
|
||||
wrong_match = 0
|
||||
confusions = defaultdict(set)
|
||||
|
||||
for _ in tqdm(range(1000)):
|
||||
for candidate, fws in FWS.items():
|
||||
fw_dict = {}
|
||||
for (_, addr, subaddr), fw_list in fws.items():
|
||||
fw_dict[(addr, subaddr)] = [random.choice(fw_list)]
|
||||
|
||||
matches = match_fw_to_car_fuzzy(fw_dict, log=False, exclude=candidate)
|
||||
|
||||
total += 1
|
||||
if len(matches) == 1:
|
||||
if list(matches)[0] == candidate:
|
||||
match += 1
|
||||
else:
|
||||
confusions[candidate] |= matches
|
||||
wrong_match += 1
|
||||
|
||||
print()
|
||||
for candidate, wrong_matches in sorted(confusions.items()):
|
||||
print(candidate, wrong_matches)
|
||||
|
||||
print()
|
||||
print(f"Total fuzz cases: {total}")
|
||||
print(f"Correct matches: {match}")
|
||||
print(f"Wrong matches: {wrong_match}")
|
||||
|
||||
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
# simple script to get a vehicle fingerprint.
|
||||
|
||||
# Instructions:
|
||||
# - connect to a Panda
|
||||
# - run openpilot/selfdrive/pandad/pandad
|
||||
# - launching this script
|
||||
# Note: it's very important that the car is in stock mode, in order to collect a complete fingerprint
|
||||
# - since some messages are published at low frequency, keep this script running for at least 30s,
|
||||
# until all messages are received at least once
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
|
||||
logcan = messaging.sub_sock('can')
|
||||
msgs = {}
|
||||
while True:
|
||||
lc = messaging.recv_sock(logcan, True)
|
||||
if lc is None:
|
||||
continue
|
||||
|
||||
for c in lc.can:
|
||||
# read also msgs sent by EON on CAN bus 0x80 and filter out the
|
||||
# addr with more than 11 bits
|
||||
if c.src % 0x80 == 0 and c.address < 0x800 and c.address not in (0x7df, 0x7e0, 0x7e8):
|
||||
msgs[c.address] = len(c.dat)
|
||||
|
||||
fingerprint = ', '.join(f"{v[0]}: {v[1]}" for v in sorted(msgs.items()))
|
||||
|
||||
print(f"number of messages {len(msgs)}:")
|
||||
print(f"fingerprint {fingerprint}")
|
||||
Executable
+106
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import numpy as np
|
||||
import capnp
|
||||
from collections import defaultdict
|
||||
|
||||
from openpilot.cereal.messaging import SubMaster
|
||||
|
||||
def cputime_total(ct):
|
||||
return ct.user + ct.nice + ct.system + ct.idle + ct.iowait + ct.irq + ct.softirq
|
||||
|
||||
|
||||
def cputime_busy(ct):
|
||||
return ct.user + ct.nice + ct.system + ct.irq + ct.softirq
|
||||
|
||||
|
||||
def proc_cputime_total(ct):
|
||||
return ct.cpuUser + ct.cpuSystem + ct.cpuChildrenUser + ct.cpuChildrenSystem
|
||||
|
||||
|
||||
def proc_name(proc):
|
||||
name = proc.name
|
||||
if len(proc.cmdline):
|
||||
name = proc.cmdline[0]
|
||||
if len(proc.exe):
|
||||
name = proc.exe + " - " + name
|
||||
|
||||
return name
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--mem', action='store_true')
|
||||
parser.add_argument('--cpu', action='store_true')
|
||||
args = parser.parse_args()
|
||||
|
||||
sm = SubMaster(['deviceState', 'procLog'])
|
||||
|
||||
last_temp = 0.0
|
||||
last_mem = 0.0
|
||||
total_times = [0.]*8
|
||||
busy_times = [0.]*8
|
||||
|
||||
prev_proclog: capnp._DynamicStructReader | None = None
|
||||
prev_proclog_t: int | None = None
|
||||
|
||||
while True:
|
||||
sm.update()
|
||||
|
||||
if sm.updated['deviceState']:
|
||||
t = sm['deviceState']
|
||||
last_temp = np.mean(t.cpuTempC)
|
||||
last_mem = t.memoryUsagePercent
|
||||
|
||||
if sm.updated['procLog']:
|
||||
m = sm['procLog']
|
||||
|
||||
cores = [0.]*8
|
||||
total_times_new = [0.]*8
|
||||
busy_times_new = [0.]*8
|
||||
|
||||
for c in m.cpuTimes:
|
||||
n = c.cpuNum
|
||||
total_times_new[n] = cputime_total(c)
|
||||
busy_times_new[n] = cputime_busy(c)
|
||||
|
||||
for n in range(8):
|
||||
t_busy = busy_times_new[n] - busy_times[n]
|
||||
t_total = total_times_new[n] - total_times[n]
|
||||
cores[n] = t_busy / t_total
|
||||
|
||||
total_times = total_times_new[:]
|
||||
busy_times = busy_times_new[:]
|
||||
|
||||
print(f"CPU {100.0 * np.mean(cores):.2f}% - RAM: {last_mem:.2f}% - Temp {last_temp:.2f}C")
|
||||
|
||||
if args.cpu and prev_proclog is not None and prev_proclog_t is not None:
|
||||
procs: dict[str, float] = defaultdict(float)
|
||||
dt = (sm.logMonoTime['procLog'] - prev_proclog_t) / 1e9
|
||||
for proc in m.procs:
|
||||
try:
|
||||
name = proc_name(proc)
|
||||
prev_proc = [p for p in prev_proclog.procs if proc.pid == p.pid][0]
|
||||
cpu_time = proc_cputime_total(proc) - proc_cputime_total(prev_proc)
|
||||
cpu_usage = cpu_time / dt * 100.
|
||||
procs[name] += cpu_usage
|
||||
except IndexError:
|
||||
pass
|
||||
|
||||
print("Top CPU usage:")
|
||||
for k, v in sorted(procs.items(), key=lambda item: item[1], reverse=True)[:10]:
|
||||
print(f"{k.rjust(70)} {v:.2f} %")
|
||||
print()
|
||||
|
||||
if args.mem:
|
||||
mems = {}
|
||||
for proc in m.procs:
|
||||
name = proc_name(proc)
|
||||
mems[name] = float(proc.memRss) / 1e6
|
||||
print("Top memory usage:")
|
||||
for k, v in sorted(mems.items(), key=lambda item: item[1], reverse=True)[:10]:
|
||||
print(f"{k.rjust(70)} {v:.2f} MB")
|
||||
print()
|
||||
|
||||
prev_proclog = m
|
||||
prev_proclog_t = sm.logMonoTime['procLog']
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
|
||||
|
||||
from openpilot.selfdrive.test.mem_usage import DEMO_ROUTE, print_report
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Analyze memory usage from route logs")
|
||||
parser.add_argument("route", nargs="?", default=None, help="route ID or local rlog path")
|
||||
parser.add_argument("--demo", action="store_true", help=f"use demo route ({DEMO_ROUTE})")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.demo:
|
||||
route = DEMO_ROUTE
|
||||
elif args.route:
|
||||
route = args.route
|
||||
else:
|
||||
parser.error("provide a route or use --demo")
|
||||
|
||||
print(f"Reading logs from: {route}")
|
||||
|
||||
proc_logs = []
|
||||
device_states = []
|
||||
for msg in LogReader(route):
|
||||
if msg.which() == 'procLog':
|
||||
proc_logs.append(msg)
|
||||
elif msg.which() == 'deviceState':
|
||||
device_states.append(msg)
|
||||
|
||||
print_report(proc_logs, device_states)
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
from opendbc.car.values import BRANDS
|
||||
|
||||
for brand in BRANDS:
|
||||
all_flags = set()
|
||||
for platform in brand:
|
||||
if platform.config.flags != 0:
|
||||
all_flags |= set(platform.config.flags)
|
||||
|
||||
if len(all_flags):
|
||||
print(brand.__module__.split('.')[-2].upper() + ':')
|
||||
for flag in sorted(all_flags):
|
||||
print(f' {flag.name:<24}:', {platform.name for platform in brand.with_flags(flag)})
|
||||
print()
|
||||
@@ -0,0 +1 @@
|
||||
clpeak/
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
|
||||
cd $DIR
|
||||
|
||||
if [ ! -d "$DIR/clpeak" ]; then
|
||||
git clone https://github.com/krrishnarraj/clpeak.git
|
||||
|
||||
cd clpeak
|
||||
git fetch
|
||||
git checkout ec2d3e70e1abc7738b81f9277c7af79d89b2133b
|
||||
git reset --hard origin/master
|
||||
git submodule update --init --recursive --remote
|
||||
|
||||
git apply ../run_continuously.patch
|
||||
fi
|
||||
|
||||
cd clpeak
|
||||
mkdir build || true
|
||||
cd build
|
||||
cmake ..
|
||||
cmake --build .
|
||||
@@ -0,0 +1,13 @@
|
||||
diff --git a/src/clpeak.cpp b/src/clpeak.cpp
|
||||
index 8cb192b..b6fe6f5 100644
|
||||
--- a/src/clpeak.cpp
|
||||
+++ b/src/clpeak.cpp
|
||||
@@ -47,7 +47,7 @@ int clPeak::runAll()
|
||||
|
||||
log->xmlOpenTag("clpeak");
|
||||
log->xmlAppendAttribs("os", OS_NAME);
|
||||
- for (size_t p = 0; p < platforms.size(); p++)
|
||||
+ for (size_t p = 0; p < platforms.size(); (p+1 % platforms.size()))
|
||||
{
|
||||
if (forcePlatform && (p != specifiedPlatform))
|
||||
continue;
|
||||
Executable
+24
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
cd /sys/kernel/tracing
|
||||
|
||||
echo 1 > tracing_on
|
||||
echo boot > trace_clock
|
||||
echo 1000 > buffer_size_kb
|
||||
|
||||
# /sys/kernel/tracing/available_events
|
||||
echo 0 > events/enable
|
||||
#echo 1 > events/irq/enable
|
||||
#echo 1 > events/sched/enable
|
||||
#echo 1 > events/kgsl/enable
|
||||
#echo 1 > events/camera/enable
|
||||
echo 1 > events/workqueue/enable
|
||||
|
||||
echo > trace
|
||||
sleep 2
|
||||
echo 0 > tracing_on
|
||||
|
||||
cp trace /tmp/trace
|
||||
chown comma: /tmp/trace
|
||||
echo /tmp/trace
|
||||
@@ -0,0 +1,2 @@
|
||||
palanteer/
|
||||
viewer
|
||||
Executable
+24
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
cd $DIR
|
||||
|
||||
if [ ! -d palanteer ]; then
|
||||
git clone https://github.com/dfeneyrou/palanteer
|
||||
pip install wheel
|
||||
sudo apt install libunwind-dev libdw-dev
|
||||
fi
|
||||
|
||||
cd palanteer
|
||||
git pull
|
||||
|
||||
mkdir -p build
|
||||
cd build
|
||||
cmake .. -DCMAKE_BUILD_TYPE=Release
|
||||
make -j$(nproc)
|
||||
|
||||
pip install --force-reinstall python/dist/palanteer*.whl
|
||||
|
||||
cp bin/palanteer $DIR/viewer
|
||||
@@ -0,0 +1,7 @@
|
||||
trace_*
|
||||
|
||||
tracebox
|
||||
trace_processor
|
||||
|
||||
perfetto/
|
||||
configs/
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [ ! -d perfetto ]; then
|
||||
git clone https://android.googlesource.com/platform/external/perfetto/
|
||||
fi
|
||||
|
||||
cd perfetto
|
||||
|
||||
tools/install-build-deps --linux-arm
|
||||
tools/gn gen --args='is_debug=false target_os="linux" target_cpu="arm64"' out/linux
|
||||
tools/ninja -C out/linux tracebox traced traced_probes perfetto
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
DEST=tici:/data/openpilot/selfdrive/debug/profiling/perfetto
|
||||
|
||||
scp -r perfetto/out/linux/tracebox $DEST
|
||||
scp -r perfetto/test/configs $DEST
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd)"
|
||||
cd $DIR
|
||||
|
||||
OUT=trace_
|
||||
sudo ./tracebox -o $OUT --txt -c configs/scheduling.cfg
|
||||
sudo chown $USER:$USER $OUT
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
curl -LO https://get.perfetto.dev/trace_processor
|
||||
chmod +x ./trace_processor
|
||||
|
||||
./trace_processor --httpd
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
DEST=tici:/data/openpilot/selfdrive/debug/profiling/perfetto
|
||||
|
||||
scp tici:/data/openpilot/selfdrive/debug/profiling/perfetto/trace_* .
|
||||
Executable
+21
@@ -0,0 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
# find process with name passed in (excluding this process)
|
||||
for PID in $(pgrep -f $1); do
|
||||
if [ "$PID" != "$$" ]; then
|
||||
ps -p $PID -o args
|
||||
TRACE_PID=$PID
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -z "$TRACE_PID" ]; then
|
||||
echo "could not find PID for $1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sudo env PATH=$PATH py-spy record -d 5 -o /tmp/perf$TRACE_PID.svg -p $TRACE_PID &&
|
||||
google-chrome /tmp/perf$TRACE_PID.svg
|
||||
@@ -0,0 +1 @@
|
||||
SnapdragonProfiler/
|
||||
@@ -0,0 +1,13 @@
|
||||
snapdragon profiler
|
||||
--------
|
||||
|
||||
|
||||
* download from https://developer.qualcomm.com/software/snapdragon-profiler/tools-archive (need a qc developer account)
|
||||
* choose v2021.5 (verified working with 24.04 dev environment)
|
||||
* unzip to openpilot/selfdrive/debug/profiling/snapdragon/SnapdragonProfiler
|
||||
* run ```./setup-profiler.sh```
|
||||
* run ```./setup-agnos.sh```
|
||||
* run ```openpilot/selfdrive/debug/adb.sh``` on device
|
||||
* run the ```adb connect xxx``` command that was given to you on local pc
|
||||
* cd to SnapdragonProfiler and run ```./run_sdp.sh```
|
||||
* connect to device -> choose device you just setup
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# TODO: there's probably a better way to do this
|
||||
|
||||
cd SnapdragonProfiler/service
|
||||
mv android real_android
|
||||
ln -s agl/ android
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# install depends
|
||||
sudo apt update
|
||||
sudo apt-get install libc++1 libc++abi1 default-jre android-tools-adb gtk-sharp2
|
||||
|
||||
# setup mono
|
||||
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
|
||||
sudo apt install apt-transport-https ca-certificates
|
||||
echo "deb https://download.mono-project.com/repo/ubuntu stable-xenial main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
|
||||
sudo apt update
|
||||
sudo apt-get install -y mono-complete
|
||||
|
||||
echo "Setup successful, you should now be able to run the profiler with cd SnapdragonProfiler and ./run_sdp.sh"
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
RUBYOPT="-W0" irqtop -d1 -R
|
||||
Executable
+78
@@ -0,0 +1,78 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import zstandard as zstd
|
||||
from collections import defaultdict
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
from openpilot.cereal.services import SERVICE_LIST
|
||||
from openpilot.common.utils import LOG_COMPRESSION_LEVEL
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
from tqdm import tqdm
|
||||
|
||||
MIN_SIZE = 0.5 # Percent size of total to show as separate entry
|
||||
|
||||
|
||||
def make_pie(msgs, typ):
|
||||
msgs_by_type = defaultdict(list)
|
||||
for m in msgs:
|
||||
msgs_by_type[m.which()].append(m.as_builder().to_bytes())
|
||||
|
||||
total = len(zstd.compress(b"".join([m.as_builder().to_bytes() for m in msgs]), LOG_COMPRESSION_LEVEL))
|
||||
uncompressed_total = len(b"".join([m.as_builder().to_bytes() for m in msgs]))
|
||||
|
||||
length_by_type = {k: len(b"".join(v)) for k, v in msgs_by_type.items()}
|
||||
# calculate compressed size by calculating diff when removed from the segment
|
||||
compressed_length_by_type = {}
|
||||
for k in tqdm(msgs_by_type.keys(), desc="Compressing"):
|
||||
compressed_length_by_type[k] = total - len(zstd.compress(b"".join([m.as_builder().to_bytes() for m in msgs if m.which() != k]), LOG_COMPRESSION_LEVEL))
|
||||
|
||||
sizes = sorted(compressed_length_by_type.items(), key=lambda kv: kv[1])
|
||||
|
||||
print("name - comp. size (uncomp. size)")
|
||||
for (name, sz) in sizes:
|
||||
print(f"{name:<22} - {sz / 1024:.2f} kB ({length_by_type[name] / 1024:.2f} kB)")
|
||||
print()
|
||||
print(f"{typ} - Real total {total / 1024:.2f} kB")
|
||||
print(f"{typ} - Breakdown total {sum(compressed_length_by_type.values()) / 1024:.2f} kB")
|
||||
print(f"{typ} - Uncompressed total {uncompressed_total / 1024 / 1024:.2f} MB")
|
||||
|
||||
sizes_large = [(k, sz) for (k, sz) in sizes if sz >= total * MIN_SIZE / 100]
|
||||
sizes_large += [('other', sum(sz for (_, sz) in sizes if sz < total * MIN_SIZE / 100))]
|
||||
|
||||
labels, sizes = zip(*sizes_large, strict=True)
|
||||
|
||||
plt.figure()
|
||||
plt.title(f"{typ}")
|
||||
plt.pie(sizes, labels=labels, autopct='%1.1f%%')
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='View log size breakdown by message type')
|
||||
parser.add_argument('route', help='route to use')
|
||||
parser.add_argument('--as-qlog', action='store_true', help='decimate rlog using latest decimation factors')
|
||||
args = parser.parse_args()
|
||||
|
||||
msgs = list(LogReader(args.route))
|
||||
|
||||
if args.as_qlog:
|
||||
new_msgs = []
|
||||
msg_cnts: dict[str, int] = defaultdict(int)
|
||||
for msg in msgs:
|
||||
msg_which = msg.which()
|
||||
if msg.which() in ("initData", "sentinel"):
|
||||
new_msgs.append(msg)
|
||||
continue
|
||||
|
||||
if msg_which not in SERVICE_LIST:
|
||||
continue
|
||||
|
||||
decimation = SERVICE_LIST[msg_which].decimation
|
||||
if decimation is not None and msg_cnts[msg_which] % decimation == 0:
|
||||
new_msgs.append(msg)
|
||||
msg_cnts[msg_which] += 1
|
||||
|
||||
msgs = new_msgs
|
||||
|
||||
make_pie(msgs, 'qlog')
|
||||
plt.show()
|
||||
Executable
+32
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
|
||||
from openpilot.selfdrive.test.process_replay.process_replay import CONFIGS, replay_process
|
||||
from openpilot.selfdrive.test.process_replay.test_processes import EXCLUDED_PROCS
|
||||
from openpilot.tools.lib.logreader import LogReader, save_log
|
||||
|
||||
ALLOW_PROCS = {c.proc_name for c in CONFIGS}
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Run process on route and create new logs",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument("route", help="The route name to use")
|
||||
parser.add_argument("--fingerprint", help="The fingerprint to use")
|
||||
parser.add_argument("--whitelist-procs", nargs='*', default=ALLOW_PROCS, help="Whitelist given processes (e.g. controlsd)")
|
||||
parser.add_argument("--blacklist-procs", nargs='*', default=EXCLUDED_PROCS, help="Blacklist given processes (e.g. controlsd)")
|
||||
args = parser.parse_args()
|
||||
|
||||
allowed_procs = set(args.whitelist_procs) - set(args.blacklist_procs)
|
||||
cfgs = [c for c in CONFIGS if c.proc_name in allowed_procs]
|
||||
|
||||
inputs = list(LogReader(args.route))
|
||||
outputs = replay_process(cfgs, inputs, fingerprint=args.fingerprint)
|
||||
|
||||
# Remove message generated by the process under test and merge in the new messages
|
||||
produces = {o.which() for o in outputs}
|
||||
inputs = [i for i in inputs if i.which() not in produces]
|
||||
outputs = sorted(inputs + outputs, key=lambda x: x.logMonoTime)
|
||||
|
||||
fn = f"{args.route.replace('/', '_')}_{'_'.join(allowed_procs)}.zst"
|
||||
print(f"Saving log to {fn}")
|
||||
save_log(fn, outputs)
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
while true; do
|
||||
if ls /dev/serial/by-id/usb-FTDI_FT230X* 2> /dev/null; then
|
||||
sudo screen /dev/serial/by-id/usb-FTDI_FT230X* 115200
|
||||
fi
|
||||
sleep 0.005
|
||||
done
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
|
||||
from opendbc.car.structs import car
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
if __name__ == "__main__":
|
||||
CP = None
|
||||
if len(sys.argv) > 1:
|
||||
r = Route(sys.argv[1])
|
||||
cps = [m for m in LogReader(r.qlog_paths()[0]) if m.which() == 'carParams']
|
||||
CP = cps[0].carParams.as_builder()
|
||||
else:
|
||||
CP = car.CarParams.new_message()
|
||||
CP.openpilotLongitudinalControl = True
|
||||
CP.alphaLongitudinalAvailable = False
|
||||
|
||||
cp_bytes = CP.to_bytes()
|
||||
for p in ("CarParams", "CarParamsCache", "CarParamsPersistent"):
|
||||
Params().put(p, cp_bytes, block=True)
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import requests
|
||||
from openpilot.common.params import Params
|
||||
import sys
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print(f"{sys.argv[0]} <github username>")
|
||||
exit(1)
|
||||
|
||||
username = sys.argv[1]
|
||||
keys = requests.get(f"https://github.com/{username}.keys", timeout=10)
|
||||
|
||||
if keys.status_code == 200:
|
||||
params = Params()
|
||||
params.put_bool("SshEnabled", True, block=True)
|
||||
params.put("GithubSshKeys", keys.text, block=True)
|
||||
params.put("GithubUsername", username, block=True)
|
||||
print("Set up ssh keys successfully")
|
||||
else:
|
||||
print("Error getting public keys from github")
|
||||
Executable
+57
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import re
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="A helper for connecting to devices over the comma prime SSH proxy.\
|
||||
Adding your SSH key to your SSH config is recommended for more convenient use; see https://docs.comma.ai/how-to/connect-to-comma/.")
|
||||
parser.add_argument("device", help="device name or dongle id")
|
||||
parser.add_argument("--host", help="ssh jump server host", default="ssh.comma.ai")
|
||||
parser.add_argument("--port", help="ssh jump server port", default=22, type=int)
|
||||
parser.add_argument("--key", help="ssh key", default=os.path.join(BASEDIR, "openpilot/common/hardware/comma/id_rsa"))
|
||||
parser.add_argument("--debug", help="enable debug output", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
r = CommaApi(get_token()).get("v1/me/devices")
|
||||
devices = {x['dongle_id']: x['alias'] for x in r}
|
||||
|
||||
if not re.match("[0-9a-zA-Z]{16}", args.device):
|
||||
user_input = args.device.replace(" ", "").lower()
|
||||
matches = { k: v for k, v in devices.items() if isinstance(v, str) and user_input in v.replace(" ", "").lower() }
|
||||
if len(matches) == 1:
|
||||
dongle_id = list(matches.keys())[0]
|
||||
else:
|
||||
print(f"failed to look up dongle id for \"{args.device}\"", file=sys.stderr)
|
||||
if len(matches) > 1:
|
||||
print("found multiple matches:", file=sys.stderr)
|
||||
for k, v in matches.items():
|
||||
print(f" \"{v}\" ({k})", file=sys.stderr)
|
||||
exit(1)
|
||||
else:
|
||||
dongle_id = args.device
|
||||
|
||||
name = dongle_id
|
||||
if dongle_id in devices:
|
||||
name = f"{devices[dongle_id]} ({dongle_id})"
|
||||
print(f"connecting to {name} through {args.host}:{args.port} ...")
|
||||
|
||||
command = [
|
||||
"ssh",
|
||||
"-i", args.key,
|
||||
"-o", f"ProxyCommand=ssh -i {args.key} -W %h:%p -p %p %h@{args.host}",
|
||||
"-p", str(args.port),
|
||||
]
|
||||
if args.debug:
|
||||
command += ["-v"]
|
||||
command += [
|
||||
f"comma@comma-{dongle_id}",
|
||||
]
|
||||
if args.debug:
|
||||
print(" ".join([f"'{c}'" if " " in c else c for c in command]))
|
||||
os.execvp(command[0], command)
|
||||
Executable
+179
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from collections import defaultdict
|
||||
import argparse
|
||||
import os
|
||||
import traceback
|
||||
from tqdm import tqdm
|
||||
from opendbc.car.car_helpers import interface_names
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.fw_versions import VERSIONS, match_fw_to_car
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
from openpilot.tools.lib.route import SegmentRange
|
||||
|
||||
|
||||
SUPPORTED_BRANDS = VERSIONS.keys()
|
||||
SUPPORTED_CARS = [brand for brand in SUPPORTED_BRANDS for brand in interface_names[brand]]
|
||||
UNKNOWN_BRAND = "unknown"
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description='Run FW fingerprint on Qlog of route or list of routes')
|
||||
parser.add_argument('route', help='Route or file with list of routes')
|
||||
parser.add_argument('--car', help='Force comparison fingerprint to known car')
|
||||
args = parser.parse_args()
|
||||
|
||||
if os.path.exists(args.route):
|
||||
routes = list(open(args.route))
|
||||
else:
|
||||
routes = [args.route]
|
||||
|
||||
mismatches = defaultdict(list)
|
||||
|
||||
not_fingerprinted = 0
|
||||
solved_by_fuzzy = 0
|
||||
|
||||
good_exact = 0
|
||||
wrong_fuzzy = 0
|
||||
good_fuzzy = 0
|
||||
|
||||
dongles = []
|
||||
for route in tqdm(routes):
|
||||
sr = SegmentRange(route)
|
||||
dongle_id = sr.dongle_id
|
||||
|
||||
if dongle_id in dongles:
|
||||
continue
|
||||
|
||||
if sr.slice == '' and sr.selector is None:
|
||||
route += '/0'
|
||||
|
||||
lr = LogReader(route, default_mode=ReadMode.QLOG)
|
||||
|
||||
try:
|
||||
dongles.append(dongle_id)
|
||||
|
||||
CP = None
|
||||
for msg in lr:
|
||||
if msg.which() == "pandaStates":
|
||||
if msg.pandaStates[0].pandaType in ('unknown', 'whitePanda', 'greyPanda', 'pedal'):
|
||||
print("wrong panda type")
|
||||
break
|
||||
|
||||
elif msg.which() == "carParams":
|
||||
CP = msg.carParams
|
||||
car_fw = [fw for fw in CP.carFw if not fw.logging]
|
||||
if len(car_fw) == 0:
|
||||
print("WARNING: no fw")
|
||||
|
||||
live_fingerprint = CP.carFingerprint
|
||||
live_fingerprint = MIGRATION.get(live_fingerprint, live_fingerprint)
|
||||
|
||||
if args.car is not None:
|
||||
live_fingerprint = args.car
|
||||
|
||||
if live_fingerprint not in SUPPORTED_CARS:
|
||||
print("not in supported cars")
|
||||
break
|
||||
|
||||
_, exact_matches = match_fw_to_car(car_fw, CP.carVin, allow_exact=True, allow_fuzzy=False)
|
||||
_, fuzzy_matches = match_fw_to_car(car_fw, CP.carVin, allow_exact=False, allow_fuzzy=True)
|
||||
|
||||
if (len(exact_matches) == 1) and (list(exact_matches)[0] == live_fingerprint):
|
||||
good_exact += 1
|
||||
print(f"Correct! Live: {live_fingerprint} - Fuzzy: {fuzzy_matches}")
|
||||
|
||||
# Check if fuzzy match was correct
|
||||
if len(fuzzy_matches) == 1:
|
||||
if list(fuzzy_matches)[0] != live_fingerprint:
|
||||
wrong_fuzzy += 1
|
||||
print("Fuzzy match wrong! Fuzzy:", fuzzy_matches, "Live:", live_fingerprint)
|
||||
else:
|
||||
good_fuzzy += 1
|
||||
break
|
||||
|
||||
print("Old style:", live_fingerprint, "Vin", CP.carVin)
|
||||
print("New style (exact):", exact_matches)
|
||||
print("New style (fuzzy):", fuzzy_matches)
|
||||
|
||||
padding = max([len(fw.brand or UNKNOWN_BRAND) for fw in car_fw] + [0])
|
||||
for version in sorted(car_fw, key=lambda fw: fw.brand):
|
||||
subaddr = None if version.subAddress == 0 else hex(version.subAddress)
|
||||
print(f" Brand: {version.brand or UNKNOWN_BRAND:{padding}}, bus: {version.bus} - " +
|
||||
f"(Ecu.{version.ecu}, {hex(version.address)}, {subaddr}): [{version.fwVersion}],")
|
||||
|
||||
print("Mismatches")
|
||||
found = False
|
||||
for brand in SUPPORTED_BRANDS:
|
||||
car_fws = VERSIONS[brand]
|
||||
if live_fingerprint in car_fws:
|
||||
found = True
|
||||
expected = car_fws[live_fingerprint]
|
||||
for (_, expected_addr, expected_sub_addr), v in expected.items():
|
||||
for version in car_fw:
|
||||
if version.brand != brand and len(version.brand):
|
||||
continue
|
||||
sub_addr = None if version.subAddress == 0 else version.subAddress
|
||||
addr = version.address
|
||||
|
||||
if (addr, sub_addr) == (expected_addr, expected_sub_addr):
|
||||
if version.fwVersion not in v:
|
||||
print(f"({hex(addr)}, {'None' if sub_addr is None else hex(sub_addr)}) - {version.fwVersion}")
|
||||
|
||||
# Add to global list of mismatches
|
||||
mismatch = (addr, sub_addr, version.fwVersion)
|
||||
if mismatch not in mismatches[live_fingerprint]:
|
||||
mismatches[live_fingerprint].append(mismatch)
|
||||
|
||||
# No FW versions for this car yet, add them all to mismatch list
|
||||
if not found:
|
||||
for version in car_fw:
|
||||
sub_addr = None if version.subAddress == 0 else version.subAddress
|
||||
addr = version.address
|
||||
mismatch = (addr, sub_addr, version.fwVersion)
|
||||
if mismatch not in mismatches[live_fingerprint]:
|
||||
mismatches[live_fingerprint].append(mismatch)
|
||||
|
||||
print()
|
||||
not_fingerprinted += 1
|
||||
|
||||
if len(fuzzy_matches) == 1:
|
||||
if list(fuzzy_matches)[0] == live_fingerprint:
|
||||
solved_by_fuzzy += 1
|
||||
else:
|
||||
wrong_fuzzy += 1
|
||||
print("Fuzzy match wrong! Fuzzy:", fuzzy_matches, "Live:", live_fingerprint)
|
||||
|
||||
break
|
||||
|
||||
if CP is None:
|
||||
print("no CarParams in logs")
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
except KeyboardInterrupt:
|
||||
break
|
||||
|
||||
print()
|
||||
# Print FW versions that need to be added separated out by car and address
|
||||
for car, m in sorted(mismatches.items()):
|
||||
print(car)
|
||||
addrs = defaultdict(list)
|
||||
for (addr, sub_addr, version) in m:
|
||||
addrs[(addr, sub_addr)].append(version)
|
||||
|
||||
for (addr, sub_addr), versions in addrs.items():
|
||||
print(f" ({hex(addr)}, {'None' if sub_addr is None else hex(sub_addr)}): [")
|
||||
for v in versions:
|
||||
print(f" {v},")
|
||||
print(" ]")
|
||||
print()
|
||||
|
||||
print()
|
||||
print(f"Number of dongle ids checked: {len(dongles)}")
|
||||
print(f"Fingerprinted: {good_exact}")
|
||||
print(f"Not fingerprinted: {not_fingerprinted}")
|
||||
print(f" of which had a fuzzy match: {solved_by_fuzzy}")
|
||||
|
||||
print()
|
||||
print(f"Correct fuzzy matches: {good_fuzzy}")
|
||||
print(f"Wrong fuzzy matches: {wrong_fuzzy}")
|
||||
print()
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
|
||||
from openpilot.cereal import log, messaging
|
||||
from opendbc.car.structs import car
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.system.manager.process_config import managed_processes, is_tinygrad_model, is_stock_model
|
||||
from openpilot.common.hardware import HARDWARE
|
||||
|
||||
if __name__ == "__main__":
|
||||
CP = car.CarParams(notCar=True, wheelbase=1, steerRatio=10)
|
||||
params = Params()
|
||||
params.put("CarParams", CP.to_bytes(), block=True)
|
||||
|
||||
if use_tinygrad_modeld := is_tinygrad_model(False, params, CP):
|
||||
print("Using TinyGrad modeld")
|
||||
if use_stock_modeld := is_stock_model(False, params, CP):
|
||||
print("Using stock modeld")
|
||||
|
||||
HARDWARE.set_power_save(False)
|
||||
|
||||
procs = ['camerad', 'ui', 'calibrationd', 'plannerd', 'dmonitoringmodeld', 'dmonitoringd']
|
||||
procs += ["modeld_tinygrad" if use_tinygrad_modeld else "modeld"]
|
||||
for p in procs:
|
||||
managed_processes[p].start()
|
||||
|
||||
pm = messaging.PubMaster(['controlsState', 'deviceState', 'pandaStates', 'carParams'])
|
||||
|
||||
msgs = {s: messaging.new_message(s) for s in ['controlsState', 'deviceState', 'carParams']}
|
||||
msgs['deviceState'].deviceState.started = True
|
||||
msgs['deviceState'].deviceState.deviceType = HARDWARE.get_device_type()
|
||||
msgs['carParams'].carParams.openpilotLongitudinalControl = True
|
||||
|
||||
msgs['pandaStates'] = messaging.new_message('pandaStates', 1)
|
||||
msgs['pandaStates'].pandaStates[0].ignitionLine = True
|
||||
msgs['pandaStates'].pandaStates[0].pandaType = log.PandaState.PandaType.uno
|
||||
|
||||
try:
|
||||
while True:
|
||||
time.sleep(1 / 100) # continually send, rate doesn't matter
|
||||
for s in msgs:
|
||||
pm.send(s, msgs[s])
|
||||
except KeyboardInterrupt:
|
||||
for p in procs:
|
||||
managed_processes[p].stop()
|
||||
Executable
+102
@@ -0,0 +1,102 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import datetime
|
||||
import time
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import numpy as np
|
||||
|
||||
import openpilot.cereal.messaging as messaging
|
||||
from openpilot.cereal.services import SERVICE_LIST
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServiceTiming:
|
||||
times: list[float] = field(default_factory=list)
|
||||
window: deque[float] = field(default_factory=lambda: deque(maxlen=100))
|
||||
valids: deque[bool] = field(default_factory=lambda: deque(maxlen=100))
|
||||
lag_events: list[tuple[float, float]] = field(default_factory=list)
|
||||
|
||||
def add(self, mono_time: float, valid: bool, expected_interval: float | None, lag_threshold: float) -> None:
|
||||
if self.times:
|
||||
dt = mono_time - self.times[-1]
|
||||
self.window.append(dt)
|
||||
if expected_interval is not None and dt > lag_threshold * expected_interval:
|
||||
self.lag_events.append((mono_time, dt))
|
||||
|
||||
self.times.append(mono_time)
|
||||
self.valids.append(valid)
|
||||
|
||||
def intervals(self, latest_only: bool) -> np.ndarray:
|
||||
if latest_only:
|
||||
return np.array(self.window)
|
||||
return np.diff(self.times)
|
||||
|
||||
|
||||
def format_row(name: str, timing: ServiceTiming, latest_only: bool) -> str:
|
||||
dts = timing.intervals(latest_only)
|
||||
if len(dts) == 0:
|
||||
return f"{name:25} waiting for messages"
|
||||
|
||||
mean = np.mean(dts)
|
||||
hz = 1.0 / mean if mean > 0 else 0.0
|
||||
valid = all(timing.valids) if timing.valids else False
|
||||
return f"{name:25} {hz:8.2f}Hz {mean * 1e3:8.2f}ms {np.std(dts) * 1e3:8.2f}ms {np.max(dts) * 1e3:8.2f}ms {np.min(dts) * 1e3:8.2f}ms valid={valid}"
|
||||
|
||||
|
||||
def print_lag_events(name: str, timing: ServiceTiming, printed_lags: dict[str, int]) -> None:
|
||||
start = printed_lags.get(name, 0)
|
||||
for mono_time, dt in timing.lag_events[start:]:
|
||||
print(f"{mono_time:.3f} {name} lag {dt:.3f}s", flush=True)
|
||||
printed_lags[name] = len(timing.lag_events)
|
||||
|
||||
|
||||
def monitor_services(socket_names: list[str], print_interval: float, lag_threshold: float, lag_only: bool) -> None:
|
||||
sockets = {name: messaging.sub_sock(name, conflate=False) for name in socket_names}
|
||||
timings = {name: ServiceTiming() for name in socket_names}
|
||||
printed_lags: dict[str, int] = {}
|
||||
|
||||
start_time = time.monotonic()
|
||||
last_print = start_time
|
||||
|
||||
try:
|
||||
while True:
|
||||
for name, sock in sockets.items():
|
||||
for msg in messaging.drain_sock(sock):
|
||||
expected_interval = 1.0 / SERVICE_LIST[name].frequency if name in SERVICE_LIST else None
|
||||
timings[name].add(msg.logMonoTime / 1e9, msg.valid, expected_interval, lag_threshold)
|
||||
|
||||
now = time.monotonic()
|
||||
if now - last_print < print_interval:
|
||||
time.sleep(0.01)
|
||||
continue
|
||||
|
||||
if not lag_only:
|
||||
print(flush=True)
|
||||
print(f"{'service':25} {'freq':>10} {'mean':>10} {'std':>10} {'max':>10} {'min':>10} valid", flush=True)
|
||||
for name in socket_names:
|
||||
print(format_row(name, timings[name], latest_only=True), flush=True)
|
||||
|
||||
for name in socket_names:
|
||||
print_lag_events(name, timings[name], printed_lags)
|
||||
|
||||
last_print = now
|
||||
except KeyboardInterrupt:
|
||||
print("\n", flush=True)
|
||||
print("=" * 5, "timing summary", "=" * 5, flush=True)
|
||||
print(f"{'service':25} {'freq':>10} {'mean':>10} {'std':>10} {'max':>10} {'min':>10} valid", flush=True)
|
||||
for name in socket_names:
|
||||
print(format_row(name, timings[name], latest_only=False), flush=True)
|
||||
print("=" * 5, datetime.timedelta(seconds=time.monotonic() - start_time), "=" * 5, flush=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Check live service timing, frequency, validity, and lag")
|
||||
parser.add_argument("socket", nargs="*", default=["carState"], help="service/socket name")
|
||||
parser.add_argument("--lag-threshold", type=float, default=10.0, help="report intervals above this multiple of the expected service interval")
|
||||
parser.add_argument("--lag-only", action="store_true", help="only print lag events")
|
||||
parser.add_argument("--print-interval", type=float, default=1.0, help="seconds between table updates")
|
||||
args = parser.parse_args()
|
||||
|
||||
monitor_services(args.socket, args.print_interval, args.lag_threshold, args.lag_only)
|
||||
Executable
+151
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
if [ -z "$OPENPILOT_ROOT" ]; then
|
||||
# default to current directory for installation
|
||||
OPENPILOT_ROOT="$(pwd)/openpilot"
|
||||
fi
|
||||
|
||||
function show_motd() {
|
||||
cat << 'EOF'
|
||||
|
||||
.~ssos+.
|
||||
+8888888888i,
|
||||
{888888888888o.
|
||||
h8888888888888k
|
||||
t888888888s888k
|
||||
`t88888d/ h88k
|
||||
``` h88l
|
||||
,88k`
|
||||
.d8h`
|
||||
+d8h
|
||||
_+d8h`
|
||||
;y8h+`
|
||||
|-`
|
||||
|
||||
openpilot installer
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
function check_platform() {
|
||||
if [[ -f /AGNOS ]]; then
|
||||
echo -e "[${RED}✗${NC}] This installer is for PCs only. The environment is pre-configured in AGNOS."
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
function check_stdin() {
|
||||
if [ -t 0 ]; then
|
||||
INTERACTIVE=1
|
||||
else
|
||||
echo "Checking for valid invocation..."
|
||||
echo -e " ↳ [${RED}✗${NC}] stdin not found! Running in non-interactive mode."
|
||||
echo -e " Run ${BOLD}'bash <(curl -fsSL openpilot.comma.ai)'${NC} to run in interactive mode.\n"
|
||||
fi
|
||||
}
|
||||
|
||||
function ask_dir() {
|
||||
echo -n "Enter directory in which to install openpilot (default $OPENPILOT_ROOT): "
|
||||
|
||||
if [[ -z $INTERACTIVE ]]; then
|
||||
echo -e "\nBecause your are running in non-interactive mode, the installation"
|
||||
echo -e "will default to $OPENPILOT_ROOT\n"
|
||||
return 0
|
||||
fi
|
||||
|
||||
read
|
||||
if [[ ! -z "$REPLY" ]]; then
|
||||
mkdir -p $REPLY
|
||||
OPENPILOT_ROOT="$(realpath $REPLY)/openpilot"
|
||||
fi
|
||||
}
|
||||
|
||||
function check_dir() {
|
||||
echo "Checking for installation directory..."
|
||||
if [ -d "$OPENPILOT_ROOT" ]; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Installation destination $OPENPILOT_ROOT already exists!"
|
||||
|
||||
# not a valid clone, can't continue
|
||||
if [[ ! -z "$(ls -A $OPENPILOT_ROOT)" && ! -f "$OPENPILOT_ROOT/launch_openpilot.sh" ]]; then
|
||||
echo -e " $OPENPILOT_ROOT already contains files but does not seems"
|
||||
echo -e " to be a valid openpilot git clone. Choose another location for"
|
||||
echo -e " installing openpilot!\n"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# already a "valid" openpilot clone, skip cloning again
|
||||
if [[ ! -z "$(ls -A $OPENPILOT_ROOT)" ]]; then
|
||||
SKIP_GIT_CLONE=1
|
||||
fi
|
||||
|
||||
# by default, don't try installing in already existing directory
|
||||
if [[ -z $INTERACTIVE ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
read -p " Would you like to attempt installation anyway? [Y/n] " -n 1 -r
|
||||
echo -e "\n"
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Successfully chosen $OPENPILOT_ROOT as installation directory\n"
|
||||
}
|
||||
|
||||
function check_git() {
|
||||
echo "Checking for git..."
|
||||
if ! command -v "git" > /dev/null 2>&1; then
|
||||
echo -e " ↳ [${RED}✗${NC}] git not found on your system, can't continue!"
|
||||
return 1
|
||||
else
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git found.\n"
|
||||
fi
|
||||
}
|
||||
|
||||
function git_clone() {
|
||||
st="$(date +%s)"
|
||||
echo "Cloning openpilot..."
|
||||
if $(git clone --filter=blob:none https://github.com/commaai/openpilot.git "$OPENPILOT_ROOT"); then
|
||||
if [[ -f $OPENPILOT_ROOT/launch_openpilot.sh ]]; then
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Successfully cloned openpilot in $((et - st)) seconds.\n"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e " ↳ [${RED}✗${NC}] failed to clone openpilot!"
|
||||
return 1
|
||||
}
|
||||
|
||||
function install_with_op() {
|
||||
cd $OPENPILOT_ROOT
|
||||
$OPENPILOT_ROOT/tools/op.sh post-commit
|
||||
|
||||
if ! $OPENPILOT_ROOT/tools/op.sh setup; then
|
||||
echo -e "\n[${RED}✗${NC}] failed to install openpilot!"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo -e "\n----------------------------------------------------------------------"
|
||||
echo -e "[${GREEN}✔${NC}] openpilot was successfully installed into ${BOLD}$OPENPILOT_ROOT${NC}"
|
||||
echo -e "Checkout the docs at https://docs.comma.ai"
|
||||
echo -e "Checkout how to contribute at https://github.com/commaai/openpilot/blob/master/docs/CONTRIBUTING.md"
|
||||
}
|
||||
|
||||
show_motd
|
||||
check_platform
|
||||
check_stdin
|
||||
ask_dir
|
||||
check_dir
|
||||
check_git
|
||||
[ -z $SKIP_GIT_CLONE ] && git_clone
|
||||
install_with_op
|
||||
Executable
+142
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT="$(git -C "$DIR" rev-parse --show-toplevel)"
|
||||
|
||||
function retry() {
|
||||
local attempts=$1
|
||||
shift
|
||||
for i in $(seq 1 "$attempts"); do
|
||||
if "$@"; then
|
||||
return 0
|
||||
fi
|
||||
if [ "$i" -lt "$attempts" ]; then
|
||||
echo " Attempt $i/$attempts failed, retrying in 5s..."
|
||||
sleep 5
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
function install_linux_deps() {
|
||||
SUDO=""
|
||||
|
||||
if [[ ! $(id -u) -eq 0 ]]; then
|
||||
if [[ -z $(which sudo) ]]; then
|
||||
echo "Please install sudo or run as root"
|
||||
exit 1
|
||||
fi
|
||||
SUDO="sudo"
|
||||
fi
|
||||
|
||||
local missing_linux_deps=0
|
||||
for cmd in gcc g++ make curl curl-config git; do
|
||||
if ! command -v "$cmd" > /dev/null 2>&1; then
|
||||
missing_linux_deps=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# ------------------------------------------------
|
||||
# dependencies should never be added to this list.
|
||||
# these are only for inflating bare docker images
|
||||
# to their desktop equivalents.
|
||||
# ------------------------------------------------
|
||||
if [[ "$missing_linux_deps" -eq 0 ]]; then
|
||||
# the native package managers are slow, so skip if we can
|
||||
echo "[ ] system packages already installed t=$SECONDS"
|
||||
elif command -v apt-get > /dev/null 2>&1; then
|
||||
$SUDO apt-get update
|
||||
$SUDO apt-get install -y --no-install-recommends ca-certificates build-essential curl libcurl4-openssl-dev locales git xclip wl-clipboard
|
||||
elif command -v dnf > /dev/null 2>&1; then
|
||||
$SUDO dnf install -y ca-certificates gcc gcc-c++ make curl libcurl-devel glibc-langpack-en git
|
||||
elif command -v yum > /dev/null 2>&1; then
|
||||
$SUDO yum install -y ca-certificates gcc gcc-c++ make curl libcurl-devel glibc-langpack-en git
|
||||
elif command -v pacman > /dev/null 2>&1; then
|
||||
$SUDO pacman -Syu --noconfirm --needed base-devel ca-certificates curl git
|
||||
elif command -v zypper > /dev/null 2>&1; then
|
||||
$SUDO zypper --non-interactive refresh
|
||||
$SUDO zypper --non-interactive install ca-certificates gcc gcc-c++ make curl libcurl-devel glibc-locale git
|
||||
elif command -v apk > /dev/null 2>&1; then
|
||||
$SUDO apk add --no-cache ca-certificates build-base curl curl-dev musl-locales git
|
||||
elif command -v xbps-install > /dev/null 2>&1; then
|
||||
$SUDO xbps-install -Syu base-devel ca-certificates curl git libcurl-devel glibc-locales
|
||||
else
|
||||
echo "Unsupported Linux distribution. Supported package managers: apt-get, dnf, yum, pacman, zypper, apk, xbps-install."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ -d "/etc/udev/rules.d/" ]]; then
|
||||
$SUDO tee /etc/udev/rules.d/11-openpilot.rules > /dev/null <<-EOF
|
||||
# Panda Jungle devices
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
|
||||
# Panda devices
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0483", ATTRS{idProduct}=="df11", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
|
||||
# comma devices over ADB
|
||||
SUBSYSTEM=="usb", ATTR{idVendor}=="04d8", ATTR{idProduct}=="1234", ENV{adb_user}="yes"
|
||||
EOF
|
||||
|
||||
# delete the old ones
|
||||
$SUDO rm -f /etc/udev/rules.d/11-panda.rules /etc/udev/rules.d/12-panda_jungle.rules /etc/udev/rules.d/50-comma-adb.rules
|
||||
|
||||
$SUDO udevadm control --reload-rules && $SUDO udevadm trigger || true
|
||||
fi
|
||||
}
|
||||
|
||||
function install_python_deps() {
|
||||
# Increase the pip timeout to handle TimeoutError
|
||||
export PIP_DEFAULT_TIMEOUT=200
|
||||
|
||||
cd "$ROOT"
|
||||
|
||||
if ! command -v "uv" > /dev/null 2>&1; then
|
||||
echo "installing uv..."
|
||||
# TODO: outer retry can be removed once https://github.com/axodotdev/cargo-dist/pull/2311 is merged
|
||||
retry 3 sh -c 'curl --retry 5 --retry-delay 5 --retry-all-errors -LsSf https://astral.sh/uv/install.sh | UV_GITHUB_TOKEN="${GITHUB_TOKEN:-}" sh'
|
||||
UV_BIN="$HOME/.local/bin"
|
||||
PATH="$UV_BIN:$PATH"
|
||||
fi
|
||||
|
||||
echo "updating uv..."
|
||||
# ok to fail, can also fail due to installing with brew
|
||||
uv self update || true
|
||||
|
||||
echo "installing python packages..."
|
||||
uv sync --frozen --all-extras
|
||||
source .venv/bin/activate
|
||||
}
|
||||
|
||||
# --- Main ---
|
||||
|
||||
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
install_linux_deps
|
||||
echo "[ ] installed system dependencies t=$SECONDS"
|
||||
elif [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
if [[ $SHELL == "/bin/zsh" ]]; then
|
||||
RC_FILE="$HOME/.zshrc"
|
||||
elif [[ $SHELL == "/bin/bash" ]]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "$ROOT/pyproject.toml" ]; then
|
||||
install_python_deps
|
||||
echo "[ ] installed python dependencies t=$SECONDS"
|
||||
fi
|
||||
|
||||
if [[ "$OSTYPE" == "darwin"* ]] && [[ -n "${RC_FILE:-}" ]]; then
|
||||
echo
|
||||
echo "---- OPENPILOT SETUP DONE ----"
|
||||
echo "Open a new shell or configure your active shell env by running:"
|
||||
echo "source $RC_FILE"
|
||||
fi
|
||||
Executable
+309
@@ -0,0 +1,309 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
from collections import Counter
|
||||
from concurrent.futures import as_completed, ProcessPoolExecutor
|
||||
from itertools import batched
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import traceback
|
||||
import unittest
|
||||
import warnings
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
IGNORED = (
|
||||
ROOT / "openpilot/selfdrive/test/process_replay/test_processes.py",
|
||||
ROOT / "openpilot/tools/sim",
|
||||
)
|
||||
FAILURES = {"failed", "error", "xpassed"}
|
||||
STATUS_MARKS = {
|
||||
"passed": (".", 32),
|
||||
"skipped": ("s", 33),
|
||||
"xfailed": ("x", 36),
|
||||
"failed": ("F", 31),
|
||||
"error": ("E", 31),
|
||||
"xpassed": ("X", 31),
|
||||
}
|
||||
|
||||
|
||||
def paint(text, code):
|
||||
if sys.stdout.isatty() and "NO_COLOR" not in os.environ:
|
||||
return f"\033[{code}m{text}\033[0m"
|
||||
return text
|
||||
|
||||
|
||||
class Capture:
|
||||
def __init__(self, enabled):
|
||||
self.enabled = enabled
|
||||
|
||||
def start(self):
|
||||
if not self.enabled:
|
||||
return
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
stream.flush()
|
||||
self.files = [tempfile.TemporaryFile() for _ in range(2)]
|
||||
self.saved = [os.dup(fd) for fd in (1, 2)]
|
||||
for fd, file in enumerate(self.files, 1):
|
||||
os.dup2(file.fileno(), fd)
|
||||
|
||||
def stop(self, keep=True):
|
||||
if not self.enabled:
|
||||
return "", ""
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
stream.flush()
|
||||
for fd, saved in enumerate(self.saved, 1):
|
||||
os.dup2(saved, fd)
|
||||
os.close(saved)
|
||||
output = []
|
||||
for file in self.files:
|
||||
if keep:
|
||||
file.seek(0)
|
||||
output.append(file.read().decode(errors="replace"))
|
||||
file.close()
|
||||
return output if keep else ("", "")
|
||||
|
||||
|
||||
def make_record(test_id, status="passed", detail=""):
|
||||
return {"id": test_id, "status": status, "detail": detail, "time": 0.0, "stdout": "", "stderr": ""}
|
||||
|
||||
|
||||
class Result(unittest.TestResult):
|
||||
def __init__(self, capture_output):
|
||||
super().__init__()
|
||||
self.records = []
|
||||
self.current = None
|
||||
self.capture = Capture(capture_output)
|
||||
|
||||
def startTest(self, test):
|
||||
self.current = make_record(test.id())
|
||||
self.started = time.monotonic()
|
||||
self.capture.start()
|
||||
|
||||
def stopTest(self, test):
|
||||
self.current["time"] = time.monotonic() - self.started
|
||||
keep = self.current["status"] in FAILURES
|
||||
stdout, stderr = self.capture.stop(keep)
|
||||
if keep:
|
||||
self.current["stdout"] = stdout
|
||||
self.current["stderr"] = stderr
|
||||
self.records.append(self.current)
|
||||
self.current = None
|
||||
|
||||
def mark(self, test, status, detail=""):
|
||||
if self.current is None: # setUpClass/setUpModule can fail before a test starts
|
||||
self.records.append(make_record(test.id(), status, detail))
|
||||
return
|
||||
if self.current["status"] not in FAILURES or status == "error":
|
||||
self.current["status"] = status
|
||||
if detail:
|
||||
self.current["detail"] += ("\n\n" if self.current["detail"] else "") + detail
|
||||
|
||||
def addFailure(self, test, err):
|
||||
self.mark(test, "failed", self._exc_info_to_string(err, test))
|
||||
|
||||
def addError(self, test, err):
|
||||
self.mark(test, "error", self._exc_info_to_string(err, test))
|
||||
|
||||
def addSkip(self, test, reason):
|
||||
self.mark(test, "skipped")
|
||||
|
||||
def addExpectedFailure(self, test, err):
|
||||
self.mark(test, "xfailed")
|
||||
|
||||
def addUnexpectedSuccess(self, test):
|
||||
self.mark(test, "xpassed", "Test was expected to fail, but passed.")
|
||||
|
||||
def addSubTest(self, test, subtest, err):
|
||||
if err:
|
||||
status = "failed" if issubclass(err[0], test.failureException) else "error"
|
||||
self.mark(test, status, f"{subtest}\n{self._exc_info_to_string(err, test)}")
|
||||
|
||||
|
||||
def flatten(suite):
|
||||
for test in suite:
|
||||
if isinstance(test, unittest.TestSuite):
|
||||
yield from flatten(test)
|
||||
else:
|
||||
yield test
|
||||
|
||||
|
||||
def module_name(path):
|
||||
return ".".join(path.resolve().relative_to(ROOT).with_suffix("").parts)
|
||||
|
||||
|
||||
def collect(targets, keyword):
|
||||
use_ignores = not targets
|
||||
targets = targets or ["openpilot"]
|
||||
loader = unittest.TestLoader()
|
||||
tests = []
|
||||
errors = []
|
||||
names = []
|
||||
for target in targets:
|
||||
path_text, *nodes = target.split("::")
|
||||
path = Path(path_text)
|
||||
try:
|
||||
if path.is_dir():
|
||||
files = sorted(path.rglob("test_*.py"))
|
||||
if use_ignores:
|
||||
files = [f for f in files if not any(f.resolve().is_relative_to(i) for i in IGNORED)]
|
||||
names.extend(module_name(file) for file in files)
|
||||
elif path.is_file():
|
||||
names.append(".".join((module_name(path), *nodes)))
|
||||
elif "/" in path_text or path_text.endswith(".py"):
|
||||
errors.append(f"{target}: file or directory not found")
|
||||
else:
|
||||
names.append(target.replace("::", "."))
|
||||
except (OSError, ValueError) as e:
|
||||
errors.append(str(e))
|
||||
for name in dict.fromkeys(names):
|
||||
before = len(loader.errors)
|
||||
try:
|
||||
suite = loader.loadTestsFromName(name)
|
||||
except Exception:
|
||||
errors.append(f"Failed to collect {name}\n{traceback.format_exc()}")
|
||||
continue
|
||||
errors.extend(loader.errors[before:])
|
||||
for test in flatten(suite):
|
||||
cls = type(test)
|
||||
if cls.__name__ == "_FailedTest":
|
||||
continue
|
||||
if getattr(cls, "__unittest_skip_why__", "") == "parameterized base class":
|
||||
continue
|
||||
if not keyword or keyword.lower() in test.id().lower():
|
||||
tests.append(test)
|
||||
return list({test.id(): test for test in tests}.values()), errors
|
||||
|
||||
|
||||
def make_batches(tests, workers):
|
||||
fixture_groups = {}
|
||||
parallel = []
|
||||
for test in tests:
|
||||
cls = type(test)
|
||||
module = sys.modules[cls.__module__]
|
||||
if hasattr(module, "setUpModule") or hasattr(module, "tearDownModule"):
|
||||
key = cls.__module__
|
||||
elif "setUpClass" in cls.__dict__ or "tearDownClass" in cls.__dict__:
|
||||
key = f"{cls.__module__}.{cls.__qualname__}"
|
||||
else:
|
||||
parallel.append(test.id())
|
||||
continue
|
||||
fixture_groups.setdefault(key, []).append(test.id())
|
||||
size = max(1, math.ceil(len(tests) / (workers * 4)))
|
||||
batches = list(fixture_groups.values())
|
||||
batches.extend(list(batch) for batch in batched(parallel, size))
|
||||
return sorted(batches, key=len, reverse=True)
|
||||
|
||||
|
||||
def run_batch(test_ids, capture_output):
|
||||
result = Result(capture_output)
|
||||
outside = Capture(capture_output)
|
||||
os.chdir(ROOT)
|
||||
outside.start()
|
||||
try:
|
||||
unittest.TestLoader().loadTestsFromNames(test_ids).run(result)
|
||||
finally:
|
||||
stdout, stderr = outside.stop()
|
||||
failures = [item for item in result.records if item["status"] in FAILURES]
|
||||
if failures: # attach class/module fixture output to the first related failure
|
||||
failures[0]["stdout"] = stdout + failures[0]["stdout"]
|
||||
failures[0]["stderr"] = stderr + failures[0]["stderr"]
|
||||
return result.records
|
||||
|
||||
|
||||
def run_parallel(batches, workers, warning_action, capture_output):
|
||||
with ProcessPoolExecutor(max_workers=workers, initializer=warnings.simplefilter, initargs=(warning_action,)) as pool:
|
||||
futures = {pool.submit(run_batch, batch, capture_output): batch for batch in batches}
|
||||
for future in as_completed(futures):
|
||||
try:
|
||||
yield future.result()
|
||||
except Exception:
|
||||
yield [make_record(futures[future][0], "error", traceback.format_exc())]
|
||||
|
||||
|
||||
def report(records, errors, duration_count, elapsed):
|
||||
width = min(100, os.get_terminal_size().columns if sys.stdout.isatty() else 80)
|
||||
for index, error in enumerate(errors, 1):
|
||||
print(paint(f"\n{'=' * 8} COLLECTION ERROR {index} {'=' * 8}", 31))
|
||||
print(error.rstrip())
|
||||
for item in sorted((r for r in records if r["status"] in FAILURES), key=lambda r: r["id"]):
|
||||
heading = f" {item['status'].upper()} {item['id']} "
|
||||
print(paint(f"\n{heading:=^{width}}", 31))
|
||||
if item["detail"]:
|
||||
print(item["detail"].rstrip())
|
||||
for stream in ("stdout", "stderr"):
|
||||
if item[stream]:
|
||||
print(paint(f"\n--- captured {stream} ---", 33))
|
||||
print(item[stream].rstrip())
|
||||
timed = sorted((r for r in records if r["time"]), key=lambda r: r["time"], reverse=True)
|
||||
if duration_count:
|
||||
timed = timed[:duration_count]
|
||||
if timed:
|
||||
print(paint("\nslowest tests", 36))
|
||||
for item in timed:
|
||||
print(f"{item['time']:8.2f}s {item['id']}")
|
||||
counts = Counter(item["status"] for item in records)
|
||||
parts = [f"{counts[name]} {name}" for name in STATUS_MARKS if counts[name]]
|
||||
if errors:
|
||||
parts.append(f"{len(errors)} collection error{'s' if len(errors) != 1 else ''}")
|
||||
failed = bool(errors) or any(counts[name] for name in FAILURES)
|
||||
print(paint(f"\n{', '.join(parts) or 'no tests ran'} in {elapsed:.2f}s", 31 if failed else 32))
|
||||
if failed:
|
||||
return 1
|
||||
if records:
|
||||
return 0
|
||||
return 5
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("targets", nargs="*", help="files, directories, dotted IDs, or path.py::Class::test")
|
||||
parser.add_argument("-j", "--jobs", type=int, default=os.cpu_count() or 1, help="workers (default: available CPUs)")
|
||||
parser.add_argument("-k", metavar="TEXT", help="only run test IDs containing TEXT")
|
||||
parser.add_argument("-s", "--no-capture", action="store_true", help="show test output live")
|
||||
parser.add_argument("-v", "--verbose", action="store_true", help="show every test")
|
||||
parser.add_argument("--durations", type=int, default=10, metavar="N", help="show N slowest tests; 0 shows all")
|
||||
parser.add_argument("-W", "--warnings", choices=("error", "default", "always", "ignore"), default="error")
|
||||
args = parser.parse_args()
|
||||
|
||||
capture_output = not args.no_capture
|
||||
os.chdir(ROOT)
|
||||
warnings.simplefilter(args.warnings)
|
||||
started = time.monotonic()
|
||||
tests, errors = collect(args.targets, args.k)
|
||||
batches = make_batches(tests, args.jobs)
|
||||
workers = min(args.jobs, len(batches))
|
||||
summary = f"collected {len(tests)} test{'s' if len(tests) != 1 else ''} in {time.monotonic() - started:.2f}s "
|
||||
summary += f"• {workers} worker{'s' if workers != 1 else ''}"
|
||||
print(summary)
|
||||
records = []
|
||||
column = 0
|
||||
try:
|
||||
if workers < 2:
|
||||
streams = (run_batch(batch, capture_output) for batch in batches)
|
||||
else:
|
||||
streams = run_parallel(batches, workers, args.warnings, capture_output)
|
||||
for batch in streams:
|
||||
records.extend(batch)
|
||||
for item in batch:
|
||||
mark, code = STATUS_MARKS[item["status"]]
|
||||
if args.verbose:
|
||||
print(f"{paint(mark, code)} {item['id']} {item['time']:.2f}s")
|
||||
else:
|
||||
print(paint(mark, code), end="", flush=True)
|
||||
column += 1
|
||||
if column == 80:
|
||||
print()
|
||||
column = 0
|
||||
except KeyboardInterrupt:
|
||||
print(paint("\ninterrupted", 31))
|
||||
return 2
|
||||
if column:
|
||||
print()
|
||||
return report(records, errors, args.durations, time.monotonic() - started)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user