mirror of
https://github.com/dragonpilot/dragonpilot.git
synced 2026-08-24 01:33:50 +08:00
dragonpilot v0.9.8
This commit is contained in:
Comma Device
@@ -0,0 +1,20 @@
|
||||
## CTF
|
||||
Welcome to the first part of the comma CTF!
|
||||
|
||||
* all the flags are contained in this route: `0c7f0c7f0c7f0c7f|2021-10-13--13-00-00`
|
||||
* there's 2 flags in each segment, with roughly increasing difficulty
|
||||
* everything you'll need to find the flags is in the openpilot repo
|
||||
* grep is also your friend
|
||||
* first, [setup](https://github.com/commaai/openpilot/tree/master/tools#setup-your-pc) your PC
|
||||
* read the docs & checkout out the tools in tools/ and selfdrive/debug/
|
||||
* tip: once you get the replay and UI up, start by familiarizing yourself with seeking in replay
|
||||
|
||||
getting started
|
||||
```bash
|
||||
# start the route replay
|
||||
cd tools/replay
|
||||
./replay '0c7f0c7f0c7f0c7f|2021-10-13--13-00-00' --dcam --ecam
|
||||
|
||||
# start the UI in another terminal
|
||||
selfdrive/ui/ui
|
||||
```
|
||||
@@ -0,0 +1,77 @@
|
||||
# openpilot tools
|
||||
|
||||
## System Requirements
|
||||
|
||||
openpilot is developed and tested on **Ubuntu 24.04**, which is the primary development target aside from the [supported embedded hardware](https://github.com/commaai/openpilot#running-on-a-dedicated-device-in-a-car).
|
||||
|
||||
Most of openpilot should work natively on macOS. On Windows you can use WSL for a nearly native Ubuntu experience. Running natively on any other system is not currently recommended and will likely require modifications.
|
||||
|
||||
## Native setup on Ubuntu 24.04 and macOS
|
||||
|
||||
**1. Clone openpilot**
|
||||
|
||||
NOTE: This repository uses Git LFS for large files. Ensure you have [Git LFS](https://git-lfs.com/) installed and set up before cloning or working with it.
|
||||
|
||||
Either do a partial clone for faster download:
|
||||
``` bash
|
||||
git clone --filter=blob:none --recurse-submodules --also-filter-submodules https://github.com/commaai/openpilot.git
|
||||
```
|
||||
|
||||
or do a full clone:
|
||||
``` bash
|
||||
git clone --recurse-submodules https://github.com/commaai/openpilot.git
|
||||
```
|
||||
|
||||
**2. Run the setup script**
|
||||
|
||||
``` bash
|
||||
cd openpilot
|
||||
tools/op.sh setup
|
||||
```
|
||||
|
||||
**3. Git LFS**
|
||||
|
||||
``` bash
|
||||
git lfs pull
|
||||
```
|
||||
|
||||
**4. Activate a python shell**
|
||||
|
||||
Activate a shell with the Python dependencies installed:
|
||||
``` bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**5. Build openpilot**
|
||||
|
||||
``` bash
|
||||
scons -u -j$(nproc)
|
||||
```
|
||||
|
||||
## WSL on Windows
|
||||
|
||||
[Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) should provide a similar experience to native Ubuntu. [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/compare-versions) specifically has been reported by several users to be a seamless experience.
|
||||
|
||||
Follow [these instructions](https://docs.microsoft.com/en-us/windows/wsl/install) to setup the WSL and install the `Ubuntu-24.04` distribution. Once your Ubuntu WSL environment is setup, follow the Linux setup instructions to finish setting up your environment. See [these instructions](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps) for running GUI apps.
|
||||
|
||||
**NOTE**: If you are running WSL and any GUIs are failing (segfaulting or other strange issues) even after following the steps above, you may need to enable software rendering with `LIBGL_ALWAYS_SOFTWARE=1`, e.g. `LIBGL_ALWAYS_SOFTWARE=1 selfdrive/ui/ui`.
|
||||
|
||||
## CTF
|
||||
Learn about the openpilot ecosystem and tools by playing our [CTF](/tools/CTF.md).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
├── ubuntu_setup.sh # Setup script for Ubuntu
|
||||
├── mac_setup.sh # Setup script for macOS
|
||||
├── cabana/ # View and plot CAN messages from drives or in realtime
|
||||
├── camerastream/ # Cameras stream over the network
|
||||
├── joystick/ # Control your car with a joystick
|
||||
├── lib/ # Libraries to support the tools and reading openpilot logs
|
||||
├── plotjuggler/ # A tool to plot openpilot logs
|
||||
├── replay/ # Replay drives and mock openpilot services
|
||||
├── scripts/ # Miscellaneous scripts
|
||||
├── serial/ # Tools for using the comma serial
|
||||
├── sim/ # Run openpilot in a simulator
|
||||
└── webcam/ # Run openpilot on a PC with webcams
|
||||
```
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env expect
|
||||
spawn adb shell
|
||||
expect "#"
|
||||
send "cd data/openpilot\r"
|
||||
send "export TERM=xterm-256color\r"
|
||||
send "su comma\r"
|
||||
send "clear\r"
|
||||
interact
|
||||
@@ -0,0 +1,4 @@
|
||||
av
|
||||
av-10.0.0/*
|
||||
key.pem
|
||||
cert.pem
|
||||
@@ -0,0 +1,103 @@
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>commabody</title>
|
||||
<link rel="stylesheet" href="/static/main.css">
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/css/bootstrap.min.css" integrity="sha512-SbiR/eusphKoMVVXysTKG/7VseWii+Y3FdHrt0EpKgpToZeemhqHeZeLWLhJutz/2ut2Vw1uQEj2MbRF+TVBUA==" crossorigin="anonymous" referrerpolicy="no-referrer" /><script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.5/font/bootstrap-icons.css">
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/js/bootstrap.min.js" integrity="sha512-1/RvZTcCDEUjY/CypiMz+iqqtaoQfAITmNSJY17Myp4Ms5mdxPS5UV7iOfdZoxcGhzFbOm6sntTKJppjvuhg4g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@^3"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/moment@^2"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-moment@^1"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="main">
|
||||
<p class="jumbo">comma body</p>
|
||||
<audio id="audio" autoplay="true"></audio>
|
||||
<video id="video" playsinline autoplay muted loop poster="/static/poster.png"></video>
|
||||
<div id="icon-panel" class="row">
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
<i class="bi bi-camera-video-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">body</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">you</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="ping-time" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-arrow-repeat small"> ping time</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="battery" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-battery-half small"> battery</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- <div class="icon-sub-panel">
|
||||
<button type="button" id="start" class="btn btn-light btn-lg">Start</button>
|
||||
<button type="button" id="stop" class="btn btn-light btn-lg">Stop</button>
|
||||
</div> -->
|
||||
</div>
|
||||
<div class="row" style="width: 100%; padding: 0px 10px 0px 10px;">
|
||||
<div id="wasd" class="col-md-12 row">
|
||||
<div class="col-md-6 col-sm-12" style="justify-content: center; display: flex; flex-direction: column;">
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-w">W</div>
|
||||
<div id="key-val"><span id="pos-vals">0,0</span><span>x,y</span></div>
|
||||
</div>
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-a">A</div>
|
||||
<div class="keys" id="key-s">S</div>
|
||||
<div class="keys" id="key-d">D</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6 col-sm-12 form-group plan-form">
|
||||
<label for="plan-text">Plan (w, a, s, d, t)</label>
|
||||
<label style="font-size: 15px;" for="plan-text">*Extremely Experimental*</label>
|
||||
<textarea class="form-control" id="plan-text" rows="7" placeholder="1,0,0,0,2"></textarea>
|
||||
<button type="button" id="plan-button" class="btn btn-light btn-lg">Execute</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-3" style="text-align: center;">
|
||||
<p>Play Sounds</p>
|
||||
</div>
|
||||
<div class="btn-group col-sm-8">
|
||||
<button type="button" id="sound-engage" class="btn btn-outline-success btn-lg sound">Engage</button>
|
||||
<button type="button" id="sound-disengage" class="btn btn-outline-warning btn-lg sound">Disengage</button>
|
||||
<button type="button" id="sound-error" class="btn btn-outline-danger btn-lg sound">Error</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-6"><canvas id="chart-ping"></canvas></div>
|
||||
<div class="col-sm-6"><canvas id="chart-battery"></canvas></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<script src="/static/js/jsmain.js" type="module"></script>
|
||||
</body>
|
||||
</html>
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
const keyVals = {w: 0, a: 0, s: 0, d: 0}
|
||||
|
||||
export function getXY() {
|
||||
let x = -keyVals.w + keyVals.s
|
||||
let y = -keyVals.d + keyVals.a
|
||||
return {x, y}
|
||||
}
|
||||
|
||||
export const handleKeyX = (key, setValue) => {
|
||||
if (['w', 'a', 's', 'd'].includes(key)){
|
||||
keyVals[key] = setValue;
|
||||
let color = "#333";
|
||||
if (setValue === 1){
|
||||
color = "#e74c3c";
|
||||
}
|
||||
$("#key-"+key).css('background', color);
|
||||
const {x, y} = getXY();
|
||||
$("#pos-vals").text(x+","+y);
|
||||
}
|
||||
};
|
||||
|
||||
export async function executePlan() {
|
||||
let plan = $("#plan-text").val();
|
||||
const planList = [];
|
||||
plan.split("\n").forEach(function(e){
|
||||
let line = e.split(",").map(k=>parseInt(k));
|
||||
if (line.length != 5 || line.slice(0, 4).map(e=>[1, 0].includes(e)).includes(false) || line[4] < 0 || line[4] > 10){
|
||||
console.log("invalid plan");
|
||||
}
|
||||
else{
|
||||
planList.push(line)
|
||||
}
|
||||
});
|
||||
|
||||
async function execute() {
|
||||
for (var i = 0; i < planList.length; i++) {
|
||||
let [w, a, s, d, t] = planList[i];
|
||||
while(t > 0){
|
||||
console.log(w, a, s, d, t);
|
||||
if(w==1){$("#key-w").mousedown();}
|
||||
if(a==1){$("#key-a").mousedown();}
|
||||
if(s==1){$("#key-s").mousedown();}
|
||||
if(d==1){$("#key-d").mousedown();}
|
||||
await sleep(50);
|
||||
$("#key-w").mouseup();
|
||||
$("#key-a").mouseup();
|
||||
$("#key-s").mouseup();
|
||||
$("#key-d").mouseup();
|
||||
t = t - 0.05;
|
||||
}
|
||||
}
|
||||
}
|
||||
execute();
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { handleKeyX, executePlan } from "./controls.js";
|
||||
import { start, stop, lastChannelMessageTime, playSoundRequest } from "./webrtc.js";
|
||||
|
||||
export var pc = null;
|
||||
export var dc = null;
|
||||
|
||||
document.addEventListener('keydown', (e)=>(handleKeyX(e.key.toLowerCase(), 1)));
|
||||
document.addEventListener('keyup', (e)=>(handleKeyX(e.key.toLowerCase(), 0)));
|
||||
$(".keys").bind("mousedown touchstart", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 1));
|
||||
$(".keys").bind("mouseup touchend", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 0));
|
||||
$("#plan-button").click(executePlan);
|
||||
$(".sound").click((e)=>{
|
||||
const sound = $(e.target).attr('id').replace('sound-', '')
|
||||
return playSoundRequest(sound);
|
||||
});
|
||||
|
||||
setInterval( () => {
|
||||
const dt = new Date().getTime();
|
||||
if ((dt - lastChannelMessageTime) > 1000) {
|
||||
$(".pre-blob").removeClass('blob');
|
||||
$("#battery").text("-");
|
||||
$("#ping-time").text('-');
|
||||
$("video")[0].load();
|
||||
}
|
||||
}, 5000);
|
||||
|
||||
start(pc, dc);
|
||||
@@ -0,0 +1,53 @@
|
||||
export const pingPoints = [];
|
||||
export const batteryPoints = [];
|
||||
|
||||
function getChartConfig(pts, color, title, ymax=100) {
|
||||
return {
|
||||
type: 'line',
|
||||
data: {
|
||||
datasets: [{
|
||||
label: title,
|
||||
data: pts,
|
||||
borderWidth: 1,
|
||||
borderColor: color,
|
||||
backgroundColor: color,
|
||||
fill: 'origin'
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
scales: {
|
||||
x: {
|
||||
type: 'time',
|
||||
time: {
|
||||
unit: 'minute',
|
||||
displayFormats: {
|
||||
second: 'h:mm a'
|
||||
}
|
||||
},
|
||||
grid: {
|
||||
color: '#222', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
source: 'data',
|
||||
fontColor: 'rgba(255, 255, 255, 1.0)', // Y-axis label color
|
||||
}
|
||||
},
|
||||
y: {
|
||||
beginAtZero: true,
|
||||
max: ymax,
|
||||
grid: {
|
||||
color: 'rgba(255, 255, 255, 0.1)', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
fontColor: 'rgba(255, 255, 255, 0.7)', // Y-axis label color
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const ctxPing = document.getElementById('chart-ping');
|
||||
const ctxBattery = document.getElementById('chart-battery');
|
||||
export const chartPing = new Chart(ctxPing, getChartConfig(pingPoints, 'rgba(192, 57, 43, 0.7)', 'Controls Ping Time (ms)', 250));
|
||||
export const chartBattery = new Chart(ctxBattery, getChartConfig(batteryPoints, 'rgba(41, 128, 185, 0.7)', 'Battery %', 100));
|
||||
@@ -0,0 +1,209 @@
|
||||
import { getXY } from "./controls.js";
|
||||
import { pingPoints, batteryPoints, chartPing, chartBattery } from "./plots.js";
|
||||
|
||||
export let controlCommandInterval = null;
|
||||
export let latencyInterval = null;
|
||||
export let lastChannelMessageTime = null;
|
||||
|
||||
|
||||
export function offerRtcRequest(sdp, type) {
|
||||
return fetch('/offer', {
|
||||
body: JSON.stringify({sdp: sdp, type: type}),
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
method: 'POST'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function playSoundRequest(sound) {
|
||||
return fetch('/sound', {
|
||||
body: JSON.stringify({sound}),
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
method: 'POST'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function pingHeadRequest() {
|
||||
return fetch('/', {
|
||||
method: 'HEAD'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function createPeerConnection(pc) {
|
||||
var config = {
|
||||
sdpSemantics: 'unified-plan'
|
||||
};
|
||||
|
||||
pc = new RTCPeerConnection(config);
|
||||
|
||||
// connect audio / video
|
||||
pc.addEventListener('track', function(evt) {
|
||||
console.log("Adding Tracks!")
|
||||
if (evt.track.kind == 'video')
|
||||
document.getElementById('video').srcObject = evt.streams[0];
|
||||
else
|
||||
document.getElementById('audio').srcObject = evt.streams[0];
|
||||
});
|
||||
return pc;
|
||||
}
|
||||
|
||||
|
||||
export function negotiate(pc) {
|
||||
return pc.createOffer({offerToReceiveAudio:true, offerToReceiveVideo:true}).then(function(offer) {
|
||||
return pc.setLocalDescription(offer);
|
||||
}).then(function() {
|
||||
return new Promise(function(resolve) {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
resolve();
|
||||
}
|
||||
else {
|
||||
function checkState() {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
pc.removeEventListener('icegatheringstatechange', checkState);
|
||||
resolve();
|
||||
}
|
||||
}
|
||||
pc.addEventListener('icegatheringstatechange', checkState);
|
||||
}
|
||||
});
|
||||
}).then(function() {
|
||||
var offer = pc.localDescription;
|
||||
return offerRtcRequest(offer.sdp, offer.type);
|
||||
}).then(function(response) {
|
||||
console.log(response);
|
||||
return response.json();
|
||||
}).then(function(answer) {
|
||||
return pc.setRemoteDescription(answer);
|
||||
}).catch(function(e) {
|
||||
alert(e);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
function isMobile() {
|
||||
let check = false;
|
||||
(function(a){if(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4))) check = true;})(navigator.userAgent||navigator.vendor||window.opera);
|
||||
return check;
|
||||
};
|
||||
|
||||
|
||||
export const constraints = {
|
||||
audio: {
|
||||
autoGainControl: false,
|
||||
sampleRate: 48000,
|
||||
sampleSize: 16,
|
||||
echoCancellation: true,
|
||||
noiseSuppression: true,
|
||||
channelCount: 1
|
||||
},
|
||||
video: isMobile()
|
||||
};
|
||||
|
||||
|
||||
export function start(pc, dc) {
|
||||
pc = createPeerConnection(pc);
|
||||
|
||||
// add audio track
|
||||
navigator.mediaDevices.enumerateDevices()
|
||||
.then(function(devices) {
|
||||
const hasAudioInput = devices.find((device) => device.kind === "audioinput");
|
||||
var modifiedConstraints = {};
|
||||
modifiedConstraints.video = constraints.video;
|
||||
modifiedConstraints.audio = hasAudioInput ? constraints.audio : false;
|
||||
|
||||
return Promise.resolve(modifiedConstraints);
|
||||
})
|
||||
.then(function(constraints) {
|
||||
if (constraints.audio || constraints.video) {
|
||||
return navigator.mediaDevices.getUserMedia(constraints);
|
||||
} else{
|
||||
return Promise.resolve(null);
|
||||
}
|
||||
})
|
||||
.then(function(stream) {
|
||||
if (stream) {
|
||||
stream.getTracks().forEach(function(track) {
|
||||
pc.addTrack(track, stream);
|
||||
});
|
||||
}
|
||||
|
||||
return negotiate(pc);
|
||||
})
|
||||
.catch(function(err) {
|
||||
alert('Could not acquire media: ' + err);
|
||||
});
|
||||
|
||||
var parameters = {"ordered": true};
|
||||
dc = pc.createDataChannel('data', parameters);
|
||||
dc.onclose = function() {
|
||||
clearInterval(controlCommandInterval);
|
||||
clearInterval(latencyInterval);
|
||||
};
|
||||
|
||||
function sendJoystickOverDataChannel() {
|
||||
const {x, y} = getXY();
|
||||
var message = JSON.stringify({type: "testJoystick", data: {axes: [x, y], buttons: [false]}})
|
||||
dc.send(message);
|
||||
}
|
||||
function checkLatency() {
|
||||
const initialTime = new Date().getTime();
|
||||
pingHeadRequest().then(function() {
|
||||
const currentTime = new Date().getTime();
|
||||
if (Math.abs(currentTime - lastChannelMessageTime) < 1000) {
|
||||
const pingtime = currentTime - initialTime;
|
||||
pingPoints.push({'x': currentTime, 'y': pingtime});
|
||||
if (pingPoints.length > 1000) {
|
||||
pingPoints.shift();
|
||||
}
|
||||
chartPing.update();
|
||||
$("#ping-time").text((pingtime) + "ms");
|
||||
}
|
||||
})
|
||||
}
|
||||
dc.onopen = function() {
|
||||
controlCommandInterval = setInterval(sendJoystickOverDataChannel, 50);
|
||||
latencyInterval = setInterval(checkLatency, 1000);
|
||||
sendJoystickOverDataChannel();
|
||||
};
|
||||
|
||||
const textDecoder = new TextDecoder();
|
||||
var carStaterIndex = 0;
|
||||
dc.onmessage = function(evt) {
|
||||
const text = textDecoder.decode(evt.data);
|
||||
const msg = JSON.parse(text);
|
||||
if (carStaterIndex % 100 == 0 && msg.type === 'carState') {
|
||||
const batteryLevel = Math.round(msg.data.fuelGauge * 100);
|
||||
$("#battery").text(batteryLevel + "%");
|
||||
batteryPoints.push({'x': new Date().getTime(), 'y': batteryLevel});
|
||||
if (batteryPoints.length > 1000) {
|
||||
batteryPoints.shift();
|
||||
}
|
||||
chartBattery.update();
|
||||
}
|
||||
carStaterIndex += 1;
|
||||
lastChannelMessageTime = new Date().getTime();
|
||||
$(".pre-blob").addClass('blob');
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
export function stop(pc, dc) {
|
||||
if (dc) {
|
||||
dc.close();
|
||||
}
|
||||
if (pc.getTransceivers) {
|
||||
pc.getTransceivers().forEach(function(transceiver) {
|
||||
if (transceiver.stop) {
|
||||
transceiver.stop();
|
||||
}
|
||||
});
|
||||
}
|
||||
pc.getSenders().forEach(function(sender) {
|
||||
sender.track.stop();
|
||||
});
|
||||
setTimeout(function() {
|
||||
pc.close();
|
||||
}, 500);
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
body {
|
||||
background: #333 !important;
|
||||
color: #fff !important;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: start;
|
||||
}
|
||||
|
||||
p {
|
||||
margin: 0px !important;
|
||||
}
|
||||
|
||||
i {
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
.small {
|
||||
font-size: 1em !important
|
||||
}
|
||||
|
||||
.jumbo {
|
||||
font-size: 8rem;
|
||||
}
|
||||
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.small {
|
||||
font-size: 0.5em !important
|
||||
}
|
||||
.jumbo {
|
||||
display: none;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
#main {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-content: center;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 30px;
|
||||
width: 100%;
|
||||
max-width: 1200px;
|
||||
}
|
||||
|
||||
video {
|
||||
width: 95%;
|
||||
}
|
||||
|
||||
.pre-blob {
|
||||
display: flex;
|
||||
background: #333;
|
||||
border-radius: 50%;
|
||||
margin: 10px;
|
||||
height: 45px;
|
||||
width: 45px;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.blob {
|
||||
background: rgba(231, 76, 60,1.0);
|
||||
box-shadow: 0 0 0 0 rgba(231, 76, 60,1.0);
|
||||
animation: pulse 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0% {
|
||||
box-shadow: 0 0 0 0px rgba(192, 57, 43, 1);
|
||||
}
|
||||
100% {
|
||||
box-shadow: 0 0 0 20px rgba(192, 57, 43, 0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
.icon-sup-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: center;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
padding: 5px;
|
||||
margin: 5px 0px 5px 0px;
|
||||
}
|
||||
|
||||
.icon-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
#icon-panel {
|
||||
display: flex;
|
||||
width: 100%;
|
||||
justify-content: space-between;
|
||||
margin-top: 5px;
|
||||
}
|
||||
|
||||
.icon-sub-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
}
|
||||
|
||||
.keys, #key-val {
|
||||
background: #333;
|
||||
padding: 2rem;
|
||||
margin: 5px;
|
||||
color: #fff;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
border-radius: 10px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#key-val {
|
||||
pointer-events: none;
|
||||
background: #fff;
|
||||
color: #333;
|
||||
line-height: 1;
|
||||
font-size: 20px;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.wasd-row {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
#wasd {
|
||||
margin: 5px 0px 5px 0px;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 20px;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: stretch;
|
||||
|
||||
user-select: none;
|
||||
-webkit-touch-callout: none;
|
||||
-webkit-user-select: none;
|
||||
-khtml-user-select: none;
|
||||
-moz-user-select: none;
|
||||
-ms-user-select: none;
|
||||
touch-action: manipulation;
|
||||
}
|
||||
|
||||
.panel {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
margin: 5px 0px 5px 0px !important;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
#ping-time, #battery {
|
||||
font-size: 25px;
|
||||
}
|
||||
|
||||
#stop {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.plan-form {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.details {
|
||||
display: flex;
|
||||
padding: 0px 10px 0px 10px;
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 8.1 KiB |
@@ -0,0 +1,127 @@
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import ssl
|
||||
import subprocess
|
||||
|
||||
import pyaudio
|
||||
import wave
|
||||
from aiohttp import web
|
||||
from aiohttp import ClientSession
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.system.webrtc.webrtcd import StreamRequestBody
|
||||
from openpilot.common.params import Params
|
||||
|
||||
logger = logging.getLogger("bodyteleop")
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
|
||||
TELEOPDIR = f"{BASEDIR}/tools/bodyteleop"
|
||||
WEBRTCD_HOST, WEBRTCD_PORT = "localhost", 5001
|
||||
|
||||
|
||||
## UTILS
|
||||
async def play_sound(sound: str):
|
||||
SOUNDS = {
|
||||
"engage": "selfdrive/assets/sounds/engage.wav",
|
||||
"disengage": "selfdrive/assets/sounds/disengage.wav",
|
||||
"error": "selfdrive/assets/sounds/warning_immediate.wav",
|
||||
}
|
||||
assert sound in SOUNDS
|
||||
|
||||
chunk = 5120
|
||||
with wave.open(os.path.join(BASEDIR, SOUNDS[sound]), "rb") as wf:
|
||||
def callback(in_data, frame_count, time_info, status):
|
||||
data = wf.readframes(frame_count)
|
||||
return data, pyaudio.paContinue
|
||||
|
||||
p = pyaudio.PyAudio()
|
||||
stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
|
||||
channels=wf.getnchannels(),
|
||||
rate=wf.getframerate(),
|
||||
output=True,
|
||||
frames_per_buffer=chunk,
|
||||
stream_callback=callback)
|
||||
stream.start_stream()
|
||||
while stream.is_active():
|
||||
await asyncio.sleep(0)
|
||||
stream.stop_stream()
|
||||
stream.close()
|
||||
p.terminate()
|
||||
|
||||
## SSL
|
||||
def create_ssl_cert(cert_path: str, key_path: str):
|
||||
try:
|
||||
proc = subprocess.run(f'openssl req -x509 -newkey rsa:4096 -nodes -out {cert_path} -keyout {key_path} \
|
||||
-days 365 -subj "/C=US/ST=California/O=commaai/OU=comma body"',
|
||||
capture_output=True, shell=True)
|
||||
proc.check_returncode()
|
||||
except subprocess.CalledProcessError as ex:
|
||||
raise ValueError(f"Error creating SSL certificate:\n[stdout]\n{proc.stdout.decode()}\n[stderr]\n{proc.stderr.decode()}") from ex
|
||||
|
||||
|
||||
def create_ssl_context():
|
||||
cert_path = os.path.join(TELEOPDIR, "cert.pem")
|
||||
key_path = os.path.join(TELEOPDIR, "key.pem")
|
||||
if not os.path.exists(cert_path) or not os.path.exists(key_path):
|
||||
logger.info("Creating certificate...")
|
||||
create_ssl_cert(cert_path, key_path)
|
||||
else:
|
||||
logger.info("Certificate exists!")
|
||||
ssl_context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_SERVER)
|
||||
ssl_context.load_cert_chain(cert_path, key_path)
|
||||
|
||||
return ssl_context
|
||||
|
||||
## ENDPOINTS
|
||||
async def index(request: 'web.Request'):
|
||||
with open(os.path.join(TELEOPDIR, "static", "index.html")) as f:
|
||||
content = f.read()
|
||||
return web.Response(content_type="text/html", text=content)
|
||||
|
||||
|
||||
async def ping(request: 'web.Request'):
|
||||
return web.Response(text="pong")
|
||||
|
||||
|
||||
async def sound(request: 'web.Request'):
|
||||
params = await request.json()
|
||||
sound_to_play = params["sound"]
|
||||
|
||||
await play_sound(sound_to_play)
|
||||
return web.json_response({"status": "ok"})
|
||||
|
||||
|
||||
async def offer(request: 'web.Request'):
|
||||
params = await request.json()
|
||||
body = StreamRequestBody(params["sdp"], ["driver"], ["testJoystick"], ["carState"])
|
||||
body_json = json.dumps(dataclasses.asdict(body))
|
||||
|
||||
logger.info("Sending offer to webrtcd...")
|
||||
webrtcd_url = f"http://{WEBRTCD_HOST}:{WEBRTCD_PORT}/stream"
|
||||
async with ClientSession() as session, session.post(webrtcd_url, data=body_json) as resp:
|
||||
assert resp.status == 200
|
||||
answer = await resp.json()
|
||||
return web.json_response(answer)
|
||||
|
||||
|
||||
def main():
|
||||
# Enable joystick debug mode
|
||||
Params().put_bool("JoystickDebugMode", True)
|
||||
|
||||
# App needs to be HTTPS for microphone and audio autoplay to work on the browser
|
||||
ssl_context = create_ssl_context()
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_get("/", index)
|
||||
app.router.add_get("/ping", ping, allow_head=True)
|
||||
app.router.add_post("/offer", offer)
|
||||
app.router.add_post("/sound", sound)
|
||||
app.router.add_static('/static', os.path.join(TELEOPDIR, 'static'))
|
||||
web.run_app(app, access_log=None, host="0.0.0.0", port=5000, ssl_context=ssl_context)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
moc_*
|
||||
*.moc
|
||||
|
||||
cabana
|
||||
dbc/car_fingerprint_to_dbc.json
|
||||
tests/test_cabana
|
||||
@@ -0,0 +1,98 @@
|
||||
# Cabana
|
||||
|
||||
Cabana is a tool developed to view raw CAN data. One use for this is creating and editing [CAN Dictionaries](http://socialledge.com/sjsu/index.php/DBC_Format) (DBC files), and the tool provides direct integration with [commaai/opendbc](https://github.com/commaai/opendbc) (a collection of DBC files), allowing you to load the DBC files direct from source, and save to your fork. In addition, you can load routes from [comma connect](https://connect.comma.ai).
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
```bash
|
||||
$ ./cabana -h
|
||||
Usage: ./cabana [options] route
|
||||
|
||||
Options:
|
||||
-h, --help Displays help on commandline options.
|
||||
--help-all Displays help including Qt specific options.
|
||||
--demo use a demo route instead of providing your own
|
||||
--qcam load qcamera
|
||||
--ecam load wide road camera
|
||||
--msgq read can messages from msgq
|
||||
--panda read can messages from panda
|
||||
--panda-serial <panda-serial> read can messages from panda with given serial
|
||||
--socketcan <socketcan> read can messages from given SocketCAN device
|
||||
--zmq <ip-address> read can messages from zmq at the specified ip-address
|
||||
messages
|
||||
--data_dir <data_dir> local directory with routes
|
||||
--no-vipc do not output video
|
||||
--dbc <dbc> dbc file to open
|
||||
|
||||
Arguments:
|
||||
route the drive to replay. find your drives at
|
||||
connect.comma.ai
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Running Cabana in Demo Mode
|
||||
To run Cabana using a built-in demo route, use the following command:
|
||||
|
||||
```shell
|
||||
cabana --demo
|
||||
```
|
||||
|
||||
### Loading a Specific Route
|
||||
|
||||
To load a specific route for replay, provide the route as an argument:
|
||||
|
||||
```shell
|
||||
cabana "a2a0ccea32023010|2023-07-27--13-01-19"
|
||||
```
|
||||
|
||||
Replace "0ccea32023010|2023-07-27--13-01-19" with your desired route identifier.
|
||||
|
||||
|
||||
### Running Cabana with multiple cameras
|
||||
To run Cabana with multiple cameras, use the following command:
|
||||
|
||||
```shell
|
||||
cabana "a2a0ccea32023010|2023-07-27--13-01-19" --dcam --ecam
|
||||
```
|
||||
|
||||
### Streaming CAN Messages from a comma Device
|
||||
|
||||
[SSH into your device](https://github.com/commaai/openpilot/wiki/SSH) and start the bridge with the following command:
|
||||
|
||||
```shell
|
||||
cd /data/openpilot/cereal/messaging/
|
||||
./bridge &
|
||||
```
|
||||
|
||||
Then Run Cabana with the device's IP address:
|
||||
|
||||
```shell
|
||||
cabana --zmq <ipaddress>
|
||||
```
|
||||
|
||||
Replace <ipaddress> with your comma device's IP address.
|
||||
|
||||
While streaming from the device, Cabana will log the CAN messages to a local directory. By default, this directory is ~/cabana_live_stream/. You can change the log directory in Cabana by navigating to menu -> tools -> settings.
|
||||
|
||||
After disconnecting from the device, you can replay the logged CAN messages from the stream selector dialog -> browse local route.
|
||||
|
||||
### Streaming CAN Messages from Panda
|
||||
|
||||
To read CAN messages from a connected Panda, use the following command:
|
||||
|
||||
```shell
|
||||
cabana --panda
|
||||
```
|
||||
|
||||
### Using the Stream Selector Dialog
|
||||
|
||||
If you run Cabana without any arguments, a stream selector dialog will pop up, allowing you to choose the stream.
|
||||
|
||||
```shell
|
||||
cabana
|
||||
```
|
||||
|
||||
## Additional Information
|
||||
|
||||
For more information, see the [openpilot wiki](https://github.com/commaai/openpilot/wiki/Cabana)
|
||||
@@ -0,0 +1,42 @@
|
||||
Import('qt_env', 'arch', 'common', 'messaging', 'visionipc', 'replay_lib', 'cereal', 'widgets')
|
||||
|
||||
base_frameworks = qt_env['FRAMEWORKS']
|
||||
base_libs = [common, messaging, cereal, visionipc, 'qt_util', 'm', 'ssl', 'crypto', 'pthread'] + qt_env["LIBS"]
|
||||
|
||||
if arch == "Darwin":
|
||||
base_frameworks.append('OpenCL')
|
||||
base_frameworks.append('QtCharts')
|
||||
base_frameworks.append('QtSerialBus')
|
||||
else:
|
||||
base_libs.append('OpenCL')
|
||||
base_libs.append('Qt5Charts')
|
||||
base_libs.append('Qt5SerialBus')
|
||||
|
||||
qt_libs = ['qt_util'] + base_libs
|
||||
|
||||
cabana_env = qt_env.Clone()
|
||||
cabana_libs = [widgets, cereal, messaging, visionipc, replay_lib, 'panda', 'avutil', 'avcodec', 'avformat', 'bz2', 'zstd', 'curl', 'yuv', 'usb-1.0'] + qt_libs
|
||||
opendbc_path = '-DOPENDBC_FILE_PATH=\'"%s"\'' % (cabana_env.Dir("../../opendbc/dbc").abspath)
|
||||
cabana_env['CXXFLAGS'] += [opendbc_path]
|
||||
|
||||
# build assets
|
||||
assets = "assets/assets.cc"
|
||||
assets_src = "assets/assets.qrc"
|
||||
cabana_env.Command(assets, assets_src, f"rcc $SOURCES -o $TARGET")
|
||||
cabana_env.Depends(assets, Glob('/assets/*', exclude=[assets, assets_src, "assets/assets.o"]))
|
||||
|
||||
cabana_lib = cabana_env.Library("cabana_lib", ['mainwin.cc', 'streams/socketcanstream.cc', 'streams/pandastream.cc', 'streams/devicestream.cc', 'streams/livestream.cc', 'streams/abstractstream.cc', 'streams/replaystream.cc', 'binaryview.cc', 'historylog.cc', 'videowidget.cc', 'signalview.cc',
|
||||
'streams/routes.cc', 'dbc/dbc.cc', 'dbc/dbcfile.cc', 'dbc/dbcmanager.cc',
|
||||
'utils/export.cc', 'utils/util.cc',
|
||||
'chart/chartswidget.cc', 'chart/chart.cc', 'chart/signalselector.cc', 'chart/tiplabel.cc', 'chart/sparkline.cc',
|
||||
'commands.cc', 'messageswidget.cc', 'streamselector.cc', 'settings.cc', 'detailwidget.cc', 'tools/findsimilarbits.cc', 'tools/findsignal.cc'], LIBS=cabana_libs, FRAMEWORKS=base_frameworks)
|
||||
cabana_env.Program('cabana', ['cabana.cc', cabana_lib, assets], LIBS=cabana_libs, FRAMEWORKS=base_frameworks)
|
||||
|
||||
if GetOption('extras'):
|
||||
cabana_env.Program('tests/test_cabana', ['tests/test_runner.cc', 'tests/test_cabana.cc', cabana_lib], LIBS=[cabana_libs])
|
||||
|
||||
output_json_file = 'tools/cabana/dbc/car_fingerprint_to_dbc.json'
|
||||
generate_dbc = cabana_env.Command('#' + output_json_file,
|
||||
['dbc/generate_dbc_json.py'],
|
||||
"python3 tools/cabana/dbc/generate_dbc_json.py --out " + output_json_file)
|
||||
cabana_env.Depends(generate_dbc, ["#common", "#selfdrive/pandad", '#opendbc_repo', "#cereal", "#msgq_repo"])
|
||||
@@ -0,0 +1 @@
|
||||
*.cc
|
||||
@@ -0,0 +1,6 @@
|
||||
<!DOCTYPE RCC><RCC version="1.0">
|
||||
<qresource>
|
||||
<file alias="bootstrap-icons.svg">../../../third_party/bootstrap/bootstrap-icons.svg</file>
|
||||
<file>cabana-icon.png</file>
|
||||
</qresource>
|
||||
</RCC>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
@@ -0,0 +1,104 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
#include <QList>
|
||||
#include <QSet>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class BinaryItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
BinaryItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool hasSignal(const QModelIndex &index, int dx, int dy, const cabana::Signal *sig) const;
|
||||
void drawSignalCell(QPainter* painter, const QStyleOptionViewItem &option, const QModelIndex &index, const cabana::Signal *sig) const;
|
||||
|
||||
QFont small_font, hex_font;
|
||||
std::array<QStaticText, 256> hex_text_table;
|
||||
std::array<QStaticText, 2> bin_text_table;
|
||||
};
|
||||
|
||||
class BinaryViewModel : public QAbstractTableModel {
|
||||
public:
|
||||
BinaryViewModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void refresh();
|
||||
void updateState();
|
||||
void updateItem(int row, int col, uint8_t val, const QColor &color);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return row_count; }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return column_count; }
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override {
|
||||
return createIndex(row, column, (void *)&items[row * column_count + column]);
|
||||
}
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override {
|
||||
return (index.column() == column_count - 1) ? Qt::ItemIsEnabled : Qt::ItemIsEnabled | Qt::ItemIsSelectable;
|
||||
}
|
||||
const std::vector<std::array<uint32_t, 8>> &getBitFlipChanges(size_t msg_size);
|
||||
|
||||
struct BitFlipTracker {
|
||||
std::optional<std::pair<double, double>> time_range;
|
||||
std::vector<std::array<uint32_t, 8>> flip_counts;
|
||||
} bit_flip_tracker;
|
||||
|
||||
struct Item {
|
||||
QColor bg_color = QColor(102, 86, 169, 255);
|
||||
bool is_msb = false;
|
||||
bool is_lsb = false;
|
||||
uint8_t val;
|
||||
QList<const cabana::Signal *> sigs;
|
||||
bool valid = false;
|
||||
};
|
||||
std::vector<Item> items;
|
||||
bool heatmap_live_mode = true;
|
||||
MessageId msg_id;
|
||||
int row_count = 0;
|
||||
const int column_count = 9;
|
||||
};
|
||||
|
||||
class BinaryView : public QTableView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
BinaryView(QWidget *parent = nullptr);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void highlight(const cabana::Signal *sig);
|
||||
QSet<const cabana::Signal*> getOverlappingSignals() const;
|
||||
void updateState() { model->updateState(); }
|
||||
void paintEvent(QPaintEvent *event) override {
|
||||
is_message_active = can->isMessageActive(model->msg_id);
|
||||
QTableView::paintEvent(event);
|
||||
}
|
||||
QSize minimumSizeHint() const override;
|
||||
void setHeatmapLiveMode(bool live) { model->heatmap_live_mode = live; updateState(); }
|
||||
|
||||
signals:
|
||||
void signalClicked(const cabana::Signal *sig);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void editSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void addShortcuts();
|
||||
void refresh();
|
||||
std::tuple<int, int, bool> getSelection(QModelIndex index);
|
||||
void setSelection(const QRect &rect, QItemSelectionModel::SelectionFlags flags) override;
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void leaveEvent(QEvent *event) override;
|
||||
void highlightPosition(const QPoint &pt);
|
||||
|
||||
QModelIndex anchor_index;
|
||||
BinaryViewModel *model;
|
||||
BinaryItemDelegate *delegate;
|
||||
bool is_message_active = false;
|
||||
const cabana::Signal *resize_sig = nullptr;
|
||||
const cabana::Signal *hovered_sig = nullptr;
|
||||
friend class BinaryItemDelegate;
|
||||
};
|
||||
@@ -0,0 +1,123 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QMenu>
|
||||
#include <QGraphicsPixmapItem>
|
||||
#include <QGraphicsProxyWidget>
|
||||
#include <QtCharts/QChartView>
|
||||
#include <QtCharts/QLegendMarker>
|
||||
#include <QtCharts/QLineSeries>
|
||||
#include <QtCharts/QScatterSeries>
|
||||
#include <QtCharts/QValueAxis>
|
||||
using namespace QtCharts;
|
||||
|
||||
#include "tools/cabana/chart/tiplabel.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
enum class SeriesType {
|
||||
Line = 0,
|
||||
StepLine,
|
||||
Scatter
|
||||
};
|
||||
|
||||
class ChartsWidget;
|
||||
class ChartView : public QChartView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartView(const std::pair<double, double> &x_range, ChartsWidget *parent = nullptr);
|
||||
void addSignal(const MessageId &msg_id, const cabana::Signal *sig);
|
||||
bool hasSignal(const MessageId &msg_id, const cabana::Signal *sig) const;
|
||||
void updateSeries(const cabana::Signal *sig = nullptr, const MessageEventsMap *msg_new_events = nullptr);
|
||||
void updatePlot(double cur, double min, double max);
|
||||
void setSeriesType(SeriesType type);
|
||||
void updatePlotArea(int left, bool force = false);
|
||||
void showTip(double sec);
|
||||
void hideTip();
|
||||
void startAnimation();
|
||||
double secondsAtPoint(const QPointF &pt) const { return chart()->mapToValue(pt).x(); }
|
||||
|
||||
struct SigItem {
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QXYSeries *series = nullptr;
|
||||
std::vector<QPointF> vals;
|
||||
std::vector<QPointF> step_vals;
|
||||
QPointF track_pt{};
|
||||
SegmentTree segment_tree;
|
||||
double min = 0;
|
||||
double max = 0;
|
||||
};
|
||||
|
||||
signals:
|
||||
void axisYLabelWidthChanged(int w);
|
||||
|
||||
private slots:
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void manageSignals();
|
||||
void handleMarkerClicked();
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id) { removeIf([=](auto &s) { return s.msg_id.address == id.address && !dbc()->msg(id); }); }
|
||||
void signalRemoved(const cabana::Signal *sig) { removeIf([=](auto &s) { return s.sig == sig; }); }
|
||||
|
||||
private:
|
||||
void appendCanEvents(const cabana::Signal *sig, const std::vector<const CanEvent *> &events,
|
||||
std::vector<QPointF> &vals, std::vector<QPointF> &step_vals);
|
||||
void createToolButtons();
|
||||
void addSeries(QXYSeries *series);
|
||||
void contextMenuEvent(QContextMenuEvent *event) override;
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *ev) override;
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator(false); }
|
||||
void dragMoveEvent(QDragMoveEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void resizeEvent(QResizeEvent *event) override;
|
||||
QSize sizeHint() const override;
|
||||
void updateAxisY();
|
||||
void updateTitle();
|
||||
void resetChartCache();
|
||||
void setTheme(QChart::ChartTheme theme);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void drawForeground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawBackground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawDropIndicator(bool draw) { if (std::exchange(can_drop, draw) != can_drop) viewport()->update(); }
|
||||
void drawSignalValue(QPainter *painter);
|
||||
void drawTimeline(QPainter *painter);
|
||||
void drawRubberBandTimeRange(QPainter *painter);
|
||||
std::tuple<double, double, int> getNiceAxisNumbers(qreal min, qreal max, int tick_count);
|
||||
qreal niceNumber(qreal x, bool ceiling);
|
||||
QXYSeries *createSeries(SeriesType type, QColor color);
|
||||
void setSeriesColor(QXYSeries *, QColor color);
|
||||
void updateSeriesPoints();
|
||||
void removeIf(std::function<bool(const SigItem &)> predicate);
|
||||
inline void clearTrackPoints() { for (auto &s : sigs) s.track_pt = {}; }
|
||||
|
||||
int y_label_width = 0;
|
||||
int align_to = 0;
|
||||
QValueAxis *axis_x;
|
||||
QValueAxis *axis_y;
|
||||
QMenu *menu;
|
||||
QAction *split_chart_act;
|
||||
QAction *close_act;
|
||||
QGraphicsPixmapItem *move_icon;
|
||||
QGraphicsProxyWidget *close_btn_proxy;
|
||||
QGraphicsProxyWidget *manage_btn_proxy;
|
||||
TipLabel *tip_label;
|
||||
std::vector<SigItem> sigs;
|
||||
double cur_sec = 0;
|
||||
SeriesType series_type = SeriesType::Line;
|
||||
bool is_scrubbing = false;
|
||||
bool resume_after_scrub = false;
|
||||
QPixmap chart_pixmap;
|
||||
bool can_drop = false;
|
||||
double tooltip_x = -1;
|
||||
QFont signal_value_font;
|
||||
ChartsWidget *charts_widget;
|
||||
friend class ChartsWidget;
|
||||
};
|
||||
@@ -0,0 +1,129 @@
|
||||
#pragma once
|
||||
|
||||
#include <unordered_map>
|
||||
#include <utility>
|
||||
|
||||
#include <QGridLayout>
|
||||
#include <QLabel>
|
||||
#include <QScrollArea>
|
||||
#include <QTimer>
|
||||
#include <QToolBar>
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/chart/signalselector.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
const int CHART_MIN_WIDTH = 300;
|
||||
const QString CHART_MIME_TYPE = "application/x-cabanachartview";
|
||||
|
||||
class ChartView;
|
||||
class ChartsWidget;
|
||||
|
||||
class ChartsContainer : public QWidget {
|
||||
public:
|
||||
ChartsContainer(ChartsWidget *parent);
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator({}); }
|
||||
void drawDropIndicator(const QPoint &pt) { drop_indictor_pos = pt; update(); }
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
ChartView *getDropAfter(const QPoint &pos) const;
|
||||
|
||||
QGridLayout *charts_layout;
|
||||
ChartsWidget *charts_widget;
|
||||
QPoint drop_indictor_pos;
|
||||
};
|
||||
|
||||
class ChartsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartsWidget(QWidget *parent = nullptr);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
inline bool hasSignal(const MessageId &id, const cabana::Signal *sig) { return findChart(id, sig) != nullptr; }
|
||||
|
||||
public slots:
|
||||
void setColumnCount(int n);
|
||||
void removeAll();
|
||||
void timeRangeChanged(const std::optional<std::pair<double, double>> &time_range);
|
||||
void setIsDocked(bool dock);
|
||||
|
||||
signals:
|
||||
void toggleChartsDocking();
|
||||
void seriesChanged();
|
||||
void showTip(double seconds);
|
||||
|
||||
private:
|
||||
QSize minimumSizeHint() const override;
|
||||
bool event(QEvent *event) override;
|
||||
void alignCharts();
|
||||
void newChart();
|
||||
ChartView *createChart(int pos = 0);
|
||||
void removeChart(ChartView *chart);
|
||||
void splitChart(ChartView *chart);
|
||||
QRect chartVisibleRect(ChartView *chart);
|
||||
void eventsMerged(const MessageEventsMap &new_events);
|
||||
void updateState();
|
||||
void zoomReset();
|
||||
void startAutoScroll();
|
||||
void stopAutoScroll();
|
||||
void doAutoScroll();
|
||||
void updateToolBar();
|
||||
void updateTabBar();
|
||||
void setMaxChartRange(int value);
|
||||
void updateLayout(bool force = false);
|
||||
void settingChanged();
|
||||
void showValueTip(double sec);
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
void newTab();
|
||||
void removeTab(int index);
|
||||
inline QList<ChartView *> ¤tCharts() { return tab_charts[tabbar->tabData(tabbar->currentIndex()).toInt()]; }
|
||||
ChartView *findChart(const MessageId &id, const cabana::Signal *sig);
|
||||
|
||||
QLabel *title_label;
|
||||
QLabel *range_lb;
|
||||
LogSlider *range_slider;
|
||||
QAction *range_lb_action;
|
||||
QAction *range_slider_action;
|
||||
bool is_docked = true;
|
||||
ToolButton *dock_btn;
|
||||
|
||||
QToolBar *toolbar;
|
||||
QAction *undo_zoom_action;
|
||||
QAction *redo_zoom_action;
|
||||
QAction *reset_zoom_action;
|
||||
ToolButton *reset_zoom_btn;
|
||||
QUndoStack *zoom_undo_stack;
|
||||
|
||||
ToolButton *remove_all_btn;
|
||||
QList<ChartView *> charts;
|
||||
std::unordered_map<int, QList<ChartView *>> tab_charts;
|
||||
TabBar *tabbar;
|
||||
ChartsContainer *charts_container;
|
||||
QScrollArea *charts_scroll;
|
||||
uint32_t max_chart_range = 0;
|
||||
std::pair<double, double> display_range;
|
||||
QAction *columns_action;
|
||||
int column_count = 1;
|
||||
int current_column_count = 0;
|
||||
int auto_scroll_count = 0;
|
||||
QTimer *auto_scroll_timer;
|
||||
QTimer *align_timer;
|
||||
int current_theme = 0;
|
||||
bool value_tip_visible_ = false;
|
||||
friend class ChartView;
|
||||
friend class ChartsContainer;
|
||||
};
|
||||
|
||||
class ZoomCommand : public QUndoCommand {
|
||||
public:
|
||||
ZoomCommand(std::pair<double, double> range) : range(range), QUndoCommand() {
|
||||
prev_range = can->timeRange();
|
||||
setText(QObject::tr("Zoom to %1-%2").arg(range.first, 0, 'f', 2).arg(range.second, 0, 'f', 2));
|
||||
}
|
||||
void undo() override { can->setTimeRange(prev_range); }
|
||||
void redo() override { can->setTimeRange(range); }
|
||||
std::optional<std::pair<double, double>> prev_range, range;
|
||||
};
|
||||
@@ -0,0 +1,30 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QListWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class SignalSelector : public QDialog {
|
||||
public:
|
||||
struct ListItem : public QListWidgetItem {
|
||||
ListItem(const MessageId &msg_id, const cabana::Signal *sig, QListWidget *parent) : msg_id(msg_id), sig(sig), QListWidgetItem(parent) {}
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig;
|
||||
};
|
||||
|
||||
SignalSelector(QString title, QWidget *parent);
|
||||
QList<ListItem *> seletedItems();
|
||||
inline void addSelected(const MessageId &id, const cabana::Signal *sig) { addItemToList(selected_list, id, sig, true); }
|
||||
|
||||
private:
|
||||
void updateAvailableList(int index);
|
||||
void addItemToList(QListWidget *parent, const MessageId id, const cabana::Signal *sig, bool show_msg_name = false);
|
||||
void add(QListWidgetItem *item);
|
||||
void remove(QListWidgetItem *item);
|
||||
|
||||
QComboBox *msgs_combo;
|
||||
QListWidget *available_list;
|
||||
QListWidget *selected_list;
|
||||
};
|
||||
@@ -0,0 +1,25 @@
|
||||
#pragma once
|
||||
|
||||
#include <QPixmap>
|
||||
#include <QPointF>
|
||||
#include <vector>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class Sparkline {
|
||||
public:
|
||||
void update(const MessageId &msg_id, const cabana::Signal *sig, double last_msg_ts, int range, QSize size);
|
||||
inline double freq() const { return freq_; }
|
||||
bool isEmpty() const { return pixmap.isNull(); }
|
||||
|
||||
QPixmap pixmap;
|
||||
double min_val = 0;
|
||||
double max_val = 0;
|
||||
|
||||
private:
|
||||
void render(const QColor &color, int range, QSize size);
|
||||
|
||||
std::vector<QPointF> points;
|
||||
double freq_ = 0;
|
||||
};
|
||||
@@ -0,0 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#include <QLabel>
|
||||
|
||||
class TipLabel : public QLabel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
TipLabel(QWidget *parent = nullptr);
|
||||
void showText(const QPoint &pt, const QString &sec, QWidget *w, const QRect &rect);
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
};
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
#include <utility>
|
||||
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class EditMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
EditMsgCommand(const MessageId &id, const QString &name, int size, const QString &node,
|
||||
const QString &comment, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QString old_name, new_name, old_comment, new_comment, old_node, new_node;
|
||||
int old_size = 0, new_size = 0;
|
||||
};
|
||||
|
||||
class RemoveMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveMsgCommand(const MessageId &id, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
cabana::Msg message;
|
||||
};
|
||||
|
||||
class AddSigCommand : public QUndoCommand {
|
||||
public:
|
||||
AddSigCommand(const MessageId &id, const cabana::Signal &sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
bool msg_created = false;
|
||||
cabana::Signal signal = {};
|
||||
};
|
||||
|
||||
class RemoveSigCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveSigCommand(const MessageId &id, const cabana::Signal *sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<cabana::Signal> sigs;
|
||||
};
|
||||
|
||||
class EditSignalCommand : public QUndoCommand {
|
||||
public:
|
||||
EditSignalCommand(const MessageId &id, const cabana::Signal *sig, const cabana::Signal &new_sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<std::pair<cabana::Signal, cabana::Signal>> sigs; // QList<{old_sig, new_sig}>
|
||||
};
|
||||
|
||||
namespace UndoStack {
|
||||
QUndoStack *instance();
|
||||
inline void push(QUndoCommand *cmd) { instance()->push(cmd); }
|
||||
};
|
||||
@@ -0,0 +1,120 @@
|
||||
#pragma once
|
||||
|
||||
#include <limits>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QMetaType>
|
||||
#include <QString>
|
||||
|
||||
const QString UNTITLED = "untitled";
|
||||
const QString DEFAULT_NODE_NAME = "XXX";
|
||||
constexpr int CAN_MAX_DATA_BYTES = 64;
|
||||
|
||||
struct MessageId {
|
||||
uint8_t source = 0;
|
||||
uint32_t address = 0;
|
||||
|
||||
QString toString() const {
|
||||
return QString("%1:%2").arg(source).arg(QString::number(address, 16).toUpper());
|
||||
}
|
||||
|
||||
bool operator==(const MessageId &other) const {
|
||||
return source == other.source && address == other.address;
|
||||
}
|
||||
|
||||
bool operator!=(const MessageId &other) const {
|
||||
return !(*this == other);
|
||||
}
|
||||
|
||||
bool operator<(const MessageId &other) const {
|
||||
return std::tie(source, address) < std::tie(other.source, other.address);
|
||||
}
|
||||
|
||||
bool operator>(const MessageId &other) const {
|
||||
return std::tie(source, address) > std::tie(other.source, other.address);
|
||||
}
|
||||
};
|
||||
|
||||
uint qHash(const MessageId &item);
|
||||
Q_DECLARE_METATYPE(MessageId);
|
||||
|
||||
template <>
|
||||
struct std::hash<MessageId> {
|
||||
std::size_t operator()(const MessageId &k) const noexcept { return qHash(k); }
|
||||
};
|
||||
|
||||
typedef std::vector<std::pair<double, QString>> ValueDescription;
|
||||
|
||||
namespace cabana {
|
||||
|
||||
class Signal {
|
||||
public:
|
||||
Signal() = default;
|
||||
Signal(const Signal &other) = default;
|
||||
void update();
|
||||
bool getValue(const uint8_t *data, size_t data_size, double *val) const;
|
||||
QString formatValue(double value, bool with_unit = true) const;
|
||||
bool operator==(const cabana::Signal &other) const;
|
||||
inline bool operator!=(const cabana::Signal &other) const { return !(*this == other); }
|
||||
|
||||
enum class Type {
|
||||
Normal = 0,
|
||||
Multiplexed,
|
||||
Multiplexor
|
||||
};
|
||||
|
||||
Type type = Type::Normal;
|
||||
QString name;
|
||||
int start_bit, msb, lsb, size;
|
||||
double factor = 1.0;
|
||||
double offset = 0;
|
||||
bool is_signed;
|
||||
bool is_little_endian;
|
||||
double min, max;
|
||||
QString unit;
|
||||
QString comment;
|
||||
QString receiver_name;
|
||||
ValueDescription val_desc;
|
||||
int precision = 0;
|
||||
QColor color;
|
||||
|
||||
// Multiplexed
|
||||
int multiplex_value = 0;
|
||||
Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
class Msg {
|
||||
public:
|
||||
Msg() = default;
|
||||
Msg(const Msg &other) { *this = other; }
|
||||
~Msg();
|
||||
cabana::Signal *addSignal(const cabana::Signal &sig);
|
||||
cabana::Signal *updateSignal(const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const QString &sig_name);
|
||||
Msg &operator=(const Msg &other);
|
||||
int indexOf(const cabana::Signal *sig) const;
|
||||
cabana::Signal *sig(const QString &sig_name) const;
|
||||
QString newSignalName();
|
||||
void update();
|
||||
inline const std::vector<cabana::Signal *> &getSignals() const { return sigs; }
|
||||
|
||||
uint32_t address;
|
||||
QString name;
|
||||
uint32_t size;
|
||||
QString comment;
|
||||
QString transmitter;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
|
||||
std::vector<uint8_t> mask;
|
||||
cabana::Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
} // namespace cabana
|
||||
|
||||
// Helper functions
|
||||
double get_raw_value(const uint8_t *data, size_t data_size, const cabana::Signal &sig);
|
||||
void updateMsbLsb(cabana::Signal &s);
|
||||
inline int flipBitPos(int start_bit) { return 8 * (start_bit / 8) + 7 - start_bit % 8; }
|
||||
inline QString doubleToString(double value) { return QString::number(value, 'g', std::numeric_limits<double>::digits10); }
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
#include <map>
|
||||
#include <QTextStream>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
|
||||
class DBCFile {
|
||||
public:
|
||||
DBCFile(const QString &dbc_file_name);
|
||||
DBCFile(const QString &name, const QString &content);
|
||||
~DBCFile() {}
|
||||
|
||||
bool save();
|
||||
bool saveAs(const QString &new_filename);
|
||||
bool writeContents(const QString &fn);
|
||||
QString generateDBC();
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &node, const QString &comment);
|
||||
inline void removeMsg(const MessageId &id) { msgs.erase(id.address); }
|
||||
|
||||
inline const std::map<uint32_t, cabana::Msg> &getMessages() const { return msgs; }
|
||||
cabana::Msg *msg(uint32_t address);
|
||||
cabana::Msg *msg(const QString &name);
|
||||
inline cabana::Msg *msg(const MessageId &id) { return msg(id.address); }
|
||||
cabana::Signal *signal(uint32_t address, const QString &name);
|
||||
|
||||
inline QString name() const { return name_.isEmpty() ? "untitled" : name_; }
|
||||
inline bool isEmpty() const { return msgs.empty() && name_.isEmpty(); }
|
||||
|
||||
QString filename;
|
||||
|
||||
private:
|
||||
void parse(const QString &content);
|
||||
cabana::Msg *parseBO(const QString &line);
|
||||
void parseSG(const QString &line, cabana::Msg *current_msg, int &multiplexor_cnt);
|
||||
void parseCM_BO(const QString &line, const QString &content, const QString &raw_line, const QTextStream &stream);
|
||||
void parseCM_SG(const QString &line, const QString &content, const QString &raw_line, const QTextStream &stream);
|
||||
void parseVAL(const QString &line);
|
||||
|
||||
QString header;
|
||||
std::map<uint32_t, cabana::Msg> msgs;
|
||||
QString name_;
|
||||
};
|
||||
@@ -0,0 +1,69 @@
|
||||
#pragma once
|
||||
|
||||
#include <QObject>
|
||||
#include <memory>
|
||||
#include <map>
|
||||
#include <set>
|
||||
|
||||
#include "tools/cabana/dbc/dbcfile.h"
|
||||
|
||||
typedef std::set<int> SourceSet;
|
||||
const SourceSet SOURCE_ALL = {-1};
|
||||
const int INVALID_SOURCE = 0xff;
|
||||
inline bool operator<(const std::shared_ptr<DBCFile> &l, const std::shared_ptr<DBCFile> &r) { return l.get() < r.get(); }
|
||||
|
||||
class DBCManager : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DBCManager(QObject *parent) : QObject(parent) {}
|
||||
~DBCManager() {}
|
||||
bool open(const SourceSet &sources, const QString &dbc_file_name, QString *error = nullptr);
|
||||
bool open(const SourceSet &sources, const QString &name, const QString &content, QString *error = nullptr);
|
||||
void close(const SourceSet &sources);
|
||||
void close(DBCFile *dbc_file);
|
||||
void closeAll();
|
||||
|
||||
void addSignal(const MessageId &id, const cabana::Signal &sig);
|
||||
void updateSignal(const MessageId &id, const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const MessageId &id, const QString &sig_name);
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &node, const QString &comment);
|
||||
void removeMsg(const MessageId &id);
|
||||
|
||||
QString newMsgName(const MessageId &id);
|
||||
QString newSignalName(const MessageId &id);
|
||||
|
||||
const std::map<uint32_t, cabana::Msg> &getMessages(uint8_t source);
|
||||
cabana::Msg *msg(const MessageId &id);
|
||||
cabana::Msg* msg(uint8_t source, const QString &name);
|
||||
|
||||
QStringList signalNames();
|
||||
inline int dbcCount() { return allDBCFiles().size(); }
|
||||
int nonEmptyDBCCount();
|
||||
|
||||
const SourceSet sources(const DBCFile *dbc_file) const;
|
||||
DBCFile *findDBCFile(const uint8_t source);
|
||||
inline DBCFile *findDBCFile(const MessageId &id) { return findDBCFile(id.source); }
|
||||
std::set<DBCFile *> allDBCFiles();
|
||||
|
||||
signals:
|
||||
void signalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void signalRemoved(const cabana::Signal *sig);
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id);
|
||||
void DBCFileChanged();
|
||||
void maskUpdated();
|
||||
|
||||
private:
|
||||
std::map<int, std::shared_ptr<DBCFile>> dbc_files;
|
||||
};
|
||||
|
||||
DBCManager *dbc();
|
||||
|
||||
QString toString(const SourceSet &ss);
|
||||
inline QString msgName(const MessageId &id) {
|
||||
auto msg = dbc()->msg(id);
|
||||
return msg ? msg->name : UNTITLED;
|
||||
}
|
||||
Executable
+39
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from opendbc.car import Bus
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.values import PLATFORMS
|
||||
|
||||
|
||||
def generate_dbc_dict() -> dict[str, str]:
|
||||
dbc_map = {}
|
||||
for platform in PLATFORMS.values():
|
||||
if platform != "MOCK":
|
||||
if Bus.pt in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.pt]
|
||||
elif Bus.main in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.main]
|
||||
elif Bus.party in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.party]
|
||||
else:
|
||||
raise ValueError("Unknown main type")
|
||||
|
||||
for m in MIGRATION:
|
||||
if MIGRATION[m] in dbc_map:
|
||||
dbc_map[m] = dbc_map[MIGRATION[m]]
|
||||
|
||||
return dbc_map
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Generate mapping for all car fingerprints to DBC names and outputs json file",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--out", required=True, help="Generated json filepath")
|
||||
args = parser.parse_args()
|
||||
|
||||
with open(args.out, 'w') as f:
|
||||
f.write(json.dumps(dict(sorted(generate_dbc_dict().items())), indent=2))
|
||||
print(f"Generated and written to {args.out}")
|
||||
@@ -0,0 +1,70 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialogButtonBox>
|
||||
#include <QSplitter>
|
||||
#include <QTabWidget>
|
||||
#include <QTextEdit>
|
||||
#include <set>
|
||||
|
||||
#include "selfdrive/ui/qt/widgets/controls.h"
|
||||
#include "tools/cabana/binaryview.h"
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/historylog.h"
|
||||
#include "tools/cabana/signalview.h"
|
||||
|
||||
class EditMessageDialog : public QDialog {
|
||||
public:
|
||||
EditMessageDialog(const MessageId &msg_id, const QString &title, int size, QWidget *parent);
|
||||
void validateName(const QString &text);
|
||||
|
||||
MessageId msg_id;
|
||||
QString original_name;
|
||||
QDialogButtonBox *btn_box;
|
||||
QLineEdit *name_edit;
|
||||
QLineEdit *node;
|
||||
QTextEdit *comment_edit;
|
||||
QLabel *error_label;
|
||||
QSpinBox *size_spin;
|
||||
};
|
||||
|
||||
class DetailWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DetailWidget(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void refresh();
|
||||
|
||||
private:
|
||||
void createToolBar();
|
||||
void showTabBarContextMenu(const QPoint &pt);
|
||||
void editMsg();
|
||||
void removeMsg();
|
||||
void updateState(const std::set<MessageId> *msgs = nullptr);
|
||||
|
||||
MessageId msg_id;
|
||||
QLabel *warning_icon, *warning_label;
|
||||
ElidedLabel *name_label;
|
||||
QWidget *warning_widget;
|
||||
TabBar *tabbar;
|
||||
QTabWidget *tab_widget;
|
||||
QAction *action_remove_msg;
|
||||
LogsWidget *history_log;
|
||||
BinaryView *binary_view;
|
||||
SignalView *signal_view;
|
||||
ChartsWidget *charts;
|
||||
QSplitter *splitter;
|
||||
};
|
||||
|
||||
class CenterWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
CenterWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &msg_id);
|
||||
void clear();
|
||||
|
||||
private:
|
||||
QWidget *createWelcomeWidget();
|
||||
DetailWidget *detail_widget = nullptr;
|
||||
QWidget *welcome_widget = nullptr;
|
||||
};
|
||||
@@ -0,0 +1,81 @@
|
||||
#pragma once
|
||||
|
||||
#include <deque>
|
||||
#include <vector>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QHeaderView>
|
||||
#include <QLineEdit>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class HeaderView : public QHeaderView {
|
||||
public:
|
||||
HeaderView(Qt::Orientation orientation, QWidget *parent = nullptr) : QHeaderView(orientation, parent) {}
|
||||
QSize sectionSizeFromContents(int logicalIndex) const override;
|
||||
void paintSection(QPainter *painter, const QRect &rect, int logicalIndex) const;
|
||||
};
|
||||
|
||||
class HistoryLogModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
HistoryLogModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void setMessage(const MessageId &message_id);
|
||||
void updateState(bool clear = false);
|
||||
void setFilter(int sig_idx, const QString &value, std::function<bool(double, double)> cmp);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
void fetchMore(const QModelIndex &parent) override;
|
||||
bool canFetchMore(const QModelIndex &parent) const override;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return messages.size(); }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return !isHexMode() ? sigs.size() + 1 : 2; }
|
||||
inline bool isHexMode() const { return sigs.empty() || hex_mode; }
|
||||
void reset();
|
||||
void setHexMode(bool hex_mode);
|
||||
|
||||
struct Message {
|
||||
uint64_t mono_time = 0;
|
||||
std::vector<double> sig_values;
|
||||
std::vector<uint8_t> data;
|
||||
std::vector<QColor> colors;
|
||||
};
|
||||
|
||||
void fetchData(std::deque<Message>::iterator insert_pos, uint64_t from_time, uint64_t min_time);
|
||||
|
||||
MessageId msg_id;
|
||||
CanData hex_colors;
|
||||
const int batch_size = 50;
|
||||
int filter_sig_idx = -1;
|
||||
double filter_value = 0;
|
||||
std::function<bool(double, double)> filter_cmp = nullptr;
|
||||
std::deque<Message> messages;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
bool hex_mode = false;
|
||||
};
|
||||
|
||||
class LogsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogsWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &message_id) { model->setMessage(message_id); }
|
||||
void updateState() { model->updateState(); }
|
||||
void showEvent(QShowEvent *event) override { model->updateState(true); }
|
||||
|
||||
private slots:
|
||||
void filterChanged();
|
||||
void exportToCSV();
|
||||
void modelReset();
|
||||
|
||||
private:
|
||||
QTableView *logs;
|
||||
HistoryLogModel *model;
|
||||
QComboBox *signals_cb, *comp_box, *display_type_cb;
|
||||
QLineEdit *value_edit;
|
||||
QWidget *filters_widget;
|
||||
ToolButton *export_btn;
|
||||
MessageBytesDelegate *delegate;
|
||||
};
|
||||
@@ -0,0 +1,110 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDockWidget>
|
||||
#include <QJsonDocument>
|
||||
#include <QMainWindow>
|
||||
#include <QMenu>
|
||||
#include <QProgressBar>
|
||||
#include <QSplitter>
|
||||
#include <QStatusBar>
|
||||
#include <set>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/detailwidget.h"
|
||||
#include "tools/cabana/messageswidget.h"
|
||||
#include "tools/cabana/videowidget.h"
|
||||
#include "tools/cabana/tools/findsimilarbits.h"
|
||||
|
||||
class MainWindow : public QMainWindow {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MainWindow(AbstractStream *stream, const QString &dbc_file);
|
||||
void toggleChartsDocking();
|
||||
void showStatusMessage(const QString &msg, int timeout = 0) { statusBar()->showMessage(msg, timeout); }
|
||||
void loadFile(const QString &fn, SourceSet s = SOURCE_ALL);
|
||||
ChartsWidget *charts_widget = nullptr;
|
||||
|
||||
public slots:
|
||||
void selectAndOpenStream();
|
||||
void openStream(AbstractStream *stream, const QString &dbc_file = {});
|
||||
void closeStream();
|
||||
void exportToCSV();
|
||||
|
||||
void newFile(SourceSet s = SOURCE_ALL);
|
||||
void openFile(SourceSet s = SOURCE_ALL);
|
||||
void loadDBCFromOpendbc(const QString &name);
|
||||
void save();
|
||||
void saveAs();
|
||||
void saveToClipboard();
|
||||
|
||||
signals:
|
||||
void showMessage(const QString &msg, int timeout);
|
||||
void updateProgressBar(uint64_t cur, uint64_t total, bool success);
|
||||
|
||||
protected:
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
void remindSaveChanges();
|
||||
void closeFile(SourceSet s = SOURCE_ALL);
|
||||
void closeFile(DBCFile *dbc_file);
|
||||
void saveFile(DBCFile *dbc_file);
|
||||
void saveFileAs(DBCFile *dbc_file);
|
||||
void saveFileToClipboard(DBCFile *dbc_file);
|
||||
void loadFingerprints();
|
||||
void loadFromClipboard(SourceSet s = SOURCE_ALL, bool close_all = true);
|
||||
void updateRecentFiles(const QString &fn);
|
||||
void updateRecentFileMenu();
|
||||
void createActions();
|
||||
void createDockWindows();
|
||||
void createStatusBar();
|
||||
void createShortcuts();
|
||||
void closeEvent(QCloseEvent *event) override;
|
||||
void DBCFileChanged();
|
||||
void updateDownloadProgress(uint64_t cur, uint64_t total, bool success);
|
||||
void setOption();
|
||||
void findSimilarBits();
|
||||
void findSignal();
|
||||
void undoStackCleanChanged(bool clean);
|
||||
void onlineHelp();
|
||||
void toggleFullScreen();
|
||||
void updateStatus();
|
||||
void updateLoadSaveMenus();
|
||||
void createDockWidgets();
|
||||
void eventsMerged();
|
||||
|
||||
VideoWidget *video_widget = nullptr;
|
||||
QDockWidget *video_dock;
|
||||
QDockWidget *messages_dock;
|
||||
MessagesWidget *messages_widget = nullptr;
|
||||
CenterWidget *center_widget;
|
||||
QWidget *floating_window = nullptr;
|
||||
QVBoxLayout *charts_layout;
|
||||
QProgressBar *progress_bar;
|
||||
QLabel *status_label;
|
||||
QJsonDocument fingerprint_to_dbc;
|
||||
QSplitter *video_splitter = nullptr;
|
||||
enum { MAX_RECENT_FILES = 15 };
|
||||
QMenu *open_recent_menu = nullptr;
|
||||
QMenu *manage_dbcs_menu = nullptr;
|
||||
QMenu *tools_menu = nullptr;
|
||||
QAction *close_stream_act = nullptr;
|
||||
QAction *export_to_csv_act = nullptr;
|
||||
QAction *save_dbc = nullptr;
|
||||
QAction *save_dbc_as = nullptr;
|
||||
QAction *copy_dbc_to_clipboard = nullptr;
|
||||
QString car_fingerprint;
|
||||
QByteArray default_state;
|
||||
};
|
||||
|
||||
class HelpOverlay : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
HelpOverlay(MainWindow *parent);
|
||||
|
||||
protected:
|
||||
void drawHelpForWidget(QPainter &painter, QWidget *w);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
};
|
||||
@@ -0,0 +1,121 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <optional>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QHeaderView>
|
||||
#include <QLineEdit>
|
||||
#include <QMenu>
|
||||
#include <QTreeView>
|
||||
#include <QWheelEvent>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class MessageListModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum Column {
|
||||
NAME = 0,
|
||||
SOURCE,
|
||||
ADDRESS,
|
||||
NODE,
|
||||
FREQ,
|
||||
COUNT,
|
||||
DATA,
|
||||
};
|
||||
|
||||
MessageListModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return Column::DATA + 1; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return items_.size(); }
|
||||
void sort(int column, Qt::SortOrder order = Qt::AscendingOrder) override;
|
||||
void setFilterStrings(const QMap<int, QString> &filters);
|
||||
void showInactivemessages(bool show);
|
||||
void msgsReceived(const std::set<MessageId> *new_msgs, bool has_new_ids);
|
||||
bool filterAndSort();
|
||||
void dbcModified();
|
||||
|
||||
struct Item {
|
||||
MessageId id;
|
||||
QString name;
|
||||
QString node;
|
||||
bool operator==(const Item &other) const {
|
||||
return id == other.id && name == other.name && node == other.node;
|
||||
}
|
||||
};
|
||||
std::vector<Item> items_;
|
||||
bool show_inactive_messages = true;
|
||||
|
||||
private:
|
||||
void sortItems(std::vector<MessageListModel::Item> &items);
|
||||
bool match(const MessageListModel::Item &id);
|
||||
|
||||
QMap<int, QString> filters_;
|
||||
std::set<MessageId> dbc_messages_;
|
||||
int sort_column = 0;
|
||||
Qt::SortOrder sort_order = Qt::AscendingOrder;
|
||||
int sort_threshold_ = 0;
|
||||
};
|
||||
|
||||
class MessageView : public QTreeView {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageView(QWidget *parent) : QTreeView(parent) {}
|
||||
void updateBytesSectionSize();
|
||||
|
||||
protected:
|
||||
void drawRow(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void drawBranches(QPainter *painter, const QRect &rect, const QModelIndex &index) const override {}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override;
|
||||
void wheelEvent(QWheelEvent *event) override;
|
||||
};
|
||||
|
||||
class MessageViewHeader : public QHeaderView {
|
||||
// https://stackoverflow.com/a/44346317
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageViewHeader(QWidget *parent);
|
||||
void updateHeaderPositions();
|
||||
void updateGeometries() override;
|
||||
QSize sizeHint() const override;
|
||||
void updateFilters();
|
||||
|
||||
QMap<int, QLineEdit *> editors;
|
||||
};
|
||||
|
||||
class MessagesWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MessagesWidget(QWidget *parent);
|
||||
void selectMessage(const MessageId &message_id);
|
||||
QByteArray saveHeaderState() const { return view->header()->saveState(); }
|
||||
bool restoreHeaderState(const QByteArray &state) const { return view->header()->restoreState(state); }
|
||||
void suppressHighlighted();
|
||||
|
||||
signals:
|
||||
void msgSelectionChanged(const MessageId &message_id);
|
||||
void titleChanged(const QString &title);
|
||||
|
||||
protected:
|
||||
QWidget *createToolBar();
|
||||
void headerContextMenuEvent(const QPoint &pos);
|
||||
void menuAboutToShow();
|
||||
void setMultiLineBytes(bool multi);
|
||||
void updateTitle();
|
||||
|
||||
MessageView *view;
|
||||
MessageViewHeader *header;
|
||||
MessageBytesDelegate *delegate;
|
||||
std::optional<MessageId> current_msg_id;
|
||||
MessageListModel *model;
|
||||
QPushButton *suppress_add;
|
||||
QPushButton *suppress_clear;
|
||||
QMenu *menu;
|
||||
};
|
||||
@@ -0,0 +1,67 @@
|
||||
#pragma once
|
||||
|
||||
#include <QByteArray>
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QGroupBox>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
|
||||
#define LIGHT_THEME 1
|
||||
#define DARK_THEME 2
|
||||
|
||||
class Settings : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum DragDirection {
|
||||
MsbFirst,
|
||||
LsbFirst,
|
||||
AlwaysLE,
|
||||
AlwaysBE,
|
||||
};
|
||||
|
||||
Settings();
|
||||
~Settings();
|
||||
|
||||
bool absolute_time = false;
|
||||
int fps = 10;
|
||||
int max_cached_minutes = 30;
|
||||
int chart_height = 200;
|
||||
int chart_column_count = 1;
|
||||
int chart_range = 3 * 60; // 3 minutes
|
||||
int chart_series_type = 0;
|
||||
int theme = 0;
|
||||
int sparkline_range = 15; // 15 seconds
|
||||
bool multiple_lines_hex = false;
|
||||
bool log_livestream = true;
|
||||
bool suppress_defined_signals = false;
|
||||
QString log_path;
|
||||
QString last_dir;
|
||||
QString last_route_dir;
|
||||
QByteArray geometry;
|
||||
QByteArray video_splitter_state;
|
||||
QByteArray window_state;
|
||||
QStringList recent_files;
|
||||
QByteArray message_header_state;
|
||||
DragDirection drag_direction = MsbFirst;
|
||||
|
||||
signals:
|
||||
void changed();
|
||||
};
|
||||
|
||||
class SettingsDlg : public QDialog {
|
||||
public:
|
||||
SettingsDlg(QWidget *parent);
|
||||
void save();
|
||||
QSpinBox *fps;
|
||||
QSpinBox *cached_minutes;
|
||||
QSpinBox *chart_height;
|
||||
QComboBox *chart_series_type;
|
||||
QComboBox *theme;
|
||||
QGroupBox *log_livestream;
|
||||
QLineEdit *log_path;
|
||||
QComboBox *drag_direction;
|
||||
};
|
||||
|
||||
extern Settings settings;
|
||||
@@ -0,0 +1,147 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <utility>
|
||||
|
||||
#include <QAbstractItemModel>
|
||||
#include <QLabel>
|
||||
#include <QLineEdit>
|
||||
#include <QSlider>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableWidget>
|
||||
#include <QTreeView>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/chart/sparkline.h"
|
||||
|
||||
class SignalModel : public QAbstractItemModel {
|
||||
Q_OBJECT
|
||||
public:
|
||||
struct Item {
|
||||
enum Type {Root, Sig, Name, Size, Node, Endian, Signed, Offset, Factor, SignalType, MultiplexValue, ExtraInfo, Unit, Comment, Min, Max, Desc };
|
||||
~Item() { qDeleteAll(children); }
|
||||
inline int row() { return parent->children.indexOf(this); }
|
||||
|
||||
Type type = Type::Root;
|
||||
Item *parent = nullptr;
|
||||
QList<Item *> children;
|
||||
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QString title;
|
||||
bool highlight = false;
|
||||
QString sig_val = "-";
|
||||
Sparkline sparkline;
|
||||
};
|
||||
|
||||
SignalModel(QObject *parent);
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 2; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override;
|
||||
QModelIndex parent(const QModelIndex &index) const override;
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override;
|
||||
bool setData(const QModelIndex &index, const QVariant &value, int role = Qt::EditRole) override;
|
||||
void setMessage(const MessageId &id);
|
||||
void setFilter(const QString &txt);
|
||||
bool saveSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
Item *getItem(const QModelIndex &index) const;
|
||||
int signalRow(const cabana::Signal *sig) const;
|
||||
|
||||
private:
|
||||
void insertItem(SignalModel::Item *root_item, int pos, const cabana::Signal *sig);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void handleSignalRemoved(const cabana::Signal *sig);
|
||||
void handleMsgChanged(MessageId id);
|
||||
void refresh();
|
||||
|
||||
MessageId msg_id;
|
||||
QString filter_str;
|
||||
std::unique_ptr<Item> root;
|
||||
friend class SignalView;
|
||||
friend class SignalItemDelegate;
|
||||
};
|
||||
|
||||
class ValueDescriptionDlg : public QDialog {
|
||||
public:
|
||||
ValueDescriptionDlg(const ValueDescription &descriptions, QWidget *parent);
|
||||
ValueDescription val_desc;
|
||||
|
||||
private:
|
||||
struct Delegate : public QStyledItemDelegate {
|
||||
Delegate(QWidget *parent) : QStyledItemDelegate(parent) {}
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
};
|
||||
|
||||
void save();
|
||||
QTableWidget *table;
|
||||
};
|
||||
|
||||
class SignalItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
SignalItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void updateEditorGeometry(QWidget *editor, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void setModelData(QWidget *editor, QAbstractItemModel *model, const QModelIndex &index) const override;
|
||||
|
||||
QValidator *name_validator, *double_validator, *node_validator;
|
||||
QFont label_font, minmax_font;
|
||||
const int color_label_width = 18;
|
||||
mutable QSize button_size;
|
||||
};
|
||||
|
||||
class SignalView : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
SignalView(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &id);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void updateChartState();
|
||||
void selectSignal(const cabana::Signal *sig, bool expand = false);
|
||||
void rowClicked(const QModelIndex &index);
|
||||
SignalModel *model = nullptr;
|
||||
|
||||
signals:
|
||||
void highlight(const cabana::Signal *sig);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void rowsChanged();
|
||||
void resizeEvent(QResizeEvent* event) override;
|
||||
void updateToolBar();
|
||||
void setSparklineRange(int value);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void updateState(const std::set<MessageId> *msgs = nullptr);
|
||||
std::pair<QModelIndex, QModelIndex> visibleSignalRange();
|
||||
|
||||
struct TreeView : public QTreeView {
|
||||
TreeView(QWidget *parent) : QTreeView(parent) {}
|
||||
void rowsInserted(const QModelIndex &parent, int start, int end) override {
|
||||
((SignalView *)parentWidget())->rowsChanged();
|
||||
// update widget geometries in QTreeView::rowsInserted
|
||||
QTreeView::rowsInserted(parent, start, end);
|
||||
}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override {
|
||||
// Bypass the slow call to QTreeView::dataChanged.
|
||||
QAbstractItemView::dataChanged(topLeft, bottomRight, roles);
|
||||
}
|
||||
void leaveEvent(QEvent *event) override {
|
||||
emit static_cast<SignalView *>(parentWidget())->highlight(nullptr);
|
||||
QTreeView::leaveEvent(event);
|
||||
}
|
||||
};
|
||||
int max_value_width = 0;
|
||||
int value_column_width = 0;
|
||||
TreeView *tree;
|
||||
QLabel *sparkline_label;
|
||||
QSlider *sparkline_range_slider;
|
||||
QLineEdit *filter_edit;
|
||||
ChartsWidget *charts;
|
||||
QLabel *signal_count_lb;
|
||||
SignalItemDelegate *delegate;
|
||||
};
|
||||
@@ -0,0 +1,157 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <condition_variable>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <set>
|
||||
#include <unordered_map>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QDateTime>
|
||||
|
||||
#include "cereal/messaging/messaging.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/utils/util.h"
|
||||
#include "tools/replay/util.h"
|
||||
|
||||
struct CanData {
|
||||
void compute(const MessageId &msg_id, const uint8_t *dat, const int size, double current_sec,
|
||||
double playback_speed, const std::vector<uint8_t> &mask, double in_freq = 0);
|
||||
|
||||
double ts = 0.;
|
||||
uint32_t count = 0;
|
||||
double freq = 0;
|
||||
std::vector<uint8_t> dat;
|
||||
std::vector<QColor> colors;
|
||||
|
||||
struct ByteLastChange {
|
||||
double ts = 0;
|
||||
int delta = 0;
|
||||
int same_delta_counter = 0;
|
||||
bool suppressed = false;
|
||||
};
|
||||
std::vector<ByteLastChange> last_changes;
|
||||
std::vector<std::array<uint32_t, 8>> bit_flip_counts;
|
||||
double last_freq_update_ts = 0;
|
||||
};
|
||||
|
||||
struct CanEvent {
|
||||
uint8_t src;
|
||||
uint32_t address;
|
||||
uint64_t mono_time;
|
||||
uint8_t size;
|
||||
uint8_t dat[];
|
||||
};
|
||||
|
||||
struct CompareCanEvent {
|
||||
constexpr bool operator()(const CanEvent *const e, uint64_t ts) const { return e->mono_time < ts; }
|
||||
constexpr bool operator()(uint64_t ts, const CanEvent *const e) const { return ts < e->mono_time; }
|
||||
};
|
||||
|
||||
typedef std::unordered_map<MessageId, std::vector<const CanEvent *>> MessageEventsMap;
|
||||
using CanEventIter = std::vector<const CanEvent *>::const_iterator;
|
||||
|
||||
class AbstractStream : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
AbstractStream(QObject *parent);
|
||||
virtual ~AbstractStream() {}
|
||||
virtual void start() = 0;
|
||||
virtual bool liveStreaming() const { return true; }
|
||||
virtual void seekTo(double ts) {}
|
||||
virtual QString routeName() const = 0;
|
||||
virtual QString carFingerprint() const { return ""; }
|
||||
virtual QDateTime beginDateTime() const { return {}; }
|
||||
virtual uint64_t beginMonoTime() const { return 0; }
|
||||
virtual double minSeconds() const { return 0; }
|
||||
virtual double maxSeconds() const { return 0; }
|
||||
virtual void setSpeed(float speed) {}
|
||||
virtual double getSpeed() { return 1; }
|
||||
virtual bool isPaused() const { return false; }
|
||||
virtual void pause(bool pause) {}
|
||||
void setTimeRange(const std::optional<std::pair<double, double>> &range);
|
||||
const std::optional<std::pair<double, double>> &timeRange() const { return time_range_; }
|
||||
|
||||
inline double currentSec() const { return current_sec_; }
|
||||
inline uint64_t toMonoTime(double sec) const { return beginMonoTime() + std::max(sec, 0.0) * 1e9; }
|
||||
inline double toSeconds(uint64_t mono_time) const { return std::max(0.0, (mono_time - beginMonoTime()) / 1e9); }
|
||||
|
||||
inline const std::unordered_map<MessageId, CanData> &lastMessages() const { return last_msgs; }
|
||||
bool isMessageActive(const MessageId &id) const;
|
||||
inline const MessageEventsMap &eventsMap() const { return events_; }
|
||||
inline const std::vector<const CanEvent *> &allEvents() const { return all_events_; }
|
||||
const CanData &lastMessage(const MessageId &id) const;
|
||||
const std::vector<const CanEvent *> &events(const MessageId &id) const;
|
||||
std::pair<CanEventIter, CanEventIter> eventsInRange(const MessageId &id, std::optional<std::pair<double, double>> time_range) const;
|
||||
|
||||
size_t suppressHighlighted();
|
||||
void clearSuppressed();
|
||||
void suppressDefinedSignals(bool suppress);
|
||||
|
||||
signals:
|
||||
void paused();
|
||||
void resume();
|
||||
void seeking(double sec);
|
||||
void seekedTo(double sec);
|
||||
void timeRangeChanged(const std::optional<std::pair<double, double>> &range);
|
||||
void eventsMerged(const MessageEventsMap &events_map);
|
||||
void msgsReceived(const std::set<MessageId> *new_msgs, bool has_new_ids);
|
||||
void sourcesUpdated(const SourceSet &s);
|
||||
void privateUpdateLastMsgsSignal();
|
||||
|
||||
public:
|
||||
SourceSet sources;
|
||||
|
||||
protected:
|
||||
void mergeEvents(const std::vector<const CanEvent *> &events);
|
||||
const CanEvent *newEvent(uint64_t mono_time, const cereal::CanData::Reader &c);
|
||||
void updateEvent(const MessageId &id, double sec, const uint8_t *data, uint8_t size);
|
||||
void waitForSeekFinshed();
|
||||
std::vector<const CanEvent *> all_events_;
|
||||
double current_sec_ = 0;
|
||||
std::optional<std::pair<double, double>> time_range_;
|
||||
|
||||
private:
|
||||
void updateLastMessages();
|
||||
void updateLastMsgsTo(double sec);
|
||||
void updateMasks();
|
||||
|
||||
MessageEventsMap events_;
|
||||
std::unordered_map<MessageId, CanData> last_msgs;
|
||||
std::unique_ptr<MonotonicBuffer> event_buffer_;
|
||||
|
||||
// Members accessed in multiple threads. (mutex protected)
|
||||
std::mutex mutex_;
|
||||
std::condition_variable seek_finished_cv_;
|
||||
bool seek_finished_ = false;
|
||||
std::set<MessageId> new_msgs_;
|
||||
std::unordered_map<MessageId, CanData> messages_;
|
||||
std::unordered_map<MessageId, std::vector<uint8_t>> masks_;
|
||||
};
|
||||
|
||||
class AbstractOpenStreamWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
AbstractOpenStreamWidget(QWidget *parent = nullptr) : QWidget(parent) {}
|
||||
virtual AbstractStream *open() = 0;
|
||||
|
||||
signals:
|
||||
void enableOpenButton(bool);
|
||||
};
|
||||
|
||||
class DummyStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DummyStream(QObject *parent) : AbstractStream(parent) {}
|
||||
QString routeName() const override { return tr("No Stream"); }
|
||||
void start() override {}
|
||||
};
|
||||
|
||||
// A global pointer referring to the unique AbstractStream object
|
||||
extern AbstractStream *can;
|
||||
@@ -0,0 +1,28 @@
|
||||
#pragma once
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
class DeviceStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DeviceStream(QObject *parent, QString address = {});
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From %1").arg(zmq_address.isEmpty() ? "127.0.0.1" : zmq_address);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
const QString zmq_address;
|
||||
};
|
||||
|
||||
class OpenDeviceWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenDeviceWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
QLineEdit *ip_address;
|
||||
QButtonGroup *group;
|
||||
};
|
||||
@@ -0,0 +1,56 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QBasicTimer>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class LiveStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LiveStream(QObject *parent);
|
||||
virtual ~LiveStream();
|
||||
void start() override;
|
||||
void stop();
|
||||
inline QDateTime beginDateTime() const { return begin_date_time; }
|
||||
inline uint64_t beginMonoTime() const override { return begin_event_ts; }
|
||||
double maxSeconds() const override { return std::max(1.0, (lastest_event_ts - begin_event_ts) / 1e9); }
|
||||
void setSpeed(float speed) override { speed_ = speed; }
|
||||
double getSpeed() override { return speed_; }
|
||||
bool isPaused() const override { return paused_; }
|
||||
void pause(bool pause) override;
|
||||
void seekTo(double sec) override;
|
||||
|
||||
protected:
|
||||
virtual void streamThread() = 0;
|
||||
void handleEvent(kj::ArrayPtr<capnp::word> event);
|
||||
|
||||
private:
|
||||
void startUpdateTimer();
|
||||
void timerEvent(QTimerEvent *event) override;
|
||||
void updateEvents();
|
||||
|
||||
std::mutex lock;
|
||||
QThread *stream_thread;
|
||||
std::vector<const CanEvent *> received_events_;
|
||||
|
||||
int timer_id;
|
||||
QBasicTimer update_timer;
|
||||
|
||||
QDateTime begin_date_time;
|
||||
uint64_t begin_event_ts = 0;
|
||||
uint64_t lastest_event_ts = 0;
|
||||
uint64_t current_event_ts = 0;
|
||||
uint64_t first_event_ts = 0;
|
||||
uint64_t first_update_ts = 0;
|
||||
bool post_last_event = true;
|
||||
double speed_ = 1;
|
||||
bool paused_ = false;
|
||||
|
||||
struct Logger;
|
||||
std::unique_ptr<Logger> logger;
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QFormLayout>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
#include "selfdrive/pandad/panda.h"
|
||||
|
||||
const uint32_t speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U};
|
||||
const uint32_t data_speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U, 2000U, 5000U};
|
||||
|
||||
struct BusConfig {
|
||||
int can_speed_kbps = 500;
|
||||
int data_speed_kbps = 2000;
|
||||
bool can_fd = false;
|
||||
};
|
||||
|
||||
struct PandaStreamConfig {
|
||||
QString serial = "";
|
||||
std::vector<BusConfig> bus_config;
|
||||
};
|
||||
|
||||
class PandaStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
PandaStream(QObject *parent, PandaStreamConfig config_ = {});
|
||||
~PandaStream() { stop(); }
|
||||
inline QString routeName() const override {
|
||||
return QString("Panda: %1").arg(config.serial);
|
||||
}
|
||||
|
||||
protected:
|
||||
bool connect();
|
||||
void streamThread() override;
|
||||
|
||||
std::unique_ptr<Panda> panda;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
|
||||
class OpenPandaWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenPandaWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
void refreshSerials();
|
||||
void buildConfigForm();
|
||||
|
||||
QComboBox *serial_edit;
|
||||
QFormLayout *form_layout;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
#include <QCheckBox>
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include "common/prefix.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
#include "tools/replay/replay.h"
|
||||
|
||||
Q_DECLARE_METATYPE(std::shared_ptr<LogReader>);
|
||||
|
||||
class ReplayStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ReplayStream(QObject *parent);
|
||||
void start() override { replay->start(); }
|
||||
bool loadRoute(const QString &route, const QString &data_dir, uint32_t replay_flags = REPLAY_FLAG_NONE);
|
||||
bool eventFilter(const Event *event);
|
||||
void seekTo(double ts) override { replay->seekTo(std::max(double(0), ts), false); }
|
||||
bool liveStreaming() const override { return false; }
|
||||
inline QString routeName() const override { return QString::fromStdString(replay->route().name()); }
|
||||
inline QString carFingerprint() const override { return replay->carFingerprint().c_str(); }
|
||||
double minSeconds() const override { return replay->minSeconds(); }
|
||||
double maxSeconds() const { return replay->maxSeconds(); }
|
||||
inline QDateTime beginDateTime() const { return QDateTime::fromSecsSinceEpoch(replay->routeDateTime()); }
|
||||
inline uint64_t beginMonoTime() const override { return replay->routeStartNanos(); }
|
||||
inline void setSpeed(float speed) override { replay->setSpeed(speed); }
|
||||
inline float getSpeed() const { return replay->getSpeed(); }
|
||||
inline Replay *getReplay() const { return replay.get(); }
|
||||
inline bool isPaused() const override { return replay->isPaused(); }
|
||||
void pause(bool pause) override;
|
||||
|
||||
signals:
|
||||
void qLogLoaded(std::shared_ptr<LogReader> qlog);
|
||||
|
||||
private:
|
||||
void mergeSegments();
|
||||
std::unique_ptr<Replay> replay = nullptr;
|
||||
std::set<int> processed_segments;
|
||||
std::unique_ptr<OpenpilotPrefix> op_prefix;
|
||||
};
|
||||
|
||||
class OpenReplayWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenReplayWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
QLineEdit *route_edit;
|
||||
std::vector<QCheckBox *> cameras;
|
||||
};
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
|
||||
#include "selfdrive/ui/qt/api.h"
|
||||
|
||||
class RouteListWidget;
|
||||
class OneShotHttpRequest;
|
||||
|
||||
class RoutesDialog : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
RoutesDialog(QWidget *parent);
|
||||
QString route();
|
||||
|
||||
protected:
|
||||
void parseDeviceList(const QString &json, bool success, QNetworkReply::NetworkError err);
|
||||
void parseRouteList(const QString &json, bool success, QNetworkReply::NetworkError err);
|
||||
void fetchRoutes();
|
||||
|
||||
QComboBox *device_list_;
|
||||
QComboBox *period_selector_;
|
||||
RouteListWidget *route_list_;
|
||||
OneShotHttpRequest *route_requester_;
|
||||
};
|
||||
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include <QtSerialBus/QCanBus>
|
||||
#include <QtSerialBus/QCanBusDevice>
|
||||
#include <QtSerialBus/QCanBusDeviceInfo>
|
||||
#include <QComboBox>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
struct SocketCanStreamConfig {
|
||||
QString device = ""; // TODO: support multiple devices/buses at once
|
||||
};
|
||||
|
||||
class SocketCanStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
SocketCanStream(QObject *parent, SocketCanStreamConfig config_ = {});
|
||||
~SocketCanStream() { stop(); }
|
||||
static bool available();
|
||||
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From Socket CAN %1").arg(config.device);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
bool connect();
|
||||
|
||||
SocketCanStreamConfig config = {};
|
||||
std::unique_ptr<QCanBusDevice> device;
|
||||
};
|
||||
|
||||
class OpenSocketCanWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenSocketCanWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
void refreshDevices();
|
||||
|
||||
QComboBox *device_edit;
|
||||
SocketCanStreamConfig config = {};
|
||||
};
|
||||
@@ -0,0 +1,24 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialogButtonBox>
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QTabWidget>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class StreamSelector : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
StreamSelector(QWidget *parent = nullptr);
|
||||
void addStreamWidget(AbstractOpenStreamWidget *w, const QString &title);
|
||||
QString dbcFile() const { return dbc_file->text(); }
|
||||
AbstractStream *stream() const { return stream_; }
|
||||
|
||||
private:
|
||||
AbstractStream *stream_ = nullptr;
|
||||
QLineEdit *dbc_file;
|
||||
QTabWidget *tab;
|
||||
QDialogButtonBox *btn_box;
|
||||
};
|
||||
@@ -0,0 +1,64 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <limits>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QCheckBox>
|
||||
#include <QLabel>
|
||||
#include <QPushButton>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/commands.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class FindSignalModel : public QAbstractTableModel {
|
||||
public:
|
||||
struct SearchSignal {
|
||||
MessageId id = {};
|
||||
uint64_t mono_time = 0;
|
||||
cabana::Signal sig = {};
|
||||
double value = 0.;
|
||||
QStringList values;
|
||||
};
|
||||
|
||||
FindSignalModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 3; }
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return std::min(filtered_signals.size(), 300); }
|
||||
void search(std::function<bool(double)> cmp);
|
||||
void reset();
|
||||
void undo();
|
||||
|
||||
QList<SearchSignal> filtered_signals;
|
||||
QList<SearchSignal> initial_signals;
|
||||
QList<QList<SearchSignal>> histories;
|
||||
uint64_t last_time = std::numeric_limits<uint64_t>::max();
|
||||
};
|
||||
|
||||
class FindSignalDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
FindSignalDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &id);
|
||||
|
||||
private:
|
||||
void search();
|
||||
void modelReset();
|
||||
void setInitialSignals();
|
||||
void customMenuRequested(const QPoint &pos);
|
||||
|
||||
QLineEdit *value1, *value2, *factor_edit, *offset_edit;
|
||||
QLineEdit *bus_edit, *address_edit, *first_time_edit, *last_time_edit;
|
||||
QComboBox *compare_cb;
|
||||
QSpinBox *min_size, *max_size;
|
||||
QCheckBox *litter_endian, *is_signed;
|
||||
QPushButton *search_btn, *reset_btn, *undo_btn;
|
||||
QGroupBox *properties_group, *message_group;
|
||||
QTableView *view;
|
||||
QLabel *to_label, *stats_label;
|
||||
FindSignalModel *model;
|
||||
};
|
||||
@@ -0,0 +1,34 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
#include <QTableWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class FindSimilarBitsDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
FindSimilarBitsDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &msg_id);
|
||||
|
||||
private:
|
||||
struct mismatched_struct {
|
||||
uint32_t address, byte_idx, bit_idx, mismatches, total;
|
||||
float perc;
|
||||
};
|
||||
QList<mismatched_struct> calcBits(uint8_t bus, uint32_t selected_address, int byte_idx, int bit_idx, uint8_t find_bus,
|
||||
bool equal, int min_msgs_cnt);
|
||||
void find();
|
||||
|
||||
QTableWidget *table;
|
||||
QComboBox *src_bus_combo, *find_bus_combo, *msg_cb, *equal_combo;
|
||||
QSpinBox *byte_idx_sb, *bit_idx_sb;
|
||||
QPushButton *search_btn;
|
||||
QLineEdit *min_msgs;
|
||||
};
|
||||
@@ -0,0 +1,10 @@
|
||||
#pragma once
|
||||
|
||||
#include <optional>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
namespace utils {
|
||||
void exportToCSV(const QString &file_name, std::optional<MessageId> msg_id = std::nullopt);
|
||||
void exportSignalsToCSV(const QString &file_name, const MessageId &msg_id);
|
||||
} // namespace utils
|
||||
@@ -0,0 +1,166 @@
|
||||
#pragma once
|
||||
|
||||
#include <array>
|
||||
#include <cmath>
|
||||
#include <vector>
|
||||
#include <utility>
|
||||
|
||||
#include <QApplication>
|
||||
#include <QByteArray>
|
||||
#include <QDoubleValidator>
|
||||
#include <QFont>
|
||||
#include <QFontMetrics>
|
||||
#include <QPainter>
|
||||
#include <QRegExpValidator>
|
||||
#include <QSocketNotifier>
|
||||
#include <QStaticText>
|
||||
#include <QStringBuilder>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QToolButton>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class LogSlider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogSlider(double factor, Qt::Orientation orientation, QWidget *parent = nullptr) : factor(factor), QSlider(orientation, parent) {}
|
||||
|
||||
void setRange(double min, double max) {
|
||||
log_min = factor * std::log10(min);
|
||||
log_max = factor * std::log10(max);
|
||||
QSlider::setRange(min, max);
|
||||
setValue(QSlider::value());
|
||||
}
|
||||
int value() const {
|
||||
double v = log_min + (log_max - log_min) * ((QSlider::value() - minimum()) / double(maximum() - minimum()));
|
||||
return std::lround(std::pow(10, v / factor));
|
||||
}
|
||||
void setValue(int v) {
|
||||
double log_v = std::clamp(factor * std::log10(v), log_min, log_max);
|
||||
v = minimum() + (maximum() - minimum()) * ((log_v - log_min) / (log_max - log_min));
|
||||
QSlider::setValue(v);
|
||||
}
|
||||
|
||||
private:
|
||||
double factor, log_min = 0, log_max = 1;
|
||||
};
|
||||
|
||||
enum {
|
||||
ColorsRole = Qt::UserRole + 1,
|
||||
BytesRole = Qt::UserRole + 2
|
||||
};
|
||||
|
||||
class SegmentTree {
|
||||
public:
|
||||
SegmentTree() = default;
|
||||
void build(const std::vector<QPointF> &arr);
|
||||
inline std::pair<double, double> minmax(int left, int right) const { return get_minmax(1, 0, size - 1, left, right); }
|
||||
|
||||
private:
|
||||
std::pair<double, double> get_minmax(int n, int left, int right, int range_left, int range_right) const;
|
||||
void build_tree(const std::vector<QPointF> &arr, int n, int left, int right);
|
||||
std::vector<std::pair<double, double>> tree;
|
||||
int size = 0;
|
||||
};
|
||||
|
||||
class MessageBytesDelegate : public QStyledItemDelegate {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageBytesDelegate(QObject *parent, bool multiple_lines = false);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool multipleLines() const { return multiple_lines; }
|
||||
void setMultipleLines(bool v) { multiple_lines = v; }
|
||||
QSize sizeForBytes(int n) const;
|
||||
|
||||
private:
|
||||
std::array<QStaticText, 256> hex_text_table;
|
||||
QFontMetrics font_metrics;
|
||||
QFont fixed_font;
|
||||
QSize byte_size = {};
|
||||
bool multiple_lines = false;
|
||||
int h_margin, v_margin;
|
||||
};
|
||||
|
||||
class NameValidator : public QRegExpValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
NameValidator(QObject *parent=nullptr);
|
||||
QValidator::State validate(QString &input, int &pos) const override;
|
||||
};
|
||||
|
||||
class DoubleValidator : public QDoubleValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DoubleValidator(QObject *parent = nullptr);
|
||||
};
|
||||
|
||||
namespace utils {
|
||||
QPixmap icon(const QString &id);
|
||||
void setTheme(int theme);
|
||||
QString formatSeconds(double sec, bool include_milliseconds = false, bool absolute_time = false);
|
||||
inline void drawStaticText(QPainter *p, const QRect &r, const QStaticText &text) {
|
||||
auto size = (r.size() - text.size()) / 2;
|
||||
p->drawStaticText(r.left() + size.width(), r.top() + size.height(), text);
|
||||
}
|
||||
inline QString toHex(const std::vector<uint8_t> &dat, char separator = '\0') {
|
||||
return QByteArray::fromRawData((const char *)dat.data(), dat.size()).toHex(separator).toUpper();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
class ToolButton : public QToolButton {
|
||||
Q_OBJECT
|
||||
public:
|
||||
ToolButton(const QString &icon, const QString &tooltip = {}, QWidget *parent = nullptr) : QToolButton(parent) {
|
||||
setIcon(icon);
|
||||
setToolTip(tooltip);
|
||||
setAutoRaise(true);
|
||||
const int metric = QApplication::style()->pixelMetric(QStyle::PM_SmallIconSize);
|
||||
setIconSize({metric, metric});
|
||||
theme = settings.theme;
|
||||
connect(&settings, &Settings::changed, this, &ToolButton::updateIcon);
|
||||
}
|
||||
void setIcon(const QString &icon) {
|
||||
icon_str = icon;
|
||||
QToolButton::setIcon(utils::icon(icon_str));
|
||||
}
|
||||
|
||||
private:
|
||||
void updateIcon() { if (std::exchange(theme, settings.theme) != theme) setIcon(icon_str); }
|
||||
QString icon_str;
|
||||
int theme;
|
||||
};
|
||||
|
||||
class TabBar : public QTabBar {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
TabBar(QWidget *parent) : QTabBar(parent) {}
|
||||
int addTab(const QString &text);
|
||||
|
||||
private:
|
||||
void closeTabClicked();
|
||||
};
|
||||
|
||||
class UnixSignalHandler : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
UnixSignalHandler(QObject *parent = nullptr);
|
||||
~UnixSignalHandler();
|
||||
static void signalHandler(int s);
|
||||
|
||||
public slots:
|
||||
void handleSigTerm();
|
||||
|
||||
private:
|
||||
inline static int sig_fd[2] = {};
|
||||
QSocketNotifier *sn;
|
||||
};
|
||||
|
||||
int num_decimals(double num);
|
||||
QString signalToolTip(const cabana::Signal *sig);
|
||||
inline QString toHexString(int value) { return QString("0x%1").arg(QString::number(value, 16).toUpper(), 2, '0'); }
|
||||
@@ -0,0 +1,82 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
|
||||
#include <QFrame>
|
||||
#include <QPropertyAnimation>
|
||||
#include <QSlider>
|
||||
#include <QToolBar>
|
||||
#include <QTabBar>
|
||||
|
||||
#include "selfdrive/ui/qt/widgets/cameraview.h"
|
||||
#include "tools/cabana/utils/util.h"
|
||||
#include "tools/replay/logreader.h"
|
||||
#include "tools/cabana/streams/replaystream.h"
|
||||
|
||||
class Slider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
Slider(QWidget *parent);
|
||||
double currentSecond() const { return value() / factor; }
|
||||
void setCurrentSecond(double sec) { setValue(sec * factor); }
|
||||
void setTimeRange(double min, double max) { setRange(min * factor, max * factor); }
|
||||
void mousePressEvent(QMouseEvent *e) override;
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
const double factor = 1000.0;
|
||||
double thumbnail_dispaly_time = -1;
|
||||
};
|
||||
|
||||
class StreamCameraView : public CameraWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
StreamCameraView(std::string stream_name, VisionStreamType stream_type, QWidget *parent = nullptr);
|
||||
void paintGL() override;
|
||||
void showPausedOverlay() { fade_animation->start(); }
|
||||
void parseQLog(std::shared_ptr<LogReader> qlog);
|
||||
|
||||
private:
|
||||
QPixmap generateThumbnail(QPixmap thumbnail, double seconds);
|
||||
void drawAlert(QPainter &p, const QRect &rect, const Timeline::Entry &alert);
|
||||
void drawThumbnail(QPainter &p);
|
||||
void drawScrubThumbnail(QPainter &p);
|
||||
void drawTime(QPainter &p, const QRect &rect, double seconds);
|
||||
|
||||
QPropertyAnimation *fade_animation;
|
||||
QMap<uint64_t, QPixmap> big_thumbnails;
|
||||
QMap<uint64_t, QPixmap> thumbnails;
|
||||
double thumbnail_dispaly_time = -1;
|
||||
friend class VideoWidget;
|
||||
};
|
||||
|
||||
class VideoWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
VideoWidget(QWidget *parnet = nullptr);
|
||||
void showThumbnail(double seconds);
|
||||
|
||||
protected:
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
QString formatTime(double sec, bool include_milliseconds = false);
|
||||
void timeRangeChanged();
|
||||
void updateState();
|
||||
void updatePlayBtnState();
|
||||
QWidget *createCameraWidget();
|
||||
void createPlaybackController();
|
||||
void createSpeedDropdown(QToolBar *toolbar);
|
||||
void loopPlaybackClicked();
|
||||
void vipcAvailableStreamsUpdated(std::set<VisionStreamType> streams);
|
||||
|
||||
StreamCameraView *cam_widget;
|
||||
QAction *time_display_action = nullptr;
|
||||
QAction *play_toggle_action = nullptr;
|
||||
QToolButton *speed_btn = nullptr;
|
||||
QAction *skip_to_end_action = nullptr;
|
||||
Slider *slider = nullptr;
|
||||
QTabBar *camera_tab = nullptr;
|
||||
};
|
||||
@@ -0,0 +1,66 @@
|
||||
# Camera stream
|
||||
|
||||
`compressed_vipc.py` connects to a remote device running openpilot, decodes the video streams, and republishes them over VisionIPC.
|
||||
|
||||
## Usage
|
||||
|
||||
### On the device
|
||||
SSH into the device and run following in separate terminals:
|
||||
|
||||
`cd /data/openpilot/cereal/messaging && ./bridge`
|
||||
|
||||
`cd /data/openpilot/system/loggerd && ./encoderd`
|
||||
|
||||
`cd /data/openpilot/system/camerad && ./camerad`
|
||||
|
||||
Note that both the device and your PC must be on the same openpilot commit.
|
||||
|
||||
Alternatively paste this as a single command:
|
||||
```
|
||||
(
|
||||
cd /data/openpilot/cereal/messaging/
|
||||
./bridge &
|
||||
|
||||
cd /data/openpilot/system/camerad/
|
||||
./camerad &
|
||||
|
||||
cd /data/openpilot/system/loggerd/
|
||||
./encoderd &
|
||||
|
||||
wait
|
||||
) ; trap 'kill $(jobs -p)' SIGINT
|
||||
```
|
||||
Ctrl+C will stop all three processes.
|
||||
|
||||
### On the PC
|
||||
Decode the stream with `compressed_vipc.py`:
|
||||
|
||||
```cd ~/openpilot/tools/camerastream && ./compressed_vipc.py <ip>```
|
||||
|
||||
To actually display the stream, run `watch3` in separate terminal:
|
||||
|
||||
```cd ~/openpilot/selfdrive/ui/ && ./watch3```
|
||||
|
||||
## compressed_vipc.py usage
|
||||
```
|
||||
$ python3 compressed_vipc.py -h
|
||||
usage: compressed_vipc.py [-h] [--nvidia] [--cams CAMS] [--silent] addr
|
||||
|
||||
Decode video streams and broadcast on VisionIPC
|
||||
|
||||
positional arguments:
|
||||
addr Address of comma three
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--nvidia Use nvidia instead of ffmpeg
|
||||
--cams CAMS Cameras to decode
|
||||
--silent Suppress debug output
|
||||
```
|
||||
|
||||
|
||||
## Example:
|
||||
```
|
||||
cd ~/openpilot/tools/camerastream && ./compressed_vipc.py comma-ffffffff --cams 0
|
||||
cd ~/openpilot/selfdrive/ui/ && ./watch3
|
||||
```
|
||||
Executable
+162
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env python3
|
||||
import av
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import numpy as np
|
||||
import multiprocessing
|
||||
import time
|
||||
import signal
|
||||
|
||||
|
||||
import cereal.messaging as messaging
|
||||
from msgq.visionipc import VisionIpcServer, VisionStreamType
|
||||
|
||||
V4L2_BUF_FLAG_KEYFRAME = 8
|
||||
|
||||
# start encoderd
|
||||
# also start cereal messaging bridge
|
||||
# then run this "./compressed_vipc.py <ip>"
|
||||
|
||||
ENCODE_SOCKETS = {
|
||||
VisionStreamType.VISION_STREAM_ROAD: "roadEncodeData",
|
||||
VisionStreamType.VISION_STREAM_DRIVER: "driverEncodeData",
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD: "wideRoadEncodeData",
|
||||
}
|
||||
|
||||
def decoder(addr, vipc_server, vst, nvidia, W, H, debug=False):
|
||||
sock_name = ENCODE_SOCKETS[vst]
|
||||
if debug:
|
||||
print(f"start decoder for {sock_name}, {W}x{H}")
|
||||
|
||||
if nvidia:
|
||||
os.environ["NV_LOW_LATENCY"] = "3" # both bLowLatency and CUVID_PKT_ENDOFPICTURE
|
||||
sys.path += os.environ["LD_LIBRARY_PATH"].split(":")
|
||||
import PyNvCodec as nvc
|
||||
|
||||
nvDec = nvc.PyNvDecoder(W, H, nvc.PixelFormat.NV12, nvc.CudaVideoCodec.HEVC, 0)
|
||||
cc1 = nvc.ColorspaceConversionContext(nvc.ColorSpace.BT_709, nvc.ColorRange.JPEG)
|
||||
conv_yuv = nvc.PySurfaceConverter(W, H, nvc.PixelFormat.NV12, nvc.PixelFormat.YUV420, 0)
|
||||
nvDwn_yuv = nvc.PySurfaceDownloader(W, H, nvc.PixelFormat.YUV420, 0)
|
||||
img_yuv = np.ndarray((H*W//2*3), dtype=np.uint8)
|
||||
else:
|
||||
codec = av.CodecContext.create("hevc", "r")
|
||||
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
sock = messaging.sub_sock(sock_name, None, addr=addr, conflate=False)
|
||||
cnt = 0
|
||||
last_idx = -1
|
||||
seen_iframe = False
|
||||
|
||||
time_q = []
|
||||
while 1:
|
||||
msgs = messaging.drain_sock(sock, wait_for_one=True)
|
||||
for evt in msgs:
|
||||
evta = getattr(evt, evt.which())
|
||||
if debug and evta.idx.encodeId != 0 and evta.idx.encodeId != (last_idx+1):
|
||||
print("DROP PACKET!")
|
||||
last_idx = evta.idx.encodeId
|
||||
if not seen_iframe and not (evta.idx.flags & V4L2_BUF_FLAG_KEYFRAME):
|
||||
if debug:
|
||||
print("waiting for iframe")
|
||||
continue
|
||||
time_q.append(time.monotonic())
|
||||
network_latency = (int(time.time()*1e9) - evta.unixTimestampNanos)/1e6
|
||||
frame_latency = ((evta.idx.timestampEof/1e9) - (evta.idx.timestampSof/1e9))*1000
|
||||
process_latency = ((evt.logMonoTime/1e9) - (evta.idx.timestampEof/1e9))*1000
|
||||
|
||||
# put in header (first)
|
||||
if not seen_iframe:
|
||||
if nvidia:
|
||||
nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.header, dtype=np.uint8))
|
||||
else:
|
||||
codec.decode(av.packet.Packet(evta.header))
|
||||
seen_iframe = True
|
||||
|
||||
if nvidia:
|
||||
rawSurface = nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.data, dtype=np.uint8))
|
||||
if rawSurface.Empty():
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
convSurface = conv_yuv.Execute(rawSurface, cc1)
|
||||
nvDwn_yuv.DownloadSingleSurface(convSurface, img_yuv)
|
||||
else:
|
||||
frames = codec.decode(av.packet.Packet(evta.data))
|
||||
if len(frames) == 0:
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
assert len(frames) == 1
|
||||
img_yuv = frames[0].to_ndarray(format=av.video.format.VideoFormat('yuv420p')).flatten()
|
||||
uv_offset = H*W
|
||||
y = img_yuv[:uv_offset]
|
||||
uv = img_yuv[uv_offset:].reshape(2, -1).ravel('F')
|
||||
img_yuv = np.hstack((y, uv))
|
||||
|
||||
vipc_server.send(vst, img_yuv.data, cnt, int(time_q[0]*1e9), int(time.monotonic()*1e9))
|
||||
cnt += 1
|
||||
|
||||
pc_latency = (time.monotonic()-time_q[0])*1000
|
||||
time_q = time_q[1:]
|
||||
if debug:
|
||||
print(f"{len(msgs):2d} {evta.idx.encodeId:4d} {evt.logMonoTime/1e9:.3f} {evta.idx.timestampEof/1e6:.3f} \
|
||||
roll {frame_latency:6.2f} ms latency {process_latency:6.2f} ms + {network_latency:6.2f} ms + {pc_latency:6.2f} ms \
|
||||
= {process_latency+network_latency+pc_latency:6.2f} ms", len(evta.data), sock_name)
|
||||
|
||||
|
||||
class CompressedVipc:
|
||||
def __init__(self, addr, vision_streams, nvidia=False, debug=False):
|
||||
print("getting frame sizes")
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
sm = messaging.SubMaster([ENCODE_SOCKETS[s] for s in vision_streams], addr=addr)
|
||||
while min(sm.recv_frame.values()) == 0:
|
||||
sm.update(100)
|
||||
os.environ.pop("ZMQ")
|
||||
messaging.reset_context()
|
||||
|
||||
self.vipc_server = VisionIpcServer("camerad")
|
||||
for vst in vision_streams:
|
||||
ed = sm[ENCODE_SOCKETS[vst]]
|
||||
self.vipc_server.create_buffers(vst, 4, ed.width, ed.height)
|
||||
self.vipc_server.start_listener()
|
||||
|
||||
self.procs = []
|
||||
for vst in vision_streams:
|
||||
ed = sm[ENCODE_SOCKETS[vst]]
|
||||
p = multiprocessing.Process(target=decoder, args=(addr, self.vipc_server, vst, nvidia, ed.width, ed.height, debug))
|
||||
p.start()
|
||||
self.procs.append(p)
|
||||
|
||||
def join(self):
|
||||
for p in self.procs:
|
||||
p.join()
|
||||
|
||||
def kill(self):
|
||||
for p in self.procs:
|
||||
p.terminate()
|
||||
self.join()
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Decode video streams and broadcast on VisionIPC")
|
||||
parser.add_argument("addr", help="Address of comma three")
|
||||
parser.add_argument("--nvidia", action="store_true", help="Use nvidia instead of ffmpeg")
|
||||
parser.add_argument("--cams", default="0,1,2", help="Cameras to decode")
|
||||
parser.add_argument("--silent", action="store_true", help="Suppress debug output")
|
||||
args = parser.parse_args()
|
||||
|
||||
vision_streams = [
|
||||
VisionStreamType.VISION_STREAM_ROAD,
|
||||
VisionStreamType.VISION_STREAM_DRIVER,
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD,
|
||||
]
|
||||
|
||||
vsts = [vision_streams[int(x)] for x in args.cams.split(",")]
|
||||
cvipc = CompressedVipc(args.addr, vsts, args.nvidia, debug=(not args.silent))
|
||||
|
||||
# register exit handler
|
||||
signal.signal(signal.SIGINT, lambda sig, frame: cvipc.kill())
|
||||
|
||||
cvipc.join()
|
||||
@@ -0,0 +1,129 @@
|
||||
# tools/car_porting
|
||||
|
||||
Check out [this blog post](https://blog.comma.ai/how-to-write-a-car-port-for-openpilot/) for a high-level overview of porting a car.
|
||||
|
||||
## Useful car porting utilities
|
||||
|
||||
Testing car ports in your car is very time-consuming. Check out these utilities to do basic checks on your work before running it in your car.
|
||||
|
||||
### [Cabana](/tools/cabana/README.md)
|
||||
|
||||
View your car's CAN signals through DBC files, which openpilot uses to parse and create messages that talk to the car.
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> tools/cabana/cabana '1bbe6bf2d62f58a8|2022-07-14--17-11-43'
|
||||
```
|
||||
|
||||
### [tools/car_porting/auto_fingerprint.py](/tools/car_porting/auto_fingerprint.py)
|
||||
|
||||
Given a route and platform, automatically inserts FW fingerprints from the platform into the correct place in fingerprints.py
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> python3 tools/car_porting/auto_fingerprint.py '1bbe6bf2d62f58a8|2022-07-14--17-11-43' 'OUTBACK'
|
||||
Attempting to add fw version for: OUTBACK
|
||||
```
|
||||
|
||||
### [selfdrive/car/tests/test_car_interfaces.py](/selfdrive/car/tests/test_car_interfaces.py)
|
||||
|
||||
Finds common bugs for car interfaces, without even requiring a route.
|
||||
|
||||
|
||||
#### Example: Typo in signal name
|
||||
```bash
|
||||
> pytest selfdrive/car/tests/test_car_interfaces.py -k subaru # replace with the brand you are working on
|
||||
|
||||
=====================================================================
|
||||
FAILED selfdrive/car/tests/test_car_interfaces.py::TestCarInterfaces::test_car_interfaces_165_SUBARU_LEGACY_7TH_GEN - KeyError: 'CruiseControlOOPS'
|
||||
|
||||
```
|
||||
|
||||
### [tools/car_porting/test_car_model.py](/tools/car_porting/test_car_model.py)
|
||||
|
||||
Given a route, runs most of the car interface to check for common errors like missing signals, blocked panda messages, and safety mismatches.
|
||||
|
||||
#### Example: panda safety mismatch for gasPressed
|
||||
```bash
|
||||
> python3 tools/car_porting/test_car_model.py '4822a427b188122a|2023-08-14--16-22-21'
|
||||
|
||||
=====================================================================
|
||||
FAIL: test_panda_safety_carstate (__main__.CarModelTestCase.test_panda_safety_carstate)
|
||||
Assert that panda safety matches openpilot's carState
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/home/batman/xx/openpilot/openpilot/selfdrive/car/tests/test_models.py", line 380, in test_panda_safety_carstate
|
||||
self.assertFalse(len(failed_checks), f"panda safety doesn't agree with openpilot: {failed_checks}")
|
||||
AssertionError: 1 is not false : panda safety doesn't agree with openpilot: {'gasPressed': 116}
|
||||
```
|
||||
|
||||
## Jupyter notebooks
|
||||
|
||||
To use these notebooks, install Jupyter within your [openpilot virtual environment](/tools/README.md).
|
||||
|
||||
```bash
|
||||
uv pip install jupyter ipykernel
|
||||
```
|
||||
|
||||
Launching:
|
||||
|
||||
```bash
|
||||
jupyter notebook
|
||||
```
|
||||
|
||||
### [examples/subaru_steer_temp_fault.ipynb](/tools/car_porting/examples/subaru_steer_temp_fault.ipynb)
|
||||
|
||||
An example of searching through a database of segments for a specific condition, and plotting the results.
|
||||
|
||||
|
||||
|
||||
*a plot of the steer_warning vs steering angle, where we can see it is clearly caused by a large steering angle change*
|
||||
|
||||
### [examples/subaru_long_accel.ipynb](/tools/car_porting/examples/subaru_long_accel.ipynb)
|
||||
|
||||
An example of plotting the response of an actuator when it is active.
|
||||
|
||||
|
||||
|
||||
*a plot of the brake_pressure vs acceleration, where we can see it is a fairly linear response.*
|
||||
|
||||
### [examples/ford_vin_fingerprint.ipynb](/tools/car_porting/examples/ford_vin_fingerprint.ipynb)
|
||||
|
||||
In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.
|
||||
|
||||
```
|
||||
vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 00000000000XXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8F98NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True
|
||||
vin: 1FM5K7LC0MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False
|
||||
vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True
|
||||
vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 1FM5K8GC7NGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC8MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E31PRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
```
|
||||
|
||||
### [examples/find_segments_with_message.ipynb](/tools/car_porting/examples/find_segments_with_message.ipynb)
|
||||
|
||||
Searches for segments where a set of given CAN message IDs are present. In the example, we search for all messages
|
||||
used for CAN-based ignition detection.
|
||||
|
||||
```
|
||||
Match found: 46b21f1c5f7aa885/2024-01-23--15-19-34/20/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: a63a23c3e628f288/2023-11-05--18-36-20/8/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: ce31b7a998781ba8/2024-01-19--07-05-29/23/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2023-12-25--19-31-00/4/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2024-01-10--14-33-57/2/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: ae679616266f4096/2023-12-05--15-43-46/4/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2023-11-18--17-49-42/3/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2024-01-03--21-57-09/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-10--11-53-15/17/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-03--17-31-17/29/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-11-27--23-29-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
```
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
from openpilot.selfdrive.debug.format_fingerprints import format_brand_fw_versions
|
||||
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.fw_versions import MODEL_TO_BRAND, match_fw_to_car
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Auto fingerprint from a route")
|
||||
parser.add_argument("route", help="The route name to use")
|
||||
parser.add_argument("platform", help="The platform, or leave empty to auto-determine using fuzzy", default=None, nargs="?")
|
||||
args = parser.parse_args()
|
||||
|
||||
lr = LogReader(args.route, ReadMode.QLOG)
|
||||
CP = lr.first("carParams")
|
||||
assert CP is not None, "No carParams in route"
|
||||
|
||||
carPlatform = MIGRATION.get(CP.carFingerprint, CP.carFingerprint)
|
||||
|
||||
if args.platform is not None:
|
||||
platform = args.platform
|
||||
elif carPlatform != "MOCK":
|
||||
platform = carPlatform
|
||||
else:
|
||||
_, matches = match_fw_to_car(CP.carFw, CP.carVin, log=False)
|
||||
assert len(matches) == 1, f"Unable to auto-determine platform, matches: {matches}"
|
||||
platform = list(matches)[0]
|
||||
|
||||
print("Attempting to add fw version for:", platform)
|
||||
|
||||
fw_versions: dict[str, dict[tuple, list[bytes]]] = defaultdict(lambda: defaultdict(list))
|
||||
brand = MODEL_TO_BRAND[platform]
|
||||
|
||||
for fw in CP.carFw:
|
||||
if fw.brand == brand and not fw.logging:
|
||||
addr = fw.address
|
||||
subAddr = None if fw.subAddress == 0 else fw.subAddress
|
||||
key = (fw.ecu.raw, addr, subAddr)
|
||||
|
||||
fw_versions[platform][key].append(fw.fwVersion)
|
||||
|
||||
format_brand_fw_versions(brand, fw_versions)
|
||||
@@ -0,0 +1,232 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 85,
|
||||
"id": "facb8edc-9924-491a-a4dd-fe6135b0c6c4",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import all cars from opendbc\n",
|
||||
"\n",
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.values import PLATFORMS as TEST_PLATFORMS\n",
|
||||
"\n",
|
||||
"# Example: add additional platforms/segments to test outside of commaCarSegments\n",
|
||||
"\n",
|
||||
"EXTRA_SEGMENTS = {\n",
|
||||
" # \"81dd9e9fe256c397/0000001f--97c42cf98d\", # Volkswagen ID.4 test route, new car port, not in public dataset\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 86,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Searching 221 platforms\n",
|
||||
"No segments available for DODGE_DURANGO\n",
|
||||
"No segments available for FORD_RANGER_MK2\n",
|
||||
"No segments available for HOLDEN_ASTRA\n",
|
||||
"No segments available for CADILLAC_ATS\n",
|
||||
"No segments available for CHEVROLET_MALIBU\n",
|
||||
"No segments available for CADILLAC_XT4\n",
|
||||
"No segments available for CHEVROLET_VOLT_2019\n",
|
||||
"No segments available for CHEVROLET_TRAVERSE\n",
|
||||
"No segments available for GMC_YUKON\n",
|
||||
"No segments available for HONDA_ODYSSEY_CHN\n",
|
||||
"No segments available for HYUNDAI_KONA_2022\n",
|
||||
"No segments available for HYUNDAI_NEXO_1ST_GEN\n",
|
||||
"No segments available for GENESIS_GV70_ELECTRIFIED_1ST_GEN\n",
|
||||
"No segments available for GENESIS_G80_2ND_GEN_FL\n",
|
||||
"No segments available for RIVIAN_R1_GEN1\n",
|
||||
"No segments available for SUBARU_FORESTER_HYBRID\n",
|
||||
"No segments available for TESLA_MODEL_3\n",
|
||||
"No segments available for TESLA_MODEL_Y\n",
|
||||
"No segments available for TOYOTA_RAV4_PRIME\n",
|
||||
"No segments available for TOYOTA_SIENNA_4TH_GEN\n",
|
||||
"No segments available for LEXUS_LC_TSS2\n",
|
||||
"No segments available for VOLKSWAGEN_CADDY_MK3\n",
|
||||
"No segments available for VOLKSWAGEN_CRAFTER_MK2\n",
|
||||
"No segments available for VOLKSWAGEN_JETTA_MK6\n",
|
||||
"Searching 577 segments\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 3 # Increase this to search more segments\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_PLATFORMS)} platforms\")\n",
|
||||
"\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"No segments available for {platform}\")\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" all_segments = database[platform]\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"TEST_SEGMENTS.extend(EXTRA_SEGMENTS)\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_SEGMENTS)} segments\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "27a243c33de44498b2b946190df44b23",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"segments searched: 0%| | 0/577 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Match found: 0f53b336851e1384/2023-11-20--09-44-03/12/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7620ad20d3cefc64/2023-10-28--08-14-40/3/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 00d247a9bb1f9196/2023-11-06--13-33-17/9/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 120a432f63cb0de2/2023-10-30--20-01-34/1/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: b70b56b76a6217f2/2023-12-19--08-30-22/35/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 97e388680a6716ed/2024-01-17--10-15-13/9/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2137b01aa0ca63f9/2024-01-06--22-06-14/70/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 8fc6a1b72c8b1357/2023-11-06--07-50-05/8/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7e705eb5c27a49cc/2024-01-18--16-51-20/3/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2024-01-20--14-46-24/12/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2023-11-30--12-01-09/2/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9cad19e0efce3650/2024-01-26--10-24-52/27/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9db428338427dec2/2023-11-05--18-40-09/21/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: d50ada8ee55a5e74/2023-12-11--13-38-09/0/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 900dfa83b4addfe6/2023-12-30--19-20-08/28/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 20acda0eb23d7f23/2024-01-19--17-33-26/41/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 1cc3b46843cad2ca/2024-01-10--20-20-54/24/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2d9b6425552c52c1/2023-12-07--10-31-46/22/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-04--13-13-56/16/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2024-01-08--07-58-12/65/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-05--15-43-46/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-09--21-41-11/4/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 440a155809ba2b6d/2023-12-30--08-51-53/2/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-06--10-11-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/19/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-10--14-13-14/0/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/4/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-12-10--19-02-33/3/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 0b91b433b9332780/2023-12-28--14-02-49/4/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-11-09--07-28-12/1/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/59/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/82/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/106/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from tqdm.notebook import tqdm, tnrange\n",
|
||||
"\n",
|
||||
"# Example search for CAN ignition messages\n",
|
||||
"# Be careful when filtering by bus, account for odd harness arrangements on Honda/HKG\n",
|
||||
"\n",
|
||||
"BUSES_TO_SEARCH = [0, 1, 2]\n",
|
||||
"\n",
|
||||
"# Support for external Red Panda\n",
|
||||
"EXTERNAL_PANDA_BUSES = [bus + 4 for bus in BUSES_TO_SEARCH]\n",
|
||||
"\n",
|
||||
"MESSAGES_TO_FIND = {\n",
|
||||
" 0x1F1: \"GM CAN Ign\",\n",
|
||||
" 0x152: \"Rivian CAN Ign\",\n",
|
||||
" 0x221: \"Tesla 3/Y CAN Ign\",\n",
|
||||
" 0x9E: \"Mazda CAN Ign\",\n",
|
||||
" 0x3C0: \"VW CAN Ign\",\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"progress_bar = tnrange(len(TEST_SEGMENTS), desc=\"segments searched\")\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" progress_bar.update()\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_packets = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" matched_messages = set()\n",
|
||||
"\n",
|
||||
" for packet in can_packets:\n",
|
||||
" for msg in packet.can:\n",
|
||||
" if msg.address in MESSAGES_TO_FIND and msg.src in (BUSES_TO_SEARCH + EXTERNAL_PANDA_BUSES):\n",
|
||||
" # print(msg)\n",
|
||||
" matched_messages.add(msg.address)\n",
|
||||
"\n",
|
||||
" if len(matched_messages) > 0:\n",
|
||||
" message_names = [MESSAGES_TO_FIND[message] for message in matched_messages]\n",
|
||||
" print(f\"Match found: {segment:<45} {CP.carFingerprint:<38} {message_names}\")\n",
|
||||
"\n",
|
||||
" progress_bar.update()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"jupyter": {
|
||||
"is_executing": true
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Got 9 Ford cars from opendbc\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"\"\"\"In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.\"\"\"\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.ford.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"\n",
|
||||
"platforms = [c.value for c in CAR]\n",
|
||||
"print(f\"Got {len(platforms)} Ford cars from opendbc\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Adapted from https://github.com/commaai/openpilot/issues/31052#issuecomment-1902690083\n",
|
||||
"\n",
|
||||
"MODEL_YEAR_CODES = {'M': 2021, 'N': 2022, 'P': 2023, 'R': 2024, 'S': 2025}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"F150_CODES = ['F1C', 'F1E', 'W1C', 'W1E', 'X1C', 'X1E', 'W1R', 'W1P', 'W1S', 'W1T']\n",
|
||||
"LIGHTNING_CODES = ['L', 'V']\n",
|
||||
"MACHE_CODES = ['K1R', 'K1S', 'K2S', 'K3R', 'K3S', 'K4S']\n",
|
||||
"\n",
|
||||
"FORD_VIN_START = ['1FT', '3FM', '5LM']\n",
|
||||
"\n",
|
||||
"def ford_vin_fingerprint(vin): # Check if it's a Ford vehicle and determine the model\n",
|
||||
" vin_positions_567 = vin[4:7]\n",
|
||||
"\n",
|
||||
" if vin.startswith('1FT'):\n",
|
||||
" if vin_positions_567 in F150_CODES:\n",
|
||||
" if vin[7] in LIGHTNING_CODES:\n",
|
||||
" return f\"FORD F-150 LIGHTNING 1ST GEN\"\n",
|
||||
" else:\n",
|
||||
" return f\"FORD F-150 14TH GEN\"\n",
|
||||
" elif vin.startswith('3FM'):\n",
|
||||
" if vin_positions_567 in MACHE_CODES:\n",
|
||||
" return f\"FORD MUSTANG MACH-E 1ST GEN\"\n",
|
||||
" elif vin.startswith('5LM'):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" return \"mock\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 287 segments for platform FORD_BRONCO_SPORT_MK1, sampling 5 segments\n",
|
||||
"Got 137 segments for platform FORD_ESCAPE_MK4, sampling 5 segments\n",
|
||||
"Got 1041 segments for platform FORD_EXPLORER_MK6, sampling 5 segments\n",
|
||||
"Got 5 segments for platform FORD_F_150_MK14, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_F_150_LIGHTNING_MK1, sampling 3 segments\n",
|
||||
"Got 56 segments for platform FORD_FOCUS_MK4, sampling 5 segments\n",
|
||||
"Got 637 segments for platform FORD_MAVERICK_MK1, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_MUSTANG_MACH_E_MK1, sampling 3 segments\n",
|
||||
"Skipping platform: FORD_RANGER_MK2, no data available\n",
|
||||
"Segment collection finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5\n",
|
||||
"\n",
|
||||
"VINS_TO_CHECK = set()\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in platforms:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" segments = random.sample(all_segments, NUM_SEGMENTS)\n",
|
||||
"\n",
|
||||
" for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if \"FORD\" not in CP.carFingerprint:\n",
|
||||
" print(segment, CP.carFingerprint)\n",
|
||||
" VINS_TO_CHECK.add((CP.carVin, CP.carFingerprint))\n",
|
||||
"\n",
|
||||
"print(\"Segment collection finished\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 00000000000XXXXXX real platform: FORD F-150 14TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True\n",
|
||||
"vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True\n",
|
||||
"vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E33NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC1LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E3XPRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for vin, real_fingerprint in VINS_TO_CHECK:\n",
|
||||
" determined_fingerprint = ford_vin_fingerprint(vin)\n",
|
||||
" print(f\"vin: {vin} real platform: {real_fingerprint: <30} determined platform: {determined_fingerprint: <30} correct: {real_fingerprint == determined_fingerprint}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,277 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"id": "228a6736-de31-4255-9d72-a6ff391b968d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Found 6 qualifying vehicles:\n",
|
||||
" KIA_EV6\n",
|
||||
" HYUNDAI_KONA_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_5\n",
|
||||
" KIA_NIRO_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_6\n",
|
||||
" GENESIS_GV60_EV_1ST_GEN\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.hyundai.values import CAR, HyundaiFlags\n",
|
||||
"from opendbc.car.hyundai.fingerprints import FW_VERSIONS\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) & set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD electric vehicles only\n",
|
||||
"#TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) - set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD hybrid and ICE vehicles only\n",
|
||||
"\n",
|
||||
"print(f\"Found {len(TEST_PLATFORMS)} qualifying vehicles:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" print(f\" {platform}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 1300 segments for platform KIA_EV6, sampling 5 segments\n",
|
||||
"Got 9 segments for platform HYUNDAI_KONA_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 1570 segments for platform HYUNDAI_IONIQ_5, sampling 5 segments\n",
|
||||
"Got 34 segments for platform KIA_NIRO_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 974 segments for platform HYUNDAI_IONIQ_6, sampling 5 segments\n",
|
||||
"Got 157 segments for platform GENESIS_GV60_EV_1ST_GEN, sampling 5 segments\n",
|
||||
"Collected 30 segments for analysis\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.hyundai.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5 # TODO: Increase this to search more segments\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" assert(platform in database)\n",
|
||||
" #if platform not in database:\n",
|
||||
" # print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" # continue\n",
|
||||
" \n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"print(f\"Collected {len(TEST_SEGMENTS)} segments for analysis\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Analyzing segment ff2bd20623fcaeaa/2023-11-26--16-27-04/5/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 3f1a6480f940cf9a/2024-01-10--23-06-11/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b0a9998109ed0053/2023-12-15--11-10-18/12/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 6e14aa2ed85025df/2023-11-15--13-18-12/24/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment a43f21df3a1ca12d/2024-01-25--08-56-22/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--13-49-24/11/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-11-26--12-31-18/17/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--11-51-44/3/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-08-27--09-32-14/13/s for HYUNDAI KONA 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2024-01-25--15-07-04/24/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 223780ed74116bc2/2023-11-16--09-44-56/15/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment ba9951252624f37d/2024-01-20--22-33-23/118/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 8379b28e51ceb3b1/2023-11-09--23-21-58/92/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26fac43e27cd6091/2023-11-06--12-23-21/9/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=1.0\n",
|
||||
" ACCELERATOR gear=0.0\n",
|
||||
"Analyzing segment 5edb897a0ec7a477/2024-01-13--20-41-36/101/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 66cf8ea23b7c2789/2023-12-04--13-48-53/5/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-10--20-31-37/2/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-03--21-08-30/14/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-11-07--19-52-23/0/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-07-12--19-25-18/6/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-18--07-59-26/11/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 0ad7facc77922c3e/2023-12-21--17-47-25/18/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26968f888e7330d3/2024-01-02--11-18-37/8/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-27--21-03-24/33/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment df7fdd56970d90fe/2024-01-07--01-04-39/26/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--14-45-44/0/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--20-57-09/8/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--12-52-38/5/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-19--19-57-52/47/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--13-01-23/1/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analysis finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.hyundai.values import DBC\n",
|
||||
"from opendbc.car.hyundai.hyundaicanfd import CanBus\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"message_names = [\"GEAR_SHIFTER\", \"ACCELERATOR\", \"GEAR\", \"GEAR_ALT\", \"GEAR_ALT_2\"]\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" parser_messages = []\n",
|
||||
" for name in message_names:\n",
|
||||
" parser_messages.append((name, 0))\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], parser_messages, CanBus(CP).ECAN)\n",
|
||||
"\n",
|
||||
" parsed_message_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" parsed_message_history.append(copy.copy(cp.vl))\n",
|
||||
"\n",
|
||||
" print(f\"Analyzing segment {segment:<44} for {CP.carFingerprint}\")\n",
|
||||
" for name in message_names:\n",
|
||||
" if parsed_message_history[0][name][\"CHECKSUM\"] != 0: # Message is present for this segment\n",
|
||||
" gear_prev = parsed_message_history[0][name][\"GEAR\"]\n",
|
||||
" print(f\" {name:<15} gear={gear_prev}\")\n",
|
||||
" for i, parsed_messages in enumerate(parsed_message_history):\n",
|
||||
" gear = parsed_messages[name][\"GEAR\"]\n",
|
||||
" if gear != gear_prev:\n",
|
||||
" print(f\" *** Signal transition found! ***\")\n",
|
||||
" examples.append(i)\n",
|
||||
" gear_prev = gear\n",
|
||||
"\n",
|
||||
"print(f\"Analysis finished\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.subaru.values import CAR, SubaruFlags\n",
|
||||
"from opendbc.car.subaru.fingerprints import FW_VERSIONS\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR) - CAR.with_flags(SubaruFlags.PREGLOBAL)\n",
|
||||
"\n",
|
||||
"Ecu = structs.CarParams.Ecu\n",
|
||||
"\n",
|
||||
"FW_BY_ECU = {platform: {ecu: versions for (ecu, addr, sub_addr), versions in fw_versions.items()} for platform, fw_versions in FW_VERSIONS.items()}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PLATFORM_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 0: {\n",
|
||||
" b'\\xa5': [CAR.SUBARU_ASCENT, CAR.SUBARU_ASCENT_2023],\n",
|
||||
" b'\\xa2': [CAR.SUBARU_IMPREZA, CAR.SUBARU_IMPREZA_2020, CAR.SUBARU_CROSSTREK_HYBRID],\n",
|
||||
" b'\\xa1': [CAR.SUBARU_OUTBACK, CAR.SUBARU_LEGACY, CAR.SUBARU_OUTBACK_2023],\n",
|
||||
" b'\\xa3': [CAR.SUBARU_FORESTER, CAR.SUBARU_FORESTER_HYBRID, CAR.SUBARU_FORESTER_2022],\n",
|
||||
" b'z': [CAR.SUBARU_IMPREZA],\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"YEAR_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 2: {\n",
|
||||
" b'\\x18': 2018,\n",
|
||||
" b'\\x19': 2019,\n",
|
||||
" b'\\x20': 2020,\n",
|
||||
" b'\\x21': 2021,\n",
|
||||
" b'\\x22': 2022,\n",
|
||||
" b'\\x23': 2023,\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_codes(platforms, codes):\n",
|
||||
" results = []\n",
|
||||
" for platform in platforms:\n",
|
||||
" for ecu in codes:\n",
|
||||
" for i in codes[ecu]:\n",
|
||||
" if isinstance(i, tuple):\n",
|
||||
" j = slice(i[0], i[1])\n",
|
||||
" else:\n",
|
||||
" j = slice(i, i+1)\n",
|
||||
" for version in FW_BY_ECU[platform][ecu]:\n",
|
||||
" code = version[j]\n",
|
||||
" if code not in codes[ecu][i]:\n",
|
||||
" print(f\"{platform} {code.hex()} not in {codes[ecu][i].keys()}\")\n",
|
||||
" else:\n",
|
||||
" results.append((platform, codes[ecu][i][code]))\n",
|
||||
" return results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 2e not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 3f not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2021 years=[2020]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2018 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2021 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_year_code(platform, year):\n",
|
||||
" car_docs = CAR(platform).config.car_docs\n",
|
||||
" if isinstance(car_docs, list):\n",
|
||||
" car_docs = car_docs[0]\n",
|
||||
" years = [int(y) for y in car_docs.year_list]\n",
|
||||
" correct_year = year in years\n",
|
||||
" print(f\"{correct_year=!s: <6} {platform=: <32} {year=: <5} {years=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, YEAR_CODES)\n",
|
||||
"for platform, year in codes:\n",
|
||||
" test_year_code(platform, year)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_HYBRID platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT_2023 platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_platform_code(platform, platforms):\n",
|
||||
" platforms = [str(p) for p in platforms]\n",
|
||||
" in_possible_platforms = platform in platforms\n",
|
||||
" print(f\"{in_possible_platforms=!s: <6} {platform=: <32} {platforms=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, PLATFORM_CODES)\n",
|
||||
"for platform, possible_platforms in codes:\n",
|
||||
" test_platform_code(platform, possible_platforms)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"segments = [\n",
|
||||
" \"d9df6f87e8feff94|2023-03-28--17-41-10/1:12\"\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we plot the relationship between Cruise_Brake and Acceleration for stock eyesight.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"ES_Distance\", 20),\n",
|
||||
" (\"ES_Brake\", 20),\n",
|
||||
" (\"ES_Status\", 20),\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, 1)\n",
|
||||
"\n",
|
||||
" es_distance_history = []\n",
|
||||
" es_status_history = []\n",
|
||||
" es_brake_history = []\n",
|
||||
" acceleration_history = []\n",
|
||||
"\n",
|
||||
" last_acc = 0\n",
|
||||
"\n",
|
||||
" for msg in lr:\n",
|
||||
" if msg.which() == \"can\":\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" es_distance_history.append(copy.copy(cp.vl[\"ES_Distance\"]))\n",
|
||||
" es_brake_history.append(copy.copy(cp.vl[\"ES_Brake\"]))\n",
|
||||
" es_status_history.append(copy.copy(cp.vl[\"ES_Status\"]))\n",
|
||||
"\n",
|
||||
" acceleration_history.append(last_acc)\n",
|
||||
"\n",
|
||||
" if msg.which() == \"carState\":\n",
|
||||
" last_acc = msg.carState.aEgo"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def process(history, func):\n",
|
||||
" return np.array([func(h) for h in history])\n",
|
||||
"\n",
|
||||
"cruise_activated = process(es_status_history, lambda es_status: es_status[\"Cruise_Activated\"])\n",
|
||||
"cruise_throttle = process(es_distance_history, lambda es_distance: es_distance[\"Cruise_Throttle\"])\n",
|
||||
"cruise_rpm = process(es_status_history, lambda es_status: es_status[\"Cruise_RPM\"])\n",
|
||||
"cruise_brake = process(es_brake_history, lambda es_brake: es_brake[\"Brake_Pressure\"])\n",
|
||||
"acceleration = process(acceleration_history, lambda acc: acc)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"valid_brake = (cruise_activated==1) & (cruise_brake>0) # only when cruise is activated and eyesight is braking\n",
|
||||
"\n",
|
||||
"ax = plt.figure().add_subplot()\n",
|
||||
"\n",
|
||||
"ax.set_title(\"Brake_Pressure vs Acceleration\")\n",
|
||||
"ax.set_xlabel(\"Brake_Pessure\")\n",
|
||||
"ax.set_ylabel(\"Acceleration\")\n",
|
||||
"ax.scatter(cruise_brake[valid_brake], -acceleration[valid_brake])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# An example of searching through a database of segments for a specific condition, and plotting the results.\n",
|
||||
"\n",
|
||||
"segments = [\n",
|
||||
" \"c3d1ccb52f5f9d65|2023-07-22--01-23-20/6:10\",\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import CanBus, DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we search for positive transitions of Steer_Warning, which indicate that the EPS\n",
|
||||
"has stopped responding to our messages. This analysis would allow you to find the cause of these\n",
|
||||
"steer warnings and potentially work around them.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"Steering_Torque\", 50)\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, CanBus.main)\n",
|
||||
"\n",
|
||||
" steering_torque_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" steering_torque_history.append(copy.copy(cp.vl[\"Steering_Torque\"]))\n",
|
||||
"\n",
|
||||
" steer_warning_last = False\n",
|
||||
" for i, steering_torque_msg in enumerate(steering_torque_history):\n",
|
||||
" steer_warning = steering_torque_msg[\"Steer_Warning\"]\n",
|
||||
"\n",
|
||||
" steer_angle = steering_torque_msg[\"Steering_Angle\"]\n",
|
||||
"\n",
|
||||
" if steer_warning and not steer_warning_last: # positive transition of \"Steer_Warning\"\n",
|
||||
" examples.append(i)\n",
|
||||
"\n",
|
||||
" steer_warning_last = steer_warning\n",
|
||||
"\n",
|
||||
" FRAME_DELTA = 100 # plot this many frames around the positive transition\n",
|
||||
"\n",
|
||||
" for example in examples:\n",
|
||||
" fig, axs = plt.subplots(2)\n",
|
||||
"\n",
|
||||
" min_frame = int(example-FRAME_DELTA/2)\n",
|
||||
" max_frame = int(example+FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" steering_angle_history = [msg[\"Steering_Angle\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
" steering_warning_history = [msg[\"Steer_Warning\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
"\n",
|
||||
" xs = np.arange(-FRAME_DELTA/2, FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" axs[0].plot(xs, steering_angle_history)\n",
|
||||
" axs[0].set_ylabel(\"Steering Angle (deg)\")\n",
|
||||
" axs[1].plot(xs, steering_warning_history)\n",
|
||||
" axs[1].set_ylabel(\"Steer Warning\")\n",
|
||||
"\n",
|
||||
" plt.show()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import sys
|
||||
import unittest # noqa: TID251
|
||||
|
||||
from opendbc.car.tests.routes import CarTestRoute
|
||||
from openpilot.selfdrive.car.tests.test_models import TestCarModel
|
||||
from openpilot.tools.lib.route import SegmentRange
|
||||
|
||||
|
||||
def create_test_models_suite(routes: list[CarTestRoute]) -> unittest.TestSuite:
|
||||
test_suite = unittest.TestSuite()
|
||||
for test_route in routes:
|
||||
# create new test case and discover tests
|
||||
test_case_args = {"platform": test_route.car_model, "test_route": test_route}
|
||||
CarModelTestCase = type("CarModelTestCase", (TestCarModel,), test_case_args)
|
||||
test_suite.addTest(unittest.TestLoader().loadTestsFromTestCase(CarModelTestCase))
|
||||
return test_suite
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Test any route against common issues with a new car port. " +
|
||||
"Uses selfdrive/car/tests/test_models.py")
|
||||
parser.add_argument("route_or_segment_name", help="Specify route to run tests on")
|
||||
parser.add_argument("--car", help="Specify car model for test route")
|
||||
args = parser.parse_args()
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
|
||||
sr = SegmentRange(args.route_or_segment_name)
|
||||
|
||||
test_routes = [CarTestRoute(sr.route_name, args.car, segment=seg_idx) for seg_idx in sr.seg_idxs]
|
||||
test_suite = create_test_models_suite(test_routes)
|
||||
|
||||
unittest.TextTestRunner().run(test_suite)
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# Increase the pip timeout to handle TimeoutError
|
||||
export PIP_DEFAULT_TIMEOUT=200
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT="$DIR"/../
|
||||
cd "$ROOT"
|
||||
|
||||
if ! command -v "uv" > /dev/null 2>&1; then
|
||||
echo "installing uv..."
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
UV_BIN="$HOME/.local/bin"
|
||||
PATH="$UV_BIN:$PATH"
|
||||
fi
|
||||
|
||||
echo "updating uv..."
|
||||
# ok to fail, can also fail due to installing with brew
|
||||
uv self update || true
|
||||
|
||||
echo "installing python packages..."
|
||||
uv sync --frozen --all-extras
|
||||
source .venv/bin/activate
|
||||
|
||||
echo "PYTHONPATH=${PWD}" > "$ROOT"/.env
|
||||
if [[ "$(uname)" == 'Darwin' ]]; then
|
||||
echo "# msgq doesn't work on mac" >> "$ROOT"/.env
|
||||
echo "export ZMQ=1" >> "$ROOT"/.env
|
||||
echo "export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" >> "$ROOT"/.env
|
||||
fi
|
||||
Executable
+125
@@ -0,0 +1,125 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
SUDO=""
|
||||
|
||||
# Use sudo if not root
|
||||
if [[ ! $(id -u) -eq 0 ]]; then
|
||||
if [[ -z $(which sudo) ]]; then
|
||||
echo "Please install sudo or run as root"
|
||||
exit 1
|
||||
fi
|
||||
SUDO="sudo"
|
||||
fi
|
||||
|
||||
# Check if stdin is open
|
||||
if [ -t 0 ]; then
|
||||
INTERACTIVE=1
|
||||
fi
|
||||
|
||||
# Install common packages
|
||||
function install_ubuntu_common_requirements() {
|
||||
$SUDO apt-get update
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
clang \
|
||||
build-essential \
|
||||
gcc-arm-none-eabi \
|
||||
liblzma-dev \
|
||||
capnproto \
|
||||
libcapnp-dev \
|
||||
curl \
|
||||
libcurl4-openssl-dev \
|
||||
git \
|
||||
git-lfs \
|
||||
ffmpeg \
|
||||
libavformat-dev \
|
||||
libavcodec-dev \
|
||||
libavdevice-dev \
|
||||
libavutil-dev \
|
||||
libavfilter-dev \
|
||||
libbz2-dev \
|
||||
libeigen3-dev \
|
||||
libffi-dev \
|
||||
libglew-dev \
|
||||
libgles2-mesa-dev \
|
||||
libglfw3-dev \
|
||||
libglib2.0-0 \
|
||||
libjpeg-dev \
|
||||
libqt5charts5-dev \
|
||||
libncurses5-dev \
|
||||
libssl-dev \
|
||||
libusb-1.0-0-dev \
|
||||
libzmq3-dev \
|
||||
libzstd-dev \
|
||||
libsqlite3-dev \
|
||||
libsystemd-dev \
|
||||
locales \
|
||||
opencl-headers \
|
||||
ocl-icd-libopencl1 \
|
||||
ocl-icd-opencl-dev \
|
||||
portaudio19-dev \
|
||||
qttools5-dev-tools \
|
||||
libqt5svg5-dev \
|
||||
libqt5serialbus5-dev \
|
||||
libqt5x11extras5-dev \
|
||||
libqt5opengl5-dev
|
||||
}
|
||||
|
||||
# Install Ubuntu 24.04 LTS packages
|
||||
function install_ubuntu_lts_latest_requirements() {
|
||||
install_ubuntu_common_requirements
|
||||
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
g++-12 \
|
||||
qtbase5-dev \
|
||||
qtchooser \
|
||||
qt5-qmake \
|
||||
qtbase5-dev-tools \
|
||||
python3-dev \
|
||||
python3-venv
|
||||
}
|
||||
|
||||
# Detect OS using /etc/os-release file
|
||||
if [ -f "/etc/os-release" ]; then
|
||||
source /etc/os-release
|
||||
case "$VERSION_CODENAME" in
|
||||
"jammy" | "kinetic" | "noble")
|
||||
install_ubuntu_lts_latest_requirements
|
||||
;;
|
||||
*)
|
||||
echo "$ID $VERSION_ID is unsupported. This setup script is written for Ubuntu 24.04."
|
||||
read -p "Would you like to attempt installation anyway? " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
exit 1
|
||||
fi
|
||||
install_ubuntu_lts_latest_requirements
|
||||
esac
|
||||
|
||||
if [[ -d "/etc/udev/rules.d/" ]]; then
|
||||
# Setup jungle udev rules
|
||||
$SUDO tee /etc/udev/rules.d/12-panda_jungle.rules > /dev/null <<EOF
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
|
||||
EOF
|
||||
|
||||
# Setup panda udev rules
|
||||
$SUDO tee /etc/udev/rules.d/11-panda.rules > /dev/null <<EOF
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0483", ATTRS{idProduct}=="df11", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
EOF
|
||||
|
||||
$SUDO udevadm control --reload-rules && $SUDO udevadm trigger || true
|
||||
fi
|
||||
|
||||
else
|
||||
echo "No /etc/os-release in the system. Make sure you're running on Ubuntu, or similar."
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,53 @@
|
||||
# Joystick
|
||||
|
||||
**Hardware needed**: device running openpilot, laptop, joystick (optional)
|
||||
|
||||
With joystick_control, you can connect your laptop to your comma device over the network and debug controls using a joystick or keyboard.
|
||||
joystick_control uses [inputs](https://pypi.org/project/inputs) which supports many common gamepads and joysticks.
|
||||
|
||||
## Usage
|
||||
|
||||
The car must be off, and openpilot must be offroad before starting `joystick_control`.
|
||||
|
||||
### Using a keyboard
|
||||
|
||||
SSH into your comma device and start joystick_control with the following command:
|
||||
|
||||
```shell
|
||||
tools/joystick/joystick_control.py --keyboard
|
||||
```
|
||||
|
||||
The available buttons and axes will print showing their key mappings. In general, the WASD keys control gas and brakes and steering torque in 5% increments.
|
||||
|
||||
### Joystick on your comma three
|
||||
|
||||
Plug the joystick into your comma three aux USB-C port. Then, SSH into the device and start `joystick_control.py`.
|
||||
|
||||
### Joystick on your laptop
|
||||
|
||||
In order to use a joystick over the network, we need to run joystick_control locally from your laptop and have it send `testJoystick` packets over the network to the comma device.
|
||||
|
||||
1. Connect a joystick to your PC.
|
||||
2. Connect your laptop to your comma device's hotspot and open a new SSH shell. Since joystick_control is being run on your laptop, we need to write a parameter to let controlsd know to start in joystick debug mode:
|
||||
```shell
|
||||
# on your comma device
|
||||
echo -n "1" > /data/params/d/JoystickDebugMode
|
||||
```
|
||||
3. Run bridge with your laptop's IP address. This republishes the `testJoystick` packets sent from your laptop so that openpilot can receive them:
|
||||
```shell
|
||||
# on your comma device
|
||||
cereal/messaging/bridge {LAPTOP_IP} testJoystick
|
||||
```
|
||||
4. Start joystick_control on your laptop in ZMQ mode.
|
||||
```shell
|
||||
# on your laptop
|
||||
export ZMQ=1
|
||||
tools/joystick/joystick_control.py
|
||||
```
|
||||
|
||||
---
|
||||
Now start your car and openpilot should go into joystick mode with an alert on startup! The status of the axes will display on the alert, while button statuses print in the shell.
|
||||
|
||||
Make sure the conditions are met in the panda to allow controls (e.g. cruise control engaged). You can also make a modification to the panda code to always allow controls.
|
||||
|
||||
|
||||
Executable
+147
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import argparse
|
||||
import threading
|
||||
import numpy as np
|
||||
from inputs import UnpluggedError, get_gamepad
|
||||
|
||||
from cereal import messaging
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.common.realtime import Ratekeeper
|
||||
from openpilot.system.hardware import HARDWARE
|
||||
from openpilot.tools.lib.kbhit import KBHit
|
||||
|
||||
EXPO = 0.4
|
||||
|
||||
|
||||
class Keyboard:
|
||||
def __init__(self):
|
||||
self.kb = KBHit()
|
||||
self.axis_increment = 0.05 # 5% of full actuation each key press
|
||||
self.axes_map = {'w': 'gb', 's': 'gb',
|
||||
'a': 'steer', 'd': 'steer'}
|
||||
self.axes_values = {'gb': 0., 'steer': 0.}
|
||||
self.axes_order = ['gb', 'steer']
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
key = self.kb.getch().lower()
|
||||
self.cancel = False
|
||||
if key == 'r':
|
||||
self.axes_values = {ax: 0. for ax in self.axes_values}
|
||||
elif key == 'c':
|
||||
self.cancel = True
|
||||
elif key in self.axes_map:
|
||||
axis = self.axes_map[key]
|
||||
incr = self.axis_increment if key in ['w', 'a'] else -self.axis_increment
|
||||
self.axes_values[axis] = float(np.clip(self.axes_values[axis] + incr, -1, 1))
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class Joystick:
|
||||
def __init__(self):
|
||||
# This class supports a PlayStation 5 DualSense controller on the comma 3X
|
||||
# TODO: find a way to get this from API or detect gamepad/PC, perhaps "inputs" doesn't support it
|
||||
self.cancel_button = 'BTN_NORTH' # BTN_NORTH=X/triangle
|
||||
if HARDWARE.get_device_type() == 'pc':
|
||||
accel_axis = 'ABS_Z'
|
||||
steer_axis = 'ABS_RX'
|
||||
# TODO: once the longcontrol API is finalized, we can replace this with outputting gas/brake and steering
|
||||
self.flip_map = {'ABS_RZ': accel_axis}
|
||||
else:
|
||||
accel_axis = 'ABS_RX'
|
||||
steer_axis = 'ABS_Z'
|
||||
self.flip_map = {'ABS_RY': accel_axis}
|
||||
|
||||
self.min_axis_value = {accel_axis: 0., steer_axis: 0.}
|
||||
self.max_axis_value = {accel_axis: 255., steer_axis: 255.}
|
||||
self.axes_values = {accel_axis: 0., steer_axis: 0.}
|
||||
self.axes_order = [accel_axis, steer_axis]
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
try:
|
||||
joystick_event = get_gamepad()[0]
|
||||
except (OSError, UnpluggedError):
|
||||
self.axes_values = {ax: 0. for ax in self.axes_values}
|
||||
return False
|
||||
|
||||
event = (joystick_event.code, joystick_event.state)
|
||||
|
||||
# flip left trigger to negative accel
|
||||
if event[0] in self.flip_map:
|
||||
event = (self.flip_map[event[0]], -event[1])
|
||||
|
||||
if event[0] == self.cancel_button:
|
||||
if event[1] == 1:
|
||||
self.cancel = True
|
||||
elif event[1] == 0: # state 0 is falling edge
|
||||
self.cancel = False
|
||||
elif event[0] in self.axes_values:
|
||||
self.max_axis_value[event[0]] = max(event[1], self.max_axis_value[event[0]])
|
||||
self.min_axis_value[event[0]] = min(event[1], self.min_axis_value[event[0]])
|
||||
|
||||
norm = -float(np.interp(event[1], [self.min_axis_value[event[0]], self.max_axis_value[event[0]]], [-1., 1.]))
|
||||
norm = norm if abs(norm) > 0.03 else 0. # center can be noisy, deadzone of 3%
|
||||
self.axes_values[event[0]] = EXPO * norm ** 3 + (1 - EXPO) * norm # less action near center for fine control
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def send_thread(joystick):
|
||||
pm = messaging.PubMaster(['testJoystick'])
|
||||
|
||||
rk = Ratekeeper(100, print_delay_threshold=None)
|
||||
|
||||
while True:
|
||||
if rk.frame % 20 == 0:
|
||||
print('\n' + ', '.join(f'{name}: {round(v, 3)}' for name, v in joystick.axes_values.items()))
|
||||
|
||||
joystick_msg = messaging.new_message('testJoystick')
|
||||
joystick_msg.valid = True
|
||||
joystick_msg.testJoystick.axes = [joystick.axes_values[ax] for ax in joystick.axes_order]
|
||||
|
||||
pm.send('testJoystick', joystick_msg)
|
||||
|
||||
rk.keep_time()
|
||||
|
||||
|
||||
def joystick_control_thread(joystick):
|
||||
Params().put_bool('JoystickDebugMode', True)
|
||||
threading.Thread(target=send_thread, args=(joystick,), daemon=True).start()
|
||||
while True:
|
||||
joystick.update()
|
||||
|
||||
|
||||
def main():
|
||||
joystick_control_thread(Joystick())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Publishes events from your joystick to control your car.\n' +
|
||||
'openpilot must be offroad before starting joystick_control. This tool supports ' +
|
||||
'a PlayStation 5 DualSense controller on the comma 3X.',
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument('--keyboard', action='store_true', help='Use your keyboard instead of a joystick')
|
||||
args = parser.parse_args()
|
||||
|
||||
if not Params().get_bool("IsOffroad") and "ZMQ" not in os.environ:
|
||||
print("The car must be off before running joystick_control.")
|
||||
exit()
|
||||
|
||||
print()
|
||||
if args.keyboard:
|
||||
print('Gas/brake control: `W` and `S` keys')
|
||||
print('Steering control: `A` and `D` keys')
|
||||
print('Buttons')
|
||||
print('- `R`: Resets axes')
|
||||
print('- `C`: Cancel cruise control')
|
||||
else:
|
||||
print('Using joystick, make sure to run cereal/messaging/bridge on your device if running over the network!')
|
||||
print('If not running on a comma device, the mapping may need to be adjusted.')
|
||||
|
||||
joystick = Keyboard() if args.keyboard else Joystick()
|
||||
joystick_control_thread(joystick)
|
||||
Executable
+80
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
from cereal import messaging, car
|
||||
from opendbc.car.vehicle_model import VehicleModel
|
||||
from openpilot.common.realtime import DT_CTRL, Ratekeeper
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
|
||||
LongCtrlState = car.CarControl.Actuators.LongControlState
|
||||
MAX_LAT_ACCEL = 2.5
|
||||
|
||||
|
||||
def joystickd_thread():
|
||||
params = Params()
|
||||
cloudlog.info("joystickd is waiting for CarParams")
|
||||
CP = messaging.log_from_bytes(params.get("CarParams", block=True), car.CarParams)
|
||||
VM = VehicleModel(CP)
|
||||
|
||||
sm = messaging.SubMaster(['carState', 'onroadEvents', 'liveParameters', 'selfdriveState', 'testJoystick'], frequency=1. / DT_CTRL)
|
||||
pm = messaging.PubMaster(['carControl', 'controlsState'])
|
||||
|
||||
rk = Ratekeeper(100, print_delay_threshold=None)
|
||||
while 1:
|
||||
sm.update(0)
|
||||
|
||||
cc_msg = messaging.new_message('carControl')
|
||||
cc_msg.valid = True
|
||||
CC = cc_msg.carControl
|
||||
CC.enabled = sm['selfdriveState'].enabled
|
||||
CC.latActive = sm['selfdriveState'].active and not sm['carState'].steerFaultTemporary and not sm['carState'].steerFaultPermanent
|
||||
CC.longActive = CC.enabled and not any(e.overrideLongitudinal for e in sm['onroadEvents']) and CP.openpilotLongitudinalControl
|
||||
CC.cruiseControl.cancel = sm['carState'].cruiseState.enabled and (not CC.enabled or not CP.pcmCruise)
|
||||
CC.hudControl.leadDistanceBars = 2
|
||||
|
||||
actuators = CC.actuators
|
||||
|
||||
# reset joystick if it hasn't been received in a while
|
||||
should_reset_joystick = sm.recv_frame['testJoystick'] == 0 or (sm.frame - sm.recv_frame['testJoystick'])*DT_CTRL > 0.2
|
||||
|
||||
if not should_reset_joystick:
|
||||
joystick_axes = sm['testJoystick'].axes
|
||||
else:
|
||||
joystick_axes = [0.0, 0.0]
|
||||
|
||||
if CC.longActive:
|
||||
actuators.accel = 4.0 * float(np.clip(joystick_axes[0], -1, 1))
|
||||
actuators.longControlState = LongCtrlState.pid if sm['carState'].vEgo > CP.vEgoStopping else LongCtrlState.stopping
|
||||
|
||||
if CC.latActive:
|
||||
max_curvature = MAX_LAT_ACCEL / max(sm['carState'].vEgo ** 2, 5)
|
||||
max_angle = math.degrees(VM.get_steer_from_curvature(max_curvature, sm['carState'].vEgo, sm['liveParameters'].roll))
|
||||
|
||||
actuators.torque = float(np.clip(joystick_axes[1], -1, 1))
|
||||
actuators.steeringAngleDeg, actuators.curvature = actuators.torque * max_angle, actuators.torque * -max_curvature
|
||||
|
||||
pm.send('carControl', cc_msg)
|
||||
|
||||
cs_msg = messaging.new_message('controlsState')
|
||||
cs_msg.valid = True
|
||||
controlsState = cs_msg.controlsState
|
||||
controlsState.lateralControlState.init('debugState')
|
||||
|
||||
lp = sm['liveParameters']
|
||||
steer_angle_without_offset = math.radians(sm['carState'].steeringAngleDeg - lp.angleOffsetDeg)
|
||||
controlsState.curvature = -VM.calc_curvature(steer_angle_without_offset, sm['carState'].vEgo, lp.roll)
|
||||
|
||||
pm.send('controlsState', cs_msg)
|
||||
|
||||
rk.keep_time()
|
||||
|
||||
|
||||
def main():
|
||||
joystickd_thread()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,59 @@
|
||||
## LogReader
|
||||
|
||||
Route is a class for conveniently accessing all the [logs](/system/loggerd/) from your routes. The LogReader class reads the non-video logs, i.e. rlog.bz2 and qlog.bz2. There's also a matching FrameReader class for reading the videos.
|
||||
|
||||
```python
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
r = Route("a2a0ccea32023010|2023-07-27--13-01-19")
|
||||
|
||||
# get a list of paths for the route's rlog files
|
||||
print(r.log_paths())
|
||||
|
||||
# and road camera (fcamera.hevc) files
|
||||
print(r.camera_paths())
|
||||
|
||||
# setup a LogReader to read the route's first rlog
|
||||
lr = LogReader(r.log_paths()[0])
|
||||
|
||||
# print out all the messages in the log
|
||||
import codecs
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
for msg in lr:
|
||||
print(msg)
|
||||
|
||||
# setup a LogReader for the route's second qlog
|
||||
lr = LogReader(r.log_paths()[1])
|
||||
|
||||
# print all the steering angles values from the log
|
||||
for msg in lr:
|
||||
if msg.which() == "carState":
|
||||
print(msg.carState.steeringAngleDeg)
|
||||
```
|
||||
|
||||
### Segment Ranges
|
||||
|
||||
We also support a new format called a "segment range":
|
||||
|
||||
```
|
||||
344c5c15b34f2d8a / 2024-01-03--09-37-12 / 2:6 / q
|
||||
[ dongle id ] [ timestamp ] [ selector ] [ query type]
|
||||
```
|
||||
|
||||
you can specify which segments from a route to load
|
||||
|
||||
```python
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4") # 4th segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4:6") # 4th and 5th segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/-1") # last segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/:5") # first 5 segments
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/1:") # all except first segment
|
||||
```
|
||||
|
||||
and can select which type of logs to grab
|
||||
|
||||
```python
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/q") # get qlogs
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/r") # get rlogs (default)
|
||||
```
|
||||
@@ -0,0 +1,34 @@
|
||||
import os
|
||||
import requests
|
||||
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
|
||||
|
||||
class CommaApi:
|
||||
def __init__(self, token=None):
|
||||
self.session = requests.Session()
|
||||
self.session.headers['User-agent'] = 'OpenpilotTools'
|
||||
if token:
|
||||
self.session.headers['Authorization'] = 'JWT ' + token
|
||||
|
||||
def request(self, method, endpoint, **kwargs):
|
||||
with self.session.request(method, API_HOST + '/' + endpoint, **kwargs) as resp:
|
||||
resp_json = resp.json()
|
||||
if isinstance(resp_json, dict) and resp_json.get('error'):
|
||||
if resp.status_code in [401, 403]:
|
||||
raise UnauthorizedError('Unauthorized. Authenticate with tools/lib/auth.py')
|
||||
|
||||
e = APIError(str(resp.status_code) + ":" + resp_json.get('description', str(resp_json['error'])))
|
||||
e.status_code = resp.status_code
|
||||
raise e
|
||||
return resp_json
|
||||
|
||||
def get(self, endpoint, **kwargs):
|
||||
return self.request('GET', endpoint, **kwargs)
|
||||
|
||||
def post(self, endpoint, **kwargs):
|
||||
return self.request('POST', endpoint, **kwargs)
|
||||
|
||||
class APIError(Exception):
|
||||
pass
|
||||
|
||||
class UnauthorizedError(Exception):
|
||||
pass
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Usage::
|
||||
|
||||
usage: auth.py [-h] [{google,apple,github,jwt}] [jwt]
|
||||
|
||||
Login to your comma account
|
||||
|
||||
positional arguments:
|
||||
{google,apple,github,jwt}
|
||||
jwt
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
|
||||
|
||||
Examples::
|
||||
|
||||
./auth.py # Log in with google account
|
||||
./auth.py github # Log in with GitHub Account
|
||||
./auth.py jwt ey......hw # Log in with a JWT from https://jwt.comma.ai, for use in CI
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import pprint
|
||||
import webbrowser
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from typing import Any
|
||||
from urllib.parse import parse_qs, urlencode
|
||||
|
||||
from openpilot.tools.lib.api import APIError, CommaApi, UnauthorizedError
|
||||
from openpilot.tools.lib.auth_config import set_token, get_token
|
||||
|
||||
PORT = 3000
|
||||
|
||||
|
||||
class ClientRedirectServer(HTTPServer):
|
||||
query_params: dict[str, Any] = {}
|
||||
|
||||
|
||||
class ClientRedirectHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
if not self.path.startswith('/auth'):
|
||||
self.send_response(204)
|
||||
return
|
||||
|
||||
query = self.path.split('?', 1)[-1]
|
||||
query_parsed = parse_qs(query, keep_blank_values=True)
|
||||
self.server.query_params = query_parsed
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header('Content-type', 'text/plain')
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Return to the CLI to continue')
|
||||
|
||||
def log_message(self, *args):
|
||||
pass # this prevent http server from dumping messages to stdout
|
||||
|
||||
|
||||
def auth_redirect_link(method):
|
||||
provider_id = {
|
||||
'google': 'g',
|
||||
'apple': 'a',
|
||||
'github': 'h',
|
||||
}[method]
|
||||
|
||||
params = {
|
||||
'redirect_uri': f"https://api.comma.ai/v2/auth/{provider_id}/redirect/",
|
||||
'state': f'service,localhost:{PORT}',
|
||||
}
|
||||
|
||||
if method == 'google':
|
||||
params.update({
|
||||
'type': 'web_server',
|
||||
'client_id': '45471411055-ornt4svd2miog6dnopve7qtmh5mnu6id.apps.googleusercontent.com',
|
||||
'response_type': 'code',
|
||||
'scope': 'https://www.googleapis.com/auth/userinfo.email',
|
||||
'prompt': 'select_account',
|
||||
})
|
||||
return 'https://accounts.google.com/o/oauth2/auth?' + urlencode(params)
|
||||
elif method == 'github':
|
||||
params.update({
|
||||
'client_id': '28c4ecb54bb7272cb5a4',
|
||||
'scope': 'read:user',
|
||||
})
|
||||
return 'https://github.com/login/oauth/authorize?' + urlencode(params)
|
||||
elif method == 'apple':
|
||||
params.update({
|
||||
'client_id': 'ai.comma.login',
|
||||
'response_type': 'code',
|
||||
'response_mode': 'form_post',
|
||||
'scope': 'name email',
|
||||
})
|
||||
return 'https://appleid.apple.com/auth/authorize?' + urlencode(params)
|
||||
else:
|
||||
raise NotImplementedError(f"no redirect implemented for method {method}")
|
||||
|
||||
|
||||
def login(method):
|
||||
oauth_uri = auth_redirect_link(method)
|
||||
|
||||
web_server = ClientRedirectServer(('localhost', PORT), ClientRedirectHandler)
|
||||
print(f'To sign in, use your browser and navigate to {oauth_uri}')
|
||||
webbrowser.open(oauth_uri, new=2)
|
||||
|
||||
while True:
|
||||
web_server.handle_request()
|
||||
if 'code' in web_server.query_params:
|
||||
break
|
||||
elif 'error' in web_server.query_params:
|
||||
print('Authentication Error: "{}". Description: "{}" '.format(
|
||||
web_server.query_params['error'],
|
||||
web_server.query_params.get('error_description')), file=sys.stderr)
|
||||
break
|
||||
|
||||
try:
|
||||
auth_resp = CommaApi().post('v2/auth/', data={'code': web_server.query_params['code'], 'provider': web_server.query_params['provider']})
|
||||
set_token(auth_resp['access_token'])
|
||||
except APIError as e:
|
||||
print(f'Authentication Error: {e}', file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Login to your comma account')
|
||||
parser.add_argument('method', default='google', const='google', nargs='?', choices=['google', 'apple', 'github', 'jwt'])
|
||||
parser.add_argument('jwt', nargs='?')
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.method == 'jwt':
|
||||
if args.jwt is None:
|
||||
print("method JWT selected, but no JWT was provided")
|
||||
exit(1)
|
||||
|
||||
set_token(args.jwt)
|
||||
else:
|
||||
login(args.method)
|
||||
|
||||
try:
|
||||
me = CommaApi(token=get_token()).get('/v1/me')
|
||||
print("Authenticated!")
|
||||
pprint.pprint(me)
|
||||
except UnauthorizedError:
|
||||
print("Got invalid JWT")
|
||||
exit(1)
|
||||
@@ -0,0 +1,29 @@
|
||||
import json
|
||||
import os
|
||||
from openpilot.system.hardware.hw import Paths
|
||||
|
||||
|
||||
class MissingAuthConfigError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_token():
|
||||
try:
|
||||
with open(os.path.join(Paths.config_root(), 'auth.json')) as f:
|
||||
auth = json.load(f)
|
||||
return auth['access_token']
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def set_token(token):
|
||||
os.makedirs(Paths.config_root(), exist_ok=True)
|
||||
with open(os.path.join(Paths.config_root(), 'auth.json'), 'w') as f:
|
||||
json.dump({'access_token': token}, f)
|
||||
|
||||
|
||||
def clear_token():
|
||||
try:
|
||||
os.unlink(os.path.join(Paths.config_root(), 'auth.json'))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
@@ -0,0 +1,73 @@
|
||||
import os
|
||||
from datetime import datetime, timedelta, UTC
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import IO
|
||||
|
||||
|
||||
TOKEN_PATH = Path("/data/azure_token")
|
||||
|
||||
@lru_cache
|
||||
def get_azure_credential():
|
||||
if "AZURE_TOKEN" in os.environ:
|
||||
return os.environ["AZURE_TOKEN"]
|
||||
elif TOKEN_PATH.is_file():
|
||||
return TOKEN_PATH.read_text().strip()
|
||||
else:
|
||||
from azure.identity import AzureCliCredential
|
||||
return AzureCliCredential()
|
||||
|
||||
@lru_cache
|
||||
def get_container_sas(account_name: str, container_name: str):
|
||||
from azure.storage.blob import BlobServiceClient, ContainerSasPermissions, generate_container_sas
|
||||
start_time = datetime.now(UTC).replace(tzinfo=None)
|
||||
expiry_time = start_time + timedelta(hours=1)
|
||||
blob_service = BlobServiceClient(
|
||||
account_url=f"https://{account_name}.blob.core.windows.net",
|
||||
credential=get_azure_credential(),
|
||||
)
|
||||
return generate_container_sas(
|
||||
account_name,
|
||||
container_name,
|
||||
user_delegation_key=blob_service.get_user_delegation_key(start_time, expiry_time),
|
||||
permission=ContainerSasPermissions(read=True, write=True, list=True),
|
||||
expiry=expiry_time,
|
||||
)
|
||||
|
||||
class AzureContainer:
|
||||
def __init__(self, account, container):
|
||||
self.ACCOUNT = account
|
||||
self.CONTAINER = container
|
||||
|
||||
@property
|
||||
def ACCOUNT_URL(self) -> str:
|
||||
return f"https://{self.ACCOUNT}.blob.core.windows.net"
|
||||
|
||||
@property
|
||||
def BASE_URL(self) -> str:
|
||||
return f"{self.ACCOUNT_URL}/{self.CONTAINER}/"
|
||||
|
||||
def get_client_and_key(self):
|
||||
from azure.storage.blob import ContainerClient
|
||||
client = ContainerClient(self.ACCOUNT_URL, self.CONTAINER, credential=get_azure_credential())
|
||||
key = get_container_sas(self.ACCOUNT, self.CONTAINER)
|
||||
return client, key
|
||||
|
||||
def get_url(self, route_name: str, segment_num: str, filename: str) -> str:
|
||||
return self.BASE_URL + f"{route_name.replace('|', '/')}/{segment_num}/{filename}"
|
||||
|
||||
def upload_bytes(self, data: bytes | IO, blob_name: str, overwrite=False) -> str:
|
||||
from azure.storage.blob import BlobClient
|
||||
blob = BlobClient(
|
||||
account_url=self.ACCOUNT_URL,
|
||||
container_name=self.CONTAINER,
|
||||
blob_name=blob_name,
|
||||
credential=get_azure_credential(),
|
||||
overwrite=overwrite,
|
||||
)
|
||||
blob.upload_blob(data, overwrite=overwrite)
|
||||
return self.BASE_URL + blob_name
|
||||
|
||||
def upload_file(self, path: str | os.PathLike, blob_name: str, overwrite=False) -> str:
|
||||
with open(path, "rb") as f:
|
||||
return self.upload_bytes(f, blob_name, overwrite)
|
||||
@@ -0,0 +1,57 @@
|
||||
import functools
|
||||
import re
|
||||
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
from openpilot.tools.lib.helpers import RE
|
||||
|
||||
|
||||
@functools.total_ordering
|
||||
class Bootlog:
|
||||
def __init__(self, url: str):
|
||||
self._url = url
|
||||
|
||||
r = re.search(RE.BOOTLOG_NAME, url)
|
||||
if not r:
|
||||
raise Exception(f"Unable to parse: {url}")
|
||||
|
||||
self._id = r.group('log_id')
|
||||
self._dongle_id = r.group('dongle_id')
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return self._url
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str:
|
||||
return self._dongle_id
|
||||
|
||||
@property
|
||||
def id(self) -> str:
|
||||
return self._id
|
||||
|
||||
def __str__(self):
|
||||
return f"{self._dongle_id}/{self._id}"
|
||||
|
||||
def __eq__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.id == b.id
|
||||
|
||||
def __lt__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.id < b.id
|
||||
|
||||
def get_bootlog_from_id(bootlog_id: str) -> Bootlog | None:
|
||||
# TODO: implement an API endpoint for this
|
||||
bl = Bootlog(bootlog_id)
|
||||
for b in get_bootlogs(bl.dongle_id):
|
||||
if b == bl:
|
||||
return b
|
||||
return None
|
||||
|
||||
def get_bootlogs(dongle_id: str) -> list[Bootlog]:
|
||||
api = CommaApi(get_token())
|
||||
r = api.get(f'v1/devices/{dongle_id}/bootlogs')
|
||||
return [Bootlog(b) for b in r]
|
||||
@@ -0,0 +1,14 @@
|
||||
import os
|
||||
import urllib.parse
|
||||
|
||||
DEFAULT_CACHE_DIR = os.getenv("CACHE_ROOT", os.path.expanduser("~/.commacache"))
|
||||
|
||||
def cache_path_for_file_path(fn, cache_dir=DEFAULT_CACHE_DIR):
|
||||
dir_ = os.path.join(cache_dir, "local")
|
||||
os.makedirs(dir_, exist_ok=True)
|
||||
fn_parsed = urllib.parse.urlparse(fn)
|
||||
if fn_parsed.scheme == '':
|
||||
cache_fn = os.path.abspath(fn).replace("/", "_")
|
||||
else:
|
||||
cache_fn = f'{fn_parsed.hostname}_{fn_parsed.path.replace("/", "_")}'
|
||||
return os.path.join(dir_, cache_fn)
|
||||
@@ -0,0 +1,90 @@
|
||||
import os
|
||||
import requests
|
||||
|
||||
|
||||
# Forks with additional car support can fork the commaCarSegments repo on huggingface or host the LFS files themselves
|
||||
COMMA_CAR_SEGMENTS_REPO = os.environ.get("COMMA_CAR_SEGMENTS_REPO", "https://huggingface.co/datasets/commaai/commaCarSegments")
|
||||
COMMA_CAR_SEGMENTS_BRANCH = os.environ.get("COMMA_CAR_SEGMENTS_BRANCH", "main")
|
||||
COMMA_CAR_SEGMENTS_LFS_INSTANCE = os.environ.get("COMMA_CAR_SEGMENTS_LFS_INSTANCE", COMMA_CAR_SEGMENTS_REPO)
|
||||
|
||||
def get_comma_car_segments_database():
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
|
||||
database = requests.get(get_repo_raw_url("database.json")).json()
|
||||
|
||||
ret = {}
|
||||
for platform in database:
|
||||
ret[MIGRATION.get(platform, platform)] = database[platform]
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
# Helpers related to interfacing with the commaCarSegments repository, which contains a collection of public segments for users to perform validation on.
|
||||
|
||||
def parse_lfs_pointer(text):
|
||||
header, lfs_version = text.splitlines()[0].split(" ")
|
||||
assert header == "version"
|
||||
assert lfs_version == "https://git-lfs.github.com/spec/v1"
|
||||
|
||||
header, oid_raw = text.splitlines()[1].split(" ")
|
||||
assert header == "oid"
|
||||
header, oid = oid_raw.split(":")
|
||||
assert header == "sha256"
|
||||
|
||||
header, size = text.splitlines()[2].split(" ")
|
||||
assert header == "size"
|
||||
|
||||
return oid, size
|
||||
|
||||
def get_lfs_file_url(oid, size):
|
||||
data = {
|
||||
"operation": "download",
|
||||
"transfers": [ "basic" ],
|
||||
"objects": [
|
||||
{
|
||||
"oid": oid,
|
||||
"size": int(size)
|
||||
}
|
||||
],
|
||||
"hash_algo": "sha256"
|
||||
}
|
||||
|
||||
headers = {
|
||||
"Accept": "application/vnd.git-lfs+json",
|
||||
"Content-Type": "application/vnd.git-lfs+json"
|
||||
}
|
||||
|
||||
response = requests.post(f"{COMMA_CAR_SEGMENTS_LFS_INSTANCE}.git/info/lfs/objects/batch", json=data, headers=headers)
|
||||
|
||||
assert response.ok
|
||||
|
||||
obj = response.json()["objects"][0]
|
||||
|
||||
assert "error" not in obj, obj
|
||||
|
||||
return obj["actions"]["download"]["href"]
|
||||
|
||||
def get_repo_raw_url(path):
|
||||
if "huggingface" in COMMA_CAR_SEGMENTS_REPO:
|
||||
return f"{COMMA_CAR_SEGMENTS_REPO}/raw/{COMMA_CAR_SEGMENTS_BRANCH}/{path}"
|
||||
|
||||
def get_repo_url(path):
|
||||
# Automatically switch to LFS if we are requesting a file that is stored in LFS
|
||||
|
||||
response = requests.head(get_repo_raw_url(path))
|
||||
|
||||
if "text/plain" in response.headers.get("content-type"):
|
||||
# This is an LFS pointer, so download the raw data from lfs
|
||||
response = requests.get(get_repo_raw_url(path))
|
||||
assert response.status_code == 200
|
||||
oid, size = parse_lfs_pointer(response.text)
|
||||
|
||||
return get_lfs_file_url(oid, size)
|
||||
else:
|
||||
# File has not been uploaded to LFS yet
|
||||
# (either we are on a fork where the data hasn't been pushed to LFS yet, or the CI job to push hasn't finished)
|
||||
return get_repo_raw_url(path)
|
||||
|
||||
|
||||
def get_url(route, segment, file="rlog.bz2"):
|
||||
return get_repo_url(f"segments/{route.replace('|', '/')}/{segment}/{file}")
|
||||
@@ -0,0 +1,2 @@
|
||||
class DataUnreadableError(Exception):
|
||||
pass
|
||||
@@ -0,0 +1,44 @@
|
||||
import os
|
||||
import posixpath
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from openpilot.tools.lib.url_file import URLFile
|
||||
|
||||
DATA_ENDPOINT = os.getenv("DATA_ENDPOINT", "http://data-raw.comma.internal/")
|
||||
|
||||
|
||||
def internal_source_available(url=DATA_ENDPOINT):
|
||||
if os.path.isdir(url):
|
||||
return True
|
||||
|
||||
try:
|
||||
hostname = urlparse(url).hostname
|
||||
port = urlparse(url).port or 80
|
||||
with socket.socket(socket.AF_INET,socket.SOCK_STREAM) as s:
|
||||
s.settimeout(0.5)
|
||||
s.connect((hostname, port))
|
||||
return True
|
||||
except (socket.gaierror, ConnectionRefusedError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def resolve_name(fn):
|
||||
if fn.startswith("cd:/"):
|
||||
return posixpath.join(DATA_ENDPOINT, fn[4:])
|
||||
return fn
|
||||
|
||||
|
||||
def file_exists(fn):
|
||||
fn = resolve_name(fn)
|
||||
if fn.startswith(("http://", "https://")):
|
||||
return URLFile(fn).get_length_online() != -1
|
||||
return os.path.exists(fn)
|
||||
|
||||
|
||||
def FileReader(fn, debug=False):
|
||||
fn = resolve_name(fn)
|
||||
if fn.startswith(("http://", "https://")):
|
||||
return URLFile(fn, debug=debug)
|
||||
return open(fn, "rb")
|
||||
@@ -0,0 +1,559 @@
|
||||
import json
|
||||
import os
|
||||
import pickle
|
||||
import struct
|
||||
import subprocess
|
||||
import threading
|
||||
from enum import IntEnum
|
||||
from functools import wraps
|
||||
|
||||
import numpy as np
|
||||
from lru import LRU
|
||||
|
||||
import _io
|
||||
from openpilot.tools.lib.cache import cache_path_for_file_path, DEFAULT_CACHE_DIR
|
||||
from openpilot.tools.lib.exceptions import DataUnreadableError
|
||||
from openpilot.tools.lib.vidindex import hevc_index
|
||||
from openpilot.common.file_helpers import atomic_write_in_dir
|
||||
|
||||
from openpilot.tools.lib.filereader import FileReader, resolve_name
|
||||
|
||||
HEVC_SLICE_B = 0
|
||||
HEVC_SLICE_P = 1
|
||||
HEVC_SLICE_I = 2
|
||||
|
||||
|
||||
class GOPReader:
|
||||
def get_gop(self, num):
|
||||
# returns (start_frame_num, num_frames, frames_to_skip, gop_data)
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class DoNothingContextManager:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *x):
|
||||
pass
|
||||
|
||||
|
||||
class FrameType(IntEnum):
|
||||
raw = 1
|
||||
h265_stream = 2
|
||||
|
||||
|
||||
def fingerprint_video(fn):
|
||||
with FileReader(fn) as f:
|
||||
header = f.read(4)
|
||||
if len(header) == 0:
|
||||
raise DataUnreadableError(f"{fn} is empty")
|
||||
elif header == b"\x00\xc0\x12\x00":
|
||||
return FrameType.raw
|
||||
elif header == b"\x00\x00\x00\x01":
|
||||
if 'hevc' in fn:
|
||||
return FrameType.h265_stream
|
||||
else:
|
||||
raise NotImplementedError(fn)
|
||||
else:
|
||||
raise NotImplementedError(fn)
|
||||
|
||||
|
||||
def ffprobe(fn, fmt=None):
|
||||
fn = resolve_name(fn)
|
||||
cmd = ["ffprobe", "-v", "quiet", "-print_format", "json", "-show_format", "-show_streams"]
|
||||
if fmt:
|
||||
cmd += ["-f", fmt]
|
||||
cmd += ["-i", "-"]
|
||||
|
||||
try:
|
||||
with FileReader(fn) as f:
|
||||
ffprobe_output = subprocess.check_output(cmd, input=f.read(4096))
|
||||
except subprocess.CalledProcessError as e:
|
||||
raise DataUnreadableError(fn) from e
|
||||
|
||||
return json.loads(ffprobe_output)
|
||||
|
||||
|
||||
def cache_fn(func):
|
||||
@wraps(func)
|
||||
def cache_inner(fn, *args, **kwargs):
|
||||
if kwargs.pop('no_cache', None):
|
||||
cache_path = None
|
||||
else:
|
||||
cache_dir = kwargs.pop('cache_dir', DEFAULT_CACHE_DIR)
|
||||
cache_path = cache_path_for_file_path(fn, cache_dir)
|
||||
|
||||
if cache_path and os.path.exists(cache_path):
|
||||
with open(cache_path, "rb") as cache_file:
|
||||
cache_value = pickle.load(cache_file)
|
||||
else:
|
||||
cache_value = func(fn, *args, **kwargs)
|
||||
if cache_path:
|
||||
with atomic_write_in_dir(cache_path, mode="wb", overwrite=True) as cache_file:
|
||||
pickle.dump(cache_value, cache_file, -1)
|
||||
|
||||
return cache_value
|
||||
|
||||
return cache_inner
|
||||
|
||||
|
||||
@cache_fn
|
||||
def index_stream(fn, ft):
|
||||
if ft != FrameType.h265_stream:
|
||||
raise NotImplementedError("Only h265 supported")
|
||||
|
||||
frame_types, dat_len, prefix = hevc_index(fn)
|
||||
index = np.array(frame_types + [(0xFFFFFFFF, dat_len)], dtype=np.uint32)
|
||||
probe = ffprobe(fn, "hevc")
|
||||
|
||||
return {
|
||||
'index': index,
|
||||
'global_prefix': prefix,
|
||||
'probe': probe
|
||||
}
|
||||
|
||||
|
||||
def get_video_index(fn, frame_type, cache_dir=DEFAULT_CACHE_DIR):
|
||||
return index_stream(fn, frame_type, cache_dir=cache_dir)
|
||||
|
||||
def read_file_check_size(f, sz, cookie):
|
||||
buff = bytearray(sz)
|
||||
bytes_read = f.readinto(buff)
|
||||
assert bytes_read == sz, (bytes_read, sz)
|
||||
return buff
|
||||
|
||||
|
||||
def rgb24toyuv(rgb):
|
||||
yuv_from_rgb = np.array([[ 0.299 , 0.587 , 0.114 ],
|
||||
[-0.14714119, -0.28886916, 0.43601035 ],
|
||||
[ 0.61497538, -0.51496512, -0.10001026 ]])
|
||||
img = np.dot(rgb.reshape(-1, 3), yuv_from_rgb.T).reshape(rgb.shape)
|
||||
|
||||
|
||||
|
||||
ys = img[:, :, 0]
|
||||
us = (img[::2, ::2, 1] + img[1::2, ::2, 1] + img[::2, 1::2, 1] + img[1::2, 1::2, 1]) / 4 + 128
|
||||
vs = (img[::2, ::2, 2] + img[1::2, ::2, 2] + img[::2, 1::2, 2] + img[1::2, 1::2, 2]) / 4 + 128
|
||||
|
||||
return ys, us, vs
|
||||
|
||||
|
||||
def rgb24toyuv420(rgb):
|
||||
ys, us, vs = rgb24toyuv(rgb)
|
||||
|
||||
y_len = rgb.shape[0] * rgb.shape[1]
|
||||
uv_len = y_len // 4
|
||||
|
||||
yuv420 = np.empty(y_len + 2 * uv_len, dtype=rgb.dtype)
|
||||
yuv420[:y_len] = ys.reshape(-1)
|
||||
yuv420[y_len:y_len + uv_len] = us.reshape(-1)
|
||||
yuv420[y_len + uv_len:y_len + 2 * uv_len] = vs.reshape(-1)
|
||||
|
||||
return yuv420.clip(0, 255).astype('uint8')
|
||||
|
||||
|
||||
def rgb24tonv12(rgb):
|
||||
ys, us, vs = rgb24toyuv(rgb)
|
||||
|
||||
y_len = rgb.shape[0] * rgb.shape[1]
|
||||
uv_len = y_len // 4
|
||||
|
||||
nv12 = np.empty(y_len + 2 * uv_len, dtype=rgb.dtype)
|
||||
nv12[:y_len] = ys.reshape(-1)
|
||||
nv12[y_len::2] = us.reshape(-1)
|
||||
nv12[y_len+1::2] = vs.reshape(-1)
|
||||
|
||||
return nv12.clip(0, 255).astype('uint8')
|
||||
|
||||
|
||||
def decompress_video_data(rawdat, vid_fmt, w, h, pix_fmt):
|
||||
threads = os.getenv("FFMPEG_THREADS", "0")
|
||||
cuda = os.getenv("FFMPEG_CUDA", "0") == "1"
|
||||
args = ["ffmpeg", "-v", "quiet",
|
||||
"-threads", threads,
|
||||
"-hwaccel", "none" if not cuda else "cuda",
|
||||
"-c:v", "hevc",
|
||||
"-vsync", "0",
|
||||
"-f", vid_fmt,
|
||||
"-flags2", "showall",
|
||||
"-i", "-",
|
||||
"-threads", threads,
|
||||
"-f", "rawvideo",
|
||||
"-pix_fmt", pix_fmt,
|
||||
"-"]
|
||||
dat = subprocess.check_output(args, input=rawdat)
|
||||
|
||||
if pix_fmt == "rgb24":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, h, w, 3)
|
||||
elif pix_fmt == "nv12":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, (h*w*3//2))
|
||||
elif pix_fmt == "yuv420p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, (h*w*3//2))
|
||||
elif pix_fmt == "yuv444p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, 3, h, w)
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
class BaseFrameReader:
|
||||
# properties: frame_type, frame_count, w, h
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
def FrameReader(fn, cache_dir=DEFAULT_CACHE_DIR, readahead=False, readbehind=False, index_data=None):
|
||||
frame_type = fingerprint_video(fn)
|
||||
if frame_type == FrameType.raw:
|
||||
return RawFrameReader(fn)
|
||||
elif frame_type in (FrameType.h265_stream,):
|
||||
if not index_data:
|
||||
index_data = get_video_index(fn, frame_type, cache_dir)
|
||||
return StreamFrameReader(fn, frame_type, index_data, readahead=readahead, readbehind=readbehind)
|
||||
else:
|
||||
raise NotImplementedError(frame_type)
|
||||
|
||||
|
||||
class RawData:
|
||||
def __init__(self, f):
|
||||
self.f = _io.FileIO(f, 'rb')
|
||||
self.lenn = struct.unpack("I", self.f.read(4))[0]
|
||||
self.count = os.path.getsize(f) / (self.lenn+4)
|
||||
|
||||
def read(self, i):
|
||||
self.f.seek((self.lenn+4)*i + 4)
|
||||
return self.f.read(self.lenn)
|
||||
|
||||
|
||||
class RawFrameReader(BaseFrameReader):
|
||||
def __init__(self, fn):
|
||||
# raw camera
|
||||
self.fn = fn
|
||||
self.frame_type = FrameType.raw
|
||||
self.rawfile = RawData(self.fn)
|
||||
self.frame_count = self.rawfile.count
|
||||
self.w, self.h = 640, 480
|
||||
|
||||
def load_and_debayer(self, img):
|
||||
img = np.frombuffer(img, dtype='uint8').reshape(960, 1280)
|
||||
cimg = np.dstack([img[0::2, 1::2], ((img[0::2, 0::2].astype("uint16") + img[1::2, 1::2].astype("uint16")) >> 1).astype("uint8"), img[1::2, 0::2]])
|
||||
return cimg
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
assert self.frame_count is not None
|
||||
assert num+count <= self.frame_count
|
||||
|
||||
if pix_fmt not in ("nv12", "yuv420p", "rgb24"):
|
||||
raise ValueError(f"Unsupported pixel format {pix_fmt!r}")
|
||||
|
||||
app = []
|
||||
for i in range(num, num+count):
|
||||
dat = self.rawfile.read(i)
|
||||
rgb_dat = self.load_and_debayer(dat)
|
||||
if pix_fmt == "rgb24":
|
||||
app.append(rgb_dat)
|
||||
elif pix_fmt == "nv12":
|
||||
app.append(rgb24tonv12(rgb_dat))
|
||||
elif pix_fmt == "yuv420p":
|
||||
app.append(rgb24toyuv420(rgb_dat))
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
return app
|
||||
|
||||
|
||||
class VideoStreamDecompressor:
|
||||
def __init__(self, fn, vid_fmt, w, h, pix_fmt):
|
||||
self.fn = fn
|
||||
self.vid_fmt = vid_fmt
|
||||
self.w = w
|
||||
self.h = h
|
||||
self.pix_fmt = pix_fmt
|
||||
|
||||
if pix_fmt in ("nv12", "yuv420p"):
|
||||
self.out_size = w*h*3//2 # yuv420p
|
||||
elif pix_fmt in ("rgb24", "yuv444p"):
|
||||
self.out_size = w*h*3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
self.proc = None
|
||||
self.t = threading.Thread(target=self.write_thread)
|
||||
self.t.daemon = True
|
||||
|
||||
def write_thread(self):
|
||||
try:
|
||||
with FileReader(self.fn) as f:
|
||||
while True:
|
||||
r = f.read(1024*1024)
|
||||
if len(r) == 0:
|
||||
break
|
||||
self.proc.stdin.write(r)
|
||||
except BrokenPipeError:
|
||||
pass
|
||||
finally:
|
||||
self.proc.stdin.close()
|
||||
|
||||
def read(self):
|
||||
threads = os.getenv("FFMPEG_THREADS", "0")
|
||||
cuda = os.getenv("FFMPEG_CUDA", "0") == "1"
|
||||
cmd = [
|
||||
"ffmpeg",
|
||||
"-threads", threads,
|
||||
"-hwaccel", "none" if not cuda else "cuda",
|
||||
"-c:v", "hevc",
|
||||
# "-avioflags", "direct",
|
||||
"-analyzeduration", "0",
|
||||
"-probesize", "32",
|
||||
"-flush_packets", "0",
|
||||
# "-fflags", "nobuffer",
|
||||
"-vsync", "0",
|
||||
"-f", self.vid_fmt,
|
||||
"-i", "pipe:0",
|
||||
"-threads", threads,
|
||||
"-f", "rawvideo",
|
||||
"-pix_fmt", self.pix_fmt,
|
||||
"pipe:1"
|
||||
]
|
||||
self.proc = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
self.t.start()
|
||||
|
||||
while True:
|
||||
dat = self.proc.stdout.read(self.out_size)
|
||||
if len(dat) == 0:
|
||||
break
|
||||
assert len(dat) == self.out_size
|
||||
if self.pix_fmt == "rgb24":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape((self.h, self.w, 3))
|
||||
elif self.pix_fmt == "yuv420p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8)
|
||||
elif self.pix_fmt == "nv12":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8)
|
||||
elif self.pix_fmt == "yuv444p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape((3, self.h, self.w))
|
||||
else:
|
||||
raise RuntimeError(f"unknown pix_fmt: {self.pix_fmt}")
|
||||
yield ret
|
||||
|
||||
result_code = self.proc.wait()
|
||||
assert result_code == 0, result_code
|
||||
finally:
|
||||
self.proc.kill()
|
||||
self.t.join()
|
||||
|
||||
class StreamGOPReader(GOPReader):
|
||||
def __init__(self, fn, frame_type, index_data):
|
||||
assert frame_type == FrameType.h265_stream
|
||||
|
||||
self.fn = fn
|
||||
|
||||
self.frame_type = frame_type
|
||||
self.frame_count = None
|
||||
self.w, self.h = None, None
|
||||
|
||||
self.prefix = None
|
||||
self.index = None
|
||||
|
||||
self.index = index_data['index']
|
||||
self.prefix = index_data['global_prefix']
|
||||
probe = index_data['probe']
|
||||
|
||||
self.prefix_frame_data = None
|
||||
self.num_prefix_frames = 0
|
||||
self.vid_fmt = "hevc"
|
||||
|
||||
i = 0
|
||||
while i < self.index.shape[0] and self.index[i, 0] != HEVC_SLICE_I:
|
||||
i += 1
|
||||
self.first_iframe = i
|
||||
|
||||
assert self.first_iframe == 0
|
||||
|
||||
self.frame_count = len(self.index) - 1
|
||||
|
||||
self.w = probe['streams'][0]['width']
|
||||
self.h = probe['streams'][0]['height']
|
||||
|
||||
def _lookup_gop(self, num):
|
||||
frame_b = num
|
||||
while frame_b > 0 and self.index[frame_b, 0] != HEVC_SLICE_I:
|
||||
frame_b -= 1
|
||||
|
||||
frame_e = num + 1
|
||||
while frame_e < (len(self.index) - 1) and self.index[frame_e, 0] != HEVC_SLICE_I:
|
||||
frame_e += 1
|
||||
|
||||
offset_b = self.index[frame_b, 1]
|
||||
offset_e = self.index[frame_e, 1]
|
||||
|
||||
return (frame_b, frame_e, offset_b, offset_e)
|
||||
|
||||
def get_gop(self, num):
|
||||
frame_b, frame_e, offset_b, offset_e = self._lookup_gop(num)
|
||||
assert frame_b <= num < frame_e
|
||||
|
||||
num_frames = frame_e - frame_b
|
||||
|
||||
with FileReader(self.fn) as f:
|
||||
f.seek(offset_b)
|
||||
rawdat = f.read(offset_e - offset_b)
|
||||
|
||||
if num < self.first_iframe:
|
||||
assert self.prefix_frame_data
|
||||
rawdat = self.prefix_frame_data + rawdat
|
||||
|
||||
rawdat = self.prefix + rawdat
|
||||
|
||||
skip_frames = 0
|
||||
if num < self.first_iframe:
|
||||
skip_frames = self.num_prefix_frames
|
||||
|
||||
return frame_b, num_frames, skip_frames, rawdat
|
||||
|
||||
|
||||
class GOPFrameReader(BaseFrameReader):
|
||||
#FrameReader with caching and readahead for formats that are group-of-picture based
|
||||
|
||||
def __init__(self, readahead=False, readbehind=False):
|
||||
self.open_ = True
|
||||
|
||||
self.readahead = readahead
|
||||
self.readbehind = readbehind
|
||||
self.frame_cache = LRU(64)
|
||||
|
||||
if self.readahead:
|
||||
self.cache_lock = threading.RLock()
|
||||
self.readahead_last = None
|
||||
self.readahead_len = 30
|
||||
self.readahead_c = threading.Condition()
|
||||
self.readahead_thread = threading.Thread(target=self._readahead_thread)
|
||||
self.readahead_thread.daemon = True
|
||||
self.readahead_thread.start()
|
||||
else:
|
||||
self.cache_lock = DoNothingContextManager()
|
||||
|
||||
def close(self):
|
||||
if not self.open_:
|
||||
return
|
||||
self.open_ = False
|
||||
|
||||
if self.readahead:
|
||||
self.readahead_c.acquire()
|
||||
self.readahead_c.notify()
|
||||
self.readahead_c.release()
|
||||
self.readahead_thread.join()
|
||||
|
||||
def _readahead_thread(self):
|
||||
while True:
|
||||
self.readahead_c.acquire()
|
||||
try:
|
||||
if not self.open_:
|
||||
break
|
||||
self.readahead_c.wait()
|
||||
finally:
|
||||
self.readahead_c.release()
|
||||
if not self.open_:
|
||||
break
|
||||
assert self.readahead_last
|
||||
num, pix_fmt = self.readahead_last
|
||||
|
||||
if self.readbehind:
|
||||
for k in range(num - 1, max(0, num - self.readahead_len), -1):
|
||||
self._get_one(k, pix_fmt)
|
||||
else:
|
||||
for k in range(num, min(self.frame_count, num + self.readahead_len)):
|
||||
self._get_one(k, pix_fmt)
|
||||
|
||||
def _get_one(self, num, pix_fmt):
|
||||
assert num < self.frame_count
|
||||
|
||||
if (num, pix_fmt) in self.frame_cache:
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
with self.cache_lock:
|
||||
if (num, pix_fmt) in self.frame_cache:
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
frame_b, num_frames, skip_frames, rawdat = self.get_gop(num)
|
||||
|
||||
ret = decompress_video_data(rawdat, self.vid_fmt, self.w, self.h, pix_fmt)
|
||||
ret = ret[skip_frames:]
|
||||
assert ret.shape[0] == num_frames
|
||||
|
||||
for i in range(ret.shape[0]):
|
||||
self.frame_cache[(frame_b+i, pix_fmt)] = ret[i]
|
||||
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
assert self.frame_count is not None
|
||||
|
||||
if num + count > self.frame_count:
|
||||
raise ValueError(f"{num + count} > {self.frame_count}")
|
||||
|
||||
if pix_fmt not in ("nv12", "yuv420p", "rgb24", "yuv444p"):
|
||||
raise ValueError(f"Unsupported pixel format {pix_fmt!r}")
|
||||
|
||||
ret = [self._get_one(num + i, pix_fmt) for i in range(count)]
|
||||
|
||||
if self.readahead:
|
||||
self.readahead_last = (num+count, pix_fmt)
|
||||
self.readahead_c.acquire()
|
||||
self.readahead_c.notify()
|
||||
self.readahead_c.release()
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
class StreamFrameReader(StreamGOPReader, GOPFrameReader):
|
||||
def __init__(self, fn, frame_type, index_data, readahead=False, readbehind=False):
|
||||
StreamGOPReader.__init__(self, fn, frame_type, index_data)
|
||||
GOPFrameReader.__init__(self, readahead, readbehind)
|
||||
|
||||
|
||||
def GOPFrameIterator(gop_reader, pix_fmt):
|
||||
dec = VideoStreamDecompressor(gop_reader.fn, gop_reader.vid_fmt, gop_reader.w, gop_reader.h, pix_fmt)
|
||||
yield from dec.read()
|
||||
|
||||
|
||||
def FrameIterator(fn, pix_fmt, **kwargs):
|
||||
fr = FrameReader(fn, **kwargs)
|
||||
if isinstance(fr, GOPReader):
|
||||
yield from GOPFrameIterator(fr, pix_fmt)
|
||||
else:
|
||||
for i in range(fr.frame_count):
|
||||
yield fr.get(i, pix_fmt=pix_fmt)[0]
|
||||
|
||||
|
||||
class NumpyFrameReader:
|
||||
def __init__(self, name, w, h, cache_size):
|
||||
self.name = name
|
||||
self.pos = -1
|
||||
self.frames = None
|
||||
self.w = w
|
||||
self.h = h
|
||||
self.cache_size = cache_size
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
def get(self, num, count=1, pix_fmt="nv12"):
|
||||
num -= 1
|
||||
q = num // self.cache_size
|
||||
if q != self.pos:
|
||||
del self.frames
|
||||
self.pos = q
|
||||
self.frames = np.load(f'{self.name}_{self.pos}.npy')
|
||||
return [self.frames[num % self.cache_size]]
|
||||
@@ -0,0 +1,114 @@
|
||||
import base64
|
||||
import requests
|
||||
from http import HTTPMethod
|
||||
|
||||
class GithubUtils:
|
||||
def __init__(self, api_token, data_token, owner='commaai', api_repo='openpilot', data_repo='ci-artifacts'):
|
||||
self.OWNER = owner
|
||||
self.API_REPO = api_repo
|
||||
self.DATA_REPO = data_repo
|
||||
self.API_TOKEN = api_token
|
||||
self.DATA_TOKEN = data_token
|
||||
|
||||
@property
|
||||
def API_ROUTE(self):
|
||||
return f"https://api.github.com/repos/{self.OWNER}/{self.API_REPO}"
|
||||
|
||||
@property
|
||||
def DATA_ROUTE(self):
|
||||
return f"https://api.github.com/repos/{self.OWNER}/{self.DATA_REPO}"
|
||||
|
||||
def api_call(self, path, data="", method=HTTPMethod.GET, accept="", data_call=False, raise_on_failure=True):
|
||||
token = self.DATA_TOKEN if data_call else self.API_TOKEN
|
||||
if token:
|
||||
headers = {"Authorization": f"Bearer {self.DATA_TOKEN if data_call else self.API_TOKEN}", \
|
||||
"Accept": f"application/vnd.github{accept}+json"}
|
||||
else:
|
||||
headers = {}
|
||||
path = f'{self.DATA_ROUTE if data_call else self.API_ROUTE}/{path}'
|
||||
r = requests.request(method, path, headers=headers, data=data)
|
||||
if not r.ok and raise_on_failure:
|
||||
raise Exception(f"Call to {path} failed with {r.status_code}")
|
||||
else:
|
||||
return r
|
||||
|
||||
def upload_file(self, bucket, path, file_name):
|
||||
with open(path, "rb") as f:
|
||||
encoded = base64.b64encode(f.read()).decode()
|
||||
|
||||
# check if file already exists
|
||||
sha = self.get_file_sha(bucket, file_name)
|
||||
sha = f'"sha":"{sha}",' if sha else ''
|
||||
|
||||
data = f'{{"message":"uploading {file_name}", \
|
||||
"branch":"{bucket}", \
|
||||
"committer":{{"name":"Vehicle Researcher", "email": "user@comma.ai"}}, \
|
||||
{sha} \
|
||||
"content":"{encoded}"}}'
|
||||
github_path = f"contents/{file_name}"
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.PUT, data_call=True)
|
||||
|
||||
def upload_files(self, bucket, files):
|
||||
self.create_bucket(bucket)
|
||||
for file_name,path in files:
|
||||
self.upload_file(bucket, path, file_name)
|
||||
|
||||
def create_bucket(self, bucket):
|
||||
if self.get_bucket_sha(bucket):
|
||||
return
|
||||
master_sha = self.get_bucket_sha('master')
|
||||
github_path = "git/refs"
|
||||
data = f'{{"ref":"refs/heads/{bucket}", "sha":"{master_sha}"}}'
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.POST, data_call=True)
|
||||
|
||||
def get_bucket_sha(self, bucket):
|
||||
github_path = f"git/refs/heads/{bucket}"
|
||||
r = self.api_call(github_path, data_call=True, raise_on_failure=False)
|
||||
return r.json()['object']['sha'] if r.ok else None
|
||||
|
||||
def get_file_url(self, bucket, file_name):
|
||||
github_path = f"contents/{file_name}?ref={bucket}"
|
||||
r = self.api_call(github_path, data_call=True)
|
||||
return r.json()['download_url']
|
||||
|
||||
def get_file_sha(self, bucket, file_name):
|
||||
github_path = f"contents/{file_name}?ref={bucket}"
|
||||
r = self.api_call(github_path, data_call=True, raise_on_failure=False)
|
||||
return r.json()['sha'] if r.ok else None
|
||||
|
||||
def get_pr_number(self, pr_branch):
|
||||
github_path = f"commits/{pr_branch}/pulls"
|
||||
r = self.api_call(github_path)
|
||||
return r.json()[0]['number']
|
||||
|
||||
def get_bucket_link(self, bucket):
|
||||
return f'https://raw.githubusercontent.com/{self.OWNER}/{self.DATA_REPO}/refs/heads/{bucket}'
|
||||
|
||||
def comment_on_pr(self, comment, pr_branch, commenter="", overwrite=False):
|
||||
pr_number = self.get_pr_number(pr_branch)
|
||||
data = f'{{"body": "{comment}"}}'
|
||||
if overwrite:
|
||||
github_path = f'issues/{pr_number}/comments'
|
||||
r = self.api_call(github_path)
|
||||
comments = [x['id'] for x in r.json() if x['user']['login'] == commenter]
|
||||
if comments:
|
||||
github_path = f'issues/comments/{comments[0]}'
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.PATCH)
|
||||
return
|
||||
|
||||
github_path=f'issues/{pr_number}/comments'
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.POST)
|
||||
|
||||
# upload files to github and comment them on the pr
|
||||
def comment_images_on_pr(self, title, commenter, pr_branch, bucket, images):
|
||||
self.upload_files(bucket, images)
|
||||
table = [f'<details><summary>{title}</summary><table>']
|
||||
for i,f in enumerate(images):
|
||||
if not (i % 2):
|
||||
table.append('<tr>')
|
||||
table.append(f'<td><img src=\\"https://raw.githubusercontent.com/{self.OWNER}/{self.DATA_REPO}/{bucket}/{f[0]}\\"></td>')
|
||||
if (i % 2):
|
||||
table.append('</tr>')
|
||||
table.append('</table></details>')
|
||||
table = ''.join(table)
|
||||
self.comment_on_pr(table, commenter, pr_branch)
|
||||
@@ -0,0 +1,17 @@
|
||||
# regex patterns
|
||||
class RE:
|
||||
DONGLE_ID = r'(?P<dongle_id>[a-f0-9]{16})'
|
||||
TIMESTAMP = r'(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})'
|
||||
LOG_ID_V2 = r'(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10})'
|
||||
LOG_ID = fr'(?P<log_id>(?:{TIMESTAMP}|{LOG_ID_V2}))'
|
||||
ROUTE_NAME = fr'(?P<route_name>{DONGLE_ID}[|_/]{LOG_ID})'
|
||||
SEGMENT_NAME = fr'{ROUTE_NAME}(?:--|/)(?P<segment_num>[0-9]+)'
|
||||
|
||||
INDEX = r'-?[0-9]+'
|
||||
SLICE = fr'(?P<start>{INDEX})?:?(?P<end>{INDEX})?:?(?P<step>{INDEX})?'
|
||||
SEGMENT_RANGE = fr'{ROUTE_NAME}(?:(--|/)(?P<slice>({SLICE})))?(?:/(?P<selector>([qras])))?'
|
||||
|
||||
BOOTLOG_NAME = ROUTE_NAME
|
||||
|
||||
EXPLORER_FILE = fr'^(?P<segment_name>{SEGMENT_NAME})--(?P<file_name>[a-z]+\.[a-z0-9]+)$'
|
||||
OP_SEGMENT_DIR = fr'^(?P<segment_name>{SEGMENT_NAME})$'
|
||||
Executable
+81
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import termios
|
||||
import atexit
|
||||
from select import select
|
||||
|
||||
|
||||
class KBHit:
|
||||
def __init__(self) -> None:
|
||||
''' Creates a KBHit object that you can call to do various keyboard things.
|
||||
'''
|
||||
|
||||
self.stdin_fd = sys.stdin.fileno()
|
||||
self.set_kbhit_terminal()
|
||||
|
||||
def set_kbhit_terminal(self) -> None:
|
||||
''' Save old terminal settings for closure, remove ICANON & ECHO flags.
|
||||
'''
|
||||
|
||||
# Save the terminal settings
|
||||
self.old_term = termios.tcgetattr(self.stdin_fd)
|
||||
self.new_term = self.old_term.copy()
|
||||
|
||||
# New terminal setting unbuffered
|
||||
self.new_term[3] &= ~(termios.ICANON | termios.ECHO)
|
||||
termios.tcsetattr(self.stdin_fd, termios.TCSAFLUSH, self.new_term)
|
||||
|
||||
# Support normal-terminal reset at exit
|
||||
atexit.register(self.set_normal_term)
|
||||
|
||||
def set_normal_term(self) -> None:
|
||||
''' Resets to normal terminal. On Windows this is a no-op.
|
||||
'''
|
||||
|
||||
termios.tcsetattr(self.stdin_fd, termios.TCSAFLUSH, self.old_term)
|
||||
|
||||
@staticmethod
|
||||
def getch() -> str:
|
||||
''' Returns a keyboard character after kbhit() has been called.
|
||||
Should not be called in the same program as getarrow().
|
||||
'''
|
||||
return sys.stdin.read(1)
|
||||
|
||||
@staticmethod
|
||||
def getarrow() -> int:
|
||||
''' Returns an arrow-key code after kbhit() has been called. Codes are
|
||||
0 : up
|
||||
1 : right
|
||||
2 : down
|
||||
3 : left
|
||||
Should not be called in the same program as getch().
|
||||
'''
|
||||
|
||||
c = sys.stdin.read(3)[2]
|
||||
vals = [65, 67, 66, 68]
|
||||
|
||||
return vals.index(ord(c))
|
||||
|
||||
@staticmethod
|
||||
def kbhit():
|
||||
''' Returns True if keyboard character was hit, False otherwise.
|
||||
'''
|
||||
return select([sys.stdin], [], [], 0)[0] != []
|
||||
|
||||
|
||||
# Test
|
||||
if __name__ == "__main__":
|
||||
|
||||
kb = KBHit()
|
||||
|
||||
print('Hit any key, or ESC to exit')
|
||||
|
||||
while True:
|
||||
|
||||
if kb.kbhit():
|
||||
c = kb.getch()
|
||||
if c == '\x1b': # ESC
|
||||
break
|
||||
print(c)
|
||||
|
||||
kb.set_normal_term()
|
||||
@@ -0,0 +1,30 @@
|
||||
import os
|
||||
from cereal import log as capnp_log, messaging
|
||||
from cereal.services import SERVICE_LIST
|
||||
|
||||
from openpilot.tools.lib.logreader import LogIterable, RawLogIterable
|
||||
|
||||
|
||||
ALL_SERVICES = list(SERVICE_LIST.keys())
|
||||
|
||||
def raw_live_logreader(services: list[str] = ALL_SERVICES, addr: str = '127.0.0.1') -> RawLogIterable:
|
||||
if addr != "127.0.0.1":
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
|
||||
poller = messaging.Poller()
|
||||
|
||||
for m in services:
|
||||
messaging.sub_sock(m, poller, addr=addr)
|
||||
|
||||
while True:
|
||||
polld = poller.poll(100)
|
||||
for sock in polld:
|
||||
msg = sock.receive()
|
||||
yield msg
|
||||
|
||||
|
||||
def live_logreader(services: list[str] = ALL_SERVICES, addr: str = '127.0.0.1') -> LogIterable:
|
||||
for m in raw_live_logreader(services, addr):
|
||||
with capnp_log.Event.from_bytes(m) as evt:
|
||||
yield evt
|
||||
@@ -0,0 +1,84 @@
|
||||
import numpy as np
|
||||
|
||||
|
||||
def flatten_type_dict(d, sep="/", prefix=None):
|
||||
res = {}
|
||||
if isinstance(d, dict):
|
||||
for key, val in d.items():
|
||||
if prefix is None:
|
||||
res.update(flatten_type_dict(val, prefix=key))
|
||||
else:
|
||||
res.update(flatten_type_dict(val, prefix=prefix + sep + key))
|
||||
return res
|
||||
elif isinstance(d, list):
|
||||
return {prefix: np.array(d)}
|
||||
else:
|
||||
return {prefix: d}
|
||||
|
||||
|
||||
def get_message_dict(message, typ):
|
||||
valid = message.valid
|
||||
message = message._get(typ)
|
||||
if not hasattr(message, 'to_dict') or typ in ('qcomGnss', 'ubloxGnss'):
|
||||
# TODO: support these
|
||||
#print("skipping", typ)
|
||||
return
|
||||
|
||||
msg_dict = message.to_dict(verbose=True)
|
||||
msg_dict = flatten_type_dict(msg_dict)
|
||||
msg_dict['_valid'] = valid
|
||||
return msg_dict
|
||||
|
||||
|
||||
def append_dict(path, t, d, values):
|
||||
if path not in values:
|
||||
group = {}
|
||||
group["t"] = []
|
||||
for k in d:
|
||||
group[k] = []
|
||||
values[path] = group
|
||||
else:
|
||||
group = values[path]
|
||||
|
||||
group["t"].append(t)
|
||||
for k, v in d.items():
|
||||
group[k].append(v)
|
||||
|
||||
|
||||
def potentially_ragged_array(arr, dtype=None, **kwargs):
|
||||
# TODO: is there a better way to detect inhomogeneous shapes?
|
||||
try:
|
||||
return np.array(arr, dtype=dtype, **kwargs)
|
||||
except ValueError:
|
||||
return np.array(arr, dtype=object, **kwargs)
|
||||
|
||||
def msgs_to_time_series(msgs):
|
||||
"""
|
||||
Convert an iterable of canonical capnp messages into a dictionary of time series.
|
||||
Each time series has a value with key "t" which consists of monotonically increasing timestamps
|
||||
in seconds.
|
||||
"""
|
||||
values = {}
|
||||
for msg in msgs:
|
||||
typ = msg.which()
|
||||
|
||||
tm = msg.logMonoTime / 1.0e9
|
||||
msg_dict = get_message_dict(msg, typ)
|
||||
if msg_dict is not None:
|
||||
append_dict(typ, tm, msg_dict, values)
|
||||
|
||||
# Sort values by time.
|
||||
for group in values.values():
|
||||
order = np.argsort(group["t"])
|
||||
for name, group_values in group.items():
|
||||
group[name] = potentially_ragged_array(group_values)[order]
|
||||
|
||||
return values
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
m = msgs_to_time_series(LogReader(sys.argv[1]))
|
||||
print(m['driverCameraState']['t'])
|
||||
print(np.diff(m['driverCameraState']['timestampSof']))
|
||||
Executable
+339
@@ -0,0 +1,339 @@
|
||||
#!/usr/bin/env python3
|
||||
import bz2
|
||||
from functools import cache, partial
|
||||
import multiprocessing
|
||||
import capnp
|
||||
import enum
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
import tqdm
|
||||
import urllib.parse
|
||||
import warnings
|
||||
import zstandard as zstd
|
||||
|
||||
from collections.abc import Callable, Iterable, Iterator
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from cereal import log as capnp_log
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
from openpilot.tools.lib.comma_car_segments import get_url as get_comma_segments_url
|
||||
from openpilot.tools.lib.openpilotci import get_url
|
||||
from openpilot.tools.lib.filereader import FileReader, file_exists, internal_source_available
|
||||
from openpilot.tools.lib.route import Route, SegmentRange
|
||||
from openpilot.tools.lib.log_time_series import msgs_to_time_series
|
||||
|
||||
LogMessage = type[capnp._DynamicStructReader]
|
||||
LogIterable = Iterable[LogMessage]
|
||||
RawLogIterable = Iterable[bytes]
|
||||
|
||||
|
||||
def save_log(dest, log_msgs, compress=True):
|
||||
dat = b"".join(msg.as_builder().to_bytes() for msg in log_msgs)
|
||||
|
||||
if compress and dest.endswith(".bz2"):
|
||||
dat = bz2.compress(dat)
|
||||
elif compress and dest.endswith(".zst"):
|
||||
dat = zstd.compress(dat, 10)
|
||||
|
||||
with open(dest, "wb") as f:
|
||||
f.write(dat)
|
||||
|
||||
def decompress_stream(data: bytes):
|
||||
dctx = zstd.ZstdDecompressor()
|
||||
decompressed_data = b""
|
||||
|
||||
with dctx.stream_reader(data) as reader:
|
||||
decompressed_data = reader.read()
|
||||
|
||||
return decompressed_data
|
||||
|
||||
class _LogFileReader:
|
||||
def __init__(self, fn, canonicalize=True, only_union_types=False, sort_by_time=False, dat=None):
|
||||
self.data_version = None
|
||||
self._only_union_types = only_union_types
|
||||
|
||||
ext = None
|
||||
if not dat:
|
||||
_, ext = os.path.splitext(urllib.parse.urlparse(fn).path)
|
||||
if ext not in ('', '.bz2', '.zst'):
|
||||
# old rlogs weren't compressed
|
||||
raise ValueError(f"unknown extension {ext}")
|
||||
|
||||
with FileReader(fn) as f:
|
||||
dat = f.read()
|
||||
|
||||
if ext == ".bz2" or dat.startswith(b'BZh9'):
|
||||
dat = bz2.decompress(dat)
|
||||
elif ext == ".zst" or dat.startswith(b'\x28\xB5\x2F\xFD'):
|
||||
# https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#zstandard-frames
|
||||
dat = decompress_stream(dat)
|
||||
|
||||
ents = capnp_log.Event.read_multiple_bytes(dat)
|
||||
|
||||
self._ents = []
|
||||
try:
|
||||
for e in ents:
|
||||
self._ents.append(e)
|
||||
except capnp.KjException:
|
||||
warnings.warn("Corrupted events detected", RuntimeWarning, stacklevel=1)
|
||||
|
||||
if sort_by_time:
|
||||
self._ents.sort(key=lambda x: x.logMonoTime)
|
||||
|
||||
def __iter__(self) -> Iterator[capnp._DynamicStructReader]:
|
||||
for ent in self._ents:
|
||||
if self._only_union_types:
|
||||
try:
|
||||
ent.which()
|
||||
yield ent
|
||||
except capnp.lib.capnp.KjException:
|
||||
pass
|
||||
else:
|
||||
yield ent
|
||||
|
||||
|
||||
class ReadMode(enum.StrEnum):
|
||||
RLOG = "r" # only read rlogs
|
||||
QLOG = "q" # only read qlogs
|
||||
SANITIZED = "s" # read from the commaCarSegments database
|
||||
AUTO = "a" # default to rlogs, fallback to qlogs
|
||||
AUTO_INTERACTIVE = "i" # default to rlogs, fallback to qlogs with a prompt from the user
|
||||
|
||||
|
||||
LogPath = str | None
|
||||
ValidFileCallable = Callable[[LogPath], bool]
|
||||
Source = Callable[[SegmentRange, ReadMode], list[LogPath]]
|
||||
|
||||
InternalUnavailableException = Exception("Internal source not available")
|
||||
|
||||
|
||||
class LogsUnavailable(Exception):
|
||||
pass
|
||||
|
||||
|
||||
@cache
|
||||
def default_valid_file(fn: LogPath) -> bool:
|
||||
return fn is not None and file_exists(fn)
|
||||
|
||||
|
||||
def auto_strategy(rlog_paths: list[LogPath], qlog_paths: list[LogPath], interactive: bool, valid_file: ValidFileCallable) -> list[LogPath]:
|
||||
# auto select logs based on availability
|
||||
missing_rlogs = [rlog is None or not valid_file(rlog) for rlog in rlog_paths].count(True)
|
||||
if missing_rlogs != 0:
|
||||
if interactive:
|
||||
if input(f"{missing_rlogs}/{len(rlog_paths)} rlogs were not found, would you like to fallback to qlogs for those segments? (y/n) ").lower() != "y":
|
||||
return rlog_paths
|
||||
else:
|
||||
cloudlog.warning(f"{missing_rlogs}/{len(rlog_paths)} rlogs were not found, falling back to qlogs for those segments...")
|
||||
|
||||
return [rlog if valid_file(rlog) else (qlog if valid_file(qlog) else None)
|
||||
for (rlog, qlog) in zip(rlog_paths, qlog_paths, strict=True)]
|
||||
return rlog_paths
|
||||
|
||||
|
||||
def apply_strategy(mode: ReadMode, rlog_paths: list[LogPath], qlog_paths: list[LogPath], valid_file: ValidFileCallable = default_valid_file) -> list[LogPath]:
|
||||
if mode == ReadMode.RLOG:
|
||||
return rlog_paths
|
||||
elif mode == ReadMode.QLOG:
|
||||
return qlog_paths
|
||||
elif mode == ReadMode.AUTO:
|
||||
return auto_strategy(rlog_paths, qlog_paths, False, valid_file)
|
||||
elif mode == ReadMode.AUTO_INTERACTIVE:
|
||||
return auto_strategy(rlog_paths, qlog_paths, True, valid_file)
|
||||
raise ValueError(f"invalid mode: {mode}")
|
||||
|
||||
|
||||
def comma_api_source(sr: SegmentRange, mode: ReadMode) -> list[LogPath]:
|
||||
route = Route(sr.route_name)
|
||||
|
||||
rlog_paths = [route.log_paths()[seg] for seg in sr.seg_idxs]
|
||||
qlog_paths = [route.qlog_paths()[seg] for seg in sr.seg_idxs]
|
||||
|
||||
# comma api will have already checked if the file exists
|
||||
def valid_file(fn):
|
||||
return fn is not None
|
||||
|
||||
return apply_strategy(mode, rlog_paths, qlog_paths, valid_file=valid_file)
|
||||
|
||||
|
||||
def internal_source(sr: SegmentRange, mode: ReadMode, file_ext: str = "bz2") -> list[LogPath]:
|
||||
if not internal_source_available():
|
||||
raise InternalUnavailableException
|
||||
|
||||
def get_internal_url(sr: SegmentRange, seg, file):
|
||||
return f"cd:/{sr.dongle_id}/{sr.log_id}/{seg}/{file}.{file_ext}"
|
||||
|
||||
# TODO: list instead of using static URLs to support routes with multiple file extensions
|
||||
rlog_paths = [get_internal_url(sr, seg, "rlog") for seg in sr.seg_idxs]
|
||||
qlog_paths = [get_internal_url(sr, seg, "qlog") for seg in sr.seg_idxs]
|
||||
|
||||
return apply_strategy(mode, rlog_paths, qlog_paths)
|
||||
|
||||
|
||||
def internal_source_zst(sr: SegmentRange, mode: ReadMode, file_ext: str = "zst") -> list[LogPath]:
|
||||
return internal_source(sr, mode, file_ext)
|
||||
|
||||
|
||||
def openpilotci_source(sr: SegmentRange, mode: ReadMode, file_ext: str = "bz2") -> list[LogPath]:
|
||||
rlog_paths = [get_url(sr.route_name, seg, f"rlog.{file_ext}") for seg in sr.seg_idxs]
|
||||
qlog_paths = [get_url(sr.route_name, seg, f"qlog.{file_ext}") for seg in sr.seg_idxs]
|
||||
|
||||
return apply_strategy(mode, rlog_paths, qlog_paths)
|
||||
|
||||
|
||||
def openpilotci_source_zst(sr: SegmentRange, mode: ReadMode) -> list[LogPath]:
|
||||
return openpilotci_source(sr, mode, "zst")
|
||||
|
||||
|
||||
def comma_car_segments_source(sr: SegmentRange, mode=ReadMode.RLOG) -> list[LogPath]:
|
||||
return [get_comma_segments_url(sr.route_name, seg) for seg in sr.seg_idxs]
|
||||
|
||||
|
||||
def testing_closet_source(sr: SegmentRange, mode=ReadMode.RLOG) -> list[LogPath]:
|
||||
if not internal_source_available('http://testing.comma.life'):
|
||||
raise InternalUnavailableException
|
||||
return [f"http://testing.comma.life/download/{sr.route_name.replace('|', '/')}/{seg}/rlog" for seg in sr.seg_idxs]
|
||||
|
||||
|
||||
def direct_source(file_or_url: str) -> list[LogPath]:
|
||||
return [file_or_url]
|
||||
|
||||
|
||||
def get_invalid_files(files):
|
||||
for f in files:
|
||||
if f is None or not file_exists(f):
|
||||
yield f
|
||||
|
||||
|
||||
def check_source(source: Source, *args) -> list[LogPath]:
|
||||
files = source(*args)
|
||||
assert len(files) > 0, "No files on source"
|
||||
assert next(get_invalid_files(files), False) is False, "Some files are invalid"
|
||||
return files
|
||||
|
||||
|
||||
def auto_source(sr: SegmentRange, mode=ReadMode.RLOG, sources: list[Source] = None) -> list[LogPath]:
|
||||
if mode == ReadMode.SANITIZED:
|
||||
return comma_car_segments_source(sr, mode)
|
||||
|
||||
if sources is None:
|
||||
sources = [internal_source, internal_source_zst, openpilotci_source, openpilotci_source_zst,
|
||||
comma_api_source, comma_car_segments_source, testing_closet_source]
|
||||
exceptions = {}
|
||||
|
||||
# for automatic fallback modes, auto_source needs to first check if rlogs exist for any source
|
||||
if mode in [ReadMode.AUTO, ReadMode.AUTO_INTERACTIVE]:
|
||||
for source in sources:
|
||||
try:
|
||||
return check_source(source, sr, ReadMode.RLOG)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Automatically determine viable source
|
||||
for source in sources:
|
||||
try:
|
||||
return check_source(source, sr, mode)
|
||||
except Exception as e:
|
||||
exceptions[source.__name__] = e
|
||||
|
||||
raise LogsUnavailable("auto_source could not find any valid source, exceptions for sources:\n - " +
|
||||
"\n - ".join([f"{k}: {repr(v)}" for k, v in exceptions.items()]))
|
||||
|
||||
|
||||
def parse_indirect(identifier: str) -> str:
|
||||
if "useradmin.comma.ai" in identifier:
|
||||
query = parse_qs(urlparse(identifier).query)
|
||||
return query["onebox"][0]
|
||||
return identifier
|
||||
|
||||
|
||||
def parse_direct(identifier: str):
|
||||
if identifier.startswith(("http://", "https://", "cd:/")) or pathlib.Path(identifier).exists():
|
||||
return identifier
|
||||
return None
|
||||
|
||||
|
||||
class LogReader:
|
||||
def _parse_identifier(self, identifier: str) -> list[LogPath]:
|
||||
# useradmin, etc.
|
||||
identifier = parse_indirect(identifier)
|
||||
|
||||
# direct url or file
|
||||
direct_parsed = parse_direct(identifier)
|
||||
if direct_parsed is not None:
|
||||
return direct_source(identifier)
|
||||
|
||||
sr = SegmentRange(identifier)
|
||||
mode = self.default_mode if sr.selector is None else ReadMode(sr.selector)
|
||||
|
||||
identifiers = self.source(sr, mode)
|
||||
|
||||
invalid_count = len(list(get_invalid_files(identifiers)))
|
||||
assert invalid_count == 0, (f"{invalid_count}/{len(identifiers)} invalid log(s) found, please ensure all logs " +
|
||||
"are uploaded or auto fallback to qlogs with '/a' selector at the end of the route name.")
|
||||
return identifiers
|
||||
|
||||
def __init__(self, identifier: str | list[str], default_mode: ReadMode = ReadMode.RLOG,
|
||||
source: Source = auto_source, sort_by_time=False, only_union_types=False):
|
||||
self.default_mode = default_mode
|
||||
self.source = source
|
||||
self.identifier = identifier
|
||||
if isinstance(identifier, str):
|
||||
self.identifier = [identifier]
|
||||
|
||||
self.sort_by_time = sort_by_time
|
||||
self.only_union_types = only_union_types
|
||||
|
||||
self.__lrs: dict[int, _LogFileReader] = {}
|
||||
self.reset()
|
||||
|
||||
def _get_lr(self, i):
|
||||
if i not in self.__lrs:
|
||||
self.__lrs[i] = _LogFileReader(self.logreader_identifiers[i], sort_by_time=self.sort_by_time, only_union_types=self.only_union_types)
|
||||
return self.__lrs[i]
|
||||
|
||||
def __iter__(self):
|
||||
for i in range(len(self.logreader_identifiers)):
|
||||
yield from self._get_lr(i)
|
||||
|
||||
def _run_on_segment(self, func, i):
|
||||
return func(self._get_lr(i))
|
||||
|
||||
def run_across_segments(self, num_processes, func, desc=None):
|
||||
with multiprocessing.Pool(num_processes) as pool:
|
||||
ret = []
|
||||
num_segs = len(self.logreader_identifiers)
|
||||
for p in tqdm.tqdm(pool.imap(partial(self._run_on_segment, func), range(num_segs)), total=num_segs, desc=desc):
|
||||
ret.extend(p)
|
||||
return ret
|
||||
|
||||
def reset(self):
|
||||
self.logreader_identifiers = []
|
||||
for identifier in self.identifier:
|
||||
self.logreader_identifiers.extend(self._parse_identifier(identifier))
|
||||
|
||||
@staticmethod
|
||||
def from_bytes(dat):
|
||||
return _LogFileReader("", dat=dat)
|
||||
|
||||
def filter(self, msg_type: str):
|
||||
return (getattr(m, m.which()) for m in filter(lambda m: m.which() == msg_type, self))
|
||||
|
||||
def first(self, msg_type: str):
|
||||
return next(self.filter(msg_type), None)
|
||||
|
||||
@property
|
||||
def time_series(self):
|
||||
return msgs_to_time_series(self)
|
||||
|
||||
if __name__ == "__main__":
|
||||
import codecs
|
||||
|
||||
# capnproto <= 0.8.0 throws errors converting byte data to string
|
||||
# below line catches those errors and replaces the bytes with \x__
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
log_path = sys.argv[1]
|
||||
lr = LogReader(log_path, sort_by_time=True)
|
||||
for msg in lr:
|
||||
print(msg)
|
||||
@@ -0,0 +1,12 @@
|
||||
from openpilot.tools.lib.openpilotcontainers import OpenpilotCIContainer
|
||||
|
||||
def get_url(*args, **kwargs):
|
||||
return OpenpilotCIContainer.get_url(*args, **kwargs)
|
||||
|
||||
def upload_file(*args, **kwargs):
|
||||
return OpenpilotCIContainer.upload_file(*args, **kwargs)
|
||||
|
||||
def upload_bytes(*args, **kwargs):
|
||||
return OpenpilotCIContainer.upload_bytes(*args, **kwargs)
|
||||
|
||||
BASE_URL = OpenpilotCIContainer.BASE_URL
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
from openpilot.tools.lib.azure_container import AzureContainer
|
||||
|
||||
OpenpilotCIContainer = AzureContainer("commadataci", "openpilotci")
|
||||
DataCIContainer = AzureContainer("commadataci", "commadataci")
|
||||
DataProdContainer = AzureContainer("commadata2", "commadata2")
|
||||
@@ -0,0 +1,298 @@
|
||||
import os
|
||||
import re
|
||||
from functools import cache
|
||||
from urllib.parse import urlparse
|
||||
from collections import defaultdict
|
||||
from itertools import chain
|
||||
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
from openpilot.tools.lib.helpers import RE
|
||||
|
||||
QLOG_FILENAMES = ['qlog', 'qlog.bz2', 'qlog.zst']
|
||||
QCAMERA_FILENAMES = ['qcamera.ts']
|
||||
LOG_FILENAMES = ['rlog', 'rlog.bz2', 'raw_log.bz2', 'rlog.zst']
|
||||
CAMERA_FILENAMES = ['fcamera.hevc', 'video.hevc']
|
||||
DCAMERA_FILENAMES = ['dcamera.hevc']
|
||||
ECAMERA_FILENAMES = ['ecamera.hevc']
|
||||
|
||||
|
||||
class Route:
|
||||
def __init__(self, name, data_dir=None):
|
||||
self._name = RouteName(name)
|
||||
self.files = None
|
||||
if data_dir is not None:
|
||||
self._segments = self._get_segments_local(data_dir)
|
||||
else:
|
||||
self._segments = self._get_segments_remote()
|
||||
self.max_seg_number = self._segments[-1].name.segment_num
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
@property
|
||||
def segments(self):
|
||||
return self._segments
|
||||
|
||||
def log_paths(self):
|
||||
log_path_by_seg_num = {s.name.segment_num: s.log_path for s in self._segments}
|
||||
return [log_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
def qlog_paths(self):
|
||||
qlog_path_by_seg_num = {s.name.segment_num: s.qlog_path for s in self._segments}
|
||||
return [qlog_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
def camera_paths(self):
|
||||
camera_path_by_seg_num = {s.name.segment_num: s.camera_path for s in self._segments}
|
||||
return [camera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
def dcamera_paths(self):
|
||||
dcamera_path_by_seg_num = {s.name.segment_num: s.dcamera_path for s in self._segments}
|
||||
return [dcamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
def ecamera_paths(self):
|
||||
ecamera_path_by_seg_num = {s.name.segment_num: s.ecamera_path for s in self._segments}
|
||||
return [ecamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
def qcamera_paths(self):
|
||||
qcamera_path_by_seg_num = {s.name.segment_num: s.qcamera_path for s in self._segments}
|
||||
return [qcamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number + 1)]
|
||||
|
||||
# TODO: refactor this, it's super repetitive
|
||||
def _get_segments_remote(self):
|
||||
api = CommaApi(get_token())
|
||||
route_files = api.get('v1/route/' + self.name.canonical_name + '/files')
|
||||
self.files = list(chain.from_iterable(route_files.values()))
|
||||
|
||||
segments = {}
|
||||
for url in self.files:
|
||||
_, dongle_id, time_str, segment_num, fn = urlparse(url).path.rsplit('/', maxsplit=4)
|
||||
segment_name = f'{dongle_id}|{time_str}--{segment_num}'
|
||||
if segments.get(segment_name):
|
||||
segments[segment_name] = Segment(
|
||||
segment_name,
|
||||
url if fn in LOG_FILENAMES else segments[segment_name].log_path,
|
||||
url if fn in QLOG_FILENAMES else segments[segment_name].qlog_path,
|
||||
url if fn in CAMERA_FILENAMES else segments[segment_name].camera_path,
|
||||
url if fn in DCAMERA_FILENAMES else segments[segment_name].dcamera_path,
|
||||
url if fn in ECAMERA_FILENAMES else segments[segment_name].ecamera_path,
|
||||
url if fn in QCAMERA_FILENAMES else segments[segment_name].qcamera_path,
|
||||
)
|
||||
else:
|
||||
segments[segment_name] = Segment(
|
||||
segment_name,
|
||||
url if fn in LOG_FILENAMES else None,
|
||||
url if fn in QLOG_FILENAMES else None,
|
||||
url if fn in CAMERA_FILENAMES else None,
|
||||
url if fn in DCAMERA_FILENAMES else None,
|
||||
url if fn in ECAMERA_FILENAMES else None,
|
||||
url if fn in QCAMERA_FILENAMES else None,
|
||||
)
|
||||
|
||||
return sorted(segments.values(), key=lambda seg: seg.name.segment_num)
|
||||
|
||||
def _get_segments_local(self, data_dir):
|
||||
files = os.listdir(data_dir)
|
||||
segment_files = defaultdict(list)
|
||||
|
||||
for f in files:
|
||||
fullpath = os.path.join(data_dir, f)
|
||||
explorer_match = re.match(RE.EXPLORER_FILE, f)
|
||||
op_match = re.match(RE.OP_SEGMENT_DIR, f)
|
||||
|
||||
if explorer_match:
|
||||
segment_name = explorer_match.group('segment_name')
|
||||
fn = explorer_match.group('file_name')
|
||||
if segment_name.replace('_', '|').startswith(self.name.canonical_name):
|
||||
segment_files[segment_name].append((fullpath, fn))
|
||||
elif op_match and os.path.isdir(fullpath):
|
||||
segment_name = op_match.group('segment_name')
|
||||
if segment_name.startswith(self.name.canonical_name):
|
||||
for seg_f in os.listdir(fullpath):
|
||||
segment_files[segment_name].append((os.path.join(fullpath, seg_f), seg_f))
|
||||
elif f == self.name.canonical_name:
|
||||
for seg_num in os.listdir(fullpath):
|
||||
if not seg_num.isdigit():
|
||||
continue
|
||||
|
||||
segment_name = f'{self.name.canonical_name}--{seg_num}'
|
||||
for seg_f in os.listdir(os.path.join(fullpath, seg_num)):
|
||||
segment_files[segment_name].append((os.path.join(fullpath, seg_num, seg_f), seg_f))
|
||||
|
||||
segments = []
|
||||
for segment, files in segment_files.items():
|
||||
|
||||
try:
|
||||
log_path = next(path for path, filename in files if filename in LOG_FILENAMES)
|
||||
except StopIteration:
|
||||
log_path = None
|
||||
|
||||
try:
|
||||
qlog_path = next(path for path, filename in files if filename in QLOG_FILENAMES)
|
||||
except StopIteration:
|
||||
qlog_path = None
|
||||
|
||||
try:
|
||||
camera_path = next(path for path, filename in files if filename in CAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
camera_path = None
|
||||
|
||||
try:
|
||||
dcamera_path = next(path for path, filename in files if filename in DCAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
dcamera_path = None
|
||||
|
||||
try:
|
||||
ecamera_path = next(path for path, filename in files if filename in ECAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
ecamera_path = None
|
||||
|
||||
try:
|
||||
qcamera_path = next(path for path, filename in files if filename in QCAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
qcamera_path = None
|
||||
|
||||
segments.append(Segment(segment, log_path, qlog_path, camera_path, dcamera_path, ecamera_path, qcamera_path))
|
||||
|
||||
if len(segments) == 0:
|
||||
raise ValueError(f'Could not find segments for route {self.name.canonical_name} in data directory {data_dir}')
|
||||
return sorted(segments, key=lambda seg: seg.name.segment_num)
|
||||
|
||||
|
||||
class Segment:
|
||||
def __init__(self, name, log_path, qlog_path, camera_path, dcamera_path, ecamera_path, qcamera_path):
|
||||
self._name = SegmentName(name)
|
||||
self.log_path = log_path
|
||||
self.qlog_path = qlog_path
|
||||
self.camera_path = camera_path
|
||||
self.dcamera_path = dcamera_path
|
||||
self.ecamera_path = ecamera_path
|
||||
self.qcamera_path = qcamera_path
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
|
||||
class RouteName:
|
||||
def __init__(self, name_str: str):
|
||||
self._name_str = name_str
|
||||
delim = next(c for c in self._name_str if c in ("|", "/"))
|
||||
self._dongle_id, self._time_str = self._name_str.split(delim)
|
||||
|
||||
assert len(self._dongle_id) == 16, self._name_str
|
||||
assert len(self._time_str) == 20, self._name_str
|
||||
self._canonical_name = f"{self._dongle_id}|{self._time_str}"
|
||||
|
||||
@property
|
||||
def canonical_name(self) -> str: return self._canonical_name
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str: return self._dongle_id
|
||||
|
||||
@property
|
||||
def time_str(self) -> str: return self._time_str
|
||||
|
||||
def __str__(self) -> str: return self._canonical_name
|
||||
|
||||
|
||||
class SegmentName:
|
||||
# TODO: add constructor that takes dongle_id, time_str, segment_num and then create instances
|
||||
# of this class instead of manually constructing a segment name (use canonical_name prop instead)
|
||||
def __init__(self, name_str: str, allow_route_name=False):
|
||||
data_dir_path_separator_index = name_str.rsplit("|", 1)[0].rfind("/")
|
||||
use_data_dir = (data_dir_path_separator_index != -1) and ("|" in name_str)
|
||||
self._name_str = name_str[data_dir_path_separator_index + 1:] if use_data_dir else name_str
|
||||
self._data_dir = name_str[:data_dir_path_separator_index] if use_data_dir else None
|
||||
|
||||
seg_num_delim = "--" if self._name_str.count("--") == 2 else "/"
|
||||
name_parts = self._name_str.rsplit(seg_num_delim, 1)
|
||||
if allow_route_name and len(name_parts) == 1:
|
||||
name_parts.append("-1") # no segment number
|
||||
self._route_name = RouteName(name_parts[0])
|
||||
self._num = int(name_parts[1])
|
||||
self._canonical_name = f"{self._route_name._dongle_id}|{self._route_name._time_str}--{self._num}"
|
||||
|
||||
@property
|
||||
def canonical_name(self) -> str: return self._canonical_name
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str: return self._route_name.dongle_id
|
||||
|
||||
@property
|
||||
def time_str(self) -> str: return self._route_name.time_str
|
||||
|
||||
@property
|
||||
def segment_num(self) -> int: return self._num
|
||||
|
||||
@property
|
||||
def route_name(self) -> RouteName: return self._route_name
|
||||
|
||||
@property
|
||||
def data_dir(self) -> str | None: return self._data_dir
|
||||
|
||||
def __str__(self) -> str: return self._canonical_name
|
||||
|
||||
|
||||
@cache
|
||||
def get_max_seg_number_cached(sr: 'SegmentRange') -> int:
|
||||
try:
|
||||
api = CommaApi(get_token())
|
||||
max_seg_number = api.get("/v1/route/" + sr.route_name.replace("/", "|"))["maxqlog"]
|
||||
assert isinstance(max_seg_number, int)
|
||||
return max_seg_number
|
||||
except Exception as e:
|
||||
raise Exception("unable to get max_segment_number. ensure you have access to this route or the route is public.") from e
|
||||
|
||||
|
||||
class SegmentRange:
|
||||
def __init__(self, segment_range: str):
|
||||
m = re.fullmatch(RE.SEGMENT_RANGE, segment_range)
|
||||
assert m is not None, f"Segment range is not valid {segment_range}"
|
||||
self.m = m
|
||||
|
||||
@property
|
||||
def route_name(self) -> str:
|
||||
return self.m.group("route_name")
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str:
|
||||
return self.m.group("dongle_id")
|
||||
|
||||
@property
|
||||
def log_id(self) -> str:
|
||||
return self.m.group("log_id")
|
||||
|
||||
@property
|
||||
def slice(self) -> str:
|
||||
return self.m.group("slice") or ""
|
||||
|
||||
@property
|
||||
def selector(self) -> str | None:
|
||||
return self.m.group("selector")
|
||||
|
||||
@property
|
||||
def seg_idxs(self) -> list[int]:
|
||||
m = re.fullmatch(RE.SLICE, self.slice)
|
||||
assert m is not None, f"Invalid slice: {self.slice}"
|
||||
start, end, step = (None if s is None else int(s) for s in m.groups())
|
||||
|
||||
# one segment specified
|
||||
if start is not None and end is None and ':' not in self.slice:
|
||||
if start < 0:
|
||||
start += get_max_seg_number_cached(self) + 1
|
||||
return [start]
|
||||
|
||||
s = slice(start, end, step)
|
||||
# no specified end or using relative indexing, need number of segments
|
||||
if end is None or end < 0 or (start is not None and start < 0):
|
||||
return list(range(get_max_seg_number_cached(self) + 1))[s]
|
||||
else:
|
||||
return list(range(end + 1))[s]
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.dongle_id}/{self.log_id}" + (f"/{self.slice}" if self.slice else "") + (f"/{self.selector}" if self.selector else "")
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return self.__str__()
|
||||
@@ -0,0 +1,26 @@
|
||||
# Utilities for sanitizing routes of only essential data for testing car ports and doing validation.
|
||||
|
||||
from openpilot.tools.lib.logreader import LogIterable, LogMessage
|
||||
|
||||
|
||||
def sanitize_vin(vin: str):
|
||||
# (last 6 digits of vin are serial number https://en.wikipedia.org/wiki/Vehicle_identification_number)
|
||||
VIN_SENSITIVE = 6
|
||||
return vin[:-VIN_SENSITIVE] + "X" * VIN_SENSITIVE
|
||||
|
||||
|
||||
def sanitize_msg(msg: LogMessage) -> LogMessage:
|
||||
if msg.which() == "carParams":
|
||||
msg = msg.as_builder()
|
||||
msg.carParams.carVin = sanitize_vin(msg.carParams.carVin)
|
||||
msg = msg.as_reader()
|
||||
return msg
|
||||
|
||||
|
||||
PRESERVE_SERVICES = ["can", "carParams", "pandaStates", "pandaStateDEPRECATED"]
|
||||
|
||||
|
||||
def sanitize(lr: LogIterable) -> LogIterable:
|
||||
filtered = filter(lambda msg: msg.which() in PRESERVE_SERVICES, lr)
|
||||
sanitized = map(sanitize_msg, filtered)
|
||||
return sanitized
|
||||
@@ -0,0 +1,164 @@
|
||||
import logging
|
||||
import os
|
||||
import socket
|
||||
import time
|
||||
from hashlib import sha256
|
||||
from urllib3 import PoolManager, Retry
|
||||
from urllib3.response import BaseHTTPResponse
|
||||
from urllib3.util import Timeout
|
||||
|
||||
from openpilot.common.file_helpers import atomic_write_in_dir
|
||||
from openpilot.system.hardware.hw import Paths
|
||||
# Cache chunk size
|
||||
K = 1000
|
||||
CHUNK_SIZE = 1000 * K
|
||||
|
||||
logging.getLogger("urllib3").setLevel(logging.WARNING)
|
||||
|
||||
def hash_256(link: str) -> str:
|
||||
hsh = str(sha256((link.split("?")[0]).encode('utf-8')).hexdigest())
|
||||
return hsh
|
||||
|
||||
|
||||
class URLFileException(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class URLFile:
|
||||
_pool_manager: PoolManager|None = None
|
||||
|
||||
@staticmethod
|
||||
def reset() -> None:
|
||||
URLFile._pool_manager = None
|
||||
|
||||
@staticmethod
|
||||
def pool_manager() -> PoolManager:
|
||||
if URLFile._pool_manager is None:
|
||||
socket_options = [(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),]
|
||||
retries = Retry(total=5, backoff_factor=0.5, status_forcelist=[409, 429, 503, 504])
|
||||
URLFile._pool_manager = PoolManager(num_pools=10, maxsize=100, socket_options=socket_options, retries=retries)
|
||||
return URLFile._pool_manager
|
||||
|
||||
def __init__(self, url: str, timeout: int=10, debug: bool=False, cache: bool|None=None):
|
||||
self._url = url
|
||||
self._timeout = Timeout(connect=timeout, read=timeout)
|
||||
self._pos = 0
|
||||
self._length: int|None = None
|
||||
self._debug = debug
|
||||
# True by default, false if FILEREADER_CACHE is defined, but can be overwritten by the cache input
|
||||
self._force_download = not int(os.environ.get("FILEREADER_CACHE", "0"))
|
||||
if cache is not None:
|
||||
self._force_download = not cache
|
||||
|
||||
if not self._force_download:
|
||||
os.makedirs(Paths.download_cache_root(), exist_ok=True)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback) -> None:
|
||||
pass
|
||||
|
||||
def _request(self, method: str, url: str, headers: dict[str, str]|None=None) -> BaseHTTPResponse:
|
||||
return URLFile.pool_manager().request(method, url, timeout=self._timeout, headers=headers)
|
||||
|
||||
def get_length_online(self) -> int:
|
||||
response = self._request('HEAD', self._url)
|
||||
if not (200 <= response.status <= 299):
|
||||
return -1
|
||||
length = response.headers.get('content-length', 0)
|
||||
return int(length)
|
||||
|
||||
def get_length(self) -> int:
|
||||
if self._length is not None:
|
||||
return self._length
|
||||
|
||||
file_length_path = os.path.join(Paths.download_cache_root(), hash_256(self._url) + "_length")
|
||||
if not self._force_download and os.path.exists(file_length_path):
|
||||
with open(file_length_path) as file_length:
|
||||
content = file_length.read()
|
||||
self._length = int(content)
|
||||
return self._length
|
||||
|
||||
self._length = self.get_length_online()
|
||||
if not self._force_download and self._length != -1:
|
||||
with atomic_write_in_dir(file_length_path, mode="w", overwrite=True) as file_length:
|
||||
file_length.write(str(self._length))
|
||||
return self._length
|
||||
|
||||
def read(self, ll: int|None=None) -> bytes:
|
||||
if self._force_download:
|
||||
return self.read_aux(ll=ll)
|
||||
|
||||
file_begin = self._pos
|
||||
file_end = self._pos + ll if ll is not None else self.get_length()
|
||||
assert file_end != -1, f"Remote file is empty or doesn't exist: {self._url}"
|
||||
# We have to align with chunks we store. Position is the begginiing of the latest chunk that starts before or at our file
|
||||
position = (file_begin // CHUNK_SIZE) * CHUNK_SIZE
|
||||
response = b""
|
||||
while True:
|
||||
self._pos = position
|
||||
chunk_number = self._pos / CHUNK_SIZE
|
||||
file_name = hash_256(self._url) + "_" + str(chunk_number)
|
||||
full_path = os.path.join(Paths.download_cache_root(), str(file_name))
|
||||
data = None
|
||||
# If we don't have a file, download it
|
||||
if not os.path.exists(full_path):
|
||||
data = self.read_aux(ll=CHUNK_SIZE)
|
||||
with atomic_write_in_dir(full_path, mode="wb", overwrite=True) as new_cached_file:
|
||||
new_cached_file.write(data)
|
||||
else:
|
||||
with open(full_path, "rb") as cached_file:
|
||||
data = cached_file.read()
|
||||
|
||||
response += data[max(0, file_begin - position): min(CHUNK_SIZE, file_end - position)]
|
||||
|
||||
position += CHUNK_SIZE
|
||||
if position >= file_end:
|
||||
self._pos = file_end
|
||||
return response
|
||||
|
||||
def read_aux(self, ll: int|None=None) -> bytes:
|
||||
download_range = False
|
||||
headers = {}
|
||||
if self._pos != 0 or ll is not None:
|
||||
if ll is None:
|
||||
end = self.get_length() - 1
|
||||
else:
|
||||
end = min(self._pos + ll, self.get_length()) - 1
|
||||
if self._pos >= end:
|
||||
return b""
|
||||
headers['Range'] = f"bytes={self._pos}-{end}"
|
||||
download_range = True
|
||||
|
||||
if self._debug:
|
||||
t1 = time.time()
|
||||
|
||||
response = self._request('GET', self._url, headers=headers)
|
||||
ret = response.data
|
||||
|
||||
if self._debug:
|
||||
t2 = time.time()
|
||||
if t2 - t1 > 0.1:
|
||||
print(f"get {self._url} {headers!r} {t2 - t1:.3f} slow")
|
||||
|
||||
response_code = response.status
|
||||
if response_code == 416: # Requested Range Not Satisfiable
|
||||
raise URLFileException(f"Error, range out of bounds {response_code} {headers} ({self._url}): {repr(ret)[:500]}")
|
||||
if download_range and response_code != 206: # Partial Content
|
||||
raise URLFileException(f"Error, requested range but got unexpected response {response_code} {headers} ({self._url}): {repr(ret)[:500]}")
|
||||
if (not download_range) and response_code != 200: # OK
|
||||
raise URLFileException(f"Error {response_code} {headers} ({self._url}): {repr(ret)[:500]}")
|
||||
|
||||
self._pos += len(ret)
|
||||
return ret
|
||||
|
||||
def seek(self, pos:int) -> None:
|
||||
self._pos = pos
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return self._url
|
||||
|
||||
|
||||
os.register_at_fork(after_in_child=URLFile.reset)
|
||||
Executable
+311
@@ -0,0 +1,311 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import os
|
||||
import struct
|
||||
from enum import IntEnum
|
||||
|
||||
from openpilot.tools.lib.filereader import FileReader
|
||||
|
||||
DEBUG = int(os.getenv("DEBUG", "0"))
|
||||
|
||||
# compare to ffmpeg parsing
|
||||
# ffmpeg -i <input.hevc> -c copy -bsf:v trace_headers -f null - 2>&1 | grep -B4 -A32 '] 0 '
|
||||
|
||||
# H.265 specification
|
||||
# https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.265-201802-S!!PDF-E&type=items
|
||||
|
||||
NAL_UNIT_START_CODE = b"\x00\x00\x01"
|
||||
NAL_UNIT_START_CODE_SIZE = len(NAL_UNIT_START_CODE)
|
||||
NAL_UNIT_HEADER_SIZE = 2
|
||||
|
||||
class HevcNalUnitType(IntEnum):
|
||||
TRAIL_N = 0 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
TRAIL_R = 1 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
TSA_N = 2 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
TSA_R = 3 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
STSA_N = 4 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
STSA_R = 5 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RADL_N = 6 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RADL_R = 7 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RASL_N = 8 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RASL_R = 9 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RSV_VCL_N10 = 10
|
||||
RSV_VCL_R11 = 11
|
||||
RSV_VCL_N12 = 12
|
||||
RSV_VCL_R13 = 13
|
||||
RSV_VCL_N14 = 14
|
||||
RSV_VCL_R15 = 15
|
||||
BLA_W_LP = 16 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
BLA_W_RADL = 17 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
BLA_N_LP = 18 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
IDR_W_RADL = 19 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
IDR_N_LP = 20 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
CRA_NUT = 21 # RBSP structure: slice_segment_layer_rbsp( )
|
||||
RSV_IRAP_VCL22 = 22
|
||||
RSV_IRAP_VCL23 = 23
|
||||
RSV_VCL24 = 24
|
||||
RSV_VCL25 = 25
|
||||
RSV_VCL26 = 26
|
||||
RSV_VCL27 = 27
|
||||
RSV_VCL28 = 28
|
||||
RSV_VCL29 = 29
|
||||
RSV_VCL30 = 30
|
||||
RSV_VCL31 = 31
|
||||
VPS_NUT = 32 # RBSP structure: video_parameter_set_rbsp( )
|
||||
SPS_NUT = 33 # RBSP structure: seq_parameter_set_rbsp( )
|
||||
PPS_NUT = 34 # RBSP structure: pic_parameter_set_rbsp( )
|
||||
AUD_NUT = 35
|
||||
EOS_NUT = 36
|
||||
EOB_NUT = 37
|
||||
FD_NUT = 38
|
||||
PREFIX_SEI_NUT = 39
|
||||
SUFFIX_SEI_NUT = 40
|
||||
RSV_NVCL41 = 41
|
||||
RSV_NVCL42 = 42
|
||||
RSV_NVCL43 = 43
|
||||
RSV_NVCL44 = 44
|
||||
RSV_NVCL45 = 45
|
||||
RSV_NVCL46 = 46
|
||||
RSV_NVCL47 = 47
|
||||
UNSPEC48 = 48
|
||||
UNSPEC49 = 49
|
||||
UNSPEC50 = 50
|
||||
UNSPEC51 = 51
|
||||
UNSPEC52 = 52
|
||||
UNSPEC53 = 53
|
||||
UNSPEC54 = 54
|
||||
UNSPEC55 = 55
|
||||
UNSPEC56 = 56
|
||||
UNSPEC57 = 57
|
||||
UNSPEC58 = 58
|
||||
UNSPEC59 = 59
|
||||
UNSPEC60 = 60
|
||||
UNSPEC61 = 61
|
||||
UNSPEC62 = 62
|
||||
UNSPEC63 = 63
|
||||
|
||||
# B.2.2 Byte stream NAL unit semantics
|
||||
# - The nal_unit_type within the nal_unit( ) syntax structure is equal to VPS_NUT, SPS_NUT or PPS_NUT.
|
||||
# - The byte stream NAL unit syntax structure contains the first NAL unit of an access unit in decoding
|
||||
# order, as specified in clause 7.4.2.4.4.
|
||||
HEVC_PARAMETER_SET_NAL_UNITS = (
|
||||
HevcNalUnitType.VPS_NUT,
|
||||
HevcNalUnitType.SPS_NUT,
|
||||
HevcNalUnitType.PPS_NUT,
|
||||
)
|
||||
|
||||
# 3.29 coded slice segment NAL unit: A NAL unit that has nal_unit_type in the range of TRAIL_N to RASL_R,
|
||||
# inclusive, or in the range of BLA_W_LP to RSV_IRAP_VCL23, inclusive, which indicates that the NAL unit
|
||||
# contains a coded slice segment
|
||||
HEVC_CODED_SLICE_SEGMENT_NAL_UNITS = (
|
||||
HevcNalUnitType.TRAIL_N,
|
||||
HevcNalUnitType.TRAIL_R,
|
||||
HevcNalUnitType.TSA_N,
|
||||
HevcNalUnitType.TSA_R,
|
||||
HevcNalUnitType.STSA_N,
|
||||
HevcNalUnitType.STSA_R,
|
||||
HevcNalUnitType.RADL_N,
|
||||
HevcNalUnitType.RADL_R,
|
||||
HevcNalUnitType.RASL_N,
|
||||
HevcNalUnitType.RASL_R,
|
||||
HevcNalUnitType.BLA_W_LP,
|
||||
HevcNalUnitType.BLA_W_RADL,
|
||||
HevcNalUnitType.BLA_N_LP,
|
||||
HevcNalUnitType.IDR_W_RADL,
|
||||
HevcNalUnitType.IDR_N_LP,
|
||||
HevcNalUnitType.CRA_NUT,
|
||||
)
|
||||
|
||||
class VideoFileInvalid(Exception):
|
||||
pass
|
||||
|
||||
def get_ue(dat: bytes, start_idx: int, skip_bits: int) -> tuple[int, int]:
|
||||
prefix_val = 0
|
||||
prefix_len = 0
|
||||
suffix_val = 0
|
||||
suffix_len = 0
|
||||
|
||||
i = start_idx
|
||||
while i < len(dat):
|
||||
j = 7
|
||||
while j >= 0:
|
||||
if skip_bits > 0:
|
||||
skip_bits -= 1
|
||||
elif prefix_val == 0:
|
||||
prefix_val = (dat[i] >> j) & 1
|
||||
prefix_len += 1
|
||||
else:
|
||||
suffix_val = (suffix_val << 1) | ((dat[i] >> j) & 1)
|
||||
suffix_len += 1
|
||||
j -= 1
|
||||
|
||||
if prefix_val == 1 and prefix_len - 1 == suffix_len:
|
||||
val = 2**(prefix_len-1) - 1 + suffix_val
|
||||
size = prefix_len + suffix_len
|
||||
return val, size
|
||||
i += 1
|
||||
|
||||
raise VideoFileInvalid("invalid exponential-golomb code")
|
||||
|
||||
def require_nal_unit_start(dat: bytes, nal_unit_start: int) -> None:
|
||||
if nal_unit_start < 1:
|
||||
raise ValueError("start index must be greater than zero")
|
||||
|
||||
if dat[nal_unit_start:nal_unit_start + NAL_UNIT_START_CODE_SIZE] != NAL_UNIT_START_CODE:
|
||||
raise VideoFileInvalid("data must begin with start code")
|
||||
|
||||
def get_hevc_nal_unit_length(dat: bytes, nal_unit_start: int) -> int:
|
||||
try:
|
||||
pos = dat.index(NAL_UNIT_START_CODE, nal_unit_start + NAL_UNIT_START_CODE_SIZE)
|
||||
except ValueError:
|
||||
pos = -1
|
||||
|
||||
# length of NAL unit is byte count up to next NAL unit start index
|
||||
nal_unit_len = (pos if pos != -1 else len(dat)) - nal_unit_start
|
||||
if DEBUG:
|
||||
print(" nal_unit_len:", nal_unit_len)
|
||||
return nal_unit_len
|
||||
|
||||
def get_hevc_nal_unit_type(dat: bytes, nal_unit_start: int) -> HevcNalUnitType:
|
||||
# 7.3.1.2 NAL unit header syntax
|
||||
# nal_unit_header( ) { // descriptor
|
||||
# forbidden_zero_bit f(1)
|
||||
# nal_unit_type u(6)
|
||||
# nuh_layer_id u(6)
|
||||
# nuh_temporal_id_plus1 u(3)
|
||||
# }
|
||||
header_start = nal_unit_start + NAL_UNIT_START_CODE_SIZE
|
||||
nal_unit_header = dat[header_start:header_start + NAL_UNIT_HEADER_SIZE]
|
||||
if len(nal_unit_header) != 2:
|
||||
raise VideoFileInvalid("data to short to contain nal unit header")
|
||||
nal_unit_type = HevcNalUnitType((nal_unit_header[0] >> 1) & 0x3F)
|
||||
if DEBUG:
|
||||
print(" nal_unit_type:", nal_unit_type.name, f"({nal_unit_type.value})")
|
||||
return nal_unit_type
|
||||
|
||||
def get_hevc_slice_type(dat: bytes, nal_unit_start: int, nal_unit_type: HevcNalUnitType) -> tuple[int, bool]:
|
||||
# 7.3.2.9 Slice segment layer RBSP syntax
|
||||
# slice_segment_layer_rbsp( ) {
|
||||
# slice_segment_header( )
|
||||
# slice_segment_data( )
|
||||
# rbsp_slice_segment_trailing_bits( )
|
||||
# }
|
||||
# ...
|
||||
# 7.3.6.1 General slice segment header syntax
|
||||
# slice_segment_header( ) { // descriptor
|
||||
# first_slice_segment_in_pic_flag u(1)
|
||||
# if( nal_unit_type >= BLA_W_LP && nal_unit_type <= RSV_IRAP_VCL23 )
|
||||
# no_output_of_prior_pics_flag u(1)
|
||||
# slice_pic_parameter_set_id ue(v)
|
||||
# if( !first_slice_segment_in_pic_flag ) {
|
||||
# if( dependent_slice_segments_enabled_flag )
|
||||
# dependent_slice_segment_flag u(1)
|
||||
# slice_segment_address u(v)
|
||||
# }
|
||||
# if( !dependent_slice_segment_flag ) {
|
||||
# for( i = 0; i < num_extra_slice_header_bits; i++ )
|
||||
# slice_reserved_flag[ i ] u(1)
|
||||
# slice_type ue(v)
|
||||
# ...
|
||||
|
||||
rbsp_start = nal_unit_start + NAL_UNIT_START_CODE_SIZE + NAL_UNIT_HEADER_SIZE
|
||||
skip_bits = 0
|
||||
|
||||
# 7.4.7.1 General slice segment header semantics
|
||||
# first_slice_segment_in_pic_flag equal to 1 specifies that the slice segment is the first slice segment of the picture in
|
||||
# decoding order. first_slice_segment_in_pic_flag equal to 0 specifies that the slice segment is not the first slice segment
|
||||
# of the picture in decoding order.
|
||||
is_first_slice = dat[rbsp_start] >> 7 & 1 == 1
|
||||
if not is_first_slice:
|
||||
# TODO: parse dependent_slice_segment_flag and slice_segment_address and get real slice_type
|
||||
# for now since we don't use it return -1 for slice_type
|
||||
return (-1, is_first_slice)
|
||||
skip_bits += 1 # skip past first_slice_segment_in_pic_flag
|
||||
|
||||
if nal_unit_type >= HevcNalUnitType.BLA_W_LP and nal_unit_type <= HevcNalUnitType.RSV_IRAP_VCL23:
|
||||
# 7.4.7.1 General slice segment header semantics
|
||||
# no_output_of_prior_pics_flag affects the output of previously-decoded pictures in the decoded picture buffer after the
|
||||
# decoding of an IDR or a BLA picture that is not the first picture in the bitstream as specified in Annex C.
|
||||
skip_bits += 1 # skip past no_output_of_prior_pics_flag
|
||||
|
||||
# 7.4.7.1 General slice segment header semantics
|
||||
# slice_pic_parameter_set_id specifies the value of pps_pic_parameter_set_id for the PPS in use.
|
||||
# The value of slice_pic_parameter_set_id shall be in the range of 0 to 63, inclusive.
|
||||
_, size = get_ue(dat, rbsp_start, skip_bits)
|
||||
skip_bits += size # skip past slice_pic_parameter_set_id
|
||||
|
||||
# 7.4.3.3.1 General picture parameter set RBSP semanal_unit_lenntics
|
||||
# num_extra_slice_header_bits specifies the number of extra slice header bits that are present in the slice header RBSP
|
||||
# for coded pictures referring to the PPS. The value of num_extra_slice_header_bits shall be in the range of 0 to 2, inclusive,
|
||||
# in bitstreams conforming to this version of this Specification. Other values for num_extra_slice_header_bits are reserved
|
||||
# for future use by ITU-T | ISO/IEC. However, decoders shall allow num_extra_slice_header_bits to have any value.
|
||||
# TODO: get from PPS_NUT pic_parameter_set_rbsp( ) for corresponding slice_pic_parameter_set_id
|
||||
num_extra_slice_header_bits = 0
|
||||
skip_bits += num_extra_slice_header_bits
|
||||
|
||||
# 7.4.7.1 General slice segment header semantics
|
||||
# slice_type specifies the coding type of the slice according to Table 7-7.
|
||||
# Table 7-7 - Name association to slice_type
|
||||
# slice_type | Name of slice_type
|
||||
# 0 | B (B slice)
|
||||
# 1 | P (P slice)
|
||||
# 2 | I (I slice)
|
||||
# unsigned integer 0-th order Exp-Golomb-coded syntax element with the left bit first
|
||||
slice_type, _ = get_ue(dat, rbsp_start, skip_bits)
|
||||
if DEBUG:
|
||||
print(" slice_type:", slice_type, f"(first slice: {is_first_slice})")
|
||||
if slice_type > 2:
|
||||
raise VideoFileInvalid("slice_type must be 0, 1, or 2")
|
||||
return slice_type, is_first_slice
|
||||
|
||||
def hevc_index(hevc_file_name: str, allow_corrupt: bool=False) -> tuple[list, int, bytes]:
|
||||
with FileReader(hevc_file_name) as f:
|
||||
dat = f.read()
|
||||
|
||||
if len(dat) < NAL_UNIT_START_CODE_SIZE + 1:
|
||||
raise VideoFileInvalid("data is too short")
|
||||
|
||||
if dat[0] != 0x00:
|
||||
raise VideoFileInvalid("first byte must be 0x00")
|
||||
|
||||
prefix_dat = b""
|
||||
frame_types = list()
|
||||
|
||||
i = 1 # skip past first byte 0x00
|
||||
try:
|
||||
while i < len(dat):
|
||||
require_nal_unit_start(dat, i)
|
||||
nal_unit_len = get_hevc_nal_unit_length(dat, i)
|
||||
nal_unit_type = get_hevc_nal_unit_type(dat, i)
|
||||
if nal_unit_type in HEVC_PARAMETER_SET_NAL_UNITS:
|
||||
prefix_dat += dat[i:i+nal_unit_len]
|
||||
elif nal_unit_type in HEVC_CODED_SLICE_SEGMENT_NAL_UNITS:
|
||||
slice_type, is_first_slice = get_hevc_slice_type(dat, i, nal_unit_type)
|
||||
if is_first_slice:
|
||||
frame_types.append((slice_type, i))
|
||||
i += nal_unit_len
|
||||
except Exception as e:
|
||||
if not allow_corrupt:
|
||||
raise
|
||||
print(f"ERROR: NAL unit skipped @ {i}\n", str(e))
|
||||
|
||||
return frame_types, len(dat), prefix_dat
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("input_file", type=str)
|
||||
parser.add_argument("output_prefix_file", type=str)
|
||||
parser.add_argument("output_index_file", type=str)
|
||||
args = parser.parse_args()
|
||||
|
||||
frame_types, dat_len, prefix_dat = hevc_index(args.input_file)
|
||||
with open(args.output_prefix_file, "wb") as f:
|
||||
f.write(prefix_dat)
|
||||
|
||||
with open(args.output_index_file, "wb") as f:
|
||||
for ft, fp in frame_types:
|
||||
f.write(struct.pack("<II", ft, fp))
|
||||
f.write(struct.pack("<II", 0xFFFFFFFF, dat_len))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
/longitudinal_reports/
|
||||
@@ -0,0 +1,48 @@
|
||||
# Longitudinal Maneuvers Testing Tool
|
||||
|
||||
Test your vehicle's longitudinal control tuning with this tool. The tool will test the vehicle's ability to follow a few longitudinal maneuvers and includes a tool to generate a report from the route.
|
||||
|
||||
<details><summary>Sample snapshot of a report.</summary><img width="600px" src="https://github.com/user-attachments/assets/d18d0c7d-2bde-44c1-8e86-1741ed442ad8"></details>
|
||||
|
||||
## Instructions
|
||||
|
||||
1. Check out a development branch such as `master` on your comma device.
|
||||
2. Locate either a large empty parking lot or road devoid of any car or foot traffic. Flat, straight road is preferred. The full maneuver suite can take 1 mile or more if left running, however it is recommended to disengage openpilot between maneuvers and turn around if there is not enough space.
|
||||
3. Turn off the vehicle and set this parameter which will signal to openpilot to start the longitudinal maneuver daemon:
|
||||
|
||||
```sh
|
||||
echo -n 1 > /data/params/d/LongitudinalManeuverMode
|
||||
```
|
||||
|
||||
4. Turn your vehicle back on. You will see the "Longitudinal Maneuver Mode" alert:
|
||||
|
||||
|
||||
|
||||
5. Ensure the road ahead is clear, as openpilot will not brake for any obstructions in this mode. Once you are ready, press "Set" on your steering wheel to start the tests. The tests will run for about 4 minutes. If you need to pause the tests, press "Cancel" on your steering wheel. You can resume the tests by pressing "Resume" on your steering wheel.
|
||||
|
||||
**Note:** For GM cars, it is recommended to hold down the resume button for all low-speed tests (starting, stopping and creep) to avoid the car entering standstill.
|
||||
|
||||
|
||||
|
||||
6. When the testing is complete, you'll see an alert that says "Maneuvers Finished." Complete the route by pulling over and turning off the vehicle.
|
||||
|
||||
|
||||
|
||||
7. Visit https://connect.comma.ai and locate the route(s). They will stand out with lots of orange intervals in their timeline. Ensure "All logs" show as "uploaded."
|
||||
|
||||
|
||||
|
||||
8. Gather the route ID and then run the report generator. The file will be exported to the same directory:
|
||||
|
||||
```sh
|
||||
$ python tools/longitudinal_maneuvers/generate_report.py 57048cfce01d9625/0000010e--5b26bc3be7 'pcm accel compensation'
|
||||
|
||||
processing report for LEXUS_ES_TSS2
|
||||
plotting maneuver: start from stop, runs: 4
|
||||
plotting maneuver: creep: alternate between +1m/s^2 and -1m/s^2, runs: 2
|
||||
plotting maneuver: gas step response: +1m/s^2 from 20mph, runs: 2
|
||||
|
||||
Report written to /home/batman/openpilot/tools/longitudinal_maneuvers/longitudinal_reports/LEXUS_ES_TSS2_57048cfce01d9625_0000010e--5b26bc3be7.html
|
||||
```
|
||||
|
||||
You can reach out on [Discord](https://discord.comma.ai) if you have any questions about these instructions or the tool itself.
|
||||
+187
@@ -0,0 +1,187 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import base64
|
||||
import io
|
||||
import os
|
||||
import math
|
||||
import pprint
|
||||
import webbrowser
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
import matplotlib.pyplot as plt
|
||||
from tabulate import tabulate
|
||||
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
from openpilot.system.hardware.hw import Paths
|
||||
|
||||
|
||||
def format_car_params(CP):
|
||||
return pprint.pformat({k: v for k, v in CP.to_dict().items() if not k.endswith('DEPRECATED')}, indent=2)
|
||||
|
||||
|
||||
def report(platform, route, _description, CP, ID, maneuvers):
|
||||
output_path = Path(__file__).resolve().parent / "longitudinal_reports"
|
||||
output_fn = output_path / f"{platform}_{route.replace('/', '_')}.html"
|
||||
output_path.mkdir(exist_ok=True)
|
||||
target_cross_times = defaultdict(list)
|
||||
|
||||
builder = [
|
||||
"<style>summary { cursor: pointer; }\n td, th { padding: 8px; } </style>\n",
|
||||
"<h1>Longitudinal maneuver report</h1>\n",
|
||||
f"<h3>{platform}</h3>\n",
|
||||
f"<h3>{route}</h3>\n",
|
||||
f"<h3>{ID.gitCommit}, {ID.gitBranch}, {ID.gitRemote}</h3>\n",
|
||||
]
|
||||
if _description is not None:
|
||||
builder.append(f"<h3>Description: {_description}</h3>\n")
|
||||
builder.append(f"<details><summary><h3 style='display: inline-block;'>CarParams</h3></summary><pre>{format_car_params(CP)}</pre></details>\n")
|
||||
builder.append('{ summary }') # to be replaced below
|
||||
for description, runs in maneuvers:
|
||||
print(f'plotting maneuver: {description}, runs: {len(runs)}')
|
||||
builder.append("<div style='border-top: 1px solid #000; margin: 20px 0;'></div>\n")
|
||||
builder.append(f"<h2>{description}</h2>\n")
|
||||
for run, msgs in enumerate(runs):
|
||||
t_carControl, carControl = zip(*[(m.logMonoTime, m.carControl) for m in msgs if m.which() == 'carControl'], strict=True)
|
||||
t_carOutput, carOutput = zip(*[(m.logMonoTime, m.carOutput) for m in msgs if m.which() == 'carOutput'], strict=True)
|
||||
t_carState, carState = zip(*[(m.logMonoTime, m.carState) for m in msgs if m.which() == 'carState'], strict=True)
|
||||
t_livePose, livePose = zip(*[(m.logMonoTime, m.livePose) for m in msgs if m.which() == 'livePose'], strict=True)
|
||||
t_longitudinalPlan, longitudinalPlan = zip(*[(m.logMonoTime, m.longitudinalPlan) for m in msgs if m.which() == 'longitudinalPlan'], strict=True)
|
||||
|
||||
# make time relative seconds
|
||||
t_carControl = [(t - t_carControl[0]) / 1e9 for t in t_carControl]
|
||||
t_carOutput = [(t - t_carOutput[0]) / 1e9 for t in t_carOutput]
|
||||
t_carState = [(t - t_carState[0]) / 1e9 for t in t_carState]
|
||||
t_livePose = [(t - t_livePose[0]) / 1e9 for t in t_livePose]
|
||||
t_longitudinalPlan = [(t - t_longitudinalPlan[0]) / 1e9 for t in t_longitudinalPlan]
|
||||
|
||||
# maneuver validity
|
||||
longActive = [m.longActive for m in carControl]
|
||||
maneuver_valid = all(longActive) and (not any(cs.cruiseState.standstill for cs in carState) or CP.autoResumeSng)
|
||||
|
||||
_open = 'open' if maneuver_valid else ''
|
||||
title = f'Run #{int(run)+1}' + (' <span style="color: red">(invalid maneuver!)</span>' if not maneuver_valid else '')
|
||||
|
||||
builder.append(f"<details {_open}><summary><h3 style='display: inline-block;'>{title}</h3></summary>\n")
|
||||
|
||||
# get first acceleration target and first intersection
|
||||
aTarget = longitudinalPlan[0].aTarget
|
||||
target_cross_time = None
|
||||
builder.append(f'<h3 style="font-weight: normal">Initial aTarget: {round(aTarget, 2)} m/s^2')
|
||||
|
||||
# Localizer is noisy, require two consecutive 20Hz frames above threshold
|
||||
prev_crossed = False
|
||||
for t, lp in zip(t_livePose, livePose, strict=True):
|
||||
crossed = (0 < aTarget < lp.accelerationDevice.x) or (0 > aTarget > lp.accelerationDevice.x)
|
||||
if crossed and prev_crossed:
|
||||
builder.append(f', <strong>crossed in {t:.3f}s</strong>')
|
||||
target_cross_time = t
|
||||
if maneuver_valid:
|
||||
target_cross_times[description].append(t)
|
||||
break
|
||||
prev_crossed = crossed
|
||||
else:
|
||||
builder.append(', <strong>not crossed</strong>')
|
||||
builder.append('</h3>')
|
||||
|
||||
pitches = [math.degrees(m.orientationNED[1]) for m in carControl]
|
||||
builder.append(f'<h3 style="font-weight: normal">Average pitch: <strong>{sum(pitches) / len(pitches):0.2f} degrees</strong></h3>')
|
||||
|
||||
plt.rcParams['font.size'] = 40
|
||||
fig = plt.figure(figsize=(30, 26))
|
||||
ax = fig.subplots(4, 1, sharex=True, gridspec_kw={'height_ratios': [5, 3, 1, 1]})
|
||||
|
||||
ax[0].grid(linewidth=4)
|
||||
ax[0].plot(t_carControl, [m.actuators.accel for m in carControl], label='carControl.actuators.accel', linewidth=6)
|
||||
ax[0].plot(t_carOutput, [m.actuatorsOutput.accel for m in carOutput], label='carOutput.actuatorsOutput.accel', linewidth=6)
|
||||
ax[0].plot(t_longitudinalPlan, [m.aTarget for m in longitudinalPlan], label='longitudinalPlan.aTarget', linewidth=6)
|
||||
ax[0].plot(t_carState, [m.aEgo for m in carState], label='carState.aEgo', linewidth=6)
|
||||
ax[0].plot(t_livePose, [m.accelerationDevice.x for m in livePose], label='livePose.accelerationDevice.x', linewidth=6)
|
||||
# TODO localizer accel
|
||||
ax[0].set_ylabel('Acceleration (m/s^2)')
|
||||
#ax[0].set_ylim(-6.5, 6.5)
|
||||
ax[0].legend(prop={'size': 30})
|
||||
|
||||
if target_cross_time is not None:
|
||||
ax[0].plot(target_cross_time, aTarget, marker='o', markersize=50, markeredgewidth=7, markeredgecolor='black', markerfacecolor='None')
|
||||
|
||||
ax[1].grid(linewidth=4)
|
||||
ax[1].plot(t_carState, [m.vEgo for m in carState], 'g', label='vEgo', linewidth=6)
|
||||
ax[1].set_ylabel('Velocity (m/s)')
|
||||
ax[1].legend()
|
||||
|
||||
ax[2].plot(t_carControl, longActive, label='longActive', linewidth=6)
|
||||
ax[3].plot(t_carState, [m.gasPressed for m in carState], label='gasPressed', linewidth=6)
|
||||
ax[3].plot(t_carState, [m.brakePressed for m in carState], label='brakePressed', linewidth=6)
|
||||
for i in (2, 3):
|
||||
ax[i].set_yticks([0, 1], minor=False)
|
||||
ax[i].set_ylim(-1, 2)
|
||||
ax[i].legend()
|
||||
|
||||
ax[-1].set_xlabel("Time (s)")
|
||||
fig.tight_layout()
|
||||
|
||||
buffer = io.BytesIO()
|
||||
fig.savefig(buffer, format='webp')
|
||||
plt.close(fig)
|
||||
buffer.seek(0)
|
||||
builder.append(f"<img src='data:image/webp;base64,{base64.b64encode(buffer.getvalue()).decode()}' style='width:100%; max-width:800px;'>\n")
|
||||
builder.append("</details>\n")
|
||||
|
||||
summary = ["<h2>Summary</h2>\n"]
|
||||
cols = ['maneuver', 'crossed', 'runs', 'mean', 'min', 'max']
|
||||
table = []
|
||||
for description, runs in maneuvers:
|
||||
times = target_cross_times[description]
|
||||
l = [description, len(times), len(runs)]
|
||||
if len(times):
|
||||
l.extend([round(sum(times) / len(times), 2), round(min(times), 2), round(max(times), 2)])
|
||||
table.append(l)
|
||||
summary.append(tabulate(table, headers=cols, tablefmt='html', numalign='left') + '\n')
|
||||
|
||||
sum_idx = builder.index('{ summary }')
|
||||
builder[sum_idx:sum_idx + 1] = summary
|
||||
|
||||
with open(output_fn, "w") as f:
|
||||
f.write(''.join(builder))
|
||||
|
||||
print(f"\nOpening report: {output_fn}\n")
|
||||
webbrowser.open_new_tab(str(output_fn))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Generate longitudinal maneuver report from route')
|
||||
parser.add_argument('route', type=str, help='Route name (e.g. 00000000--5f742174be)')
|
||||
parser.add_argument('description', type=str, nargs='?')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if '/' in args.route or '|' in args.route:
|
||||
lr = LogReader(args.route)
|
||||
else:
|
||||
segs = [seg for seg in os.listdir(Paths.log_root()) if args.route in seg]
|
||||
lr = LogReader([os.path.join(Paths.log_root(), seg, 'rlog.zst') for seg in segs])
|
||||
|
||||
CP = lr.first('carParams')
|
||||
ID = lr.first('initData')
|
||||
platform = CP.carFingerprint
|
||||
print('processing report for', platform)
|
||||
|
||||
maneuvers: list[tuple[str, list[list]]] = []
|
||||
active_prev = False
|
||||
description_prev = None
|
||||
|
||||
for msg in lr:
|
||||
if msg.which() == 'alertDebug':
|
||||
active = 'Maneuver Active' in msg.alertDebug.alertText1
|
||||
if active and not active_prev:
|
||||
if msg.alertDebug.alertText2 == description_prev:
|
||||
maneuvers[-1][1].append([])
|
||||
else:
|
||||
maneuvers.append((msg.alertDebug.alertText2, [[]]))
|
||||
description_prev = maneuvers[-1][0]
|
||||
active_prev = active
|
||||
|
||||
if active_prev:
|
||||
maneuvers[-1][1][-1].append(msg)
|
||||
|
||||
report(platform, args.route, args.description, CP, ID, maneuvers)
|
||||
Executable
+185
@@ -0,0 +1,185 @@
|
||||
#!/usr/bin/env python3
|
||||
import numpy as np
|
||||
from dataclasses import dataclass
|
||||
|
||||
from cereal import messaging, car
|
||||
from opendbc.car.common.conversions import Conversions as CV
|
||||
from openpilot.common.realtime import DT_MDL
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
|
||||
|
||||
@dataclass
|
||||
class Action:
|
||||
accel_bp: list[float] # m/s^2
|
||||
time_bp: list[float] # seconds
|
||||
|
||||
def __post_init__(self):
|
||||
assert len(self.accel_bp) == len(self.time_bp)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Maneuver:
|
||||
description: str
|
||||
actions: list[Action]
|
||||
repeat: int = 0
|
||||
initial_speed: float = 0. # m/s
|
||||
|
||||
_active: bool = False
|
||||
_finished: bool = False
|
||||
_action_index: int = 0
|
||||
_action_frames: int = 0
|
||||
_ready_cnt: int = 0
|
||||
_repeated: int = 0
|
||||
|
||||
def get_accel(self, v_ego: float, long_active: bool, standstill: bool, cruise_standstill: bool) -> float:
|
||||
ready = abs(v_ego - self.initial_speed) < 0.3 and long_active and not cruise_standstill
|
||||
if self.initial_speed < 0.01:
|
||||
ready = ready and standstill
|
||||
self._ready_cnt = (self._ready_cnt + 1) if ready else 0
|
||||
|
||||
if self._ready_cnt > (3. / DT_MDL):
|
||||
self._active = True
|
||||
|
||||
if not self._active:
|
||||
return min(max(self.initial_speed - v_ego, -2.), 2.)
|
||||
|
||||
action = self.actions[self._action_index]
|
||||
action_accel = np.interp(self._action_frames * DT_MDL, action.time_bp, action.accel_bp)
|
||||
|
||||
self._action_frames += 1
|
||||
|
||||
# reached duration of action
|
||||
if self._action_frames > (action.time_bp[-1] / DT_MDL):
|
||||
# next action
|
||||
if self._action_index < len(self.actions) - 1:
|
||||
self._action_index += 1
|
||||
self._action_frames = 0
|
||||
# repeat maneuver
|
||||
elif self._repeated < self.repeat:
|
||||
self._repeated += 1
|
||||
self._action_index = 0
|
||||
self._action_frames = 0
|
||||
self._active = False
|
||||
# finish maneuver
|
||||
else:
|
||||
self._finished = True
|
||||
|
||||
return float(action_accel)
|
||||
|
||||
@property
|
||||
def finished(self):
|
||||
return self._finished
|
||||
|
||||
@property
|
||||
def active(self):
|
||||
return self._active
|
||||
|
||||
|
||||
MANEUVERS = [
|
||||
Maneuver(
|
||||
"come to stop",
|
||||
[Action([-0.5], [12])],
|
||||
repeat=2,
|
||||
initial_speed=5.,
|
||||
),
|
||||
Maneuver(
|
||||
"start from stop",
|
||||
[Action([1.5], [6])],
|
||||
repeat=2,
|
||||
initial_speed=0.,
|
||||
),
|
||||
Maneuver(
|
||||
"creep: alternate between +1m/s^2 and -1m/s^2",
|
||||
[
|
||||
Action([1], [3]), Action([-1], [3]),
|
||||
Action([1], [3]), Action([-1], [3]),
|
||||
Action([1], [3]), Action([-1], [3]),
|
||||
],
|
||||
repeat=2,
|
||||
initial_speed=0.,
|
||||
),
|
||||
Maneuver(
|
||||
"brake step response: -1m/s^2 from 20mph",
|
||||
[Action([-1], [3])],
|
||||
repeat=2,
|
||||
initial_speed=20. * CV.MPH_TO_MS,
|
||||
),
|
||||
Maneuver(
|
||||
"brake step response: -4m/s^2 from 20mph",
|
||||
[Action([-4], [3])],
|
||||
repeat=2,
|
||||
initial_speed=20. * CV.MPH_TO_MS,
|
||||
),
|
||||
Maneuver(
|
||||
"gas step response: +1m/s^2 from 20mph",
|
||||
[Action([1], [3])],
|
||||
repeat=2,
|
||||
initial_speed=20. * CV.MPH_TO_MS,
|
||||
),
|
||||
Maneuver(
|
||||
"gas step response: +4m/s^2 from 20mph",
|
||||
[Action([4], [3])],
|
||||
repeat=2,
|
||||
initial_speed=20. * CV.MPH_TO_MS,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def main():
|
||||
params = Params()
|
||||
cloudlog.info("joystickd is waiting for CarParams")
|
||||
CP = messaging.log_from_bytes(params.get("CarParams", block=True), car.CarParams)
|
||||
|
||||
sm = messaging.SubMaster(['carState', 'carControl', 'controlsState', 'selfdriveState', 'modelV2'], poll='modelV2')
|
||||
pm = messaging.PubMaster(['longitudinalPlan', 'driverAssistance', 'alertDebug'])
|
||||
|
||||
maneuvers = iter(MANEUVERS)
|
||||
maneuver = None
|
||||
|
||||
while True:
|
||||
sm.update()
|
||||
|
||||
if maneuver is None:
|
||||
maneuver = next(maneuvers, None)
|
||||
|
||||
alert_msg = messaging.new_message('alertDebug')
|
||||
alert_msg.valid = True
|
||||
|
||||
plan_send = messaging.new_message('longitudinalPlan')
|
||||
plan_send.valid = sm.all_checks()
|
||||
|
||||
longitudinalPlan = plan_send.longitudinalPlan
|
||||
accel = 0
|
||||
v_ego = max(sm['carState'].vEgo, 0)
|
||||
|
||||
if maneuver is not None:
|
||||
accel = maneuver.get_accel(v_ego, sm['carControl'].longActive, sm['carState'].standstill, sm['carState'].cruiseState.standstill)
|
||||
|
||||
if maneuver.active:
|
||||
alert_msg.alertDebug.alertText1 = f'Maneuver Active: {accel:0.2f} m/s^2'
|
||||
else:
|
||||
alert_msg.alertDebug.alertText1 = f'Setting up to {maneuver.initial_speed * CV.MS_TO_MPH:0.2f} mph'
|
||||
alert_msg.alertDebug.alertText2 = f'{maneuver.description}'
|
||||
else:
|
||||
alert_msg.alertDebug.alertText1 = 'Maneuvers Finished'
|
||||
|
||||
pm.send('alertDebug', alert_msg)
|
||||
|
||||
longitudinalPlan.aTarget = accel
|
||||
longitudinalPlan.shouldStop = v_ego < CP.vEgoStopping and accel < 1e-2
|
||||
|
||||
longitudinalPlan.allowBrake = True
|
||||
longitudinalPlan.allowThrottle = True
|
||||
longitudinalPlan.hasLead = True
|
||||
|
||||
longitudinalPlan.speeds = [0.2] # triggers carControl.cruiseControl.resume in controlsd
|
||||
|
||||
pm.send('longitudinalPlan', plan_send)
|
||||
|
||||
assistance_send = messaging.new_message('driverAssistance')
|
||||
assistance_send.valid = True
|
||||
pm.send('driverAssistance', assistance_send)
|
||||
|
||||
if maneuver is not None and maneuver.finished:
|
||||
maneuver = None
|
||||
Executable
+97
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT="$(cd $DIR/../ && pwd)"
|
||||
ARCH=$(uname -m)
|
||||
|
||||
# homebrew update is slow
|
||||
export HOMEBREW_NO_AUTO_UPDATE=1
|
||||
|
||||
if [[ $SHELL == "/bin/zsh" ]]; then
|
||||
RC_FILE="$HOME/.zshrc"
|
||||
elif [[ $SHELL == "/bin/bash" ]]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
|
||||
# Install brew if required
|
||||
if [[ $(command -v brew) == "" ]]; then
|
||||
echo "Installing Homebrew"
|
||||
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
||||
echo "[ ] installed brew t=$SECONDS"
|
||||
|
||||
# make brew available now
|
||||
if [[ $ARCH == "x86_64" ]]; then
|
||||
echo 'eval "$(/usr/local/bin/brew shellenv)"' >> $RC_FILE
|
||||
eval "$(/usr/local/bin/brew shellenv)"
|
||||
else
|
||||
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> $RC_FILE
|
||||
eval "$(/opt/homebrew/bin/brew shellenv)"
|
||||
fi
|
||||
fi
|
||||
|
||||
brew bundle --file=- <<-EOS
|
||||
brew "git-lfs"
|
||||
brew "zlib"
|
||||
brew "capnp"
|
||||
brew "coreutils"
|
||||
brew "eigen"
|
||||
brew "ffmpeg"
|
||||
brew "glfw"
|
||||
brew "libarchive"
|
||||
brew "libusb"
|
||||
brew "libtool"
|
||||
brew "llvm"
|
||||
brew "openssl@3.0"
|
||||
brew "qt@5"
|
||||
brew "zeromq"
|
||||
cask "gcc-arm-embedded"
|
||||
brew "portaudio"
|
||||
brew "gcc@13"
|
||||
EOS
|
||||
|
||||
echo "[ ] finished brew install t=$SECONDS"
|
||||
|
||||
BREW_PREFIX=$(brew --prefix)
|
||||
|
||||
# archive backend tools for pip dependencies
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/zlib/lib"
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/bzip2/lib"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/zlib/include"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/bzip2/include"
|
||||
|
||||
# pycurl curl/openssl backend dependencies
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/openssl@3/lib"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/openssl@3/include"
|
||||
export PYCURL_CURL_CONFIG=/usr/bin/curl-config
|
||||
export PYCURL_SSL_LIBRARY=openssl
|
||||
|
||||
# install python dependencies
|
||||
$DIR/install_python_dependencies.sh
|
||||
echo "[ ] installed python dependencies t=$SECONDS"
|
||||
|
||||
# brew does not link qt5 by default
|
||||
# check if qt5 can be linked, if not, prompt the user to link it
|
||||
QT_BIN_LOCATION="$(command -v lupdate || :)"
|
||||
if [ -n "$QT_BIN_LOCATION" ]; then
|
||||
# if qt6 is linked, prompt the user to unlink it and link the right version
|
||||
QT_BIN_VERSION="$(lupdate -version | awk '{print $NF}')"
|
||||
if [[ ! "$QT_BIN_VERSION" =~ 5\.[0-9]+\.[0-9]+ ]]; then
|
||||
echo
|
||||
echo "lupdate/lrelease available at PATH is $QT_BIN_VERSION"
|
||||
if [[ "$QT_BIN_LOCATION" == "$(brew --prefix)/"* ]]; then
|
||||
echo "Run the following command to link qt5:"
|
||||
echo "brew unlink qt@6 && brew link qt@5"
|
||||
else
|
||||
echo "Remove conflicting qt entries from PATH and run the following command to link qt5:"
|
||||
echo "brew link qt@5"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
brew link qt@5
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "---- OPENPILOT SETUP DONE ----"
|
||||
echo "Open a new shell or configure your active shell env by running:"
|
||||
echo "source $RC_FILE"
|
||||
Executable
+460
@@ -0,0 +1,460 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [[ ! "${BASH_SOURCE[0]}" = "${0}" ]]; then
|
||||
echo "Invalid invocation! This script must not be sourced."
|
||||
echo "Run 'op.sh' directly or check your .bashrc for a valid alias"
|
||||
return 0
|
||||
fi
|
||||
|
||||
set -e
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
UNDERLINE='\033[4m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
SHELL_NAME="$(basename ${SHELL})"
|
||||
RC_FILE="${HOME}/.$(basename ${SHELL})rc"
|
||||
if [ "$(uname)" == "Darwin" ] && [ $SHELL == "/bin/bash" ]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
function op_install() {
|
||||
echo "Installing op system-wide..."
|
||||
CMD="\nalias op='"$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )/op.sh" \"\$@\"'\n"
|
||||
grep "alias op=" "$RC_FILE" &> /dev/null || printf "$CMD" >> $RC_FILE
|
||||
echo -e " ↳ [${GREEN}✔${NC}] op installed successfully. Open a new shell to use it."
|
||||
}
|
||||
|
||||
function loge() {
|
||||
if [[ -f "$LOG_FILE" ]]; then
|
||||
# error type
|
||||
echo "$1" >> $LOG_FILE
|
||||
# error log
|
||||
echo "$2" >> $LOG_FILE
|
||||
fi
|
||||
}
|
||||
|
||||
function op_run_command() {
|
||||
CMD="$@"
|
||||
|
||||
echo -e "${BOLD}Running command →${NC} $CMD │"
|
||||
for ((i=0; i<$((19 + ${#CMD})); i++)); do
|
||||
echo -n "─"
|
||||
done
|
||||
echo -e "┘\n"
|
||||
|
||||
if [[ -z "$DRY" ]]; then
|
||||
eval "$CMD"
|
||||
fi
|
||||
}
|
||||
|
||||
# be default, assume openpilot dir is in current directory
|
||||
OPENPILOT_ROOT=$(pwd)
|
||||
function op_get_openpilot_dir() {
|
||||
while [[ "$OPENPILOT_ROOT" != '/' ]];
|
||||
do
|
||||
if find "$OPENPILOT_ROOT/launch_openpilot.sh" -maxdepth 1 -mindepth 1 &> /dev/null; then
|
||||
return 0
|
||||
fi
|
||||
OPENPILOT_ROOT="$(readlink -f "$OPENPILOT_ROOT/"..)"
|
||||
done
|
||||
}
|
||||
|
||||
function op_install_post_commit() {
|
||||
op_get_openpilot_dir
|
||||
if [[ ! -d $OPENPILOT_ROOT/.git/hooks/post-commit.d ]]; then
|
||||
mkdir $OPENPILOT_ROOT/.git/hooks/post-commit.d
|
||||
mv $OPENPILOT_ROOT/.git/hooks/post-commit $OPENPILOT_ROOT/.git/hooks/post-commit.d 2>/dev/null || true
|
||||
fi
|
||||
cd $OPENPILOT_ROOT/.git/hooks
|
||||
ln -sf ../../scripts/post-commit post-commit
|
||||
}
|
||||
|
||||
function op_check_openpilot_dir() {
|
||||
echo "Checking for openpilot directory..."
|
||||
if [[ -f "$OPENPILOT_ROOT/launch_openpilot.sh" ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] openpilot found."
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo -e " ↳ [${RED}✗${NC}] openpilot directory not found! Make sure that you are"
|
||||
echo " inside the openpilot directory or specify one with the"
|
||||
echo " --dir option!"
|
||||
return 1
|
||||
}
|
||||
|
||||
function op_check_git() {
|
||||
echo "Checking for git..."
|
||||
if ! command -v "git" > /dev/null 2>&1; then
|
||||
echo -e " ↳ [${RED}✗${NC}] git not found on your system!"
|
||||
return 1
|
||||
else
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git found."
|
||||
fi
|
||||
|
||||
echo "Checking for git lfs files..."
|
||||
if [[ $(file -b $OPENPILOT_ROOT/selfdrive/modeld/models/dmonitoring_model.onnx) == "data" ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git lfs files found."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] git lfs files not found! Run 'git lfs pull'"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "Checking for git submodules..."
|
||||
for name in $(git config --file .gitmodules --get-regexp path | awk '{ print $2 }' | tr '\n' ' '); do
|
||||
if [[ -z $(ls $OPENPILOT_ROOT/$name) ]]; then
|
||||
echo -e " ↳ [${RED}✗${NC}] git submodule $name not found! Run 'git submodule update --init --recursive'"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
echo -e " ↳ [${GREEN}✔${NC}] git submodules found."
|
||||
}
|
||||
|
||||
function op_check_os() {
|
||||
echo "Checking for compatible os version..."
|
||||
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
|
||||
if [ -f "/etc/os-release" ]; then
|
||||
source /etc/os-release
|
||||
case "$VERSION_CODENAME" in
|
||||
"jammy" | "kinetic" | "noble" | "focal")
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Ubuntu $VERSION_CODENAME detected."
|
||||
;;
|
||||
* )
|
||||
echo -e " ↳ [${RED}✗${NC}] Incompatible Ubuntu version $VERSION_CODENAME detected!"
|
||||
loge "ERROR_INCOMPATIBLE_UBUNTU" "$VERSION_CODENAME"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] No /etc/os-release on your system. Make sure you're running on Ubuntu, or similar!"
|
||||
loge "ERROR_UNKNOWN_UBUNTU"
|
||||
return 1
|
||||
fi
|
||||
|
||||
elif [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] macOS detected."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] OS type $OSTYPE not supported!"
|
||||
loge "ERROR_UNKNOWN_OS" "$OSTYPE"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
function op_check_python() {
|
||||
echo "Checking for compatible python version..."
|
||||
REQUIRED_PYTHON_VERSION=$(grep "requires-python" $OPENPILOT_ROOT/pyproject.toml)
|
||||
INSTALLED_PYTHON_VERSION=$(python3 --version 2> /dev/null || true)
|
||||
|
||||
if [[ -z $INSTALLED_PYTHON_VERSION ]]; then
|
||||
echo -e " ↳ [${RED}✗${NC}] python3 not found on your system. You need python version satisfying $(echo $REQUIRED_PYTHON_VERSION | cut -d '=' -f2-) to continue!"
|
||||
loge "ERROR_PYTHON_NOT_FOUND"
|
||||
return 1
|
||||
else
|
||||
LB=$(echo $REQUIRED_PYTHON_VERSION | tr -d -c '[0-9,]' | cut -d ',' -f1)
|
||||
UB=$(echo $REQUIRED_PYTHON_VERSION | tr -d -c '[0-9,]' | cut -d ',' -f2)
|
||||
VERSION=$(echo $INSTALLED_PYTHON_VERSION | grep -o '[0-9]\+\.[0-9]\+' | tr -d -c '[0-9]')
|
||||
if [[ $VERSION -ge LB && $VERSION -lt UB ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] $INSTALLED_PYTHON_VERSION detected."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] You need a python version satisfying $(echo $REQUIRED_PYTHON_VERSION | cut -d '=' -f2-) to continue!"
|
||||
loge "ERROR_PYTHON_VERSION" "$INSTALLED_PYTHON_VERSION"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
function op_check_venv() {
|
||||
echo "Checking for venv..."
|
||||
if [[ -f $OPENPILOT_ROOT/.venv/bin/activate ]]; then
|
||||
echo -e " ↳ [${GREEN}✔${NC}] venv detected."
|
||||
else
|
||||
echo -e " ↳ [${RED}✗${NC}] Can't activate venv in $OPENPILOT_ROOT. Assuming global env!"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_before_cmd() {
|
||||
if [[ ! -z "$NO_VERIFY" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
op_get_openpilot_dir
|
||||
cd $OPENPILOT_ROOT
|
||||
|
||||
result="$((op_check_openpilot_dir ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_git ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_os ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
result="${result}\n$(( op_check_venv ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
|
||||
op_activate_venv
|
||||
|
||||
result="${result}\n$(( op_check_python ) 2>&1)" || (echo -e "$result" && return 1)
|
||||
|
||||
if [[ -z $VERBOSE ]]; then
|
||||
echo -e "${BOLD}Checking system →${NC} [${GREEN}✔${NC}]"
|
||||
else
|
||||
echo -e "$result"
|
||||
fi
|
||||
}
|
||||
|
||||
function op_setup() {
|
||||
op_get_openpilot_dir
|
||||
cd $OPENPILOT_ROOT
|
||||
|
||||
op_check_openpilot_dir
|
||||
op_check_os
|
||||
|
||||
echo "Installing dependencies..."
|
||||
st="$(date +%s)"
|
||||
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
SETUP_SCRIPT="tools/ubuntu_setup.sh"
|
||||
elif [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
SETUP_SCRIPT="tools/mac_setup.sh"
|
||||
fi
|
||||
if ! $OPENPILOT_ROOT/$SETUP_SCRIPT; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Dependencies installation failed!"
|
||||
loge "ERROR_DEPENDENCIES_INSTALLATION"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Dependencies installed successfully in $((et - st)) seconds."
|
||||
|
||||
echo "Getting git submodules..."
|
||||
st="$(date +%s)"
|
||||
if ! git submodule update --filter=blob:none --jobs 4 --init --recursive; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Getting git submodules failed!"
|
||||
loge "ERROR_GIT_SUBMODULES"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Submodules installed successfully in $((et - st)) seconds."
|
||||
|
||||
echo "Pulling git lfs files..."
|
||||
st="$(date +%s)"
|
||||
if ! git lfs pull; then
|
||||
echo -e " ↳ [${RED}✗${NC}] Pulling git lfs files failed!"
|
||||
loge "ERROR_GIT_LFS"
|
||||
return 1
|
||||
fi
|
||||
et="$(date +%s)"
|
||||
echo -e " ↳ [${GREEN}✔${NC}] Files pulled successfully in $((et - st)) seconds."
|
||||
|
||||
op_check
|
||||
}
|
||||
|
||||
function op_auth() {
|
||||
op_before_cmd
|
||||
op_run_command tools/lib/auth.py
|
||||
}
|
||||
|
||||
function op_activate_venv() {
|
||||
# bash 3.2 can't handle this without the 'set +e'
|
||||
set +e
|
||||
source $OPENPILOT_ROOT/.venv/bin/activate &> /dev/null || true
|
||||
set -e
|
||||
}
|
||||
|
||||
function op_venv() {
|
||||
op_before_cmd
|
||||
|
||||
if [[ ! -f $OPENPILOT_ROOT/.venv/bin/activate ]]; then
|
||||
echo -e "No venv found in $OPENPILOT_ROOT"
|
||||
return 1
|
||||
fi
|
||||
|
||||
case $SHELL_NAME in
|
||||
"zsh")
|
||||
ZSHRC_DIR=$(mktemp -d 2>/dev/null || mktemp -d -t 'tmp_zsh')
|
||||
echo "source $RC_FILE; source $OPENPILOT_ROOT/.venv/bin/activate" >> $ZSHRC_DIR/.zshrc
|
||||
ZDOTDIR=$ZSHRC_DIR zsh ;;
|
||||
*)
|
||||
bash --rcfile <(echo "source $RC_FILE; source $OPENPILOT_ROOT/.venv/bin/activate") ;;
|
||||
esac
|
||||
}
|
||||
|
||||
function op_adb() {
|
||||
op_before_cmd
|
||||
op_run_command tools/adb_shell.sh
|
||||
}
|
||||
|
||||
function op_check() {
|
||||
VERBOSE=1
|
||||
op_before_cmd
|
||||
unset VERBOSE
|
||||
}
|
||||
|
||||
function op_build() {
|
||||
CDIR=$(pwd)
|
||||
op_before_cmd
|
||||
cd "$CDIR"
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
# needed on AGNOS to not run out of memory
|
||||
op_run_command system/manager/build.py
|
||||
else
|
||||
# scons is fine on PC
|
||||
op_run_command scons $@
|
||||
fi
|
||||
}
|
||||
|
||||
function op_juggle() {
|
||||
op_before_cmd
|
||||
op_run_command tools/plotjuggler/juggle.py $@
|
||||
}
|
||||
|
||||
function op_lint() {
|
||||
op_before_cmd
|
||||
op_run_command scripts/lint/lint.sh $@
|
||||
}
|
||||
|
||||
function op_test() {
|
||||
op_before_cmd
|
||||
op_run_command pytest $@
|
||||
}
|
||||
|
||||
function op_replay() {
|
||||
op_before_cmd
|
||||
op_run_command tools/replay/replay $@
|
||||
}
|
||||
|
||||
function op_cabana() {
|
||||
op_before_cmd
|
||||
op_run_command tools/cabana/cabana $@
|
||||
}
|
||||
|
||||
function op_sim() {
|
||||
op_before_cmd
|
||||
op_run_command exec tools/sim/run_bridge.py &
|
||||
op_run_command exec tools/sim/launch_openpilot.sh
|
||||
}
|
||||
|
||||
function op_switch() {
|
||||
REMOTE="origin"
|
||||
if [ "$#" -gt 1 ]; then
|
||||
REMOTE="$1"
|
||||
shift
|
||||
fi
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo -e "${BOLD}${UNDERLINE}Usage:${NC} op switch [REMOTE] <BRANCH>"
|
||||
return 1
|
||||
fi
|
||||
BRANCH="$1"
|
||||
|
||||
git fetch "$REMOTE" "$BRANCH"
|
||||
git checkout -f FETCH_HEAD
|
||||
git checkout -B "$BRANCH" --track "$REMOTE"/"$BRANCH"
|
||||
git reset --hard "${REMOTE}/${BRANCH}"
|
||||
git clean -df
|
||||
git submodule update --init --recursive
|
||||
git submodule foreach git reset --hard
|
||||
git submodule foreach git clean -df
|
||||
}
|
||||
|
||||
function op_start() {
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
op_before_cmd
|
||||
op_run_command sudo systemctl restart comma $@
|
||||
fi
|
||||
}
|
||||
|
||||
function op_stop() {
|
||||
if [[ -f "/AGNOS" ]]; then
|
||||
op_before_cmd
|
||||
op_run_command sudo systemctl stop comma $@
|
||||
fi
|
||||
}
|
||||
|
||||
function op_default() {
|
||||
echo "An openpilot helper"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Description:${NC}"
|
||||
echo " op is your entry point for all things related to openpilot development."
|
||||
echo " op is only a wrapper for existing scripts, tools, and commands."
|
||||
echo " op will always show you what it will run on your system."
|
||||
echo ""
|
||||
echo " op will try to find your openpilot directory in the following order:"
|
||||
echo " 1: use the directory specified with the --dir option"
|
||||
echo " 2: use the current working directory"
|
||||
echo " 3: go up the file tree non-recursively"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Usage:${NC} op [OPTIONS] <COMMAND>"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [System]:${NC}"
|
||||
echo -e " ${BOLD}auth${NC} Authenticate yourself for API use"
|
||||
echo -e " ${BOLD}check${NC} Check the development environment (git, os, python) to start using openpilot"
|
||||
echo -e " ${BOLD}venv${NC} Activate the python virtual environment"
|
||||
echo -e " ${BOLD}setup${NC} Install openpilot dependencies"
|
||||
echo -e " ${BOLD}build${NC} Run the openpilot build system in the current working directory"
|
||||
echo -e " ${BOLD}install${NC} Install the 'op' tool system wide"
|
||||
echo -e " ${BOLD}switch${NC} Switch to a different git branch with a clean slate (nukes any changes)"
|
||||
echo -e " ${BOLD}start${NC} Starts (or restarts) openpilot"
|
||||
echo -e " ${BOLD}stop${NC} Stops openpilot"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [Tooling]:${NC}"
|
||||
echo -e " ${BOLD}juggle${NC} Run PlotJuggler"
|
||||
echo -e " ${BOLD}replay${NC} Run Replay"
|
||||
echo -e " ${BOLD}cabana${NC} Run Cabana"
|
||||
echo -e " ${BOLD}adb${NC} Run adb shell"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Commands [Testing]:${NC}"
|
||||
echo -e " ${BOLD}sim${NC} Run openpilot in a simulator"
|
||||
echo -e " ${BOLD}lint${NC} Run the linter"
|
||||
echo -e " ${BOLD}post-commit${NC} Install the linter as a post-commit hook"
|
||||
echo -e " ${BOLD}test${NC} Run all unit tests from pytest"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Options:${NC}"
|
||||
echo -e " ${BOLD}-d, --dir${NC}"
|
||||
echo " Specify the openpilot directory you want to use"
|
||||
echo -e " ${BOLD}--dry${NC}"
|
||||
echo " Don't actually run anything, just print what would be run"
|
||||
echo -e " ${BOLD}-n, --no-verify${NC}"
|
||||
echo " Skip environment check before running commands"
|
||||
echo ""
|
||||
echo -e "${BOLD}${UNDERLINE}Examples:${NC}"
|
||||
echo " op setup"
|
||||
echo " Run the setup script to install"
|
||||
echo " openpilot's dependencies."
|
||||
echo ""
|
||||
echo " op build -j4"
|
||||
echo " Compile openpilot using 4 cores"
|
||||
echo ""
|
||||
echo " op juggle --demo"
|
||||
echo " Run PlotJuggler on the demo route"
|
||||
}
|
||||
|
||||
|
||||
function _op() {
|
||||
# parse Options
|
||||
case $1 in
|
||||
-d | --dir ) shift 1; OPENPILOT_ROOT="$1"; shift 1 ;;
|
||||
--dry ) shift 1; DRY="1" ;;
|
||||
-n | --no-verify ) shift 1; NO_VERIFY="1" ;;
|
||||
-l | --log ) shift 1; LOG_FILE="$1" ; shift 1 ;;
|
||||
esac
|
||||
|
||||
# parse Commands
|
||||
case $1 in
|
||||
auth ) shift 1; op_auth "$@" ;;
|
||||
venv ) shift 1; op_venv "$@" ;;
|
||||
check ) shift 1; op_check "$@" ;;
|
||||
setup ) shift 1; op_setup "$@" ;;
|
||||
build ) shift 1; op_build "$@" ;;
|
||||
juggle ) shift 1; op_juggle "$@" ;;
|
||||
cabana ) shift 1; op_cabana "$@" ;;
|
||||
lint ) shift 1; op_lint "$@" ;;
|
||||
test ) shift 1; op_test "$@" ;;
|
||||
replay ) shift 1; op_replay "$@" ;;
|
||||
sim ) shift 1; op_sim "$@" ;;
|
||||
install ) shift 1; op_install "$@" ;;
|
||||
switch ) shift 1; op_switch "$@" ;;
|
||||
start ) shift 1; op_start "$@" ;;
|
||||
stop ) shift 1; op_stop "$@" ;;
|
||||
restart ) shift 1; op_restart "$@" ;;
|
||||
post-commit ) shift 1; op_install_post_commit "$@" ;;
|
||||
adb ) shift 1; op_adb "$@" ;;
|
||||
* ) op_default "$@" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
_op $@
|
||||
@@ -0,0 +1,3 @@
|
||||
bin/
|
||||
bin
|
||||
*.rlog
|
||||
@@ -0,0 +1,74 @@
|
||||
# PlotJuggler
|
||||
|
||||
[PlotJuggler](https://github.com/facontidavide/PlotJuggler) is a tool to quickly visualize time series data, and we've written plugins to parse openpilot logs. Check out our plugins: https://github.com/commaai/PlotJuggler.
|
||||
|
||||
## Installation
|
||||
|
||||
Once you've [set up the openpilot environment](../README.md), this command will download PlotJuggler and install our plugins:
|
||||
|
||||
`cd tools/plotjuggler && ./juggle.py --install`
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
$ ./juggle.py -h
|
||||
usage: juggle.py [-h] [--demo] [--can] [--stream] [--layout [LAYOUT]] [--install] [--dbc DBC]
|
||||
[route_or_segment_name] [segment_count]
|
||||
|
||||
A helper to run PlotJuggler on openpilot routes
|
||||
|
||||
positional arguments:
|
||||
route_or_segment_name
|
||||
The route or segment name to plot (cabana share URL accepted) (default: None)
|
||||
segment_count The number of segments to plot (default: None)
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--demo Use the demo route instead of providing one (default: False)
|
||||
--can Parse CAN data (default: False)
|
||||
--stream Start PlotJuggler in streaming mode (default: False)
|
||||
--layout [LAYOUT] Run PlotJuggler with a pre-defined layout (default: None)
|
||||
--install Install or update PlotJuggler + plugins (default: False)
|
||||
--dbc DBC Set the DBC name to load for parsing CAN data. If not set, the DBC will be automatically
|
||||
inferred from the logs. (default: None)
|
||||
|
||||
```
|
||||
|
||||
Examples using route name:
|
||||
|
||||
`./juggle.py "a2a0ccea32023010/2023-07-27--13-01-19"`
|
||||
|
||||
Examples using segment range:
|
||||
|
||||
`./juggle.py "a2a0ccea32023010/2023-07-27--13-01-19/1"`
|
||||
|
||||
`./juggle.py "a2a0ccea32023010/2023-07-27--13-01-19/1/q" # use qlogs`
|
||||
|
||||
## Streaming
|
||||
|
||||
Explore live data from your car! Follow these steps to stream from your comma device to your laptop:
|
||||
- Enable wifi tethering on your comma device
|
||||
- [SSH into your device](https://github.com/commaai/openpilot/wiki/SSH) and run `cd /data/openpilot && ./cereal/messaging/bridge`
|
||||
- On your laptop, connect to the device's wifi hotspot
|
||||
- Start PlotJuggler with `ZMQ=1 ./juggle.py --stream`, find the `Cereal Subscriber` plugin in the dropdown under Streaming, and click `Start`.
|
||||
|
||||
If streaming to PlotJuggler from a replay on your PC, simply run: `./juggle.py --stream` and start the cereal subscriber.
|
||||
|
||||
## Demo
|
||||
|
||||
For a quick demo, go through the installation step and run this command:
|
||||
|
||||
`./juggle.py --demo --layout=layouts/tuning.xml`
|
||||
|
||||
## Layouts
|
||||
|
||||
If you create a layout that's useful for others, consider upstreaming it.
|
||||
|
||||
### Tuning
|
||||
|
||||
Use this layout to improve your car's tuning and generate plots for tuning PRs. Also see the [tuning wiki](https://github.com/commaai/openpilot/wiki/Tuning) and tuning PR template.
|
||||
|
||||
`--layout layouts/tuning.xml`
|
||||
|
||||
|
||||
|
||||
Executable
+139
@@ -0,0 +1,139 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import sys
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import tarfile
|
||||
import tempfile
|
||||
import requests
|
||||
import argparse
|
||||
from functools import partial
|
||||
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
from openpilot.tools.cabana.dbc.generate_dbc_json import generate_dbc_dict
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode, save_log
|
||||
from openpilot.selfdrive.test.process_replay.migration import migrate_all
|
||||
|
||||
juggle_dir = os.path.dirname(os.path.realpath(__file__))
|
||||
|
||||
os.environ['LD_LIBRARY_PATH'] = os.environ.get('LD_LIBRARY_PATH', '') + f":{juggle_dir}/bin/"
|
||||
|
||||
DEMO_ROUTE = "a2a0ccea32023010|2023-07-27--13-01-19"
|
||||
RELEASES_URL = "https://github.com/commaai/PlotJuggler/releases/download/latest"
|
||||
INSTALL_DIR = os.path.join(juggle_dir, "bin")
|
||||
PLOTJUGGLER_BIN = os.path.join(juggle_dir, "bin/plotjuggler")
|
||||
MINIMUM_PLOTJUGGLER_VERSION = (3, 5, 2)
|
||||
MAX_STREAMING_BUFFER_SIZE = 1000
|
||||
|
||||
|
||||
def install():
|
||||
m = f"{platform.system()}-{platform.machine()}"
|
||||
supported = ("Linux-x86_64", "Linux-aarch64", "Darwin-arm64", "Darwin-x86_64")
|
||||
if m not in supported:
|
||||
raise Exception(f"Unsupported platform: '{m}'. Supported platforms: {supported}")
|
||||
|
||||
if os.path.exists(INSTALL_DIR):
|
||||
shutil.rmtree(INSTALL_DIR)
|
||||
os.mkdir(INSTALL_DIR)
|
||||
|
||||
url = os.path.join(RELEASES_URL, m + ".tar.gz")
|
||||
with requests.get(url, stream=True, timeout=10) as r, tempfile.NamedTemporaryFile() as tmp:
|
||||
r.raise_for_status()
|
||||
with open(tmp.name, 'wb') as tmpf:
|
||||
for chunk in r.iter_content(chunk_size=1024 * 1024):
|
||||
tmpf.write(chunk)
|
||||
|
||||
with tarfile.open(tmp.name) as tar:
|
||||
tar.extractall(path=INSTALL_DIR)
|
||||
|
||||
|
||||
def get_plotjuggler_version():
|
||||
out = subprocess.check_output([PLOTJUGGLER_BIN, "-v"], encoding="utf-8").strip()
|
||||
version = out.split(" ")[1]
|
||||
return tuple(map(int, version.split(".")))
|
||||
|
||||
|
||||
def start_juggler(fn=None, dbc=None, layout=None, route_or_segment_name=None, platform=None):
|
||||
env = os.environ.copy()
|
||||
env["BASEDIR"] = BASEDIR
|
||||
env["PATH"] = f"{INSTALL_DIR}:{os.getenv('PATH', '')}"
|
||||
if dbc:
|
||||
env["DBC_NAME"] = dbc
|
||||
|
||||
extra_args = ""
|
||||
if fn is not None:
|
||||
extra_args += f" -d {fn}"
|
||||
if layout is not None:
|
||||
extra_args += f" -l {layout}"
|
||||
if route_or_segment_name is not None:
|
||||
extra_args += f" --window_title \"{route_or_segment_name}{f' ({platform})' if platform is not None else ''}\""
|
||||
|
||||
cmd = f'{PLOTJUGGLER_BIN} --buffer_size {MAX_STREAMING_BUFFER_SIZE} --plugin_folders {INSTALL_DIR}{extra_args}'
|
||||
subprocess.call(cmd, shell=True, env=env, cwd=juggle_dir)
|
||||
|
||||
|
||||
def process(can, lr):
|
||||
return [d for d in lr if can or d.which() not in ['can', 'sendcan'] and not d.which().startswith('customReserved')]
|
||||
|
||||
|
||||
def juggle_route(route_or_segment_name, can, layout, dbc, should_migrate):
|
||||
lr = LogReader(route_or_segment_name, default_mode=ReadMode.AUTO_INTERACTIVE)
|
||||
|
||||
all_data = lr.run_across_segments(24, partial(process, can))
|
||||
if should_migrate:
|
||||
all_data = migrate_all(all_data)
|
||||
|
||||
# Infer DBC name from logs
|
||||
platform = None
|
||||
if dbc is None:
|
||||
try:
|
||||
CP = lr.first('carParams')
|
||||
platform = MIGRATION.get(CP.carFingerprint, CP.carFingerprint)
|
||||
dbc = generate_dbc_dict()[platform]
|
||||
except Exception:
|
||||
cloudlog.exception("Failed to get DBC name from logs!")
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix='.rlog', dir=juggle_dir) as tmp:
|
||||
save_log(tmp.name, all_data, compress=False)
|
||||
del all_data
|
||||
start_juggler(tmp.name, dbc, layout, route_or_segment_name, platform)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="A helper to run PlotJuggler on openpilot routes",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--demo", action="store_true", help="Use the demo route instead of providing one")
|
||||
parser.add_argument("--can", action="store_true", help="Parse CAN data")
|
||||
parser.add_argument("--stream", action="store_true", help="Start PlotJuggler in streaming mode")
|
||||
parser.add_argument("--no-migration", action="store_true", help="Do not perform log migration")
|
||||
parser.add_argument("--layout", nargs='?', help="Run PlotJuggler with a pre-defined layout")
|
||||
parser.add_argument("--install", action="store_true", help="Install or update PlotJuggler + plugins")
|
||||
parser.add_argument("--dbc", help="Set the DBC name to load for parsing CAN data. If not set, the DBC will be automatically inferred from the logs.")
|
||||
parser.add_argument("route_or_segment_name", nargs='?', help="The route or segment name to plot (cabana share URL accepted)")
|
||||
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.install:
|
||||
install()
|
||||
sys.exit()
|
||||
|
||||
if not os.path.exists(PLOTJUGGLER_BIN):
|
||||
print("PlotJuggler is missing. Downloading...")
|
||||
install()
|
||||
|
||||
if get_plotjuggler_version() < MINIMUM_PLOTJUGGLER_VERSION:
|
||||
print("PlotJuggler is out of date. Installing update...")
|
||||
install()
|
||||
|
||||
if args.stream:
|
||||
start_juggler(layout=args.layout)
|
||||
else:
|
||||
route_or_segment_name = DEMO_ROUTE if args.demo else args.route_or_segment_name.strip()
|
||||
juggle_route(route_or_segment_name, args.can, args.layout, args.dbc, not args.no_migration)
|
||||
@@ -0,0 +1,86 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab containers="1" tab_name="tab1">
|
||||
<Container>
|
||||
<DockSplitter sizes="0.33362;0.33276;0.33362" count="3" orientation="-">
|
||||
<DockArea name="CAN RX">
|
||||
<plot style="Lines" flip_y="false" mode="TimeSeries" flip_x="false">
|
||||
<range top="1101.875000" left="0.000000" bottom="-26.875000" right="60.526742"/>
|
||||
<limitY/>
|
||||
<curve name="/pandaStates/0/canState0/totalRxCnt" color="#f14cc1">
|
||||
<transform alias="/pandaStates/0/canState0/totalRxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState1/totalRxCnt" color="#9467bd">
|
||||
<transform alias="/pandaStates/0/canState1/totalRxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState2/totalRxCnt" color="#ff7f0e">
|
||||
<transform alias="/pandaStates/0/canState2/totalRxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="CAN TX">
|
||||
<plot style="Lines" flip_y="false" mode="TimeSeries" flip_x="false">
|
||||
<range top="455.100000" left="0.000000" bottom="-11.100000" right="60.526742"/>
|
||||
<limitY/>
|
||||
<curve name="/pandaStates/0/canState0/totalTxCnt" color="#17becf">
|
||||
<transform alias="/pandaStates/0/canState0/totalTxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState1/totalTxCnt" color="#bcbd22">
|
||||
<transform alias="/pandaStates/0/canState1/totalTxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState2/totalTxCnt" color="#1f77b4">
|
||||
<transform alias="/pandaStates/0/canState2/totalTxCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="CAN errors">
|
||||
<plot style="Lines" flip_y="false" mode="TimeSeries" flip_x="false">
|
||||
<range top="2515.350000" left="0.000000" bottom="-61.350000" right="60.526742"/>
|
||||
<limitY/>
|
||||
<curve name="/pandaStates/0/canState0/totalErrorCnt" color="#1f77b4">
|
||||
<transform alias="/pandaStates/0/canState0/totalErrorCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState1/totalErrorCnt" color="#d62728">
|
||||
<transform alias="/pandaStates/0/canState1/totalErrorCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve name="/pandaStates/0/canState2/totalErrorCnt" color="#1ac938">
|
||||
<transform alias="/pandaStates/0/canState2/totalErrorCnt[Derivative]" name="Derivative">
|
||||
<options radioChecked="radioCustom" lineEdit="1.0"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab tab_name="SOF / EOF (encodeIdx)" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.500885;0.499115" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#1f77b4" name="/driverEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/driverEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#d62728" name="/roadEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/roadEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#1ac938" name="/wideRoadEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/wideRoadEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#f14cc1" name="/driverEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/driverEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#9467bd" name="/roadEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/roadEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#17becf" name="/wideRoadEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/wideRoadEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="model timings" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.5;0.5" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="0.015143" left="0.000000" top="0.016865" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#ff7f0e" name="/modelV2/modelExecutionTime"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="-0.100000" left="0.000000" top="0.100000" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#f14cc1" name="/modelV2/frameDropPerc"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="sensor info" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="1" count="1">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="-0.100000" left="0.000000" top="0.100000" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#bcbd22" name="/driverCameraState/sensor"/>
|
||||
<curve color="#1f77b4" name="/roadCameraState/sensor"/>
|
||||
<curve color="#d62728" name="/wideRoadCameraState/sensor"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="SOF / EOF (cameraState)" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.500885;0.499115" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#1f77b4" name="/driverCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/driverCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#d62728" name="/roadCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/roadCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#1ac938" name="/wideRoadCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/wideRoadCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#ff7f0e" name="/driverCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/driverCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#f14cc1" name="/roadCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/roadCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#9467bd" name="/wideRoadCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/wideRoadCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user