mirror of
https://github.com/dragonpilot/dragonpilot.git
synced 2026-08-23 01:04:05 +08:00
dragonpilot v0.10.2
This commit is contained in:
Comma Device
@@ -0,0 +1,20 @@
|
||||
## CTF
|
||||
Welcome to the first part of the comma CTF!
|
||||
|
||||
* all the flags are contained in this route: `0c7f0c7f0c7f0c7f|2021-10-13--13-00-00`
|
||||
* there's 2 flags in each segment, with roughly increasing difficulty
|
||||
* everything you'll need to find the flags is in the openpilot repo
|
||||
* grep is also your friend
|
||||
* first, [setup](https://github.com/commaai/openpilot/tree/master/tools#setup-your-pc) your PC
|
||||
* read the docs & checkout out the tools in tools/ and selfdrive/debug/
|
||||
* tip: once you get the replay and UI up, start by familiarizing yourself with seeking in replay
|
||||
|
||||
getting started
|
||||
```bash
|
||||
# start the route replay
|
||||
cd tools/replay
|
||||
./replay '0c7f0c7f0c7f0c7f|2021-10-13--13-00-00' --dcam --ecam
|
||||
|
||||
# start the UI in another terminal
|
||||
selfdrive/ui/ui
|
||||
```
|
||||
@@ -0,0 +1,59 @@
|
||||
# openpilot tools
|
||||
|
||||
## System Requirements
|
||||
|
||||
openpilot is developed and tested on **Ubuntu 24.04**, which is the primary development target aside from the [supported embedded hardware](https://github.com/commaai/openpilot#running-on-a-dedicated-device-in-a-car).
|
||||
|
||||
Most of openpilot should work natively on macOS. On Windows you can use WSL for a nearly native Ubuntu experience. Running natively on any other system is not currently recommended and will likely require modifications.
|
||||
|
||||
## Native setup on Ubuntu 24.04 and macOS
|
||||
|
||||
Follow these instructions for a fully managed setup experience. If you'd like to manage the dependencies yourself, just read the setup scripts in this directory.
|
||||
|
||||
**1. Clone openpilot**
|
||||
``` bash
|
||||
git clone https://github.com/commaai/openpilot.git
|
||||
```
|
||||
|
||||
**2. Run the setup script**
|
||||
``` bash
|
||||
cd openpilot
|
||||
tools/op.sh setup
|
||||
```
|
||||
|
||||
**3. Activate a Python shell**
|
||||
Activate a shell with the Python dependencies installed:
|
||||
``` bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
**4. Build openpilot**
|
||||
``` bash
|
||||
scons -u -j$(nproc)
|
||||
```
|
||||
|
||||
## WSL on Windows
|
||||
|
||||
[Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) should provide a similar experience to native Ubuntu. [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/compare-versions) specifically has been reported by several users to be a seamless experience.
|
||||
|
||||
Follow [these instructions](https://docs.microsoft.com/en-us/windows/wsl/install) to setup the WSL and install the `Ubuntu-24.04` distribution. Once your Ubuntu WSL environment is setup, follow the Linux setup instructions to finish setting up your environment. See [these instructions](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps) for running GUI apps.
|
||||
|
||||
**NOTE**: If you are running WSL and any GUIs are failing (segfaulting or other strange issues) even after following the steps above, you may need to enable software rendering with `LIBGL_ALWAYS_SOFTWARE=1`, e.g. `LIBGL_ALWAYS_SOFTWARE=1 selfdrive/ui/ui`.
|
||||
|
||||
## CTF
|
||||
Learn about the openpilot ecosystem and tools by playing our [CTF](/tools/CTF.md).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
├── cabana/ # View and plot CAN messages from drives or in realtime
|
||||
├── camerastream/ # Cameras stream over the network
|
||||
├── joystick/ # Control your car with a joystick
|
||||
├── lib/ # Libraries to support the tools and reading openpilot logs
|
||||
├── plotjuggler/ # A tool to plot openpilot logs
|
||||
├── replay/ # Replay drives and mock openpilot services
|
||||
├── scripts/ # Miscellaneous scripts
|
||||
├── serial/ # Tools for using the comma serial
|
||||
├── sim/ # Run openpilot in a simulator
|
||||
└── webcam/ # Run openpilot on a PC with webcams
|
||||
```
|
||||
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2:
|
||||
print("Usage: python auto_source.py <log_path>")
|
||||
sys.exit(1)
|
||||
|
||||
log_path = sys.argv[1]
|
||||
lr = LogReader(log_path, default_mode=ReadMode.AUTO, sort_by_time=True)
|
||||
print("\n".join(lr.logreader_identifiers))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,4 @@
|
||||
av
|
||||
av-10.0.0/*
|
||||
key.pem
|
||||
cert.pem
|
||||
@@ -0,0 +1,103 @@
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>commabody</title>
|
||||
<link rel="stylesheet" href="/static/main.css">
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/css/bootstrap.min.css" integrity="sha512-SbiR/eusphKoMVVXysTKG/7VseWii+Y3FdHrt0EpKgpToZeemhqHeZeLWLhJutz/2ut2Vw1uQEj2MbRF+TVBUA==" crossorigin="anonymous" referrerpolicy="no-referrer" /><script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.5/font/bootstrap-icons.css">
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/js/bootstrap.min.js" integrity="sha512-1/RvZTcCDEUjY/CypiMz+iqqtaoQfAITmNSJY17Myp4Ms5mdxPS5UV7iOfdZoxcGhzFbOm6sntTKJppjvuhg4g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@^3"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/moment@^2"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-moment@^1"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="main">
|
||||
<p class="jumbo">comma body</p>
|
||||
<audio id="audio" autoplay="true"></audio>
|
||||
<video id="video" playsinline autoplay muted loop poster="/static/poster.png"></video>
|
||||
<div id="icon-panel" class="row">
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
<i class="bi bi-camera-video-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">body</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">you</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="ping-time" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-arrow-repeat small"> ping time</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="battery" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-battery-half small"> battery</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- <div class="icon-sub-panel">
|
||||
<button type="button" id="start" class="btn btn-light btn-lg">Start</button>
|
||||
<button type="button" id="stop" class="btn btn-light btn-lg">Stop</button>
|
||||
</div> -->
|
||||
</div>
|
||||
<div class="row" style="width: 100%; padding: 0px 10px 0px 10px;">
|
||||
<div id="wasd" class="col-md-12 row">
|
||||
<div class="col-md-6 col-sm-12" style="justify-content: center; display: flex; flex-direction: column;">
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-w">W</div>
|
||||
<div id="key-val"><span id="pos-vals">0,0</span><span>x,y</span></div>
|
||||
</div>
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-a">A</div>
|
||||
<div class="keys" id="key-s">S</div>
|
||||
<div class="keys" id="key-d">D</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6 col-sm-12 form-group plan-form">
|
||||
<label for="plan-text">Plan (w, a, s, d, t)</label>
|
||||
<label style="font-size: 15px;" for="plan-text">*Extremely Experimental*</label>
|
||||
<textarea class="form-control" id="plan-text" rows="7" placeholder="1,0,0,0,2"></textarea>
|
||||
<button type="button" id="plan-button" class="btn btn-light btn-lg">Execute</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-3" style="text-align: center;">
|
||||
<p>Play Sounds</p>
|
||||
</div>
|
||||
<div class="btn-group col-sm-8">
|
||||
<button type="button" id="sound-engage" class="btn btn-outline-success btn-lg sound">Engage</button>
|
||||
<button type="button" id="sound-disengage" class="btn btn-outline-warning btn-lg sound">Disengage</button>
|
||||
<button type="button" id="sound-error" class="btn btn-outline-danger btn-lg sound">Error</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-6"><canvas id="chart-ping"></canvas></div>
|
||||
<div class="col-sm-6"><canvas id="chart-battery"></canvas></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<script src="/static/js/jsmain.js" type="module"></script>
|
||||
</body>
|
||||
</html>
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
const keyVals = {w: 0, a: 0, s: 0, d: 0}
|
||||
|
||||
export function getXY() {
|
||||
let x = -keyVals.w + keyVals.s
|
||||
let y = -keyVals.d + keyVals.a
|
||||
return {x, y}
|
||||
}
|
||||
|
||||
export const handleKeyX = (key, setValue) => {
|
||||
if (['w', 'a', 's', 'd'].includes(key)){
|
||||
keyVals[key] = setValue;
|
||||
let color = "#333";
|
||||
if (setValue === 1){
|
||||
color = "#e74c3c";
|
||||
}
|
||||
$("#key-"+key).css('background', color);
|
||||
const {x, y} = getXY();
|
||||
$("#pos-vals").text(x+","+y);
|
||||
}
|
||||
};
|
||||
|
||||
export async function executePlan() {
|
||||
let plan = $("#plan-text").val();
|
||||
const planList = [];
|
||||
plan.split("\n").forEach(function(e){
|
||||
let line = e.split(",").map(k=>parseInt(k));
|
||||
if (line.length != 5 || line.slice(0, 4).map(e=>[1, 0].includes(e)).includes(false) || line[4] < 0 || line[4] > 10){
|
||||
console.log("invalid plan");
|
||||
}
|
||||
else{
|
||||
planList.push(line)
|
||||
}
|
||||
});
|
||||
|
||||
async function execute() {
|
||||
for (var i = 0; i < planList.length; i++) {
|
||||
let [w, a, s, d, t] = planList[i];
|
||||
while(t > 0){
|
||||
console.log(w, a, s, d, t);
|
||||
if(w==1){$("#key-w").mousedown();}
|
||||
if(a==1){$("#key-a").mousedown();}
|
||||
if(s==1){$("#key-s").mousedown();}
|
||||
if(d==1){$("#key-d").mousedown();}
|
||||
await sleep(50);
|
||||
$("#key-w").mouseup();
|
||||
$("#key-a").mouseup();
|
||||
$("#key-s").mouseup();
|
||||
$("#key-d").mouseup();
|
||||
t = t - 0.05;
|
||||
}
|
||||
}
|
||||
}
|
||||
execute();
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { handleKeyX, executePlan } from "./controls.js";
|
||||
import { start, stop, lastChannelMessageTime, playSoundRequest } from "./webrtc.js";
|
||||
|
||||
export var pc = null;
|
||||
export var dc = null;
|
||||
|
||||
document.addEventListener('keydown', (e)=>(handleKeyX(e.key.toLowerCase(), 1)));
|
||||
document.addEventListener('keyup', (e)=>(handleKeyX(e.key.toLowerCase(), 0)));
|
||||
$(".keys").bind("mousedown touchstart", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 1));
|
||||
$(".keys").bind("mouseup touchend", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 0));
|
||||
$("#plan-button").click(executePlan);
|
||||
$(".sound").click((e)=>{
|
||||
const sound = $(e.target).attr('id').replace('sound-', '')
|
||||
return playSoundRequest(sound);
|
||||
});
|
||||
|
||||
setInterval( () => {
|
||||
const dt = new Date().getTime();
|
||||
if ((dt - lastChannelMessageTime) > 1000) {
|
||||
$(".pre-blob").removeClass('blob');
|
||||
$("#battery").text("-");
|
||||
$("#ping-time").text('-');
|
||||
$("video")[0].load();
|
||||
}
|
||||
}, 5000);
|
||||
|
||||
start(pc, dc);
|
||||
@@ -0,0 +1,53 @@
|
||||
export const pingPoints = [];
|
||||
export const batteryPoints = [];
|
||||
|
||||
function getChartConfig(pts, color, title, ymax=100) {
|
||||
return {
|
||||
type: 'line',
|
||||
data: {
|
||||
datasets: [{
|
||||
label: title,
|
||||
data: pts,
|
||||
borderWidth: 1,
|
||||
borderColor: color,
|
||||
backgroundColor: color,
|
||||
fill: 'origin'
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
scales: {
|
||||
x: {
|
||||
type: 'time',
|
||||
time: {
|
||||
unit: 'minute',
|
||||
displayFormats: {
|
||||
second: 'h:mm a'
|
||||
}
|
||||
},
|
||||
grid: {
|
||||
color: '#222', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
source: 'data',
|
||||
fontColor: 'rgba(255, 255, 255, 1.0)', // Y-axis label color
|
||||
}
|
||||
},
|
||||
y: {
|
||||
beginAtZero: true,
|
||||
max: ymax,
|
||||
grid: {
|
||||
color: 'rgba(255, 255, 255, 0.1)', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
fontColor: 'rgba(255, 255, 255, 0.7)', // Y-axis label color
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const ctxPing = document.getElementById('chart-ping');
|
||||
const ctxBattery = document.getElementById('chart-battery');
|
||||
export const chartPing = new Chart(ctxPing, getChartConfig(pingPoints, 'rgba(192, 57, 43, 0.7)', 'Controls Ping Time (ms)', 250));
|
||||
export const chartBattery = new Chart(ctxBattery, getChartConfig(batteryPoints, 'rgba(41, 128, 185, 0.7)', 'Battery %', 100));
|
||||
@@ -0,0 +1,209 @@
|
||||
import { getXY } from "./controls.js";
|
||||
import { pingPoints, batteryPoints, chartPing, chartBattery } from "./plots.js";
|
||||
|
||||
export let controlCommandInterval = null;
|
||||
export let latencyInterval = null;
|
||||
export let lastChannelMessageTime = null;
|
||||
|
||||
|
||||
export function offerRtcRequest(sdp, type) {
|
||||
return fetch('/offer', {
|
||||
body: JSON.stringify({sdp: sdp, type: type}),
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
method: 'POST'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function playSoundRequest(sound) {
|
||||
return fetch('/sound', {
|
||||
body: JSON.stringify({sound}),
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
method: 'POST'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function pingHeadRequest() {
|
||||
return fetch('/', {
|
||||
method: 'HEAD'
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
export function createPeerConnection(pc) {
|
||||
var config = {
|
||||
sdpSemantics: 'unified-plan'
|
||||
};
|
||||
|
||||
pc = new RTCPeerConnection(config);
|
||||
|
||||
// connect audio / video
|
||||
pc.addEventListener('track', function(evt) {
|
||||
console.log("Adding Tracks!")
|
||||
if (evt.track.kind == 'video')
|
||||
document.getElementById('video').srcObject = evt.streams[0];
|
||||
else
|
||||
document.getElementById('audio').srcObject = evt.streams[0];
|
||||
});
|
||||
return pc;
|
||||
}
|
||||
|
||||
|
||||
export function negotiate(pc) {
|
||||
return pc.createOffer({offerToReceiveAudio:true, offerToReceiveVideo:true}).then(function(offer) {
|
||||
return pc.setLocalDescription(offer);
|
||||
}).then(function() {
|
||||
return new Promise(function(resolve) {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
resolve();
|
||||
}
|
||||
else {
|
||||
function checkState() {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
pc.removeEventListener('icegatheringstatechange', checkState);
|
||||
resolve();
|
||||
}
|
||||
}
|
||||
pc.addEventListener('icegatheringstatechange', checkState);
|
||||
}
|
||||
});
|
||||
}).then(function() {
|
||||
var offer = pc.localDescription;
|
||||
return offerRtcRequest(offer.sdp, offer.type);
|
||||
}).then(function(response) {
|
||||
console.log(response);
|
||||
return response.json();
|
||||
}).then(function(answer) {
|
||||
return pc.setRemoteDescription(answer);
|
||||
}).catch(function(e) {
|
||||
alert(e);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
function isMobile() {
|
||||
let check = false;
|
||||
(function(a){if(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4))) check = true;})(navigator.userAgent||navigator.vendor||window.opera);
|
||||
return check;
|
||||
};
|
||||
|
||||
|
||||
export const constraints = {
|
||||
audio: {
|
||||
autoGainControl: false,
|
||||
sampleRate: 48000,
|
||||
sampleSize: 16,
|
||||
echoCancellation: true,
|
||||
noiseSuppression: true,
|
||||
channelCount: 1
|
||||
},
|
||||
video: isMobile()
|
||||
};
|
||||
|
||||
|
||||
export function start(pc, dc) {
|
||||
pc = createPeerConnection(pc);
|
||||
|
||||
// add audio track
|
||||
navigator.mediaDevices.enumerateDevices()
|
||||
.then(function(devices) {
|
||||
const hasAudioInput = devices.find((device) => device.kind === "audioinput");
|
||||
var modifiedConstraints = {};
|
||||
modifiedConstraints.video = constraints.video;
|
||||
modifiedConstraints.audio = hasAudioInput ? constraints.audio : false;
|
||||
|
||||
return Promise.resolve(modifiedConstraints);
|
||||
})
|
||||
.then(function(constraints) {
|
||||
if (constraints.audio || constraints.video) {
|
||||
return navigator.mediaDevices.getUserMedia(constraints);
|
||||
} else{
|
||||
return Promise.resolve(null);
|
||||
}
|
||||
})
|
||||
.then(function(stream) {
|
||||
if (stream) {
|
||||
stream.getTracks().forEach(function(track) {
|
||||
pc.addTrack(track, stream);
|
||||
});
|
||||
}
|
||||
|
||||
return negotiate(pc);
|
||||
})
|
||||
.catch(function(err) {
|
||||
alert('Could not acquire media: ' + err);
|
||||
});
|
||||
|
||||
var parameters = {"ordered": true};
|
||||
dc = pc.createDataChannel('data', parameters);
|
||||
dc.onclose = function() {
|
||||
clearInterval(controlCommandInterval);
|
||||
clearInterval(latencyInterval);
|
||||
};
|
||||
|
||||
function sendJoystickOverDataChannel() {
|
||||
const {x, y} = getXY();
|
||||
var message = JSON.stringify({type: "testJoystick", data: {axes: [x, y], buttons: [false]}})
|
||||
dc.send(message);
|
||||
}
|
||||
function checkLatency() {
|
||||
const initialTime = new Date().getTime();
|
||||
pingHeadRequest().then(function() {
|
||||
const currentTime = new Date().getTime();
|
||||
if (Math.abs(currentTime - lastChannelMessageTime) < 1000) {
|
||||
const pingtime = currentTime - initialTime;
|
||||
pingPoints.push({'x': currentTime, 'y': pingtime});
|
||||
if (pingPoints.length > 1000) {
|
||||
pingPoints.shift();
|
||||
}
|
||||
chartPing.update();
|
||||
$("#ping-time").text((pingtime) + "ms");
|
||||
}
|
||||
})
|
||||
}
|
||||
dc.onopen = function() {
|
||||
controlCommandInterval = setInterval(sendJoystickOverDataChannel, 50);
|
||||
latencyInterval = setInterval(checkLatency, 1000);
|
||||
sendJoystickOverDataChannel();
|
||||
};
|
||||
|
||||
const textDecoder = new TextDecoder();
|
||||
var carStaterIndex = 0;
|
||||
dc.onmessage = function(evt) {
|
||||
const text = textDecoder.decode(evt.data);
|
||||
const msg = JSON.parse(text);
|
||||
if (carStaterIndex % 100 == 0 && msg.type === 'carState') {
|
||||
const batteryLevel = Math.round(msg.data.fuelGauge * 100);
|
||||
$("#battery").text(batteryLevel + "%");
|
||||
batteryPoints.push({'x': new Date().getTime(), 'y': batteryLevel});
|
||||
if (batteryPoints.length > 1000) {
|
||||
batteryPoints.shift();
|
||||
}
|
||||
chartBattery.update();
|
||||
}
|
||||
carStaterIndex += 1;
|
||||
lastChannelMessageTime = new Date().getTime();
|
||||
$(".pre-blob").addClass('blob');
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
export function stop(pc, dc) {
|
||||
if (dc) {
|
||||
dc.close();
|
||||
}
|
||||
if (pc.getTransceivers) {
|
||||
pc.getTransceivers().forEach(function(transceiver) {
|
||||
if (transceiver.stop) {
|
||||
transceiver.stop();
|
||||
}
|
||||
});
|
||||
}
|
||||
pc.getSenders().forEach(function(sender) {
|
||||
sender.track.stop();
|
||||
});
|
||||
setTimeout(function() {
|
||||
pc.close();
|
||||
}, 500);
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
body {
|
||||
background: #333 !important;
|
||||
color: #fff !important;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: start;
|
||||
}
|
||||
|
||||
p {
|
||||
margin: 0px !important;
|
||||
}
|
||||
|
||||
i {
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
.small {
|
||||
font-size: 1em !important
|
||||
}
|
||||
|
||||
.jumbo {
|
||||
font-size: 8rem;
|
||||
}
|
||||
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.small {
|
||||
font-size: 0.5em !important
|
||||
}
|
||||
.jumbo {
|
||||
display: none;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
#main {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-content: center;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 30px;
|
||||
width: 100%;
|
||||
max-width: 1200px;
|
||||
}
|
||||
|
||||
video {
|
||||
width: 95%;
|
||||
}
|
||||
|
||||
.pre-blob {
|
||||
display: flex;
|
||||
background: #333;
|
||||
border-radius: 50%;
|
||||
margin: 10px;
|
||||
height: 45px;
|
||||
width: 45px;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.blob {
|
||||
background: rgba(231, 76, 60,1.0);
|
||||
box-shadow: 0 0 0 0 rgba(231, 76, 60,1.0);
|
||||
animation: pulse 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0% {
|
||||
box-shadow: 0 0 0 0px rgba(192, 57, 43, 1);
|
||||
}
|
||||
100% {
|
||||
box-shadow: 0 0 0 20px rgba(192, 57, 43, 0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
.icon-sup-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: center;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
padding: 5px;
|
||||
margin: 5px 0px 5px 0px;
|
||||
}
|
||||
|
||||
.icon-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
#icon-panel {
|
||||
display: flex;
|
||||
width: 100%;
|
||||
justify-content: space-between;
|
||||
margin-top: 5px;
|
||||
}
|
||||
|
||||
.icon-sub-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
}
|
||||
|
||||
.keys, #key-val {
|
||||
background: #333;
|
||||
padding: 2rem;
|
||||
margin: 5px;
|
||||
color: #fff;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
border-radius: 10px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#key-val {
|
||||
pointer-events: none;
|
||||
background: #fff;
|
||||
color: #333;
|
||||
line-height: 1;
|
||||
font-size: 20px;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.wasd-row {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
#wasd {
|
||||
margin: 5px 0px 5px 0px;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 20px;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: stretch;
|
||||
|
||||
user-select: none;
|
||||
-webkit-touch-callout: none;
|
||||
-webkit-user-select: none;
|
||||
-khtml-user-select: none;
|
||||
-moz-user-select: none;
|
||||
-ms-user-select: none;
|
||||
touch-action: manipulation;
|
||||
}
|
||||
|
||||
.panel {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
margin: 5px 0px 5px 0px !important;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
#ping-time, #battery {
|
||||
font-size: 25px;
|
||||
}
|
||||
|
||||
#stop {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.plan-form {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.details {
|
||||
display: flex;
|
||||
padding: 0px 10px 0px 10px;
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 8.1 KiB |
@@ -0,0 +1,127 @@
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import ssl
|
||||
import subprocess
|
||||
|
||||
import pyaudio
|
||||
import wave
|
||||
from aiohttp import web
|
||||
from aiohttp import ClientSession
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.system.webrtc.webrtcd import StreamRequestBody
|
||||
from openpilot.common.params import Params
|
||||
|
||||
logger = logging.getLogger("bodyteleop")
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
|
||||
TELEOPDIR = f"{BASEDIR}/tools/bodyteleop"
|
||||
WEBRTCD_HOST, WEBRTCD_PORT = "localhost", 5001
|
||||
|
||||
|
||||
## UTILS
|
||||
async def play_sound(sound: str):
|
||||
SOUNDS = {
|
||||
"engage": "selfdrive/assets/sounds/engage.wav",
|
||||
"disengage": "selfdrive/assets/sounds/disengage.wav",
|
||||
"error": "selfdrive/assets/sounds/warning_immediate.wav",
|
||||
}
|
||||
assert sound in SOUNDS
|
||||
|
||||
chunk = 5120
|
||||
with wave.open(os.path.join(BASEDIR, SOUNDS[sound]), "rb") as wf:
|
||||
def callback(in_data, frame_count, time_info, status):
|
||||
data = wf.readframes(frame_count)
|
||||
return data, pyaudio.paContinue
|
||||
|
||||
p = pyaudio.PyAudio()
|
||||
stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
|
||||
channels=wf.getnchannels(),
|
||||
rate=wf.getframerate(),
|
||||
output=True,
|
||||
frames_per_buffer=chunk,
|
||||
stream_callback=callback)
|
||||
stream.start_stream()
|
||||
while stream.is_active():
|
||||
await asyncio.sleep(0)
|
||||
stream.stop_stream()
|
||||
stream.close()
|
||||
p.terminate()
|
||||
|
||||
## SSL
|
||||
def create_ssl_cert(cert_path: str, key_path: str):
|
||||
try:
|
||||
proc = subprocess.run(f'openssl req -x509 -newkey rsa:4096 -nodes -out {cert_path} -keyout {key_path} \
|
||||
-days 365 -subj "/C=US/ST=California/O=commaai/OU=comma body"',
|
||||
capture_output=True, shell=True)
|
||||
proc.check_returncode()
|
||||
except subprocess.CalledProcessError as ex:
|
||||
raise ValueError(f"Error creating SSL certificate:\n[stdout]\n{proc.stdout.decode()}\n[stderr]\n{proc.stderr.decode()}") from ex
|
||||
|
||||
|
||||
def create_ssl_context():
|
||||
cert_path = os.path.join(TELEOPDIR, "cert.pem")
|
||||
key_path = os.path.join(TELEOPDIR, "key.pem")
|
||||
if not os.path.exists(cert_path) or not os.path.exists(key_path):
|
||||
logger.info("Creating certificate...")
|
||||
create_ssl_cert(cert_path, key_path)
|
||||
else:
|
||||
logger.info("Certificate exists!")
|
||||
ssl_context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_SERVER)
|
||||
ssl_context.load_cert_chain(cert_path, key_path)
|
||||
|
||||
return ssl_context
|
||||
|
||||
## ENDPOINTS
|
||||
async def index(request: 'web.Request'):
|
||||
with open(os.path.join(TELEOPDIR, "static", "index.html")) as f:
|
||||
content = f.read()
|
||||
return web.Response(content_type="text/html", text=content)
|
||||
|
||||
|
||||
async def ping(request: 'web.Request'):
|
||||
return web.Response(text="pong")
|
||||
|
||||
|
||||
async def sound(request: 'web.Request'):
|
||||
params = await request.json()
|
||||
sound_to_play = params["sound"]
|
||||
|
||||
await play_sound(sound_to_play)
|
||||
return web.json_response({"status": "ok"})
|
||||
|
||||
|
||||
async def offer(request: 'web.Request'):
|
||||
params = await request.json()
|
||||
body = StreamRequestBody(params["sdp"], ["driver"], ["testJoystick"], ["carState"])
|
||||
body_json = json.dumps(dataclasses.asdict(body))
|
||||
|
||||
logger.info("Sending offer to webrtcd...")
|
||||
webrtcd_url = f"http://{WEBRTCD_HOST}:{WEBRTCD_PORT}/stream"
|
||||
async with ClientSession() as session, session.post(webrtcd_url, data=body_json) as resp:
|
||||
assert resp.status == 200
|
||||
answer = await resp.json()
|
||||
return web.json_response(answer)
|
||||
|
||||
|
||||
def main():
|
||||
# Enable joystick debug mode
|
||||
Params().put_bool("JoystickDebugMode", True)
|
||||
|
||||
# App needs to be HTTPS for microphone and audio autoplay to work on the browser
|
||||
ssl_context = create_ssl_context()
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_get("/", index)
|
||||
app.router.add_get("/ping", ping, allow_head=True)
|
||||
app.router.add_post("/offer", offer)
|
||||
app.router.add_post("/sound", sound)
|
||||
app.router.add_static('/static', os.path.join(TELEOPDIR, 'static'))
|
||||
web.run_app(app, access_log=None, host="0.0.0.0", port=5000, ssl_context=ssl_context)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
moc_*
|
||||
*.moc
|
||||
|
||||
cabana
|
||||
dbc/car_fingerprint_to_dbc.json
|
||||
tests/test_cabana
|
||||
@@ -0,0 +1,100 @@
|
||||
# Cabana
|
||||
|
||||
Cabana is a tool developed to view raw CAN data. One use for this is creating and editing [CAN Dictionaries](http://socialledge.com/sjsu/index.php/DBC_Format) (DBC files), and the tool provides direct integration with [commaai/opendbc](https://github.com/commaai/opendbc) (a collection of DBC files), allowing you to load the DBC files direct from source, and save to your fork. In addition, you can load routes from [comma connect](https://connect.comma.ai).
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
```bash
|
||||
$ ./cabana -h
|
||||
Usage: ./cabana [options] route
|
||||
|
||||
Options:
|
||||
-h, --help Displays help on commandline options.
|
||||
--help-all Displays help including Qt specific options.
|
||||
--demo use a demo route instead of providing your own
|
||||
--auto Auto load the route from the best available source (no video):
|
||||
internal, openpilotci, comma_api, car_segments, testing_closet
|
||||
--qcam load qcamera
|
||||
--ecam load wide road camera
|
||||
--msgq read can messages from msgq
|
||||
--panda read can messages from panda
|
||||
--panda-serial <panda-serial> read can messages from panda with given serial
|
||||
--socketcan <socketcan> read can messages from given SocketCAN device
|
||||
--zmq <ip-address> read can messages from zmq at the specified ip-address
|
||||
messages
|
||||
--data_dir <data_dir> local directory with routes
|
||||
--no-vipc do not output video
|
||||
--dbc <dbc> dbc file to open
|
||||
|
||||
Arguments:
|
||||
route the drive to replay. find your drives at
|
||||
connect.comma.ai
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Running Cabana in Demo Mode
|
||||
To run Cabana using a built-in demo route, use the following command:
|
||||
|
||||
```shell
|
||||
cabana --demo
|
||||
```
|
||||
|
||||
### Loading a Specific Route
|
||||
|
||||
To load a specific route for replay, provide the route as an argument:
|
||||
|
||||
```shell
|
||||
cabana "a2a0ccea32023010|2023-07-27--13-01-19"
|
||||
```
|
||||
|
||||
Replace "0ccea32023010|2023-07-27--13-01-19" with your desired route identifier.
|
||||
|
||||
|
||||
### Running Cabana with multiple cameras
|
||||
To run Cabana with multiple cameras, use the following command:
|
||||
|
||||
```shell
|
||||
cabana "a2a0ccea32023010|2023-07-27--13-01-19" --dcam --ecam
|
||||
```
|
||||
|
||||
### Streaming CAN Messages from a comma Device
|
||||
|
||||
[SSH into your device](https://github.com/commaai/openpilot/wiki/SSH) and start the bridge with the following command:
|
||||
|
||||
```shell
|
||||
cd /data/openpilot/cereal/messaging/
|
||||
./bridge &
|
||||
```
|
||||
|
||||
Then Run Cabana with the device's IP address:
|
||||
|
||||
```shell
|
||||
cabana --zmq <ipaddress>
|
||||
```
|
||||
|
||||
Replace <ipaddress> with your comma device's IP address.
|
||||
|
||||
While streaming from the device, Cabana will log the CAN messages to a local directory. By default, this directory is ~/cabana_live_stream/. You can change the log directory in Cabana by navigating to menu -> tools -> settings.
|
||||
|
||||
After disconnecting from the device, you can replay the logged CAN messages from the stream selector dialog -> browse local route.
|
||||
|
||||
### Streaming CAN Messages from Panda
|
||||
|
||||
To read CAN messages from a connected Panda, use the following command:
|
||||
|
||||
```shell
|
||||
cabana --panda
|
||||
```
|
||||
|
||||
### Using the Stream Selector Dialog
|
||||
|
||||
If you run Cabana without any arguments, a stream selector dialog will pop up, allowing you to choose the stream.
|
||||
|
||||
```shell
|
||||
cabana
|
||||
```
|
||||
|
||||
## Additional Information
|
||||
|
||||
For more information, see the [openpilot wiki](https://github.com/commaai/openpilot/wiki/Cabana)
|
||||
@@ -0,0 +1,97 @@
|
||||
import subprocess
|
||||
import os
|
||||
|
||||
Import('env', 'arch', 'common', 'messaging', 'visionipc', 'replay_lib', 'cereal')
|
||||
|
||||
qt_env = env.Clone()
|
||||
qt_modules = ["Widgets", "Gui", "Core", "Network", "Concurrent", "DBus", "Xml"]
|
||||
|
||||
qt_libs = []
|
||||
if arch == "Darwin":
|
||||
brew_prefix = subprocess.check_output(['brew', '--prefix'], encoding='utf8').strip()
|
||||
qt_env['QTDIR'] = f"{brew_prefix}/opt/qt@5"
|
||||
qt_dirs = [
|
||||
os.path.join(qt_env['QTDIR'], "include"),
|
||||
]
|
||||
qt_dirs += [f"{qt_env['QTDIR']}/include/Qt{m}" for m in qt_modules]
|
||||
qt_env["LINKFLAGS"] += ["-F" + os.path.join(qt_env['QTDIR'], "lib")]
|
||||
qt_env["FRAMEWORKS"] += [f"Qt{m}" for m in qt_modules] + ["OpenGL"]
|
||||
qt_env.AppendENVPath('PATH', os.path.join(qt_env['QTDIR'], "bin"))
|
||||
else:
|
||||
qt_install_prefix = subprocess.check_output(['qmake', '-query', 'QT_INSTALL_PREFIX'], encoding='utf8').strip()
|
||||
qt_install_headers = subprocess.check_output(['qmake', '-query', 'QT_INSTALL_HEADERS'], encoding='utf8').strip()
|
||||
|
||||
qt_env['QTDIR'] = qt_install_prefix
|
||||
qt_dirs = [
|
||||
f"{qt_install_headers}",
|
||||
]
|
||||
|
||||
qt_gui_path = os.path.join(qt_install_headers, "QtGui")
|
||||
qt_gui_dirs = [d for d in os.listdir(qt_gui_path) if os.path.isdir(os.path.join(qt_gui_path, d))]
|
||||
qt_dirs += [f"{qt_install_headers}/QtGui/{qt_gui_dirs[0]}/QtGui", ] if qt_gui_dirs else []
|
||||
qt_dirs += [f"{qt_install_headers}/Qt{m}" for m in qt_modules]
|
||||
|
||||
qt_libs = [f"Qt5{m}" for m in qt_modules]
|
||||
if arch == "larch64":
|
||||
qt_libs += ["GLESv2", "wayland-client"]
|
||||
qt_env.PrependENVPath('PATH', Dir("#third_party/qt5/larch64/bin/").abspath)
|
||||
elif arch != "Darwin":
|
||||
qt_libs += ["GL"]
|
||||
qt_env['QT3DIR'] = qt_env['QTDIR']
|
||||
qt_env.Tool('qt3')
|
||||
|
||||
qt_env['CPPPATH'] += qt_dirs + ["#third_party/qrcode"]
|
||||
qt_flags = [
|
||||
"-D_REENTRANT",
|
||||
"-DQT_NO_DEBUG",
|
||||
"-DQT_WIDGETS_LIB",
|
||||
"-DQT_GUI_LIB",
|
||||
"-DQT_CORE_LIB",
|
||||
"-DQT_MESSAGELOGCONTEXT",
|
||||
]
|
||||
qt_env['CXXFLAGS'] += qt_flags
|
||||
qt_env['LIBPATH'] += ['#selfdrive/ui', ]
|
||||
qt_env['LIBS'] = qt_libs
|
||||
|
||||
base_frameworks = qt_env['FRAMEWORKS']
|
||||
base_libs = [common, messaging, cereal, visionipc, 'm', 'ssl', 'crypto', 'pthread'] + qt_env["LIBS"]
|
||||
|
||||
if arch == "Darwin":
|
||||
base_frameworks.append('OpenCL')
|
||||
base_frameworks.append('QtCharts')
|
||||
base_frameworks.append('QtSerialBus')
|
||||
else:
|
||||
base_libs.append('OpenCL')
|
||||
base_libs.append('Qt5Charts')
|
||||
base_libs.append('Qt5SerialBus')
|
||||
|
||||
qt_libs = base_libs
|
||||
|
||||
cabana_env = qt_env.Clone()
|
||||
|
||||
cabana_libs = [cereal, messaging, visionipc, replay_lib, 'avutil', 'avcodec', 'avformat', 'bz2', 'zstd', 'curl', 'yuv', 'usb-1.0'] + qt_libs
|
||||
opendbc_path = '-DOPENDBC_FILE_PATH=\'"%s"\'' % (cabana_env.Dir("../../opendbc/dbc").abspath)
|
||||
cabana_env['CXXFLAGS'] += [opendbc_path]
|
||||
|
||||
# build assets
|
||||
assets = "assets/assets.cc"
|
||||
assets_src = "assets/assets.qrc"
|
||||
cabana_env.Command(assets, assets_src, f"rcc $SOURCES -o $TARGET")
|
||||
cabana_env.Depends(assets, Glob('/assets/*', exclude=[assets, assets_src, "assets/assets.o"]))
|
||||
|
||||
cabana_lib = cabana_env.Library("cabana_lib", ['mainwin.cc', 'streams/socketcanstream.cc', 'streams/pandastream.cc', 'streams/devicestream.cc', 'streams/livestream.cc', 'streams/abstractstream.cc', 'streams/replaystream.cc', 'binaryview.cc', 'historylog.cc', 'videowidget.cc', 'signalview.cc',
|
||||
'streams/routes.cc', 'dbc/dbc.cc', 'dbc/dbcfile.cc', 'dbc/dbcmanager.cc',
|
||||
'utils/export.cc', 'utils/util.cc', 'utils/elidedlabel.cc', 'utils/api.cc',
|
||||
'chart/chartswidget.cc', 'chart/chart.cc', 'chart/signalselector.cc', 'chart/tiplabel.cc', 'chart/sparkline.cc',
|
||||
'commands.cc', 'messageswidget.cc', 'streamselector.cc', 'settings.cc', 'panda.cc',
|
||||
'cameraview.cc', 'detailwidget.cc', 'tools/findsimilarbits.cc', 'tools/findsignal.cc', 'tools/routeinfo.cc'], LIBS=cabana_libs, FRAMEWORKS=base_frameworks)
|
||||
cabana_env.Program('cabana', ['cabana.cc', cabana_lib, assets], LIBS=cabana_libs, FRAMEWORKS=base_frameworks)
|
||||
|
||||
if GetOption('extras'):
|
||||
cabana_env.Program('tests/test_cabana', ['tests/test_runner.cc', 'tests/test_cabana.cc', cabana_lib], LIBS=[cabana_libs])
|
||||
|
||||
output_json_file = 'tools/cabana/dbc/car_fingerprint_to_dbc.json'
|
||||
generate_dbc = cabana_env.Command('#' + output_json_file,
|
||||
['dbc/generate_dbc_json.py'],
|
||||
"python3 tools/cabana/dbc/generate_dbc_json.py --out " + output_json_file)
|
||||
cabana_env.Depends(generate_dbc, ["#common", '#opendbc_repo', "#cereal", "#msgq_repo"])
|
||||
@@ -0,0 +1 @@
|
||||
*.cc
|
||||
@@ -0,0 +1,6 @@
|
||||
<!DOCTYPE RCC><RCC version="1.0">
|
||||
<qresource>
|
||||
<file alias="bootstrap-icons.svg">../../../third_party/bootstrap/bootstrap-icons.svg</file>
|
||||
<file>cabana-icon.png</file>
|
||||
</qresource>
|
||||
</RCC>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
@@ -0,0 +1,104 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
#include <QList>
|
||||
#include <QSet>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class BinaryItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
BinaryItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool hasSignal(const QModelIndex &index, int dx, int dy, const cabana::Signal *sig) const;
|
||||
void drawSignalCell(QPainter* painter, const QStyleOptionViewItem &option, const QModelIndex &index, const cabana::Signal *sig) const;
|
||||
|
||||
QFont small_font, hex_font;
|
||||
std::array<QStaticText, 256> hex_text_table;
|
||||
std::array<QStaticText, 2> bin_text_table;
|
||||
};
|
||||
|
||||
class BinaryViewModel : public QAbstractTableModel {
|
||||
public:
|
||||
BinaryViewModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void refresh();
|
||||
void updateState();
|
||||
void updateItem(int row, int col, uint8_t val, const QColor &color);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return row_count; }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return column_count; }
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override {
|
||||
return createIndex(row, column, (void *)&items[row * column_count + column]);
|
||||
}
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override {
|
||||
return (index.column() == column_count - 1) ? Qt::ItemIsEnabled : Qt::ItemIsEnabled | Qt::ItemIsSelectable;
|
||||
}
|
||||
const std::vector<std::array<uint32_t, 8>> &getBitFlipChanges(size_t msg_size);
|
||||
|
||||
struct BitFlipTracker {
|
||||
std::optional<std::pair<double, double>> time_range;
|
||||
std::vector<std::array<uint32_t, 8>> flip_counts;
|
||||
} bit_flip_tracker;
|
||||
|
||||
struct Item {
|
||||
QColor bg_color = QColor(102, 86, 169, 255);
|
||||
bool is_msb = false;
|
||||
bool is_lsb = false;
|
||||
uint8_t val;
|
||||
QList<const cabana::Signal *> sigs;
|
||||
bool valid = false;
|
||||
};
|
||||
std::vector<Item> items;
|
||||
bool heatmap_live_mode = true;
|
||||
MessageId msg_id;
|
||||
int row_count = 0;
|
||||
const int column_count = 9;
|
||||
};
|
||||
|
||||
class BinaryView : public QTableView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
BinaryView(QWidget *parent = nullptr);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void highlight(const cabana::Signal *sig);
|
||||
QSet<const cabana::Signal*> getOverlappingSignals() const;
|
||||
void updateState() { model->updateState(); }
|
||||
void paintEvent(QPaintEvent *event) override {
|
||||
is_message_active = can->isMessageActive(model->msg_id);
|
||||
QTableView::paintEvent(event);
|
||||
}
|
||||
QSize minimumSizeHint() const override;
|
||||
void setHeatmapLiveMode(bool live) { model->heatmap_live_mode = live; updateState(); }
|
||||
|
||||
signals:
|
||||
void signalClicked(const cabana::Signal *sig);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void editSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void addShortcuts();
|
||||
void refresh();
|
||||
std::tuple<int, int, bool> getSelection(QModelIndex index);
|
||||
void setSelection(const QRect &rect, QItemSelectionModel::SelectionFlags flags) override;
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void leaveEvent(QEvent *event) override;
|
||||
void highlightPosition(const QPoint &pt);
|
||||
|
||||
QModelIndex anchor_index;
|
||||
BinaryViewModel *model;
|
||||
BinaryItemDelegate *delegate;
|
||||
bool is_message_active = false;
|
||||
const cabana::Signal *resize_sig = nullptr;
|
||||
const cabana::Signal *hovered_sig = nullptr;
|
||||
friend class BinaryItemDelegate;
|
||||
};
|
||||
@@ -0,0 +1,64 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
|
||||
#include <QOpenGLFunctions>
|
||||
#include <QOpenGLShaderProgram>
|
||||
#include <QOpenGLWidget>
|
||||
#include <QThread>
|
||||
|
||||
#include "msgq/visionipc/visionipc_client.h"
|
||||
|
||||
class CameraWidget : public QOpenGLWidget, protected QOpenGLFunctions {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
using QOpenGLWidget::QOpenGLWidget;
|
||||
explicit CameraWidget(std::string stream_name, VisionStreamType stream_type, QWidget* parent = nullptr);
|
||||
~CameraWidget();
|
||||
void setStreamType(VisionStreamType type) { requested_stream_type = type; }
|
||||
VisionStreamType getStreamType() { return active_stream_type; }
|
||||
void stopVipcThread();
|
||||
|
||||
signals:
|
||||
void clicked();
|
||||
void vipcThreadConnected(VisionIpcClient *);
|
||||
void vipcThreadFrameReceived();
|
||||
void vipcAvailableStreamsUpdated(std::set<VisionStreamType>);
|
||||
|
||||
protected:
|
||||
void paintGL() override;
|
||||
void initializeGL() override;
|
||||
void showEvent(QShowEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override { emit clicked(); }
|
||||
void vipcThread();
|
||||
void clearFrames();
|
||||
|
||||
GLuint frame_vao, frame_vbo, frame_ibo;
|
||||
GLuint textures[2];
|
||||
std::unique_ptr<QOpenGLShaderProgram> shader_program_;
|
||||
QColor bg = Qt::black;
|
||||
|
||||
std::string stream_name;
|
||||
int stream_width = 0;
|
||||
int stream_height = 0;
|
||||
int stream_stride = 0;
|
||||
std::atomic<VisionStreamType> active_stream_type;
|
||||
std::atomic<VisionStreamType> requested_stream_type;
|
||||
std::set<VisionStreamType> available_streams;
|
||||
QThread *vipc_thread = nullptr;
|
||||
std::recursive_mutex frame_lock;
|
||||
VisionBuf* current_frame_ = nullptr;
|
||||
VisionIpcBufExtra frame_meta_ = {};
|
||||
|
||||
protected slots:
|
||||
void vipcConnected(VisionIpcClient *vipc_client);
|
||||
void vipcFrameReceived();
|
||||
void availableStreamsUpdated(std::set<VisionStreamType> streams);
|
||||
};
|
||||
|
||||
Q_DECLARE_METATYPE(std::set<VisionStreamType>);
|
||||
@@ -0,0 +1,123 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QMenu>
|
||||
#include <QGraphicsPixmapItem>
|
||||
#include <QGraphicsProxyWidget>
|
||||
#include <QtCharts/QChartView>
|
||||
#include <QtCharts/QLegendMarker>
|
||||
#include <QtCharts/QLineSeries>
|
||||
#include <QtCharts/QScatterSeries>
|
||||
#include <QtCharts/QValueAxis>
|
||||
using namespace QtCharts;
|
||||
|
||||
#include "tools/cabana/chart/tiplabel.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
enum class SeriesType {
|
||||
Line = 0,
|
||||
StepLine,
|
||||
Scatter
|
||||
};
|
||||
|
||||
class ChartsWidget;
|
||||
class ChartView : public QChartView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartView(const std::pair<double, double> &x_range, ChartsWidget *parent = nullptr);
|
||||
void addSignal(const MessageId &msg_id, const cabana::Signal *sig);
|
||||
bool hasSignal(const MessageId &msg_id, const cabana::Signal *sig) const;
|
||||
void updateSeries(const cabana::Signal *sig = nullptr, const MessageEventsMap *msg_new_events = nullptr);
|
||||
void updatePlot(double cur, double min, double max);
|
||||
void setSeriesType(SeriesType type);
|
||||
void updatePlotArea(int left, bool force = false);
|
||||
void showTip(double sec);
|
||||
void hideTip();
|
||||
void startAnimation();
|
||||
double secondsAtPoint(const QPointF &pt) const { return chart()->mapToValue(pt).x(); }
|
||||
|
||||
struct SigItem {
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QXYSeries *series = nullptr;
|
||||
std::vector<QPointF> vals;
|
||||
std::vector<QPointF> step_vals;
|
||||
QPointF track_pt{};
|
||||
SegmentTree segment_tree;
|
||||
double min = 0;
|
||||
double max = 0;
|
||||
};
|
||||
|
||||
signals:
|
||||
void axisYLabelWidthChanged(int w);
|
||||
|
||||
private slots:
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void manageSignals();
|
||||
void handleMarkerClicked();
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id) { removeIf([=](auto &s) { return s.msg_id.address == id.address && !dbc()->msg(id); }); }
|
||||
void signalRemoved(const cabana::Signal *sig) { removeIf([=](auto &s) { return s.sig == sig; }); }
|
||||
|
||||
private:
|
||||
void appendCanEvents(const cabana::Signal *sig, const std::vector<const CanEvent *> &events,
|
||||
std::vector<QPointF> &vals, std::vector<QPointF> &step_vals);
|
||||
void createToolButtons();
|
||||
void addSeries(QXYSeries *series);
|
||||
void contextMenuEvent(QContextMenuEvent *event) override;
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *ev) override;
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator(false); }
|
||||
void dragMoveEvent(QDragMoveEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void resizeEvent(QResizeEvent *event) override;
|
||||
QSize sizeHint() const override;
|
||||
void updateAxisY();
|
||||
void updateTitle();
|
||||
void resetChartCache();
|
||||
void setTheme(QChart::ChartTheme theme);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void drawForeground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawBackground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawDropIndicator(bool draw) { if (std::exchange(can_drop, draw) != can_drop) viewport()->update(); }
|
||||
void drawSignalValue(QPainter *painter);
|
||||
void drawTimeline(QPainter *painter);
|
||||
void drawRubberBandTimeRange(QPainter *painter);
|
||||
std::tuple<double, double, int> getNiceAxisNumbers(qreal min, qreal max, int tick_count);
|
||||
qreal niceNumber(qreal x, bool ceiling);
|
||||
QXYSeries *createSeries(SeriesType type, QColor color);
|
||||
void setSeriesColor(QXYSeries *, QColor color);
|
||||
void updateSeriesPoints();
|
||||
void removeIf(std::function<bool(const SigItem &)> predicate);
|
||||
inline void clearTrackPoints() { for (auto &s : sigs) s.track_pt = {}; }
|
||||
|
||||
int y_label_width = 0;
|
||||
int align_to = 0;
|
||||
QValueAxis *axis_x;
|
||||
QValueAxis *axis_y;
|
||||
QMenu *menu;
|
||||
QAction *split_chart_act;
|
||||
QAction *close_act;
|
||||
QGraphicsPixmapItem *move_icon;
|
||||
QGraphicsProxyWidget *close_btn_proxy;
|
||||
QGraphicsProxyWidget *manage_btn_proxy;
|
||||
TipLabel *tip_label;
|
||||
std::vector<SigItem> sigs;
|
||||
double cur_sec = 0;
|
||||
SeriesType series_type = SeriesType::Line;
|
||||
bool is_scrubbing = false;
|
||||
bool resume_after_scrub = false;
|
||||
QPixmap chart_pixmap;
|
||||
bool can_drop = false;
|
||||
double tooltip_x = -1;
|
||||
QFont signal_value_font;
|
||||
ChartsWidget *charts_widget;
|
||||
friend class ChartsWidget;
|
||||
};
|
||||
@@ -0,0 +1,129 @@
|
||||
#pragma once
|
||||
|
||||
#include <unordered_map>
|
||||
#include <utility>
|
||||
|
||||
#include <QGridLayout>
|
||||
#include <QLabel>
|
||||
#include <QScrollArea>
|
||||
#include <QTimer>
|
||||
#include <QToolBar>
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/chart/signalselector.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
const int CHART_MIN_WIDTH = 300;
|
||||
const QString CHART_MIME_TYPE = "application/x-cabanachartview";
|
||||
|
||||
class ChartView;
|
||||
class ChartsWidget;
|
||||
|
||||
class ChartsContainer : public QWidget {
|
||||
public:
|
||||
ChartsContainer(ChartsWidget *parent);
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator({}); }
|
||||
void drawDropIndicator(const QPoint &pt) { drop_indictor_pos = pt; update(); }
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
ChartView *getDropAfter(const QPoint &pos) const;
|
||||
|
||||
QGridLayout *charts_layout;
|
||||
ChartsWidget *charts_widget;
|
||||
QPoint drop_indictor_pos;
|
||||
};
|
||||
|
||||
class ChartsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartsWidget(QWidget *parent = nullptr);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
inline bool hasSignal(const MessageId &id, const cabana::Signal *sig) { return findChart(id, sig) != nullptr; }
|
||||
|
||||
public slots:
|
||||
void setColumnCount(int n);
|
||||
void removeAll();
|
||||
void timeRangeChanged(const std::optional<std::pair<double, double>> &time_range);
|
||||
void setIsDocked(bool dock);
|
||||
|
||||
signals:
|
||||
void toggleChartsDocking();
|
||||
void seriesChanged();
|
||||
void showTip(double seconds);
|
||||
|
||||
private:
|
||||
QSize minimumSizeHint() const override;
|
||||
bool event(QEvent *event) override;
|
||||
void alignCharts();
|
||||
void newChart();
|
||||
ChartView *createChart(int pos = 0);
|
||||
void removeChart(ChartView *chart);
|
||||
void splitChart(ChartView *chart);
|
||||
QRect chartVisibleRect(ChartView *chart);
|
||||
void eventsMerged(const MessageEventsMap &new_events);
|
||||
void updateState();
|
||||
void zoomReset();
|
||||
void startAutoScroll();
|
||||
void stopAutoScroll();
|
||||
void doAutoScroll();
|
||||
void updateToolBar();
|
||||
void updateTabBar();
|
||||
void setMaxChartRange(int value);
|
||||
void updateLayout(bool force = false);
|
||||
void settingChanged();
|
||||
void showValueTip(double sec);
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
void newTab();
|
||||
void removeTab(int index);
|
||||
inline QList<ChartView *> ¤tCharts() { return tab_charts[tabbar->tabData(tabbar->currentIndex()).toInt()]; }
|
||||
ChartView *findChart(const MessageId &id, const cabana::Signal *sig);
|
||||
|
||||
QLabel *title_label;
|
||||
QLabel *range_lb;
|
||||
LogSlider *range_slider;
|
||||
QAction *range_lb_action;
|
||||
QAction *range_slider_action;
|
||||
bool is_docked = true;
|
||||
ToolButton *dock_btn;
|
||||
|
||||
QToolBar *toolbar;
|
||||
QAction *undo_zoom_action;
|
||||
QAction *redo_zoom_action;
|
||||
QAction *reset_zoom_action;
|
||||
ToolButton *reset_zoom_btn;
|
||||
QUndoStack *zoom_undo_stack;
|
||||
|
||||
ToolButton *remove_all_btn;
|
||||
QList<ChartView *> charts;
|
||||
std::unordered_map<int, QList<ChartView *>> tab_charts;
|
||||
TabBar *tabbar;
|
||||
ChartsContainer *charts_container;
|
||||
QScrollArea *charts_scroll;
|
||||
uint32_t max_chart_range = 0;
|
||||
std::pair<double, double> display_range;
|
||||
QAction *columns_action;
|
||||
int column_count = 1;
|
||||
int current_column_count = 0;
|
||||
int auto_scroll_count = 0;
|
||||
QTimer *auto_scroll_timer;
|
||||
QTimer *align_timer;
|
||||
int current_theme = 0;
|
||||
bool value_tip_visible_ = false;
|
||||
friend class ChartView;
|
||||
friend class ChartsContainer;
|
||||
};
|
||||
|
||||
class ZoomCommand : public QUndoCommand {
|
||||
public:
|
||||
ZoomCommand(std::pair<double, double> range) : range(range), QUndoCommand() {
|
||||
prev_range = can->timeRange();
|
||||
setText(QObject::tr("Zoom to %1-%2").arg(range.first, 0, 'f', 2).arg(range.second, 0, 'f', 2));
|
||||
}
|
||||
void undo() override { can->setTimeRange(prev_range); }
|
||||
void redo() override { can->setTimeRange(range); }
|
||||
std::optional<std::pair<double, double>> prev_range, range;
|
||||
};
|
||||
@@ -0,0 +1,30 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QListWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class SignalSelector : public QDialog {
|
||||
public:
|
||||
struct ListItem : public QListWidgetItem {
|
||||
ListItem(const MessageId &msg_id, const cabana::Signal *sig, QListWidget *parent) : msg_id(msg_id), sig(sig), QListWidgetItem(parent) {}
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig;
|
||||
};
|
||||
|
||||
SignalSelector(QString title, QWidget *parent);
|
||||
QList<ListItem *> seletedItems();
|
||||
inline void addSelected(const MessageId &id, const cabana::Signal *sig) { addItemToList(selected_list, id, sig, true); }
|
||||
|
||||
private:
|
||||
void updateAvailableList(int index);
|
||||
void addItemToList(QListWidget *parent, const MessageId id, const cabana::Signal *sig, bool show_msg_name = false);
|
||||
void add(QListWidgetItem *item);
|
||||
void remove(QListWidgetItem *item);
|
||||
|
||||
QComboBox *msgs_combo;
|
||||
QListWidget *available_list;
|
||||
QListWidget *selected_list;
|
||||
};
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
#include <QPixmap>
|
||||
#include <QPointF>
|
||||
#include <vector>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class Sparkline {
|
||||
public:
|
||||
void update(const cabana::Signal *sig, CanEventIter first, CanEventIter last, int range, QSize size);
|
||||
inline double freq() const { return freq_; }
|
||||
bool isEmpty() const { return pixmap.isNull(); }
|
||||
|
||||
QPixmap pixmap;
|
||||
double min_val = 0;
|
||||
double max_val = 0;
|
||||
|
||||
private:
|
||||
void render(const QColor &color, int range, QSize size);
|
||||
|
||||
std::vector<QPointF> points_;
|
||||
std::vector<QPointF> render_points_;
|
||||
double freq_ = 0;
|
||||
};
|
||||
@@ -0,0 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#include <QLabel>
|
||||
|
||||
class TipLabel : public QLabel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
TipLabel(QWidget *parent = nullptr);
|
||||
void showText(const QPoint &pt, const QString &sec, QWidget *w, const QRect &rect);
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
};
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
#include <utility>
|
||||
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class EditMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
EditMsgCommand(const MessageId &id, const QString &name, int size, const QString &node,
|
||||
const QString &comment, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QString old_name, new_name, old_comment, new_comment, old_node, new_node;
|
||||
int old_size = 0, new_size = 0;
|
||||
};
|
||||
|
||||
class RemoveMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveMsgCommand(const MessageId &id, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
cabana::Msg message;
|
||||
};
|
||||
|
||||
class AddSigCommand : public QUndoCommand {
|
||||
public:
|
||||
AddSigCommand(const MessageId &id, const cabana::Signal &sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
bool msg_created = false;
|
||||
cabana::Signal signal = {};
|
||||
};
|
||||
|
||||
class RemoveSigCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveSigCommand(const MessageId &id, const cabana::Signal *sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<cabana::Signal> sigs;
|
||||
};
|
||||
|
||||
class EditSignalCommand : public QUndoCommand {
|
||||
public:
|
||||
EditSignalCommand(const MessageId &id, const cabana::Signal *sig, const cabana::Signal &new_sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<std::pair<cabana::Signal, cabana::Signal>> sigs; // QList<{old_sig, new_sig}>
|
||||
};
|
||||
|
||||
namespace UndoStack {
|
||||
QUndoStack *instance();
|
||||
inline void push(QUndoCommand *cmd) { instance()->push(cmd); }
|
||||
};
|
||||
@@ -0,0 +1,120 @@
|
||||
#pragma once
|
||||
|
||||
#include <limits>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QMetaType>
|
||||
#include <QString>
|
||||
|
||||
const QString UNTITLED = "untitled";
|
||||
const QString DEFAULT_NODE_NAME = "XXX";
|
||||
constexpr int CAN_MAX_DATA_BYTES = 64;
|
||||
|
||||
struct MessageId {
|
||||
uint8_t source = 0;
|
||||
uint32_t address = 0;
|
||||
|
||||
QString toString() const {
|
||||
return QString("%1:%2").arg(source).arg(QString::number(address, 16).toUpper());
|
||||
}
|
||||
|
||||
bool operator==(const MessageId &other) const {
|
||||
return source == other.source && address == other.address;
|
||||
}
|
||||
|
||||
bool operator!=(const MessageId &other) const {
|
||||
return !(*this == other);
|
||||
}
|
||||
|
||||
bool operator<(const MessageId &other) const {
|
||||
return std::tie(source, address) < std::tie(other.source, other.address);
|
||||
}
|
||||
|
||||
bool operator>(const MessageId &other) const {
|
||||
return std::tie(source, address) > std::tie(other.source, other.address);
|
||||
}
|
||||
};
|
||||
|
||||
uint qHash(const MessageId &item);
|
||||
Q_DECLARE_METATYPE(MessageId);
|
||||
|
||||
template <>
|
||||
struct std::hash<MessageId> {
|
||||
std::size_t operator()(const MessageId &k) const noexcept { return qHash(k); }
|
||||
};
|
||||
|
||||
typedef std::vector<std::pair<double, QString>> ValueDescription;
|
||||
|
||||
namespace cabana {
|
||||
|
||||
class Signal {
|
||||
public:
|
||||
Signal() = default;
|
||||
Signal(const Signal &other) = default;
|
||||
void update();
|
||||
bool getValue(const uint8_t *data, size_t data_size, double *val) const;
|
||||
QString formatValue(double value, bool with_unit = true) const;
|
||||
bool operator==(const cabana::Signal &other) const;
|
||||
inline bool operator!=(const cabana::Signal &other) const { return !(*this == other); }
|
||||
|
||||
enum class Type {
|
||||
Normal = 0,
|
||||
Multiplexed,
|
||||
Multiplexor
|
||||
};
|
||||
|
||||
Type type = Type::Normal;
|
||||
QString name;
|
||||
int start_bit, msb, lsb, size;
|
||||
double factor = 1.0;
|
||||
double offset = 0;
|
||||
bool is_signed;
|
||||
bool is_little_endian;
|
||||
double min, max;
|
||||
QString unit;
|
||||
QString comment;
|
||||
QString receiver_name;
|
||||
ValueDescription val_desc;
|
||||
int precision = 0;
|
||||
QColor color;
|
||||
|
||||
// Multiplexed
|
||||
int multiplex_value = 0;
|
||||
Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
class Msg {
|
||||
public:
|
||||
Msg() = default;
|
||||
Msg(const Msg &other) { *this = other; }
|
||||
~Msg();
|
||||
cabana::Signal *addSignal(const cabana::Signal &sig);
|
||||
cabana::Signal *updateSignal(const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const QString &sig_name);
|
||||
Msg &operator=(const Msg &other);
|
||||
int indexOf(const cabana::Signal *sig) const;
|
||||
cabana::Signal *sig(const QString &sig_name) const;
|
||||
QString newSignalName();
|
||||
void update();
|
||||
inline const std::vector<cabana::Signal *> &getSignals() const { return sigs; }
|
||||
|
||||
uint32_t address;
|
||||
QString name;
|
||||
uint32_t size;
|
||||
QString comment;
|
||||
QString transmitter;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
|
||||
std::vector<uint8_t> mask;
|
||||
cabana::Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
} // namespace cabana
|
||||
|
||||
// Helper functions
|
||||
double get_raw_value(const uint8_t *data, size_t data_size, const cabana::Signal &sig);
|
||||
void updateMsbLsb(cabana::Signal &s);
|
||||
inline int flipBitPos(int start_bit) { return 8 * (start_bit / 8) + 7 - start_bit % 8; }
|
||||
inline QString doubleToString(double value) { return QString::number(value, 'g', std::numeric_limits<double>::digits10); }
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
#include <map>
|
||||
#include <QTextStream>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
|
||||
class DBCFile {
|
||||
public:
|
||||
DBCFile(const QString &dbc_file_name);
|
||||
DBCFile(const QString &name, const QString &content);
|
||||
~DBCFile() {}
|
||||
|
||||
bool save();
|
||||
bool saveAs(const QString &new_filename);
|
||||
bool writeContents(const QString &fn);
|
||||
QString generateDBC();
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &node, const QString &comment);
|
||||
inline void removeMsg(const MessageId &id) { msgs.erase(id.address); }
|
||||
|
||||
inline const std::map<uint32_t, cabana::Msg> &getMessages() const { return msgs; }
|
||||
cabana::Msg *msg(uint32_t address);
|
||||
cabana::Msg *msg(const QString &name);
|
||||
inline cabana::Msg *msg(const MessageId &id) { return msg(id.address); }
|
||||
cabana::Signal *signal(uint32_t address, const QString &name);
|
||||
|
||||
inline QString name() const { return name_.isEmpty() ? "untitled" : name_; }
|
||||
inline bool isEmpty() const { return msgs.empty() && name_.isEmpty(); }
|
||||
|
||||
QString filename;
|
||||
|
||||
private:
|
||||
void parse(const QString &content);
|
||||
cabana::Msg *parseBO(const QString &line);
|
||||
void parseSG(const QString &line, cabana::Msg *current_msg, int &multiplexor_cnt);
|
||||
void parseCM_BO(const QString &line, const QString &content, const QString &raw_line, const QTextStream &stream);
|
||||
void parseCM_SG(const QString &line, const QString &content, const QString &raw_line, const QTextStream &stream);
|
||||
void parseVAL(const QString &line);
|
||||
|
||||
QString header;
|
||||
std::map<uint32_t, cabana::Msg> msgs;
|
||||
QString name_;
|
||||
};
|
||||
@@ -0,0 +1,69 @@
|
||||
#pragma once
|
||||
|
||||
#include <QObject>
|
||||
#include <memory>
|
||||
#include <map>
|
||||
#include <set>
|
||||
|
||||
#include "tools/cabana/dbc/dbcfile.h"
|
||||
|
||||
typedef std::set<int> SourceSet;
|
||||
const SourceSet SOURCE_ALL = {-1};
|
||||
const int INVALID_SOURCE = 0xff;
|
||||
inline bool operator<(const std::shared_ptr<DBCFile> &l, const std::shared_ptr<DBCFile> &r) { return l.get() < r.get(); }
|
||||
|
||||
class DBCManager : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DBCManager(QObject *parent) : QObject(parent) {}
|
||||
~DBCManager() {}
|
||||
bool open(const SourceSet &sources, const QString &dbc_file_name, QString *error = nullptr);
|
||||
bool open(const SourceSet &sources, const QString &name, const QString &content, QString *error = nullptr);
|
||||
void close(const SourceSet &sources);
|
||||
void close(DBCFile *dbc_file);
|
||||
void closeAll();
|
||||
|
||||
void addSignal(const MessageId &id, const cabana::Signal &sig);
|
||||
void updateSignal(const MessageId &id, const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const MessageId &id, const QString &sig_name);
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &node, const QString &comment);
|
||||
void removeMsg(const MessageId &id);
|
||||
|
||||
QString newMsgName(const MessageId &id);
|
||||
QString newSignalName(const MessageId &id);
|
||||
|
||||
const std::map<uint32_t, cabana::Msg> &getMessages(uint8_t source);
|
||||
cabana::Msg *msg(const MessageId &id);
|
||||
cabana::Msg* msg(uint8_t source, const QString &name);
|
||||
|
||||
QStringList signalNames();
|
||||
inline int dbcCount() { return allDBCFiles().size(); }
|
||||
int nonEmptyDBCCount();
|
||||
|
||||
const SourceSet sources(const DBCFile *dbc_file) const;
|
||||
DBCFile *findDBCFile(const uint8_t source);
|
||||
inline DBCFile *findDBCFile(const MessageId &id) { return findDBCFile(id.source); }
|
||||
std::set<DBCFile *> allDBCFiles();
|
||||
|
||||
signals:
|
||||
void signalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void signalRemoved(const cabana::Signal *sig);
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id);
|
||||
void DBCFileChanged();
|
||||
void maskUpdated();
|
||||
|
||||
private:
|
||||
std::map<int, std::shared_ptr<DBCFile>> dbc_files;
|
||||
};
|
||||
|
||||
DBCManager *dbc();
|
||||
|
||||
QString toString(const SourceSet &ss);
|
||||
inline QString msgName(const MessageId &id) {
|
||||
auto msg = dbc()->msg(id);
|
||||
return msg ? msg->name : UNTITLED;
|
||||
}
|
||||
Executable
+39
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from opendbc.car import Bus
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.values import PLATFORMS
|
||||
|
||||
|
||||
def generate_dbc_dict() -> dict[str, str]:
|
||||
dbc_map = {}
|
||||
for platform in PLATFORMS.values():
|
||||
if platform != "MOCK":
|
||||
if Bus.pt in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.pt]
|
||||
elif Bus.main in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.main]
|
||||
elif Bus.party in platform.config.dbc_dict:
|
||||
dbc_map[platform.name] = platform.config.dbc_dict[Bus.party]
|
||||
else:
|
||||
raise ValueError("Unknown main type")
|
||||
|
||||
for m in MIGRATION:
|
||||
if MIGRATION[m] in dbc_map:
|
||||
dbc_map[m] = dbc_map[MIGRATION[m]]
|
||||
|
||||
return dbc_map
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Generate mapping for all car fingerprints to DBC names and outputs json file",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--out", required=True, help="Generated json filepath")
|
||||
args = parser.parse_args()
|
||||
|
||||
with open(args.out, 'w') as f:
|
||||
f.write(json.dumps(dict(sorted(generate_dbc_dict().items())), indent=2))
|
||||
print(f"Generated and written to {args.out}")
|
||||
@@ -0,0 +1,70 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialogButtonBox>
|
||||
#include <QSplitter>
|
||||
#include <QTabWidget>
|
||||
#include <QTextEdit>
|
||||
#include <set>
|
||||
|
||||
#include "tools/cabana/binaryview.h"
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/historylog.h"
|
||||
#include "tools/cabana/signalview.h"
|
||||
#include "tools/cabana/utils/elidedlabel.h"
|
||||
|
||||
class EditMessageDialog : public QDialog {
|
||||
public:
|
||||
EditMessageDialog(const MessageId &msg_id, const QString &title, int size, QWidget *parent);
|
||||
void validateName(const QString &text);
|
||||
|
||||
MessageId msg_id;
|
||||
QString original_name;
|
||||
QDialogButtonBox *btn_box;
|
||||
QLineEdit *name_edit;
|
||||
QLineEdit *node;
|
||||
QTextEdit *comment_edit;
|
||||
QLabel *error_label;
|
||||
QSpinBox *size_spin;
|
||||
};
|
||||
|
||||
class DetailWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DetailWidget(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void refresh();
|
||||
|
||||
private:
|
||||
void createToolBar();
|
||||
void showTabBarContextMenu(const QPoint &pt);
|
||||
void editMsg();
|
||||
void removeMsg();
|
||||
void updateState(const std::set<MessageId> *msgs = nullptr);
|
||||
|
||||
MessageId msg_id;
|
||||
QLabel *warning_icon, *warning_label;
|
||||
ElidedLabel *name_label;
|
||||
QWidget *warning_widget;
|
||||
TabBar *tabbar;
|
||||
QTabWidget *tab_widget;
|
||||
QAction *action_remove_msg;
|
||||
LogsWidget *history_log;
|
||||
BinaryView *binary_view;
|
||||
SignalView *signal_view;
|
||||
ChartsWidget *charts;
|
||||
QSplitter *splitter;
|
||||
};
|
||||
|
||||
class CenterWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
CenterWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &msg_id);
|
||||
void clear();
|
||||
|
||||
private:
|
||||
QWidget *createWelcomeWidget();
|
||||
DetailWidget *detail_widget = nullptr;
|
||||
QWidget *welcome_widget = nullptr;
|
||||
};
|
||||
@@ -0,0 +1,81 @@
|
||||
#pragma once
|
||||
|
||||
#include <deque>
|
||||
#include <vector>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QHeaderView>
|
||||
#include <QLineEdit>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class HeaderView : public QHeaderView {
|
||||
public:
|
||||
HeaderView(Qt::Orientation orientation, QWidget *parent = nullptr) : QHeaderView(orientation, parent) {}
|
||||
QSize sectionSizeFromContents(int logicalIndex) const override;
|
||||
void paintSection(QPainter *painter, const QRect &rect, int logicalIndex) const;
|
||||
};
|
||||
|
||||
class HistoryLogModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
HistoryLogModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void setMessage(const MessageId &message_id);
|
||||
void updateState(bool clear = false);
|
||||
void setFilter(int sig_idx, const QString &value, std::function<bool(double, double)> cmp);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
void fetchMore(const QModelIndex &parent) override;
|
||||
bool canFetchMore(const QModelIndex &parent) const override;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return messages.size(); }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return !isHexMode() ? sigs.size() + 1 : 2; }
|
||||
inline bool isHexMode() const { return sigs.empty() || hex_mode; }
|
||||
void reset();
|
||||
void setHexMode(bool hex_mode);
|
||||
|
||||
struct Message {
|
||||
uint64_t mono_time = 0;
|
||||
std::vector<double> sig_values;
|
||||
std::vector<uint8_t> data;
|
||||
std::vector<QColor> colors;
|
||||
};
|
||||
|
||||
void fetchData(std::deque<Message>::iterator insert_pos, uint64_t from_time, uint64_t min_time);
|
||||
|
||||
MessageId msg_id;
|
||||
CanData hex_colors;
|
||||
const int batch_size = 50;
|
||||
int filter_sig_idx = -1;
|
||||
double filter_value = 0;
|
||||
std::function<bool(double, double)> filter_cmp = nullptr;
|
||||
std::deque<Message> messages;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
bool hex_mode = false;
|
||||
};
|
||||
|
||||
class LogsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogsWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &message_id) { model->setMessage(message_id); }
|
||||
void updateState() { model->updateState(); }
|
||||
void showEvent(QShowEvent *event) override { model->updateState(true); }
|
||||
|
||||
private slots:
|
||||
void filterChanged();
|
||||
void exportToCSV();
|
||||
void modelReset();
|
||||
|
||||
private:
|
||||
QTableView *logs;
|
||||
HistoryLogModel *model;
|
||||
QComboBox *signals_cb, *comp_box, *display_type_cb;
|
||||
QLineEdit *value_edit;
|
||||
QWidget *filters_widget;
|
||||
ToolButton *export_btn;
|
||||
MessageBytesDelegate *delegate;
|
||||
};
|
||||
@@ -0,0 +1,110 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDockWidget>
|
||||
#include <QJsonDocument>
|
||||
#include <QMainWindow>
|
||||
#include <QMenu>
|
||||
#include <QProgressBar>
|
||||
#include <QSplitter>
|
||||
#include <QStatusBar>
|
||||
#include <set>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/detailwidget.h"
|
||||
#include "tools/cabana/messageswidget.h"
|
||||
#include "tools/cabana/videowidget.h"
|
||||
#include "tools/cabana/tools/findsimilarbits.h"
|
||||
|
||||
class MainWindow : public QMainWindow {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MainWindow(AbstractStream *stream, const QString &dbc_file);
|
||||
void toggleChartsDocking();
|
||||
void showStatusMessage(const QString &msg, int timeout = 0) { statusBar()->showMessage(msg, timeout); }
|
||||
void loadFile(const QString &fn, SourceSet s = SOURCE_ALL);
|
||||
ChartsWidget *charts_widget = nullptr;
|
||||
|
||||
public slots:
|
||||
void selectAndOpenStream();
|
||||
void openStream(AbstractStream *stream, const QString &dbc_file = {});
|
||||
void closeStream();
|
||||
void exportToCSV();
|
||||
|
||||
void newFile(SourceSet s = SOURCE_ALL);
|
||||
void openFile(SourceSet s = SOURCE_ALL);
|
||||
void loadDBCFromOpendbc(const QString &name);
|
||||
void save();
|
||||
void saveAs();
|
||||
void saveToClipboard();
|
||||
|
||||
signals:
|
||||
void showMessage(const QString &msg, int timeout);
|
||||
void updateProgressBar(uint64_t cur, uint64_t total, bool success);
|
||||
|
||||
protected:
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
void remindSaveChanges();
|
||||
void closeFile(SourceSet s = SOURCE_ALL);
|
||||
void closeFile(DBCFile *dbc_file);
|
||||
void saveFile(DBCFile *dbc_file);
|
||||
void saveFileAs(DBCFile *dbc_file);
|
||||
void saveFileToClipboard(DBCFile *dbc_file);
|
||||
void loadFingerprints();
|
||||
void loadFromClipboard(SourceSet s = SOURCE_ALL, bool close_all = true);
|
||||
void updateRecentFiles(const QString &fn);
|
||||
void updateRecentFileMenu();
|
||||
void createActions();
|
||||
void createDockWindows();
|
||||
void createStatusBar();
|
||||
void createShortcuts();
|
||||
void closeEvent(QCloseEvent *event) override;
|
||||
void DBCFileChanged();
|
||||
void updateDownloadProgress(uint64_t cur, uint64_t total, bool success);
|
||||
void setOption();
|
||||
void findSimilarBits();
|
||||
void findSignal();
|
||||
void undoStackCleanChanged(bool clean);
|
||||
void onlineHelp();
|
||||
void toggleFullScreen();
|
||||
void updateStatus();
|
||||
void updateLoadSaveMenus();
|
||||
void createDockWidgets();
|
||||
void eventsMerged();
|
||||
|
||||
VideoWidget *video_widget = nullptr;
|
||||
QDockWidget *video_dock;
|
||||
QDockWidget *messages_dock;
|
||||
MessagesWidget *messages_widget = nullptr;
|
||||
CenterWidget *center_widget;
|
||||
QWidget *floating_window = nullptr;
|
||||
QVBoxLayout *charts_layout;
|
||||
QProgressBar *progress_bar;
|
||||
QLabel *status_label;
|
||||
QJsonDocument fingerprint_to_dbc;
|
||||
QSplitter *video_splitter = nullptr;
|
||||
enum { MAX_RECENT_FILES = 15 };
|
||||
QMenu *open_recent_menu = nullptr;
|
||||
QMenu *manage_dbcs_menu = nullptr;
|
||||
QMenu *tools_menu = nullptr;
|
||||
QAction *close_stream_act = nullptr;
|
||||
QAction *export_to_csv_act = nullptr;
|
||||
QAction *save_dbc = nullptr;
|
||||
QAction *save_dbc_as = nullptr;
|
||||
QAction *copy_dbc_to_clipboard = nullptr;
|
||||
QString car_fingerprint;
|
||||
QByteArray default_state;
|
||||
};
|
||||
|
||||
class HelpOverlay : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
HelpOverlay(MainWindow *parent);
|
||||
|
||||
protected:
|
||||
void drawHelpForWidget(QPainter &painter, QWidget *w);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
};
|
||||
@@ -0,0 +1,121 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <optional>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QHeaderView>
|
||||
#include <QLineEdit>
|
||||
#include <QMenu>
|
||||
#include <QTreeView>
|
||||
#include <QWheelEvent>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class MessageListModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum Column {
|
||||
NAME = 0,
|
||||
SOURCE,
|
||||
ADDRESS,
|
||||
NODE,
|
||||
FREQ,
|
||||
COUNT,
|
||||
DATA,
|
||||
};
|
||||
|
||||
MessageListModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return Column::DATA + 1; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return items_.size(); }
|
||||
void sort(int column, Qt::SortOrder order = Qt::AscendingOrder) override;
|
||||
void setFilterStrings(const QMap<int, QString> &filters);
|
||||
void showInactivemessages(bool show);
|
||||
void msgsReceived(const std::set<MessageId> *new_msgs, bool has_new_ids);
|
||||
bool filterAndSort();
|
||||
void dbcModified();
|
||||
|
||||
struct Item {
|
||||
MessageId id;
|
||||
QString name;
|
||||
QString node;
|
||||
bool operator==(const Item &other) const {
|
||||
return id == other.id && name == other.name && node == other.node;
|
||||
}
|
||||
};
|
||||
std::vector<Item> items_;
|
||||
bool show_inactive_messages = true;
|
||||
|
||||
private:
|
||||
void sortItems(std::vector<MessageListModel::Item> &items);
|
||||
bool match(const MessageListModel::Item &id);
|
||||
|
||||
QMap<int, QString> filters_;
|
||||
std::set<MessageId> dbc_messages_;
|
||||
int sort_column = 0;
|
||||
Qt::SortOrder sort_order = Qt::AscendingOrder;
|
||||
int sort_threshold_ = 0;
|
||||
};
|
||||
|
||||
class MessageView : public QTreeView {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageView(QWidget *parent) : QTreeView(parent) {}
|
||||
void updateBytesSectionSize();
|
||||
|
||||
protected:
|
||||
void drawRow(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void drawBranches(QPainter *painter, const QRect &rect, const QModelIndex &index) const override {}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override;
|
||||
void wheelEvent(QWheelEvent *event) override;
|
||||
};
|
||||
|
||||
class MessageViewHeader : public QHeaderView {
|
||||
// https://stackoverflow.com/a/44346317
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageViewHeader(QWidget *parent);
|
||||
void updateHeaderPositions();
|
||||
void updateGeometries() override;
|
||||
QSize sizeHint() const override;
|
||||
void updateFilters();
|
||||
|
||||
QMap<int, QLineEdit *> editors;
|
||||
};
|
||||
|
||||
class MessagesWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MessagesWidget(QWidget *parent);
|
||||
void selectMessage(const MessageId &message_id);
|
||||
QByteArray saveHeaderState() const { return view->header()->saveState(); }
|
||||
bool restoreHeaderState(const QByteArray &state) const { return view->header()->restoreState(state); }
|
||||
void suppressHighlighted();
|
||||
|
||||
signals:
|
||||
void msgSelectionChanged(const MessageId &message_id);
|
||||
void titleChanged(const QString &title);
|
||||
|
||||
protected:
|
||||
QWidget *createToolBar();
|
||||
void headerContextMenuEvent(const QPoint &pos);
|
||||
void menuAboutToShow();
|
||||
void setMultiLineBytes(bool multi);
|
||||
void updateTitle();
|
||||
|
||||
MessageView *view;
|
||||
MessageViewHeader *header;
|
||||
MessageBytesDelegate *delegate;
|
||||
std::optional<MessageId> current_msg_id;
|
||||
MessageListModel *model;
|
||||
QPushButton *suppress_add;
|
||||
QPushButton *suppress_clear;
|
||||
QMenu *menu;
|
||||
};
|
||||
@@ -0,0 +1,95 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <ctime>
|
||||
#include <functional>
|
||||
#include <list>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include <atomic>
|
||||
#include <mutex>
|
||||
#include <libusb-1.0/libusb.h>
|
||||
|
||||
#include "cereal/gen/cpp/car.capnp.h"
|
||||
#include "cereal/gen/cpp/log.capnp.h"
|
||||
#include "panda/board/health.h"
|
||||
#include "panda/board/can.h"
|
||||
|
||||
#define USB_TX_SOFT_LIMIT (0x100U)
|
||||
#define USBPACKET_MAX_SIZE (0x40)
|
||||
#define RECV_SIZE (0x4000U)
|
||||
#define TIMEOUT 0
|
||||
|
||||
#define CAN_REJECTED_BUS_OFFSET 0xC0U
|
||||
#define CAN_RETURNED_BUS_OFFSET 0x80U
|
||||
|
||||
#define PANDA_BUS_OFFSET 4
|
||||
|
||||
struct __attribute__((packed)) can_header {
|
||||
uint8_t reserved : 1;
|
||||
uint8_t bus : 3;
|
||||
uint8_t data_len_code : 4;
|
||||
uint8_t rejected : 1;
|
||||
uint8_t returned : 1;
|
||||
uint8_t extended : 1;
|
||||
uint32_t addr : 29;
|
||||
uint8_t checksum : 8;
|
||||
};
|
||||
|
||||
struct can_frame {
|
||||
long address;
|
||||
std::string dat;
|
||||
long src;
|
||||
};
|
||||
|
||||
|
||||
class Panda {
|
||||
public:
|
||||
Panda(std::string serial="", uint32_t bus_offset=0);
|
||||
~Panda();
|
||||
|
||||
cereal::PandaState::PandaType hw_type = cereal::PandaState::PandaType::UNKNOWN;
|
||||
const uint32_t bus_offset;
|
||||
|
||||
bool connected();
|
||||
bool comms_healthy();
|
||||
std::string hw_serial();
|
||||
|
||||
// Static functions
|
||||
static std::vector<std::string> list(bool usb_only=false);
|
||||
|
||||
// Panda functionality
|
||||
cereal::PandaState::PandaType get_hw_type();
|
||||
void set_safety_model(cereal::CarParams::SafetyModel safety_model, uint16_t safety_param=0U);
|
||||
void send_heartbeat(bool engaged);
|
||||
void set_can_speed_kbps(uint16_t bus, uint16_t speed);
|
||||
void set_data_speed_kbps(uint16_t bus, uint16_t speed);
|
||||
bool can_receive(std::vector<can_frame>& out_vec);
|
||||
void can_reset_communications();
|
||||
|
||||
private:
|
||||
// USB connection members
|
||||
libusb_context *ctx = nullptr;
|
||||
libusb_device_handle *dev_handle = nullptr;
|
||||
std::string hw_serial_str;
|
||||
std::atomic<bool> connected_flag = true;
|
||||
std::atomic<bool> comms_healthy_flag = true;
|
||||
|
||||
// CAN buffer members
|
||||
uint8_t receive_buffer[RECV_SIZE + sizeof(can_header) + 64];
|
||||
uint32_t receive_buffer_size = 0;
|
||||
|
||||
// Internal methods
|
||||
bool init_usb_connection(const std::string& serial);
|
||||
void cleanup_usb();
|
||||
void handle_usb_issue(int err, const char func[]);
|
||||
int control_write(uint8_t request, uint16_t param1, uint16_t param2, unsigned int timeout=TIMEOUT);
|
||||
int control_read(uint8_t request, uint16_t param1, uint16_t param2, unsigned char *data, uint16_t length, unsigned int timeout=TIMEOUT);
|
||||
int bulk_write(unsigned char endpoint, unsigned char* data, int length, unsigned int timeout=TIMEOUT);
|
||||
int bulk_read(unsigned char endpoint, unsigned char* data, int length, unsigned int timeout=TIMEOUT);
|
||||
bool unpack_can_buffer(uint8_t *data, uint32_t &size, std::vector<can_frame> &out_vec);
|
||||
uint8_t calculate_checksum(uint8_t *data, uint32_t len);
|
||||
};
|
||||
@@ -0,0 +1,67 @@
|
||||
#pragma once
|
||||
|
||||
#include <QByteArray>
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QGroupBox>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
|
||||
#define LIGHT_THEME 1
|
||||
#define DARK_THEME 2
|
||||
|
||||
class Settings : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum DragDirection {
|
||||
MsbFirst,
|
||||
LsbFirst,
|
||||
AlwaysLE,
|
||||
AlwaysBE,
|
||||
};
|
||||
|
||||
Settings();
|
||||
~Settings();
|
||||
|
||||
bool absolute_time = false;
|
||||
int fps = 10;
|
||||
int max_cached_minutes = 30;
|
||||
int chart_height = 200;
|
||||
int chart_column_count = 1;
|
||||
int chart_range = 3 * 60; // 3 minutes
|
||||
int chart_series_type = 0;
|
||||
int theme = 0;
|
||||
int sparkline_range = 15; // 15 seconds
|
||||
bool multiple_lines_hex = false;
|
||||
bool log_livestream = true;
|
||||
bool suppress_defined_signals = false;
|
||||
QString log_path;
|
||||
QString last_dir;
|
||||
QString last_route_dir;
|
||||
QByteArray geometry;
|
||||
QByteArray video_splitter_state;
|
||||
QByteArray window_state;
|
||||
QStringList recent_files;
|
||||
QByteArray message_header_state;
|
||||
DragDirection drag_direction = MsbFirst;
|
||||
|
||||
signals:
|
||||
void changed();
|
||||
};
|
||||
|
||||
class SettingsDlg : public QDialog {
|
||||
public:
|
||||
SettingsDlg(QWidget *parent);
|
||||
void save();
|
||||
QSpinBox *fps;
|
||||
QSpinBox *cached_minutes;
|
||||
QSpinBox *chart_height;
|
||||
QComboBox *chart_series_type;
|
||||
QComboBox *theme;
|
||||
QGroupBox *log_livestream;
|
||||
QLineEdit *log_path;
|
||||
QComboBox *drag_direction;
|
||||
};
|
||||
|
||||
extern Settings settings;
|
||||
@@ -0,0 +1,147 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <utility>
|
||||
|
||||
#include <QAbstractItemModel>
|
||||
#include <QLabel>
|
||||
#include <QLineEdit>
|
||||
#include <QSlider>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableWidget>
|
||||
#include <QTreeView>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/chart/sparkline.h"
|
||||
|
||||
class SignalModel : public QAbstractItemModel {
|
||||
Q_OBJECT
|
||||
public:
|
||||
struct Item {
|
||||
enum Type {Root, Sig, Name, Size, Node, Endian, Signed, Offset, Factor, SignalType, MultiplexValue, ExtraInfo, Unit, Comment, Min, Max, Desc };
|
||||
~Item() { qDeleteAll(children); }
|
||||
inline int row() { return parent->children.indexOf(this); }
|
||||
|
||||
Type type = Type::Root;
|
||||
Item *parent = nullptr;
|
||||
QList<Item *> children;
|
||||
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QString title;
|
||||
bool highlight = false;
|
||||
QString sig_val = "-";
|
||||
Sparkline sparkline;
|
||||
};
|
||||
|
||||
SignalModel(QObject *parent);
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 2; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override;
|
||||
QModelIndex parent(const QModelIndex &index) const override;
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override;
|
||||
bool setData(const QModelIndex &index, const QVariant &value, int role = Qt::EditRole) override;
|
||||
void setMessage(const MessageId &id);
|
||||
void setFilter(const QString &txt);
|
||||
bool saveSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
Item *getItem(const QModelIndex &index) const;
|
||||
int signalRow(const cabana::Signal *sig) const;
|
||||
|
||||
private:
|
||||
void insertItem(SignalModel::Item *root_item, int pos, const cabana::Signal *sig);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void handleSignalRemoved(const cabana::Signal *sig);
|
||||
void handleMsgChanged(MessageId id);
|
||||
void refresh();
|
||||
|
||||
MessageId msg_id;
|
||||
QString filter_str;
|
||||
std::unique_ptr<Item> root;
|
||||
friend class SignalView;
|
||||
friend class SignalItemDelegate;
|
||||
};
|
||||
|
||||
class ValueDescriptionDlg : public QDialog {
|
||||
public:
|
||||
ValueDescriptionDlg(const ValueDescription &descriptions, QWidget *parent);
|
||||
ValueDescription val_desc;
|
||||
|
||||
private:
|
||||
struct Delegate : public QStyledItemDelegate {
|
||||
Delegate(QWidget *parent) : QStyledItemDelegate(parent) {}
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
};
|
||||
|
||||
void save();
|
||||
QTableWidget *table;
|
||||
};
|
||||
|
||||
class SignalItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
SignalItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void updateEditorGeometry(QWidget *editor, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void setModelData(QWidget *editor, QAbstractItemModel *model, const QModelIndex &index) const override;
|
||||
|
||||
QValidator *name_validator, *double_validator, *node_validator;
|
||||
QFont label_font, minmax_font;
|
||||
const int color_label_width = 18;
|
||||
mutable QSize button_size;
|
||||
};
|
||||
|
||||
class SignalView : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
SignalView(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &id);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void updateChartState();
|
||||
void selectSignal(const cabana::Signal *sig, bool expand = false);
|
||||
void rowClicked(const QModelIndex &index);
|
||||
SignalModel *model = nullptr;
|
||||
|
||||
signals:
|
||||
void highlight(const cabana::Signal *sig);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void rowsChanged();
|
||||
void resizeEvent(QResizeEvent* event) override;
|
||||
void updateToolBar();
|
||||
void setSparklineRange(int value);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void updateState(const std::set<MessageId> *msgs = nullptr);
|
||||
std::pair<QModelIndex, QModelIndex> visibleSignalRange();
|
||||
|
||||
struct TreeView : public QTreeView {
|
||||
TreeView(QWidget *parent) : QTreeView(parent) {}
|
||||
void rowsInserted(const QModelIndex &parent, int start, int end) override {
|
||||
((SignalView *)parentWidget())->rowsChanged();
|
||||
// update widget geometries in QTreeView::rowsInserted
|
||||
QTreeView::rowsInserted(parent, start, end);
|
||||
}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override {
|
||||
// Bypass the slow call to QTreeView::dataChanged.
|
||||
QAbstractItemView::dataChanged(topLeft, bottomRight, roles);
|
||||
}
|
||||
void leaveEvent(QEvent *event) override {
|
||||
emit static_cast<SignalView *>(parentWidget())->highlight(nullptr);
|
||||
QTreeView::leaveEvent(event);
|
||||
}
|
||||
};
|
||||
int max_value_width = 0;
|
||||
int value_column_width = 0;
|
||||
TreeView *tree;
|
||||
QLabel *sparkline_label;
|
||||
QSlider *sparkline_range_slider;
|
||||
QLineEdit *filter_edit;
|
||||
ChartsWidget *charts;
|
||||
QLabel *signal_count_lb;
|
||||
SignalItemDelegate *delegate;
|
||||
};
|
||||
@@ -0,0 +1,157 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <condition_variable>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <set>
|
||||
#include <unordered_map>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QDateTime>
|
||||
|
||||
#include "cereal/messaging/messaging.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/utils/util.h"
|
||||
#include "tools/replay/util.h"
|
||||
|
||||
struct CanData {
|
||||
void compute(const MessageId &msg_id, const uint8_t *dat, const int size, double current_sec,
|
||||
double playback_speed, const std::vector<uint8_t> &mask, double in_freq = 0);
|
||||
|
||||
double ts = 0.;
|
||||
uint32_t count = 0;
|
||||
double freq = 0;
|
||||
std::vector<uint8_t> dat;
|
||||
std::vector<QColor> colors;
|
||||
|
||||
struct ByteLastChange {
|
||||
double ts = 0;
|
||||
int delta = 0;
|
||||
int same_delta_counter = 0;
|
||||
bool suppressed = false;
|
||||
};
|
||||
std::vector<ByteLastChange> last_changes;
|
||||
std::vector<std::array<uint32_t, 8>> bit_flip_counts;
|
||||
double last_freq_update_ts = 0;
|
||||
};
|
||||
|
||||
struct CanEvent {
|
||||
uint8_t src;
|
||||
uint32_t address;
|
||||
uint64_t mono_time;
|
||||
uint8_t size;
|
||||
uint8_t dat[];
|
||||
};
|
||||
|
||||
struct CompareCanEvent {
|
||||
constexpr bool operator()(const CanEvent *const e, uint64_t ts) const { return e->mono_time < ts; }
|
||||
constexpr bool operator()(uint64_t ts, const CanEvent *const e) const { return ts < e->mono_time; }
|
||||
};
|
||||
|
||||
typedef std::unordered_map<MessageId, std::vector<const CanEvent *>> MessageEventsMap;
|
||||
using CanEventIter = std::vector<const CanEvent *>::const_iterator;
|
||||
|
||||
class AbstractStream : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
AbstractStream(QObject *parent);
|
||||
virtual ~AbstractStream() {}
|
||||
virtual void start() = 0;
|
||||
virtual bool liveStreaming() const { return true; }
|
||||
virtual void seekTo(double ts) {}
|
||||
virtual QString routeName() const = 0;
|
||||
virtual QString carFingerprint() const { return ""; }
|
||||
virtual QDateTime beginDateTime() const { return {}; }
|
||||
virtual uint64_t beginMonoTime() const { return 0; }
|
||||
virtual double minSeconds() const { return 0; }
|
||||
virtual double maxSeconds() const { return 0; }
|
||||
virtual void setSpeed(float speed) {}
|
||||
virtual double getSpeed() { return 1; }
|
||||
virtual bool isPaused() const { return false; }
|
||||
virtual void pause(bool pause) {}
|
||||
void setTimeRange(const std::optional<std::pair<double, double>> &range);
|
||||
const std::optional<std::pair<double, double>> &timeRange() const { return time_range_; }
|
||||
|
||||
inline double currentSec() const { return current_sec_; }
|
||||
inline uint64_t toMonoTime(double sec) const { return beginMonoTime() + std::max(sec, 0.0) * 1e9; }
|
||||
inline double toSeconds(uint64_t mono_time) const { return std::max(0.0, (mono_time - beginMonoTime()) / 1e9); }
|
||||
|
||||
inline const std::unordered_map<MessageId, CanData> &lastMessages() const { return last_msgs; }
|
||||
bool isMessageActive(const MessageId &id) const;
|
||||
inline const MessageEventsMap &eventsMap() const { return events_; }
|
||||
inline const std::vector<const CanEvent *> &allEvents() const { return all_events_; }
|
||||
const CanData &lastMessage(const MessageId &id) const;
|
||||
const std::vector<const CanEvent *> &events(const MessageId &id) const;
|
||||
std::pair<CanEventIter, CanEventIter> eventsInRange(const MessageId &id, std::optional<std::pair<double, double>> time_range) const;
|
||||
|
||||
size_t suppressHighlighted();
|
||||
void clearSuppressed();
|
||||
void suppressDefinedSignals(bool suppress);
|
||||
|
||||
signals:
|
||||
void paused();
|
||||
void resume();
|
||||
void seeking(double sec);
|
||||
void seekedTo(double sec);
|
||||
void timeRangeChanged(const std::optional<std::pair<double, double>> &range);
|
||||
void eventsMerged(const MessageEventsMap &events_map);
|
||||
void msgsReceived(const std::set<MessageId> *new_msgs, bool has_new_ids);
|
||||
void sourcesUpdated(const SourceSet &s);
|
||||
void privateUpdateLastMsgsSignal();
|
||||
|
||||
public:
|
||||
SourceSet sources;
|
||||
|
||||
protected:
|
||||
void mergeEvents(const std::vector<const CanEvent *> &events);
|
||||
const CanEvent *newEvent(uint64_t mono_time, const cereal::CanData::Reader &c);
|
||||
void updateEvent(const MessageId &id, double sec, const uint8_t *data, uint8_t size);
|
||||
void waitForSeekFinshed();
|
||||
std::vector<const CanEvent *> all_events_;
|
||||
double current_sec_ = 0;
|
||||
std::optional<std::pair<double, double>> time_range_;
|
||||
|
||||
private:
|
||||
void updateLastMessages();
|
||||
void updateLastMsgsTo(double sec);
|
||||
void updateMasks();
|
||||
|
||||
MessageEventsMap events_;
|
||||
std::unordered_map<MessageId, CanData> last_msgs;
|
||||
std::unique_ptr<MonotonicBuffer> event_buffer_;
|
||||
|
||||
// Members accessed in multiple threads. (mutex protected)
|
||||
std::mutex mutex_;
|
||||
std::condition_variable seek_finished_cv_;
|
||||
bool seek_finished_ = false;
|
||||
std::set<MessageId> new_msgs_;
|
||||
std::unordered_map<MessageId, CanData> messages_;
|
||||
std::unordered_map<MessageId, std::vector<uint8_t>> masks_;
|
||||
};
|
||||
|
||||
class AbstractOpenStreamWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
AbstractOpenStreamWidget(QWidget *parent = nullptr) : QWidget(parent) {}
|
||||
virtual AbstractStream *open() = 0;
|
||||
|
||||
signals:
|
||||
void enableOpenButton(bool);
|
||||
};
|
||||
|
||||
class DummyStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DummyStream(QObject *parent) : AbstractStream(parent) {}
|
||||
QString routeName() const override { return tr("No Stream"); }
|
||||
void start() override {}
|
||||
};
|
||||
|
||||
// A global pointer referring to the unique AbstractStream object
|
||||
extern AbstractStream *can;
|
||||
@@ -0,0 +1,28 @@
|
||||
#pragma once
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
class DeviceStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DeviceStream(QObject *parent, QString address = {});
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From %1").arg(zmq_address.isEmpty() ? "127.0.0.1" : zmq_address);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
const QString zmq_address;
|
||||
};
|
||||
|
||||
class OpenDeviceWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenDeviceWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
QLineEdit *ip_address;
|
||||
QButtonGroup *group;
|
||||
};
|
||||
@@ -0,0 +1,56 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QBasicTimer>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class LiveStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LiveStream(QObject *parent);
|
||||
virtual ~LiveStream();
|
||||
void start() override;
|
||||
void stop();
|
||||
inline QDateTime beginDateTime() const { return begin_date_time; }
|
||||
inline uint64_t beginMonoTime() const override { return begin_event_ts; }
|
||||
double maxSeconds() const override { return std::max(1.0, (lastest_event_ts - begin_event_ts) / 1e9); }
|
||||
void setSpeed(float speed) override { speed_ = speed; }
|
||||
double getSpeed() override { return speed_; }
|
||||
bool isPaused() const override { return paused_; }
|
||||
void pause(bool pause) override;
|
||||
void seekTo(double sec) override;
|
||||
|
||||
protected:
|
||||
virtual void streamThread() = 0;
|
||||
void handleEvent(kj::ArrayPtr<capnp::word> event);
|
||||
|
||||
private:
|
||||
void startUpdateTimer();
|
||||
void timerEvent(QTimerEvent *event) override;
|
||||
void updateEvents();
|
||||
|
||||
std::mutex lock;
|
||||
QThread *stream_thread;
|
||||
std::vector<const CanEvent *> received_events_;
|
||||
|
||||
int timer_id;
|
||||
QBasicTimer update_timer;
|
||||
|
||||
QDateTime begin_date_time;
|
||||
uint64_t begin_event_ts = 0;
|
||||
uint64_t lastest_event_ts = 0;
|
||||
uint64_t current_event_ts = 0;
|
||||
uint64_t first_event_ts = 0;
|
||||
uint64_t first_update_ts = 0;
|
||||
bool post_last_event = true;
|
||||
double speed_ = 1;
|
||||
bool paused_ = false;
|
||||
|
||||
struct Logger;
|
||||
std::unique_ptr<Logger> logger;
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QFormLayout>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
#include "tools/cabana/panda.h"
|
||||
|
||||
const uint32_t speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U};
|
||||
const uint32_t data_speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U, 2000U, 5000U};
|
||||
|
||||
struct BusConfig {
|
||||
int can_speed_kbps = 500;
|
||||
int data_speed_kbps = 2000;
|
||||
bool can_fd = false;
|
||||
};
|
||||
|
||||
struct PandaStreamConfig {
|
||||
QString serial = "";
|
||||
std::vector<BusConfig> bus_config;
|
||||
};
|
||||
|
||||
class PandaStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
PandaStream(QObject *parent, PandaStreamConfig config_ = {});
|
||||
~PandaStream() { stop(); }
|
||||
inline QString routeName() const override {
|
||||
return QString("Panda: %1").arg(config.serial);
|
||||
}
|
||||
|
||||
protected:
|
||||
bool connect();
|
||||
void streamThread() override;
|
||||
|
||||
std::unique_ptr<Panda> panda;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
|
||||
class OpenPandaWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenPandaWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
void refreshSerials();
|
||||
void buildConfigForm();
|
||||
|
||||
QComboBox *serial_edit;
|
||||
QFormLayout *form_layout;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
#include <QCheckBox>
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include "common/prefix.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
#include "tools/replay/replay.h"
|
||||
|
||||
Q_DECLARE_METATYPE(std::shared_ptr<LogReader>);
|
||||
|
||||
class ReplayStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ReplayStream(QObject *parent);
|
||||
void start() override { replay->start(); }
|
||||
bool loadRoute(const QString &route, const QString &data_dir, uint32_t replay_flags = REPLAY_FLAG_NONE, bool auto_source = false);
|
||||
bool eventFilter(const Event *event);
|
||||
void seekTo(double ts) override { replay->seekTo(std::max(double(0), ts), false); }
|
||||
bool liveStreaming() const override { return false; }
|
||||
inline QString routeName() const override { return QString::fromStdString(replay->route().name()); }
|
||||
inline QString carFingerprint() const override { return replay->carFingerprint().c_str(); }
|
||||
double minSeconds() const override { return replay->minSeconds(); }
|
||||
double maxSeconds() const { return replay->maxSeconds(); }
|
||||
inline QDateTime beginDateTime() const { return QDateTime::fromSecsSinceEpoch(replay->routeDateTime()); }
|
||||
inline uint64_t beginMonoTime() const override { return replay->routeStartNanos(); }
|
||||
inline void setSpeed(float speed) override { replay->setSpeed(speed); }
|
||||
inline float getSpeed() const { return replay->getSpeed(); }
|
||||
inline Replay *getReplay() const { return replay.get(); }
|
||||
inline bool isPaused() const override { return replay->isPaused(); }
|
||||
void pause(bool pause) override;
|
||||
|
||||
signals:
|
||||
void qLogLoaded(std::shared_ptr<LogReader> qlog);
|
||||
|
||||
private:
|
||||
void mergeSegments();
|
||||
std::unique_ptr<Replay> replay = nullptr;
|
||||
std::set<int> processed_segments;
|
||||
std::unique_ptr<OpenpilotPrefix> op_prefix;
|
||||
};
|
||||
|
||||
class OpenReplayWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenReplayWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
QLineEdit *route_edit;
|
||||
std::vector<QCheckBox *> cameras;
|
||||
};
|
||||
@@ -0,0 +1,25 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include "tools/cabana/utils/api.h"
|
||||
|
||||
class RouteListWidget;
|
||||
class OneShotHttpRequest;
|
||||
|
||||
class RoutesDialog : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
RoutesDialog(QWidget *parent);
|
||||
QString route();
|
||||
|
||||
protected:
|
||||
void parseDeviceList(const QString &json, bool success, QNetworkReply::NetworkError err);
|
||||
void parseRouteList(const QString &json, bool success, QNetworkReply::NetworkError err);
|
||||
void fetchRoutes();
|
||||
|
||||
QComboBox *device_list_;
|
||||
QComboBox *period_selector_;
|
||||
RouteListWidget *route_list_;
|
||||
OneShotHttpRequest *route_requester_;
|
||||
};
|
||||
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include <QtSerialBus/QCanBus>
|
||||
#include <QtSerialBus/QCanBusDevice>
|
||||
#include <QtSerialBus/QCanBusDeviceInfo>
|
||||
#include <QComboBox>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
struct SocketCanStreamConfig {
|
||||
QString device = ""; // TODO: support multiple devices/buses at once
|
||||
};
|
||||
|
||||
class SocketCanStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
SocketCanStream(QObject *parent, SocketCanStreamConfig config_ = {});
|
||||
~SocketCanStream() { stop(); }
|
||||
static bool available();
|
||||
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From Socket CAN %1").arg(config.device);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
bool connect();
|
||||
|
||||
SocketCanStreamConfig config = {};
|
||||
std::unique_ptr<QCanBusDevice> device;
|
||||
};
|
||||
|
||||
class OpenSocketCanWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenSocketCanWidget(QWidget *parent = nullptr);
|
||||
AbstractStream *open() override;
|
||||
|
||||
private:
|
||||
void refreshDevices();
|
||||
|
||||
QComboBox *device_edit;
|
||||
SocketCanStreamConfig config = {};
|
||||
};
|
||||
@@ -0,0 +1,24 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialogButtonBox>
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QTabWidget>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class StreamSelector : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
StreamSelector(QWidget *parent = nullptr);
|
||||
void addStreamWidget(AbstractOpenStreamWidget *w, const QString &title);
|
||||
QString dbcFile() const { return dbc_file->text(); }
|
||||
AbstractStream *stream() const { return stream_; }
|
||||
|
||||
private:
|
||||
AbstractStream *stream_ = nullptr;
|
||||
QLineEdit *dbc_file;
|
||||
QTabWidget *tab;
|
||||
QDialogButtonBox *btn_box;
|
||||
};
|
||||
@@ -0,0 +1,64 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <limits>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QCheckBox>
|
||||
#include <QLabel>
|
||||
#include <QPushButton>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/commands.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class FindSignalModel : public QAbstractTableModel {
|
||||
public:
|
||||
struct SearchSignal {
|
||||
MessageId id = {};
|
||||
uint64_t mono_time = 0;
|
||||
cabana::Signal sig = {};
|
||||
double value = 0.;
|
||||
QStringList values;
|
||||
};
|
||||
|
||||
FindSignalModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 3; }
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return std::min(filtered_signals.size(), 300); }
|
||||
void search(std::function<bool(double)> cmp);
|
||||
void reset();
|
||||
void undo();
|
||||
|
||||
QList<SearchSignal> filtered_signals;
|
||||
QList<SearchSignal> initial_signals;
|
||||
QList<QList<SearchSignal>> histories;
|
||||
uint64_t last_time = std::numeric_limits<uint64_t>::max();
|
||||
};
|
||||
|
||||
class FindSignalDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
FindSignalDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &id);
|
||||
|
||||
private:
|
||||
void search();
|
||||
void modelReset();
|
||||
void setInitialSignals();
|
||||
void customMenuRequested(const QPoint &pos);
|
||||
|
||||
QLineEdit *value1, *value2, *factor_edit, *offset_edit;
|
||||
QLineEdit *bus_edit, *address_edit, *first_time_edit, *last_time_edit;
|
||||
QComboBox *compare_cb;
|
||||
QSpinBox *min_size, *max_size;
|
||||
QCheckBox *litter_endian, *is_signed;
|
||||
QPushButton *search_btn, *reset_btn, *undo_btn;
|
||||
QGroupBox *properties_group, *message_group;
|
||||
QTableView *view;
|
||||
QLabel *to_label, *stats_label;
|
||||
FindSignalModel *model;
|
||||
};
|
||||
@@ -0,0 +1,34 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
#include <QTableWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class FindSimilarBitsDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
FindSimilarBitsDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &msg_id);
|
||||
|
||||
private:
|
||||
struct mismatched_struct {
|
||||
uint32_t address, byte_idx, bit_idx, mismatches, total;
|
||||
float perc;
|
||||
};
|
||||
QList<mismatched_struct> calcBits(uint8_t bus, uint32_t selected_address, int byte_idx, int bit_idx, uint8_t find_bus,
|
||||
bool equal, int min_msgs_cnt);
|
||||
void find();
|
||||
|
||||
QTableWidget *table;
|
||||
QComboBox *src_bus_combo, *find_bus_combo, *msg_cb, *equal_combo;
|
||||
QSpinBox *byte_idx_sb, *bit_idx_sb;
|
||||
QPushButton *search_btn;
|
||||
QLineEdit *min_msgs;
|
||||
};
|
||||
@@ -0,0 +1,8 @@
|
||||
#pragma once
|
||||
#include <QDialog>
|
||||
|
||||
class RouteInfoDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
RouteInfoDlg(QWidget *parent = nullptr);
|
||||
};
|
||||
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
#include <QJsonObject>
|
||||
#include <QNetworkReply>
|
||||
#include <QString>
|
||||
#include <QTimer>
|
||||
|
||||
#include "common/util.h"
|
||||
|
||||
namespace CommaApi {
|
||||
|
||||
const QString BASE_URL = util::getenv("API_HOST", "https://api.commadotai.com").c_str();
|
||||
QByteArray rsa_sign(const QByteArray &data);
|
||||
QString create_jwt(const QJsonObject &payloads = {}, int expiry = 3600);
|
||||
|
||||
} // namespace CommaApi
|
||||
|
||||
/**
|
||||
* Makes a request to the request endpoint.
|
||||
*/
|
||||
|
||||
class HttpRequest : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum class Method {GET, DELETE};
|
||||
|
||||
explicit HttpRequest(QObject* parent, bool create_jwt = true, int timeout = 20000);
|
||||
void sendRequest(const QString &requestURL, const Method method = Method::GET);
|
||||
bool active() const;
|
||||
bool timeout() const;
|
||||
|
||||
signals:
|
||||
void requestDone(const QString &response, bool success, QNetworkReply::NetworkError error);
|
||||
|
||||
protected:
|
||||
QNetworkReply *reply = nullptr;
|
||||
|
||||
private:
|
||||
static QNetworkAccessManager *nam();
|
||||
QTimer *networkTimer = nullptr;
|
||||
bool create_jwt;
|
||||
|
||||
private slots:
|
||||
void requestTimeout();
|
||||
void requestFinished();
|
||||
};
|
||||
@@ -0,0 +1,25 @@
|
||||
#pragma once
|
||||
|
||||
#include <QLabel>
|
||||
#include <QMouseEvent>
|
||||
|
||||
class ElidedLabel : public QLabel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
explicit ElidedLabel(QWidget *parent = 0);
|
||||
explicit ElidedLabel(const QString &text, QWidget *parent = 0);
|
||||
|
||||
signals:
|
||||
void clicked();
|
||||
|
||||
protected:
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void resizeEvent(QResizeEvent* event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override {
|
||||
if (rect().contains(event->pos())) {
|
||||
emit clicked();
|
||||
}
|
||||
}
|
||||
QString lastText_, elidedText_;
|
||||
};
|
||||
@@ -0,0 +1,10 @@
|
||||
#pragma once
|
||||
|
||||
#include <optional>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
namespace utils {
|
||||
void exportToCSV(const QString &file_name, std::optional<MessageId> msg_id = std::nullopt);
|
||||
void exportSignalsToCSV(const QString &file_name, const MessageId &msg_id);
|
||||
} // namespace utils
|
||||
@@ -0,0 +1,170 @@
|
||||
#pragma once
|
||||
|
||||
#include <array>
|
||||
#include <cmath>
|
||||
#include <vector>
|
||||
#include <utility>
|
||||
|
||||
#include <QApplication>
|
||||
#include <QByteArray>
|
||||
#include <QDoubleValidator>
|
||||
#include <QFont>
|
||||
#include <QFontMetrics>
|
||||
#include <QPainter>
|
||||
#include <QRegExpValidator>
|
||||
#include <QSocketNotifier>
|
||||
#include <QStaticText>
|
||||
#include <QStringBuilder>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QToolButton>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class LogSlider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogSlider(double factor, Qt::Orientation orientation, QWidget *parent = nullptr) : factor(factor), QSlider(orientation, parent) {}
|
||||
|
||||
void setRange(double min, double max) {
|
||||
log_min = factor * std::log10(min);
|
||||
log_max = factor * std::log10(max);
|
||||
QSlider::setRange(min, max);
|
||||
setValue(QSlider::value());
|
||||
}
|
||||
int value() const {
|
||||
double v = log_min + (log_max - log_min) * ((QSlider::value() - minimum()) / double(maximum() - minimum()));
|
||||
return std::lround(std::pow(10, v / factor));
|
||||
}
|
||||
void setValue(int v) {
|
||||
double log_v = std::clamp(factor * std::log10(v), log_min, log_max);
|
||||
v = minimum() + (maximum() - minimum()) * ((log_v - log_min) / (log_max - log_min));
|
||||
QSlider::setValue(v);
|
||||
}
|
||||
|
||||
private:
|
||||
double factor, log_min = 0, log_max = 1;
|
||||
};
|
||||
|
||||
enum {
|
||||
ColorsRole = Qt::UserRole + 1,
|
||||
BytesRole = Qt::UserRole + 2
|
||||
};
|
||||
|
||||
class SegmentTree {
|
||||
public:
|
||||
SegmentTree() = default;
|
||||
void build(const std::vector<QPointF> &arr);
|
||||
inline std::pair<double, double> minmax(int left, int right) const { return get_minmax(1, 0, size - 1, left, right); }
|
||||
|
||||
private:
|
||||
std::pair<double, double> get_minmax(int n, int left, int right, int range_left, int range_right) const;
|
||||
void build_tree(const std::vector<QPointF> &arr, int n, int left, int right);
|
||||
std::vector<std::pair<double, double>> tree;
|
||||
int size = 0;
|
||||
};
|
||||
|
||||
class MessageBytesDelegate : public QStyledItemDelegate {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageBytesDelegate(QObject *parent, bool multiple_lines = false);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool multipleLines() const { return multiple_lines; }
|
||||
void setMultipleLines(bool v) { multiple_lines = v; }
|
||||
QSize sizeForBytes(int n) const;
|
||||
|
||||
private:
|
||||
std::array<QStaticText, 256> hex_text_table;
|
||||
QFontMetrics font_metrics;
|
||||
QFont fixed_font;
|
||||
QSize byte_size = {};
|
||||
bool multiple_lines = false;
|
||||
int h_margin, v_margin;
|
||||
};
|
||||
|
||||
class NameValidator : public QRegExpValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
NameValidator(QObject *parent=nullptr);
|
||||
QValidator::State validate(QString &input, int &pos) const override;
|
||||
};
|
||||
|
||||
class DoubleValidator : public QDoubleValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DoubleValidator(QObject *parent = nullptr);
|
||||
};
|
||||
|
||||
namespace utils {
|
||||
|
||||
QPixmap icon(const QString &id);
|
||||
bool isDarkTheme();
|
||||
void setTheme(int theme);
|
||||
QString formatSeconds(double sec, bool include_milliseconds = false, bool absolute_time = false);
|
||||
inline void drawStaticText(QPainter *p, const QRect &r, const QStaticText &text) {
|
||||
auto size = (r.size() - text.size()) / 2;
|
||||
p->drawStaticText(r.left() + size.width(), r.top() + size.height(), text);
|
||||
}
|
||||
inline QString toHex(const std::vector<uint8_t> &dat, char separator = '\0') {
|
||||
return QByteArray::fromRawData((const char *)dat.data(), dat.size()).toHex(separator).toUpper();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
class ToolButton : public QToolButton {
|
||||
Q_OBJECT
|
||||
public:
|
||||
ToolButton(const QString &icon, const QString &tooltip = {}, QWidget *parent = nullptr) : QToolButton(parent) {
|
||||
setIcon(icon);
|
||||
setToolTip(tooltip);
|
||||
setAutoRaise(true);
|
||||
const int metric = QApplication::style()->pixelMetric(QStyle::PM_SmallIconSize);
|
||||
setIconSize({metric, metric});
|
||||
theme = settings.theme;
|
||||
connect(&settings, &Settings::changed, this, &ToolButton::updateIcon);
|
||||
}
|
||||
void setIcon(const QString &icon) {
|
||||
icon_str = icon;
|
||||
QToolButton::setIcon(utils::icon(icon_str));
|
||||
}
|
||||
|
||||
private:
|
||||
void updateIcon() { if (std::exchange(theme, settings.theme) != theme) setIcon(icon_str); }
|
||||
QString icon_str;
|
||||
int theme;
|
||||
};
|
||||
|
||||
class TabBar : public QTabBar {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
TabBar(QWidget *parent) : QTabBar(parent) {}
|
||||
int addTab(const QString &text);
|
||||
|
||||
private:
|
||||
void closeTabClicked();
|
||||
};
|
||||
|
||||
class UnixSignalHandler : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
UnixSignalHandler(QObject *parent = nullptr);
|
||||
~UnixSignalHandler();
|
||||
static void signalHandler(int s);
|
||||
|
||||
public slots:
|
||||
void handleSigTerm();
|
||||
|
||||
private:
|
||||
inline static int sig_fd[2] = {};
|
||||
QSocketNotifier *sn;
|
||||
};
|
||||
|
||||
int num_decimals(double num);
|
||||
QString signalToolTip(const cabana::Signal *sig);
|
||||
inline QString toHexString(int value) { return QString("0x%1").arg(QString::number(value, 16).toUpper(), 2, '0'); }
|
||||
void initApp(int argc, char *argv[], bool disable_hidpi = true);
|
||||
QPixmap bootstrapPixmap(const QString &id);
|
||||
@@ -0,0 +1,83 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
|
||||
#include <QFrame>
|
||||
#include <QPropertyAnimation>
|
||||
#include <QSlider>
|
||||
#include <QToolBar>
|
||||
#include <QTabBar>
|
||||
|
||||
#include "tools/cabana/cameraview.h"
|
||||
#include "tools/cabana/utils/util.h"
|
||||
#include "tools/replay/logreader.h"
|
||||
#include "tools/cabana/streams/replaystream.h"
|
||||
|
||||
class Slider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
Slider(QWidget *parent);
|
||||
double currentSecond() const { return value() / factor; }
|
||||
void setCurrentSecond(double sec) { setValue(sec * factor); }
|
||||
void setTimeRange(double min, double max) { setRange(min * factor, max * factor); }
|
||||
void mousePressEvent(QMouseEvent *e) override;
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
const double factor = 1000.0;
|
||||
double thumbnail_dispaly_time = -1;
|
||||
};
|
||||
|
||||
class StreamCameraView : public CameraWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
StreamCameraView(std::string stream_name, VisionStreamType stream_type, QWidget *parent = nullptr);
|
||||
void paintGL() override;
|
||||
void showPausedOverlay() { fade_animation->start(); }
|
||||
void parseQLog(std::shared_ptr<LogReader> qlog);
|
||||
|
||||
private:
|
||||
QPixmap generateThumbnail(QPixmap thumbnail, double seconds);
|
||||
void drawAlert(QPainter &p, const QRect &rect, const Timeline::Entry &alert);
|
||||
void drawThumbnail(QPainter &p);
|
||||
void drawScrubThumbnail(QPainter &p);
|
||||
void drawTime(QPainter &p, const QRect &rect, double seconds);
|
||||
|
||||
QPropertyAnimation *fade_animation;
|
||||
QMap<uint64_t, QPixmap> big_thumbnails;
|
||||
QMap<uint64_t, QPixmap> thumbnails;
|
||||
double thumbnail_dispaly_time = -1;
|
||||
friend class VideoWidget;
|
||||
};
|
||||
|
||||
class VideoWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
VideoWidget(QWidget *parnet = nullptr);
|
||||
void showThumbnail(double seconds);
|
||||
|
||||
protected:
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
QString formatTime(double sec, bool include_milliseconds = false);
|
||||
void timeRangeChanged();
|
||||
void updateState();
|
||||
void updatePlayBtnState();
|
||||
QWidget *createCameraWidget();
|
||||
void createPlaybackController();
|
||||
void createSpeedDropdown(QToolBar *toolbar);
|
||||
void loopPlaybackClicked();
|
||||
void vipcAvailableStreamsUpdated(std::set<VisionStreamType> streams);
|
||||
void showRouteInfo();
|
||||
|
||||
StreamCameraView *cam_widget;
|
||||
QAction *time_display_action = nullptr;
|
||||
QAction *play_toggle_action = nullptr;
|
||||
QToolButton *speed_btn = nullptr;
|
||||
QAction *skip_to_end_action = nullptr;
|
||||
Slider *slider = nullptr;
|
||||
QTabBar *camera_tab = nullptr;
|
||||
};
|
||||
@@ -0,0 +1,66 @@
|
||||
# Camera stream
|
||||
|
||||
`compressed_vipc.py` connects to a remote device running openpilot, decodes the video streams, and republishes them over VisionIPC.
|
||||
|
||||
## Usage
|
||||
|
||||
### On the device
|
||||
SSH into the device and run following in separate terminals:
|
||||
|
||||
`cd /data/openpilot/cereal/messaging && ./bridge`
|
||||
|
||||
`cd /data/openpilot/system/loggerd && ./encoderd`
|
||||
|
||||
`cd /data/openpilot/system/camerad && ./camerad`
|
||||
|
||||
Note that both the device and your PC must be on the same openpilot commit.
|
||||
|
||||
Alternatively paste this as a single command:
|
||||
```
|
||||
(
|
||||
cd /data/openpilot/cereal/messaging/
|
||||
./bridge &
|
||||
|
||||
cd /data/openpilot/system/camerad/
|
||||
./camerad &
|
||||
|
||||
cd /data/openpilot/system/loggerd/
|
||||
./encoderd &
|
||||
|
||||
wait
|
||||
) ; trap 'kill $(jobs -p)' SIGINT
|
||||
```
|
||||
Ctrl+C will stop all three processes.
|
||||
|
||||
### On the PC
|
||||
Decode the stream with `compressed_vipc.py`:
|
||||
|
||||
```cd ~/openpilot/tools/camerastream && ./compressed_vipc.py <ip>```
|
||||
|
||||
To actually display the stream, run `watch3` in separate terminal:
|
||||
|
||||
```cd ~/openpilot/selfdrive/ui/ && ./watch3.py```
|
||||
|
||||
## compressed_vipc.py usage
|
||||
```
|
||||
$ python3 compressed_vipc.py -h
|
||||
usage: compressed_vipc.py [-h] [--nvidia] [--cams CAMS] [--silent] addr
|
||||
|
||||
Decode video streams and broadcast on VisionIPC
|
||||
|
||||
positional arguments:
|
||||
addr Address of comma three
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--nvidia Use nvidia instead of ffmpeg
|
||||
--cams CAMS Cameras to decode
|
||||
--silent Suppress debug output
|
||||
```
|
||||
|
||||
|
||||
## Example:
|
||||
```
|
||||
cd ~/openpilot/tools/camerastream && ./compressed_vipc.py comma-ffffffff --cams 0
|
||||
cd ~/openpilot/selfdrive/ui/ && ./watch3.py
|
||||
```
|
||||
Executable
+163
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env python3
|
||||
import av
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import numpy as np
|
||||
import multiprocessing
|
||||
import time
|
||||
import signal
|
||||
|
||||
|
||||
import cereal.messaging as messaging
|
||||
from msgq.visionipc import VisionIpcServer, VisionStreamType
|
||||
|
||||
V4L2_BUF_FLAG_KEYFRAME = 8
|
||||
|
||||
# start encoderd
|
||||
# also start cereal messaging bridge
|
||||
# then run this "./compressed_vipc.py <ip>"
|
||||
|
||||
ENCODE_SOCKETS = {
|
||||
VisionStreamType.VISION_STREAM_ROAD: "roadEncodeData",
|
||||
VisionStreamType.VISION_STREAM_DRIVER: "driverEncodeData",
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD: "wideRoadEncodeData",
|
||||
}
|
||||
|
||||
def decoder(addr, vipc_server, vst, nvidia, W, H, debug=False):
|
||||
sock_name = ENCODE_SOCKETS[vst]
|
||||
if debug:
|
||||
print(f"start decoder for {sock_name}, {W}x{H}")
|
||||
|
||||
if nvidia:
|
||||
os.environ["NV_LOW_LATENCY"] = "3" # both bLowLatency and CUVID_PKT_ENDOFPICTURE
|
||||
sys.path += os.environ["LD_LIBRARY_PATH"].split(":")
|
||||
import PyNvCodec as nvc
|
||||
|
||||
nvDec = nvc.PyNvDecoder(W, H, nvc.PixelFormat.NV12, nvc.CudaVideoCodec.HEVC, 0)
|
||||
cc1 = nvc.ColorspaceConversionContext(nvc.ColorSpace.BT_709, nvc.ColorRange.JPEG)
|
||||
conv_yuv = nvc.PySurfaceConverter(W, H, nvc.PixelFormat.NV12, nvc.PixelFormat.YUV420, 0)
|
||||
nvDwn_yuv = nvc.PySurfaceDownloader(W, H, nvc.PixelFormat.YUV420, 0)
|
||||
img_yuv = np.ndarray((H*W//2*3), dtype=np.uint8)
|
||||
else:
|
||||
codec = av.CodecContext.create("hevc", "r")
|
||||
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
sock = messaging.sub_sock(sock_name, None, addr=addr, conflate=False)
|
||||
cnt = 0
|
||||
last_idx = -1
|
||||
seen_iframe = False
|
||||
|
||||
time_q = []
|
||||
while 1:
|
||||
msgs = messaging.drain_sock(sock, wait_for_one=True)
|
||||
for evt in msgs:
|
||||
evta = getattr(evt, evt.which())
|
||||
if debug and evta.idx.encodeId != 0 and evta.idx.encodeId != (last_idx+1):
|
||||
print("DROP PACKET!")
|
||||
last_idx = evta.idx.encodeId
|
||||
if not seen_iframe and not (evta.idx.flags & V4L2_BUF_FLAG_KEYFRAME):
|
||||
if debug:
|
||||
print("waiting for iframe")
|
||||
continue
|
||||
time_q.append(time.monotonic())
|
||||
network_latency = (int(time.time()*1e9) - evta.unixTimestampNanos)/1e6 # noqa: TID251
|
||||
frame_latency = ((evta.idx.timestampEof/1e9) - (evta.idx.timestampSof/1e9))*1000
|
||||
process_latency = ((evt.logMonoTime/1e9) - (evta.idx.timestampEof/1e9))*1000
|
||||
|
||||
# put in header (first)
|
||||
if not seen_iframe:
|
||||
if nvidia:
|
||||
nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.header, dtype=np.uint8))
|
||||
else:
|
||||
codec.decode(av.packet.Packet(evta.header))
|
||||
seen_iframe = True
|
||||
|
||||
if nvidia:
|
||||
rawSurface = nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.data, dtype=np.uint8))
|
||||
if rawSurface.Empty():
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
convSurface = conv_yuv.Execute(rawSurface, cc1)
|
||||
nvDwn_yuv.DownloadSingleSurface(convSurface, img_yuv)
|
||||
else:
|
||||
frames = codec.decode(av.packet.Packet(evta.data))
|
||||
if len(frames) == 0:
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
assert len(frames) == 1
|
||||
img_yuv = frames[0].to_ndarray(format=av.video.format.VideoFormat('yuv420p')).flatten()
|
||||
uv_offset = H*W
|
||||
y = img_yuv[:uv_offset]
|
||||
uv = img_yuv[uv_offset:].reshape(2, -1).ravel('F')
|
||||
img_yuv = np.hstack((y, uv))
|
||||
|
||||
vipc_server.send(vst, img_yuv.data, cnt, int(time_q[0]*1e9), int(time.monotonic()*1e9))
|
||||
cnt += 1
|
||||
|
||||
pc_latency = (time.monotonic()-time_q[0])*1000
|
||||
time_q = time_q[1:]
|
||||
if debug:
|
||||
print(f"{len(msgs):2d} {evta.idx.encodeId:4d} {evt.logMonoTime/1e9:.3f} {evta.idx.timestampEof/1e6:.3f} \
|
||||
roll {frame_latency:6.2f} ms latency {process_latency:6.2f} ms + {network_latency:6.2f} ms + {pc_latency:6.2f} ms \
|
||||
= {process_latency+network_latency+pc_latency:6.2f} ms", len(evta.data), sock_name)
|
||||
|
||||
|
||||
class CompressedVipc:
|
||||
def __init__(self, addr, vision_streams, server_name, nvidia=False, debug=False):
|
||||
print("getting frame sizes")
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.reset_context()
|
||||
sm = messaging.SubMaster([ENCODE_SOCKETS[s] for s in vision_streams], addr=addr)
|
||||
while min(sm.recv_frame.values()) == 0:
|
||||
sm.update(100)
|
||||
os.environ.pop("ZMQ")
|
||||
messaging.reset_context()
|
||||
|
||||
self.vipc_server = VisionIpcServer(server_name)
|
||||
for vst in vision_streams:
|
||||
ed = sm[ENCODE_SOCKETS[vst]]
|
||||
self.vipc_server.create_buffers(vst, 4, ed.width, ed.height)
|
||||
self.vipc_server.start_listener()
|
||||
|
||||
self.procs = []
|
||||
for vst in vision_streams:
|
||||
ed = sm[ENCODE_SOCKETS[vst]]
|
||||
p = multiprocessing.Process(target=decoder, args=(addr, self.vipc_server, vst, nvidia, ed.width, ed.height, debug))
|
||||
p.start()
|
||||
self.procs.append(p)
|
||||
|
||||
def join(self):
|
||||
for p in self.procs:
|
||||
p.join()
|
||||
|
||||
def kill(self):
|
||||
for p in self.procs:
|
||||
p.terminate()
|
||||
self.join()
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Decode video streams and broadcast on VisionIPC")
|
||||
parser.add_argument("addr", help="Address of comma three")
|
||||
parser.add_argument("--nvidia", action="store_true", help="Use nvidia instead of ffmpeg")
|
||||
parser.add_argument("--cams", default="0,1,2", help="Cameras to decode")
|
||||
parser.add_argument("--server", default="camerad", help="choose vipc server name")
|
||||
parser.add_argument("--silent", action="store_true", help="Suppress debug output")
|
||||
args = parser.parse_args()
|
||||
|
||||
vision_streams = [
|
||||
VisionStreamType.VISION_STREAM_ROAD,
|
||||
VisionStreamType.VISION_STREAM_DRIVER,
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD,
|
||||
]
|
||||
|
||||
vsts = [vision_streams[int(x)] for x in args.cams.split(",")]
|
||||
cvipc = CompressedVipc(args.addr, vsts, args.server, args.nvidia, debug=(not args.silent))
|
||||
|
||||
# register exit handler
|
||||
signal.signal(signal.SIGINT, lambda sig, frame: cvipc.kill())
|
||||
|
||||
cvipc.join()
|
||||
@@ -0,0 +1,129 @@
|
||||
# tools/car_porting
|
||||
|
||||
Check out [this blog post](https://blog.comma.ai/how-to-write-a-car-port-for-openpilot/) for a high-level overview of porting a car.
|
||||
|
||||
## Useful car porting utilities
|
||||
|
||||
Testing car ports in your car is very time-consuming. Check out these utilities to do basic checks on your work before running it in your car.
|
||||
|
||||
### [Cabana](/tools/cabana/README.md)
|
||||
|
||||
View your car's CAN signals through DBC files, which openpilot uses to parse and create messages that talk to the car.
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> tools/cabana/cabana '1bbe6bf2d62f58a8|2022-07-14--17-11-43'
|
||||
```
|
||||
|
||||
### [tools/car_porting/auto_fingerprint.py](/tools/car_porting/auto_fingerprint.py)
|
||||
|
||||
Given a route and platform, automatically inserts FW fingerprints from the platform into the correct place in fingerprints.py
|
||||
|
||||
Example:
|
||||
```bash
|
||||
> python3 tools/car_porting/auto_fingerprint.py '1bbe6bf2d62f58a8|2022-07-14--17-11-43' 'OUTBACK'
|
||||
Attempting to add fw version for: OUTBACK
|
||||
```
|
||||
|
||||
### [selfdrive/car/tests/test_car_interfaces.py](/selfdrive/car/tests/test_car_interfaces.py)
|
||||
|
||||
Finds common bugs for car interfaces, without even requiring a route.
|
||||
|
||||
|
||||
#### Example: Typo in signal name
|
||||
```bash
|
||||
> pytest selfdrive/car/tests/test_car_interfaces.py -k subaru # replace with the brand you are working on
|
||||
|
||||
=====================================================================
|
||||
FAILED selfdrive/car/tests/test_car_interfaces.py::TestCarInterfaces::test_car_interfaces_165_SUBARU_LEGACY_7TH_GEN - KeyError: 'CruiseControlOOPS'
|
||||
|
||||
```
|
||||
|
||||
### [tools/car_porting/test_car_model.py](/tools/car_porting/test_car_model.py)
|
||||
|
||||
Given a route, runs most of the car interface to check for common errors like missing signals, blocked panda messages, and safety mismatches.
|
||||
|
||||
#### Example: panda safety mismatch for gasPressed
|
||||
```bash
|
||||
> python3 tools/car_porting/test_car_model.py '4822a427b188122a|2023-08-14--16-22-21'
|
||||
|
||||
=====================================================================
|
||||
FAIL: test_panda_safety_carstate (__main__.CarModelTestCase.test_panda_safety_carstate)
|
||||
Assert that panda safety matches openpilot's carState
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/home/batman/xx/openpilot/openpilot/selfdrive/car/tests/test_models.py", line 380, in test_panda_safety_carstate
|
||||
self.assertFalse(len(failed_checks), f"panda safety doesn't agree with openpilot: {failed_checks}")
|
||||
AssertionError: 1 is not false : panda safety doesn't agree with openpilot: {'gasPressed': 116}
|
||||
```
|
||||
|
||||
## Jupyter notebooks
|
||||
|
||||
To use these notebooks, install Jupyter within your [openpilot virtual environment](/tools/README.md).
|
||||
|
||||
```bash
|
||||
uv pip install jupyter ipykernel
|
||||
```
|
||||
|
||||
Launching:
|
||||
|
||||
```bash
|
||||
jupyter notebook
|
||||
```
|
||||
|
||||
### [examples/subaru_steer_temp_fault.ipynb](/tools/car_porting/examples/subaru_steer_temp_fault.ipynb)
|
||||
|
||||
An example of searching through a database of segments for a specific condition, and plotting the results.
|
||||
|
||||
|
||||
|
||||
*a plot of the steer_warning vs steering angle, where we can see it is clearly caused by a large steering angle change*
|
||||
|
||||
### [examples/subaru_long_accel.ipynb](/tools/car_porting/examples/subaru_long_accel.ipynb)
|
||||
|
||||
An example of plotting the response of an actuator when it is active.
|
||||
|
||||
|
||||
|
||||
*a plot of the brake_pressure vs acceleration, where we can see it is a fairly linear response.*
|
||||
|
||||
### [examples/ford_vin_fingerprint.ipynb](/tools/car_porting/examples/ford_vin_fingerprint.ipynb)
|
||||
|
||||
In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.
|
||||
|
||||
```
|
||||
vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 00000000000XXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8F98NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True
|
||||
vin: 1FM5K7LC0MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False
|
||||
vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True
|
||||
vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 1FM5K8GC7NGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 5LM5J7XC8MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E31PRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False
|
||||
```
|
||||
|
||||
### [examples/find_segments_with_message.ipynb](/tools/car_porting/examples/find_segments_with_message.ipynb)
|
||||
|
||||
Searches for segments where a set of given CAN message IDs are present. In the example, we search for all messages
|
||||
used for CAN-based ignition detection.
|
||||
|
||||
```
|
||||
Match found: 46b21f1c5f7aa885/2024-01-23--15-19-34/20/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: a63a23c3e628f288/2023-11-05--18-36-20/8/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']
|
||||
Match found: ce31b7a998781ba8/2024-01-19--07-05-29/23/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2023-12-25--19-31-00/4/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: e1dfba62a4e33f7b/2024-01-10--14-33-57/2/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']
|
||||
Match found: ae679616266f4096/2023-12-05--15-43-46/4/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2023-11-18--17-49-42/3/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: ae679616266f4096/2024-01-03--21-57-09/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-10--11-53-15/17/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-12-03--17-31-17/29/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
Match found: 6dae2984cc53cd7f/2023-11-27--23-29-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']
|
||||
```
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
from opendbc.car.debug.format_fingerprints import format_brand_fw_versions
|
||||
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
from opendbc.car.fw_versions import MODEL_TO_BRAND, match_fw_to_car
|
||||
from openpilot.tools.lib.logreader import LogReader, ReadMode
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Auto fingerprint from a route")
|
||||
parser.add_argument("route", help="The route name to use")
|
||||
parser.add_argument("platform", help="The platform, or leave empty to auto-determine using fuzzy", default=None, nargs="?")
|
||||
args = parser.parse_args()
|
||||
|
||||
lr = LogReader(args.route, ReadMode.QLOG)
|
||||
CP = lr.first("carParams")
|
||||
assert CP is not None, "No carParams in route"
|
||||
|
||||
carPlatform = MIGRATION.get(CP.carFingerprint, CP.carFingerprint)
|
||||
|
||||
if args.platform is not None:
|
||||
platform = args.platform
|
||||
elif carPlatform != "MOCK":
|
||||
platform = carPlatform
|
||||
else:
|
||||
_, matches = match_fw_to_car(CP.carFw, CP.carVin, log=False)
|
||||
assert len(matches) == 1, f"Unable to auto-determine platform, matches: {matches}"
|
||||
platform = list(matches)[0]
|
||||
|
||||
print("Attempting to add fw version for:", platform)
|
||||
|
||||
fw_versions: dict[str, dict[tuple, list[bytes]]] = defaultdict(lambda: defaultdict(list))
|
||||
brand = MODEL_TO_BRAND[platform]
|
||||
|
||||
for fw in CP.carFw:
|
||||
if fw.brand == brand and not fw.logging:
|
||||
addr = fw.address
|
||||
subAddr = None if fw.subAddress == 0 else fw.subAddress
|
||||
key = (fw.ecu.raw, addr, subAddr)
|
||||
|
||||
fw_versions[platform][key].append(fw.fwVersion)
|
||||
|
||||
format_brand_fw_versions(brand, fw_versions)
|
||||
@@ -0,0 +1,232 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 85,
|
||||
"id": "facb8edc-9924-491a-a4dd-fe6135b0c6c4",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Import all cars from opendbc\n",
|
||||
"\n",
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.values import PLATFORMS as TEST_PLATFORMS\n",
|
||||
"\n",
|
||||
"# Example: add additional platforms/segments to test outside of commaCarSegments\n",
|
||||
"\n",
|
||||
"EXTRA_SEGMENTS = {\n",
|
||||
" # \"81dd9e9fe256c397/0000001f--97c42cf98d\", # Volkswagen ID.4 test route, new car port, not in public dataset\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 86,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Searching 221 platforms\n",
|
||||
"No segments available for DODGE_DURANGO\n",
|
||||
"No segments available for FORD_RANGER_MK2\n",
|
||||
"No segments available for HOLDEN_ASTRA\n",
|
||||
"No segments available for CADILLAC_ATS\n",
|
||||
"No segments available for CHEVROLET_MALIBU\n",
|
||||
"No segments available for CADILLAC_XT4\n",
|
||||
"No segments available for CHEVROLET_VOLT_2019\n",
|
||||
"No segments available for CHEVROLET_TRAVERSE\n",
|
||||
"No segments available for GMC_YUKON\n",
|
||||
"No segments available for HONDA_ODYSSEY_CHN\n",
|
||||
"No segments available for HYUNDAI_KONA_2022\n",
|
||||
"No segments available for HYUNDAI_NEXO_1ST_GEN\n",
|
||||
"No segments available for GENESIS_GV70_ELECTRIFIED_1ST_GEN\n",
|
||||
"No segments available for GENESIS_G80_2ND_GEN_FL\n",
|
||||
"No segments available for RIVIAN_R1_GEN1\n",
|
||||
"No segments available for SUBARU_FORESTER_HYBRID\n",
|
||||
"No segments available for TESLA_MODEL_3\n",
|
||||
"No segments available for TESLA_MODEL_Y\n",
|
||||
"No segments available for TOYOTA_RAV4_PRIME\n",
|
||||
"No segments available for TOYOTA_SIENNA_4TH_GEN\n",
|
||||
"No segments available for LEXUS_LC_TSS2\n",
|
||||
"No segments available for VOLKSWAGEN_CADDY_MK3\n",
|
||||
"No segments available for VOLKSWAGEN_CRAFTER_MK2\n",
|
||||
"No segments available for VOLKSWAGEN_JETTA_MK6\n",
|
||||
"Searching 577 segments\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 3 # Increase this to search more segments\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_PLATFORMS)} platforms\")\n",
|
||||
"\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"No segments available for {platform}\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"TEST_SEGMENTS.extend(EXTRA_SEGMENTS)\n",
|
||||
"\n",
|
||||
"print(f\"Searching {len(TEST_SEGMENTS)} segments\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "27a243c33de44498b2b946190df44b23",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"segments searched: 0%| | 0/577 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Match found: 0f53b336851e1384/2023-11-20--09-44-03/12/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7620ad20d3cefc64/2023-10-28--08-14-40/3/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 00d247a9bb1f9196/2023-11-06--13-33-17/9/s CHRYSLER PACIFICA HYBRID 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 120a432f63cb0de2/2023-10-30--20-01-34/1/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: b70b56b76a6217f2/2023-12-19--08-30-22/35/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 97e388680a6716ed/2024-01-17--10-15-13/9/s CHRYSLER PACIFICA HYBRID 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2137b01aa0ca63f9/2024-01-06--22-06-14/70/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 8fc6a1b72c8b1357/2023-11-06--07-50-05/8/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 7e705eb5c27a49cc/2024-01-18--16-51-20/3/s CHRYSLER PACIFICA 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2024-01-20--14-46-24/12/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 12208e5acdc97eb3/2023-11-30--12-01-09/2/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9cad19e0efce3650/2024-01-26--10-24-52/27/s CHRYSLER PACIFICA 2020 ['VW CAN Ign']\n",
|
||||
"Match found: 9db428338427dec2/2023-11-05--18-40-09/21/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: d50ada8ee55a5e74/2023-12-11--13-38-09/0/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 900dfa83b4addfe6/2023-12-30--19-20-08/28/s JEEP GRAND CHEROKEE V6 2018 ['VW CAN Ign']\n",
|
||||
"Match found: 20acda0eb23d7f23/2024-01-19--17-33-26/41/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 1cc3b46843cad2ca/2024-01-10--20-20-54/24/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: 2d9b6425552c52c1/2023-12-07--10-31-46/22/s JEEP GRAND CHEROKEE 2019 ['VW CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-04--13-13-56/16/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2024-01-08--07-58-12/65/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: ae679616266f4096/2023-12-05--15-43-46/25/s RAM HD 5TH GEN ['Tesla 3/Y CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-09--21-41-11/4/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 440a155809ba2b6d/2023-12-30--08-51-53/2/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 6dae2984cc53cd7f/2024-01-06--10-11-07/1/s FORD BRONCO SPORT 1ST GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/19/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-10--14-13-14/0/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: a4218e6416dfd978/2023-11-27--13-48-46/4/s FORD ESCAPE 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-12-10--19-02-33/3/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 0b91b433b9332780/2023-12-28--14-02-49/4/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: 8a732841c3a8d5ef/2023-11-09--07-28-12/1/s FORD EXPLORER 6TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/59/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/82/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n",
|
||||
"Match found: e886087f430e7fe7/2023-11-05--19-59-40/106/s FORD FOCUS 4TH GEN ['Rivian CAN Ign']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from openpilot.tools.lib.logreader import LogReader, comma_car_segments_source\n",
|
||||
"from tqdm.notebook import tqdm, tnrange\n",
|
||||
"\n",
|
||||
"# Example search for CAN ignition messages\n",
|
||||
"# Be careful when filtering by bus, account for odd harness arrangements on Honda/HKG\n",
|
||||
"\n",
|
||||
"BUSES_TO_SEARCH = [0, 1, 2]\n",
|
||||
"\n",
|
||||
"# Support for external Red Panda\n",
|
||||
"EXTERNAL_PANDA_BUSES = [bus + 4 for bus in BUSES_TO_SEARCH]\n",
|
||||
"\n",
|
||||
"MESSAGES_TO_FIND = {\n",
|
||||
" 0x1F1: \"GM CAN Ign\",\n",
|
||||
" 0x152: \"Rivian CAN Ign\",\n",
|
||||
" 0x221: \"Tesla 3/Y CAN Ign\",\n",
|
||||
" 0x9E: \"Mazda CAN Ign\",\n",
|
||||
" 0x3C0: \"VW CAN Ign\",\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"progress_bar = tnrange(len(TEST_SEGMENTS), desc=\"segments searched\")\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" progress_bar.update()\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_packets = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" matched_messages = set()\n",
|
||||
"\n",
|
||||
" for packet in can_packets:\n",
|
||||
" for msg in packet.can:\n",
|
||||
" if msg.address in MESSAGES_TO_FIND and msg.src in (BUSES_TO_SEARCH + EXTERNAL_PANDA_BUSES):\n",
|
||||
" # print(msg)\n",
|
||||
" matched_messages.add(msg.address)\n",
|
||||
"\n",
|
||||
" if len(matched_messages) > 0:\n",
|
||||
" message_names = [MESSAGES_TO_FIND[message] for message in matched_messages]\n",
|
||||
" print(f\"Match found: {segment:<45} {CP.carFingerprint:<38} {message_names}\")\n",
|
||||
"\n",
|
||||
" progress_bar.update()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.0"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {
|
||||
"jupyter": {
|
||||
"is_executing": true
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Got 9 Ford cars from opendbc\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"\"\"\"In this example, we use the public comma car segments database to check if vin fingerprinting is feasible for ford.\"\"\"\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader, comma_car_segments_source\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.ford.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"\n",
|
||||
"platforms = [c.value for c in CAR]\n",
|
||||
"print(f\"Got {len(platforms)} Ford cars from opendbc\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Adapted from https://github.com/commaai/openpilot/issues/31052#issuecomment-1902690083\n",
|
||||
"\n",
|
||||
"MODEL_YEAR_CODES = {'M': 2021, 'N': 2022, 'P': 2023, 'R': 2024, 'S': 2025}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"F150_CODES = ['F1C', 'F1E', 'W1C', 'W1E', 'X1C', 'X1E', 'W1R', 'W1P', 'W1S', 'W1T']\n",
|
||||
"LIGHTNING_CODES = ['L', 'V']\n",
|
||||
"MACHE_CODES = ['K1R', 'K1S', 'K2S', 'K3R', 'K3S', 'K4S']\n",
|
||||
"\n",
|
||||
"FORD_VIN_START = ['1FT', '3FM', '5LM']\n",
|
||||
"\n",
|
||||
"def ford_vin_fingerprint(vin): # Check if it's a Ford vehicle and determine the model\n",
|
||||
" vin_positions_567 = vin[4:7]\n",
|
||||
"\n",
|
||||
" if vin.startswith('1FT'):\n",
|
||||
" if vin_positions_567 in F150_CODES:\n",
|
||||
" if vin[7] in LIGHTNING_CODES:\n",
|
||||
" return f\"FORD F-150 LIGHTNING 1ST GEN\"\n",
|
||||
" else:\n",
|
||||
" return f\"FORD F-150 14TH GEN\"\n",
|
||||
" elif vin.startswith('3FM'):\n",
|
||||
" if vin_positions_567 in MACHE_CODES:\n",
|
||||
" return f\"FORD MUSTANG MACH-E 1ST GEN\"\n",
|
||||
" elif vin.startswith('5LM'):\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" return \"mock\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 287 segments for platform FORD_BRONCO_SPORT_MK1, sampling 5 segments\n",
|
||||
"Got 137 segments for platform FORD_ESCAPE_MK4, sampling 5 segments\n",
|
||||
"Got 1041 segments for platform FORD_EXPLORER_MK6, sampling 5 segments\n",
|
||||
"Got 5 segments for platform FORD_F_150_MK14, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_F_150_LIGHTNING_MK1, sampling 3 segments\n",
|
||||
"Got 56 segments for platform FORD_FOCUS_MK4, sampling 5 segments\n",
|
||||
"Got 637 segments for platform FORD_MAVERICK_MK1, sampling 5 segments\n",
|
||||
"Got 3 segments for platform FORD_MUSTANG_MACH_E_MK1, sampling 3 segments\n",
|
||||
"Skipping platform: FORD_RANGER_MK2, no data available\n",
|
||||
"Segment collection finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5\n",
|
||||
"\n",
|
||||
"VINS_TO_CHECK = set()\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in platforms:\n",
|
||||
" if platform not in database:\n",
|
||||
" print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" segments = random.sample(all_segments, NUM_SEGMENTS)\n",
|
||||
"\n",
|
||||
" for segment in segments:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if \"FORD\" not in CP.carFingerprint:\n",
|
||||
" print(segment, CP.carFingerprint)\n",
|
||||
" VINS_TO_CHECK.add((CP.carVin, CP.carFingerprint))\n",
|
||||
"\n",
|
||||
"print(\"Segment collection finished\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"vin: 3FMCR9B69NRXXXXXX real platform: FORD BRONCO SPORT 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 00000000000XXXXXX real platform: FORD F-150 14TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FMCU9J94MUXXXXXX real platform: FORD ESCAPE 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FMTK3SU0MMXXXXXX real platform: FORD MUSTANG MACH-E 1ST GEN determined platform: FORD MUSTANG MACH-E 1ST GEN correct: True\n",
|
||||
"vin: 1FM5K8HC7MGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC9LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FTVW1EL4NWXXXXXX real platform: FORD F-150 LIGHTNING 1ST GEN determined platform: FORD F-150 LIGHTNING 1ST GEN correct: True\n",
|
||||
"vin: WF0NXXGCHNJXXXXXX real platform: FORD FOCUS 4TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E99NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 1FM5K8GC7LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E33NRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n",
|
||||
"vin: 5LM5J7XC1LGXXXXXX real platform: FORD EXPLORER 6TH GEN determined platform: mock correct: False\n",
|
||||
"vin: 3FTTW8E3XPRXXXXXX real platform: FORD MAVERICK 1ST GEN determined platform: mock correct: False\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"for vin, real_fingerprint in VINS_TO_CHECK:\n",
|
||||
" determined_fingerprint = ford_vin_fingerprint(vin)\n",
|
||||
" print(f\"vin: {vin} real platform: {real_fingerprint: <30} determined platform: {determined_fingerprint: <30} correct: {real_fingerprint == determined_fingerprint}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,277 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"id": "228a6736-de31-4255-9d72-a6ff391b968d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Found 6 qualifying vehicles:\n",
|
||||
" KIA_EV6\n",
|
||||
" HYUNDAI_KONA_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_5\n",
|
||||
" KIA_NIRO_EV_2ND_GEN\n",
|
||||
" HYUNDAI_IONIQ_6\n",
|
||||
" GENESIS_GV60_EV_1ST_GEN\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.hyundai.values import CAR, HyundaiFlags\n",
|
||||
"from opendbc.car.hyundai.fingerprints import FW_VERSIONS\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) & set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD electric vehicles only\n",
|
||||
"#TEST_PLATFORMS = set(CAR.with_flags(HyundaiFlags.CANFD)) - set(CAR.with_flags(HyundaiFlags.EV)) # CAN-FD hybrid and ICE vehicles only\n",
|
||||
"\n",
|
||||
"print(f\"Found {len(TEST_PLATFORMS)} qualifying vehicles:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" print(f\" {platform}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"id": "ed1c8aec-c274-4c61-b83d-711ea194bf86",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Collecting segments from commaCarSegments dataset:\n",
|
||||
"Got 1300 segments for platform KIA_EV6, sampling 5 segments\n",
|
||||
"Got 9 segments for platform HYUNDAI_KONA_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 1570 segments for platform HYUNDAI_IONIQ_5, sampling 5 segments\n",
|
||||
"Got 34 segments for platform KIA_NIRO_EV_2ND_GEN, sampling 5 segments\n",
|
||||
"Got 974 segments for platform HYUNDAI_IONIQ_6, sampling 5 segments\n",
|
||||
"Got 157 segments for platform GENESIS_GV60_EV_1ST_GEN, sampling 5 segments\n",
|
||||
"Collected 30 segments for analysis\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"from openpilot.tools.lib.comma_car_segments import get_comma_car_segments_database\n",
|
||||
"from opendbc.car.hyundai.values import CAR\n",
|
||||
"\n",
|
||||
"database = get_comma_car_segments_database()\n",
|
||||
"TEST_SEGMENTS = []\n",
|
||||
"\n",
|
||||
"MAX_SEGS_PER_PLATFORM = 5 # TODO: Increase this to search more segments\n",
|
||||
"\n",
|
||||
"print(\"Collecting segments from commaCarSegments dataset:\")\n",
|
||||
"for platform in TEST_PLATFORMS:\n",
|
||||
" assert(platform in database)\n",
|
||||
" #if platform not in database:\n",
|
||||
" # print(f\"Skipping platform: {platform}, no data available\")\n",
|
||||
" # continue\n",
|
||||
"\n",
|
||||
" all_segments = database[platform]\n",
|
||||
"\n",
|
||||
" NUM_SEGMENTS = min(len(all_segments), MAX_SEGS_PER_PLATFORM)\n",
|
||||
"\n",
|
||||
" print(f\"Got {len(all_segments)} segments for platform {platform}, sampling {NUM_SEGMENTS} segments\")\n",
|
||||
"\n",
|
||||
" TEST_SEGMENTS.extend(random.sample(all_segments, NUM_SEGMENTS))\n",
|
||||
"\n",
|
||||
"print(f\"Collected {len(TEST_SEGMENTS)} segments for analysis\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"id": "0c75e8f2-4f5f-4f89-b8db-5223a6534a9f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Analyzing segment ff2bd20623fcaeaa/2023-11-26--16-27-04/5/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 3f1a6480f940cf9a/2024-01-10--23-06-11/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b0a9998109ed0053/2023-12-15--11-10-18/12/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 6e14aa2ed85025df/2023-11-15--13-18-12/24/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment a43f21df3a1ca12d/2024-01-25--08-56-22/16/s for KIA EV6 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--13-49-24/11/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-11-26--12-31-18/17/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-12-05--11-51-44/3/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2023-08-27--09-32-14/13/s for HYUNDAI KONA 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 1618132d68afc876/2024-01-25--15-07-04/24/s for HYUNDAI KONA ELECTRIC 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 223780ed74116bc2/2023-11-16--09-44-56/15/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment ba9951252624f37d/2024-01-20--22-33-23/118/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 8379b28e51ceb3b1/2023-11-09--23-21-58/92/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26fac43e27cd6091/2023-11-06--12-23-21/9/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=1.0\n",
|
||||
" ACCELERATOR gear=0.0\n",
|
||||
"Analyzing segment 5edb897a0ec7a477/2024-01-13--20-41-36/101/s for HYUNDAI IONIQ 5 2022\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 66cf8ea23b7c2789/2023-12-04--13-48-53/5/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-10--20-31-37/2/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-12-03--21-08-30/14/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-11-07--19-52-23/0/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment b153671049a867b3/2023-07-12--19-25-18/6/s for KIA NIRO EV 2ND GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-18--07-59-26/11/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 0ad7facc77922c3e/2023-12-21--17-47-25/18/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 26968f888e7330d3/2024-01-02--11-18-37/8/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 9ea4578ee2b1abcb/2023-11-27--21-03-24/33/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment df7fdd56970d90fe/2024-01-07--01-04-39/26/s for HYUNDAI IONIQ 6 2023\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--14-45-44/0/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2023-12-11--20-57-09/8/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--12-52-38/5/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-19--19-57-52/47/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analyzing segment 94542b2d06f7a9a6/2024-01-03--13-01-23/1/s for GENESIS GV60 ELECTRIC 1ST GEN\n",
|
||||
" GEAR_SHIFTER gear=4.0\n",
|
||||
" ACCELERATOR gear=5.0\n",
|
||||
"Analysis finished\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.hyundai.values import DBC\n",
|
||||
"from opendbc.car.hyundai.hyundaicanfd import CanBus\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader, comma_car_segments_source\n",
|
||||
"\n",
|
||||
"message_names = [\"GEAR_SHIFTER\", \"ACCELERATOR\", \"GEAR\", \"GEAR_ALT\", \"GEAR_ALT_2\"]\n",
|
||||
"\n",
|
||||
"for segment in TEST_SEGMENTS:\n",
|
||||
" lr = LogReader(segment, sources=[comma_car_segments_source])\n",
|
||||
" CP = lr.first(\"carParams\")\n",
|
||||
" if CP is None:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
" parser_messages = []\n",
|
||||
" for name in message_names:\n",
|
||||
" parser_messages.append((name, 0))\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], parser_messages, CanBus(CP).ECAN)\n",
|
||||
"\n",
|
||||
" parsed_message_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" parsed_message_history.append(copy.copy(cp.vl))\n",
|
||||
"\n",
|
||||
" print(f\"Analyzing segment {segment:<44} for {CP.carFingerprint}\")\n",
|
||||
" for name in message_names:\n",
|
||||
" if parsed_message_history[0][name][\"CHECKSUM\"] != 0: # Message is present for this segment\n",
|
||||
" gear_prev = parsed_message_history[0][name][\"GEAR\"]\n",
|
||||
" print(f\" {name:<15} gear={gear_prev}\")\n",
|
||||
" for i, parsed_messages in enumerate(parsed_message_history):\n",
|
||||
" gear = parsed_messages[name][\"GEAR\"]\n",
|
||||
" if gear != gear_prev:\n",
|
||||
" print(f\" *** Signal transition found! ***\")\n",
|
||||
" examples.append(i)\n",
|
||||
" gear_prev = gear\n",
|
||||
"\n",
|
||||
"print(f\"Analysis finished\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7724dd97-f62e-4fd3-9f64-63d49be669d2",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9f393e00-8efd-40fb-a41e-d312531a83e8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from opendbc.car import structs\n",
|
||||
"from opendbc.car.subaru.values import CAR, SubaruFlags\n",
|
||||
"from opendbc.car.subaru.fingerprints import FW_VERSIONS\n",
|
||||
"\n",
|
||||
"TEST_PLATFORMS = set(CAR) - CAR.with_flags(SubaruFlags.PREGLOBAL)\n",
|
||||
"\n",
|
||||
"Ecu = structs.CarParams.Ecu\n",
|
||||
"\n",
|
||||
"FW_BY_ECU = {platform: {ecu: versions for (ecu, addr, sub_addr), versions in fw_versions.items()} for platform, fw_versions in FW_VERSIONS.items()}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"PLATFORM_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 0: {\n",
|
||||
" b'\\xa5': [CAR.SUBARU_ASCENT, CAR.SUBARU_ASCENT_2023],\n",
|
||||
" b'\\xa2': [CAR.SUBARU_IMPREZA, CAR.SUBARU_IMPREZA_2020, CAR.SUBARU_CROSSTREK_HYBRID],\n",
|
||||
" b'\\xa1': [CAR.SUBARU_OUTBACK, CAR.SUBARU_LEGACY, CAR.SUBARU_OUTBACK_2023],\n",
|
||||
" b'\\xa3': [CAR.SUBARU_FORESTER, CAR.SUBARU_FORESTER_HYBRID, CAR.SUBARU_FORESTER_2022],\n",
|
||||
" b'z': [CAR.SUBARU_IMPREZA],\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"YEAR_CODES = {\n",
|
||||
" Ecu.abs: {\n",
|
||||
" 2: {\n",
|
||||
" b'\\x18': 2018,\n",
|
||||
" b'\\x19': 2019,\n",
|
||||
" b'\\x20': 2020,\n",
|
||||
" b'\\x21': 2021,\n",
|
||||
" b'\\x22': 2022,\n",
|
||||
" b'\\x23': 2023,\n",
|
||||
" }\n",
|
||||
" }\n",
|
||||
"}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_codes(platforms, codes):\n",
|
||||
" results = []\n",
|
||||
" for platform in platforms:\n",
|
||||
" for ecu in codes:\n",
|
||||
" for i in codes[ecu]:\n",
|
||||
" if isinstance(i, tuple):\n",
|
||||
" j = slice(i[0], i[1])\n",
|
||||
" else:\n",
|
||||
" j = slice(i, i+1)\n",
|
||||
" for version in FW_BY_ECU[platform][ecu]:\n",
|
||||
" code = version[j]\n",
|
||||
" if code not in codes[ecu][i]:\n",
|
||||
" print(f\"{platform} {code.hex()} not in {codes[ecu][i].keys()}\")\n",
|
||||
" else:\n",
|
||||
" results.append((platform, codes[ecu][i][code]))\n",
|
||||
" return results"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 08 not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 0c not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 2e not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"SUBARU_IMPREZA 3f not in dict_keys([b'\\x18', b'\\x19', b' ', b'!', b'\"', b'#'])\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER year=2018 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER year=2020 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK year=2022 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2019 years=[2020]\n",
|
||||
"correct_year=False platform=SUBARU_CROSSTREK_HYBRID year=2021 years=[2020]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2018 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA year=2019 years=[2017, 2018, 2019]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=False platform=SUBARU_FORESTER_2022 year=2021 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_FORESTER_2022 year=2022 years=[2022, 2023, 2024]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2019 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_ASCENT year=2021 years=[2019, 2020, 2021]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=True platform=SUBARU_OUTBACK_2023 year=2023 years=[2023]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=False platform=SUBARU_IMPREZA_2020 year=2019 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_IMPREZA_2020 year=2021 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n",
|
||||
"correct_year=True platform=SUBARU_LEGACY year=2020 years=[2020, 2021, 2022]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_year_code(platform, year):\n",
|
||||
" car_docs = CAR(platform).config.car_docs\n",
|
||||
" if isinstance(car_docs, list):\n",
|
||||
" car_docs = car_docs[0]\n",
|
||||
" years = [int(y) for y in car_docs.year_list]\n",
|
||||
" correct_year = year in years\n",
|
||||
" print(f\"{correct_year=!s: <6} {platform=: <32} {year=: <5} {years=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, YEAR_CODES)\n",
|
||||
"for platform, year in codes:\n",
|
||||
" test_year_code(platform, year)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_HYBRID platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_CROSSTREK_HYBRID platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT_2023 platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_FORESTER_2022 platforms=['SUBARU_FORESTER', 'SUBARU_FORESTER_HYBRID', 'SUBARU_FORESTER_2022']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_ASCENT platforms=['SUBARU_ASCENT', 'SUBARU_ASCENT_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_OUTBACK_2023 platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_IMPREZA_2020 platforms=['SUBARU_IMPREZA', 'SUBARU_IMPREZA_2020', 'SUBARU_CROSSTREK_HYBRID']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n",
|
||||
"in_possible_platforms=True platform=SUBARU_LEGACY platforms=['SUBARU_OUTBACK', 'SUBARU_LEGACY', 'SUBARU_OUTBACK_2023']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"def test_platform_code(platform, platforms):\n",
|
||||
" platforms = [str(p) for p in platforms]\n",
|
||||
" in_possible_platforms = platform in platforms\n",
|
||||
" print(f\"{in_possible_platforms=!s: <6} {platform=: <32} {platforms=}\")\n",
|
||||
"\n",
|
||||
"codes = get_codes(TEST_PLATFORMS, PLATFORM_CODES)\n",
|
||||
"for platform, possible_platforms in codes:\n",
|
||||
" test_platform_code(platform, possible_platforms)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"segments = [\n",
|
||||
" \"d9df6f87e8feff94|2023-03-28--17-41-10/1:12\"\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we plot the relationship between Cruise_Brake and Acceleration for stock eyesight.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"ES_Distance\", 20),\n",
|
||||
" (\"ES_Brake\", 20),\n",
|
||||
" (\"ES_Status\", 20),\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, 1)\n",
|
||||
"\n",
|
||||
" es_distance_history = []\n",
|
||||
" es_status_history = []\n",
|
||||
" es_brake_history = []\n",
|
||||
" acceleration_history = []\n",
|
||||
"\n",
|
||||
" last_acc = 0\n",
|
||||
"\n",
|
||||
" for msg in lr:\n",
|
||||
" if msg.which() == \"can\":\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" es_distance_history.append(copy.copy(cp.vl[\"ES_Distance\"]))\n",
|
||||
" es_brake_history.append(copy.copy(cp.vl[\"ES_Brake\"]))\n",
|
||||
" es_status_history.append(copy.copy(cp.vl[\"ES_Status\"]))\n",
|
||||
"\n",
|
||||
" acceleration_history.append(last_acc)\n",
|
||||
"\n",
|
||||
" if msg.which() == \"carState\":\n",
|
||||
" last_acc = msg.carState.aEgo"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def process(history, func):\n",
|
||||
" return np.array([func(h) for h in history])\n",
|
||||
"\n",
|
||||
"cruise_activated = process(es_status_history, lambda es_status: es_status[\"Cruise_Activated\"])\n",
|
||||
"cruise_throttle = process(es_distance_history, lambda es_distance: es_distance[\"Cruise_Throttle\"])\n",
|
||||
"cruise_rpm = process(es_status_history, lambda es_status: es_status[\"Cruise_RPM\"])\n",
|
||||
"cruise_brake = process(es_brake_history, lambda es_brake: es_brake[\"Brake_Pressure\"])\n",
|
||||
"acceleration = process(acceleration_history, lambda acc: acc)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"valid_brake = (cruise_activated==1) & (cruise_brake>0) # only when cruise is activated and eyesight is braking\n",
|
||||
"\n",
|
||||
"ax = plt.figure().add_subplot()\n",
|
||||
"\n",
|
||||
"ax.set_title(\"Brake_Pressure vs Acceleration\")\n",
|
||||
"ax.set_xlabel(\"Brake_Pessure\")\n",
|
||||
"ax.set_ylabel(\"Acceleration\")\n",
|
||||
"ax.scatter(cruise_brake[valid_brake], -acceleration[valid_brake])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# An example of searching through a database of segments for a specific condition, and plotting the results.\n",
|
||||
"\n",
|
||||
"segments = [\n",
|
||||
" \"c3d1ccb52f5f9d65|2023-07-22--01-23-20/6:10\",\n",
|
||||
"]\n",
|
||||
"platform = \"SUBARU_OUTBACK\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import copy\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"from opendbc.can.parser import CANParser\n",
|
||||
"from opendbc.car.subaru.values import CanBus, DBC\n",
|
||||
"\n",
|
||||
"from openpilot.selfdrive.pandad import can_capnp_to_list\n",
|
||||
"from openpilot.tools.lib.logreader import LogReader\n",
|
||||
"\n",
|
||||
"\"\"\"\n",
|
||||
"In this example, we search for positive transitions of Steer_Warning, which indicate that the EPS\n",
|
||||
"has stopped responding to our messages. This analysis would allow you to find the cause of these\n",
|
||||
"steer warnings and potentially work around them.\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"for segment in segments:\n",
|
||||
" lr = LogReader(segment)\n",
|
||||
"\n",
|
||||
" can_msgs = [msg for msg in lr if msg.which() == \"can\"]\n",
|
||||
"\n",
|
||||
" messages = [\n",
|
||||
" (\"Steering_Torque\", 50)\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" cp = CANParser(DBC[platform][\"pt\"], messages, CanBus.main)\n",
|
||||
"\n",
|
||||
" steering_torque_history = []\n",
|
||||
" examples = []\n",
|
||||
"\n",
|
||||
" for msg in can_msgs:\n",
|
||||
" cp.update_strings(can_capnp_to_list([msg.as_builder().to_bytes()]))\n",
|
||||
" steering_torque_history.append(copy.copy(cp.vl[\"Steering_Torque\"]))\n",
|
||||
"\n",
|
||||
" steer_warning_last = False\n",
|
||||
" for i, steering_torque_msg in enumerate(steering_torque_history):\n",
|
||||
" steer_warning = steering_torque_msg[\"Steer_Warning\"]\n",
|
||||
"\n",
|
||||
" steer_angle = steering_torque_msg[\"Steering_Angle\"]\n",
|
||||
"\n",
|
||||
" if steer_warning and not steer_warning_last: # positive transition of \"Steer_Warning\"\n",
|
||||
" examples.append(i)\n",
|
||||
"\n",
|
||||
" steer_warning_last = steer_warning\n",
|
||||
"\n",
|
||||
" FRAME_DELTA = 100 # plot this many frames around the positive transition\n",
|
||||
"\n",
|
||||
" for example in examples:\n",
|
||||
" fig, axs = plt.subplots(2)\n",
|
||||
"\n",
|
||||
" min_frame = int(example-FRAME_DELTA/2)\n",
|
||||
" max_frame = int(example+FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" steering_angle_history = [msg[\"Steering_Angle\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
" steering_warning_history = [msg[\"Steer_Warning\"] for msg in steering_torque_history[min_frame:max_frame]]\n",
|
||||
"\n",
|
||||
" xs = np.arange(-FRAME_DELTA/2, FRAME_DELTA/2)\n",
|
||||
"\n",
|
||||
" axs[0].plot(xs, steering_angle_history)\n",
|
||||
" axs[0].set_ylabel(\"Steering Angle (deg)\")\n",
|
||||
" axs[1].plot(xs, steering_warning_history)\n",
|
||||
" axs[1].set_ylabel(\"Steer Warning\")\n",
|
||||
"\n",
|
||||
" plt.show()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import sys
|
||||
import unittest # noqa: TID251
|
||||
|
||||
from opendbc.car.tests.routes import CarTestRoute
|
||||
from openpilot.selfdrive.car.tests.test_models import TestCarModel
|
||||
from openpilot.tools.lib.route import SegmentRange
|
||||
|
||||
|
||||
def create_test_models_suite(routes: list[CarTestRoute]) -> unittest.TestSuite:
|
||||
test_suite = unittest.TestSuite()
|
||||
for test_route in routes:
|
||||
# create new test case and discover tests
|
||||
test_case_args = {"platform": test_route.car_model, "test_route": test_route}
|
||||
CarModelTestCase = type("CarModelTestCase", (TestCarModel,), test_case_args)
|
||||
test_suite.addTest(unittest.TestLoader().loadTestsFromTestCase(CarModelTestCase))
|
||||
return test_suite
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Test any route against common issues with a new car port. " +
|
||||
"Uses selfdrive/car/tests/test_models.py")
|
||||
parser.add_argument("route_or_segment_name", help="Specify route to run tests on")
|
||||
parser.add_argument("--car", help="Specify car model for test route")
|
||||
args = parser.parse_args()
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
|
||||
sr = SegmentRange(args.route_or_segment_name)
|
||||
|
||||
test_routes = [CarTestRoute(sr.route_name, args.car, segment=seg_idx) for seg_idx in sr.seg_idxs]
|
||||
test_suite = create_test_models_suite(test_routes)
|
||||
|
||||
unittest.TextTestRunner().run(test_suite)
|
||||
Executable
+310
@@ -0,0 +1,310 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import logging
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from argparse import ArgumentParser, ArgumentTypeError
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
from random import randint
|
||||
from subprocess import Popen
|
||||
from typing import Literal
|
||||
|
||||
from cereal.messaging import SubMaster
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.common.params import Params, UnknownKeyName
|
||||
from openpilot.common.prefix import OpenpilotPrefix
|
||||
from openpilot.common.utils import managed_proc
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
DEFAULT_OUTPUT = 'output.mp4'
|
||||
DEMO_START = 90
|
||||
DEMO_END = 105
|
||||
DEMO_ROUTE = 'a2a0ccea32023010/2023-07-27--13-01-19'
|
||||
FRAMERATE = 20
|
||||
PIXEL_DEPTH = '24'
|
||||
RESOLUTION = '2160x1080'
|
||||
SECONDS_TO_WARM = 2
|
||||
PROC_WAIT_SECONDS = 30*10
|
||||
|
||||
OPENPILOT_FONT = str(Path(BASEDIR, 'selfdrive/assets/fonts/Inter-Regular.ttf').resolve())
|
||||
REPLAY = str(Path(BASEDIR, 'tools/replay/replay').resolve())
|
||||
UI = str(Path(BASEDIR, 'selfdrive/ui/ui').resolve())
|
||||
|
||||
logger = logging.getLogger('clip.py')
|
||||
|
||||
|
||||
def check_for_failure(procs: list[Popen]):
|
||||
for proc in procs:
|
||||
exit_code = proc.poll()
|
||||
if exit_code is not None and exit_code != 0:
|
||||
cmd = str(proc.args)
|
||||
if isinstance(proc.args, str):
|
||||
cmd = proc.args
|
||||
elif isinstance(proc.args, Sequence):
|
||||
cmd = str(proc.args[0])
|
||||
msg = f'{cmd} failed, exit code {exit_code}'
|
||||
logger.error(msg)
|
||||
stdout, stderr = proc.communicate()
|
||||
if stdout:
|
||||
logger.error(stdout.decode())
|
||||
if stderr:
|
||||
logger.error(stderr.decode())
|
||||
raise ChildProcessError(msg)
|
||||
|
||||
|
||||
def escape_ffmpeg_text(value: str):
|
||||
special_chars = {',': '\\,', ':': '\\:', '=': '\\=', '[': '\\[', ']': '\\]'}
|
||||
value = value.replace('\\', '\\\\\\\\\\\\\\\\')
|
||||
for char, escaped in special_chars.items():
|
||||
value = value.replace(char, escaped)
|
||||
return value
|
||||
|
||||
|
||||
def get_logreader(route: Route):
|
||||
return LogReader(route.qlog_paths()[0] if len(route.qlog_paths()) else route.name.canonical_name)
|
||||
|
||||
|
||||
def get_meta_text(lr: LogReader, route: Route):
|
||||
init_data = lr.first('initData')
|
||||
car_params = lr.first('carParams')
|
||||
origin_parts = init_data.gitRemote.split('/')
|
||||
origin = origin_parts[3] if len(origin_parts) > 3 else 'unknown'
|
||||
return ', '.join([
|
||||
f"openpilot v{init_data.version}",
|
||||
f"route: {route.name.canonical_name}",
|
||||
f"car: {car_params.carFingerprint}",
|
||||
f"origin: {origin}",
|
||||
f"branch: {init_data.gitBranch}",
|
||||
f"commit: {init_data.gitCommit[:7]}",
|
||||
f"modified: {str(init_data.dirty).lower()}",
|
||||
])
|
||||
|
||||
|
||||
def parse_args(parser: ArgumentParser):
|
||||
args = parser.parse_args()
|
||||
if args.demo:
|
||||
args.route = DEMO_ROUTE
|
||||
if args.start is None or args.end is None:
|
||||
args.start = DEMO_START
|
||||
args.end = DEMO_END
|
||||
elif args.route.count('/') == 1:
|
||||
if args.start is None or args.end is None:
|
||||
parser.error('must provide both start and end if timing is not in the route ID')
|
||||
elif args.route.count('/') == 3:
|
||||
if args.start is not None or args.end is not None:
|
||||
parser.error('don\'t provide timing when including it in the route ID')
|
||||
parts = args.route.split('/')
|
||||
args.route = '/'.join(parts[:2])
|
||||
args.start = int(parts[2])
|
||||
args.end = int(parts[3])
|
||||
if args.end <= args.start:
|
||||
parser.error(f'end ({args.end}) must be greater than start ({args.start})')
|
||||
if args.start < SECONDS_TO_WARM:
|
||||
parser.error(f'start must be greater than {SECONDS_TO_WARM}s to allow the UI time to warm up')
|
||||
|
||||
try:
|
||||
args.route = Route(args.route, data_dir=args.data_dir)
|
||||
except Exception as e:
|
||||
parser.error(f'failed to get route: {e}')
|
||||
|
||||
# FIXME: length isn't exactly max segment seconds, simplify to replay exiting at end of data
|
||||
length = round(args.route.max_seg_number * 60)
|
||||
if args.start >= length:
|
||||
parser.error(f'start ({args.start}s) cannot be after end of route ({length}s)')
|
||||
if args.end > length:
|
||||
parser.error(f'end ({args.end}s) cannot be after end of route ({length}s)')
|
||||
|
||||
return args
|
||||
|
||||
|
||||
def populate_car_params(lr: LogReader):
|
||||
init_data = lr.first('initData')
|
||||
assert init_data is not None
|
||||
|
||||
params = Params()
|
||||
entries = init_data.params.entries
|
||||
for cp in entries:
|
||||
key, value = cp.key, cp.value
|
||||
try:
|
||||
params.put(key, params.cpp2python(key, value))
|
||||
except UnknownKeyName:
|
||||
# forks of openpilot may have other Params keys configured. ignore these
|
||||
logger.warning(f"unknown Params key '{key}', skipping")
|
||||
logger.debug('persisted CarParams')
|
||||
|
||||
|
||||
def validate_env(parser: ArgumentParser):
|
||||
if platform.system() not in ['Linux']:
|
||||
parser.exit(1, f'clip.py: error: {platform.system()} is not a supported operating system\n')
|
||||
for proc in ['Xvfb', 'ffmpeg']:
|
||||
if shutil.which(proc) is None:
|
||||
parser.exit(1, f'clip.py: error: missing {proc} command, is it installed?\n')
|
||||
for proc in [REPLAY, UI]:
|
||||
if shutil.which(proc) is None:
|
||||
parser.exit(1, f'clip.py: error: missing {proc} command, did you build openpilot yet?\n')
|
||||
|
||||
|
||||
def validate_output_file(output_file: str):
|
||||
if not output_file.endswith('.mp4'):
|
||||
raise ArgumentTypeError('output must be an mp4')
|
||||
return output_file
|
||||
|
||||
|
||||
def validate_route(route: str):
|
||||
if route.count('/') not in (1, 3):
|
||||
raise ArgumentTypeError(f'route must include or exclude timing, example: {DEMO_ROUTE}')
|
||||
return route
|
||||
|
||||
|
||||
def validate_title(title: str):
|
||||
if len(title) > 80:
|
||||
raise ArgumentTypeError('title must be no longer than 80 chars')
|
||||
return title
|
||||
|
||||
|
||||
def wait_for_frames(procs: list[Popen]):
|
||||
sm = SubMaster(['uiDebug'])
|
||||
no_frames_drawn = True
|
||||
while no_frames_drawn:
|
||||
sm.update()
|
||||
no_frames_drawn = sm['uiDebug'].drawTimeMillis == 0.
|
||||
check_for_failure(procs)
|
||||
|
||||
|
||||
def clip(
|
||||
data_dir: str | None,
|
||||
quality: Literal['low', 'high'],
|
||||
prefix: str,
|
||||
route: Route,
|
||||
out: str,
|
||||
start: int,
|
||||
end: int,
|
||||
speed: int,
|
||||
target_mb: int,
|
||||
title: str | None,
|
||||
):
|
||||
logger.info(f'clipping route {route.name.canonical_name}, start={start} end={end} quality={quality} target_filesize={target_mb}MB')
|
||||
lr = get_logreader(route)
|
||||
|
||||
begin_at = max(start - SECONDS_TO_WARM, 0)
|
||||
duration = end - start
|
||||
bit_rate_kbps = int(round(target_mb * 8 * 1024 * 1024 / duration / 1000))
|
||||
|
||||
# TODO: evaluate creating fn that inspects /tmp/.X11-unix and creates unused display to avoid possibility of collision
|
||||
display = f':{randint(99, 999)}'
|
||||
|
||||
box_style = 'box=1:boxcolor=black@0.33:boxborderw=7'
|
||||
meta_text = get_meta_text(lr, route)
|
||||
overlays = [
|
||||
# metadata overlay
|
||||
f"drawtext=text='{escape_ffmpeg_text(meta_text)}':fontfile={OPENPILOT_FONT}:fontcolor=white:fontsize=15:{box_style}:x=(w-text_w)/2:y=5.5:enable='between(t,1,5)'",
|
||||
# route time overlay
|
||||
f"drawtext=text='%{{eif\\:floor(({start}+t)/60)\\:d\\:2}}\\:%{{eif\\:mod({start}+t\\,60)\\:d\\:2}}':fontfile={OPENPILOT_FONT}:fontcolor=white:fontsize=24:{box_style}:x=w-text_w-38:y=38"
|
||||
]
|
||||
if title:
|
||||
overlays.append(f"drawtext=text='{escape_ffmpeg_text(title)}':fontfile={OPENPILOT_FONT}:fontcolor=white:fontsize=32:{box_style}:x=(w-text_w)/2:y=53")
|
||||
|
||||
if speed > 1:
|
||||
overlays += [
|
||||
f"setpts=PTS/{speed}",
|
||||
"fps=60",
|
||||
]
|
||||
|
||||
ffmpeg_cmd = [
|
||||
'ffmpeg', '-y',
|
||||
'-video_size', RESOLUTION,
|
||||
'-framerate', str(FRAMERATE),
|
||||
'-f', 'x11grab',
|
||||
'-rtbufsize', '100M',
|
||||
'-draw_mouse', '0',
|
||||
'-i', display,
|
||||
'-c:v', 'libx264',
|
||||
'-maxrate', f'{bit_rate_kbps}k',
|
||||
'-bufsize', f'{bit_rate_kbps*2}k',
|
||||
'-crf', '23',
|
||||
'-filter:v', ','.join(overlays),
|
||||
'-preset', 'ultrafast',
|
||||
'-tune', 'zerolatency',
|
||||
'-pix_fmt', 'yuv420p',
|
||||
'-movflags', '+faststart',
|
||||
'-f', 'mp4',
|
||||
'-t', str(duration),
|
||||
out,
|
||||
]
|
||||
|
||||
replay_cmd = [REPLAY, '--ecam', '-c', '1', '-s', str(begin_at), '--prefix', prefix]
|
||||
if data_dir:
|
||||
replay_cmd.extend(['--data_dir', data_dir])
|
||||
if quality == 'low':
|
||||
replay_cmd.append('--qcam')
|
||||
replay_cmd.append(route.name.canonical_name)
|
||||
|
||||
ui_cmd = [UI, '-platform', 'xcb']
|
||||
xvfb_cmd = ['Xvfb', display, '-terminate', '-screen', '0', f'{RESOLUTION}x{PIXEL_DEPTH}']
|
||||
|
||||
with OpenpilotPrefix(prefix, shared_download_cache=True):
|
||||
populate_car_params(lr)
|
||||
env = os.environ.copy()
|
||||
env['DISPLAY'] = display
|
||||
|
||||
with managed_proc(xvfb_cmd, env) as xvfb_proc, managed_proc(ui_cmd, env) as ui_proc, managed_proc(replay_cmd, env) as replay_proc:
|
||||
procs = [xvfb_proc, ui_proc, replay_proc]
|
||||
logger.info('waiting for replay to begin (loading segments, may take a while)...')
|
||||
wait_for_frames(procs)
|
||||
logger.debug(f'letting UI warm up ({SECONDS_TO_WARM}s)...')
|
||||
time.sleep(SECONDS_TO_WARM)
|
||||
check_for_failure(procs)
|
||||
with managed_proc(ffmpeg_cmd, env) as ffmpeg_proc:
|
||||
procs.append(ffmpeg_proc)
|
||||
logger.info(f'recording in progress ({duration}s)...')
|
||||
ffmpeg_proc.wait(duration + PROC_WAIT_SECONDS)
|
||||
check_for_failure(procs)
|
||||
logger.info(f'recording complete: {Path(out).resolve()}')
|
||||
|
||||
|
||||
def main():
|
||||
p = ArgumentParser(prog='clip.py', description='clip your openpilot route.', epilog='comma.ai')
|
||||
validate_env(p)
|
||||
route_group = p.add_mutually_exclusive_group(required=True)
|
||||
route_group.add_argument('route', nargs='?', type=validate_route, help=f'The route (e.g. {DEMO_ROUTE} or {DEMO_ROUTE}/{DEMO_START}/{DEMO_END})')
|
||||
route_group.add_argument('--demo', help='use the demo route', action='store_true')
|
||||
p.add_argument('-d', '--data-dir', help='local directory where route data is stored')
|
||||
p.add_argument('-e', '--end', help='stop clipping at <end> seconds', type=int)
|
||||
p.add_argument('-f', '--file-size', help='target file size (Discord/GitHub support max 10MB, default is 9MB)', type=float, default=9.)
|
||||
p.add_argument('-o', '--output', help='output clip to (.mp4)', type=validate_output_file, default=DEFAULT_OUTPUT)
|
||||
p.add_argument('-p', '--prefix', help='openpilot prefix', default=f'clip_{randint(100, 99999)}')
|
||||
p.add_argument('-q', '--quality', help='quality of camera (low = qcam, high = hevc)', choices=['low', 'high'], default='high')
|
||||
p.add_argument('-x', '--speed', help='record the clip at this speed multiple', type=int, default=1)
|
||||
p.add_argument('-s', '--start', help='start clipping at <start> seconds', type=int)
|
||||
p.add_argument('-t', '--title', help='overlay this title on the video (e.g. "Chill driving across the Golden Gate Bridge")', type=validate_title)
|
||||
args = parse_args(p)
|
||||
exit_code = 1
|
||||
try:
|
||||
clip(
|
||||
data_dir=args.data_dir,
|
||||
quality=args.quality,
|
||||
prefix=args.prefix,
|
||||
route=args.route,
|
||||
out=args.output,
|
||||
start=args.start,
|
||||
end=args.end,
|
||||
speed=args.speed,
|
||||
target_mb=args.file_size,
|
||||
title=args.title,
|
||||
)
|
||||
exit_code = 0
|
||||
except KeyboardInterrupt as e:
|
||||
logger.exception('interrupted by user', exc_info=e)
|
||||
except Exception as e:
|
||||
logger.exception('encountered error', exc_info=e)
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
logging.basicConfig(level=logging.INFO, format='%(asctime)s %(name)s %(levelname)s\t%(message)s')
|
||||
main()
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
# Increase the pip timeout to handle TimeoutError
|
||||
export PIP_DEFAULT_TIMEOUT=200
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT="$DIR"/../
|
||||
cd "$ROOT"
|
||||
|
||||
if ! command -v "uv" > /dev/null 2>&1; then
|
||||
echo "installing uv..."
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
UV_BIN="$HOME/.local/bin"
|
||||
PATH="$UV_BIN:$PATH"
|
||||
fi
|
||||
|
||||
echo "updating uv..."
|
||||
# ok to fail, can also fail due to installing with brew
|
||||
uv self update || true
|
||||
|
||||
echo "installing python packages..."
|
||||
uv sync --frozen --all-extras
|
||||
source .venv/bin/activate
|
||||
|
||||
if [[ "$(uname)" == 'Darwin' ]]; then
|
||||
touch "$ROOT"/.env
|
||||
echo "# msgq doesn't work on mac" >> "$ROOT"/.env
|
||||
echo "export ZMQ=1" >> "$ROOT"/.env
|
||||
echo "export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" >> "$ROOT"/.env
|
||||
fi
|
||||
Executable
+132
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
SUDO=""
|
||||
|
||||
# Use sudo if not root
|
||||
if [[ ! $(id -u) -eq 0 ]]; then
|
||||
if [[ -z $(which sudo) ]]; then
|
||||
echo "Please install sudo or run as root"
|
||||
exit 1
|
||||
fi
|
||||
SUDO="sudo"
|
||||
fi
|
||||
|
||||
# Check if stdin is open
|
||||
if [ -t 0 ]; then
|
||||
INTERACTIVE=1
|
||||
fi
|
||||
|
||||
# Install common packages
|
||||
function install_ubuntu_common_requirements() {
|
||||
$SUDO apt-get update
|
||||
|
||||
# normal stuff, mostly for the bare docker image
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
clang \
|
||||
build-essential \
|
||||
curl \
|
||||
libssl-dev \
|
||||
libcurl4-openssl-dev \
|
||||
locales \
|
||||
git \
|
||||
git-lfs \
|
||||
xvfb
|
||||
|
||||
# TODO: vendor the rest of these in third_party/
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
gcc-arm-none-eabi \
|
||||
capnproto \
|
||||
libcapnp-dev \
|
||||
ffmpeg \
|
||||
libavformat-dev \
|
||||
libavcodec-dev \
|
||||
libavdevice-dev \
|
||||
libavutil-dev \
|
||||
libavfilter-dev \
|
||||
libbz2-dev \
|
||||
libeigen3-dev \
|
||||
libffi-dev \
|
||||
libgles2-mesa-dev \
|
||||
libglfw3-dev \
|
||||
libglib2.0-0 \
|
||||
libjpeg-dev \
|
||||
libqt5charts5-dev \
|
||||
libncurses5-dev \
|
||||
libusb-1.0-0-dev \
|
||||
libzmq3-dev \
|
||||
libzstd-dev \
|
||||
libsqlite3-dev \
|
||||
opencl-headers \
|
||||
ocl-icd-libopencl1 \
|
||||
ocl-icd-opencl-dev \
|
||||
portaudio19-dev \
|
||||
qttools5-dev-tools \
|
||||
libqt5svg5-dev \
|
||||
libqt5serialbus5-dev \
|
||||
libqt5x11extras5-dev \
|
||||
libqt5opengl5-dev \
|
||||
gettext
|
||||
}
|
||||
|
||||
# Install Ubuntu 24.04 LTS packages
|
||||
function install_ubuntu_lts_latest_requirements() {
|
||||
install_ubuntu_common_requirements
|
||||
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
g++-12 \
|
||||
qtbase5-dev \
|
||||
qtbase5-dev-tools \
|
||||
python3-dev \
|
||||
python3-venv
|
||||
}
|
||||
|
||||
# Detect OS using /etc/os-release file
|
||||
if [ -f "/etc/os-release" ]; then
|
||||
source /etc/os-release
|
||||
case "$VERSION_CODENAME" in
|
||||
"jammy" | "kinetic" | "noble")
|
||||
install_ubuntu_lts_latest_requirements
|
||||
;;
|
||||
*)
|
||||
echo "$ID $VERSION_ID is unsupported. This setup script is written for Ubuntu 24.04."
|
||||
read -p "Would you like to attempt installation anyway? " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
exit 1
|
||||
fi
|
||||
install_ubuntu_lts_latest_requirements
|
||||
esac
|
||||
|
||||
if [[ -d "/etc/udev/rules.d/" ]]; then
|
||||
# Setup jungle udev rules
|
||||
$SUDO tee /etc/udev/rules.d/12-panda_jungle.rules > /dev/null <<EOF
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcf", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddef", MODE="0666"
|
||||
|
||||
EOF
|
||||
|
||||
# Setup panda udev rules
|
||||
$SUDO tee /etc/udev/rules.d/11-panda.rules > /dev/null <<EOF
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="0483", ATTRS{idProduct}=="df11", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="3801", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddcc", MODE="0666"
|
||||
SUBSYSTEM=="usb", ATTRS{idVendor}=="bbaa", ATTRS{idProduct}=="ddee", MODE="0666"
|
||||
EOF
|
||||
|
||||
# Setup adb udev rules
|
||||
$SUDO tee /etc/udev/rules.d/50-comma-adb.rules > /dev/null <<EOF
|
||||
SUBSYSTEM=="usb", ATTR{idVendor}=="04d8", ATTR{idProduct}=="1234", ENV{adb_user}="yes"
|
||||
EOF
|
||||
|
||||
$SUDO udevadm control --reload-rules && $SUDO udevadm trigger || true
|
||||
fi
|
||||
|
||||
else
|
||||
echo "No /etc/os-release in the system. Make sure you're running on Ubuntu, or similar."
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,67 @@
|
||||
# JotPluggler
|
||||
|
||||
JotPluggler is a tool to quickly visualize openpilot logs.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
$ ./jotpluggler/pluggle.py -h
|
||||
usage: pluggle.py [-h] [--demo] [--layout LAYOUT] [route]
|
||||
|
||||
A tool for visualizing openpilot logs.
|
||||
|
||||
positional arguments:
|
||||
route Optional route name to load on startup.
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--demo Use the demo route instead of providing one
|
||||
--layout LAYOUT Path to YAML layout file to load on startup
|
||||
```
|
||||
|
||||
Example using route name:
|
||||
|
||||
`./pluggle.py "a2a0ccea32023010/2023-07-27--13-01-19"`
|
||||
|
||||
Examples using segment:
|
||||
|
||||
`./pluggle.py "a2a0ccea32023010/2023-07-27--13-01-19/1"`
|
||||
|
||||
`./pluggle.py "a2a0ccea32023010/2023-07-27--13-01-19/1/q" # use qlogs`
|
||||
|
||||
Example using segment range:
|
||||
|
||||
`./pluggle.py "a2a0ccea32023010/2023-07-27--13-01-19/0:1"`
|
||||
|
||||
## Demo
|
||||
|
||||
For a quick demo, run this command:
|
||||
|
||||
`./pluggle.py --demo --layout=layouts/torque-controller.yaml`
|
||||
|
||||
|
||||
## Basic Usage/Features:
|
||||
- The text box to load a route is a the top left of the page, accepts standard openpilot format routes (e.g. `a2a0ccea32023010/2023-07-27--13-01-19/0:1`, `https://connect.comma.ai/a2a0ccea32023010/2023-07-27--13-01-19/`)
|
||||
- The Play/Pause button is at the bottom of the screen, you can drag the bottom slider to seek. The timeline in timeseries plots are synced with the slider.
|
||||
- The Timeseries List sidebar has several dropdowns, the fields each show the field name and value, synced with the timeline (will show N/A until the time of the first message in that field is reached).
|
||||
- There is a search bar for the timeseries list, you can search for structs or fields, or both by separating with a "/"
|
||||
- You can drag and drop any numeric/boolean field from the timeseries list into a timeseries panel.
|
||||
- You can create more panels with the split buttons (buttons with two rectangles, either horizontal or vertical). You can resize the panels by dragging the grip in between any panel.
|
||||
- You can load and save layouts with the corresponding buttons. Layouts will save all tabs, panels, titles, timeseries, etc.
|
||||
|
||||
## Layouts
|
||||
|
||||
If you create a layout that's useful for others, consider upstreaming it.
|
||||
|
||||
## Plot Interaction Controls
|
||||
|
||||
- **Left click and drag within the plot area** to pan X
|
||||
- Left click and drag on an axis to pan an individual axis (disabled for Y-axis)
|
||||
- **Scroll in the plot area** to zoom in X axes, Y-axis is autofit
|
||||
- Scroll on an axis to zoom an individual axis
|
||||
- **Right click and drag** to select data and zoom into the selected data
|
||||
- Left click while box selecting to cancel the selection
|
||||
- **Double left click** to fit all visible data
|
||||
- Double left click on an axis to fit the individual axis (disabled for Y-axis, always autofit)
|
||||
- **Double right click** to open the plot context menu
|
||||
- **Click legend label icons** to show/hide plot items
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 2.3 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.3 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.8 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.6 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.0 KiB |
@@ -0,0 +1,360 @@
|
||||
import numpy as np
|
||||
import threading
|
||||
import multiprocessing
|
||||
import bisect
|
||||
from collections import defaultdict
|
||||
from tqdm import tqdm
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
from openpilot.selfdrive.test.process_replay.migration import migrate_all
|
||||
from openpilot.tools.lib.logreader import _LogFileReader, LogReader
|
||||
|
||||
|
||||
def flatten_dict(d: dict, sep: str = "/", prefix: str = None) -> dict:
|
||||
result = {}
|
||||
stack: list[tuple] = [(d, prefix)]
|
||||
|
||||
while stack:
|
||||
obj, current_prefix = stack.pop()
|
||||
|
||||
if isinstance(obj, dict):
|
||||
for key, val in obj.items():
|
||||
new_prefix = key if current_prefix is None else f"{current_prefix}{sep}{key}"
|
||||
if isinstance(val, (dict, list)):
|
||||
stack.append((val, new_prefix))
|
||||
else:
|
||||
result[new_prefix] = val
|
||||
elif isinstance(obj, list):
|
||||
for i, item in enumerate(obj):
|
||||
new_prefix = f"{current_prefix}{sep}{i}"
|
||||
if isinstance(item, (dict, list)):
|
||||
stack.append((item, new_prefix))
|
||||
else:
|
||||
result[new_prefix] = item
|
||||
else:
|
||||
if current_prefix is not None:
|
||||
result[current_prefix] = obj
|
||||
return result
|
||||
|
||||
|
||||
def extract_field_types(schema, prefix, field_types_dict):
|
||||
stack = [(schema, prefix)]
|
||||
|
||||
while stack:
|
||||
current_schema, current_prefix = stack.pop()
|
||||
|
||||
for field in current_schema.fields_list:
|
||||
field_name = field.proto.name
|
||||
field_path = f"{current_prefix}/{field_name}"
|
||||
field_proto = field.proto
|
||||
field_which = field_proto.which()
|
||||
|
||||
field_type = field_proto.slot.type.which() if field_which == 'slot' else field_which
|
||||
field_types_dict[field_path] = field_type
|
||||
|
||||
if field_which == 'slot':
|
||||
slot_type = field_proto.slot.type
|
||||
type_which = slot_type.which()
|
||||
|
||||
if type_which == 'list':
|
||||
element_type = slot_type.list.elementType.which()
|
||||
list_path = f"{field_path}/*"
|
||||
field_types_dict[list_path] = element_type
|
||||
|
||||
if element_type == 'struct':
|
||||
stack.append((field.schema.elementType, list_path))
|
||||
|
||||
elif type_which == 'struct':
|
||||
stack.append((field.schema, field_path))
|
||||
|
||||
elif field_which == 'group':
|
||||
stack.append((field.schema, field_path))
|
||||
|
||||
|
||||
def _convert_to_optimal_dtype(values_list, capnp_type):
|
||||
dtype_mapping = {
|
||||
'bool': np.bool_, 'int8': np.int8, 'int16': np.int16, 'int32': np.int32, 'int64': np.int64,
|
||||
'uint8': np.uint8, 'uint16': np.uint16, 'uint32': np.uint32, 'uint64': np.uint64,
|
||||
'float32': np.float32, 'float64': np.float64, 'text': object, 'data': object,
|
||||
'enum': object, 'anyPointer': object,
|
||||
}
|
||||
|
||||
target_dtype = dtype_mapping.get(capnp_type, object)
|
||||
return np.array(values_list, dtype=target_dtype)
|
||||
|
||||
|
||||
def _match_field_type(field_path, field_types):
|
||||
if field_path in field_types:
|
||||
return field_types[field_path]
|
||||
|
||||
path_parts = field_path.split('/')
|
||||
template_parts = [p if not p.isdigit() else '*' for p in path_parts]
|
||||
template_path = '/'.join(template_parts)
|
||||
return field_types.get(template_path)
|
||||
|
||||
|
||||
def _get_field_times_values(segment, field_name):
|
||||
if field_name not in segment:
|
||||
return None, None
|
||||
|
||||
field_data = segment[field_name]
|
||||
segment_times = segment['t']
|
||||
|
||||
if field_data['sparse']:
|
||||
if len(field_data['t_index']) == 0:
|
||||
return None, None
|
||||
return segment_times[field_data['t_index']], field_data['values']
|
||||
else:
|
||||
return segment_times, field_data['values']
|
||||
|
||||
|
||||
def msgs_to_time_series(msgs):
|
||||
"""Extract scalar fields and return (time_series_data, start_time, end_time)."""
|
||||
collected_data = defaultdict(lambda: {'timestamps': [], 'columns': defaultdict(list), 'sparse_fields': set()})
|
||||
field_types = {}
|
||||
extracted_schemas = set()
|
||||
min_time = max_time = None
|
||||
|
||||
for msg in msgs:
|
||||
typ = msg.which()
|
||||
timestamp = msg.logMonoTime * 1e-9
|
||||
if typ != 'initData':
|
||||
if min_time is None:
|
||||
min_time = timestamp
|
||||
max_time = timestamp
|
||||
|
||||
sub_msg = getattr(msg, typ)
|
||||
if not hasattr(sub_msg, 'to_dict'):
|
||||
continue
|
||||
|
||||
if hasattr(sub_msg, 'schema') and typ not in extracted_schemas:
|
||||
extract_field_types(sub_msg.schema, typ, field_types)
|
||||
extracted_schemas.add(typ)
|
||||
|
||||
try:
|
||||
msg_dict = sub_msg.to_dict(verbose=True)
|
||||
except Exception as e:
|
||||
cloudlog.warning(f"Failed to convert sub_msg.to_dict() for message of type: {typ}: {e}")
|
||||
continue
|
||||
|
||||
flat_dict = flatten_dict(msg_dict)
|
||||
flat_dict['_valid'] = msg.valid
|
||||
field_types[f"{typ}/_valid"] = 'bool'
|
||||
|
||||
type_data = collected_data[typ]
|
||||
columns, sparse_fields = type_data['columns'], type_data['sparse_fields']
|
||||
known_fields = set(columns.keys())
|
||||
missing_fields = known_fields - flat_dict.keys()
|
||||
|
||||
for field, value in flat_dict.items():
|
||||
if field not in known_fields and type_data['timestamps']:
|
||||
sparse_fields.add(field)
|
||||
columns[field].append(value)
|
||||
if value is None:
|
||||
sparse_fields.add(field)
|
||||
|
||||
for field in missing_fields:
|
||||
columns[field].append(None)
|
||||
sparse_fields.add(field)
|
||||
|
||||
type_data['timestamps'].append(timestamp)
|
||||
|
||||
final_result = {}
|
||||
for typ, data in collected_data.items():
|
||||
if not data['timestamps']:
|
||||
continue
|
||||
|
||||
typ_result = {'t': np.array(data['timestamps'], dtype=np.float64)}
|
||||
sparse_fields = data['sparse_fields']
|
||||
|
||||
for field_name, values in data['columns'].items():
|
||||
if len(values) < len(data['timestamps']):
|
||||
values = [None] * (len(data['timestamps']) - len(values)) + values
|
||||
sparse_fields.add(field_name)
|
||||
|
||||
capnp_type = _match_field_type(f"{typ}/{field_name}", field_types)
|
||||
|
||||
if field_name in sparse_fields: # extract non-None values and their indices
|
||||
non_none_indices = []
|
||||
non_none_values = []
|
||||
for i, value in enumerate(values):
|
||||
if value is not None:
|
||||
non_none_indices.append(i)
|
||||
non_none_values.append(value)
|
||||
|
||||
if non_none_values: # check if indices > uint16 max, currently would require a 1000+ Hz signal since indices are within segments
|
||||
assert max(non_none_indices) <= 65535, f"Sparse field {typ}/{field_name} has timestamp indices exceeding uint16 max. Max: {max(non_none_indices)}"
|
||||
|
||||
typ_result[field_name] = {
|
||||
'values': _convert_to_optimal_dtype(non_none_values, capnp_type),
|
||||
'sparse': True,
|
||||
't_index': np.array(non_none_indices, dtype=np.uint16),
|
||||
}
|
||||
else: # dense representation
|
||||
typ_result[field_name] = {'values': _convert_to_optimal_dtype(values, capnp_type), 'sparse': False}
|
||||
|
||||
final_result[typ] = typ_result
|
||||
|
||||
return final_result, min_time or 0.0, max_time or 0.0
|
||||
|
||||
|
||||
def _process_segment(segment_identifier: str):
|
||||
try:
|
||||
lr = _LogFileReader(segment_identifier, sort_by_time=True)
|
||||
migrated_msgs = migrate_all(lr)
|
||||
return msgs_to_time_series(migrated_msgs)
|
||||
except Exception as e:
|
||||
cloudlog.warning(f"Warning: Failed to process segment {segment_identifier}: {e}")
|
||||
return {}, 0.0, 0.0
|
||||
|
||||
|
||||
class DataManager:
|
||||
def __init__(self):
|
||||
self._segments = []
|
||||
self._segment_starts = []
|
||||
self._start_time = 0.0
|
||||
self._duration = 0.0
|
||||
self._paths = set()
|
||||
self._observers = []
|
||||
self._loading = False
|
||||
self._lock = threading.RLock()
|
||||
|
||||
def load_route(self, route: str) -> None:
|
||||
if self._loading:
|
||||
return
|
||||
self._reset()
|
||||
threading.Thread(target=self._load_async, args=(route,), daemon=True).start()
|
||||
|
||||
def get_timeseries(self, path: str):
|
||||
with self._lock:
|
||||
msg_type, field = path.split('/', 1)
|
||||
times, values = [], []
|
||||
|
||||
for segment in self._segments:
|
||||
if msg_type in segment:
|
||||
field_times, field_values = _get_field_times_values(segment[msg_type], field)
|
||||
if field_times is not None:
|
||||
times.append(field_times)
|
||||
values.append(field_values)
|
||||
|
||||
if not times:
|
||||
return np.array([]), np.array([])
|
||||
|
||||
combined_times = np.concatenate(times) - self._start_time
|
||||
|
||||
if len(values) > 1:
|
||||
first_dtype = values[0].dtype
|
||||
if all(arr.dtype == first_dtype for arr in values): # check if all arrays have compatible dtypes
|
||||
combined_values = np.concatenate(values)
|
||||
else:
|
||||
combined_values = np.concatenate([arr.astype(object) for arr in values])
|
||||
else:
|
||||
combined_values = values[0] if values else np.array([])
|
||||
|
||||
return combined_times, combined_values
|
||||
|
||||
def get_value_at(self, path: str, time: float):
|
||||
with self._lock:
|
||||
MAX_LOOKBACK = 5.0 # seconds
|
||||
absolute_time = self._start_time + time
|
||||
message_type, field = path.split('/', 1)
|
||||
current_index = bisect.bisect_right(self._segment_starts, absolute_time) - 1
|
||||
for index in (current_index, current_index - 1):
|
||||
if not 0 <= index < len(self._segments):
|
||||
continue
|
||||
segment = self._segments[index].get(message_type)
|
||||
if not segment:
|
||||
continue
|
||||
times, values = _get_field_times_values(segment, field)
|
||||
if times is None or len(times) == 0 or (index != current_index and absolute_time - times[-1] > MAX_LOOKBACK):
|
||||
continue
|
||||
position = np.searchsorted(times, absolute_time, 'right') - 1
|
||||
if position >= 0 and absolute_time - times[position] <= MAX_LOOKBACK:
|
||||
return values[position]
|
||||
return None
|
||||
|
||||
def get_all_paths(self):
|
||||
with self._lock:
|
||||
return sorted(self._paths)
|
||||
|
||||
def get_duration(self):
|
||||
with self._lock:
|
||||
return self._duration
|
||||
|
||||
def is_plottable(self, path: str):
|
||||
_, values = self.get_timeseries(path)
|
||||
if len(values) == 0:
|
||||
return False
|
||||
return np.issubdtype(values.dtype, np.number) or np.issubdtype(values.dtype, np.bool_)
|
||||
|
||||
def add_observer(self, callback):
|
||||
with self._lock:
|
||||
self._observers.append(callback)
|
||||
|
||||
def remove_observer(self, callback):
|
||||
with self._lock:
|
||||
if callback in self._observers:
|
||||
self._observers.remove(callback)
|
||||
|
||||
def _reset(self):
|
||||
with self._lock:
|
||||
self._loading = True
|
||||
self._segments.clear()
|
||||
self._segment_starts.clear()
|
||||
self._paths.clear()
|
||||
self._start_time = self._duration = 0.0
|
||||
observers = self._observers.copy()
|
||||
|
||||
for callback in observers:
|
||||
callback({'reset': True})
|
||||
|
||||
def _load_async(self, route: str):
|
||||
try:
|
||||
lr = LogReader(route, sort_by_time=True)
|
||||
if not lr.logreader_identifiers:
|
||||
cloudlog.warning(f"Warning: No log segments found for route: {route}")
|
||||
return
|
||||
|
||||
total_segments = len(lr.logreader_identifiers)
|
||||
with self._lock:
|
||||
observers = self._observers.copy()
|
||||
for callback in observers:
|
||||
callback({'metadata_loaded': True, 'total_segments': total_segments})
|
||||
|
||||
num_processes = max(1, multiprocessing.cpu_count() // 2)
|
||||
with multiprocessing.Pool(processes=num_processes) as pool, tqdm(total=len(lr.logreader_identifiers), desc="Processing Segments") as pbar:
|
||||
for segment_result, start_time, end_time in pool.imap(_process_segment, lr.logreader_identifiers):
|
||||
pbar.update(1)
|
||||
if segment_result:
|
||||
self._add_segment(segment_result, start_time, end_time)
|
||||
except Exception:
|
||||
cloudlog.exception(f"Error loading route {route}:")
|
||||
finally:
|
||||
self._finalize_loading()
|
||||
|
||||
def _add_segment(self, segment_data: dict, start_time: float, end_time: float):
|
||||
with self._lock:
|
||||
self._segments.append(segment_data)
|
||||
self._segment_starts.append(start_time)
|
||||
|
||||
if len(self._segments) == 1:
|
||||
self._start_time = start_time
|
||||
self._duration = end_time - self._start_time
|
||||
|
||||
for msg_type, data in segment_data.items():
|
||||
for field_name in data.keys():
|
||||
if field_name != 't':
|
||||
self._paths.add(f"{msg_type}/{field_name}")
|
||||
|
||||
observers = self._observers.copy()
|
||||
|
||||
for callback in observers:
|
||||
callback({'segment_added': True, 'duration': self._duration, 'segment_count': len(self._segments)})
|
||||
|
||||
def _finalize_loading(self):
|
||||
with self._lock:
|
||||
self._loading = False
|
||||
observers = self._observers.copy()
|
||||
duration = self._duration
|
||||
|
||||
for callback in observers:
|
||||
callback({'loading_complete': True, 'duration': duration})
|
||||
@@ -0,0 +1,315 @@
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import numpy as np
|
||||
import dearpygui.dearpygui as dpg
|
||||
|
||||
|
||||
class DataTreeNode:
|
||||
def __init__(self, name: str, full_path: str = "", parent=None):
|
||||
self.name = name
|
||||
self.full_path = full_path
|
||||
self.parent = parent
|
||||
self.children: dict[str, DataTreeNode] = {}
|
||||
self.filtered_children: dict[str, DataTreeNode] = {}
|
||||
self.created_children: dict[str, DataTreeNode] = {}
|
||||
self.is_leaf = False
|
||||
self.is_plottable: bool | None = None
|
||||
self.ui_created = False
|
||||
self.children_ui_created = False
|
||||
self.ui_tag: str | None = None
|
||||
|
||||
|
||||
class DataTree:
|
||||
MAX_NODES_PER_FRAME = 50
|
||||
|
||||
def __init__(self, data_manager, playback_manager):
|
||||
self.data_manager = data_manager
|
||||
self.playback_manager = playback_manager
|
||||
self.current_search = ""
|
||||
self.data_tree = DataTreeNode(name="root")
|
||||
self._build_queue: dict[str, tuple[DataTreeNode, DataTreeNode, str | int]] = {} # full_path -> (node, parent, before_tag)
|
||||
self._current_created_paths: set[str] = set()
|
||||
self._current_filtered_paths: set[str] = set()
|
||||
self._path_to_node: dict[str, DataTreeNode] = {} # full_path -> node
|
||||
self._expanded_tags: set[str] = set()
|
||||
self._item_handlers: dict[str, str] = {} # ui_tag -> handler_tag
|
||||
self._char_width = None
|
||||
self._queued_search = None
|
||||
self._new_data = False
|
||||
self._ui_lock = threading.RLock()
|
||||
self._handlers_to_delete = []
|
||||
self.data_manager.add_observer(self._on_data_loaded)
|
||||
|
||||
def create_ui(self, parent_tag: str):
|
||||
with dpg.child_window(parent=parent_tag, border=False, width=-1, height=-1):
|
||||
dpg.add_text("Timeseries List")
|
||||
dpg.add_separator()
|
||||
dpg.add_input_text(tag="search_input", width=-1, hint="Search fields...", callback=self.search_data)
|
||||
dpg.add_separator()
|
||||
with dpg.child_window(border=False, width=-1, height=-1):
|
||||
with dpg.group(tag="data_tree_container"):
|
||||
pass
|
||||
|
||||
def _on_data_loaded(self, data: dict):
|
||||
with self._ui_lock:
|
||||
if data.get('segment_added') or data.get('reset'):
|
||||
self._new_data = True
|
||||
|
||||
def update_frame(self, font):
|
||||
if self._handlers_to_delete: # we need to do everything in main thread, frame callbacks are flaky
|
||||
dpg.render_dearpygui_frame() # wait a frame to ensure queued callbacks are done
|
||||
with self._ui_lock:
|
||||
for handler in self._handlers_to_delete:
|
||||
dpg.delete_item(handler)
|
||||
self._handlers_to_delete.clear()
|
||||
|
||||
with self._ui_lock:
|
||||
if self._char_width is None:
|
||||
if size := dpg.get_text_size(" ", font=font):
|
||||
self._char_width = size[0] / 2 # we scale font 2x and downscale to fix hidpi bug
|
||||
|
||||
if self._new_data:
|
||||
self._process_path_change()
|
||||
self._new_data = False
|
||||
return
|
||||
|
||||
if self._queued_search is not None:
|
||||
self.current_search = self._queued_search
|
||||
self._process_path_change()
|
||||
self._queued_search = None
|
||||
return
|
||||
|
||||
nodes_processed = 0
|
||||
while self._build_queue and nodes_processed < self.MAX_NODES_PER_FRAME:
|
||||
child_node, parent, before_tag = self._build_queue.pop(next(iter(self._build_queue)))
|
||||
parent_tag = "data_tree_container" if parent.name == "root" else parent.ui_tag
|
||||
if not child_node.ui_created:
|
||||
if child_node.is_leaf:
|
||||
self._create_leaf_ui(child_node, parent_tag, before_tag)
|
||||
else:
|
||||
self._create_tree_node_ui(child_node, parent_tag, before_tag)
|
||||
parent.created_children[child_node.name] = parent.children[child_node.name]
|
||||
self._current_created_paths.add(child_node.full_path)
|
||||
nodes_processed += 1
|
||||
|
||||
def _process_path_change(self):
|
||||
self._build_queue.clear()
|
||||
search_term = self.current_search.strip().lower()
|
||||
all_paths = set(self.data_manager.get_all_paths())
|
||||
new_filtered_leafs = {path for path in all_paths if self._should_show_path(path, search_term)}
|
||||
new_filtered_paths = set(new_filtered_leafs)
|
||||
for path in new_filtered_leafs:
|
||||
parts = path.split('/')
|
||||
for i in range(1, len(parts)):
|
||||
prefix = '/'.join(parts[:i])
|
||||
new_filtered_paths.add(prefix)
|
||||
created_paths_to_remove = self._current_created_paths - new_filtered_paths
|
||||
filtered_paths_to_remove = self._current_filtered_paths - new_filtered_leafs
|
||||
|
||||
if created_paths_to_remove or filtered_paths_to_remove:
|
||||
self._remove_paths_from_tree(created_paths_to_remove, filtered_paths_to_remove)
|
||||
self._apply_expansion_to_tree(self.data_tree, search_term)
|
||||
|
||||
paths_to_add = new_filtered_leafs - self._current_created_paths
|
||||
if paths_to_add:
|
||||
self._add_paths_to_tree(paths_to_add)
|
||||
self._apply_expansion_to_tree(self.data_tree, search_term)
|
||||
self._current_filtered_paths = new_filtered_paths
|
||||
|
||||
def _remove_paths_from_tree(self, created_paths_to_remove, filtered_paths_to_remove):
|
||||
for path in sorted(created_paths_to_remove, reverse=True):
|
||||
current_node = self._path_to_node[path]
|
||||
|
||||
if len(current_node.created_children) == 0:
|
||||
self._current_created_paths.remove(current_node.full_path)
|
||||
if item_handler_tag := self._item_handlers.get(current_node.ui_tag):
|
||||
dpg.configure_item(item_handler_tag, show=False)
|
||||
self._handlers_to_delete.append(item_handler_tag)
|
||||
del self._item_handlers[current_node.ui_tag]
|
||||
dpg.delete_item(current_node.ui_tag)
|
||||
current_node.ui_created = False
|
||||
current_node.ui_tag = None
|
||||
current_node.children_ui_created = False
|
||||
del current_node.parent.created_children[current_node.name]
|
||||
del current_node.parent.filtered_children[current_node.name]
|
||||
|
||||
for path in filtered_paths_to_remove:
|
||||
parts = path.split('/')
|
||||
current_node = self._path_to_node[path]
|
||||
|
||||
part_array_index = -1
|
||||
while len(current_node.filtered_children) == 0 and part_array_index >= -len(parts):
|
||||
current_node = current_node.parent
|
||||
if parts[part_array_index] in current_node.filtered_children:
|
||||
del current_node.filtered_children[parts[part_array_index]]
|
||||
part_array_index -= 1
|
||||
|
||||
def _add_paths_to_tree(self, paths):
|
||||
parent_nodes_to_recheck = set()
|
||||
for path in sorted(paths):
|
||||
parts = path.split('/')
|
||||
current_node = self.data_tree
|
||||
current_path_prefix = ""
|
||||
|
||||
for i, part in enumerate(parts):
|
||||
current_path_prefix = f"{current_path_prefix}/{part}" if current_path_prefix else part
|
||||
if i < len(parts):
|
||||
parent_nodes_to_recheck.add(current_node) # for incremental changes from new data
|
||||
if part not in current_node.children:
|
||||
current_node.children[part] = DataTreeNode(name=part, full_path=current_path_prefix, parent=current_node)
|
||||
self._path_to_node[current_path_prefix] = current_node.children[part]
|
||||
current_node.filtered_children[part] = current_node.children[part]
|
||||
current_node = current_node.children[part]
|
||||
|
||||
if not current_node.is_leaf:
|
||||
current_node.is_leaf = True
|
||||
|
||||
for p_node in parent_nodes_to_recheck:
|
||||
p_node.children_ui_created = False
|
||||
self._request_children_build(p_node)
|
||||
|
||||
def _get_node_label_and_expand(self, node: DataTreeNode, search_term: str):
|
||||
label = f"{node.name} ({len(node.filtered_children)} fields)"
|
||||
expand = len(search_term) > 0 and any(search_term in path for path in self._get_descendant_paths(node))
|
||||
if expand and node.parent and len(node.parent.filtered_children) > 100 and len(node.filtered_children) > 2:
|
||||
label += " (+)" # symbol for large lists which aren't fully expanded for performance (only affects procLog rn)
|
||||
expand = False
|
||||
return label, expand
|
||||
|
||||
def _apply_expansion_to_tree(self, node: DataTreeNode, search_term: str):
|
||||
if node.ui_created and not node.is_leaf and node.ui_tag and dpg.does_item_exist(node.ui_tag):
|
||||
label, expand = self._get_node_label_and_expand(node, search_term)
|
||||
if expand:
|
||||
self._expanded_tags.add(node.ui_tag)
|
||||
dpg.set_value(node.ui_tag, expand)
|
||||
elif node.ui_tag in self._expanded_tags: # not expanded and was expanded
|
||||
self._expanded_tags.remove(node.ui_tag)
|
||||
dpg.set_value(node.ui_tag, expand)
|
||||
dpg.delete_item(node.ui_tag, children_only=True) # delete children (not visible since collapsed)
|
||||
self._reset_ui_state_recursive(node)
|
||||
node.children_ui_created = False
|
||||
dpg.set_item_label(node.ui_tag, label)
|
||||
for child in node.created_children.values():
|
||||
self._apply_expansion_to_tree(child, search_term)
|
||||
|
||||
def _reset_ui_state_recursive(self, node: DataTreeNode):
|
||||
for child in node.created_children.values():
|
||||
if child.ui_tag is not None:
|
||||
if item_handler_tag := self._item_handlers.get(child.ui_tag):
|
||||
self._handlers_to_delete.append(item_handler_tag)
|
||||
dpg.configure_item(item_handler_tag, show=False)
|
||||
del self._item_handlers[child.ui_tag]
|
||||
self._reset_ui_state_recursive(child)
|
||||
child.ui_created = False
|
||||
child.ui_tag = None
|
||||
child.children_ui_created = False
|
||||
self._current_created_paths.remove(child.full_path)
|
||||
node.created_children.clear()
|
||||
|
||||
def search_data(self):
|
||||
with self._ui_lock:
|
||||
self._queued_search = dpg.get_value("search_input")
|
||||
|
||||
def _create_tree_node_ui(self, node: DataTreeNode, parent_tag: str, before: str | int):
|
||||
node.ui_tag = f"tree_{node.full_path}"
|
||||
search_term = self.current_search.strip().lower()
|
||||
label, expand = self._get_node_label_and_expand(node, search_term)
|
||||
if expand:
|
||||
self._expanded_tags.add(node.ui_tag)
|
||||
elif node.ui_tag in self._expanded_tags:
|
||||
self._expanded_tags.remove(node.ui_tag)
|
||||
|
||||
with dpg.tree_node(
|
||||
label=label, parent=parent_tag, tag=node.ui_tag, default_open=expand, open_on_arrow=True, open_on_double_click=True, before=before, delay_search=True
|
||||
):
|
||||
with dpg.item_handler_registry() as handler_tag:
|
||||
dpg.add_item_toggled_open_handler(callback=lambda s, a, u: self._request_children_build(node))
|
||||
dpg.add_item_visible_handler(callback=lambda s, a, u: self._request_children_build(node))
|
||||
dpg.bind_item_handler_registry(node.ui_tag, handler_tag)
|
||||
self._item_handlers[node.ui_tag] = handler_tag
|
||||
node.ui_created = True
|
||||
|
||||
def _create_leaf_ui(self, node: DataTreeNode, parent_tag: str, before: str | int):
|
||||
node.ui_tag = f"leaf_{node.full_path}"
|
||||
with dpg.group(parent=parent_tag, tag=node.ui_tag, before=before, delay_search=True):
|
||||
with dpg.table(header_row=False, policy=dpg.mvTable_SizingStretchProp, delay_search=True):
|
||||
dpg.add_table_column(init_width_or_weight=0.5)
|
||||
dpg.add_table_column(init_width_or_weight=0.5)
|
||||
with dpg.table_row():
|
||||
dpg.add_text(node.name)
|
||||
dpg.add_text("N/A", tag=f"value_{node.full_path}")
|
||||
|
||||
if node.is_plottable is None:
|
||||
node.is_plottable = self.data_manager.is_plottable(node.full_path)
|
||||
if node.is_plottable:
|
||||
with dpg.drag_payload(parent=node.ui_tag, drag_data=node.full_path, payload_type="TIMESERIES_PAYLOAD"):
|
||||
dpg.add_text(f"Plot: {node.full_path}")
|
||||
|
||||
with dpg.item_handler_registry() as handler_tag:
|
||||
dpg.add_item_visible_handler(callback=self._on_item_visible, user_data=node.full_path)
|
||||
dpg.bind_item_handler_registry(node.ui_tag, handler_tag)
|
||||
self._item_handlers[node.ui_tag] = handler_tag
|
||||
node.ui_created = True
|
||||
|
||||
def _on_item_visible(self, sender, app_data, user_data):
|
||||
with self._ui_lock:
|
||||
path = user_data
|
||||
value_tag = f"value_{path}"
|
||||
if not dpg.does_item_exist(value_tag):
|
||||
return
|
||||
value_column_width = dpg.get_item_rect_size(f"leaf_{path}")[0] // 2
|
||||
value = self.data_manager.get_value_at(path, self.playback_manager.current_time_s)
|
||||
if value is not None:
|
||||
formatted_value = self.format_and_truncate(value, value_column_width, self._char_width)
|
||||
dpg.set_value(value_tag, formatted_value)
|
||||
else:
|
||||
dpg.set_value(value_tag, "N/A")
|
||||
|
||||
def _request_children_build(self, node: DataTreeNode):
|
||||
with self._ui_lock:
|
||||
if not node.children_ui_created and (node.name == "root" or (node.ui_tag is not None and dpg.get_value(node.ui_tag))): # check root or node expanded
|
||||
sorted_children = sorted(node.filtered_children.values(), key=self._natural_sort_key)
|
||||
next_existing: list[int | str] = [0] * len(sorted_children)
|
||||
current_before_tag: int | str = 0
|
||||
|
||||
for i in range(len(sorted_children) - 1, -1, -1): # calculate "before_tag" for correct ordering when incrementally building tree
|
||||
child = sorted_children[i]
|
||||
next_existing[i] = current_before_tag
|
||||
if child.ui_created:
|
||||
candidate_tag = f"leaf_{child.full_path}" if child.is_leaf else f"tree_{child.full_path}"
|
||||
if dpg.does_item_exist(candidate_tag):
|
||||
current_before_tag = candidate_tag
|
||||
|
||||
for i, child_node in enumerate(sorted_children):
|
||||
if not child_node.ui_created:
|
||||
before_tag = next_existing[i]
|
||||
self._build_queue[child_node.full_path] = (child_node, node, before_tag)
|
||||
node.children_ui_created = True
|
||||
|
||||
def _should_show_path(self, path: str, search_term: str) -> bool:
|
||||
if 'DEPRECATED' in path and not os.environ.get('SHOW_DEPRECATED'):
|
||||
return False
|
||||
return not search_term or search_term in path.lower()
|
||||
|
||||
def _natural_sort_key(self, node: DataTreeNode):
|
||||
node_type_key = node.is_leaf
|
||||
parts = [int(p) if p.isdigit() else p.lower() for p in re.split(r'(\d+)', node.name) if p]
|
||||
return (node_type_key, parts)
|
||||
|
||||
def _get_descendant_paths(self, node: DataTreeNode):
|
||||
for child_name, child_node in node.filtered_children.items():
|
||||
child_name_lower = child_name.lower()
|
||||
if child_node.is_leaf:
|
||||
yield child_name_lower
|
||||
else:
|
||||
for path in self._get_descendant_paths(child_node):
|
||||
yield f"{child_name_lower}/{path}"
|
||||
|
||||
@staticmethod
|
||||
def format_and_truncate(value, available_width: float, char_width: float) -> str:
|
||||
s = f"{value:.5f}" if np.issubdtype(type(value), np.floating) else str(value)
|
||||
max_chars = int(available_width / char_width)
|
||||
if len(s) > max_chars:
|
||||
return s[: max(0, max_chars - 3)] + "..."
|
||||
return s
|
||||
@@ -0,0 +1,477 @@
|
||||
import dearpygui.dearpygui as dpg
|
||||
from openpilot.tools.jotpluggler.data import DataManager
|
||||
from openpilot.tools.jotpluggler.views import TimeSeriesPanel
|
||||
|
||||
GRIP_SIZE = 4
|
||||
MIN_PANE_SIZE = 60
|
||||
|
||||
class LayoutManager:
|
||||
def __init__(self, data_manager, playback_manager, worker_manager, scale: float = 1.0):
|
||||
self.data_manager = data_manager
|
||||
self.playback_manager = playback_manager
|
||||
self.worker_manager = worker_manager
|
||||
self.scale = scale
|
||||
self.container_tag = "plot_layout_container"
|
||||
self.tab_bar_tag = "tab_bar_container"
|
||||
self.tab_content_tag = "tab_content_area"
|
||||
|
||||
self.active_tab = 0
|
||||
initial_panel_layout = PanelLayoutManager(data_manager, playback_manager, worker_manager, scale)
|
||||
self.tabs: dict = {0: {"name": "Tab 1", "panel_layout": initial_panel_layout}}
|
||||
self._next_tab_id = self.active_tab + 1
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"tabs": {
|
||||
str(tab_id): {
|
||||
"name": tab_data["name"],
|
||||
"panel_layout": tab_data["panel_layout"].to_dict()
|
||||
}
|
||||
for tab_id, tab_data in self.tabs.items()
|
||||
}
|
||||
}
|
||||
|
||||
def clear_and_load_from_dict(self, data: dict):
|
||||
tab_ids_to_close = list(self.tabs.keys())
|
||||
for tab_id in tab_ids_to_close:
|
||||
self.close_tab(tab_id, force=True)
|
||||
|
||||
for tab_id_str, tab_data in data["tabs"].items():
|
||||
tab_id = int(tab_id_str)
|
||||
panel_layout = PanelLayoutManager.load_from_dict(
|
||||
tab_data["panel_layout"], self.data_manager, self.playback_manager,
|
||||
self.worker_manager, self.scale
|
||||
)
|
||||
self.tabs[tab_id] = {
|
||||
"name": tab_data["name"],
|
||||
"panel_layout": panel_layout
|
||||
}
|
||||
|
||||
self.active_tab = min(self.tabs.keys()) if self.tabs else 0
|
||||
self._next_tab_id = max(self.tabs.keys()) + 1 if self.tabs else 1
|
||||
|
||||
def create_ui(self, parent_tag: str):
|
||||
if dpg.does_item_exist(self.container_tag):
|
||||
dpg.delete_item(self.container_tag)
|
||||
|
||||
with dpg.child_window(tag=self.container_tag, parent=parent_tag, border=False, width=-1, height=-1, no_scrollbar=True, no_scroll_with_mouse=True):
|
||||
self._create_tab_bar()
|
||||
self._create_tab_content()
|
||||
dpg.bind_item_theme(self.tab_bar_tag, "tab_bar_theme")
|
||||
|
||||
def _create_tab_bar(self):
|
||||
text_size = int(13 * self.scale)
|
||||
with dpg.child_window(tag=self.tab_bar_tag, parent=self.container_tag, height=(text_size + 8), border=False, horizontal_scrollbar=True):
|
||||
with dpg.group(horizontal=True, tag="tab_bar_group"):
|
||||
for tab_id, tab_data in self.tabs.items():
|
||||
self._create_tab_ui(tab_id, tab_data["name"])
|
||||
dpg.add_image_button(texture_tag="plus_texture", callback=self.add_tab, width=text_size, height=text_size, tag="add_tab_button")
|
||||
dpg.bind_item_theme("add_tab_button", "inactive_tab_theme")
|
||||
|
||||
def _create_tab_ui(self, tab_id: int, tab_name: str):
|
||||
text_size = int(13 * self.scale)
|
||||
tab_width = int(140 * self.scale)
|
||||
with dpg.child_window(width=tab_width, height=-1, border=False, no_scrollbar=True, tag=f"tab_window_{tab_id}", parent="tab_bar_group"):
|
||||
with dpg.group(horizontal=True, tag=f"tab_group_{tab_id}"):
|
||||
dpg.add_input_text(
|
||||
default_value=tab_name, width=tab_width - text_size - 16, callback=lambda s, v, u: self.rename_tab(u, v), user_data=tab_id, tag=f"tab_input_{tab_id}"
|
||||
)
|
||||
dpg.add_image_button(
|
||||
texture_tag="x_texture", callback=lambda s, a, u: self.close_tab(u), user_data=tab_id, width=text_size, height=text_size, tag=f"tab_close_{tab_id}"
|
||||
)
|
||||
with dpg.item_handler_registry(tag=f"tab_handler_{tab_id}"):
|
||||
dpg.add_item_clicked_handler(callback=lambda s, a, u: self.switch_tab(u), user_data=tab_id)
|
||||
dpg.bind_item_handler_registry(f"tab_group_{tab_id}", f"tab_handler_{tab_id}")
|
||||
|
||||
theme_tag = "active_tab_theme" if tab_id == self.active_tab else "inactive_tab_theme"
|
||||
dpg.bind_item_theme(f"tab_window_{tab_id}", theme_tag)
|
||||
|
||||
def _create_tab_content(self):
|
||||
with dpg.child_window(tag=self.tab_content_tag, parent=self.container_tag, border=False, width=-1, height=-1, no_scrollbar=True, no_scroll_with_mouse=True):
|
||||
if self.active_tab in self.tabs:
|
||||
active_panel_layout = self.tabs[self.active_tab]["panel_layout"]
|
||||
active_panel_layout.create_ui()
|
||||
|
||||
def add_tab(self):
|
||||
new_panel_layout = PanelLayoutManager(self.data_manager, self.playback_manager, self.worker_manager, self.scale)
|
||||
new_tab = {"name": f"Tab {self._next_tab_id + 1}", "panel_layout": new_panel_layout}
|
||||
self.tabs[self._next_tab_id] = new_tab
|
||||
self._create_tab_ui(self._next_tab_id, new_tab["name"])
|
||||
dpg.move_item("add_tab_button", parent="tab_bar_group") # move plus button to end
|
||||
self.switch_tab(self._next_tab_id)
|
||||
self._next_tab_id += 1
|
||||
|
||||
def close_tab(self, tab_id: int, force = False):
|
||||
if len(self.tabs) <= 1 and not force:
|
||||
return # don't allow closing the last tab
|
||||
|
||||
tab_to_close = self.tabs[tab_id]
|
||||
tab_to_close["panel_layout"].destroy_ui()
|
||||
for suffix in ["window", "group", "input", "close", "handler"]:
|
||||
tag = f"tab_{suffix}_{tab_id}"
|
||||
if dpg.does_item_exist(tag):
|
||||
dpg.delete_item(tag)
|
||||
del self.tabs[tab_id]
|
||||
|
||||
if self.active_tab == tab_id and self.tabs: # switch to another tab if we closed the active one
|
||||
self.active_tab = next(iter(self.tabs.keys()))
|
||||
self._switch_tab_content()
|
||||
dpg.bind_item_theme(f"tab_window_{self.active_tab}", "active_tab_theme")
|
||||
|
||||
def switch_tab(self, tab_id: int):
|
||||
if tab_id == self.active_tab or tab_id not in self.tabs:
|
||||
return
|
||||
|
||||
current_panel_layout = self.tabs[self.active_tab]["panel_layout"]
|
||||
current_panel_layout.destroy_ui()
|
||||
dpg.bind_item_theme(f"tab_window_{self.active_tab}", "inactive_tab_theme") # deactivate old tab
|
||||
self.active_tab = tab_id
|
||||
dpg.bind_item_theme(f"tab_window_{tab_id}", "active_tab_theme") # activate new tab
|
||||
self._switch_tab_content()
|
||||
|
||||
def _switch_tab_content(self):
|
||||
dpg.delete_item(self.tab_content_tag, children_only=True)
|
||||
active_panel_layout = self.tabs[self.active_tab]["panel_layout"]
|
||||
active_panel_layout.create_ui()
|
||||
active_panel_layout.update_all_panels()
|
||||
|
||||
def rename_tab(self, tab_id: int, new_name: str):
|
||||
if tab_id in self.tabs:
|
||||
self.tabs[tab_id]["name"] = new_name
|
||||
|
||||
def update_all_panels(self):
|
||||
self.tabs[self.active_tab]["panel_layout"].update_all_panels()
|
||||
|
||||
def on_viewport_resize(self):
|
||||
self.tabs[self.active_tab]["panel_layout"].on_viewport_resize()
|
||||
|
||||
class PanelLayoutManager:
|
||||
def __init__(self, data_manager: DataManager, playback_manager, worker_manager, scale: float = 1.0):
|
||||
self.data_manager = data_manager
|
||||
self.playback_manager = playback_manager
|
||||
self.worker_manager = worker_manager
|
||||
self.scale = scale
|
||||
self.active_panels: list = []
|
||||
self.parent_tag = "tab_content_area"
|
||||
self._queue_resize = False
|
||||
self._created_handler_tags: set[str] = set()
|
||||
|
||||
self.grip_size = int(GRIP_SIZE * self.scale)
|
||||
self.min_pane_size = int(MIN_PANE_SIZE * self.scale)
|
||||
|
||||
initial_panel = TimeSeriesPanel(data_manager, playback_manager, worker_manager)
|
||||
self.layout: dict = {"type": "panel", "panel": initial_panel}
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return self._layout_to_dict(self.layout)
|
||||
|
||||
def _layout_to_dict(self, layout: dict) -> dict:
|
||||
if layout["type"] == "panel":
|
||||
return {
|
||||
"type": "panel",
|
||||
"panel": layout["panel"].to_dict()
|
||||
}
|
||||
else: # split
|
||||
return {
|
||||
"type": "split",
|
||||
"orientation": layout["orientation"],
|
||||
"proportions": layout["proportions"],
|
||||
"children": [self._layout_to_dict(child) for child in layout["children"]]
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def load_from_dict(cls, data: dict, data_manager, playback_manager, worker_manager, scale: float = 1.0):
|
||||
manager = cls(data_manager, playback_manager, worker_manager, scale)
|
||||
manager.layout = manager._dict_to_layout(data)
|
||||
return manager
|
||||
|
||||
def _dict_to_layout(self, data: dict) -> dict:
|
||||
if data["type"] == "panel":
|
||||
panel_data = data["panel"]
|
||||
if panel_data["type"] == "timeseries":
|
||||
panel = TimeSeriesPanel.load_from_dict(
|
||||
panel_data, self.data_manager, self.playback_manager, self.worker_manager
|
||||
)
|
||||
return {"type": "panel", "panel": panel}
|
||||
else:
|
||||
# Handle future panel types here or make a general mapping
|
||||
raise ValueError(f"Unknown panel type: {panel_data['type']}")
|
||||
else: # split
|
||||
return {
|
||||
"type": "split",
|
||||
"orientation": data["orientation"],
|
||||
"proportions": data["proportions"],
|
||||
"children": [self._dict_to_layout(child) for child in data["children"]]
|
||||
}
|
||||
|
||||
def create_ui(self):
|
||||
self.active_panels.clear()
|
||||
if dpg.does_item_exist(self.parent_tag):
|
||||
dpg.delete_item(self.parent_tag, children_only=True)
|
||||
self._cleanup_all_handlers()
|
||||
|
||||
container_width, container_height = dpg.get_item_rect_size(self.parent_tag)
|
||||
if container_width == 0 and container_height == 0:
|
||||
self._queue_resize = True
|
||||
self._create_ui_recursive(self.layout, self.parent_tag, [], container_width, container_height)
|
||||
|
||||
def destroy_ui(self):
|
||||
self._cleanup_ui_recursive(self.layout, [])
|
||||
self._cleanup_all_handlers()
|
||||
self.active_panels.clear()
|
||||
|
||||
def _cleanup_all_handlers(self):
|
||||
for handler_tag in list(self._created_handler_tags):
|
||||
if dpg.does_item_exist(handler_tag):
|
||||
dpg.delete_item(handler_tag)
|
||||
self._created_handler_tags.clear()
|
||||
|
||||
def _create_ui_recursive(self, layout: dict, parent_tag: str, path: list[int], width: int, height: int):
|
||||
if layout["type"] == "panel":
|
||||
self._create_panel_ui(layout, parent_tag, path, width, height)
|
||||
else:
|
||||
self._create_split_ui(layout, parent_tag, path, width, height)
|
||||
|
||||
def _create_panel_ui(self, layout: dict, parent_tag: str, path: list[int], width: int, height: int):
|
||||
panel_tag = self._path_to_tag(path, "panel")
|
||||
panel = layout["panel"]
|
||||
self.active_panels.append(panel)
|
||||
text_size = int(13 * self.scale)
|
||||
bar_height = (text_size + 24) if width < int(329 * self.scale + 64) else (text_size + 8) # adjust height to allow for scrollbar
|
||||
|
||||
with dpg.child_window(parent=parent_tag, border=False, width=-1, height=-1, no_scrollbar=True):
|
||||
with dpg.group(horizontal=True):
|
||||
with dpg.child_window(tag=panel_tag, width=-(text_size + 16), height=bar_height, horizontal_scrollbar=True, no_scroll_with_mouse=True, border=False):
|
||||
with dpg.group(horizontal=True):
|
||||
# if you change the widths make sure to change the sum of widths (currently 329 * scale)
|
||||
dpg.add_input_text(default_value=panel.title, width=int(150 * self.scale), callback=lambda s, v: setattr(panel, "title", v))
|
||||
dpg.add_combo(items=["Time Series"], default_value="Time Series", width=int(100 * self.scale))
|
||||
dpg.add_button(label="Clear", callback=lambda: self.clear_panel(panel), width=int(40 * self.scale))
|
||||
dpg.add_image_button(texture_tag="split_h_texture", callback=lambda: self.split_panel(path, 0), width=text_size, height=text_size)
|
||||
dpg.add_image_button(texture_tag="split_v_texture", callback=lambda: self.split_panel(path, 1), width=text_size, height=text_size)
|
||||
dpg.add_image_button(texture_tag="x_texture", callback=lambda: self.delete_panel(path), width=text_size, height=text_size)
|
||||
|
||||
dpg.add_separator()
|
||||
|
||||
content_tag = self._path_to_tag(path, "content")
|
||||
with dpg.child_window(tag=content_tag, border=False, height=-1, width=-1, no_scrollbar=True):
|
||||
panel.create_ui(content_tag)
|
||||
|
||||
def _create_split_ui(self, layout: dict, parent_tag: str, path: list[int], width: int, height: int):
|
||||
split_tag = self._path_to_tag(path, "split")
|
||||
orientation, _, pane_sizes = self._get_split_geometry(layout, (width, height))
|
||||
|
||||
with dpg.group(tag=split_tag, parent=parent_tag, horizontal=orientation == 0):
|
||||
for i, child_layout in enumerate(layout["children"]):
|
||||
child_path = path + [i]
|
||||
container_tag = self._path_to_tag(child_path, "container")
|
||||
pane_width, pane_height = [(pane_sizes[i], -1), (-1, pane_sizes[i])][orientation] # fill 2nd dim up to the border
|
||||
with dpg.child_window(tag=container_tag, width=pane_width, height=pane_height, border=False, no_scrollbar=True):
|
||||
child_width, child_height = [(pane_sizes[i], height), (width, pane_sizes[i])][orientation]
|
||||
self._create_ui_recursive(child_layout, container_tag, child_path, child_width, child_height)
|
||||
if i < len(layout["children"]) - 1:
|
||||
self._create_grip(split_tag, path, i, orientation)
|
||||
|
||||
def clear_panel(self, panel):
|
||||
panel.clear()
|
||||
|
||||
def delete_panel(self, panel_path: list[int]):
|
||||
if not panel_path: # Root deletion
|
||||
old_panel = self.layout["panel"]
|
||||
old_panel.destroy_ui()
|
||||
self.active_panels.remove(old_panel)
|
||||
new_panel = TimeSeriesPanel(self.data_manager, self.playback_manager, self.worker_manager)
|
||||
self.layout = {"type": "panel", "panel": new_panel}
|
||||
self._rebuild_ui_at_path([])
|
||||
return
|
||||
|
||||
parent, child_index = self._get_parent_and_index(panel_path)
|
||||
layout_to_delete = parent["children"][child_index]
|
||||
self._cleanup_ui_recursive(layout_to_delete, panel_path)
|
||||
|
||||
parent["children"].pop(child_index)
|
||||
parent["proportions"].pop(child_index)
|
||||
|
||||
if len(parent["children"]) == 1: # remove parent and collapse
|
||||
remaining_child = parent["children"][0]
|
||||
if len(panel_path) == 1: # parent is at root level - promote remaining child to root
|
||||
self.layout = remaining_child
|
||||
self._rebuild_ui_at_path([])
|
||||
else: # replace parent with remaining child in grandparent
|
||||
grandparent_path = panel_path[:-2]
|
||||
parent_index = panel_path[-2]
|
||||
self._replace_layout_at_path(grandparent_path + [parent_index], remaining_child)
|
||||
self._rebuild_ui_at_path(grandparent_path + [parent_index])
|
||||
else: # redistribute proportions
|
||||
equal_prop = 1.0 / len(parent["children"])
|
||||
parent["proportions"] = [equal_prop] * len(parent["children"])
|
||||
self._rebuild_ui_at_path(panel_path[:-1])
|
||||
|
||||
def split_panel(self, panel_path: list[int], orientation: int):
|
||||
current_layout = self._get_layout_at_path(panel_path)
|
||||
existing_panel = current_layout["panel"]
|
||||
new_panel = TimeSeriesPanel(self.data_manager, self.playback_manager, self.worker_manager)
|
||||
parent, child_index = self._get_parent_and_index(panel_path)
|
||||
|
||||
if parent is None: # Root split
|
||||
self.layout = {
|
||||
"type": "split",
|
||||
"orientation": orientation,
|
||||
"children": [{"type": "panel", "panel": existing_panel}, {"type": "panel", "panel": new_panel}],
|
||||
"proportions": [0.5, 0.5],
|
||||
}
|
||||
self._rebuild_ui_at_path([])
|
||||
elif parent["type"] == "split" and parent["orientation"] == orientation: # Same orientation - insert into existing split
|
||||
parent["children"].insert(child_index + 1, {"type": "panel", "panel": new_panel})
|
||||
parent["proportions"] = [1.0 / len(parent["children"])] * len(parent["children"])
|
||||
self._rebuild_ui_at_path(panel_path[:-1])
|
||||
else: # Different orientation - create new split level
|
||||
new_split = {"type": "split", "orientation": orientation, "children": [current_layout, {"type": "panel", "panel": new_panel}], "proportions": [0.5, 0.5]}
|
||||
self._replace_layout_at_path(panel_path, new_split)
|
||||
self._rebuild_ui_at_path(panel_path)
|
||||
|
||||
def _rebuild_ui_at_path(self, path: list[int]):
|
||||
layout = self._get_layout_at_path(path)
|
||||
if path:
|
||||
container_tag = self._path_to_tag(path, "container")
|
||||
else: # Root update
|
||||
container_tag = self.parent_tag
|
||||
|
||||
self._cleanup_ui_recursive(layout, path)
|
||||
dpg.delete_item(container_tag, children_only=True)
|
||||
width, height = dpg.get_item_rect_size(container_tag)
|
||||
self._create_ui_recursive(layout, container_tag, path, width, height)
|
||||
|
||||
def _cleanup_ui_recursive(self, layout: dict, path: list[int]):
|
||||
if layout["type"] == "panel":
|
||||
panel = layout["panel"]
|
||||
panel.destroy_ui()
|
||||
if panel in self.active_panels:
|
||||
self.active_panels.remove(panel)
|
||||
else:
|
||||
for i in range(len(layout["children"]) - 1):
|
||||
handler_tag = f"{self._path_to_tag(path, f'grip_{i}')}_handler"
|
||||
if dpg.does_item_exist(handler_tag):
|
||||
dpg.delete_item(handler_tag)
|
||||
self._created_handler_tags.discard(handler_tag)
|
||||
|
||||
for i, child in enumerate(layout["children"]):
|
||||
self._cleanup_ui_recursive(child, path + [i])
|
||||
|
||||
def update_all_panels(self):
|
||||
if self._queue_resize:
|
||||
if (size := dpg.get_item_rect_size(self.parent_tag)) != [0, 0]:
|
||||
self._queue_resize = False
|
||||
self._resize_splits_recursive(self.layout, [], *size)
|
||||
for panel in self.active_panels:
|
||||
panel.update()
|
||||
|
||||
def on_viewport_resize(self):
|
||||
self._resize_splits_recursive(self.layout, [])
|
||||
|
||||
def _resize_splits_recursive(self, layout: dict, path: list[int], width: int | None = None, height: int | None = None):
|
||||
if layout["type"] == "split":
|
||||
split_tag = self._path_to_tag(path, "split")
|
||||
if dpg.does_item_exist(split_tag):
|
||||
available_sizes = (width, height) if width and height else dpg.get_item_rect_size(dpg.get_item_parent(split_tag))
|
||||
orientation, _, pane_sizes = self._get_split_geometry(layout, available_sizes)
|
||||
size_properties = ("width", "height")
|
||||
|
||||
for i, child_layout in enumerate(layout["children"]):
|
||||
child_path = path + [i]
|
||||
container_tag = self._path_to_tag(child_path, "container")
|
||||
if dpg.does_item_exist(container_tag):
|
||||
dpg.configure_item(container_tag, **{size_properties[orientation]: pane_sizes[i]})
|
||||
child_width, child_height = [(pane_sizes[i], available_sizes[1]), (available_sizes[0], pane_sizes[i])][orientation]
|
||||
self._resize_splits_recursive(child_layout, child_path, child_width, child_height)
|
||||
else: # leaf node/panel - adjust bar height to allow for scrollbar
|
||||
panel_tag = self._path_to_tag(path, "panel")
|
||||
if width is not None and width < int(329 * self.scale + 64): # scaled widths of the elements in top bar + fixed 8 padding on left and right of each item
|
||||
dpg.configure_item(panel_tag, height=(int(13 * self.scale) + 24))
|
||||
else:
|
||||
dpg.configure_item(panel_tag, height=(int(13 * self.scale) + 8))
|
||||
|
||||
def _get_split_geometry(self, layout: dict, available_size: tuple[int, int]) -> tuple[int, int, list[int]]:
|
||||
orientation = layout["orientation"]
|
||||
num_grips = len(layout["children"]) - 1
|
||||
usable_size = max(self.min_pane_size, available_size[orientation] - (num_grips * (self.grip_size + 8 * (2 - orientation)))) # approximate, scaling is weird
|
||||
pane_sizes = [max(self.min_pane_size, int(usable_size * prop)) for prop in layout["proportions"]]
|
||||
return orientation, usable_size, pane_sizes
|
||||
|
||||
def _get_layout_at_path(self, path: list[int]) -> dict:
|
||||
current = self.layout
|
||||
for index in path:
|
||||
current = current["children"][index]
|
||||
return current
|
||||
|
||||
def _get_parent_and_index(self, path: list[int]) -> tuple:
|
||||
return (None, -1) if not path else (self._get_layout_at_path(path[:-1]), path[-1])
|
||||
|
||||
def _replace_layout_at_path(self, path: list[int], new_layout: dict):
|
||||
if not path:
|
||||
self.layout = new_layout
|
||||
else:
|
||||
parent, index = self._get_parent_and_index(path)
|
||||
parent["children"][index] = new_layout
|
||||
|
||||
def _path_to_tag(self, path: list[int], prefix: str = "") -> str:
|
||||
path_str = "_".join(map(str, path)) if path else "root"
|
||||
return f"{prefix}_{path_str}" if prefix else path_str
|
||||
|
||||
def _create_grip(self, parent_tag: str, path: list[int], grip_index: int, orientation: int):
|
||||
grip_tag = self._path_to_tag(path, f"grip_{grip_index}")
|
||||
handler_tag = f"{grip_tag}_handler"
|
||||
width, height = [(self.grip_size, -1), (-1, self.grip_size)][orientation]
|
||||
|
||||
with dpg.child_window(tag=grip_tag, parent=parent_tag, width=width, height=height, no_scrollbar=True, border=False):
|
||||
button_tag = dpg.add_button(label="", width=-1, height=-1)
|
||||
|
||||
with dpg.item_handler_registry(tag=handler_tag):
|
||||
user_data = (path, grip_index, orientation)
|
||||
dpg.add_item_active_handler(callback=self._on_grip_drag, user_data=user_data)
|
||||
dpg.add_item_deactivated_handler(callback=self._on_grip_end, user_data=user_data)
|
||||
dpg.bind_item_handler_registry(button_tag, handler_tag)
|
||||
self._created_handler_tags.add(handler_tag)
|
||||
|
||||
def _on_grip_drag(self, sender, app_data, user_data):
|
||||
path, grip_index, orientation = user_data
|
||||
layout = self._get_layout_at_path(path)
|
||||
|
||||
if "_drag_data" not in layout:
|
||||
layout["_drag_data"] = {"initial_proportions": layout["proportions"][:], "start_mouse": dpg.get_mouse_pos(local=False)[orientation]}
|
||||
return
|
||||
|
||||
drag_data = layout["_drag_data"]
|
||||
split_tag = self._path_to_tag(path, "split")
|
||||
if not dpg.does_item_exist(split_tag):
|
||||
return
|
||||
|
||||
_, usable_size, _ = self._get_split_geometry(layout, dpg.get_item_rect_size(split_tag))
|
||||
current_coord = dpg.get_mouse_pos(local=False)[orientation]
|
||||
delta = current_coord - drag_data["start_mouse"]
|
||||
delta_prop = delta / usable_size
|
||||
|
||||
left_idx = grip_index
|
||||
right_idx = left_idx + 1
|
||||
initial = drag_data["initial_proportions"]
|
||||
min_prop = self.min_pane_size / usable_size
|
||||
|
||||
new_left = max(min_prop, initial[left_idx] + delta_prop)
|
||||
new_right = max(min_prop, initial[right_idx] - delta_prop)
|
||||
|
||||
total_available = initial[left_idx] + initial[right_idx]
|
||||
if new_left + new_right > total_available:
|
||||
if new_left > new_right:
|
||||
new_left = total_available - new_right
|
||||
else:
|
||||
new_right = total_available - new_left
|
||||
|
||||
layout["proportions"] = initial[:]
|
||||
layout["proportions"][left_idx] = new_left
|
||||
layout["proportions"][right_idx] = new_right
|
||||
|
||||
self._resize_splits_recursive(layout, path)
|
||||
|
||||
def _on_grip_end(self, sender, app_data, user_data):
|
||||
path, _, _ = user_data
|
||||
self._get_layout_at_path(path).pop("_drag_data", None)
|
||||
@@ -0,0 +1,128 @@
|
||||
tabs:
|
||||
'0':
|
||||
name: Lateral Plan Conformance
|
||||
panel_layout:
|
||||
type: split
|
||||
orientation: 1
|
||||
proportions:
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
children:
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: desired vs actual
|
||||
series_paths:
|
||||
- controlsState/lateralControlState/torqueState/desiredLateralAccel
|
||||
- controlsState/lateralControlState/torqueState/actualLateralAccel
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: ff vs output
|
||||
series_paths:
|
||||
- controlsState/lateralControlState/torqueState/f
|
||||
- carState/steeringPressed
|
||||
- carControl/actuators/torque
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: vehicle speed
|
||||
series_paths:
|
||||
- carState/vEgo
|
||||
'1':
|
||||
name: Actuator Performance
|
||||
panel_layout:
|
||||
type: split
|
||||
orientation: 1
|
||||
proportions:
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
children:
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: calc vs learned latAccelFactor
|
||||
series_paths:
|
||||
- liveTorqueParameters/latAccelFactorFiltered
|
||||
- liveTorqueParameters/latAccelFactorRaw
|
||||
- carParams/lateralTuning/torque/latAccelFactor
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: learned latAccelOffset
|
||||
series_paths:
|
||||
- liveTorqueParameters/latAccelOffsetRaw
|
||||
- liveTorqueParameters/latAccelOffsetFiltered
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: calc vs learned friction
|
||||
series_paths:
|
||||
- liveTorqueParameters/frictionCoefficientFiltered
|
||||
- liveTorqueParameters/frictionCoefficientRaw
|
||||
- carParams/lateralTuning/torque/friction
|
||||
'2':
|
||||
name: Vehicle Dynamics
|
||||
panel_layout:
|
||||
type: split
|
||||
orientation: 1
|
||||
proportions:
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
children:
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: initial vs learned steerRatio
|
||||
series_paths:
|
||||
- carParams/steerRatio
|
||||
- liveParameters/steerRatio
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: initial vs learned tireStiffnessFactor
|
||||
series_paths:
|
||||
- carParams/tireStiffnessFactor
|
||||
- liveParameters/stiffnessFactor
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: live steering angle offsets
|
||||
series_paths:
|
||||
- liveParameters/angleOffsetDeg
|
||||
- liveParameters/angleOffsetAverageDeg
|
||||
'3':
|
||||
name: Controller PIF Terms
|
||||
panel_layout:
|
||||
type: split
|
||||
orientation: 1
|
||||
proportions:
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
- 0.3333333333333333
|
||||
children:
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: ff vs output
|
||||
series_paths:
|
||||
- carControl/actuators/torque
|
||||
- controlsState/lateralControlState/torqueState/f
|
||||
- carState/steeringPressed
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: PIF terms
|
||||
series_paths:
|
||||
- controlsState/lateralControlState/torqueState/f
|
||||
- controlsState/lateralControlState/torqueState/p
|
||||
- controlsState/lateralControlState/torqueState/i
|
||||
- type: panel
|
||||
panel:
|
||||
type: timeseries
|
||||
title: road roll angle
|
||||
series_paths:
|
||||
- liveParameters/roll
|
||||
Executable
+370
@@ -0,0 +1,370 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import os
|
||||
import pyautogui
|
||||
import subprocess
|
||||
import dearpygui.dearpygui as dpg
|
||||
import multiprocessing
|
||||
import uuid
|
||||
import signal
|
||||
import yaml # type: ignore
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.tools.jotpluggler.data import DataManager
|
||||
from openpilot.tools.jotpluggler.datatree import DataTree
|
||||
from openpilot.tools.jotpluggler.layout import LayoutManager
|
||||
|
||||
DEMO_ROUTE = "a2a0ccea32023010|2023-07-27--13-01-19"
|
||||
|
||||
|
||||
class WorkerManager:
|
||||
def __init__(self, max_workers=None):
|
||||
self.pool = multiprocessing.Pool(max_workers or min(4, multiprocessing.cpu_count()), initializer=WorkerManager.worker_initializer)
|
||||
self.active_tasks = {}
|
||||
|
||||
def submit_task(self, func, args_list, callback=None, task_id=None):
|
||||
task_id = task_id or str(uuid.uuid4())
|
||||
|
||||
if task_id in self.active_tasks:
|
||||
try:
|
||||
self.active_tasks[task_id].terminate()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def handle_success(result):
|
||||
self.active_tasks.pop(task_id, None)
|
||||
if callback:
|
||||
try:
|
||||
callback(result)
|
||||
except Exception as e:
|
||||
print(f"Callback for task {task_id} failed: {e}")
|
||||
|
||||
def handle_error(error):
|
||||
self.active_tasks.pop(task_id, None)
|
||||
print(f"Task {task_id} failed: {error}")
|
||||
|
||||
async_result = self.pool.starmap_async(func, args_list, callback=handle_success, error_callback=handle_error)
|
||||
self.active_tasks[task_id] = async_result
|
||||
return task_id
|
||||
|
||||
@staticmethod
|
||||
def worker_initializer():
|
||||
signal.signal(signal.SIGINT, signal.SIG_IGN)
|
||||
|
||||
def shutdown(self):
|
||||
for task in self.active_tasks.values():
|
||||
try:
|
||||
task.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
self.pool.terminate()
|
||||
self.pool.join()
|
||||
|
||||
|
||||
class PlaybackManager:
|
||||
def __init__(self):
|
||||
self.is_playing = False
|
||||
self.current_time_s = 0.0
|
||||
self.duration_s = 0.0
|
||||
self.num_segments = 0
|
||||
|
||||
self.x_axis_bounds = (0.0, 0.0) # (min_time, max_time)
|
||||
self.x_axis_observers = [] # callbacks for x-axis changes
|
||||
self._updating_x_axis = False
|
||||
|
||||
def set_route_duration(self, duration: float):
|
||||
self.duration_s = duration
|
||||
self.seek(min(self.current_time_s, duration))
|
||||
|
||||
def toggle_play_pause(self):
|
||||
if not self.is_playing and self.current_time_s >= self.duration_s:
|
||||
self.seek(0.0)
|
||||
self.is_playing = not self.is_playing
|
||||
texture_tag = "pause_texture" if self.is_playing else "play_texture"
|
||||
dpg.configure_item("play_pause_button", texture_tag=texture_tag)
|
||||
|
||||
def seek(self, time_s: float):
|
||||
self.current_time_s = max(0.0, min(time_s, self.duration_s))
|
||||
|
||||
def update_time(self, delta_t: float):
|
||||
if self.is_playing:
|
||||
self.current_time_s = min(self.current_time_s + delta_t, self.duration_s)
|
||||
if self.current_time_s >= self.duration_s:
|
||||
self.is_playing = False
|
||||
dpg.configure_item("play_pause_button", texture_tag="play_texture")
|
||||
return self.current_time_s
|
||||
|
||||
def set_x_axis_bounds(self, min_time: float, max_time: float, source_panel=None):
|
||||
if self._updating_x_axis:
|
||||
return
|
||||
|
||||
new_bounds = (min_time, max_time)
|
||||
if new_bounds == self.x_axis_bounds:
|
||||
return
|
||||
|
||||
self.x_axis_bounds = new_bounds
|
||||
self._updating_x_axis = True # prevent recursive updates
|
||||
|
||||
try:
|
||||
for callback in self.x_axis_observers:
|
||||
try:
|
||||
callback(min_time, max_time, source_panel)
|
||||
except Exception as e:
|
||||
print(f"Error in x-axis sync callback: {e}")
|
||||
finally:
|
||||
self._updating_x_axis = False
|
||||
|
||||
def add_x_axis_observer(self, callback):
|
||||
if callback not in self.x_axis_observers:
|
||||
self.x_axis_observers.append(callback)
|
||||
|
||||
def remove_x_axis_observer(self, callback):
|
||||
if callback in self.x_axis_observers:
|
||||
self.x_axis_observers.remove(callback)
|
||||
|
||||
class MainController:
|
||||
def __init__(self, scale: float = 1.0):
|
||||
self.scale = scale
|
||||
self.data_manager = DataManager()
|
||||
self.playback_manager = PlaybackManager()
|
||||
self.worker_manager = WorkerManager()
|
||||
self._create_global_themes()
|
||||
self.data_tree = DataTree(self.data_manager, self.playback_manager)
|
||||
self.layout_manager = LayoutManager(self.data_manager, self.playback_manager, self.worker_manager, scale=self.scale)
|
||||
self.data_manager.add_observer(self.on_data_loaded)
|
||||
self._total_segments = 0
|
||||
|
||||
def _create_global_themes(self):
|
||||
with dpg.theme(tag="line_theme"):
|
||||
with dpg.theme_component(dpg.mvLineSeries):
|
||||
scaled_thickness = max(1.0, self.scale)
|
||||
dpg.add_theme_style(dpg.mvPlotStyleVar_LineWeight, scaled_thickness, category=dpg.mvThemeCat_Plots)
|
||||
|
||||
with dpg.theme(tag="timeline_theme"):
|
||||
with dpg.theme_component(dpg.mvInfLineSeries):
|
||||
scaled_thickness = max(1.0, self.scale)
|
||||
dpg.add_theme_style(dpg.mvPlotStyleVar_LineWeight, scaled_thickness, category=dpg.mvThemeCat_Plots)
|
||||
dpg.add_theme_color(dpg.mvPlotCol_Line, (255, 0, 0, 128), category=dpg.mvThemeCat_Plots)
|
||||
|
||||
for tag, color in (("active_tab_theme", (37, 37, 38, 255)), ("inactive_tab_theme", (70, 70, 75, 255))):
|
||||
with dpg.theme(tag=tag):
|
||||
for cmp, target in ((dpg.mvChildWindow, dpg.mvThemeCol_ChildBg), (dpg.mvInputText, dpg.mvThemeCol_FrameBg), (dpg.mvImageButton, dpg.mvThemeCol_Button)):
|
||||
with dpg.theme_component(cmp):
|
||||
dpg.add_theme_color(target, color)
|
||||
|
||||
with dpg.theme(tag="tab_bar_theme"):
|
||||
with dpg.theme_component(dpg.mvChildWindow):
|
||||
dpg.add_theme_color(dpg.mvThemeCol_ChildBg, (51, 51, 55, 255))
|
||||
|
||||
def on_data_loaded(self, data: dict):
|
||||
duration = data.get('duration', 0.0)
|
||||
self.playback_manager.set_route_duration(duration)
|
||||
|
||||
if data.get('metadata_loaded'):
|
||||
self.playback_manager.num_segments = data.get('total_segments', 0)
|
||||
self._total_segments = data.get('total_segments', 0)
|
||||
dpg.set_value("load_status", f"Loading... 0/{self._total_segments} segments processed")
|
||||
elif data.get('reset'):
|
||||
self.playback_manager.current_time_s = 0.0
|
||||
self.playback_manager.duration_s = 0.0
|
||||
self.playback_manager.is_playing = False
|
||||
self._total_segments = 0
|
||||
dpg.set_value("load_status", "Loading...")
|
||||
dpg.set_value("timeline_slider", 0.0)
|
||||
dpg.configure_item("timeline_slider", max_value=0.0)
|
||||
dpg.configure_item("play_pause_button", texture_tag="play_texture")
|
||||
dpg.configure_item("load_button", enabled=True)
|
||||
elif data.get('loading_complete'):
|
||||
num_paths = len(self.data_manager.get_all_paths())
|
||||
dpg.set_value("load_status", f"Loaded {num_paths} data paths")
|
||||
dpg.configure_item("load_button", enabled=True)
|
||||
elif data.get('segment_added'):
|
||||
segment_count = data.get('segment_count', 0)
|
||||
dpg.set_value("load_status", f"Loading... {segment_count}/{self._total_segments} segments processed")
|
||||
|
||||
dpg.configure_item("timeline_slider", max_value=duration)
|
||||
|
||||
def save_layout_to_yaml(self, filepath: str):
|
||||
layout_dict = self.layout_manager.to_dict()
|
||||
with open(filepath, 'w') as f:
|
||||
yaml.dump(layout_dict, f, default_flow_style=False, sort_keys=False)
|
||||
|
||||
def load_layout_from_yaml(self, filepath: str):
|
||||
with open(filepath) as f:
|
||||
layout_dict = yaml.safe_load(f)
|
||||
self.layout_manager.clear_and_load_from_dict(layout_dict)
|
||||
self.layout_manager.create_ui("main_plot_area")
|
||||
|
||||
def save_layout_dialog(self):
|
||||
if dpg.does_item_exist("save_layout_dialog"):
|
||||
dpg.delete_item("save_layout_dialog")
|
||||
with dpg.file_dialog(
|
||||
callback=self._save_layout_callback, tag="save_layout_dialog", width=int(700 * self.scale), height=int(400 * self.scale),
|
||||
default_filename="layout", default_path=os.path.join(os.path.dirname(os.path.realpath(__file__)), "layouts")
|
||||
):
|
||||
dpg.add_file_extension(".yaml")
|
||||
|
||||
def load_layout_dialog(self):
|
||||
if dpg.does_item_exist("load_layout_dialog"):
|
||||
dpg.delete_item("load_layout_dialog")
|
||||
with dpg.file_dialog(
|
||||
callback=self._load_layout_callback, tag="load_layout_dialog", width=int(700 * self.scale), height=int(400 * self.scale),
|
||||
default_path=os.path.join(os.path.dirname(os.path.realpath(__file__)), "layouts")
|
||||
):
|
||||
dpg.add_file_extension(".yaml")
|
||||
|
||||
def _save_layout_callback(self, sender, app_data):
|
||||
filepath = app_data['file_path_name']
|
||||
try:
|
||||
self.save_layout_to_yaml(filepath)
|
||||
dpg.set_value("load_status", f"Layout saved to {os.path.basename(filepath)}")
|
||||
except Exception:
|
||||
dpg.set_value("load_status", "Error saving layout")
|
||||
cloudlog.exception(f"Error saving layout to {filepath}")
|
||||
dpg.delete_item("save_layout_dialog")
|
||||
|
||||
def _load_layout_callback(self, sender, app_data):
|
||||
filepath = app_data['file_path_name']
|
||||
try:
|
||||
self.load_layout_from_yaml(filepath)
|
||||
dpg.set_value("load_status", f"Layout loaded from {os.path.basename(filepath)}")
|
||||
except Exception:
|
||||
dpg.set_value("load_status", "Error loading layout")
|
||||
cloudlog.exception(f"Error loading layout from {filepath}:")
|
||||
dpg.delete_item("load_layout_dialog")
|
||||
|
||||
def setup_ui(self):
|
||||
with dpg.texture_registry():
|
||||
script_dir = os.path.dirname(os.path.realpath(__file__))
|
||||
for image in ["play", "pause", "x", "split_h", "split_v", "plus"]:
|
||||
texture = dpg.load_image(os.path.join(script_dir, "assets", f"{image}.png"))
|
||||
dpg.add_static_texture(width=texture[0], height=texture[1], default_value=texture[3], tag=f"{image}_texture")
|
||||
|
||||
with dpg.window(tag="Primary Window"):
|
||||
with dpg.group(horizontal=True):
|
||||
# Left panel - Data tree
|
||||
with dpg.child_window(label="Sidebar", width=int(300 * self.scale), tag="sidebar_window", border=True, resizable_x=True):
|
||||
with dpg.group(horizontal=True):
|
||||
dpg.add_input_text(tag="route_input", width=int(-75 * self.scale), hint="Enter route name...")
|
||||
dpg.add_button(label="Load", callback=self.load_route, tag="load_button", width=-1)
|
||||
dpg.add_text("Ready to load route", tag="load_status")
|
||||
dpg.add_separator()
|
||||
|
||||
with dpg.table(header_row=False, policy=dpg.mvTable_SizingStretchProp):
|
||||
dpg.add_table_column(init_width_or_weight=0.5)
|
||||
dpg.add_table_column(init_width_or_weight=0.5)
|
||||
with dpg.table_row():
|
||||
dpg.add_button(label="Save Layout", callback=self.save_layout_dialog, width=-1)
|
||||
dpg.add_button(label="Load Layout", callback=self.load_layout_dialog, width=-1)
|
||||
dpg.add_separator()
|
||||
|
||||
self.data_tree.create_ui("sidebar_window")
|
||||
|
||||
# Right panel - Plots and timeline
|
||||
with dpg.group(tag="right_panel"):
|
||||
with dpg.child_window(label="Plot Window", border=True, height=int(-(32 + 13 * self.scale)), tag="main_plot_area"):
|
||||
self.layout_manager.create_ui("main_plot_area")
|
||||
|
||||
with dpg.child_window(label="Timeline", border=True):
|
||||
with dpg.table(header_row=False):
|
||||
btn_size = int(13 * self.scale)
|
||||
dpg.add_table_column(width_fixed=True, init_width_or_weight=(btn_size + 8)) # Play button
|
||||
dpg.add_table_column(width_stretch=True) # Timeline slider
|
||||
dpg.add_table_column(width_fixed=True, init_width_or_weight=int(50 * self.scale)) # FPS counter
|
||||
with dpg.table_row():
|
||||
dpg.add_image_button(texture_tag="play_texture", tag="play_pause_button", callback=self.toggle_play_pause, width=btn_size, height=btn_size)
|
||||
dpg.add_slider_float(tag="timeline_slider", default_value=0.0, label="", width=-1, callback=self.timeline_drag)
|
||||
dpg.add_text("", tag="fps_counter")
|
||||
with dpg.item_handler_registry(tag="plot_resize_handler"):
|
||||
dpg.add_item_resize_handler(callback=self.on_plot_resize)
|
||||
dpg.bind_item_handler_registry("right_panel", "plot_resize_handler")
|
||||
|
||||
dpg.set_primary_window("Primary Window", True)
|
||||
|
||||
def on_plot_resize(self, sender, app_data, user_data):
|
||||
self.layout_manager.on_viewport_resize()
|
||||
|
||||
def load_route(self):
|
||||
route_name = dpg.get_value("route_input").strip()
|
||||
if route_name:
|
||||
dpg.set_value("load_status", "Loading route...")
|
||||
dpg.configure_item("load_button", enabled=False)
|
||||
self.data_manager.load_route(route_name)
|
||||
|
||||
def toggle_play_pause(self, sender):
|
||||
self.playback_manager.toggle_play_pause()
|
||||
|
||||
def timeline_drag(self, sender, app_data):
|
||||
self.playback_manager.seek(app_data)
|
||||
|
||||
def update_frame(self, font):
|
||||
self.data_tree.update_frame(font)
|
||||
|
||||
new_time = self.playback_manager.update_time(dpg.get_delta_time())
|
||||
if not dpg.is_item_active("timeline_slider"):
|
||||
dpg.set_value("timeline_slider", new_time)
|
||||
|
||||
self.layout_manager.update_all_panels()
|
||||
|
||||
dpg.set_value("fps_counter", f"{dpg.get_frame_rate():.1f} FPS")
|
||||
|
||||
def shutdown(self):
|
||||
self.worker_manager.shutdown()
|
||||
|
||||
|
||||
def main(route_to_load=None, layout_to_load=None):
|
||||
dpg.create_context()
|
||||
|
||||
# TODO: find better way of calculating display scaling
|
||||
try:
|
||||
w, h = next(tuple(map(int, l.split()[0].split('x'))) for l in subprocess.check_output(['xrandr']).decode().split('\n') if '*' in l) # actual resolution
|
||||
scale = pyautogui.size()[0] / w # scaled resolution
|
||||
except Exception:
|
||||
scale = 1
|
||||
|
||||
with dpg.font_registry():
|
||||
default_font = dpg.add_font(os.path.join(BASEDIR, "selfdrive/assets/fonts/JetBrainsMono-Medium.ttf"), int(13 * scale * 2)) # 2x then scale for hidpi
|
||||
dpg.bind_font(default_font)
|
||||
dpg.set_global_font_scale(0.5)
|
||||
|
||||
viewport_width, viewport_height = int(1200 * scale), int(800 * scale)
|
||||
mouse_x, mouse_y = pyautogui.position() # TODO: find better way of creating the window where the user is (default dpg behavior annoying on multiple displays)
|
||||
dpg.create_viewport(
|
||||
title='JotPluggler', width=viewport_width, height=viewport_height, x_pos=mouse_x - viewport_width // 2, y_pos=mouse_y - viewport_height // 2
|
||||
)
|
||||
dpg.setup_dearpygui()
|
||||
|
||||
controller = MainController(scale=scale)
|
||||
controller.setup_ui()
|
||||
|
||||
if layout_to_load:
|
||||
try:
|
||||
controller.load_layout_from_yaml(layout_to_load)
|
||||
print(f"Loaded layout from {layout_to_load}")
|
||||
except Exception as e:
|
||||
print(f"Failed to load layout from {layout_to_load}: {e}")
|
||||
cloudlog.exception(f"Error loading layout from {layout_to_load}")
|
||||
|
||||
if route_to_load:
|
||||
dpg.set_value("route_input", route_to_load)
|
||||
controller.load_route()
|
||||
|
||||
dpg.show_viewport()
|
||||
|
||||
# Main loop
|
||||
try:
|
||||
while dpg.is_dearpygui_running():
|
||||
controller.update_frame(default_font)
|
||||
dpg.render_dearpygui_frame()
|
||||
finally:
|
||||
controller.shutdown()
|
||||
dpg.destroy_context()
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="A tool for visualizing openpilot logs.")
|
||||
parser.add_argument("--demo", action="store_true", help="Use the demo route instead of providing one")
|
||||
parser.add_argument("--layout", type=str, help="Path to YAML layout file to load on startup")
|
||||
parser.add_argument("route", nargs='?', default=None, help="Optional route name to load on startup.")
|
||||
args = parser.parse_args()
|
||||
route = DEMO_ROUTE if args.demo else args.route
|
||||
main(route_to_load=route, layout_to_load=args.layout)
|
||||
@@ -0,0 +1,294 @@
|
||||
import uuid
|
||||
import threading
|
||||
import numpy as np
|
||||
from collections import deque
|
||||
import dearpygui.dearpygui as dpg
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class ViewPanel(ABC):
|
||||
"""Abstract base class for all view panels that can be displayed in a plot container"""
|
||||
|
||||
def __init__(self, panel_id: str = None):
|
||||
self.panel_id = panel_id or str(uuid.uuid4())
|
||||
self.title = "Untitled Panel"
|
||||
|
||||
@abstractmethod
|
||||
def clear(self):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def create_ui(self, parent_tag: str):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def destroy_ui(self):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_panel_type(self) -> str:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def update(self):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def to_dict(self) -> dict:
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
@abstractmethod
|
||||
def load_from_dict(cls, data: dict, data_manager, playback_manager, worker_manager):
|
||||
pass
|
||||
|
||||
|
||||
class TimeSeriesPanel(ViewPanel):
|
||||
def __init__(self, data_manager, playback_manager, worker_manager, panel_id: str | None = None):
|
||||
super().__init__(panel_id)
|
||||
self.data_manager = data_manager
|
||||
self.playback_manager = playback_manager
|
||||
self.worker_manager = worker_manager
|
||||
self.title = "Time Series Plot"
|
||||
self.plot_tag = f"plot_{self.panel_id}"
|
||||
self.x_axis_tag = f"{self.plot_tag}_x_axis"
|
||||
self.y_axis_tag = f"{self.plot_tag}_y_axis"
|
||||
self.timeline_indicator_tag = f"{self.plot_tag}_timeline"
|
||||
self._ui_created = False
|
||||
self._series_data: dict[str, tuple[np.ndarray, np.ndarray]] = {}
|
||||
self._last_plot_duration = 0
|
||||
self._update_lock = threading.RLock()
|
||||
self._results_deque: deque[tuple[str, list, list]] = deque()
|
||||
self._new_data = False
|
||||
self._last_x_limits = (0.0, 0.0)
|
||||
self._queued_x_sync: tuple | None = None
|
||||
self._queued_reallow_x_zoom = False
|
||||
self._total_segments = self.playback_manager.num_segments
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"type": "timeseries",
|
||||
"title": self.title,
|
||||
"series_paths": list(self._series_data.keys())
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def load_from_dict(cls, data: dict, data_manager, playback_manager, worker_manager):
|
||||
panel = cls(data_manager, playback_manager, worker_manager)
|
||||
panel.title = data.get("title", "Time Series Plot")
|
||||
panel._series_data = {path: (np.array([]), np.array([])) for path in data.get("series_paths", [])}
|
||||
return panel
|
||||
|
||||
def create_ui(self, parent_tag: str):
|
||||
self.data_manager.add_observer(self.on_data_loaded)
|
||||
self.playback_manager.add_x_axis_observer(self._on_x_axis_sync)
|
||||
with dpg.plot(height=-1, width=-1, tag=self.plot_tag, parent=parent_tag, drop_callback=self._on_series_drop, payload_type="TIMESERIES_PAYLOAD"):
|
||||
dpg.add_plot_legend()
|
||||
dpg.add_plot_axis(dpg.mvXAxis, no_label=True, tag=self.x_axis_tag)
|
||||
dpg.add_plot_axis(dpg.mvYAxis, no_label=True, tag=self.y_axis_tag)
|
||||
timeline_series_tag = dpg.add_inf_line_series(x=[0], label="Timeline", parent=self.y_axis_tag, tag=self.timeline_indicator_tag)
|
||||
dpg.bind_item_theme(timeline_series_tag, "timeline_theme")
|
||||
|
||||
self._new_data = True
|
||||
self._queued_x_sync = self.playback_manager.x_axis_bounds
|
||||
self._ui_created = True
|
||||
|
||||
def update(self):
|
||||
with self._update_lock:
|
||||
if not self._ui_created:
|
||||
return
|
||||
|
||||
if self._queued_x_sync:
|
||||
min_time, max_time = self._queued_x_sync
|
||||
self._queued_x_sync = None
|
||||
dpg.set_axis_limits(self.x_axis_tag, min_time, max_time)
|
||||
self._last_x_limits = (min_time, max_time)
|
||||
self._fit_y_axis(min_time, max_time)
|
||||
self._queued_reallow_x_zoom = True # must wait a frame before allowing user changes so that axis limits take effect
|
||||
return
|
||||
|
||||
if self._queued_reallow_x_zoom:
|
||||
self._queued_reallow_x_zoom = False
|
||||
if tuple(dpg.get_axis_limits(self.x_axis_tag)) == self._last_x_limits:
|
||||
dpg.set_axis_limits_auto(self.x_axis_tag)
|
||||
else:
|
||||
self._queued_x_sync = self._last_x_limits # retry, likely too early
|
||||
return
|
||||
|
||||
if self._new_data: # handle new data in main thread
|
||||
self._new_data = False
|
||||
if self._total_segments > 0:
|
||||
dpg.set_axis_limits_constraints(self.x_axis_tag, -10, self._total_segments * 60 + 10)
|
||||
self._fit_y_axis(*dpg.get_axis_limits(self.x_axis_tag))
|
||||
for series_path in list(self._series_data.keys()):
|
||||
self.add_series(series_path, update=True)
|
||||
|
||||
current_limits = dpg.get_axis_limits(self.x_axis_tag)
|
||||
# downsample if plot zoom changed significantly
|
||||
plot_duration = current_limits[1] - current_limits[0]
|
||||
if plot_duration > self._last_plot_duration * 2 or plot_duration < self._last_plot_duration * 0.5:
|
||||
self._downsample_all_series(plot_duration)
|
||||
# sync x-axis if changed by user
|
||||
if self._last_x_limits != current_limits:
|
||||
self.playback_manager.set_x_axis_bounds(current_limits[0], current_limits[1], source_panel=self)
|
||||
self._last_x_limits = current_limits
|
||||
self._fit_y_axis(current_limits[0], current_limits[1])
|
||||
|
||||
while self._results_deque: # handle downsampled results in main thread
|
||||
results = self._results_deque.popleft()
|
||||
for series_path, downsampled_time, downsampled_values in results:
|
||||
series_tag = f"series_{self.panel_id}_{series_path}"
|
||||
if dpg.does_item_exist(series_tag):
|
||||
dpg.set_value(series_tag, (downsampled_time, downsampled_values.astype(float)))
|
||||
|
||||
# update timeline
|
||||
current_time_s = self.playback_manager.current_time_s
|
||||
dpg.set_value(self.timeline_indicator_tag, [[current_time_s], [0]])
|
||||
|
||||
# update timeseries legend label
|
||||
for series_path, (time_array, value_array) in self._series_data.items():
|
||||
position = np.searchsorted(time_array, current_time_s, side='right') - 1
|
||||
if position >= 0 and (current_time_s - time_array[position]) <= 1.0:
|
||||
value = value_array[position]
|
||||
formatted_value = f"{value:.5f}" if np.issubdtype(type(value), np.floating) else str(value)
|
||||
series_tag = f"series_{self.panel_id}_{series_path}"
|
||||
if dpg.does_item_exist(series_tag):
|
||||
dpg.configure_item(series_tag, label=f"{series_path}: {formatted_value}")
|
||||
|
||||
def _on_x_axis_sync(self, min_time: float, max_time: float, source_panel):
|
||||
with self._update_lock:
|
||||
if source_panel != self:
|
||||
self._queued_x_sync = (min_time, max_time)
|
||||
|
||||
def _fit_y_axis(self, x_min: float, x_max: float):
|
||||
if not self._series_data:
|
||||
dpg.set_axis_limits(self.y_axis_tag, -1, 1)
|
||||
return
|
||||
|
||||
global_min = float('inf')
|
||||
global_max = float('-inf')
|
||||
found_data = False
|
||||
|
||||
for time_array, value_array in self._series_data.values():
|
||||
if len(time_array) == 0:
|
||||
continue
|
||||
start_idx, end_idx = np.searchsorted(time_array, [x_min, x_max])
|
||||
end_idx = min(end_idx, len(time_array) - 1)
|
||||
if start_idx <= end_idx:
|
||||
y_slice = value_array[start_idx:end_idx + 1]
|
||||
series_min, series_max = np.min(y_slice), np.max(y_slice)
|
||||
global_min = min(global_min, series_min)
|
||||
global_max = max(global_max, series_max)
|
||||
found_data = True
|
||||
|
||||
if not found_data:
|
||||
dpg.set_axis_limits(self.y_axis_tag, -1, 1)
|
||||
return
|
||||
|
||||
if global_min == global_max:
|
||||
padding = max(abs(global_min) * 0.1, 1.0)
|
||||
y_min, y_max = global_min - padding, global_max + padding
|
||||
else:
|
||||
range_size = global_max - global_min
|
||||
padding = range_size * 0.1
|
||||
y_min, y_max = global_min - padding, global_max + padding
|
||||
|
||||
dpg.set_axis_limits(self.y_axis_tag, y_min, y_max)
|
||||
|
||||
def _downsample_all_series(self, plot_duration):
|
||||
plot_width = dpg.get_item_rect_size(self.plot_tag)[0]
|
||||
if plot_width <= 0 or plot_duration <= 0:
|
||||
return
|
||||
|
||||
self._last_plot_duration = plot_duration
|
||||
target_points_per_second = plot_width / plot_duration
|
||||
work_items = []
|
||||
for series_path, (time_array, value_array) in self._series_data.items():
|
||||
if len(time_array) == 0:
|
||||
continue
|
||||
series_duration = time_array[-1] - time_array[0] if len(time_array) > 1 else 1
|
||||
points_per_second = len(time_array) / series_duration
|
||||
if points_per_second > target_points_per_second * 2:
|
||||
target_points = max(int(target_points_per_second * series_duration), plot_width)
|
||||
work_items.append((series_path, time_array, value_array, target_points))
|
||||
elif dpg.does_item_exist(f"series_{self.panel_id}_{series_path}"):
|
||||
dpg.set_value(f"series_{self.panel_id}_{series_path}", (time_array, value_array.astype(float)))
|
||||
|
||||
if work_items:
|
||||
self.worker_manager.submit_task(
|
||||
TimeSeriesPanel._downsample_worker, work_items, callback=lambda results: self._results_deque.append(results), task_id=f"downsample_{self.panel_id}"
|
||||
)
|
||||
|
||||
def add_series(self, series_path: str, update: bool = False):
|
||||
with self._update_lock:
|
||||
if update or series_path not in self._series_data:
|
||||
self._series_data[series_path] = self.data_manager.get_timeseries(series_path)
|
||||
|
||||
time_array, value_array = self._series_data[series_path]
|
||||
series_tag = f"series_{self.panel_id}_{series_path}"
|
||||
if dpg.does_item_exist(series_tag):
|
||||
dpg.set_value(series_tag, (time_array, value_array.astype(float)))
|
||||
else:
|
||||
line_series_tag = dpg.add_line_series(x=time_array, y=value_array.astype(float), label=series_path, parent=self.y_axis_tag, tag=series_tag)
|
||||
dpg.bind_item_theme(line_series_tag, "line_theme")
|
||||
self._fit_y_axis(*dpg.get_axis_limits(self.x_axis_tag))
|
||||
plot_duration = dpg.get_axis_limits(self.x_axis_tag)[1] - dpg.get_axis_limits(self.x_axis_tag)[0]
|
||||
self._downsample_all_series(plot_duration)
|
||||
|
||||
def destroy_ui(self):
|
||||
with self._update_lock:
|
||||
self.data_manager.remove_observer(self.on_data_loaded)
|
||||
self.playback_manager.remove_x_axis_observer(self._on_x_axis_sync)
|
||||
if dpg.does_item_exist(self.plot_tag):
|
||||
dpg.delete_item(self.plot_tag)
|
||||
self._ui_created = False
|
||||
|
||||
def get_panel_type(self) -> str:
|
||||
return "timeseries"
|
||||
|
||||
def clear(self):
|
||||
with self._update_lock:
|
||||
for series_path in list(self._series_data.keys()):
|
||||
self.remove_series(series_path)
|
||||
|
||||
def remove_series(self, series_path: str):
|
||||
with self._update_lock:
|
||||
if series_path in self._series_data:
|
||||
if dpg.does_item_exist(f"series_{self.panel_id}_{series_path}"):
|
||||
dpg.delete_item(f"series_{self.panel_id}_{series_path}")
|
||||
del self._series_data[series_path]
|
||||
|
||||
def on_data_loaded(self, data: dict):
|
||||
with self._update_lock:
|
||||
self._new_data = True
|
||||
if data.get('metadata_loaded'):
|
||||
self._total_segments = data.get('total_segments', 0)
|
||||
limits = (-10, self._total_segments * 60 + 10)
|
||||
self._queued_x_sync = limits
|
||||
|
||||
def _on_series_drop(self, sender, app_data, user_data):
|
||||
self.add_series(app_data)
|
||||
|
||||
@staticmethod
|
||||
def _downsample_worker(series_path, time_array, value_array, target_points):
|
||||
if len(time_array) <= target_points:
|
||||
return series_path, time_array, value_array
|
||||
|
||||
step = len(time_array) / target_points
|
||||
indices = []
|
||||
|
||||
for i in range(target_points):
|
||||
start_idx = int(i * step)
|
||||
end_idx = int((i + 1) * step)
|
||||
if start_idx == end_idx:
|
||||
indices.append(start_idx)
|
||||
else:
|
||||
bucket_values = value_array[start_idx:end_idx]
|
||||
min_idx = start_idx + np.argmin(bucket_values)
|
||||
max_idx = start_idx + np.argmax(bucket_values)
|
||||
if min_idx != max_idx:
|
||||
indices.extend([min(min_idx, max_idx), max(min_idx, max_idx)])
|
||||
else:
|
||||
indices.append(min_idx)
|
||||
indices = sorted(set(indices))
|
||||
return series_path, time_array[indices], value_array[indices]
|
||||
@@ -0,0 +1,53 @@
|
||||
# Joystick
|
||||
|
||||
**Hardware needed**: device running openpilot, laptop, joystick (optional)
|
||||
|
||||
With joystick_control, you can connect your laptop to your comma device over the network and debug controls using a joystick or keyboard.
|
||||
joystick_control uses [inputs](https://pypi.org/project/inputs) which supports many common gamepads and joysticks.
|
||||
|
||||
## Usage
|
||||
|
||||
The car must be off, and openpilot must be offroad before starting `joystick_control`.
|
||||
|
||||
### Using a keyboard
|
||||
|
||||
SSH into your comma device and start joystick_control with the following command:
|
||||
|
||||
```shell
|
||||
tools/joystick/joystick_control.py --keyboard
|
||||
```
|
||||
|
||||
The available buttons and axes will print showing their key mappings. In general, the WASD keys control gas and brakes and steering torque in 5% increments.
|
||||
|
||||
### Joystick on your comma three
|
||||
|
||||
Plug the joystick into your comma three aux USB-C port. Then, SSH into the device and start `joystick_control.py`.
|
||||
|
||||
### Joystick on your laptop
|
||||
|
||||
In order to use a joystick over the network, we need to run joystick_control locally from your laptop and have it send `testJoystick` packets over the network to the comma device.
|
||||
|
||||
1. Connect a joystick to your PC.
|
||||
2. Connect your laptop to your comma device's hotspot and open a new SSH shell. Since joystick_control is being run on your laptop, we need to write a parameter to let controlsd know to start in joystick debug mode:
|
||||
```shell
|
||||
# on your comma device
|
||||
echo -n "1" > /data/params/d/JoystickDebugMode
|
||||
```
|
||||
3. Run bridge with your laptop's IP address. This republishes the `testJoystick` packets sent from your laptop so that openpilot can receive them:
|
||||
```shell
|
||||
# on your comma device
|
||||
cereal/messaging/bridge {LAPTOP_IP} testJoystick
|
||||
```
|
||||
4. Start joystick_control on your laptop in ZMQ mode.
|
||||
```shell
|
||||
# on your laptop
|
||||
export ZMQ=1
|
||||
tools/joystick/joystick_control.py
|
||||
```
|
||||
|
||||
---
|
||||
Now start your car and openpilot should go into joystick mode with an alert on startup! The status of the axes will display on the alert, while button statuses print in the shell.
|
||||
|
||||
Make sure the conditions are met in the panda to allow controls (e.g. cruise control engaged). You can also make a modification to the panda code to always allow controls.
|
||||
|
||||
|
||||
Executable
+147
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import argparse
|
||||
import threading
|
||||
import numpy as np
|
||||
from inputs import UnpluggedError, get_gamepad
|
||||
|
||||
from cereal import messaging
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.common.realtime import Ratekeeper
|
||||
from openpilot.system.hardware import HARDWARE
|
||||
from openpilot.tools.lib.kbhit import KBHit
|
||||
|
||||
EXPO = 0.4
|
||||
|
||||
|
||||
class Keyboard:
|
||||
def __init__(self):
|
||||
self.kb = KBHit()
|
||||
self.axis_increment = 0.05 # 5% of full actuation each key press
|
||||
self.axes_map = {'w': 'gb', 's': 'gb',
|
||||
'a': 'steer', 'd': 'steer'}
|
||||
self.axes_values = {'gb': 0., 'steer': 0.}
|
||||
self.axes_order = ['gb', 'steer']
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
key = self.kb.getch().lower()
|
||||
self.cancel = False
|
||||
if key == 'r':
|
||||
self.axes_values = dict.fromkeys(self.axes_values, 0.)
|
||||
elif key == 'c':
|
||||
self.cancel = True
|
||||
elif key in self.axes_map:
|
||||
axis = self.axes_map[key]
|
||||
incr = self.axis_increment if key in ['w', 'a'] else -self.axis_increment
|
||||
self.axes_values[axis] = float(np.clip(self.axes_values[axis] + incr, -1, 1))
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class Joystick:
|
||||
def __init__(self):
|
||||
# This class supports a PlayStation 5 DualSense controller on the comma 3X
|
||||
# TODO: find a way to get this from API or detect gamepad/PC, perhaps "inputs" doesn't support it
|
||||
self.cancel_button = 'BTN_NORTH' # BTN_NORTH=X/triangle
|
||||
if HARDWARE.get_device_type() == 'pc':
|
||||
accel_axis = 'ABS_Z'
|
||||
steer_axis = 'ABS_RX'
|
||||
# TODO: once the longcontrol API is finalized, we can replace this with outputting gas/brake and steering
|
||||
self.flip_map = {'ABS_RZ': accel_axis}
|
||||
else:
|
||||
accel_axis = 'ABS_RX'
|
||||
steer_axis = 'ABS_Z'
|
||||
self.flip_map = {'ABS_RY': accel_axis}
|
||||
|
||||
self.min_axis_value = {accel_axis: 0., steer_axis: 0.}
|
||||
self.max_axis_value = {accel_axis: 255., steer_axis: 255.}
|
||||
self.axes_values = {accel_axis: 0., steer_axis: 0.}
|
||||
self.axes_order = [accel_axis, steer_axis]
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
try:
|
||||
joystick_event = get_gamepad()[0]
|
||||
except (OSError, UnpluggedError):
|
||||
self.axes_values = dict.fromkeys(self.axes_values, 0.)
|
||||
return False
|
||||
|
||||
event = (joystick_event.code, joystick_event.state)
|
||||
|
||||
# flip left trigger to negative accel
|
||||
if event[0] in self.flip_map:
|
||||
event = (self.flip_map[event[0]], -event[1])
|
||||
|
||||
if event[0] == self.cancel_button:
|
||||
if event[1] == 1:
|
||||
self.cancel = True
|
||||
elif event[1] == 0: # state 0 is falling edge
|
||||
self.cancel = False
|
||||
elif event[0] in self.axes_values:
|
||||
self.max_axis_value[event[0]] = max(event[1], self.max_axis_value[event[0]])
|
||||
self.min_axis_value[event[0]] = min(event[1], self.min_axis_value[event[0]])
|
||||
|
||||
norm = -float(np.interp(event[1], [self.min_axis_value[event[0]], self.max_axis_value[event[0]]], [-1., 1.]))
|
||||
norm = norm if abs(norm) > 0.03 else 0. # center can be noisy, deadzone of 3%
|
||||
self.axes_values[event[0]] = EXPO * norm ** 3 + (1 - EXPO) * norm # less action near center for fine control
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def send_thread(joystick):
|
||||
pm = messaging.PubMaster(['testJoystick'])
|
||||
|
||||
rk = Ratekeeper(100, print_delay_threshold=None)
|
||||
|
||||
while True:
|
||||
if rk.frame % 20 == 0:
|
||||
print('\n' + ', '.join(f'{name}: {round(v, 3)}' for name, v in joystick.axes_values.items()))
|
||||
|
||||
joystick_msg = messaging.new_message('testJoystick')
|
||||
joystick_msg.valid = True
|
||||
joystick_msg.testJoystick.axes = [joystick.axes_values[ax] for ax in joystick.axes_order]
|
||||
|
||||
pm.send('testJoystick', joystick_msg)
|
||||
|
||||
rk.keep_time()
|
||||
|
||||
|
||||
def joystick_control_thread(joystick):
|
||||
Params().put_bool('JoystickDebugMode', True)
|
||||
threading.Thread(target=send_thread, args=(joystick,), daemon=True).start()
|
||||
while True:
|
||||
joystick.update()
|
||||
|
||||
|
||||
def main():
|
||||
joystick_control_thread(Joystick())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Publishes events from your joystick to control your car.\n' +
|
||||
'openpilot must be offroad before starting joystick_control. This tool supports ' +
|
||||
'a PlayStation 5 DualSense controller on the comma 3X.',
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument('--keyboard', action='store_true', help='Use your keyboard instead of a joystick')
|
||||
args = parser.parse_args()
|
||||
|
||||
if not Params().get_bool("IsOffroad") and "ZMQ" not in os.environ:
|
||||
print("The car must be off before running joystick_control.")
|
||||
exit()
|
||||
|
||||
print()
|
||||
if args.keyboard:
|
||||
print('Gas/brake control: `W` and `S` keys')
|
||||
print('Steering control: `A` and `D` keys')
|
||||
print('Buttons')
|
||||
print('- `R`: Resets axes')
|
||||
print('- `C`: Cancel cruise control')
|
||||
else:
|
||||
print('Using joystick, make sure to run cereal/messaging/bridge on your device if running over the network!')
|
||||
print('If not running on a comma device, the mapping may need to be adjusted.')
|
||||
|
||||
joystick = Keyboard() if args.keyboard else Joystick()
|
||||
joystick_control_thread(joystick)
|
||||
Executable
+80
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
from cereal import messaging, car
|
||||
from opendbc.car.vehicle_model import VehicleModel
|
||||
from openpilot.common.realtime import DT_CTRL, Ratekeeper
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.common.swaglog import cloudlog
|
||||
|
||||
LongCtrlState = car.CarControl.Actuators.LongControlState
|
||||
MAX_LAT_ACCEL = 3.0
|
||||
|
||||
|
||||
def joystickd_thread():
|
||||
params = Params()
|
||||
cloudlog.info("joystickd is waiting for CarParams")
|
||||
CP = messaging.log_from_bytes(params.get("CarParams", block=True), car.CarParams)
|
||||
VM = VehicleModel(CP)
|
||||
|
||||
sm = messaging.SubMaster(['carState', 'onroadEvents', 'liveParameters', 'selfdriveState', 'testJoystick'], frequency=1. / DT_CTRL)
|
||||
pm = messaging.PubMaster(['carControl', 'controlsState'])
|
||||
|
||||
rk = Ratekeeper(100, print_delay_threshold=None)
|
||||
while 1:
|
||||
sm.update(0)
|
||||
|
||||
cc_msg = messaging.new_message('carControl')
|
||||
cc_msg.valid = True
|
||||
CC = cc_msg.carControl
|
||||
CC.enabled = sm['selfdriveState'].enabled
|
||||
CC.latActive = sm['selfdriveState'].active and not sm['carState'].steerFaultTemporary and not sm['carState'].steerFaultPermanent
|
||||
CC.longActive = CC.enabled and not any(e.overrideLongitudinal for e in sm['onroadEvents']) and CP.openpilotLongitudinalControl
|
||||
CC.cruiseControl.cancel = sm['carState'].cruiseState.enabled and (not CC.enabled or not CP.pcmCruise)
|
||||
CC.hudControl.leadDistanceBars = 2
|
||||
|
||||
actuators = CC.actuators
|
||||
|
||||
# reset joystick if it hasn't been received in a while
|
||||
should_reset_joystick = sm.recv_frame['testJoystick'] == 0 or (sm.frame - sm.recv_frame['testJoystick'])*DT_CTRL > 0.2
|
||||
|
||||
if not should_reset_joystick:
|
||||
joystick_axes = sm['testJoystick'].axes
|
||||
else:
|
||||
joystick_axes = [0.0, 0.0]
|
||||
|
||||
if CC.longActive:
|
||||
actuators.accel = 4.0 * float(np.clip(joystick_axes[0], -1, 1))
|
||||
actuators.longControlState = LongCtrlState.pid if sm['carState'].vEgo > CP.vEgoStopping else LongCtrlState.stopping
|
||||
|
||||
if CC.latActive:
|
||||
max_curvature = MAX_LAT_ACCEL / max(sm['carState'].vEgo ** 2, 5)
|
||||
max_angle = math.degrees(VM.get_steer_from_curvature(max_curvature, sm['carState'].vEgo, sm['liveParameters'].roll))
|
||||
|
||||
actuators.torque = float(np.clip(joystick_axes[1], -1, 1))
|
||||
actuators.steeringAngleDeg, actuators.curvature = actuators.torque * max_angle, actuators.torque * -max_curvature
|
||||
|
||||
pm.send('carControl', cc_msg)
|
||||
|
||||
cs_msg = messaging.new_message('controlsState')
|
||||
cs_msg.valid = True
|
||||
controlsState = cs_msg.controlsState
|
||||
controlsState.lateralControlState.init('debugState')
|
||||
|
||||
lp = sm['liveParameters']
|
||||
steer_angle_without_offset = math.radians(sm['carState'].steeringAngleDeg - lp.angleOffsetDeg)
|
||||
controlsState.curvature = -VM.calc_curvature(steer_angle_without_offset, sm['carState'].vEgo, lp.roll)
|
||||
|
||||
pm.send('controlsState', cs_msg)
|
||||
|
||||
rk.keep_time()
|
||||
|
||||
|
||||
def main():
|
||||
joystickd_thread()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,59 @@
|
||||
## LogReader
|
||||
|
||||
Route is a class for conveniently accessing all the [logs](/system/loggerd/) from your routes. The LogReader class reads the non-video logs, i.e. rlog.bz2 and qlog.bz2. There's also a matching FrameReader class for reading the videos.
|
||||
|
||||
```python
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
r = Route("a2a0ccea32023010|2023-07-27--13-01-19")
|
||||
|
||||
# get a list of paths for the route's rlog files
|
||||
print(r.log_paths())
|
||||
|
||||
# and road camera (fcamera.hevc) files
|
||||
print(r.camera_paths())
|
||||
|
||||
# setup a LogReader to read the route's first rlog
|
||||
lr = LogReader(r.log_paths()[0])
|
||||
|
||||
# print out all the messages in the log
|
||||
import codecs
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
for msg in lr:
|
||||
print(msg)
|
||||
|
||||
# setup a LogReader for the route's second qlog
|
||||
lr = LogReader(r.log_paths()[1])
|
||||
|
||||
# print all the steering angles values from the log
|
||||
for msg in lr:
|
||||
if msg.which() == "carState":
|
||||
print(msg.carState.steeringAngleDeg)
|
||||
```
|
||||
|
||||
### Segment Ranges
|
||||
|
||||
We also support a new format called a "segment range":
|
||||
|
||||
```
|
||||
344c5c15b34f2d8a / 2024-01-03--09-37-12 / 2:6 / q
|
||||
[ dongle id ] [ timestamp ] [ selector ] [ query type]
|
||||
```
|
||||
|
||||
you can specify which segments from a route to load
|
||||
|
||||
```python
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4") # 4th segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4:6") # 4th and 5th segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/-1") # last segment
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/:5") # first 5 segments
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/1:") # all except first segment
|
||||
```
|
||||
|
||||
and can select which type of logs to grab
|
||||
|
||||
```python
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/q") # get qlogs
|
||||
lr = LogReader("a2a0ccea32023010|2023-07-27--13-01-19/4/r") # get rlogs (default)
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
import os
|
||||
import requests
|
||||
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
|
||||
|
||||
# TODO: this should be merged into common.api
|
||||
|
||||
class CommaApi:
|
||||
def __init__(self, token=None):
|
||||
self.session = requests.Session()
|
||||
self.session.headers['User-agent'] = 'OpenpilotTools'
|
||||
if token:
|
||||
self.session.headers['Authorization'] = 'JWT ' + token
|
||||
|
||||
def request(self, method, endpoint, **kwargs):
|
||||
with self.session.request(method, API_HOST + '/' + endpoint, **kwargs) as resp:
|
||||
resp_json = resp.json()
|
||||
if isinstance(resp_json, dict) and resp_json.get('error'):
|
||||
if resp.status_code in [401, 403]:
|
||||
raise UnauthorizedError('Unauthorized. Authenticate with tools/lib/auth.py')
|
||||
|
||||
e = APIError(str(resp.status_code) + ":" + resp_json.get('description', str(resp_json['error'])))
|
||||
e.status_code = resp.status_code
|
||||
raise e
|
||||
return resp_json
|
||||
|
||||
def get(self, endpoint, **kwargs):
|
||||
return self.request('GET', endpoint, **kwargs)
|
||||
|
||||
def post(self, endpoint, **kwargs):
|
||||
return self.request('POST', endpoint, **kwargs)
|
||||
|
||||
class APIError(Exception):
|
||||
pass
|
||||
|
||||
class UnauthorizedError(Exception):
|
||||
pass
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Usage::
|
||||
|
||||
usage: auth.py [-h] [{google,apple,github,jwt}] [jwt]
|
||||
|
||||
Login to your comma account
|
||||
|
||||
positional arguments:
|
||||
{google,apple,github,jwt}
|
||||
jwt
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
|
||||
|
||||
Examples::
|
||||
|
||||
./auth.py # Log in with google account
|
||||
./auth.py github # Log in with GitHub Account
|
||||
./auth.py jwt ey......hw # Log in with a JWT from https://jwt.comma.ai, for use in CI
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import pprint
|
||||
import webbrowser
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from typing import Any
|
||||
from urllib.parse import parse_qs, urlencode
|
||||
|
||||
from openpilot.tools.lib.api import APIError, CommaApi, UnauthorizedError
|
||||
from openpilot.tools.lib.auth_config import set_token, get_token
|
||||
|
||||
PORT = 3000
|
||||
|
||||
|
||||
class ClientRedirectServer(HTTPServer):
|
||||
query_params: dict[str, Any] = {}
|
||||
|
||||
|
||||
class ClientRedirectHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
if not self.path.startswith('/auth'):
|
||||
self.send_response(204)
|
||||
return
|
||||
|
||||
query = self.path.split('?', 1)[-1]
|
||||
query_parsed = parse_qs(query, keep_blank_values=True)
|
||||
self.server.query_params = query_parsed
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header('Content-type', 'text/plain')
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Return to the CLI to continue')
|
||||
|
||||
def log_message(self, *args):
|
||||
pass # this prevent http server from dumping messages to stdout
|
||||
|
||||
|
||||
def auth_redirect_link(method):
|
||||
provider_id = {
|
||||
'google': 'g',
|
||||
'apple': 'a',
|
||||
'github': 'h',
|
||||
}[method]
|
||||
|
||||
params = {
|
||||
'redirect_uri': f"https://api.comma.ai/v2/auth/{provider_id}/redirect/",
|
||||
'state': f'service,localhost:{PORT}',
|
||||
}
|
||||
|
||||
if method == 'google':
|
||||
params.update({
|
||||
'type': 'web_server',
|
||||
'client_id': '45471411055-ornt4svd2miog6dnopve7qtmh5mnu6id.apps.googleusercontent.com',
|
||||
'response_type': 'code',
|
||||
'scope': 'https://www.googleapis.com/auth/userinfo.email',
|
||||
'prompt': 'select_account',
|
||||
})
|
||||
return 'https://accounts.google.com/o/oauth2/auth?' + urlencode(params)
|
||||
elif method == 'github':
|
||||
params.update({
|
||||
'client_id': '28c4ecb54bb7272cb5a4',
|
||||
'scope': 'read:user',
|
||||
})
|
||||
return 'https://github.com/login/oauth/authorize?' + urlencode(params)
|
||||
elif method == 'apple':
|
||||
params.update({
|
||||
'client_id': 'ai.comma.login',
|
||||
'response_type': 'code',
|
||||
'response_mode': 'form_post',
|
||||
'scope': 'name email',
|
||||
})
|
||||
return 'https://appleid.apple.com/auth/authorize?' + urlencode(params)
|
||||
else:
|
||||
raise NotImplementedError(f"no redirect implemented for method {method}")
|
||||
|
||||
|
||||
def login(method):
|
||||
oauth_uri = auth_redirect_link(method)
|
||||
|
||||
web_server = ClientRedirectServer(('localhost', PORT), ClientRedirectHandler)
|
||||
print(f'To sign in, use your browser and navigate to {oauth_uri}')
|
||||
webbrowser.open(oauth_uri, new=2)
|
||||
|
||||
while True:
|
||||
web_server.handle_request()
|
||||
if 'code' in web_server.query_params:
|
||||
break
|
||||
elif 'error' in web_server.query_params:
|
||||
print('Authentication Error: "{}". Description: "{}" '.format(
|
||||
web_server.query_params['error'],
|
||||
web_server.query_params.get('error_description')), file=sys.stderr)
|
||||
break
|
||||
|
||||
try:
|
||||
auth_resp = CommaApi().post('v2/auth/', data={'code': web_server.query_params['code'], 'provider': web_server.query_params['provider']})
|
||||
set_token(auth_resp['access_token'])
|
||||
except APIError as e:
|
||||
print(f'Authentication Error: {e}', file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Login to your comma account')
|
||||
parser.add_argument('method', default='google', const='google', nargs='?', choices=['google', 'apple', 'github', 'jwt'])
|
||||
parser.add_argument('jwt', nargs='?')
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.method == 'jwt':
|
||||
if args.jwt is None:
|
||||
print("method JWT selected, but no JWT was provided")
|
||||
exit(1)
|
||||
|
||||
set_token(args.jwt)
|
||||
else:
|
||||
login(args.method)
|
||||
|
||||
try:
|
||||
me = CommaApi(token=get_token()).get('/v1/me')
|
||||
print("Authenticated!")
|
||||
pprint.pprint(me)
|
||||
except UnauthorizedError:
|
||||
print("Got invalid JWT")
|
||||
exit(1)
|
||||
@@ -0,0 +1,29 @@
|
||||
import json
|
||||
import os
|
||||
from openpilot.system.hardware.hw import Paths
|
||||
|
||||
|
||||
class MissingAuthConfigError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_token():
|
||||
try:
|
||||
with open(os.path.join(Paths.config_root(), 'auth.json')) as f:
|
||||
auth = json.load(f)
|
||||
return auth['access_token']
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def set_token(token):
|
||||
os.makedirs(Paths.config_root(), exist_ok=True)
|
||||
with open(os.path.join(Paths.config_root(), 'auth.json'), 'w') as f:
|
||||
json.dump({'access_token': token}, f)
|
||||
|
||||
|
||||
def clear_token():
|
||||
try:
|
||||
os.unlink(os.path.join(Paths.config_root(), 'auth.json'))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
@@ -0,0 +1,73 @@
|
||||
import os
|
||||
from datetime import datetime, timedelta, UTC
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import IO
|
||||
|
||||
|
||||
TOKEN_PATH = Path("/data/azure_token")
|
||||
|
||||
@lru_cache
|
||||
def get_azure_credential():
|
||||
if "AZURE_TOKEN" in os.environ:
|
||||
return os.environ["AZURE_TOKEN"]
|
||||
elif TOKEN_PATH.is_file():
|
||||
return TOKEN_PATH.read_text().strip()
|
||||
else:
|
||||
from azure.identity import AzureCliCredential
|
||||
return AzureCliCredential()
|
||||
|
||||
@lru_cache
|
||||
def get_container_sas(account_name: str, container_name: str):
|
||||
from azure.storage.blob import BlobServiceClient, ContainerSasPermissions, generate_container_sas
|
||||
start_time = datetime.now(UTC).replace(tzinfo=None)
|
||||
expiry_time = start_time + timedelta(hours=1)
|
||||
blob_service = BlobServiceClient(
|
||||
account_url=f"https://{account_name}.blob.core.windows.net",
|
||||
credential=get_azure_credential(),
|
||||
)
|
||||
return generate_container_sas(
|
||||
account_name,
|
||||
container_name,
|
||||
user_delegation_key=blob_service.get_user_delegation_key(start_time, expiry_time),
|
||||
permission=ContainerSasPermissions(read=True, write=True, list=True),
|
||||
expiry=expiry_time,
|
||||
)
|
||||
|
||||
class AzureContainer:
|
||||
def __init__(self, account, container):
|
||||
self.ACCOUNT = account
|
||||
self.CONTAINER = container
|
||||
|
||||
@property
|
||||
def ACCOUNT_URL(self) -> str:
|
||||
return f"https://{self.ACCOUNT}.blob.core.windows.net"
|
||||
|
||||
@property
|
||||
def BASE_URL(self) -> str:
|
||||
return f"{self.ACCOUNT_URL}/{self.CONTAINER}/"
|
||||
|
||||
def get_client_and_key(self):
|
||||
from azure.storage.blob import ContainerClient
|
||||
client = ContainerClient(self.ACCOUNT_URL, self.CONTAINER, credential=get_azure_credential())
|
||||
key = get_container_sas(self.ACCOUNT, self.CONTAINER)
|
||||
return client, key
|
||||
|
||||
def get_url(self, route_name: str, segment_num: str, filename: str) -> str:
|
||||
return self.BASE_URL + f"{route_name.replace('|', '/')}/{segment_num}/{filename}"
|
||||
|
||||
def upload_bytes(self, data: bytes | IO, blob_name: str, overwrite=False) -> str:
|
||||
from azure.storage.blob import BlobClient
|
||||
blob = BlobClient(
|
||||
account_url=self.ACCOUNT_URL,
|
||||
container_name=self.CONTAINER,
|
||||
blob_name=blob_name,
|
||||
credential=get_azure_credential(),
|
||||
overwrite=overwrite,
|
||||
)
|
||||
blob.upload_blob(data, overwrite=overwrite)
|
||||
return self.BASE_URL + blob_name
|
||||
|
||||
def upload_file(self, path: str | os.PathLike, blob_name: str, overwrite=False) -> str:
|
||||
with open(path, "rb") as f:
|
||||
return self.upload_bytes(f, blob_name, overwrite)
|
||||
@@ -0,0 +1,57 @@
|
||||
import functools
|
||||
import re
|
||||
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
from openpilot.tools.lib.helpers import RE
|
||||
|
||||
|
||||
@functools.total_ordering
|
||||
class Bootlog:
|
||||
def __init__(self, url: str):
|
||||
self._url = url
|
||||
|
||||
r = re.search(RE.BOOTLOG_NAME, url)
|
||||
if not r:
|
||||
raise Exception(f"Unable to parse: {url}")
|
||||
|
||||
self._id = r.group('log_id')
|
||||
self._dongle_id = r.group('dongle_id')
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return self._url
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str:
|
||||
return self._dongle_id
|
||||
|
||||
@property
|
||||
def id(self) -> str:
|
||||
return self._id
|
||||
|
||||
def __str__(self):
|
||||
return f"{self._dongle_id}/{self._id}"
|
||||
|
||||
def __eq__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.id == b.id
|
||||
|
||||
def __lt__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.id < b.id
|
||||
|
||||
def get_bootlog_from_id(bootlog_id: str) -> Bootlog | None:
|
||||
# TODO: implement an API endpoint for this
|
||||
bl = Bootlog(bootlog_id)
|
||||
for b in get_bootlogs(bl.dongle_id):
|
||||
if b == bl:
|
||||
return b
|
||||
return None
|
||||
|
||||
def get_bootlogs(dongle_id: str) -> list[Bootlog]:
|
||||
api = CommaApi(get_token())
|
||||
r = api.get(f'v1/devices/{dongle_id}/bootlogs')
|
||||
return [Bootlog(b) for b in r]
|
||||
@@ -0,0 +1,14 @@
|
||||
import os
|
||||
import urllib.parse
|
||||
|
||||
DEFAULT_CACHE_DIR = os.getenv("CACHE_ROOT", os.path.expanduser("~/.commacache"))
|
||||
|
||||
def cache_path_for_file_path(fn, cache_dir=DEFAULT_CACHE_DIR):
|
||||
dir_ = os.path.join(cache_dir, "local")
|
||||
os.makedirs(dir_, exist_ok=True)
|
||||
fn_parsed = urllib.parse.urlparse(fn)
|
||||
if fn_parsed.scheme == '':
|
||||
cache_fn = os.path.abspath(fn).replace("/", "_")
|
||||
else:
|
||||
cache_fn = f'{fn_parsed.hostname}_{fn_parsed.path.replace("/", "_")}'
|
||||
return os.path.join(dir_, cache_fn)
|
||||
@@ -0,0 +1,91 @@
|
||||
import os
|
||||
import requests
|
||||
|
||||
|
||||
# Forks with additional car support can fork the commaCarSegments repo on huggingface or host the LFS files themselves
|
||||
COMMA_CAR_SEGMENTS_REPO = os.environ.get("COMMA_CAR_SEGMENTS_REPO", "https://huggingface.co/datasets/commaai/commaCarSegments")
|
||||
COMMA_CAR_SEGMENTS_BRANCH = os.environ.get("COMMA_CAR_SEGMENTS_BRANCH", "main")
|
||||
COMMA_CAR_SEGMENTS_LFS_INSTANCE = os.environ.get("COMMA_CAR_SEGMENTS_LFS_INSTANCE", COMMA_CAR_SEGMENTS_REPO)
|
||||
|
||||
def get_comma_car_segments_database():
|
||||
from opendbc.car.fingerprints import MIGRATION
|
||||
|
||||
database = requests.get(get_repo_raw_url("database.json")).json()
|
||||
|
||||
ret = {}
|
||||
for platform in database:
|
||||
# TODO: remove this when commaCarSegments is updated to remove selector
|
||||
ret[MIGRATION.get(platform, platform)] = [s.rstrip('/s') for s in database[platform]]
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
# Helpers related to interfacing with the commaCarSegments repository, which contains a collection of public segments for users to perform validation on.
|
||||
|
||||
def parse_lfs_pointer(text):
|
||||
header, lfs_version = text.splitlines()[0].split(" ")
|
||||
assert header == "version"
|
||||
assert lfs_version == "https://git-lfs.github.com/spec/v1"
|
||||
|
||||
header, oid_raw = text.splitlines()[1].split(" ")
|
||||
assert header == "oid"
|
||||
header, oid = oid_raw.split(":")
|
||||
assert header == "sha256"
|
||||
|
||||
header, size = text.splitlines()[2].split(" ")
|
||||
assert header == "size"
|
||||
|
||||
return oid, size
|
||||
|
||||
def get_lfs_file_url(oid, size):
|
||||
data = {
|
||||
"operation": "download",
|
||||
"transfers": [ "basic" ],
|
||||
"objects": [
|
||||
{
|
||||
"oid": oid,
|
||||
"size": int(size)
|
||||
}
|
||||
],
|
||||
"hash_algo": "sha256"
|
||||
}
|
||||
|
||||
headers = {
|
||||
"Accept": "application/vnd.git-lfs+json",
|
||||
"Content-Type": "application/vnd.git-lfs+json"
|
||||
}
|
||||
|
||||
response = requests.post(f"{COMMA_CAR_SEGMENTS_LFS_INSTANCE}.git/info/lfs/objects/batch", json=data, headers=headers)
|
||||
|
||||
assert response.ok
|
||||
|
||||
obj = response.json()["objects"][0]
|
||||
|
||||
assert "error" not in obj, obj
|
||||
|
||||
return obj["actions"]["download"]["href"]
|
||||
|
||||
def get_repo_raw_url(path):
|
||||
if "huggingface" in COMMA_CAR_SEGMENTS_REPO:
|
||||
return f"{COMMA_CAR_SEGMENTS_REPO}/raw/{COMMA_CAR_SEGMENTS_BRANCH}/{path}"
|
||||
|
||||
def get_repo_url(path):
|
||||
# Automatically switch to LFS if we are requesting a file that is stored in LFS
|
||||
|
||||
response = requests.head(get_repo_raw_url(path))
|
||||
|
||||
if "text/plain" in response.headers.get("content-type"):
|
||||
# This is an LFS pointer, so download the raw data from lfs
|
||||
response = requests.get(get_repo_raw_url(path))
|
||||
assert response.status_code == 200
|
||||
oid, size = parse_lfs_pointer(response.text)
|
||||
|
||||
return get_lfs_file_url(oid, size)
|
||||
else:
|
||||
# File has not been uploaded to LFS yet
|
||||
# (either we are on a fork where the data hasn't been pushed to LFS yet, or the CI job to push hasn't finished)
|
||||
return get_repo_raw_url(path)
|
||||
|
||||
|
||||
def get_url(route, segment, file="rlog.zst"):
|
||||
return get_repo_url(f"segments/{route.replace('|', '/')}/{segment}/{file}")
|
||||
@@ -0,0 +1,2 @@
|
||||
class DataUnreadableError(Exception):
|
||||
pass
|
||||
Executable
+57
@@ -0,0 +1,57 @@
|
||||
from collections.abc import Callable
|
||||
|
||||
from openpilot.tools.lib.comma_car_segments import get_url as get_comma_segments_url
|
||||
from openpilot.tools.lib.openpilotci import get_url
|
||||
from openpilot.tools.lib.filereader import DATA_ENDPOINT, file_exists, internal_source_available
|
||||
from openpilot.tools.lib.route import Route, SegmentRange, FileName
|
||||
|
||||
# When passed a tuple of file names, each source will return the first that exists (rlog.zst, rlog.bz2)
|
||||
FileNames = tuple[str, ...]
|
||||
Source = Callable[[SegmentRange, list[int], FileNames], dict[int, str]]
|
||||
|
||||
InternalUnavailableException = Exception("Internal source not available")
|
||||
|
||||
|
||||
def comma_api_source(sr: SegmentRange, seg_idxs: list[int], fns: FileNames) -> dict[int, str]:
|
||||
route = Route(sr.route_name)
|
||||
|
||||
# comma api will have already checked if the file exists
|
||||
if fns == FileName.RLOG:
|
||||
return {seg: route.log_paths()[seg] for seg in seg_idxs if route.log_paths()[seg] is not None}
|
||||
else:
|
||||
return {seg: route.qlog_paths()[seg] for seg in seg_idxs if route.qlog_paths()[seg] is not None}
|
||||
|
||||
|
||||
def internal_source(sr: SegmentRange, seg_idxs: list[int], fns: FileNames, endpoint_url: str = DATA_ENDPOINT) -> dict[int, str]:
|
||||
if not internal_source_available(endpoint_url):
|
||||
raise InternalUnavailableException
|
||||
|
||||
def get_internal_url(sr: SegmentRange, seg, file):
|
||||
return f"{endpoint_url.rstrip('/')}/{sr.dongle_id}/{sr.log_id}/{seg}/{file}"
|
||||
|
||||
return eval_source({seg: [get_internal_url(sr, seg, fn) for fn in fns] for seg in seg_idxs})
|
||||
|
||||
|
||||
def openpilotci_source(sr: SegmentRange, seg_idxs: list[int], fns: FileNames) -> dict[int, str]:
|
||||
return eval_source({seg: [get_url(sr.route_name, seg, fn) for fn in fns] for seg in seg_idxs})
|
||||
|
||||
|
||||
def comma_car_segments_source(sr: SegmentRange, seg_idxs: list[int], fns: FileNames) -> dict[int, str]:
|
||||
return eval_source({seg: get_comma_segments_url(sr.route_name, seg) for seg in seg_idxs})
|
||||
|
||||
|
||||
def eval_source(files: dict[int, list[str] | str]) -> dict[int, str]:
|
||||
# Returns valid file URLs given a list of possible file URLs for each segment (e.g. rlog.bz2, rlog.zst)
|
||||
valid_files: dict[int, str] = {}
|
||||
|
||||
for seg_idx, urls in files.items():
|
||||
if isinstance(urls, str):
|
||||
urls = [urls]
|
||||
|
||||
# Add first valid file URL
|
||||
for url in urls:
|
||||
if file_exists(url):
|
||||
valid_files[seg_idx] = url
|
||||
break
|
||||
|
||||
return valid_files
|
||||
@@ -0,0 +1,49 @@
|
||||
import os
|
||||
import posixpath
|
||||
import socket
|
||||
from functools import cache
|
||||
from openpilot.common.utils import retry
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from openpilot.tools.lib.url_file import URLFile
|
||||
|
||||
DATA_ENDPOINT = os.getenv("DATA_ENDPOINT", "http://data-raw.comma.internal/")
|
||||
|
||||
|
||||
@cache
|
||||
@retry(delay=0.0)
|
||||
def internal_source_available(url: str) -> bool:
|
||||
if os.path.isdir(url):
|
||||
return True
|
||||
|
||||
try:
|
||||
hostname = urlparse(url).hostname
|
||||
port = urlparse(url).port or 80
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
|
||||
s.settimeout(0.5)
|
||||
s.connect((hostname, port))
|
||||
return True
|
||||
except (socket.gaierror, ConnectionRefusedError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def resolve_name(fn):
|
||||
if fn.startswith("cd:/"):
|
||||
return posixpath.join(DATA_ENDPOINT, fn[4:])
|
||||
return fn
|
||||
|
||||
|
||||
@cache
|
||||
def file_exists(fn):
|
||||
fn = resolve_name(fn)
|
||||
if fn.startswith(("http://", "https://")):
|
||||
return URLFile(fn).get_length_online() != -1
|
||||
return os.path.exists(fn)
|
||||
|
||||
|
||||
def FileReader(fn, debug=False):
|
||||
fn = resolve_name(fn)
|
||||
if fn.startswith(("http://", "https://")):
|
||||
return URLFile(fn, debug=debug)
|
||||
return open(fn, "rb")
|
||||
@@ -0,0 +1,173 @@
|
||||
import os
|
||||
import subprocess
|
||||
import json
|
||||
from collections.abc import Iterator
|
||||
from collections import OrderedDict
|
||||
|
||||
import numpy as np
|
||||
from openpilot.tools.lib.filereader import FileReader, resolve_name
|
||||
from openpilot.tools.lib.exceptions import DataUnreadableError
|
||||
from openpilot.tools.lib.vidindex import hevc_index
|
||||
|
||||
|
||||
HEVC_SLICE_B = 0
|
||||
HEVC_SLICE_P = 1
|
||||
HEVC_SLICE_I = 2
|
||||
|
||||
|
||||
class LRUCache:
|
||||
def __init__(self, capacity: int):
|
||||
self._cache: OrderedDict = OrderedDict()
|
||||
self.capacity = capacity
|
||||
|
||||
def __getitem__(self, key):
|
||||
self._cache.move_to_end(key)
|
||||
return self._cache[key]
|
||||
|
||||
def __setitem__(self, key, value):
|
||||
self._cache[key] = value
|
||||
if len(self._cache) > self.capacity:
|
||||
self._cache.popitem(last=False)
|
||||
|
||||
def __contains__(self, key):
|
||||
return key in self._cache
|
||||
|
||||
|
||||
def assert_hvec(fn: str) -> None:
|
||||
with FileReader(fn) as f:
|
||||
header = f.read(4)
|
||||
if len(header) == 0:
|
||||
raise DataUnreadableError(f"{fn} is empty")
|
||||
elif header == b"\x00\x00\x00\x01":
|
||||
if 'hevc' not in fn:
|
||||
raise NotImplementedError(fn)
|
||||
|
||||
def decompress_video_data(rawdat, w, h, pix_fmt="rgb24", vid_fmt='hevc') -> np.ndarray:
|
||||
threads = os.getenv("FFMPEG_THREADS", "0")
|
||||
args = ["ffmpeg", "-v", "quiet",
|
||||
"-threads", threads,
|
||||
"-c:v", "hevc",
|
||||
"-vsync", "0",
|
||||
"-f", vid_fmt,
|
||||
"-flags2", "showall",
|
||||
"-i", "-",
|
||||
"-f", "rawvideo",
|
||||
"-pix_fmt", pix_fmt,
|
||||
"-"]
|
||||
dat = subprocess.check_output(args, input=rawdat)
|
||||
|
||||
ret: np.ndarray
|
||||
if pix_fmt == "rgb24":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, h, w, 3)
|
||||
elif pix_fmt in ["nv12", "yuv420p"]:
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, (h*w*3//2))
|
||||
else:
|
||||
raise NotImplementedError(f"Unsupported pixel format: {pix_fmt}")
|
||||
return ret
|
||||
|
||||
def ffprobe(fn, fmt=None):
|
||||
fn = resolve_name(fn)
|
||||
cmd = ["ffprobe", "-v", "quiet", "-print_format", "json", "-show_format", "-show_streams"]
|
||||
if fmt:
|
||||
cmd += ["-f", fmt]
|
||||
cmd += ["-i", "-"]
|
||||
|
||||
try:
|
||||
with FileReader(fn) as f:
|
||||
ffprobe_output = subprocess.check_output(cmd, input=f.read(4096))
|
||||
except subprocess.CalledProcessError as e:
|
||||
raise DataUnreadableError(fn) from e
|
||||
return json.loads(ffprobe_output)
|
||||
|
||||
def get_index_data(fn: str, index_data: dict|None = None):
|
||||
if index_data is None:
|
||||
index_data = get_video_index(fn)
|
||||
if index_data is None:
|
||||
raise DataUnreadableError(f"Failed to index {fn!r}")
|
||||
stream = index_data["probe"]["streams"][0]
|
||||
return index_data["index"], index_data["global_prefix"], stream["width"], stream["height"]
|
||||
|
||||
def get_video_index(fn):
|
||||
assert_hvec(fn)
|
||||
frame_types, dat_len, prefix = hevc_index(fn)
|
||||
index = np.array(frame_types + [(0xFFFFFFFF, dat_len)], dtype=np.uint32)
|
||||
probe = ffprobe(fn, "hevc")
|
||||
return {
|
||||
'index': index,
|
||||
'global_prefix': prefix,
|
||||
'probe': probe
|
||||
}
|
||||
|
||||
|
||||
class FfmpegDecoder:
|
||||
def __init__(self, fn: str, index_data: dict|None = None,
|
||||
pix_fmt: str = "rgb24"):
|
||||
self.fn = fn
|
||||
self.index, self.prefix, self.w, self.h = get_index_data(fn, index_data)
|
||||
self.frame_count = len(self.index) - 1 # sentinel row at the end
|
||||
self.iframes = np.where(self.index[:, 0] == HEVC_SLICE_I)[0]
|
||||
self.pix_fmt = pix_fmt
|
||||
|
||||
def _gop_bounds(self, frame_idx: int):
|
||||
f_b = frame_idx
|
||||
while f_b > 0 and self.index[f_b, 0] != HEVC_SLICE_I:
|
||||
f_b -= 1
|
||||
f_e = frame_idx + 1
|
||||
while f_e < self.frame_count and self.index[f_e, 0] != HEVC_SLICE_I:
|
||||
f_e += 1
|
||||
return f_b, f_e, self.index[f_b, 1], self.index[f_e, 1]
|
||||
|
||||
def _decode_gop(self, raw: bytes) -> Iterator[np.ndarray]:
|
||||
yield from decompress_video_data(raw, self.w, self.h, self.pix_fmt)
|
||||
|
||||
def get_gop_start(self, frame_idx: int):
|
||||
return self.iframes[np.searchsorted(self.iframes, frame_idx, side="right") - 1]
|
||||
|
||||
def get_iterator(self, start_fidx: int = 0, end_fidx: int|None = None,
|
||||
frame_skip: int = 1) -> Iterator[tuple[int, np.ndarray]]:
|
||||
end_fidx = end_fidx or self.frame_count
|
||||
fidx = start_fidx
|
||||
while fidx < end_fidx:
|
||||
f_b, f_e, off_b, off_e = self._gop_bounds(fidx)
|
||||
with FileReader(self.fn) as f:
|
||||
f.seek(off_b)
|
||||
raw = self.prefix + f.read(off_e - off_b)
|
||||
# number of frames to discard inside this GOP before the wanted one
|
||||
for i, frm in enumerate(decompress_video_data(raw, self.w, self.h, self.pix_fmt)):
|
||||
fidx = f_b + i
|
||||
if fidx >= end_fidx:
|
||||
return
|
||||
elif fidx >= start_fidx and (fidx - start_fidx) % frame_skip == 0:
|
||||
yield fidx, frm
|
||||
fidx += 1
|
||||
|
||||
def FrameIterator(fn: str, index_data: dict|None=None,
|
||||
pix_fmt: str = "rgb24",
|
||||
start_fidx:int=0, end_fidx=None, frame_skip:int=1) -> Iterator[np.ndarray]:
|
||||
dec = FfmpegDecoder(fn, pix_fmt=pix_fmt, index_data=index_data)
|
||||
for _, frame in dec.get_iterator(start_fidx=start_fidx, end_fidx=end_fidx, frame_skip=frame_skip):
|
||||
yield frame
|
||||
|
||||
class FrameReader:
|
||||
def __init__(self, fn: str, index_data: dict|None = None,
|
||||
cache_size: int = 30, pix_fmt: str = "rgb24"):
|
||||
self.decoder = FfmpegDecoder(fn, index_data, pix_fmt)
|
||||
self.iframes = self.decoder.iframes
|
||||
self._cache: LRUCache = LRUCache(cache_size)
|
||||
self.w, self.h, self.frame_count, = self.decoder.w, self.decoder.h, self.decoder.frame_count
|
||||
self.pix_fmt = pix_fmt
|
||||
|
||||
self.it: Iterator[tuple[int, np.ndarray]] | None = None
|
||||
self.fidx = -1
|
||||
|
||||
def get(self, fidx:int):
|
||||
if fidx in self._cache: # If frame is cached, return it
|
||||
return self._cache[fidx]
|
||||
read_start = self.decoder.get_gop_start(fidx)
|
||||
if not self.it or fidx < self.fidx or read_start != self.decoder.get_gop_start(self.fidx): # If the frame is in a different GOP, reset the iterator
|
||||
self.it = self.decoder.get_iterator(read_start)
|
||||
self.fidx = -1
|
||||
while self.fidx < fidx:
|
||||
self.fidx, frame = next(self.it)
|
||||
self._cache[self.fidx] = frame
|
||||
return self._cache[fidx]
|
||||
@@ -0,0 +1,113 @@
|
||||
import base64
|
||||
import requests
|
||||
from http import HTTPMethod
|
||||
|
||||
class GithubUtils:
|
||||
def __init__(self, api_token, data_token, owner='commaai', api_repo='openpilot', data_repo='ci-artifacts'):
|
||||
self.OWNER = owner
|
||||
self.API_REPO = api_repo
|
||||
self.DATA_REPO = data_repo
|
||||
self.API_TOKEN = api_token
|
||||
self.DATA_TOKEN = data_token
|
||||
|
||||
@property
|
||||
def API_ROUTE(self):
|
||||
return f"https://api.github.com/repos/{self.OWNER}/{self.API_REPO}"
|
||||
|
||||
@property
|
||||
def DATA_ROUTE(self):
|
||||
return f"https://api.github.com/repos/{self.OWNER}/{self.DATA_REPO}"
|
||||
|
||||
def api_call(self, path, data="", method=HTTPMethod.GET, accept="", data_call=False, raise_on_failure=True):
|
||||
token = self.DATA_TOKEN if data_call else self.API_TOKEN
|
||||
if token:
|
||||
headers = {"Authorization": f"Bearer {self.DATA_TOKEN if data_call else self.API_TOKEN}", \
|
||||
"Accept": f"application/vnd.github{accept}+json"}
|
||||
else:
|
||||
headers = {}
|
||||
path = f'{self.DATA_ROUTE if data_call else self.API_ROUTE}/{path}'
|
||||
r = requests.request(method, path, headers=headers, data=data)
|
||||
if not r.ok and raise_on_failure:
|
||||
raise Exception(f"Call to {path} failed with {r.status_code}")
|
||||
else:
|
||||
return r
|
||||
|
||||
def upload_file(self, bucket, path, file_name):
|
||||
with open(path, "rb") as f:
|
||||
encoded = base64.b64encode(f.read()).decode()
|
||||
|
||||
# check if file already exists
|
||||
sha = self.get_file_sha(bucket, file_name)
|
||||
sha = f'"sha":"{sha}",' if sha else ''
|
||||
|
||||
data = f'{{"message":"uploading {file_name}", \
|
||||
"branch":"{bucket}", \
|
||||
"committer":{{"name":"Vehicle Researcher", "email": "user@comma.ai"}}, \
|
||||
{sha} \
|
||||
"content":"{encoded}"}}'
|
||||
github_path = f"contents/{file_name}"
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.PUT, data_call=True)
|
||||
|
||||
def upload_files(self, bucket, files):
|
||||
self.create_bucket(bucket)
|
||||
for file_name,path in files:
|
||||
self.upload_file(bucket, path, file_name)
|
||||
|
||||
def create_bucket(self, bucket):
|
||||
if self.get_bucket_sha(bucket):
|
||||
return
|
||||
master_sha = self.get_bucket_sha('master')
|
||||
github_path = "git/refs"
|
||||
data = f'{{"ref":"refs/heads/{bucket}", "sha":"{master_sha}"}}'
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.POST, data_call=True)
|
||||
|
||||
def get_bucket_sha(self, bucket):
|
||||
github_path = f"git/refs/heads/{bucket}"
|
||||
r = self.api_call(github_path, data_call=True, raise_on_failure=False)
|
||||
return r.json()['object']['sha'] if r.ok else None
|
||||
|
||||
def get_file_url(self, bucket, file_name):
|
||||
github_path = f"contents/{file_name}?ref={bucket}"
|
||||
r = self.api_call(github_path, data_call=True)
|
||||
return r.json()['download_url']
|
||||
|
||||
def get_file_sha(self, bucket, file_name):
|
||||
github_path = f"contents/{file_name}?ref={bucket}"
|
||||
r = self.api_call(github_path, data_call=True, raise_on_failure=False)
|
||||
return r.json()['sha'] if r.ok else None
|
||||
|
||||
def get_pr_number(self, pr_branch):
|
||||
github_path = f"commits/{pr_branch}/pulls"
|
||||
r = self.api_call(github_path)
|
||||
return r.json()[0]['number']
|
||||
|
||||
def get_bucket_link(self, bucket):
|
||||
return f'https://raw.githubusercontent.com/{self.OWNER}/{self.DATA_REPO}/refs/heads/{bucket}'
|
||||
|
||||
def comment_on_pr(self, comment, pr_branch, commenter="", overwrite=False):
|
||||
pr_number = self.get_pr_number(pr_branch)
|
||||
data = f'{{"body": "{comment}"}}'
|
||||
github_path = f'issues/{pr_number}/comments'
|
||||
if overwrite:
|
||||
r = self.api_call(github_path)
|
||||
comments = [x['id'] for x in r.json() if x['user']['login'] == commenter]
|
||||
if comments:
|
||||
github_path = f'issues/comments/{comments[0]}'
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.PATCH)
|
||||
return
|
||||
|
||||
self.api_call(github_path, data=data, method=HTTPMethod.POST)
|
||||
|
||||
# upload files to github and comment them on the pr
|
||||
def comment_images_on_pr(self, title, commenter, pr_branch, bucket, images):
|
||||
self.upload_files(bucket, images)
|
||||
table = [f'<details><summary>{title}</summary><table>']
|
||||
for i,f in enumerate(images):
|
||||
if not (i % 2):
|
||||
table.append('<tr>')
|
||||
table.append(f'<td><img src=\\"https://raw.githubusercontent.com/{self.OWNER}/{self.DATA_REPO}/{bucket}/{f[0]}\\"></td>')
|
||||
if (i % 2):
|
||||
table.append('</tr>')
|
||||
table.append('</table></details>')
|
||||
table = ''.join(table)
|
||||
self.comment_on_pr(table, commenter, pr_branch)
|
||||
@@ -0,0 +1,17 @@
|
||||
# regex patterns
|
||||
class RE:
|
||||
DONGLE_ID = r'(?P<dongle_id>[a-f0-9]{16})'
|
||||
TIMESTAMP = r'(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})'
|
||||
LOG_ID_V2 = r'(?P<count>[a-f0-9]{8})--(?P<uid>[a-z0-9]{10})'
|
||||
LOG_ID = fr'(?P<log_id>(?:{TIMESTAMP}|{LOG_ID_V2}))'
|
||||
ROUTE_NAME = fr'(?P<route_name>{DONGLE_ID}[|_/]{LOG_ID})'
|
||||
SEGMENT_NAME = fr'{ROUTE_NAME}(?:--|/)(?P<segment_num>[0-9]+)'
|
||||
|
||||
INDEX = r'-?[0-9]+'
|
||||
SLICE = fr'(?P<start>{INDEX})?:?(?P<end>{INDEX})?:?(?P<step>{INDEX})?'
|
||||
SEGMENT_RANGE = fr'{ROUTE_NAME}(?:(--|/)(?P<slice>({SLICE})))?(?:/(?P<selector>([qra])))?'
|
||||
|
||||
BOOTLOG_NAME = ROUTE_NAME
|
||||
|
||||
EXPLORER_FILE = fr'^(?P<segment_name>{SEGMENT_NAME})--(?P<file_name>[a-z]+\.[a-z0-9]+)$'
|
||||
OP_SEGMENT_DIR = fr'^(?P<segment_name>{SEGMENT_NAME})$'
|
||||
Executable
+81
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import termios
|
||||
import atexit
|
||||
from select import select
|
||||
|
||||
|
||||
class KBHit:
|
||||
def __init__(self) -> None:
|
||||
''' Creates a KBHit object that you can call to do various keyboard things.
|
||||
'''
|
||||
|
||||
self.stdin_fd = sys.stdin.fileno()
|
||||
self.set_kbhit_terminal()
|
||||
|
||||
def set_kbhit_terminal(self) -> None:
|
||||
''' Save old terminal settings for closure, remove ICANON & ECHO flags.
|
||||
'''
|
||||
|
||||
# Save the terminal settings
|
||||
self.old_term = termios.tcgetattr(self.stdin_fd)
|
||||
self.new_term = self.old_term.copy()
|
||||
|
||||
# New terminal setting unbuffered
|
||||
self.new_term[3] &= ~(termios.ICANON | termios.ECHO)
|
||||
termios.tcsetattr(self.stdin_fd, termios.TCSAFLUSH, self.new_term)
|
||||
|
||||
# Support normal-terminal reset at exit
|
||||
atexit.register(self.set_normal_term)
|
||||
|
||||
def set_normal_term(self) -> None:
|
||||
''' Resets to normal terminal. On Windows this is a no-op.
|
||||
'''
|
||||
|
||||
termios.tcsetattr(self.stdin_fd, termios.TCSAFLUSH, self.old_term)
|
||||
|
||||
@staticmethod
|
||||
def getch() -> str:
|
||||
''' Returns a keyboard character after kbhit() has been called.
|
||||
Should not be called in the same program as getarrow().
|
||||
'''
|
||||
return sys.stdin.read(1)
|
||||
|
||||
@staticmethod
|
||||
def getarrow() -> int:
|
||||
''' Returns an arrow-key code after kbhit() has been called. Codes are
|
||||
0 : up
|
||||
1 : right
|
||||
2 : down
|
||||
3 : left
|
||||
Should not be called in the same program as getch().
|
||||
'''
|
||||
|
||||
c = sys.stdin.read(3)[2]
|
||||
vals = [65, 67, 66, 68]
|
||||
|
||||
return vals.index(ord(c))
|
||||
|
||||
@staticmethod
|
||||
def kbhit():
|
||||
''' Returns True if keyboard character was hit, False otherwise.
|
||||
'''
|
||||
return select([sys.stdin], [], [], 0)[0] != []
|
||||
|
||||
|
||||
# Test
|
||||
if __name__ == "__main__":
|
||||
|
||||
kb = KBHit()
|
||||
|
||||
print('Hit any key, or ESC to exit')
|
||||
|
||||
while True:
|
||||
|
||||
if kb.kbhit():
|
||||
c = kb.getch()
|
||||
if c == '\x1b': # ESC
|
||||
break
|
||||
print(c)
|
||||
|
||||
kb.set_normal_term()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user