Compare commits

...

95 Commits

Author SHA1 Message Date
royjr
669f3b7945 fix 2025-11-07 21:51:35 -05:00
royjr
63122e1a33 Merge branch 'master' into visual-style 2025-11-07 21:25:46 -05:00
Jason Wen
c1d3ae427b version: bump to 2025.003.000 2025-11-06 23:12:41 -05:00
Jason Wen
2ab45b552d Update CHANGELOG.md 2025-11-06 23:10:03 -05:00
github-actions[bot]
8c1d59fecd [bot] Update Python packages (#1434)
Update Python packages

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-06 22:47:55 -05:00
DevTekVE
cde88fd8ed bug: Fix initial registration for sunnylink (#1457)
refactor(sunnylink): defer `SunnylinkApi` initialization to function scope

- Moved `SunnylinkApi` object creation into individual functions as needed.
- Prevents unnecessary initialization when the object isn't used.
2025-11-06 12:13:15 +01:00
DevTekVE
4b5de0eddb stats: sunnylink integration (#1454)
* sunnylink: add statsd process and related telemetry logging infrastructure

- Introduced `statsd_sp` process for handling Sunnylink-specific stats.
- Enhanced metrics logging with improved directory structure and data handling.

* sunnylink: re-enable and refine stat_handler for telemetry processing

- Reactivated `stat_handler` thread with improved path handling.
- Made `stat_handler` more flexible by allowing directory injection.

* statsd: fix formatting issue in telemetry string generation

- Corrected missing comma between `sunnylink_dongle_id` and `comma_dongle_id`.

* update statsd_sp process configuration for enhanced readiness logic

- Modified `statsd_sp` initialization to include `always_run` alongside `sunnylink_ready_shim`.
- Ensures robust process activation conditions.

* refactor(statsd): enhance and unify StatLogSP implementation

- Replaced custom `StatLogSP` in sunnylink with centralized implementation from `system.statsd`.
- Ensures consistent logic for StatLogSP handling across modules.

* fix

* refactor(statsd): add intercept parameter to StatLogSP for configurable logging

- Introduced optional `intercept` parameter to `StatLogSP` to manage `comma_statlog` initialization.
- Updated usage in `sunnylink` to disable interception where unnecessary.

* Dont complain

* feat(statsd): add raw metric type and SunnyPilot-specific stats collection

- Introduced `METRIC_TYPE.RAW` for base64-encoded raw data metrics.
- Added `sp_stats` thread to export SunnyPilot params as raw metrics.
- Enhanced telemetry handling with decoding and serialization updates.

* refactor(statsd): improve `sp_stats` error handling and param processing

- Enhanced exception handling for `params.get` to prevent crashes.
- Added support for nested dict values to be included in stats.

* refactor(statsd): adjust imports and minor code formatting updates

- Updated `Ratekeeper` import path for consistency with the `openpilot` module structure.
- Fixed minor formatting for improved readability.

* refactor(statsd): update typings and remove unused NoReturn annotation

- Removed unnecessary `NoReturn` typing for `stats_main` to simplify function definition.
- Adjusted `get_influxdb_line_raw` to refine typing for `value` parameter.

* cleanup

* init

* init

* slightly more

* staticmethod

* handle them all

* get them models

* log with route

* more

* car

* Revert "car"

This reverts commit fe1c90cf4d.

* handle capnp

* Revert "handle capnp"

This reverts commit c5aea68803.

* 1 more time

* Revert "1 more time"

This reverts commit a364474fa5.

* Cleaning to expose wider

* feat(interfaces, statsd): log car params to stats system

- Added `STATSLOGSP` import and logging to capture `carFingerprint` in metrics.
- Improved error handling in `get_influxdb_line_raw` for robust metric generation.

* refactor(interfaces): streamline car params logging to stats

- Simplified logging by directly converting `CP` to a dictionary.
- Removed legacy stats aggregation for clarity.

* feat(sunnylink): enable compression for stats in SunnyLink

- Added optional compression for stats payload to support large data.
- Updated `stat_handler` to handle compression and base64 encoding.

* fix(statsd): filter complex types in `get_influxdb_line_raw`

- Skips unsupported types (dict, list, bytes) to prevent formatting errors.
- Simplifies type annotation for `value` parameter.

* fix(statsd): use `json.dumps` for string conversion in `get_influxdb_line_raw`

- Ensures proper handling of special characters in values.
- Prevents potential formatting issues with raw `str()` conversion.

* refactor(interfaces, statsd): update parameter keys for stats logging

- Renamed logged keys for better clarity (`sunnypilot_params` → `sunnypilot.car_params`, `device_params`).
- Ensures consistency across data logs.

* bet

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-11-04 16:53:31 -05:00
royjr
c9b1afb154 Merge branch 'master' into visual-style 2025-10-10 00:06:55 -04:00
royjr
6e3bd3fbed explicit radius 2025-10-09 23:56:37 -04:00
royjr
42592dd550 match what we currently send 2025-10-09 23:54:34 -04:00
royjr
b2f7d72a33 no need 2025-10-09 23:53:07 -04:00
royjr
2e0ce18c84 group 2025-10-09 23:52:51 -04:00
royjr
2a9a4a9263 Merge branch 'master' into visual-style 2025-10-09 23:51:18 -04:00
royjr
e63fb10fdb Revert "metric threshold"
This reverts commit b54941928d.
2025-10-08 21:35:23 -04:00
royjr
b54941928d metric threshold 2025-10-08 21:35:15 -04:00
royjr
b85b8ffacf better VisualStyleOverheadThreshold 2025-10-08 21:28:10 -04:00
royjr
3ff2e9b26a reorder 2025-10-08 21:25:37 -04:00
royjr
4b3ffc722a show visual_radar_tracks_delay_settings on VisualRadarTracks 2025-10-08 21:24:52 -04:00
royjr
5f49066829 more 2025-10-08 21:18:51 -04:00
royjr
ab2eb218d5 clean 2025-10-08 21:16:06 -04:00
royjr
6212b174e9 descs 2025-10-08 21:12:08 -04:00
royjr
a8a6e5708a Merge branch 'master' into visual-style 2025-10-08 20:47:03 -04:00
royjr
54b060f178 move with 2025-10-08 20:45:17 -04:00
royjr
8266386cd0 better 2025-10-08 20:44:06 -04:00
royjr
8bbe87ee22 simple 2025-10-08 20:40:23 -04:00
royjr
d35ac0c145 a bit better 2025-10-08 20:34:26 -04:00
royjr
a134ae1e29 combine 2025-10-08 20:25:55 -04:00
royjr
6a69759b9e hide options based on options 2025-10-08 20:24:51 -04:00
royjr
b9063e2966 cleanup 2025-10-08 19:55:40 -04:00
royjr
4397a4387a mooooore fps 2025-10-08 19:49:14 -04:00
royjr
32dc384524 not needed for now 2025-10-08 19:47:08 -04:00
royjr
68fa239b97 unused 2025-10-08 19:45:37 -04:00
royjr
c8367fbc25 more fps remove 2025-10-08 19:44:45 -04:00
royjr
76fc4514e1 reorder 2025-10-08 19:43:32 -04:00
royjr
073ce2b4df remove VisualFPS 2025-10-08 19:35:37 -04:00
royjr
5d516ba89f Merge branch 'master' into visual-style 2025-10-08 12:26:38 -04:00
royjr
e819a0dcb1 Merge branch 'master' into visual-style 2025-10-08 02:14:22 -04:00
royjr
ba4b583e6e fix visual_style_overhead_zoom 2025-10-08 01:55:38 -04:00
royjr
5954354356 fix visual_style_overhead 2025-10-08 01:48:06 -04:00
royjr
23ff232333 fix visual_style_overhead_settings 2025-10-08 01:40:10 -04:00
royjr
0bb47fcfa9 fix visual_style_overhead_threshold_settings 2025-10-08 01:29:15 -04:00
royjr
99a5682371 todo 2025-10-08 01:09:08 -04:00
royjr
22ca343050 fix visual_style_overhead_threshold_settings 2025-10-08 01:07:53 -04:00
royjr
0049d20151 VisualStyleZoom fix 2025-10-08 01:05:17 -04:00
royjr
b9f8f4e8ac visual style better 2025-10-08 00:55:20 -04:00
royjr
26564dd42f better VisualWideCam 2025-10-08 00:07:31 -04:00
royjr
ec440e4568 Merge branch 'master' into visual-style 2025-10-07 22:38:57 -04:00
royjr
69817f887b Merge branch 'master' into visual-style 2025-09-30 15:01:21 -04:00
royjr
df21208b7c Merge branch 'master' into visual-style 2025-09-25 02:08:38 -04:00
royjr
660c994c5e visual_style_blend vs visual_style_overhead_blend 2025-09-25 02:08:28 -04:00
royjr
904cc796b0 prevent jitter 2025-09-25 01:44:06 -04:00
royjr
1a9a1e1b8a Merge branch 'master' into visual-style 2025-09-23 23:00:46 -04:00
royjr
2ae7078c0f use ParamWatcher 2025-09-23 14:37:21 -04:00
royjr
0fde830a30 fix params 2025-09-23 10:35:46 -04:00
royjr
7c744a42e5 safe font 2025-09-23 08:18:51 -04:00
royjr
90cc169dec VisualRadarTracksDelay 2025-09-23 08:17:59 -04:00
royjr
36c1e11a9e add FPS toggle 2025-09-23 07:47:16 -04:00
royjr
5e26a99337 better fps 2025-09-23 07:40:59 -04:00
royjr
53683cf90b constant track size 2025-09-23 07:25:00 -04:00
royjr
95dcc23887 better tracks for overhead 2025-09-23 07:21:57 -04:00
royjr
cef3163c7c more more more cached params 2025-09-23 06:54:46 -04:00
royjr
cbc34dd1f6 more cached params 2025-09-23 06:49:41 -04:00
royjr
fbd1f6bad1 cached params 2025-09-23 06:42:14 -04:00
royjr
f2ddf9abba debug ui lag 2025-09-23 04:04:52 -04:00
royjr
8a5eaf5ba6 prepare for lag compensation 2025-09-23 03:35:06 -04:00
royjr
ddb377ac5b radar lag compensate 2025-09-23 03:24:15 -04:00
royjr
512a01d28c VisualWideCam toggle (untested) 2025-09-23 01:33:50 -04:00
royjr
b30e275169 Merge branch 'master' into visual-style 2025-09-23 01:15:18 -04:00
royjr
ee9fe5c9ed VisualRadarTracks toggle 2025-09-23 01:10:44 -04:00
royjr
e672a352d3 bigger points 2025-09-23 00:24:38 -04:00
royjr
361d107040 basic radar 2025-09-22 23:55:55 -04:00
royjr
36eb047cd3 Merge branch 'master' into visual-style 2025-09-22 00:10:31 -04:00
royjr
218c6172e6 darker fills 2025-09-22 00:03:29 -04:00
royjr
8f35e4fc3c Revert "smooooth"
This reverts commit c965df39d6.
2025-09-21 23:02:48 -04:00
royjr
c965df39d6 smooooth 2025-09-21 22:10:35 -04:00
royjr
53ef69f3c3 hide horizon if no data 2025-09-21 21:55:23 -04:00
royjr
ea9ca18c8b horizon at end 2025-09-21 21:49:01 -04:00
royjr
225858261e better horizon 2025-09-21 21:34:14 -04:00
royjr
43de43b34a better lines 2025-09-21 21:25:39 -04:00
royjr
96ddbe35a1 dynamically adjust background 2025-09-21 15:37:54 -04:00
royjr
7d547ad533 fix default 2025-09-21 15:30:13 -04:00
royjr
13de58b845 add more speeds 2025-09-21 15:30:06 -04:00
royjr
3d8c563a4b dynamic zoom 2025-09-21 15:19:57 -04:00
royjr
bc75199d5a overhead 2025-09-21 14:34:38 -04:00
royjr
ba6e18ed91 only for vision 2025-09-21 14:32:30 -04:00
royjr
74dbcd699b add road edges to vision 2025-09-21 14:31:58 -04:00
royjr
35c6af0190 theme 2025-09-21 13:55:56 -04:00
royjr
827de88c8b stump top down 2025-09-21 13:03:37 -04:00
royjr
54ca4b537d VisualStyleBlendThreshold 2025-09-21 12:48:32 -04:00
royjr
c61f327076 allow always overhead view 2025-09-20 13:04:07 -04:00
royjr
d569913e5a VisualStyleBlend 2025-09-20 12:57:17 -04:00
royjr
418c93be06 animate overhead 2025-09-20 12:16:49 -04:00
royjr
5acc040a89 overhead 2025-09-20 11:24:24 -04:00
royjr
e45b17c230 better 2025-09-20 03:00:00 -04:00
royjr
b14b6246d7 visual style init 2025-09-20 00:31:07 -04:00
20 changed files with 891 additions and 37 deletions

View File

@@ -1,6 +1,30 @@
sunnypilot Version 2025.002.000 (2025-xx-xx)
sunnypilot Version 2025.003.000 (20xx-xx-xx)
========================
sunnypilot Version 2025.002.000 (2025-11-06)
========================
* What's Changed (sunnypilot/sunnypilot)
* models: bump model json to v8 by @Discountchubbs
* Bug: Model UI Crash Fix by @nayan8teen
* controlsd: add `CP_SP` to `get_pid_accel_limits` by @THERoenPR
* sunnylink: update uploader button logic to support novice tier and above by @devtekve
* Tesla: Coop Steering by @AmyJeanes
* ui: update discord references and add forum widget by @devtekve
* ui: Fix spacing in sunnylink panel by @devtekve
* docs: Update README installation branches and discord links by @mpurnell1 in
* stats: sunnylink integration by @devtekve
* bug: Fix initial registration for sunnylink by @devtekve
* What's Changed (sunnypilot/opendbc)
* Honda: add brake hold messages for Clarity by @mvl-boston
* interface: add `CP_SP` to `get_pid_accel_limits` method signature by @roenthomas
* Honda: use fixed accel min/max constants for Gas Interceptor by @roenthomas
* Tesla: Coop Steering by @AmyJeanes
* New Contributors (sunnypilot/sunnypilot)
* @THERoenPR made their first contribution in "controlsd: add `CP_SP` to `get_pid_accel_limits`"
* @AmyJeanes made their first contribution in "Tesla: Coop Steering"
* @mpurnell1 made their first contribution in "docs: Update README installation branches and discord links"
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2025.001.000...v2025.002.000
sunnypilot Version 2025.001.000 (2025-10-25)
========================
* 🛠️ Major rewrite

View File

@@ -172,6 +172,14 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"ShowTurnSignals", {PERSISTENT | BACKUP, BOOL, "0"}},
{"StandstillTimer", {PERSISTENT | BACKUP, BOOL, "0"}},
{"TrueVEgoUI", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualRadarTracks", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualRadarTracksDelay", {PERSISTENT | BACKUP, FLOAT, "0.0"}},
{"VisualWideCam", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualStyle", {PERSISTENT | BACKUP, INT, "0"}},
{"VisualStyleZoom", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualStyleOverhead", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualStyleOverheadZoom", {PERSISTENT | BACKUP, BOOL, "0"}},
{"VisualStyleOverheadThreshold", {PERSISTENT | BACKUP, INT, "20"}},
// MADS params
{"Mads", {PERSISTENT | BACKUP, BOOL, "1"}},

View File

@@ -25,6 +25,11 @@ void AnnotatedCameraWidget::updateState(const UIState &s) {
// update engageability/experimental mode button
experimental_btn->updateState(s);
dmon.updateState(s);
if (s.scene.visual_style == 0) {
setBackgroundColor(bg_colors[STATUS_DISENGAGED]);
} else {
setBackgroundColor(QColor(0, 0, 0));
}
}
void AnnotatedCameraWidget::initializeGL() {
@@ -35,7 +40,12 @@ void AnnotatedCameraWidget::initializeGL() {
qInfo() << "OpenGL language version:" << QString((const char*)glGetString(GL_SHADING_LANGUAGE_VERSION));
prev_draw_t = millis_since_boot();
setBackgroundColor(bg_colors[STATUS_DISENGAGED]);
auto *s = uiState();
if (s->scene.visual_style == 0) {
setBackgroundColor(bg_colors[STATUS_DISENGAGED]);
} else {
setBackgroundColor(QColor(0, 0, 0));
}
}
mat4 AnnotatedCameraWidget::calcFrameMatrix() {
@@ -118,7 +128,13 @@ void AnnotatedCameraWidget::paintGL() {
} else if (v_ego > 15) {
wide_cam_requested = false;
}
wide_cam_requested = wide_cam_requested && sm["selfdriveState"].getSelfdriveState().getExperimentalMode();
if (s->scene.visual_wide_cam == 1) {
wide_cam_requested = true;
} else if (s->scene.visual_wide_cam == 2) {
wide_cam_requested = false;
} else {
wide_cam_requested = wide_cam_requested && sm["selfdriveState"].getSelfdriveState().getExperimentalMode();
}
}
CameraWidget::setStreamType(wide_cam_requested ? VISION_STREAM_WIDE_ROAD : VISION_STREAM_ROAD);
CameraWidget::setFrameId(sm["modelV2"].getModelV2().getFrameId());

View File

@@ -1,4 +1,5 @@
#include "selfdrive/ui/qt/onroad/model.h"
#include <algorithm>
void ModelRenderer::draw(QPainter &painter, const QRect &surface_rect) {
auto *s = uiState();
@@ -49,8 +50,14 @@ void ModelRenderer::update_leads(const cereal::RadarState::Reader &radar_state,
}
void ModelRenderer::update_model(const cereal::ModelDataV2::Reader &model, const cereal::RadarState::LeadData::Reader &lead) {
auto *s = uiState();
const auto &model_position = model.getPosition();
float max_distance = std::clamp(*(model_position.getX().end() - 1), MIN_DRAW_DISTANCE, MAX_DRAW_DISTANCE);
float max_distance;
if (s->scene.visual_style == 0) {
max_distance = std::clamp(*(model_position.getX().end() - 1), MIN_DRAW_DISTANCE, MAX_DRAW_DISTANCE);
} else {
max_distance = std::clamp(*(model_position.getX().end() - 1), MIN_DRAW_DISTANCE, MAX_DRAW_DISTANCE);
}
// update lane lines
const auto &lane_lines = model.getLaneLines();
@@ -58,7 +65,11 @@ void ModelRenderer::update_model(const cereal::ModelDataV2::Reader &model, const
int max_idx = get_path_length_idx(lane_lines[0], max_distance);
for (int i = 0; i < std::size(lane_line_vertices); i++) {
lane_line_probs[i] = line_probs[i];
mapLineToPolygon(lane_lines[i], 0.025 * lane_line_probs[i], 0, &lane_line_vertices[i], max_idx);
if (s->scene.visual_style == 2) {
mapLineToPolygon(lane_lines[i], 0.075 * lane_line_probs[i], 0, &lane_line_vertices[i], max_idx);
} else {
mapLineToPolygon(lane_lines[i], 0.025 * lane_line_probs[i], 0, &lane_line_vertices[i], max_idx);
}
}
// update road edges
@@ -66,7 +77,11 @@ void ModelRenderer::update_model(const cereal::ModelDataV2::Reader &model, const
const auto &edge_stds = model.getRoadEdgeStds();
for (int i = 0; i < std::size(road_edge_vertices); i++) {
road_edge_stds[i] = edge_stds[i];
mapLineToPolygon(road_edges[i], 0.025, 0, &road_edge_vertices[i], max_idx);
if (s->scene.visual_style == 2) {
mapLineToPolygon(road_edges[i], 0.1, 0, &road_edge_vertices[i], max_idx);
} else {
mapLineToPolygon(road_edges[i], 0.025, 0, &road_edge_vertices[i], max_idx);
}
}
// update path
@@ -79,16 +94,112 @@ void ModelRenderer::update_model(const cereal::ModelDataV2::Reader &model, const
}
void ModelRenderer::drawLaneLines(QPainter &painter) {
// lanelines
for (int i = 0; i < std::size(lane_line_vertices); ++i) {
painter.setBrush(QColor::fromRgbF(1.0, 1.0, 1.0, std::clamp<float>(lane_line_probs[i], 0.0, 0.7)));
painter.drawPolygon(lane_line_vertices[i]);
}
auto *s = uiState();
if (s->scene.visual_style == 2) {
QRectF r = clip_region;
// road edges
for (int i = 0; i < std::size(road_edge_vertices); ++i) {
painter.setBrush(QColor::fromRgbF(1.0, 0, 0, std::clamp<float>(1.0 - road_edge_stds[i], 0.0, 1.0)));
painter.drawPolygon(road_edge_vertices[i]);
qreal horizonY = r.bottom();
if (!road_edge_vertices[0].isEmpty() || !road_edge_vertices[1].isEmpty()) {
qreal leftH = r.top();
qreal rightH = r.top();
if (!road_edge_vertices[0].isEmpty()) {
leftH = std::numeric_limits<qreal>::max();
for (const QPointF &pt : road_edge_vertices[0]) {
if (pt.y() < leftH) leftH = pt.y();
}
}
if (!road_edge_vertices[1].isEmpty()) {
rightH = std::numeric_limits<qreal>::max();
for (const QPointF &pt : road_edge_vertices[1]) {
if (pt.y() < rightH) rightH = pt.y();
}
}
horizonY = std::max(leftH, rightH);
}
painter.fillRect(QRectF(r.left(), horizonY + 0, r.width(), r.bottom() - (horizonY + 0)), QColor("#111111"));
auto buildFill = [&](const QPolygonF &edgeRibbon, bool isLeftSide) -> QPolygonF {
if (edgeRibbon.isEmpty()) return {};
QMap<int, QPointF> byY;
for (const QPointF &pt : edgeRibbon) {
int yi = int(std::round(pt.y()));
if (!byY.contains(yi)) {
byY[yi] = pt;
} else {
if (isLeftSide) {
if (pt.x() > byY[yi].x()) byY[yi] = pt;
} else {
if (pt.x() < byY[yi].x()) byY[yi] = pt;
}
}
}
if (byY.isEmpty()) return {};
QPolygonF curve;
for (auto it = byY.cbegin(); it != byY.cend(); ++it) {
curve << it.value();
}
if (curve.size() < 2) return {};
const qreal topY = curve.first().y();
QPolygonF fill;
if (isLeftSide) {
fill << QPointF(r.left(), topY);
for (const QPointF &pt : curve) fill << pt;
fill << QPointF(r.left(), r.bottom());
} else {
fill << QPointF(r.right(), topY);
for (const QPointF &pt : curve) fill << pt;
fill << QPointF(r.right(), r.bottom());
}
return fill;
};
QPolygonF leftFill = buildFill(road_edge_vertices[0], true);
QPolygonF rightFill = buildFill(road_edge_vertices[1], false);
if (!leftFill.isEmpty()) {
painter.setBrush(QColor("#222222"));
painter.drawPolygon(leftFill);
}
if (!rightFill.isEmpty()) {
painter.setBrush(QColor("#222222"));
painter.drawPolygon(rightFill);
}
for (int i = 0; i < std::size(lane_line_vertices); ++i) {
painter.setBrush(QColor::fromRgbF(0.902, 0.902, 0.902, std::clamp<float>(lane_line_probs[i], 0.0, 0.7)));
painter.drawPolygon(lane_line_vertices[i]);
}
for (int i = 0; i < std::size(road_edge_vertices); ++i) {
painter.setBrush(QColor(0x55, 0x55, 0x55, 255));
painter.drawPolygon(road_edge_vertices[i]);
}
QLinearGradient bgGrad(r.left(), horizonY - 100, r.left(), horizonY + 100);
bgGrad.setColorAt(0.0, QColor("#000000"));
bgGrad.setColorAt(0.5, QColor("#111111"));
bgGrad.setColorAt(1.0, QColor("#111111"));
painter.fillRect(QRectF(r.left(), horizonY - 200, r.width(), 200), bgGrad);
} else {
// lanelines
for (int i = 0; i < std::size(lane_line_vertices); ++i) {
painter.setBrush(QColor::fromRgbF(1.0, 1.0, 1.0, std::clamp<float>(lane_line_probs[i], 0.0, 0.7)));
painter.drawPolygon(lane_line_vertices[i]);
}
// road edges
for (int i = 0; i < std::size(road_edge_vertices); ++i) {
painter.setBrush(QColor::fromRgbF(1.0, 0, 0, std::clamp<float>(1.0 - road_edge_stds[i], 0.0, 1.0)));
painter.drawPolygon(road_edge_vertices[i]);
}
}
}
@@ -175,6 +286,7 @@ QColor ModelRenderer::blendColors(const QColor &start, const QColor &end, float
void ModelRenderer::drawLead(QPainter &painter, const cereal::RadarState::LeadData::Reader &lead_data,
const QPointF &vd, const QRect &surface_rect) {
auto *s = uiState();
const float speedBuff = 10.;
const float leadBuff = 40.;
const float d_rel = lead_data.getDRel();
@@ -197,20 +309,133 @@ void ModelRenderer::drawLead(QPainter &painter, const cereal::RadarState::LeadDa
float g_yo = sz / 10;
QPointF glow[] = {{x + (sz * 1.35) + g_xo, y + sz + g_yo}, {x, y - g_yo}, {x - (sz * 1.35) - g_xo, y + sz + g_yo}};
painter.setBrush(QColor(218, 202, 37, 255));
if (s->scene.visual_style == 2) {
painter.setBrush(QColor(0xE6, 0xE6, 0xE6, 255));
} else {
painter.setBrush(QColor(218, 202, 37, 255));
}
painter.drawPolygon(glow, std::size(glow));
// chevron
QPointF chevron[] = {{x + (sz * 1.25), y + sz}, {x, y}, {x - (sz * 1.25), y + sz}};
painter.setBrush(QColor(201, 34, 49, fillAlpha));
if (s->scene.visual_style == 2) {
painter.setBrush(QColor(0, 0, 0, fillAlpha));
} else {
painter.setBrush(QColor(201, 34, 49, fillAlpha));
}
painter.drawPolygon(chevron, std::size(chevron));
}
// Projects a point in car to space to the corresponding point in full frame image space.
float mapRange(float x, float in_min, float in_max, float out_min, float out_max) {
if (in_min < in_max) {
x = std::clamp(x, in_min, in_max);
} else {
x = std::clamp(x, in_max, in_min);
}
return out_min + (x - in_min) * (out_max - out_min) / (in_max - in_min);
}
// Projects a point in car space to the corresponding point in full frame image space.
bool ModelRenderer::mapToScreen(float in_x, float in_y, float in_z, QPointF *out) {
auto *s = uiState();
auto &sm = *(s->sm);
float blend_speed_mph = fabsf(sm["carState"].getCarState().getVEgo() * 2.23694f);
Eigen::Vector3f input(in_x, in_y, in_z);
if ((s->scene.visual_style_zoom == 1 || s->scene.visual_style_zoom == 2) && s->scene.visual_style != 0) {
float zoom_start = 20.0f;
float zoom_end = 50.0f;
if (s->scene.visual_style_zoom == 2) {
std::swap(zoom_start, zoom_end);
}
float IN_X_OFFSET = mapRange(blend_speed_mph, zoom_start, zoom_end, 0.0f, 24.0f);
float IN_Y_OFFSET = mapRange(blend_speed_mph, zoom_start, zoom_end, 1.0f, 2.0f);
float IN_Z_OFFSET = mapRange(blend_speed_mph, zoom_start, zoom_end, 0.0f, 5.0f);
float PITCH_DEG = mapRange(blend_speed_mph, zoom_start, zoom_end, 0.0f, 5.0f);
input = Eigen::Vector3f(in_x + IN_X_OFFSET, in_y / IN_Y_OFFSET, in_z + IN_Z_OFFSET);
Eigen::AngleAxisf pitch_rot(PITCH_DEG * M_PI / 180.0f, Eigen::Vector3f::UnitY());
input = pitch_rot * input;
}
auto pt = car_space_transform * input;
*out = QPointF(pt.x() / pt.z(), pt.y() / pt.z());
bool normal_valid = (pt.z() > 1e-3f &&
std::isfinite(pt.x()) && std::isfinite(pt.y()));
QPointF normal_view;
if (normal_valid) {
normal_view = QPointF(pt.x() / pt.z(), pt.y() / pt.z());
}
const float base_scale_x = 20.0f;
const float base_scale_y = 15.0f;
const float y_offset = 450.0f;
float factor_scale_x = 0.0f;
if (blend_speed_mph > 0.0f) {
if (s->scene.visual_style_overhead_zoom == 1) {
factor_scale_x = mapRange(blend_speed_mph, 0.0f, 50.0f, 30.0f, 0.0f);
} else if (s->scene.visual_style_overhead_zoom == 2) {
factor_scale_x = mapRange(blend_speed_mph, 50.0f, 0.0f, 30.0f, 0.0f);
}
}
float scale_x = base_scale_x + factor_scale_x;
float scale_y = base_scale_y;
QPointF topdown_view(
clip_region.center().x() + in_y * scale_x,
(clip_region.bottom() - y_offset) - in_x * scale_y
);
if ((s->scene.visual_style_overhead == 1 || s->scene.visual_style_overhead == 2) && s->scene.visual_style != 0) {
static float blend = 0.0f;
static float target_blend = 0.0f;
static double last_t = millis_since_boot();
const bool inverted = (s->scene.visual_style_overhead == 2);
const float threshold = s->scene.visual_style_overhead_threshold;
const float hysteresis = 5.0f;
if (!inverted) {
if (target_blend < 0.5f && blend_speed_mph > threshold) {
target_blend = 1.0f;
} else if (target_blend > 0.5f && blend_speed_mph < threshold - hysteresis) {
target_blend = 0.0f;
}
} else {
if (target_blend < 0.5f && blend_speed_mph < threshold) {
target_blend = 1.0f;
} else if (target_blend > 0.5f && blend_speed_mph > threshold + hysteresis) {
target_blend = 0.0f;
}
}
double now = millis_since_boot();
double dt = (now - last_t) / 1000.0;
last_t = now;
const float transition_time = 1.50f;
float step = dt / transition_time;
if (blend < target_blend) {
blend = std::min(blend + step, target_blend);
} else if (blend > target_blend) {
blend = std::max(blend - step, target_blend);
}
if (!normal_valid) return false;
*out = QPointF(
(1 - blend) * normal_view.x() + blend * topdown_view.x(),
(1 - blend) * normal_view.y() + blend * topdown_view.y()
);
} else {
if (!normal_valid) return false;
*out = normal_view;
}
return clip_region.contains(*out);
}

View File

@@ -196,6 +196,7 @@ mat4 CameraWidget::calcFrameMatrix() {
}
void CameraWidget::paintGL() {
auto *s = uiState();
glClearColor(bg.redF(), bg.greenF(), bg.blueF(), bg.alphaF());
glClear(GL_STENCIL_BUFFER_BIT | GL_COLOR_BUFFER_BIT);
@@ -248,7 +249,9 @@ void CameraWidget::paintGL() {
glUniformMatrix4fv(program->uniformLocation("uTransform"), 1, GL_TRUE, frame_mat.v);
glEnableVertexAttribArray(0);
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_BYTE, (const void *)0);
if (s->scene.visual_style == 0) {
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_BYTE, (const void *)0);
}
glDisableVertexAttribArray(0);
glBindVertexArray(0);
glBindTexture(GL_TEXTURE_2D, 0);

View File

@@ -11,6 +11,18 @@ VisualsPanel::VisualsPanel(QWidget *parent) : QWidget(parent) {
param_watcher = new ParamWatcher(this);
connect(param_watcher, &ParamWatcher::paramChanged, [=](const QString &param_name, const QString &param_value) {
paramsRefresh();
if (param_name == "VisualStyle") {
visual_style_value = param_value.toInt();
} else if (param_name == "VisualStyleOverhead") {
visual_style_overhead_value = param_value.toInt();
} else if (param_name == "VisualRadarTracks") {
bool radar_tracks_enabled = param_value.toInt() != 0;
visual_radar_tracks_delay_settings->setVisible(radar_tracks_enabled);
}
visual_style_zoom_settings->setVisible(visual_style_value != 0);
visual_style_overhead_settings->setVisible(visual_style_value != 0);
visual_style_overhead_zoom_settings->setVisible(visual_style_value != 0 && visual_style_overhead_value != 0);
visual_style_overhead_threshold_settings->setVisible(visual_style_value != 0 && visual_style_overhead_value != 0);
});
main_layout = new QStackedLayout(this);
@@ -90,6 +102,13 @@ VisualsPanel::VisualsPanel(QWidget *parent) : QWidget(parent) {
"",
false,
},
{
"VisualRadarTracks",
tr("Show Radar Tracks"),
tr("Shows what the cars radar sees."),
"",
false,
},
};
// Add regular toggles first
@@ -116,6 +135,111 @@ VisualsPanel::VisualsPanel(QWidget *parent) : QWidget(parent) {
param_watcher->addParam(param);
}
// Visuals: Radar Tracks Delay
visual_radar_tracks_delay_settings = new OptionControlSP("VisualRadarTracksDelay", tr("Adjust Visual Radar Tracks Delay"),
tr("Delays radar tracks to better match what you see through the camera."),
"", {0, 100}, 10, false, nullptr, true);
connect(visual_radar_tracks_delay_settings, &OptionControlSP::updateLabels, [=]() {
float radar_tracks_delay_value = QString::fromStdString(params.get("VisualRadarTracksDelay")).toFloat();
visual_radar_tracks_delay_settings->setLabel(QString::number(radar_tracks_delay_value, 'f', 1) + " s");
});
float radar_tracks_delay_value = QString::fromStdString(params.get("VisualRadarTracksDelay")).toFloat();
visual_radar_tracks_delay_settings->setLabel(QString::number(radar_tracks_delay_value, 'f', 1) + " s");
list->addItem(visual_radar_tracks_delay_settings);
// Wide Cam
std::vector<QString> visual_wide_cam_settings_texts{tr("Auto"), tr("On"), tr("Off")};
visual_wide_cam_settings = new ButtonParamControlSP(
"VisualWideCam", tr("Wide Cam"), tr("Override the wide cam view regardless of experimental mode status."),
"",
visual_wide_cam_settings_texts,
250);
list->addItem(visual_wide_cam_settings);
// Visual Style
std::vector<QString> visual_style_settings_texts{tr("Default"), tr("Minimal"), tr("Vision")};
visual_style_settings = new ButtonParamControlSP(
"VisualStyle", tr("Visual Style"),
tr(
"Switch between different on-road visualization layouts."
"<ul style='margin-left: 10px; margin-top: 4px;'>"
"<li><b>Default:</b> Standard OpenPilot layout with camera and path view.</li>"
"<li><b>Minimal:</b> Clean interface without camera feed or extra elements.</li>"
"<li><b>Vision:</b> Experimental layout that focuses on model perception and environment.</li>"
"</ul>"
),
"",
visual_style_settings_texts,
380);
list->addItem(visual_style_settings);
// Visual Style Zoom
std::vector<QString> visual_style_zoom_settings_texts{tr("Disabled"), tr("Enabled"), tr("Inverted")};
visual_style_zoom_settings = new ButtonParamControlSP(
"VisualStyleZoom", tr("Visual Style Zoom"),
tr(
"Enables dynamic zooming based on driving speed in the selected visual style."
"<ul style='margin-left: 10px; margin-top: 4px;'>"
"<li><b>Disabled:</b> Keeps the zoom fixed.</li>"
"<li><b>Enabled:</b> Zooms in at low speed and out at high speed.</li>"
"<li><b>Inverted:</b> Reverses the zoom behavior.</li>"
"</ul>"
),
"",
visual_style_zoom_settings_texts,
380);
list->addItem(visual_style_zoom_settings);
// Visual Style Overhead
std::vector<QString> visual_style_overhead_settings_texts{tr("Disabled"), tr("Enabled"), tr("Inverted")};
visual_style_overhead_settings = new ButtonParamControlSP(
"VisualStyleOverhead", tr("Visual Style Overhead"),
tr(
"Toggles an overhead (top-down) camera view for a 2D-style perspective."
"<ul style='margin-left: 10px; margin-top: 4px;'>"
"<li><b>Disabled:</b> Keeps the standard forward 3D view.</li>"
"<li><b>Enabled:</b> Switches to overhead view when active.</li>"
"<li><b>Inverted:</b> Reverses when the transition happens.</li>"
"</ul>"
),
"",
visual_style_overhead_settings_texts,
380);
list->addItem(visual_style_overhead_settings);
// Visual Style Overhead Zoom
std::vector<QString> visual_style_overhead_zoom_settings_texts{tr("Disabled"), tr("Enabled"), tr("Inverted")};
visual_style_overhead_zoom_settings = new ButtonParamControlSP(
"VisualStyleOverheadZoom", tr("Visual Style Overhead Zoom"),
tr(
"Controls zooming behavior while in overhead mode."
"<ul style='margin-left: 10px; margin-top: 4px;'>"
"<li><b>Disabled:</b> Keeps a fixed zoom level in overhead mode.</li>"
"<li><b>Enabled:</b> Zooms dynamically based on speed while overhead.</li>"
"<li><b>Inverted:</b> Opposite zoom direction.</li>"
"</ul>"
),
"",
visual_style_overhead_zoom_settings_texts,
380);
list->addItem(visual_style_overhead_zoom_settings);
// Visual Style Overhead Threshold
visual_style_overhead_threshold_settings = new OptionControlSP(
"VisualStyleOverheadThreshold", tr("Visual Style Overhead Threshold"),
tr("Sets the speed (in mph) where the display transitions between normal and overhead view."),
"", {10, 80}, 5, false, nullptr, false);
auto updateThresholdLabel = [=]() {
int mph = QString::fromStdString(params.get("VisualStyleOverheadThreshold")).toInt();
visual_style_overhead_threshold_settings->setLabel(QString("%1 mph").arg(mph));
};
connect(visual_style_overhead_threshold_settings, &OptionControlSP::updateLabels, updateThresholdLabel);
updateThresholdLabel();
list->addItem(visual_style_overhead_threshold_settings);
// Visuals: Display Metrics below Chevron
std::vector<QString> chevron_info_settings_texts{tr("Off"), tr("Distance"), tr("Speed"), tr("Time"), tr("All")};
chevron_info_settings = new ButtonParamControlSP(
@@ -136,6 +260,19 @@ VisualsPanel::VisualsPanel(QWidget *parent) : QWidget(parent) {
380);
list->addItem(dev_ui_settings);
bool radar_tracks_enabled = QString::fromStdString(params.get("VisualRadarTracks")).toInt() != 0;
visual_radar_tracks_delay_settings->setVisible(radar_tracks_enabled);
param_watcher->addParam("VisualRadarTracks");
visual_style_value = QString::fromStdString(params.get("VisualStyle")).toInt();
visual_style_overhead_value = QString::fromStdString(params.get("VisualStyleOverhead")).toInt();
visual_style_zoom_settings->setVisible(visual_style_value != 0);
visual_style_overhead_settings->setVisible(visual_style_value != 0);
visual_style_overhead_zoom_settings->setVisible(visual_style_value != 0 && visual_style_overhead_value != 0);
visual_style_overhead_threshold_settings->setVisible(visual_style_value != 0 && visual_style_overhead_value != 0);
param_watcher->addParam("VisualStyle");
param_watcher->addParam("VisualStyleOverhead");
sunnypilotScroller = new ScrollViewSP(list, this);
vlayout->addWidget(sunnypilotScroller);
@@ -191,4 +328,19 @@ void VisualsPanel::paramsRefresh() {
if (dev_ui_settings) {
dev_ui_settings->refresh();
}
if (visual_wide_cam_settings) {
visual_wide_cam_settings->refresh();
}
if (visual_style_settings) {
visual_style_settings->refresh();
}
if (visual_style_zoom_settings) {
visual_style_zoom_settings->refresh();
}
if (visual_style_overhead_settings) {
visual_style_overhead_settings->refresh();
}
if (visual_style_overhead_zoom_settings) {
visual_style_overhead_zoom_settings->refresh();
}
}

View File

@@ -32,4 +32,14 @@ protected:
ButtonParamControlSP *dev_ui_settings;
bool has_longitudinal_control = false;
OptionControlSP *visual_radar_tracks_delay_settings;
ButtonParamControlSP *visual_wide_cam_settings;
int visual_style_value = 0;
int visual_style_overhead_value = 0;
ButtonParamControlSP *visual_style_settings;
ButtonParamControlSP *visual_style_zoom_settings;
ButtonParamControlSP *visual_style_overhead_settings;
ButtonParamControlSP *visual_style_overhead_zoom_settings;
OptionControlSP *visual_style_overhead_threshold_settings;
};

View File

@@ -8,6 +8,12 @@
#include "selfdrive/ui/sunnypilot/qt/onroad/model.h"
void ModelRendererSP::drawRadarPoint(QPainter &painter, const QPointF &pos, float v_rel, float radius) {
painter.setBrush(QColor(255, 255, 255, 200));
painter.setPen(Qt::NoPen);
painter.drawEllipse(pos, radius, radius);
}
void ModelRendererSP::update_model(const cereal::ModelDataV2::Reader &model, const cereal::RadarState::LeadData::Reader &lead) {
ModelRenderer::update_model(model, lead);
const auto &model_position = model.getPosition();
@@ -67,6 +73,26 @@ void ModelRendererSP::draw(QPainter &painter, const QRect &surface_rect) {
const bool right_blindspot = car_state.getRightBlindspot();
drawBlindspot(painter, surface_rect, left_blindspot, right_blindspot);
}
if (s->scene.visual_radar_tracks) {
if (sm.alive("liveTracks") && sm.rcv_frame("liveTracks") >= s->scene.started_frame) {
const auto &tracks = sm["liveTracks"].getLiveTracks().getPoints();
for (const auto &track : tracks) {
if (!std::isfinite(track.getDRel()) || !std::isfinite(track.getYRel())) continue;
float t_lag = s->scene.visual_radar_tracks_delay;
float d_pred = track.getDRel();
float y_pred = track.getYRel();
if (t_lag > 0.0f) {
d_pred += track.getVRel() * t_lag + 0.5f * track.getARel() * t_lag * t_lag;
}
QPointF screen_pt;
if (mapToScreen(d_pred, -y_pred, path_offset_z, &screen_pt)) {
drawRadarPoint(painter, screen_pt, track.getVRel(), 10.0f);
}
}
}
}
drawLeadStatus(painter, surface_rect.height(), surface_rect.width());
painter.restore();

View File

@@ -28,4 +28,6 @@ private:
// Lead status animation
float lead_status_alpha = 0.0f;
void drawRadarPoint(QPainter &painter, const QPointF &pos, float v_rel, float radius = 10.0f);
};

View File

@@ -29,7 +29,7 @@ UIStateSP::UIStateSP(QObject *parent) : UIState(parent) {
"wideRoadCameraState", "managerState", "selfdriveState", "longitudinalPlan",
"modelManagerSP", "selfdriveStateSP", "longitudinalPlanSP", "backupManagerSP",
"carControl", "gpsLocationExternal", "gpsLocation", "liveTorqueParameters",
"carStateSP", "liveParameters", "liveMapDataSP", "carParamsSP"
"carStateSP", "liveParameters", "liveMapDataSP", "carParamsSP", "liveTracks"
});
// update timer
@@ -44,6 +44,14 @@ UIStateSP::UIStateSP(QObject *parent) : UIState(parent) {
});
param_watcher->addParam("DevUIInfo");
param_watcher->addParam("StandstillTimer");
param_watcher->addParam("VisualRadarTracks");
param_watcher->addParam("VisualRadarTracksDelay");
param_watcher->addParam("VisualWideCam");
param_watcher->addParam("VisualStyle");
param_watcher->addParam("VisualStyleZoom");
param_watcher->addParam("VisualStyleOverhead");
param_watcher->addParam("VisualStyleOverheadZoom");
param_watcher->addParam("VisualStyleOverheadThreshold");
}
// This method overrides completely the update method from the parent class intentionally.
@@ -76,6 +84,17 @@ void ui_update_params_sp(UIStateSP *s) {
s->scene.chevron_info = std::atoi(params.get("ChevronInfo").c_str());
s->scene.blindspot_ui = params.getBool("BlindSpot");
s->scene.rainbow_mode = params.getBool("RainbowMode");
s->scene.visual_radar_tracks = QString::fromStdString(params.get("VisualRadarTracks")).toInt();
s->scene.visual_radar_tracks_delay = QString::fromStdString(params.get("VisualRadarTracksDelay")).toFloat();
s->scene.visual_wide_cam = QString::fromStdString(params.get("VisualWideCam")).toInt();
s->scene.visual_style = QString::fromStdString(params.get("VisualStyle")).toInt();
s->scene.visual_style_zoom = QString::fromStdString(params.get("VisualStyleZoom")).toInt();
s->scene.visual_style_overhead = QString::fromStdString(params.get("VisualStyleOverhead")).toInt();
s->scene.visual_style_overhead_zoom = QString::fromStdString(params.get("VisualStyleOverheadZoom")).toInt();
s->scene.visual_style_overhead_threshold = QString::fromStdString(params.get("VisualStyleOverheadThreshold")).toInt();
}
void UIStateSP::reset_onroad_sleep_timer(OnroadTimerStatusToggle toggleTimerStatus) {

View File

@@ -21,4 +21,12 @@ typedef struct UISceneSP : UIScene {
int chevron_info;
bool blindspot_ui;
bool rainbow_mode;
int visual_radar_tracks = 0;
float visual_radar_tracks_delay = 0;
int visual_wide_cam = 0;
int visual_style = 0;
int visual_style_zoom = 0;
int visual_style_overhead = 0;
int visual_style_overhead_zoom = 0;
int visual_style_overhead_threshold = 20.0;
} UISceneSP;

View File

@@ -1 +1 @@
#define SUNNYPILOT_VERSION "2025.002.000"
#define SUNNYPILOT_VERSION "2025.003.000"

View File

@@ -15,6 +15,8 @@ from openpilot.sunnypilot.selfdrive.controls.lib.speed_limit.helpers import set_
import openpilot.system.sentry as sentry
from sunnypilot.sunnylink.statsd import STATSLOGSP
def log_fingerprint(CP: structs.CarParams) -> None:
if CP.carFingerprint == "MOCK":
@@ -100,6 +102,9 @@ def setup_interfaces(CI: CarInterfaceBase, params: Params = None) -> None:
_initialize_torque_lateral_control(CI, CP, enforce_torque, nnlc_enabled)
_cleanup_unsupported_params(CP, CP_SP)
STATSLOGSP.raw('sunnypilot.car_params', CP.to_dict())
# STATSLOGSP.raw('sunnypilot_params.car_params_sp', CP_SP.to_dict()) # https://github.com/sunnypilot/opendbc/pull/361
def initialize_params(params) -> list[dict[str, Any]]:
keys: list = []

View File

@@ -16,8 +16,9 @@ from functools import partial
from openpilot.common.params import Params
from openpilot.common.realtime import set_core_affinity
from openpilot.common.swaglog import cloudlog
from openpilot.system.hardware.hw import Paths
from openpilot.system.athena.athenad import ws_send, jsonrpc_handler, \
recv_queue, UploadQueueCache, upload_queue, cur_upload_items, backoff, ws_manage, log_handler, start_local_proxy_shim, upload_handler
recv_queue, UploadQueueCache, upload_queue, cur_upload_items, backoff, ws_manage, log_handler, start_local_proxy_shim, upload_handler, stat_handler
from websocket import (ABNF, WebSocket, WebSocketException, WebSocketTimeoutException,
create_connection, WebSocketConnectionClosedException)
@@ -33,9 +34,6 @@ SUNNYLINK_RECONNECT_TIMEOUT_S = 70 # FYI changing this will also would require
DISALLOW_LOG_UPLOAD = threading.Event()
params = Params()
sunnylink_dongle_id = params.get("SunnylinkDongleId")
sunnylink_api = SunnylinkApi(sunnylink_dongle_id)
def handle_long_poll(ws: WebSocket, exit_event: threading.Event | None) -> None:
cloudlog.info("sunnylinkd.handle_long_poll started")
@@ -51,7 +49,7 @@ def handle_long_poll(ws: WebSocket, exit_event: threading.Event | None) -> None:
threading.Thread(target=ws_queue, args=(end_event,), name='ws_queue'),
threading.Thread(target=upload_handler, args=(end_event,), name='upload_handler'),
# threading.Thread(target=sunny_log_handler, args=(end_event, comma_prime_cellular_end_event), name='log_handler'),
# threading.Thread(target=stat_handler, args=(end_event,), name='stat_handler'),
threading.Thread(target=stat_handler, args=(end_event, Paths.stats_sp_root(), True), name='stat_handler'),
] + [
threading.Thread(target=jsonrpc_handler, args=(end_event, partial(startLocalProxy, end_event),), name=f'worker_{x}')
for x in range(HANDLER_THREADS)
@@ -132,6 +130,8 @@ def ws_ping(ws: WebSocket, end_event: threading.Event) -> None:
def ws_queue(end_event: threading.Event) -> None:
sunnylink_dongle_id = params.get("SunnylinkDongleId")
sunnylink_api = SunnylinkApi(sunnylink_dongle_id)
resume_requested = False
tries = 0
@@ -233,6 +233,9 @@ def saveParams(params_to_update: dict[str, str], compression: bool = False) -> N
def startLocalProxy(global_end_event: threading.Event, remote_ws_uri: str, local_port: int) -> dict[str, int]:
sunnylink_dongle_id = params.get("SunnylinkDongleId")
sunnylink_api = SunnylinkApi(sunnylink_dongle_id)
cloudlog.debug("athena.startLocalProxy.starting")
ws = create_connection(
remote_ws_uri,
@@ -254,6 +257,8 @@ def main(exit_event: threading.Event = None):
cloudlog.info("Waiting for sunnylink registration to complete")
time.sleep(10)
sunnylink_dongle_id = params.get("SunnylinkDongleId")
sunnylink_api = SunnylinkApi(sunnylink_dongle_id)
UploadQueueCache.initialize(upload_queue)
ws_uri = f"{SUNNYLINK_ATHENA_HOST}"

278
sunnypilot/sunnylink/statsd.py Executable file
View File

@@ -0,0 +1,278 @@
#!/usr/bin/env python3
import base64
import json
import os
import threading
import traceback
import zmq
import time
import uuid
from pathlib import Path
from collections import defaultdict
from datetime import datetime, UTC
from openpilot.common.params import Params
from cereal.messaging import SubMaster
from openpilot.system.hardware.hw import Paths
from openpilot.common.swaglog import cloudlog
from openpilot.system.hardware import HARDWARE
from openpilot.common.file_helpers import atomic_write_in_dir
from openpilot.system.version import get_build_metadata
from openpilot.system.loggerd.config import STATS_DIR_FILE_LIMIT, STATS_SOCKET, STATS_FLUSH_TIME_S
from openpilot.system.statsd import METRIC_TYPE, StatLogSP
from openpilot.common.realtime import Ratekeeper
STATSLOGSP = StatLogSP(intercept=False)
def sp_stats(end_event):
"""Collect sunnypilot-specific statistics and send as raw metrics."""
rk = Ratekeeper(.1, print_delay_threshold=None)
statlogsp = STATSLOGSP
params = Params()
def flatten_dict(d, parent_key='', sep='.'):
items = {}
if isinstance(d, dict):
for k, v in d.items():
new_key = f"{parent_key}{sep}{k}" if parent_key else k
items.update(flatten_dict(v, new_key, sep=sep))
elif isinstance(d, (list, tuple)):
for i, v in enumerate(d):
new_key = f"{parent_key}[{i}]"
items.update(flatten_dict(v, new_key, sep=sep))
else:
items[parent_key] = d
return items
# Collect sunnypilot parameters
stats_dict = {}
param_keys = [
'SunnylinkEnabled',
'AutoLaneChangeBsmDelay',
'AutoLaneChangeTimer',
'CarPlatformBundle',
'CurrentRoute',
'DevUIInfo',
'EnableCopyparty',
'IntelligentCruiseButtonManagement',
'QuietMode',
'RainbowMode',
'ShowAdvancedControls',
'Mads',
'MadsMainCruiseAllowed',
'MadsSteeringMode',
'MadsUnifiedEngagementMode',
'ModelManager_ActiveBundle',
'ModelManager_Favs',
'EnableSunnylinkUploader',
'SunnylinkEnabled',
'InstallDate',
'UptimeOffroad',
'UptimeOnroad',
]
while not end_event.is_set():
try:
for key in param_keys:
try:
value = params.get(key)
except Exception as e:
stats_dict[key] = e
continue
if value is None:
continue
if isinstance(value, (dict, list, tuple)):
stats_dict.update(flatten_dict(value, key))
else:
stats_dict[key] = value
if stats_dict:
statlogsp.raw('sunnypilot.device_params', stats_dict)
except Exception as e:
cloudlog.error(f"Exception {e}")
finally:
rk.keep_time()
def stats_main(end_event):
comma_dongle_id = Params().get("DongleId")
sunnylink_dongle_id = Params().get("SunnylinkDongleId")
def get_influxdb_line(measurement: str, value: float | dict[str, float], timestamp: datetime, tags: dict) -> str:
res = f"{measurement}"
for k, v in tags.items():
res += f",{k}={str(v)}"
res += " "
if isinstance(value, float):
value = {'value': value}
for k, v in value.items():
res += f"{k}={str(v)},"
res += f"sunnylink_dongle_id=\"{sunnylink_dongle_id}\",comma_dongle_id=\"{comma_dongle_id}\" {int(timestamp.timestamp() * 1e9)}\n"
return res
def get_influxdb_line_raw(measurement: str, value: dict, timestamp: datetime, tags: dict) -> str:
res = f"{measurement}"
try:
custom_tags = ""
for k, v in tags.items():
custom_tags += f",{k}={str(v)}"
res += custom_tags
fields = ""
for k, v in value.items():
# Skip complex types - only keep simple scalar values
if isinstance(v, (dict, list, bytes, bytearray)):
continue
fields += f"{k}={json.dumps(v)},"
res += f" {fields}"
except Exception as e:
cloudlog.error(f"Unable to get influxdb line for: {value}")
res += f",invalid=1 reason={e},"
res += f"sunnylink_dongle_id=\"{sunnylink_dongle_id}\",comma_dongle_id=\"{comma_dongle_id}\" {int(timestamp.timestamp() * 1e9)}\n"
return res
# open statistics socket
ctx = zmq.Context.instance()
sock = ctx.socket(zmq.PULL)
sock.bind(f"{STATS_SOCKET}_sp")
STATS_DIR = Paths.stats_sp_root()
# initialize stats directory
Path(STATS_DIR).mkdir(parents=True, exist_ok=True)
build_metadata = get_build_metadata()
# initialize tags
tags = {
'started': False,
'version': build_metadata.openpilot.version,
'branch': build_metadata.channel,
'dirty': build_metadata.openpilot.is_dirty,
'origin': build_metadata.openpilot.git_normalized_origin,
'deviceType': HARDWARE.get_device_type(),
}
# subscribe to deviceState for started state
sm = SubMaster(['deviceState'])
idx = 0
boot_uid = str(uuid.uuid4())[:8]
last_flush_time = time.monotonic()
gauges = {}
samples: dict[str, list[float]] = defaultdict(list)
raws: dict = defaultdict()
try:
while not end_event.is_set():
started_prev = sm['deviceState'].started
sm.update()
# Update metrics
while True:
try:
metric = sock.recv_string(zmq.NOBLOCK)
try:
metric_type = metric.split('|')[1]
metric_name = metric.split(':')[0]
metric_value_raw = metric.split('|')[0].split(':')[1]
if metric_type == METRIC_TYPE.GAUGE:
metric_value = float(metric_value_raw)
gauges[metric_name] = metric_value
elif metric_type == METRIC_TYPE.SAMPLE:
metric_value = float(metric_value_raw)
samples[metric_name].append(metric_value)
elif metric_type == METRIC_TYPE.RAW:
raws[metric_name] = metric_value_raw
else:
cloudlog.event("unknown metric type", metric_type=metric_type)
except Exception:
print(traceback.format_exc())
cloudlog.event("malformed metric", metric=metric)
except zmq.error.Again:
break
# flush when started state changes or after FLUSH_TIME_S
if (time.monotonic() > last_flush_time + STATS_FLUSH_TIME_S) or (sm['deviceState'].started != started_prev):
result = ""
current_time = datetime.now(UTC)
tags['started'] = sm['deviceState'].started
for key, value in raws.items():
decoded_value = json.loads(base64.b64decode(value).decode('utf-8'))
result += get_influxdb_line_raw(key, decoded_value, current_time, tags)
for key, value in gauges.items():
result += get_influxdb_line(f"gauge.{key}", value, current_time, tags)
for key, values in samples.items():
values.sort()
sample_count = len(values)
sample_sum = sum(values)
stats = {
'count': sample_count,
'min': values[0],
'max': values[-1],
'mean': sample_sum / sample_count,
}
for percentile in [0.05, 0.5, 0.95]:
value = values[int(round(percentile * (sample_count - 1)))]
stats[f"p{int(percentile * 100)}"] = value
result += get_influxdb_line(f"sample.{key}", stats, current_time, tags)
# clear intermediate data
gauges.clear()
samples.clear()
last_flush_time = time.monotonic()
# check that we aren't filling up the drive
if len(os.listdir(STATS_DIR)) < STATS_DIR_FILE_LIMIT:
if len(result) > 0:
stats_path = os.path.join(STATS_DIR, f"{boot_uid}_{idx}")
with atomic_write_in_dir(stats_path) as f:
f.write(result)
idx += 1
else:
cloudlog.error("stats dir full")
finally:
sock.close()
ctx.term()
def main():
rk = Ratekeeper(1, print_delay_threshold=None)
end_event = threading.Event()
threads = [
threading.Thread(target=stats_main, args=(end_event,)),
threading.Thread(target=sp_stats, args=(end_event,)),
]
for t in threads:
t.start()
try:
while all(t.is_alive() for t in threads):
rk.keep_time()
finally:
end_event.set()
for t in threads:
t.join()
if __name__ == "__main__":
main()

View File

@@ -744,26 +744,40 @@ def log_handler(end_event: threading.Event, log_attr_name=LOG_ATTR_NAME) -> None
cloudlog.exception("athena.log_handler.exception")
def stat_handler(end_event: threading.Event) -> None:
STATS_DIR = Paths.stats_root()
def stat_handler(end_event: threading.Event, stats_dir=None, is_sunnylink=False) -> None:
stats_dir = stats_dir or Paths.stats_root()
last_scan = 0.0
while not end_event.is_set():
curr_scan = time.monotonic()
try:
if curr_scan - last_scan > 10:
stat_filenames = list(filter(lambda name: not name.startswith(tempfile.gettempprefix()), os.listdir(STATS_DIR)))
stat_filenames = list(filter(lambda name: not name.startswith(tempfile.gettempprefix()), os.listdir(stats_dir)))
if len(stat_filenames) > 0:
stat_path = os.path.join(STATS_DIR, stat_filenames[0])
stat_path = os.path.join(stats_dir, stat_filenames[0])
with open(stat_path) as f:
payload = f.read()
is_compressed = False
# Log the current size of the file
if is_sunnylink:
# Compress and encode the data if it exceeds the maximum size
compressed_data = gzip.compress(payload.encode())
payload = base64.b64encode(compressed_data).decode()
is_compressed = True
jsonrpc = {
"method": "storeStats",
"params": {
"stats": f.read()
"stats": payload
},
"jsonrpc": "2.0",
"id": stat_filenames[0]
}
if is_sunnylink and is_compressed:
jsonrpc["params"]["compressed"] = is_compressed
low_priority_send_queue.put_nowait(json.dumps(jsonrpc))
os.remove(stat_path)
last_scan = curr_scan

View File

@@ -55,6 +55,13 @@ class Paths:
else:
return "/data/stats/"
@staticmethod
def stats_sp_root() -> str:
if PC:
return str(Path(Paths.comma_home()) / "stats")
else:
return "/data/stats_sp/"
@staticmethod
def config_root() -> str:
if PC:

View File

@@ -164,6 +164,7 @@ procs = [
# sunnylink <3
DaemonProcess("manage_sunnylinkd", "sunnypilot.sunnylink.athena.manage_sunnylinkd", "SunnylinkdPid"),
PythonProcess("sunnylink_registration_manager", "sunnypilot.sunnylink.registration_manager", sunnylink_need_register_shim),
PythonProcess("statsd_sp", "sunnypilot.sunnylink.statsd", and_(always_run, sunnylink_ready_shim)),
]
# sunnypilot

View File

@@ -1,11 +1,15 @@
#!/usr/bin/env python3
import base64
import json
import os
from decimal import Decimal
import zmq
import time
import uuid
from pathlib import Path
from collections import defaultdict
from datetime import datetime, UTC
from datetime import datetime, UTC, date
from typing import NoReturn
from openpilot.common.params import Params
@@ -21,18 +25,21 @@ from openpilot.system.loggerd.config import STATS_DIR_FILE_LIMIT, STATS_SOCKET,
class METRIC_TYPE:
GAUGE = 'g'
SAMPLE = 'sa'
RAW = 'r'
class StatLog:
def __init__(self):
self.pid = None
self.zctx = None
self.sock = None
self.stats_socket = STATS_SOCKET
def connect(self) -> None:
self.zctx = zmq.Context()
self.zctx = zmq.Context.instance() or zmq.Context()
self.sock = self.zctx.socket(zmq.PUSH)
self.sock.setsockopt(zmq.LINGER, 10)
self.sock.connect(STATS_SOCKET)
self.sock.connect(self.stats_socket)
self.pid = os.getpid()
def __del__(self):
@@ -60,6 +67,50 @@ class StatLog:
self._send(f"{name}:{value}|{METRIC_TYPE.SAMPLE}")
class StatLogSP(StatLog):
def __init__(self, intercept=True):
"""
Initializes the class instance with an optional parameter to determine
if statistical logging should be configured or not.
:param intercept: A boolean flag that indicates whether to initialize
the `comma_statlog`. If True, the `comma_statlog` attribute is
instantiated as a `StatLog` object. Defaults to True.
"""
super().__init__()
self.comma_statlog = StatLog() if intercept else None
self.stats_socket = f"{STATS_SOCKET}_sp"
def connect(self) -> None:
super().connect()
if self.comma_statlog:
self.comma_statlog.connect()
def __del__(self):
super().__del__()
if self.comma_statlog:
self.comma_statlog.__del__()
def _send(self, metric: str) -> None:
super()._send(metric)
if self.comma_statlog:
self.comma_statlog._send(metric)
@staticmethod
def default_converter(obj):
if isinstance(obj, (datetime, date)):
return obj.isoformat()
if isinstance(obj, set):
return list(obj)
if isinstance(obj, Decimal):
return float(obj)
return str(obj) # fallback for unknown types
def raw(self, name: str, value: dict) -> None:
encoded_dict = base64.b64encode(json.dumps(value, default=self.default_converter).encode("utf-8")).decode("utf-8")
self._send(f"{name}:{encoded_dict}|{METRIC_TYPE.RAW}")
def main() -> NoReturn:
dongle_id = Params().get("DongleId")
def get_influxdb_line(measurement: str, value: float | dict[str, float], timestamp: datetime, tags: dict) -> str:
@@ -180,4 +231,4 @@ def main() -> NoReturn:
if __name__ == "__main__":
main()
else:
statlog = StatLog()
statlog = StatLogSP(intercept=True)