Compare commits

..

23 Commits

Author SHA1 Message Date
rav4kumar 0268dfe79a Fix Mici UI replay auto-scroll hook 2026-06-20 13:23:36 -07:00
rav4kumar bfe5c079a6 Fix Mici scroll and speed limit UI layout 2026-06-20 13:09:28 -07:00
Kumar f7543e47a0 Merge branch 'master' into mici-sla-ui 2026-06-20 12:24:10 -07:00
Kumar b94f8eaecb Merge branch 'master' into mici-sla-ui 2026-05-10 11:27:45 -07:00
Jason Wen c41d953be5 less 2026-05-01 00:29:52 -04:00
Jason Wen c603d07012 fix(mici/onroad): suppress click on outer drag and bookmark gesture
OnroadViewContainerSP nests a vertical scroller between the outer horizontal
scroller and road_view/info_panel. Stock click suppression relies on a child's
_touch_valid_callback being gated by its parent scroll panel — with the
container in the middle, children were only gated by the inner panel, so an
outer L→R drag never invalidated their touch and the click fired on release
(immediately popping to home).

- Chain children's touch_valid through the container's own touch_valid so
  outer-scroll state propagates to nested children.
- Add OnroadInfoPanel._handle_mouse_release mirroring AugmentedRoadView's
  bookmark guard, so an R→L bookmark drag on the info panel doesn't fire the
  click on release.
2026-05-01 00:26:02 -04:00
Jason Wen d2ab08eaf2 some changes 2026-05-01 00:15:14 -04:00
Jason Wen 1086abcf6b Merge remote-tracking branch 'sunnypilot/sunnypilot/master' into mici-sla-ui 2026-04-28 15:43:17 -04:00
Jason Wen 617a98d769 Merge branch 'master' into mici-sla-ui 2026-04-19 13:59:12 -04:00
rav4kumar 7e9de97596 mapd pannel now onroad info 2026-04-18 10:28:17 -07:00
Kumar 21ab8c9daf Merge branch 'master' into mici-sla-ui 2026-04-18 10:27:19 -07:00
Jason Wen d671c7e03a Merge branch 'master' into mici-sla-ui 2026-04-14 16:40:46 -04:00
Jason Wen 7cd9a76a25 Merge branch 'master' into mici-sla-ui 2026-03-18 01:05:05 -04:00
rav4kumar 7292d409a7 allow bookmarking for mapd view and align text 2026-03-17 21:44:00 -07:00
rav4kumar 78009bdf39 max speed 2026-03-17 21:28:58 -07:00
rav4kumar 5e097b2682 not needed 2026-03-15 17:52:10 -07:00
rav4kumar 31a61a24f8 no 2026-03-15 17:46:40 -07:00
rav4kumar 000c6ce261 final design 2026-03-15 17:45:08 -07:00
rav4kumar f6998b3d98 show current speed when disable and set speed when enabled 2026-03-15 17:25:22 -07:00
rav4kumar c197a1a45a Add alert rendering to MapdPanel 2026-03-15 17:14:53 -07:00
rav4kumar 33e3809330 Clean up:
- now wipe on the onroad view to change.
 - move to sp dir.
2026-03-15 13:55:45 -07:00
rav4kumar df5c0a2690 codespell 2026-03-15 12:03:31 -07:00
rav4kumar cce352cccb sla ui 2026-03-15 11:56:30 -07:00
25 changed files with 1019 additions and 112 deletions
+1 -30
View File
@@ -1,34 +1,5 @@
sunnypilot Version 2026.002.000 (2026-06-28)
sunnypilot Version 2026.002.000 (2026-xx-xx)
========================
* What's Changed (sunnypilot/sunnypilot)
* ui: update gates for certain toggles by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1830
* release: ignore upstream IsReleaseBranch by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1831
* manager: disable DEVELOPMENT_ONLY reset by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1833
* sunnylink: fix max time offroad values by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1835
* ui: show default model name by @nayan8teen in https://github.com/sunnypilot/sunnypilot/pull/1837
* sunnylink: add CarParams fallback for brand-specific capabilities by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1839
* sunnylink SDUI: tweak DisableUpdate param for clarity by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1842
* Revert "DM: Lancia Delta HF Integrale model" by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1849
* modeld_v2: safe model validation by @Discountchubbs in https://github.com/sunnypilot/sunnypilot/pull/1855
* Revert "deprecate `carState.brake`" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/sunnypilot/pull/1860
* sunnylink: deprecate legacy params metadata by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1862
* ui: reset Enforce Torque Control and NNLC if both are enabled by @sunnyhaibin in https://github.com/sunnypilot/sunnypilot/pull/1863
* What's Changed (sunnypilot/opendbc)
* Rivian: suppress ACM hold-the-wheel warning during MADS-only lateral by @lukasloetkolben in https://github.com/sunnypilot/opendbc/pull/465
* Sync: `commaai/opendbc:master``sunnypilot/opendbc:master` by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/479
* safety: add option to ignore frequency check for RX checks by @sunnyhaibin in https://github.com/sunnypilot/opendbc/pull/480
* Revert "deprecate carState.brake" for Honda Gas Interceptor by @mvl-boston in https://github.com/sunnypilot/opendbc/pull/481
* New Contributors (sunnypilot/sunnypilot)
* @mvl-boston made their first contribution in https://github.com/sunnypilot/sunnypilot/pull/1860
* Full Changelog: https://github.com/sunnypilot/sunnypilot/compare/v2026.001.007...v2026.002.000
************************
* Synced with commaai's openpilot (v0.11.1)
* master commit 69e2c321e49760e52f7983eaa0a5f77cb95de637 (June 02, 2026)
* New driver monitoring model
* Improved image processing pipeline for driver camera
* Improved thermal policy for comma four
* Acura MDX 2022-24 support thanks to mvl-boston!
* Rivian R1S and R1T 2025 support thanks to lukasloetkolben!
sunnypilot Version 2026.001.000 (2026-05-06)
========================
+1
View File
@@ -80,6 +80,7 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
{"LiveDelay", {PERSISTENT | BACKUP, BYTES}},
{"LiveParameters", {PERSISTENT, JSON}},
{"LiveParametersV2", {PERSISTENT, BYTES}},
{"LivestreamEncoderBitrate", {CLEAR_ON_MANAGER_START | DONT_LOG, INT}},
{"LiveTorqueParameters", {PERSISTENT | DONT_LOG, BYTES}},
{"LocationFilterInitialState", {PERSISTENT, BYTES}},
{"LateralManeuverMode", {CLEAR_ON_MANAGER_START | CLEAR_ON_OFFROAD_TRANSITION, BOOL}},
+8 -2
View File
@@ -13,6 +13,7 @@ from openpilot.system.ui.lib.application import gui_app
if gui_app.sunnypilot_ui():
from openpilot.selfdrive.ui.sunnypilot.mici.layouts.settings import SettingsLayoutSP as SettingsLayout
from openpilot.selfdrive.ui.sunnypilot.mici.layouts.onroad import OnroadViewContainerSP as AugmentedRoadView
ONROAD_DELAY = 2.5 # seconds
@@ -68,6 +69,9 @@ class MiciMainLayout(Scroller):
# For scroll_to
return self._body_onroad_layout if ui_state.is_body else self._car_onroad_layout
def _should_auto_scroll_to_onroad(self) -> bool:
return True
def _setup_callbacks(self):
self._home_layout.set_callbacks(
on_settings=lambda: gui_app.push_widget(self._settings_layout),
@@ -118,13 +122,15 @@ class MiciMainLayout(Scroller):
# FIXME: these two pops can interrupt user interacting in the settings
if self._onroad_time_delay is not None and rl.get_time() - self._onroad_time_delay >= ONROAD_DELAY:
gui_app.pop_widgets_to(self, lambda: self._scroll_to(self._onroad_layout))
if not gui_app.sunnypilot_ui() or self._should_auto_scroll_to_onroad():
gui_app.pop_widgets_to(self, lambda: self._scroll_to(self._onroad_layout))
self._onroad_time_delay = None
# When car leaves standstill, pop nav stack and scroll to onroad
CS = ui_state.sm["carState"]
if not CS.standstill and self._prev_standstill:
gui_app.pop_widgets_to(self, lambda: self._scroll_to(self._onroad_layout))
if not gui_app.sunnypilot_ui() or self._should_auto_scroll_to_onroad():
gui_app.pop_widgets_to(self, lambda: self._scroll_to(self._onroad_layout))
self._prev_standstill = CS.standstill
def _on_interactive_timeout(self):
@@ -0,0 +1,19 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from openpilot.selfdrive.ui.mici.layouts.main import MiciMainLayout
from openpilot.selfdrive.ui.sunnypilot.mici.widgets.scroll_panel_sp import GuiScrollPanel2SP
class MiciMainLayoutSP(MiciMainLayout):
def __init__(self):
super().__init__()
scroller = self._scroller
scroller.scroll_panel = GuiScrollPanel2SP(scroller._horizontal, handle_out_of_bounds=not scroller._snap_items)
def _should_auto_scroll_to_onroad(self) -> bool:
return not self._onroad_layout.is_on_info_panel()
@@ -0,0 +1,63 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import pyray as rl
from openpilot.system.ui.lib.application import gui_app
from openpilot.selfdrive.ui.sunnypilot.mici.widgets.scroller_sp import ScrollerSP
from openpilot.selfdrive.ui.sunnypilot.mici.onroad.augmented_road_view import AugmentedRoadViewSP
from openpilot.selfdrive.ui.sunnypilot.mici.layouts.onroad_info_panel import OnroadInfoPanel
CONFIDENCE_BALL_VISIBLE_RATIO = 0.4
HORIZONTAL_SETTLE_PX = 5
HORIZONTAL_RESET_RATIO = 0.5
class OnroadViewContainerSP(ScrollerSP):
def __init__(self, bookmark_callback=None):
super().__init__(horizontal=False, snap_items=True, spacing=0, pad=0, scroll_indicator=False, edge_shadows=False)
self.road_view = AugmentedRoadViewSP(bookmark_callback=bookmark_callback)
self.onroad_info_panel = OnroadInfoPanel(bookmark_callback=bookmark_callback)
self._scroller.add_widgets([
self.road_view,
self.onroad_info_panel,
])
self._scroller.set_reset_scroll_at_show(False)
self._scroller.set_scrolling_enabled(lambda: abs(self.rect.x) < HORIZONTAL_SETTLE_PX)
for child in (self.road_view, self.onroad_info_panel):
inner_touch_valid = child._touch_valid_callback
child.set_touch_valid_callback(
lambda inner=inner_touch_valid: self._touch_valid() and (inner() if inner else True)
)
def set_rect(self, rect: rl.Rectangle):
super().set_rect(rect)
self.road_view.set_rect(rect)
self.onroad_info_panel.set_rect(rect)
return self
def is_swiping_left(self) -> bool:
return self.road_view.is_swiping_left() or self.onroad_info_panel.is_swiping_left()
def set_click_callback(self, callback) -> None:
self.road_view.set_click_callback(callback)
self.onroad_info_panel.set_click_callback(callback)
def is_on_info_panel(self) -> bool:
"""True when scrolled past halfway toward onroad_info_panel (used by main layout
to skip auto-pop-back-to-camera while user is reading the info panel)."""
return abs(self._scroller.scroll_panel.get_offset()) > self._rect.height / 2
def _render(self, rect: rl.Rectangle):
if abs(self.rect.x) > gui_app.width * HORIZONTAL_RESET_RATIO:
self._scroller.scroll_panel.set_offset(0)
vertical_offset = self._scroller.scroll_panel.get_offset()
show_ball = abs(vertical_offset) < rect.height * CONFIDENCE_BALL_VISIBLE_RATIO
self.road_view.set_show_confidence_ball(show_ball)
super()._render(rect)
@@ -0,0 +1,403 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import pyray as rl
from dataclasses import dataclass
from openpilot.common.constants import CV
from openpilot.common.filter_simple import FirstOrderFilter
from openpilot.selfdrive.ui.ui_state import ui_state
from openpilot.system.ui.lib.application import gui_app, FontWeight, MousePos
from openpilot.system.ui.lib.multilang import tr
from openpilot.system.ui.lib.text_measure import measure_text_cached
from openpilot.system.ui.widgets import Widget
from openpilot.selfdrive.ui.mici.onroad.alert_renderer import AlertRenderer
from openpilot.selfdrive.ui.mici.onroad.augmented_road_view import BookmarkIcon
METER_TO_KM = 0.001
METER_TO_MILE = 0.000621371
CONTENT_MARGIN = 16
SPEED_LIMIT_SIGN_WIDTH = 146
VIENNA_SIGN_SIZE = 146
MUTCD_SIGN_HEIGHT = 178
OFFSET_BADGE_SIZE = 50
OFFSET_BADGE_PANEL_PADDING = 4
MUTCD_OFFSET_SIGN_Y_SHIFT = 6
VIENNA_BADGE_X_RATIO = 0.80
VIENNA_BADGE_UPCOMING_X_RATIO = 0.70
VIENNA_BADGE_Y_RATIO = -0.82
UPCOMING_SIGN_SIZE_RATIO = 0.76
UPCOMING_SIGN_OVERLAP_RATIO = 0.05
UNIT_FONT_SIZE = 40
SPEED_FONT_SIZE = 114
ROAD_FONT_SIZE = 32
SCC_TAG_WIDTH = 78
SCC_TAG_HEIGHT = 30
SCC_TAG_GAP = 5
COLUMN_GAP = 12
@dataclass(frozen=True)
class OnroadInfoPanelColors:
white: rl.Color = rl.WHITE
black: rl.Color = rl.BLACK
red: rl.Color = rl.Color(255, 0, 0, 255)
green: rl.Color = rl.Color(0, 255, 0, 255)
grey: rl.Color = rl.Color(190, 195, 190, 255)
light_grey: rl.Color = rl.Color(200, 200, 200, 255)
dark_grey: rl.Color = rl.Color(100, 100, 100, 255)
bg_dark: rl.Color = rl.Color(0, 0, 0, 255)
card_bg: rl.Color = rl.Color(50, 50, 50, 200)
badge_bg: rl.Color = rl.Color(60, 60, 60, 255)
COLORS = OnroadInfoPanelColors()
class OnroadInfoPanel(Widget):
def __init__(self, bookmark_callback=None):
super().__init__()
self.speed_limit: float = 0.0
self.speed_limit_valid: bool = False
self.speed_limit_offset: float = 0.0
self.next_speed_limit: float = 0.0
self.next_speed_limit_distance: float = 0.0
self.road_name: str = ""
self.current_speed: float = 0.0
self.set_speed: float = 0.0
self.cruise_enabled: bool = False
self._sign_slide: float = 0.0
self._font_bold: rl.Font = gui_app.font(FontWeight.BOLD)
self._font_semi_bold: rl.Font = gui_app.font(FontWeight.SEMI_BOLD)
self._font_medium: rl.Font = gui_app.font(FontWeight.MEDIUM)
self._marquee_offset: float = 0.0
self._marquee_direction: int = 1
self._marquee_pause_timer: float = 0.0
self._marquee_speed: float = 40.0
self._marquee_pause_duration: float = 1.5
self._alert_renderer = AlertRenderer()
self._alert_alpha_filter = FirstOrderFilter(0, 0.05, 1 / gui_app.target_fps)
self._bookmark_icon = BookmarkIcon(bookmark_callback)
def is_swiping_left(self) -> bool:
return self._bookmark_icon.is_swiping_left()
def _handle_mouse_release(self, mouse_pos: MousePos) -> None:
# Mirror stock AugmentedRoadView: suppress click while bookmark gesture active
if not self._bookmark_icon.interacting():
super()._handle_mouse_release(mouse_pos)
def _update_state(self) -> None:
sm = ui_state.sm
speed_conv = CV.MS_TO_KPH if ui_state.is_metric else CV.MS_TO_MPH
if sm.valid["longitudinalPlanSP"]:
lp_sp = sm["longitudinalPlanSP"]
resolver = lp_sp.speedLimit.resolver
self.speed_limit = resolver.speedLimit * speed_conv
self.speed_limit_valid = resolver.speedLimitValid
self.speed_limit_offset = resolver.speedLimitOffset * speed_conv
if sm.valid["liveMapDataSP"]:
lmd = sm["liveMapDataSP"]
self.next_speed_limit = lmd.speedLimitAhead * speed_conv
self.next_speed_limit_distance = lmd.speedLimitAheadDistance
self.road_name = lmd.roadName
if sm.updated["carState"]:
self.current_speed = sm["carState"].vEgo * speed_conv
if sm.valid["carState"] and sm.valid["controlsState"]:
self.cruise_enabled = sm["carState"].cruiseState.enabled
v_cruise_cluster = sm["carState"].vCruiseCluster
set_speed_kph = sm["controlsState"].vCruiseDEPRECATED if v_cruise_cluster == 0.0 else v_cruise_cluster
self.set_speed = set_speed_kph * (METER_TO_MILE / METER_TO_KM) if not ui_state.is_metric else set_speed_kph
def _render(self, rect: rl.Rectangle) -> None:
self._update_state()
rl.draw_rectangle(int(rect.x), int(rect.y), int(rect.width), int(rect.height), COLORS.bg_dark)
left_x = rect.x + CONTENT_MARGIN
if self.cruise_enabled:
unit = tr("MAX")
display_speed = self.set_speed
else:
unit = tr("km/h") if ui_state.is_metric else tr("MPH")
display_speed = self.current_speed
display_speed_text = str(round(display_speed))
if self.speed_limit_valid and display_speed > self.speed_limit:
speed_color = COLORS.red
else:
speed_color = COLORS.white
sign_width = min(SPEED_LIMIT_SIGN_WIDTH, rect.width * 0.30)
sign_height = VIENNA_SIGN_SIZE if ui_state.is_metric else MUTCD_SIGN_HEIGHT
has_upcoming_limit = self.next_speed_limit > 0 and self.next_speed_limit != self.speed_limit
target_sign_slide = 1.0 if has_upcoming_limit else 0.0
slide_speed = 3.0 * rl.get_frame_time()
if self._sign_slide < target_sign_slide:
self._sign_slide = min(self._sign_slide + slide_speed, target_sign_slide)
elif self._sign_slide > target_sign_slide:
self._sign_slide = max(self._sign_slide - slide_speed, target_sign_slide)
upcoming_width = int(sign_width * UPCOMING_SIGN_SIZE_RATIO)
upcoming_height = int(sign_height * UPCOMING_SIGN_SIZE_RATIO)
upcoming_reserved_width = int(upcoming_width * 0.85) + 5
sign_x_without_upcoming = rect.x + rect.width - sign_width - CONTENT_MARGIN
sign_x_with_upcoming = rect.x + rect.width - sign_width - CONTENT_MARGIN - upcoming_reserved_width
sign_x = sign_x_without_upcoming + (sign_x_with_upcoming - sign_x_without_upcoming) * self._sign_slide
sign_y = rect.y + (rect.height - sign_height) / 2
if not ui_state.is_metric and self.speed_limit_offset != 0 and self.speed_limit_valid:
sign_y += MUTCD_OFFSET_SIGN_Y_SHIFT
readout_right = sign_x - COLUMN_GAP
readout_width = max(1, readout_right - left_x)
road_y = rect.y + rect.height - 44
unit_font_size = self._fit_font_size(self._font_semi_bold, unit, readout_width, 46, UNIT_FONT_SIZE, 28)
speed_font_size = self._fit_font_size(self._font_bold, display_speed_text, readout_width, road_y - (rect.y + 54) - 8,
SPEED_FONT_SIZE, 76)
speed_size = measure_text_cached(self._font_bold, display_speed_text, speed_font_size)
speed_y = min(rect.y + 54, road_y - speed_size.y - 8)
unit_y = max(rect.y + 14, speed_y - unit_font_size - 6)
rl.draw_text_ex(self._font_semi_bold, unit, rl.Vector2(left_x, unit_y), unit_font_size, 0, COLORS.grey)
rl.draw_text_ex(self._font_bold, display_speed_text, rl.Vector2(left_x, speed_y), speed_font_size, 0, speed_color)
self._draw_road_name(left_x, road_y, readout_width)
if has_upcoming_limit and self._sign_slide > 0.01:
upcoming_speed_text = str(round(self.next_speed_limit))
distance_text = self._format_distance(self.next_speed_limit_distance)
upcoming_x = sign_x + sign_width - int(upcoming_width * UPCOMING_SIGN_OVERLAP_RATIO)
upcoming_y = sign_y + (sign_height - upcoming_height) / 2
upcoming_speed_color = COLORS.black
if ui_state.is_metric:
self._draw_vienna_sign(upcoming_x, upcoming_y, upcoming_width, upcoming_height, upcoming_speed_text, upcoming_speed_color, is_upcoming=True)
else:
self._draw_mutcd_sign(upcoming_x, upcoming_y, upcoming_width, upcoming_height, upcoming_speed_text, upcoming_speed_color, is_upcoming=True)
distance_font_size = self._fit_font_size(self._font_medium, distance_text, upcoming_width, 30, 24, 16)
distance_size = measure_text_cached(self._font_medium, distance_text, distance_font_size)
rl.draw_text_ex(self._font_medium, distance_text, rl.Vector2(upcoming_x + upcoming_width / 2 - distance_size.x / 2, upcoming_y + upcoming_height),
distance_font_size, 0, COLORS.grey)
self._draw_speed_limit_sign(sign_x, sign_y, sign_width, sign_height)
if self.speed_limit_offset != 0 and self.speed_limit_valid:
offset_text = str(abs(round(self.speed_limit_offset)))
badge_size = OFFSET_BADGE_SIZE
badge_rect = self._offset_badge_rect(rect, sign_x, sign_y, sign_width, sign_height, badge_size, has_upcoming_limit)
if ui_state.is_metric:
badge_radius = badge_size / 2
badge_center_x = badge_rect.x + badge_radius
badge_center_y = badge_rect.y + badge_radius
rl.draw_circle(int(badge_center_x), int(badge_center_y), badge_radius + 2, COLORS.dark_grey)
rl.draw_circle(int(badge_center_x), int(badge_center_y), badge_radius, COLORS.badge_bg)
self._draw_text_centered_fit(self._font_bold, offset_text, 32, rl.Vector2(badge_center_x, badge_center_y), COLORS.white,
badge_size - 10, badge_size - 8, min_size=24)
else:
rl.draw_rectangle_rounded(badge_rect, 0.25, 10, COLORS.badge_bg)
rl.draw_rectangle_rounded_lines_ex(badge_rect, 0.25, 10, 2, COLORS.dark_grey)
self._draw_text_centered_fit(self._font_bold, offset_text, 32, rl.Vector2(badge_rect.x + badge_size / 2, badge_rect.y + badge_size / 2),
COLORS.white, badge_size - 10, badge_size - 8, min_size=24)
scc_tag_x = min(left_x + speed_size.x + COLUMN_GAP, readout_right - SCC_TAG_WIDTH)
scc_tag_y = speed_y + (speed_size.y - (SCC_TAG_HEIGHT * 2 + SCC_TAG_GAP)) / 2
if scc_tag_x >= left_x + speed_size.x + 8:
self._draw_scc_icons(scc_tag_x, scc_tag_y, readout_right)
self._bookmark_icon.render(rect)
if ui_state.started:
alert_obj, no_alert = self._alert_renderer.will_render()
self._alert_alpha_filter.update(0 if no_alert else 1)
alpha = self._alert_alpha_filter.x
if alpha > 0.01:
rl.draw_rectangle(int(rect.x), int(rect.y), int(rect.width), int(rect.height), rl.Color(0, 0, 0, int(150 * alpha)))
self._alert_renderer.render(rect)
def _draw_scc_icons(self, x: float, y: float, right_limit: float) -> None:
sm = ui_state.sm
if not sm.valid["longitudinalPlanSP"]:
return
scc = sm["longitudinalPlanSP"].smartCruiseControl
drawn = 0
for label, active in [("SCC-V", scc.vision.active), ("SCC-M", scc.map.active)]:
if not active:
continue
tag_x = x
if tag_x + SCC_TAG_WIDTH > right_limit:
return
tag_y = y + drawn * (SCC_TAG_HEIGHT + SCC_TAG_GAP)
rl.draw_rectangle_rounded(rl.Rectangle(tag_x, tag_y, SCC_TAG_WIDTH, SCC_TAG_HEIGHT), 0.3, 10, COLORS.green)
self._draw_text_centered_fit(self._font_bold, label, 18, rl.Vector2(tag_x + SCC_TAG_WIDTH / 2, tag_y + SCC_TAG_HEIGHT / 2), COLORS.black,
SCC_TAG_WIDTH - 10, SCC_TAG_HEIGHT - 4, min_size=14)
drawn += 1
def _draw_speed_limit_sign(self, x: float, y: float, sign_width: float, sign_height: float) -> None:
speed_str = str(round(self.speed_limit)) if self.speed_limit_valid and self.speed_limit > 0 else "--"
speed_color = COLORS.black if not self.speed_limit_valid or self.current_speed <= self.speed_limit else COLORS.red
if ui_state.is_metric:
self._draw_vienna_sign(x, y, sign_width, sign_height, speed_str, speed_color, is_upcoming=False)
else:
self._draw_mutcd_sign(x, y, sign_width, sign_height, speed_str, speed_color, is_upcoming=False)
def _draw_road_name(self, x: float, y: float, width: float) -> None:
if width <= 0:
return
road_display = self.road_name if self.road_name else "--"
font_size = self._fit_font_size(self._font_semi_bold, road_display, width, 38, ROAD_FONT_SIZE, 28)
road_size = measure_text_cached(self._font_semi_bold, road_display, font_size)
text_width = road_size.x
if text_width <= width:
self._marquee_offset = 0.0
self._marquee_direction = 1
self._marquee_pause_timer = 0.0
rl.draw_text_ex(self._font_semi_bold, road_display, rl.Vector2(x, y), font_size, 0, COLORS.white)
else:
overflow = text_width - width
dt = rl.get_frame_time()
if self._marquee_pause_timer > 0:
self._marquee_pause_timer -= dt
else:
self._marquee_offset += self._marquee_direction * self._marquee_speed * dt
if self._marquee_offset >= overflow:
self._marquee_offset = overflow
self._marquee_direction = -1
self._marquee_pause_timer = self._marquee_pause_duration
elif self._marquee_offset <= 0:
self._marquee_offset = 0
self._marquee_direction = 1
self._marquee_pause_timer = self._marquee_pause_duration
rl.begin_scissor_mode(int(x), int(y), int(width), int(road_size.y + 4))
text_pos = rl.Vector2(x - self._marquee_offset, y)
rl.draw_text_ex(self._font_semi_bold, road_display, text_pos, font_size, 0, COLORS.white)
rl.end_scissor_mode()
def _draw_vienna_sign(self, x: float, y: float, width: float, height: float, speed_str: str, speed_color: rl.Color, is_upcoming: bool = False) -> None:
center = rl.Vector2(x + width / 2, y + height / 2)
outer_radius = min(width, height) / 2
rl.draw_circle_v(center, outer_radius, COLORS.white)
ring_width = outer_radius * 0.18
rl.draw_ring(center, outer_radius - ring_width, outer_radius, 0, 360, 36, COLORS.red)
font_size = outer_radius * (0.7 if len(speed_str) >= 3 else 0.9)
self._draw_text_centered_fit(self._font_bold, speed_str, int(font_size), center, speed_color, width * 0.72, height * 0.50, min_size=24)
def _draw_mutcd_sign(self, x: float, y: float, width: float, height: float, speed_str: str, speed_color: rl.Color, is_upcoming: bool = False) -> None:
sign_rect = rl.Rectangle(x, y, width, height)
rl.draw_rectangle_rounded(sign_rect, 0.35, 10, COLORS.white)
inset = max(4, width * 0.05)
inner_rect = rl.Rectangle(x + inset, y + inset, width - inset * 2, height - inset * 2)
outer_radius = 0.35 * width / 2.0
inner_radius = outer_radius - inset
inner_roundness = inner_radius / (inner_rect.width / 2.0)
rl.draw_rectangle_rounded_lines_ex(inner_rect, inner_roundness, 10, 3, COLORS.black)
mid_x = x + width / 2
label_size = max(18, int(width * 0.26))
if is_upcoming:
self._draw_text_centered_fit(self._font_bold, tr("AHEAD"), int(width * 0.34), rl.Vector2(mid_x, y + height * 0.28), COLORS.black,
width * 0.94, height * 0.32, min_size=20)
else:
self._draw_text_centered_fit(self._font_bold, tr("SPEED"), label_size, rl.Vector2(mid_x, y + height * 0.20), COLORS.black,
width * 0.84, height * 0.24, min_size=16)
self._draw_text_centered_fit(self._font_bold, tr("LIMIT"), label_size, rl.Vector2(mid_x, y + height * 0.40), COLORS.black,
width * 0.84, height * 0.24, min_size=16)
speed_font_size = int(width * 0.60) if len(speed_str) >= 3 else int(width * 0.72)
self._draw_text_centered_fit(self._font_bold, speed_str, speed_font_size, rl.Vector2(mid_x, y + height * 0.72), speed_color,
width * 0.90, height * 0.52, min_size=32)
def _draw_text_centered(self, font, text, size, pos_center, color):
sz = measure_text_cached(font, text, size)
rl.draw_text_ex(font, text, rl.Vector2(pos_center.x - sz.x / 2, pos_center.y - sz.y / 2), size, 0, color)
def _draw_text_centered_fit(self, font, text, size, pos_center, color, max_width: float, max_height: float, min_size: int = 10):
size = self._fit_font_size(font, text, max_width, max_height, size, min_size)
self._draw_text_centered(font, text, size, pos_center, color)
def _fit_font_size(self, font, text: str, max_width: float, max_height: float, max_size: int | float, min_size: int) -> int:
size = int(max_size)
while size > min_size:
text_size = measure_text_cached(font, text, size)
if text_size.x <= max_width and text_size.y <= max_height:
return size
size -= 2
return min_size
def _offset_badge_rect(self, panel_rect: rl.Rectangle, sign_x: float, sign_y: float, sign_width: float, sign_height: float,
badge_size: float, has_upcoming_limit: bool) -> rl.Rectangle:
if ui_state.is_metric:
radius = min(sign_width, sign_height) / 2
center_x = sign_x + sign_width / 2
center_y = sign_y + sign_height / 2
badge_x_ratio = VIENNA_BADGE_UPCOMING_X_RATIO if has_upcoming_limit else VIENNA_BADGE_X_RATIO
badge_center_x = center_x + radius * badge_x_ratio
badge_center_y = center_y + radius * VIENNA_BADGE_Y_RATIO
badge_x = badge_center_x - badge_size / 2
badge_y = badge_center_y - badge_size / 2
else:
badge_x = sign_x + sign_width - badge_size * 0.45
badge_y = sign_y - badge_size * 0.75
return rl.Rectangle(
self._clamp(
badge_x,
panel_rect.x + OFFSET_BADGE_PANEL_PADDING,
panel_rect.x + panel_rect.width - badge_size - OFFSET_BADGE_PANEL_PADDING,
),
self._clamp(
badge_y,
panel_rect.y + OFFSET_BADGE_PANEL_PADDING,
panel_rect.y + panel_rect.height - badge_size - OFFSET_BADGE_PANEL_PADDING,
),
badge_size,
badge_size,
)
@staticmethod
def _clamp(value: float, min_value: float, max_value: float) -> float:
return max(min_value, min(max_value, value))
def _format_distance(self, distance: float) -> str:
if ui_state.is_metric:
if distance < 50:
return tr("Near")
if distance >= 1000:
return f"{distance * METER_TO_KM:.1f}" + tr("km")
if distance < 200:
rounded = max(10, int(distance / 10) * 10)
else:
rounded = int(distance / 100) * 100
return str(rounded) + tr("m")
else:
distance_mi = distance * METER_TO_MILE
if distance_mi < 0.1:
return tr("Near")
return f"{distance_mi:.1f}" + tr("mi")
@@ -0,0 +1,30 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import pyray as rl
from openpilot.selfdrive.ui.mici.onroad.augmented_road_view import AugmentedRoadView
class _SuppressedConfidenceBall:
def render(self, *_):
pass
class AugmentedRoadViewSP(AugmentedRoadView):
def __init__(self, **kwargs):
super().__init__(**kwargs)
self._show_confidence_ball: bool = True
self._real_confidence_ball = self._confidence_ball
self._confidence_ball = _SuppressedConfidenceBall()
def set_show_confidence_ball(self, show: bool) -> None:
self._show_confidence_ball = show
def _render(self, rect: rl.Rectangle) -> None:
super()._render(rect)
if self._show_confidence_ball:
self._real_confidence_ball.render(self.rect)
@@ -0,0 +1,83 @@
import pyray as rl
from openpilot.system.ui.lib.application import MouseEvent, MousePos, gui_app
from openpilot.system.ui.lib.scroll_panel2 import ScrollState
from openpilot.system.ui.widgets import Widget
from openpilot.system.ui.widgets import scroller as scroller_mod
from openpilot.selfdrive.ui.sunnypilot.mici.widgets.scroll_panel_sp import GuiScrollPanel2SP
class DummyScrollIndicator:
def update(self, *_) -> None:
pass
def render(self) -> None:
pass
class DummyWidget(Widget):
def __init__(self, rect: rl.Rectangle):
super().__init__()
self.set_rect(rect)
def _render(self, _) -> None:
pass
def _mouse_event(x: float, y: float, *, pressed: bool = False, released: bool = False,
down: bool = True, t: float = 0.0) -> MouseEvent:
return MouseEvent(MousePos(x, y), 0, pressed, released, down, t)
def test_vertical_snap_items_are_supported(monkeypatch):
monkeypatch.setattr(scroller_mod, "ScrollIndicator", DummyScrollIndicator)
scroller = scroller_mod._Scroller([], horizontal=False, snap_items=True, scroll_indicator=False)
scroller.set_rect(rl.Rectangle(0, 0, 100, 100))
scroller.scroll_panel.set_offset(-60)
captured_snap_target = None
def update(_, __, snap_target=None):
nonlocal captured_snap_target
captured_snap_target = snap_target
return scroller.scroll_panel.get_offset()
monkeypatch.setattr(scroller.scroll_panel, "update", update)
visible_items = [
DummyWidget(rl.Rectangle(0, -60, 100, 100)),
DummyWidget(rl.Rectangle(0, 40, 100, 100)),
]
scroller._get_scroll(visible_items, 200)
assert captured_snap_target == -100
def test_scroll_panel_sp_rejects_orthogonal_drags(monkeypatch):
panel = GuiScrollPanel2SP(horizontal=True)
bounds = rl.Rectangle(0, 0, 100, 100)
monkeypatch.setattr(gui_app, "_mouse_events", [_mouse_event(10, 10, pressed=True, t=1.0)])
panel.update(bounds, 200)
assert panel.state == ScrollState.PRESSED
monkeypatch.setattr(gui_app, "_mouse_events", [_mouse_event(23, 60, t=1.1)])
panel.update(bounds, 200)
assert panel.state == ScrollState.STEADY
assert panel.get_offset() == 0
def test_scroll_panel_sp_can_disable_out_of_bounds_handling(monkeypatch):
panel = GuiScrollPanel2SP(horizontal=False, handle_out_of_bounds=False)
bounds = rl.Rectangle(0, 0, 100, 100)
monkeypatch.setattr(gui_app, "_mouse_events", [])
panel.set_offset(20)
panel.update(bounds, 200)
assert panel.get_offset() == 0
panel.set_offset(-150)
panel.update(bounds, 200)
assert panel.get_offset() == -100
@@ -0,0 +1,33 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
import pyray as rl
from openpilot.system.ui.lib.application import MouseEvent
from openpilot.system.ui.lib.scroll_panel2 import GuiScrollPanel2, ScrollState
class GuiScrollPanel2SP(GuiScrollPanel2):
"""Scroll panel behavior for nested Mici pagers."""
def __init__(self, horizontal: bool = True, handle_out_of_bounds: bool = True) -> None:
super().__init__(horizontal, handle_out_of_bounds=handle_out_of_bounds)
def _handle_mouse_event(self, mouse_event: MouseEvent, bounds: rl.Rectangle, bounds_size: float,
content_size: float) -> None:
state_before_update = self._state
super()._handle_mouse_event(mouse_event, bounds, bounds_size, content_size)
if self._state == ScrollState.MANUAL_SCROLL and state_before_update == ScrollState.PRESSED and \
self._initial_click_event is not None:
drag_x = abs(mouse_event.pos.x - self._initial_click_event.pos.x)
drag_y = abs(mouse_event.pos.y - self._initial_click_event.pos.y)
primary_drag = drag_x if self._horizontal else drag_y
cross_drag = drag_y if self._horizontal else drag_x
if cross_drag > primary_drag:
self._state = ScrollState.STEADY
self._velocity = 0.0
self._velocity_buffer.clear()
@@ -0,0 +1,16 @@
"""
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
from openpilot.system.ui.widgets.scroller import Scroller
from openpilot.selfdrive.ui.sunnypilot.mici.widgets.scroll_panel_sp import GuiScrollPanel2SP
class ScrollerSP(Scroller):
def __init__(self, **kwargs):
super().__init__(**kwargs)
inner = self._scroller
inner.scroll_panel = GuiScrollPanel2SP(inner._horizontal, handle_out_of_bounds=not inner._snap_items)
+3
View File
@@ -10,6 +10,9 @@ from openpilot.selfdrive.ui.layouts.main import MainLayout
from openpilot.selfdrive.ui.mici.layouts.main import MiciMainLayout
from openpilot.selfdrive.ui.ui_state import ui_state
if gui_app.sunnypilot_ui():
from openpilot.selfdrive.ui.sunnypilot.mici.layouts.main import MiciMainLayoutSP as MiciMainLayout
BIG_UI = gui_app.big_ui()
+41 -1
View File
@@ -29,7 +29,7 @@ from websocket import (ABNF, WebSocket, WebSocketException, WebSocketTimeoutExce
create_connection)
import cereal.messaging as messaging
from cereal import log
from cereal import car, log
from cereal.services import SERVICE_LIST
from openpilot.common.api import Api, get_key_pair
from openpilot.common.utils import CallbackReader, get_upload_stream
@@ -45,6 +45,7 @@ from openpilot.system.hardware.hw import Paths
ATHENA_HOST = os.getenv('ATHENA_HOST', 'wss://athena.comma.ai')
HANDLER_THREADS = int(os.getenv('HANDLER_THREADS', "4"))
LOCAL_PORT_WHITELIST = {22, } # SSH
WEBRTCD_PORT = 5001
LOG_ATTR_NAME = 'user.upload'
LOG_ATTR_VALUE_MAX_UNIX_TIME = int.to_bytes(2147483647, 4, sys.byteorder)
@@ -567,6 +568,16 @@ def getSshAuthorizedKeys() -> str:
def getGithubUsername() -> str:
return cast(str, Params().get("GithubUsername") or "")
@dispatcher.add_method
def getNotCar() -> bool:
cp_bytes = Params().get("CarParamsPersistent")
if cp_bytes is not None:
with car.CarParams.from_bytes(cp_bytes) as CP:
return CP.notCar
return False
@dispatcher.add_method
def getSimInfo():
return HARDWARE.get_sim_info()
@@ -588,6 +599,35 @@ def getNetworks():
return HARDWARE.get_networks()
@dispatcher.add_method
def startStream(sdp: str) -> dict:
from openpilot.system.webrtc.webrtcd import StreamRequestBody
bridge_services_in = []
# get live car params to avoid stale notCar edge case
cp_bytes = Params().get("CarParams")
if cp_bytes is not None:
with car.CarParams.from_bytes(cp_bytes) as CP:
if CP.notCar:
bridge_services_in.append("testJoystick")
body = StreamRequestBody(sdp, "wideRoad", bridge_services_in, ["carState"])
try:
resp = requests.post(f"http://localhost:{WEBRTCD_PORT}/stream",
json=asdict(body), timeout=10)
if not resp.ok:
try:
error_body = resp.json()
raise Exception(error_body.get("message", f"webrtcd returned {resp.status_code}"))
except ValueError:
resp.raise_for_status()
return resp.json()
except requests.ConnectTimeout as e:
raise Exception("webrtc took too long to respond. is it on?") from e
except requests.ConnectionError as e:
raise Exception("webrtc is not running. turn on comma body ignition.") from e
@dispatcher.add_method
def takeSnapshot() -> str | dict[str, str] | None:
from openpilot.system.camerad.snapshot import jpeg_write, snapshot
-3
View File
@@ -313,9 +313,6 @@ class Tici(HardwareBase):
continue
gov = 'ondemand' if powersave_enabled else 'performance'
sudo_write(gov, f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_governor')
if not powersave_enabled:
# cap max core freq to 1689 Mhz
sudo_write('1689600', f'/sys/devices/system/cpu/cpufreq/policy{n}/scaling_max_freq')
# *** IRQ config ***
+1
View File
@@ -26,6 +26,7 @@ public:
virtual int encode_frame(VisionBuf* buf, VisionIpcBufExtra *extra) = 0;
virtual void encoder_open() = 0;
virtual void encoder_close() = 0;
virtual void set_bitrate(int bitrate) = 0;
void publisher_publish(int segment_num, uint32_t idx, VisionIpcBufExtra &extra, unsigned int flags, kj::ArrayPtr<capnp::byte> header, kj::ArrayPtr<capnp::byte> dat);
+4
View File
@@ -72,6 +72,10 @@ void FfmpegEncoder::encoder_close() {
is_open = false;
}
void FfmpegEncoder::set_bitrate(int bitrate) {
LOGE("adaptive bitrate is not supported for ffmpeg encoder %s", encoder_info.publish_name);
}
int FfmpegEncoder::encode_frame(VisionBuf* buf, VisionIpcBufExtra *extra) {
assert(buf->width == this->in_width);
assert(buf->height == this->in_height);
+1
View File
@@ -21,6 +21,7 @@ public:
int encode_frame(VisionBuf* buf, VisionIpcBufExtra *extra);
void encoder_open();
void encoder_close();
void set_bitrate(int bitrate);
private:
int segment_num = -1;
+21
View File
@@ -155,6 +155,8 @@ V4LEncoder::V4LEncoder(const EncoderInfo &encoder_info, int in_width, int in_hei
assert(strcmp((const char *)cap.card, "msm_vidc_venc") == 0);
EncoderSettings encoder_settings = encoder_info.get_settings(in_width);
current_bitrate = encoder_settings.bitrate;
adaptive_bitrate = encoder_info.adaptive_bitrate;
bool is_h265 = encoder_settings.encode_type == cereal::EncodeIndex::Type::FULL_H_E_V_C;
struct v4l2_format fmt_out = {
@@ -304,6 +306,25 @@ void V4LEncoder::encoder_close() {
this->is_open = false;
}
void V4LEncoder::set_bitrate(int bitrate) {
if (!adaptive_bitrate || bitrate == current_bitrate) return;
if (bitrate <= 0) {
LOGE("invalid livestream encoder bitrate %d", bitrate);
return;
}
struct v4l2_control ctrl = {
.id = V4L2_CID_MPEG_VIDEO_BITRATE,
.value = bitrate,
};
if (util::safe_ioctl(fd, VIDIOC_S_CTRL, &ctrl) == -1) {
LOGE("failed to update %s bitrate to %d", encoder_info.publish_name, bitrate);
return;
}
current_bitrate = bitrate;
}
V4LEncoder::~V4LEncoder() {
encoder_close();
v4l2_buf_type buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+3
View File
@@ -13,6 +13,7 @@ public:
int encode_frame(VisionBuf* buf, VisionIpcBufExtra *extra);
void encoder_open();
void encoder_close();
void set_bitrate(int bitrate);
private:
int fd;
@@ -20,6 +21,8 @@ private:
bool is_open = false;
int segment_num = -1;
int counter = 0;
int current_bitrate = -1;
bool adaptive_bitrate;
SafeQueue<VisionIpcBufExtra> extras;
+15
View File
@@ -44,11 +44,24 @@ bool sync_encoders(EncoderdState *s, VisionStreamType cam_type, uint32_t frame_i
}
}
void apply_bitrate(std::vector<std::unique_ptr<Encoder>> &encoders) {
static Params params;
std::string val = params.get("LivestreamEncoderBitrate");
if (val.empty()) return;
int bitrate = std::stoi(val);
for (auto &e : encoders) {
e->set_bitrate(bitrate);
}
}
void encoder_thread(EncoderdState *s, const LogCameraInfo &cam_info) {
util::set_thread_name(cam_info.thread_name);
std::vector<std::unique_ptr<Encoder>> encoders;
bool has_adaptive = std::any_of(cam_info.encoder_infos.begin(), cam_info.encoder_infos.end(),
[](const auto &ei) { return ei.adaptive_bitrate; });
VisionIpcClient vipc_client = VisionIpcClient("camerad", cam_info.stream_type, false);
std::unique_ptr<JpegEncoder> jpeg_encoder;
@@ -108,6 +121,8 @@ void encoder_thread(EncoderdState *s, const LogCameraInfo &cam_info) {
++cur_seg;
}
if (has_adaptive) apply_bitrate(encoders);
// encode a frame
for (int i = 0; i < encoders.size(); ++i) {
int out_id = encoders[i]->encode_frame(buf, &extra);
+9 -5
View File
@@ -47,8 +47,8 @@ struct EncoderSettings {
}
static EncoderSettings StreamEncoderSettings() {
int _stream_bitrate = getenv("STREAM_BITRATE") ? atoi(getenv("STREAM_BITRATE")) : 1'000'000;
return EncoderSettings{.encode_type = cereal::EncodeIndex::Type::QCAMERA_H264, .bitrate = _stream_bitrate , .gop_size = 15};
int _stream_bitrate = getenv("STREAM_BITRATE") ? atoi(getenv("STREAM_BITRATE")) : 5'000'000;
return EncoderSettings{.encode_type = cereal::EncodeIndex::Type::QCAMERA_H264, .bitrate = _stream_bitrate , .gop_size = 5};
}
};
@@ -59,6 +59,7 @@ public:
const char *filename = NULL;
bool record = true;
bool include_audio = false;
bool adaptive_bitrate = false;
int frame_width = -1;
int frame_height = -1;
int fps = MAIN_FPS;
@@ -104,6 +105,7 @@ const EncoderInfo stream_road_encoder_info = {
.publish_name = "livestreamRoadEncodeData",
//.thumbnail_name = "thumbnail",
.record = false,
.adaptive_bitrate = true,
.get_settings = [](int){return EncoderSettings::StreamEncoderSettings();},
INIT_ENCODE_FUNCTIONS(LivestreamRoadEncode),
};
@@ -111,6 +113,7 @@ const EncoderInfo stream_road_encoder_info = {
const EncoderInfo stream_wide_road_encoder_info = {
.publish_name = "livestreamWideRoadEncodeData",
.record = false,
.adaptive_bitrate = true,
.get_settings = [](int){return EncoderSettings::StreamEncoderSettings();},
INIT_ENCODE_FUNCTIONS(LivestreamWideRoadEncode),
};
@@ -118,6 +121,7 @@ const EncoderInfo stream_wide_road_encoder_info = {
const EncoderInfo stream_driver_encoder_info = {
.publish_name = "livestreamDriverEncodeData",
.record = false,
.adaptive_bitrate = true,
.get_settings = [](int){return EncoderSettings::StreamEncoderSettings();},
INIT_ENCODE_FUNCTIONS(LivestreamDriverEncode),
};
@@ -153,19 +157,19 @@ const LogCameraInfo driver_camera_info{
const LogCameraInfo stream_road_camera_info{
.thread_name = "road_cam_encoder",
.stream_type = VISION_STREAM_ROAD,
.encoder_infos = {stream_road_encoder_info}
.encoder_infos = {stream_road_encoder_info},
};
const LogCameraInfo stream_wide_road_camera_info{
.thread_name = "wide_road_cam_encoder",
.stream_type = VISION_STREAM_WIDE_ROAD,
.encoder_infos = {stream_wide_road_encoder_info}
.encoder_infos = {stream_wide_road_encoder_info},
};
const LogCameraInfo stream_driver_camera_info{
.thread_name = "driver_cam_encoder",
.stream_type = VISION_STREAM_DRIVER,
.encoder_infos = {stream_driver_encoder_info}
.encoder_infos = {stream_driver_encoder_info},
};
const LogCameraInfo cameras_logged[] = {road_camera_info, wide_road_camera_info, driver_camera_info};
+18 -1
View File
@@ -45,8 +45,9 @@ class ScrollState(Enum):
class GuiScrollPanel2:
def __init__(self, horizontal: bool = True) -> None:
def __init__(self, horizontal: bool = True, handle_out_of_bounds: bool = True) -> None:
self._horizontal = horizontal
self._handle_out_of_bounds = handle_out_of_bounds
self._state = ScrollState.STEADY
self._offset: rl.Vector2 = rl.Vector2(0, 0)
self._initial_click_event: MouseEvent | None = None
@@ -85,6 +86,20 @@ class GuiScrollPanel2:
"""Returns (max_offset, min_offset) for the given bounds and content size."""
return 0.0, min(0.0, bounds_size - content_size)
def _clamp_offset(self, bounds_size: float, content_size: float) -> None:
if self._handle_out_of_bounds:
return
max_offset, min_offset = self._get_offset_bounds(bounds_size, content_size)
offset = self.get_offset()
clamped_offset = max(min_offset, min(max_offset, offset))
if clamped_offset == offset:
return
self.set_offset(clamped_offset)
if (clamped_offset == max_offset and self._velocity > 0) or (clamped_offset == min_offset and self._velocity < 0):
self._velocity = 0.0
def _update_state(self, bounds_size: float, content_size: float, snap_target: float | None) -> None:
"""Runs per render frame, independent of mouse events. Updates auto-scrolling state and velocity."""
max_offset, min_offset = self._get_offset_bounds(bounds_size, content_size)
@@ -138,6 +153,8 @@ class GuiScrollPanel2:
factor = 1.0 - math.exp(-SNAP_RATE * dt)
self.set_offset(self.get_offset() + dist * factor)
self._clamp_offset(bounds_size, content_size)
def _handle_mouse_event(self, mouse_event: MouseEvent, bounds: rl.Rectangle, bounds_size: float,
content_size: float) -> None:
max_offset, min_offset = self._get_offset_bounds(bounds_size, content_size)
+10 -3
View File
@@ -75,7 +75,6 @@ class _Scroller(Widget):
self._items: list[Widget] = []
self._horizontal = horizontal
self._snap_items = snap_items
assert not self._snap_items or self._horizontal, "Snapping is only supported for horizontal scrolling"
self._spacing = spacing
self._pad = pad
@@ -191,12 +190,20 @@ class _Scroller(Widget):
snap_target: float | None = None
if self._snap_items and visible_items and self._scrolling_to[0] is None:
# TODO: this doesn't handle two small buttons at the edges well
center_pos = self._rect.x + self._rect.width / 2
closest_delta_pos = min((((item.rect.x + item.rect.width / 2) - center_pos) for item in visible_items), key=abs)
center_pos = (self._rect.x + self._rect.width / 2) if self._horizontal else (self._rect.y + self._rect.height / 2)
closest_delta_pos = min(
(self._item_center_pos(item) - center_pos for item in visible_items),
key=abs,
)
snap_target = self.scroll_panel.get_offset() - closest_delta_pos
return self.scroll_panel.update(self._rect, content_size, snap_target=snap_target)
def _item_center_pos(self, item: Widget) -> float:
if self._horizontal:
return item.rect.x + item.rect.width / 2
return item.rect.y + item.rect.height / 2
@property
def moving_items(self) -> bool:
return len(self._move_animations) > 0 or len(self._move_lift) > 0
+31 -4
View File
@@ -1,4 +1,5 @@
import asyncio
import struct
import time
import av
@@ -7,6 +8,13 @@ from teleoprtc.tracks import TiciVideoStreamTrack
from cereal import messaging
from openpilot.common.realtime import DT_MDL, DT_DMON
# arbitrary 16-byte UUID identifying openpilot frame-timing SEI messages
TIMING_SEI_UUID = bytes([
0xa5, 0xe0, 0xc4, 0xa4, 0x5b, 0x6e, 0x4e, 0x1e,
0x9c, 0x7e, 0x12, 0x34, 0x56, 0x78, 0x9a, 0xbc,
])
_SEI_PREFIX = b'\x00\x00\x00\x01\x06\x05\x30' + TIMING_SEI_UUID
class LiveStreamVideoStreamTrack(TiciVideoStreamTrack):
camera_to_sock_mapping = {
@@ -19,9 +27,30 @@ class LiveStreamVideoStreamTrack(TiciVideoStreamTrack):
dt = DT_DMON if camera_type == "driver" else DT_MDL
super().__init__(camera_type, dt)
self._sock = messaging.sub_sock(self.camera_to_sock_mapping[camera_type], conflate=True)
self._sock = self._make_sock(camera_type)
self._pts = 0
self._t0_ns = time.monotonic_ns()
self.timing_sei_enabled = False
def _make_sock(self, camera_type: str) -> messaging.SubSocket:
return messaging.sub_sock(self.camera_to_sock_mapping[camera_type], conflate=True)
def switch_camera(self, camera_type: str) -> None:
self._sock = self._make_sock(camera_type)
def _build_frame_data(self, msg) -> bytes:
encode_data = getattr(msg, msg.which())
if not self.timing_sei_enabled:
return encode_data.header + encode_data.data
idx = encode_data.idx
sei_nal = _SEI_PREFIX + struct.pack('>4d',
(idx.timestampEof - idx.timestampSof) / 1e6,
(msg.logMonoTime - idx.timestampEof) / 1e6,
(time.monotonic_ns() - msg.logMonoTime) / 1e6,
time.time() * 1000, # noqa: TID251
) + b'\x80'
return encode_data.header + sei_nal + encode_data.data
async def recv(self):
while True:
@@ -30,9 +59,7 @@ class LiveStreamVideoStreamTrack(TiciVideoStreamTrack):
break
await asyncio.sleep(0.005)
evta = getattr(msg, msg.which())
packet = av.Packet(evta.header + evta.data)
packet = av.Packet(self._build_frame_data(msg))
packet.time_base = self._time_base
self._pts = ((time.monotonic_ns() - self._t0_ns) * self._clock_rate) // 1_000_000_000
+204 -62
View File
@@ -1,7 +1,11 @@
#!/usr/bin/env python3
from abc import abstractmethod
import os
import time
import argparse
import asyncio
import contextlib
import json
import uuid
import logging
@@ -19,11 +23,39 @@ if TYPE_CHECKING:
from aiortc.rtcdatachannel import RTCDataChannel
from openpilot.system.webrtc.schema import generate_field
from openpilot.common.params import Params
from cereal import messaging, log
class CerealOutgoingMessageProxy:
class AsyncTaskRunner:
def __init__(self):
self.is_running = False
self.task = None
self.logger = logging.getLogger("webrtcd")
def start(self):
assert self.task is None
self.task = asyncio.create_task(self.run())
async def stop(self):
if self.task is None:
return
task = self.task
self.task = None
if task.done():
return
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
@abstractmethod
async def run(self):
pass
class CerealOutgoingMessageProxy(AsyncTaskRunner):
def __init__(self, sm: messaging.SubMaster):
super().__init__()
self.sm = sm
self.channels: list[RTCDataChannel] = []
@@ -55,6 +87,19 @@ class CerealOutgoingMessageProxy:
for channel in self.channels:
channel.send(encoded_msg)
async def run(self):
from aiortc.exceptions import InvalidStateError
while True:
try:
self.update()
except InvalidStateError:
self.logger.warning("Cereal outgoing proxy invalid state (connection closed)")
break
except Exception:
self.logger.exception("Cereal outgoing proxy failure")
await asyncio.sleep(0.01)
class CerealIncomingMessageProxy:
def __init__(self, pm: messaging.PubMaster):
@@ -72,37 +117,6 @@ class CerealIncomingMessageProxy:
self.pm.send(msg_type, msg)
class CerealProxyRunner:
def __init__(self, proxy: CerealOutgoingMessageProxy):
self.proxy = proxy
self.is_running = False
self.task = None
self.logger = logging.getLogger("webrtcd")
def start(self):
assert self.task is None
self.task = asyncio.create_task(self.run())
def stop(self):
if self.task is None or self.task.done():
return
self.task.cancel()
self.task = None
async def run(self):
from aiortc.exceptions import InvalidStateError
while True:
try:
self.proxy.update()
except InvalidStateError:
self.logger.warning("Cereal outgoing proxy invalid state (connection closed)")
break
except Exception:
self.logger.exception("Cereal outgoing proxy failure")
await asyncio.sleep(0.01)
class DynamicPubMaster(messaging.PubMaster):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
@@ -115,21 +129,90 @@ class DynamicPubMaster(messaging.PubMaster):
self.sock[service] = messaging.pub_sock(service)
class LivestreamBitrateController(AsyncTaskRunner):
bitrates = [500_000, 1_500_000, int(os.environ.get("STREAM_BITRATE", 5_000_000))]
label_to_bitrate = { "high": bitrates[2], "med": bitrates[1], "low": bitrates[0]}
sample_interval = 0.2
high_level = 0.1 # drop immediately
med_level = 0.05 # drop after # of samples
low_level = 0 # raise after # of samples
down_samples = 5 # 1s
param_name = "LivestreamEncoderBitrate"
def __init__(self, peer_connection: Any):
super().__init__()
self.pc = peer_connection
self.params = Params()
self.level = 0
self.prev_lost, self.prev_sent = None, None
self.counter = 0
self.up_samples = 5 # 1s
self._auto = True
async def run(self):
while True:
await asyncio.sleep(self.sample_interval)
if not self._auto:
continue
loss_rate = await self._sample()
if loss_rate is None:
continue
if loss_rate >= self.med_level and self.level > 0:
self.counter += 1
if self.counter >= self.down_samples or loss_rate >= self.high_level:
self.level -= 1
self.up_samples *= 2 # exponential backoff before raising again
self.counter = 0
self._publish(self.bitrates[self.level])
elif loss_rate <= self.low_level and self.level < len(self.bitrates) - 1:
self.counter -= 1
if -self.counter >= self.up_samples:
self.level += 1
self.counter = 0
self._publish(self.bitrates[self.level])
async def _sample(self) -> float | None:
report = await self.pc.getStats()
packets_lost = packets_sent = 0
for s in report.values():
if s.type == "remote-inbound-rtp":
packets_lost += s.packetsLost
elif s.type == "outbound-rtp":
packets_sent += s.packetsSent
if self.prev_lost is None:
self.prev_lost, self.prev_sent = packets_lost, packets_sent
return None
lost_delta = max(0, packets_lost - self.prev_lost)
sent_delta = max(0, packets_sent - self.prev_sent)
self.prev_lost, self.prev_sent = packets_lost, packets_sent
return lost_delta / sent_delta if sent_delta else 0.0
def _publish(self, bitrate: float):
self.params.put(self.param_name, bitrate)
def set_quality(self, quality):
if quality in self.label_to_bitrate:
self._publish(self.label_to_bitrate[quality])
self._auto = False
elif quality == "auto":
self._auto = True
class StreamSession:
shared_pub_master = DynamicPubMaster([])
def __init__(self, sdp: str, cameras: list[str], incoming_services: list[str], outgoing_services: list[str], debug_mode: bool = False):
def __init__(self, sdp: str, init_camera: str, incoming_services: list[str], outgoing_services: list[str], debug_mode: bool = False):
from aiortc.mediastreams import VideoStreamTrack
from openpilot.system.webrtc.device.video import LiveStreamVideoStreamTrack
from teleoprtc import WebRTCAnswerBuilder
from teleoprtc.info import parse_info_from_offer
config = parse_info_from_offer(sdp)
builder = WebRTCAnswerBuilder(sdp)
assert len(cameras) == config.n_expected_camera_tracks, "Incoming stream has misconfigured number of video tracks"
for cam in cameras:
builder.add_video_stream(cam, LiveStreamVideoStreamTrack(cam) if not debug_mode else VideoStreamTrack())
self.video_track = LiveStreamVideoStreamTrack(init_camera) if not debug_mode else VideoStreamTrack()
builder.add_video_stream(init_camera, self.video_track)
self.stream = builder.stream()
self.identifier = str(uuid.uuid4())
@@ -137,35 +220,60 @@ class StreamSession:
self.incoming_bridge: CerealIncomingMessageProxy | None = None
self.incoming_bridge_services = incoming_services
self.outgoing_bridge: CerealOutgoingMessageProxy | None = None
self.outgoing_bridge_runner: CerealProxyRunner | None = None
self.bitrate_controller: LivestreamBitrateController | None = None
if len(incoming_services) > 0:
self.incoming_bridge = CerealIncomingMessageProxy(self.shared_pub_master)
if len(outgoing_services) > 0:
self.outgoing_bridge = CerealOutgoingMessageProxy(messaging.SubMaster(outgoing_services))
self.outgoing_bridge_runner = CerealProxyRunner(self.outgoing_bridge)
self.bitrate_controller = LivestreamBitrateController(self.stream.peer_connection)
self.run_task: asyncio.Task | None = None
self._cleanup_lock = asyncio.Lock()
self._cleanup_done = False
self.logger = logging.getLogger("webrtcd")
self.logger.info("New stream session (%s), cameras %s, incoming services %s, outgoing services %s",
self.identifier, cameras, incoming_services, outgoing_services)
self.logger.info(
"New stream session (%s), init camera %s, incoming services %s, outgoing services %s",
self.identifier, init_camera, incoming_services, outgoing_services,
)
def start(self):
self.run_task = asyncio.create_task(self.run())
def stop(self):
if self.run_task.done():
return
self.run_task.cancel()
async def stop(self):
if self.run_task is not None and not self.run_task.done() and self.run_task is not asyncio.current_task():
self.run_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self.run_task
self.run_task = None
asyncio.run(self.post_run_cleanup())
await self.post_run_cleanup()
async def get_answer(self):
return await self.stream.start()
async def message_handler(self, message: bytes):
def message_handler(self, message: bytes):
assert self.incoming_bridge is not None
try:
self.incoming_bridge.send(message)
payload = json.loads(message) if isinstance(message, (bytes, str)) else None
if isinstance(payload, dict):
msg_type = payload.get("type")
match msg_type:
case "livestreamCameraSwitch":
self.video_track.switch_camera(payload["data"]["camera"])
case "livestreamSettings":
self.bitrate_controller.set_quality(payload["data"]["quality"])
case "clockSync":
pong = json.dumps({"type": "clockSync", "data": {
"action": "pong", "browserSendTime": payload["data"]["browserSendTime"], "deviceTime": time.time() * 1000, # noqa: TID251
}})
self.stream.get_messaging_channel().send(pong)
case "enableTimingSei":
if hasattr(self.video_track, 'timing_sei_enabled'):
self.video_track.timing_sei_enabled = bool(payload["data"]["enabled"])
case _:
if payload.get("type") not in self.incoming_bridge_services:
return
self.incoming_bridge.send(message)
except Exception:
self.logger.exception("Cereal incoming proxy failure")
@@ -176,29 +284,38 @@ class StreamSession:
if self.incoming_bridge is not None:
await self.shared_pub_master.add_services_if_needed(self.incoming_bridge_services)
self.stream.set_message_handler(self.message_handler)
if self.outgoing_bridge_runner is not None:
if self.outgoing_bridge is not None:
channel = self.stream.get_messaging_channel()
self.outgoing_bridge_runner.proxy.add_channel(channel)
self.outgoing_bridge_runner.start()
self.outgoing_bridge.add_channel(channel)
self.outgoing_bridge.start()
self.bitrate_controller.start()
self.logger.info("Stream session (%s) connected", self.identifier)
await self.stream.wait_for_disconnection()
await self.post_run_cleanup()
self.logger.info("Stream session (%s) ended", self.identifier)
except Exception:
self.logger.exception("Stream session failure")
finally:
await self.post_run_cleanup()
async def post_run_cleanup(self):
await self.stream.stop()
if self.outgoing_bridge is not None:
self.outgoing_bridge_runner.stop()
async with self._cleanup_lock:
if self._cleanup_done:
return
self._cleanup_done = True
if self.bitrate_controller is not None:
await self.bitrate_controller.stop()
if self.outgoing_bridge is not None:
await self.outgoing_bridge.stop()
await self.stream.stop()
@dataclass
class StreamRequestBody:
sdp: str
cameras: list[str]
initCamera: str
bridge_services_in: list[str] = field(default_factory=list)
bridge_services_out: list[str] = field(default_factory=list)
@@ -208,11 +325,33 @@ async def get_stream(request: 'web.Request'):
raw_body = await request.json()
body = StreamRequestBody(**raw_body)
session = StreamSession(body.sdp, body.cameras, body.bridge_services_in, body.bridge_services_out, debug_mode)
answer = await session.get_answer()
session.start()
async with request.app['stream_lock']:
# Fully disconnect any other active stream before starting the replacement.
for sid, s in list(stream_dict.items()):
if s.run_task and not s.run_task.done():
try:
ch = s.stream.get_messaging_channel()
ch.send(json.dumps({"type": "connectionReplaced", "data": "Another device has connected, closing this session."}))
except Exception:
pass
await s.stop()
del stream_dict[sid]
stream_dict[session.identifier] = session
session = StreamSession(body.sdp, body.initCamera, body.bridge_services_in, body.bridge_services_out, debug_mode)
try:
answer = await session.get_answer()
except ValueError as e:
await session.stop()
raise web.HTTPBadRequest(
text=json.dumps({"error": "invalid_sdp", "message": str(e)}),
content_type="application/json",
) from e
except Exception:
await session.stop()
raise
session.start()
stream_dict[session.identifier] = session
return web.json_response({"sdp": answer.sdp, "type": answer.type})
@@ -224,6 +363,7 @@ async def get_schema(request: 'web.Request'):
schema_dict = {s: generate_field(log.Event.schema.fields[s]) for s in services}
return web.json_response(schema_dict)
async def post_notify(request: 'web.Request'):
try:
payload = await request.json()
@@ -239,9 +379,10 @@ async def post_notify(request: 'web.Request'):
return web.Response(status=200, text="OK")
async def on_shutdown(app: 'web.Application'):
for session in app['streams'].values():
session.stop()
await session.stop()
del app['streams']
@@ -254,6 +395,7 @@ def webrtcd_thread(host: str, port: int, debug: bool):
app = web.Application()
app['streams'] = dict()
app['stream_lock'] = asyncio.Lock()
app['debug'] = debug
app.on_shutdown.append(on_shutdown)
app.router.add_post("/stream", get_stream)
+1 -1
View File
@@ -56,7 +56,7 @@ async def ping(request: 'web.Request'):
async def offer(request: 'web.Request'):
params = await request.json()
body = StreamRequestBody(params["sdp"], ["driver"], ["testJoystick"], ["carState"])
body = StreamRequestBody(params["sdp"], "driver", ["testJoystick"], ["carState"])
body_json = json.dumps(dataclasses.asdict(body))
logger.info("Sending offer to webrtcd...")