set var

* models: download whole-file artifacts with parallel byte ranges

HuggingFace's Xet CAS throttles each connection to roughly 1-2 MB/s but does
not limit connection count, so the single-stream _download_file left the link
idle most of the time. Replace it with a downloader that probes the size with
a one-byte Range request, creates the file at its final size and streams each
8 MB piece straight to its offset from one of 12 worker threads. The file is
never copied or assembled and only ever occupies its own size on disk. A
server that answers the probe with 200 gets one plain stream instead.
Every request sends Accept-Encoding: identity, because byte ranges count
encoded bytes while requests hands back decoded ones, so a compressed
response would land at the wrong offsets and fail as a short read.

Workers only write bytes; a reporter coroutine on the event loop publishes
progress every 0.5 s (instead of every 128 KB), re-raises the first worker
failure and polls the cancel ref. Transient errors (connection, timeout,
truncated body, HTTP 408/429/500/502/503/504) give a piece up to three
attempts; any other HTTP error fails the download at once. The whole-file
path now also publishes "verifying" while hashing before and after the
transfer, and the download is hashed against the manifest exactly as before.

A cancel has to wait for in-flight pieces to unwind (a blocked socket read
cannot be interrupted), so the row shows "canceling..." through a new
DownloadProgress.canceling flag instead of a frozen bar, and the mici cancel
button hides once the ref is gone, as it already does on the big UI.

DownloadProgress gains speed, canceling and resuming, appended to the capnp so
older readers are unaffected. All three are declared here so the wire format
lands in one place; speed and resuming are filled in by the follow-up speed
and resume changes.

Builds on #2050, which drops chunking and publishes models as whole files;
this makes that whole-file download parallel.

Measured on a comma 3X on WiFi: ~2 MB/s single stream, 17-24 MB/s steady and
30-60 MB/s peak with 12 range connections, the count picked from an on-device
sweep. Disk is not the bottleneck (eMMC writes at 56-67 MB/s with fdatasync).

Tests drive the real downloader against a local threaded range server: piece
tiling, actual parallelism (a stalled piece does not block the others), the
no-range fallback, per-piece retry, 4xx, cancel via ModelManager_DownloadRef
and mid-piece, verify/cached/downloaded state, a probe failure moving
past verifying, and hash mismatch cleanup.

Also drops two chunk-era leftovers in the tests: a stale "no chunks" note on
manifest_bundle and a one-item URL loop in the live manifest check.

Assisted-by: Claude Code (Claude Opus 4.8)
Assisted-by: Claude Code (Claude Opus 5.5)

* remove unneeded comments

* models: trim the manager tests to the regressions they catch

Each remaining test is there because a deliberate break in manager.py,
helpers.py or fetcher.py fails it. Tests that only re-checked what another
already catches are merged into a happy-path download or a table, and
checks of constants rather than behaviour are dropped. The fake server
now starts per test, keeps its state in one dataclass, and can cut a
response short.

Against 66 deliberate breaks, every one the old tests caught still fails,
and a 404 being retried, a short read and its retry, and the display
flags choosing the big model without a GPU are now caught too.

Assisted-by: Claude Code (Claude Opus 5.5)

* Update test_manager_download.py

* Update manager.py

---------

Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>

.github/workflows/test_models.yml

@@ -3,9 +3,62 @@ on:
   pull_request:
     paths:
       - 'tinygrad_repo'
+      - 'openpilot/sunnypilot/modeld_v2/**'
   workflow_dispatch:
 
+env:
+  CI: 1
+  GIT_CONFIG_COUNT: 1
+  GIT_CONFIG_KEY_0: lfs.fetchexclude
+  GIT_CONFIG_VALUE_0: openpilot/selfdrive/modeld/models/big_driving_tinygrad.pkl
+
 jobs:
+  plain-artifact:
+    name: Plain artifact round trip and pickle converter
+    runs-on: ubuntu-24.04
+    env:
+      PYTHONPATH: ${{ github.workspace }}/tinygrad_repo:${{ github.workspace }}
+      TINYGRAD_TESTS: 1
+    steps:
+      - uses: actions/checkout@v7
+      - run: ./tools/op.sh setup
+      - name: Build openpilot
+        run: scons -j$(nproc)
+      - name: Compile, load, cache and fuse a model with the pinned tinygrad
+        env:
+          DEV: CPU
+        run: tools/op.sh test -v openpilot/sunnypilot/modeld_v2/tests/test_plain_artifact.py
+      - name: Install LLVM 20 and ROCm comgr for the amd mock, as tinygrad_repo/.github/actions/setup-tinygrad does
+        run: |
+          . /etc/os-release
+          curl -fsSL https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc > /dev/null
+          echo "deb http://apt.llvm.org/$VERSION_CODENAME/ llvm-toolchain-$VERSION_CODENAME-20 main" | sudo tee /etc/apt/sources.list.d/llvm.list
+          sudo mkdir -p /etc/apt/keyrings
+          curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
+          echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.1 $VERSION_CODENAME main" | sudo tee /etc/apt/sources.list.d/rocm.list
+          printf 'Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600\n' | sudo tee /etc/apt/preferences.d/rocm-pin-600
+          sudo apt-get -qq update && sudo apt-get -qq install -y --no-install-recommends libllvm20 clang-20 lld-20 comgr
+          printf '/opt/rocm/lib\n/opt/rocm/lib64\n' | sudo tee /etc/ld.so.conf.d/rocm.conf && sudo ldconfig
+          echo "/usr/lib/llvm-20/bin" >> "$GITHUB_PATH"
+      - name: Convert a pickled USB+AMD jit and replay it under tinygrad's amd mock
+        env:
+          DEV: MOCKUSB+AMD
+          CHECK_OOB: 0
+          GMMU: 0
+        run: tools/op.sh test -v openpilot/sunnypilot/modeld_v2/tests/test_convert_upstream_pkl.py
+      - name: Where tinygrad moved, for the helper or failure point named above
+        if: failure() && github.event_name == 'pull_request'
+        run: |
+          git fetch -q --depth=1 origin "${{ github.base_ref }}"
+          old=$(git rev-parse "origin/${{ github.base_ref }}:tinygrad_repo") new=$(git rev-parse HEAD:tinygrad_repo)
+          git -C tinygrad_repo fetch -q origin "$old" || true
+          echo "tinygrad_repo $old -> $new, in what helpers.py and plain_artifact_compiler.py read:"
+          git -C tinygrad_repo diff "$old" "$new" -- tinygrad/uop/ops.py tinygrad/engine/jit.py tinygrad/engine/realize.py tinygrad/device.py \
+            tinygrad/helpers.py tinygrad/dtype.py tinygrad/renderer/__init__.py
+          echo "and in what convert_upstream_pkl.py decodes:"
+          git -C tinygrad_repo diff --stat "$old" "$new" -- tinygrad/runtime/autogen/am/pm4_nv.py tinygrad/runtime/autogen/amd_gpu.py
+          git -C tinygrad_repo diff "$old" "$new" -- tinygrad/runtime/support/hcq2.py tinygrad/runtime/support/usb.py tinygrad/runtime/ops_amd.py
+
   generate-matrix:
     runs-on: ubuntu-latest
     outputs:

.gitmodules

@@ -4,7 +4,6 @@
 [submodule "opendbc"]
   path = opendbc_repo
   url = https://github.com/sunnypilot/opendbc.git
-  branch = toyotabsmhack
 [submodule "msgq"]
   path = msgq_repo
   url = https://github.com/sunnypilot/msgq.git

openpilot/cereal/custom.capnp

@@ -138,6 +138,9 @@ struct ModelManagerSP @0xaedffd8f31e7b55d {
     status @0 :DownloadStatus;
     progress @1 :Float32;
     eta @2 :UInt32;
+    speed @3 :Float32;
+    canceling @4 :Bool;
+    resuming @5 :Bool;
   }
 
   struct Chunk {
@@ -465,7 +468,6 @@ struct ModelDataV2SP @0xa1680744031fdb2d {
   laneTurnDirection @0 :TurnDirection;
   leftLaneChangeEdgeBlock @1 :Bool;
   rightLaneChangeEdgeBlock @2 :Bool;
-  lanefulCurvature @3 :Float32;
 
   enum TurnDirection {
     none @0;

openpilot/common/params_keys.h

@@ -170,7 +170,6 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
     {"DevUIInfo", {PERSISTENT | BACKUP, INT, "0"}},
     {"EnableCopyparty", {PERSISTENT | BACKUP, BOOL}},
     {"EnableGithubRunner", {PERSISTENT | BACKUP, BOOL}},
-    {"FollowDistanceAccelBoost", {PERSISTENT | BACKUP, BOOL, "0"}},
     {"GreenLightAlert", {PERSISTENT | BACKUP, BOOL, "0"}},
     {"GithubRunnerSufficientVoltage", {CLEAR_ON_MANAGER_START , BOOL}},
     {"HasAcceptedTermsSP", {PERSISTENT, STRING, "0"}},
@@ -247,7 +246,6 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
     {"TeslaMadsScreenButton", {PERSISTENT | BACKUP, INT, "0"}},
     {"ToyotaEnforceStockLongitudinal", {PERSISTENT | BACKUP, BOOL, "0"}},
     {"ToyotaStopAndGoHack", {PERSISTENT | BACKUP, BOOL, "0"}},
-    {"ToyotaEnhancedBsm", {PERSISTENT | BACKUP, BOOL, "0"}},
 
     {"BlindSpot", {PERSISTENT | BACKUP, BOOL, "0"}},
 
@@ -258,7 +256,6 @@ inline static std::unordered_map<std::string, ParamKeyAttributes> keys = {
     {"LagdValueCache", {PERSISTENT, FLOAT, "0.2"}},
     {"LaneTurnDesire", {PERSISTENT | BACKUP, BOOL, "0"}},
     {"LaneTurnValue", {PERSISTENT | BACKUP, FLOAT, "19.0"}},
-    {"Laneful", {PERSISTENT | BACKUP, BOOL, "0"}},
     {"PlanplusControl", {PERSISTENT | BACKUP, FLOAT, "1.0"}},
 
     // mapd

openpilot/selfdrive/controls/lib/longcontrol.py

@@ -9,12 +9,6 @@ CONTROL_N_T_IDX = ModelConstants.T_IDXS[:CONTROL_N]
 
 LongCtrlState = car.CarControl.Actuators.LongControlState
 
-STOPPING_JERK_MIN = 0.3  # m/s^3
-STOPPING_JERK_MAX = 2.0  # m/s^3
-STOPPING_HOLD_SPEED = 0.05  # m/s, below this ramp into the standstill hold
-STOPPING_HOLD_JERK = 1.0  # m/s^3, command ramp rate for the standstill hold
-STOPPING_EXIT_FRAMES = 15  # 0.15 s at 100 Hz, debounce should_stop at standstill
-
 
 def long_control_state_trans(CP_SP, active, long_control_state,
                              should_stop, brake_pressed, cruise_standstill):
@@ -53,57 +47,28 @@ class LongControl:
     self.pid = PIDController(0.0, (CP.longitudinalTuning.kiBP, CP.longitudinalTuning.kiV),
                              rate=1 / DT_CTRL)
     self.last_output_accel = 0.0
-    self.stopping_jerk = STOPPING_JERK_MIN
-    self.stopping_exit_frames = 0
 
   def reset(self):
     self.pid.reset()
 
-  def _stopping_accel(self, v_ego, standstill):
-    v_ego = max(float(v_ego), 0.0)
-    if standstill or v_ego < STOPPING_HOLD_SPEED:
-      goal = float(self.CP.stopAccel)
-      jerk = STOPPING_HOLD_JERK
-    else:
-      taper = -np.sqrt(2.0 * self.stopping_jerk * v_ego)
-      goal = max(self.last_output_accel, taper) if self.last_output_accel < 0.0 else taper
-      jerk = self.stopping_jerk
-
-    output_accel = np.clip(min(goal, 0.0), self.last_output_accel - jerk * DT_CTRL,
-                           self.last_output_accel + jerk * DT_CTRL)
-    return float(output_accel)
-
   def update(self, active, CS, a_target, should_stop, accel_limits):
     """Update longitudinal control. This updates the state machine and runs a PID loop"""
     self.pid.neg_limit = accel_limits[0]
     self.pid.pos_limit = accel_limits[1]
 
-    if self.long_control_state == LongCtrlState.stopping and not should_stop:
-      self.stopping_exit_frames += 1
-    else:
-      self.stopping_exit_frames = 0
-    should_stop_trans = should_stop or (self.long_control_state == LongCtrlState.stopping and
-                                        self.stopping_exit_frames < STOPPING_EXIT_FRAMES)
-
-    prev_state = self.long_control_state
     self.long_control_state = long_control_state_trans(self.CP_SP, active, self.long_control_state,
-                                                       should_stop_trans, CS.brakePressed,
+                                                       should_stop, CS.brakePressed,
                                                        CS.cruiseState.standstill)
-
-    if self.long_control_state == LongCtrlState.stopping and prev_state != LongCtrlState.stopping:
-      entry_v = max(float(CS.vEgo), STOPPING_HOLD_SPEED)
-      entry_a = min(float(self.last_output_accel), 0.0)
-      if entry_a == 0.0:
-        entry_a = min(float(a_target), 0.0)
-      self.stopping_jerk = float(np.clip(entry_a ** 2 / (2.0 * entry_v),
-                                         STOPPING_JERK_MIN, STOPPING_JERK_MAX))
-
     if self.long_control_state == LongCtrlState.off:
       self.reset()
       output_accel = 0.
 
     elif self.long_control_state == LongCtrlState.stopping:
-      output_accel = self._stopping_accel(CS.vEgo, CS.standstill)
+      output_accel = self.last_output_accel
+      if output_accel > self.CP.stopAccel:
+        output_accel = min(output_accel, 0.0)
+        # TODO: can we just go straight to stopAccel?
+        output_accel -= 1.0 * DT_CTRL  # m/s^2/s while trying to stop
       self.reset()
 
     else:  # LongCtrlState.pid

openpilot/selfdrive/controls/tests/test_accel_boost.py

@@ -1,101 +0,0 @@
-from types import SimpleNamespace
-
-from openpilot.cereal import log
-from openpilot.common.realtime import DT_MDL
-from openpilot.common.test import OpenpilotTestCase
-from openpilot.selfdrive.controls.lib.accel_boost import (
-  AccelBoost, ACCEL_BOOST_MAX, ACCEL_BOOST_PER_OVERRIDE, ACCEL_BOOST_RATE
-)
-
-E2E_ACCEL = 0.5
-PLANNER_ACCEL = 2.0
-RATE_PER_STEP = ACCEL_BOOST_RATE * DT_MDL
-
-
-class FakeSubMaster:
-  def __init__(self, enabled=True, gas_pressed=False, v_ego=15.0,
-               personality=log.LongitudinalPersonality.standard):
-    self._messages = {
-      'selfdriveState': SimpleNamespace(enabled=enabled, personality=personality),
-      'carState': SimpleNamespace(gasPressed=gas_pressed, vEgo=v_ego),
-    }
-
-  def __getitem__(self, key):
-    return self._messages[key]
-
-
-def run_boost(boost, sm, steps=1):
-  out = E2E_ACCEL
-  for _ in range(steps):
-    out = boost.update(sm, E2E_ACCEL, PLANNER_ACCEL, PLANNER_ACCEL)
-  return out
-
-
-class TestAccelBoost(OpenpilotTestCase):
-  def _boost_to_max(self):
-    boost = AccelBoost()
-    sm = FakeSubMaster()
-    steps = round(ACCEL_BOOST_PER_OVERRIDE / RATE_PER_STEP)
-    # Eligibility is latched on a frame without gas, then applied on the next override.
-    run_boost(boost, sm)
-    for _ in range(10):
-      sm['carState'].gasPressed = True
-      run_boost(boost, sm, steps)
-      sm['carState'].gasPressed = False
-      run_boost(boost, sm)
-    return boost
-
-  def test_no_boost_without_gas(self):
-    boost = AccelBoost()
-    sm = FakeSubMaster(gas_pressed=False)
-    out = run_boost(boost, sm)
-    self.assertEqual(boost.total_boost, 0.0)
-    self.assertAlmostEqual(out, E2E_ACCEL)
-
-  def test_boost_requires_e2e_to_be_limiting(self):
-    boost = AccelBoost()
-    sm = FakeSubMaster(gas_pressed=False)
-    # Eligibility is latched before the override, so keep the e2e plan at the planner limit here.
-    boost.update(sm, E2E_ACCEL, E2E_ACCEL, E2E_ACCEL)
-    sm['carState'].gasPressed = True
-    for _ in range(10):
-      boost.update(sm, E2E_ACCEL, E2E_ACCEL, E2E_ACCEL)
-    self.assertEqual(boost.total_boost, 0.0)
-
-  def test_per_override_cap(self):
-    boost = AccelBoost()
-    sm = FakeSubMaster(gas_pressed=False)
-    run_boost(boost, sm)
-    sm['carState'].gasPressed = True
-    run_boost(boost, sm, 1000)
-    self.assertAlmostEqual(boost.total_boost, ACCEL_BOOST_PER_OVERRIDE)
-
-  def test_max_boost(self):
-    boost = self._boost_to_max()
-    self.assertAlmostEqual(boost.total_boost, ACCEL_BOOST_MAX)
-
-  def test_boost_is_applied_to_e2e_accel(self):
-    boost = self._boost_to_max()
-    sm = FakeSubMaster(gas_pressed=True)
-    out = run_boost(boost, sm)
-    self.assertAlmostEqual(out, E2E_ACCEL + boost.total_boost)
-
-  def test_boost_does_not_depend_on_following_distance(self):
-    totals = []
-    for personality in (log.LongitudinalPersonality.relaxed,
-                        log.LongitudinalPersonality.standard,
-                        log.LongitudinalPersonality.aggressive):
-      boost = AccelBoost()
-      sm = FakeSubMaster(personality=personality)
-      steps = round(ACCEL_BOOST_PER_OVERRIDE / RATE_PER_STEP)
-      run_boost(boost, sm)
-      for _ in range(10):
-        sm['carState'].gasPressed = True
-        run_boost(boost, sm, steps)
-        sm['carState'].gasPressed = False
-        run_boost(boost, sm)
-      totals.append(boost.total_boost)
-
-    self.assertEqual(len(totals), 3)
-    for total in totals:
-      self.assertAlmostEqual(total, ACCEL_BOOST_MAX)

openpilot/selfdrive/controls/tests/test_longcontrol.py

@@ -1,8 +1,6 @@
-from opendbc.car.structs import car
-from openpilot.common.realtime import DT_CTRL
 from openpilot.common.test import OpenpilotTestCase
 from openpilot.cereal import custom
-from openpilot.selfdrive.controls.lib.longcontrol import LongControl, LongCtrlState, long_control_state_trans
+from openpilot.selfdrive.controls.lib.longcontrol import LongCtrlState, long_control_state_trans
 
 
 class TestLongControlStateTransition(OpenpilotTestCase):
@@ -44,112 +42,3 @@ class TestLongControlStateTransition(OpenpilotTestCase):
     next_state = long_control_state_trans(CP_SP, active, current_state,
                              should_stop=False, brake_pressed=False, cruise_standstill=False)
     assert next_state == LongCtrlState.pid
-
-
-class TestLongControlStopping(OpenpilotTestCase):
-  ACCEL_LIMITS = (-2.0, 2.0)
-
-  def make_control(self):
-    CP = car.CarParams.new_message()
-    CP.stopAccel = -2.0
-    CP.longitudinalTuning.kiBP = [0.0]
-    CP.longitudinalTuning.kiV = [0.0]
-    CP_SP = custom.CarParamsSP.new_message()
-    return LongControl(CP, CP_SP)
-
-  def make_cs(self, v_ego, a_ego, standstill=False, cruise_standstill=None):
-    CS = car.CarState.new_message()
-    CS.vEgo = float(v_ego)
-    CS.aEgo = float(a_ego)
-    CS.standstill = standstill
-    CS.brakePressed = False
-    CS.cruiseState.standstill = standstill if cruise_standstill is None else cruise_standstill
-    return CS
-
-  def enter_stopping(self, loc, v_ego=0.23, a_ego=-0.55):
-    # Run in pid first so last_output_accel is a realistic decel, then enter stopping.
-    out = 0.0
-    for _ in range(100):
-      out = loc.update(True, self.make_cs(2.0, a_ego), a_ego, False, self.ACCEL_LIMITS)
-    assert loc.long_control_state == LongCtrlState.pid
-    out = loc.update(True, self.make_cs(v_ego, a_ego), a_ego, True, self.ACCEL_LIMITS)
-    assert loc.long_control_state == LongCtrlState.stopping
-    return out
-
-  def test_entry_is_continuous(self):
-    loc = self.make_control()
-    out = self.enter_stopping(loc)
-    assert abs(out - loc.last_output_accel) < 1e-9
-    assert abs(out - (-0.55)) < 0.02
-
-  def test_positive_entry_ramps_down(self):
-    loc = self.make_control()
-    loc.last_output_accel = 0.3
-    loc.stopping_jerk = 0.66
-    out = loc._stopping_accel(0.23, False)
-    assert abs(out - (0.3 - 0.66 * DT_CTRL)) < 1e-6  # no jump to zero
-
-  def test_taper_never_steps(self):
-    loc = self.make_control()
-    v_ego = 0.23
-    out = self.enter_stopping(loc)
-    prev = out
-    max_step = 0.0
-    for _ in range(400):
-      out = loc.update(True, self.make_cs(v_ego, out), -0.55, True, self.ACCEL_LIMITS)
-      max_step = max(max_step, abs(out - prev))
-      prev = out
-      v_ego = max(0.0, v_ego + out * DT_CTRL)
-
-    # bounded by the release jerk (and the 1.0 m/s^3 hold ramp), never a step to stopAccel
-    assert max_step <= 1.0 * DT_CTRL + 1e-6
-    assert -2.01 < out < -1.9
-
-  def test_taper_releases_before_standstill(self):
-    loc = self.make_control()
-    v_ego = 0.23
-    out = self.enter_stopping(loc)
-    while v_ego > 0.11:
-      out = loc.update(True, self.make_cs(v_ego, out), -0.55, True, self.ACCEL_LIMITS)
-      v_ego = max(0.0, v_ego + out * DT_CTRL)
-    assert out > -0.5  # noticeably released before the hold takes over
-
-  def test_hot_entry_still_bounded(self):
-    loc = self.make_control()
-    out = self.enter_stopping(loc, v_ego=0.25, a_ego=-1.2)
-    prev = out
-    assert abs(out - (-1.2)) < 0.021
-    for _ in range(30):
-      out = loc.update(True, self.make_cs(0.25, out), -1.2, True, self.ACCEL_LIMITS)
-      assert abs(out - prev) <= 2.0 * DT_CTRL + 1e-6
-      prev = out
-
-  def test_hold_ramps_instead_of_stepping(self):
-    loc = self.make_control()
-    loc.last_output_accel = -0.3
-    out = loc._stopping_accel(0.0, True)
-    assert out > -0.32  # first hold frame is a ramp, not stopAccel
-    for _ in range(300):
-      prev = out
-      loc.last_output_accel = out
-      out = loc._stopping_accel(0.0, True)
-      assert abs(out - prev) <= 1.0 * DT_CTRL + 1e-6
-    assert -2.01 < out < -1.9
-
-  def test_should_stop_debounce(self):
-    loc = self.make_control()
-    self.enter_stopping(loc)
-    # settle at standstill
-    out = -0.55
-    for _ in range(300):
-      out = loc.update(True, self.make_cs(0.0, 0.0, standstill=True), 0.2, True, self.ACCEL_LIMITS)
-    assert out < -1.9
-    # one dropped should_stop frame must not release the brake. cruiseState.standstill is
-    # false here, matching the observed standstill where should_stop briefly drops.
-    out = loc.update(True, self.make_cs(0.0, 0.0, standstill=True, cruise_standstill=False), 0.2, False, self.ACCEL_LIMITS)
-    assert loc.long_control_state == LongCtrlState.stopping
-    assert out < -1.9
-    # and it must eventually exit when the request to move persists
-    for _ in range(20):
-      out = loc.update(True, self.make_cs(0.0, 0.0, standstill=True, cruise_standstill=False), 0.2, False, self.ACCEL_LIMITS)
-    assert loc.long_control_state == LongCtrlState.pid

openpilot/selfdrive/ui/mici/onroad/accel_boost_indicator.py

@@ -1,86 +0,0 @@
-import pyray as rl
-
-from openpilot.common.filter_simple import FirstOrderFilter
-from openpilot.selfdrive.ui.mici.onroad import SIDE_PANEL_WIDTH
-from openpilot.selfdrive.ui.ui_state import ui_state
-from openpilot.system.ui.lib.application import FontWeight, gui_app
-from openpilot.system.ui.lib.text_measure import measure_text_cached
-from openpilot.system.ui.widgets import Widget
-from openpilot.sunnypilot.selfdrive.controls.lib.accel_boost_ext import get_accel_boost_max
-
-BAR_WIDTH = 10
-VALUE_FONT_SIZE = 17
-LABEL_FONT_SIZE = 10
-
-COLOR_TRACK = rl.Color(255, 255, 255, 40)
-COLOR_TEXT = rl.Color(255, 255, 255, 235)
-
-PROFILE_STYLES = {
-  0: ("AGGR", rl.Color(255, 80, 60, 255)),
-  1: ("STD", rl.Color(0, 230, 120, 255)),
-  2: ("RELAX", rl.Color(70, 150, 255, 255)),
-}
-
-
-class AccelBoostIndicator(Widget):
-  def __init__(self):
-    super().__init__()
-    self._boost_filter = FirstOrderFilter(0.0, 0.15, 1 / gui_app.target_fps)
-    self._alpha_filter = FirstOrderFilter(0.0, 0.1, 1 / gui_app.target_fps)
-    self._font = gui_app.font(FontWeight.BOLD)
-    self._label_font = gui_app.font(FontWeight.MEDIUM)
-    self._profile_label = "STD"
-    self._profile_color = PROFILE_STYLES[1][1]
-    self._max_boost = 0.0
-
-  def _update_state(self):
-    sm = ui_state.sm
-    visible = (ui_state.has_longitudinal_control and sm.alive['selfdriveState'] and sm.alive['longitudinalPlan'] and
-               sm['selfdriveState'].enabled)
-    self._alpha_filter.update(1.0 if visible else 0.0)
-    self._boost_filter.update(float(sm['longitudinalPlan'].accelBoost) if visible else 0.0)
-
-    if sm.alive['selfdriveState']:
-      personality = sm['selfdriveState'].personality
-      self._profile_label, self._profile_color = PROFILE_STYLES.get(personality.raw, PROFILE_STYLES[1])
-      self._max_boost = get_accel_boost_max(personality)
-
-  def _render(self, _):
-    alpha = self._alpha_filter.x
-    if alpha < 0.01:
-      return
-
-    panel_x = self.rect.x + self.rect.width - SIDE_PANEL_WIDTH
-    margin = min(120.0, max(56.0, self.rect.height * 0.14))
-    track_top = self.rect.y + margin
-    track_bottom = self.rect.y + self.rect.height - margin
-    track_h = max(1.0, track_bottom - track_top)
-    cx = panel_x + SIDE_PANEL_WIDTH / 2
-    bar_x = cx - BAR_WIDTH / 2
-
-    # track
-    rl.draw_rectangle_rounded(rl.Rectangle(bar_x, track_top, BAR_WIDTH, track_h), 0.5, 12,
-                              rl.Color(COLOR_TRACK.r, COLOR_TRACK.g, COLOR_TRACK.b, int(COLOR_TRACK.a * alpha)))
-
-    if self._max_boost <= 0.0:
-      return
-
-    # Scale to the current profile's cap, so a maxed boost fills the bar.
-    boost = min(max(self._boost_filter.x, 0.0), self._max_boost)
-    fill_h = boost / self._max_boost * track_h
-    if fill_h > 0.0:
-      fill = rl.Color(self._profile_color.r, self._profile_color.g, self._profile_color.b, int(255 * alpha))
-      if fill_h < BAR_WIDTH:
-        rl.draw_circle(int(cx), int(track_bottom - fill_h / 2), max(1, int(BAR_WIDTH / 2)), fill)
-      else:
-        rl.draw_rectangle_rounded(rl.Rectangle(bar_x, track_bottom - fill_h, BAR_WIDTH, fill_h), 0.5, 12, fill)
-
-    label_size = measure_text_cached(self._label_font, self._profile_label, LABEL_FONT_SIZE)
-    rl.draw_text_ex(self._label_font, self._profile_label, rl.Vector2(cx - label_size.x / 2, track_top - 42),
-                    LABEL_FONT_SIZE, 0,
-                    rl.Color(self._profile_color.r, self._profile_color.g, self._profile_color.b, int(230 * alpha)))
-
-    value = f"+{boost:.2f}"
-    value_size = measure_text_cached(self._font, value, VALUE_FONT_SIZE)
-    rl.draw_text_ex(self._font, value, rl.Vector2(cx - value_size.x / 2, track_top - 30), VALUE_FONT_SIZE, 0,
-                    rl.Color(COLOR_TEXT.r, COLOR_TEXT.g, COLOR_TEXT.b, int(COLOR_TEXT.a * alpha)))

openpilot/selfdrive/ui/mici/onroad/augmented_road_view.py

@@ -5,13 +5,13 @@ from opendbc.car.structs import car
 from openpilot.cereal.visionipc import VisionStreamType
 from openpilot.selfdrive.ui.ui_state import ui_state, UIStatus
 from openpilot.selfdrive.ui.mici.onroad import SIDE_PANEL_WIDTH
-from openpilot.selfdrive.ui.mici.onroad.accel_boost_indicator import AccelBoostIndicator
 from openpilot.selfdrive.ui.mici.onroad.alert_renderer import AlertRenderer
 from openpilot.selfdrive.ui.mici.onroad.driver_state import DriverStateRenderer
 from openpilot.selfdrive.ui.mici.onroad.hud_renderer import HudRenderer
 from openpilot.selfdrive.ui.mici.onroad.long_indicator import LongIndicator
 from openpilot.selfdrive.ui.mici.onroad.model_renderer import ModelRenderer
 from openpilot.selfdrive.ui.mici.onroad.navigation import NavigationPanel, PANEL_WIDTH
+from openpilot.selfdrive.ui.mici.onroad.confidence_ball import ConfidenceBall
 from openpilot.selfdrive.ui.mici.onroad.cameraview import CameraView
 from openpilot.system.ui.lib.application import FontWeight, gui_app, MousePos, MouseEvent, TextAlignment, TextAlignmentVertical
 from openpilot.system.ui.widgets.label import UnifiedLabel
@@ -180,7 +180,7 @@ class AugmentedRoadView(CameraView):
     self._long_indicator = LongIndicator()
     self._alert_renderer = AlertRenderer()
     self._driver_state_renderer = DriverStateRenderer()
-    self._accel_boost_indicator = AccelBoostIndicator()
+    self._confidence_ball = ConfidenceBall()
     self._offroad_label = UnifiedLabel("start the car to\nuse sunnypilot", 54, FontWeight.DISPLAY,
                                        text_color=rl.Color(255, 255, 255, int(255 * 0.9)),
                                        alignment=TextAlignment.CENTER,
@@ -206,8 +206,6 @@ class AugmentedRoadView(CameraView):
   def _handle_mouse_release(self, mouse_pos: MousePos):
     # Don't trigger click callback if bookmark was triggered
     if not self._bookmark_icon.interacting():
-      if self._model_renderer.laneful.handle_tap(mouse_pos):
-        return
       super()._handle_mouse_release(mouse_pos)
 
   def _render(self, _):
@@ -277,7 +275,7 @@ class AugmentedRoadView(CameraView):
     if show_navigation:
       self._navigation.render(rl.Rectangle(self._content_rect.x + self._content_rect.width, self.rect.y, PANEL_WIDTH, self.rect.height))
     else:
-      self._accel_boost_indicator.render(self.rect)
+      self._confidence_ball.render(self.rect)
 
     self._bookmark_icon.render(self.rect)
 

openpilot/selfdrive/ui/mici/onroad/model_renderer.py

@@ -141,7 +141,6 @@ class ModelRenderer(Widget, ModelRendererSP):
     if ui_state.status != UIStatus.DISENGAGED:
       self._draw_lane_lines()
       self._draw_path()
-      self.laneful.draw(self._rect)
 
     if render_lead_indicator:
       self._update_leads(sm)
@@ -220,7 +219,6 @@ class ModelRenderer(Widget, ModelRendererSP):
     # Update road edges using raw points
     for road_edge in self._road_edges:
       road_edge.projected_points = self._map_line_to_polygon(road_edge.raw_points, line_width_factor, 0.0, max_idx)
-    self.laneful.update(self, max_idx)
 
     # Update path using raw points
     if lead and lead.present:

openpilot/selfdrive/ui/sunnypilot/layouts/settings/cruise.py

@@ -30,11 +30,6 @@ ACC_NOLONG_DESCRIPTION = tr_noop("This feature can only be used with sunnypilot
 ACC_PCMCRUISE_DISABLED_DESCRIPTION = tr_noop("This feature is not supported on this platform due to vehicle limitations.")
 ONROAD_ONLY_DESCRIPTION = tr_noop("Start the vehicle to check vehicle compatibility.")
 
-ACCEL_BOOST_EXT_DESCRIPTION = tr_noop("Adds extra acceleration on top of the stock gas-override boost when you hold the gas pedal " +
-                                       "while sunnypilot longitudinal control is active. The extra boost follows the Following Distance setting: " +
-                                       "strongest when Aggressive, weakest when Relaxed. The stock boost remains active when this is off.")
-ACCEL_BOOST_EXT_NOLONG_DESCRIPTION = tr_noop("This feature requires sunnypilot longitudinal control to be available.")
-
 
 class CruiseLayout(Widget):
   def __init__(self):
@@ -81,11 +76,6 @@ class CruiseLayout(Widget):
       min_value=1, max_value=3, value_change_step=1,
       inline=True)
 
-    self.accel_boost_toggle = toggle_item_sp(
-      title=tr("Extra Acceleration Boost by Following Distance"),
-      description="",
-      param="FollowDistanceAccelBoost")
-
     self.sla_settings_button = simple_button_item_sp(
       button_text=lambda: tr("Speed Limit"),
       button_width=800,
@@ -99,7 +89,6 @@ class CruiseLayout(Widget):
       self.custom_acc_toggle,
       self.custom_acc_short_increment,
       self.custom_acc_long_increment,
-      self.accel_boost_toggle,
       self.sla_settings_button,
     ]
     return items
@@ -115,7 +104,6 @@ class CruiseLayout(Widget):
     self._scroller.show_event()
     self.icbm_toggle.show_description(True)
     self.custom_acc_toggle.show_description(True)
-    self.accel_boost_toggle.show_description(True)
 
   def _set_current_panel(self, panel: PanelType):
     self._current_panel = panel
@@ -160,17 +148,10 @@ class CruiseLayout(Widget):
         self.scc_v_toggle.action_item.set_enabled(False)
         self.scc_m_toggle.action_item.set_enabled(False)
 
-      self.accel_boost_toggle.action_item.set_enabled(has_long and ui_state.is_offroad())
-      boost_desc = tr(ACCEL_BOOST_EXT_DESCRIPTION) if has_long else tr(ACCEL_BOOST_EXT_NOLONG_DESCRIPTION)
-      if self.accel_boost_toggle.description != boost_desc:
-        self.accel_boost_toggle.set_description(boost_desc)
-        self.accel_boost_toggle.show_description(True)
-
     else:
       has_icbm = has_long = False
       self.icbm_toggle.action_item.set_enabled(False)
       self.icbm_toggle.set_description(tr(ONROAD_ONLY_DESCRIPTION))
-      self.accel_boost_toggle.action_item.set_enabled(False)
 
     show_custom_acc_desc = False
 

openpilot/selfdrive/ui/sunnypilot/layouts/settings/models.py

@@ -235,6 +235,8 @@ class ModelsLayout(Widget):
     progress = sum(p.progress for p in progresses) / len(progresses)
     ds = custom.ModelManagerSP.DownloadStatus
 
+    if any(p.canceling for p in progresses):
+      return {"name": name, "downloading": True, "progress": progress, "status_text": tr("canceling...")}
     if ds.failed in statuses:
       # close.png is authored black and a tint cannot lift it, hence close2
       return {"name": name, "status_text": tr("download failed"), "text_color": rl.RED, "icon": "icons/close2.png"}

openpilot/selfdrive/ui/sunnypilot/layouts/settings/vehicle/brands/toyota.py

@@ -4,8 +4,6 @@ Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
 This file is part of sunnypilot and is licensed under the MIT License.
 See the LICENSE.md file in the root directory for more details.
 """
-from opendbc.car.toyota.values import ToyotaFlags
-from opendbc.sunnypilot.car.toyota.values import ToyotaFlagsSP
 from openpilot.selfdrive.ui.sunnypilot.layouts.settings.vehicle.brands.base import BrandSettings
 from openpilot.selfdrive.ui.ui_state import ui_state
 from openpilot.system.ui.lib.application import gui_app
@@ -17,10 +15,6 @@ from openpilot.system.ui.sunnypilot.widgets.list_view import toggle_item_sp
 
 ONROAD_ONLY_DESCRIPTION = tr_noop("Start the vehicle to check vehicle compatibility.")
 SNG_HACK_UNAVAILABLE = tr_noop("sunnypilot Longitudinal Control must be available and enabled for your vehicle to use this feature.")
-BSM_UNAVAILABLE = tr_noop("Enhanced Blind Spot Monitoring requires a TSS2 Toyota/Lexus without SecOC.")
-BSM_STOCK_LONG_UNAVAILABLE = tr_noop("Enhanced Blind Spot Monitoring is not available while Enforce Factory Longitudinal Control is enabled.")
-BSM_REQUIRED = tr_noop("This vehicle does not report blind spot status over the standard CAN signals. " +
-                        "Enhanced Blind Spot Monitoring is required for blind spot detection.")
 
 DESCRIPTIONS = {
   'enforce_stock_longitudinal': tr_noop(
@@ -29,10 +23,6 @@ DESCRIPTIONS = {
   'stop_and_go_hack': tr_noop(
     'sunnypilot will allow some Toyota/Lexus cars to auto resume during stop and go traffic. ' +
     'This feature is only applicable to certain models that are able to use longitudinal control. This is an alpha feature. Use at your own risk.'
-  ),
-  'enhanced_bsm': tr_noop(
-    'sunnypilot will poll the blind spot monitor diagnostics directly from the radar. Use this on vehicles whose blind spot monitors ' +
-    'do not report blind spot status over the standard CAN signals. This is an alpha feature. Use at your own risk.'
   )
 }
 
@@ -57,18 +47,9 @@ class ToyotaSettings(BrandSettings):
       enabled=lambda: not ui_state.engaged,
     )
 
-    self.enhanced_bsm = toggle_item_sp(
-      lambda: tr("Enhanced Blind Spot Monitoring (Alpha)"),
-      description=lambda: tr(DESCRIPTIONS["enhanced_bsm"]),
-      initial_state=ui_state.params.get_bool("ToyotaEnhancedBsm"),
-      callback=self._on_enable_enhanced_bsm,
-      enabled=lambda: not ui_state.engaged,
-    )
-
     self.items = [
       self.enforce_stock_longitudinal,
       self.stop_and_go_hack,
-      self.enhanced_bsm,
     ]
 
   def _on_enable_enforce_stock_longitudinal(self, state: bool):
@@ -113,25 +94,6 @@ class ToyotaSettings(BrandSettings):
       ui_state.params.put_bool("ToyotaStopAndGoHack", False)
       ui_state.params.put_bool("OnroadCycleRequested", True)
 
-  def _on_enable_enhanced_bsm(self, state: bool):
-    if state:
-      def confirm_callback(result: int):
-        if result == DialogResult.CONFIRM:
-          ui_state.params.put_bool("ToyotaEnhancedBsm", True)
-          ui_state.params.put_bool("OnroadCycleRequested", True)
-        else:
-          self.enhanced_bsm.action_item.set_state(False)
-
-      content = (f"<h1>{self.enhanced_bsm.title}</h1><br>" +
-                 f"<p>{self.enhanced_bsm.description}</p>")
-
-      dlg = ConfirmDialog(content, tr("Enable"), rich=True, callback=confirm_callback)
-      gui_app.push_widget(dlg)
-
-    else:
-      ui_state.params.put_bool("ToyotaEnhancedBsm", False)
-      ui_state.params.put_bool("OnroadCycleRequested", True)
-
   def update_settings(self):
     if ui_state.CP is not None:
       longitudinal = ui_state.CP.openpilotLongitudinalControl
@@ -151,36 +113,9 @@ class ToyotaSettings(BrandSettings):
         self.stop_and_go_hack.set_description(new_desc)
         if show_desc:
           self.stop_and_go_hack.show_description(True)
-
-      # Enhanced BSM needs the radar diagnostic address, which is only in the TX allowlist for
-      # TSS2 cars without SecOC that are not enforcing factory longitudinal control
-      bsm_supported = bool(ui_state.CP.flags & ToyotaFlags.TSS2) and not bool(ui_state.CP.flags & ToyotaFlags.SECOC)
-      bsm_available = bsm_supported and not enforce_stock
-      self.enhanced_bsm.action_item.set_enabled(bsm_available and not ui_state.engaged)
-
-      bsm_desc = tr(DESCRIPTIONS["enhanced_bsm"])
-      if not bsm_supported:
-        self.enhanced_bsm.action_item.set_state(False)
-        bsm_desc = "<b>" + tr(BSM_UNAVAILABLE) + "</b>\n\n" + bsm_desc
-      elif enforce_stock:
-        self.enhanced_bsm.action_item.set_state(False)
-        bsm_desc = "<b>" + tr(BSM_STOCK_LONG_UNAVAILABLE) + "</b>\n\n" + bsm_desc
-      elif ui_state.CP_SP is not None and ui_state.CP_SP.flags & ToyotaFlagsSP.SP_NEED_DEBUG_BSM:
-        bsm_desc = "<b>" + tr(BSM_REQUIRED) + "</b>\n\n" + bsm_desc
-
-      if self.enhanced_bsm.description != bsm_desc:
-        self.enhanced_bsm.set_description(bsm_desc)
-        if not bsm_available:
-          self.enhanced_bsm.show_description(True)
     else:
       self.stop_and_go_hack.action_item.set_enabled(False)
       new_desc = "<b>" + tr(ONROAD_ONLY_DESCRIPTION) + "</b>\n\n" + tr(DESCRIPTIONS["stop_and_go_hack"])
       if self.stop_and_go_hack.description != new_desc:
         self.stop_and_go_hack.set_description(new_desc)
         self.stop_and_go_hack.show_description(True)
-
-      self.enhanced_bsm.action_item.set_enabled(False)
-      bsm_desc = "<b>" + tr(ONROAD_ONLY_DESCRIPTION) + "</b>\n\n" + tr(DESCRIPTIONS["enhanced_bsm"])
-      if self.enhanced_bsm.description != bsm_desc:
-        self.enhanced_bsm.set_description(bsm_desc)
-        self.enhanced_bsm.show_description(True)

openpilot/selfdrive/ui/sunnypilot/mici/layouts/laneful.py

@@ -1,21 +0,0 @@
-"""
-Copyright (c) 2021-, sunnypilot, and a number of other contributors.
-
-This file is part of sunnypilot and is licensed under the MIT License.
-See the LICENSE.md file in the root directory for more details.
-"""
-from openpilot.selfdrive.ui.mici.widgets.button import BigParamControl
-from openpilot.system.ui.widgets.scroller import NavScroller
-
-
-class LanefulLayoutMici(NavScroller):
-  def __init__(self):
-    super().__init__()
-    self._toggle = BigParamControl("laneful", "Laneful",
-                                   description="Keeps the car centered in its lane.\n" +
-                                               "Leans on the lane lines when they are clear, drives laneless when they are not.")
-    self._scroller.add_widget(self._toggle)
-
-  def show_event(self):
-    super().show_event()
-    self._toggle.refresh()

openpilot/selfdrive/ui/sunnypilot/mici/layouts/models.py

@@ -251,11 +251,12 @@ class ModelsLayoutMici(NavScroller):
       self.current_model_info.info_text.set_text(tr("download failed"))
 
     elif manager.selectedBundle and manager.selectedBundle.status == custom.ModelManagerSP.DownloadStatus.downloading:
-      self.cancel_download_btn.set_visible(True)
+      self.cancel_download_btn.set_visible(ui_state.params.get("ModelManager_DownloadRef") is not None)
       device.set_override_interactive_timeout(5)
       progress = 0.0
       count = 0
       verifying = False
+      canceling = False
       for model in manager.selectedBundle.models:
         count += 1
         p = model.artifact.downloadProgress
@@ -263,11 +264,13 @@ class ModelsLayoutMici(NavScroller):
                         custom.ModelManagerSP.DownloadStatus.verifying):
           progress += p.progress
           verifying = verifying or p.status == custom.ModelManagerSP.DownloadStatus.verifying
+          canceling = canceling or p.canceling
         elif p.status in (custom.ModelManagerSP.DownloadStatus.downloaded,
                           custom.ModelManagerSP.DownloadStatus.cached):
           progress += 100.0
 
-      self.current_model_info.current_model_header.set_text(tr("verifying") if verifying else tr("downloading"))
+      header = tr("canceling...") if canceling else tr("verifying") if verifying else tr("downloading")
+      self.current_model_info.current_model_header.set_text(header)
       self.cancel_download_btn.set_text(tr("cancel verification") if verifying else tr("cancel download"))
       self.current_model_info.current_model_header._shimmer = True
       name_text = manager.selectedBundle.internalName.lower()

openpilot/selfdrive/ui/sunnypilot/mici/layouts/settings.py

@@ -11,7 +11,6 @@ from openpilot.selfdrive.ui.mici.widgets.button import BigCircleButton
 from openpilot.selfdrive.ui.mici.widgets.dialog import BigConfirmationDialog, BigDialog
 from openpilot.selfdrive.ui.sunnypilot.mici.layouts.sunnylink import SunnylinkLayoutMici
 from openpilot.selfdrive.ui.sunnypilot.mici.layouts.models import ModelsLayoutMici
-from openpilot.selfdrive.ui.sunnypilot.mici.layouts.laneful import LanefulLayoutMici
 from openpilot.selfdrive.ui.ui_state import ui_state
 from openpilot.system.ui.lib.application import gui_app, FontWeight
 from openpilot.system.ui.lib.multilang import tr
@@ -51,10 +50,6 @@ class SettingsLayoutSP(OP.SettingsLayout):
     models_btn = SettingsBigButton(tr("models"), "", gui_app.texture("../../sunnypilot/selfdrive/assets/offroad/icon_models.png", ICON_SIZE, ICON_SIZE))
     models_btn.set_click_callback(lambda: gui_app.push_widget(models_panel))
 
-    laneful_panel = LanefulLayoutMici()
-    laneful_btn = SettingsBigButton(tr("laneful"), "", gui_app.texture("icons_mici/settings/device/lkas.png", ICON_SIZE, ICON_SIZE))
-    laneful_btn.set_click_callback(lambda: gui_app.push_widget(laneful_panel))
-
     # onroad: enable button sits at the front (left of toggles)
     self._enable_offroad_btn_onroad = BigCircleButton(self.icon_offroad_enable, red=True)
     self._enable_offroad_btn_onroad.set_click_callback(lambda: self._handle_always_offroad(True))
@@ -72,8 +67,7 @@ class SettingsLayoutSP(OP.SettingsLayout):
     items = self._scroller._items.copy()
 
     items.insert(1, models_btn)
-    items.insert(2, laneful_btn)
-    items.insert(6, sunnylink_btn)
+    items.insert(5, sunnylink_btn)
 
     # front slots (only one ever visible at a time): exit-always-offroad, then enable-onroad
     items.insert(0, self._enable_offroad_btn_onroad)

openpilot/selfdrive/ui/sunnypilot/mici/onroad/laneful.py

@@ -1,117 +0,0 @@
-"""
-Copyright (c) 2021-, sunnypilot, and a number of other contributors.
-
-This file is part of sunnypilot and is licensed under the MIT License.
-See the LICENSE.md file in the root directory for more details.
-
-Laneful visuals: cyan lane-center target, thin white "after" path (model path bent by laneful's nudge),
-and a top-center readout of the pull, from the curvature modeld publishes in modelDataV2SP.lanefulCurvature.
-"""
-import numpy as np
-import pyray as rl
-from openpilot.common.filter_simple import FirstOrderFilter
-from openpilot.common.realtime import DT_MDL
-from openpilot.selfdrive.ui.ui_state import ui_state
-from openpilot.sunnypilot.modeld_v2.laneful import LOOKAHEAD_TIME, MIN_LOOKAHEAD, MAX_ERROR
-from openpilot.system.ui.lib.application import gui_app, FontWeight
-from openpilot.system.ui.lib.shader_polygon import draw_polygon
-from openpilot.system.ui.lib.text_measure import measure_text_cached
-
-CENTER_COLOR = (0, 200, 255)
-AFTER_COLOR = (255, 255, 255)
-
-
-class LanefulRenderer:
-  def __init__(self):
-    self._center_points = np.empty((0, 2), dtype=np.float32)
-    self._after_points = np.empty((0, 2), dtype=np.float32)
-    self._trust = 0.0
-    self._font = gui_app.font(FontWeight.BOLD)
-    # smoothed copies for the readout so the numbers don't flicker
-    self._pull_display = FirstOrderFilter(0.0, 0.5, DT_MDL)
-    self._trust_display = FirstOrderFilter(0.0, 0.5, DT_MDL)
-    self._alpha = FirstOrderFilter(0.0, 0.3, DT_MDL) # fades that hoe
-    self._readout_rect: rl.Rectangle | None = None
-    self._settings_page = None
-
-  def update(self, renderer, max_idx: int) -> None:
-    """Called from the model renderer's update with its lane lines and path already in raw points."""
-    laneful_curvature = ui_state.sm['modelDataV2SP'].lanefulCurvature
-    self._alpha.update(float(laneful_curvature != 0.0))
-
-    left, right = renderer._lane_lines[1].raw_points, renderer._lane_lines[2].raw_points
-    stds = list(ui_state.sm['modelV2'].laneLineStds)
-    if self._alpha.x < 0.01 or len(left) == 0 or len(right) != len(left) or len(stds) < 3:
-      self._center_points = np.empty((0, 2), dtype=np.float32)
-      self._after_points = np.empty((0, 2), dtype=np.float32)
-      return
-
-    probs = renderer._lane_line_probs
-    l_prob = float(probs[1] * np.interp(stds[1], [0.15, 0.3], [1.0, 0.0]))
-    r_prob = float(probs[2] * np.interp(stds[2], [0.15, 0.3], [1.0, 0.0]))
-    width = np.clip(right[:, 1] - left[:, 1], 2.8, 4.5) if l_prob * r_prob > 0.5 else 3.7
-    center = left.copy()
-    center[:, 1] = (l_prob * (left[:, 1] + width / 2) + r_prob * (right[:, 1] - width / 2)) / (l_prob + r_prob + 1e-4)
-    self._trust = l_prob + r_prob - l_prob * r_prob
-    self._center_points = renderer._map_line_to_polygon(center, 0.1, 0.0, max_idx)
-
-    # "after" path: model path bent by laneful's curvature, ending at the lookahead
-    path = renderer._path.raw_points
-    lookahead = max(ui_state.sm['carState'].vEgo * LOOKAHEAD_TIME, MIN_LOOKAHEAD)
-    self._pull_display.update(laneful_curvature * lookahead ** 2 / 2)  # how far laneful moves the car at the lookahead
-    self._trust_display.update(self._trust)
-    after = path.copy()
-    after[:, 1] += laneful_curvature * path[:, 0] ** 2 / 2
-    self._after_points = renderer._map_line_to_polygon(after, 0.15, renderer._path_offset_z,
-                                                       renderer._get_path_length_idx(path[:, 0], lookahead))
-
-  def _color(self, rgb, alpha: float) -> rl.Color:
-    return rl.Color(*rgb, int(alpha * self._alpha.x))
-
-  def draw(self, rect: rl.Rectangle) -> None:
-    if self._center_points.size and self._trust > 0.05:
-      draw_polygon(rect, self._center_points, self._color(CENTER_COLOR, np.clip(self._trust, 0.2, 1.0) * 255))
-    self._readout_rect = None
-    if self._after_points.size:
-      draw_polygon(rect, self._after_points, self._color(AFTER_COLOR, 200))
-      self._draw_readout(rect)
-
-  def handle_tap(self, pos) -> bool:
-    """Tapping the readout opens the laneful settings page. Returns True if the tap was consumed."""
-    r = self._readout_rect
-    if r is None or not (r.x - 20 <= pos.x <= r.x + r.width + 20 and r.y - 20 <= pos.y <= r.y + r.height + 20):
-      return False
-    if self._settings_page is None:
-      from openpilot.selfdrive.ui.sunnypilot.mici.layouts.laneful import LanefulLayoutMici
-      self._settings_page = LanefulLayoutMici()
-    gui_app.push_widget(self._settings_page)
-    return True
-
-  def _draw_readout(self, rect: rl.Rectangle) -> None:
-    """Top-center pill: pull toward lane center in cm at the lookahead, arrow = side,
-    bar grows from center toward the pull (full at the cap), brighter with more lane line trust."""
-    pull, trust = self._pull_display.x, self._trust_display.x
-    w, h = 190, 76
-    x, y = rect.x + (rect.width - w) / 2, rect.y + 10
-    cx, cy = x + w / 2, y + 30
-    self._readout_rect = rl.Rectangle(x, y, w, h)
-    rl.draw_rectangle_rounded(self._readout_rect, 0.5, 12, self._color((0, 0, 0), 170))
-
-    text = f"{abs(pull) * 100:.0f}"
-    text_w = measure_text_cached(self._font, text, 44, 0).x
-    rl.draw_text_ex(self._font, text, rl.Vector2(cx - text_w / 2, cy - 24), 44, 0, self._color((255, 255, 255), 255))
-
-    active = self._color(CENTER_COLOR, np.clip(abs(pull) / 0.1, 0.0, 1.0) * 255)
-    idle = self._color((255, 255, 255), 40)
-    lx, rx = x + 18, x + w - 18
-    rl.draw_triangle(rl.Vector2(lx, cy), rl.Vector2(lx + 16, cy + 12), rl.Vector2(lx + 16, cy - 12), active if pull < 0 else idle)
-    rl.draw_triangle(rl.Vector2(rx - 16, cy - 12), rl.Vector2(rx - 16, cy + 12), rl.Vector2(rx, cy), active if pull > 0 else idle)
-
-    bar = rl.Rectangle(x + 24, y + h - 16, w - 48, 6)
-    rl.draw_rectangle_rounded(bar, 1.0, 6, self._color((255, 255, 255), 40))
-    rl.draw_rectangle(int(cx - 1), int(bar.y - 2), 2, int(bar.height + 4), self._color((255, 255, 255), 120))
-    fill = bar.width / 2 * float(np.clip(abs(pull) / MAX_ERROR, 0.0, 1.0))
-    if fill > 1:
-      fill_x = cx if pull > 0 else cx - fill
-      rl.draw_rectangle_rounded(rl.Rectangle(fill_x, bar.y, fill, bar.height), 1.0, 6,
-                                self._color(CENTER_COLOR, np.clip(0.3 + 0.7 * trust, 0.0, 1.0) * 255))

openpilot/selfdrive/ui/sunnypilot/mici/onroad/model_renderer.py

@@ -6,7 +6,6 @@ See the LICENSE.md file in the root directory for more details.
 """
 import pyray as rl
 from openpilot.selfdrive.ui.ui_state import UIStatus
-from openpilot.selfdrive.ui.sunnypilot.mici.onroad.laneful import LanefulRenderer
 from openpilot.selfdrive.ui.sunnypilot.onroad.rainbow_path import RainbowPath
 
 LANE_LINE_COLORS_SP = {
@@ -18,4 +17,3 @@ LANE_LINE_COLORS_SP = {
 class ModelRendererSP:
   def __init__(self):
     self.rainbow_path = RainbowPath()
-    self.laneful = LanefulRenderer()

openpilot/selfdrive/ui/sunnypilot/ui_state.py

@@ -36,7 +36,7 @@ class UIStateSP:
     self.is_sp_release: bool = self.params.get_bool("IsReleaseSpBranch")
     self.sm_services_ext = [
       "modelManagerSP", "selfdriveStateSP", "longitudinalPlanSP", "backupManagerSP",
-      "lateralTorqueParameters", "carStateSP", "liveMapDataSP", "carParamsSP", "lateralDelay", "modelDataV2SP"
+      "lateralTorqueParameters", "carStateSP", "liveMapDataSP", "carParamsSP", "lateralDelay"
     ]
 
     self.sunnylink_state = SunnylinkState()

openpilot/sunnypilot/modeld_v2/convert_upstream_pkl.py

@@ -7,6 +7,7 @@ See the LICENSE.md file in the root directory for more details.
 """
 import argparse
 import dataclasses
+import inspect
 import itertools
 import mmap
 import re
@@ -45,6 +46,11 @@ def _address(v):
   return _root(v.src[0]), add + start, size
 
 
+def _buffer(u):
+  from tinygrad.uop.ops import Ops
+  return u.buffer if u.op is Ops.BUFFER else getattr(u.arg, "buffer", None)
+
+
 def _patched(buf, topo) -> tuple[bytearray, dict]:
   from tinygrad.uop.ops import Ops
   blob, rows = bytearray(buf.max_numel() * buf.dtype.itemsize), {}
@@ -72,7 +78,7 @@ def _evaluate(u, env):
     table, index = _root(u.src[0].src[0]), _evaluate(u.src[0].src[1], env)
     return struct.unpack_from(u.dtype.fmt, table.arg, index * u.dtype.itemsize)[0] if table.op is Ops.BINARY and index is not None else None
   if u.op in GroupOp.ALU and None not in (values := [_evaluate(x, env) for x in u.src]):
-    return exec_alu(u.op, u.dtype, values, False)
+    return exec_alu(u.op, u.dtype, values, *([False] if "truncate_output" in inspect.signature(exec_alu).parameters else []))
   return None
 
 
@@ -112,7 +118,7 @@ def convert(jit, kwargs: dict, frame: dict, target) -> tuple[dict, memoryview]:
   from tinygrad.uop.ops import Ops
   submit = jit.captured._linear.src[0].without_after
   kernels, topo = submit.arg.aux.kernels, list(jit.captured._linear.toposort(enter_calls=True))
-  allocs = {str(u.tag): u for u in topo if u.op is Ops.ALLOC and u.tag is not None}
+  allocs = {str(name): u for u in topo if u.op is Ops.ALLOC and (name := getattr(u, "tag", None) or u.arg.name) is not None}
   (commands, patched), kernargs = _patched(allocs["cmdbuf_compute_0"], topo), allocs["kernargs_compute_0"]
   argument_words, pokes = _patched(kernargs, topo)[1], _input_pokes(submit, kernargs)
   sh = {name: getattr(amd_gpu, f"reg{name}") + amd_gpu.GC_BASE__INST0_SEG0 - pm4.PACKET3_SET_SH_REG_START
@@ -158,7 +164,7 @@ def convert(jit, kwargs: dict, frame: dict, target) -> tuple[dict, memoryview]:
         args.append({"input": pokes[slot][0]})
       elif slot in argument_words:
         base, at, nbytes = _address(argument_words[slot])
-        args.append({"buffer": buffers.setdefault(base, len(buffers)), "offset": at, "nbytes": nbytes or base.arg.buffer.nbytes, "dtype": u8})
+        args.append({"buffer": buffers.setdefault(base, len(buffers)), "offset": at, "nbytes": nbytes or base.max_numel() * base.dtype.itemsize, "dtype": u8})
     outs, ins = kernel[6]
     calls.append({"kind": "program", "name": re.sub(r"\x1b\[[0-9;]*m", "", kernel[1]), "lib": programs[image_alloc], "target": dataclasses.asdict(target),
                   "signature": [[f"data{i}", i, u8, [a.get("nbytes", 1)]] for i, a in enumerate(args)], "global_size": grid, "local_size": local,
@@ -166,12 +172,13 @@ def convert(jit, kwargs: dict, frame: dict, target) -> tuple[dict, memoryview]:
     needs_init |= {args[i]["buffer"] for i in ins if "buffer" in args[i] and args[i]["buffer"] not in written}
     written |= {args[i]["buffer"] for i in outs if "buffer" in args[i]}
 
+  assert all(_buffer(b) is not None for b, i in buffers.items() if i in needs_init), "a kernel reads a scratch before any kernel writes it"
   names = list(jit.captured.expected_names)
   inputs = [{"slot": slot, "name": names[slot] if slot < len(names) else None, "dtype": repr(u.dtype), "numel": int(u.max_numel()), "device": u.device}
             for slot, u in enumerate(_prepare_jit_inputs((), kwargs)[0])]
   header = {"input_names": names, "calls": calls, "returns": {"kind": "NoneType", "tensors": []}, "inputs": inputs,
-            "buffers": [{"nbytes": b.arg.buffer.nbytes, "device": b.arg.buffer.device, "dtype": u8}
-                        | ({"init": blob.add(b.arg.buffer.ensure_allocated().as_memoryview())} if i in needs_init else {}) for b, i in buffers.items()],
+            "buffers": [{"nbytes": bb.nbytes if (bb := _buffer(b)) else b.max_numel() * b.dtype.itemsize, "device": bb.device if bb else b.device, "dtype": u8}
+                        | ({"init": blob.add(bb.ensure_allocated().as_memoryview())} if i in needs_init else {}) for b, i in buffers.items()],
             "check": {"inputs": [[path, store(array), device] for path, array, device in frame["inputs"]],
                       "outputs": [store(array) for array in frame["outputs"]], "output_container": frame["output_container"]}}
   with sink:

openpilot/sunnypilot/modeld_v2/helpers.py

@@ -250,6 +250,12 @@ def _allocate_buffers(header: dict, blob: memoryview, device_map: dict | None, s
   return buffers
 
 
+def _fields(what, /, **values):  # a tinygrad dataclass built or replaced with the fields it has today: a dropped field is fine, a new required one is not
+  known = {field.name for field in dataclasses.fields(what)}
+  values = {name: value for name, value in values.items() if name in known}
+  return what(**values) if isinstance(what, type) else dataclasses.replace(what, **values)
+
+
 def _program_uop(call: dict, lib: bytes):
   from tinygrad.helpers import Target
   from tinygrad.uop.ops import KernelInfo, Ops, ProgramInfo, UOp
@@ -257,9 +263,9 @@ def _program_uop(call: dict, lib: bytes):
   for name, compact_slot, dtype, shape in call["signature"]:
     parameter = UOp.param(call["globals"][compact_slot], _dtype(dtype), math.prod(shape))
     image = (shape[0], shape[1]) if len(shape) == 3 and shape[2] == 4 else None
-    parameters.append(parameter.replace(arg=dataclasses.replace(parameter.arg, name=name, image=image)))
-  info = ProgramInfo(global_size=tuple(call["global_size"]), local_size=tuple(call["local_size"]), vars=(), globals=tuple(call["globals"]),
-                     outs=tuple(call["outs"]), ins=tuple(call["ins"]), target=Target(**call["target"]))
+    parameters.append(parameter.replace(arg=_fields(parameter.arg, name=name, image=image)))
+  info = _fields(ProgramInfo, global_size=tuple(call["global_size"]), local_size=tuple(call["local_size"]), globals=tuple(call["globals"]),
+                 outs=tuple(call["outs"]), ins=tuple(call["ins"]), target=_fields(Target, **call["target"]))
   return UOp(Ops.PROGRAM, src=(UOp(Ops.SINK, arg=KernelInfo(name=call["name"])), UOp(Ops.LINEAR, src=tuple(parameters)),
                                UOp(Ops.SOURCE, arg=""), UOp(Ops.BINARY, arg=lib)), arg=info)
 

openpilot/sunnypilot/modeld_v2/laneful.py

@@ -1,117 +0,0 @@
-"""
-Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
-
-This file is part of sunnypilot and is licensed under the MIT License.
-See the LICENSE.md file in the root directory for more details.
-"""
-import numpy as np
-
-from openpilot.common.filter_simple import FirstOrderFilter
-from openpilot.common.realtime import DT_MDL
-from openpilot.sunnypilot.modeld_v2.constants import ModelConstants
-
-LANE_WIDTH_DEFAULT = 3.6  # m (reseeded to model's lane width upon initialization)
-LANE_WIDTH_CLIP = (2.8, 4.5)  # m
-LOOKAHEAD_TIME = 1.5  # s
-MIN_LOOKAHEAD = 10.0  # m
-MIN_SPEED = 5.0  # m/s
-LINE_STD = (0.2, 0.6)
-POINT_STD = 0.1
-MAX_ERROR = 1.0
-YIELD = (0.2, 0.45)
-X_IDXS = np.array(ModelConstants.X_IDXS)
-
-
-class Laneful: # TODO-SP: Rename this to blendful or something similar because it is not pure laneful anymore.
-  def __init__(self):
-    self.lane_width = FirstOrderFilter(LANE_WIDTH_DEFAULT, 10.0, DT_MDL, initialized=False)
-    self.line_gone = FirstOrderFilter(np.zeros(2), 2.0, DT_MDL)
-    self.line_probs_slow = FirstOrderFilter(np.zeros(2), 3.0, DT_MDL)
-    self.line_probs = FirstOrderFilter(np.zeros(2), 0.5, DT_MDL)
-    self.usual_offset = FirstOrderFilter(0.0, 10.0, DT_MDL, initialized=False)
-    self.going_around = False
-    self.trust = 0.0
-    self.curvature = 0.0
-
-  def update(self, model_output: dict[str, np.ndarray], v_ego: float, active: bool, model_curvature: float) -> float:
-    fading = not active or v_ego < MIN_SPEED
-    if fading:
-      self.trust = 0.0
-      self.usual_offset.initialized = False
-      self.lane_width.initialized = False
-      self.going_around = False
-    target = 0.0 if fading else self._target(model_output, v_ego, model_curvature)
-    releasing = not fading and (abs(target) < abs(self.curvature) or target * self.curvature < 0)
-    step = (2.0 if releasing else 0.5) / max(v_ego, 1.0) ** 2 * DT_MDL  # jerk over 1 second
-    self.curvature = float(np.clip(target, self.curvature - step, self.curvature + step))
-    return self.curvature
-
-  def _target(self, model_output: dict[str, np.ndarray], v_ego: float, model_curvature: float) -> float:
-    if v_ego < MIN_SPEED or 'lane_lines' not in model_output:
-      return 0.0
-
-    lookahead = max(v_ego * LOOKAHEAD_TIME, MIN_LOOKAHEAD)
-    lines = model_output['lane_lines'][0, :, :, 0]
-    stds = model_output['lane_lines_stds'][0, :, :, 0]
-
-    probs = self.line_probs.update(model_output['lane_lines_prob'][0, 1::2][1:3])
-    gone = self.line_gone.update((model_output['lane_lines_prob'][0, 1::2][1:3] < 0.3).astype(float))
-    dropping = bool(np.any(probs < 0.6 * self.line_probs_slow.update(model_output['lane_lines_prob'][0, 1::2][1:3])))
-    l_prob = float(probs[0] * np.interp(np.interp(0.5 * lookahead, X_IDXS, stds[1]), LINE_STD, [1.0, 0.0]))
-    r_prob = float(probs[1] * np.interp(np.interp(0.5 * lookahead, X_IDXS, stds[2]), LINE_STD, [1.0, 0.0]))
-    # one line steadily on its own still centers at half say conf. a line that is disappearing gives control back to laneless
-    if gone[1] > 0.8 and l_prob > 0.5:
-      lane_trust = l_prob / 2
-    elif gone[0] > 0.8 and r_prob > 0.5:
-      lane_trust = r_prob / 2
-    elif dropping:
-      lane_trust = min(l_prob, r_prob)
-    else:
-      lane_trust = l_prob + r_prob - l_prob * r_prob
-    if lane_trust <= 0.0:
-      self.trust = 0.0
-      return 0.0
-
-    win = (X_IDXS >= 0.5 * lookahead) & (X_IDXS <= 1.5 * lookahead)
-    x = X_IDXS[win]
-    half_width = self.lane_width.x / 2
-    if l_prob * r_prob > 0.5:
-      width = np.interp(lookahead, X_IDXS, lines[2]) - np.interp(lookahead, X_IDXS, lines[1])
-      self.lane_width.update(float(np.clip(width, *LANE_WIDTH_CLIP)))
-      half_width = np.clip(lines[2, win] - lines[1, win], *LANE_WIDTH_CLIP) / 2
-    l_var = stds[1, win] ** 2 / max(l_prob, 1e-4)
-    r_var = stds[2, win] ** 2 / max(r_prob, 1e-4)
-    center = (r_var * (lines[1, win] + half_width) + l_var * (lines[2, win] - half_width)) / (l_var + r_var)
-    weight = 1.0 / (l_var * r_var / (l_var + r_var) + POINT_STD ** 2)
-
-    # Fit curvature to place the plant in center: e ~ dk x^2 / 2
-    plan_x = np.interp(x, model_output['plan'][0, :, 0], model_output['plan'][0, :, 1])
-    plan_share = np.interp(abs(model_curvature) * lookahead ** 2 / 2, [0.5, 2.0], [0.3, 0.5]) # blending up to 50% plan with action during sharp curvature
-    error = np.clip(center - ((1 - plan_share) * model_curvature * x ** 2 / 2 + plan_share * plan_x), -1.0, 1.0)
-    dk = 2.0 * np.sum(weight * error * x ** 2) / np.sum(weight * x ** 4)
-
-    # A model planning to end up further from its usual spot in the lane than the plant is now is likely going around something
-    hw_now = np.clip(lines[2, 0] - lines[1, 0], *LANE_WIDTH_CLIP) / 2 if l_prob * r_prob > 0.5 else self.lane_width.x / 2
-    center_now = (l_prob * (lines[1, 0] + hw_now) + r_prob * (lines[2, 0] - hw_now)) / (l_prob + r_prob)
-    plan_y = np.interp(lookahead, model_output['plan'][0, :, 0], model_output['plan'][0, :, 1])
-    offset = plan_y - np.interp(lookahead, x, center)
-    usual = self.usual_offset.x if self.usual_offset.initialized else offset
-    swerve = abs(offset - usual) - 0.25 * abs(model_curvature) * lookahead ** 2 / 2
-    away = swerve - abs(-center_now - usual)
-    lane_curvature = 2.0 * np.polyfit(x, center, 2)[0]
-    inside_curve = abs(lane_curvature) * lookahead ** 2 / 2 > 0.1 and (offset - usual) * lane_curvature > 0
-    if inside_curve:
-      self.going_around = False
-    elif away > YIELD[0]:
-      self.going_around = True
-    elif swerve < YIELD[0] / 2:
-      self.going_around = False
-    self.usual_offset.update_alpha(20.0 if self.going_around else 10.0)
-    self.usual_offset.update(offset)
-    if inside_curve:
-      lane_trust *= np.interp(abs(offset) - abs(center_now) - 0.25 * abs(model_curvature) * lookahead ** 2 / 2, (0.5, 1.0), [1.0, 0.0])
-    else:
-      lane_trust *= np.interp(swerve, (YIELD[0] / 2, YIELD[0]), [1.0, 0.0]) if self.going_around else np.interp(away, YIELD, [1.0, 0.0])
-
-    self.trust = lane_trust if lane_trust < self.trust else self.trust + (lane_trust - self.trust) * DT_MDL / (1.5 + DT_MDL)
-    return float(self.trust * dk)

openpilot/sunnypilot/modeld_v2/modeld.py

@@ -7,7 +7,6 @@ See the LICENSE.md file in the root directory for more details.
 """
 
 from collections.abc import Callable
-from functools import partial
 import gc
 import os
 os.environ['GMMU'] = '0'
@@ -40,7 +39,6 @@ from openpilot.selfdrive.modeld.modeld import ChestnutGpuState
 
 from openpilot.sunnypilot.modeld_v2.fill_model_msg import fill_model_msg, fill_pose_msg, PublishState, get_curvature_from_output
 from openpilot.sunnypilot.modeld_v2.parse_model_outputs import Parser
-from openpilot.sunnypilot.modeld_v2.laneful import Laneful
 from openpilot.sunnypilot.modeld_v2.constants import ModelConstants, Plan
 from openpilot.sunnypilot.modeld_v2.meta_helper import load_meta_constants
 from openpilot.sunnypilot.modeld_v2.camera_offset_helper import CameraOffsetHelper
@@ -230,8 +228,7 @@ class ModelState(ModelStateBase):
     return outputs
 
   def get_action_from_model(self, model_output: dict[str, np.ndarray], prev_action: log.ModelDataV2.Action,
-                            lat_action_t: float, long_action_t: float, v_ego: float,
-                            laneful: Callable[[float], float] | None = None) -> log.ModelDataV2.Action:
+                            lat_action_t: float, long_action_t: float, v_ego: float) -> log.ModelDataV2.Action:
     if 'action' not in model_output:
       plan = model_output['plan'][0]
       desired_accel = get_accel_from_plan(plan[:, Plan.VELOCITY][:, 0], plan[:, Plan.ACCELERATION][:, 0], self.constants.T_IDXS,
@@ -243,8 +240,6 @@ class ModelState(ModelStateBase):
     else:
       desired_accel = model_output['action'][0, 1]
       desired_curvature = model_output['action'][0, 0] / (max(1.0, v_ego))**2
-    if laneful is not None:
-      desired_curvature += laneful(desired_curvature)
 
     stop = v_ego < 0.3 and desired_accel < 0.1
     desired_accel = smooth_value(desired_accel, prev_action.desiredAcceleration, self.LONG_SMOOTH_SECONDS)
@@ -371,8 +366,6 @@ def main(demo=False):
   DH = DesireHelper()
   meta_constants = load_meta_constants()
   RELC = RoadEdgeLaneChangeController()
-  laneful = Laneful()
-  laneful_enabled = False
 
   while True:
     # Keep receiving frames until we are at least 1 frame ahead of previous extra frame
@@ -416,7 +409,6 @@ def main(demo=False):
       model.lat_delay = get_lat_delay(params, sm["lateralDelay"].lateralDelay)
       model.PLANPLUS_CONTROL = params.get("PlanplusControl", return_default=True)
       camera_offset_helper.set_offset(params.get("CameraOffset", return_default=True))
-      laneful_enabled = params.get_bool("Laneful")
     lat_delay = model.lat_delay + model.LAT_SMOOTH_SECONDS
     if sm.updated["extrinsicsCalibration"] and sm.seen['narrowRoadCameraState'] and sm.seen['deviceState']:
       device_from_calib_euler = np.array(sm["extrinsicsCalibration"].rpyCalib, dtype=np.float32)
@@ -490,9 +482,7 @@ def main(demo=False):
 
       if not sm['carControl'].latActive:
         prev_action.desiredCurvature = sm['controlsState'].curvature
-      laneful_active = laneful_enabled and desire == log.Desire.none and DH.lane_change_state == log.LaneChangeState.off
-      action = model.get_action_from_model(model_output, prev_action, lat_action_t, long_action_t, v_ego,
-                                           partial(laneful.update, model_output, v_ego, laneful_active))
+      action = model.get_action_from_model(model_output, prev_action, lat_action_t, long_action_t, v_ego)
       prev_action = action
       fill_model_msg(drivingdata_send, modelv2_send, model_output, action,
                      publish_state, meta_main.frame_id, meta_extra.frame_id, frame_id,
@@ -509,7 +499,6 @@ def main(demo=False):
       modelv2_send.modelV2.meta.laneChangeDirection = DH.lane_change_direction
       mdv2sp_send.valid = modelv2_send.valid
       mdv2sp_send.modelDataV2SP.laneTurnDirection = DH.lane_turn_direction
-      mdv2sp_send.modelDataV2SP.lanefulCurvature = laneful.curvature
       drivingdata_send.drivingModelData.meta.laneChangeState = DH.lane_change_state
       drivingdata_send.drivingModelData.meta.laneChangeDirection = DH.lane_change_direction
 

openpilot/sunnypilot/modeld_v2/plain_artifact_compiler.py

@@ -23,7 +23,7 @@ def record_kernel_linear():
   recorded, hcq_compile, compile_linear = [], realize.hcq_compile, jit.compile_linear
 
   def recording_compile_linear(*args, **kwargs):
-    with mock.patch.object(realize, "hcq_compile", lambda linear, *a, **k: (recorded.append(linear), hcq_compile(linear, *a, **k))[1]):
+    with mock.patch.object(realize, "hcq_compile", lambda linear, *a, **k: (linear.src and recorded.append(linear), hcq_compile(linear, *a, **k))[1]):
       return compile_linear(*args, **kwargs)
 
   with mock.patch.object(jit, "compile_linear", recording_compile_linear):
@@ -38,6 +38,18 @@ def capture_frame(jit, kwargs: dict, output_container: str | None = None):
   return {"inputs": inputs, "outputs": [tensor.numpy().copy() for tensor in outputs], "output_container": output_container}, result
 
 
+def kernel_signature(ast, elf) -> list:
+  from tinygrad.dtype import AddrSpace
+  if hasattr(ast, "kernel_params"):
+    if any(p.arg.addrspace is AddrSpace.ALU for p in ast.kernel_params):
+      raise ValueError(f"{elf.name} takes a variable, which plain artifacts don't support")
+    compact = {slot: i for i, slot in enumerate(ast.arg.globals)}
+    return [[p.arg.name, compact[p.arg.slot], repr(p.dtype), [int(x) for x in p._shape]] for p in ast.kernel_params]
+  if not all(isinstance(slot, int) for _, slot, _, _ in elf.signature):
+    raise ValueError("TinyELF.signature lost its slots and this tinygrad has no UOp.kernel_params: update kernel_signature")
+  return [[name, slot, repr(dtype), [int(x) for x in shape]] for name, slot, dtype, shape in elf.signature]
+
+
 def serialize_jit(jit, kernel_linear, check: dict | None = None) -> tuple[dict, memoryview]:
   from tinygrad.uop.ops import Ops
   sink = tempfile.TemporaryFile(dir=".")
@@ -51,6 +63,12 @@ def serialize_jit(jit, kernel_linear, check: dict | None = None) -> tuple[dict,
     if uop.op is Ops.PARAM:
       input_specs.setdefault(uop.arg.slot, {"dtype": repr(uop.dtype), "numel": int(uop.max_numel()), "device": uop.device})
       return {"input": uop.arg.slot}
+    root, offset = uop, 0
+    while root.op in (Ops.AFTER, Ops.BITCAST, Ops.SHRINK):
+      offset, root = offset + (root.src[1].arg * root.src[0].dtype.itemsize if root.op is Ops.SHRINK else 0), root.src[0]
+    if root.op is Ops.ALLOC:
+      base = bases.setdefault(id(root), (len(bases), root, root.dtype))
+      return {"buffer": base[0], "offset": offset, "nbytes": int(uop.max_numel()) * uop.dtype.itemsize, "dtype": repr(uop.dtype)}
     buffer = uop.buffer
     root = buffer._base if buffer._base is not None else buffer
     base = bases.setdefault(id(root), (len(bases), root, uop.base.dtype if buffer._base is None else uop.dtype))
@@ -63,7 +81,7 @@ def serialize_jit(jit, kernel_linear, check: dict | None = None) -> tuple[dict,
       elf, info = ast.to_elf(), ast.arg
       global_size, local_size = info.launch_dims({})
       calls.append({"kind": "program", "name": elf.name, "lib": blob.add(bytes(elf.lib)), "target": dataclasses.asdict(elf.target),
-                    "signature": [[name, slot, repr(dtype), [int(x) for x in shape]] for name, slot, dtype, shape in elf.signature],
+                    "signature": kernel_signature(ast, elf),
                     "global_size": [int(x) for x in global_size], "local_size": [int(x) for x in local_size],
                     "globals": list(info.globals), "outs": list(info.outs), "ins": list(info.ins), "args": arguments})
       outs, ins = info.outs, info.ins
@@ -77,7 +95,9 @@ def serialize_jit(jit, kernel_linear, check: dict | None = None) -> tuple[dict,
 
   ret = jit.captured.ret
   returns = {"kind": type(ret).__name__, "tensors": [describe(t.uop.base) | {"shape": [int(x) for x in t.shape]} for t in flatten(ret)]}
-  buffers = [{"nbytes": base.nbytes, "device": base.device, "dtype": repr(dtype)}
+  from tinygrad.device import Buffer
+  assert not [base for index, base, _ in bases.values() if index in needs_init and not isinstance(base, Buffer)], "a kernel reads a scratch before writing it"
+  buffers = [{"nbytes": base.nbytes if isinstance(base, Buffer) else int(base.max_numel()) * dtype.itemsize, "device": base.device, "dtype": repr(dtype)}
              | ({"init": blob.add(base.as_memoryview())} if index in needs_init else {}) for index, base, dtype in bases.values()]
   names = list(jit.captured.expected_names)
   if check is not None:

openpilot/sunnypilot/modeld_v2/tests/test_convert_upstream_pkl.py

@@ -0,0 +1,121 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+import os
+import re
+import sys
+import tempfile
+import unittest
+from unittest import mock
+import numpy as np
+
+os.environ.setdefault("CHECK_OOB", "0")
+os.environ.setdefault("GMMU", "0")
+if sys.platform == "darwin" and os.environ.get("DEV", "").startswith("MOCKUSB"):  # the mock's VRAM is a memfd, which macOS lacks: an unlinked file is the same
+  from tinygrad.runtime.autogen import libc
+
+  def _memfd_create(name, flags):
+    fd, path = tempfile.mkstemp()
+    os.unlink(path)
+    return fd
+  libc.memfd_create = _memfd_create
+
+from openpilot.common.test import OpenpilotTestCase
+from openpilot.sunnypilot.modeld_v2 import convert_upstream_pkl
+from openpilot.sunnypilot.modeld_v2.convert_upstream_pkl import convert
+from openpilot.sunnypilot.modeld_v2.helpers import PlainJit, load_model
+from openpilot.sunnypilot.modeld_v2.plain_artifact_compiler import capture_frame
+
+IMG, DESIRE, STATE, OUT = (6, 8, 8), (8,), (4, 16), (1, 32)
+
+
+class TestConvertUpstreamPkl(OpenpilotTestCase):
+  @classmethod
+  def setUpClass(cls):
+    super().setUpClass()
+    if not os.environ.get("TINYGRAD_TESTS"):
+      raise unittest.SkipTest("TINYGRAD_TESTS not set. skipping converter test")
+    from tinygrad import Device, Tensor
+    assert Device.DEFAULT == "AMD" and getattr(Device["AMD"], "is_usb", False), f"{Device.DEFAULT} is not an AMD device behind the USB bridge"
+    assert (Tensor([1, 2, 3]) * 2).tolist() == [2, 4, 6]
+    cls.tmp = tempfile.TemporaryDirectory()
+    cls.rng = np.random.default_rng(0)
+    cls.weights = Tensor(cls.rng.standard_normal((16, 32)).astype(np.float32)).realize()
+    cls.bias = Tensor(cls.rng.standard_normal((1, 32)).astype(np.float32)).realize()
+    cls.input_specs = {"input_imgs": (IMG, "uint8", Device.DEFAULT), "desire": (DESIRE, "float32", Device.DEFAULT),
+                       "features": (STATE, "float32", Device.DEFAULT)}
+    cls.output_specs = {"outputs": (OUT, "float32", Device.DEFAULT), "next_features": (STATE, "float32", Device.DEFAULT)}
+
+  @classmethod
+  def tearDownClass(cls):
+    cls.tmp.cleanup()
+    super().tearDownClass()
+
+  def model(self, input_imgs, desire, features, output_buffers):
+    from tinygrad import Tensor
+    feat = features.mean(axis=0).reshape(1, 16) + input_imgs.float().mean() + desire.sum()
+    Tensor.realize(output_buffers["outputs"].assign((feat @ self.weights + self.bias).relu()),
+                   output_buffers["next_features"].assign(features[1:].cat(feat * 2)))
+
+  def inputs(self, seed: int) -> dict:
+    from examples.openpilot.helpers import allocate_inputs
+    rng = np.random.default_rng(seed)
+
+    def initialize(arrays: dict):
+      for value in arrays.values():
+        value[...] = rng.standard_normal(value.shape) if value.dtype.kind == "f" else rng.integers(0, 256, value.shape)
+    return allocate_inputs(self.input_specs, initialize) | {"output_buffers": allocate_inputs(self.output_specs)}
+
+  def jit(self):
+    from tinygrad import TinyJit
+    run = TinyJit(self.model, prune=True)
+    kwargs = self.inputs(1)
+    run(**kwargs)
+    run(**kwargs)
+    return run
+
+  def assert_matches_eager(self, plain, seed: int):
+    fresh, expected = self.inputs(seed), self.inputs(seed)
+    plain(**fresh)
+    self.model(**expected)
+    for name in self.output_specs:
+      assert np.array_equal(fresh["output_buffers"][name].numpy(), expected["output_buffers"][name].numpy()), name
+
+  def convert(self, jit) -> tuple[dict, memoryview]:
+    from tinygrad import Device
+    kwargs = self.inputs(2)
+    frame, _ = capture_frame(jit, kwargs, "output_buffers")
+    header, blob = convert(jit, kwargs, frame, Device[Device.DEFAULT].renderer.target)
+    kernels = jit.captured._linear.src[0].without_after.arg.aux.kernels
+    assert [call["name"] for call in header["calls"]] == [re.sub(r"\x1b\[[0-9;]*m", "", kernel[1]) for kernel in kernels]
+    assert all(re.fullmatch(r"\w+", call["name"]) for call in header["calls"]), "kernel names: failure point 1"
+    assert [(tuple(call["outs"]), tuple(call["ins"])) for call in header["calls"]] == [tuple(kernel[6]) for kernel in kernels], "outs/ins: failure point 1"
+    bound = {argument["input"] for call in header["calls"] for argument in call["args"] if "input" in argument}
+    assert bound == {spec["slot"] for spec in header["inputs"]}, f"input slots bound by pokes {sorted(bound)}: failure point 4"
+    assert [spec["name"] for spec in header["inputs"]][:len(self.input_specs)] == sorted(self.input_specs)
+    return header, blob
+
+  def test_converted_jit_matches_eager(self):  # the submit decoded into plain calls, replayed against its own frame
+    jit = self.jit()
+    plain = PlainJit(*self.convert(jit))
+    self.assert_matches_eager(plain, 3)
+    self.assert_matches_eager(plain, 4)
+
+  def test_pickle_round_trip_through_main(self):
+    from examples.openpilot.helpers import dump_pickle, load_pickle
+    from tinygrad.engine.realize import lower_and_compile
+    pkl, out = os.path.join(self.tmp.name, "tiny_tinygrad.pkl"), os.path.join(self.tmp.name, "tiny_plain.pkl")
+    dump_pickle({"run": self.jit(), "input_specs": self.input_specs, "output_specs": self.output_specs, "metadata": {"name": "tiny"}}, pkl)
+
+    artifact = load_pickle(pkl, out_of_band=True)
+    artifact["run"].captured._linear = lower_and_compile(artifact["run"].captured._linear)
+    self.assert_matches_eager(PlainJit(*self.convert(artifact["run"])), 5)
+
+    with mock.patch.object(sys, "argv", ["convert_upstream_pkl", pkl, out]):
+      convert_upstream_pkl.main()
+    bundle = load_model(out, cache=False, verify=True)
+    assert bundle["metadata"] == {"name": "tiny"} and bundle["input_specs"] == self.input_specs and bundle["output_specs"] == self.output_specs
+    self.assert_matches_eager(bundle["run"], 6)

openpilot/sunnypilot/modeld_v2/tests/test_plain_artifact.py

@@ -0,0 +1,162 @@
+"""
+Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
+
+This file is part of sunnypilot and is licensed under the MIT License.
+See the LICENSE.md file in the root directory for more details.
+"""
+import dataclasses
+import functools
+import inspect
+import os
+import sys
+import tempfile
+from unittest import mock
+import numpy as np
+import unittest
+
+import openpilot.sunnypilot.modeld_v2.helpers as helpers
+from openpilot.common.test import OpenpilotTestCase
+from openpilot.sunnypilot.modeld_v2 import compile_warp
+from openpilot.sunnypilot.modeld_v2.frame_fusion import frame_part, pack_layout
+from openpilot.sunnypilot.modeld_v2.helpers import PlainJit, cache_is_fresh, load_model, read_parts, write_bundle
+from openpilot.sunnypilot.modeld_v2.plain_artifact_compiler import capture_frame, record_kernel_linear, serialize_jit
+
+CAM_W, CAM_H, MODEL_W, MODEL_H = 32, 16, 16, 8
+FRAME = f"{CAM_W},{CAM_H},{CAM_W},{CAM_H},{CAM_H // 2},{CAM_W * CAM_H * 3 // 2}"
+FRAME_SIZE = CAM_W * CAM_H * 3 // 2
+HOST = {"desire": (8,), "traffic_convention": (2,)}
+STATE, OUT = (4, 16), (1, 32)
+
+
+def compile_jit(fn, inputs: dict, output_container: str | None = None) -> tuple[dict, memoryview]:
+  from tinygrad import TinyJit
+  run = TinyJit(fn, prune=True)
+  with record_kernel_linear() as recorded:
+    run(**inputs)
+    run(**inputs)
+    check, _ = capture_frame(run, inputs, output_container)
+  assert len(recorded) == 1, f"expected one captured kernel list, got {len(recorded)}"
+  return serialize_jit(run, recorded[0], check)
+
+
+class TestPlainArtifact(OpenpilotTestCase):
+  @classmethod
+  def setUpClass(cls):
+    super().setUpClass()
+    if not os.environ.get("TINYGRAD_TESTS"):
+      raise unittest.SkipTest("TINYGRAD_TESTS env var not set, skipping artifact test.")
+    from tinygrad import Tensor
+    cls.tmp = tempfile.TemporaryDirectory()
+    cls.rng = np.random.default_rng(0)
+    cls.weights = Tensor(cls.rng.standard_normal((16, 32)).astype(np.float32)).realize()
+    cls.bias = Tensor(cls.rng.standard_normal((1, 32)).astype(np.float32)).realize()
+
+  @classmethod
+  def tearDownClass(cls):
+    cls.tmp.cleanup()
+    super().tearDownClass()
+
+  def model(self, x, h):  # a two input, one weight model: the shape of a policy without the size
+    return ((x @ self.weights + self.bias).relu() + h.mean()).contiguous()
+
+  def tensors(self, seed: int):
+    from tinygrad import Tensor
+    rng = np.random.default_rng(seed)
+    return {"x": Tensor(rng.standard_normal((1, 16)).astype(np.float32)).realize(), "h": Tensor(rng.standard_normal((4, 16)).astype(np.float32)).realize()}
+
+  def artifact(self, name: str = "model") -> str:
+    path = os.path.join(self.tmp.name, f"{name}_tinygrad.pkl")
+    if not os.path.exists(path):
+      write_bundle(path, {"run": compile_jit(self.model, self.tensors(1))}, {"metadata": {"name": name}})
+    return path
+
+  def test_round_trip_matches_jit(self):  # compiler -> bundle -> PlainJit, replay verified, then fresh inputs bit exact with eager tinygrad
+    bundle = load_model(self.artifact(), cache=False, verify=True)
+    assert bundle["metadata"] == {"name": "model"} and isinstance(bundle["run"], PlainJit)
+    fresh = self.tensors(2)
+    assert np.array_equal(bundle["run"](**fresh).numpy(), self.model(**fresh).numpy())
+
+  def test_replay_check_catches_a_changed_kernel(self):
+    parts, extras = read_parts(self.artifact())
+    header, blob = parts["run"]
+    poisoned = dict(header, check=dict(header["check"], outputs=[header["check"]["inputs"][0][1]]))
+    with self.assertRaises(helpers.ArtifactMismatch):
+      PlainJit(poisoned, blob)
+
+  def test_cache_round_trip_holds_no_weights_and_is_rebuilt_when_stale(self):
+    path = self.artifact("cached")
+    expected = load_model(path, cache=False, verify=False)["run"](**self.tensors(3)).numpy()
+    load_model(path)
+    assert cache_is_fresh(path)
+    with open(f"{path}.cache", "rb") as file:
+      cache = file.read()
+    assert self.weights.numpy().tobytes() not in cache and len(cache) < os.path.getsize(path)
+    with mock.patch.object(helpers, "PlainJit", side_effect=AssertionError("a fresh cache must not rebuild")):
+      cached = load_model(path)
+    assert np.array_equal(cached["run"](**self.tensors(3)).numpy(), expected)
+    with mock.patch.object(helpers, "tinygrad_identity", return_value="another tinygrad"):
+      assert not cache_is_fresh(path)
+      assert isinstance(load_model(path)["run"], PlainJit)
+      assert cache_is_fresh(path)
+    assert not cache_is_fresh(path)  # keyed on the tinygrad that wrote it, so the next boot after a bump rebuilds once
+    os.utime(path)
+    with mock.patch.object(helpers, "tinygrad_identity", return_value="another tinygrad"):
+      assert not cache_is_fresh(path)
+
+  def test_fused_with_warp_matches_warp_then_model(self):
+    from tinygrad import Tensor
+    warp_path = os.path.join(self.tmp.name, "warp_tinygrad.pkl")
+    with mock.patch.object(sys, "argv", ["compile_warp", "--frame", FRAME, "--warp-to", f"{MODEL_W}x{MODEL_H}", "--output", warp_path]):
+      compile_warp.main()
+    warp = load_model(warp_path, cache=False)["run"]
+
+    def native(new_img, desire, traffic_convention, state_feat_q, output_buffers):
+      feat = state_feat_q.mean(axis=0).reshape(1, 16) + new_img.float().mean() + desire.sum() + traffic_convention[0]
+      Tensor.realize(output_buffers["outputs"].assign((feat @ self.weights + self.bias).relu()),
+                     output_buffers["next_state_feat_q"].assign(state_feat_q[1:].cat(feat * 2)))
+
+    def model_inputs(new_img):
+      return {"new_img": new_img} | {name: Tensor(np.ones(shape, np.float32) * 0.5).realize() for name, shape in HOST.items()} | \
+             {"state_feat_q": Tensor(self.rng.standard_normal(STATE).astype(np.float32)).realize(),
+              "output_buffers": {"outputs": Tensor.zeros(OUT).contiguous().realize(), "next_state_feat_q": Tensor.zeros(STATE).contiguous().realize()}}
+    frames = Tensor(self.rng.integers(0, 256, (2, FRAME_SIZE), dtype=np.uint8)).realize()
+    tfm = Tensor(np.tile(np.eye(3, dtype=np.float32), (2, 1, 1))).realize()
+    path = os.path.join(self.tmp.name, "native_tinygrad.pkl")
+    part = compile_jit(native, model_inputs(warp(input_frame=frames, M_inv=tfm).realize()), "output_buffers")
+    extras = {"input_specs": {"new_img": ((2, 6, MODEL_H // 2, MODEL_W // 2), "|u1", "CPU"), "state_feat_q": (STATE, "<f4", "CPU")}
+              | {name: (shape, "<f4", "CPU") for name, shape in HOST.items()},
+              "output_specs": {"outputs": (OUT, "<f4", "CPU"), "next_state_feat_q": (STATE, "<f4", "CPU")},
+              "metadata": {"output_shapes": {"outputs": OUT, "next_state_feat_q": STATE}}}
+    write_bundle(path, {"run": part}, extras)
+
+    fused = load_model(path, cache=False, extend=functools.partial(frame_part, warp_path))
+    assert fused["run"].input_names == ["host", "state_feat_q"]
+    offsets, packed = pack_layout(HOST, FRAME_SIZE)
+    host = np.zeros(packed, np.uint8)
+    inputs = model_inputs(None)
+    for name, shape in ({"tfm": (2, 3, 3)} | HOST).items():
+      np.ndarray(shape, np.float32, host, offsets[name])[...] = tfm.numpy() if name == "tfm" else inputs[name].numpy()
+    host[offsets["frames"]:] = frames.numpy().ravel()
+    fused["run"](host=Tensor(host, device="NPY"), state_feat_q=inputs["state_feat_q"], output_buffers=inputs["output_buffers"])
+
+    expected = model_inputs(warp(input_frame=frames, M_inv=tfm).realize()) | {k: inputs[k] for k in ("state_feat_q",)}
+    native(**expected)
+    for name in ("outputs", "next_state_feat_q"):
+      assert np.array_equal(inputs["output_buffers"][name].numpy(), expected["output_buffers"][name].numpy()), name
+
+  def test_tinygrad_api_the_loader_reads(self):
+    import tinygrad.engine.jit as jit
+    from tinygrad.device import Buffer, TinyELF
+    from tinygrad.helpers import Target
+    from tinygrad.uop.ops import ParamArg, ProgramInfo, UOp
+    fields = {cls: {f.name for f in dataclasses.fields(cls)} for cls in (ProgramInfo, ParamArg, Target)}
+    assert fields[ProgramInfo] >= {"global_size", "local_size", "globals", "outs", "ins", "target"}, "ProgramInfo changed: update helpers._program_uop"
+    assert fields[ParamArg] >= {"slot", "dtype", "name", "image"}, "ParamArg changed: update helpers._program_uop"
+    assert fields[Target] >= {"device", "renderer", "arch"}, "Target changed: update the 'target' key in plain_artifact_compiler.serialize_jit"
+    assert "signature" in getattr(TinyELF, "__annotations__", {}) or hasattr(UOp, "kernel_params"), \
+      "kernel signatures moved: update plain_artifact_compiler.serialize_jit"
+    assert callable(getattr(UOp, "param", None)) and callable(getattr(UOp, "from_buffer", None)), "UOp.param/from_buffer: update helpers.PlainJit"
+    assert callable(getattr(UOp, "store_call", None)) and callable(getattr(UOp, "call", None)), "UOp.call/store_call: update helpers.PlainJit"
+    assert callable(getattr(Buffer, "view", None)) and callable(getattr(Buffer, "ensure_allocated", None)), "Buffer.view: update helpers._upload"
+    assert callable(getattr(jit, "compile_linear", None)) and callable(getattr(jit, "_prepare_jit_inputs", None)), "engine.jit: update helpers._Jit"
+    assert "expected_names" in inspect.signature(jit.CapturedJit).parameters, "CapturedJit: update helpers.PlainJit and plain_artifact_compiler.serialize_jit"

openpilot/sunnypilot/models/manager.py

@@ -7,7 +7,10 @@ See the LICENSE.md file in the root directory for more details.
 
 import asyncio
 import os
+import re
+import threading
 import time
+from concurrent.futures import ThreadPoolExecutor
 
 import requests
 from openpilot.common.params import Params
@@ -22,12 +25,99 @@ from openpilot.sunnypilot.models.helpers import (ACTIVE_BUNDLE_KEYS, get_active_
 
 # (connect, read) seconds. read is per-request inactivity, not a total cap
 DOWNLOAD_TIMEOUT = (30, 30)
+# HuggingFace throttles each connection to ~1-2 MB/s but not the connection count; 12 saturates the device link
+MAX_CONCURRENT_PIECES = 12
+PIECE_SIZE = 8 * 1024 * 1024
+PIECE_ATTEMPTS = 3
+PIECE_BACKOFF = 1.0  # seconds, growing linearly: waits 1 s then 2 s between a piece's three attempts
+TRANSIENT_HTTP_STATUS = frozenset({408, 429, 500, 502, 503, 504})
+REPORT_INTERVAL = 0.5  # seconds between progress publications
+# byte ranges count encoded bytes but iter_content yields decoded ones, so ask for an unencoded body
+NO_COMPRESSION = {"Accept-Encoding": "identity"}
 
 
 class DownloadCancelled(Exception):
   pass
 
 
+def _is_transient(e: BaseException) -> bool:
+  if isinstance(e, requests.HTTPError):
+    return e.response is not None and e.response.status_code in TRANSIENT_HTTP_STATUS
+  return isinstance(e, (requests.ConnectionError, requests.Timeout, requests.exceptions.ChunkedEncodingError))
+
+
+class _Progress:
+  def __init__(self):
+    self._bytes = 0
+    self._lock = threading.Lock()
+
+  def add_bytes(self, n: int) -> None:
+    with self._lock:
+      self._bytes += n
+
+  def snapshot(self) -> int:
+    with self._lock:
+      return self._bytes
+
+
+def _piece_ranges(total: int, piece_size: int) -> list[tuple[int, int]]:
+  return [(start, min(start + piece_size, total)) for start in range(0, total, piece_size)]
+
+
+def _prepare_target(path: str, total: int) -> None:
+  with open(path, "wb") as f:
+    f.truncate(total)
+
+
+def _content_range_total(response: requests.Response) -> int | None:
+  if response.status_code != 206:
+    return None
+  match = re.fullmatch(r"bytes \d+-\d+/(\d+)", response.headers.get("Content-Range", ""))
+  return int(match.group(1)) if match else None
+
+
+def _probe_size(url: str) -> tuple[int, bool]:
+  with requests.get(url, headers={**NO_COMPRESSION, "Range": "bytes=0-0"}, stream=True, timeout=DOWNLOAD_TIMEOUT) as response:
+    response.raise_for_status()
+    total = _content_range_total(response)
+    if total is not None:
+      return total, True
+    return int(response.headers.get("Content-Length", 0)), False
+
+
+def _fetch_piece(url: str, path: str, index: int, start: int, end: int | None, progress: _Progress,
+                 cancel: threading.Event, block_size: int) -> None:
+  headers = {**NO_COMPRESSION, "Range": f"bytes={start}-{end - 1}"} if end is not None else NO_COMPRESSION
+  label = f"{os.path.basename(path)} piece {index}"
+  for attempt in range(PIECE_ATTEMPTS):
+    written = 0
+    try:
+      with requests.get(url, headers=headers, stream=True, timeout=DOWNLOAD_TIMEOUT) as response:
+        response.raise_for_status()
+        if end is not None and response.status_code != 206:
+          raise ValueError(f"server ignored byte range for {label}")
+        with open(path, "r+b") as f:
+          f.seek(start)
+          for block in response.iter_content(chunk_size=block_size):
+            if cancel.is_set():
+              raise DownloadCancelled("Download cancelled")
+            f.write(block)
+            written += len(block)
+            progress.add_bytes(len(block))
+          if end is None:
+            f.truncate()  # unknown length: the body defines the file size
+          if end is not None and written != end - start:
+            raise requests.exceptions.ConnectionError(f"short read: {written} of {end - start} bytes")
+      return
+    except requests.RequestException as e:
+      progress.add_bytes(-written)  # the piece restarts from scratch
+      if not _is_transient(e) or attempt == PIECE_ATTEMPTS - 1:
+        raise
+      cloudlog.warning(f"retrying {label} after {type(e).__name__}: {e}")
+      if cancel.wait(PIECE_BACKOFF * (1 + attempt)):
+        raise DownloadCancelled("Download cancelled") from None
+
+
 class ModelManagerSP:
   """Manages model downloads and status reporting"""
 
@@ -41,7 +131,7 @@ class ModelManagerSP:
     self.source_models: dict[str, list[custom.ModelManagerSP.ModelBundle]] = {}
     self.selected_bundle: custom.ModelManagerSP.ModelBundle = None
     self.active_bundle: custom.ModelManagerSP.ModelBundle = get_active_bundle(self.params, chestnut=self.chestnut_present)
-    self._chunk_size = 128 * 1000  # 128 KB chunks
+    self._block_size = 128 * 1000  # 128 KB network read blocks
     self._download_start_times: dict[str, float] = {}  # Track start time per model
     self._download_ref: bytes | str | None = None
 
@@ -56,7 +146,6 @@ class ModelManagerSP:
     self._download_ref = None
 
   def _sync_artifact_progress(self, source_artifact) -> None:
-    """Mirror download progress to all artifacts sharing the same filename in the selected bundle."""
     if not self.selected_bundle:
       return
     for model in self.selected_bundle.models:
@@ -67,7 +156,6 @@ class ModelManagerSP:
         artifact.downloadProgress.eta = source_artifact.downloadProgress.eta
 
   def _calculate_eta(self, filename: str, progress: float) -> int:
-    """Calculate ETA based on elapsed time and current progress"""
     if filename not in self._download_start_times or progress <= 0:
       return 60  # Default ETA for new downloads
 
@@ -75,39 +163,58 @@ class ModelManagerSP:
     if elapsed_time <= 0:
       return 60
 
-    # If we're at X% after Y seconds, we can estimate total time as (Y / X) * 100
     total_estimated_time = (elapsed_time / progress) * 100
     eta = total_estimated_time - elapsed_time
-
     return max(1, int(eta))  # Return at least 1 second if download is ongoing
 
-  async def _download_file(self, url: str, path: str, model) -> None:
-    """Downloads a file with progress tracking"""
-    self._download_start_times[model.fileName] = time.monotonic()
+  def _set_progress(self, artifact, status, progress: float, eta: int = 0) -> None:
+    artifact.downloadProgress.status = status
+    artifact.downloadProgress.progress = progress
+    artifact.downloadProgress.eta = eta
+    self._sync_artifact_progress(artifact)
+    self._report_status()
 
-    with requests.get(url, stream=True, timeout=DOWNLOAD_TIMEOUT) as response:  # noqa: ASYNC210
-      response.raise_for_status()
-      total_size = int(response.headers.get("content-length", 0))
-      bytes_downloaded = 0
+  def _publish_progress(self, artifact, done_bytes: int, total: int) -> None:
+    # held at 99 until the file passes its hash check
+    progress = min(99.0, done_bytes / total * 100) if total > 0 else 0.0
+    eta = self._calculate_eta(artifact.fileName, progress)
+    self._set_progress(artifact, custom.ModelManagerSP.DownloadStatus.downloading, progress, eta)
 
-      with open(path, 'wb') as f:  # noqa: ASYNC230
-        for chunk in response.iter_content(chunk_size=self._chunk_size):  # type: bytes
-          f.write(chunk)
-          bytes_downloaded += len(chunk)
+  async def _report_until_done(self, tasks: list[asyncio.Future], artifact, progress: _Progress, total: int) -> None:
+    pending = set(tasks)
+    while pending:
+      done, pending = await asyncio.wait(pending, timeout=REPORT_INTERVAL)
+      for task in done:
+        task.result()  # re-raises a worker failure
+      if self._download_interrupted():
+        raise DownloadCancelled("Download cancelled")
+      self._publish_progress(artifact, progress.snapshot(), total)
 
-          if self._download_interrupted():
-            raise DownloadCancelled("Download cancelled")
+  async def _download_file(self, url: str, path: str, artifact) -> None:
+    self._download_start_times[artifact.fileName] = time.monotonic()
+    loop = asyncio.get_running_loop()
 
-          if total_size > 0:
-            progress = (bytes_downloaded / total_size) * 100
-            model.downloadProgress.status = custom.ModelManagerSP.DownloadStatus.downloading
-            model.downloadProgress.progress = progress
-            model.downloadProgress.eta = self._calculate_eta(model.fileName, progress)
-            self._sync_artifact_progress(model)
-            self._report_status()
+    total, ranged = await loop.run_in_executor(None, _probe_size, url)
+    pieces: list[tuple[int, int | None]] = list(_piece_ranges(total, PIECE_SIZE)) if ranged else [(0, None)]
+    await loop.run_in_executor(None, _prepare_target, path, total)
+    progress = _Progress()
+    cancel = threading.Event()
 
-    # Clean up start time after download completes
-    del self._download_start_times[model.fileName]
+    with ThreadPoolExecutor(max_workers=MAX_CONCURRENT_PIECES) as pool:
+      tasks = [loop.run_in_executor(pool, _fetch_piece, url, path, i, start, end, progress, cancel, self._block_size)
+               for i, (start, end) in enumerate(pieces)]
+      try:
+        await self._report_until_done(tasks, artifact, progress, total)
+      except BaseException as e:
+        cancel.set()
+        if isinstance(e, DownloadCancelled):
+          artifact.downloadProgress.canceling = True
+          self._set_progress(artifact, custom.ModelManagerSP.DownloadStatus.downloading, artifact.downloadProgress.progress)
+        for task in tasks:
+          task.cancel()  # drops queued pieces; running ones see the event at their next block
+        await asyncio.gather(*tasks, return_exceptions=True)
+        raise
+    del self._download_start_times[artifact.fileName]
 
   async def _process_artifact(self, artifact, destination_path: str) -> None:
     if not artifact.downloadUri.uri:
@@ -119,33 +226,30 @@ class ModelManagerSP:
     expected_hash = artifact.downloadUri.sha256
     filename = artifact.fileName
     full_path = os.path.join(destination_path, filename)
+    status = custom.ModelManagerSP.DownloadStatus
+    artifact.downloadProgress.canceling = False
 
     try:
+      self._set_progress(artifact, status.verifying, 0)
       if await verify_file(full_path, expected_hash):
-        artifact.downloadProgress.status = custom.ModelManagerSP.DownloadStatus.cached
-        artifact.downloadProgress.progress = 100
-        artifact.downloadProgress.eta = 0
-        self._sync_artifact_progress(artifact)
-        self._report_status()
+        self._set_progress(artifact, status.cached, 100)
         return
 
+      self._set_progress(artifact, status.downloading, 0)
       await self._download_file(url, full_path, artifact)
+      self._set_progress(artifact, status.verifying, 99)
       if not await verify_file(full_path, expected_hash):
         raise ValueError(f"Hash validation failed for {filename}")
 
-      artifact.downloadProgress.status = custom.ModelManagerSP.DownloadStatus.downloaded
-      artifact.downloadProgress.progress = 100
-      artifact.downloadProgress.eta = 0
-      self._sync_artifact_progress(artifact)
-      self._report_status()
+      self._set_progress(artifact, status.downloaded, 100)
 
     except DownloadCancelled:
       self._download_start_times.pop(artifact.fileName, None)
-      artifact.downloadProgress.status = custom.ModelManagerSP.DownloadStatus.failed
+      artifact.downloadProgress.status = status.failed
       artifact.downloadProgress.eta = 0
       self._sync_artifact_progress(artifact)
       if self.selected_bundle:
-        self.selected_bundle.status = custom.ModelManagerSP.DownloadStatus.failed
+        self.selected_bundle.status = status.failed
       self._report_status()
       raise
 
@@ -153,11 +257,11 @@ class ModelManagerSP:
       cloudlog.error(f"Error downloading {filename}: {str(e)}")
       if os.path.isfile(full_path):  # noqa: ASYNC240
         os.remove(full_path)
-      artifact.downloadProgress.status = custom.ModelManagerSP.DownloadStatus.failed
+      artifact.downloadProgress.status = status.failed
       artifact.downloadProgress.eta = 0
       self._sync_artifact_progress(artifact)
       if self.selected_bundle:
-        self.selected_bundle.status = custom.ModelManagerSP.DownloadStatus.failed
+        self.selected_bundle.status = status.failed
       self._report_status()
       self._download_start_times.pop(artifact.fileName, None)
       raise
@@ -272,12 +376,7 @@ class ModelManagerSP:
         rk.keep_time()
 
   def clear_model_cache(self) -> None:
-    """
-    Clears the model cache directory of all files except those in the active model bundle.
-    """
-
-    # Get list of files used by both slots' selected bundles (either may become
-    # the truly active bundle depending on hardware availability)
+    """ Clears the model cache directory of all files except those in the active model bundle."""
     active_files = []
     for source in ACTIVE_BUNDLE_KEYS:
       if selected_bundle := get_selected_bundle(self.params, source):
@@ -285,7 +384,6 @@ class ModelManagerSP:
           if model.artifact.fileName:
             active_files.append(model.artifact.fileName)
 
-    # Remove all files except active ones
     model_dir = Paths.model_root()
     try:
       for filename in os.listdir(model_dir):

openpilot/sunnypilot/selfdrive/car/interfaces.py

@@ -126,7 +126,6 @@ def initialize_params(params) -> list[dict[str, Any]]:
   keys.extend([
     "ToyotaEnforceStockLongitudinal",
     "ToyotaStopAndGoHack",
-    "ToyotaEnhancedBsm",
   ])
 
   return [{k: params.get(k, return_default=True)} for k in keys]

openpilot/sunnypilot/selfdrive/controls/lib/accel_boost_ext.py

@@ -1,67 +0,0 @@
-"""
-Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
-
-This file is part of sunnypilot and is licensed under the MIT License.
-See the LICENSE.md file in the root directory for more details.
-"""
-from openpilot.cereal import log
-from openpilot.common.params import Params
-from openpilot.common.realtime import DT_MDL
-from openpilot.selfdrive.controls.lib.accel_boost import AccelBoost, ACCEL_BOOST_MAX
-from openpilot.sunnypilot import PARAMS_UPDATE_PERIOD
-
-SCALE_RELAXED = 0.3
-SCALE_STANDARD = 0.5
-SCALE_AGGRESSIVE = 0.7
-
-
-def get_accel_boost_scale(personality) -> float:
-  if personality == log.LongitudinalPersonality.aggressive:
-    return SCALE_AGGRESSIVE
-  if personality == log.LongitudinalPersonality.relaxed:
-    return SCALE_RELAXED
-  return SCALE_STANDARD
-
-
-def get_accel_boost_max(personality) -> float:
-  """Maximum reported boost: the stock cap plus the profile-scaled extra cap."""
-  return ACCEL_BOOST_MAX * (1.0 + get_accel_boost_scale(personality))
-
-
-class AccelBoostExt:
-  def __init__(self, stock: AccelBoost | None = None):
-    self._stock = stock if stock is not None else AccelBoost()
-    self._extra = AccelBoost()
-    self.params = Params()
-    self.frame = -1
-    self.enabled = self.params.get_bool("FollowDistanceAccelBoost")
-
-    self.stock_boost = 0.0
-    self.extra_boost = 0.0
-
-  @property
-  def total_boost(self) -> float:
-    return self._stock.total_boost + self.extra_boost
-
-  def _update_params(self) -> None:
-    if self.frame % int(PARAMS_UPDATE_PERIOD / DT_MDL) == 0:
-      self.enabled = self.params.get_bool("FollowDistanceAccelBoost")
-
-  def update(self, sm, output_a_target_e2e, output_a_target_mpc, a_cruise):
-    self.frame += 1
-    self._update_params()
-
-    if self.enabled and sm['selfdriveState'].enabled:
-      scale = get_accel_boost_scale(sm['selfdriveState'].personality)
-      extra_output = self._extra.update(sm, output_a_target_e2e, output_a_target_mpc, a_cruise)
-      self.extra_boost = (extra_output - output_a_target_e2e) * scale
-    else:
-      self.extra_boost = 0.0
-      self._extra.total_boost = 0.0
-      self._extra.boost_this_override = 0.0
-      self._extra.boost_eligible = False
-
-    target = output_a_target_e2e + self.extra_boost
-    output = self._stock.update(sm, target, output_a_target_mpc, a_cruise)
-    self.stock_boost = output - target
-    return output

openpilot/sunnypilot/selfdrive/controls/lib/longitudinal_planner.py

@@ -9,7 +9,6 @@ from openpilot.cereal import messaging, custom
 from opendbc.car import structs
 from openpilot.common.constants import CV
 from openpilot.selfdrive.car.cruise import V_CRUISE_MAX
-from openpilot.sunnypilot.selfdrive.controls.lib.accel_boost_ext import AccelBoostExt
 from openpilot.sunnypilot.selfdrive.controls.lib.e2e_alerts_helper import E2EAlertsHelper
 from openpilot.sunnypilot.selfdrive.controls.lib.smart_cruise_control.smart_cruise_control import SmartCruiseControl
 from openpilot.sunnypilot.selfdrive.controls.lib.speed_limit.speed_limit_assist import SpeedLimitAssist
@@ -33,16 +32,6 @@ class LongitudinalPlannerSP:
 
     self.output_v_target = 0.
     self.output_a_target = 0.
-    self._accel_boost: AccelBoostExt | None = None
-
-  @property
-  def accel_boost(self) -> AccelBoostExt:
-    assert self._accel_boost is not None
-    return self._accel_boost
-
-  @accel_boost.setter
-  def accel_boost(self, controller) -> None:
-    self._accel_boost = AccelBoostExt(controller)
 
   def update_targets(self, sm: messaging.SubMaster, v_ego: float, a_ego: float, v_cruise: float) -> tuple[float, float]:
     CS = sm['carState']

openpilot/sunnypilot/selfdrive/controls/lib/tests/test_accel_boost_ext.py

@@ -1,236 +0,0 @@
-from types import SimpleNamespace
-from unittest import mock
-
-from openpilot.cereal import log
-from openpilot.common.realtime import DT_MDL
-from openpilot.common.test import OpenpilotTestCase
-from openpilot.selfdrive.controls.lib.accel_boost import AccelBoost, ACCEL_BOOST_MAX, ACCEL_BOOST_PER_OVERRIDE
-from openpilot.sunnypilot import PARAMS_UPDATE_PERIOD
-from openpilot.sunnypilot.selfdrive.controls.lib.accel_boost_ext import AccelBoostExt, get_accel_boost_max
-from openpilot.sunnypilot.selfdrive.controls.lib.longitudinal_planner import LongitudinalPlannerSP
-
-E2E_ACCEL = 0.5
-PLANNER_ACCEL = 2.0
-STEPS_PER_OVERRIDE = round(ACCEL_BOOST_PER_OVERRIDE / (0.025 * DT_MDL))
-EXPECTED_SCALE = {
-  log.LongitudinalPersonality.relaxed: 0.3,
-  log.LongitudinalPersonality.standard: 0.5,
-  log.LongitudinalPersonality.aggressive: 0.7,
-}
-
-
-class FakeParams:
-  def __init__(self, enabled=True):
-    self.enabled = enabled
-
-  def get_bool(self, key):
-    return self.enabled
-
-
-class FakeSubMaster:
-  def __init__(self, enabled=True, gas_pressed=False, v_ego=15.0,
-               personality=log.LongitudinalPersonality.standard):
-    self._messages = {
-      'selfdriveState': SimpleNamespace(enabled=enabled, personality=personality),
-      'carState': SimpleNamespace(gasPressed=gas_pressed, vEgo=v_ego),
-    }
-
-  def __getitem__(self, key):
-    return self._messages[key]
-
-
-def update(boost, sm, e2e=E2E_ACCEL):
-  return boost.update(sm, e2e, PLANNER_ACCEL, PLANNER_ACCEL)
-
-
-class PlannerStub(LongitudinalPlannerSP):
-  def __init__(self):
-    self._accel_boost = None
-
-
-class TestAccelBoostExt(OpenpilotTestCase):
-  def setUp(self):
-    super().setUp()
-    self.params = FakeParams()
-    patcher = mock.patch("openpilot.sunnypilot.selfdrive.controls.lib.accel_boost_ext.Params", return_value=self.params)
-    patcher.start()
-    self.addCleanup(patcher.stop)
-
-  def _boost_to_max(self, personality):
-    boost = AccelBoostExt()
-    sm = FakeSubMaster(personality=personality)
-    # Eligibility is latched on a frame without gas, then applied on the next override.
-    update(boost, sm)
-    for _ in range(10):
-      sm['carState'].gasPressed = True
-      for _ in range(STEPS_PER_OVERRIDE):
-        update(boost, sm)
-      sm['carState'].gasPressed = False
-      update(boost, sm)
-    return boost, sm
-
-  def _applied_extra(self, boost, sm):
-    update(boost, sm)
-    return boost.extra_boost
-
-  def test_off_matches_stock_boost_exactly(self):
-    self.params.enabled = False
-    sm = FakeSubMaster()
-    stock = AccelBoost()
-    ext = AccelBoostExt()
-
-    stock_out = update(stock, sm)
-    ext_out = update(ext, sm)
-    self.assertAlmostEqual(ext_out, stock_out)
-
-    for _ in range(10):
-      sm['carState'].gasPressed = True
-      for _ in range(STEPS_PER_OVERRIDE):
-        stock_out = update(stock, sm)
-        ext_out = update(ext, sm)
-        self.assertAlmostEqual(ext_out, stock_out)
-      sm['carState'].gasPressed = False
-      stock_out = update(stock, sm)
-      ext_out = update(ext, sm)
-      self.assertAlmostEqual(ext_out, stock_out)
-
-    self.assertAlmostEqual(ext.total_boost, stock.total_boost)
-    self.assertAlmostEqual(ext.stock_boost, stock_out - E2E_ACCEL)
-    self.assertEqual(ext.extra_boost, 0.0)
-    self.assertEqual(ext._extra.total_boost, 0.0)
-
-  def test_no_boost_without_gas(self):
-    ext = AccelBoostExt()
-    sm = FakeSubMaster(gas_pressed=False)
-    out = update(ext, sm)
-    self.assertEqual(ext.total_boost, 0.0)
-    self.assertEqual(ext._extra.total_boost, 0.0)
-    self.assertAlmostEqual(out, E2E_ACCEL)
-
-  def test_boost_requires_e2e_to_be_limiting(self):
-    ext = AccelBoostExt()
-    sm = FakeSubMaster(gas_pressed=False)
-    ext.update(sm, E2E_ACCEL, E2E_ACCEL, E2E_ACCEL)
-    sm['carState'].gasPressed = True
-    for _ in range(10):
-      ext.update(sm, E2E_ACCEL, E2E_ACCEL, E2E_ACCEL)
-    self.assertEqual(ext.total_boost, 0.0)
-    self.assertEqual(ext.stock_boost, 0.0)
-    self.assertEqual(ext.extra_boost, 0.0)
-    self.assertEqual(ext._extra.total_boost, 0.0)
-
-  def test_extra_layer_per_override_cap(self):
-    ext = AccelBoostExt()
-    sm = FakeSubMaster(gas_pressed=False)
-    update(ext, sm)
-    sm['carState'].gasPressed = True
-    for _ in range(1000):
-      update(ext, sm)
-
-    self.assertAlmostEqual(ext._extra.total_boost, ACCEL_BOOST_PER_OVERRIDE)
-    applied = self._applied_extra(ext, sm)
-    expected = ACCEL_BOOST_PER_OVERRIDE * EXPECTED_SCALE[log.LongitudinalPersonality.standard]
-    self.assertAlmostEqual(applied, expected)
-
-  def test_extra_scales_with_following_distance(self):
-    applied = {}
-    for personality in (log.LongitudinalPersonality.relaxed,
-                        log.LongitudinalPersonality.standard,
-                        log.LongitudinalPersonality.aggressive):
-      ext, sm = self._boost_to_max(personality)
-      sm['carState'].gasPressed = True
-      applied[personality] = self._applied_extra(ext, sm)
-
-    for personality, value in applied.items():
-      self.assertAlmostEqual(value, ACCEL_BOOST_MAX * EXPECTED_SCALE[personality])
-
-    self.assertLess(applied[log.LongitudinalPersonality.relaxed], applied[log.LongitudinalPersonality.standard])
-    self.assertLess(applied[log.LongitudinalPersonality.standard], applied[log.LongitudinalPersonality.aggressive])
-
-  def test_capnp_dynamic_enum_personality_scales(self):
-    # Real SubMaster messages expose personality as a capnp DynamicEnum; equality still matches the enum value.
-    msg = log.SelfdriveState.new_message()
-    msg.personality = log.LongitudinalPersonality.aggressive
-    ext, sm = self._boost_to_max(msg.personality)
-    sm['carState'].gasPressed = True
-    applied = self._applied_extra(ext, sm)
-    self.assertAlmostEqual(applied, ACCEL_BOOST_MAX * EXPECTED_SCALE[log.LongitudinalPersonality.aggressive])
-
-  def test_extra_feeds_into_stock_last(self):
-    ext, sm = self._boost_to_max(log.LongitudinalPersonality.standard)
-    sm['carState'].gasPressed = True
-    inputs = []
-    original_update = AccelBoost.update
-
-    def spy(boost, sm_, e2e, mpc, cruise):
-      inputs.append(e2e)
-      return original_update(boost, sm_, e2e, mpc, cruise)
-
-    with mock.patch.object(AccelBoost, "update", spy):
-      out = update(ext, sm)
-
-    # First call is our extra layer on the raw target, last call is stock on the boosted target.
-    self.assertAlmostEqual(inputs[0], E2E_ACCEL)
-    self.assertAlmostEqual(inputs[-1], E2E_ACCEL + ext.extra_boost)
-    self.assertAlmostEqual(out, inputs[-1] + ext.stock_boost)
-
-  def test_extra_is_added_on_top_of_stock_boost(self):
-    ext, sm = self._boost_to_max(log.LongitudinalPersonality.aggressive)
-    sm['carState'].gasPressed = True
-    out = update(ext, sm)
-    extra = out - E2E_ACCEL - ext.stock_boost
-    self.assertAlmostEqual(ext._stock.total_boost, ACCEL_BOOST_MAX)
-    self.assertAlmostEqual(ext.total_boost, ACCEL_BOOST_MAX + ext.extra_boost)
-    self.assertAlmostEqual(ext.stock_boost, ACCEL_BOOST_MAX)
-    self.assertAlmostEqual(ext.extra_boost, ACCEL_BOOST_MAX * EXPECTED_SCALE[log.LongitudinalPersonality.aggressive])
-    self.assertAlmostEqual(extra, ext.extra_boost)
-
-  def test_sp_planner_wraps_assigned_stock_controller(self):
-    planner = PlannerStub()
-    stock = AccelBoost()
-    planner.accel_boost = stock
-
-    self.assertIsInstance(planner.accel_boost, AccelBoostExt)
-    self.assertIs(planner.accel_boost._stock, stock)
-
-  def test_wraps_existing_stock_controller(self):
-    stock = AccelBoost()
-    sm = FakeSubMaster()
-    update(stock, sm)
-    sm['carState'].gasPressed = True
-    update(stock, sm)
-
-    ext = AccelBoostExt(stock)
-    self.assertIs(ext._stock, stock)
-    self.assertAlmostEqual(ext.total_boost, stock.total_boost)
-
-  def test_boost_bar_max_matches_profile_cap(self):
-    for personality, scale in EXPECTED_SCALE.items():
-      self.assertAlmostEqual(get_accel_boost_max(personality), ACCEL_BOOST_MAX * (1.0 + scale))
-
-  def test_total_boost_is_stock_accumulated_plus_extra_applied(self):
-    ext, sm = self._boost_to_max(log.LongitudinalPersonality.aggressive)
-    sm['carState'].gasPressed = True
-    update(ext, sm)
-    self.assertAlmostEqual(ext.total_boost, ext._stock.total_boost + ext.extra_boost)
-
-  def test_turning_toggle_off_restores_stock_only(self):
-    ext, sm = self._boost_to_max(log.LongitudinalPersonality.aggressive)
-    self.assertGreater(ext._extra.total_boost, 0.0)
-
-    self.params.enabled = False
-    sm['carState'].gasPressed = True
-    # Params are re-read every PARAMS_UPDATE_PERIOD; the extra layer must clear on the next poll.
-    for _ in range(int(PARAMS_UPDATE_PERIOD / DT_MDL) + 1):
-      out = update(ext, sm)
-
-    self.assertEqual(ext._extra.total_boost, 0.0)
-    self.assertEqual(ext.extra_boost, 0.0)
-    self.assertAlmostEqual(out, E2E_ACCEL + ext.stock_boost)
-
-  def test_relaxing_following_distance_applies_immediately(self):
-    ext, sm = self._boost_to_max(log.LongitudinalPersonality.aggressive)
-    sm['selfdriveState'].personality = log.LongitudinalPersonality.relaxed
-    sm['carState'].gasPressed = True
-    applied = self._applied_extra(ext, sm)
-    self.assertAlmostEqual(applied, ACCEL_BOOST_MAX * EXPECTED_SCALE[log.LongitudinalPersonality.relaxed])

openpilot/sunnypilot/sunnylink/capabilities.py

@@ -10,7 +10,6 @@ from openpilot.cereal import custom, messaging
 from opendbc.car.structs import car
 from opendbc.car.hyundai.values import CAR as HYUNDAI_CAR, UNSUPPORTED_LONGITUDINAL_CAR
 from opendbc.car.subaru.values import CAR as SUBARU_CAR, SubaruFlags
-from opendbc.car.toyota.values import CAR as TOYOTA_CAR, ToyotaFlags
 from opendbc.sunnypilot.car.interfaces import has_bsm
 from opendbc.sunnypilot.car.tesla.values import TeslaFlagsSP
 from openpilot.common.params import Params
@@ -44,7 +43,6 @@ CAPABILITY_FIELDS = (
   "device_type",
   "subaru_has_sng",
   "hyundai_alpha_long_available",
-  "toyota_enhanced_bsm_available",
 )
 
 CAPABILITY_LABELS: dict[str, str] = {
@@ -67,7 +65,6 @@ CAPABILITY_LABELS: dict[str, str] = {
   "device_type": "Device type",
   "subaru_has_sng": "Subaru Stop-and-Go available",
   "hyundai_alpha_long_available": "Hyundai Alpha Longitudinal available",
-  "toyota_enhanced_bsm_available": "Toyota Enhanced BSM available",
 }
 
 # Explicit defaults for non-boolean capability fields
@@ -114,16 +111,6 @@ def _resolve_brand_capabilities(caps: dict, bundle_platform: str, CP) -> None:
       caps["subaru_has_sng"] = not bool(CP.flags & (SubaruFlags.GLOBAL_GEN2 | SubaruFlags.HYBRID))
       caps["has_stop_and_go"] = caps["subaru_has_sng"]
 
-  elif brand == "toyota":
-    if bundle_platform:
-      try:
-        flags = TOYOTA_CAR[bundle_platform].config.flags
-        caps["toyota_enhanced_bsm_available"] = bool(flags & ToyotaFlags.TSS2) and not bool(flags & ToyotaFlags.SECOC)
-      except KeyError:
-        cloudlog.exception(f"capabilities: unknown toyota platform {bundle_platform!r}")
-    elif CP is not None:
-      caps["toyota_enhanced_bsm_available"] = bool(CP.flags & ToyotaFlags.TSS2) and not bool(CP.flags & ToyotaFlags.SECOC)
-
 
 def generate_capabilities(params: Params | None = None) -> dict:
   """Generate a SettingsCapabilities dict from CarParams + boolean params.

openpilot/sunnypilot/sunnylink/docs/README.md

@@ -156,7 +156,7 @@ The tables below describe the **compiled** `settings_ui.json` schema — what th
 
 **Enablement rules**: Grayed out (disabled) when rules fail. Frontend shows a contextual badge explaining why.
 
-**Capability fields** (referenced in rules): `has_longitudinal_control`, `has_icbm`, `icbm_available`, `torque_allowed`, `brand`, `pcm_cruise`, `alpha_long_available`, `steer_control_type`, `has_bsm`, `is_release`, `is_sp_release`, `is_development`, `tesla_has_vehicle_bus`, `has_stop_and_go`, `stock_longitudinal`, `toyota_enhanced_bsm_available`
+**Capability fields** (referenced in rules): `has_longitudinal_control`, `has_icbm`, `icbm_available`, `torque_allowed`, `brand`, `pcm_cruise`, `alpha_long_available`, `steer_control_type`, `has_bsm`, `is_release`, `is_sp_release`, `is_development`, `tesla_has_vehicle_bus`, `has_stop_and_go`, `stock_longitudinal`
 
 ---
 

openpilot/sunnypilot/sunnylink/settings_ui.json

@@ -620,22 +620,6 @@
                 }
               ]
             },
-            {
-              "key": "FollowDistanceAccelBoost",
-              "widget": "toggle",
-              "title": "Extra Acceleration Boost by Following Distance",
-              "description": "Adds extra acceleration on top of the stock gas-override boost. The extra boost follows the Following Distance setting, strongest when Aggressive and weakest when Relaxed. The stock boost remains active when this is off.",
-              "enablement": [
-                {
-                  "type": "offroad_only"
-                },
-                {
-                  "type": "capability",
-                  "field": "has_longitudinal_control",
-                  "equals": true
-                }
-              ]
-            },
             {
               "key": "IntelligentCruiseButtonManagement",
               "widget": "toggle",
@@ -2340,28 +2324,6 @@
               "equals": false
             }
           ]
-        },
-        {
-          "key": "ToyotaEnhancedBsm",
-          "widget": "toggle",
-          "needs_onroad_cycle": true,
-          "title": "Enhanced Blind Spot Monitoring (Alpha)",
-          "description": "sunnypilot will poll the blind spot monitor diagnostics directly from the radar. Use this on vehicles whose blind spot monitors do not report blind spot status over the standard CAN signals. This is an alpha feature. Use at your own risk.",
-          "enablement": [
-            {
-              "type": "not_engaged"
-            },
-            {
-              "type": "capability",
-              "field": "toyota_enhanced_bsm_available",
-              "equals": true
-            },
-            {
-              "type": "capability",
-              "field": "stock_longitudinal",
-              "equals": false
-            }
-          ]
         }
       ]
     }

openpilot/sunnypilot/sunnylink/settings_ui_src/pages/cruise.yaml

@@ -30,15 +30,6 @@ sections:
       label: Relaxed
     enablement:
     - $ref: '#/macros/longitudinal'
-  - key: FollowDistanceAccelBoost
-    widget: toggle
-    title: Extra Acceleration Boost by Following Distance
-    description: Adds extra acceleration on top of the stock gas-override boost. The extra boost follows
-      the Following Distance setting, strongest when Aggressive and weakest when Relaxed. The stock boost
-      remains active when this is off.
-    enablement:
-    - $ref: '#/macros/offroad'
-    - $ref: '#/macros/longitudinal'
   - key: IntelligentCruiseButtonManagement
     widget: toggle
     title: Intelligent Cruise Button Management (ICBM) (Alpha)

openpilot/sunnypilot/sunnylink/settings_ui_src/pages/vehicle.yaml

@@ -101,18 +101,3 @@ sections:
     - type: param
       key: ToyotaEnforceStockLongitudinal
       equals: false
-  - key: ToyotaEnhancedBsm
-    widget: toggle
-    needs_onroad_cycle: true
-    title: Enhanced Blind Spot Monitoring (Alpha)
-    description: sunnypilot will poll the blind spot monitor diagnostics directly from the radar. Use this on vehicles whose
-      blind spot monitors do not report blind spot status over the standard CAN signals. This is an alpha feature. Use at
-      your own risk.
-    enablement:
-    - $ref: '#/macros/not_engaged'
-    - type: capability
-      field: toyota_enhanced_bsm_available
-      equals: true
-    - type: capability
-      field: stock_longitudinal
-      equals: false

openpilot/sunnypilot/sunnylink/tests/test_capabilities.py

@@ -69,18 +69,12 @@ class TestOpaquePerBrandFlags(OpenpilotTestCase):
   def test_hyundai_alpha_long_available_field_present(self):
     assert "hyundai_alpha_long_available" in CAPABILITY_FIELDS
 
-  def test_toyota_enhanced_bsm_available_field_present(self):
-    assert "toyota_enhanced_bsm_available" in CAPABILITY_FIELDS
-
   def test_subaru_has_sng_default_false(self, caps):
     assert caps["subaru_has_sng"] is False
 
   def test_hyundai_alpha_long_available_default_false(self, caps):
     assert caps["hyundai_alpha_long_available"] is False
 
-  def test_toyota_enhanced_bsm_available_default_false(self, caps):
-    assert caps["toyota_enhanced_bsm_available"] is False
-
 
 class TestCapabilitiesShape(OpenpilotTestCase):
   def test_all_fields_present(self, caps):

openpilot/sunnypilot/sunnylink/tests/test_settings_changes.py

@@ -207,7 +207,6 @@ class TestNotEngagedReplacement(OpenpilotTestCase):
     "AlphaLongitudinalEnabled",
     "ToyotaEnforceStockLongitudinal",
     "ToyotaStopAndGoHack",
-    "ToyotaEnhancedBsm",
   ], names=["key"])
   def test_offroad_only_replaced_with_not_engaged(self, schema, key):
     """These items should use not_engaged, not offroad_only."""

openpilot/sunnypilot/sunnylink/tests/test_settings_schema.py

@@ -286,7 +286,6 @@ class TestKnownVehicleSettings(OpenpilotTestCase):
     keys = {i["key"] for i in _brand_items(schema["vehicle_settings"].get("toyota"))}
     assert "ToyotaEnforceStockLongitudinal" in keys
     assert "ToyotaStopAndGoHack" in keys
-    assert "ToyotaEnhancedBsm" in keys
 
   def test_tesla_has_coop_steering(self, schema):
     keys = {i["key"] for i in _brand_items(schema["vehicle_settings"].get("tesla"))}

openpilot/system/ui/sunnypilot/widgets/download_status.py

@@ -57,7 +57,7 @@ class DownloadStatusAction(ItemAction):
     self.icon: str | None = None
     self.icon_color: rl.Color | None = None
     self._font = gui_app.font(FontWeight.NORMAL)
-    # raw progress arrives in steps, one per 128KB chunk the manager publishes
+    # raw progress arrives in steps, one per progress report from the manager
     self._progress = FirstOrderFilter(0.0, 0.5, 1 / gui_app.target_fps)
     # integrated per frame; (t * speed) % span jumps whenever the fill width changes
     self._sweep = 0.0