* models: publish the download speed and derive the eta from it
Fills in DownloadProgress.speed (bytes per second, declared with the parallel
downloader). The reporter measures the rate between its 0.5 s reports, smooths
it, and derives the eta from the bytes remaining at that rate instead of
extrapolating elapsed time. Speed is zeroed when a download ends, fails or is
cancelled and for a cached artifact, and copied across artifacts that share a
file like the other progress fields.
The big UI's Models page shows "12.3 MB/s" next to the progress bar and the
mici models panel shows it beside the percentage. Both take the max across a
bundle's artifacts, not the sum, since artifacts sharing a file mirror the same
transfer and only one file is ever in flight.
A stall decays the smoothed speed toward 0 without ever reaching it, so below
0.05 MB/s, which would only display as "0.0 MB/s", it snaps to 0. Both UIs
then show "--.- MB/s" while downloading with no rate (before the first reading
or during a stall) instead of a stuck "0.0 MB/s" or nothing, and the speed
recovers straight to the real rate once data flows again. The snap also keeps
the eta in range: without it a stall of about 25 s drove remaining / speed past
the capnp UInt32 eta, and the capnp error failed the download outright instead
of leaving the stall to the read timeout and retries. A test holds a piece
stalled until the published speed reaches exactly 0, and errors on that
overflow without the snap.
Assisted-by: Claude Code (Claude Opus 4.8)
Assisted-by: Claude Code (Claude Opus 5.5)
* remove unneeded comments
Assisted-by: Claude Code (Claude Opus 5.5)
* models: take the speed baseline before the workers start
The reporter took its starting byte count after the piece workers were
submitted, and starting each pool thread lets earlier workers run, so
bytes landed in that gap never counted toward a speed sample. Take the
baseline in _download_file before any worker exists and pass it in.
The test runs every piece inline in submit, so the whole file lands
before the reporter starts; with the late baseline the speed stays 0.
Assisted-by: Claude Code (Claude Opus 5.5)
* models: download whole-file artifacts with parallel byte ranges
HuggingFace's Xet CAS throttles each connection to roughly 1-2 MB/s but does
not limit connection count, so the single-stream _download_file left the link
idle most of the time. Replace it with a downloader that probes the size with
a one-byte Range request, creates the file at its final size and streams each
8 MB piece straight to its offset from one of 12 worker threads. The file is
never copied or assembled and only ever occupies its own size on disk. A
server that answers the probe with 200 gets one plain stream instead.
Every request sends Accept-Encoding: identity, because byte ranges count
encoded bytes while requests hands back decoded ones, so a compressed
response would land at the wrong offsets and fail as a short read.
Workers only write bytes; a reporter coroutine on the event loop publishes
progress every 0.5 s (instead of every 128 KB), re-raises the first worker
failure and polls the cancel ref. Transient errors (connection, timeout,
truncated body, HTTP 408/429/500/502/503/504) give a piece up to three
attempts; any other HTTP error fails the download at once. The whole-file
path now also publishes "verifying" while hashing before and after the
transfer, and the download is hashed against the manifest exactly as before.
A cancel has to wait for in-flight pieces to unwind (a blocked socket read
cannot be interrupted), so the row shows "canceling..." through a new
DownloadProgress.canceling flag instead of a frozen bar, and the mici cancel
button hides once the ref is gone, as it already does on the big UI.
DownloadProgress gains speed, canceling and resuming, appended to the capnp so
older readers are unaffected. All three are declared here so the wire format
lands in one place; speed and resuming are filled in by the follow-up speed
and resume changes.
Builds on #2050, which drops chunking and publishes models as whole files;
this makes that whole-file download parallel.
Measured on a comma 3X on WiFi: ~2 MB/s single stream, 17-24 MB/s steady and
30-60 MB/s peak with 12 range connections, the count picked from an on-device
sweep. Disk is not the bottleneck (eMMC writes at 56-67 MB/s with fdatasync).
Tests drive the real downloader against a local threaded range server: piece
tiling, actual parallelism (a stalled piece does not block the others), the
no-range fallback, per-piece retry, 4xx, cancel via ModelManager_DownloadRef
and mid-piece, verify/cached/downloaded state, a probe failure moving
past verifying, and hash mismatch cleanup.
Also drops two chunk-era leftovers in the tests: a stale "no chunks" note on
manifest_bundle and a one-item URL loop in the live manifest check.
Assisted-by: Claude Code (Claude Opus 4.8)
Assisted-by: Claude Code (Claude Opus 5.5)
* remove unneeded comments
* models: trim the manager tests to the regressions they catch
Each remaining test is there because a deliberate break in manager.py,
helpers.py or fetcher.py fails it. Tests that only re-checked what another
already catches are merged into a happy-path download or a table, and
checks of constants rather than behaviour are dropped. The fake server
now starts per test, keeps its state in one dataclass, and can cut a
response short.
Against 66 deliberate breaks, every one the old tests caught still fails,
and a 404 being retried, a short read and its retry, and the display
flags choosing the big model without a GPU are now caught too.
Assisted-by: Claude Code (Claude Opus 5.5)
* Update test_manager_download.py
* Update manager.py
---------
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
- Introduced new tabs for `Smoothing and Torque ceilings`.
- Updated custom math equations for cleaner logic and added new snippets for angle smoothing, ceiling brackets, and roll compensation.
- Deleted `_hyundai_canfd_ccnc.dbc` and its import references across related files.
- Merged relevant signals and comments into `hyundai_canfd_og.dbc` for consolidated usage.
- Cleaned up obsolete imports in `hyundai_canfd.dbc`.
- Adjusted `SMOOTHING_ANGLE_VEGO_MATRIX` to refine torque smoothing at mid-range speeds.
- Removed unused deadzone logic for cleaner and more consistent angle smoothing.
- Deleted `<previouslyLoaded_Datafiles>` sections from PlotJuggler layouts to streamline configuration files.
- Added `.ipynb` files to the `pyproject.toml` ignore list for cleaner tooling.
Reimplemented an Exponential Moving Average (EMA) filter on the requested
steering angle (`apply_angle`). The model's raw high-frequency micro-corrections
at low speeds cause acoustic resonance (whine) in the EPS motor. This filter
dynamically adjusts the smoothing factor (alpha) based on vehicle speed to
silence the EPS at a crawl while maintaining zero-latency precision on the highway.
Key Behaviors & Speed Matrix:
* Deadzone: Ignores angle changes ≤ 0.1° to preserve straight-line tracking.
* 0 km/h (0 mph) -> Alpha: 0.05 (Max smoothing to eliminate stationary vibration)
* 30.6 km/h (19 mph) -> Alpha: 0.10 (Heavy smoothing for stable residential turning)
* 39.6 km/h (25 mph) -> Alpha: 0.30 (Moderate smoothing)
* 49.7 km/h (31 mph) -> Alpha: 0.60 (Light smoothing for responsive city driving)
* 80.0 km/h (50 mph) -> Alpha: 1.00 (Zero smoothing / raw signal for high-speed precision)
- Replaced `parameterized.expand` with new `parameterized` syntax for cleaner test definitions.
- Added 2026 model year to Hyundai Ioniq 5 PE in `car_list.json`.
- Introduced `StatLogSP` for sunnypilot-specific metrics.
- Integrated stats collection and submission pathways for sunnylink.
- Extended parameters and handlers to support additional metrics.
- Added gzip compression and base64 encoding for oversized payload handling.
- Adjusted torque tuning configuration to avoid reliance on torque controller for Hyundai angle steering.
- Simplified control logic by removing unnecessary checks for torque control type.
refactor: clean up code formatting and improve test structure for torque reduction gain
- Adjusted `slip_factor` in Hyundai CANFD safety modes for improved consistency and accuracy.
- Ensured proper representation of `slip_factor` output in test logs.
This reverts commit b95f8c5929.
Revert "Add baseline safety model and improve steering angle limiting logic"
This reverts commit b53cbb2e18.
Revert "Disable lateral accel/jerk params and ensure float consistency in angle limits"
This reverts commit 165d7c7b36.
- Introduced a baseline safety model (`GENESIS_GV80_2025`) for comparison.
- Enhanced steer angle limit calculation using both baseline and current limits for improved safety and precision.
- Reduced max lateral acceleration and jerk by 20% for smoother handling.
- Removed unused `get_safety_CP` function, simplifying `VehicleModel` initialization.
- Replaced conservative GENESIS_GV80_2025 model with IONIQ 5 PE parameters.
- Adjusted steering parameters (ratio, slip factor, wheelbase) for better lateral control performance.
Ensure `actuators.torque` uses its absolute value in the `calculate_angle_torque_reduction_gain` method to prevent sign-related issues during Hyundai steering angle control.
The parameter name "HkgTuningAngleIdleTorqueReductionGain" was updated to "HkgTuningAngleActiveTorqueReductionGain" across multiple files for better clarity and alignment with its functionality. This change ensures consistency in naming conventions and improves code readability.
Introduced `ANGLE_IDLE_TORQUE_REDUCTION_GAIN` to manage torque when the vehicle is stationary, ensuring smoother handling and better lane centering. Updated parsing, parameters, and UI settings to support this new idle torque parameter. Adjusted torque calculation logic and smoothing factor behavior for enhanced control flexibility.
Introduced a toggle for angle smoothing factor and renamed related parameters for clarity. Refactored backend settings to use new parameter names and expanded smoothing matrices for better tuning granularity. Updated UI elements to reflect these changes, emphasizing usability and consistency.
Rename methods and variables for clarity in torque reduction and override calculations. Adjust logic to streamline handling of steering inputs and improve maintainability.
Extract torque ramping and override functionality into dedicated methods within `LkasTorqueManager` to improve maintainability and reduce redundancy. Simplify `update` logic by delegating state-specific operations to new methods.
Encapsulate LKAS torque calculations, ramping, and override logic into the new `LkasTorqueManager` class to improve modularity and maintainability. Replace existing torque logic with calls to the manager.
Add handling for IMU lateral acceleration to refine steering angle limits in CAN FD configurations. Parse and utilize `IMU_LatAccelVal` signal for enhanced lateral control accuracy.
Introduced support for CAN FD angle steering, including updated parameters, signal parsing, and new tests. Refactored related steering logic for clarity, reducing unused code and enhancing maintainability.
Streamline steering angle calculations and fault avoidance logic by removing redundant comments and unused code. Simplified `round_angle` implementation for clarity and consistency.
Revised the determination of `steeringPressed` to account for both hands-on-wheel detection and torque overriding in CAN FD setups. Simplified fallback logic for non-CAN FD configurations for better code clarity and maintainability.
Adjusted the sensitivity and threshold values for `HOD_Dir_Status` in steering press updates, improving accuracy in detecting steering input. This change aligns with updated parameter requirements for better responsiveness.
Updated the condition to detect steering press by changing HOD_Dir_Status threshold from `> 2` to `>= 2`. This ensures the detection logic aligns correctly with expected behavior.
Introduced handling for the `HOD_FD_01_100ms` message when the CANFD angle steering flag is enabled. This ensures proper message parsing and extends compatibility for specific Hyundai vehicle configurations.
Adjusted the steering override frame window and incorporated new HOD_Dir_Status to improve hands-on detection. Added parsing for new signals in Hyundai CAN FD, enhancing steering override responsiveness and reliability.
Increased the minimum override angle cap from 0.01 to 0.1 and explicitly cast the maximum cap to a float. This change improves consistency and ensures proper handling of steering limits.
Reduced the override frame window and updated the angle cap logic to use MAX_ANGLE_RATE. These changes aim to enhance steering responsiveness and safety by fine-tuning steer angle limits.
The `smoothing_factor` parameter and related logic have been removed to simplify the steering angle smoothing approach. All references and usage of this parameter have been eliminated, relying solely on speed-based dynamic interpolation. This change streamlines the code while maintaining functionality.
Reduced `OVERRIDE_FRAME_WINDOW` and updated condition to properly respect override frame limits. This ensures smoother handling and more precise steering adjustments under certain driving scenarios.
Replaced `recently_overridden` with `frames_since_override` for better granularity and added dynamic override angle limits using interpolation. These changes enhance steering control accuracy during user overrides and improve overall code readability.
Adjusted logic for recently overridden steering to improve angle limits and torque smoothing. Removed unused or redundant code, optimizing the functionality and maintaining cleaner readability.
Extracted the steering angle smoothing logic into a standalone function `sp_smooth_angle` to enhance readability and reusability. Adjusted angle smoothing parameters and introduced a maximum vehicle speed threshold for applying smoothing. Minor updates improve maintainability and ensure consistent behavior across speed ranges.
Updated `STEER_THRESHOLD` to 350 and `NO_LONGER_OVERRIDING_THRESHOLD` to 150 for better alignment with Hyundai CAN FD steering behavior. These changes ensure improved compatibility and more accurate steering response.
Re-enables resetting `apply_angle_last` to `steering_angle` when steering is recently overridden. This ensures proper handling of steering angle limits during transitions.
Simplified torque ramp-up logic by combining conditions and adjusted `STEER_THRESHOLD` for CANFD angle steering. These changes aim to enhance control precision and maintain consistency in overrides.
Decrease the override timeout from 100 to 50 frames, ensuring quicker recognition of driver input override. This improves responsiveness and aligns with refined control behavior.
Removed restrictive rate limiting during recent user overrides to improve steering response. Adjusted logic to ensure correct handling of steering angle when lateral control is inactive or overridden.
Adjust steering behavior to account for recent user overrides, improving safety and control. Introduced a "recently_overridden" check to limit angle rates and torque adjustments when user intervention is detected.
Added logic to use the current steering angle when the steering wheel is pressed, ensuring smoother transitions during user overrides. Updated function parameters and implementation to reflect this enhancement.
Removed redundant `recently_overridden` logic and introduced a more robust approach for tracking user steering overrides. Added `NO_LONGER_OVERRIDING_THRESHOLD` and updated conditions to improve steer override handling. Adjustments ensure smoother torque transitions and more accurate steering state detection.
Replaced hardcoded `angle_min_active_torque` with `ANGLE_MIN_TORQUE` from params for better configurability and consistency. This ensures the torque clamping logic aligns with defined parameters.
Updated torque calculation logic with a new optional parameter for minimum active torque, streamlining control behavior. Deactivated and cleaned up references to HkgAngleLiveTuning, simplifying configuration and reducing runtime complexities. Updated relevant UI and parameter descriptions for clarity.
Increase the override window and refine torque ramp-up behavior to avoid conflicts during recent overrides. Updated steering driver allowance and threshold values for CANFD angle steering to improve compatibility and performance.
Added logic to track recent steering overrides and adjust LKAS torque behavior accordingly. This ensures smoother transitions when the steering is overridden and reduces potential conflicts with driver input. Updated CANFD-specific steering thresholds for enhanced compatibility.
This new layout visualizes actuator data, CAN steering messages, and car state variables. It provides multiple time-series plots to aid in debugging and analysis. Plugin configurations are also included for extended functionality.
Simplified control flag handling for angle steering, adjusted torque calculations for smoother ramp rates, and updated tuning parameters for the Hyundai Ioniq 5 PE. Minor adjustment to return value handling in lateral control functions.
Remove duplicate STEER_ANGLE_SATURATION_THRESHOLD import
Cleaned up an unnecessary duplicate import of STEER_ANGLE_SATURATION_THRESHOLD from latcontrol_angle_torque. This simplifies the module imports and prevents potential redundancy or confusion.
Refactor lateral control to combine torque and angle logic
Merged functionalities of LatControlTorque and LatControlAngle into a single LatControlAngleTorque class. Refactored code to utilize methods from both parent classes, reducing duplication and improving maintainability.
Add angle-torque hybrid lateral control for Hyundai CAN FD
Introduces `LatControlAngleTorque` to enable hybrid angle and torque-based steering for specific Hyundai models. Updates related logic in carcontroller, interface, and controlsd to accommodate this new lateral control method. Adjusts torque parameters for enhanced control in supported models.
Replaced direct access to `params` with instance variables for torque parameters to improve code clarity and maintainability. Updated smoothing factor description in angle tuning settings to include speed-related behavior. This enhances readability and prepares for further tuning adjustments.
Updated the value of HkgTuningOverridingCycles to a string for consistency with other parameters in the tuning configuration. This ensures proper handling and avoids potential issues with type mismatches.
Add overriding cycles parameter for torque adjustment
Introduced "HkgTuningOverridingCycles" for configurable user override torque ramp-down cycles. Updated relevant logic in torque control and UI settings to handle the new parameter. This improves flexibility in adjusting steering torque override behavior.
Introduce separate angle tuning controls for HKG vehicles, including smoothing factor, min torque, and max torque parameters. Refactor developer panel to integrate the new settings into a dedicated UI panel, enhancing modularity and customization capabilities.
Enhanced the description to clarify its effect on steering behavior. Included details on how the smoothing factor impacts steering smoothness using EMA, aiding user understanding.
Refactored the calculation and application of the steering angle to improve code clarity and ensure smoother transitions. Removed unused parameter update logic in `latcontrol_angle.py` and enhanced handling of driver overrides in `carcontroller.py`.
Replaced `self._params` with `self.params` to correctly access the parameter `HkgTuningAngleSmoothingFactor`. This ensures the smoothing factor is updated as intended during the control loop.
Introduced a dynamic smoothing factor using the `HkgTuningAngleSmoothingFactor` parameter. This allows more granular control over curvature smoothing based on customizable user input, enhancing driving smoothness. Added necessary logic to process and apply this parameter efficiently.
Introduced a new parameter, `HkgTuningAngleSmoothingFactor`, to apply exponential moving average (EMA) smoothing to steering angle changes, reducing sudden adjustments. Added associated UI controls, parameter persistence, and integration into Hyundai carcontroller logic for improved steering stability.
Wrap the pygame import in a try-except block to catch ImportError. This prevents the script from crashing and provides a clear message prompting the user to install pygame if it's missing.
Remove "inputs" package and update "pygame" dependency
The "inputs" package has been removed from the lockfile and dependency list, while "pygame" is now included universally without the "dev" extra marker. This change simplifies dependencies and ensures consistency across environments.
Update dependencies: replace 'inputs' with 'pygame'
Replaced the 'inputs' library with 'pygame' for joystickd dependencies in `pyproject.toml`. Additionally, removed a redundant 'pygame' entry from the general dependencies.
Ugly, I know, but soundd is unhappy with joystick
Allowing lat with mads
Invert steering input for joystick control
The steering axis input is now multiplied by -1 to reverse its direction. This ensures correct handling of the left stick's horizontal input, aligning behavior with expected control dynamics.
Refactor joystick control to use pygame for broader support
Replaced the `inputs` library with `pygame` for joystick handling, providing improved compatibility with Xbox and PlayStation controllers. Added initialization, adaptive mappings, deadzone handling, and enhanced event processing for robust joystick operation. Updated README with dependencies and usage information for Xbox controllers.
Updated the `filter_speed_matrox` values to improve curvature filtering behavior at different speeds. This change ensures better handling and stability across a wider range of driving conditions.
Introduced speed-based dynamic alpha adjustment using interpolation for smoother curvature filtering. This improves steering angle calculations by adapting filter sensitivity to vehicle speed, enhancing control performance.
Updated curvature breakpoints and torque scaling for improved control in sharp turns. Increased filter alpha for faster curvature response while maintaining system stability.
Updated curvature breakpoints in Hyundai carcontroller to improve torque scaling for curved driving. Slightly refined the filter coefficient in lateral control for smoother curvature filtering and more accurate steering adjustments.
Introduced dynamic torque scaling based on curvature for smoother and more adaptive steering control. Replaced raw curvature inputs with filtered curvature for enhanced stability and reduced noise in steering angle calculations. Removed unused speed scaling logic to simplify the lateral control flow.
This reverts commit ea1af879ba2905b076ccfe65993a9db701d689dd.
Revert "More improvement but still not quite"
This reverts commit ad95493c5c61b2ace7c459d2ebc151ddaa80040f.
Revert "Adjust low-speed scaling for lateral control angle"
This reverts commit 6f789ac1ebb66b0239b4028303573c2d7d386b39.
Revert "Refactor speed-based steering scaling logic."
This reverts commit 1d40735ab8db8d470ff3b287a6b42847beffff7d.
Updated the steering angle computation to use a clearer and more descriptive speed-scaling configuration. Replaced low-speed-specific logic with a generalized approach based on speed breakpoints and corresponding influence factors. This improves maintainability and ensures smoother steering adjustments at varying speeds.
Refined the low-speed scaling parameters by modifying speed breakpoints and factors. This improves handling at lower speeds for smoother and more predictable behavior.
<configurationdefault="false"name="Replay for controls + ui"type="Multirun"separateTabs="false"reuseTabsWithFailures="false"startOneByOne="true"markFailedProcess="true"hideSuccessProcess="false"delayTime="0.0">
<runConfigurationname="replay for controls"type="Native Application"/>
<configurationdefault="false"name="replay for controls"type="CLionNativeAppRunConfigurationType"focusToolWindowBeforeRun="true"PROGRAM_PARAMS=""$Prompt$" --block "sendcan,carState,carParams,carOutput,liveTracks,carParamsSP,carStateSP,bookmarkButton""REDIRECT_INPUT="false"ELEVATE="false"USE_EXTERNAL_CONSOLE="false"EMULATE_TERMINAL="true"WORKING_DIR="file://$ProjectFileDir$/tools/replay"PASS_PARENT_ENVS_2="true"PROJECT_NAME="openpilot-special"TARGET_NAME="replay"CONFIG_NAME="replay"version="1"RUN_PATH="replay">
Copyright (c) 2021-, Haibin Wen, sunnypilot, and a number of other contributors.
This file is part of sunnypilot and is licensed under the MIT License.
See the LICENSE.md file in the root directory for more details.
"""
importos
importre
importsys
importtempfile
importunittest
fromunittestimportmock
importnumpyasnp
os.environ.setdefault("CHECK_OOB","0")
os.environ.setdefault("GMMU","0")
ifsys.platform=="darwin"andos.environ.get("DEV","").startswith("MOCKUSB"):# the mock's VRAM is a memfd, which macOS lacks: an unlinked file is the same
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Jason Wen