Amy Jeanes
ddd7b14ebc
* models: download whole-file artifacts with parallel byte ranges HuggingFace's Xet CAS throttles each connection to roughly 1-2 MB/s but does not limit connection count, so the single-stream _download_file left the link idle most of the time. Replace it with a downloader that probes the size with a one-byte Range request, creates the file at its final size and streams each 8 MB piece straight to its offset from one of 12 worker threads. The file is never copied or assembled and only ever occupies its own size on disk. A server that answers the probe with 200 gets one plain stream instead. Every request sends Accept-Encoding: identity, because byte ranges count encoded bytes while requests hands back decoded ones, so a compressed response would land at the wrong offsets and fail as a short read. Workers only write bytes; a reporter coroutine on the event loop publishes progress every 0.5 s (instead of every 128 KB), re-raises the first worker failure and polls the cancel ref. Transient errors (connection, timeout, truncated body, HTTP 408/429/500/502/503/504) give a piece up to three attempts; any other HTTP error fails the download at once. The whole-file path now also publishes "verifying" while hashing before and after the transfer, and the download is hashed against the manifest exactly as before. A cancel has to wait for in-flight pieces to unwind (a blocked socket read cannot be interrupted), so the row shows "canceling..." through a new DownloadProgress.canceling flag instead of a frozen bar, and the mici cancel button hides once the ref is gone, as it already does on the big UI. DownloadProgress gains speed, canceling and resuming, appended to the capnp so older readers are unaffected. All three are declared here so the wire format lands in one place; speed and resuming are filled in by the follow-up speed and resume changes. Builds on #2050, which drops chunking and publishes models as whole files; this makes that whole-file download parallel. Measured on a comma 3X on WiFi: ~2 MB/s single stream, 17-24 MB/s steady and 30-60 MB/s peak with 12 range connections, the count picked from an on-device sweep. Disk is not the bottleneck (eMMC writes at 56-67 MB/s with fdatasync). Tests drive the real downloader against a local threaded range server: piece tiling, actual parallelism (a stalled piece does not block the others), the no-range fallback, per-piece retry, 4xx, cancel via ModelManager_DownloadRef and mid-piece, verify/cached/downloaded state, a probe failure moving past verifying, and hash mismatch cleanup. Also drops two chunk-era leftovers in the tests: a stale "no chunks" note on manifest_bundle and a one-item URL loop in the live manifest check. Assisted-by: Claude Code (Claude Opus 4.8) Assisted-by: Claude Code (Claude Opus 5.5) * remove unneeded comments * models: trim the manager tests to the regressions they catch Each remaining test is there because a deliberate break in manager.py, helpers.py or fetcher.py fails it. Tests that only re-checked what another already catches are merged into a happy-path download or a table, and checks of constants rather than behaviour are dropped. The fake server now starts per test, keeps its state in one dataclass, and can cut a response short. Against 66 deliberate breaks, every one the old tests caught still fails, and a 404 being retried, a short read and its retry, and the display flags choosing the big model without a GPU are now caught too. Assisted-by: Claude Code (Claude Opus 5.5) * Update test_manager_download.py * Update manager.py --------- Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
🌞 What is sunnypilot?
sunnypilot is a fork of comma.ai's openpilot, an open source driver assistance system. sunnypilot offers the user a unique driving experience for over 300+ supported car makes and models with modified behaviors of driving assist engagements. sunnypilot complies with comma.ai's safety rules as accurately as possible.
💭 Join our Community Forum
Join the official sunnypilot community forum to stay up to date with all the latest features and be a part of shaping the future of sunnypilot!
Documentation
https://docs.sunnypilot.ai/ is your one stop shop for everything from features to installation to FAQ about the sunnypilot
🚘 Running on a dedicated device in a car
First, check out this list of items you'll need to get started.
Installation
Next, refer to the sunnypilot community forum for installation instructions, as well as a complete list of Recommended Branch Installations.
🎆 Pull Requests
We welcome both pull requests and issues on GitHub. Bug fixes are encouraged.
Pull requests should be against the most current master branch.
📊 User Data
By default, sunnypilot uploads the driving data to comma servers. You can also access your data through comma connect.
sunnypilot is open source software. The user is free to disable data collection if they wish to do so.
sunnypilot logs the road-facing camera, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs. The driver-facing camera and microphone are only logged if you explicitly opt-in in settings.
By using this software, you understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma for the use of this data.
Licensing
sunnypilot is released under the MIT License. This repository includes original work as well as significant portions of code derived from openpilot by comma.ai, which is also released under the MIT license with additional disclaimers.
The original openpilot license notice, including comma.ai’s indemnification and alpha software disclaimer, is reproduced below as required:
openpilot is released under the MIT license. Some parts of the software are released under other licenses as specified.
Any user of this software shall indemnify and hold harmless Comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys’ fees and costs) which arise out of, relate to or result from any use of this software by user.
THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED.
For full license terms, please see the LICENSE file.
💰 Support sunnypilot
If you find any of the features useful, consider becoming a sponsor on GitHub to support future feature development and improvements.
By becoming a sponsor, you will gain access to exclusive content, early access to new features, and the opportunity to directly influence the project's development.
GitHub Sponsor
PayPal
Your continuous love and support are greatly appreciated! Enjoy 🥰
- Jason, Founder of sunnypilot