* athenad and webrtcd updates
* remove feature stream services from webrtcd split
* stream encoder thread
* reduce diff
* wire webrtc to livestream camera encoder
* request livestream camera switch service
* remove camera list in favour of init camera field
* remove cors
* clean
* remove unused
* remove extra try except
* add back exception trace
* add stream road camera info to stream cameras
* fix
* clean diff
* clean diff
* add testJoystick only on body
* fix camera list
* remove reference to future service
* encode all cameras and swap in video track in webrtc
* clean
* explicitly gate bridge send
* clean leftover
* make local bodyteleop work still
* athenad and webrtcd updates
* remove feature stream services from webrtcd split
* remove cors
* clean
* remove unused
* remove extra try except
* add back exception trace
* clean diff
* clean diff
* add testJoystick only on body
* fix camera list
* remove reference to future service
* sunnylink: add statsd process and related telemetry logging infrastructure
- Introduced `statsd_sp` process for handling Sunnylink-specific stats.
- Enhanced metrics logging with improved directory structure and data handling.
* sunnylink: re-enable and refine stat_handler for telemetry processing
- Reactivated `stat_handler` thread with improved path handling.
- Made `stat_handler` more flexible by allowing directory injection.
* statsd: fix formatting issue in telemetry string generation
- Corrected missing comma between `sunnylink_dongle_id` and `comma_dongle_id`.
* update statsd_sp process configuration for enhanced readiness logic
- Modified `statsd_sp` initialization to include `always_run` alongside `sunnylink_ready_shim`.
- Ensures robust process activation conditions.
* refactor(statsd): enhance and unify StatLogSP implementation
- Replaced custom `StatLogSP` in sunnylink with centralized implementation from `system.statsd`.
- Ensures consistent logic for StatLogSP handling across modules.
* fix
* refactor(statsd): add intercept parameter to StatLogSP for configurable logging
- Introduced optional `intercept` parameter to `StatLogSP` to manage `comma_statlog` initialization.
- Updated usage in `sunnylink` to disable interception where unnecessary.
* Dont complain
* feat(statsd): add raw metric type and SunnyPilot-specific stats collection
- Introduced `METRIC_TYPE.RAW` for base64-encoded raw data metrics.
- Added `sp_stats` thread to export SunnyPilot params as raw metrics.
- Enhanced telemetry handling with decoding and serialization updates.
* refactor(statsd): improve `sp_stats` error handling and param processing
- Enhanced exception handling for `params.get` to prevent crashes.
- Added support for nested dict values to be included in stats.
* refactor(statsd): adjust imports and minor code formatting updates
- Updated `Ratekeeper` import path for consistency with the `openpilot` module structure.
- Fixed minor formatting for improved readability.
* refactor(statsd): update typings and remove unused NoReturn annotation
- Removed unnecessary `NoReturn` typing for `stats_main` to simplify function definition.
- Adjusted `get_influxdb_line_raw` to refine typing for `value` parameter.
* cleanup
* init
* init
* slightly more
* staticmethod
* handle them all
* get them models
* log with route
* more
* car
* Revert "car"
This reverts commit fe1c90cf4d.
* handle capnp
* Revert "handle capnp"
This reverts commit c5aea68803.
* 1 more time
* Revert "1 more time"
This reverts commit a364474fa5.
* Cleaning to expose wider
* feat(interfaces, statsd): log car params to stats system
- Added `STATSLOGSP` import and logging to capture `carFingerprint` in metrics.
- Improved error handling in `get_influxdb_line_raw` for robust metric generation.
* refactor(interfaces): streamline car params logging to stats
- Simplified logging by directly converting `CP` to a dictionary.
- Removed legacy stats aggregation for clarity.
* feat(sunnylink): enable compression for stats in SunnyLink
- Added optional compression for stats payload to support large data.
- Updated `stat_handler` to handle compression and base64 encoding.
* fix(statsd): filter complex types in `get_influxdb_line_raw`
- Skips unsupported types (dict, list, bytes) to prevent formatting errors.
- Simplifies type annotation for `value` parameter.
* fix(statsd): use `json.dumps` for string conversion in `get_influxdb_line_raw`
- Ensures proper handling of special characters in values.
- Prevents potential formatting issues with raw `str()` conversion.
* refactor(interfaces, statsd): update parameter keys for stats logging
- Renamed logged keys for better clarity (`sunnypilot_params` → `sunnypilot.car_params`, `device_params`).
- Ensures consistency across data logs.
* bet
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* external storage
* fix mountStorage
* fix perms
* works for now
* better
* lagless
* move to sp qt
* orderish
* fix ui crash
* cleanup
* fix format
* offroad only
* debug external storage
* dont care about delete
* just use cloudlog
* show logs if using external storage
* better text
* wipe entire drive
* allow partitionless drive to be formatted
* label while formatting
* this works
* better
* cleaner
* cleaner logs
* keep upstream happy
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
* first
* fix
* fix
* this
* real
* time
* time
* more
* more
* fix
* subclass
* fix
* fix
* test
* fix
* ruff
* revert
* default
* json
* more
* test
* test
* d
* better
* better
* test
* one
* test
* space
* ban it!
* some legitimate blunders
* even more!
* this is from commaai/one
* here too
* this too
* ignore rest
* one liner
* Revert "one liner"
This reverts commit d96a1624032aaad829a005b523e579d953cfe690.
* Adding capabilities to route localProxy via sunnylink
* Undo
* Thx lint
* get api token
* cert is not valid when it's an IP. Still use cert, but don't validate
Today, Firehose uploads and user-requested uploads from Connect are treated the same. This is not ideal behavior if the user wants to upload routes immediately for a bug report and the queue is full of uploads for Firehose. The workaround is to clear the queue and retry the upload from Connect.
This PR adds an optional `priority` to requested file uploads in `athenad`. By default, all requests are marked `Low`. However, if the caller wishes to mark their uploads as "more important," then the upload queue will prioritize those requests when uploading.
The only caveat to this PR is that we won't reorder files currently being uploaded. Most connections are quick enough to finish uploading max 4 files before polling the new high-priority items in the queue.
Closes https://github.com/commaai/openpilot/issues/34836
**Verification**
Added test case to insert upload tasks with differing priorities. Polling the queue produces items in the correct order (smallest # to largest).
PR to mark Connect uploads as high priority: https://github.com/commaai/connect/pull/557
---------
Co-authored-by: Cameron Clough <cameronjclough@gmail.com>
* switch logging from raw file format to zstd compressed files
* more zst suffix
* compress bootlog
* remove class RawFile
* Optimize ZstdFileWriter by adding input caching
* use ZSTD_compressStream2
* cleanup
* LOG_COMPRESSION_LEVEL=10
* space
* add zst suffix to LOGS_SIZE_RATE
Jason Wen