* feat(params): add support for parameter metadata retrieval
- Introduced `getKeyMetadata` method for accessing metadata associated with params.
- Enhanced `getParamsAllKeysV1` to include metadata parsing and optional dynamic enum generation.
- Extended unit tests to verify metadata parsing, enum mapping, and edge cases.
* Revert "feat(params): add support for parameter metadata retrieval"
This reverts commit 865b695ff966bfab7dab4d211fb2d91f06fe92db.
* update: integrate params metadata management and unit tests
- Added `update_params_metadata.py` to manage and update parameters metadata.
- Enhanced `getParamsAllKeysV1` to include metadata for params.
- Created comprehensive tests (`test_params_metadata.py`, `test_params_sync.py`) to validate metadata integrity and params consistency.
* update: improve params metadata readability and enhance enums
- Renamed params titles for clarity and consistency.
- Added enum options and mappings to selected params for better usability.
* update: enhance params metadata with improved enum structures
- Replaced plain enum lists with detailed objects (`value`, `label`) for clarity.
- Standardized parameter options for consistency across metadata.
* update: add validation constraints to params metadata
- Introduced `min`, `max`, and `step` attributes for improved parameter range validation.
- Enhances user input handling and ensures consistency in metadata.
* lint
* more lint stuff and permissions
* does this suffice?
* more lint
* update: refine params type hinting and remove unused shebang
- Adjusted type annotation in `params_dict` for better compatibility.
- Removed unnecessary shebang from `test_params_metadata.py`.
* update: expand test coverage for params metadata validation
- Added detailed test cases to ensure metadata consistency (`options`, `constraints`, `titles`).
- Validates API response alignment with `params_metadata.json`.
* the finals
* names
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
refactor(sunnylink): defer `SunnylinkApi` initialization to function scope
- Moved `SunnylinkApi` object creation into individual functions as needed.
- Prevents unnecessary initialization when the object isn't used.
* sunnylink: add statsd process and related telemetry logging infrastructure
- Introduced `statsd_sp` process for handling Sunnylink-specific stats.
- Enhanced metrics logging with improved directory structure and data handling.
* sunnylink: re-enable and refine stat_handler for telemetry processing
- Reactivated `stat_handler` thread with improved path handling.
- Made `stat_handler` more flexible by allowing directory injection.
* statsd: fix formatting issue in telemetry string generation
- Corrected missing comma between `sunnylink_dongle_id` and `comma_dongle_id`.
* update statsd_sp process configuration for enhanced readiness logic
- Modified `statsd_sp` initialization to include `always_run` alongside `sunnylink_ready_shim`.
- Ensures robust process activation conditions.
* refactor(statsd): enhance and unify StatLogSP implementation
- Replaced custom `StatLogSP` in sunnylink with centralized implementation from `system.statsd`.
- Ensures consistent logic for StatLogSP handling across modules.
* fix
* refactor(statsd): add intercept parameter to StatLogSP for configurable logging
- Introduced optional `intercept` parameter to `StatLogSP` to manage `comma_statlog` initialization.
- Updated usage in `sunnylink` to disable interception where unnecessary.
* Dont complain
* feat(statsd): add raw metric type and SunnyPilot-specific stats collection
- Introduced `METRIC_TYPE.RAW` for base64-encoded raw data metrics.
- Added `sp_stats` thread to export SunnyPilot params as raw metrics.
- Enhanced telemetry handling with decoding and serialization updates.
* refactor(statsd): improve `sp_stats` error handling and param processing
- Enhanced exception handling for `params.get` to prevent crashes.
- Added support for nested dict values to be included in stats.
* refactor(statsd): adjust imports and minor code formatting updates
- Updated `Ratekeeper` import path for consistency with the `openpilot` module structure.
- Fixed minor formatting for improved readability.
* refactor(statsd): update typings and remove unused NoReturn annotation
- Removed unnecessary `NoReturn` typing for `stats_main` to simplify function definition.
- Adjusted `get_influxdb_line_raw` to refine typing for `value` parameter.
* cleanup
* init
* init
* slightly more
* staticmethod
* handle them all
* get them models
* log with route
* more
* car
* Revert "car"
This reverts commit fe1c90cf4d422864388e4a73d5cd25c9e585a9b3.
* handle capnp
* Revert "handle capnp"
This reverts commit c5aea6880333863838a7c258541fb7a2812f2e2b.
* 1 more time
* Revert "1 more time"
This reverts commit a364474fa580d9ef9760a8896f9db76d526abead.
* Cleaning to expose wider
* feat(interfaces, statsd): log car params to stats system
- Added `STATSLOGSP` import and logging to capture `carFingerprint` in metrics.
- Improved error handling in `get_influxdb_line_raw` for robust metric generation.
* refactor(interfaces): streamline car params logging to stats
- Simplified logging by directly converting `CP` to a dictionary.
- Removed legacy stats aggregation for clarity.
* feat(sunnylink): enable compression for stats in SunnyLink
- Added optional compression for stats payload to support large data.
- Updated `stat_handler` to handle compression and base64 encoding.
* fix(statsd): filter complex types in `get_influxdb_line_raw`
- Skips unsupported types (dict, list, bytes) to prevent formatting errors.
- Simplifies type annotation for `value` parameter.
* fix(statsd): use `json.dumps` for string conversion in `get_influxdb_line_raw`
- Ensures proper handling of special characters in values.
- Prevents potential formatting issues with raw `str()` conversion.
* refactor(interfaces, statsd): update parameter keys for stats logging
- Renamed logged keys for better clarity (`sunnypilot_params` → `sunnypilot.car_params`, `device_params`).
- Ensures consistency across data logs.
* bet
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
Reapply "sunnylink: enhanced param keys fetch with data type" (#1316)"
This reverts commit 282a8b093d.
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* refactor: enhance parameter handling and add new parameter retrieval method
- Refactored `get_param_as_byte` for better modularity and added `_to_bytes` helper function.
- Introduced `getParamsAllKeysV1` to retrieve all keys with enhanced metadata.
* refactor: update parameter handling and response structure in sunnylink
- Modified `_to_bytes` to accept `bytes` type for improved type consistency.
- Adjusted response keys in `sunnylinkd` for clarity.
* fix: update `get_param_as_byte` to use corrected method for default values
- Replaced `get_default` with `get_default_value` for accurate param retrieval.
- Ensures consistent handling of default parameter values.
* refactor: remove redundant `None` check in `sunnylinkd.py`
- Streamlined parameter iteration by eliminating unnecessary `None` validation.
- Simplifies logic for constructing `params_dict`.
* refactor: streamline `sunnylinkd` response by removing redundant `keys_v1` field
- Simplified return structure by excluding unused `keys_v1` key.
- Reduces response payload for improved efficiency.
* refactor: split `getParamsAllKeys` into distinct functions for improved clarity
- Added `getParamsAllKeysV1` to preserve original metadata-rich behavior.
- Revised `getParamsAllKeys` to return a simplified list of parameter keys.
* style: remove extraneous newline in `sunnylinkd.py`
- Ensures consistent formatting and adheres to style guidelines.
* Test
* refactor: improve parameter handling in sunnylink for robustness
- Updated `get_param_as_byte` to return `None` for nonexistent parameters.
- Enhanced param compression and encoding in `sunnylinkd`.
* refactor: centralize parameter restoration with new helper function
- Added `save_param_from_base64_encoded_string` to handle param decoding and saving.
- Updated backup manager and sunnylinkd to use the new method.
- Improved code readability and reduced duplication in parameter handling logic.
* don't bother
* clean
* Hotfix for the params stuff until I rework this properly and leverage the new data types
* Revert "Hotfix for the params stuff until I rework this properly and leverage the new data types"
This reverts commit c6031b29d92d3ff5b679ffce3ba53611bb2dba0e.
* refactor: enhance getParams function to support JSON and bytes types with optional compression
* refactor: add TODO for enhancing server support of metadata in sunnylinkd.py
* lint and clean
* refactor: update value handling in getParams to return decoded values for JSON serialization
* refactor: simplify params_dict initialization by removing type hint
* refactor: update response handling in getParams to include JSON serialization of params
* refactor: update response handling in getParams to include JSON serialization of params
* Add to dic types
* refactor: extract get_param_as_byte function for improved parameter handling and fix backup inconsistencies
* cleanup
* ensure error propagates on backup fail
* Adding capabilities to route localProxy via sunnylink
* Undo
* Thx lint
* get api token
* cert is not valid when it's an IP. Still use cert, but don't validate
Refactor sunnylinkd for better readability and SSL support
Reorganized variable handling for Sunnylink API initialization. Updated WebSocket connection to use Authorization header and added SSL configuration based on the connection type. This enhances code clarity and ensures secure connections when not on localhost.
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* Add Sunnylink integration for improved device communication
This commit introduces Sunnylink support, including modules for API interactions, device registration, logging, and uploader processes. Key changes involve adding Sunnylink-related components, such as sunnylinkd, manage_sunnylinkd, and associated utilities, along with seamless integration into process management.
* Refactor Sunnylink modules and update import paths
Standardize parameter handling in Sunnylink functions by initializing Params within functions as needed. Update imports to use fully-qualified paths for better clarity and consistency. Also, refactor logging messages for improved readability and maintainability.
* Add Sunnylink support and improve log handling
Introduced Sunnylink-specific functionality, including compression for oversized logs and platform-specific socket handling for macOS. Improved logging mechanisms, refactored log queue management, and fixed exception handling in sunnylinkd.
* Refactor and fix minor coding style inconsistencies
Remove unnecessary string concatenation, adjust spacing for better readability, and ensure cleaner code in `athenad.py` and `sunnylink.py`. Added a macOS-specific comment for TCP_KEEPALIVE configuration to improve code clarity.
* Replace platform system check with sys platform in athenad.py
To check for macOS platform, the code in athenad.py has been altered. Originally, the platform.system() function was used. However, the function has been replaced with sys.platform for a more consistent and preferable syntax. Particularly, this has been modified in the context of setting socket options.
* Apply suggestions from code review
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Simplify imports and reformat API function.
Removed unused `platform` import for cleanup in `athenad.py`. Improved readability of `api_get` in `__init__.py` by reformatting the long return statement into multiple lines.
* Adjust backoff logic and refactor API call formatting.
Introduce randomness to backoff calculation in Sunnylink API to reduce synchronization issues. Minor code refactoring improves readability in the API call logic.
* Refactor Sunnylink network check logic.
Removed hardware-based network check due to performance concerns and replaced it with a real-time device state monitoring loop. This improves efficiency and ensures accurate online status before proceeding with Sunnylink registration.
* Apply suggestions from code review
* `Refactor saveParams error handling and simplify logic`
Removed redundant try-except block wrapping the entire method for clarity. Moved error logging directly inside the loop to handle individual parameter exceptions more effectively. Simplified dictionary construction and improved error logging format.
* Add BACKUP flag to select persistent parameters
This commit introduces a new BACKUP flag and applies it to specific persistent parameters in `params.cc` and `params.h`. The BACKUP flag enhances data retention by designating parameters for inclusion in backups, ensuring crucial information is preserved across sessions.
* Simplify Sunnypilot params formatting
Removed unnecessary blank lines and adjusted the Sunnypilot comment format for better readability and consistency. No functional changes were made.
* SP: Move Sunnypilot-related code to sunnypilot/sunnylink (#504)
* Refactor and relocate sunnylink-related modules
sunnylink components have been reorganized for better modularity and clarity, with files moved under `sunnypilot/sunnylink`. Unused code was removed, and reusable utilities were separated for easier maintenance. Adjusted references across the project to reflect these changes.
* Permissions
* adding init py
* more
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
DevTekVE