mirror of
https://github.com/infiniteCable2/openpilot.git
synced 2026-08-05 16:26:10 +08:00
bb407ae9bf159e02f83ec8dbede425befb09f947
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DevTekVE
|
c392b2b269 |
modeld: legacy MLSIM driving models support (#595)
* Add buffer length parameter for enhanced frame handling
Introduce a configurable `buffer_length` parameter to `DrivingModelFrame` to support dynamic buffer sizes, enabling better handling of different frame rates like 20Hz. Updates include necessary adjustments in buffer initialization, copying logic, and related model inputs for improved compatibility and flexibility.
* Rename variable `len` to `length` to avoid shadowing built-in.
Replaced the usage of `len` with `length` across the code to prevent conflicts with Python's built-in `len` function. This improves code clarity and reduces potential errors or misunderstandings in variable usage.
* Fix spacing inconsistency in modeld.py
Added a missing newline for better code readability and consistency. This change has no impact on functionality but improves code formatting.
* Move numpy_inputs initialization to correct position
Repositioned the `numpy_inputs` initialization to align with the input shape processing logic. This ensures consistency in buffer management and clarifies the flow of code execution related to input handling.
* Add 20Hz model state, smart input, and model switcher classes
Introduce `ModelState20Hz`, `ModelSmartInput`, and `ModelSwitcher` for enhanced modularity and flexibility in modeld. Refactor `ModelState` to inherit from these new classes, enabling support for 20Hz processing and smart input initialization. Update associated files to handle the new buffer length parameter and metadata management.
* Refactor `modeld` to streamline feature handling logic
Simplified feature processing for both standard and "smart input" modes by consolidating logic into reusable methods. Updated variable naming, formatting, and spacing for consistency and readability. This refactor enhances maintainability and reduces redundancy in feature update operations.
* Silence debug print statements and use cloudlog for warnings.
Commented out a debug print statement in `commonmodel.cc` to reduce noise. Replaced `print` statements with `cloudlog.warning` in `model_smart_input.py` for improved logging consistency and better integration with the logging system.
* Clean up formatting and fix minor style inconsistencies
Removed unnecessary blank lines and adjusted spacing to standardize code style across the file. These changes improve readability without altering functionality or logic.
* Refactor modeld logic and remove unused 20Hz and smart inputs
Eliminated `ModelSmartInput`, `ModelSwitcher`, and `ModelState20Hz` classes, simplifying model state handling. Centralized model processing within a unified `ModelState` class and moved related code into `sunnypilot/modeld_20hz`. This improves maintainability by removing unused features and consolidating model execution logic, aligning with current system requirements.
* clean
* Remove debug print statement in commonmodel.cc
The `printf` statement logging buffer movement details was removed as it is unnecessary for release builds. This helps streamline the code and avoid excessive console output during execution.
* Refactor model handling for 20Hz and introduce model runners
Introduce ModelRunner abstraction with TinygradRunner and ONNXRunner to streamline model handling for TICI and non-TICI hardware. Added support for dynamic input preparation and 20Hz models while simplifying the model parsing logic. This improves modularity, readability, and extensibility for future updates.
* Remove unused import and fix import order in model_runner.py
This commit removes the unused 'dtypes' import from tinygrad.tensor and adjusts the import order for cleaner code. These changes enhance readability and maintain coding standards.
* Add is20hz field to custom.capnp schema
Introduce a new boolean field `is20hz` to the `custom.capnp` schema. This allows the system to identify 20Hz-specific configurations or data processing. No changes to existing behavior are introduced for non-20Hz cases.
* Add Meta20hz class for 20Hz model message handling.
Introduces a new Meta20hz class for filling 20Hz model messages, encapsulating functionality for curvature, lane lines, road edges, and more. Refactored `modeld.py` to utilize the new class, improving modularity and maintainability. Minor adjustments were made to initialize and handle model metadata.
* Refactor import paths to align with `openpilot` structure.
Updated several import statements to use the `openpilot` namespace for better consistency and organization. This aligns the sunnypilot components more closely with the overall project structure.
* Refactor modeld to support 20Hz models and modularize runners
Replaced legacy runner logic with a unified ONNX and Tinygrad runner to support 20Hz models. Centralized model metadata management and optimized input preparation for adaptability. Updated curvature handling and output parsing for improved modularity and maintainability.
* Add 20Hz metadata handling for model predictions
Introduce `Meta20hz` class for 20Hz-specific metadata and implement dynamic loading of meta model classes in `meta_helper.py`. Update `fill_model_msg.py` to use the new metadata structure, ensuring seamless integration with 20Hz models. Adjust imports in `model_runner.py` to align with project structure.
* "Refactor modeld_20hz to modeld_v2 with cleanup"
Refactored `modeld_20hz` module to `modeld_v2` for improved clarity and consistency. Removed unused code and aligned imports across modules to reflect the new structure. Enhanced maintainability by restructuring model-related files and updating references accordingly.
* Refactor variable names and adjust imports for clarity.
Renamed `len` to `length` to avoid conflict with the built-in function and improve readability. Reorganized imports in `fill_model_msg.py` for better structure and consistency.
* "Add missing newline at end of file in __init__.py
Ensure proper formatting by adding a newline at the end of the file. This adheres to POSIX standards and improves compatibility with some tools and version control systems."
* Handle model runner initialization errors gracefully
Wrap the model runner initialization in a try-except block to catch and log exceptions. This ensures that failures during initialization are logged with detailed information, improving debugging and error tracing.
* Refactor curvature calculation for clarity and reuse.
Introduce a dedicated `get_curvature_from_output` function to handle desired curvature retrieval, improving code readability and reusability. Replace redundant logic in curvature calculation with the new function to streamline the flow.
* Make 20Hz-specific variables conditional in modeld.py
Moved the initialization of 20Hz-specific variables to be conditional based on the `is_20hz` flag. This ensures that unnecessary memory allocations are avoided when the model is not running at 20Hz, improving efficiency and clarity.
* cleanup
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
|
||
|
Jason Wen
|
213b977774 |
Car: Migrate sunnypilot CarControl to its own cereal (#606)
* carControlSP and move MADS to outside structs * publish it * apply to all car controller * migrate sunnypilotParams * migrate madsEnabled * tldr * convert to capnp * unused * wrong module * fix name * cancer is right (all tests should be passing now) * bump opendbc |
||
|
Jason Wen
|
3e7240516e |
Car: Migrate sunnypilot CarParams to its own cereal (#604)
* sp flags * pass CP_SP to card and car interfaces * CP_SP in radar interface * bump opendbc * use dataclass like old times * bump opendbc * write to params for controls * fix test models * fix * need to use copy instead * fix data type * add service * more * fix * Revert "fix" This reverts commit 74723d7fb277907e174c021af14ecbcdb54f1586. * Revert "fix data type" This reverts commit 02355f44df2e9fca06d565b2e30641ff8d62f1a3. * missed * more * no more lagging * Reapply "fix data type" This reverts commit dbf1b8583fe1843db91c6fb5c96cf1fb3be5ef79. * Reapply "fix" This reverts commit 9cbce9968a6bf245087681d5061a39db90873226. * Revert "Reapply "fix"" This reverts commit 1871919b633c6f94627b2574299740c6c3f6738e. * Revert "Reapply "fix data type"" This reverts commit 5e95752fd5e7d754acdaa83855ec8868ece290f2. * no longer * Revert "no longer" This reverts commit 66ee1ba1513b43b0b07d8344e0dfb13752f6872d. * Reapply "Reapply "fix data type"" This reverts commit 670a3843338490fa7fbf23bbe7bf47c261ded9c8. * Reapply "Reapply "fix"" This reverts commit 42f09f955c84391d01866f3d48717382547f8b85. * only for car params sp * rename * fix more test * no need for process replay * pass stock car params to sp set car params * pass stock car params to sp set car params * deprecate CarParams.sunnypilotFlags to CarParamsSP.flags * missed arg * fix tests * tests fixed * need to pass this too * must generate cp_sp! * fix typing * must be initialized prior can comm callback! * no more cancer (@devtekve) * remove more cancer * Refactor `get_non_essential_params_sp` to simplify arguments (#612) * Refactor 'get_non_essential_params_sp' function calls in tests In both `test_latcontrol.py` and `process_replay.py`, simplified the function calls to 'get_non_essential_params_sp'. Removed an unnecessary call to 'get_non_essential_params'. This change makes the code cleaner and more efficient by reducing redundancy in the function calls. This modification also ensures consistency across different code files. * Refactor get_non_essential_params_sp to take car_params. Simplify parameters by modifying `get_non_essential_params_sp` to use `car_params` as input. Adjust related calls in test files and process replay to match the updated method signature. This improves code clarity and reduces redundancy. * bump opendbc * Refactor parameter handling for `get_params_sp`. Removed unnecessary reassignment of `car_params` in calls to `get_params_sp`, ensuring a cleaner and more streamlined code structure. This change improves code clarity and eliminates redundant operations. All relevant assertions and behavior remain unaffected. * bumping opedbc * bump opendbc --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
d08fd25784 |
Events: Migrate sunnypilot onroad events to its own cereal (#603)
* Events: Migrate sunnypilot onroad events to its own cereal * more * slightly more * typing * fix more * fix mads state machine tests * readjust order * fix event * abstract * need these * move around * let's make sure it cleared on every loop * Update selfdrive/selfdrived/alertmanager.py Co-authored-by: DevTekVE <devtekve@gmail.com> * use upstream custom struct --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
4730a192b1 |
Revert "Events: Migrate sunnypilot onroad events to its own cereal" (#602)
Revert "Events: Migrate sunnypilot onroad events to its own cereal (#598)"
This reverts commit
|
||
|
Jason Wen
|
c9961f1590 |
Events: Migrate sunnypilot onroad events to its own cereal (#598)
* Events: Migrate sunnypilot onroad events to its own cereal * more * slightly more * typing * fix more * fix mads state machine tests * readjust order * fix event * abstract * need these * move around * let's make sure it cleared on every loop * Update selfdrive/selfdrived/alertmanager.py Co-authored-by: DevTekVE <devtekve@gmail.com> --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
5c38aeae0b |
Longitudinal: Dynamic Experimental Control (#572)
* init dec * Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * fix static test * ff * fix static test * unitee testt * Refactor test_dynamic_controller and fix formatting issues Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency. * Refactor test indentation for dynamic controller tests Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code. * Migrated to pytest using claude * Integrate radar parameter into dynamic controller's pytest tests Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases. * Disabling unittest file to allow checks on the pipeline to succeed. Pending to remove this, but leaving it to validate the move to pytest is okay before merging * Replace unittest with pytest for dynamic controller tests Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage. * new line... * Refactor and modularize DynamicExperimentalController logic Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports. * Add missing import for messaging in helpers.py The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code. * Format * Formatting * rebase fix * Refactor MpcSource definition and update references. Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality. * Format * Refactor DEC into a dedicated longitudinal planner class Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity. * **Refactor DEC module structure for better organization** Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes. * static test * static * had moved to car_state * cleanup * some more * static method * move around * more cleanup * stuff * into their own * rename * check live param * sync with stock * type hint * unused * smoother trans * window time * fix type hint * pass sm.frame from plannerd * more fixes * more * more explicit * fix test * Revert "fix test" This reverts commit 635b15f2bc11a8ddd5c4194546d51df33de324a0. * Revert "pass sm.frame from plannerd" This reverts commit a8deaa69b89f599f86ae517827a570789566e079. * use internal frame * update name * never used * this is why it was never using DEC * more logs * slight cleanup * remove to fail test * update name * more * rename * move around * explicit type hints * move to constants py * Revert "explicit type hints" This reverts commit c205497b * more * don't set to exp mode initial if DEC is active * use walrus for None * Revert "use walrus for None" This reverts commit 5f2396d490deded87d43e378bd9e74738fe307b0. * fix wrong typing and variable name * use walrus (needs cleanup) * fix tests * revert smooht lead for now * dec: how good is FirstOrderFilter? * Update dec.py * dec: faster ? * Revert "dec: faster ?" This reverts commit 40259cd22ad23d01e63a9faa47a0cf0edcc73399. * Revert "Update dec.py" This reverts commit 3f29ccbd99c538e9f3d11c6178bc90b0a2ed3f7b. * Revert "dec: how good is FirstOrderFilter?" This reverts commit 01e06df542894b468fb9460c60bca1b09bed4eec. * Update slow-down logic and constants for improved behavior Adjust the slowdown scaling factor and anomaly handling to refine behavior without abrupt resets. Modify constants to increase window size and adjust probabilities and distances for smoother adaptation. Update version to reflect the new changes. * Fix lead detection fallback for weighted average check. Add a fallback value of -1 when computing the weighted average to prevent errors caused by invalid or None values. This ensures robust lead detection and avoids potential crashes or undefined behavior. * visuals for DEC * try this * add opacity * should be active and dimmer * even dimmer * Update dec.py * Update constants.py * use another method for drawing * migrate to sp only * fix * init --------- Co-authored-by: rav4kumar <meetkumardesai@gmail.com> Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
2792ce472a |
Merge branch 'upstream/openpilot/master' into sync-20240118
# Conflicts: # cereal/custom.capnp # cereal/log.capnp |
||
|
Adeeb Shihadeh
|
ae58be87bc |
cereal fork compatibility (#34414)
* more custom * cleanup * lil more |
||
|
Kumar
|
c3bfd17028 |
Revert "Longitudinal: Dynamic Experimental Control" (#571)
Revert "Longitudinal: Dynamic Experimental Control (#564)"
This reverts commit
|
||
|
Kumar
|
bba3c39e2f |
Longitudinal: Dynamic Experimental Control (#564)
* init dec * Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Update sunnypilot/selfdrive/controls/lib/dynamic_experimental_controller.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * fix static test * ff * fix static test * unitee testt * Refactor test_dynamic_controller and fix formatting issues Added a new import for STOP_AND_GO_FRAME and corrected a float initialization for v_ego in MockCarState. Also fixed indentation in the test_standstill_detection method for consistency. * Refactor test indentation for dynamic controller tests Adjust indentation and formatting in test_dynamic_controller.py to ensure consistency and readability. This change does not alter functionality but improves the maintainability of the test code. * Migrated to pytest using claude * Integrate radar parameter into dynamic controller's pytest tests Added a `has_radar` parameter to the test functions in the dynamic controller's pytest file. This allows each function to run both with and without radar inputs, thus enhancing the coverage of our test cases. * Disabling unittest file to allow checks on the pipeline to succeed. Pending to remove this, but leaving it to validate the move to pytest is okay before merging * Replace unittest with pytest for dynamic controller tests Migrated dynamic controller tests from unittest to pytest for improved readability and maintainability. Refactored mock setup using pytest fixtures and monkeypatching while preserving test coverage. * new line... * Refactor and modularize DynamicExperimentalController logic Moved DynamicExperimentalController logic and helper functions to a dedicated module for better readability and maintainability. Simplified longitudinal planner logic by introducing reusable methods to manage MPC mode and longitudinal plan publishing. Adjusted file structure for dynamic controller-related components and updated relevant imports. * Add missing import for messaging in helpers.py The `messaging` module was added to resolve potential issues with undefined references. This change ensures all required imports are present, improving the reliability and maintainability of the code. * Format * Formatting * rebase fix * Refactor MpcSource definition and update references. Moved MpcSource enum into LongitudinalPlanSP for better encapsulation. Updated references in helpers.py to use the new path. This change improves code organization and maintains functionality. * Format * Refactor DEC into a dedicated longitudinal planner class Move Dynamic Experimental Control (DEC) logic to a new `DecLongitudinalPlanner` class for better modularity and maintainability. This simplifies the `LongitudinalPlanner` by delegating DEC-specific behavior and consolidates related methods into a single file. Additionally, redundant code was removed to improve readability and reduce complexity. * **Refactor DEC module structure for better organization** Moved DEC-related files from `dec` to `lib` for improved clarity and consistency within the project structure. Updated all relevant import paths to reflect the new locations. Ensured functionality remains unaffected with these changes. * static test * static * had moved to car_state * cleanup * some more * static method * move around * more cleanup * stuff * into their own * rename * check live param * sync with stock * type hint * unused * smoother trans * window time * fix type hint * pass sm.frame from plannerd * more fixes * more * more explicit * fix test * Revert "fix test" This reverts commit 635b15f2bc11a8ddd5c4194546d51df33de324a0. * Revert "pass sm.frame from plannerd" This reverts commit a8deaa69b89f599f86ae517827a570789566e079. * use internal frame --------- Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: DevTekVE <devtekve@gmail.com> Co-authored-by: Jason Wen <haibin.wen3@gmail.com> |
||
|
DevTekVE
|
9ccc5cc3ea |
models: retain pre-20hz drive model support (#531)
* modeld: Retain pre-20hz drive model support * Method not available anymore on OP * some fixes * Revert "Long planner get accel: new function args (#34288)" * Revert "Fix low-speed allow_throttle behavior in long planner (#33894)" * Revert "long planner: allow throttle reflects usage (#33792)" * Revert "Gate acceleration on model gas press predictions (#33643)" * Reapply "Gate acceleration on model gas press predictions (#33643)" This reverts commit 76b08e37cb8eb94266ad9f6fed80db227e7c3428. * Reapply "long planner: allow throttle reflects usage (#33792)" This reverts commit c75244ca4e9c48084b0205b7c871e1a4e0f4e693. * Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)" This reverts commit b2b7d21b7b685a2785d1beede3d223f0bb954807. * Reapply "Long planner get accel: new function args (#34288)" This reverts commit 74dca2fccf4da59cc8ac62ba9c0ad10ba3fc264b. * don't need * retain snpe * wrong * they're symlinks * remove * put back into VCS * add back * don't include built * Refactor model runner retrieval with caching support Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions. * Update model runner determination logic with caching fix Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks. * parse inputs via metadata * load model and metadata dynamically * cherry pick from devtekve as base * lateral_control_params & prev_desired_curv: MLSIM V0 to Null Pointer * old desired_curv data: MLSIM V1 to Postal Service * Bringing what was on master back then * Cleaning up * Refactor model pipeline for modularity and dynamic input handling Refactored the model pipeline by introducing helper functions to modularize model loading, metadata extraction, and input preparation. Improved flexibility in handling dynamic input keys and parsing outputs based on model configuration. Removed deprecated or unused code segments for cleaner and more maintainable structure. * Push NDv2 because why not and fix modeld * `Refactor model parsing and clean unused code dependencies` Simplified `parse_outputs` by removing unnecessary `input_keys` parameter, ensuring cleaner logic. Updated `PROCESS_NAME` for standardization and eliminated deprecated `Pathlib` dependencies in model paths. Minor adjustments improve input handling for lateral control parameters. * Refactor model and metadata loading functions. Simplified and clarified `load_model` by renaming it to `get_model_path` and removing redundant variable assignments. Streamlined `load_metadata` by directly returning the loaded metadata without intermediate variables. These changes improve code readability and maintainability. * Refactor modeld process selection based on SNPE support. Introduce conditional logic to determine and start the appropriate modeld process (SNPE or default) based on hardware support. This improves flexibility and ensures correct process management. * Walrus baby * Update longitudinal_planner.py * Improve model download progress handling and translations Refactored model download status handling logic for better clarity and added mechanisms to track status changes efficiently. Updated UI text and translations across multiple languages to reflect consistent and accurate model download states. * Revert "Update longitudinal_planner.py" This reverts commit b44a687e4cfb1e2708457478e8626c731f42db0c. * Fix variable naming for curvature size in modeld.py Renamed the variable `len` to `length` to avoid conflict with the built-in `len()` function, improving code clarity and preventing potential errors. Also removed trailing spaces in commented-out sections for better formatting consistency. * sync with upstream * some are different * should work * sim_pose only exists in in ndv3 and prior * dynamic meta constants * fix * uiview: disable power saving * fix this * ain't coasting for y'all * Static analysis * Refactor initialization of model inputs for clarity. Removed redundant pre-initialization of `driving_style`, `nav_features`, and `nav_instructions` variables. Instead, directly initialize these within their respective conditional blocks for better readability and reduced memory usage. * default to none * enable in next PR * more * Revert "more" This reverts commit f5a4220588c5b014b29173f8f89e1fecdec25336. * Revert "enable in next PR" This reverts commit 621cc4f18ead8489cb05a22a73b48e5e4fe33787. * no need to cast bool * nuke * fix desired curvature for pre LAv1 models * mypy * static * fix * new json * Going to a test branch with a different model json list * renamed model json * Update model runner handling in custom.capnp and helpers Refactored model runner logic by introducing a `runner` field in `custom.capnp` and simplifying `get_active_model_runner` logic. Removed deprecated function `get_model_runner_by_filename` and added a temporary filter in `fetcher.py` to enforce `snpe` until full tinygrad support is implemented. * Revert "Update model runner handling in custom.capnp and helpers" This reverts commit f34d872c1369fa5479496ba4e0cce7b3063ab358. * Revert "renamed model json" This reverts commit 15c6ed303b01ab78d1a576dc1d4ad105db7518df. * Revert "Going to a test branch with a different model json list" This reverts commit 4c1408fee524656049ac6502ac4f578e5b632d54. * Reapply "renamed model json" This reverts commit c6fec6912a1517d161dc9b23dce43ddf46866312. Reapply "Going to a test branch with a different model json list" This reverts commit 83e253e9a3b6573191d907584d6368d991395b0a. * Add 'runner' property to 'ModelBundle' and update relevant functions The 'ModelBundle' class in 'custom.capnp' has been extended to include a 'runner' property. This required updating 'fetcher.py' to handle the 'runner' property when parsing model bundles. Additionally, the helper function 'get_model_runner_by_filename' has been removed from 'helpers.py' as it is no longer needed because the 'runner' property provides this information. The 'get_active_model_runner' function has also been updated in light of these changes. * Refine bundle selection logic in SoftwarePanelSP Improve logic for determining which model bundle to display by considering download status and failure state. Remove unnecessary function call to enhance clarity and maintainability. * tmp * Add retrieval of active model bundle in manager loop Introduce a call to `get_active_bundle` to fetch the active model bundle and store it in `self.active_bundle`. This ensures the active bundle is always up-to-date during the model management process. * Add "Use Default" option for model selection Introduced a "Use Default" option in the model selection dropdown, allowing users to reset to the default model. Adjusted logic to handle default selection and ensure proper parameter updates. Fixed bundle status handling during the download process in the model manager. * Refactor type hint in active_bundle assignment. Removed an unnecessary type hint in the active_bundle assignment for cleaner and more maintainable code. This change does not affect functionality but improves code readability. * no nested * update json url * split out * format * condense * more --------- Co-authored-by: Jason Wen <haibin.wen3@gmail.com> Co-authored-by: Kumar <36933347+rav4kumar@users.noreply.github.com> |
||
|
Jason Wen
|
acd46aa94b |
modeld: retain SNPE and thneed drive model support (#555)
* modeld: Retain pre-20hz drive model support * Method not available anymore on OP * some fixes * Revert "Long planner get accel: new function args (#34288)" * Revert "Fix low-speed allow_throttle behavior in long planner (#33894)" * Revert "long planner: allow throttle reflects usage (#33792)" * Revert "Gate acceleration on model gas press predictions (#33643)" * Reapply "Gate acceleration on model gas press predictions (#33643)" This reverts commit 76b08e37cb8eb94266ad9f6fed80db227e7c3428. * Reapply "long planner: allow throttle reflects usage (#33792)" This reverts commit c75244ca4e9c48084b0205b7c871e1a4e0f4e693. * Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)" This reverts commit b2b7d21b7b685a2785d1beede3d223f0bb954807. * Reapply "Long planner get accel: new function args (#34288)" This reverts commit 74dca2fccf4da59cc8ac62ba9c0ad10ba3fc264b. * don't need * retain snpe * wrong * they're symlinks * remove * put back into VCS * add back * don't include built * Refactor model runner retrieval with caching support Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions. * Update model runner determination logic with caching fix Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks. * default to none * enable in next PR * more --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
402ee444b7 |
Revert "modeld: retain SNPE and thneed drive model support" (#551)
Revert "modeld: retain SNPE and thneed drive model support (#530)"
This reverts commit
|
||
|
Jason Wen
|
56aa07844a |
modeld: retain SNPE and thneed drive model support (#530)
* modeld: Retain pre-20hz drive model support * Method not available anymore on OP * some fixes * Revert "Long planner get accel: new function args (#34288)" * Revert "Fix low-speed allow_throttle behavior in long planner (#33894)" * Revert "long planner: allow throttle reflects usage (#33792)" * Revert "Gate acceleration on model gas press predictions (#33643)" * Reapply "Gate acceleration on model gas press predictions (#33643)" This reverts commit 76b08e37cb8eb94266ad9f6fed80db227e7c3428. * Reapply "long planner: allow throttle reflects usage (#33792)" This reverts commit c75244ca4e9c48084b0205b7c871e1a4e0f4e693. * Reapply "Fix low-speed allow_throttle behavior in long planner (#33894)" This reverts commit b2b7d21b7b685a2785d1beede3d223f0bb954807. * Reapply "Long planner get accel: new function args (#34288)" This reverts commit 74dca2fccf4da59cc8ac62ba9c0ad10ba3fc264b. * don't need * retain snpe * wrong * they're symlinks * remove * put back into VCS * add back * don't include built * Refactor model runner retrieval with caching support Added caching for active model runner type via `ModelRunnerTypeCache` to enhance performance and avoid redundant checks. Introduced a `force_check` flag to bypass the cache when necessary. Updated related code to handle cache clearing during onroad transitions. * Update model runner determination logic with caching fix Enhances `get_active_model_runner` to utilize caching more effectively by ensuring type consistency and updating cache only when necessary. Also updates `is_snpe_model` to pass the `started` state to the runner determination function, improving behavior for dynamic checks. * default to none * enable in next PR * more --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
Jason Wen
|
c13f159a77 |
Revert "models: retain SNPE runner support" (#529)
Revert "models: retain SNPE runner support (#522)"
This reverts commit
|
||
|
Jason Wen
|
f84d27c4ee |
models: retain SNPE runner support (#522)
* tinygrad with snpe * force with snpe to validate * fix path * fix more paths * Adjust modeld execution logic based on active model runner Introduced a check to conditionally execute `modeld` based on the active model runner. Added support for distinguishing between SNPE and TinyGrad runners using new helper functions and updated `custom.capnp` definitions. This change optimizes process management by ensuring compatibility with the selected model runner. * Refactor modeld process function checks. Introduce `is_stock_model` to clarify logic and replace direct uses of `is_snpe_model` where the stock model condition is needed. Additionally, rename the duplicate "modeld" process in sunnyPilot to "modeld_snpe" for clarity and consistency. * ignore tg * fix process name * ruff * fix thneed paths * mypy * remove our own * use upstream compile3 * fix thneed * try this * Revert "remove our own" This reverts commit 1cf4f57502565c274628e185c66f1eb04c4d9bbe. * try using compile2.py again * add back symlink * fix path * more fix * wrong path again * Revert "wrong path again" This reverts commit f5301c19d594defb23cda2604168bcb02c830f3e. * update * hardcode path to our submodule * force path * try this * fix file name * try this * again * Revert "again" This reverts commit 17c8cd73768a2de5aaf2a34d202aa6f03d59d72d. * Revert "try this" This reverts commit 767f78bbcf17b08edbc20f283d10a1990bf4264f. * Revert "fix file name" This reverts commit 485eef68da981da571fe9a9ebf3aecdb95e5588c. * Revert "try this" This reverts commit 41fef87680cdda1ba8cbd7e5c59256e6c57010a0. * Revert "force path" This reverts commit 5c3b408937bff0f61b150971b328d127501c5bf0. * Revert "hardcode path to our submodule" This reverts commit 5ee1950b6f4fce73e7e73b6971b996a9989d0a93. * Revert "update" This reverts commit fb313bd7fbeb50111f3c4a11137b4cfd9cabc3fa. * Reapply "wrong path again" This reverts commit 309639aeb3575e20b215bbbcc27e48923ee95c87. * Revert "wrong path again" This reverts commit f5301c19d594defb23cda2604168bcb02c830f3e. * Revert "more fix" This reverts commit 23dd423e78e60bd6a0bd73e921bfba9a5aad88bd. * Revert "fix path" This reverts commit 75d338f2bda2dfa77044e33f4c6553390621b13b. * Revert "add back symlink" This reverts commit 9f71ad0b8ae2068e431c455af9d4f954d024eec7. * Revert "try using compile2.py again" This reverts commit 914117d2e1c6c4270f7bef1a5305847a1d7bab17. * Reapply "remove our own" This reverts commit b1996377b346658f2094cba9032a66230784cb3e. * don't even compile anymore * need it for default snpe model * add to lfs * bring onnx back for sim * must add this back * need this --------- Co-authored-by: DevTekVE <devtekve@gmail.com> |
||
|
DevTekVE
|
ee5c1c4507 |
Modular Assistive Driving System (MADS) (#446)
* allow re-regage
* bump opendbc
* bump panda
* apply pause/resume fix for hyundai (should do this in a separate PR)
* bump opendbc
* fix
* rename
* Fix?
* make sure to disengage for allow always cars
* fix
* combine
* more fix
* not needed
* check if engagement is from openpilot's state machine
* Rename
* fix panda safety
* fix
* no fake lfa button for @devtekve ;)
* fix non drive gear re-engage
* fix settings
* combine
* add replace method
* use replace
* remoev already checks if it exists
* fix
* group
* add todo
* reserve events
* cleaner
* hyundai: only allow for cars with lfa button
* sunnyParams
* make sure it's car only
* Move car-specific changes to opendbc
* no need
* bump opendbc
* more fixes
* no more available
* more!
* final?
* always emit user disable
* no longer needed
* move unit test
* add sunnypilot to unit tests
* bump opendbc
* use new cereal
* bump opendbc
* static analysis
* no unittest
* no need available
* UI border update
* show MADS updates
* Add TODO
* no longer needed
* fix changed events
* fix cluster enabled
* don't add pre enable if not long
* should use enabled
* enabled <-> active
* better format
* bump opendbc
* static analysis
* static analysis
* Rename test as collector was dying
* Show our overriding
* Revert "show MADS updates"
This reverts commit daf0ad62
Revert "fix changed events"
This reverts commit 31d8c97f
* ignoring reserved events
* adjusting creation delays
* back to stock
removing allow_cancel
* should be enabled
* revert
* silent lkas disable
* no need
* user disable tests
* just warning
* MUST REMOVE test process replay
* fix no entry
* fixme
* bump opendbc
* need this check
* cleanup
* allow entering paused state if no entry from disabled
* brake hold should apply to all
* in lists
* update unit test
* simpler
* unused
* same thing
* fix
* only mads in enabled state and long in disabled state
* unify silent enable
* do this for dlob
* bump submodules
* fix
* bump submodules
* bump opendbc
* less frequent
* more events
* fix
* allow no entry to paused for non-drive gears
* fix
* use cereal
* Revert "allow no entry to paused for non-drive gears"
This reverts commit 6d64a4dd9c6de5ad14ef1e88dbc3b6ad174232d8.
* allow in all
* Revert "allow in all"
This reverts commit 6375f1489176dc6c69cb7fa388676e482713f80a.
* should not be all!
* rename for clarity
* silent park brake
* flipped
* bump submodules
* Bump to latest mads-new panda
* bump panda
* more nissan
* bump panda
* bump msgq
* bump panda
* bump submodules
* bump opendbc
* bump opendbc
* improving the state
* Revert "PlayStation® model (#34133)"
This reverts commit
|
||
|
DevTekVE
|
993c4a0d2a |
Driving Model Manager (#457)
* Introduce Model Manager to handle downloads and verification
This commit introduces a new Model Manager responsible for handling model downloads, including driving and navigation application models. The manager also verifies file hashes and communicates download progress for an improved user experience.
The Model Manager is asynchronous and utilizes asyncio and aiohttp for enhanced performance, including robust error handling.
Impacted files in the 'cereal', 'common', 'sunnypilot', and 'system' directories have been updated accordingly. The 'ModelsFetcher' process configuration has been modified to run only when off-road, ensuring optimum resource management.
This update aims to enhance code clarity, improve performance, and streamline the handling of model downloads.
* "Update model management and fetching for SunnyPilot"
This update refactors the model management, downloading and cache verification mechanisms of SunnyPilot. New functionalities, such as smart cache handling, have been implemented in ModelFetcher in sunnypilot/models/model_fetcher.py. Also, the model downloading process has been moved to a separate async function _download_bundle in ModelManagerSP in sunnypilot/models/model_manager.py. Hash verification of files is now performed in an async function verify_file in sunnypilot/models/model_helper.py. Changes in system parameters related to model management have been reflected in system/manager/manager.py.
* Integrate download ETA calculations in model manager
This commit introduces a new feature that calculates and tracks the Estimated Time of Arrival (ETA) for downloading models in the model manager component. The 'eta' property in the 'DownloadProgress' structure in 'custom.capnp' is changed from 'Float32' to 'UInt32'.
In the 'model_manager.py' file, a new method '_calculate_eta' has been added to perform ETA calculations. An additional dictionary '_download_start_times' has been created to keep track of the start time of each model download. The ETA is calculated every time a portion of the model file is downloaded, and it gets updated in the 'DownloadProgress' structure. Finally, the start time is cleared after the download completes.
In 'model_manager_audit.py', an additional check is added to only print downloadProgress for the downloads currently in progress.
* format
* no default model cache {} because it can be considered a valid json, we do not want that
* Refactor typing annotations to use PEP 604 syntax.
Updated type hints to adopt PEP 604 union syntax (`X | None`) and replaced `List` and `Dict` with modern built-in `list` and `dict`. This change improves consistency and readability while aligning with Python 3.10+ standards.
* Simplify logging messages and remove unused imports.
Removed an unused import from `model_manager.py` to improve clarity and maintainability. Also refined a log message in `model_fetcher.py` by removing unnecessary formatting for consistency.
* Refactor model handling and simplify cache fallback logic.
Updated type annotations for `selected_bundle` in `model_manager.py` for clarity. Streamlined cache fallback logic in `model_fetcher.py` by removing redundant conditionals while preserving functionality. These changes improve code readability and maintainability.
* "Fix formatting for ModelManager_DownloadIndex retrieval
Condensed parameter alignment in the get method for improved readability and adherence to style guidelines. This change does not affect functionality but ensures consistent code formatting."
* Need to have main defined for process_config to be able to run it
* Refactor model management to support active bundle tracking
Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.
* UI commit (#515)
* Refactor model management to support active bundle tracking
Introduce the concept of an active model bundle with a new persistent parameter and API updates. Added fields for `generation` and `environment` in model metadata, improved caching, and updated methods to manage active model states efficiently.
* Add new driving model selection feature to settings
This commit introduces a new feature to the settings that allows users to select different driving models. It fetches available models and displays their download progress. The created UI also suggests a calibration reset after model download. The changes include the creation of 'SoftwarePanelSP' within 'settings.' Additionally, 'sunnypilot/SConscript' has been updated to include 'settings.cc' and 'software_panel.cc'. Changes also include localization for this feature.
* Show model description during download status
This update ensures the model description is displayed when a model is in the downloading state. It improves the user interface by providing real-time feedback during the download process.
* Update translations for multiple languages
Added new and updated translation strings in several language files, including Spanish, Arabic, Chinese (Simplified and Traditional), Turkish, Korean, Thai, Japanese, and Brazilian Portuguese. These updates include placeholder translations for new UI elements and features.
* Refactor model name handling and add generation check.
Replaced `bundleName` with `model_name` for better clarity in status messages. Added a generation mismatch check before showing the reset parameters dialog to avoid unnecessary prompts.
* Update model handling in SoftwarePanelSP
Remove unused "common/model.h" and replace "CURRENT_MODEL" with "..." as the default return value in GetModelName. Adjust logic to check for active bundles instead of selected bundles for improved accuracy. Minor text change for clarity in UI label.
* Rename `GetModelName` to `GetActiveModelName` for clarity.
The new name better reflects the function's purpose of retrieving the active model name, improving code readability. All relevant calls and references have been updated to ensure consistency across the codebase.
* Update sunnypilot/models/model_helper.py
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* Refactor download status handling and add 'cached' state
Introduce 'cached' as a new download status and adjust relevant logic across components to support it. Simplify and streamline model status handling in the software panel for better readability and maintainability. Ensure consistent status reporting for all model types.
* Update translations for multiple languages
Refined and expanded translations across various languages, replacing placeholders with meaningful text. This improves clarity and user experience in the multilingual interface.
* Update terminology from 'bundle' to 'model' in UI texts
Replaced occurrences of 'bundle' with 'model' in button labels, dialog titles, and messages in the SoftwarePanelSP code. This improves clarity and aligns terminology with current functionality.
* Update translation placeholders for model fetching texts
Replaced "Fetching bundles" with "Fetching models" across multiple languages to align text placeholders with the updated functionality. Adjusted related background download messages for clarity and consistency.
* cleanup
* not used, and likely not needed
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
* cleaning up
* Update system/manager/process_config.py
* Simplify model parsing and index handling logic.
Refactored `ModelManager_DownloadIndex` retrieval to use the walrus operator, streamlining the conditional logic. Additionally, restructured model list initialization in `_parse_bundle` for improved readability and maintainability. These changes enhance code clarity and reduce redundancy.
* `Improve error handling in model cache retrieval`
Revised the `get` method to ensure it returns an empty dictionary on errors or missing data, avoiding potential `None`-related issues. Added logging for clearer diagnostics when cached model data is unavailable or retrieval fails. This improves reliability and debuggability of the model fetching process.
* Fix cached model data handling by parsing JSON response
Previously, cached model data was returned as a raw string, causing potential issues when using the data. The change ensures the cached data is properly parsed into JSON format before returning, improving reliability and consistency.
* Adjust modelManagerSP rate and Ratekeeper frequency
Reduced the rate for modelManagerSP in services and aligned the Ratekeeper frequency in model_manager.py to 0.1.
* Update model fetcher URL and adjust modelManagerSP rate
Updated the model fetcher URL to point to the correct resource for driving models. Adjusted the rate of modelManagerSP in both its service definition and the corresponding Ratekeeper initialization to 1 Hz for improved consistency.
* Refactor model download logic for clarity and efficiency
Simplify the logic for finding the model to download by combining redundant constructs into a single line. This improves code readability and reduces unnecessary variable assignments.
* Fix cache keys for manual prebuilt actions because they were missing the cache when manually built
* no need to log
* formatting
* revert ci changes
* Refactor and restructure `modeld` to `models` module.
Renamed `modeld` directory to `models` for clarity and consistency. Updated all references and imports to reflect the new structure. This improves maintainability and aligns with naming conventions.
---------
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
|
||
|
Harald Schäfer
|
f870a968e9 |
Split cereal into cereal/msgq (#32631)
* squash * fix doc * compile device * compile device * Update ref * add msgq to precommit exclusions * No service ports * fix compile * address comments * More comments * Delete logger * Update opendbc * Linting * bump msgq old-commit-hash: e71ec4e8d7e39a56235f590227e7cb361cd8b79d |