Commit Graph

606 Commits

Author SHA1 Message Date
Jason Wen 9dba62e686 Merge branch 'upstream/openpilot/master' into sync-20250403
# Conflicts:
#	.github/workflows/auto_pr_review.yaml
#	.github/workflows/selfdrive_tests.yaml
#	.github/workflows/ui_preview.yaml
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/controls/controlsd.py
2025-04-03 21:53:04 -04:00
Harald Schäfer 05f7519628 Remove T_IDXS from road edges and lanelines (#34950)
* Not useful anymore

* add type
2025-03-28 10:43:43 -07:00
Adeeb Shihadeh 2382f1cab0 Run DM model twice, prep for bigger models (#34894)
* twice

* swap

* adjust
2025-03-24 15:54:42 -07:00
Jason Wen 70f8225efc Merge branch 'upstream/openpilot/master' into sync-20250312
# Conflicts:
#    opendbc_repo
#    selfdrive/car/tests/test_models.py
2025-03-13 02:41:16 -04:00
Dean Lee dbbb938ae0 commonmodel: release input_frame_cl and input_frames_cl (#34850)
release input_frame_cl and input_frames_cl
2025-03-12 10:44:04 -07:00
DevTekVE a4ef5ad982 Merge remote-tracking branch 'comma/master' into sync-20250309
# Conflicts:
#	.github/workflows/ui_preview.yaml
#	common/params_keys.h
#	msgq_repo
#	opendbc_repo
#	panda
#	selfdrive/ui/qt/offroad/firehose.h
#	selfdrive/ui/tests/test_ui/run.py
#	system/manager/process_config.py
2025-03-09 14:53:03 +01:00
Jason Wen 290383128e Sync: Fix conflicts 2025-03-09 00:53:06 -05:00
ZwX1616 d807b5c476 Filet o Fish model (#34798)
* 690b01c3 seems ok

* correct temporal

* push

* inplace

* bs

* what thw

* is this wrong

* frames are skipped

* new models

* simplify decimation

* clean up

* clean up modelframe

* need attr

* lint

* 0

* use all samples

* this should break - Revert "use all samples"

This reverts commit 6c0d7943ac5fbb7ae60af1a1201e2423e4c3c105.

* add lc probs

* Revert "this should break - Revert "use all samples""

This reverts commit ca38c54333555266a0d2c885c28af28941841431.

* Reapply "this should break - Revert "use all samples""

This reverts commit a3f0246f209f85f06b9090d9492bfba32ed8cfed.

* Revert "Reapply "this should break - Revert "use all samples"""

This reverts commit 7fd3d2a191b688e5ef7b4dcc8f5379e900af10f8.

* new fish

* e07ce1de-bdea-463e-b5bc-a38ce8d43f4f/700

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-03-07 14:24:22 -08:00
Shane Smiskol a82db2d331 Revert "Filet o Fish model" (#34780)
Revert "Filet o Fish model (#34775)"

This reverts commit 31076b818d.
2025-03-05 01:41:16 -08:00
ZwX1616 31076b818d Filet o Fish model (#34775)
* 690b01c3 seems ok

* correct temporal

* push

* inplace

* bs

* what thw

* is this wrong

* frames are skipped

* new models

* simplify decimation

* clean up

* clean up modelframe

* need attr

* lint

* 0

* use all samples

* this should break - Revert "use all samples"

This reverts commit 6c0d7943ac5fbb7ae60af1a1201e2423e4c3c105.

* add lc probs

* Revert "this should break - Revert "use all samples""

This reverts commit ca38c54333555266a0d2c885c28af28941841431.

* Reapply "this should break - Revert "use all samples""

This reverts commit a3f0246f209f85f06b9090d9492bfba32ed8cfed.

* Revert "Reapply "this should break - Revert "use all samples"""

This reverts commit 7fd3d2a191b688e5ef7b4dcc8f5379e900af10f8.

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-03-04 20:35:57 -08:00
Adeeb Shihadeh e5ba737db0 Revert "Filet o Fish model (#34637)"
This reverts commit 159b1c9eb4.
2025-03-03 22:56:59 -08:00
ZwX1616 159b1c9eb4 Filet o Fish model (#34637)
* 690b01c3 seems ok

* correct temporal

* push

* inplace

* bs

* what thw

* is this wrong

* frames are skipped

* new models

* simplify decimation

* clean up

* clean up modelframe

* need attr

* lint

* 0

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-03-03 15:15:36 -08:00
Adeeb Shihadeh c32c405f65 remove model shims (#34750) 2025-03-02 13:39:36 -08:00
Adeeb Shihadeh 0abe348283 gc old stuff in modeld 2025-03-02 13:23:33 -08:00
DevTekVE b791d4ed55 Model: Handle missing 'sim_pose' gracefully in model outputs. (#644)
Handle missing 'sim_pose' gracefully in model outputs.

Check for the presence of 'sim_pose' in output data before processing. If absent, fallback to using 'plan' data to populate temporal pose fields, ensuring robustness and preventing potential runtime errors.
2025-03-02 20:52:44 +01:00
ZwX1616 29ca4aa7de modeld refactor: supercombo -> vision+policy (#34718)
* should be ok

* update tg

* type

* wrong model

* is no more

* update readme

* add back

---------

Co-authored-by: Comma Device <device@comma.ai>
2025-02-27 17:00:56 -08:00
Harald Schäfer d41311ca20 Switch PC running to tinygrad with LLVM (#34660)
* try llvm on cpu

* update lock

* Make sure test fails

* With BEAM=3 its fast

* small fixes

* more fixes

* Fix macos

* do dm too

* rm ort
2025-02-24 14:51:10 -08:00
YassineYousfi 9ec69ddb36 modeld: trigger model metadata on script change (#34623) 2025-02-19 15:11:33 -08:00
YassineYousfi f06701ea24 modeld: add model names to metadata (#34602)
* modeld: add model names to metadata

* lint

* type hint

* oops

* assert

* ok Any
2025-02-17 14:15:34 -08:00
Jason Wen a0a1635c55 Merge branch 'upstream/openpilot/master' into sync-20250215
# Conflicts:
#	common/params.cc
#	opendbc_repo
#	panda
#	selfdrive/car/tests/test_models.py
#	selfdrive/modeld/fill_model_msg.py
#	selfdrive/test/process_replay/process_replay.py
#	selfdrive/test/process_replay/ref_commit
#	selfdrive/ui/qt/offroad/developer_panel.cc
#	selfdrive/ui/tests/test_ui/run.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_fr.ts
#	selfdrive/ui/translations/main_ja.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
#	system/athena/athenad.py
2025-02-16 22:10:32 -05:00
Adeeb Shihadeh d98a554b17 tici: set core affinity for all realtime processes (#34574) 2025-02-11 21:20:31 -08:00
YassineYousfi 2eb3585dae Not Too Shabby Model (#34484)
* b25c27a8-263b-4f3f-b1f8-72ac49b4f9c7/700

* replay

* model_replay: support empty
2025-01-28 12:16:13 -08:00
YassineYousfi 8bfce0979d modeld: use temporalPose from model (#34460)
* modeld: use temporalPose from model

* cleanup
2025-01-26 10:29:58 -08:00
Jason Wen 7e5a84b897 Merge branch 'upstream/openpilot/master' into sync-20250125
# Conflicts:
#	opendbc_repo
#	panda
#	selfdrive/modeld/fill_model_msg.py
#	selfdrive/pandad/pandad.cc
#	selfdrive/test/process_replay/ref_commit
#	selfdrive/ui/qt/home.h
#	selfdrive/ui/qt/offroad/developer_panel.cc
#	selfdrive/ui/qt/offroad/developer_panel.h
#	selfdrive/ui/tests/test_ui/run.py
#	selfdrive/ui/translations/main_ar.ts
#	selfdrive/ui/translations/main_de.ts
#	selfdrive/ui/translations/main_es.ts
#	selfdrive/ui/translations/main_ko.ts
#	selfdrive/ui/translations/main_pt-BR.ts
#	selfdrive/ui/translations/main_th.ts
#	selfdrive/ui/translations/main_tr.ts
#	selfdrive/ui/translations/main_zh-CHS.ts
#	selfdrive/ui/translations/main_zh-CHT.ts
2025-01-25 22:50:03 -05:00
Shane Smiskol 1d86afdf83 carName -> brand (#34474)
* bump

* brand

* body is lower
2025-01-25 01:39:06 -06:00
DevTekVE c392b2b269 modeld: legacy MLSIM driving models support (#595)
* Add buffer length parameter for enhanced frame handling

Introduce a configurable `buffer_length` parameter to `DrivingModelFrame` to support dynamic buffer sizes, enabling better handling of different frame rates like 20Hz. Updates include necessary adjustments in buffer initialization, copying logic, and related model inputs for improved compatibility and flexibility.

* Rename variable `len` to `length` to avoid shadowing built-in.

Replaced the usage of `len` with `length` across the code to prevent conflicts with Python's built-in `len` function. This improves code clarity and reduces potential errors or misunderstandings in variable usage.

* Fix spacing inconsistency in modeld.py

    Added a missing newline for better code readability and consistency. This change has no impact on functionality but improves code formatting.

* Move numpy_inputs initialization to correct position

Repositioned the `numpy_inputs` initialization to align with the input shape processing logic. This ensures consistency in buffer management and clarifies the flow of code execution related to input handling.

* Add 20Hz model state, smart input, and model switcher classes

Introduce `ModelState20Hz`, `ModelSmartInput`, and `ModelSwitcher` for enhanced modularity and flexibility in modeld. Refactor `ModelState` to inherit from these new classes, enabling support for 20Hz processing and smart input initialization. Update associated files to handle the new buffer length parameter and metadata management.

* Refactor `modeld` to streamline feature handling logic

Simplified feature processing for both standard and "smart input" modes by consolidating logic into reusable methods. Updated variable naming, formatting, and spacing for consistency and readability. This refactor enhances maintainability and reduces redundancy in feature update operations.

* Silence debug print statements and use cloudlog for warnings.

Commented out a debug print statement in `commonmodel.cc` to reduce noise. Replaced `print` statements with `cloudlog.warning` in `model_smart_input.py` for improved logging consistency and better integration with the logging system.

* Clean up formatting and fix minor style inconsistencies

Removed unnecessary blank lines and adjusted spacing to standardize code style across the file. These changes improve readability without altering functionality or logic.

* Refactor modeld logic and remove unused 20Hz and smart inputs

Eliminated `ModelSmartInput`, `ModelSwitcher`, and `ModelState20Hz` classes, simplifying model state handling. Centralized model processing within a unified `ModelState` class and moved related code into `sunnypilot/modeld_20hz`. This improves maintainability by removing unused features and consolidating model execution logic, aligning with current system requirements.

* clean

* Remove debug print statement in commonmodel.cc

The `printf` statement logging buffer movement details was removed as it is unnecessary for release builds. This helps streamline the code and avoid excessive console output during execution.

* Refactor model handling for 20Hz and introduce model runners

Introduce ModelRunner abstraction with TinygradRunner and ONNXRunner to streamline model handling for TICI and non-TICI hardware. Added support for dynamic input preparation and 20Hz models while simplifying the model parsing logic. This improves modularity, readability, and extensibility for future updates.

* Remove unused import and fix import order in model_runner.py

This commit removes the unused 'dtypes' import from tinygrad.tensor and adjusts the import order for cleaner code. These changes enhance readability and maintain coding standards.

* Add is20hz field to custom.capnp schema

Introduce a new boolean field `is20hz` to the `custom.capnp` schema. This allows the system to identify 20Hz-specific configurations or data processing. No changes to existing behavior are introduced for non-20Hz cases.

* Add Meta20hz class for 20Hz model message handling.

Introduces a new Meta20hz class for filling 20Hz model messages, encapsulating functionality for curvature, lane lines, road edges, and more. Refactored `modeld.py` to utilize the new class, improving modularity and maintainability. Minor adjustments were made to initialize and handle model metadata.

* Refactor import paths to align with `openpilot` structure.

Updated several import statements to use the `openpilot` namespace for better consistency and organization. This aligns the sunnypilot components more closely with the overall project structure.

* Refactor modeld to support 20Hz models and modularize runners

Replaced legacy runner logic with a unified ONNX and Tinygrad runner to support 20Hz models. Centralized model metadata management and optimized input preparation for adaptability. Updated curvature handling and output parsing for improved modularity and maintainability.

* Add 20Hz metadata handling for model predictions

Introduce `Meta20hz` class for 20Hz-specific metadata and implement dynamic loading of meta model classes in `meta_helper.py`. Update `fill_model_msg.py` to use the new metadata structure, ensuring seamless integration with 20Hz models. Adjust imports in `model_runner.py` to align with project structure.

* "Refactor modeld_20hz to modeld_v2 with cleanup"

Refactored `modeld_20hz` module to `modeld_v2` for improved clarity and consistency. Removed unused code and aligned imports across modules to reflect the new structure. Enhanced maintainability by restructuring model-related files and updating references accordingly.

* Refactor variable names and adjust imports for clarity.

Renamed `len` to `length` to avoid conflict with the built-in function and improve readability. Reorganized imports in `fill_model_msg.py` for better structure and consistency.

* "Add missing newline at end of file in __init__.py

Ensure proper formatting by adding a newline at the end of the file. This adheres to POSIX standards and improves compatibility with some tools and version control systems."

* Handle model runner initialization errors gracefully

Wrap the model runner initialization in a try-except block to catch and log exceptions. This ensures that failures during initialization are logged with detailed information, improving debugging and error tracing.

* Refactor curvature calculation for clarity and reuse.

Introduce a dedicated `get_curvature_from_output` function to handle desired curvature retrieval, improving code readability and reusability. Replace redundant logic in curvature calculation with the new function to streamline the flow.

* Make 20Hz-specific variables conditional in modeld.py

Moved the initialization of 20Hz-specific variables to be conditional based on the `is_20hz` flag. This ensures that unnecessary memory allocations are avoided when the model is not running at 20Hz, improving efficiency and clarity.

* cleanup

---------

Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
2025-01-24 05:01:14 +00:00
Shane Smiskol 02ec9e5bf8 dmonitoringmodeld: clean exit (#34454)
* nice exit

* correct spacing

* sentry
2025-01-22 20:39:52 -08:00
Dean Lee 205c2b63aa modeld: simplify model message construction by removing redundant variables (#34354)
Simplify model message construction by removing redundant variables
2025-01-19 16:21:02 -08:00
Dean Lee 1068779294 modeld: properly release OpenCL context in __dealloc__ method (#34353)
release OpenCL context in __dealloc__
2025-01-12 10:15:26 -08:00
Harald Schäfer c8264cbf2c Notre Dame model in tinygrad (#34324)
* release model: 6f23a03f-486b-4d3e-a314-19d149644c7c/700

* old style model in tinygrad

* fix desire

* tg hack

* 20Hz

* no gas probs

* No gas here

* better indexing

---------

Co-authored-by: Yassine Yousfi <yyousfi1@binghamton.edu>
2025-01-06 20:52:36 -08:00
Adeeb Shihadeh 420d6f0d5e Quick GC pass heading into 2025 (#34330)
* first pass

* bye bye snpe
2025-01-05 15:10:24 -08:00
YassineYousfi 1a7c284445 National Public Radio Model 📻 (#34259)
* f3a009b7-dcb9-41f3-8917-6fcb3cec37bf/400

* 65f26b40-56c9-4c6c-a3ac-e1788bd52567/400
2024-12-18 16:13:47 -08:00
Harald Schäfer 17ca6389e1 Tinygrad runner (#34261)
* squash

* dmonitoringmodeld: use cl transform (#34235)

* needs cleanup

* only if tici

* bump tinygrad

* check width

* base modelframe

* .

* need to be args

* more cleanup

* no _frame in base

* tici only

* its DrivingModelFrame

* .6 is fair

---------

Co-authored-by: Comma Device <device@comma.ai>

* Update tinygrad

* tg upstream

* bump tg

* bump tg

* debug

* attr

* misc cleanup

* whitespace

* remove

* Add TODOs to make python proc for modelrunners

* whitespace

---------

Co-authored-by: ZwX1616 <zwx1616@gmail.com>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com>
2024-12-18 11:58:59 -08:00
ZwX1616 7352e612a2 dmonitoringmodeld: use cl transform without tinygrad (#34266)
* merge

* why

* self.buf_size

* 0.05 more than with tg due to copy

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-16 16:29:06 -08:00
Harald Schäfer 5018cf75ff North America Model (#34260)
* 0e1c9c12-0472-4a0c-8963-611ad240ec62/400

* rm outputs
2024-12-16 14:17:31 -08:00
Harald Schäfer a98210aeec modeld: ort helpers (#34258)
* ort helpers

* import from ort helpers

* import that too

* linter

* linter

* linter
2024-12-16 13:10:00 -08:00
Harald Schäfer f6885dcbec Revert Tinygrad (#34243)
* Revert "dmonitoringmodeld: use cl transform (#34235)"

This reverts commit 684b0b9d4d.

* Revert "load model before calling convert_fp16_to_fp32"

This reverts commit 31606a7d15.

* Revert "bump tinygrad"

This reverts commit 44f58ff758.

* Revert "Tinygrad runner (#34171)"

This reverts commit 7b5a4fbb03.

* Allow init buffer

* typo
2024-12-14 12:15:36 -08:00
ZwX1616 684b0b9d4d dmonitoringmodeld: use cl transform (#34235)
* needs cleanup

* only if tici

* bump tinygrad

* check width

* base modelframe

* .

* need to be args

* more cleanup

* no _frame in base

* tici only

* its DrivingModelFrame

* .6 is fair

---------

Co-authored-by: Comma Device <device@comma.ai>
2024-12-14 00:44:15 -08:00
Harald Schäfer 8743bc4fe2 Null Pointer Model (#34111)
* e8cb7f27-e448-4c15-90c2-ac440cd5a042/400

* 0078ad07-4d46-4086-820f-23d61c90e07f/400

* 4bd74082-70af-47da-8156-e84ebf4d4812/400

* 2a074022-5c2c-4628-97f9-f54849a936a6/400

* 0660aa81-93c5-41b7-9cc2-dc8816a512cd/400

* Clip curvature to reasonable limits

* Better curvature and speed clips

* typo

* typo

* 31aa62c3-b373-4878-8f2e-5107305de187/400

* 384690ca-9b8a-41fe-9bcd-389b20fc6aa4/400

* ref commit

---------

Co-authored-by: Yassine <yassine.y10@gmail.com>
2024-12-13 19:14:21 -08:00
mitchellgoffpc 31606a7d15 load model before calling convert_fp16_to_fp32 2024-12-12 15:04:54 -08:00
Harald Schäfer 7b5a4fbb03 Tinygrad runner (#34171)
* squash

* bump tg

* bump tg

* debump tinygrad

* bump tinygrad

* bump tg

* Skip init iteration

* fixes

* cleanups

* skip first test sample

* typos

* linter unhappy

* update cpu usage

* OPENCL just zeros for now

* imports

* Try printing

* Runs again, but slower

* unused import

* Allow more buffer with tg and all on gpu

* bump tinygrad

* seems ok

* stricter timings for driving looser for dm

* try llvm

* check nvidia

* More timeout for now

* make test pass

* Revert "try llvm"

This reverts commit ef136e478320101fea262bae3579e558da991902.

* small fixes

* whitespace

* revert test timeout

* No model runners

* Always CPU always fast

* No onnx runtime GPU

* more cores

* cleanup

* Is this faster

* Is this faster

* at least runs

* FP32 is faster than 16

* fix deps

* whitespace

* comment

---------

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2024-12-11 23:15:20 -08:00
Maxime Desroches 216ebcaa50 Fix model runtime on PC (#34210)
exhaustive
2024-12-11 13:27:39 -08:00
YassineYousfi 57fc4f79d1 Postal Service Model (#34183)
0d68d465-9938-4327-a015-0cef58bc9f3a/400
2024-12-09 13:13:01 -08:00
Harald Schäfer 7a4169379d Revert "PlayStation® model (#34133)" (#34170)
This reverts commit 5160bee543.
2024-12-06 23:56:53 -08:00
Adeeb Shihadeh 1ec2c56b4e gc old DM init param 2024-12-03 14:59:28 -08:00
YassineYousfi 5160bee543 PlayStation® model (#34133)
* e2a90efd-8986-4c97-8f55-22725cdee8b9/400

* bea47b6f-c2f9-460f-9658-73f202756294/400

* 7e8efd49-f1a4-4355-912e-610633e0e7f9/390

* 7e8efd49-f1a4-4355-912e-610633e0e7f9/400

---------

Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com>
2024-12-02 23:44:21 -08:00
YassineYousfi 81252f549c Alabama model (#34108)
e22d5e0c-1441-4953-b878-8f0f36e527c6/400
2024-11-25 14:34:36 -08:00
Harald Schäfer fee1f29ce9 Last Horizon Model (#34090)
* 516b3968-82ec-4d7c-89ff-57ded21b3966/400

* 074f0168-aa4a-456b-a82c-464a6fc4ecdf/400

* f3dede04-b52d-4ca1-83aa-9686bd9d49ae/400
2024-11-22 19:19:55 -08:00
Bruce Wayne 612dbb32e1 Revert "Last Horizon Model (#34024)"
This reverts commit e123ac3d32.
2024-11-18 09:32:54 -08:00
Harald Schäfer e123ac3d32 Last Horizon Model (#34024)
516b3968-82ec-4d7c-89ff-57ded21b3966/400
2024-11-18 09:31:13 -08:00