* Add support for TinyGrad model runner processing
Introduced a new function `is_tinygrad_model` to detect TinyGrad as an active model runner. Updated the `is_stock_model` logic to account for TinyGrad models and added a new process entry for TinyGrad in the model manager. This enables handling TinyGrad models alongside existing configurations.
adding modeld back
Add support for `modeld_v2` and update paths for consistency
Updated `SConscript` files to integrate `modeld_v2` alongside `modeld` and adjusted script paths for correct metadata handling. Adjusted various configurations and scripts, such as `labeler.yaml` and `build_release.sh`, to include `modeld_v2` and ensure cohesive project structure.
Refactor imports to use updated `modeld_v2` paths.
Replaced outdated `modeld` references with their `modeld_v2` counterparts for consistency and clarity across the codebase. Also updated `.gitignore` to accommodate new directory structure. This change ensures better maintainability and alignment with the new directory schema.
Refactor and reorganize modeld to sunnypilot/modeld_v2 structure.
Moved and renamed `modeld` components to the new `sunnypilot/modeld_v2` directory for better organization and modularity. Updated imports and file references to align with the new structure, ensuring compatibility and functionality. Streamlined project structure to improve maintainability and future development.
* typo
* Use `stock` model runner and refactor model checks.
Replaces outdated model detection logic with unified `stock` runner integration, simplifying the decision flow for model selection. Includes `stock` as a new enum in the `Runner` type and updates affected references accordingly.
* Handle missing 'sim_pose' in model outputs gracefully.
Added conditional checks to ensure the code handles cases where 'sim_pose' is absent in the model outputs. Fallback behaviors use 'plan' data when 'sim_pose' is unavailable, preventing potential errors and enhancing robustness.
Fix branch status check for merge readiness
Previously, any non-"CLEAN" status prevented merging. The check now specifically fails only if the branch is "BEHIND", allowing other statuses to proceed if valid. This ensures more accurate merge validations.
* Add nightly branch reset workflow and improve squash script
Introduced a GitHub Actions workflow to reset and squash PRs nightly for the `master-dev-c3-new-test` branch. Enhanced `squash_and_merge.py` to handle more specific exit codes and `squash_and_merge_prs.py` to streamline PR processing. Updated argument handling in scripts and added validation for squash script execution.
* Forcing to show up
* UnForcing to show up
* Refactor branch handling to use inputs/environments directly
Removed intermediate step for setting branch variables and updated logic to use `inputs` or environment variables directly. Simplifies the workflow and improves maintainability by reducing redundancy and reliance on unnecessary outputs.
* Fix Python script invocation in CI workflow.
Replaced implicit script execution with an explicit `python3` command to ensure compatibility and consistency in the workflow. This change resolves potential issues with shell defaults or system configurations.
* Update branch reset logic in workflow script
Replaces checkout-based branch reset with deletion and recreation to ensure the target branch correctly points to the source branch. This change handles cases where the target branch may already exist.
* Refactor PR data handling to parse JSON input.
Updated the script to parse and handle PR data as JSON, ensuring proper data structure during processing. Adjusted functions to operate on parsed JSON instead of raw arguments for improved clarity and error handling.
* Refactor PR processing to streamline branch handling
Removed redundant `fetch_pr_branches` function and integrated branch fetching directly into `process_pr`. Simplified subprocess calls for clarity and added branch cleanup to prevent conflicts. This improves code maintainability and execution efficiency.
* Add PR number to squash and merge commit titles
This change appends the PR number to the title used in squash and merge commits, improving traceability and clarity in the commit history. The modification ensures easier identification of commits linked to specific pull requests.
* Enhance PR processing with sorting and additional checks
Implemented sorting of PRs by creation date and added checks for merge conflicts, commit data availability, and status check completion. Updated nightly squash script to provide detailed feedback via PR comments for skipped PRs. These changes improve the reliability and traceability of the merge process.
* Add traceback logging to error handling in squash_and_merge_prs
Enhanced error reporting by including full traceback details when an exception occurs in the `process_pr` function. This aids in debugging by providing more context on failures.
* Refactor and add debug output to PR merge check logic
Simplified multi-line statements into single lines for clarity and added debug `print` statements to log `merge_status` and its output. These changes enhance readability and facilitate debugging of the merge conflict check process.
* Add GITHUB_TOKEN to environment for CI workflow
Ensure the workflow has access to the GITHUB_TOKEN for authentication. This is necessary for interacting securely with the GitHub API during the CI process. Without this, some steps may fail due to lack of authorization.
* Enable scheduled workflow execution at midnight UTC
Reactivates the cron schedule for the workflow to run daily at midnight UTC. This ensures the workflow executes automatically without manual triggers, maintaining regular updates or checks.
* test
* Update workflow to trigger and monitor selfdrive tests
Replaced the direct triggering of the prebuilt workflow with a step to trigger and wait for the completion of selfdrive tests. Ensures prebuilt workflow runs only if selfdrive tests succeed, improving reliability of the CI process.
* Refine workflow trigger to fetch run URL and ID
Updated the workflow script to extract the run URL and derive the workflow ID from it. This ensures more accurate handling and tracking of GitHub Actions runs.
* Simplify selfdrive test triggering in workflow
Replaces custom script with a reusable GitHub Action to trigger and wait for selfdrive test completion. This improves maintainability and reduces complexity in the workflow file. Adjusts subsequent prebuilt workflow trigger to ensure compatibility.
* Remove duplicate prebuilt workflow trigger step
The redundant step for triggering the sunnypilot prebuilt workflow has been removed. This cleanup avoids unnecessary duplication and ensures a more streamlined workflow definition.
* Update selfdrive test trigger to use GitHub CLI commands
Replaced the third-party action with GitHub CLI for triggering and monitoring selfdrive tests. This change improves maintainability and reduces reliance on external dependencies. Updated related steps to ensure compatibility with the new approach.
* Add delay to ensure selfdrive tests workflow starts
Introduce a 120-second sleep before fetching the latest run ID to allow sufficient time for the selfdrive tests action to initialize. This prevents potential issues caused by attempting to retrieve the run ID too early.
* Improve push step to check for diffs before execution
Added logic to verify if there are differences between local and remote branches before attempting a push. This prevents unnecessary pushes and skips subsequent workflows when no changes are detected. Updated dependent steps to conditionally run based on the presence of changes.
* Update target branch and improve squash comment clarity
Renamed the default target branch from `master-dev-c3-new-test` to `nightly` in the workflow configuration. Enhanced squash process comments to dynamically reflect the `target_branch` value for better clarity and consistency.
* Add missing newline at end of file
Ensures the file complies with POSIX standards by including a newline at the end. This improves consistency and prevents potential issues with some tools or systems.
* Update default target branch and disable nightly schedule
Changed the default target branch to 'master-dev-c3-new' for workflow consistency. Commented out the nightly schedule to temporarily disable automated runs. No functional changes were made to other parts of the workflow.
* Refactored squash and merge scripts for improved PR handling
In this commit, significant updates have been made to the 'squash_and_merge.py' and 'squash_and_merge_prs.py' scripts related to how pull requests (PRs) are processed.
A 'source-branch' argument has been added to the argument parser. The merging command has been changed from 'merge' to 'rebase'.
The PR processing function has been refined. Specifically, PR validation is now a separate function confirming the conditions 'branch name', 'commit data', 'check pass status', and 'mergeability'. Now, any failures under these conditions result in skipping the PR with an appropriate warning.
The target branch is deleted if it exists, before a new one is created from the source branch. The squash script now runs with more structured arguments.
These changes generally improve PR handling in CI/CD pipelines, making them more efficient and error-resistant.
* Add 'PullRequest' to .codespellignore
This update includes 'PullRequest' in the .codespellignore file to prevent it from being flagged as a spelling error. It helps streamline code reviews and reduces false positives during spell checks.
* Refactor GitHub runner installation script
Enhance the script with modular functions for better readability and maintenance. Add support for restoration mode, enabling the recovery of existing runner configurations. Improve error handling, argument parsing, and system configuration setup.
* Ensure runner directory existence before restoration
Add a prerequisite check to verify the runner directory exists, preventing restoration failures due to a missing directory. Adjust exit code logic for proper handling when no restoration is required.
* rw not needed until a bit later
* Refactor system configuration handling and remount logic.
Reorganized system setup functions and centralized remount logic for consistency and clarity. Improved script structure by separating utility and setup functions, ensuring proper remounting behavior with a robust exit trap. Reduced redundancy and enhanced maintainability.
* silly change
* Add script for squash and merge workflow automation
Introduced `squash_and_merge.py` to automate the process of squashing a source branch and merging it into a target branch. The script handles backups, commit message creation, stashing, cleanup, and includes optional push functionality. Provides user prompts and error handling to simplify complex merge workflows.
* temp1
* Switch to PEP 604 style for optional and generic type hints
Replaces `Optional` and `List` with the simpler `X | None` and `list[X]` syntax introduced in Python 3.10. Updated all function signatures accordingly for consistency and modernity. Also made minor string formatting consistency adjustments.
* Fix type annotation for temp_branch variable
Updated the temp_branch variable to include a proper type annotation (`str | None`). This change improves code clarity and aligns with Python typing standards.
* more hints
* Add support for GitHub Actions runner management
Introduce a new process and configuration to manage a GitHub Actions runner. Added a persistent "EnableGithubRunner" parameter and a script to control the runner service. Integrated the feature into the system's process manager logic.
* Restrict GitHub runner usage on metered networks.
This update modifies the `use_github_runner` function to include a check for metered networks using `HARDWARE.get_network_metered()`. This ensures the GitHub runner is not enabled when the network is metered, improving network usage efficiency.
* Add GitHub runner service toggle to developer panel
Introduces a new toggle in the developer panel to enable or disable the GitHub runner service. This provides developers with a convenient way to control the service from the UI.
* translations
* Refactor network condition check for GitHub runner.
Updated `use_github_runner` to include network type in metered network checks. This ensures more accurate validation by considering specific network types when determining metered status.
* Mark as executable
* Update paths and shebangs for consistency across scripts
Modified file paths to align with the new directory structure under `/data/media/0/github` and updated the shebang in `github_runner.sh` for better environment compatibility. Adjusted the `BUILD_DIR` in the GitHub workflow to reflect the new path.
* Fix string translation for GitHub runner parameter text
Added translation support for the GitHub runner service description text. This ensures consistent localization across the UI.
* Remove gitlab_runner.sh from Sunnypilot blacklist.
The script is no longer required to be blacklisted, likely due to updates or changes in its usage. This improves the maintainability of the blacklist by removing unnecessary entries.
* lang stuff
* Update BASE_DIR determination based on mount point
Refactored scripts to dynamically set BASE_DIR depending on whether /data/media is a valid mount point. This ensures compatibility with different environments and improves robustness of path resolution.
* Refactor GitHub runner logic in process_config.
Simplified enabling conditions for the GitHub runner by removing dependency on hardware network checks and adding a logical combination of offroad and runner-related functions. This improves code readability and reduces hardware coupling.
* Create a GitHub Actions workflow for synchronizing the repository to GitLab and enhance GitLab CI settings
A new GitHub Actions workflow for mirroring the current repository to GitLab is added. This workflow is triggered by both push and delete events in addition to manual triggering. The role of GitHub actions runner is defined through a series of steps. A new '.gitlab-ci.yml' build configuration file is also introduced, with a more comprehensive definition of variables, jobs, and pipeline rules for its utilization in the GitLab CI/CD pipeline.
Further, other changes include the addition of scripts for installing and uninstalling GitLab CI runner, along with modifying the SCons build system configuration file to include custom cache directory. Moreover, 'release_files.py' has been revised to include additional blacklisted and whitelisted files specific to Sunnypilot, ensuring suitable settings for the CI flow.
The improvements facilitate a smoother integration between GitHub and GitLab, powerfully harnessing the capabilities of both platforms for more efficient and effective CI/CD pipelines and version control management.
* not needed for this
* Update workflow to build model from upstream repository
Revised the CI workflow to build directly from the upstream `commaai/openpilot` repository. Simplified configuration, removed unused steps, and added support for specifying the upstream branch dynamically via inputs.
* Update SConstruct to allow passing arbitrary cache_dir
Modified the SConstruct file to enable setting a custom cache directory via arguments. This enhances flexibility in configuring cache paths during the build process.
* Refactor build workflow to improve branch handling logic.
Reorganized conditions for setting environment variables, replacing repository_dispatch with workflow_dispatch for prebuilt builds. Added a fallback error message for unsupported triggers to improve robustness. This enhances clarity and ensures compatibility with the intended workflow triggers.
* test test test
* test test test
* test test test
* Enable publication flag during build configurations
Added `SHOULD_PUBLISH=true` to all relevant build configuration steps to ensure proper handling of publishing logic. Updated environment variables to include this flag for downstream usage. Removed the error-check step for unsupported configurations.
* Simplify publish condition in workflow logic
Replaced the previous condition for publishing with a single output variable, `should_publish`, to streamline logic and improve maintainability. This change reduces redundancy and makes the workflow more adaptable to future updates.
* Simplify restore key patterns in build workflow.
Removed unnecessary trailing dashes in SCons cache restore keys to streamline and slightly improve key matching logic. This ensures better consistency with the current workflow setup.
* Update cache key usage in build workflow
Replaced `github.ref_name` with `github.head_ref` for cache keys to ensure accurate branch-specific caching. Added fallback restore-keys for master branches to improve cache efficiency and reduce redundant builds.
* Improved debug logging in GitHub actions workflow
This commit refines the debug logging in our GitHub Actions workflow for the Sunnypilot build. This change provides more granularity, enabling logging only during debug mode, which helps to keep the runtime logs less cluttered during normal operations. This includes the conditionally displaying of environmental variables, GitHub output contents, and directory listings.
Additionally, debug mode verbosity was added to rsync commands to aid troubleshooting file transfers during the build process.
Blank lines were also reduced for better readability and cleaner code presentation.
* test diff path
* Refactor SCons cache handling in build workflow
Replaced hardcoded cache directory paths with an environment variable (`SCONS_CACHE_DIR`) for better maintainability and flexibility. Updated related workflow steps to utilize the new variable and adjusted cache key usage. Removed unused `BASE_BUILD_NUMBER` variable to clean up the configuration.
* Update cache key to include commit SHA in workflow
This change adds the commit SHA to the cache key in the GitHub Actions workflow. It ensures more precise caching by differentiating builds based on the specific commit, reducing potential conflicts.
* clean
Update GitHub runner service to set environment variables
Updated the ExecStart command to explicitly set HOME, USER, LOGNAME, and MAIL environment variables. This ensures the runner operates with the correct environment configuration, improving reliability and compatibility.
Refactor GitHub runner service ExecStart command.
Replaced direct command execution with running the service as a specific user using `su`. This improves compatibility and aligns with best practices for user-based execution. No functional changes are expected.
Update GitHub runner service to set environment variables
The ExecStart command now sets HOME, USER, LOGNAME, and MAIL environment variables for the runner process. This ensures proper environment initialization for the designated user, improving compatibility and reliability during execution.
Refactor GitHub runner service template handling.
Revised the `modify_service_template` function to create a properly structured service template for the GitHub runner. Updated service permissions, execution parameters, and enabled the function call to ensure usage during runner setup.
* Refactor GitHub Runner installer to improve argument parsing.
Reworked the script to implement flexible and explicit command-line argument parsing using flags like `--token`, `--repo`, and `--start-at-boot`. Added support for setting default values and enabling/disabling auto-start based on the `--start-at-boot` flag. Improved error handling and usage messaging for better user experience.
* Update cache keys in sunnypilot build workflow
Modified the cache keys to include `github.ref_name` for more precise caching and restore behavior. This improves build consistency by better differentiating between branches and refs. No changes to the overall workflow logic.
* Remove 'tinygrad/*' from release file exclusions
This change modifies the release file exclusions by removing 'tinygrad/*'. The adjustment ensures that files in the 'tinygrad' directory are now included in the release process, aligning with updated packaging requirements.
* Simplify release file exclusions list.
Removed redundant and unnecessary file patterns from the exclusions list in `release_files.py`. This streamlines the file handling process and reduces maintenance overhead.
* Refactor SCons cache path and key structure.
Updated the SCons cache directory path to `SCONS_CACHE_DIR` for clarity and consistency. Improved the caching key structure to include `github.head_ref` for better cache differentiation and restore hierarchy. Adjusted related build instructions to reflect the new environment variable.
* Refactor branch configuration in CI workflow
Standardize branch name handling by replacing hardcoded values with environment variables. This improves maintainability and simplifies updates to branch names across the workflow. Updated references to use the new dynamic environment variable approach.
* squash
* bump tg
* bump tg
* debump tinygrad
* bump tinygrad
* bump tg
* Skip init iteration
* fixes
* cleanups
* skip first test sample
* typos
* linter unhappy
* update cpu usage
* OPENCL just zeros for now
* imports
* Try printing
* Runs again, but slower
* unused import
* Allow more buffer with tg and all on gpu
* bump tinygrad
* seems ok
* stricter timings for driving looser for dm
* try llvm
* check nvidia
* More timeout for now
* make test pass
* Revert "try llvm"
This reverts commit ef136e478320101fea262bae3579e558da991902.
* small fixes
* whitespace
* revert test timeout
* No model runners
* Always CPU always fast
* No onnx runtime GPU
* more cores
* cleanup
* Is this faster
* Is this faster
* at least runs
* FP32 is faster than 16
* fix deps
* whitespace
* comment
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
* Revert "Replace ThneedModel with TinygradModel (#33532)"
This reverts commit da952e9b64.
* Revert "camerad: move E + D cams image pipelines to the IFE (#33959)"
This reverts commit f2a1cce42b.
* add longitudinal profiles
* stash
* unfortunately even longitudinalPlan causes circle
* add to process config
* reach target speed smoothly
* stash
* works
* clean up
* debug alert
* rename
* fix
* better text
* toggle via exp button
* try coming to a stop better, smoother target reaching
* closer to target
* revert controlsd migration
* add description to alert
* generate report from local logs
* hide bad maneuvers
* pdflike
* Revert "pdflike"
This reverts commit 6d4af1bf9be2e9e0798eaecf026a53d860da7613.
* try this
* use alert manager
* fix that check
* wat
* Revert "wat"
This reverts commit 93d0d27ab838d3f580d06ff212f380e0b912d599.
* some clean up
* rm
* cleanup
* move
* fix test
* more fix
* clean up
* fix that
* remove pre-commit (explorations)
* lint.sh
* remove rest
* add to release files
* no source
* 1min is all you need
old-commit-hash: 13511e383c1679d17843e8cbf1bcb0a926f74c35
* blacklist more files from release
* dbcs
* fix
* just use the lfs
* rm that too
* cleanup
* cleanup
old-commit-hash: 5c3c06dfdf087c74214a6cfc350cbcc02346b41e
* updated: finish system/ move
* add those back
* skip that
* skip whole file
* one more time
old-commit-hash: 73b02f2cda858d4ad08bc7fa4116d0d11ff79a02
* one call does all
* no need
* update etst
* filename
* dbf5b05ff480145a79b5941e360d0698b70979cd
old-commit-hash: 204219695f57f6ba76347df4df99e00a28b51e9f
