sunnypilot modeld: Sync Tinygrad (#1081)

* Tinygrad bump from sync-20250627 * bump tinygrad_repo * Reformat metadata generator to match driving_models.json * bump tinygrad * Revert "bump tinygrad" This reverts commit f479dfd5021266e22b34deca06cc3c4d531fb7e8. * revert me after SP model compiled * Model recompiled successfully, initiate "revert me after SP model compiled" This reverts commit 95706eb688c9f5522903bee21892b8871ff8f957. * The "FillMe" placeholder caused an extra 10 seconds of work * bump to 22Jul2025 * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * Allow more dynamic short names This should hopefully be future-proof for now.. It's robust enough to return the correct word-digit format (see example on how it generates from given display name below): 'Last Horizon V2 (November 22, 2024)' -> LHV2 'Alabama (November 25, 2024)' -> ALABAMA 'PlayStation (December 03, 2024)' -> PLAYSTAT 'Postal Service (December 09, 2024)' -> PS 'Null Pointer (December 13, 2024)' -> NP 'North America (December 16, 2024)' -> NA 'National Public Radio (December 18, 2024)' -> NPR 'Filet o Fish (March 7, 2025)' -> FOF 'Tomb Raider 2 (April 18, 2025)' -> TR2 'Tomb Raider 3 (April 22, 2025)' -> TR3 'Tomb Raider 4 (April 25, 2025)' -> TR4 'Tomb Raider 5 (April 25, 2025)' -> TR5 'Tomb Raider 6 (April 30, 2025)' -> TR6 'Tomb Raider 7 (May 07, 2025)' -> TR7 'Down to Ride (Revision: May 10, 2025)' -> DTR 'SP Vikander Model (May 16, 2025)' -> SPVM 'VFF Driving (May 15, 2025)' -> VFFD 'Secret Good Openpilot (May 16, 2025)' -> SGO 'Vegetarian Filet o Fish (May 29, 2025)' -> VFOF 'Down To Ride (Revision: May 30, 2025)' -> DTR 'Vegetarian Filet o Fish v2 (June 05, 2025)' -> VFOFV2 'Kerrygold Driving (June 08, 2025)' -> KD 'Tomb Raider 10 (June 16, 2025)' -> TR10 'Organic Kerrygold (June 17, 2025)' -> OK 'Liquid Crystal Driving (June 21, 2025)' -> LCD 'Vegetarian Filet o Fish v3 (June 21, 2025)' -> VFOFV3 'Vibe Model [Custom Model]' -> VMCM 'Tomb Raider 13 (June 27, 2025)' -> TR13 'Aggressive TR (June 28, 2025)' -> ATR 'Tomb Raider 14 (June 30, 2025)' -> TR14 'Cookiemonster Tomb Raider (July 02, 2025)' -> CTR 'Down to Ride (Revision: July 07, 2025)' -> DTR 'Simple Plan Driving (July 07, 2025)' -> SPD 'Down to Ride (Revision: July 08, 2025)' -> DTR 'Tomb Raider 15 (July 09, 2025)' -> TR15 'Tomb Raider 15 rev-2 (July 11, 2025)' -> TR15R2 'Le Tomb Raider 14 (July 14, 2025)' -> LTR14 'Le Tomb Raider 14h (July 17, 2025)' -> LTR14H 'Tomb Raider 16 (July 18, 2025)' -> TR16 'Tomb Raider 16v2 (July 21, 2025)' -> TR16V2 * Update build-all-tinygrad-models.yaml * Update build-all-tinygrad-models.yaml * No need to sleep 3 seconds, just send it * try dynamic * cleanup * Update build-single-tinygrad-model.yaml * bc devtekve said. also, this is repetitive af * Revert "bc devtekve said. also, this is repetitive af" This reverts commit 3a0c1562de6eae1d2d213d2f53cfb2edb4eef40c. * maybe we could use a script instead that both build all That both build all and sunnypilot-build-model reference * refactor: consolidate model building steps into a single workflow * tweak * tweakx2 * tweakx3 * tweakx4 * dunno dunno... * output dir * lots of changes * Revert "lots of changes" This reverts commit 4aadb0ee2929fdf9d902335161b22072bef4e068. * fail if all fail * no inputs needed * make it easier for us * note failure and exit 0 * Update build-all-tinygrad-models.yaml * not needed unless we really want it * Update build-single-tinygrad-model.yaml * Merge branch 'sync-20250627-tinygrad' of github.com:sunnypilot/sunnypilot into sync-20250627-tinygrad * retry for failed ? * always run this step because sometimes one build fails which causes the matrix to fail, but most builds still have uploaded artifacts. * strip * no escape * Update build-all-tinygrad-models.yaml * Test case from terminal run (openpilot) james@Mac sunnypilot % jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \$[^)]*\$"; "")), is_20hz}]' \ /Users/james/Documents/GitHub/sunnypilot-docs/docs/driving_models_v6.json > matrix.json mkdir -p output touch "output/model-Tomb Raider 16v2 (July 21, 2025)-544" touch "output/model-Space Lab Model (July 24, 2025)-547" touch "output/model-Space Lab Model v1 (July 24, 2025)-548" built=(); while IFS= read -r line; do built+=("$line"); done < <( ls output | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ $[^)]*$//' | awk '{gsub(/^ +| +$/, ""); print}' ) jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \ 'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' \ matrix.json > retry_matrix.json cat retry_matrix.json [] (openpilot) james@Mac sunnypilot % * always * great success * add suffix to retry artifact so it doesn't conflict * retry to get_model too * and there haha * unnecessary hyphen * compare built to missing. include retries * adjust copy of artifacts. * Update build-all-tinygrad-models.yaml * Update model selector versioning and add documentation * Update retry condition for failed models in build-all-tinygrad-models.yaml * Update retry condition for failed models in build-all-tinygrad-models.yaml * Update build-single-tinygrad-model.yaml * false * default none because why not * red diff? i think? * meh ... not needed i guess * error error error * Nayan is watching... always watching mike wazowski * string all the way * lots of retries just in case because im scared * more robust * ONLY ONE!!!!!! * delete.... a lot * fix artifacts * fix artifacts * make sure each is unique :) * skip files like artifact duhhhh * artifact name dir * concurrency * copy here * Update build-single-tinygrad-model.yaml * Update build-single-tinygrad-model.yaml * bump * bump tinygrad * max parallel? if not, i have the other remedy ready in build-all * revert me! * I resynced tinygrad woo hoo * setup shouldnt fail * pull * big ole diff * condition * Update build-all-tinygrad-models.yaml * not always() never always() never!!! * not failure instead of great success * Update build-all-tinygrad-models.yaml * yay that worked. lets invoke build-single one last time * these arent used and are just taking up 250MB space * really frog? * bump back to 3 * self-hosted, tici * rename to trigger tests * split sync --------- Co-authored-by: DevTekVE <devtekve@gmail.com>
2026-08-06 00:36:29 +08:00 · 2025-07-31 06:50:23 -07:00
parent 72a6c9fd65
commit c47aadee19
6 changed files with 71 additions and 7 deletions
@@ -55,7 +55,7 @@ jobs:
          path: ${{ github.workspace }}/selfdrive/modeld/models/*.onnx

  build_model:
-    runs-on: self-hosted
+    runs-on: [self-hosted, tici]
    needs: get_model
    env:
      MODEL_NAME: ${{ inputs.custom_name || inputs.upstream_branch }} (${{ needs.get_model.outputs.model_date }})
@@ -71,7 +71,7 @@ jobs:
        with:
          path: ${{env.SCONS_CACHE_DIR}}
          key: scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model-${{ github.sha }}
-          # Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.) 
+          # Note: GitHub Actions enforces cache isolation between different build sources (PR builds, workflow dispatches, etc.)
          #   for security. Only caches from the default branch are shared across all builds. This is by design and cannot be overridden.
          restore-keys: |
            scons-${{ runner.os }}-${{ runner.arch }}-${{ github.head_ref || github.ref_name }}-model
@@ -50,7 +50,7 @@ jobs:
    concurrency:
      group: build-${{ github.head_ref || github.ref_name }}
      cancel-in-progress: false
-    runs-on: self-hosted
+    runs-on: [self-hosted, tici]
    outputs:
      new_branch: ${{ steps.set-env.outputs.new_branch }}
      version: ${{ steps.set-env.outputs.version }}
@@ -0,0 +1,63 @@
+# Model Selector Version Compatibility
+
+This document explains the version compatibility mechanism used by the Model Selector system, and the rationale behind certain version constraints and JSON file management strategies.
+
+## Overview
+
+The Model Selector is responsible for selecting and validating model bundles based on their metadata and version constraints. Each model bundle is distributed via a JSON file and includes a `minimumSelectorVersion` field indicating the minimum selector version required to load it.
+
+To ensure robust compatibility and prevent mismatches between model expectations and selector capabilities, the selector enforces two version boundaries:
+
+* **`REQUIRED_MIN_SELECTOR_VERSION`**: the oldest selector version we support.
+* **`CURRENT_SELECTOR_VERSION`**: the current version of the selector logic.
+
+## Version Compatibility Check
+
+A model bundle is considered compatible if:
+
+```python
+REQUIRED_MIN_SELECTOR_VERSION <= bundle["minimumSelectorVersion"] <= CURRENT_SELECTOR_VERSION
+```
+
+This ensures:
+
+* **Old bundles are rejected** if they rely on deprecated selector behavior.
+* **Future bundles are ignored** if they expect logic that the current selector doesn't yet implement.
+
+## Handling Breaking Changes
+
+When a deep change in selector behavior requires *all* models to be recompiled (e.g., due to a major architectural update), we:
+
+1. **Create a new JSON file** (e.g., from `models_v4.json` to `models_v5.json`).
+2. **Assign updated `minimumSelectorVersion` values** in the new bundles.
+
+This allows older selector versions to continue using the previous JSON file, while newer versions point to the new one, preventing cross-contamination.
+
+## Why `REQUIRED_MIN_SELECTOR_VERSION` Still Matters
+
+Despite using new JSON files to isolate breaking changes, `REQUIRED_MIN_SELECTOR_VERSION` plays a critical role:
+
+### 1. **Cached Bundle Validation**
+
+Model bundles are cached locally (e.g., in-memory or on disk). A user might have previously loaded a now-invalid bundle from an older JSON file.
+
+`REQUIRED_MIN_SELECTOR_VERSION` prevents the selector from reloading or trusting that stale cached bundle, even if the original JSON is gone.
+
+### 2. **Explicit Deprecation Boundary**
+
+By raising `REQUIRED_MIN_SELECTOR_VERSION`, we declare older bundles officially unsupported, even if they technically still exist in a legacy JSON file.
+
+### 3. **Avoiding Race Conditions**
+
+Some clients may have intermittent access to updated JSONs. The runtime check ensures version compatibility is enforced independently of external file state.
+
+## Summary
+
+| Component                       | Purpose                                                               |
+| ------------------------------- | --------------------------------------------------------------------- |
+| `minimumSelectorVersion`        | Declares the minimum selector version required to load a model bundle |
+| `REQUIRED_MIN_SELECTOR_VERSION` | Prevents loading bundles that are too old (e.g., from stale cache)    |
+| `CURRENT_SELECTOR_VERSION`      | Prevents loading bundles that are too new or forward-incompatible     |
+| JSON file renaming              | Isolates bundles by selector generation to handle full recompiles     |
+
+This layered strategy ensures safe evolution of the model selection system while maintaining backward compatibility and runtime protection against stale or incompatible bundles.
@@ -115,7 +115,7 @@ class ModelCache:

 class ModelFetcher:
  """Handles fetching and caching of model data from remote source"""
-  MODEL_URL = "https://docs.sunnypilot.ai/driving_models_v4.json"
+  MODEL_URL = "https://docs.sunnypilot.ai/driving_models_v6.json"

  def __init__(self, params: Params):
    self.params = params
@@ -19,8 +19,9 @@ from openpilot.system.hardware import PC
 from openpilot.system.hardware.hw import Paths
 from pathlib import Path

-CURRENT_SELECTOR_VERSION = 7
-REQUIRED_MIN_SELECTOR_VERSION = 5
+# see the README.md for more details on the model selector versioning
+CURRENT_SELECTOR_VERSION = 8
+REQUIRED_MIN_SELECTOR_VERSION = 6

 USE_ONNX = os.getenv('USE_ONNX', PC)