* modeld_v2: Support eGpu
* bump tg
* egpu
* no pkls please
* god no onnx either
* fix test
* done in build model now
* lint
* rip
* egpu ready build all split
* manual seed , reuse memory buffers across runs
* dont download big when we dont have big lol
* whoops
* no i and x
* cd
* who put those there. ??
* reduce flakiness by using artifact-name from build-model to regex, speed up pub b y checking the name before trying to clone and publish again
* try hf as a trusted publisher :)
* mf its a dataaset. i knew that
* fucking validation wants raw to fetch and full to push. grr
* smh
* dude i am missing so much
* pkl name
* move build all to hf
* tests: migrate sunnypilot tests to unittest and remove pytest
* red diff mf
* im scared , this may be a bad idea lol
* fetch latest commit.
* transition to requests
* models: use requests instead of aiohttp
* tici
* fix
* gpu fixes from upstream
* lint
* bump
* needed to say
* old
* how??
* epgu flag reduce
* this made me cry
* support monolith still
* precache warp in legacy
* Move jsons to param for sunnylink
---------
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* modeld_v2: safe model validation
* fix string
* numpy
* dumb
* god use full attribute names please
* modeld_v2: refactor compile_modeld
* redundant
* CREAM AND SUGAR
* gpu stuffs
* Update fetcher.py
* Update compile_modeld.py
* Update compile_modeld.py
* Update compile_modeld.py
* Update compile_modeld.py
* summary
* Update compile_modeld.py
* Update compile_modeld.py
* Update compile_modeld.py
* i could lie say
* needed
* i could
* Update compile_modeld.py
* done done done
* bye metadata
* simplify
* deeeeep
* oopsie
* i don't know what i'm doing
* it's a supercombo
* wtf. ghostwriter
* i might be blind
* fuck. i AM blind. or dumb. or both.
* hmmmm
* read/open - what's the difference
* realize. that i don't know shit
* whatever
* fix paths
* fuckit. dynamic everything.
* simplify everything.
* fix macos cabana
* np.random.seed is deprecated, use Generator or default rng instead @nayan
* this is annoying me
* too much fluff
* no
* lint be crazy now, man i've been away a while
* back
* fix pkl loader test
* that comment was wrong, its still per model, just compiled at with the input/output shapes
* fix chunking
* dump and assign to cpu
* prev_feat in cpu to prevent corruption
* add todo-sp
* desire me?
* allow legacy
* lint
* shapey
* bye
---------
Co-authored-by: discountchubbs <alexgrant990@gmail.com>
Co-authored-by: James Vecellio-Grant <159560811+Discountchubbs@users.noreply.github.com>
Co-authored-by: Jason Wen <haibin.wen3@gmail.com>
* zero ll patched big model
* probe in a subprocess so usbgpu lock gets released
* compiles
* runs
* num_jobs gets overwritten, use side effect
* poll tg devices
* make sure build crashes on missing gpu
* fine not to rely on Device.default
* seperate tg env for each model runner
* comment
* Revert "seperate tg env for each model runner"
This reverts commit f6470cc4258eaeb3e8e37907ef370871c9af5aa4.
* env is shared, gate on flag
* no fallback warp dev must be set
* build for current device only, unless pc/release
* comment
* list
* listen for plug in
* add icon to status bar, read params on every frame (?)
* log available devices
* try copy out when loading?
* Revert "log available devices"
This reverts commit e8c52a5d59456d4820ecb13b99a6c46ea1386a20.
* Revert "try copy out when loading?"
This reverts commit 518f403aa03faeda1950fe3dbce0d9e4c1584455.
* don't trigger device probe/caching on modeld prepare
* re-export with ll and road edges
* dont cache devices in manager process
* get USBGPU from params
* no usbgpu env
* missed one
* sconscript don't poll
* unconditional env
* always explicitely set devices on input tensors
* set DEV so amd uses right compiler and iface??
* fix flag
* bump tg
* rm xdg_cache_home
* tg don't bump all the way
* missing gmmu=0 at compile time
* dm set dev
* tg backend
* update gitignore
* missing import
* unused imports
* rely on Device.DEFAULT at compile time (already the case bc onnxrunner)
* comments
* dm warp needs DEV set too
* build both smol and big
* misc typos
* set dev at compile time
* don't need
* DEV=CPU when getting metadata, ensure we don't grab gpu lock
* this would also grab lock
* put bool
* warp compile always prepare only
* missed one
* poll ui
* missing here
* don't force usbgpu at build time
* tmp patch fetch_fw
* catch all, follow hardwared patterns
* simpler
* compile make input queues
* revert this
* group this more readable
* rm empty line
* make dummy frame using numpy
* revert compile make input queues
* no compiler at runtime
* cleanup
* fine to rebuild all on change to device node for now
* fix usbgpu_present
* fix sconscript
* no size in header stream decompress
* DEBUG=2
* minimal viable feedback
* egpu gray
* oops
* gotta do this actually
* modeld build only depends on modeld devices
* don't ship onnx to release? or chunk
* don't need
* can only set compiler on dev=
* none device works, will use default
* make linter happy
* chunk agnostic onnx input to compile_modeld
* chunk big onnx
* +x chunker
* fix #!
* and don't ship chunked onnx to release
* firmware now in correct location
* better err on missing onnx/chunk
* SConscript also need to accept chunked onnx
* metadata also need to load maybe chunked
* dedupe cmd
* this needs to be on cpu
* devices are set in the tgflags, we already depend on them
* rebuilding on changed order is fine
* read file chunked can already load either chunked or not
* chunk all big onnx
* less confusing
* unused import
* python device to load onnx bytes
* default device for runners, python for metadata
* why not
* chunked to shm
* version: a new beginning
* changelog
* singular
* show ours
* actual
* readjust
* updated
* more
* official spelling
* more
* sync
* fix
* send it
* push
* we never had this lol
* syncs
* ci: add validate-test-on-staging-c3 branch to deployment triggers and enhance stable branch handling
* fix long overdue mistake lol
* ci: add condition to wait for start on push events in build workflow
* Fix extra version identifier
* no need for this, i validated what I needed
* only care for release tags, not any
* fix: update versioning logic to use build date and run number for tagging
* fix: update tagging logic and enhance commit message format in build scripts
* fix: refine tagging condition to exclude tag pushes for stable branches
* fix: add extra version identifier to output for better version tracking
* trying to keep things clean and simple
* bugfix
* ci: dynamic submodule check for build_release
* test opendbc diff
* somem fix
* this way
* use path
* use master branch instead
* less verbose
* test bump
* test 1 more sub change
* unbump
* only echo if there's a diff
* Tinygrad bump from sync-20250627
* bump tinygrad_repo
* Reformat metadata generator to match driving_models.json
* bump tinygrad
* Revert "bump tinygrad"
This reverts commit f479dfd502.
* revert me after SP model compiled
* Model recompiled successfully, initiate "revert me after SP model compiled"
This reverts commit 95706eb688.
* The "FillMe" placeholder caused an extra 10 seconds of work
* bump to 22Jul2025
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* Allow more dynamic short names
This should hopefully be future-proof for now.. It's robust enough to return the correct word-digit format (see example on how it generates from given display name below):
'Last Horizon V2 (November 22, 2024)' -> LHV2
'Alabama (November 25, 2024)' -> ALABAMA
'PlayStation (December 03, 2024)' -> PLAYSTAT
'Postal Service (December 09, 2024)' -> PS
'Null Pointer (December 13, 2024)' -> NP
'North America (December 16, 2024)' -> NA
'National Public Radio (December 18, 2024)' -> NPR
'Filet o Fish (March 7, 2025)' -> FOF
'Tomb Raider 2 (April 18, 2025)' -> TR2
'Tomb Raider 3 (April 22, 2025)' -> TR3
'Tomb Raider 4 (April 25, 2025)' -> TR4
'Tomb Raider 5 (April 25, 2025)' -> TR5
'Tomb Raider 6 (April 30, 2025)' -> TR6
'Tomb Raider 7 (May 07, 2025)' -> TR7
'Down to Ride (Revision: May 10, 2025)' -> DTR
'SP Vikander Model (May 16, 2025)' -> SPVM
'VFF Driving (May 15, 2025)' -> VFFD
'Secret Good Openpilot (May 16, 2025)' -> SGO
'Vegetarian Filet o Fish (May 29, 2025)' -> VFOF
'Down To Ride (Revision: May 30, 2025)' -> DTR
'Vegetarian Filet o Fish v2 (June 05, 2025)' -> VFOFV2
'Kerrygold Driving (June 08, 2025)' -> KD
'Tomb Raider 10 (June 16, 2025)' -> TR10
'Organic Kerrygold (June 17, 2025)' -> OK
'Liquid Crystal Driving (June 21, 2025)' -> LCD
'Vegetarian Filet o Fish v3 (June 21, 2025)' -> VFOFV3
'Vibe Model [Custom Model]' -> VMCM
'Tomb Raider 13 (June 27, 2025)' -> TR13
'Aggressive TR (June 28, 2025)' -> ATR
'Tomb Raider 14 (June 30, 2025)' -> TR14
'Cookiemonster Tomb Raider (July 02, 2025)' -> CTR
'Down to Ride (Revision: July 07, 2025)' -> DTR
'Simple Plan Driving (July 07, 2025)' -> SPD
'Down to Ride (Revision: July 08, 2025)' -> DTR
'Tomb Raider 15 (July 09, 2025)' -> TR15
'Tomb Raider 15 rev-2 (July 11, 2025)' -> TR15R2
'Le Tomb Raider 14 (July 14, 2025)' -> LTR14
'Le Tomb Raider 14h (July 17, 2025)' -> LTR14H
'Tomb Raider 16 (July 18, 2025)' -> TR16
'Tomb Raider 16v2 (July 21, 2025)' -> TR16V2
* Update build-all-tinygrad-models.yaml
* Update build-all-tinygrad-models.yaml
* No need to sleep 3 seconds, just send it
* try dynamic
* cleanup
* Update build-single-tinygrad-model.yaml
* bc devtekve said. also, this is repetitive af
* Revert "bc devtekve said. also, this is repetitive af"
This reverts commit 3a0c1562de.
* maybe we could use a script instead that both build all
That both build all and sunnypilot-build-model reference
* refactor: consolidate model building steps into a single workflow
* tweak
* tweakx2
* tweakx3
* tweakx4
* dunno dunno...
* output dir
* lots of changes
* Revert "lots of changes"
This reverts commit 4aadb0ee29.
* fail if all fail
* no inputs needed
* make it easier for us
* note failure and exit 0
* Update build-all-tinygrad-models.yaml
* not needed unless we really want it
* Update build-single-tinygrad-model.yaml
* Merge branch 'sync-20250627-tinygrad' of github.com:sunnypilot/sunnypilot into sync-20250627-tinygrad
* retry for failed ?
* always run this step because sometimes one build fails
which causes the matrix to fail, but most builds still have uploaded artifacts.
* strip
* no escape
* Update build-all-tinygrad-models.yaml
* Test case from terminal run
(openpilot) james@Mac sunnypilot % jq -c '[.bundles[] | select(.runner=="tinygrad") | {ref, display_name: (.display_name | gsub(" \\([^)]*\\)"; "")), is_20hz}]' \
/Users/james/Documents/GitHub/sunnypilot-docs/docs/driving_models_v6.json > matrix.json
mkdir -p output
touch "output/model-Tomb Raider 16v2 (July 21, 2025)-544"
touch "output/model-Space Lab Model (July 24, 2025)-547"
touch "output/model-Space Lab Model v1 (July 24, 2025)-548"
built=(); while IFS= read -r line; do built+=("$line"); done < <(
ls output | sed -E 's/^model-//' | sed -E 's/-[0-9]+$//' | sed -E 's/ \([^)]*\)//' | awk '{gsub(/^ +| +$/, ""); print}'
)
jq -c --argjson built "$(printf '%s\n' "${built[@]}" | jq -R . | jq -s .)" \
'map(select(.display_name as $n | ($built | index($n | gsub("^ +| +$"; "")) | not)))' \
matrix.json > retry_matrix.json
cat retry_matrix.json
[]
(openpilot) james@Mac sunnypilot %
* always
* great success
* add suffix to retry artifact so it doesn't conflict
* retry to get_model too
* and there haha
* unnecessary hyphen
* compare built to missing. include retries
* adjust copy of artifacts.
* Update build-all-tinygrad-models.yaml
* Update model selector versioning and add documentation
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update retry condition for failed models in build-all-tinygrad-models.yaml
* Update build-single-tinygrad-model.yaml
* false
* default none because why not
* red diff? i think?
* meh ... not needed i guess
* error error error
* Nayan is watching... always watching mike wazowski
* string all the way
* lots of retries just in case because im scared
* more robust
* ONLY ONE!!!!!!
* delete.... a lot
* fix artifacts
* fix artifacts
* make sure each is unique :)
* skip files like artifact duhhhh
* artifact name dir
* concurrency
* copy here
* Update build-single-tinygrad-model.yaml
* Update build-single-tinygrad-model.yaml
* bump
* bump tinygrad
* max parallel? if not, i have the other remedy ready in build-all
* revert me!
* I resynced tinygrad woo hoo
* setup shouldnt fail
* pull
* big ole diff
* condition
* Update build-all-tinygrad-models.yaml
* not always() never always() never!!!
* not failure instead of great success
* Update build-all-tinygrad-models.yaml
* yay that worked. lets invoke build-single one last time
* these arent used and are just taking up 250MB space
* really frog?
* bump back to 3
* self-hosted, tici
* rename to trigger tests
* 2 and done
---------
Co-authored-by: DevTekVE <devtekve@gmail.com>
Jason Wen