16dda06a0c
Reapply chunker ( #37292 )
...
* Reapply chunker
* good size
* rm glob
* cleaner
* back to 45mb
* warp need not be fixed
* add manifest path
* lil cleaner
2026-02-23 16:49:48 -08:00
d6af0e6eb5
Revert "Simpler file chunker ( #37276 )"
...
This reverts commit b27fa58444
2026-02-20 16:43:43 -08:00
b27fa58444
Simpler file chunker ( #37276 )
...
* Chunk tinygrad pkl below GitHub max size
* pull that out
* rm glob
* make work
* Single name def
* unused comment
* more cleanups
* revert that
* 10MB overhead
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
2026-02-20 10:37:14 -08:00
2ba6df2506
chunk tinygrad pkl below GitHub max size - NoCache and AlwaysBuild ( #37194 )
...
* nocache
* +
* fixes
* lint
* not split
* use pathlib
* cleanup
* better
* even better
2026-02-13 10:14:24 -08:00
af1583cdfc
Reapply tgwarp w NV12 fix ( #37168 )
...
* Revert "Revert tgwarp again (#37161 )"
This reverts commit 45099e7fcd
2026-02-12 08:59:19 -08:00
45099e7fcd
Revert tgwarp again ( #37161 )
...
* Reapply "revert tg calib and opencl cleanup (#37113 )" (#37115 )
This reverts commit 667f3bb32f
2026-02-10 23:12:41 -08:00
3d11e8ef36
Revert "Chunk big model files ( #37134 )" ( #37139 )
...
This reverts commit a941e8f78f
2026-02-09 20:58:22 -08:00
73f720220b
modeld: simplify model run processing ( #37138 )
...
Hi! The point of this pr is to make the model run easier to read. On the latest tinygrad numpy().flatten() empirically does the same thing as the internal contiguous().realize().uop.base.buffer.numpy(). numpy() is also documented (docstrings), which can assist new contributors in learning what each potential execution does. Torq_boi or yassine, I know you want proof in the code base, so here it is. As of tinygrad commit 2f55005:
in tinygrad_repo/tinygrad/tensor.py
Lines 316-318 (def _buffer): ensure the tenso is contiguous() and realized() before accessing the raw buffer.
Line 378 (def numpy): Wraps the buffer access and adds a reshape to match the tensor shape.
self._buffer() is what executes contiguous().realize() and returns the buffer object.
Calling numpy() on that buffer object returns a 1D array (defined in tinygrad/device.py:193 via np.frombuffer).
The reshape(self.shape) at the end of Tensor.numpy() then adds dimensions to that 1D array. The added .flatten() removes those dimensions, flattening it back to a 1D array. Effectively the same as what is currently done, but less complex.
2026-02-09 20:24:25 -08:00
a941e8f78f
Chunk big model files ( #37134 )
...
* file chunking
* try this
* more cleanup
* cleaner
2026-02-09 15:29:50 -08:00
667f3bb32f
Revert "revert tg calib and opencl cleanup ( #37113 )" ( #37115 )
...
* Revert "revert tg calib and opencl cleanup (#37113 )"
This reverts commit 51312afd3d
2026-02-07 21:36:44 -08:00
51312afd3d
revert tg calib and opencl cleanup ( #37113 )
...
* Revert "Remove all the OpenCL (#37105 )"
This reverts commit d5cbb89d844ce701150a#36621 )"
This reverts commit 593c3a0c8e
2026-02-07 09:10:29 -08:00
d5cbb89d84
Remove all the OpenCL ( #37105 )
...
* Remove all the OpenCL
* lil more
* bump msgq
2026-02-06 16:36:47 -08:00
593c3a0c8e
Calibrate in tg ( #36621 )
...
* squash
* bump tg
* fix linmt
* Ready to merge
* cleaner
* match modeld
* more dead stuff
2026-02-06 14:13:46 -08:00
1398bdb10e
dmonitoringmodeld: follow same pattern as modeld ( #36636 )
...
* dmonitoringmodeld: follow same pattern as modeld
* lint
* oops
* rename
2025-11-19 09:23:13 -08:00
b778da1d7c
dmonitoringmodeld: clean up data structures ( #36624 )
...
* update onnx
* get meta
* start
* cast
* deprecate notready
* more
* line too long
* 2
2025-11-14 14:29:04 -08:00
d24a14cb39
DM: Large Donut model 🍩 ( #36198 )
...
* 59cfd731-6f80-4857-9271-10d952165079/225
* deprecate at the end
2025-09-30 20:32:19 -07:00
b6e0d4807a
[bot] Update Python packages ( #36184 )
...
* Update Python packages
* not available anymore
* also this
* also this
* maybe?
* version
* try
* Revert "version"
This reverts commit 9ac4401b9ca59677b82736faff8baf66861df5f2.
* revert
* cffi
* issue
* comment
---------
Co-authored-by: Vehicle Researcher <user@comma.ai >
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
2025-09-20 20:10:51 -07:00
35ed6bc3a9
Tinygrad DEV=DEVICE ( #35814 )
...
* Reapply "Tinygrad DEV=DEVICE (#35809 )"
This reverts commit 5e07636d54
2025-07-26 21:21:25 -07:00
5e07636d54
Revert "Tinygrad DEV=DEVICE ( #35809 )"
...
This reverts commit 47f23828d2
2025-07-25 12:54:11 -07:00
47f23828d2
Tinygrad DEV=DEVICE ( #35809 )
...
* bump tg
* step one cleanup
* cleanup
* typo
* cleaner
* cleaner
* Revert "cleaner"
This reverts commit 9c1abd0dc06b4564e61dd32b0e93375badbc9ca5.
* usbgpu
* bit cleaner
* cleaner sconscript
2025-07-25 11:53:08 -07:00
7e0d3c22f4
modeld: skip redundant cast, reshape, and flatten ( #35735 )
...
* skip redundant cast, reshape, and flatten
* accidentally deleted a newline lol
2025-07-22 16:35:19 -07:00
6f1a1b3213
Revert "modeld: autodetect tinygrad backend" ( #35701 )
...
Revert "modeld: autodetect tinygrad backend (#35405 )"
This reverts commit ce92fd1a0f
2025-07-12 00:52:18 -07:00
ce92fd1a0f
modeld: autodetect tinygrad backend ( #35405 )
...
* modeld: autodetect tinygrad backend
* modeld: autodetect tinygrad CUDA backend
* Revert "modeld: autodetect tinygrad CUDA backend"
This reverts commit 0e9755fb3c5c2021de27f4d230bd0a162883bc37.
* comment why llvm@19
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
* backend from jit
* fix static analysis
* simplify
* compile flags log
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
2025-07-11 19:48:35 -07:00
a8ec08e5bb
modeld: remove redundant process config ( #35639 )
2025-07-05 13:52:57 -07:00
e47d105b97
revert extra GPU power draw to prep for release
2025-05-20 14:28:30 -07:00
2382f1cab0
Run DM model twice, prep for bigger models ( #34894 )
...
* twice
* swap
* adjust
2025-03-24 15:54:42 -07:00
d41311ca20
Switch PC running to tinygrad with LLVM ( #34660 )
...
* try llvm on cpu
* update lock
* Make sure test fails
* With BEAM=3 its fast
* small fixes
* more fixes
* Fix macos
* do dm too
* rm ort
2025-02-24 14:51:10 -08:00
d98a554b17
tici: set core affinity for all realtime processes ( #34574 )
2025-02-11 21:20:31 -08:00
02ec9e5bf8
dmonitoringmodeld: clean exit ( #34454 )
...
* nice exit
* correct spacing
* sentry
2025-01-22 20:39:52 -08:00
17ca6389e1
Tinygrad runner ( #34261 )
...
* squash
* dmonitoringmodeld: use cl transform (#34235 )
* needs cleanup
* only if tici
* bump tinygrad
* check width
* base modelframe
* .
* need to be args
* more cleanup
* no _frame in base
* tici only
* its DrivingModelFrame
* .6 is fair
---------
Co-authored-by: Comma Device <device@comma.ai >
* Update tinygrad
* tg upstream
* bump tg
* bump tg
* debug
* attr
* misc cleanup
* whitespace
* remove
* Add TODOs to make python proc for modelrunners
* whitespace
---------
Co-authored-by: ZwX1616 <zwx1616@gmail.com >
Co-authored-by: Comma Device <device@comma.ai >
Co-authored-by: Maxime Desroches <desroches.maxime@gmail.com >
2024-12-18 11:58:59 -08:00
7352e612a2
dmonitoringmodeld: use cl transform without tinygrad ( #34266 )
...
* merge
* why
* self.buf_size
* 0.05 more than with tg due to copy
---------
Co-authored-by: Comma Device <device@comma.ai >
2024-12-16 16:29:06 -08:00
f6885dcbec
Revert Tinygrad ( #34243 )
...
* Revert "dmonitoringmodeld: use cl transform (#34235 )"
This reverts commit 684b0b9d4d31606a7d1544f58ff758#34171 )"
This reverts commit 7b5a4fbb03
2024-12-14 12:15:36 -08:00
684b0b9d4d
dmonitoringmodeld: use cl transform ( #34235 )
...
* needs cleanup
* only if tici
* bump tinygrad
* check width
* base modelframe
* .
* need to be args
* more cleanup
* no _frame in base
* tici only
* its DrivingModelFrame
* .6 is fair
---------
Co-authored-by: Comma Device <device@comma.ai >
2024-12-14 00:44:15 -08:00
7b5a4fbb03
Tinygrad runner ( #34171 )
...
* squash
* bump tg
* bump tg
* debump tinygrad
* bump tinygrad
* bump tg
* Skip init iteration
* fixes
* cleanups
* skip first test sample
* typos
* linter unhappy
* update cpu usage
* OPENCL just zeros for now
* imports
* Try printing
* Runs again, but slower
* unused import
* Allow more buffer with tg and all on gpu
* bump tinygrad
* seems ok
* stricter timings for driving looser for dm
* try llvm
* check nvidia
* More timeout for now
* make test pass
* Revert "try llvm"
This reverts commit ef136e478320101fea262bae3579e558da991902.
* small fixes
* whitespace
* revert test timeout
* No model runners
* Always CPU always fast
* No onnx runtime GPU
* more cores
* cleanup
* Is this faster
* Is this faster
* at least runs
* FP32 is faster than 16
* fix deps
* whitespace
* comment
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
2024-12-11 23:15:20 -08:00
1ec2c56b4e
gc old DM init param
2024-12-03 14:59:28 -08:00
d9d57e5d6f
Revert ISP image processing + tinygrad bump ( #34020 )
...
* Revert "Replace ThneedModel with TinygradModel (#33532 )"
This reverts commit da952e9b64#33959 )"
This reverts commit f2a1cce42b
2024-11-13 19:27:11 -08:00
da952e9b64
Replace ThneedModel with TinygradModel ( #33532 )
...
* squash
* bump tg
* bump tg
* debump tinygrad
* bump tinygrad
* bump tg
* Skip init iteration
* fixes
* cleanups
* skip first test sample
* typos
* linter unhappy
* update cpu usage
* OPENCL just zeros for now
* imports
* Try printing
* Runs again, but slower
* unused import
* Allow more buffer with tg and all on gpu
* bump tinygrad
---------
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com >
Co-authored-by: Bruce Wayne <harald.the.engineer@gmail.com >
2024-11-11 13:32:21 -08:00
a4848ceee9
Add driverStateV2 plots to model replay ( #33967 )
2024-11-07 21:49:15 -08:00
876f192112
exec DM model with gpu ( #33609 )
...
* half
old-commit-hash: 9f72eca003d4637ca7fe847ebaf925c694fc2e84
* optimed
old-commit-hash: 6e36e2a12e09275ec21d1590012a92b05ca52ff5
* thneed
old-commit-hash: 419a06c09c0c767d828bcd1e118bc575898c343a
* exec
old-commit-hash: 0059c27ec11b076a37f65d604ed135ea6541b1a6
* runner
old-commit-hash: 34232ada94450ce541eaef546197fa219810a891
* runs but
old-commit-hash: 3db37c00b6a64908293b4de8d8b56e80308cd8f2
* it is 01
old-commit-hash: a160d81eb1a7e77abbef959b44f602610f68f665
* np
old-commit-hash: c1caff6ba648cc2c0094c71b2ea074f01c3c2dc8
* module url
old-commit-hash: 6f4902c4d384263a53e2c1d14d93b5ff864b6a5f
* new
old-commit-hash: 779ae79b1bc3df6374fb6663ac8592e107a6e504
* ds fast
* is this work
* corcention
* real timing
* no reg
* interim gather
* 0e4a9c7b
* fa69be01, and halve
* list
* cleanup
* slighly faster
* setprotlt
* expected
* replay ref
* more powar
* reluctantly
* bump tg
* 8
* less
* less
* bump tg
* better than exp
* closer
* cc
* see diff
* commits
* was right
* to 32 cast
* remove dlc file
* support both
* dspExecutionTime -> gpuExecutionTime
* ignore
* time ref
* ref commit
* last
---------
Co-authored-by: Comma Device <device@comma.ai >
2024-09-26 16:40:44 -07:00
251e2e9400
Clip sigmoid overlfow ( #33635 )
...
* No more C sigmoid
* Clipping
* Need to cast to float
* Update ref
* Clip
* Model refg
2024-09-23 19:06:52 -07:00
ffb6e11f96
Revert "exec DM model with gpu" ( #33604 )
...
Revert "exec DM model with gpu (#33397 )"
This reverts commit f79aca8e1e
2024-09-19 20:55:11 -07:00
f79aca8e1e
exec DM model with gpu ( #33397 )
...
* half
old-commit-hash: 9f72eca003d4637ca7fe847ebaf925c694fc2e84
* optimed
old-commit-hash: 6e36e2a12e09275ec21d1590012a92b05ca52ff5
* thneed
old-commit-hash: 419a06c09c0c767d828bcd1e118bc575898c343a
* exec
old-commit-hash: 0059c27ec11b076a37f65d604ed135ea6541b1a6
* runner
old-commit-hash: 34232ada94450ce541eaef546197fa219810a891
* runs but
old-commit-hash: 3db37c00b6a64908293b4de8d8b56e80308cd8f2
* it is 01
old-commit-hash: a160d81eb1a7e77abbef959b44f602610f68f665
* np
old-commit-hash: c1caff6ba648cc2c0094c71b2ea074f01c3c2dc8
* module url
old-commit-hash: 6f4902c4d384263a53e2c1d14d93b5ff864b6a5f
* new
old-commit-hash: 779ae79b1bc3df6374fb6663ac8592e107a6e504
* ds fast
* is this work
* corcention
* real timing
* no reg
* interim gather
* 0e4a9c7b
* fa69be01, and halve
* list
* cleanup
* slighly faster
* setprotlt
* expected
* replay ref
* more powar
* reluctantly
* bump tg
* 8
* less
* less
* bump tg
* better than exp
* closer
* cc
* see diff
* commits
* was right
* to 32 cast
* remove dlc file
* support both
---------
Co-authored-by: Comma Device <device@comma.ai >
2024-09-19 17:42:44 -07:00
f870a968e9
Split cereal into cereal/msgq ( #32631 )
...
* squash
* fix doc
* compile device
* compile device
* Update ref
* add msgq to precommit exclusions
* No service ports
* fix compile
* address comments
* More comments
* Delete logger
* Update opendbc
* Linting
* bump msgq
old-commit-hash: e71ec4e8d7e39a56235f590227e7cb361cd8b79d
2024-06-06 14:31:56 -07:00
9bd90112d0
use pyupgrade to update to new typing syntax ( #31580 )
...
* add pyupgrade hook
* run pyupgrade (pre-commit run -a)
* ruff --fix
* Revert "add pyupgrade hook"
This reverts commit 56ec18bb6b8602a0b612f3803d96cdad14b52066.
* revert changes to third_party/
* manual type fixes
* explicit Optional wrapping capnp objects
old-commit-hash: 995250ae4967943ee33a3699a1d89a7a770d95e9
2024-02-24 16:41:23 -08:00
59eb2988de
swaglog.py goes in common/ ( #30631 )
...
* swaglog.py goes in common/
* all of these go in release
* we'll even include the pyx
old-commit-hash: 35f819c8239b8ac3a000087a0f791b91ccea5e5e
2023-12-06 17:27:51 -08:00
1cc4f1a634
new msgs default to invalid ( #30587 )
...
* new msgs default to invalid
* fix lm
* set more valid
* update rest
* update refs
* fix logMessage
* more valids
* cleanup
* fix llk test
* pigeond is also valid
* more valids
old-commit-hash: a1d36961cf9fa0c6a9c41fe92d6504de5d36cbaa
2023-12-03 10:50:17 -08:00
65fc9f14b3
Update model input names to match onnx inputs ( #29855 )
...
Updated model input names to match onnx inputs
old-commit-hash: 2d69f3d63e5462e8b095fab097c4aff00bd1c83a
2023-09-08 16:06:16 -07:00
5732002296
Rewrite dmonitoringmodeld in python ( #29740 )
...
* Added dmonitoringmodeld.py
* Removed dmonitoringmodeld.cc
* Use ModelRunner helpers from runners/__init__.py
* Fixed DriverStateResult field ordering
* Some bug fixes
* Set calib input
* Look ma, no loop!
* Bump dmonitoringmodeld cpu usage in test_onroad
* Fixed memory leak caused by np.ctypes.data_as
* Formatting fixes
* chmod +x
* remove USE_ONNX_MODEL
* Realtime priority 1, formatting fixes
old-commit-hash: 503fa121ee83469763e9386b9e5df7a733693e2c
2023-09-07 19:46:43 -07:00
Harald Schäfer