wozeparrot
f11f63007d
llama: immediate scaling on flag ( #16494 )
2026-06-04 10:30:00 -07:00
chenyu
0faaf6df26
remove kwargs from arange and linspace [PR] ( #16505 )
...
it used to have requires_grad and device, now both are removed
2026-06-04 10:32:37 -04:00
qazal
3b1a5f9770
llama: a_bT and aT_b bf16 gemms ( #16487 )
...
* hk_bf16_gemm
* enable in 8b
* cleanups
* rename to USE_HK_BF16_GEMM
* work
* work
* work
* work
* change the gemms
* work
* work
* set as default
* work
* change
2026-06-04 23:30:21 +09:00
wozeparrot
fd13080636
deviceless const skip axis check ( #16496 )
2026-06-03 19:13:20 -07:00
wozeparrot
7dcfd144b6
llama: columnwise fp8 scaling ( #16480 )
2026-06-02 18:55:45 -07:00
wozeparrot
aab9a5a8a3
llama: allow specifying layer count ( #16464 )
2026-06-01 15:36:04 -07:00
wozeparrot
6795c2d5c9
llama: zero grad this way ( #16445 )
2026-05-29 20:25:21 -07:00
wozeparrot
c23652e486
llama: minimize peak init mem ( #16440 )
2026-05-29 18:00:37 -07:00
wozeparrot
6787de9f52
llama: fix mp ( #16434 )
2026-05-29 11:21:43 -07:00
wozeparrot
f86966af56
llama: optim amax margin ( #16425 )
2026-05-28 20:18:11 -07:00
wozeparrot
36c8ff70c1
llama: use old scale for dequant in optim ( #16417 )
2026-05-28 15:21:19 -07:00
George Hotz
edca5df25a
flip offset and shape in pad and shrink ( #16414 )
...
* flip offset and shape in pad and shrink
* dumb test
2026-05-28 11:58:19 -07:00
wozeparrot
dac3743d75
llama: delayed scaling in optim ( #16407 )
2026-05-27 15:40:03 -07:00
George Hotz
8ee3a37524
shrink/pad use (new_shape, offset) ( #16405 )
...
* shrink uses offset and shape
* pad does too
* fix
2026-05-27 15:13:08 -07:00
wozeparrot
3a7a6da7d5
llama: fakedata uses real vocab size ( #16389 )
2026-05-26 18:58:55 -07:00
qazal
b73d2d17b9
viz/cli: add --interval ( #16363 )
...
* interval support
* add test_interval
* llama uses interval
2026-05-26 03:35:06 +09:00
wozeparrot
68d2102fd2
llama: offload master weights ( #16355 )
2026-05-25 08:48:13 -07:00
Christopher Milan
c2d06570a5
remove getenv(CI) from core tinygrad ( #16326 )
2026-05-21 22:20:33 -04:00
chenyu
31424cda71
Tensor.requires_grad -> is_param ( #16325 )
...
for optimizer
2026-05-21 19:39:57 -04:00
wozeparrot
0c41317a59
llama: update 405b scripts ( #16309 )
2026-05-21 14:03:34 -07:00
wozeparrot
fb718a5e9d
llama: realize amax ( #16308 )
2026-05-21 14:00:48 -07:00
wozeparrot
afc5bfa183
llama: remove fused grad accum ( #16301 )
2026-05-21 09:38:40 -07:00
qazal
e33e058d34
set SPLIT_W13=0 for 8b DP by default ( #16302 )
2026-05-21 22:09:10 +09:00
Christopher Milan
172f9493e1
move is_dtype_supported to renderer ( #16226 )
2026-05-20 21:19:37 -04:00
George Hotz
58d58c1659
remove DEVECTORIZE ( #16290 )
...
* remove DEVECTORIZE
* fully remove DEVECTORIZE
2026-05-20 13:25:49 -07:00
wozeparrot
825f30bf18
llama: apply_grad saves memory ( #16275 )
2026-05-20 13:14:06 -07:00
chenyu
7af7b6703a
relax policy ASSERT_MIN_STEP_TIME to 3.2 ( #16273 )
2026-05-19 22:29:09 -04:00
wozeparrot
361553c0a8
llama: match flat_llama with model_train ( #16269 )
2026-05-19 17:25:56 -07:00
George Hotz
da7414d6dc
fix RUN_PICKLE and test it ( #16272 )
...
* add test for openpilot RUN_PICKLE
* fix RUN_PICKLE and test it
2026-05-19 17:00:25 -07:00
wozeparrot
a3d59faef6
llama: don't save weight ( #16252 )
2026-05-18 17:05:45 -07:00
qazal
18b102f355
llama: also use 7.1 comgr, update startup_walltime.sh ( #16253 )
2026-05-19 08:59:02 +09:00
qazal
98b8a2b407
llama: use hipcc 7.1 version ( #16250 )
2026-05-19 08:09:57 +09:00
chenyu
dcee90aa3f
remove requires_grad use in extra/examples ( #16238 )
...
except the ones fed into optimizer
2026-05-16 18:40:26 -04:00
qazal
ebcb7b7cc0
fp8 gemm tests with scale args ( #16231 )
...
* update atol
* update fp8 path
* more work
* update profile.sh
2026-05-16 20:47:58 +09:00
wozeparrot
159694347e
llama: fix running flat_llama ( #16224 )
2026-05-15 20:16:48 -07:00
George Hotz
2549b14ec2
fix caformer onnx run ( #16222 )
2026-05-15 15:08:36 -07:00
chenyu
07a172dbbb
remove noop requires_grad_ calls ( #16213 )
2026-05-15 13:31:10 -04:00
chenyu
409bb0c9ad
requires_grad cannot be None ( #16212 )
...
final goal is to remove requires_grad, first change the default to True, and don't allow None
2026-05-15 02:01:04 -04:00
wozeparrot
b4d267dfd4
llama: only save when small ( #16208 )
2026-05-14 17:46:29 -07:00
wozeparrot
88ac2ac1fd
llama: cleanups ( #16189 )
2026-05-13 17:08:06 -07:00
wozeparrot
e97f2c1114
llama: only gemm + fa custom kernel ( #16180 )
...
* llama: tie store to grad directly
* llama: set mp flags
* llama: non fused grad fp8 quantize path
2026-05-12 21:03:49 -07:00
wozeparrot
e9359d9e7d
more llama mp fixes ( #16151 )
...
* llama: SPLIT_W13
* llama: fix with no fused kernels
* llama: cast to bf16 on non asm_gemm patH
* llama: new mp flags
2026-05-11 21:29:23 -07:00
wozeparrot
026688f03f
llama: move to correct dir ( #16118 )
2026-05-08 19:42:16 -07:00
chenyu
072db9924c
div to mixin ( #16078 )
...
also deleted idiv method
2026-05-07 12:52:37 -04:00
qazal
a9a87ad8fd
viz/cli: less flags ( #16076 )
...
* viz/cli: merge -s and -i flags
* only -t
* merge parser
* fix
2026-05-08 00:22:40 +09:00
wozeparrot
730fa66bf3
llama speed 6 ( #16071 )
2026-05-06 20:51:03 -07:00
wozeparrot
ab6218bc92
llama mp fixes ( #16050 )
2026-05-05 15:35:32 -07:00
wozeparrot
528d35e306
llama speed 4 ( #15993 )
2026-04-30 17:14:41 -07:00
wozeparrot
0080489abe
llama: use env vars ( #15978 )
2026-04-29 12:37:15 -07:00
wozeparrot
ef09071073
llama: speed 2 ( #15960 )
2026-04-28 20:44:37 -07:00
