Jaret Burkett
bbaef7852a
Do not materalize weights on ostris quantizer when getting state dict. Require dequantization of weight like other quantization methods.
2026-07-16 08:55:51 -06:00
Jaret Burkett
e28727d5cb
Made a fused GEMV kernel for convrot unpacking to increase speed further. Fix bug in test script that made train time add additional grads to bf16.
2026-07-15 10:24:52 -06:00
Jaret Burkett
933ca1c517
Apply w8a8 on the forward when training with convrot for more speed. Fix issue where quantizing a model had a pin memory leak.
2026-07-11 10:53:50 -06:00
Jaret Burkett
5a9b5bde3f
Added experimental orbit quant
2026-07-09 15:14:51 -06:00
Jaret Burkett
5f04ae7ad5
Rework merge_network_on_save to handle dequantization on merging and saving for much more efficient full finetuning.
2026-06-25 13:19:38 -06:00
Jaret Burkett
c8cd78b1a4
Allow nested transformer block names for quantization, lora targeting, quantizing
2026-06-13 14:33:16 -06:00
Jaret Burkett
171535833a
Add Mac OS support for Apple Silicon ( #770 )
...
* Made an install script and auto updates env for mac
* GPU sensors and initial training working for MAC. Still WIP.
* Switch dataloader to single threaded until I can work around some mac pickeling issues.
* Get quantization working on mac
* Fix mac exclusive imports so they don't break other builds.
* Add mac instructions to the UI
2026-03-30 09:37:47 -06:00
Jaret Burkett
55b8b0e23e
Fix issue where ARA was not working when using memory manager
2025-10-07 13:39:44 -06:00
Jaret Burkett
4e5707854f
Initial support for RamTorch. Still a WIP
2025-10-05 13:03:26 -06:00
Jaret Burkett
ea01a1c7d0
Fixed a bug where samples would fail if merging in lora on sampling for unquantized models. Quantize non ARA modules as uint8 when using an ARA
2025-08-25 09:21:40 -06:00
Jaret Burkett
e12bb21780
Quantize blocks sequentialls without a ARA
2025-08-14 09:59:58 -06:00
Jaret Burkett
77b10d884d
Add support for training with an accuracy recovery adapter with qwen image
2025-08-12 08:21:36 -06:00
Jaret Burkett
9da8b5408e
Initial but untested support for qwen_image
2025-08-04 13:29:37 -06:00
Jaret Burkett
f5aa4232fa
Added ability to quantize with torchao
2025-03-20 16:28:54 -06:00
Jaret Burkett
25341c4613
Got wan 14b training to work on 24GB card.
2025-03-07 17:04:10 -07:00
Jaret Burkett
4fe33f51c1
Fix issue with picking layers for quantization, adjust layers fo better quantization of cogview4
2025-03-05 13:44:40 -07:00
