Jaret Burkett
bbaef7852a
Do not materalize weights on ostris quantizer when getting state dict. Require dequantization of weight like other quantization methods.
2026-07-16 08:55:51 -06:00
Jaret Burkett
e28727d5cb
Made a fused GEMV kernel for convrot unpacking to increase speed further. Fix bug in test script that made train time add additional grads to bf16.
2026-07-15 10:24:52 -06:00
Jaret Burkett
cf0db39ede
Fix some errors for specific quants. Enable saving quantizations.
2026-07-14 07:25:09 -06:00
Jaret Burkett
28b05ee4ed
Added convrotbitnet 1.58 bit quantization
2026-07-13 07:52:48 -06:00
Jaret Burkett
b6ff367633
Convrot improvements. Add argitrary bit packed byte quantizations.
2026-07-13 06:33:54 -06:00
Jaret Burkett
095d6e7418
Add convrot4 and convrot8 quantizations to the ui. A lot of improvements to the convrot quantization methods.
2026-07-11 13:49:41 -06:00
Jaret Burkett
933ca1c517
Apply w8a8 on the forward when training with convrot for more speed. Fix issue where quantizing a model had a pin memory leak.
2026-07-11 10:53:50 -06:00
Jaret Burkett
065ac27353
Fix issue with compiling convrot on some models
2026-07-11 09:26:12 -06:00
Jaret Burkett
96a3a06111
Added initial experimental support for convrot4 and convrot8 quantization.
2026-07-10 11:01:14 -06:00
Jaret Burkett
6fac83d068
Fix casting issue with orbit quant
2026-07-10 10:23:38 -06:00
Jaret Burkett
5a9b5bde3f
Added experimental orbit quant
2026-07-09 15:14:51 -06:00
Jaret Burkett
5f04ae7ad5
Rework merge_network_on_save to handle dequantization on merging and saving for much more efficient full finetuning.
2026-06-25 13:19:38 -06:00
Jaret Burkett
c8cd78b1a4
Allow nested transformer block names for quantization, lora targeting, quantizing
2026-06-13 14:33:16 -06:00
Jaret Burkett
171535833a
Add Mac OS support for Apple Silicon ( #770 )
...
* Made an install script and auto updates env for mac
* GPU sensors and initial training working for MAC. Still WIP.
* Switch dataloader to single threaded until I can work around some mac pickeling issues.
* Get quantization working on mac
* Fix mac exclusive imports so they don't break other builds.
* Add mac instructions to the UI
2026-03-30 09:37:47 -06:00
Jaret Burkett
55b8b0e23e
Fix issue where ARA was not working when using memory manager
2025-10-07 13:39:44 -06:00
Jaret Burkett
4e5707854f
Initial support for RamTorch. Still a WIP
2025-10-05 13:03:26 -06:00
Jaret Burkett
f74475161e
Add stepped loss type
2025-09-22 15:50:12 -06:00
Jaret Burkett
ea01a1c7d0
Fixed a bug where samples would fail if merging in lora on sampling for unquantized models. Quantize non ARA modules as uint8 when using an ARA
2025-08-25 09:21:40 -06:00
Jaret Burkett
e12bb21780
Quantize blocks sequentialls without a ARA
2025-08-14 09:59:58 -06:00
Jaret Burkett
77b10d884d
Add support for training with an accuracy recovery adapter with qwen image
2025-08-12 08:21:36 -06:00
Jaret Burkett
9da8b5408e
Initial but untested support for qwen_image
2025-08-04 13:29:37 -06:00
Jaret Burkett
5890e67a46
Various bug fixes
2025-04-29 09:30:33 -06:00
Jaret Burkett
88b3fbae37
Various experiments and minor bug fixes for edge cases
2025-04-25 13:44:38 -06:00
Jaret Burkett
bfe29e2151
Removed all submodules. Submodule free now, yay.
2025-04-18 10:39:15 -06:00
Jaret Burkett
5f312cd46b
Remove ip adapter submodule
2025-04-18 09:59:42 -06:00
Jaret Burkett
4a43589666
Use a shuffled embedding as unconditional for i2v adapter
2025-04-11 10:44:43 -06:00
Jaret Burkett
059155174a
Added mask diffirential mask dialation for flex2. Handle video for the i2v adapter
2025-04-10 11:50:01 -06:00
Jaret Burkett
a8680c75eb
Added initial support for finetuning wan i2v WIP
2025-04-07 20:34:38 -06:00
Jaret Burkett
5365200da1
Added ability to add models to finetune as plugins. Also added flux2 new arch via that method.
2025-03-27 16:07:00 -06:00
Jaret Burkett
ce4c5291a0
Added experimental wavelet loss
2025-03-26 18:11:23 -06:00
Jaret Burkett
4595965e06
Added an inpainting mask generator for training inpainting if inpaint mask is not provided
2025-03-25 12:16:10 -06:00
Jaret Burkett
f5aa4232fa
Added ability to quantize with torchao
2025-03-20 16:28:54 -06:00
Jaret Burkett
25341c4613
Got wan 14b training to work on 24GB card.
2025-03-07 17:04:10 -07:00
Jaret Burkett
4fe33f51c1
Fix issue with picking layers for quantization, adjust layers fo better quantization of cogview4
2025-03-05 13:44:40 -07:00
Jaret Burkett
6f6fb90812
Added cogview4. Loss still needs work.
2025-03-04 18:43:52 -07:00
Jaret Burkett
acc79956aa
WIP create new class to add new models more easily
2025-03-01 13:49:02 -07:00
Jaret Burkett
58f9d01c2b
Added adafactor implementation that handles stochastic rounding of update and accumulation
2024-10-30 05:25:57 -06:00
Jaret Burkett
5d47244c57
Added support for pixart sigma loras
2024-06-16 11:56:30 -06:00
Jaret Burkett
3f3636b788
Bug fixes and little improvements here and there.
2024-06-08 06:24:20 -06:00
Jaret Burkett
5a70b7f38d
Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded.
2024-04-20 10:46:56 -06:00
Jaret Burkett
427847ac4c
Small tweaks and fixes for specialized ip adapter training
2024-03-26 11:35:26 -06:00
Jaret Burkett
b01e8d889a
Added stochastic rounding to adafactor. ILora adjustments
2024-03-05 07:07:09 -07:00
