github/ai-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
1,222 Commits 9 Branches 0 Tags
e20b42e84aeecbefd8cae565fa3b51216bf23d39
Commit Graph

671 Commits

Author SHA1 Message Date
Jaret Burkett
af594061ab Add ability to do hidden states with tipsv2 2026-06-22 11:00:04 -06:00
Jaret Burkett
820d534d6e Add features for models that may need a non masked loss such as inpainiting. 2026-06-22 10:59:17 -06:00
Jaret Burkett
c133c55cf5 Fix issue with mask generator revision 2026-06-22 08:19:27 -06:00
Jaret Burkett
d51463ca52 Add ability to recover from a truncated image file in the dataset 2026-06-22 05:07:30 -06:00
Jaret Burkett
ba0b3dbb65 Force batch size when bucket is too small by duplicating items in the batch 2026-06-21 20:01:29 -06:00
Jaret Burkett
99f8fd44e3 Add a fallback bbox adjustment when json parsing fails on ideogram4 captioner 2026-06-19 07:25:54 -06:00
Jaret Burkett
60c1ac6a50 Add support for Boogu Image and Boogu Image Edit 2026-06-18 15:05:49 -06:00
Jaret Burkett
178eb5fbbe Add unconditional lora support so Ideogram 4 inference will more closely resemble the full pipeline results. I pushed a finetuned unconditional lora to the hub as an adapter. 2026-06-16 13:27:43 -06:00
Jaret Burkett
86b19589a0 Update the Ideogram 4 prompt generation/parsing/ui to handle the updated format notes better. 2026-06-16 09:44:38 -06:00
Jaret Burkett
c730d64478 Added a flag to keep loading the image when latents are cached. Useful for DFE and other methods that target pixelspace losses. 2026-06-15 05:31:48 -06:00
Jaret Burkett
c8cd78b1a4 Allow nested transformer block names for quantization, lora targeting, quantizing 2026-06-13 14:33:16 -06:00
Jaret Burkett
2f91db8363 Defauly to compiling full graph to false 2026-06-13 07:28:51 -06:00
PlagueKind
e908d85f5e Allow quantized unet offload compile and force fullgraph false (#881) 2026-06-13 07:27:28 -06:00
Jaret Burkett
d83f7dd4d9 Fix a few issues with compile. Changed defaults. Future proofed block layer compile. 2026-06-12 11:43:43 -06:00
PlagueKind
6b1f89f30b Enhanced torch.compile System with Block-Level Compilation and Unified Whole-Model Fallback (#866) 
* Add block-level compile and qcompile torch.compile whole model  modes

* Update torch compile system
 2026-06-12 10:35:12 -06:00
Jaret Burkett
55ce6570f2 Automagic 3 rework. Stable in my testing. 2026-06-12 07:52:44 -06:00
Jaret Burkett
01b6a9806b Another complete rework of automagic3. Added a decay to the LR spread to the mean to prevent LRs fighting with eachother 2026-06-09 12:08:45 -06:00
Jaret Burkett
acb1548722 Updated the comments and doc for Automagic v3 2026-06-09 07:05:48 -06:00
Jaret Burkett
a1ac6e8b01 Reworked automagic v3 again. Seems more stable. Still testing. 2026-06-08 22:03:54 -06:00
Jaret Burkett
5d6887fd98 Major updates to automagic3 optimizer. Seems to be functioning more ideally and naturally decays, as it should. 2026-06-08 14:09:02 -06:00
Jaret Burkett
e281df70dd Allow automagic3 to run unfused. Add some clipping. 2026-06-08 08:59:06 -06:00
Jaret Burkett
c78b1404e3 Deepen offload prefetch pipeline with per-slot events 
Replace the 2-slot ping-pong + single global "compute-started" event
with a depth-N ring buffer where each transfer waits only on the slot
it's reusing (D layers back) instead of the most-recent compute. Applies
to forward and backward, Linear and Conv. Depth is tunable via
AI_TOOLKIT_OFFLOAD_DEPTH (default 4).

Bit-exact vs non-offload (output, grad_input, weight grads). No speedup
on a bandwidth-bound PCIe link (already saturated at depth 2), but the
cleaner per-slot design removes the fragile shared-event serialization
and lets deeper prefetch help on faster buses.
 2026-06-07 16:07:13 -06:00
Jaret Burkett
cdff6e36aa Pin inner stores of torachao to speed up layer offloading for quantized models around 25% 2026-06-07 15:52:01 -06:00
Jaret Burkett
75781fb5a5 Fix float8 weights not offloading to CPU in layer offloading 2026-06-07 15:36:17 -06:00
PlagueKind
7c1a76f336 Fix text encoder offload bug when caching embeddings (#868) 2026-06-07 14:09:32 -06:00
Jaret Burkett
1dc9a797cf Added Automagic v3 2026-06-07 12:06:42 -06:00
Jaret Burkett
41157b460c Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template. 2026-06-06 08:32:24 -06:00
Jaret Burkett
6a8d9333b6 Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts 2026-06-04 13:00:23 -06:00
Jaret Burkett
d9f26c2f87 Add gradient checkpointing to tipsv2 heads 2026-06-01 05:00:15 -06:00
Jaret Burkett
e5439509b5 Added pure lpips dfe 2026-05-31 11:52:03 -06:00
Jaret Burkett
87bac27513 Fixed issue with new bucket scaler 2026-05-28 11:34:16 -06:00
Jaret Burkett
3e86d81fc6 Adjust bucket sizes to achieve maximum pixels without going over. 2026-05-28 09:36:17 -06:00
Jaret Burkett
15082cfb8a Round buckets for divisibility instead of always rounding down. 2026-05-28 09:08:21 -06:00
Jaret Burkett
c9264bdd0b Reworked the bucketing system to precisly match model specific divisibility. The old SDXL bucket sizes needed to go. 2026-05-28 08:35:58 -06:00
Jaret Burkett
2aa60e4ca5 Update default agreement threshold for automagic v2 to be 0.5 2026-05-26 07:57:54 -06:00
Jaret Burkett
12304e170f Added some experimental loss targets 2026-05-24 14:13:23 -06:00
Jaret Burkett
aecd554128 Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders. 2026-05-20 08:56:16 -06:00
Jaret Burkett
b42acb988f Remove future steps from loss log if resuming from an earlier step 2026-05-18 09:23:55 -06:00
Jaret Burkett
4bfe944792 Scale dfe 7 with velocity_equiv_weight 2026-05-13 09:06:59 -06:00
Jaret Burkett (Ostris)
fc4d6ebf39 Add support for fine-tuning Hidream O1 (#831) 
* Initial support for hidream. Lora keys likely need work

* Fix saving for hidream-o1

* Remove dependence on flash attention for hidream o1

* Fix gradient checkpointing for hidream o1

* A lot of fixes for hidream. Handle loading and saving as comfy model.

* Omit layers not used in comfy. Fix issue with lora loading keys in comfy

* Version bumpo
 2026-05-12 11:15:16 -06:00
Jaret Burkett
f38de2a2fe Add tipsv2 locally and fix gradient checkpointing for it 2026-05-10 14:47:44 -06:00
Jaret Burkett
a12ddd72a1 Change the velocity weight cap on dfe 9 2026-05-07 07:37:05 -06:00
Jaret Burkett
6bb8acbffc Add agreement_threshold default of 0.6 to automagic 2 2026-05-05 19:13:00 -06:00
Jaret Burkett
963a9f42b2 Add decode latent to wan 2.1 models. Add gradinet checkpointing to wan vae. 2026-05-05 11:30:16 -06:00
Jaret Burkett
4260a3c5b6 Add optimizer test suite and make minor speed adjustments to Automagicv2 2026-05-05 10:02:30 -06:00
Jaret Burkett
aeca7fe404 Add Automagic v2 optimizer. It uses significantly less vram and is much more efficient. 2026-05-05 09:09:07 -06:00
Jaret Burkett
af6458d1b5 Enable caching of ACE step latents. 2026-04-28 13:39:20 -06:00
Jaret Burkett
acc6a36214 Scale DFE 9 to a velocity equiv weight to match flow matching gradient strength. Probably need to rework all DFEs to do this as the math checks out. 2026-04-28 09:10:02 -06:00
Jaret Burkett
1fc4ad3979 Add sapiens2 as a diffusion feature extractor 2026-04-27 15:59:03 -06:00
Jaret Burkett
f4445cd78c Added psuedo_huber loss 2026-04-19 09:51:46 -06:00