github/ai-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
1,441 Commits 9 Branches 0 Tags
4900e5e866dcba3317d6c8d3a71361bb0901ef93
Commit Graph

461 Commits

Author SHA1 Message Date
Jaret Burkett
6ea281973d Add support for MiniMax H3 Ref2Vid training 2026-08-13 12:36:03 -06:00
Jaret Burkett
ab18528fdb Let omni captioner handle just images as well. Set it as the new default captioner. 2026-08-13 10:30:17 -06:00
Jaret Burkett
4e91fb2d0a Add thinking and abliterated versions of qwen omni. 2026-08-13 08:56:31 -06:00
Jaret Burkett
6c88e3d138 Add layer offloading to omni captioner 2026-08-13 07:21:54 -06:00
Jaret Burkett
ca42a72f4c Fixed issue with flash attention on omni captioner 2026-08-12 21:14:23 -06:00
Jaret Burkett
7f9a142dfd Generate thumbnails for dataset items so things load faster and the ui is more stable. Especially good for videos. 2026-08-12 20:33:42 -06:00
Jaret Burkett
175cc1e151 Add Qwen 3 Omni for captioning videos with sound. 2026-08-12 19:57:01 -06:00
Jaret Burkett
4b00b61257 When doing an audio loss, show the img and audio loss in the loss log 2026-08-12 18:58:54 -06:00
Jaret Burkett
a1ddeeef13 Fixed issue with offloading text encoder on ltx 2.5 2026-08-12 10:46:58 -06:00
Jaret Burkett
cbf910ac02 Add support for LTX 2.5 2026-08-12 05:50:30 -06:00
Jaret Burkett
924c426675 Resolve the new ltx 2.5 path subfolder for ltx 2.5. Sill checking compatability. 2026-08-11 13:30:17 -06:00
Jaret Burkett
21dc65972d Only add a blank control if the model has to have it, for unconditionals. Previously it always encoded a blank control on unconditional if the model could take one. Affects DOP and blank prompt preservations. 2026-08-11 08:25:23 -06:00
Jaret Burkett
8d4beedd04 Move guidance loss target selection when given a range before preservation so it is avaliable during preservation. 2026-08-11 06:57:33 -06:00
Jaret Burkett
257da9b586 Rework DOP so it works with caching text embeddings 2026-08-09 22:13:49 -06:00
Jaret Burkett
f4e9130547 Fix race condition that can corrupt grads under certain conditions. 2026-08-07 14:52:47 -06:00
Jaret Burkett
817f3dcbcb Fix finite check on inverted masked prior 2026-08-07 09:49:12 -06:00
Jaret Burkett
685ce37a8d Adjust sample timestep sigmas to be model evals for h3 for 1 extra step 2026-08-06 19:54:04 -06:00
Jaret Burkett
b811636ae4 Check for finite vs isnan on loss before backpropigating. to catch infinity overflows 2026-08-06 09:18:56 -06:00
Jaret Burkett
edacd406b3 Handle minimax loading of non pruned model 2026-08-06 09:18:00 -06:00
Jaret Burkett
7309db4d74 Fix layer offloaded adaln projection layer move 2026-08-06 08:10:20 -06:00
Jaret Burkett
139a38f5bd Upcast adaln pruned layers to fp32 on h3 to prevent overflow. 2026-08-06 07:51:15 -06:00
Jaret Burkett
1e1418b22c Apply sigma to contrastive guidance to balance loss better. Prevent noise grads on images/non audio datasets. Prep for training adapters on MiniMax H3 2026-08-05 13:27:12 -06:00
Jaret Burkett
3afa270ab5 Fix audio losses for DOP and other preservation losses 2026-08-04 22:08:48 -06:00
Jaret Burkett
0f9094db95 Fix audio loss when doing do_guidance_loss 2026-08-04 21:43:31 -06:00
Jaret Burkett
d20a17c10e Limit max tokens to 512. Allow override with model kwargs. 2026-08-04 07:55:21 -06:00
Jaret Burkett
602306da77 Add gradient checkpointing to vae 2026-08-04 07:54:51 -06:00
Jaret Burkett
546eb7daff Look for existing models in folders recursivly 2026-08-03 15:39:55 -06:00
Jaret Burkett
d3a3f70a2a Speed up quantization processing on H3 2026-08-03 15:21:39 -06:00
Jaret Burkett
88ac27fc8f Handle images with MiniMax H3. 2026-08-03 11:58:44 -06:00
Jaret Burkett
bf739ff966 Fix issue with layer offloading with MinMax H3 2026-08-03 11:32:36 -06:00
Jaret Burkett
8502a845b1 Add support for MiniMax H3 T2V and I2V training 2026-08-03 10:17:39 -06:00
Jaret Burkett
aa762103b3 Add build support for Nvidia Spark 2026-07-28 12:27:51 -06:00
Jaret Burkett
461e798708 Rework windows start and stop methods so that command windows dont appear. Stop with signal since we cannot signint. 2026-07-27 17:43:01 -06:00
Jaret Burkett
fb204b7677 Fixed for DFEs with pixelspace and video models 2026-07-27 11:12:02 -06:00
Jaret Burkett
92bdb6e473 Add support for Mage-Flow and Mage-Flow Edit 2026-07-25 11:34:20 -06:00
Jaret Burkett
e8573dad34 Fix trailing progress bar print when stopping a job in the ui 2026-07-22 10:11:08 -06:00
Jaret Burkett
479c72ada2 Add replacing triggers on prompts when caching text encoder 2026-07-19 09:03:05 -06:00
Jaret Burkett
f1bc6508ad Fix issue with qwen image edit models. 2026-07-17 09:14:04 -06:00
Jaret Burkett
988d891102 Added a Sample Next Step in the job gear dropdown to force a sample on the next step. 2026-07-17 07:59:30 -06:00
Jaret Burkett
5cb54ba9cc Allow setting weight saving flag on hidream_o1 2026-07-16 07:40:25 -06:00
DasPauluteli
a92f18bf71 krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend (#933) 
* krea2: don't hardcode NVIDIA-only cuDNN SDPA backend

The krea2 attention() forced SDPBackend.CUDNN_ATTENTION, which is
NVIDIA-only. On non-NVIDIA backends (AMD ROCm, Intel XPU, Apple MPS)
every forward pass fails with 'RuntimeError: No available kernel.
Aborting execution.', so Krea 2 LoRA training cannot run at all there.

Pass a priority list [CUDNN, FLASH, EFFICIENT, MATH] instead. NVIDIA
still selects cuDNN; other backends fall back to flash/efficient/math.
Verified training end-to-end on an AMD Radeon 8060S (gfx1151, ROCm 7.2).

* Version bump

* Add set priority flag so CUDNN_ATTENTION is selected on cuda devices first.

---------

Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
 2026-07-15 12:44:34 -06:00
Jaret Burkett
b8f8a08ba4 Fix sampling bar with anima 2026-07-15 12:17:06 -06:00
rmatif
3e6bd874c4 feat: Add Anima support (#860) 
* Add Anima training support

* Update Anima modular training

* Use sample guidance for Anima

* Fix Anima sampling

* Limit Anima LoRA targets

* Convert Anima LoRA exports

* Fix Anima local loading

* Update Anima default model

* Pin upstream Anima diffusers

* Adjust template defaults to be consistent with other models. Update README

---------

Co-authored-by: Jaret Burkett (Ostris) <jaretburkett@gmail.com>
 2026-07-15 11:59:01 -06:00
fatalis
8bbd051667 Add sample_start_step setting to configure when sampling starts (#949) 
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
 2026-07-15 11:15:50 -06:00
Jaret Burkett
30162c0602 Improvements for captioner quantization to speed it up. Block compile on captioners. 2026-07-15 10:25:47 -06:00
Jaret Burkett
691ddf434e Add Qwen3.6 VL captioner. 2026-07-15 07:02:33 -06:00
Jaret Burkett
abba6b5845 Show better errors on captioner 2026-07-14 07:19:03 -06:00
Jaret Burkett
28f2c0acbe Move z_image over to the new modeling class 2026-07-13 17:12:30 -06:00
Jaret Burkett
7602e476eb Exclude sensative layers from quantization in krea 2026-07-13 10:14:53 -06:00
Jaret Burkett
ad07b06de5 Use cached first frame for wan22_5 model 2026-07-10 10:21:34 -06:00