github/ai-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
1,199 Commits 9 Branches 0 Tags
c730d644785cdca4bd69c4894406fac47da1aaf1
Commit Graph

377 Commits

Author SHA1 Message Date
Jaret Burkett
c0d600b5d6 Allow using flash backend for ideogram 2026-06-13 15:17:20 -06:00
Jaret Burkett
c8cd78b1a4 Allow nested transformer block names for quantization, lora targeting, quantizing 2026-06-13 14:33:16 -06:00
Jaret Burkett
6c3b82696e Add support for PRX Pixel T2I 2026-06-13 09:47:53 -06:00
Jaret Burkett
a01c83073a Added an example model with docs so people and agents can add models easier. 2026-06-13 08:17:06 -06:00
Jaret Burkett
0f580f0663 Add compiling option for captioning. Added progress bar for captioning on active job widget. Track steps on caption. Added reverse proxy iframe that plugins can use to add ui functionality. 2026-06-12 10:26:52 -06:00
Jaret Burkett
88127557f5 Comment out logging supression that was keeping weight loadings from being shown with transformers library 2026-06-11 07:53:41 -06:00
Jaret Burkett
cac3815b2c Make DOP run in a single backward pass, should be faster and more stable. Show dop loss on ui 2026-06-08 13:55:47 -06:00
Jaret Burkett
35588726de Fixed issue where a buffer was stuck on cpu when offloading ideogram4 2026-06-07 13:31:03 -06:00
Jaret Burkett
8968e41234 Set ideogram to use new loke saving format 2026-06-07 09:49:57 -06:00
Jaret Burkett
c994398850 Rework prompts and captioning systems for ideogram to more strictly match the format provided by ideogram. 2026-06-05 11:14:36 -06:00
Jaret Burkett
ef12260b80 Add a prompt upsample ui for upsampling prompts to ideogram format prompts. 2026-06-05 09:56:02 -06:00
Jaret Burkett
edcc7415d1 Added an autocaptioner for Ideogram 4 captions 2026-06-04 13:04:23 -06:00
Jaret Burkett
6a8d9333b6 Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts 2026-06-04 13:00:23 -06:00
Jaret Burkett
63b3181262 Added experimental support for Ideogram 4 2026-06-04 09:03:00 -06:00
Jaret Burkett
5e84bf0d0b Fixed issue with hidream-01 that could cause a weird nan state. Took forever to track down as it was 1 in 10 starts. 2026-05-28 12:48:24 -06:00
Jaret Burkett
ef57c1077c Change z image divisibility 2026-05-28 09:09:36 -06:00
Jaret Burkett
68e9b38220 Add ability to trigger a save from the ui which whill make the trainer save on the next step 2026-05-26 08:38:49 -06:00
Jaret Burkett
7354def271 Added decode latent method to qwen image model 2026-05-25 10:02:09 -06:00
Jaret Burkett
fbac1cb7f5 Dont force flash attention on hidream 01. Causes random issues and is slower. 2026-05-24 16:05:22 -06:00
Jaret Burkett
12304e170f Added some experimental loss targets 2026-05-24 14:13:23 -06:00
Jaret Burkett
644a6f9246 Fix device casting for zimage in some instances 2026-05-23 10:37:33 -06:00
Jaret Burkett
c6ecc03ccd Fixed saving full model of z_image l2p for finetuning 2026-05-23 07:36:24 -06:00
Jaret Burkett
6102370df9 Add support for ZImage L2P 2026-05-22 14:58:28 -06:00
Jaret Burkett
aecd554128 Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders. 2026-05-20 08:56:16 -06:00
Jaret Burkett
ec58dcde92 Add better gradient checkpointing to dfes 2026-05-18 09:25:23 -06:00
Jaret Burkett
e03c6e4dc9 Fix potential inconsistency with different attention mentods in hidream01 2026-05-13 09:08:11 -06:00
Jaret Burkett (Ostris)
fc4d6ebf39 Add support for fine-tuning Hidream O1 (#831) 
* Initial support for hidream. Lora keys likely need work

* Fix saving for hidream-o1

* Remove dependence on flash attention for hidream o1

* Fix gradient checkpointing for hidream o1

* A lot of fixes for hidream. Handle loading and saving as comfy model.

* Omit layers not used in comfy. Fix issue with lora loading keys in comfy

* Version bumpo
 2026-05-12 11:15:16 -06:00
Jaret Burkett
e9ab387dfd Fixed issue with qwen image edit models when using multiple control images when not caching text embeddings. 2026-04-30 11:59:12 +00:00
Jaret Burkett
deb409085a Made it possible to use the flux 2 small decoder VAE when setting the vae_path manually for flux2 models 2026-04-30 05:42:31 -06:00
Jaret Burkett
7ccec8ec2c Add checkpointing and a proper decode for flux 2 VAEs so they can be used with DFE 2026-04-30 04:27:13 -06:00
Jaret Burkett
af6458d1b5 Enable caching of ACE step latents. 2026-04-28 13:39:20 -06:00
Jaret Burkett
f972b750e6 Performance improvements to captioner. Add ability to set a default caption for ace captioner to avoid needing to caption the audio so we only transcribe 2026-04-28 10:15:11 -06:00
Jaret Burkett
1fc4ad3979 Add sapiens2 as a diffusion feature extractor 2026-04-27 15:59:03 -06:00
Jaret Burkett
67d67f8c1d Ignore hidden files when captioning 2026-04-25 17:38:31 -06:00
Jaret Burkett
998a02f30e Yet another pseudo_huber fix. Still drinking coffee. 2026-04-19 10:05:40 -06:00
Jaret Burkett
fc85410c9a Fix issue with precision on pseudo_huber loss 2026-04-19 10:02:27 -06:00
Jaret Burkett
20a99258b8 Fix issue is is else on pseudo_huber loss 2026-04-19 09:59:18 -06:00
Jaret Burkett
f4445cd78c Added psuedo_huber loss 2026-04-19 09:51:46 -06:00
Jaret Burkett
beb40ae29b Add DFE8 with partial step 2026-04-17 17:40:16 -06:00
Jaret Burkett
7c4f18ce51 Fix ernie unpatchify 2026-04-17 12:03:39 -06:00
Jaret Burkett
8cb9649382 Add decode latent to ernie pipe 2026-04-17 12:01:09 -06:00
Jaret Burkett
a513a1583e Fixed issue where Qwen VL MOE captioner produced nonsense 2026-04-16 21:24:43 +00:00
Jaret Burkett
ab1ee4df34 Hotfix some issues with Wan models caused by diffusers and transformers updates 2026-04-16 20:53:50 +00:00
Jaret Burkett
afb62b1fa5 Add support for Nucleus-Image 2026-04-16 13:09:10 -06:00
Jaret Burkett
dd7074a21f Fix issue with layer offloading on ernie 2026-04-14 19:19:02 -06:00
Jaret Burkett
97d8c05d75 Fix issue with captioner sometimes outputting not utf-8 characters 2026-04-14 14:37:21 -06:00
Jaret Burkett (Ostris)
3e0c904054 Add support for Baidu's ERNIE-Image (#793) 
* Add support for ERNIE Image

* change float64 to float32

* Version bump

* Update ERNIE defaults
 2026-04-14 09:45:12 -06:00
Jaret Burkett
233e292256 Added some experimental low step things for zeta 2026-04-13 09:37:34 -06:00
Jaret Burkett
78cf049c29 Add support for ACE-Step 1.5 and ACE-Step 1.5 XL. Also added dataset captioning through the UI. (#785) 
* Base ace step 1.5 xl added. Generating, still wip on training and ui

* Base training code done

* Fix some issues with caching text embeddings. Update sample cards to show audio

* Fix issue with quantizing ace step

* Add album artwork to samples with waveform.

* Cleanup logs

* Add album art endpoint to speed up album art loading

* Made an make video with artwork script

* Make ui handle basic audio models. Make multi line adjustments to the editor and better syntax hilighting.

* Add prompt tagging system for special tagged models.

* prompt tagging processing for ui working.

* Moved default samples to a special file so we can add more when needed and they can be adjusted for a specific model

* Add a captioner job with music captioner that is prepped for use with the ui

* Add basit ui setup for captioning modal and handeling captioning jobs

* Starting captioning job from ui working. Still better management for it.

* Better filtering of job options in the job view for captioning jobs

* Added qwen3 vl as a captioner for images

* Have an indicator when a dataset is being captioned.

* Adjust the way caption jobs look in the queue

* Fix a few issues. Adjust defaults.

* Version bump

* Added ace step to the readme.
 2026-04-09 15:02:03 -06:00
Jaret Burkett
9ca58e9aa2 Fixed offload and quantize order of ltx 2.3 text encoder. 2026-04-07 15:11:50 -06:00