Jaret Burkett c78b1404e3 Deepen offload prefetch pipeline with per-slot events ...

Replace the 2-slot ping-pong + single global "compute-started" event
with a depth-N ring buffer where each transfer waits only on the slot
it's reusing (D layers back) instead of the most-recent compute. Applies
to forward and backward, Linear and Conv. Depth is tunable via
AI_TOOLKIT_OFFLOAD_DEPTH (default 4).

Bit-exact vs non-offload (output, grad_input, weight grads). No speedup
on a bandwidth-bound PCIe link (already saturated at depth 2), but the
cleaner per-slot design removes the fragile shared-event serialization
and lets deeper prefetch help on faster buses.

2026-06-07 16:07:13 -06:00

..

audio

Add support for ACE-Step 1.5 and ACE-Step 1.5 XL. Also added dataset captioning through the UI. (#785)

2026-04-09 15:02:03 -06:00

data_transfer_object

Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.

2026-06-06 08:32:24 -06:00

keymaps

Fixed issue with grad scaling

2024-07-20 08:21:57 -06:00

memory_management

Deepen offload prefetch pipeline with per-slot events

2026-06-07 16:07:13 -06:00

models

Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts

2026-06-04 13:00:23 -06:00

optimizers

Added Automagic v3

2026-06-07 12:06:42 -06:00

orig_configs

Added refiner fine tuning. Works, but needs some polish.

2023-11-05 17:15:03 -07:00

samplers

fix custom_flowmatch_sampler (#783)

2026-04-13 09:42:05 -06:00

timestep_weighing

Added new timestep weighing strategy

2025-06-04 01:16:02 -06:00

util

Add Mac OS support for Apple Silicon (#770)

2026-03-30 09:37:47 -06:00

__init__.py

Initial commit

2023-07-05 16:44:58 -06:00

accelerator.py

Fallback unwrapping logic if fails

2025-05-21 03:10:33 -06:00

advanced_prompt_embeds.py

Add support for fine-tuning Hidream O1 (#831)

2026-05-12 11:15:16 -06:00

assistant_lora.py

fixed huge flux training bug. Added ability to use an assistatn lora

2024-08-14 10:14:13 -06:00

basic.py

Add Mac OS support for Apple Silicon (#770)

2026-03-30 09:37:47 -06:00

buckets.py

Fixed issue with new bucket scaler

2026-05-28 11:34:16 -06:00

civitai.py

Added ability to use civit ai url ar model name and built a model downloader and cache manager for it

2023-08-13 13:09:51 -06:00

clip_vision_adapter.py

Varous bug fixes

2024-07-10 15:20:04 -06:00

config_modules.py

Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.

2026-06-06 08:32:24 -06:00

config.py

Added specialized scaler training to ip adapters

2024-04-05 08:17:09 -06:00

control_generator.py

Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders.

2026-05-20 08:56:16 -06:00

cuda_malloc.py

Memory optimizations. Default to using cudamalloc when torch 2.0 for mem allocation

2023-09-12 04:30:23 -06:00

custom_adapter.py

Prep for future breaking changes in newer versions of transformers library

2026-04-10 12:04:32 -06:00

data_loader.py

Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.

2026-06-06 08:32:24 -06:00

dataloader_mixins.py

Added ability to set the caption extention in dataset viewer, captioner, and trainer so one dataset can have multiple caption styles in different files with different extensions. Added dataset caption template for a blank ideogram 4 formatted template.

2026-06-06 08:32:24 -06:00

dequantize.py

Improvements for full tuning flux. Added debugging launch config for vscode

2024-10-29 04:54:08 -06:00

ema.py

Increase ema feedback amount. Normalize the dfe 4 image embeds

2025-06-10 08:01:13 -06:00

embedding.py

Work on ipadapters and custom adapters

2024-05-13 06:37:54 -06:00

esrgan_utils.py

Added training for a custom version of ERSGAN arcitecture. Testing training now

2023-08-07 18:04:23 -06:00

extension.py

Added support for finetuning OmniGen2.

2025-06-25 13:58:16 -06:00

guidance.py

Add model hooks to polarity loss

2025-04-17 09:00:10 -06:00

image_utils.py

Adjust dataloader tester to handle videos to test them

2025-10-21 14:47:23 -06:00

inversion_utils.py

Various bug fixes, wip stuff, and tweaks

2023-11-02 18:19:20 -06:00

ip_adapter.py

Prep for future breaking changes in newer versions of transformers library

2026-04-10 12:04:32 -06:00

job.py

Moved the run job command

2023-08-13 10:25:56 -06:00

kohya_lora.py

Removed all submodules. Submodule free now, yay.

2025-04-18 10:39:15 -06:00

kohya_model_util.py

Hotfix to handle latest transformers clip model missing key suddenly

2023-08-02 12:26:18 -06:00

layers.py

Complete reqork of how slider training works and optimized it to hell. Can run entire algorythm in 1 batch now with less VRAM consumption than a quarter of it used to take

2023-08-05 18:46:08 -06:00

llvae.py

Bug fixes. Added some functionality to help with private extensions

2023-10-05 07:09:34 -06:00

logging_aitk.py

Remove future steps from loss log if resuming from an earlier step

2026-05-18 09:23:55 -06:00

lora_special.py

Add LTX-2 Support (#644)

2026-01-13 04:55:30 -07:00

lorm.py

Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use)

2024-11-20 09:16:55 -07:00

losses.py

Small updates and bug fixes for various things

2025-06-03 20:08:35 -06:00

lycoris_special.py

Improved lorm extraction and training

2023-10-28 08:21:59 -06:00

lycoris_utils.py

Updated module names to be compatable with ney koyha unet

2023-07-31 13:16:29 -06:00

metadata.py

Many bug fixes. Ip adapter bug fixes. Added noise to unconditional, it works better. added an ilora adapter for 1 shotting LoRAs

2024-01-28 08:20:03 -07:00

network_mixins.py

Add method to do continuious lora merging in for low vram full finetuning.

2026-02-26 09:00:41 -07:00

optimizer.py

Added Automagic v3

2026-06-07 12:06:42 -06:00

paths.py

Add sapiens2 matting as a mask generator. Begin transition to model paths and model folders.

2026-05-20 08:56:16 -06:00

photomaker_pipeline.py

Initial training script for photomaker training. Needs a little more work.

2024-01-15 18:46:26 -07:00

photomaker.py

Added an experimental clip fusion model that is showing promise for embedding concepts

2024-01-17 13:13:04 -07:00

pipelines.py

Change inpainting mask to zero out on latents instead of image for inpaint area.

2025-03-24 14:16:52 -06:00

pixel_shuffle_encoder.py

Pixel shuffle adapter. Some bug fixes thrown in

2025-03-29 21:15:01 -06:00

print.py

Fixed issue on some systems where Logger didnt have atty

2026-04-16 21:09:52 +00:00

progress_bar.py

implemented device placement preset system more places. Vastly improved speed on setting network multiplier and activating network. Fixed timing issues on progress bar

2023-09-14 08:31:54 -06:00

prompt_utils.py

Made AdvancedPromptEmbeds that is compatable with previous PromptEmbeds functionality, but is more streamlined and can accomidate more model embedding paradigms.

2026-04-11 10:45:47 -06:00

reference_adapter.py

Prep for future breaking changes in newer versions of transformers library

2026-04-10 12:04:32 -06:00

resampler.py

Added a clip vision adapter trainer. Only works for sd15 for now

2023-12-24 13:26:04 -07:00

sampler.py

WIP on mean flow loss. Still a WIP.

2025-06-12 08:00:51 -06:00

saving.py

Added training for pixart-a

2024-02-13 16:00:04 -07:00

scheduler.py

Modal cloud training support, fixed typo in toolkit/scheduler.py, Schnell training support for Colab, issue #92 , issue #114 (#115)

2024-08-22 21:25:44 -06:00

sd_device_states_presets.py

Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications

2024-12-15 08:59:27 -07:00

stable_diffusion_model.py

Improved the prompt handeling of ideogram4 model. Now used advanced prompts class to store them smaller and allow longer prompts

2026-06-04 13:00:23 -06:00

style.py

Bug fixes and little improvements here and there.

2024-06-08 06:24:20 -06:00

timer.py

Show the console log on the job overview in the ui.

2025-03-21 11:45:36 -06:00

train_pipelines.py

Information trainer

2023-07-28 08:16:29 -06:00

train_tools.py

Added initial support for f-lite model

2025-05-01 11:15:18 -06:00

unloader.py

Fix text encoder offload bug when caching embeddings (#868)

2026-06-07 14:09:32 -06:00