From ad474e3d061f39ca5d654ba3af6c98376f500085 Mon Sep 17 00:00:00 2001 From: Jaret Burkett Date: Tue, 31 Mar 2026 12:31:05 -0600 Subject: [PATCH] Update and reformat the readme --- README.md | 576 +++++++++++++++++++++++++----------------------------- 1 file changed, 266 insertions(+), 310 deletions(-) diff --git a/README.md b/README.md index e865139..00b3798 100644 --- a/README.md +++ b/README.md @@ -1,237 +1,50 @@ -# AI Toolkit by Ostris +# Ostris AI Toolkit -AI Toolkit is an all in one training suite for diffusion models. I try to support all the latest models on consumer grade hardware. Image and video models. It can be run as a GUI or CLI. It is designed to be easy to use but still have every feature imaginable. - -## Support My Work - -If you enjoy my projects or use them commercially, please consider sponsoring me. Every bit helps! 💖 - -[Sponsor on GitHub](https://github.com/orgs/ostris) | [Support on Patreon](https://www.patreon.com/ostris) | [Donate on PayPal](https://www.paypal.com/donate/?hosted_button_id=9GEFUKC8T9R9W) - -### Current Sponsors - -All of these people / organizations are the ones who selflessly make this project possible. Thank you!! - -_Last updated: 2026-03-03 15:01 UTC_ - -

-a16z -Replicate -Hugging Face -

-
-

-Pixelcut -Weights -josephrocca -Joseph Rocca -Vladimir Sotnikov -clement Delangue -Misch Strotz -RunComfy -

-
-

-nitish PNR -Kristjan Retter -Mohamed Oumoumad -Steve Hanff -Timothy Bielec -Travis Harrington -

-
-

-Infinite -Jimmy Simmons -xv -E2GO -nuliajuk -David Garrido -Randy McEntee -tungsten -L D -squewel -Christopher Williams -IR-Entertainment Ltd -Alastair Green -D G -David Shorey -Paul Kroll -EmmanuelMr18 -Slarti -Jean-Tristan Marin -Armin Behjati -Un Defined -Jürgen Stein -Khoi Nguyen -Al H -Doron Adler -John Dopamine -njgnfhahfnhnwir -The Local Lab -HestoySeghuro . -Jack Blakely -Marc -Neil Murray -Albert Bukoski -Ben Ward -Bharat Prabhakar -Julian Tsependa -Marko jak -the biitz -william tatum -Blanchon -Brian Smith -Kelevra Quackenstien -Nicholas Agranoff -Sapjes -Frank Vance -RalFinger -Dmitry Spikhalsky -Dylan -el Chavo -Gary Joseph -Jake Blakeley -James Thompson -q5sys -Sylvain Fayette -

-
-

-Andrew Park -William Tatum -Daniel Partzsch -ByteC -Ian R -jarrett towe -Marek P -Htango2 -Chris Canterbury -Dutchman5oh -Gergely Madácsi -James -Koray Birand -Mert Guvencli -Michael Carychao -S.Hasan Rizvi -Theta Graphics -Tyssel -michele carlone -Yves Poezevara -Gage Siuniak -Chip Johansen -Chris Dermody -Fredrik Normann Johansen -kingroka -Newtown -PizzaOrNot -Russell Norris -stev -zappazack -Marcus Rass -Rainer Kulow -Stavros Glezakos -Stavros Glezakos -Derrick Schultz -Kukee -Metryman55 -Tri3Ax -Guillaume Roy -Colin Boyd -DavidO -Jadev1311 -Cyril Diagne -Christopher Frey -keonmin lee -Speedy2023 -yvggeniy romanskiy -Ben May | sofsy -Chris Day -Greg Lemons -Ken Finlayson -Kenneth Loebenberg -Le_Fourbe -Manuel2Santos -Mollie -Osman Bayazit -Pablo Fonseca -Phil Ring -renderartist -Rob Stevens -Saftle -Virtamouse -Yi Chen -Joakim Sällström -Craig Penn -Andrey Sorokin -Blane -Brian M -Caleb O'Leary -Cora Nox -Dave Talbott -Dustin Lausch -Greg Richards -Gribbly -Joshua Genke -Tiny Tsuruta -v33ts -Dimitar A. -Heikki Rinkinen -Josh Lindo -Phan Dao -StrictLine e.U. -The Rope Dude -Till Meyer -Valarm, LLC -Valarm, LLC -Xavier Climent -Leo -Adel Gamal -Bennett Waisbren -Brian Buie -Charles Blakemore -claygraffix -David Hooper -Dhruv Sharma -Evan Forster -GameChanger -H.W.Prinz -james salanitri -Jason Briney -Rudolf Goertz -ShinChven ✨ -Tommy Falkowski -Victor-Ray Valdez -Wolkenfels -Jefferderp -John D -ekgreen7 -Markus / Mark -Alex Kovalchuk -Florian Fiegl -Kai Buddensiek -Karol Stępień -manuel landron -Paul Vu Nguyen -R132 -Mal Mallabar -Moritz Hutten -AAYUSH BHADANI -Chris Williams -Dfence -Greg Abousleman -Khaleesi -ProPatte -Boris HANSSEN -Juan Franco -Edwin Bustillos -Fabrizio Pasqualicchio -

- ---- +AI Toolkit is an easy to use all in one training suite for diffusion models. I try to support all the latest models on consumer grade hardware. Image and video models. It can be run as a GUI or CLI. It is designed to be easy to use but still have every feature imaginable. Free and open source. +## Supported Models + +### Image +- [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) (FLUX.1) +- [black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) (FLUX.2) +- [black-forest-labs/FLUX.2-klein-base-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B) (FLUX.2-klein-base-4B) +- [black-forest-labs/FLUX.2-klein-base-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B) (FLUX.2-klein-base-9B) +- [ostris/Flex.1-alpha](https://huggingface.co/ostris/Flex.1-alpha) (Flex.1) +- [ostris/Flex.2-preview](https://huggingface.co/ostris/Flex.2-preview) (Flex.2) +- [lodestones/Chroma1-Base](https://huggingface.co/lodestones/Chroma1-Base) (Chroma) +- [Alpha-VLLM/Lumina-Image-2.0](https://huggingface.co/Alpha-VLLM/Lumina-Image-2.0) (Lumina2) +- [Qwen/Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) (Qwen-Image) +- [Qwen/Qwen-Image-2512](https://huggingface.co/Qwen/Qwen-Image-2512) (Qwen-Image-2512) +- [HiDream-ai/HiDream-I1-Full](https://huggingface.co/HiDream-ai/HiDream-I1-Full) (HiDream) +- [OmniGen2/OmniGen2](https://huggingface.co/OmniGen2/OmniGen2) (OmniGen2) +- [Tongyi-MAI/Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) (Z-Image Turbo) +- [Tongyi-MAI/Z-Image](https://huggingface.co/Tongyi-MAI/Z-Image) (Z-Image) +- [ostris/Z-Image-De-Turbo](https://huggingface.co/ostris/Z-Image-De-Turbo) (Z-Image De-Turbo) +- [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) (SDXL) +- [stable-diffusion-v1-5/stable-diffusion-v1-5](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5) (SD 1.5) + +### Instruction / Edit +- [black-forest-labs/FLUX.1-Kontext-dev](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev) (FLUX.1-Kontext-dev) +- [Qwen/Qwen-Image-Edit](https://huggingface.co/Qwen/Qwen-Image-Edit) (Qwen-Image-Edit) +- [Qwen/Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) (Qwen-Image-Edit-2509) +- [Qwen/Qwen-Image-Edit-2511](https://huggingface.co/Qwen/Qwen-Image-Edit-2511) (Qwen-Image-Edit-2511) +- [HiDream-ai/HiDream-E1-1](https://huggingface.co/HiDream-ai/HiDream-E1-1) (HiDream E1) + +### Video +- [Wan-AI/Wan2.1-T2V-1.3B-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers) (Wan 2.1 1.3B) +- [Wan-AI/Wan2.1-I2V-14B-480P-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-480P-Diffusers) (Wan 2.1 I2V 14B-480P) +- [Wan-AI/Wan2.1-I2V-14B-720P-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P-Diffusers) (Wan 2.1 I2V 14B-720P) +- [Wan-AI/Wan2.1-T2V-14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-T2V-14B-Diffusers) (Wan 2.1 14B) +- [Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers) (Wan 2.2 14B) +- [Wan-AI/Wan2.2-I2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers) (Wan 2.2 I2V 14B) +- [Wan-AI/Wan2.2-TI2V-5B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers) (Wan 2.2 TI2V 5B) +- [Lightricks/LTX-2](https://huggingface.co/Lightricks/LTX-2) (LTX-2) +- [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) (LTX-2.3) + +### Experimental +- [lodestones/Zeta-Chroma](https://huggingface.co/lodestones/Zeta-Chroma) (Zeta Chroma) ## Installation @@ -322,56 +135,6 @@ set AI_TOOLKIT_AUTH=super_secure_password && npm run build_and_start $env:AI_TOOLKIT_AUTH="super_secure_password"; npm run build_and_start ``` - -## FLUX.1 Training - -### Tutorial - -To get started quickly, check out [@araminta_k](https://x.com/araminta_k) tutorial on [Finetuning Flux Dev on a 3090](https://www.youtube.com/watch?v=HzGW_Kyermg) with 24GB VRAM. - - -### Requirements -You currently need a GPU with **at least 24GB of VRAM** to train FLUX.1. If you are using it as your GPU to control -your monitors, you probably need to set the flag `low_vram: true` in the config file under `model:`. This will quantize -the model on CPU and should allow it to train with monitors attached. Users have gotten it to work on Windows with WSL, -but there are some reports of a bug when running on windows natively. -I have only tested on linux for now. This is still extremely experimental -and a lot of quantizing and tricks had to happen to get it to fit on 24GB at all. - -### FLUX.1-dev - -FLUX.1-dev has a non-commercial license. Which means anything you train will inherit the -non-commercial license. It is also a gated model, so you need to accept the license on HF before using it. -Otherwise, this will fail. Here are the required steps to setup a license. - -1. Sign into HF and accept the model access here [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) -2. Make a file named `.env` in the root on this folder -3. [Get a READ key from huggingface](https://huggingface.co/settings/tokens/new?) and add it to the `.env` file like so `HF_TOKEN=your_key_here` - -### FLUX.1-schnell - -FLUX.1-schnell is Apache 2.0. Anything trained on it can be licensed however you want and it does not require a HF_TOKEN to train. -However, it does require a special adapter to train with it, [ostris/FLUX.1-schnell-training-adapter](https://huggingface.co/ostris/FLUX.1-schnell-training-adapter). -It is also highly experimental. For best overall quality, training on FLUX.1-dev is recommended. - -To use it, You just need to add the assistant to the `model` section of your config file like so: - -```yaml - model: - name_or_path: "black-forest-labs/FLUX.1-schnell" - assistant_lora_path: "ostris/FLUX.1-schnell-training-adapter" - is_flux: true - quantize: true -``` - -You also need to adjust your sample steps since schnell does not require as many - -```yaml - sample: - guidance_scale: 1 # schnell does not do guidance - sample_steps: 4 # 1 - 4 works well -``` - ### Training 1. Copy the example config file located at `config/examples/train_lora_flux_24gb.yaml` (`config/examples/train_lora_flux_schnell_24gb.yaml` for schnell) to the `config` folder and rename it to `whatever_you_want.yml` 2. Edit the file following the comments in the file @@ -537,41 +300,234 @@ To learn more about LoKr, read more about it at [KohakuBlueleaf/LyCORIS](https:/ Everything else should work the same including layer targeting. -## Updates +## Support My Work -Only larger updates are listed here. There are usually smaller daily updated that are omitted. +If you enjoy my projects or use them commercially, please consider sponsoring me. Every bit helps! 💖 -### Jul 17, 2025 -- Make it easy to add control images to the samples in the ui +Support my work -### Jul 11, 2025 -- Added better video config settings to the UI for video models. -- Added Wan I2V training to the UI +### Current Sponsors -### June 29, 2025 -- Fixed issue where Kontext forced sizes on sampling +All of these people / organizations are the ones who selflessly make this project possible. Thank you!! -### June 26, 2025 -- Added support for FLUX.1 Kontext training -- added support for instruction dataset training +_Last updated: 2026-03-31 18:10 UTC_ -### June 25, 2025 -- Added support for OmniGen2 training -- -### June 17, 2025 -- Performance optimizations for batch preparation -- Added some docs via a popup for items in the simple ui explaining what settings do. Still a WIP +

+a16z +Replicate +Hugging Face +

+
+

+Pixelcut +Weights +josephrocca +Joseph Rocca +Vladimir Sotnikov +clement Delangue +Misch Strotz +RunComfy +

+
+

+nitish PNR +Mohamed Oumoumad +Steve Hanff +Timothy Bielec +Travis Harrington +Infinite +Jimmy Simmons +

+
+

+xv +nuliajuk +Randy McEntee +David Garrido +E2GO +Alastair Green +D G +L D +Paul Kroll +tungsten +David Shorey +squewel +Christopher Williams +IR-Entertainment Ltd +Alexander Korchemniy +Jürgen Stein +Khoi Nguyen +EmmanuelMr18 +Jean-Tristan Marin +Armin Behjati +Drama Labs GmbH +Slarti +Al H +Doron Adler +njgnfhahfnhnwir +The Local Lab +Marc +HestoySeghuro . +Jack Blakely +John Dopamine +Neil Murray +Albert Bukoski +Ben Ward +Bharat Prabhakar +Julian Tsependa +Brian Smith +Kelevra Quackenstien +Marko jak +Nicholas Agranoff +Sapjes +the biitz +william tatum +Blanchon +el Chavo +James Thompson +Htango2 +Frank Vance +RalFinger +Andrew Park +Dmitry Spikhalsky +Dylan +Gary Joseph +Jake Blakeley +q5sys +Sylvain Fayette +Tri3Ax +William Tatum +

+
+

+Daniel Partzsch +ByteC +jarrett towe +Marek P +Chris Canterbury +Dutchman5oh +Gergely Madácsi +Ian R +James +Koray Birand +S.Hasan Rizvi +Theta Graphics +Tyssel +Chris Dermody +kingroka +Mert Guvencli +Newtown +Russell Norris +stev +Gage Siuniak +Derrick Schultz +Gribbly +Kukee +Metryman55 +zappazack +Guillaume Roy +Colin Boyd +DavidO +Jadev1311 +Cyril Diagne +Speedy2023 +Karl Brewer +Marcus Rass +Rainer Kulow +Stavros Glezakos +Stavros Glezakos +Geno Machino +Greg Lemons +Ken Finlayson +Kenneth Loebenberg +Le_Fourbe +Manuel2Santos +Osman Bayazit +Patrick Gallagher +Phil Ring +Rob113 +Saftle +Virtamouse +Yi Chen +Craig Penn +Christopher Frey +keonmin lee +yvggeniy romanskiy +Andrey Sorokin +Blane +Brian M +Cora Nox +Dave Talbott +Greg Richards +Joshua Genke +Mollie +Pablo Fonseca +renderartist +Tiny Tsuruta +v33ts +Joakim Sällström +Leo +Bennett Waisbren +Brian Buie +Caleb O'Leary +Dustin Lausch +GameChanger +H.W.Prinz +james salanitri +Jason Briney +Wolkenfels +John D +Dimitar A. +R132 +Heikki Rinkinen +Josh Lindo +Michael Styne +Michael Styne +Phan Dao +StrictLine e.U. +The Rope Dude +Till Meyer +Valarm, LLC +Valarm, LLC +Mal Mallabar +Xavier Climent +Moritz Hutten +AAYUSH BHADANI +Adel Gamal +Charles Blakemore +Chris Williams +claygraffix +David Hooper +Dfence +Evan Forster +Greg Abousleman +Jean-Paul Lerault +Rudolf Goertz +ShinChven ✨ +Tommy Falkowski +Victor-Ray Valdez +Jefferderp +ekgreen7 +Markus / Mark +Alex Kovalchuk +Florian Fiegl +Kai Buddensiek +Karol Stępień +manuel landron +Paul Vu Nguyen +Jamie Colpitts +Alchemist +Andrew Gould +Jeroen Van Harten +Kevin Metz +Patreon2000 +Paul Bergen +ProPatte +That's Ridiculous +Boris HANSSEN +Juan Franco +Fabrizio Pasqualicchio +

-### June 16, 2025 -- Hide control images in the UI when viewing datasets -- WIP on mean flow loss +--- -### June 12, 2025 -- Fixed issue that resulted in blank captions in the dataloader - -### June 10, 2025 -- Decided to keep track up updates in the readme -- Added support for SDXL in the UI -- Added support for SD 1.5 in the UI -- Fixed UI Wan 2.1 14b name bug -- Added support for for conv training in the UI for models that support it \ No newline at end of file