Fix FLUX2 Klein load-time VRAM spikes on low-memory GPUs. (#726)

Keep the transformer and Qwen text encoder off CUDA during initial load/quantization in low-VRAM mode so model startup avoids full-model OOM before offloading and quantization can take effect.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com>
This commit is contained in:
M. Hofer
2026-04-01 17:36:55 +02:00
committed by GitHub
parent da2a79590f
commit f213e3b1e5
2 changed files with 14 additions and 9 deletions

View File

@@ -44,15 +44,14 @@ class Flux2KleinModel(Flux2Model):
self.flux2_klein_te_path,
torch_dtype=dtype,
)
text_encoder.to(self.device_torch, dtype=dtype)
flush()
if self.model_config.quantize_te:
self.print_and_status_update("Quantizing Qwen3")
quantize(text_encoder, weights=get_qtype(self.model_config.qtype))
quantize(text_encoder, weights=get_qtype(self.model_config.qtype_te))
freeze(text_encoder)
flush()
elif not self.model_config.low_vram:
text_encoder.to(self.device_torch, dtype=dtype)
flush()
if (
self.model_config.layer_offloading