Jaret Burkett
|
cdff6e36aa
|
Pin inner stores of torachao to speed up layer offloading for quantized models around 25%
|
2026-06-07 15:52:01 -06:00 |
|
Jaret Burkett
|
75781fb5a5
|
Fix float8 weights not offloading to CPU in layer offloading
|
2026-06-07 15:36:17 -06:00 |
|
Jaret Burkett
|
323b4aaf5a
|
Do not copy pin memory if it fails, just move
|
2025-11-17 18:04:00 +00:00 |
|
Jaret Burkett
|
0d8a33dc16
|
Offload ARA with the layer if doing layer offloading. Add support to offload the LoRA. Still needs optimizer support
|
2025-10-21 06:03:27 -06:00 |
|
Jaret Burkett
|
55b8b0e23e
|
Fix issue where ARA was not working when using memory manager
|
2025-10-07 13:39:44 -06:00 |
|
Jaret Burkett
|
c9f982af83
|
Add support for using quantized models with ramtorch
|
2025-10-06 13:46:57 -06:00 |
|
Jaret Burkett
|
4e5707854f
|
Initial support for RamTorch. Still a WIP
|
2025-10-05 13:03:26 -06:00 |
|