Jaret Burkett 5d6887fd98 Major updates to automagic3 optimizer. Seems to be functioning more ideally and naturally decays, as it should.

2026-06-08 14:09:02 -06:00

..

adafactor.py

Added a basic torch profiler that can be used in config during development to find some obvious issues.

2025-06-17 13:03:39 -06:00

adam8bit.py

Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use)

2024-11-20 09:16:55 -07:00

automagic2.py

Update default agreement threshold for automagic v2 to be 0.5

2026-05-26 07:57:54 -06:00

automagic3.py

Major updates to automagic3 optimizer. Seems to be functioning more ideally and naturally decays, as it should.

2026-06-08 14:09:02 -06:00

automagic.py

Reworked automagic optimizer and did more testing. Starting to really like it. Working well.

2025-04-28 08:01:10 -06:00

optimizer_utils.py

Added a basic torch profiler that can be used in config during development to find some obvious issues.

2025-06-17 13:03:39 -06:00

prodigy_8bit.py

Added adafactor implementation that handles stochastic rounding of update and accumulation

2024-10-30 05:25:57 -06:00

test_optimizers.py

Allow automagic3 to run unfused. Add some clipping.

2026-06-08 08:59:06 -06:00