Added Differential Guidance training target

This commit is contained in:
Jaret Burkett
2025-11-10 09:38:25 -07:00
parent 9b89bab8fe
commit 2e7b2d9926
9 changed files with 101 additions and 6 deletions

View File

@@ -258,6 +258,25 @@ const docs: { [key: string]: ConfigDoc } = {
</>
),
},
'train.do_differential_guidance': {
title: 'Differential Guidance',
description: (
<>
Differential Guidance will amplify the difference of the model prediction and the target during training to make
a new target. Differential Guidance Scale will be the multiplier for the difference. This is still experimental,
but in my tests, it makes the model train faster, and learns details better in every scenario I have tried with
it.
<br />
<br />
The idea is that normal training inches closer to the target but never actually gets there, because it is
limited by the learning rate. With differential guidance, we amplify the difference for a new target beyond the
actual target, this would make the model learn to hit or overshoot the target instead of falling short.
<br />
<br />
<img src="/imgs/diff_guidance.png" alt="Differential Guidance Diagram" className="max-w-full mx-auto" />
</>
),
},
};
export const getDoc = (key: string | null | undefined): ConfigDoc | null => {