sample_gs_* vs samples_cfg_scale3.*

[gianlucaasti]

Hello everyone,
I am fine tuning this repo on a dataset composed of 12045 images. Unfortunately my gpu can only handle 1 image per batch, therefore I have batch_size=1 and accumulate_grad_batches=16.
At Epoch 14, the sample_gs_* images are actually quite good, but the sample_cfg_scale3.* images are quite bad.
Is this normal or am I overfitting? Do I need to train it for more epochs? The loss is around 0.19 at this point of the training.

Being afraid of overfitting, I decided to reduce the lr to 1e-5 and change accumulate_grad_batches=6.
However, even though I am at epoch 8, I can already see good sample_gs_* images but bad samples_cfg_scale3.* images.

Is this normal during training?