Training vgnet generator loss keeps increasing

[ssinha89]

I am training VGNET on GRID dataset using the code in vgnet.py. The input to the network are the dlib landmarks after procrustes alignment, and the ground-truth image labels are the cropped face images after warping to the neutral head pose. While the discriminator losses for real and fake images keep reducing to a very low value, the generator loss keeps increasing throughout training. The quality of the generated images keeps decreasing after a few epochs, i.e, the GAN training does not converge. Could you please suggest what could be the issue.

[lelechen63]

I am training VGNET on GRID dataset using the code in vgnet.py. The input to the network are the dlib landmarks after procrustes alignment, and the ground-truth image labels are the cropped face images after warping to the neutral head pose. While the discriminator losses for real and fake images keep reducing to a very low value, the generator loss keeps increasing throughout training. The quality of the generated images keeps decreasing after a few epochs, i.e, the GAN training does not converge. Could you please suggest what could be the issue.



Are you using pytorch 0.4? Also, you need to preprocess the landmark to -0.2 to 0.2 I think. Meanwhile, you need to have pixel loss to stable the training.

[ssinha89]

I have done the preprocessing (landmark -0.2 to 0.2 then multiplied by 5), using pytorch 0.4 and using the same combination of loss functions as in vgnet.py. After a few iterations the discriminator loss goes down rapidly due to which the gradient in the generator disappears, and the attention mask learnt by the generator also becomes zero. Have you observed this kind of phenomenon during training, if so how did you overcome the problem.

[ssinha89]

I am still facing this issue of GAN convergence while training VGNET with ground truth 2D landmarks(scaled) and face images(cropped and warped). The generator GAN loss keeps increasing during the training process, even with the additional supervised L1 loss on pixel intensities. Could you please mention if you faced such a problem during the training.

[lelechen63]

I am still facing this issue of GAN convergence while training VGNET with ground truth 2D landmarks(scaled) and face images(cropped and warped). The generator GAN loss keeps increasing during the training process, even with the additional supervised L1 loss on pixel intensities. Could you please mention if you faced such a problem during the training.

I didn't encounter this problem. I tested this training code when it was released. My suggestion is that you can first remove other loss functions and only use L1 loss to supervise the network. It should also generate reasonable results. Then you can add the GAN loss to see it is loss problem or it is network problem.

[ssinha89]

Thanks for the suggestion. I trained the network with only the L1 loss on GRID dataset. As you said the results are reasonably good, however there is blur in the texture near the mouth region. On adding the GAN loss along the with L1 loss, there is the same problem which I described earlier, i.e, the discriminator loss keeps decreasing and the generator loss keeps increasing. I tried changing the relative weightage of the loss terms, (L1 loss 10 times or 100 times the GAN loss), with not much improvement. While training the GAN, did you use the same learning rate and Adam optimization parameters as given in the vgnet.py (lr = 0.0002, beta1 = 0.5, beta2=0.999)?

[lelechen63]

Thanks for the suggestion. I trained the network with only the L1 loss on GRID dataset. As you said the results are reasonably good, however there is blur in the texture near the mouth region. On adding the GAN loss along the with L1 loss, there is the same problem which I described earlier, i.e, the discriminator loss keeps decreasing and the generator loss keeps increasing. I tried changing the relative weightage of the loss terms, (L1 loss 10 times or 100 times the GAN loss), with not much improvement. While training the GAN, did you use the same learning rate and Adam optimization parameters as given in the vgnet.py (lr = 0.0002, beta1 = 0.5, beta2=0.999)?

Yes, the optimation is the same one. Can you change the gan loss to LSGAN. Basically, remove the sigmoid layer and change the discriminator loss with mseloss ranther than bceloss. My new experiments show that lsgan is more stable than vanila gan.

[gogozhaoya]

I got similar results as you.