Color inversion, black image and nan in loss after ~20 epochs

[victor-felicitas]

I've tried to train the model on original summer2winter_yosemite dataset.
After ~20 epochs all sample images turned completely black, and all all loss parameters turned to nan.
However, the model continued to run for 30 more epochs regularly saving checkpoints until I stopped it.

I've also used another, my own dataset, and it ran correctly for 70 epochs at least, unfortunately the only result I had was color inversion of images.
Any advice on changing training parameters (I used default)?