LOL dataset's PSNR is unstable #10
-
|
Why LOL dataset's PSNR is unstable ? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
The LOLdataset's validation dataset only contains 15 images, leading to a very strong bias. This may cause: problem 3. Also, because of the feature of pytorch-lightning, even if you set the same seed, using the same ckpt, the results will be different on different devices (like A6000 V.S. 4090). We trained our ckpt on two RTX A6000 for LOLdataset, and reported the best PSNR. Also, we directly provide the image results to avoid problem 3. |
Beta Was this translation helpful? Give feedback.
The LOLdataset's validation dataset only contains 15 images, leading to a very strong bias.
This may cause:
problem 1. very different PSNR on the same ckpt with different seeds.
problem 2. very different PSNR (both ckpt and seeds) on different devices.
Results on 8 * 4090:
problem 3. Also, because of the feature of pytorch-lightning, even if you set the same seed, using the same ckpt, the results will be different on different devices (like A6000 V.S. 4090).
Results on 8 * A6000:
Results on 8 * 4090:
We trained our ckpt on two RTX A6000 for LOLdataset, and reported the best PSNR. Also, we directly provide the image results to avoid problem 3.
Here is our environment: