scores correct?

[tfriedel]

Hi,
thanks for implementing this paper!
I just played a little bit with it, and I noticed almost every image, even really good or really bad ones all had scores of at least 4.8 and rarely more than 5.x.
Could this be a bug? Or maybe is more training needed? I was using your weights.
How long did you train for?

[titu1994]

Yes this is most likely due to insufficient training. I only trained for 10 epochs on the AVA dataset, whereas they recommend a few hundred epochs. That isnt possibly for me since it takes roughly 1.3 hours per epoch on my laptop.

[titu1994]

Turns out, the loss function was grossly incorrect. It needed a K.cumsum for the y_true and y_pred. This is the reason the vast majority of the images give extremely similar scores in the 4.7x to 5.3x range.

I'm retraining the network, which will take 16 hours for 10 epochs. Hopefully the results are much better this time around.

[aijianiula0601]

thanks for your jobs!
I want to know that if your training data is all the ava images or part of it?
The total number of ava images is 255530,has you train for all of them?

[titu1994]

I'm training on the first 250,000 images, and validating on the remaining 5000~ images. I don't think the nima paper gave a clear validation set.

[aijianiula0601]

HI
How can I get the Ava dataset?

[titu1994]

@aijianiula0601 Please search on Google.

[titu1994]

Just pushed a commit with the updated weights (trained from scratch on the fix #2).

Turns out, there is a small error in the calculation (cause it does it batchwise directly rather than batch of samples, thereby the losses are slightly higher than expected). Therefore this model is being further finetuned for 10 more epochs on this loss from #3