EfficientNet implementation

[leonid-pishchulin]

Hey, are there any EfficientNet (https://arxiv.org/abs/1905.11946) implementations available? If not, is somebody working on it?

[KellenSunderland]

Just for some background: we're mostly interested so no one duplicates effort.

[zhreshold]

here's one candidate @sufeidechabei
To summarize, it might be easy to write the definitions(https://github.com/mnikitin/EfficientNet/blob/master/efficientnet_model.py), but training part is unpredictable yet due to the missing training hyper-parameters in paper.

Should we split the risk by training the network simultanously?

[bermanmaxim]

Note that https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/main.py has the training script that seems to have been used and provides hyperparameters.

[bermanmaxim]

Also there might be some differences with the paper, author says "source code is correct": see tensorflow/tpu#383, tensorflow/tpu#390

[ryanjay0]

They never said "source code is correct" about tpu issue tensorflow/tpu#390. Did they? Seems like a much larger discrepancy than the padding issues in tensorflow/tpu#383

[bermanmaxim]

True, what I meant is that since the author said source code is correct on tensorflow/tpu#383 I was assuming the code is what they actually used, including concerning tensorflow/tpu#390. But you are right that this resolution discrepancy is a big difference 🤔

[bermanmaxim]

Regarding the training of efficientnet, see the remarks of Ross Wightman here: https://forums.fast.ai/t/efficientnet/46978/67 ; it might be that keeping an exponential moving average of the weights during training, for use at testing, helps this family of models a lot.

[bermanmaxim]

Similar discussion over here: pytorch/vision#980

[hetong007]

@sufeidechabei please check your PR with the resources above.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.