3.2

1. Faster beam search
2. support h5 feature file
3. allow beam search + scst (doesn't work as well though)
4. Add a few models, BertCapModel and m2transformer (usefulness still question marked)
5. Add projects.

Official py3 support; MODEL ZOO

1. Since it's 2020, py3 is officially supported. Open an issue if there is still something wrong.
2. Finally, there is a model zoo which is relatively complete. Feel free to try the provided models.

v3

1. Add structure loss inspired by Classical Structured Prediction Losses for Sequence to Sequence Learning
2. Add a function of sample n captions. Support methods described in https://www.dropbox.com/s/tdqr9efrjdkeicz/iccv.pdf?dl=0.
3. More pytorchy design of dataloader. Also, the dataloader now don't repeat image features according to seq_per_img. The repeating is now moved to the model forward function.
4. Add multi-sentence sampling evaluation metrics like mBleu, Self-CIDEr etc. (those described in https://www.dropbox.com/s/tdqr9efrjdkeicz/iccv.pdf?dl=0)
5. Use detectron type of config to setup experiments.
6. A better self critical objective. (Named as new_self_critical now.)
   Use config ymls that end with nsc to test the performance. A technical report will be out soon.
   Basically, it performs better than original SCST on all metrics (by a small margin), but also faster (by a little bit).

Add flickr30k support

2.3

Update ADVANCED.md

Add a few more things

1 Refactor the code a little bit.
2 Add BPE (didn’t seem to work much different)
3 Add nucleus sampling, topk and gumbel softmax sampling.
4 Make AttEnsemble compatible with transformer
5 Add remove bad ending from Improving Reinforcement Learning Based Image Captioning with Natural Language Prior

Just add a few things

1. Add loss_wrapper for multi-gpu loss computation
2. Fix some bugs
3. Add transformer.

Add new features.

1. Add support for bleu4 optimization or combination of bleu4 and cider
2. Add bottom-up feature support
3. Add ensemble during evaluation.
4. Add multi-gpu support.
5. Add miscellaneous things. (box features; experimental models etc.)

Self critical model

This version can replicate the self-critical sequence training paper.