This repository implements the Verb-Noun Marginal Cross Entropy Loss (VNMCE) proposed in the paper (download here):
A. Furnari, S. Battiato, G. M. Farinella (2018). Leveraging Uncertainty to Rethink Loss Functions and Evaluation Measures for Egocentric Action Anticipation . In International Workshop on Egocentric Perception, Interaction and Computing (EPIC) in conjunction with ECCV.
We also report an implementation of the Truncated Top-5 Entropy Loss proposed in the paper:
Lapin, Maksim, Matthias Hein, and Bernt Schiele. "Loss functions for top-k error: Analysis and insights." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
Please see the code for usage examples.
At the time of the publication, the evaluation server for the egocentric action recognition/anticipation challenges was not available. Therefore the results of the proposed method are not reported in the paper. To allow for fair comparisons with other methods, the results on the test sets are reported in the following.
These results are related to the TSN model trained for the task of egocentric action anticipation with the VNMCE loss (please refer to the paper for more details):
| Top-1 Accuracy (%) | Top-5 Accuracy (%) | Precision (%) | Recall (%) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Set | V | N | A | V | N | A | V | N | A | V | N | A |
| Seen | 27.92 | 16.09 | 10.76 | 73.59 | 39.32 | 25.28 | 23.43 | 17.53 | 6.05 | 14.79 | 11.65 | 5.11 |
| Unseen | 21.27 | 9.90 | 5.57 | 63.33 | 25.50 | 15.71 | 10.02 | 6.88 | 1.99 | 7.68 | 6.61 | 2.39 |
These results are related to the TSN model trained for the task of egocentric action recognition with the VNMCE loss (please refer to the paper for more details):
| Top-1 Accuracy (%) | Top-5 Accuracy (%) | Precision (%) | Recall (%) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Set | V | N | A | V | N | A | V | N | A | V | N | A |
| Seen | 54.22 | 38.85 | 29.00 | 85.22 | 61.80 | 49.62 | 53.87 | 38.18 | 18.22 | 35.88 | 32.27 | 16.56 |
| Unseen | 40.90 | 23.46 | 16.39 | 72.11 | 43.05 | 31.34 | 26.62 | 16.83 | 7.10 | 15.56 | 17.70 | 10.17 |
Please reference this publication if you find this code useful:
@inproceedings{furnari2018Leveraging,
author = { A. Furnari and S. Battiato and G. M. Farinella },
title = { Leveraging Uncertainty to Rethink Loss Functions and Evaluation Measures for Egocentric Action Anticipation },
booktitle = { International Workshop on Egocentric Perception, Interaction and Computing (EPIC) in conjunction with ECCV },
year = { 2018 },
}
You can find related works at the following page: http://iplab.dmi.unict.it/fpv/.