What is the submission file format for the challenge？

[Baboom-l]

Is it json, pkl, or other. Can you provide a sample submission file？It would be very helpful for me.

[xiaofeng94]

Hey @Baboom-l, the submission file should be a .json in the format described at https://github.com/samschulter/omnilabeltools?tab=readme-ov-file#submitting-prediction-results

You may also refer to https://www.omnilabel.org/task for more descriptions on the task

[Baboom-l]

I submitted it on OmniLabel 2023 Challenge - Track C, but there were no results

[xiaofeng94]

Hey, it takes some time depending on the size of file you upload...

BTW, you may upload results to challenge 2024, https://codalab.lisn.upsaclay.fr/competitions/19200#learn_the_details
The evaluation dataset for challenge 2024 is more compact than challenge 2023, which may facilitate your work.
A more comprehensive test set will come out in the later stage of this challenge.

You may also do evaluation on the validation set with your own device using the tool, https://github.com/samschulter/omnilabeltools?tab=readme-ov-file#evaluate-your-results

[Baboom-l]

Hi, how are the AP metrics calculated? the mean values of categ and descr don't seem to be equal to AP.

[samschulter]

Hi @Baboom-l . The AP value is the harmonic mean of AP_categ and AP_descr, not the arithmetic mean. The reason is to encourage good numbers on both simple categories and more complex descriptions. See the paper for details.

[Baboom-l]

@samschulter @xiaofeng94 Hi, where can I submit the technical report about OmniLabel 2

[yuminsuh]

Hi @Baboom-l . Please submit the technical report to [email protected]

[Baboom-l]

Hi, @yuminsuh. When will the final ranking be announced