Details of out-of-distribution

[Chenyyyyyyyyy]

Thank you for sharing your work and code!

In the paper, you claim to use M= 4 and 16 shots when testing the generalization of prompts.

But in the weights you give, it is M= 1 and 100 shots, could you please clarify the detailed experimental setting of out-of-distribution (such as M, shots, batch, and epoch)?

Thanks

[TreeLLi]

Hi,

thank you for your interest in our work. For OOD experiments, M=4 and 16 shots were used. The weights you mentioned, M=1 and 100 shots, were used for zero-shot experiments. We have not released the weights of models used for OOD experiments. If you need those, we could try to find them but not sure if they are still available since the file system of our server recently crashed.

Best wishes,
Lin

[Chenyyyyyyyyy]

Hi, Thank you for your response.

What is the difference between zero-shot experiments and OOD experiments since both of them trained on Imagenet and tested on downstream tasks.

I would appreciate it if you could provide the weights for the OOD experiments. If those are not available, the batch and epoch would also be helpful. I can reproduce it myself.

Thanks！

[TreeLLi]

Hi,

they differ in this work regarding the data distribution gap between training and test data. OOD involves the distribution shift only in the input space so the training and test datasets have the consistent category system, while zero-shot involves the distribution shift in both input space and output category so the test input data and/or category may never be seen in the training data.

The detailed experimental setups are described in the appendix. Please let me know if there are something missing in the appendix.

Lin

[Chenyyyyyyyyy]

So the cross datasets experiments is the zero-shot experiments right?

Thanks for your reply!

[TreeLLi]

yes, we follow the zero-shot setup in this work: https://arxiv.org/abs/2212.07016. Kindly please close the issue if you feel it is resolved, otherwise, I am happy to answer any other question you have.