Evaluation output uncorrectly

[shacker101]

Hello,

I followed the README to train the model, but during evaluation, I encountered the following warning:

Some weights of Prot2TextModel were not initialized from the model checkpoint at ./models/prot2text_base/model and are newly initialized: ['decoder.lm_head.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Additionally, the evaluation model only output: "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!".

Could this indicate that the weights were not loaded correctly?

However, evaluating your pretrained model (Prot2Text SMALL) works fine.

The main difference in my setup is that I downloaded GPT2 and ESM2 weights for offline training.