What is the default token, UNK(2) or PAD(0)?

[shiyegao]

As can be seen in from esm.utils.constants import esm3 as C, there are two kinds of tokens.

SS8_UNK_TOKEN = 2
SS8_PAD_TOKEN = 0

1. If we just use ESM3.forward(sequence_tokens=xxx), the ss8_tokens will be 2 by defaults.
2. However, if we use ESM3.generate(xxx), the ss8_tokens will be 0 by default_protein_tensor.

Although the PAD tokens may be learned as UNK tokens by training, I wonder what the best none token is during only-sequence embedding extraction.

Besides, is this inconsistency a bug? Will you fix this inconsistency later?

[ebetica]

The default ss8_token should be 0 for an all masked sequence. Will fix in the next release.

[ebetica]

This should be fixed on main, expect a release to pip in the next week.