About models in the original 'model' folders

[a09358872999]

I am working on a fine-tuning project, which is a multi-label task with more than 1000 categories.
So I write a script similar to this file(09_fine_tune_bert_on_uspto_1k_tpl.ipynb).
When finetuning the model, we have to select a pre-training model for starting.
So I do the same thing like you do in the script.(Use the below model as starting input)
model_path = pkg_resources.resource_filename("rxnfp", "models/transformers/bert_mlm_1k_tpl")

I'm so confusing to my result with 'bert_mlm_1k_tpl' as starting input, because my model didn't learning anything at all.
All prediction value from last layer is between 0.3-0.5(there doesn't exist a threshold value for it to have a good prediction.)
After 1 week, I noticed that maybe I should try another model('bert_pretrained').
It works. my prediction can have a normal values distribution.

I browsed the paper and the information on github, I still don't know difference between those files in model folder(rxnfp/models/transformers/) and which is pre-train on pistachio 2.3M reactions datasets？
If you have time, you can introduce what have those models been made.

Thank you for reading. Have a nice day.