How to fine-tune downstream tasks

[GeorgeBGM]

Dear developer，

The dna language model is trained using gpt2 or gpt3 ？How to fine-tune it in downstream tasks? It's seems that this model does not appear to be available directly through the Hugging face platform at this time.

Best, Du

[doublechenching]

The model is trained using GPT2. Weights is not compatible with huggingface. You need to write a script to convert weights. or refer to the training code of nanoGPT

[GeorgeBGM]

Dear developer，

Is there any plan to release it to the huggingface platform in the future? I would like to fine-tune it and use it for sequence classification tasks, are there any suggestions? At this moment, it looks like it can only be used directly for sequence classification.

Best, Du

[doublechenching]

Yes, we are preparing to release the model on huggingface now，it will take about two or three weeks。You can also use the current code and it can also do a variety of tasks, like sequence classification, regression and generation， just comment out the task branches that are not needed

[GeorgeBGM]

well, sound good.