Add new FT instructions (#1143)

mosaicml · Apr 26, 2024 · f97f02e · f97f02e
1 parent 6cfd2a3
commit f97f02e
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/scripts/train/yamls/finetune/README.md b/scripts/train/yamls/finetune/README.md
@@ -0,0 +1,6 @@
+# How to finetune a new Hugging Face model
+Using the [DBRX yaml](https://github.com/mosaicml/llm-foundry/blob/main/scripts/train/yamls/finetune/dbrx-full-ft.yaml) as a template, change the following fields for the new model:
+- `max_seq_len`
+- `model/pretrained_model_name_or_path`
+- `tokenizer/name`
+Open a GitHub issue if these steps don't work. Most new causal LMs should work out of the box.