diff --git a/scripts/train/yamls/finetune/README.md b/scripts/train/yamls/finetune/README.md
new file mode 100644
index 0000000000..3ed238a4ca
--- /dev/null
+++ b/scripts/train/yamls/finetune/README.md
@@ -0,0 +1,6 @@
+# How to finetune a new Hugging Face model
+Using the [DBRX yaml](https://github.com/mosaicml/llm-foundry/blob/main/scripts/train/yamls/finetune/dbrx-full-ft.yaml) as a template, change the following fields for the new model:
+- `max_seq_len`
+- `model/pretrained_model_name_or_path`
+- `tokenizer/name`
+Open a GitHub issue if these steps don't work. Most new causal LMs should work out of the box.