Skip to content

Commit

Permalink
updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
KevinMusgrave committed Aug 28, 2024
1 parent fc93739 commit b9b3d51
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions blog/llm-finetuning-4/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Finetuning Mistral-7B using LoRA and DeepSpeed
# Experimenting with LoRA parameters

We finetune [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) using [LoRA](https://arxiv.org/abs/2106.09685) and [DeepSpeed](https://github.com/microsoft/DeepSpeed). We ran LoRA on two 40 GB A100 GPUs utilizing DeepSpeed.

See our [blog post](determined.ai/blog/lora-parameters) for our experiment results.

To get started, first install Determined on your local machine:
```bash
pip install determined
Expand All @@ -28,4 +30,5 @@ DeepSpeed configuration files are in the `ds_configs` folder.

## Contributors

- [Sze Wai Yuen](https://github.com/szewaiyuen6)
- By [Sze Wai Yuen](https://github.com/szewaiyuen6)
- Built on `llm-finetuning` code by [Agnieszka Ciborowska](https://github.com/aciborowska) and [Kevin Musgrave](https://github.com/KevinMusgrave).

0 comments on commit b9b3d51

Please sign in to comment.