nllb-alpaca-dataset-translation

Uses the nllb model released by meta to translate the alpaca dataset.

Getting Started

Steps

Make sure you have a working installation of Pytorch

1. git clone https://github.com/KhmerAILab/nllb-alpaca-dataset-translation

2. cd nllb-alpaca-dataset-translation

3. pip install huggingface_hub transformers pandas numpy tqdm accelerate bitsandbytes

4. set which dataset you want to translate at line 15

5. run python3 index.py

6. profit!