Skip to content

Latest commit

 

History

History
25 lines (18 loc) · 1.52 KB

README.md

File metadata and controls

25 lines (18 loc) · 1.52 KB

Global-MMLU

Paper

Title: Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

Abstract: https://arxiv.org/abs/2412.03304

Global-MMLU-Lite is a balanced collection of culturally sensitive and culturally agnostic MMLU tasks. It is designed for efficient evaluation of multilingual models in 15 languages (including English). Only languages with human translations and post-edits in the original Global-MMLU 🌍 dataset have been included in the lite version.

Homepage: https://huggingface.co/datasets/CohereForAI/Global-MMLU-Lite

Citation

@misc{singh2024globalmmluunderstandingaddressing,
      title={Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation},
      author={Shivalika Singh and Angelika Romanou and Clémentine Fourrier and David I. Adelani and Jian Gang Ngui and Daniel Vila-Suero and Peerat Limkonchotiwat and Kelly Marchisio and Wei Qi Leong and Yosephine Susanto and Raymond Ng and Shayne Longpre and Wei-Yin Ko and Madeline Smith and Antoine Bosselut and Alice Oh and Andre F. T. Martins and Leshem Choshen and Daphne Ippolito and Enzo Ferrante and Marzieh Fadaee and Beyza Ermis and Sara Hooker},
      year={2024},
      eprint={2412.03304},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.03304},
}