New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I have a question about mmlu score #1

Open

gyuwon12 opened this issue Oct 17, 2023 · 1 comment

gyuwon12 commented Oct 17, 2023

I know MMLU has a lot of subsets, so are you simply going to add 57 EM values and average them?

Owner

LZY-the-boys commented Oct 17, 2023 •

edited

Loading

Yes. This table took a long time to get (57tasks 1000items per task)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment