Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot reproduce results for RobertaGen on SVAMP #28

Open
qfkkwgd opened this issue Nov 9, 2022 · 0 comments
Open

Cannot reproduce results for RobertaGen on SVAMP #28

qfkkwgd opened this issue Nov 9, 2022 · 0 comments

Comments

@qfkkwgd
Copy link

qfkkwgd commented Nov 9, 2022

Hi,

Thanks for your great work.

I am trying to reproduce the results for RobertaGen on SVAMP.

This is my command

python run_mwptoolkit.py
--model=RobertaGen --dataset=mawps_asdiv-a_svamp --task_type=single_equation --equation_fix=prefix --k_fold=None --test_step=5 --gpu_id=1

In your paper, the performance of RoBERTaGen on SVAMP is 30.3%. But my reproduced result is only 23.4%. Could you share with me about why this happens?

Thanks again for your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant