-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to reproduce the result on WMT14 DE-EN? #2003
Comments
Has there been any advancements on this? I'm having the same problem. |
I used OpenNMT-py 1.x version to reproduce the result successfully. If you are interested, you can find it in my repository with a tutorial to reproduce it. |
I have met the same problem, have you solved it in 2.0 version? |
No... actually, I want to give up on the 2.0 version :D. |
FYI I just ran the 2.0 example from scratch with a fresh |
I have successfully run the 2.0 example now, one issue is that I need to add -share_vocab parameter which the example does not contain. |
Yes you're right I forgot to mention the vocab options may not be fully up to date in the example. |
I've been tried to reproduce the result on WMT14 DE-EN currently. According to the paper "Attention is All You Need", the transformer model should achieve 27.3 BLEU on newstest2014. But I only got 17.2 by sacrebleu.
I have a look at #637 and #1862. However, some commands and parameters are eliminated in OpenNMT-py 2.0, which makes me a little confused.
The commands I used are as follows:
1. Prepare the data
I used the script here https://github.com/OpenNMT/OpenNMT-py/blob/master/examples/scripts/prepare_wmt_data.sh
And the command is
./prepare_wmt_data.sh ../../onmt_data/wmt14-de-en
.2. Build the vocabulary
onmt_build_vocab -config wmt14-de-en.yml -n_sample -1
3. Train the model
python train.py --config ./examples/scripts/wmt14-de-en.yml
The contents in the wmt14-de-en.yml are:
4. Translate and evaluate
Is there something wrong?
The text was updated successfully, but these errors were encountered: