Can't reproduce results training

[mallorbc]

With the provided splits, I can not reproduce the results when training from scratch. I get an f1 scores of 61.33% and 47.79% for TVSum and SumMe respectfully. I am thinking that maybe the seed of 1234 is not the correct seed to be used for generating these results from scratch?
Thanks!

[ok1zjf]

Hi Mallorbc,
The published results were produces with the the github code as is, that is, including the seed. Later on we
found out that the results slightly differed with later pytorch versions. It you setup you machine with the
package versions documented in the readme.md you should be able to get the published values.
Sorry I'm not able to help more.
Best
Jiri

[mallorbc]

No, thats plenty of help and makes perfect sense. Thanks

I do have a follow up though:
How did you go about selecting the seed and the splits used for the final result? Using different seeds and or different splits leads to a noninsignificant change in the results.
Thanks again!