Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There is a discrepancy in the number of triggers #4

Open
ysf123456 opened this issue Aug 30, 2024 · 1 comment
Open

There is a discrepancy in the number of triggers #4

ysf123456 opened this issue Aug 30, 2024 · 1 comment

Comments

@ysf123456
Copy link

Taking sst2 as an example, according to the word_countall.json you provided, with selected_trigger_num set to 20, the number of triggers counted is:
wandb: trigger_num_0 65708
wandb: trigger_num_1 2429
wandb: trigger_num_2 78
wandb: trigger_num_3 6
This is somewhat different from what was given in the paper. Can you help me clarify it?
image

@yjw1029
Copy link
Owner

yjw1029 commented Aug 30, 2024

When we reproduced the results using the Github code in June 2023, we were still able to replicate the results from the paper. The records in Wandb are as follows:
image

The command recorded in Wandb was:
image

Since the trend of the data you provided is not significantly different from the actual data, it is possible that some random factors, such as modifications to the random number generator, platform differences, or other random elements, led to a different trigger selection than the one we used for the reported experiment. You may try the following:

  1. Check whether the running command matches the command mentioned above.
  2. Verify whether the running environment is consistent with the environment we provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants