Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression test: two.pf is producing somewhat different results when run twice with the same commit #1127

Open
kslong opened this issue Nov 8, 2024 · 1 comment

Comments

@kslong
Copy link
Collaborator

kslong commented Nov 8, 2024

Although all of the other tests produce identical results when run with the same commit, two.pf does not. The results are similar but not identical, as they should be since it is not being run with a random seed. Here is what the comparison looks like in the regression plot.
two

(Aside: the commands.txt file for the regression tests has not been updated and still refers to python, though this probably does not matter for anything. All of the regression tests are run with the defaults. It's more generally unclear that we have a good set of tests, which is something that should be reviewed. However the problem with two.pf is independent of these concerns)

@kslong
Copy link
Collaborator Author

kslong commented Nov 9, 2024

This problem depends on the number of threads. It does not appear with 8 threads, but does appear with 12.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant