-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] make deterministic
parameter more thorough?
#6731
Comments
deterministic
parameter more impactful?deterministic
parameter more thorough?
I generally like this! In my experience, it is always slightly annoying to get ML libraries to behave purely deterministically. Having this feature built-in would be pretty nice. That being said, I find it slightly weird to fix |
I agree that setting |
Does this comment just refer to the 2 different approaches I mentioned for dealing with multi-threading? If so, I think we should start with the other sources of non-determinism (like setting
To be clear, I'm proposing that passing That's the simplest way I can think of to achieve this behavior, and I strongly suspect that most users setting |
Yes, my comment was about those ones. |
Fully agree with this proposal. But for CUDA version, we may still encounter nondeterministic issues due to nondeterministic order of atomic operations in adding up the histograms. So far, I have now idea to solve this case yet, without sacrificing too much training speed. |
Makes sense to me, I definitely think it is acceptable for the CUDA version to have weaker guarantees about determinism in exchange for being much faster. |
Proposal
When
deterministic = true
in parameters, other configuration values should be overwritten with values that will reduce randomness, even if those values make LightGBM slower or more resource-intensive.Specifically,
deterministic = true
should result in:force_row_wise = true
gpu_use_dp = true
seed = 708
(or some other fixed, positive, non-0 number)Motivation
In my experience responding to issues here and on Stack Overflow, LightGBM's sources of non-determinism are often misunderstood. It'd be simpler (for both users and maintainers) if it was possible to get deterministic results by just setting
{"deterministic": true}
.Examples of the discussions I'm referring to:
deterministic
being introduced in Support deterministic #3494)Description
There are plenty of implementation details that would need to be worked out, but in short I'm envisioning this updating would happen only on the C++ side, not in wrappers.
And that a new documentation section should be added (not in the
deterministic
parameter's part of https://lightgbm.readthedocs.io/en/latest/Parameters.html), explaining the limitations of this approach and how to further improve reproducibility.Other Questions
How to handle multithreading with OpenMP?
Proposing we do the following:
deterministic = true
(as in Support deterministic #3494)deterministic = true
is sufficient to get completely deterministic results for the CPU versionBut alternatively, we could do something simpler:
num_threads = 1
wheneverdeterministic = true
Which is preferable?
Reviewers
@shiyu1994 @guolinke @StrikerRUS @borchero @jmoralez @btrotta I'd like to hear your thoughts on this. Thanks for your consideration!
The text was updated successfully, but these errors were encountered: