Seed Libc rng from system entropy #43606
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We're seeing lots of CI failures that look like collisions in the names of temporary variables. Currently we're seeding the libc rng from a low entropy clock, which is somewhat likely to have collisions particular if (as we are), you launch all your processes at exactly the right time. Try to fix that by seeding the libc using system entropy. Of course that still leaves the potential for birthday problems, but hopfully this will tide us over until #43597 gets fixed.
|
It looks like this has fixed the CI failures! (At least for now.) |
vtjnash
reviewed
Jan 7, 2022
Comment on lines
311
to
312
| println(stderr, | ||
| "Entropy pool not available to seed RNG; using ad-hoc entropy sources.") |
There was a problem hiding this comment.
Suggested change
| println(stderr, | |
| "Entropy pool not available to seed RNG; using ad-hoc entropy sources.") |
vtjnash
reviewed
Jan 7, 2022
| base = 16) % UInt) | ||
| catch | ||
| end | ||
| Base._ad_hoc_entropy_source() |
There was a problem hiding this comment.
Suggested change
| Base._ad_hoc_entropy_source() | |
| seed = Base._ad_hoc_entropy_source() |
vtjnash
reviewed
Jan 7, 2022
| end | ||
| end # os-test | ||
|
|
||
| function _ad_hoc_entropy() |
There was a problem hiding this comment.
Can you add a unittest for this (just literally call @test_warn "ad-hoc" Base._ad_hoc_entropy_source(), and make sure it returns a reasonable value?
vtjnash
reviewed
Jan 7, 2022
| "Entropy pool not available to seed RNG; using ad-hoc entropy sources.") | ||
| seed = reinterpret(UInt64, time()) | ||
| seed = hash(seed, getpid() % UInt) | ||
| try |
There was a problem hiding this comment.
Suggested change
| try | |
| Sys.iswindows() || try |
|
There should not be a risk of the birthday paradox anymore, after #38879. The bigger issue is perhaps actually that |
LilithHafner
pushed a commit
to LilithHafner/julia
that referenced
this pull request
Feb 22, 2022
We're seeing lots of CI failures that look like collisions in the names of temporary variables. Currently we're seeding the libc rng from a low entropy clock, which is somewhat likely to have collisions particular if (as we are), you launch all your processes at exactly the right time. Try to fix that by seeding the libc using system entropy. Of course that still leaves the potential for birthday problems, but hopfully this will tide us over until JuliaLang#43597 gets fixed.
DilumAluthge
added a commit
that referenced
this pull request
Feb 26, 2022
This reverts commit 5819775.
vtjnash
added a commit
that referenced
this pull request
Mar 3, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice.
vtjnash
added a commit
that referenced
this pull request
Mar 8, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice.
LilithHafner
pushed a commit
to LilithHafner/julia
that referenced
this pull request
Mar 8, 2022
We're seeing lots of CI failures that look like collisions in the names of temporary variables. Currently we're seeding the libc rng from a low entropy clock, which is somewhat likely to have collisions particular if (as we are), you launch all your processes at exactly the right time. Try to fix that by seeding the libc using system entropy. Of course that still leaves the potential for birthday problems, but hopfully this will tide us over until JuliaLang#43597 gets fixed.
vtjnash
added a commit
that referenced
this pull request
Mar 8, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice.
vtjnash
added a commit
that referenced
this pull request
Mar 21, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice.
vtjnash
added a commit
that referenced
this pull request
Apr 7, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice.
KristofferC
pushed a commit
that referenced
this pull request
Jul 17, 2022
Continuation from #43606 - Replaces thread-unsafe function `rand` with `jl_rand`. - Fixes `_ad_hoc_entropy_source` fallback in Random. - Uses uv_random for more direct access to quality-randomness (usually a syscall rather than a file.) - Ensures Array{Bool} are valid when created from RandomDevice. (cherry picked from commit b4bed71)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We're seeing lots of CI failures that look like collisions in the
names of temporary files. Currently we're seeding the libc
rng from a low entropy clock, which is somewhat likely to have
collisions particular if (as we are), you launch all your processes
at exactly the right time. Try to fix that by seeding the libc using
system entropy. Of course that still leaves the potential for birthday
problems, but hopfully this will tide us over until #43597 gets fixed.