Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

using ahash compile-time-rng kills reproducible builds #3271

Closed
Fuuzetsu opened this issue Dec 4, 2022 · 3 comments · Fixed by #3277
Closed

using ahash compile-time-rng kills reproducible builds #3271

Fuuzetsu opened this issue Dec 4, 2022 · 3 comments · Fixed by #3277
Labels
bug parquet Changes to the parquet crate

Comments

@Fuuzetsu
Copy link

Fuuzetsu commented Dec 4, 2022

Describe the bug
In f812d2c, ahash feature compile-time-rng was enabled. This causes builds to not be reproducible. If you now build anything with ahash in the tree multiple times, it's going to give different results a lot of the time. This completely kills any systems that cache builds and such.

To Reproduce
Build parquet-rs few times and compare outputs.

Expected behavior
Outputs should be the same each time.

Additional context
It seems it was added strictly for WASM so it's a big shame that it impacts every other platform too, if it's actually necessary.

You can see some previous precedent in people having this exact problem with the feature here: rust-lang/hashbrown#149
@Fuuzetsu Fuuzetsu added the bug label Dec 4, 2022
@Fuuzetsu
Copy link
Author

Fuuzetsu commented Dec 5, 2022

We will set CONST_RANDOM_SEED in our builds for now but would be good to not have to do this...

https://crates.io/crates/const-random

@Fuuzetsu
Copy link
Author

Fuuzetsu commented Dec 5, 2022

We will set CONST_RANDOM_SEED in our builds for now but would be good to not have to do this...

https://crates.io/crates/const-random

Actually, we patched the relevant crate to hardcode the seed instead of faffing around with it...

tustvold added a commit to tustvold/arrow-rs that referenced this issue Dec 5, 2022
@tustvold
Disable const-random ahash feature on non-WASM (apache#3271)
4e280cf
@tustvold tustvold mentioned this issue Dec 5, 2022
Merged
tustvold added a commit that referenced this issue Dec 5, 2022
@tustvold
Disable const-random ahash feature on non-WASM (#3271) (#3277)
7b38cb8
@alamb alamb added the parquet Changes to the parquet crate label Dec 9, 2022
@alamb
Copy link
Contributor

alamb commented Dec 9, 2022

label_issue.py automatically added labels {'parquet'} from #3277

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Disable const-random ahash feature on non-WASM (#3271) tustvold/arrow-rs
2 participants
@alamb @Fuuzetsu