Skip to content

Commit

Permalink
build_context: improve wheel reproducibility by sorting libs
Browse files Browse the repository at this point in the history
While tracking a reproducibility issue from maturin's output, we found
that the .so file in the output .whl were not ordered the same
every time.

Order of the external libraries in the .whl comes down to the order
`soname_map` is iterated. But, `std::HashMap` does not provide a stable
order and that create an unstable order in the wheel.

Switch to `std::BTreeMap` which keeps keys sorted and is iterable
in a stable order.

This can be tested by building current python3-cryptography :
```
SOURCE_DATE_EPOCH=1728915855 maturin build
```
Before this commit, the above give two possible outputs (~50% each), the
difference is in the order of libssl and libcrypto.

After this commit, the output is reproducible.

Signed-off-by: Yoann Congal <[email protected]>
  • Loading branch information
ycongal-smile committed Oct 17, 2024
1 parent 7bda888 commit e608c97
Showing 2 changed files with 7 additions and 2 deletions.
5 changes: 5 additions & 0 deletions Changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## [Unreleased]

* Improve wheel reproduciblity by sorting external libraries [#XXX](https://github.com/PyO3/maturin/pull/XXX)


## [1.7.4]

* Fix musllinux rpath for non-cffi bindings in [#2233](https://github.com/PyO3/maturin/pull/2233)
4 changes: 2 additions & 2 deletions src/build_context.rs
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ use normpath::PathExt;
use pep508_rs::Requirement;
use platform_info::*;
use sha2::{Digest, Sha256};
use std::collections::{HashMap, HashSet};
use std::collections::{BTreeMap, HashSet};
use std::env;
use std::fmt::{Display, Formatter};
use std::io;
@@ -396,7 +396,7 @@ impl BuildContext {
writer.add_directory(&libs_dir)?;

let temp_dir = tempfile::tempdir()?;
let mut soname_map = HashMap::new();
let mut soname_map = BTreeMap::new();
let mut libs_copied = HashSet::new();
for lib in ext_libs.iter().flatten() {
let lib_path = lib.realpath.clone().with_context(|| {

0 comments on commit e608c97

Please sign in to comment.