Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate manylinux2010 support #858

Merged
merged 4 commits into from
Aug 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ fs-err = "2.5.0"
fat-macho = { version = "0.4.5", default-features = false }
once_cell = "1.7.2"
rustc_version = "0.4.0"
semver = "1.0.13"
target-lexicon = "0.12.0"
pyproject-toml = "0.3.0"
python-pkginfo = "0.5.4"
Expand Down
3 changes: 3 additions & 0 deletions Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

* Deprecate manylinux 2010 support in [#858](https://github.com/PyO3/maturin/pull/858).
The [manylinux](https://github.com/pypa/manylinux) project already dropped its support
and the rustc compiler will [drop glibc 2.12 support in 1.64.0](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html).
* Add Linux mips64el architecture support in [#1023](https://github.com/PyO3/maturin/pull/1023)
* Add Linux mipsel architecture support in [#1024](https://github.com/PyO3/maturin/pull/1024)
* Add Linux 32-bit powerpc architecture support in [#1026](https://github.com/PyO3/maturin/pull/1026)
Expand Down
6 changes: 3 additions & 3 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,18 +234,18 @@ There's a `maturin sdist` command for only building a source distribution as wor
## Manylinux and auditwheel

For portability reasons, native python modules on linux must only dynamically link a set of very few libraries which are installed basically everywhere, hence the name manylinux.
The pypa offers special docker images and a tool called [auditwheel](https://github.com/pypa/auditwheel/) to ensure compliance with the [manylinux rules](https://www.python.org/dev/peps/pep-0571/#the-manylinux2010-policy).
The pypa offers special docker images and a tool called [auditwheel](https://github.com/pypa/auditwheel/) to ensure compliance with the [manylinux rules](https://peps.python.org/pep-0599/#the-manylinux2014-policy).
If you want to publish widely usable wheels for linux pypi, **you need to use a manylinux docker image**.

The Rust compiler since version 1.47 [requires at least glibc 2.11](https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-1470-2020-10-08), so you need to use at least manylinux2010.
The Rust compiler since version 1.64 [requires at least glibc 2.17](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html), so you need to use at least manylinux2014.
For publishing, we recommend enforcing the same manylinux version as the image with the manylinux flag, e.g. use `--manylinux 2014` if you are building in `quay.io/pypa/manylinux2014_x86_64`.
The [messense/maturin-action](https://github.com/messense/maturin-action) github action already takes care of this if you set e.g. `manylinux: 2014`.

maturin contains a reimplementation of auditwheel automatically checks the generated library and gives the wheel the proper.
If your system's glibc is too new or you link other shared libraries, it will assign the `linux` tag.
You can also manually disable those checks and directly use native linux target with `--manylinux off`.

For full manylinux compliance you need to compile in a CentOS docker container. The [pyo3/maturin](https://ghcr.io/pyo3/maturin) image is based on the manylinux2010 image,
For full manylinux compliance you need to compile in a CentOS docker container. The [pyo3/maturin](https://ghcr.io/pyo3/maturin) image is based on the manylinux2014 image,
and passes arguments to the `maturin` binary. You can use it like this:

```
Expand Down
12 changes: 6 additions & 6 deletions guide/src/distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ sdist-include = ["path/**/*"]
## Build Wheels

For portability reasons, native python modules on linux must only dynamically link a set of very few libraries which are installed basically everywhere, hence the name manylinux.
The pypa offers special docker images and a tool called [auditwheel](https://github.com/pypa/auditwheel/) to ensure compliance with the [manylinux rules](https://www.python.org/dev/peps/pep-0571/#the-manylinux2010-policy).
The pypa offers special docker images and a tool called [auditwheel](https://github.com/pypa/auditwheel/) to ensure compliance with the [manylinux rules](https://peps.python.org/pep-0599/#the-manylinux2014-policy)).
If you want to publish widely usable wheels for linux pypi, **you need to use a manylinux docker image** or [build with zig](#use-zig).

The Rust compiler since version 1.47 [requires at least glibc 2.11](https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-1470-2020-10-08), so you need to use at least manylinux2010.
The Rust compiler since version 1.64 [requires at least glibc 2.17](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html), so you need to use at least manylinux2014.
For publishing, we recommend enforcing the same manylinux version as the image with the manylinux flag, e.g. use `--manylinux 2014` if you are building in `quay.io/pypa/manylinux2014_x86_64`.
The [messense/maturin-action](https://github.com/messense/maturin-action) github action already takes care of this if you set e.g. `manylinux: 2014`.

Expand Down Expand Up @@ -93,9 +93,9 @@ OPTIONS:
Options are `manylinux` tags (for example `manylinux2014`/`manylinux_2_24`) or
`musllinux` tags (for example `musllinux_1_2`) and `linux` for the native linux tag.

Note that `manylinux1` is unsupported by the rust compiler. Wheels with the native
`linux` tag will be rejected by pypi, unless they are separately validated by
`auditwheel`.
Note that `manylinux1` and `manylinux2010` is unsupported by the rust compiler.
Wheels with the native `linux` tag will be rejected by pypi, unless they are separately
validated by `auditwheel`.

The default is the lowest compatible `manylinux` tag, or plain `linux` if nothing
matched
Expand All @@ -122,7 +122,7 @@ OPTIONS:
--zig
For manylinux targets, use zig to ensure compliance for the chosen manylinux version

Default to manylinux2010/manylinux_2_12 if you do not specify an `--compatibility`
Default to manylinux2014/manylinux_2_17 if you do not specify an `--compatibility`

Make sure you installed zig with `pip install maturin[zig]`

Expand Down
14 changes: 11 additions & 3 deletions guide/src/platform_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ GitHub action and these three platforms.

The following targets are built into wheels and downloadable binaries:

* windows: 32-bit and 64-bit x86
* linux 32-bit and 64-bit x86 as well as armv7 and aarch64 (musl)
* macOS: 64-bit and aarch64
* Windows: 32-bit and 64-bit x86 as well as arm64
* Linux: x86, x86_64, armv7, aarch64 and ppc64le (musl), as well as s390x (gnu)
* macOS: x86_64 and aarch64

## Other Operating Systems

Expand All @@ -37,3 +37,11 @@ CPython 3.7 to 3.10 are supported and tested on CI, though the entire 3.x series
This will be changed as new python versions are released and others have their end of life.

PyPy 3.6 and later also works.

## Manylinux/Musllinux

`manylinux2014` and its newer versions as well as `musllinux_1_1` and its newer versions
are supported.

Since Rust and the manylinux project drop support for old manylinux/musllinux versions sometimes,
after maturin 1.0 manylinux version bumps will be minor versions rather than major versions.
5 changes: 2 additions & 3 deletions src/auditwheel/audit.rs
Original file line number Diff line number Diff line change
Expand Up @@ -363,14 +363,13 @@ pub fn auditwheel_rs(
///
/// Currently only gcc is supported, clang doesn't have a `--print-sysroot` option
pub fn get_sysroot_path(target: &Target) -> Result<PathBuf> {
use crate::target::get_host_target;
use std::process::{Command, Stdio};

if let Some(sysroot) = std::env::var_os("TARGET_SYSROOT") {
return Ok(PathBuf::from(sysroot));
}

let host_triple = get_host_target()?;
let host_triple = target.host_triple();
let target_triple = target.target_triple();
if host_triple != target_triple {
let mut build = cc::Build::new();
Expand All @@ -379,7 +378,7 @@ pub fn get_sysroot_path(target: &Target) -> Result<PathBuf> {
.cargo_metadata(false)
// opt_level, host and target are required
.opt_level(0)
.host(&host_triple)
.host(host_triple)
.target(target_triple);
let compiler = build
.try_get_compiler()
Expand Down
9 changes: 9 additions & 0 deletions src/auditwheel/platform_tag.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,15 @@ impl PlatformTag {
pub fn is_musllinux(&self) -> bool {
matches!(self, PlatformTag::Musllinux { .. })
}

/// Is it supported by Rust compiler and manylinux project
pub fn is_supported(&self) -> bool {
match self {
PlatformTag::Manylinux { x, y } => (*x, *y) >= (2, 17),
PlatformTag::Musllinux { .. } => true,
PlatformTag::Linux => true,
}
}
}

impl fmt::Display for PlatformTag {
Expand Down
20 changes: 12 additions & 8 deletions src/build_options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,9 @@ pub struct BuildOptions {
/// or `musllinux` tags (for example `musllinux_1_2`)
/// and `linux` for the native linux tag.
///
/// Note that `manylinux1` is unsupported by the rust compiler. Wheels with the native `linux` tag
/// will be rejected by pypi, unless they are separately validated by `auditwheel`.
/// Note that `manylinux1` and `manylinux2010` is unsupported by the rust compiler.
/// Wheels with the native `linux` tag will be rejected by pypi,
/// unless they are separately validated by `auditwheel`.
///
/// The default is the lowest compatible `manylinux` tag, or plain `linux` if nothing matched
///
Expand Down Expand Up @@ -177,7 +178,7 @@ pub struct BuildOptions {

/// For manylinux targets, use zig to ensure compliance for the chosen manylinux version
///
/// Default to manylinux2010/manylinux_2_12 if you do not specify an `--compatibility`
/// Default to manylinux2014/manylinux_2_17 if you do not specify an `--compatibility`
///
/// Make sure you installed zig with `pip install maturin[zig]`
#[clap(long)]
Expand Down Expand Up @@ -606,11 +607,14 @@ impl BuildOptions {
} else {
self.platform_tag
};
if platform_tags
.iter()
.any(|tag| tag == &PlatformTag::manylinux1())
{
eprintln!("⚠️ Warning: manylinux1 is unsupported by the Rust compiler.");

for platform_tag in &platform_tags {
if !platform_tag.is_supported() {
eprintln!(
"⚠️ Warning: {} is unsupported by the Rust compiler.",
platform_tag
);
}
}

match bridge {
Expand Down
3 changes: 1 addition & 2 deletions src/cross_compile.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use crate::target::get_host_target;
use crate::{PythonInterpreter, Target};
use anyhow::{bail, Result};
use fs_err::{self as fs, DirEntry};
Expand All @@ -8,7 +7,7 @@ use std::path::{Path, PathBuf};

pub fn is_cross_compiling(target: &Target) -> Result<bool> {
let target_triple = target.target_triple();
let host = get_host_target()?;
let host = target.host_triple();
if target_triple == host {
// Not cross-compiling
return Ok(false);
Expand Down
2 changes: 1 addition & 1 deletion src/python_interpreter/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@ impl PythonInterpreter {
}
InterpreterKind::PyPy => {
// pypy uses its version as part of the ABI, e.g.
// pypy 3.7 7.3 => numpy-1.20.1-pp37-pypy37_pp73-manylinux2010_x86_64.whl
// pypy 3.7 7.3 => numpy-1.20.1-pp37-pypy37_pp73-manylinux2014_x86_64.whl
format!(
"pp{major}{minor}-pypy{major}{minor}_{abi_tag}-{platform}",
major = self.major,
Expand Down
54 changes: 34 additions & 20 deletions src/target.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ use crate::python_interpreter::InterpreterKind;
use crate::{PlatformTag, PythonInterpreter};
use anyhow::{anyhow, bail, format_err, Context, Result};
use platform_info::*;
use rustc_version::VersionMeta;
use serde::Deserialize;
use std::env;
use std::fmt;
Expand Down Expand Up @@ -132,8 +133,8 @@ pub struct Target {
env: Environment,
triple: String,
cross_compiling: bool,
/// Host machine target triple
pub(crate) host_triple: String,
/// rustc version information
pub(crate) rustc_version: VersionMeta,
/// Is user specified `--target`
pub(crate) user_specified: bool,
}
Expand All @@ -148,7 +149,8 @@ impl Target {
Architecture, ArmArchitecture, Mips32Architecture, Mips64Architecture, OperatingSystem,
};

let host_triple = get_host_target()?;
let rustc_version = rustc_version_meta()?;
let host_triple = &rustc_version.host;
let (platform, triple) = if let Some(ref target_triple) = target_triple {
let platform: Triple = target_triple
.parse()
Expand Down Expand Up @@ -204,7 +206,7 @@ impl Target {
arch,
env: platform.environment,
triple,
host_triple,
rustc_version,
user_specified: target_triple.is_some(),
cross_compiling: false,
};
Expand Down Expand Up @@ -411,7 +413,16 @@ impl Target {
Arch::Aarch64 | Arch::Armv7L | Arch::Powerpc64 | Arch::Powerpc64Le | Arch::S390X => {
PlatformTag::manylinux2014()
}
Arch::X86 | Arch::X86_64 => PlatformTag::manylinux2010(),
Arch::X86 | Arch::X86_64 => {
let rust_1_64 = semver::Version::new(1, 64, 0);
// rustc 1.64.0 bumps glibc requirement to 2.17
// see https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html
if self.rustc_version.semver >= rust_1_64 {
PlatformTag::manylinux2014()
} else {
PlatformTag::manylinux2010()
}
}
Arch::Armv6L
| Arch::Wasm32
| Arch::Riscv64
Expand Down Expand Up @@ -442,11 +453,16 @@ impl Target {
}
}

/// Returns target triple string
/// Returns target triple as string
pub fn target_triple(&self) -> &str {
&self.triple
}

/// Returns host triple as string
pub fn host_triple(&self) -> &str {
&self.rustc_version.host
}

/// Returns true if the current platform is not windows
pub fn is_unix(&self) -> bool {
match self.os {
Expand Down Expand Up @@ -624,22 +640,20 @@ impl Target {
}
}

pub(crate) fn get_host_target() -> Result<String> {
let host = rustc_version::version_meta()
.map(|meta| meta.host)
.map_err(|err| match err {
rustc_version::Error::CouldNotExecuteCommand(e)
if e.kind() == std::io::ErrorKind::NotFound =>
{
anyhow!(
"rustc, the rust compiler, is not installed or not in PATH. \
fn rustc_version_meta() -> Result<VersionMeta> {
let meta = rustc_version::version_meta().map_err(|err| match err {
rustc_version::Error::CouldNotExecuteCommand(e)
if e.kind() == std::io::ErrorKind::NotFound =>
{
anyhow!(
"rustc, the rust compiler, is not installed or not in PATH. \
This package requires Rust and Cargo to compile extensions. \
Install it through the system's package manager or via https://rustup.rs/.",
)
}
err => anyhow!(err).context("Failed to run rustc to get the host target"),
})?;
Ok(host)
)
}
err => anyhow!(err).context("Failed to run rustc to get the host target"),
})?;
Ok(meta)
}

fn macosx_deployment_target(
Expand Down
Loading