Strange behavior with PyPy in 0.10 prerelease #478

messense · 2021-03-30T05:17:05Z

Please provide the following information:

Your python version (python -V): PyPy 7.3.3 with Python (3.6.12)
Your pip version (pip -V):
- If this isn't the latest pip version, have you checked using the latest pip version?
The version of the bindings you're using, if any (e.g. pyo3, rust-cpython or cffi): pyo3
Does cargo build work? I think so
If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

Please list the exact steps required to reproduce your error with all command output and if possible with a repository:

maturin-0.10.0b6 , https://github.com/milesgranger/pyrus-cramjam/runs/2218894957?check_suite_focus=true errored with

maturin build -i /opt/hostedtoolcache/PyPy/3.6.12/x64/bin/pypy --release --out dist
🔗 Found pyo3 bindings
🐍 Found PyPy 3.6 at /opt/hostedtoolcache/PyPy/3.6.12/x64/bin/pypy
📦 Built source distribution to dist/cramjam-2.1.0.tar.gz
warning: version requirement `0.6.0+zstd.1.4.8` for dependency `zstd` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
   Compiling cfg-if v1.0.0
   Compiling scopeguard v1.1.0
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-musl` target may not be installed


error: aborting due to previous error


error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-musl` target may not be installed


error: aborting due to previous error


error: could not compile `scopeguard`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit code: 101": `cargo rustc --message-format json --manifest-path Cargo.toml --release --target x86_64-unknown-linux-musl --lib --`

It's strange that --target x86_64-unknown-linux-musl is passed to cargo rustc, it works fine with maturin 0.9.4

The text was updated successfully, but these errors were encountered:

messense · 2021-03-30T14:34:18Z

Run make pypy-build
printenv
POWERSHELL_DISTRIBUTION_CHANNEL=GitHub-Actions-ubuntu20
ANDROID_NDK_ROOT=/usr/local/lib/android/sdk/ndk-bundle
GITHUB_EVENT_NAME=push
GITHUB_ACTION=run2
GITHUB_REPOSITORY=messense/pyrus-cramjam
GRADLE_HOME=/usr/share/gradle
GITHUB_WORKFLOW=CI
PERFLOG_LOCATION_SETTING=RUNNER_PERFLOG
JOURNAL_STREAM=9:20396
LEIN_HOME=/usr/local/lib/lein
GITHUB_GRAPHQL_URL=https://api.github.com/graphql
GOROOT_1_16_X64=/opt/hostedtoolcache/go/1.16.2/x64
GITHUB_BASE_REF=
GITHUB_ENV=/home/runner/work/_temp/_runner_file_commands/set_env_1631e229-1032-45b1-8424-be397b8375f9
_=/usr/bin/make
ANDROID_NDK_LATEST_HOME=/usr/local/lib/android/sdk/ndk/22.0.7026061
RUNNER_PERFLOG=/home/runner/perflog
RUNNER_TOOL_CACHE=/opt/hostedtoolcache
DOTNET_MULTILEVEL_LOOKUP=0
DEPLOYMENT_BASEPATH=/opt/runner
DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
PATH=/opt/hostedtoolcache/PyPy/3.6.12/x64/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/opt/pipx_bin:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
ImageOS=ubuntu20
GITHUB_ACTION_REF=
GRAALVM_11_ROOT=/usr/local/graalvm/graalvm-ce-java11-21.0.0.2
GITHUB_SERVER_URL=https://github.com
PIPX_BIN_DIR=/opt/pipx_bin
PIPX_HOME=/opt/pipx
GITHUB_RUN_NUMBER=25
INVOCATION_ID=b26d52d28bae473d9271824bdf42ae91
ANDROID_HOME=/usr/local/lib/android/sdk
GITHUB_ACTOR=messense
LEIN_JAR=/usr/local/lib/lein/self-installs/leiningen-2.9.5-standalone.jar
AZURE_EXTENSION_DIR=/opt/az/azcliextensions
CONDA=/usr/share/miniconda
JAVA_HOME_8_X64=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64
HOMEBREW_CELLAR="/home/linuxbrew/.linuxbrew/Cellar"
ANT_HOME=/usr/share/ant
ImageVersion=20210318.0
GITHUB_REPOSITORY_OWNER=messense
HOME=/home/runner
GECKOWEBDRIVER=/usr/local/share/gecko_driver
RUNNER_TEMP=/home/runner/work/_temp
VCPKG_INSTALLATION_ROOT=/usr/local/share/vcpkg
GITHUB_REF=refs/heads/buffer-protocol
GITHUB_PATH=/home/runner/work/_temp/_runner_file_commands/add_path_1631e229-1032-45b1-8424-be397b8375f9
ANDROID_NDK_HOME=/usr/local/lib/android/sdk/ndk-bundle
CI=true
pythonLocation=/opt/hostedtoolcache/PyPy/3.6.12/x64/bin
HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS=3650
GOROOT_1_15_X64=/opt/hostedtoolcache/go/1.15.10/x64
GITHUB_JOB=pypy
JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64
USER=runner
GITHUB_ACTION_REPOSITORY=
BOOTSTRAP_HASKELL_NONINTERACTIVE=1
GITHUB_ACTIONS=true
GITHUB_HEAD_REF=
DOTNET_NOLOGO=1
RUNNER_TRACKING_ID=github_118b9323-74cd-4865-b595-f68a70ada47e
MAKEFLAGS=
MFLAGS=
HOMEBREW_PREFIX="/home/linuxbrew/.linuxbrew"
GITHUB_EVENT_PATH=/home/runner/work/_temp/_github_workflow/event.json
AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
HOMEBREW_REPOSITORY="/home/linuxbrew/.linuxbrew/Homebrew"
ACCEPT_EULA=Y
HOMEBREW_NO_AUTO_UPDATE=1
GOROOT_1_14_X64=/opt/hostedtoolcache/go/1.14.15/x64
RUNNER_USER=runner
GITHUB_API_URL=https://api.github.com
ANDROID_SDK_ROOT=/usr/local/lib/android/sdk
***
GITHUB_RUN_ID=701759626
SWIFT_PATH=/usr/share/swift/usr/bin
DEBIAN_FRONTEND=noninteractive
CHROME_BIN=/usr/bin/google-chrome
GITHUB_WORKSPACE=/home/runner/work/pyrus-cramjam/pyrus-cramjam
GITHUB_SHA=731a843c9d0506842470cd2b3e06f78410d6aa6b
RUNNER_WORKSPACE=/home/runner/work/pyrus-cramjam
CHROMEWEBDRIVER=/usr/local/share/chrome_driver
SHLVL=1
JAVA_HOME_11_X64=/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64
RUNNER_OS=Linux
GITHUB_RETENTION_DAYS=90
SELENIUM_JAR_PATH=/usr/share/java/selenium-server-standalone.jar
LANG=C.UTF-8
MAKELEVEL=1

printenv shows no sign of CARGO_BUILD_TARGET.

konstin · 2021-04-02T12:42:34Z

Sounds like Platform::guess_current() guessed wrong. I think this can be fixed by reverting target: &str to target: Option<&str> in compile_target, because than we aren't reliant on the guess of platforms. If you can confirm that platforms is wrong for that ci environment it would also make sense to report this upstream.

messense · 2021-04-02T12:45:45Z

Sounds like Platform::guess_current() guessed wrong. I think this can be fixed by reverting target: &str to target: Option<&str> in compile_target, because than we aren't reliant on the guess of platforms. If you can confirm that platforms is wrong for that ci environment it would also make sense to report this upstream.

🔗 Found pyo3 bindings
Target: Target {
    os: Linux,
    arch: X86_64,
    env: Some(
        GNU,
    ),
    triple: "x86_64-unknown-linux-gnu",
}
🐍 Found PyPy 3.6 at /opt/hostedtoolcache/PyPy/3.6.12/x64/bin/pypy

I've tested it in https://github.com/messense/pyrus-cramjam/runs/2228798334?check_suite_focus=true , it seems fine. 😱

al626 · 2021-04-03T22:18:31Z

Also causing issues in CPython builds for rust-numpy: https://github.com/PyO3/rust-numpy/pull/179/checks?check_run_id=2260903749

poetry run maturin develop
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at python
   Compiling cfg-if v1.0.0
   Compiling smallvec v1.6.1
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-musl` target may not be installed


error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-musl` target may not be installed


error: aborting due to previous error


error: aborting due to previous error


error: could not compile `cfg-if`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit code: 101": `cargo rustc --message-format json --manifest-path Cargo.toml --target x86_64-unknown-linux-musl --lib --`
Error: Process completed with exit code 1.

Strangely, if I run the above with a locally built (debug) version of maturin (tag v0.10.0) it succeeds. I've instrumented this to print out version info and musl is not mentioned:

diff --git a/src/target.rs b/src/target.rs
index f875c44..4b6623e 100644
--- a/src/target.rs
+++ b/src/target.rs
@@ -107,6 +107,9 @@ impl Target {
             (Os::Windows, Arch::Armv7L) => bail!("armv7l is not supported for Windows"),
             (_, _) => {}
         }
+
+        println!("Created target: {:?} {:?}, {:?}: {:?}", os, arch, platform.target_env, platform.target_triple);
+
         Ok(Target {
             os,
             arch,

Created target: Linux X86_64, Some(GNU): "x86_64-unknown-linux-gnu"

Also works with v0.9.4 from pip.

al626 · 2021-04-03T23:03:53Z

Sounds like Platform::guess_current() guessed wrong. I think this can be fixed by reverting target: &str to target: Option<&str> in compile_target, because than we aren't reliant on the guess of platforms. If you can confirm that platforms is wrong for that ci environment it would also make sense to report this upstream.

Yep, platform::guess_current looks like it works based on the env (platform is built on)[https://docs.rs/platforms/0.2.1/src/platforms/target/env.rs.html#91].

Compiling maturin for musl (as done when building the pip wheels) and then checking the platform returns musl, not gnu.

cargo build --target x86_64-unknown-linux-musl
...
poetry run maturin develop
Created target: Linux X86_64, Some(Musl): "x86_64-unknown-linux-musl"

Looks like the fix mentioned will work then

See PyO3#478

al626 mentioned this issue Apr 3, 2021

Publish with license field instead of license-file PyO3/rust-numpy#179

Merged

al626 added a commit to al626/maturin that referenced this issue Apr 4, 2021

Guessed platform may be incorrect, so do not force compilation with it

11687a9

See PyO3#478

al626 mentioned this issue Apr 4, 2021

Guessed platform may be incorrect, so do not force compilation with it #486

Closed

al626 added a commit to al626/maturin that referenced this issue Apr 4, 2021

Guessed platform may be incorrect, so do not force compilation with it

f57289a

See PyO3#478

messense mentioned this issue Apr 5, 2021

Read host target triple from rustc -vV #487

Merged

messense pushed a commit to messense/maturin that referenced this issue Apr 5, 2021

Guessed platform may be incorrect, so do not force compilation with it

4ee528c

See PyO3#478

messense pushed a commit to messense/maturin that referenced this issue Apr 5, 2021

Guessed platform may be incorrect, so do not force compilation with it

0ea33a2

See PyO3#478

wilsonzlin added a commit to wilsonzlin/minify-html that referenced this issue Apr 6, 2021

Use older maturin version due to PyO3/maturin#478

a2cc9d9

koehlma added a commit to koehlma/momba that referenced this issue Apr 6, 2021

downgrade Maturin due to PyO3/maturin#478

20be90b

rajivshah3 mentioned this issue Apr 8, 2021

failed build release aarch64-apple-ios rust-lang/cfg-if#44

Closed

messense closed this as completed in #487 Apr 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strange behavior with PyPy in 0.10 prerelease #478

Strange behavior with PyPy in 0.10 prerelease #478

messense commented Mar 30, 2021 •

edited

Loading

messense commented Mar 30, 2021

konstin commented Apr 2, 2021

messense commented Apr 2, 2021

al626 commented Apr 3, 2021

al626 commented Apr 3, 2021

Strange behavior with PyPy in 0.10 prerelease #478

Strange behavior with PyPy in 0.10 prerelease #478

Comments

messense commented Mar 30, 2021 • edited Loading

messense commented Mar 30, 2021

konstin commented Apr 2, 2021

messense commented Apr 2, 2021

al626 commented Apr 3, 2021

al626 commented Apr 3, 2021

messense commented Mar 30, 2021 •

edited

Loading