Refactor: switch py toolchain to using python-build-standalone's new plain tarballs (no zstd) #54

thundergolfer · 2022-02-20T08:40:05Z

Upgrading to 0.7.0 rules_python and using its new python-build-standalone toolchain.

(Kept for posterity)

May not merge this, but playing with new 'install only' Python interpreters from the https://github.com/indygreg/python-build-standalone project.

Linux

Running into trouble with the 'install only' interpreter containing files with spaces in them.

$ bazel run //3rdparty:requirements.update
INFO: Invocation ID: e43b2c6f-aa86-4311-bedc-f7d21f613dee
INFO: Analyzed target //3rdparty:requirements.update (7 packages loaded, 4953 targets configured).
INFO: Found 1 target...
ERROR: /workspaces/example-bazel-monorepo/3rdparty/BUILD:5:25: Creating runfiles tree bazel-out/k8-fastbuild/bin/3rdparty/requirements.update.runfiles failed: build-runfiles failed: error executing command 
  (cd /home/codespace/.cache/bazel/_bazel_codespace/e8a3c85ddfdf8cf57fd910efff34e41d/execroot/example_bazel_monorepo && \
  exec env - \
    PATH=/home/codespace/.cache/bazelisk/downloads/bazelbuild/bazel-4.1.0-linux-x86_64/bin:/home/codespace/.rbenv/shims:/vscode/bin/linux-x64/f80445acd5a3dadef24aa209168452a3d97cc326/bin/remote-cli:/home/codespace/.nvm/current/bin:/home/codespace/.npm-global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/codespace/.dotnet:/home/codespace/.dotnet/tools:/usr/local/sdkman/bin:/usr/local/sdkman/candidates/gradle/current/bin:/usr/local/sdkman/candidates/java/current/bin:/opt/maven/lts:/usr/local/cargo/bin:/usr/local/go/bin:/go/bin:/usr/local/py-utils/bin:/opt/conda/condabin:/home/codespace/.java/current/bin:/home/codespace/.nodejs/current/bin:/home/codespace/.php/current/bin:/home/codespace/.python/current/bin:/home/codespace/.ruby/current/bin:/home/codespace/.maven/current/bin:/home/codespace/.hugo/current/bin:/opt/oryx:/opt/nodejs/lts/bin:/opt/dotnet/lts:/opt/python/latest/bin:/opt/php/lts/bin:/opt/php-composer:/opt/yarn/stable/bin:/opt/hugo/lts::/opt/java/lts/bin:/opt/maven/lts/bin:/opt/ruby/lts/bin:/home/codespace/.local/bin:/usr/local/rvm/bin \
  /home/codespace/.cache/bazel/_bazel_codespace/install/f95ca91ebc34d56aa0f8ad499de91220/build-runfiles bazel-out/k8-fastbuild/bin/3rdparty/requirements.update.runfiles_manifest bazel-out/k8-fastbuild/bin/3rdparty/requirements.update.runfiles): Process exited with status 1: Process exited with status 1
/home/codespace/.cache/bazel/_bazel_codespace/install/f95ca91ebc34d56aa0f8ad499de91220/build-runfiles (args bazel-out/k8-fastbuild/bin/3rdparty/requirements.update.runfiles_manifest bazel-out/k8-fastbuild/bin/3rdparty/requirements.update.runfiles): link or target filename contains space on line 3111: 'example_bazel_monorepo/external/python_interpreter/python/lib/python3.10/site-packages/setuptools/command/launcher manifest.xml /home/codespace/.cache/bazel/_bazel_codespace/e8a3c85ddfdf8cf57fd910efff34e41d/external/python_interpreter/python/lib/python3.10/site-packages/setuptools/command/launcher manifest.xml'

Target //3rdparty:requirements.update failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.693s, Critical Path: 0.25s
INFO: 4 processes: 4 internal.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

OSX (x86)

Big errors to do with clang on my Macbook Pro 2018:

Error logs

bazel build @python_interpreter//...          ✔  2.7.2 
INFO: Invocation ID: 72dc1834-2b34-4ca7-a057-54e590e15702
INFO: Repository mypy_integration_pip_deps instantiated at:
  /Users/jonathon/Code/thundergolfer/example-bazel-monorepo/WORKSPACE:145:22: in <toplevel>
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/mypy_integration/repositories/deps.bzl:18:12: in deps
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/mypy_integration/repositories/py_repositories.bzl:18:20: in py_deps
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip.bzl:54:19: in pip_install
Repository rule pip_repository defined at:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/pip_repository.bzl:177:33: in <toplevel>
ERROR: An error occurred during the fetch of repository 'mypy_integration_pip_deps':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/pip_repository.bzl", line 114, column 13, in _pip_repository_impl
		fail("rules_python failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: rules_python failed: Collecting mypy==0.812
  Downloading mypy-0.812-py3-none-any.whl (2.5 MB)
Collecting mypy-extensions<0.5.0,>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typed-ast<1.5.0,>=1.4.0
  Using cached typed_ast-1.4.3.tar.gz (210 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Saved ./mypy-0.812-py3-none-any.whl
Saved ./mypy_extensions-0.4.3-py2.py3-none-any.whl
Saved ./typing_extensions-4.1.1-py3-none-any.whl
Building wheels for collected packages: typed-ast
  Building wheel for typed-ast (setup.py): started
  Building wheel for typed-ast (setup.py): finished with status 'error'
  Running setup.py clean for typed-ast
Failed to build typed-ast
 (  ERROR: Command errored out with exit status 1:
   command: /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"'; __file__='"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-eoe65tzy
       cwd: /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/
  Complete output (30 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.10
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/conversions.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/__init__.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast3.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast27.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  package init file 'ast3/tests/__init__.py' not found (or not a regular file)
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  copying ast3/tests/test_basics.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  running build_ext
  building '_ast27' extension
  creating build/temp.macosx-10.9-x86_64-3.10
  creating build/temp.macosx-10.9-x86_64-3.10/ast27
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Custom
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Parser
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Python
  Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
  Please check your Xcode installation
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -Iast27/Include -I/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-x86_64-3.10/ast27/Custom/typed_ast.o
  clang: warning: no such sysroot directory: '/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' [-Wmissing-sysroot]
  In file included from ast27/Custom/typed_ast.c:1:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__main__.py", line 5, in <module>
    main()
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__init__.py", line 69, in main
    subprocess.run(pip_args, check=True)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3', '-m', 'pip', '--isolated', 'wheel', '-r', '/Users/jonathon/Code/thundergolfer/example-bazel-monorepo/tools/typing/mypy_version.txt']' returned non-zero exit status 1.
)
ERROR: Error fetching repository: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/pip_repository.bzl", line 114, column 13, in _pip_repository_impl
		fail("rules_python failed: %s (%s)" % (result.stdout, result.stderr))
Error in fail: rules_python failed: Collecting mypy==0.812
  Downloading mypy-0.812-py3-none-any.whl (2.5 MB)
Collecting mypy-extensions<0.5.0,>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typed-ast<1.5.0,>=1.4.0
  Using cached typed_ast-1.4.3.tar.gz (210 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Saved ./mypy-0.812-py3-none-any.whl
Saved ./mypy_extensions-0.4.3-py2.py3-none-any.whl
Saved ./typing_extensions-4.1.1-py3-none-any.whl
Building wheels for collected packages: typed-ast
  Building wheel for typed-ast (setup.py): started
  Building wheel for typed-ast (setup.py): finished with status 'error'
  Running setup.py clean for typed-ast
Failed to build typed-ast
 (  ERROR: Command errored out with exit status 1:
   command: /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"'; __file__='"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-eoe65tzy
       cwd: /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/
  Complete output (30 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.10
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/conversions.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/__init__.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast3.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast27.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  package init file 'ast3/tests/__init__.py' not found (or not a regular file)
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  copying ast3/tests/test_basics.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  running build_ext
  building '_ast27' extension
  creating build/temp.macosx-10.9-x86_64-3.10
  creating build/temp.macosx-10.9-x86_64-3.10/ast27
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Custom
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Parser
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Python
  Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
  Please check your Xcode installation
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -Iast27/Include -I/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-x86_64-3.10/ast27/Custom/typed_ast.o
  clang: warning: no such sysroot directory: '/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' [-Wmissing-sysroot]
  In file included from ast27/Custom/typed_ast.c:1:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__main__.py", line 5, in <module>
    main()
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__init__.py", line 69, in main
    subprocess.run(pip_args, check=True)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3', '-m', 'pip', '--isolated', 'wheel', '-r', '/Users/jonathon/Code/thundergolfer/example-bazel-monorepo/tools/typing/mypy_version.txt']' returned non-zero exit status 1.
)
ERROR: /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/BUILD.bazel:8:10: @python_interpreter//:interpreter depends on @mypy_integration//mypy:mypy in repository @mypy_integration which failed to fetch. no such package '@mypy_integration_pip_deps//': rules_python failed: Collecting mypy==0.812
  Downloading mypy-0.812-py3-none-any.whl (2.5 MB)
Collecting mypy-extensions<0.5.0,>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typed-ast<1.5.0,>=1.4.0
  Using cached typed_ast-1.4.3.tar.gz (210 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Saved ./mypy-0.812-py3-none-any.whl
Saved ./mypy_extensions-0.4.3-py2.py3-none-any.whl
Saved ./typing_extensions-4.1.1-py3-none-any.whl
Building wheels for collected packages: typed-ast
  Building wheel for typed-ast (setup.py): started
  Building wheel for typed-ast (setup.py): finished with status 'error'
  Running setup.py clean for typed-ast
Failed to build typed-ast
 (  ERROR: Command errored out with exit status 1:
   command: /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"'; __file__='"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-eoe65tzy
       cwd: /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/
  Complete output (30 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.10
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/conversions.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/__init__.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast3.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast27.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  package init file 'ast3/tests/__init__.py' not found (or not a regular file)
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  copying ast3/tests/test_basics.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  running build_ext
  building '_ast27' extension
  creating build/temp.macosx-10.9-x86_64-3.10
  creating build/temp.macosx-10.9-x86_64-3.10/ast27
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Custom
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Parser
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Python
  Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
  Please check your Xcode installation
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -Iast27/Include -I/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-x86_64-3.10/ast27/Custom/typed_ast.o
  clang: warning: no such sysroot directory: '/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' [-Wmissing-sysroot]
  In file included from ast27/Custom/typed_ast.c:1:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__main__.py", line 5, in <module>
    main()
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__init__.py", line 69, in main
    subprocess.run(pip_args, check=True)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3', '-m', 'pip', '--isolated', 'wheel', '-r', '/Users/jonathon/Code/thundergolfer/example-bazel-monorepo/tools/typing/mypy_version.txt']' returned non-zero exit status 1.
)
ERROR: Analysis of aspect '@mypy_integration//:mypy.bzl%mypy_aspect of @python_interpreter//:interpreter' failed; build aborted: no such package '@mypy_integration_pip_deps//': rules_python failed: Collecting mypy==0.812
  Downloading mypy-0.812-py3-none-any.whl (2.5 MB)
Collecting mypy-extensions<0.5.0,>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typed-ast<1.5.0,>=1.4.0
  Using cached typed_ast-1.4.3.tar.gz (210 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Saved ./mypy-0.812-py3-none-any.whl
Saved ./mypy_extensions-0.4.3-py2.py3-none-any.whl
Saved ./typing_extensions-4.1.1-py3-none-any.whl
Building wheels for collected packages: typed-ast
  Building wheel for typed-ast (setup.py): started
  Building wheel for typed-ast (setup.py): finished with status 'error'
  Running setup.py clean for typed-ast
Failed to build typed-ast
 (  ERROR: Command errored out with exit status 1:
   command: /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"'; __file__='"'"'/private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-eoe65tzy
       cwd: /private/var/folders/fl/lnbyws8j5n92n577yzpbbys40000gn/T/pip-wheel-j9v7q2ca/typed-ast_bedb397de09941c39ad16642ccbd3d8a/
  Complete output (30 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.10
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/conversions.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/__init__.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast3.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  copying typed_ast/ast27.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast
  package init file 'ast3/tests/__init__.py' not found (or not a regular file)
  creating build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  copying ast3/tests/test_basics.py -> build/lib.macosx-10.9-x86_64-3.10/typed_ast/tests
  running build_ext
  building '_ast27' extension
  creating build/temp.macosx-10.9-x86_64-3.10
  creating build/temp.macosx-10.9-x86_64-3.10/ast27
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Custom
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Parser
  creating build/temp.macosx-10.9-x86_64-3.10/ast27/Python
  Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
  Please check your Xcode installation
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -Iast27/Include -I/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-x86_64-3.10/ast27/Custom/typed_ast.o
  clang: warning: no such sysroot directory: '/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' [-Wmissing-sysroot]
  In file included from ast27/Custom/typed_ast.c:1:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__main__.py", line 5, in <module>
    main()
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/rules_python/python/pip_install/extract_wheels/__init__.py", line 69, in main
    subprocess.run(pip_args, check=True)
  File "/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/bin/python3', '-m', 'pip', '--isolated', 'wheel', '-r', '/Users/jonathon/Code/thundergolfer/example-bazel-monorepo/tools/typing/mypy_version.txt']' returned non-zero exit status 1.
)
INFO: Elapsed time: 2.423s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 1 target configured)
    currently loading: @mypy_integration//mypy

The interesting part is I think this bit:

  Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
  Please check your Xcode installation
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpyuvhwu_p/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -Iast27/Include -I/private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-x86_64-3.10/ast27/Custom/typed_ast.o
  clang: warning: no such sysroot directory: '/Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' [-Wmissing-sysroot]
  In file included from ast27/Custom/typed_ast.c:1:
  /private/var/tmp/_bazel_jonathon/06ae70e3a686adbadc5f109e3045a096/external/python_interpreter/python/include/python3.10/Python.h:25:10: fatal error: 'stdio.h' file not found
  #include <stdio.h>
           ^~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
ERROR: Failed to build one or more wheels

`typing` package issue

black is pulling in mypy-extensions which is trying to get the typing package.

INFO: Invocation ID: 0e6b81f4-4b9d-4252-a439-c52d1f86219b
ERROR: /home/codespace/.cache/bazel/_bazel_codespace/e8a3c85ddfdf8cf57fd910efff34e41d/external/py_deps/pypi__mypy_extensions/BUILD.bazel:11:11: no such package '@py_deps//pypi__typing': BUILD file not found in directory 'pypi__typing' of external repository @py_deps. Add a BUILD file to a directory to mark it as a package. and referenced by '@py_deps//pypi__mypy_extensions:pypi__mypy_extensions'
ERROR: Analysis of target '//py_antilibrary:repl' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.200s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

BUILD file of the mypy-extensions package:

cat bazel-example-bazel-monorepo/external/py_deps/pypi__mypy_extensions/BUILD.bazel 
package(default_visibility = ["//visibility:public"])

load("@rules_python//python:defs.bzl", "py_library")

filegroup(
    name="whl",
    srcs=glob(["*.whl"], allow_empty = True),
    data=["//pypi__typing:whl"]
)

py_library(
    name = "pypi__mypy_extensions",
    srcs = glob(["**/*.py"], allow_empty = True),
    data = glob(["**/*"], exclude=["*.whl", "**/*.py", "**/* *", "BUILD.bazel", "WORKSPACE"]),
    # This makes this directory a top-level in the python import
    # search path for anything that depends on this.
    imports = ["."],
    deps = ["//pypi__typing"],
)

OSX (M1)

todo

Windows

todo

…alls (no zstd)

switch py toolchain to using python-build-standalone's new plain tarb…

d3e7c73

…alls (no zstd)

thundergolfer mentioned this pull request Feb 20, 2022

feat: cpython toolchains bazelbuild/rules_python#618

Merged

12 tasks

Jonathon Belotti added 3 commits March 15, 2022 18:21

upgrade to rules_python 0.7.0, changing to the new py toolchain

821a4cc

remove now obsolete py toolchain bzl code

4adc603

lint

94469bc

thundergolfer marked this pull request as ready for review March 15, 2022 11:00

thundergolfer merged commit e6e0288 into master Mar 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor: switch py toolchain to using python-build-standalone's new plain tarballs (no zstd) #54

Refactor: switch py toolchain to using python-build-standalone's new plain tarballs (no zstd) #54

thundergolfer commented Feb 20, 2022 •

edited

Loading

Refactor: switch py toolchain to using python-build-standalone's new plain tarballs (no zstd) #54

Refactor: switch py toolchain to using python-build-standalone's new plain tarballs (no zstd) #54

Conversation

thundergolfer commented Feb 20, 2022 • edited Loading

Linux

OSX (x86)

typing package issue

OSX (M1)

Windows

thundergolfer commented Feb 20, 2022 •

edited

Loading

`typing` package issue