Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update spack to v0.21.1 #909

Merged
merged 65 commits into from
Jun 20, 2024
Merged

Update spack to v0.21.1 #909

merged 65 commits into from
Jun 20, 2024

Conversation

jonasjucker
Copy link
Contributor

@jonasjucker jonasjucker commented Jan 29, 2024

Upgrade spack to v0.21.1 in order to prepare the transition to Alps.

The following packages are not ported:

  • COSMO

Todo before merge:

  • Add upstreams on Daint with new paths

@jonasjucker jonasjucker marked this pull request as draft January 29, 2024 15:33
Copy link
Contributor

github-actions bot commented Jan 29, 2024

PR Preview Action v1.4.7
Preview removed because the pull request was closed.
2024-06-20 09:33 UTC

@dominichofer
Copy link
Contributor

I think the label "spack locking problem" is misleading now. The root cause of the error is that openssl failed to install. The assertion error on releasing the lock is just a subsequent error.

spack install --test=root -n -v py-cytoolz

==> Warning: openssl-3.1.3-t66yrt7wmawr7otxyka36t2ett7hnffh failed to install
==> Warning: Skipping build of python-3.11.6-ntoetbpq376egj2ufsnrtfhzzfhnsafu since openssl-3.1.3-t66yrt7wmawr7otxyka36t2ett7hnffh failed
==> Warning: Skipping build of py-cython-3.0.4-2zd4o4gjusj2cvwm5txtu4l76seepgdx since python-3.11.6-ntoetbpq376egj2ufsnrtfhzzfhnsafu failed
==> Warning: Skipping build of py-cytoolz-0.12.0-n3d3435kd5x22vkrcat7tvygxrxujrgp since py-cython-3.0.4-2zd4o4gjusj2cvwm5txtu4l76seepgdx failed
==> Warning: Skipping build of py-wheel-0.41.2-z74asospl6ystbalyciu55bbxlypzk6l since python-3.11.6-ntoetbpq376egj2ufsnrtfhzzfhnsafu failed
==> Warning: Skipping build of py-toolz-0.12.0-2je6wdnvz2tbc6qmfqaakb2q33do6xg6 since py-wheel-0.41.2-z74asospl6ystbalyciu55bbxlypzk6l failed
==> Warning: Skipping build of py-setuptools-68.0.0-ikroy3nlerhusvsecrvi3g372i6miht4 since python-3.11.6-ntoetbpq376egj2ufsnrtfhzzfhnsafu failed
==> Warning: Skipping build of py-pip-23.1.2-6ebbxaa7vgponajymyzpvj5a75rnafpl since python-3.11.6-ntoetbpq376egj2ufsnrtfhzzfhnsafu failed
==> Warning: AssertionError exception when releasing read lock for openssl-3.1.3-t66yrt7wmawr7otxyka36t2ett7hnffh: 

@jonasjucker
Copy link
Contributor Author

All Python deps of py-gt4py build using [email protected], except py-cytoolz:

Error compiling Cython file:
  ------------------------------------------------------------
  ...
      cdef readonly object func
      cdef readonly tuple args
      cdef readonly dict keywords
      cdef public object __doc__
      cdef public object __name__
      cdef public object __module__
           ^
  ------------------------------------------------------------

  cytoolz/functoolz.pxd:18:9: Compiler crash in AnalyseDeclarationsTransform

  File 'ModuleNode.py', line 203, in analyse_declarations: ModuleNode(functoolz.pxd:1:0,
      full_module_name = 'cytoolz.functoolz',
      is_pxd = True)

@jonasjucker
Copy link
Contributor Author

found solution here:
https://www.layerzrozero.network/?_=%2Fpytoolz%2Fcytoolz%2Fissues%2F202%23w2n%2BddnGqHIZTHHkluHJC3Vn

py-cytoolz 0.12.0 is not compatible with cython >= 3.
Restriction needs to be added in recipe.

@jonasjucker
Copy link
Contributor Author

launch jenkins daint py-gt4py py-icon4py

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary

@jonasjucker
Copy link
Contributor Author

launch jenkins daint infero pytorch-fortran

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary
🟢 integration test
Test
🟢py-gt4py-spack_info
🟢py-gt4py-spack_spec
🟢py-icon4py-spack_info
🟢py-icon4py-spack_spec
🔴 system test
Test
🔴PyGt4py-install_version_1_0_1_1
🔒PyGt4py-install_version_1_0_1_1b spack locking problem
🔴PyGt4py-install_version_1_0_1_6
🔴PyGt4py-install_version_1_0_1_7
🔒PyIcon4py-install_version_0_0_3_1 spack locking problem
🔒PyIcon4py-install_version_0_0_9 spack locking problem

WARNING: Serial tests did not run for system tests

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary
🟢 integration test
Test
🟢infero-spack_info
🟢infero_+onnx_+tf_c-spack_spec
🟢pytorch-fortran-spack_info
🟢pytorch-fortran-spack_spec
🟢 system test
Test
🟢Infero-install_onnx
🟢Infero-install_tf_c
🟢PytorchFortran-install_version_0_4

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

@jonasjucker
Copy link
Contributor Author

launch jenkins icon balfrin

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary
🟢 integration test
Test
🟢icon-spack_info
🟢icon-spack_spec
🟢icon_extra-config-args=--disable-new_feature,--enable-old_config_arg-spack_spec
🟢dace_icon.-O1-spack_spec
🟢icon_serialization=create-spack_spec
🔴 system test
Test
🟢Icon-install_2024_1_gcc
🔴Icon-install_2024_1_nvhpc
🔴Icon-install_conditional_dependencies

WARNING: Serial tests did not run for system tests

@dominichofer
Copy link
Contributor

launch jenkins icon balfrin

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary

@jonasjucker
Copy link
Contributor Author

launch jenkins daint py-gt4py py-icon4py icon

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jonasjucker
Copy link
Contributor Author

launch jenkins daint icon

@jenkins-apn
Copy link

tsa

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary
🟢 integration test
Test
🟢icon-spack_info
🟢icon-spack_spec
🟢icon_extra-config-args=--disable-new_feature,--enable-old_config_arg-spack_spec
🟢dace_icon.-O1-spack_spec
🟢icon_serialization=create-spack_spec
🔴 system test
Test
🔴daint_cpu_nvhpc
🔴daint_cpu_nvhpc_out_of_source

WARNING: Serial tests did not run for system tests

@jonasjucker
Copy link
Contributor Author

launch jenkins all balfrin

@jenkins-apn
Copy link

daint

🟢 unit test
Test
🟢summary

@jenkins-apn
Copy link

balfrin

🟢 unit test
Test
🟢summary
🟢 integration test
Test
🟢clang-format-spack_info
🟢clang-format-spack_spec
🟢cosmo-dycore-spack_info
🟢cosmo-dycore-spack_spec
🟢cosmo-dycore_+cuda_+gt1_+build_tests-spack_spec
🟢cosmo-dycore_~cuda_+gt1-spack_spec
🟢cosmo-eccodes-definitions-spack_info
🟢cosmo-eccodes-definitions-spack_spec
🟢cosmo-spack_info
🟢cosmo-spack_spec
🟢cosmo_cosmo_target=gpu_~cppdycore-spack_spec
🟢eccodes-spack_info
🟢eccodes-spack_spec
🟢fckit-spack_info
🟢fckit-spack_spec
🟢fdb-fortran-spack_info
🟢fdb-fortran-spack_spec
🟢fdb-spack_info
🟢flexpart-cosmo-spack_info
🟢flexpart-cosmo-spack_spec
🟢flexpart-ifs-spack_info
🟢flexpart-ifs-spack_spec
🟢gridtools-spack_info
🟢gridtools-spack_spec
🟢gridtools_+cuda-spack_spec
🟢gridtools_~cuda-spack_spec
🟢icon-ham-spack_info
🟢icon-ham-spack_spec
🟢icon-spack_info
🟢icon-spack_spec
🟢icon_extra-config-args=--disable-new_feature,--enable-old_config_arg-spack_spec
🟢dace_icon.-O1-spack_spec
🟢icon_serialization=create-spack_spec
🟢icontools-spack_info
🟢icontools-spack_spec
🟢infero-spack_info
🟢infero_+onnx_+tf_c-spack_spec
🟢int2lm-spack_info
🟢int2lm-spack_spec
🟢int2lm_+parallel-spack_spec
🟢libcdi-pio-spack_info
🟢libcdi-pio-spack_spec
🟢libfyaml-spack_spec
🟢libgrib1-spack_info
🟢libgrib1-spack_spec
🟢libtorch-spack_info
🟢libtorch-spack_spec
🟢makedepf90-spack_info
🟢nvidia-blas-spack_info
🟢nvidia-blas-spack_spec
🟢nvidia-lapack-spack_info
🟢nvidia-lapack-spack_spec
🟢oasis-spack_info
🟢oasis-spack_spec
🟢onnx-runtime-spack_info
🟢onnx-runtime-spack_spec
🟢py-asttokens-spack_info
🟢py-asttokens-spack_spec
🟢py-black-spack_info
🟢py-black-spack_spec
🟢py-boltons-spack_info
🟢py-boltons-spack_spec
🟢py-cytoolz-spack_info
🟢py-cytoolz-spack_spec
🟢py-devtools-spack_info
🟢py-devtools-spack_spec
🟢py-editables-spack_info
🟢py-editables-spack_spec
🟢py-executing-spack_info
🟢py-executing-spack_spec
🟢py-factory-boy-spack_info
🟢py-factory-boy-spack_spec
🟢py-fprettify-spack_info
🟢py-fprettify-spack_spec
🟢py-frozendict-spack_info
🟢py-frozendict-spack_spec
🟢py-gridtools-cpp-spack_info
🟢py-gridtools-cpp-spack_spec
🟢py-gt4py-spack_info
🟢py-gt4py-spack_spec
🟢py-hatchling-spack_info
🟢py-hatchling-spack_spec
🟢py-icon4py-spack_info
🟢py-icon4py-spack_spec
🟢py-inflection-spack_info
🟢py-inflection-spack_spec
🟢py-isort-spack_info
🟢py-isort-spack_spec
🟢py-lark-spack_info
🟢py-lark-spack_spec
🟢py-nanobind-spack_info
🟢py-nanobind-spack_spec
🟢py-pathspec-spack_info
🟢py-pathspec-spack_spec
🟢py-pytest-factoryboy-spack_info
🟢py-pytest-factoryboy-spack_spec
🟢py-pytest-spack_info
🟢py-pytest-spack_spec
🟢py-setuptools-spack_info
🟢py-setuptools-spack_spec
🟢py-sphinxcontrib-jquery-spack_info
🟢py-sphinxcontrib-jquery-spack_spec
🟢py-tabulate-spack_info
🟢py-tabulate-spack_spec
🟢py-typing-extensions-spack_info
🟢py-typing-extensions-spack_spec
🟢pytorch-fortran-proxy-spack_info
🟢pytorch-fortran-proxy-spack_spec
🟢pytorch-fortran-spack_info
🟢pytorch-fortran-spack_spec
🟢rttov-spack_info
🟢rttov-spack_spec
🟢scales-ppm-spack_info
🟢scales-ppm-spack_spec
🟢tensorflowc-spack_info
🟢tensorflowc-spack_spec
🟢yaxt-spack_info
🟢yaxt-spack_spec
🟢zlib_ng-spack_info
🟢zlib_ng-spack_spec
🟢 system test
Test
🟢ClangFormat-install_default
🟢CosmoEccodesDefinitions-install_version_2_19_0_7
🟢CosmoEccodesDefinitions-install_version_2_25_0_1
🟢Eccodes-install_2_19_0
🟢Fckit-install_0_9_0
🟢Fdb-install_5_11_17_gcc
🟢Fdb-install_5_11_17_nvhpc
🟢FdbFortran-install
🟢FlexpartCosmo-install
🟢FlexpartIfs-install_10_4_4
🟢FlexpartIfs-install_fdb
🟢GridTools-install_version_1_1_3_gcc
🟢GridTools-install_version_1_1_3_nvhpc
🟢Icon-install_2024_1_gcc
🟢Icon-install_2024_1_nvhpc
🟢Icon-install_conditional_dependencies
🟢IconTools-install_2_5_2
🟢Infero-install_onnx
🟢Infero-install_tf_c
🟢LibCdiPio-install_default
🟢LibTorch-install_default
🟢Libfyaml-install_default
🟢Makedepf90-install
🟢OnnxRuntime-install_default
🟢PyAsttokens-install_default
🟢PyBlack-install_default
🟢PyBoltons-install_default
🟢PyDevtools-install_default
🟢PyEditables-install_default
🟢PyExecuting-install_default
🟢PyFactoryBoy-install_default
🟢PyFprettify-install_default
🟢PyFrozendict-install_default
🟢PyGridtoolsCpp-install_default
🟢PyGt4py-install_version_1_0_3_3
🟢PyGt4py-install_version_1_0_3_6
🟢PyGt4py-install_version_1_0_3_7
🟢PyHatchling-install_default
🟢PyIcon4py-install_version_0_0_10
🟢PyIcon4py-install_version_0_0_11
🟢PyInflection-install_default
🟢PyIsort-install_default
🟢PyLark-install_default
🟢PyNanobind-install_default
🟢PyPathspec-install_default
🟢PyPytest-install_default
🟢PyPytestFactoryboy-install_default
🟢PySetuptools-install_default
🟢PySphinxcontribJquery-install_default
🟢PyTabulate-install_default
🟢PyTypingExtensions-install_default
🟢PytorchFortran-install_version_0_4
🟢PytorchFortranProxy-install_version_0_4
🟢ScalesPPM-install_default
🟢TensorflowC-install_2_6_0
🟢Yaxt-install_default
🟢ZLibNG-install_version_2_0_0

Copy link
Contributor

@dominichofer dominichofer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jonasjucker jonasjucker merged commit fd16d04 into main Jun 20, 2024
3 of 4 checks passed
@jonasjucker jonasjucker deleted the dev_v0.21.1 branch June 20, 2024 09:33
This was referenced Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants