Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tensorflow v2.14.0 #347

Merged
merged 12 commits into from
Nov 14, 2023
Merged

Conversation

regro-cf-autotick-bot
Copy link
Contributor

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

  • Dependencies have been updated if changed: see upstream
  • Tests have passed
  • Updated license if changed and license_file is packaged

Information about this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version.
  3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
  4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
  5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
icu 2023-10-04 Anaconda-Server Badge
libjpeg-turbo 9e Anaconda-Server Badge
tensorflow 2.14.0 Anaconda-Server Badge

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/6613951688, please use this URL for debugging.

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@h-vetinari h-vetinari marked this pull request as draft October 24, 2023 00:24
@xhochy
Copy link
Member

xhochy commented Oct 27, 2023

Needs conda-forge/keras-feedstock#68

@jan-janssen
Copy link
Member

@xhochy As conda-forge/keras-feedstock#68 is merged now - is there anything missing for this PR to be merged?

@jan-janssen
Copy link
Member

@conda-forge-admin, please restart ci

@xhochy
Copy link
Member

xhochy commented Oct 31, 2023

@conda-forge-admin, please restart ci

Please don't do this on the repo. CI is disabled anyways.

@xhochy
Copy link
Member

xhochy commented Oct 31, 2023

@xhochy As conda-forge/keras-feedstock#68 is merged now - is there anything missing for this PR to be merged?

Yes, a successful build. I have a set of five patches locally that I need to push but they are not exhaustive to get it even passing without CUDA.

@xhochy xhochy marked this pull request as ready for review November 9, 2023 09:42
@xhochy
Copy link
Member

xhochy commented Nov 9, 2023

CPU builds are passing, CUDA still has issues which could also relate to my available RAM.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 9, 2023

Ok i'll try one, i have a 128GB machine availble.

@xhochy
Copy link
Member

xhochy commented Nov 9, 2023

@hmaarrfk I can also do it. My machine is a cloud instance and I can just switch it to a larger one.

@xhochy
Copy link
Member

xhochy commented Nov 9, 2023

The CUDA 11.8 build passed. I will retry the 11.2 with more RAM. In general, this PR is ready for review. I will build out all variants over the weekend.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Nov 9, 2023

is Python 3.12 compatibility a thing?

Copy link
Contributor

@ngam ngam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@@ -1 +1,3 @@
tar -C ${PREFIX} -xf $SRC_DIR/libtensorflow_cc_output.tar
mkdir -p ${PREFIX}/include/tensorflow/tsl/platform/include
cp float8.h ${PREFIX}/include/tensorflow/tsl/platform/include
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to avoid this workaround? Non-blocker, but kind of surprising...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its somewhat strange because ml_dtypes is a python package. we could repackage ml_dtypes as a library.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in general a very strange interaction as Tensorflow also builds "a" ml_dtypes as a library internally again. I would rather move forward with this hack and hope that in 2.15 things are cleared up a bit more instead of trying to split the ml_dtypes package into pieces.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do they vendor the library? it seems scary to pin so hard on ml_dtypes 0.2.0

- six >=1.12
- termcolor >=1.1.0
- typing_extensions >=3.6.6,<4.6.0
- wrapt >=1.11.0
- typing_extensions >=3.6.6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@xhochy
Copy link
Member

xhochy commented Nov 9, 2023

is Python 3.12 compatibility a thing?

Not yet.

@xhochy
Copy link
Member

xhochy commented Nov 9, 2023

is Python 3.12 compatibility a thing?

Not yet.

It seems like we need to wait for 2.16. 3.12 support is not included in the 2.15 release candidate.

@njzjz
Copy link
Member

njzjz commented Nov 9, 2023

is Python 3.12 compatibility a thing?

In addition, https://conda-forge.org/status/ reports a conflict with protobuf, which might need to be resolved.

linux_64_c_compiler_version10cuda_compilernvcccuda_compiler_version11.2cxx_compiler_version10numpy1.26python3.12.____cpython: Could not solve for environment specs
The following packages are incompatible
├─ libprotobuf-python-headers is installable with the potential options
│  ├─ libprotobuf-python-headers 3.21.12 would require
│  │  └─ libprotobuf 21.12 , which can be installed;
│  ├─ libprotobuf-python-headers 3.21.12 would require
│  │  └─ libprotobuf 3.21.12 , which can be installed;
│  ├─ libprotobuf-python-headers 4.23.3 would require
│  │  └─ libprotobuf 4.23.3 , which can be installed;
│  ├─ libprotobuf-python-headers 4.23.4 would require
│  │  └─ libprotobuf 23.4 , which can be installed;
│  ├─ libprotobuf-python-headers 4.23.4 would require
│  │  └─ libprotobuf 4.23.4 , which can be installed;
│  ├─ libprotobuf-python-headers 4.24.3 would require
│  │  └─ libprotobuf 4.24.3 , which can be installed;
│  └─ libprotobuf-python-headers 4.24.4 would require
│     └─ libprotobuf 4.24.4 , which can be installed;
├─ libprotobuf 3.21.*  is not installable because it conflicts with any installable versions previously reported;
├─ python 3.12.* *_cpython is installable with the potential options
│  ├─ python 3.12.0 would require
│  │  └─ python_abi 3.12.* *_cp312, which can be installed;
│  └─ python 3.12.0rc3 would require
│     └─ _python_rc, which does not exist (perhaps a missing channel);
└─ tensorboard >=2.13,<2.14  is not installable because it requires
   └─ protobuf >=3.19.6  but there are no viable options
      ├─ protobuf [3.19.6|3.20.0|...|4.24.4] would require
      │  └─ python_abi 3.10.* *_cp310, which conflicts with any installable versions previously reported;
      ├─ protobuf 3.20.0 would require
      │  └─ python_abi 3.7 *_pypy37_pp73, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.0|3.20.1|...|4.21.8] would require
      │  └─ python_abi 3.7.* *_cp37m, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.0|3.20.1|...|4.21.9] would require
      │  └─ python_abi 3.8 *_pypy38_pp73, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.19.6|3.20.0|...|4.24.4] would require
      │  └─ python_abi 3.8.* *_cp38, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.0|3.20.1|...|4.21.9] would require
      │  └─ python_abi 3.9 *_pypy39_pp73, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.19.6|3.20.0|...|4.24.4] would require
      │  └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.2|3.20.3|...|4.24.4] would require
      │  └─ python_abi 3.11.* *_cp311, which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.3|4.22.5|...|4.23.4] would require
      │  └─ pypy3.8 >=7.3.11 , which conflicts with any installable versions previously reported;
      ├─ protobuf [3.20.3|4.22.5|...|4.23.4] would require
      │  └─ pypy3.9 >=7.3.11 , which conflicts with any installable versions previously reported;
      ├─ protobuf 4.23.4 would require
      │  └─ libprotobuf >=4.23.4,<4.23.5.0a0 , which conflicts with any installable versions previously reported;
      ├─ protobuf [4.23.4|4.24.3|4.24.4] would require
      │  └─ pypy3.9 >=7.3.13 , which conflicts with any installable versions previously reported;
      ├─ protobuf 4.24.3 would require
      │  └─ libprotobuf >=4.24.3,<4.24.4.0a0 , which conflicts with any installable versions previously reported;
      └─ protobuf 4.24.4 would require
         └─ libprotobuf >=4.24.4,<4.24.5.0a0 , which conflicts with any installable versions previously reported.

@xhochy
Copy link
Member

xhochy commented Nov 10, 2023

Compilation breaks for CUDA 11.2 with

`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIfE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEEEvSE
_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discarded section `.
text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIfE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEE
EvSE_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceEfE6LaunchEPNS_15OpKernelContextERKNS_6TensorES8_bbbbRKNS_11MatMulBCastEPS6_]' of bazel-out/k8-opt/bin
/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIN5Eigen8bfloat16EE3RunERKNS4_16ThreadPoolDeviceERKNS1_6TensorESC_bbbbPSA_ENKUlRT_llllE1_clIKNS2_15YCombinatorIm
plISG_EEEEvSF_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discard
ed section `.text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIN5Eigen8bfloat16EE3RunERKNS4_16ThreadPoolDeviceERKNS1_6TensorESC_bbbbPSA_ENKUlRT_llllE1_clIKNS
2_15YCombinatorImplISG_EEEEvSF_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceENS1_8bfloat16EE6LaunchEPNS_15OpKernelContextERKNS_6TensorES9_bbbbRKNS_11Mat
MulBCastEPS7_]' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIiE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEEEvSE
_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discarded section `.
text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIiE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEE
EvSE_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceEiE6LaunchEPNS_15OpKernelContextERKNS_6TensorES8_bbbbRKNS_11MatMulBCastEPS6_]' of bazel-out/k8-opt/bin
/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIlE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEEEvSE
_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discarded section `.
text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIlE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEE
EvSE_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceElE6LaunchEPNS_15OpKernelContextERKNS_6TensorES8_bbbbRKNS_11MatMulBCastEPS6_]' of bazel-out/k8-opt/bin
/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIdE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEEEvSE
_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discarded section `.
text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIdE3RunERKN5Eigen16ThreadPoolDeviceERKNS1_6TensorESB_bbbbPS9_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplISF_EEE
EvSE_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceEdE6LaunchEPNS_15OpKernelContextERKNS_6TensorES8_bbbbRKNS_11MatMulBCastEPS6_]' of bazel-out/k8-opt/bin
/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
`_ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIN5Eigen4halfEE3RunERKNS4_16ThreadPoolDeviceERKNS1_6TensorESC_bbbbPSA_ENKUlRT_llllE1_clIKNS2_15YCombinatorImplIS
G_EEEEvSF_llllEUlvE_E9_M_invokeERKSt9_Any_data' referenced in section `.data.rel.ro.local' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o): defined in discarded s
ection `.text._ZNSt17_Function_handlerIFvvEZZN10tensorflow12_GLOBAL__N_131SingleBatchParallelMatMulKernelIN5Eigen4halfEE3RunERKNS4_16ThreadPoolDeviceERKNS1_6TensorESC_bbbbPSA_ENKUlRT_llllE1_clIKNS2_15YCom
binatorImplISG_EEEEvSF_llllEUlvE_E9_M_invokeERKSt9_Any_data[_ZN10tensorflow17LaunchBatchMatMulIN5Eigen16ThreadPoolDeviceENS1_4halfEE6LaunchEPNS_15OpKernelContextERKNS_6TensorES9_bbbbRKNS_11MatMulBCastEPS7
_]' of bazel-out/k8-opt/bin/tensorflow/core/kernels/libmatmul_op.pic.lo(matmul_op_real.pic.o)
collect2: error: ld returned 1 exit status

As we are dropping CUDA 11.2 with a lot of other packages, I will do the same here.

@jakirkham
Copy link
Member

Makes sense 11.8 supports the same hardware

@hmaarrfk
Copy link
Contributor

In addition, https://conda-forge.org/status/ reports a conflict with protobuf, which might need to be resolved.

I feel like updating tensorflow, and updating protobuf with tensorflow can be considered two different steps. I don't think we should necessarily hold up this progress.

It isn't ideal, but we need to continue to ask upstream to make it easier to plug in "system" dependencies.

@hmaarrfk
Copy link
Contributor

export PACKAGE_VERSION=2.14.0
export DELEGATES_CHANNEL=uwe.korn-tf-gpu
export DELEGATES_LABEL=main
for package in tensorflow-base tensorflow tensorflow-estimator libtensorflow libtensorflow_cc tensorflow-cpu tensorflow-gpu; do
  anaconda copy --from-label ${DELEGATES_LABEL} --to-label main --to-owner conda-forge ${DELEGATES_CHANNEL}/${package}/${PACKAGE_VERSION}
done

export PACKAGE_VERSION=2.14.0
export DELEGATES_CHANNEL=uwe.korn-tf-experimental
export DELEGATES_LABEL=main
for package in tensorflow-base tensorflow tensorflow-estimator libtensorflow libtensorflow_cc tensorflow-cpu tensorflow-gpu; do
  anaconda copy --from-label ${DELEGATES_LABEL} --to-label main --to-owner conda-forge ${DELEGATES_CHANNEL}/${package}/${PACKAGE_VERSION} || true
done

@hmaarrfk hmaarrfk merged commit f0f57d5 into conda-forge:main Nov 14, 2023
@hmaarrfk
Copy link
Contributor

amazing work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants