[ML] Deprecate machine learning on Intel macOS #104087
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PyTorch is no longer going to provide macOS x86_64 builds after version 2.2. This doesn't instantly affect us, as we build PyTorch from source ourselves and they've said that they will not deliberately break macOS x86_64. They just won't build or test on that platform themselves. As a result it's inevitable that we'll have to make some tweaks to the PyTorch code to get it to build, and as the years go by it will become harder and harder to make the code compile on an unsupported platform. Since PyTorch is such a critical component of Elastic ML we won't be able to keep it running on macOS x86_64 for more than a few releases after PyTorch drops support. This change gives notice of our intentions.
|
Hi @droberts195, I've created a changelog YAML for you. Note that since this PR is labelled |
|
Pinging @elastic/ml-core (Team:ML) |
droberts195
added a commit
to droberts195/elasticsearch
that referenced
this pull request
Jan 9, 2024
As previously advised in elastic#104087, machine learning functionality will no longer be available on macOS x86_64. Machine learning functionality _is_ still available on macOS by using an arm64 machine (Apple silicon). It is also possible to run Elasticsearch with machine learning functionality within a Docker container on macOS x86_64.
💚 Backport successful
|
droberts195
added a commit
to droberts195/elasticsearch
that referenced
this pull request
Jan 16, 2024
PyTorch is no longer going to provide macOS x86_64 builds after version 2.2. This doesn't instantly affect us, as we build PyTorch from source ourselves and they've said that they will not deliberately break macOS x86_64. They just won't build or test on that platform themselves. As a result it's inevitable that we'll have to make some tweaks to the PyTorch code to get it to build, and as the years go by it will become harder and harder to make the code compile on an unsupported platform. Since PyTorch is such a critical component of Elastic ML we won't be able to keep it running on macOS x86_64 for more than a few releases after PyTorch drops support. This change gives notice of our intentions.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jan 16, 2024
PyTorch is no longer going to provide macOS x86_64 builds after version 2.2. This doesn't instantly affect us, as we build PyTorch from source ourselves and they've said that they will not deliberately break macOS x86_64. They just won't build or test on that platform themselves. As a result it's inevitable that we'll have to make some tweaks to the PyTorch code to get it to build, and as the years go by it will become harder and harder to make the code compile on an unsupported platform. Since PyTorch is such a critical component of Elastic ML we won't be able to keep it running on macOS x86_64 for more than a few releases after PyTorch drops support. This change gives notice of our intentions.
edsavage
added a commit
that referenced
this pull request
Nov 29, 2024
As previously advised in #104087, machine learning functionality will no longer be available on macOS x86_64. Machine learning functionality is still available on macOS by using an arm64 machine (Apple silicon). It is also possible to run Elasticsearch with machine learning functionality within a Docker container on macOS x86_64. This PR should be merged to main after the branch is split for the last minor release scheduled for before December 2024. For example, suppose 8.17.0 is scheduled for release in November 2024 and 8.18.0 is scheduled for release in January 2025. Then this PR should be merged to main after the 8.17 branch is split. One this PR is merged a followup PR should be opened against the ml-cpp repo to remove the build system for darwin-x86_64. It has been confirmed that with this change in place the Elasticsearch build system works with an ml-cpp bundle that does not contain a platform/darwin-x86_64 directory. It still produces an Elasticsearch build that will run providing xpack.ml.enabled is not explicitly set to true. After the build system for darwin-x86_64 has been removed from the ml-cpp repo, we will be able to do another PyTorch upgrade without having to worry about tweaking the build system to work on Intel macOS. --------- Co-authored-by: Ed Savage <[email protected]> Co-authored-by: Valeriy Khakhutskyy <[email protected]>
edsavage
added a commit
to edsavage/elasticsearch
that referenced
this pull request
Nov 29, 2024
As previously advised in elastic#104087, machine learning functionality will no longer be available on macOS x86_64. Machine learning functionality is still available on macOS by using an arm64 machine (Apple silicon). It is also possible to run Elasticsearch with machine learning functionality within a Docker container on macOS x86_64. This PR should be merged to main after the branch is split for the last minor release scheduled for before December 2024. For example, suppose 8.17.0 is scheduled for release in November 2024 and 8.18.0 is scheduled for release in January 2025. Then this PR should be merged to main after the 8.17 branch is split. One this PR is merged a followup PR should be opened against the ml-cpp repo to remove the build system for darwin-x86_64. It has been confirmed that with this change in place the Elasticsearch build system works with an ml-cpp bundle that does not contain a platform/darwin-x86_64 directory. It still produces an Elasticsearch build that will run providing xpack.ml.enabled is not explicitly set to true. After the build system for darwin-x86_64 has been removed from the ml-cpp repo, we will be able to do another PyTorch upgrade without having to worry about tweaking the build system to work on Intel macOS. --------- Co-authored-by: Ed Savage <[email protected]> Co-authored-by: Valeriy Khakhutskyy <[email protected]>
edsavage
added a commit
that referenced
this pull request
Nov 29, 2024
As previously advised in #104087, machine learning functionality will no longer be available on macOS x86_64. Machine learning functionality is still available on macOS by using an arm64 machine (Apple silicon). It is also possible to run Elasticsearch with machine learning functionality within a Docker container on macOS x86_64. This PR should be merged to main after the branch is split for the last minor release scheduled for before December 2024. For example, suppose 8.17.0 is scheduled for release in November 2024 and 8.18.0 is scheduled for release in January 2025. Then this PR should be merged to main after the 8.17 branch is split. One this PR is merged a followup PR should be opened against the ml-cpp repo to remove the build system for darwin-x86_64. It has been confirmed that with this change in place the Elasticsearch build system works with an ml-cpp bundle that does not contain a platform/darwin-x86_64 directory. It still produces an Elasticsearch build that will run providing xpack.ml.enabled is not explicitly set to true. After the build system for darwin-x86_64 has been removed from the ml-cpp repo, we will be able to do another PyTorch upgrade without having to worry about tweaking the build system to work on Intel macOS. Backports #104125 --------- Co-authored-by: David Roberts <[email protected]> Co-authored-by: Valeriy Khakhutskyy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PyTorch is no longer going to provide macOS x86_64 builds after version 2.2. This doesn't instantly affect us, as we build PyTorch from source ourselves and they've said that they will not deliberately break macOS x86_64. They just won't build or test on that platform themselves. As a result it's inevitable that we'll have to make some tweaks to the PyTorch code to get it to build, and as the years go by it will become harder and harder to make the code compile on an unsupported platform. Since PyTorch is such a critical component of Elastic ML we won't be able to keep it running on macOS x86_64 for more than a few releases after PyTorch drops support. This change gives notice of our intentions.