Fix edge-case when conflicting constraints prevent k8s env creation (… #43298
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pache#43276) The apache#36930 added constraints to creation of k8s environment, but it had a side effect - the constraints could not be created if source of airflow had dependencies conflicting with constraints (which might happen for example when we upgrade FAB - because locally pinned FAB might be different than the one in constraints). Also the constraints were "hard-coded" - the python version, branch and github repository were hard-coded. This PR fixes both problems: * constraints URL is dynamically generated based on current branch, repo and python version * in case attempts to create the venv with constraints fails, we attempt to install it again without constraints (cherry picked from commit 274b6e1)
|
Backport of #43276 |
romsharon98
approved these changes
Oct 23, 2024
|
Both failures should already be fixed in tip of v2-10-test - merging |
utkarsharma2
pushed a commit
that referenced
this pull request
Oct 24, 2024
#43298) * Fix edge-case when conflicting constraints prevent k8s env creation (#43276) The #36930 added constraints to creation of k8s environment, but it had a side effect - the constraints could not be created if source of airflow had dependencies conflicting with constraints (which might happen for example when we upgrade FAB - because locally pinned FAB might be different than the one in constraints). Also the constraints were "hard-coded" - the python version, branch and github repository were hard-coded. This PR fixes both problems: * constraints URL is dynamically generated based on current branch, repo and python version * in case attempts to create the venv with constraints fails, we attempt to install it again without constraints (cherry picked from commit 274b6e1) * Update k8s_requirements.txt
This was referenced Oct 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…#43276)
The #36930 added constraints to creation of k8s environment, but it had a side effect - the constraints could not be created if source of airflow had dependencies conflicting with constraints (which might happen for example when we upgrade FAB - because locally pinned FAB might be different than the one in constraints).
Also the constraints were "hard-coded" - the python version, branch and github repository were hard-coded.
This PR fixes both problems:
(cherry picked from commit 274b6e1)
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.