Skip to content

Commit

Permalink
Remove optimization to install wheel pacakges for affected providers (#…
Browse files Browse the repository at this point in the history
…36825)

When building wheel providers took a lot of time (12 minutes) there
was an optimisation implemented to only build the affected providers
and in this case we could not run verification, because having only
subset of providers would generate errors during imports.

However, changes to make our package bulds reproducible with flit #35693
also improve building time for provider packages (all of them are
built under 1 minute) and .whl installation had always been rather
quick - so we can remove the optimisation now, because side effect
of it that in some cases (like #36799) it caused the backwards
compatibility check succeed - and subsequently continue failing in
main canary build.

(cherry picked from commit 7086192)
  • Loading branch information
potiuk committed Feb 7, 2024
1 parent 719b319 commit 5a380ff
Showing 1 changed file with 3 additions and 32 deletions.
35 changes: 3 additions & 32 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -837,12 +837,10 @@ jobs:
run: >
breeze release-management prepare-provider-documentation --include-not-ready-providers
--non-interactive
${{ needs.build-info.outputs.affected-providers-list-as-string }}
- name: "Prepare provider packages: wheel"
run: >
breeze release-management prepare-provider-packages --include-not-ready-providers
--version-suffix-for-pypi dev0
--package-format wheel ${{ needs.build-info.outputs.affected-providers-list-as-string }}
--version-suffix-for-pypi dev0 --package-format wheel
- name: "Prepare airflow package: wheel"
run: breeze release-management prepare-airflow-package --version-suffix-for-pypi dev0
- name: "Verify wheel packages with twine"
Expand All @@ -865,25 +863,11 @@ jobs:
--airflow-constraints-reference default
--providers-constraints-location
/files/constraints-${{env.PYTHON_MAJOR_MINOR_VERSION}}/constraints-source-providers-${{env.PYTHON_MAJOR_MINOR_VERSION}}.txt
if: needs.build-info.outputs.affected-providers-list-as-string == ''
env:
AIRFLOW_SKIP_CONSTRAINTS: "${{ needs.build-info.outputs.upgrade-to-newer-dependencies }}"
- name: "Install affected provider packages and airflow via wheel files"
run: >
breeze release-management install-provider-packages
--use-packages-from-dist
--package-format wheel
--use-airflow-version wheel
--airflow-constraints-reference default
--providers-constraints-location
/files/constraints-${{env.PYTHON_MAJOR_MINOR_VERSION}}/constraints-source-providers-${{env.PYTHON_MAJOR_MINOR_VERSION}}.txt
if: needs.build-info.outputs.affected-providers-list-as-string != ''
env:
AIRFLOW_SKIP_CONSTRAINTS: "${{ needs.build-info.outputs.upgrade-to-newer-dependencies }}"
- name: "Prepare airflow package: wheel without suffix and skipping the tag check"
run: >
breeze release-management prepare-provider-packages --skip-tag-check
--package-format wheel ${{ needs.build-info.outputs.affected-providers-list-as-string }}
breeze release-management prepare-provider-packages --skip-tag-check --package-format wheel
prepare-install-provider-packages-sdist:
timeout-minutes: 80
Expand Down Expand Up @@ -978,7 +962,7 @@ jobs:
- name: "Prepare provider packages: wheel"
run: >
breeze release-management prepare-provider-packages --include-not-ready-providers
--package-format wheel ${{ needs.build-info.outputs.affected-providers-list-as-string }}
--package-format wheel
- name: >
Remove incompatible Airflow
${{matrix.airflow-version}}:Python ${{matrix.python-version}} provider packages
Expand All @@ -999,19 +983,6 @@ jobs:
--use-airflow-version wheel
--airflow-constraints-reference constraints-${{matrix.airflow-version}}
--providers-skip-constraints
if: needs.build-info.outputs.affected-providers-list-as-string == ''
- name: >
Install affected provider packages and airflow on
Airflow ${{matrix.airflow-version}}:Python ${{matrix.python-version}}
run: >
breeze release-management install-provider-packages
--use-packages-from-dist
--package-format wheel
--use-airflow-version wheel
--airflow-constraints-reference constraints-${{matrix.airflow-version}}
--providers-skip-constraints
--run-in-parallel
if: needs.build-info.outputs.affected-providers-list-as-string != ''
test-airflow-release-commands:
timeout-minutes: 80
Expand Down

0 comments on commit 5a380ff

Please sign in to comment.