Skip to content

Commit

Permalink
[SPARK-45212][INFRA] Install independent Python linter dependencies f…
Browse files Browse the repository at this point in the history
…or branch-3.5

### What changes were proposed in this pull request?
This pr install independent Python linter dependencies for branch-3.5 to make the daily test of branch-3.5 uses the correct dependencies for Python linter checking.

### Why are the changes needed?
The daily test of branch-3.5 failed during Python linter check:
- https://github.com/apache/spark/actions/runs/6221638911/job/16884068430

<img width="726" alt="image" src="https://github.com/apache/spark/assets/1475305/e05cbe4f-7639-4e93-a20a-35fc00ba9fc4">

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions
- Monitor branch-3.5 daily test

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#42990 from LuciferYang/SPARK-45212.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
LuciferYang authored and dongjoon-hyun committed Sep 19, 2023
1 parent 7e14c8c commit d202c70
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -680,8 +680,15 @@ jobs:
# Should delete this section after SPARK 3.4 EOL.
python3.9 -m pip install 'flake8==3.9.0' pydata_sphinx_theme 'mypy==0.920' 'pytest==7.1.3' 'pytest-mypy-plugins==1.9.3' numpydoc 'jinja2<3.0.0' 'black==22.6.0'
python3.9 -m pip install 'pandas-stubs==1.2.0.53' ipython 'grpcio==1.48.1' 'grpc-stubs==1.24.11' 'googleapis-common-protos-stubs==2.2.0'
- name: Install Python linter dependencies for branch-3.5
if: inputs.branch == 'branch-3.5'
run: |
# SPARK-45212: Copy from https://github.com/apache/spark/blob/555c8def51e5951c7bf5165a332795e9e330ec9d/.github/workflows/build_and_test.yml#L631-L638
# Should delete this section after SPARK 3.5 EOL.
python3.9 -m pip install 'flake8==3.9.0' pydata_sphinx_theme 'mypy==0.982' 'pytest==7.1.3' 'pytest-mypy-plugins==1.9.3' numpydoc 'jinja2<3.0.0' 'black==22.6.0'
python3.9 -m pip install 'pandas-stubs==1.2.0.53' ipython 'grpcio==1.56.0' 'grpc-stubs==1.24.11' 'googleapis-common-protos-stubs==2.2.0'
- name: Install Python linter dependencies
if: inputs.branch != 'branch-3.3' && inputs.branch != 'branch-3.4'
if: inputs.branch != 'branch-3.3' && inputs.branch != 'branch-3.4' && inputs.branch != 'branch-3.5'
run: |
# TODO(SPARK-32407): Sphinx 3.1+ does not correctly index nested classes.
# See also https://github.com/sphinx-doc/sphinx/issues/7551.
Expand Down

0 comments on commit d202c70

Please sign in to comment.