Skip to content

Commit

Permalink
[KYUUBI #491] Add Spark nightly build CI
Browse files Browse the repository at this point in the history
![Yikun](https://badgen.net/badge/Hello/Yikun/green) [![Closes #495](https://badgen.net/badge/Preview/Closes%20%23495/blue)](https://github.com/yaooqinn/kyuubi/pull/495) ![24](https://badgen.net/badge/%2B/24/red) ![0](https://badgen.net/badge/-/0/green) ![2](https://badgen.net/badge/commits/2/yellow) ![Test Plan](https://badgen.net/badge/Missing/Test%20Plan/ff0000) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=yaooqinn&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

### _Why are the changes needed?_
This patch add Spark nightly build CI

### _How was this patch tested?_
github action CI passed

Closes #495 from Yikun/kyuubi-491.

Closes #491

c65e1f5 [Yikun Jiang] Skip dependency check in spark-3.2-snapshot
d57ea16 [Yikun Jiang] Add Spark nightly build CI

Authored-by: Yikun Jiang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
  • Loading branch information
Yikun authored and pan3793 committed Apr 9, 2021
1 parent b7ac3cb commit 9b3c22d
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ jobs:
- '-Pspark-3.0 -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.1.1 -Dspark.archive.name=spark-3.1.1-bin-hadoop2.7.tgz -Dmaven.plugin.scalatest.exclude.tags=org.apache.kyuubi.tags.DataLakeTest'
- '-Pspark-3.1'
- '-Pspark-3.1 -Dhadoop.binary.version=3.2'
- '-Pspark-3.2-snapshot -pl :kyuubi-spark-sql-engine,:kyuubi-common,:kyuubi-ha,:kyuubi-zookeeper -Dmaven.plugin.scalatest.exclude.tags=org.apache.kyuubi.tags.DataLakeTest'
steps:
- uses: actions/checkout@v2
- name: Setup JDK 1.8
Expand Down Expand Up @@ -72,6 +73,7 @@ jobs:
mvn clean install ${{ matrix.profiles }} -Dmaven.javadoc.skip=true -V
bash <(curl -s https://codecov.io/bash)
- name: Detected Dependency List Change
if: ${{ ! contains(matrix.profiles, 'spark-3.2-snapshot') }}
run: build/dependency.sh
- name: Upload unit tests log
if: failure()
Expand Down
4 changes: 4 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ matrix:
env:
- PROFILE="-Pspark-3.0 -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.1.1 -Dspark.archive.name=spark-3.1.1-bin-hadoop2.7.tgz"
- EXCLUDE_TAGS="org.apache.kyuubi.tags.DataLakeTest"
- name: Test Kyuubi w/ -Pspark-3.2-snapshot w/ Spark 3.2 nightly build
env:
- PROFILE="-Pspark-3.2-snapshot -pl :kyuubi-spark-sql-engine,:kyuubi-common,:kyuubi-ha,:kyuubi-zookeeper"
- EXCLUDE_TAGS="org.apache.kyuubi.tags.DataLakeTest"

install:
- mvn --version
Expand Down
18 changes: 18 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,17 @@
<enabled>false</enabled>
</snapshots>
</repository>

<repository>
<id>Apache Snapshots Repository</id>
<url>https://repository.apache.org/snapshots/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>

<pluginRepositories>
Expand Down Expand Up @@ -1418,6 +1429,13 @@
<id>spark-3.0</id>
</profile>

<profile>
<id>spark-3.2-snapshot</id>
<properties>
<spark.version>3.2.0-SNAPSHOT</spark.version>
</properties>
</profile>

<profile>
<id>spark-3.1</id>
<properties>
Expand Down

0 comments on commit 9b3c22d

Please sign in to comment.