Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating labels for on-demand runners #294

Merged
merged 35 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
83b4f02
Update release_branches.yml
MyroTk Aug 8, 2023
0ae7b4b
Update release_branches.yml
MyroTk Aug 9, 2023
b14c2d9
Update release_branches.yml
MyroTk Aug 9, 2023
4022cc7
Update release_branches.yml
MyroTk Aug 9, 2023
31023f1
Update release_branches.yml
MyroTk Aug 10, 2023
dc7ec6e
Update release_branches.yml
MyroTk Aug 10, 2023
0f5dacc
Update release_branches.yml
MyroTk Aug 10, 2023
64d21cd
Update release_branches.yml
MyroTk Aug 11, 2023
d3fbe03
retrigger
MyroTk Aug 16, 2023
ec1c343
Update release_branches.yml
MyroTk Aug 24, 2023
e5f6490
Update release_branches.yml
MyroTk Aug 24, 2023
6215331
Update release_branches.yml
MyroTk Aug 24, 2023
2e2d3ac
Update release_branches.yml
MyroTk Aug 29, 2023
fe5d41b
Update release_branches.yml
MyroTk Aug 30, 2023
98d7e84
Update ci-runner.py
MyroTk Aug 31, 2023
d02b46d
Update release_branches.yml
MyroTk Aug 31, 2023
839e58c
Update release_branches.yml
MyroTk Aug 31, 2023
763e036
Update release_branches.yml
MyroTk Sep 1, 2023
440ce15
Update release_branches.yml
MyroTk Sep 5, 2023
70ebe6b
Update release_branches.yml
MyroTk Sep 6, 2023
9fdcc64
Create action.yml
MyroTk Sep 6, 2023
041f53c
Add files via upload
MyroTk Sep 6, 2023
310010c
Update release_branches.yml
MyroTk Sep 6, 2023
ddd7d06
Update action.yml
MyroTk Sep 6, 2023
646f455
chmod
MyroTk Sep 6, 2023
dbf7641
Update release_branches.yml
MyroTk Sep 8, 2023
435b6b8
Update release_branches.yml
MyroTk Sep 12, 2023
db84892
Update release_branches.yml
MyroTk Sep 15, 2023
a62986b
Update release_branches.yml
MyroTk Sep 15, 2023
020542a
Update release_branches.yml
MyroTk Sep 15, 2023
82bc8ca
Update release_branches.yml
MyroTk Sep 15, 2023
5324f79
Delete .github/actions/submodule-checkout directory
MyroTk Sep 15, 2023
7d9f75b
Update release_branches.yml
MyroTk Sep 15, 2023
6dc7df3
Update release_branches.yml
MyroTk Sep 15, 2023
6a7e191
Fixes required for tests to work on Hetzner Cloud
ilejn Sep 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/actions/submodule-checkout/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Checkout submodules
description: 'Checkout submodules'
inputs:
submodules-depth:
description: The depth of submodules to checkout
required: true
default: 1
runs:
using: "composite"
steps:
- name: Checkout submodules
shell: bash
run: |
git -C "$GITHUB_WORKSPACE" submodule sync
git -C "$GITHUB_WORKSPACE" submodule init
git config --file "$GITHUB_WORKSPACE/.gitmodules" --null --get-regexp path | \
sed -z 's|.*\n||' | \
xargs --max-procs=100 --null --no-run-if-empty --max-args=1 \
$GITHUB_WORKSPACE/.github/retry.sh 60 5 "git -C \"$GITHUB_WORKSPACE\" submodule update --depth=\"${{ inputs.submodules-depth }}\" --single-branch"
22 changes: 22 additions & 0 deletions .github/retry.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash
# Execute command until exitcode is 0 or
# maximum number of retries is reached
# Example:
# ./retry <retries> <delay> <command>
retries=$1
delay=$2
command="${@:3}"
exitcode=0
try=0
until [ "$try" -ge $retries ]
do
echo "$command"
eval "$command"
exitcode=$?
if [ $exitcode -eq 0 ]; then
break
fi
try=$((try+1))
sleep $2
done
exit $exitcode
140 changes: 117 additions & 23 deletions .github/workflows/release_branches.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
REGRESSION_RESULTS_URL: altinity-build-artifacts/${{github.event.number}}/$GITHUB_SHA


Expand Down Expand Up @@ -43,7 +45,8 @@ jobs:
# path: ${{ runner.temp }}/docker_images_check/changed_images_aarch64.json
# Former DockerHubPushAmd64
DockerHubPush:
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Check out repository code
uses: ClickHouse/checkout@v1
Expand Down Expand Up @@ -90,7 +93,8 @@ jobs:
# path: ${{ runner.temp }}/changed_images.json
CompatibilityCheck:
needs: [BuilderDebRelease]
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx41, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand Down Expand Up @@ -155,7 +159,8 @@ jobs:
#########################################################################################
BuilderDebRelease:
needs: [DockerHubPush]
runs-on: [self-hosted, builder]
runs-on: [self-hosted, builder, on-demand, type-ccx53, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand All @@ -174,18 +179,24 @@ jobs:
path: ${{ env.IMAGES_PATH }}
- name: Trust My Directory
run: git config --global --add safe.directory * # https://stackoverflow.com/a/71940133
- name: Clear repository
shell: bash
run: |
sudo rm -fr "$GITHUB_WORKSPACE" && mkdir "$GITHUB_WORKSPACE"
- name: Check out repository code
uses: ClickHouse/checkout@v1
Copy link
Member

@Enmk Enmk Sep 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just fork the ClickHouse/checkout and add retries there? Then we can reduce clutter here (to simplify merge) and even have a chance to propose these changes upstream.

uses: actions/checkout@v3
with:
clear-repository: true
submodules: true
fetch-depth: 0 # otherwise we will have no info about contributors
fetch-depth: 0
token: ${{ github.token }}
- name: Check out submodules
uses: ./.github/actions/submodule-checkout
- name: Build
run: |
sudo rm -fr "$TEMP_PATH"
mkdir -p "$TEMP_PATH/build_check/package_release"
cd .. && tar czf $TEMP_PATH/build_source.src.tar.gz ClickHouse/
cd $TEMP_PATH && tar xvzf $TEMP_PATH/build_source.src.tar.gz
ls -l $TEMP_PATH
cd "$REPO_COPY/tests/ci" && python3 build_check.py "$BUILD_NAME"
- name: Upload build URLs to artifacts
if: ${{ success() || failure() }}
Expand Down Expand Up @@ -246,7 +257,8 @@ jobs:
needs:
- BuilderDebRelease
# - BuilderDebAarch64
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Check out repository code
uses: ClickHouse/checkout@v1
Expand All @@ -273,7 +285,8 @@ jobs:
needs:
- BuilderDebRelease
# - BuilderDebAarch64
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx31, image-x86-app-docker-ce]
timeout-minutes: 180
if: ${{ success() || failure() }}
steps:
- name: Set envs
Expand Down Expand Up @@ -352,7 +365,8 @@ jobs:
- BuilderDebRelease
- SignRelease
# - BuilderDebAarch64
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx31, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Check out repository code
uses: ClickHouse/checkout@v1
Expand All @@ -367,7 +381,8 @@ jobs:
############################################################################################
InstallPackagesTestRelease:
needs: [SignRelease]
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand Down Expand Up @@ -436,14 +451,16 @@ jobs:
## Do-nothing stage to trigger tests, makes is easier to
needs: [InstallPackagesTestRelease]
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- run: true
##############################################################################################
########################### FUNCTIONAl STATELESS TESTS #######################################
##############################################################################################
FunctionalStatelessTestRelease:
needs: [tests_start]
runs-on: [self-hosted, func-tester]
runs-on: [self-hosted, func-tester, on-demand, type-ccx53, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand All @@ -454,6 +471,18 @@ jobs:
REPO_COPY=${{runner.temp}}/stateless_debug/ClickHouse
KILL_TIMEOUT=10800
EOF
- name: Configure docker IPv6
run: |
sudo touch /etc/docker/daemon.json
sudo chown ubuntu:ubuntu /etc/docker/daemon.json
sudo cat <<EOT > /etc/docker/daemon.json
{
"ipv6": true,
"fixed-cidr-v6": "2001:db8:1::/64"
}
EOT
sudo chown root:root /etc/docker/daemon.json
sudo systemctl restart docker
- name: Download json reports
uses: actions/download-artifact@v3
with:
Expand Down Expand Up @@ -516,7 +545,8 @@ jobs:
##############################################################################################
FunctionalStatefulTestRelease:
needs: [tests_start]
runs-on: [self-hosted, func-tester]
runs-on: [self-hosted, func-tester, on-demand, type-ccx53, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand All @@ -527,6 +557,18 @@ jobs:
REPO_COPY=${{runner.temp}}/stateful_debug/ClickHouse
KILL_TIMEOUT=3600
EOF
- name: Configure docker IPv6
run: |
sudo touch /etc/docker/daemon.json
sudo chown ubuntu:ubuntu /etc/docker/daemon.json
sudo cat <<EOT > /etc/docker/daemon.json
{
"ipv6": true,
"fixed-cidr-v6": "2001:db8:1::/64"
}
EOT
sudo chown root:root /etc/docker/daemon.json
sudo systemctl restart docker
- name: Download json reports
uses: actions/download-artifact@v3
with:
Expand Down Expand Up @@ -589,7 +631,8 @@ jobs:
#############################################################################################
IntegrationTestsRelease0:
needs: [tests_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, func-tester, style-checker]
timeout-minutes: 300
steps:
- name: Set envs
run: |
Expand Down Expand Up @@ -624,7 +667,8 @@ jobs:
sudo rm -fr "$TEMP_PATH"
IntegrationTestsRelease1:
needs: [tests_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, func-tester, style-checker]
timeout-minutes: 300
steps:
- name: Set envs
run: |
Expand Down Expand Up @@ -664,6 +708,7 @@ jobs:
## Not depending on the tests above since they can fail at any given moment.
needs: [tests_start]
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- run: true

Expand All @@ -673,12 +718,20 @@ jobs:
matrix:
SUITE: [aes_encryption, aggregate_functions, atomic_insert, base_58, clickhouse_keeper, datetime64_extended_range, disk_level_encryption, dns, example, extended_precision_data_types, kafka, kerberos, lightweight_delete, map_type, part_moves_between_shards, rbac, selects, ssl_server, tiered_storage, window_functions]
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -735,12 +788,20 @@ jobs:
matrix:
STORAGE: [minio, aws_s3, gcs]
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -805,12 +866,20 @@ jobs:
matrix:
SUITE: [authentication, external_user_directory, role_mapping]
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -863,12 +932,20 @@ jobs:

parquet:
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -935,12 +1012,20 @@ jobs:
matrix:
STORAGE: [minio, aws_s3, gcs]
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -1006,12 +1091,20 @@ jobs:
matrix:
STORAGE: [minio, s3amazon, s3gcs]
needs: [regression_start]
runs-on: [self-hosted, stress-tester]
runs-on: [self-hosted, regression-tester, on-demand, type-cpx51, image-x86-app-docker-ce]
timeout-minutes: 180
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_REPORT_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_REPORT_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: ${{ secrets.AWS_REPORT_REGION }}
steps:
- name: Install aws cli
run: |
cd /home/ubuntu
sudo curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip"
sudo unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install
sudo rm -rf /home/ubuntu/awscliv2.zip /home/ubuntu/aws
- name: Checkout regression repo
uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -1072,7 +1165,8 @@ jobs:

SignRelease:
needs: [BuilderDebRelease]
runs-on: [ self-hosted ]
runs-on: [self-hosted, on-demand, type-cpx31, image-x86-app-docker-ce]
timeout-minutes: 180
steps:
- name: Set envs
run: |
Expand Down Expand Up @@ -1132,7 +1226,7 @@ jobs:
- parquet
- s3
- tiered_storage_s3
runs-on: [self-hosted, style-checker]
runs-on: [self-hosted, style-checker, on-demand, type-cpx31, image-x86-app-docker-ce]
steps:
- name: Check out repository code
uses: ClickHouse/checkout@v1
Expand Down
Loading