Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-40954: [CI] Fix use of obsolete docker-compose command on Github Actions #40949

Merged
merged 6 commits into from
Apr 3, 2024

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Apr 2, 2024

Rationale for this change

The docker-compose utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

What changes are included in this PR?

Use docker client CLI directly instead of docker-compose where possible.

Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

Are there any user-facing changes?

No, except additional optional env var ARCHERY_DEBUG.

Copy link

github-actions bot commented Apr 2, 2024

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@github-actions github-actions bot added the awaiting review Awaiting review label Apr 2, 2024
@pitrou pitrou changed the title [CI] Try to diagnose sporadic docker-compose issues [CI] Try to fix sporadic docker-compose issues Apr 2, 2024
@pitrou pitrou force-pushed the ci-docker-compose branch 5 times, most recently from b2604f0 to 1ddc67d Compare April 2, 2024 15:56
@pitrou pitrou force-pushed the ci-docker-compose branch from 1ddc67d to 6a11f94 Compare April 2, 2024 15:59
@pitrou
Copy link
Member Author

pitrou commented Apr 2, 2024

@github-actions crossbow submit -g nightly

@pitrou
Copy link
Member Author

pitrou commented Apr 2, 2024

@github-actions crossbow submit -g cpp

Copy link

github-actions bot commented Apr 2, 2024

Revision: 2304b1b

Submitted crossbow builds: ursacomputing/crossbow @ actions-81f3a4a053

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind Azure
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions

@pitrou
Copy link
Member Author

pitrou commented Apr 2, 2024

@github-actions crossbow submit -g linux

@pitrou pitrou changed the title [CI] Try to fix sporadic docker-compose issues GH-40954: [CI] Try to fix sporadic docker-compose issues Apr 2, 2024
Copy link

github-actions bot commented Apr 2, 2024

Revision: 2304b1b

Submitted crossbow builds: ursacomputing/crossbow @ actions-2847c15c67

Task Status
almalinux-8-amd64 GitHub Actions
almalinux-8-arm64 GitHub Actions
almalinux-9-amd64 GitHub Actions
almalinux-9-arm64 GitHub Actions
amazon-linux-2023-amd64 GitHub Actions
amazon-linux-2023-arm64 GitHub Actions
centos-7-amd64 GitHub Actions
centos-8-stream-amd64 GitHub Actions
centos-8-stream-arm64 GitHub Actions
centos-9-stream-amd64 GitHub Actions
centos-9-stream-arm64 GitHub Actions
debian-bookworm-amd64 GitHub Actions
debian-bookworm-arm64 GitHub Actions
debian-bullseye-amd64 GitHub Actions
debian-bullseye-arm64 GitHub Actions
debian-trixie-amd64 GitHub Actions
debian-trixie-arm64 GitHub Actions
ubuntu-focal-amd64 GitHub Actions
ubuntu-focal-arm64 GitHub Actions
ubuntu-jammy-amd64 GitHub Actions
ubuntu-jammy-arm64 GitHub Actions
ubuntu-noble-amd64 GitHub Actions
ubuntu-noble-arm64 GitHub Actions

Copy link

github-actions bot commented Apr 2, 2024

⚠️ GitHub issue #40954 has been automatically assigned in GitHub to PR creator.

@pitrou pitrou marked this pull request as ready for review April 2, 2024 16:54
@pitrou pitrou requested review from assignUser, kou and raulcd as code owners April 2, 2024 16:54
@pitrou pitrou changed the title GH-40954: [CI] Try to fix sporadic docker-compose issues GH-40954: [CI] Fix use of obsolete docker-compose command on Github Actions Apr 2, 2024
@pitrou
Copy link
Member Author

pitrou commented Apr 2, 2024

@github-actions crossbow submit cuda

@apache apache deleted a comment from github-actions bot Apr 2, 2024
Copy link

github-actions bot commented Apr 2, 2024

Revision: 6f89a99

Submitted crossbow builds: ursacomputing/crossbow @ actions-be8b558ace

Task Status
conda-linux-aarch64-cuda-py3 Azure
conda-linux-ppc64le-cuda-py3 Azure
conda-linux-x64-cuda-py3 Azure
conda-win-x64-cuda-py3 Azure
test-cuda-cpp GitHub Actions
test-cuda-python GitHub Actions

@pitrou
Copy link
Member Author

pitrou commented Apr 3, 2024

@raulcd @assignUser @kou Any further comments here? Otherwise it would be nice to merge this quickly.

@pitrou pitrou merged commit be3b789 into apache:main Apr 3, 2024
62 of 63 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Apr 3, 2024
@pitrou pitrou deleted the ci-docker-compose branch April 3, 2024 08:33
pitrou added a commit that referenced this pull request Apr 3, 2024
### Rationale for this change

Same as PR #40949, but for Crossbow builds hosted on Azure Pipelines.

### Are these changes tested?

Yes, they fix CI tests.

### Are there any user-facing changes?

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
@github-actions github-actions bot added the awaiting committer review Awaiting committer review label Apr 3, 2024
Copy link

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit be3b789.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 8 possible false positives for unstable benchmarks that are known to sometimes produce them.

kou added a commit to kou/arrow that referenced this pull request Apr 9, 2024
We can't use multiple top-level `env:` in workflow. apacheGH-40949
introduced a top-level `env:` by `macros.github_header()`. It broke
workflows that already have top-level `env:`.
kou added a commit that referenced this pull request Apr 9, 2024
…1091)

### Rationale for this change

We can't use multiple top-level `env:` in workflow. GH-40949 introduced a top-level `env:` by `macros.github_header()`. It broke workflows that already have top-level `env:`.

### What changes are included in this PR?

Omit top-level `env:` key and reuse the top-level `env:` key generated by `macros.github_header()` in workflows.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: #41088

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
raulcd pushed a commit that referenced this pull request Apr 10, 2024
…1091)

### Rationale for this change

We can't use multiple top-level `env:` in workflow. GH-40949 introduced a top-level `env:` by `macros.github_header()`. It broke workflows that already have top-level `env:`.

### What changes are included in this PR?

Omit top-level `env:` key and reuse the top-level `env:` key generated by `macros.github_header()` in workflows.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: #41088

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request Apr 15, 2024
…or (apache#41091)

### Rationale for this change

We can't use multiple top-level `env:` in workflow. apacheGH-40949 introduced a top-level `env:` by `macros.github_header()`. It broke workflows that already have top-level `env:`.

### What changes are included in this PR?

Omit top-level `env:` key and reuse the top-level `env:` key generated by `macros.github_header()` in workflows.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: apache#41088

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this pull request May 2, 2024
…thub Actions (apache#40949)

### Rationale for this change

The `docker-compose` utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

### What changes are included in this PR?

Use `docker` client CLI directly instead of `docker-compose` where possible.

### Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

### Are there any user-facing changes?

No, except additional optional env var `ARCHERY_DEBUG`.
* GitHub Issue: apache#40954

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this pull request May 2, 2024
### Rationale for this change

Same as PR apache#40949, but for Crossbow builds hosted on Azure Pipelines.

### Are these changes tested?

Yes, they fix CI tests.

### Are there any user-facing changes?

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this pull request May 2, 2024
…or (apache#41091)

### Rationale for this change

We can't use multiple top-level `env:` in workflow. apacheGH-40949 introduced a top-level `env:` by `macros.github_header()`. It broke workflows that already have top-level `env:`.

### What changes are included in this PR?

Omit top-level `env:` key and reuse the top-level `env:` key generated by `macros.github_header()` in workflows.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: apache#41088

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this pull request May 4, 2024
…thub Actions (apache#40949)

### Rationale for this change

The `docker-compose` utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

### What changes are included in this PR?

Use `docker` client CLI directly instead of `docker-compose` where possible.

### Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

### Are there any user-facing changes?

No, except additional optional env var `ARCHERY_DEBUG`.
* GitHub Issue: apache#40954

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this pull request May 4, 2024
Same as PR apache#40949, but for Crossbow builds hosted on Azure Pipelines.

Yes, they fix CI tests.

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
rok pushed a commit to tmct/arrow that referenced this pull request May 8, 2024
…thub Actions (apache#40949)

### Rationale for this change

The `docker-compose` utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

### What changes are included in this PR?

Use `docker` client CLI directly instead of `docker-compose` where possible.

### Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

### Are there any user-facing changes?

No, except additional optional env var `ARCHERY_DEBUG`.
* GitHub Issue: apache#40954

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
rok pushed a commit to tmct/arrow that referenced this pull request May 8, 2024
Same as PR apache#40949, but for Crossbow builds hosted on Azure Pipelines.

Yes, they fix CI tests.

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
rok pushed a commit to tmct/arrow that referenced this pull request May 8, 2024
…thub Actions (apache#40949)

### Rationale for this change

The `docker-compose` utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

### What changes are included in this PR?

Use `docker` client CLI directly instead of `docker-compose` where possible.

### Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

### Are there any user-facing changes?

No, except additional optional env var `ARCHERY_DEBUG`.
* GitHub Issue: apache#40954

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
rok pushed a commit to tmct/arrow that referenced this pull request May 8, 2024
Same as PR apache#40949, but for Crossbow builds hosted on Azure Pipelines.

Yes, they fix CI tests.

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
…thub Actions (apache#40949)

### Rationale for this change

The `docker-compose` utility is progressively being removed from GHA-provided runners:
actions/runner-images#9557

### What changes are included in this PR?

Use `docker` client CLI directly instead of `docker-compose` where possible.

### Are these changes tested?

Yes, this should fix the sporadic CI failures because of the above migration.

### Are there any user-facing changes?

No, except additional optional env var `ARCHERY_DEBUG`.
* GitHub Issue: apache#40954

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
### Rationale for this change

Same as PR apache#40949, but for Crossbow builds hosted on Azure Pipelines.

### Are these changes tested?

Yes, they fix CI tests.

### Are there any user-facing changes?

No.

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
…or (apache#41091)

### Rationale for this change

We can't use multiple top-level `env:` in workflow. apacheGH-40949 introduced a top-level `env:` by `macros.github_header()`. It broke workflows that already have top-level `env:`.

### What changes are included in this PR?

Omit top-level `env:` key and reuse the top-level `env:` key generated by `macros.github_header()` in workflows.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: apache#41088

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting committer review Awaiting committer review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants