Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move build_and_test_android from ci.yml to pkgci.yml. #18070

Merged
merged 1 commit into from
Aug 5, 2024

Conversation

ScottTodd
Copy link
Member

@ScottTodd ScottTodd commented Jul 31, 2024

Progress on #16203 and #17957.

This migrates .github/workflows/build_and_test_android.yml to .github/workflows/pkgci_test_android.yml.

For now, this only builds for Android, it does not run tests or use real Android devices at all.

The previous workflow

  • Relied on the "install" directory from a CMake build
  • Ran on large self-hosted CPU build machines
  • Built within Docker (using build_tools/docker/dockerfiles/android.Dockerfile)
  • Used GCP/GCS for remote ccache storage
  • Used GCP/GCS for passing files between jobs
  • Ran tests on self-hosted lab machines (I think a raspberry pi connected to some physical Android devices)

The new workflow

  • Relies on Python packages produced by pkgci_build_packages
  • Runs on standard GitHub-hosted runners
  • Installs dependencies that it needs on-demand (ninja, Android NDK), without using Docker
  • Uses caches provided by GitHub Actions for ccache storage
  • Could use Artifacts provided by GitHub Actions for passing files between jobs
  • Could run tests on self-hosted lab machines or Android emulators

I made some attempts at passing files from the build job to a test job but ran into some GitHub Actions debugging that was tricky. Leaving the remaining migration work there to contributors at Google or other parties directly interested in Android CI infrastructure.

ci-exactly: build_packages, test_android

@ScottTodd ScottTodd added infrastructure Relating to build systems, CI, or testing platform/android 🤖 Android-specific build, execution, benchmarking, and deployment labels Jul 31, 2024
Comment on lines 137 to 150
- uses: actions/[email protected]
with:
name: build-android-arm_64
path: ${{ env.TARGET_BUILD_DIR }}
- name: "Examining artifact directory structure"
run: ls -R ${{ env.TARGET_BUILD_DIR }}
- name: "Examining github workspace directory structure"
run: ls -R ${{ github.workspace }}
- name: "Running tests"
env:
LABEL_EXCLUDE: ${{ matrix.target.label-exclude }}
run: |
ctest -j 4 \
--test-dir "${TARGET_BUILD_DIR}" \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having some trouble getting into a productive debugging workflow here.

Logs: https://github.com/iree-org/iree/actions/runs/10188925129/job/28186536935

Internal ctest changing into directory: /runner-root/actions-runner/_work/iree/iree/build-android-arm_64
Test project /runner-root/actions-runner/_work/iree/iree/build-android-arm_64
          Start    1: iree/tools/test/iree_run_module_correctness_test_on_android_device
Could not find executable /home/runner/work/iree/iree/build_tools/cmake/run_android_test.sh
Looked in the following places:
Unable to find required file: /home/runner/work/iree/iree/build-android-arm_64/tools/test/iree_run_module_bytecode_module_vmvx.vmfb

May be able to test with packages from nightly releases / pypi and a non-cross-compile job doing the same steps:

  1. on runner A: build, upload build dir
  2. on runner B: download build dir, run ctest

This should be simple, but something about the paths isn't lining up the way it did when using GCS

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ScottTodd ScottTodd force-pushed the infra-pkgci-android branch from 540b3ed to 711108b Compare August 5, 2024 18:25
This currently builds but does not run any tests on real or emulated devices.
@ScottTodd ScottTodd force-pushed the infra-pkgci-android branch from 711108b to 6e149e8 Compare August 5, 2024 18:39
@ScottTodd ScottTodd marked this pull request as ready for review August 5, 2024 19:24
@ScottTodd ScottTodd merged commit a7a1be3 into iree-org:main Aug 5, 2024
28 checks passed
@ScottTodd ScottTodd deleted the infra-pkgci-android branch August 5, 2024 22:07
ScottTodd added a commit that referenced this pull request Aug 6, 2024
…18107)

Cleanup relating to #16203 and
#17957.

After #18070, no jobs in `ci.yml`
are depending on the output of the `build_all` job. All jobs using the
compiler now live in `pkgci.yml` and use packages instead of the
"install dir".

* The reusable
[`build_all.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/build_all.yml)
workflow is still in use from benchmarking workflows, but those are
unmaintained and may be deleted soon.
* After this is merged I'll set the `linux_x64_clang` check to
"required", so PRs will need it to be passing before they can be merged.
* Now that `ci.yml` is pruned to just a few remaining jobs
(`build_test_all_bazel`, `build_test_runtime`, `small_runtime`,
`tracing`), we can also limit when the workflow runs at all. I've added
some `paths` patterns so this won't run on push events if only files
under `docs/` are changed, for example.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Relating to build systems, CI, or testing platform/android 🤖 Android-specific build, execution, benchmarking, and deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants