Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement bare-metal Mac M1 podman-machine testing #20691

Merged
merged 2 commits into from
Dec 5, 2023
Merged

Implement bare-metal Mac M1 podman-machine testing #20691

merged 2 commits into from
Dec 5, 2023

Conversation

cevich
Copy link
Member

@cevich cevich commented Nov 14, 2023 •
edited
Loading

Depends on: #20852

Setup and execute podman machine testing on bare-metal M1 Macs
using a pool of shared and semi-persistent hosts. Automated
and manual processes outside this repository are responsible
for providing and maintaining all hosts. Ref.
https://github.com/containers/automation/tree/main/mac_pw_pool

Update the localmachine make target to standardize execution
across platforms. Update/simplify podman-machine e2e README to
reflect current reality.

Warning: This CI setup and supporting infrastructure was developed
in favor of expediency vs reliability and stability. There are
many possible failure-modes (known and unknown) which may lead
to undefined test behaviors. Future work may address some of
these as they are encountered or discovered.

[NO NEW TESTS NEEDED]

Does this PR introduce a user-facing change?

None

@openshift-ci openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none labels Nov 14, 2023
@cevich cevich force-pushed the mac_podman_machine_ci branch from a83f484 to e771a08 Compare November 14, 2023 20:38
@packit-as-a-service Packit-as-a-Service
Copy link

Cockpit tests failed for commit a83f484ba0b9372d69976a619b89dc3f45f5c9e6. @martinpitt, @jelly, @mvollmer please check.

@packit-as-a-service Packit-as-a-Service
Copy link

Cockpit tests failed for commit e771a0855a69695edf89c2655830a00e9e3e20d9. @martinpitt, @jelly, @mvollmer please check.

@cevich cevich force-pushed the mac_podman_machine_ci branch 6 times, most recently from 3e937f2 to 9f6151c Compare November 17, 2023 15:28
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 19, 2023
@cevich cevich force-pushed the mac_podman_machine_ci branch from 9f6151c to cc83265 Compare November 20, 2023 15:53
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 20, 2023
@cevich cevich force-pushed the mac_podman_machine_ci branch 3 times, most recently from 8a71b0a to a5182cf Compare November 28, 2023 17:37
@packit-as-a-service Packit-as-a-Service
Copy link

Ephemeral COPR build failed. @containers/packit-build please check.

@cevich cevich force-pushed the mac_podman_machine_ci branch 12 times, most recently from 378dad7 to 68fc1e6 Compare November 29, 2023 18:49
@cevich cevich force-pushed the mac_podman_machine_ci branch from 54425af to 2f40955 Compare November 30, 2023 20:11
edsantiago
edsantiago reviewed Nov 30, 2023
View reviewed changes
Copy link
Member

@edsantiago edsantiago left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only partway through but submitting before github clobbers my review comments

Makefile Show resolved Hide resolved
contrib/cirrus/mac_cleanup.sh Outdated Show resolved Hide resolved
contrib/cirrus/mac_env.sh Outdated Show resolved Hide resolved
contrib/cirrus/mac_env.sh Outdated Show resolved Hide resolved
edsantiago
edsantiago reviewed Nov 30, 2023
View reviewed changes
.cirrus.yml Outdated Show resolved Hide resolved
@cevich cevich mentioned this pull request Dec 1, 2023
Merged
@cevich
Copy link
Member Author

cevich commented Dec 1, 2023

Force push: Incremental fixes to address Ed's comments as of 2f4095529ea12aeb9ec00b2b447a3c9acf7a7391

@cevich cevich force-pushed the mac_podman_machine_ci branch from 2f40955 to aa03611 Compare December 1, 2023 15:53
cevich added 2 commits December 4, 2023 10:40
@cevich
Implement bare-metal Mac M1 podman-machine testing
2d46d05 
Setup and execute podman machine testing on bare-metal M1 Macs
using a pool of shared and semi-persistent hosts.  Automated
and manual processes outside this repository are responsible
for providing and maintaining all hosts.  Ref.
https://github.com/containers/automation/tree/main/mac_pw_pool

Update the `localmachine` make target to standardize execution
across platforms.  Update/simplify podman-machine e2e README to
reflect current reality.

Warning: This CI setup and supporting infrastructure was developed
in favor of expediency vs reliability and stability.  There are
many possible failure-modes (known and unknown) which may lead
to undefined test behaviors.  Future work may address some of
these as they are encountered or discovered.

[NO NEW TESTS NEEDED]

Signed-off-by: Chris Evich <[email protected]>
@cevich
Do not aggregate failing mac test status
f1dc126 
Issue Ref: containers#20853

Allow the tests to fail, but don't block merging PRs.
This commit should be reverted when containers#20853 is resolved.

Signed-off-by: Chris Evich <[email protected]>
@cevich cevich force-pushed the mac_podman_machine_ci branch from aa03611 to f1dc126 Compare December 4, 2023 17:09
@cevich
Copy link
Member Author

cevich commented Dec 4, 2023

force-push: conflict resolution, no code changes.

@edsantiago
Copy link
Member

LGTM with fingers crossed...

@cevich
Copy link
Member Author

cevich commented Dec 4, 2023

Thanks for all your help Ed. There are going to be hiccups - this being so drastically orthogonal to how all other CI works. I think I have enough monitoring in place to help debug instance problems, if not I'll work on adding more as needed.

@cevich
Copy link
Member Author

cevich commented Dec 4, 2023

@ashley-cui @baude @mheon PTAL 🤞

@cevich cevich requested a review from mheon December 4, 2023 21:02
@edsantiago
Copy link
Member

FWIW I restarted the machine-mac test once already, it timed out

@cevich
Copy link
Member Author

cevich commented Dec 4, 2023 •
edited
Loading

important note: The timeout failure of the main tests in machine-mac podman darwin rootless host sqlite is completely expected. At this stage, the only thing that should be green is smoke_test. Either way, the status of this task is NOT part of Total Success.

@cevich
Copy link
Member Author

cevich commented Dec 4, 2023 •
edited
Loading

Restarted sys podman fedora-38 root host boltdb again (failure log). Didn't look deeply, but the problem appears not good:

<+031ms> # # podman run -d --name=test-ctr --rm quay.io/libpod/testimage:20221018 sh -c mkdir -p /foo/test1. /foo/test2; touch /foo/test1./file1 /foo/test2/file2; sleep infinity
<+382ms> # 47d87113ab959ae2ac6019568a26f2f77e9b2e0a3938babb873f0dfcf5ba0413
         #
<+011ms> # # podman cp test-ctr:/foo/test1. /var/tmp/podman_bats.jfntbd/dst/foo
<+063ms> # Error: "/foo/test1." could not be found on container test-ctr: no such file or directory
<+006ms> # [ rc=125 (** EXPECTED 0 **) ]

Maybe the run -d is racing?

I restarted it again.

Edit: Looks like maybe #20282 ?

@cevich
Copy link
Member Author

cevich commented Dec 5, 2023

Re-ran sys podman fedora-38 root host boltdb again. Different test-failure this time 😕

@cevich
Copy link
Member Author

cevich commented Dec 5, 2023

All green now

@edsantiago
Copy link
Member

Here goes...

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 5, 2023
@edsantiago
Copy link
Member

/approve

1 similar comment
@mheon
Copy link
Member

mheon commented Dec 5, 2023

/approve

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 5, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cevich, edsantiago, mheon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 5, 2023
@openshift-merge-bot openshift-merge-bot bot merged commit e8f3098 into containers:main Dec 5, 2023
92 of 93 checks passed
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Mar 5, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@edsantiago edsantiago edsantiago left review comments

@ashley-cui ashley-cui Awaiting requested review from ashley-cui

@baude baude Awaiting requested review from baude

@mheon mheon Awaiting requested review from mheon

Assignees

@edsantiago edsantiago

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. release-note-none
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@cevich @edsantiago @mheon @openshift-merge-robot