Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rootlessport: fix potential hang #5183

Merged

Conversation

giuseppe
Copy link
Member

write to the error pipe only in case of an error. Otherwise we may
end up in a race condition in the select statement below as the read
from errChan happens before initComplete and the function returns
immediately nil.

Closes: #5182

Signed-off-by: Giuseppe Scrivano [email protected]

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS labels Feb 12, 2020
@giuseppe
Copy link
Member Author

@AkihiroSuda PTAL

Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. LGTM

}
errCh <- driverErr
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe close errCh?

@mheon
Copy link
Member

mheon commented Feb 12, 2020

LGTM once comment is addressed

write to the error pipe only in case of an error.  Otherwise we may
end up in a race condition in the select statement below as the read
from errChan happens before initComplete and the function returns
immediately nil.

Closes: containers#5182

Signed-off-by: Giuseppe Scrivano <[email protected]>
@TomSweeneyRedHat
Copy link
Member

LGTM once the comment is addressed. CI isn't buying it, probably not related.

@giuseppe
Copy link
Member Author

giuseppe commented Feb 12, 2020

it took a while as I've found another race condition.

More details in the second patch.

I've tried runtime.LockOSThread but didn't work and I could still hit the same race

@giuseppe
Copy link
Member Author

@nalind I see the reexec package is using SysProcAttr.Pdeathsig, we should probably change it as well.

there is a race condition where the child process is immediately
killed:

[pid 2576752] arch_prctl(0x3001 /* ARCH_??? */, 0x7ffdf612f170) = -1 EINVAL (Invalid argument)
[pid 2576752] access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
[pid 2576752] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=2576742, si_uid=0} ---
[pid 2576752] +++ killed by SIGTERM +++

this happens because the parent process here really means the "parent
thread".

Since there is no way of running it on the main thread,
let's skip this functionality altogether and use kill(2).

Signed-off-by: Giuseppe Scrivano <[email protected]>
@giuseppe giuseppe force-pushed the rootlessport-avoid-hang branch from 51d3693 to 5b69e7f Compare February 12, 2020 17:48
@rhatdan
Copy link
Member

rhatdan commented Feb 13, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 13, 2020
@openshift-merge-robot openshift-merge-robot merged commit 5ea6cad into containers:master Feb 13, 2020
cmd.Args = []string{reexecChildKey}
cmd.Stdin = childQuitR
cmd.Stdout = &logrusWriter{prefix: "child"}
cmd.Stderr = cmd.Stdout
cmd.Env = append(os.Environ(), reexecChildEnvOpaque+"="+string(opaqueJSON))
cmd.SysProcAttr = &syscall.SysProcAttr{
Pdeathsig: syscall.SIGTERM,
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what was the problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the child process is killed whenever the parent thread exits which we have no control about since it is managed by the Go runtime.

@giuseppe
Copy link
Member Author

PR to address the same race condition in containers/storage: containers/storage#530

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 25, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 25, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

podman run freezes on "waiting for initComplete"
8 participants