Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ETXTBSY for Coursier fetch. #13425

Closed
wants to merge 1 commit into from
Closed

Conversation

jsirois
Copy link
Contributor

@jsirois jsirois commented Oct 30, 2021

Some of our Processes use system binaries to run scripts in the input
digest of the Process. This foiled the original ETXTBSY fix which was
only used when argv0 was a member of the input digest. Coursier fetch
was an example of this, using the system provided bash to execute a
script in the input digest which in turn executed a binary in the input
digest.

Fix this by just pessimistically assuming all Process invocations will
execute files in the input digest and always performing the mitigation.

Fixes #13424

[ci skip-build-wheels]

Some of our Processes use system binaries to run scripts in the input
digest of the Process. This foiled the original ETXTBSY fix which was
only used when argv0 was a member of the input digest. Coursier fetch
was an example of this, using the system provided bash to execute a
script in the input digest which in turn executed a binary in the input
digest.

Fix this by just pessimistically assuming all Process invocations will
execute files in the input digest and always performing the mitigation.

Fixes pantsbuild#13424

[ci skip-build-wheels]
Copy link
Contributor

@tdyas tdyas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@jsirois
Copy link
Contributor Author

jsirois commented Oct 31, 2021

Hrm, I just realized it won't help the case in #13424 since there the process tree is:

/usr/bin/bash ./coursier_wrapper_script.sh
  -> ./cs-x86_64-pc-linux

In other words; although this fix mitigates the top level process encountering ETXTBSY, it does nothing for the child ./cs-x86_64-pc-linux process spawn since that spawn is fully controlled by bash and runs concurrent-to-or-after our process spawn of /usr/bin/bash ./coursier_wrapper_script.sh returns control to us and we exit our lock. In #13424 it was exactly that child process spawn from within bash that had the issue.

@jsirois
Copy link
Contributor Author

jsirois commented Oct 31, 2021

It's worse - even that 1st invocation isn't helped here because it has 0 problems: bash exists on the system and that is the only binary in that invokation - it will never trigger ETXTBSY. It spawns and then reads in ./coursier_wrapper_script.sh on its own using libc which doesn't care who else may have the file open.

Updated the issue with this more thought out scenario and suggested a bad idea. I'm going to close this PR and hope people pile on to that issue with better ideas.

@jsirois jsirois closed this Oct 31, 2021
@jsirois jsirois deleted the issues/13424 branch October 31, 2021 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ETXTBSY during Coursier fetch
2 participants