Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent non-atomic writes to repodata JSON files #3834

Merged
merged 1 commit into from
Jan 2, 2020

Conversation

sscherfke
Copy link
Contributor

Fixes: #3833

@cla-bot
Copy link

cla-bot bot commented Dec 19, 2019

We require contributors to sign our Contributor License Agreement, and we don't have one on file for @sscherfke. In order for us to review and merge your code, please e-sign the PDF at https://conda.io/en/latest/contributing.html#conda-contributor-license-agreement. We then need to manually verify your signature. We will ping the bot to refresh the PR status when we have confirmed your signature.

@sscherfke
Copy link
Contributor Author

The link to the CLA stuff in the README seems to be broken: https://www.clahub.com/agreements/conda/conda-build

@sscherfke
Copy link
Contributor Author

Hm, os.rename() does not or can not overwrite an existing file on Windows. I’ll look into it.

@cla-bot
Copy link

cla-bot bot commented Dec 19, 2019

We require contributors to sign our Contributor License Agreement, and we don't have one on file for @sscherfke. In order for us to review and merge your code, please e-sign the PDF at https://conda.io/en/latest/contributing.html#conda-contributor-license-agreement. We then need to manually verify your signature. We will ping the bot to refresh the PR status when we have confirmed your signature.

@sscherfke
Copy link
Contributor Author

Using utils.move_with_fallback() again …

Also signed the CLA.

CHANGELOG.txt Outdated
@@ -79,7 +85,7 @@ Enhancements:
-------------

* add --use-channeldata argument to conda render/build.
* Extract the part in the skeletons pypi responsible to get the package metadata to a free function.
* Extract the part in the skeletons pypi responsible to get the package metadata to a free function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please revert this file?
You can add the description and you changelog here
https://github.com/conda/conda-build/blob/master/news/

and you can also use our template
https://github.com/conda/conda-build/blob/master/news/TEMPLATE

:)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add a new rst file on that folder using our template and it should be fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do that :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it look okay now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is still here =p

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please revert it? After that, I think that should be good to go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I thought I had reverted it. Tried it again. :)

temp_path = join(gettempdir(), str(uuid4()))
# Create the temp file next "path" so that we can use an atomic move, see
# https://github.com/conda/conda-build/issues/3833
temp_path = '%s.%s' % (path, uuid4())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that possible to reproduce the error using a test?
It would be interesting to have a test for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m afraid that’s no easily possible.

The error was that path and the temp_path were on two different file systems and temp_file was big enough that there could happen a read on path while temp_path was being copied into it.

Now, both files are always in the same folder and thus on the same filesystem.

@cla-bot cla-bot bot added the cla-signed [bot] added once the contributor has signed the CLA label Dec 19, 2019
@sscherfke
Copy link
Contributor Author

Can this be merged? :)

@marcelotrevisani
Copy link
Member

Can this be merged? :)

Sorry for the delay @sscherfke , because of the end of the year, holidays and all that stuff we might be a bit slower than usual 😄

@sscherfke
Copy link
Contributor Author

The changelog should now be clean.

@marcelotrevisani marcelotrevisani merged commit 90a6de5 into conda:master Jan 2, 2020
@marcelotrevisani
Copy link
Member

Thanks for your contribution @sscherfke ! 🎉

@sscherfke
Copy link
Contributor Author

Thanks for your kind feedback 🙂

mingwandroid pushed a commit to mingwandroid/conda-build that referenced this pull request Feb 25, 2020
@github-actions
Copy link

Hi there, thank you for your contribution!

This pull request has been automatically locked because it has not had recent activity after being closed.

Please open a new issue or pull request if needed.

Thanks!

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Mar 14, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"conda index" leads to "JSONDecodeError" in "conda install|update"
2 participants