Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if the Realm folder exists before using the lock file. #4855

Merged
merged 11 commits into from
Aug 26, 2021

Conversation

DominicFrei
Copy link
Contributor

@DominicFrei DominicFrei commented Aug 19, 2021

When deleting a Realm we catch a File::NotFound exception in case the folder does not exist, see https://github.com/realm/realm-core/blob/master/src/realm/object-store/shared_realm.cpp#L917-L923 (added with #4771).

Unfortunately this check is not sufficient on Windows and crashes when trying to access the lock file (see https://ci.realm.io/blue/organizations/jenkins/realm%2Frealm-dotnet/detail/PR-2558/1/tests for example).

It couldn't have been caught earlier since the OS tests are not run on Windows. It only surfaced when run through the .NET CI with a similar check on the SDK level (see https://github.com/realm/realm-dotnet/blob/master/Tests/Realm.Tests/Database/InstanceTests.cs#L96-L105).

P.S.: This check was originally added with #4765.

☑️ ToDos

  • 📝 Changelog update
  • 🚦 Tests (or not relevant) -> There are no relevant test changes for my code changes in Core because of above mentioned reasons (it failed already, but only on Windows). The above mentioned .NET test serves as the relevant test for this change until the Core CI runs on Windows as well.
    I did, however, add a check for the did_delete flag to the non-existing folder test (which is unrelated to the code changes though).

@DominicFrei DominicFrei self-assigned this Aug 19, 2021
@DominicFrei DominicFrei requested review from tgoyne and ironage August 19, 2021 11:31
@tgoyne
Copy link
Member

tgoyne commented Aug 19, 2021

Checking if a file exists before trying to do something with it is the canonical example of a TOCTOU race: just because File::exists() returns true doesn't mean that the file will still exist when we call DB::call_with_lock().

If some exception other than File::NotFound is being thrown here on Windows, we should simply be catching that exception too (or fixing the thing throwing the exception so that it throws the correct one).

@DominicFrei
Copy link
Contributor Author

Checking if a file exists before trying to do something with it is the canonical example of a TOCTOU race: just because File::exists() returns true doesn't mean that the file will still exist when we call DB::call_with_lock().

If some exception other than File::NotFound is being thrown here on Windows, we should simply be catching that exception too (or fixing the thing throwing the exception so that it throws the correct one).

Thank you for this suggestion, it led me to some further investigations and the solution is not completely different from what I expected.

As it turns out the problem actually lies within the differing Windows and POSIX implementations for File::open_internal and their error handling.

Since File::NotFound is described as Thrown if the directory part of the specified path was not found I have added the ERROR_PATH_NOT_FOUND to also be thrown as a FileNotFound instead of the more generic AccessError which satisfies all requirements. 👍

@ironage
Copy link
Contributor

ironage commented Aug 23, 2021

Can you add a test similar to https://github.com/realm/realm-core/blob/master/test/test_file.cpp#L387-L392 but with part of the path missing instead of the file itself? The core and sync tests are run on windows CI, just not the object-store tests yet.

@DominicFrei
Copy link
Contributor Author

Can you add a test similar to https://github.com/realm/realm-core/blob/master/test/test_file.cpp#L387-L392 but with part of the path missing instead of the file itself? The core and sync tests are run on windows CI, just not the object-store tests yet.

Thanks for pointing that out. I've added a test and will let it run (temporarily commented out the fix) to see if CI catches it.

@DominicFrei
Copy link
Contributor Author

Run through CI (for the Windows part) everything's looking fine. The test was failing in https://spruce.mongodb.com/task/realm_core_stable_windows_64_vs2019_core_tests_patch_bc53af903b74c553308c91227ddfa29e6c6705c1_6126302632f417350263318a_21_08_25_11_57_27/tests?execution=0&sortBy=STATUS&sortDir=ASC and then successful in https://spruce.mongodb.com/version/612639752fbabe6e534cb6c5/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC&variant=windows-64-vs2019 when re-enabling the fix (I couldn't run it locally that's why I over-complicated it via CI to make sure everything's fine).

PR is ready again. 👍

Copy link
Contributor

@ironage ironage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the test and making sure it fails without the fix. 👍

@ironage
Copy link
Contributor

ironage commented Aug 25, 2021

I noticed that Jenkins CI didn't fail when it should have, and #4874 should fix that.

@DominicFrei DominicFrei merged commit 9b4041b into master Aug 26, 2021
@DominicFrei DominicFrei deleted the df/check-if-folder-exists-before-locking branch August 26, 2021 12:51
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants