Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature - Table Lock Policy #7

Merged
merged 3 commits into from
May 3, 2018
Merged

Conversation

cnoon
Copy link
Member

@cnoon cnoon commented May 2, 2018

This PR adds a new TableLockPolicy enum to SQift to enable table lock error handling internally for execute, prepare, and step operations.

Problem Statement

Table lock errors are SQLITE_ERROR types thrown by execute, prepare, and step operations. These errors can occur when the database is configured with a WAL journal mode as well as a shared cache. When one connection has obtained a lock on a table, another connection running on a different thread will receive table lock errors until the previous lock is released. In these situations, there are a couple of ways to proceed. The error can either be immediately thrown and handled by the client, or the calling thread can poll the operation until the lock is released.

Solution

The TableLockPolicy defines two different ways to handle table lock errors. The first option is to poll on the calling thread at a specified interval until the lock is released. The other option is to immediately fast fail by throwing the table lock error as soon as it is encountered. Connection, ConnectionPool, and Database types are set to .fastFail by default.

In order to enable polling for table lock errors, all that needs to be done is to set the policy in the Connection or Database initializer.

let connection = try Connection(storageLocation: storageLocation, tableLockPolicy: .poll(0.01))
let database = try Database(storageLocation: .onDisk("path_to_db"), tableLockPolicy: .poll(0.01))

When using a WAL journal mode and a shared cache, it is recommended to use a .poll table lock policy with a poll interval of 10 ms.

Testing

I've added as many tests as possible that reproduce the table lock error behavior. We've also been shipping these changes at scale in NRC and NTC over the past couple of months to ensure the TableLockPolicy does catch all the potential edge cases for table lock errors. We have now eliminated all the table lock errors in both apps and feel these changes are ready for release.

@cnoon cnoon added this to the 3.2.0 milestone May 2, 2018
@cnoon cnoon self-assigned this May 2, 2018
@cnoon cnoon changed the title Feature - Table Lock Error Policy Feature - Table Lock Policy May 2, 2018
@ejensen
Copy link

ejensen commented May 2, 2018

A few of the new tests are failing on certain platforms

@cnoon cnoon requested a review from jereme May 2, 2018 17:43
@cnoon
Copy link
Member Author

cnoon commented May 2, 2018

I just pushed 35dcf04 which should help stabilize the tests running on CI. If that doesn't work, then we'll probably have to disable the tests on CI since they're all timing based. These types of tests are extremely difficult to make pass locally and on CI in a consistent way.

@cnoon cnoon force-pushed the feature/table-lock-error-policy branch from 35dcf04 to 63a1bf8 Compare May 2, 2018 18:30
@cnoon cnoon force-pushed the feature/table-lock-error-policy branch from 63a1bf8 to 6ca3601 Compare May 2, 2018 19:57
@cnoon cnoon force-pushed the feature/table-lock-error-policy branch from 6ca3601 to 59b145b Compare May 2, 2018 22:02
@cnoon cnoon merged commit 4903247 into master May 3, 2018
@cnoon cnoon deleted the feature/table-lock-error-policy branch May 3, 2018 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants