Upgrade dependencies, Use GitHub Registry, Database usage updates #32
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The new stuff automatics tags and things like that
spatialite has proven to be a continual pain in the ass for something we don't actually use in the app, so drop it for now. Can continue to use it in the CLI for ad-hoc if we want. Enable WAL mode by default. This ruins the 'one file' thing, but also is what should be used in a serving environment
we shouldn't actually need that, esp. with wal mode + busy timeout etc.
rather than potentially masking over db config issues by using in-mem
we ran that in one big tx, which took a couple seconds and locked the DB. I can't think of any real reason why it can't be an incremental update, the writes are all upserts. So drop the TX
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upgrade to Go 1.19, Upgrades the SQLite dependency, and re-works the docker building a bit to use the GitHub Docker Registry.
The database is updated to use WAL mode by default. This improves concurrency, and should hopefully remove the errors we see there. It also makes the database compatible with litestream, which is great for DR. A flag is provided to opt-out from this. Using WAL mode takes us away from the nice "single self-contained file", however it can be removed from an existing database if we want to re-gain that.
The database code is updated to remove all the manual locking we tried to avoid the issues we were having there. Instead we rely on a busy timeout + WAL mode.
Spatialite is removed as well. It still seems to be a pain to use successfully, and none of the queries in the app actually requires it at this point in time. If we ever require it, we can see what the state of the art for using it in go is at that time. In the mean time, nothing precludes its use from external/direct queries via the CLI.
A transaction is dropped around venue sync. I can't see any case where this was actually necessary, and it locked writes on the DB for several seconds at startup.