sqlite: disable WAL mode #17937
sqlite: disable WAL mode #17937
Conversation
Disable the write-ahead log [1] for sqlite. Performance testing suggests that WAL mode has a negative impact on Podman. While most docs suggest that WAL mode improves performance, it does not seem to apply to Podman which tends to have very short-lived processes. What I found most curious is that commands with many writes are faster without WAL mode, for instance, `rm`, `stop`, `start`, etc. Hence, let's move to as-close-to-default config of sqlite and enable options only after thorough testing. [NO NEW TESTS NEEDED] as it's a non-functional change. [1] https://www.sqlite.org/wal.html Signed-off-by: Valentin Rothberg <[email protected]>
openshift-ci
bot
added
release-note-none
do-not-merge/work-in-progress
openshift-ci
bot
added
approved
|
@mheon @giuseppe @baude @edsantiago PTAL |
|
code is fine but i do not want to remove wal mode ... if you want to remove the sync journal, im onboard. |
Please elaborate. I tested WAL without full-sync which didn't improve things. As written in the commit message, I think that WAL has a negative impact. Podman's DB is very small and connections are short-lived. I am unsure WAL has been tuned for that scenario. |
|
I understand. I would like the opportunity to run tests with your PR if that is OK |
|
Absolutely, please all test. Different machines yield quiet different behaviors in the two configurations. |
|
None of the usual lock/unique strings[1] appear in the CI logs [1] 'is locked|unique constraint|constraint fail|config row|adding db|dbconfig.id' |
|
/hold To give everybody time to try out and compare. |
openshift-ci
bot
added
the
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: giuseppe, vrothberg The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@giuseppe did you run the benchmarks on your machine? If so, which observations did you make? |
|
ping, we now have sqlite testing in system tests, and the "stop" flake seems to be gone. If this is still desired, please rebase and repush and reevaluate/review. |
|
Still waiting on feedback from others on performance impact. My machine enjoys running without WAL mode based on running the scripts in |
|
I see fairly substantial benefits to WAL mode / normal sync (instead of full sync) and around equal performance for WAL mode with full sync, but there's enough variability involved that it's very hard to say. We need a more consistent testing environment to be sure. |
|
I think that until we have better numbers and that WAL mode is clearly better, it would be better to keep it simple and closer to the default configuration. IMO, we need to add custom tweaks to the default configuration only when it is clear that it is an improvement |
github-actions
bot
added
the
locked - please file new issue/PR
Disable the write-ahead log [1] for sqlite. Performance testing suggests that WAL mode has a negative impact on Podman. While most docs suggest that WAL mode improves performance, it does not seem to apply to Podman which tends to have very short-lived processes.
What I found most curious is that commands with many writes are faster without WAL mode, for instance,
rm,stop,start, etc.Hence, let's move to as-close-to-default config of sqlite and enable options only after thorough testing.
[NO NEW TESTS NEEDED] as it's a non-functional change.
[1] https://www.sqlite.org/wal.html
Does this PR introduce a user-facing change?