sqlite: set connection attributes on open

[vrothberg]

The symptoms in #17859 indicate that setting the PRAGMAs in individual EXECs outside of a transaction can lead to concurrency issues and failures when the DB is locked. Hence set all PRAGMAs when opening the connection. Move them into individual constants to improve documentation and readability.

Further make transactions exclusive as #17859 also mentions an error that the DB is locked during a transaction.

Fixes: #17859

Does this PR introduce a user-facing change?

None

@mheon @baude @edsantiago PTAL

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vrothberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [vrothberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[baude]

LGTM

[rhatdan]

@mheon PTAL

[edsantiago]

The string "database is locked" does not appear in any of the CI logs for this PR, yay.

There is one instance of the "UNIQUE constraint failed" bug, in f37 root.

I rebased my #17831 on top of this, so we can see what system tests do.

[vrothberg]

The string "database is locked" does not appear in any of the CI logs for this PR, yay.

There is one instance of the "UNIQUE constraint failed" bug, in f37 root.

Yes, I did not tackle #17858 in this PR.

I rebased my #17831 on top of this, so we can see what system tests do.

That is good news, thanks!

[Luap99]

LGTM

[edsantiago]

The string "database is locked" does not appear in any of the logs for the abovementioned CI run of #17831. (Two systest failures, both expected). (And, two "UNIQUE constraint" failures, and I know this PR did not set out to fix that, but I had this fantasy that the bugs were related and could be 2birds1stoned...)

cirrus-pr-timing results below (only the boltdb vs sqlite comparison). The new sqlite options seem to slow things down a little bit, I don't know how significantly.

type	distro	user	DB	local	remote	container
int	fedora-37	root		29:49	32:55	28:10
int	fedora-37	root	sqlite	29:33	33:49
int	fedora-37	rootless		28:15
int	fedora-37	rootless	sqlite	30:39

Anyhow, LGTM

[vrothberg]

cirrus-pr-timing results below (only the boltdb vs sqlite comparison). The new sqlite options seem to slow things down a little bit, I don't know how significantly.

Those numbers are really great to have, thanks, Ed! @baude, we can leverage Ed's timing tracking for the upcoming performance massages.

[mheon]

@vrothberg PTAL at #17874 - I think we can avoid transaction exclusivity

[vrothberg]

@vrothberg PTAL at #17874 - I think we can avoid transaction exclusivity

Why open a new PR that conflicts with this one? Seems like a time waste.

[vrothberg]

OK, we had a quick chat and greed to mix the two PRs together. This PR gets rid of the multiple EXECs setting the PRAGMAs which is flaking. An open question is what setting max connections does in comparison to making transactions exclusive. I assumed that transactions (i.e., writes) must be exclusive.

But I am buried in meeting til the end of my workday and let @mheon take over :)

[mheon]

Pulled it into my PR, closing this now