Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: consider reducing kv.transaction.max_intents_bytes #85313

Open
erikgrinaker opened this issue Jul 29, 2022 · 1 comment
Open

kvserver: consider reducing kv.transaction.max_intents_bytes #85313

erikgrinaker opened this issue Jul 29, 2022 · 1 comment
Labels
A-kv-transactions Relating to MVCC and the transactional model. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Jul 29, 2022

Back in 21.2, we increased the kv.transaction.max_intents_bytes default from 256 KB to 4 MB. This was done because ranged intent resolution back in 21.1 was very inefficient (it had to scan all of the data in the span), and caused several high-profile outages.

However, with separated intents also shipping in 21.2, the need for this increased value is less clear, since ranged intent resolution now only has to scan across intents. But we're now carrying around as much as 4 MB of intents in the transaction struct, which is passed around in memory, sent across the network, and stored to disk.

At this point, it seems like the additional cost of the 4 MB intent tracking probably isn't worth the marginal benefit of avoiding the intent scan, so we should consider reducing it back down to e.g. 256 KB. I suspect this may also translate to non-negligible performance improvements for medium-to-large transactions, since we avoid carrying around all of this extra baggage in the transaction struct, but we should run some benchmarks to validate this.

We should also drop the warning log message when exceeding the budget, since the consequences are much less severe now with separated intents.

Jira issue: CRDB-18179

@erikgrinaker erikgrinaker added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv-transactions Relating to MVCC and the transactional model. T-kv KV Team labels Jul 29, 2022
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-transactions Relating to MVCC and the transactional model. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
None yet
Development

No branches or pull requests

1 participant