kv: implement leader leases #123847
Assignees
Labels
A-kv-replication
Relating to Raft, consensus, and coordination.
A-leader-leases
Related to the introduction of leader leases
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-kv
KV Team
Comments
nvanbenschoten
added
C-enhancement
![blathers[crl]](https://avatars.githubusercontent.com/in/59700?s=40&u=373d9101a12edc3847d2fb6ae78e3adcfb4ca41e&v=4){kind=avatar}
|
cc @cockroachdb/replication |
This was referenced May 20, 2024
This was referenced Jun 6, 2024
nvanbenschoten
added
the
A-leader-leases
exalate-issue-sync
bot
added
A-leader-leases
This was referenced Jun 6, 2024
This was referenced Jun 6, 2024
nvanbenschoten
added a commit
to nvanbenschoten/cockroach
that referenced
this issue
Oct 2, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
nvanbenschoten
added a commit
to nvanbenschoten/cockroach
that referenced
this issue
Oct 2, 2024
Part of cockroachdb#123847. Leader leases have different availability properties than epoch leases under most failure modes. This patch adds failover test variants that use leader leases where possible. Epic: none Release note: None
|
Now that leader leases are implemented, I'll close this issue and shift the remaining work over to #132762. |
nvanbenschoten
added a commit
to nvanbenschoten/cockroach
that referenced
this issue
Oct 22, 2024
Part of cockroachdb#123847. Leader leases have different availability properties than epoch leases under most failure modes. This patch adds failover test variants that use leader leases where possible. Epic: none Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Oct 24, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Oct 24, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Oct 25, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Oct 30, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 25, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 25, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 27, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 29, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 29, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 29, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 29, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled Part of cockroachdb#123847. This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. The commit also includes a few other WIP changes to try to stabilize this. It won't be fully stable until defortification is implemented. Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Nov 30, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. For now, we only run unit tests with either all leader leases or none at all. This is less disruptive than running some percentage of leader leases, as doing so may result in a range switching its lease type from an epoch based lease to a leader lease (or vice versa) when there's a split or a merge. Doing so revokes the previous lease, which can prove disruptive to tests that can't handle TXN_RETRY_NEW_LEASE_PREVENTS_TXN errors. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Dec 2, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. For now, we only run unit tests with either all leader leases or none at all. This is less disruptive than running some percentage of leader leases, as doing so may result in a range switching its lease type from an epoch based lease to a leader lease (or vice versa) when there's a split or a merge. Doing so revokes the previous lease, which can prove disruptive to tests that can't handle TXN_RETRY_NEW_LEASE_PREVENTS_TXN errors. References cockroachdb#123847 Release note: None
arulajmani
pushed a commit
to arulajmani/cockroach
that referenced
this issue
Dec 2, 2024
…n_enabled This commit metamorphically enables the `kv.raft.leader_fortification.fraction_enabled` to exercise raft fortification and leader leases. For now, we only run unit tests with either all leader leases or none at all. This is less disruptive than running some percentage of leader leases, as doing so may result in a range switching its lease type from an epoch based lease to a leader lease (or vice versa) when there's a split or a merge. Doing so revokes the previous lease, which can prove disruptive to tests that can't handle TXN_RETRY_NEW_LEASE_PREVENTS_TXN errors. References cockroachdb#123847 Release note: None
craig bot
pushed a commit
that referenced
this issue
Dec 2, 2024
133353: kvserver: metamorphically enable leader leases r=nvanbenschoten a=arulajmani Rebased version of #131623 by `@nvanbenschoten.` Part of #123847. See individual commits. Release note: None 136521: orchestration: released CockroachDB version 24.3.0. Next version: 24.3.1 r=mw5h a=cockroach-teamcity Release note: None Epic: None Release justification: non-production (release infra) change. Co-authored-by: Arul Ajmani <[email protected]> Co-authored-by: Nathan VanBenschoten <[email protected]> Co-authored-by: Justin Beaver <[email protected]>
craig bot
pushed a commit
that referenced
this issue
Dec 2, 2024
133353: kvserver: metamorphically enable leader leases r=nvanbenschoten a=arulajmani Rebased version of #131623 by `@nvanbenschoten.` Part of #123847. See individual commits. Release note: None Co-authored-by: Arul Ajmani <[email protected]> Co-authored-by: Nathan VanBenschoten <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We want the properties of expiration-based leases, but without the cost of a replicated write per range every 3 seconds, and without the cost of ticking/heartbeating Raft ranges. In particular, we want to ensure the lease is functional (can process reads and replicated writes) and that it eagerly acquires leases.
From the current working design doc:
See the design doc for details on how this will be accomplished, the resulting scaling behavior, and the architectural simplifications that fall out.
Subtasks:
Jira issue: CRDB-38570
Epic CRDB-37522
The text was updated successfully, but these errors were encountered: