Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[5/x][lamport] Remove SequenceNumber::decrement and ::increment #6163

Merged
merged 4 commits into from
Dec 1, 2022

Conversation

amnn
Copy link
Member

@amnn amnn commented Nov 17, 2022

SequenceNumber::decrement is used primarily to revert state
updates (e.g. when an update crosses an epoch or configuration change
and then needs to be rolled back).

With the introduction of lamport timestamps, this will no longer be
feasible with decrement, because a transaction can increment an
object's version by an arbitrary amount (not always one), depending on
the versions of other input objects.

This commit re-implements revert_state_update to rely on a list of
initial versions for modified inputs that gets stored in
TransactionEffects and gets rid of the decrement API altogether.

increment is replaced by two pieces:

  • SequenceNumber::lamport_increment to calculate a lamport timestamp.
  • SequenceNumber::increment_to to increment to a number that should
    be strictly greater than the current version.

And now, rather than objects being incremented when they are modified,
versions are updated as follows:

  • Shared object scheduling pre-computes their new versions based on
    the transaction's inputs.
  • The TemporaryStore is aware of the lamport timestamp for the
    transaction it's serving, and assigns all modified objects (this
    includes inputs and child objects) this version when committing
    modifications to effects.
  • We also remember the versions modified and deleted objects had,
    in case we need to revert -- we can't rely solely on the information
    from the certificate for this, because we could have dynamically
    loaded child objects which need to be reverted as well.

Gateway also needs some special support in order to replay
certificates: Previously it took the objects that were indicated as
modified in effects and replayed those modifications into a store, but
new protections that this commit introduces to ensure that all objects
get incremented would complain (because the objects in the effects
come with thei post-modified version), so before replaying the
modifications, the gateway will decrement an object's version to their
pre-modified state, as specified by modified_at_versions in effects,
using a new SequenceNumber::decrement_to API.

Test Plan

cargo simtest
cargo nextest run

@amnn amnn self-assigned this Nov 17, 2022
Copy link
Contributor

@tnowacki tnowacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things look mostly good! At a high level, I'm wondering if things would be simpler if we just put all logic for versioning to the layer outside of the adapter.

crates/sui-types/src/messages.rs Show resolved Hide resolved
@@ -429,7 +430,7 @@ fn debit_coins_and_transfer<S>(
let new_coin = Object::new_move(
MoveObject::new_coin(
coin_type.clone(),
OBJECT_START_VERSION,
SequenceNumber::new(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just remove the sequence number from new_coin?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is one of the clean-ups I want to do after the fact. In the limit, the version could maybe even be moved entirely out of MoveObject as well (so the store becomes solely responsible for handling it, and move and the adapter layer are completely unaware.

crates/sui-adapter/src/adapter.rs Show resolved Hide resolved
crates/sui-adapter/src/adapter.rs Outdated Show resolved Hide resolved
@amnn
Copy link
Member Author

amnn commented Nov 18, 2022

Thanks for the reviews both! Agreed that with us dropping support for owned to shared upgrades, all the logic can now be moved out of the adapter, which is nice! I'll rebase and make those changes.

@amnn amnn force-pushed the lamport-5-increment branch from 9a7aa09 to c3f65f3 Compare November 22, 2022 22:40
Copy link
Contributor

@tnowacki tnowacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! I think it's finally good to go!

@amnn amnn force-pushed the lamport-5-increment branch from 719306c to af2c4ed Compare November 30, 2022 16:23
Ashok Menon and others added 4 commits November 30, 2022 19:46
`SequenceNumber::decrement` is used primarily to revert state
updates (e.g. when an update crosses an epoch or configuration change
and then needs to be rolled back).

With the introduction of lamport timestamps, this will no longer be
feasible with decrement, because a transaction can increment an
object's version by an arbitrary amount (not always one), depending on
the versions of other input objects.

This commit re-implements `revert_state_update` to rely on a list of
initial versions for modified inputs that gets stored in
`TransactionEffects` and gets rid of the `decrement` API altogether.

`increment` is replaced by two pieces:

- `SequenceNumber::lamport_increment` to calculate a lamport timestamp.
- `SequenceNumber::increment_to` to increment to a number that should
  be strictly greater than the current version.

And now, rather than objects being incremented when they are modified,
versions are updated as follows:

- Shared object scheduling pre-computes their new versions based on
  the transaction's inputs.
- The `TemporaryStore` is aware of the lamport timestamp for the
  transaction it's serving, and assigns all modified objects (this
  includes inputs and child objects) this version when committing
  modifications to effects.
- We also remember the versions modified and deleted objects had,
  in case we need to revert -- we can't rely solely on the information
  from the certificate for this, because we could have dynamically
  loaded child objects which need to be reverted as well.

Gateway also needs some special support in order to replay
certificates: Previously it took the objects that were indicated as
modified in effects and replayed those modifications into a store, but
new protections that this commit introduces to ensure that all objects
get incremented would complain (because the objects in the effects
come with thei post-modified version), so before replaying the
modifications, the gateway will decrement an object's version to their
pre-modified state, as specified by `modified_at_versions` in effects,
using a new `SequenceNumber::decrement_to` API.
@amnn amnn force-pushed the lamport-5-increment branch from af2c4ed to b3bd2e5 Compare November 30, 2022 23:34
@amnn amnn merged commit 15d1e30 into MystenLabs:main Dec 1, 2022
@amnn amnn deleted the lamport-5-increment branch December 1, 2022 00:09
Jibz1 pushed a commit that referenced this pull request Dec 7, 2022
`SequenceNumber::decrement` is used primarily to revert state
updates (e.g. when an update crosses an epoch or configuration change
and then needs to be rolled back).

With the introduction of lamport timestamps, this will no longer be
feasible with decrement, because a transaction can increment an
object's version by an arbitrary amount (not always one), depending on
the versions of other input objects.

This commit re-implements `revert_state_update` to rely on a list of
initial versions for modified inputs that gets stored in
`TransactionEffects` and gets rid of the `decrement` API altogether.

`increment` is replaced by two pieces:

- `SequenceNumber::lamport_increment` to calculate a lamport timestamp.
- `SequenceNumber::increment_to` to increment to a number that should
  be strictly greater than the current version.

And now, rather than objects being incremented when they are modified,
versions are updated as follows:

- Shared object scheduling pre-computes their new versions based on
  the transaction's inputs.
- The `TemporaryStore` is aware of the lamport timestamp for the
  transaction it's serving, and assigns all modified objects (this
  includes inputs and child objects) this version when committing
  modifications to effects.
- We also remember the versions modified and deleted objects had,
  in case we need to revert -- we can't rely solely on the information
  from the certificate for this, because we could have dynamically
  loaded child objects which need to be reverted as well.

Gateway also needs some special support in order to replay
certificates: Previously it took the objects that were indicated as
modified in effects and replayed those modifications into a store, but
new protections that this commit introduces to ensure that all objects
get incremented would complain (because the objects in the effects
come with thei post-modified version), so before replaying the
modifications, the gateway will decrement an object's version to their
pre-modified state, as specified by `modified_at_versions` in effects,
using a new `SequenceNumber::decrement_to` API.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants