Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(store-sync): rework blockLogsToStorage #1176

Merged
merged 10 commits into from
Jul 19, 2023
Merged

Conversation

holic
Copy link
Member

@holic holic commented Jul 18, 2023

Pulled out of #1113

@changeset-bot
Copy link

changeset-bot bot commented Jul 18, 2023

🦋 Changeset detected

Latest commit: 43125df

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 26 packages
Name Type
@latticexyz/block-logs-stream Major
@latticexyz/store-sync Major
@latticexyz/cli Major
@latticexyz/common Major
@latticexyz/config Major
create-mud Major
@latticexyz/dev-tools Major
@latticexyz/ecs-browser Major
@latticexyz/gas-report Major
@latticexyz/network Major
@latticexyz/noise Major
@latticexyz/phaserx Major
@latticexyz/protocol-parser Major
@latticexyz/react Major
@latticexyz/recs Major
@latticexyz/schema-type Major
@latticexyz/services Major
@latticexyz/solecs Major
solhint-config-mud Major
solhint-plugin-mud Major
@latticexyz/std-client Major
@latticexyz/std-contracts Major
@latticexyz/store-cache Major
@latticexyz/store Major
@latticexyz/utils Major
@latticexyz/world Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@holic holic marked this pull request as ready for review July 18, 2023 15:57
@holic holic requested a review from alvrs as a code owner July 18, 2023 15:57
Copy link
Member

@alvrs alvrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to double check my understanding, instead of passing in an object with handlers like like { setRecord, setField, deleteRecord } etc., users pass in a single handler storeOperations correct? for context why did you end up going this route instead of a handler per store operation?

@holic
Copy link
Member Author

holic commented Jul 19, 2023

to double check my understanding, instead of passing in an object with handlers like like { setRecord, setField, deleteRecord } etc., users pass in a single handler storeOperations correct? for context why did you end up going this route instead of a handler per store operation?

Because the store operations are (and should be) grouped by block, so that you can execute them within the context of a database transaction. Otherwise it's hard to know when to begin/commit a database tx.

It also helps us to avoid the work arounds we had before like lastEventInTx.

Comment on lines 25 to 27
// I don't love carrying all these types through. Ideally this should be the shape of the thing we want, rather than the specific return type from a function.
export type StoreEventsLog = GetLogsResult<StoreEventsAbi>[number];
export type BlockLogs = GroupLogsByBlockNumberResult<StoreEventsLog>[number];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you elaborate on this comment? having trouble following

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that using the GetLogsResult import type ties it the shape defined in the other module (and its package version) and was worried it make it harder to pass in an object from another location (rather from this specific function in this specific module). But maybe that's not how these types actually work and they just compare shapes.

This worry comes from seeing weird type errors when certain package versions get out of alignment and is why we have most deps pinned to a specific version rather than a version range.

Comment on lines 60 to 62
// TODO: standardize on calling these "fields" or "values" or maybe "columns"
valueName: TValue;
value: Value<TConfig, TTable>[TValue];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we go with fieldName and fieldValue?

[TTable in keyof TConfig["tables"]]: {
name: TTable;
key: Key<TConfig, TTable>;
record: Value<TConfig, TTable>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we go with value here based on the "consistent naming" doc?

type TableKey = `${Address}:${TableNamespace}:${TableName}`;

// hacky fix for schema registration + metadata events spanning multiple blocks
// TODO: remove this once schema registration+metadata is one event or tx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO: remove this once schema registration+metadata is one event or tx
// TODO: remove this once schema registration+metadata is one event or tx
// (https://github.com/latticexyz/mud/pull/1182)

}
});

// Then find all metadata events. These should follow schema registration events and be in the same block (since they're in the same tx).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like there is a comment missing above like "First find all schema events..." or something

});

// Then find all metadata events. These should follow schema registration events and be in the same block (since they're in the same tx).
// TODO: rework contracts so schemas+tables are combined and immutable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO: rework contracts so schemas+tables are combined and immutable
// TODO: rework contracts so schemas+tables are combined and immutable
// (https://github.com/latticexyz/mud/pull/1182)

log.args.data
);
const valueNames = decodeAbiParameters(parseAbiParameters("string[]"), abiEncodedFieldNames as Hex)[0];
// TODO: add key names to table registration when we refactor it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO: add key names to table registration when we refactor it
// TODO: add key names to table registration when we refactor it
// (https://github.com/latticexyz/mud/pull/1182)

Copy link
Member

@alvrs alvrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@holic holic merged commit eeb15cc into main Jul 19, 2023
@holic holic deleted the holic/block-logs-to-storage branch July 19, 2023 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants