Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(store-sync,store-indexer): make last updated block number not null #1972

Merged
merged 3 commits into from
Nov 29, 2023
Merged

fix(store-sync,store-indexer): make last updated block number not null #1972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6aed50e
make last updated block number not null, for easier sorting
holic Nov 29, 2023
97841da
Create cyan-vans-pay.md
holic Nov 29, 2023
fe69511
Create beige-rockets-return.md
holic Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/beige-rockets-return.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@latticexyz/store-indexer": patch
---

Records are now ordered by `lastUpdatedBlockNumber` at the Postgres SQL query level
5 changes: 5 additions & 0 deletions .changeset/cyan-vans-pay.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@latticexyz/store-sync": major
---

`lastUpdatedBlockNumber` columns in Postgres storage adapters are no longer nullable
5 changes: 3 additions & 2 deletions packages/store-indexer/src/postgres/getLogs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { PgDatabase } from "drizzle-orm/pg-core";
import { Hex } from "viem";
import { StorageAdapterLog, SyncFilter } from "@latticexyz/store-sync";
import { tables } from "@latticexyz/store-sync/postgres";
import { and, eq, or } from "drizzle-orm";
import { and, asc, eq, or } from "drizzle-orm";
import { decodeDynamicField } from "@latticexyz/protocol-parser";
import { bigIntMax } from "@latticexyz/common/utils";

Expand Down Expand Up @@ -53,7 +53,8 @@ export async function getLogs(
const records = await database
.select()
.from(tables.recordsTable)
.where(or(...conditions));
.where(or(...conditions))
.orderBy(asc(tables.recordsTable.lastUpdatedBlockNumber));
Copy link
Member Author

@holic holic Nov 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

originally had this as

.orderBy(sql`${tables.recordsTable.lastUpdatedBlockNumber} ASC NULLS FIRST`)

but then realize there's ~no case where we'd add records to the table without a block number (both RPC and another indexer will provide this info)

so I decided to make these fields not nullable


const blockNumber = records.reduce(
(max, record) => bigIntMax(max, record.lastUpdatedBlockNumber ?? 0n),
Expand Down
7 changes: 7 additions & 0 deletions packages/store-sync/src/postgres-decoded/buildTable.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,37 +21,44 @@ describe("buildTable", () => {
name: column.name,
dataType: column.dataType,
sqlName: column.sqlName,
notNull: column.notNull,
}))
).toMatchInlineSnapshot(`
{
"__keyBytes": {
"dataType": "custom",
"name": "__key_bytes",
"notNull": true,
"sqlName": "bytea",
},
"__lastUpdatedBlockNumber": {
"dataType": "custom",
"name": "__last_updated_block_number",
"notNull": false,
"sqlName": "numeric",
},
"addr": {
"dataType": "custom",
"name": "addr",
"notNull": true,
"sqlName": "bytea",
},
"name": {
"dataType": "string",
"name": "name",
"notNull": true,
"sqlName": undefined,
},
"x": {
"dataType": "custom",
"name": "x",
"notNull": true,
"sqlName": "integer",
},
"y": {
"dataType": "custom",
"name": "y",
"notNull": true,
"sqlName": "integer",
},
}
Expand Down
4 changes: 2 additions & 2 deletions packages/store-sync/src/postgres/tables.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ const schemaName = transformSchemaName("mud");
*/
const chainTable = pgSchema(schemaName).table("chain", {
chainId: asNumber("chain_id", "bigint").notNull().primaryKey(),
lastUpdatedBlockNumber: asBigInt("last_updated_block_number", "numeric"),
lastUpdatedBlockNumber: asBigInt("last_updated_block_number", "numeric").notNull(),
});

const recordsTable = pgSchema(schemaName).table(
Expand All @@ -27,7 +27,7 @@ const recordsTable = pgSchema(schemaName).table(
encodedLengths: asHex("encoded_lengths"),
dynamicData: asHex("dynamic_data"),
isDeleted: boolean("is_deleted"),
lastUpdatedBlockNumber: asBigInt("last_updated_block_number", "numeric"),
lastUpdatedBlockNumber: asBigInt("last_updated_block_number", "numeric").notNull(),
},
(table) => ({
pk: primaryKey(table.address, table.tableId, table.keyBytes),
Expand Down
Loading