Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate out logs into own table #79

Closed
ashwinphatak opened this issue Jun 1, 2021 · 7 comments
Closed

Separate out logs into own table #79

ashwinphatak opened this issue Jun 1, 2021 · 7 comments
Assignees

Comments

@ashwinphatak
Copy link
Collaborator

ashwinphatak commented Jun 1, 2021

Currently, logs don't have their own table and are stored in additional columns in the receipts table. It's desirable to create a separate table and IPLD schema for them, allowing for more efficient query and distribution of individual logs.


DEPENDENCY: vulcanize/go-codec-dageth#25.

After finishing vulcanize/go-codec-dageth#25 it will be known precisely how we want to represent logs as IPLD objects.

@i-norden
Copy link
Collaborator

When we create the new logs table, we need to also create a process to backfill the existing data.

@AlexBinesh
Copy link

Had a meeting with Ian, Rick and Ashwin. Agreed on the following structure for a new table to hold the Log data as described below:
Auto Index,

Receipt_cids.id
log.address,
(Name log.address as Address ),
log.Topics 0 (Varchar 66),
log.Topics 1 (Varchar 66),
log.Topics 2 (Varchar 66),
log.Topics 3 (Varchar 66),
Data

Additionally: Add B Tree indexes for the Topic columns and Data columns

Also : When we create the new logs table, we need to also create a process to backfill the existing data.

@AlexBinesh
Copy link

Worked on the coding. Have to parse the Logs in the receipt next

@AlexBinesh
Copy link

Created a local Logs table with the structure below using the following SQL statements:

CREATE TABLE Logs (
id SERIAL,
CONSTRAINT receipt_id
FOREIGN KEY(id)
REFERENCES eth.receipt_cids(id),

address char(42),
Topics0 Varchar (66),
Topics1 Varchar (66),
Topics2 Varchar (66),
Topics3 Varchar (66),
Data bytea);

CREATE UNIQUE INDEX topic0_idx ON Logs (Topics0);
CREATE UNIQUE INDEX topic1_idx ON Logs (Topics1);
CREATE UNIQUE INDEX topic2_idx ON Logs (Topics2);
CREATE UNIQUE INDEX topic3_idx ON Logs (Topics3);
CREATE UNIQUE INDEX address_idx ON Logs (address);

Table "public.logs"
Column | Type | Collation | Nullable | Default
---------+-----------------------+-----------+----------+----------------------------------
id | integer | | not null | nextval('logs_id_seq'::regclass)
address | character(42) | | |
topics0 | character varying(66) | | |
topics1 | character varying(66) | | |
topics2 | character varying(66) | | |
topics3 | character varying(66) | | |
data | bytea | | |
Indexes:
"address_idx" UNIQUE, btree (address)
"topic0_idx" UNIQUE, btree (topics0)
"topic1_idx" UNIQUE, btree (topics1)
"topic2_idx" UNIQUE, btree (topics2)
"topic3_idx" UNIQUE, btree (topics3)
Foreign-key constraints:
"receipt_id" FOREIGN KEY (id) REFERENCES eth.receipt_cids(id)

@i-norden
Copy link
Collaborator

Related issue: vulcanize/go-codec-dageth#25

@i-norden
Copy link
Collaborator

i-norden commented Aug 4, 2021

We need to

  1. Remove the current log rows in the receipt_cids table and add a new VARCHAR(66) row name log_root (should eventually be NOT NULL, but we need to backfill the old missing values first).
  2. Create a new table, log_cids, for indexing log IPLDs, rows in this table reference the parent receipt by fk.
  3. When processing receipts, iterate over their logs in the same manner that we iterate over txs and rcts to materialize their tries.
  4. Write the root hash of the materialized logTrie to the new log_root row in the receipt_cids table.
  5. Publish all the logTrie nodes to the IPLD blockstore like we do with the txTrie and rctTrie nodes, we don't index each individual node for these tries. Use 0x99 as the multicodec type for the CID, but it's not official yet.
  6. RLP encode and publish each log as an IPLD, in analogous fashion to how we encode and publish each individual tx and rct. Use 0x9a as the multicodec type for the CID, but it's not official yet.
  7. Index these log IPLDs with the new log_cids table by referencing them by multihash fk.

@i-norden
Copy link
Collaborator

i-norden commented Aug 20, 2021

Linking to vulcanize/statediff-migrations#7
Linking to #102

@i-norden i-norden closed this as completed Sep 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants