feat: chain indexer todos [skip changelog] #12462

akaladarshi · 2024-09-14T11:20:50Z

This PR is part of ChainIndexer tooling PR:

It addresses the TODO's and comments

aarshkshah1992 · 2024-09-16T11:56:25Z

chainindex/api.go

@@ -57,9 +57,10 @@ func (si *SqliteIndexer) getTipsetCountsAtHeight(ctx context.Context, height abi
 }

 func (si *SqliteIndexer) ChainValidateIndex(ctx context.Context, epoch abi.ChainEpoch, backfill bool) (*types.IndexValidation, error) {
-	if !si.started {
-		return nil, xerrors.Errorf("ChainValidateIndex can only be called after the indexer has been started")
+	if !si.started && si.closed {


You need to take a read lock before reading this. See how we do it elsewhere.

Updated.
Added a function isClosed, so we can reuse it.

aarshkshah1992 · 2024-09-16T11:58:20Z

chainindex/api.go

+}
+
+func (si *SqliteIndexer) getIndexedTipSetData(ctx context.Context, tsKey types.TipSetKey) (messageCount uint64, eventCount uint64, hasRevertedEvents bool, err error) {
+	tsKeyBytes := tsKey.Bytes()


let's return a struct here with these fields instead of returning multiple values here.

aarshkshah1992 · 2024-09-16T11:59:04Z

chainindex/api.go

+}
+
+func (si *SqliteIndexer) getIndexedTipSetData(ctx context.Context, tsKey types.TipSetKey) (messageCount uint64, eventCount uint64, hasRevertedEvents bool, err error) {
+	tsKeyBytes := tsKey.Bytes()


This is incorrect. You need to use the tipsetKey.Cid.Bytes path here to get the key to look up the DB.

aarshkshah1992 · 2024-09-16T12:01:30Z

chainindex/api.go

+// verifyIndexedData verifies that the indexed data for a tipset is correct
+// by comparing the number of messages in the chainstore to the number of messages indexed
+
+// TODO: verify indexed events too (to verify the events we need to load the next tipset (ts+1) and verify the events are the same)


Please can you implement this TODO to verify events as well ?

You can use the loadExecutedMessages function we have for this. Then you can just read messages and events together.

I was little skeptical about events, because we need to do something like currTipset.Height() + 1 to get execution tipset then fetch data for it through loadExecuteMessages.

So should I go ahead with this?

@akaladarshi

Hmm..good point.

Yeah let's do it. So, here are the steps:

Load tipset tsX at user supplied epoch e

Load tipset tsY at epoch e+1

Ensure parent of ``tsY==tsX` (if there is a mismatch here, return an error because it means chain has forked and user needs to retry)

Load events and messages and validate the counts.

aarshkshah1992 · 2024-09-16T12:04:27Z

chainindex/api.go

+		}
+
+		// if there are events, we need to check if there are reverted events
+		if eventCount > 0 {


@akaladarshi I think this is incorrect. eventCount will be >0 ONLY if non-reverted events exist. So, we should check for reverted events ONLY if eventCount==0.

aarshkshah1992 · 2024-09-16T14:30:53Z

chain/index/ddls.go

@@ -82,5 +82,8 @@ func preparedStatementMapping(ps *preparedStatements) map[**sql.Stmt]string {
 		&ps.hasNullRoundAtHeightStmt:              "SELECT NOT EXISTS(SELECT 1 FROM tipset_message WHERE height = ?)",
 		&ps.getNonRevertedTipsetAtHeightStmt:      "SELECT tipset_key_cid FROM tipset_message WHERE height = ? AND reverted = 0",
 		&ps.countTipsetsAtHeightStmt:              "SELECT COUNT(CASE WHEN reverted = 1 THEN 1 END) AS reverted_count, COUNT(CASE WHEN reverted = 0 THEN 1 END) AS non_reverted_count FROM (SELECT tipset_key_cid, MAX(reverted) AS reverted FROM tipset_message WHERE height = ? GROUP BY tipset_key_cid) AS unique_tipsets",
+		&ps.getNonRevertedTipsetMessageCountStmt:  "SELECT COUNT(*) FROM tipset_message WHERE tipset_key_cid = ? AND reverted = 0",
+		&ps.getNonRevertedTipsetEventCountStmt:    "SELECT COUNT(*) FROM event WHERE message_id IN (SELECT message_id FROM tipset_message WHERE tipset_key_cid = ? AND reverted = 0)",


@akaladarshi What if the tipset is NOT reverted but the events are reverted ? This query only ensures that the tipset is NOT reverted. Does it have any implications on the correctness of the code /API ?

When you say events are reverted that must mean that it's corresponding message is also reverted.
Shouldn't we mark message inside that tipset reverted as well?

would it be corruption if one were reverted and the other not? is that something we should also be checking for?

aarshkshah1992 · 2024-09-16T14:33:38Z

chain/index/api.go

@@ -57,9 +57,10 @@ func (si *SqliteIndexer) getTipsetCountsAtHeight(ctx context.Context, height abi
 }

 func (si *SqliteIndexer) ChainValidateIndex(ctx context.Context, epoch abi.ChainEpoch, backfill bool) (*types.IndexValidation, error) {
-	if !si.started {
-		return nil, xerrors.Errorf("ChainValidateIndex can only be called after the indexer has been started")
+	if !si.started && si.isClosed() {


OR closed. Not AND, right ?

aarshkshah1992 · 2024-09-16T14:34:26Z

chain/index/api.go

 		return nil, xerrors.Errorf("index corruption: non-reverted tipset at height %d has key %s, but canonical chain has %s", epoch, indexedTsKeyCid, expectedTsKeyCid)
 	}

+	// indexedTsKeyCid and expectedTsKeyCid are the same, so we can use `expectedTs` to fetch the indexed data
+	indexedData, err := si.getIndexedTipSetData(ctx, expectedTs.Key())


Just pass expectedTs here and use the toTipsetKeyCidBytes function we have to find the database key.

aarshkshah1992 · 2024-09-16T14:35:10Z

chain/index/api.go

-	if !si.started {
-		return nil, xerrors.Errorf("ChainValidateIndex can only be called after the indexer has been started")
+	if !si.started && si.isClosed() {
+		return nil, xerrors.Errorf("ChainValidateIndex can only be called after the indexer has been started and not closed")


have different if statements for both conditions so the error message can be more granular.

aarshkshah1992 · 2024-09-16T14:37:47Z

chain/index/api.go

+		Backfilled:     true,
+		TotalMessages:  indexedData.nonRevertedMessageCount,
+		TotalEvents:    indexedData.nonRevertedEventCount,
+		EventsReverted: indexedData.hasRevertedEvents,


@akaladarshi I think this variable will confuse users more than help them. Let's just get rid of it and simply have a NonRevertedEventsCount value on the IndexValidation struct.

you mean rename the IndexValidation fields?

Yes. Let's remove EventsReverted from IndexValidation. We can add it later if we need it.

Then, I think we can also remove the query for checking, if the events were reverted.

if data.nonRevertedEventCount == 0 { if err = tx.Stmt(si.stmts.hasRevertedEventsStmt).QueryRowContext(ctx, tsKeyCidBytes).Scan(&data.hasRevertedEvents); err != nil { return xerrors.Errorf("failed to check for reverted events: %w", err) } }

aarshkshah1992 · 2024-09-16T14:39:28Z

chain/index/api.go

+type indexedTipSetData struct {
+	nonRevertedMessageCount int
+	nonRevertedEventCount   int
+	hasRevertedEvents       bool


Can you add a test for these fields to TestEthGetLogsBasic in eth_filtest_test.go ?

We already call this API there and so this should be easy. See

iv, err := client.ChainValidateIndex(ctx, abi.ChainEpoch(0), false) require.NoError(err) require.NotNil(iv) fmt.Printf("index validation: %v\n", iv) iv, err = client.ChainValidateIndex(ctx, abi.ChainEpoch(22), false) require.NoError(err) require.NotNil(iv) fmt.Printf("index validation: %v\n", iv)

The test is failing right now because loadExecutedMessage is not loading any messages, and we are using it to verify the indexed data.

aarshkshah1992 · 2024-09-16T14:40:45Z

chain/index/api.go

+
+	var data indexedTipSetData
+	err = withTx(ctx, si.db, func(tx *sql.Tx) error {
+		if err = tx.Stmt(si.stmts.getNonRevertedTipsetMessageCountStmt).QueryRowContext(ctx, tsKeyBytes).Scan(&data.nonRevertedMessageCount); err != nil {


You don't need to use a transaction here as you are only reading from the DB.

I was using transaction so I get the consistent view of DB for querying.

aarshkshah1992 · 2024-09-16T14:45:24Z

chain/index/api.go

+	if err != nil {
+		return nil, xerrors.Errorf("failed to get tipset message and event counts at height %d: %w", expectedTs.Height(), err)
+	}
+


ensure indexedData is not nil before proceeding here.

Ideally, it will only be nil if error is there and if there is error and indexedData is not nil then that means issue is in DB, so we should return.

Yes but no harm in checking it and returning an error of it is nil here so user's node dosen't crash if there exists a bug that causes a nil value here.

aarshkshah1992 · 2024-09-16T14:47:35Z

chain/index/api.go

+	}
+
+	msgCount := len(msgs)
+	if msgCount != indexedData.nonRevertedMessageCount {


You don't need to call MessagesForTipset here and compare number of messages since you are calling loadExecutedMessages below which gives you both messages and events, right ? You can just compare then.

I was about to ask this as well.

So here we are getting inclusionMessage and below we are getting executedMsg, I just wanted to confirm both are same (ideally they should be)

Look at what the loadExecutedMessages in it's implementation. Reading the implementation code will give you a better understanding of how it all works together.

Just checked, you're correct it's the same thing, I will update.

rvagg · 2024-09-17T07:13:40Z

chain/index/indexer.go

@@ -227,7 +225,7 @@ func (si *SqliteIndexer) indexSignedMessage(ctx context.Context, tx *sql.Tx, msg

 func (si *SqliteIndexer) Apply(ctx context.Context, from, to *types.TipSet) error {
 	si.closeLk.RLock()


double lock here because you do it in isClosed too

rvagg · 2024-09-17T07:22:15Z

chain/index/api.go

+}
+
+// getIndexedTipSetData fetches the indexed tipset data for a tipset
+func (si *SqliteIndexer) getIndexedTipSetData(ctx context.Context, ts *types.TipSet) (*indexedTipSetData, error) {


how about you just return a mostly-filled IndexValidation from this that can be built on or merged from here? alternatively just 2 ints since that's all its doing

rvagg · 2024-09-17T07:27:27Z

chain/index/api.go

+// verifyIndexedData verifies that the indexed data for a tipset is correct
+// by comparing the number of messages and events in the chainstore to the number of messages and events indexed.
+// NOTE: Events are loaded from the executed messages of the tipset at the next epoch (ts.Height() + 1).
+func (si *SqliteIndexer) verifyIndexedData(ctx context.Context, ts *types.TipSet, indexedData *indexedTipSetData) (err error) {


ah I see why you want a struct for this, but it's still just 2 ints and it might even be clearer to provide them as named arguments here

rvagg · 2024-09-17T07:28:21Z

chain/index/api.go

+	}
+
+	// get the tipset where the messages of `ts` will be executed (deferred execution)
+	executionTs, err := si.cs.GetTipsetByHeight(ctx, ts.Height()+1, nil, false)


see discussion in #12447 (comment), same problem here I think, particularly as you get close to the tip of the chain

@rvagg Which is why we error out here if the parent of this tipset is not the original tipset. We can then ask the user to retry as that means that the chain has forked between the two tipsets.

I don't think we want to re-execute the tipset/regenerate events here.

aarshkshah1992 · 2024-09-17T07:47:29Z

Merging this and continuing all work as part of the parent PR. Will address all review comments there.

aarshkshah1992 · 2024-09-17T07:49:35Z

Thanks for all your work here @akaladarshi 👍

@jennijuju

…for the `ChainIndexer` (#12450) * fix conflicts with chain indexer * feat: chain indexer todos [skip changelog] (#12462) * feat: finish todos of validation api * feat: add indexed data verification with chain store * feat: address comments and finish TODO * fix: build issue * address comments * fix: ci issue * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * changes to Index Validation API based on Rodds first review * build chain indexer API * improve error handling * feat: lotus-shed tooling for chain indexer (#12474) * feat: add lotus-shed command for backfilling chain indexer * feat: add lotus-shed command for inspecting the chain indexer * feat: use single lotus-shed command to inspect and backfill * fix: remove the unused queries * small changes * add change log * backfilling improvements and fixes * finish chain index validation and backfill tooling * user documentation for the * validate from epoch * Apply suggestions from code review Suggestions from Steve's read of the user doc. Co-authored-by: Steve Loeppky <[email protected]> * changes to user doc as per review * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * changes to user doc as per review * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * changes as per review * feat: add event entries count in validation API (#12506) * feat: add event entry count in validation API * address comments * use sqllite defaults (#12504) * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * write chain index to a different dir * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * fix conflicts * UX improvements to backfilling * feat: tests for the chain indexer (#12521) * ddl tests * tests for the chain indexer * finish unit tests for chain indexer * fix formatting * cleanup reverted tipsets to avoid db bloat * fix logging * test for filter by address * test gc cascade delete * fix db locked error during backfilling * fix var name * increase db locked timeout * fix db locked issue * reduce db lock timeout * no lock in gc * reconcile does not need lock * improved error handling * Update chain-indexing-overview-for-rpc-providers.md Doc updates based on @jennijuju feedack. * Update chain-indexing-overview-for-rpc-providers.MD Fixes after reviewing 33c1ca1 * better metrics for backfilling * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * tests for changes to event addressing * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * changes as per review -> round 1 * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * log tipset key cid * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix docs * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix tests * fix tests * make jen * fix conflicts --------- Co-authored-by: Aryan Tikarya <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steve Loeppky <[email protected]>

@jennijuju

* chain index complete for msgs and txns * dont need observer changes for now * changes * fix tests * fix tests * use th right context * index empty tipsets correctly * implement automated backfilling * add event indexing and remove all old indices * fix test * revert deployment test changes * revert test changes and better error handling for eth tx index lookups * fix sql statments naming convention * address review for Index GC * more changes as per review * changes as per review * fix config * mark events as reverted during reconciliation * better reconciliation; pens down and code complete; also reconcile events * fix tests * improve config and docs * improve docs and error handling * improve read logic * improve docs * better logging and handle ennable event storage * improve logs and index init proc * better logging * fix bugs based on calibnet testing * create sqliite Indices * gc should be based on epochs * fix event query * foreign keys should be enabled on the DB * reverted tipsets should be removed as part of GC * release read lock * make it easy to backfill an empty index using reconciliation * better docs for reconciliation * fix conflicts with master * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix go mod * fix formatting * revert config changes * address changes in observer * remove top level chainindex package * changes as per review * changes as per review * changes as per review * handle index with reverted tipsets during reconciliation * changes as per review * fix type of max reconcile epoch * changes to reconciliation as per review * log ipld error * better logging of progress * disable chain indexer hydrate from snapshot based on config * always populate index * make config easy to reason about * fix config * fix messaging * revert config changes * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * changes as per review * make error messages homogenous * fix indentation * changes as per review * feat: recompute tipset to generate missing events if event indexing is enabled (#12463) * auto repair events * make jen * fix leaky abstraction * better docs for gc retention epoch * imrpove DB handling (#12485) * fix conflict * fix lite node config for indexer * exclude reverted events from eth get logs if client queries by epoch * Simply addressing for event lookups in the index. simply addressing for event lookups * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix tests * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * feat: migration("re-indexing"), backfilling and diasgnostics tooling for the `ChainIndexer` (#12450) * fix conflicts with chain indexer * feat: chain indexer todos [skip changelog] (#12462) * feat: finish todos of validation api * feat: add indexed data verification with chain store * feat: address comments and finish TODO * fix: build issue * address comments * fix: ci issue * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * changes to Index Validation API based on Rodds first review * build chain indexer API * improve error handling * feat: lotus-shed tooling for chain indexer (#12474) * feat: add lotus-shed command for backfilling chain indexer * feat: add lotus-shed command for inspecting the chain indexer * feat: use single lotus-shed command to inspect and backfill * fix: remove the unused queries * small changes * add change log * backfilling improvements and fixes * finish chain index validation and backfill tooling * user documentation for the * validate from epoch * Apply suggestions from code review Suggestions from Steve's read of the user doc. Co-authored-by: Steve Loeppky <[email protected]> * changes to user doc as per review * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * changes to user doc as per review * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * changes as per review * feat: add event entries count in validation API (#12506) * feat: add event entry count in validation API * address comments * use sqllite defaults (#12504) * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * write chain index to a different dir * Apply suggestions from code review Co-authored-by: Steve Loeppky <[email protected]> * fix conflicts * UX improvements to backfilling * feat: tests for the chain indexer (#12521) * ddl tests * tests for the chain indexer * finish unit tests for chain indexer * fix formatting * cleanup reverted tipsets to avoid db bloat * fix logging * test for filter by address * test gc cascade delete * fix db locked error during backfilling * fix var name * increase db locked timeout * fix db locked issue * reduce db lock timeout * no lock in gc * reconcile does not need lock * improved error handling * Update chain-indexing-overview-for-rpc-providers.md Doc updates based on @jennijuju feedack. * Update chain-indexing-overview-for-rpc-providers.MD Fixes after reviewing 33c1ca1 * better metrics for backfilling * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * Update chain/index/chain-indexing-overview-for-rpc-providers.MD Co-authored-by: Rod Vagg <[email protected]> * tests for changes to event addressing * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * changes as per review -> round 1 * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * log tipset key cid * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix docs * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix tests * fix tests * make jen * fix conflicts --------- Co-authored-by: Aryan Tikarya <[email protected]> Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Steve Loeppky <[email protected]> * fix lint * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * remove reverted flag from RPC * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix testing of events and dummy chain store * remove lotus shed commands for old Indices * change type of event counts to uint64 * only recompute events if theyre not found * short-circuit empty events path for older tipsets * chain indexer must be enabled if ETH RPC is enabled * change name of message_id column to id in tipset_message table * only expose SetRecomputeTipSetStateFunc * dont block on head indexing for reading messages * document why we're only checking for missing events for a single tipset * document when we query for reverted events * simplify event collection * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * fix test * change event_id to id in the event table * change head indexed timeout * remove deprecated config options * fail ETH RPC calls if ChainIndexer is disabled * fix docs * remove the tipset key cid func from lotus shed * address review comments * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * chore(events): remove unnecessary DisableRealTimeFilterAPI (#12610) * feat(cli): add --quiet to chainindex validate-backfill + cleanups (#12611) * fix tests * Apply suggestions from code review Co-authored-by: Rod Vagg <[email protected]> * error type for disabled chainindexer * fix(chainindex): recompute tipset when we find no receipts * fix(chainindexer): backfilling should halt when chain state data is missing and not backfill parents (#12619) * fix backfilling UX * Update chain/index/api.go Co-authored-by: Rod Vagg <[email protected]> * address review --------- Co-authored-by: Rod Vagg <[email protected]> * reduce log noise * make jen * make jen * docs: finishing chain-indexer-overview-for-operators.md (#12600) * Followup to PR #12450 for doc updates This is being used to resolve the unresolved items in #12450 since that PR is unwieldly at this point. * Incorporated some items and added TODOs based on unresolved items from #12450 * Incorporating more feedback * Pointing to issue to learn about benefits * Formatting fixes * Apply most of the suggestions from @rvagg code review Co-authored-by: Rod Vagg <[email protected]> * Incorporating feedback from #12600 (comment) * Addressing #12600 (comment) and more * Moved chain-indexer docs to documentation Renamed Added ToC We can move to lotus-docs later * Update documentation/en/chain-indexer-overview-for-operators.md Co-authored-by: Rod Vagg <[email protected]> * Update documentation/en/chain-indexer-overview-for-operators.md Co-authored-by: Rod Vagg <[email protected]> * Added upgrade path when importing chain state from a snapshot. * Typo fixes * Update documentation/en/chain-indexer-overview-for-operators.md Co-authored-by: Rod Vagg <[email protected]> * chore(doc): "regular checks" section for chainindexer docs (#12612) * Apply suggestions from @rvagg code review Co-authored-by: Rod Vagg <[email protected]> * Incorporating @aarshkshah1992 feedback * Update documentation/en/chain-indexer-overview-for-operators.md Co-authored-by: Rod Vagg <[email protected]> --------- Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Aarsh Shah <[email protected]> * remove go mod replace * remove unnecessary changes from CHANGELOG * fix test * compare events AMT root (#12632) * fix(chainindex): retry transaction if database connection is lost (#12657) * retry database lost connection * log context cancellation * address review * fix gateway itest: no chainindexer for lite nodes * fix changelog --------- Co-authored-by: Rod Vagg <[email protected]> Co-authored-by: Aryan Tikarya <[email protected]> Co-authored-by: Steve Loeppky <[email protected]>

akaladarshi changed the title ~~feat: chain indexer lotus-shed tooling and changes~~ feat: chain indexer todos Sep 16, 2024

akaladarshi marked this pull request as ready for review September 16, 2024 08:06

akaladarshi changed the title ~~feat: chain indexer todos~~ feat: chain indexer todos [skip changelog] Sep 16, 2024

rjan90 assigned akaladarshi Sep 16, 2024

rjan90 requested a review from aarshkshah1992 September 16, 2024 08:12

aarshkshah1992 reviewed Sep 16, 2024

View reviewed changes

aarshkshah1992 force-pushed the feat/implement-index-validation-api branch from a769f4e to 0f4c627 Compare September 16, 2024 12:44

akaladarshi added 3 commits September 16, 2024 19:06

feat: finish todos of validation api

632c620

feat: add indexed data verification with chain store

e84a794

feat: address comments and finish TODO

1b57672

akaladarshi force-pushed the akaladarshi/index-validation-api-todos branch from 4ded0e4 to 1b57672 Compare September 16, 2024 14:26

akaladarshi requested a review from aarshkshah1992 September 16, 2024 14:28

aarshkshah1992 reviewed Sep 16, 2024

View reviewed changes

fix: build issue

0d22bcf

aarshkshah1992 reviewed Sep 16, 2024

View reviewed changes

akaladarshi added 2 commits September 17, 2024 12:30

address comments

8f2fa81

fix: ci issue

f0b4739

rvagg reviewed Sep 17, 2024

View reviewed changes

aarshkshah1992 merged commit 1b5fed3 into filecoin-project:feat/implement-index-validation-api Sep 17, 2024
70 of 75 checks passed

		@@ -227,7 +225,7 @@ func (si SqliteIndexer) indexSignedMessage(ctx context.Context, tx sql.Tx, msg

		func (si SqliteIndexer) Apply(ctx context.Context, from, to types.TipSet) error {
		si.closeLk.RLock()

feat: chain indexer todos [skip changelog] #12462

feat: chain indexer todos [skip changelog] #12462

Conversation

akaladarshi commented Sep 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aarshkshah1992 Sep 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aarshkshah1992 Sep 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aarshkshah1992 commented Sep 17, 2024

aarshkshah1992 commented Sep 17, 2024

akaladarshi commented Sep 14, 2024 •

edited

Loading

aarshkshah1992 Sep 16, 2024 •

edited

Loading

aarshkshah1992 Sep 17, 2024 •

edited

Loading