Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sinner #1380

Merged
merged 82 commits into from
Nov 2, 2023
Merged

Sinner #1380

Show file tree
Hide file tree
Changes from 79 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
41d4839
init
nautsimon Sep 15, 2023
7717370
Merge branch 'master' into feat/sinner
nautsimon Sep 26, 2023
fa87c69
stashing at prev checkpoint to resolve build issues
nautsimon Sep 28, 2023
b5b435a
Merge branch 'master' into feat/sinner
nautsimon Sep 28, 2023
67cf049
updating graphql
nautsimon Sep 28, 2023
1e29933
live filling updates
nautsimon Sep 28, 2023
e72e9cf
stash prev changes before tests
nautsimon Oct 3, 2023
7e601d1
Merge branch 'master' into feat/sinner
nautsimon Oct 3, 2023
a4cffbc
update queries w/tests
nautsimon Oct 3, 2023
d2881fc
update api and cli
nautsimon Oct 5, 2023
d6c8ecf
stash before deploy
nautsimon Oct 10, 2023
da18611
api tests
nautsimon Oct 11, 2023
d57d24e
lint
nautsimon Oct 11, 2023
6326b08
lint
nautsimon Oct 11, 2023
1be0a97
lint + gen
nautsimon Oct 11, 2023
f635222
code cov
nautsimon Oct 12, 2023
3b67b45
gen
nautsimon Oct 12, 2023
0da7982
adding code cov
nautsimon Oct 12, 2023
a66dc9b
lint + test + coderabbit suggestions
nautsimon Oct 12, 2023
df0fdb9
Merge branch 'master' into feat/sinner
nautsimon Oct 12, 2023
1ab130e
symlink
nautsimon Oct 20, 2023
a098dcf
edits
nautsimon Oct 20, 2023
45fd438
Merge branch 'master' into feat/sinner
nautsimon Oct 20, 2023
677f4b9
hold
nautsimon Oct 21, 2023
d568368
Merge branch 'master' into feat/sinner
nautsimon Oct 23, 2023
be9cd26
Update go.mod
nautsimon Oct 23, 2023
a5c2273
sinner docker + main.go
nautsimon Oct 23, 2023
44c4b1e
sinner test flake, bytes -> hex string update, update assert use, lint
nautsimon Oct 23, 2023
b95faa6
update logging
nautsimon Oct 23, 2023
8aa4cc6
deprecate port flag
nautsimon Oct 23, 2023
69e3e9a
updating fetch ceil
nautsimon Oct 23, 2023
6aae818
adding concurrency during parse store
nautsimon Oct 23, 2023
514453d
add comments
nautsimon Oct 23, 2023
61ca558
gen (stringer)
nautsimon Oct 23, 2023
4f0ef6c
add build to .goreleaser.yml
nautsimon Oct 24, 2023
8c61076
build
trajan0x Oct 25, 2023
a769d9b
Revert "build" [goreleaser]
trajan0x Oct 25, 2023
431f63b
fix goreleaser [goreleaser]
trajan0x Oct 25, 2023
1df6180
fix replacement issue
trajan0x Oct 25, 2023
60a9434
[sinner] add start readme (#1499)
trajan0x Oct 25, 2023
a5a0960
sinner strings for contract_type (#1498)
trajan0x Oct 25, 2023
839902c
trajan edits
nautsimon Oct 26, 2023
ae7426d
nits
nautsimon Oct 26, 2023
20e23da
explorer update
trajan0x Oct 26, 2023
2a59e49
Sinner: resolvers and updating db read functions (#1514)
nautsimon Oct 30, 2023
3489db9
type fix
trajan0x Oct 30, 2023
aa5d87d
Merge branch 'feat/sinner' of https://github.com/synapsecns/sanguine …
trajan0x Oct 30, 2023
3c85e93
simplify w generics
trajan0x Oct 30, 2023
c04199a
add go error group to unified command + updated read.go
nautsimon Oct 30, 2023
5e3e77f
code rabbit nit
nautsimon Oct 30, 2023
cbcd01b
logger fix
nautsimon Oct 30, 2023
54561db
Parser, API, and config updates
nautsimon Oct 30, 2023
317df84
tests, sinner/indexer updates, db updates
nautsimon Oct 30, 2023
704bbfa
lint + refactor graphql + db
nautsimon Oct 31, 2023
31dc07d
Add log range asc to API
nautsimon Oct 31, 2023
e69f6ce
[goreleaser]
nautsimon Oct 31, 2023
4649eec
Update README.md
nautsimon Oct 31, 2023
17f9fde
refactor indexer
nautsimon Oct 31, 2023
2f910b4
exit 1 on fail
trajan0x Oct 31, 2023
1a3f396
Merge branch 'master' into feat/sinner
trajan0x Oct 31, 2023
be702d1
Merge branch 'feat/logrange-asc-scribe' into feat/sinner
nautsimon Oct 31, 2023
889b9ef
Merge branch 'feat/sinner' of https://github.com/synapsecns/sanguine …
nautsimon Oct 31, 2023
bf594fa
update scribe fetcher and pagination behavior
nautsimon Oct 31, 2023
5a2512d
mysql context
nautsimon Oct 31, 2023
90dcf30
mysql + refresh rate
nautsimon Oct 31, 2023
bda1b9b
update config and readme + [goreleaser]
nautsimon Oct 31, 2023
cd0a685
update tests
nautsimon Oct 31, 2023
0625402
typo + [goreleaser]
nautsimon Oct 31, 2023
c535bdf
prom exporter gen
nautsimon Oct 31, 2023
3f63283
revert abi - [goreleaser]
nautsimon Nov 1, 2023
141b7e1
[goreleaser]
nautsimon Nov 1, 2023
a3972c8
specify varchar length for indexing [goreleaser]
nautsimon Nov 1, 2023
88b4467
update 64 to 66 to account for 0x [goreleaser]
nautsimon Nov 1, 2023
fc7f11c
update api
nautsimon Nov 1, 2023
ff8e7a3
[goreleaser]
nautsimon Nov 1, 2023
f9c1142
gen
nautsimon Nov 2, 2023
d083937
update comment
nautsimon Nov 2, 2023
8298d74
[goreleaser]
nautsimon Nov 2, 2023
20e770f
Merge branch 'feat/logrange-asc-scribe' into feat/sinner
nautsimon Nov 2, 2023
c37b2d7
add asc flag
nautsimon Nov 2, 2023
64c1cb0
[goroutine]
nautsimon Nov 2, 2023
526c562
[goreleaser]
nautsimon Nov 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ flags:
paths:
- services/scribe/
carryforward: true
sinner:
paths:
- services/sinner/
carryforward: true
solidity:
paths:
- packages/contracts-core/
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,12 @@ jobs:
fi
done

# Check if max retries is exceeded for this package.
if [ $retries -eq $max_num_retries ]; then
echo "Max retries exceeded for $pkg"
exit 1
fi

if [ -f profile.cov ]; then
tail -n +2 profile.cov >> coverage.txt
rm profile.cov
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,12 @@ root
│ ├── <a href="./packages/explorer-ui">explorer-ui</a>: Explorer UI
│ ├── <a href="./packages/sdk-router">sdk-router</a>: SDK router
│ ├── <a href="./packages/sdk-router">synapse-interface</a>: Synapse frontend code
├── <a href="./tools">services</a>
├── <a href="./services">services</a>
│ ├── <a href="./services/cctp-relayer">CCTP Relayer</a>: CCTP message relayer
│ ├── <a href="./services/explorer">explorer</a>: Bridge/messaging explorer backend
│ ├── <a href="./services/omnirpc">omnirpc</a>: Latency aware RPC Client used across multiple-chains at once
│ ├── <a href="./services/scribe">scribe</a>: Generalized ethereum event logger
│ ├── <a href="./services/omnirpc">omnirpc</a>: Latency aware RPC Client used across multiple-chains at once
│ ├── <a href="./services/sinner">sinner</a>: [Synapse Interchain Network](https://interchain.synapseprotocol.com/) indexer & query interface
├── <a href="./tools">tools</a>
│ ├── <a href="./tools/abigen">abigen</a>: Used to generate abigen bindings for go
│ ├── <a href="./tools/bundle">bundle</a>: Modified version of <a href="https://pkg.go.dev/golang.org/x/[email protected]/cmd/bundle"> go bundler </a> with improved shadowing support
Expand Down
14 changes: 10 additions & 4 deletions contrib/promexporter/internal/gql/dfk/client.gen.go
nautsimon marked this conversation as resolved.
Show resolved Hide resolved

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

353 changes: 202 additions & 151 deletions contrib/promexporter/internal/gql/dfk/models.gen.go

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions docker/sinner.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM gcr.io/distroless/static:latest

LABEL org.label-schema.description="Sinner Docker file"
LABEL org.label-schema.name="ghcr.io/synapsecns/sanguine/sinner"
LABEL org.label-schema.schema-version="1.0.0"
LABEL org.label-schema.vcs-url="https://github.com/synapsecns/sanguine"
LABEL org.opencontainers.image.source="https://github.com/synapsecns/sanguine"
LABEL org.opencontainers.image.description="Sinner Docker image"

USER nonroot:nonroot

WORKDIR /app
COPY --chown=nonroot:nonroot sinner /app/sinner

ENTRYPOINT ["/app/sinner"]
1 change: 1 addition & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,6 @@ use (
./services/explorer
./services/omnirpc
./services/scribe
./services/sinner
./tools
)
18 changes: 14 additions & 4 deletions services/explorer/consumer/client/resolver-client/server.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 15 additions & 8 deletions services/scribe/api/data_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types"
. "github.com/stretchr/testify/assert"
"github.com/synapsecns/sanguine/core"
"github.com/synapsecns/sanguine/services/scribe/db"
"github.com/synapsecns/sanguine/services/scribe/graphql"
"github.com/synapsecns/sanguine/services/scribe/grpc/client/rest"
Expand Down Expand Up @@ -50,11 +51,17 @@ func (g APISuite) TestRetrieveData() {
logs, err := g.gqlClient.GetLogs(g.GetTestContext(), int(chainID), 1)
Nil(g.T(), err)
// there were 20 logs created (2 per loop, in a loop of 10)
Equal(g.T(), len(logs.Response), 20)
logsRange, err := g.gqlClient.GetLogsRange(g.GetTestContext(), int(chainID), 2, 5, 1)
Equal(g.T(), 20, len(logs.Response))
logsRange, err := g.gqlClient.GetLogsRange(g.GetTestContext(), int(chainID), 2, 5, 1, nil)
Nil(g.T(), err)
// from 2-5, there were 8 logs created (2 per loop, in a range of 4)
Equal(g.T(), len(logsRange.Response), 8)
Equal(g.T(), 8, len(logsRange.Response))

// Test getting logs in a range in ascending order.
logsRangeAsc, err := g.gqlClient.GetLogsRange(g.GetTestContext(), int(chainID), 2, 5, 1, core.PtrTo(true))
Nil(g.T(), err)
Equal(g.T(), 8, len(logsRangeAsc.Response))
Equal(g.T(), 2, logsRangeAsc.Response[0].BlockNumber)

// test get logs and get logs in a range (GRPC)
grpcLogs, res, err := g.grpcRestClient.ScribeServiceApi.ScribeServiceFilterLogs(g.GetTestContext(), rest.V1FilterLogsRequest{
Expand Down Expand Up @@ -213,17 +220,17 @@ func (g APISuite) TestBlockTimeDataEquality() {
Nil(g.T(), err)

// check that the data is equal
Equal(g.T(), *retrievedBlockTime.Response, int(blockTime))
Equal(g.T(), int(blockTime), *retrievedBlockTime.Response)

// check that the last stored block is correct
lastBlock, err := g.gqlClient.GetLastStoredBlockNumber(g.GetTestContext(), int(chainID))
Nil(g.T(), err)
Equal(g.T(), *lastBlock.Response, int(blockNumber))
Equal(g.T(), int(blockNumber), *lastBlock.Response)

// check that the first stored block is correct
firstBlock, err := g.gqlClient.GetFirstStoredBlockNumber(g.GetTestContext(), int(chainID))
Nil(g.T(), err)
Equal(g.T(), *firstBlock.Response, int(blockNumber))
Equal(g.T(), int(blockNumber), *firstBlock.Response)
Comment on lines 220 to +233
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test case for checking the last and first stored block numbers is not correct. The test case assumes that the last and first stored block numbers are the same, which might not be the case in a real-world scenario. You should store multiple block times with different block numbers and then check the last and first stored block numbers.

}

func (g *APISuite) buildLog(contractAddress common.Address, blockNumber uint64) types.Log {
Expand Down Expand Up @@ -291,7 +298,7 @@ func (g APISuite) TestLastContractIndexed() {
Nil(g.T(), err)

// check that the data is equal
Equal(g.T(), *retrievedBlockTime.Response, int(blockNumber))
Equal(g.T(), int(blockNumber), *retrievedBlockTime.Response)
Comment on lines 298 to +301
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test case for checking the last indexed block number is not correct. The test case assumes that the last indexed block number is the same as the block number used to store the last indexed block. However, the last indexed block number should be retrieved from the database and not assumed.

}

// nolint:dupl
Expand Down Expand Up @@ -390,7 +397,7 @@ func (g APISuite) TestLastConfirmedBlock() {
Nil(g.T(), err)

// check that the data is equal
Equal(g.T(), *retrievedBlockTime.Response, int(blockNumber))
Equal(g.T(), int(blockNumber), *retrievedBlockTime.Response)
Comment on lines 397 to +400
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test case for checking the last confirmed block number is not correct. The test case assumes that the last confirmed block number is the same as the block number used to store the last confirmed block. However, the last confirmed block number should be retrieved from the database and not assumed.

}

// nolint:dupl
Expand Down
4 changes: 2 additions & 2 deletions services/scribe/cmd/cmd.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
[![Go Reference](https://pkg.go.dev/badge/github.com/synapsecns/sanguine/services/scribe.svg)](https://pkg.go.dev/github.com/synapsecns/sanguine/services/scribe)
[![Go Report Card](https://goreportcard.com/badge/github.com/synapsecns/sanguine/services/scribe)](https://goreportcard.com/report/github.com/synapsecns/sanguine/services/scribe)

Scribe is a multi chain indexing service. Scribe is designed to take a list of contracts specified by chain id and store logs, receipts, and txs for every event, past to present, in a mysql database.
Scribe is a multi-chain indexing service. Scribe is designed to take a list of contracts specified by chain id and store logs, receipts, and txs for every event, past to present, in a mysql database.

Use cases
- Analytics for on chain events
- Analytics for on-chain events
- Stream events occurring across chains
- Monitor activity on your contracts

Expand Down
1 change: 0 additions & 1 deletion services/scribe/db/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ func NewEventDBSuite(tb testing.TB) *DBSuite {

func (t *DBSuite) SetupTest() {
t.TestSuite.SetupTest()

t.logIndex.Store(0)

sqliteStore, err := sqlite.NewSqliteStore(t.GetTestContext(), filet.TmpDir(t.T(), ""), t.scribeMetrics, false)
Expand Down
7 changes: 4 additions & 3 deletions services/scribe/graphql/client/client.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions services/scribe/graphql/client/queries/queries.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ query GetLogs ($chain_id: Int!, $page: Int!) {
}
}

query GetLogsRange ($chain_id: Int!, $start_block: Int!, $end_block: Int!, $page: Int!) {
response: logsRange (chain_id: $chain_id, start_block: $start_block, end_block: $end_block, page: $page) {
query GetLogsRange ($chain_id: Int!, $start_block: Int!, $end_block: Int!, $page: Int!, $asc: Boolean = false) {
response: logsRange (chain_id: $chain_id, start_block: $start_block, end_block: $end_block, page: $page,asc: $asc) {
contract_address
chain_id
topics
Expand Down
11 changes: 9 additions & 2 deletions services/scribe/graphql/server/graph/queries.resolvers.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading