Skip to content

Commit

Permalink
Merge pull request #13 from mdehoog/refcell/docs/cleanup
Browse files Browse the repository at this point in the history
Feat: Token List Parsing
  • Loading branch information
refcell authored Feb 2, 2023
2 parents 56f157a + 8aaa190 commit 237b72e
Show file tree
Hide file tree
Showing 6 changed files with 418 additions and 18 deletions.
1 change: 0 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,6 @@ run-optimism-mainnet-construction-check:
# For the genesis block hash, see:
# https://github.com/ethereum-optimism/optimism/blob/5e8bc3d5b4f36f0192b22b032e25b09f23cd0985/op-node/chaincfg/chains.go
run-optimism-mainnet:
make build \
CHAIN_CONFIG='{ "chainId": 10, "terminalTotalDifficultyPassed": true }' \
MODE=ONLINE \
PORT=8080 \
Expand Down
76 changes: 71 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,98 @@

# op-rosetta • [![tests](https://github.com/mdehoog/op-rosetta/actions/workflows/unit-tests.yaml/badge.svg?label=tests)](https://github.com/mdehoog/op-rosetta/actions/workflows/unit-tests.yaml) [![lints](https://github.com/mdehoog/op-rosetta/actions/workflows/lints.yaml/badge.svg)](https://github.com/mdehoog/op-rosetta/actions/workflows/lints.yaml)

> **Warning**
> WIP; Not bedrock-compatible yet.
Provides Rosetta API compatibility for Optimism Bedrock, a low-cost and lightning-fast Ethereum L2 blockchain.


## Overview

`op-rosetta` is an executable [client](./app/client.go) extending the [rosetta-geth-sdk](https://github.com/coinbase/rosetta-geth-sdk).
`op-rosetta` is an executable [client](./app/client.go) extending the [rosetta-geth-sdk](https://github.com/coinbase/rosetta-geth-sdk) which itself extends [rosetta-sdk-go](https://github.com/coinbase/rosetta-sdk-go).

To learn more about the Rosetta API, you can find more online at [rosetta-api.org](https://www.rosetta-api.org/).


## Setup

First, clone the repository:

```bash
git clone https://github.com/mdehoog/op-rosetta.git
```

Then, configure your own `.env` file by copying the contents of `.env.example` to `.env`.

```bash
cp .env.example .env
```

The `.env` variables will need to be configured.


## Usage

Run the `op-rosetta` client for goerli like so: `make run-optimism-goerli`.

As part of the `run-optimism-goerli` step inside the [makefile](./Makefile), a few parameters are specified before running the `bin/op-rosetta` executable:

```
CHAIN_CONFIG='{ "chainId": 10, "terminalTotalDifficultyPassed": true }' \
MODE=ONLINE \
PORT=8080 \
BLOCKCHAIN=Optimism \
NETWORK=Goerli \
ENABLE_TRACE_CACHE=true \
ENABLE_GETH_TRACER=true \
GETH=${OPTIMISM_GOERLI_NODE} \
GENESIS_BLOCK_HASH=${OPTIMISM_GOERLI_GENESIS_BLOCK_HASH} \
bin/op-rosetta
```

These parameters are configured as follows:
- `CHAIN_CONFIG` has the same field values as the genesis config’s. But only the `chainId` and `terminalTotalDifficultyPassed` values are needed.
- Setting `MODE` to `ONLINE` permits outbound connections.
- `PORT` is set to `8080` - the default connection for `rosetta-cli`.
- `BLOCKCHAIN` is `Optimism` and `NETWORK` is `Goerli`
- We enable geth tracing and caching with `ENABLE_TRACE_CACHE` and `ENABLE_GETH_TRACER`
- `GETH` points to a running op-geth *archive* node. This should be set in your `.env` file by setting `OPTIMISM_GOERLI_NODE` to a node url.
- `GENESIS_BLOCK_HASH` is the block hash of the genesis block. This should be set in your `.env` file by setting `OPTIMISM_GOERLI_GENESIS_BLOCK_HASH` to the goerli bedrock genesis block hash.

A mainnet client can be run with `make run-optimism-mainnet`.

> **Note**
> Mainnet will only be supported once Optimism mainnet is upgraded to its Bedrock Release.
The `run-optimism-mainnet` step inside the [makefile](./Makefile) specifies a similar set of variables as above:

```
CHAIN_CONFIG='{ "chainId": 10, "terminalTotalDifficultyPassed": true }' \
MODE=ONLINE \
PORT=8080 \
BLOCKCHAIN=Optimism \
NETWORK=Mainnet \
ENABLE_TRACE_CACHE=true \
ENABLE_GETH_TRACER=true \
GETH=${OPTIMISM_MAINNET_NODE} \
GENESIS_BLOCK_HASH=${OPTIMISM_MAINNET_GENESIS_BLOCK_HASH} \
bin/op-rosetta
```

These parameters are configured as follows:
- `CHAIN_CONFIG` has the same field values as the genesis config’s. But only the `chainId` and `terminalTotalDifficultyPassed` values are needed.
- Setting `MODE` to `ONLINE` permits outbound connections.
- `PORT` is set to `8080` - the default connection for `rosetta-cli`.
- `BLOCKCHAIN` is `Optimism` and `NETWORK` is `Mainnet`
- We enable geth tracing and caching with `ENABLE_TRACE_CACHE` and `ENABLE_GETH_TRACER`
- `GETH` points to a running op-geth *archive* node. This should be set in your `.env` file by setting `OPTIMISM_MAINNET_NODE` to a node url.
- `GENESIS_BLOCK_HASH` is the block hash of the genesis block. This should be set in your `.env` file by setting `OPTIMISM_MAINNET_GENESIS_BLOCK_HASH` to the mainnet bedrock genesis block hash.


## Testing

_NOTE: `op-rosetta` must be running on the specified host and port provided in the configuration file. For local testing, this can be done as described in the [Running](#running) section, which will run an instance on localhost, port 8080._

See [configs/README.md](./configs/README.md) for more information.


## Layout

```
Expand All @@ -40,10 +106,10 @@ See [configs/README.md](./configs/README.md) for more information.
│ └── README.md
└── pkg
├── client/ -> Optimism Rosetta API client
├── common/ -> Standalone common functionality
├── config/ -> Configuration options
├── handlers/ -> Rosetta API handlers
├── logging/ -> Logging utilities
├── topics/ -> Events topic functionality
└── utils/ -> Package utilities
```

Expand Down
39 changes: 39 additions & 0 deletions configs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,45 @@ To validate with `rosetta-cli`, run one of the following commands:

* `rosetta-cli check:data --configuration-file optimism/config.json` - This command validates that the Data API implementation is correct using the ethereum `optimism` node. It also ensures that the implementation does not miss any balance-changing operations.

> *Note*
>
> Running data validation takes a couple hours to completion. Once complete, the tool prints a report of any issues it encountered.
> If it fails to do so, there's likely an issue in the implementation of the `op-rosetta` client.

### Bedrock Compatibility

The rosetta configuration file must have its start_index set to the Bedrock fork block. If it’s set to any prior block then validation fails as op-rosetta fails to parse OVM (pre-Bedrock) transactions. Here’s a sample rosetta configuration file:

```
{
"network": {
"blockchain": "Optimism",
"network": "goerli"
},
"online_url": "http://localhost:5000",
"data_directory": "/tmp/rosetta-cli-goerli-bedrock-db",
"http_timeout": 300,
"max_retries": 30,
"max_online_connections": 30,
"max_sync_concurrency": 30,
"tip_delay": 120,
"compression_disabled": true,
"construction": null,
"data": {
"historical_balance_disabled": false,
"active_reconciliation_concurrency": 30,
"start_index": 4307133,
"log_balance_changes": true,
"log_transactions": true,
"log_blocks": true,
"end_conditions": {
"tip": true
}
}
}
```


### Reference

Expand Down
20 changes: 8 additions & 12 deletions pkg/config/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,19 +102,15 @@ func LoadConfiguration() (*configuration.Configuration, error) {
}
}

tokenListJson := os.Getenv(TokenListEnv)
if file, err := os.ReadFile(tokenListJson); err == nil {
// if the envvar points to a file, read it; otherwise the envvar contents is expected to be JSON
tokenListJson = string(file)
}
if tokenListJson == "" {
tokenListJson = "[]"
// Graceful tokenlist unmarshalling
tokenListJsonFilename := os.Getenv(TokenListEnv)
tokenListJsonFile, err := ReadTokenConfig(tokenListJsonFilename)
if err != nil {
return nil, fmt.Errorf("unable to parse token list %s: %w", tokenListJsonFilename, err)
}

var payload []configuration.Token
err = json.Unmarshal([]byte(tokenListJson), &payload)
tokenWhiteList, err := UnmarshalTokenConfig([]byte(tokenListJsonFile), *chainConfig.ChainID)
if err != nil {
return nil, fmt.Errorf("error parsing %s: %w", tokenListJson, err)
return nil, err
}

config.RosettaCfg = configuration.RosettaConfig{
Expand All @@ -126,7 +122,7 @@ func LoadConfiguration() (*configuration.Configuration, error) {
},
TracePrefix: "optrace",
FilterTokens: filterTokens,
TokenWhiteList: payload,
TokenWhiteList: tokenWhiteList,
SupportsSyncing: true,
}

Expand Down
85 changes: 85 additions & 0 deletions pkg/config/tokens.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
package config

import (
"encoding/json"
"fmt"
"math/big"
"os"
"strings"

"github.com/coinbase/rosetta-geth-sdk/configuration"
)

// ReadTokenConfig attempts to read file contents from a filename.
func ReadTokenConfig(filename string) (string, error) {
contents := ""
if file, err := os.ReadFile(filename); err == nil {
// if the envvar points to a file, read it; otherwise the envvar contents is expected to be JSON
contents = string(file)
}
if contents == "" {
contents = "[]"
}
return contents, nil
}

// UnmarshalTokenConfig attempts to construct a list of [configuration.Token] from a JSON file.
// Accepts variadic network IDs used to filter tokens.
func UnmarshalTokenConfig(contents []byte, networks ...big.Int) ([]configuration.Token, error) {
// Try to parse the file as a list of tokens.
var payload []configuration.Token
if err := json.Unmarshal(contents, &payload); err == nil {
return FilterNetworks(payload, networks...), nil
}

// If this fails, try the backwards-compatible token json format
var outer map[string]interface{}
if err := json.Unmarshal(contents, &outer); err == nil {
for k, v := range outer {
for t, b := range v.(map[string]interface{}) {
if b.(bool) {
switch strings.ToLower(k) {
case "mainnet":
payload = append(payload, configuration.Token{
ChainID: 1,
Address: t,
})
case "testnet":
payload = append(payload, configuration.Token{
ChainID: 420,
Address: t,
})
case "goerli":
payload = append(payload, configuration.Token{
ChainID: 420,
Address: t,
})
default:
return nil, fmt.Errorf("unknown chain %s found when parsing json token list", k)
}
}
}
}
return FilterNetworks(payload, networks...), nil
}

return nil, fmt.Errorf("error parsing file contents as json token list")
}

// FilterNetworks filters a list of [configuration.Token] by network ID.
func FilterNetworks(tokens []configuration.Token, networks ...big.Int) []configuration.Token {
if len(networks) == 0 {
return tokens
}

filtered := []configuration.Token{}
for _, token := range tokens {
for _, network := range networks {
if token.ChainID == network.Uint64() {
filtered = append(filtered, token)
}
}
}

return filtered
}
Loading

0 comments on commit 237b72e

Please sign in to comment.