The fault-detector
is a simple service for detecting discrepancies between your local view of the Optimism network and the L2 output proposals published to Ethereum.
Clone, install, and build the Optimism monorepo:
git clone https://github.com/ethereum-optimism/optimism.git
yarn install
yarn build
Copy .env.example
into a new file named .env
, then set the environment variables listed there.
Once your environment variables have been set, run the service via:
yarn start
- API is exposed at
$FAULT_DETECTOR__HOSTNAME:$FAULT_DETECTOR__PORT/api
- Metrics are exposed at
$FAULT_DETECTOR__HOSTNAME:$FAULT_DETECTOR__PORT/metrics
$FAULT_DETECTOR__HOSTNAME
defaults to0.0.0.0
$FAULT_DETECTOR__PORT
defaults to7300
The fault-detector
detects differences between the transaction results generated by your local Optimism node and the transaction results actually published to Ethereum.
Currently, transaction results take the form of the root of the Optimism state trie.
For every Optimism block, the state root of the block is published to the StateCommitmentChain
contract on Ethereum.
We can therefore detect differences by, for each block, checking the state root of the given block as reported by an Optimism node and the state root as published to Ethereum.
We export a series of Prometheus metrics that you can use to trigger alerting when issues are detected.
Check the list of available metrics via yarn start --help
:
> yarn start --help
yarn run v1.22.19
$ ts-node ./src/service.ts --help
Usage: service [options]
Options:
--l1rpcprovider Provider for interacting with L1 (env: FAULT_DETECTOR__L1_RPC_PROVIDER)
--l2rpcprovider Provider for interacting with L2 (env: FAULT_DETECTOR__L2_RPC_PROVIDER)
--startbatchindex Batch index to start checking from. Setting it to -1 will cause the fault detector to find the first state batch index that has not yet passed the fault proof window (env: FAULT_DETECTOR__START_BATCH_INDEX, default value: -1)
--loopintervalms Loop interval in milliseconds (env: FAULT_DETECTOR__LOOP_INTERVAL_MS)
--bedrock Whether or not the service is running against a Bedrock chain (env: FAULT_DETECTOR__BEDROCK, default value: false)
--port Port for the app server (env: FAULT_DETECTOR__PORT)
--hostname Hostname for the app server (env: FAULT_DETECTOR__HOSTNAME)
-h, --help display help for command
Metrics:
highest_checked_batch_index Highest good batch index (type: Gauge)
highest_known_batch_index Highest known batch index (type: Gauge)
is_currently_mismatched 0 if state is ok, 1 if state is mismatched (type: Gauge)
l1_node_connection_failures Number of times L1 node connection has failed (type: Gauge)
l2_node_connection_failures Number of times L2 node connection has failed (type: Gauge)
metadata Service metadata (type: Gauge)
unhandled_errors Unhandled errors (type: Counter)
Done in 2.19s.