Problems getting blocks from Ganache #375

nenadjaja · 2018-09-17T15:18:53Z

I tried the following and all of these fail because it can't get the block from the blockchain:

Adchain-subgraph with --ethereum-rpc and ganache-cli
Adchain-subgraph with --ethereum-ipc and geth --testnet
Bitsausage-subgraph with --ethereum-rpc and ganache-cli
Bitsausage-subgraph with --ethereum-ipc and geth --testnet
I am getting one of these 2 error msgs pretty much immediately after I run cargo:
https://github.com/graphprotocol/graph-node/blob/master/datasource/ethereum/src/block_ingestor.rs#L162-L166

Versions:
Ganache CLI v6.1.8 (ganache-core: 2.2.1)
Geth Version: 1.8.14-stable

The text was updated successfully, but these errors were encountered:

Jannis · 2018-09-17T19:49:25Z

We've received another report for Ganache:

Sep 17 15:33:40.443 WARN failed to poll for latest block: ErrorMessage { msg: "could not get block from Ethereum: Decoder error: Error(\"invalid length 1, expected a 0x-prefixed hex string with length of 40\", line: 0, column: 0)" }

being logged a lot. I'm fairly certain this particular error is specific to Ganache. When polling for the latest block in the Block Ingestor, Ganache must be returning something that we are not prepared for.

Jannis · 2018-09-17T20:29:30Z

Identified one type of error:

Key not found in database

Error message

Sep 17 22:17:18.482 WARN failed to poll for latest block: ErrorMessage {
  msg: "could not get block from Ethereum: RPC error: Error {
    code: ServerError(-32000),
    message: \"Key not found in database\",
    data: Some(Object({\"stack\": String(\"NotFoundError: Key not found in database\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:773948\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:776500\\n    at /usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:748581\\n    at ReadFileContext.callback (/usr/local/lib/node_modules/ganache-cli/build/cli.node.js:2:750214)\\n    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:352:13)\"), \"name\": String(\"NotFoundError\")})) 
  }" 
}

Cause

The same network name (e.g. --ethereum-rpc ganache:...) was used with two different chains (e.g. two separate instances of Ganache; or one Ethereum mainnet and one testnet endpoint).

Why is the error thrown?

Doing this results in the Graph Node having an invalid latest block for this network. It will then get an error from Ethereum when calling eth_getBlockByHash with the hash of this block, which belongs to the previous chain but not the one in use at the time of the error.

timmclean · 2018-09-17T20:33:48Z

Ahhh knew that would come up. We talked about checking for that in #168, but might be worth a new issue just for the UX improvement

Update: filed #377

Jannis · 2018-09-17T21:39:25Z

Identified another type of error:

Invalid length 1, expected a 0x-prefixed hex string with length 40

Error message

 Sep 17 23:32:10.097 WARN failed to poll for latest block: ErrorMessage {
  msg: "could not get block from Ethereum: Decoder error: Error(\"invalid length 1, expected a 0x-prefixed hex string with length of 40\", line: 0, column: 0)"
}

Cause

Contract creation transactions have no to address. However, Ganache returns "to": "0x0" instead of "to": null. This causes rust-web3's deserialization of the Transaction object to fail (it has to: Option<H160>).

How to reproduce?

Set up Ganache with a migration that creates a contract or two. Make an eth_getBlockByHash request that includes transactions for a block that contains a contract creation transaction:

➜  subgraph git:(master) ✗ curl --insecure http://localhost:8545 \
                               -v \
                               -H "Content-Type: application/json" \
                               -d '{"jsonrpc":"2.0","method":"eth_getTransactionByHash","params":["0xc9e41a85969b440b161ece21e81bc7ce01d99a5fb01c9f754d56e51614a0de54"],"id":106}'
* Rebuilt URL to: http://localhost:8545/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8545 (#0)
> POST / HTTP/1.1
> Host: localhost:8545
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 142
>
* upload completely sent off: 142 out of 142 bytes
< HTTP/1.1 200 OK
< Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: *
< Content-Type: application/json
< Date: Mon, 17 Sep 2018 22:42:09 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
<
{"id":106,"jsonrpc":"2.0","result":{"hash":"0xc9e41a85969b440b161ece21e81bc7ce01d99a5fb01c9f754d56e51614a0de54","nonce":"0x03","blockHash":"0xc151b5fc66c4f108f466fb3068bf0d3f09e72014d45dca8302a57cbcf558b08d","blockNumber":"0x04","transactionIndex":"0x00","from":"0xe207259495d9de39e92e8b280a85b4d845906ce6","to":"0x0","value":"0x0","gas":"0x6691b7" ...

The last line above has the incorrect "to": "0x0".

The Ethereum JSON-RPC API spec states that to should be null in case of a contract creation.

Jannis · 2018-09-17T22:47:17Z

There are two possible routes here:

A fix in Ganache to return "to": null.
Custom deserialization for the to field in rust-web3, translating "0x0" to None.

leoyvens · 2018-09-17T22:49:50Z

Given that Ganache is the side not conforming to the spec, I suppose the right thing to do would be to patch them.

timmclean · 2018-09-17T23:36:39Z

I filed issue trufflesuite/ganache#907 in the ganache project repo about getting "to": "0x0" instead of "to": null

timmclean · 2018-10-19T21:01:21Z

trufflesuite/ganache#907 is closed and fixed! 🎉

Jannis · 2018-10-22T15:35:00Z

@nenadjaja to revisit if Ganache works now (may need to install it from master if the fix is not in the latest release yet).

nenadjaja · 2018-10-24T03:17:35Z

Ganache didn't release the fix, so I pulled the graph-cli repo and referenced to the Github graph-core in its package.json, but...
I am still getting this issue :-(

nenadjaja · 2018-10-24T17:23:27Z

Update - Ganache issue is fixed, but the fix is not in the latest release. To get it running locally, follow the steps:

Pull ganache-core from https://github.com/trufflesuite/ganache-core
Pull ganache-cli from https://github.com/trufflesuite/ganache-cli
Edit package.json in your local ganache-cli to include this line: "ganache-core": "[email protected]:trufflesuite/ganache-core.git" (instead of "ganache-core": "2.2.1" under dependencies)
Run npm install in both ganache-core and ganache-cli
In your terminal tab access your local ganache-cli and run npm start to run the local node

Now ganache is running with the fix for this issue, and you can run graph-node with your subgraph.
NOTE: you can use this project for testing

tsuberim · 2018-10-25T16:31:16Z

@nenadjaja Is it nessesry to clone both repos, since you alreay npm installed the remote repo on ganache-cli ? Also should't we take the develop branch instead of master?

nenadjaja · 2018-10-29T15:29:46Z

Hi @tsuberim sorry for not responding earlier, I didn't see your comment. Yeah good point, you don't need to clone both repos, just ganache-cli. I originally cloned both, cause I pointed to my local repo for ganache-core, but that didn't work (I was running into npm install issues).
And yes, I am using the develop branch :)

nenadjaja · 2018-11-19T21:58:32Z

Tested with the following:

new Ganache CLI release: ganache-cli 6.2.1 that's using ganache-core version 2.3.1
Our hackathon-scaffold project with "@graphprotocol/graph-cli": "^0.4.1" and "@graphprotocol/graph-ts": "^0.4.1"

This issue is fixed and can't be reproduced anymore.

nenadjaja added the bug Something isn't working label Sep 17, 2018

Jannis mentioned this issue Sep 17, 2018

Identify and fix null block errors received from Infura #360

Closed

Jannis added the to do label Sep 17, 2018

Jannis assigned fordN Sep 17, 2018

timmclean mentioned this issue Sep 17, 2018

Show helpful error when node is run on a different network but with same network name #377

Closed

This comment has been minimized.

Sign in to view

fordN added in progress and removed to do labels Sep 18, 2018

Jannis assigned Jannis and unassigned fordN Sep 24, 2018

Jannis added to do and removed in progress labels Sep 24, 2018

timmclean mentioned this issue Sep 28, 2018

Add with_retry component, use for Eth RPC #424

Merged

Jannis changed the title ~~Not able to get the blocks~~ Problems getting blocks from Ganache Oct 1, 2018

Jannis added the chains/ethereum label Oct 1, 2018

Jannis modified the milestones: ETHBerlin, ETHSanFrancisco Oct 1, 2018

Jannis added the p3 label Oct 1, 2018

fordN added in progress and removed to do labels Oct 4, 2018

Jannis assigned nenadjaja and unassigned Jannis Oct 22, 2018

orenyodfat mentioned this issue Oct 24, 2018

Devops daostack/subgraph#4

Merged

nenadjaja added pending review and removed in progress labels Oct 25, 2018

timmclean mentioned this issue Nov 8, 2018

Omitting optional sendTransaction parameters in ganache/truffle causes block ingestor errors #561

Closed

Jannis added to do and removed pending review labels Nov 12, 2018

nenadjaja added in progress and removed to do labels Nov 19, 2018

nenadjaja added pending review and removed in progress labels Nov 19, 2018

Jannis closed this as completed Nov 26, 2018

Jannis removed the pending review label Nov 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problems getting blocks from Ganache #375

Problems getting blocks from Ganache #375

nenadjaja commented Sep 17, 2018 •

edited

Loading

Jannis commented Sep 17, 2018

Jannis commented Sep 17, 2018 •

edited

Loading

timmclean commented Sep 17, 2018 •

edited

Loading

Jannis commented Sep 17, 2018 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

Jannis commented Sep 17, 2018

leoyvens commented Sep 17, 2018

timmclean commented Sep 17, 2018 •

edited by Jannis

Loading

timmclean commented Oct 19, 2018

Jannis commented Oct 22, 2018

nenadjaja commented Oct 24, 2018 •

edited

Loading

nenadjaja commented Oct 24, 2018 •

edited

Loading

tsuberim commented Oct 25, 2018 •

edited

Loading

nenadjaja commented Oct 29, 2018 •

edited

Loading

nenadjaja commented Nov 19, 2018 •

edited

Loading

Problems getting blocks from Ganache #375

Problems getting blocks from Ganache #375

Comments

nenadjaja commented Sep 17, 2018 • edited Loading

Jannis commented Sep 17, 2018

Jannis commented Sep 17, 2018 • edited Loading

Key not found in database

Error message

Cause

Why is the error thrown?

timmclean commented Sep 17, 2018 • edited Loading

Jannis commented Sep 17, 2018 • edited Loading

Invalid length 1, expected a 0x-prefixed hex string with length 40

Error message

Cause

How to reproduce?

This comment has been minimized.

This comment has been minimized.

Jannis commented Sep 17, 2018

leoyvens commented Sep 17, 2018

timmclean commented Sep 17, 2018 • edited by Jannis Loading

timmclean commented Oct 19, 2018

Jannis commented Oct 22, 2018

nenadjaja commented Oct 24, 2018 • edited Loading

nenadjaja commented Oct 24, 2018 • edited Loading

tsuberim commented Oct 25, 2018 • edited Loading

nenadjaja commented Oct 29, 2018 • edited Loading

nenadjaja commented Nov 19, 2018 • edited Loading

nenadjaja commented Sep 17, 2018 •

edited

Loading

Jannis commented Sep 17, 2018 •

edited

Loading

timmclean commented Sep 17, 2018 •

edited

Loading

Jannis commented Sep 17, 2018 •

edited

Loading

timmclean commented Sep 17, 2018 •

edited by Jannis

Loading

nenadjaja commented Oct 24, 2018 •

edited

Loading

nenadjaja commented Oct 24, 2018 •

edited

Loading

tsuberim commented Oct 25, 2018 •

edited

Loading

nenadjaja commented Oct 29, 2018 •

edited

Loading

nenadjaja commented Nov 19, 2018 •

edited

Loading