Client JSON-RPC Support (old)

[holgerd77]

Description

Implementation of the JSON-RPC interface as described in https://github.com/ethereum/wiki/wiki/JSON-RPC.

This encompasses a server implementation and the implementation of the various API calls.

Possible Implementation

It probably best to start with an early-on implementation of 2-3 exemplary API calls and then refine this for some time to work out a good internal structure to build upon and extend with the further calls.

There are various Javascript libraries out there touching the topic, not sure if any can be of help here or be integrated/used, this should be examined more closely.

UI

The RPC server should be constantly running and be available via some configured address and port.

Implement all JSON-RPC Endpoints

Update (2018-07-10): This has been adressed in ethereumjs/ethereumjs-client#30 by @yurenju, providing a very solid structure for implementation and testing. It is now possible to implement the single RPC endpoints (limited by the functionality already implemented in the core client code).

Checklist of endpoints to support, and which are supported in master or pull requests (taken from Py-EVM:

• Support all the web_* methods
  • web3_clientVersion
  • web3_sha3

---

• support all the net_* methods
  • net_listening
  • net_peerCount
  • net_version Benchmark ci fixes #840

---

• support all the eth_* write methods
  • make list of methods

---

• support all the eth_* read-only methods
  • eth_protocolVersion
  • eth_syncing
  • eth_coinbase
  • eth_mining
  • eth_hashrate
  • eth_gasPrice
  • eth_accounts
  • eth_blockNumber
  • eth_getBalance
  • eth_getStorageAt
  • eth_getTransactionCount
  • eth_getBlockTransactionCountByHash
  • eth_getBlockTransactionCountByNumber
  • eth_getUncleCountByBlockHash
  • eth_getUncleCountByBlockNumber block int
  • eth_getUncleCountByBlockNumber block string
  • eth_getCode
  • eth_call
  • eth_estimateGas
  • eth_getBlockByHash
  • eth_getBlockByNumber block int
  • eth_getBlockByNumber block string
  • eth_getTransactionByHash
  • eth_getTransactionByBlockHashAndIndex
  • eth_getTransactionByBlockNumberAndIndex
  • eth_getTransactionReceipt
  • eth_getUncleByBlockHashAndIndex
  • eth_getUncleByBlockNumberAndIndex block int
  • eth_getUncleByBlockNumberAndIndex block string
  • eth_getCompilers
  • eth_getLogs
  • eth_getWork

Note: On a first round I've just copied over all calls, this might be limited by functionality which eventually shall not be implemented in the client (mining related stuff e.g.).

Reference Implementations / Links

• Py-EVM trinity RPC implementation

[kikoncuo]

How can we approach this?

I'm lost, here is my general impression:
Create an express web server to accept the defined petitions
Execute the transactions with ethereumjs-vm
Use ethereumjs-accounts to manage accounts
Read .ldb files to get info about blocks

[yurenju]

some clue from ganache-core
https://github.com/trufflesuite/ganache-core/blob/develop/lib/server.js

it should be a good start to use node.js built-in web server since JSON-RPC only has a single endpoint. express is pretty powerful with route path config and static file host, but JSON-RPC doesn't need these features.

maybe we should go with built-in HTTP server?

[yurenju]

before go deep into this issue I want to have a rough WIP commit to see if we are on the same page, here we go:

yurenju/ethereumjs-client@6d917a5

This is WIP commit only to verify the idea. This commit refers py-evm implementation to have a this.modules to lookup what module we should handle an RPC request. With this commit you can use --rpc to enable RPC server:

$ node lib/index.js --rpc

and use curl to try this API:

curl -X POST --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x1b4", true],"id":1}' http://127.0.0.1:8545

then you should get not implement yet on the console.

@holgerd77 any thought? Is it too rough? Let me how should I contribute on this issue :D

[holgerd77]

@yurenju cool that you are taking on this! From a first look I would say that this is going into the right direction. I am in Croatia on holidays at the moment, so can't look into this deeper, back on the weekend and will have a closer look next week.

If you want you can already submit this as a PR and we can continue from there or you can await a deeper review, as you like.

[yurenju]

Cool, if this is the right direction I would like to finish one API call like eth_getBlockByNumber with test and send a PR, thank you!

btw I also went Croatia last year, pretty cool country especially Rovinj & Dubrovnik. enjoy your vacation and no rush to reply here!!

[yurenju]

PR ethereumjs/ethereumjs-client#30 is sent

[bit-warrior]

@yurenju Thanks for putting together a clean and solid foundation for RPC. Should i implement eth_protocolVersion , or you guys are already working on it.Let me know which ones i can pickup to implement.

[yurenju]

@bit-warrior, I'm not working on it so go ahead for it, thank you :D

I will take a few weeks to work on another side project so feel free to implement the rest of APIs!

[fgimenez]

@yurenju @bit-warrior I'd like to work on these too! :) @bit-warrior can we coordinate, are you working on anything else other than protocolVersion?

[cdetrio]

As @yurenju noted, the RPC methods are already implemented javascript in truffle/ganache. So much of that code would be duplicated here (unless refactored out of truffle).

Also, it would be good to do this in a test-driven way (TDD). It will take some work but a start is it at http://cdetrio.github.io/eth-compat-table

[cdetrio]

I see some RPC testing scripts was added in ethereumjs/ethereumjs-client#30 . There is already a graveyard of umpteen unmaintained rpc test runners already, it would be good to standardize on one cross-client RPC test suite. See also ethereum/EIPs#217

[holgerd77]

@cdetrio Any suggestion on how to continue with this regarding the truffle/ganache overlap?

[holgerd77]

Maybe @benjamincburns (from Truffle/Ganache) wants to join the discussion here.

[yurenju]

@cdetrio http://cdetrio.github.io/eth-compat-table seems pretty cool, and we should leverage that. I notice it was forked from ethereum/interfaces, I'm not familar with EIP process, will it be merged back to upstream via PR?

[bit-warrior]

@fgimenez sorry i missed to reply you earlier. I have started with the easy ones :-) i m working on below right now
eth_getTransactionByBlockHashAndIndex
eth_getTransactionByBlockNumberAndIndex

After that i m looking to implement "eth_getTransactionByHash" but not sure how to get data.

[fgimenez]

@bit-warrior no worries :)

Nice, I'll take eth_getUncleBy*, hopefully I'll have something ready soon.

[benjamincburns]

Thanks for the shoutout @holgerd77 - is there anything I can do to help out?

As an aside, the metamask folks are working on a new iteration of their provider engine for implementing parts of the RPC. We haven't switched over to it in Ganache yet, but it looks nice. Might be worth a peek.

https://github.com/kumavis/json-rpc-engine
https://github.com/MetaMask/eth-json-rpc-middleware

cc @kumavis

[holgerd77]

@benjamincburns Thanks for letting us know, we will definitely have a closer look here.

[aunyks]

I've opened a PR for web3_* at ethereumjs/ethereumjs-client#65.

[aunyks]

net_* are complete.

[tomonari-t]

I guess web_* and eth_getBlockByHash are complete

[holgerd77]

@ryanio I've read in some internal chats - I think from someone from the geth team - that RPC methods would be valuable for debugging and comparing results when clients are joining the Yolo testnet. So if you are not sure what to work on next adding (some) RPC methods is always a good candidate I guess. Maybe you want to have a closer look what might be the more useful ones to have, not sure, haven't given this any thought yet (others can suggest as well of course).

[ryanio]

@holgerd77 sounds good, I really like the setup of the RPC modules code with the methods on the classes, it makes it natural to use in TypeScript. I'm seeing that it's currently callback based though, I can start with a PR updating it to async/promises then start adding some useful methods.

I started a tracking table in #1114 to visualize some more namespaces I found.

[holgerd77]

Ok, great! 😄

Will then close here in favor of #1114