Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p2p/discover: implement v5.1 wire protocol #21647

Merged
merged 70 commits into from
Oct 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
b194b04
p2p/discover: implement new packet format proposal
fjl May 20, 2020
e54f43c
p2p/discover: reset input slice for each decoder benchmark round
fjl May 26, 2020
d1a736b
p2p/discover: add header masking code for testing
fjl May 26, 2020
6f88e75
p2p/discover: replace checksum with static protocol identifier
fjl Aug 18, 2020
23a1bb1
p2p/discover: implement talk requests
fjl Aug 18, 2020
731fc13
p2p/discover: implement multi-distance FINDNODE
fjl Aug 19, 2020
a482a91
p2p/discover: improve collectTableNodes
fjl Aug 19, 2020
f608c23
p2p/discover: fix result node count in collectTableNodes
fjl Aug 27, 2020
b6a9d1d
p2p/discover: add one more test
fjl Aug 28, 2020
f276753
p2p/discover: add multi-distance findnode handler test
fjl Sep 2, 2020
665dfdb
p2p/discover: move header masking constant to the top and add comments
fjl Sep 2, 2020
a4454b4
p2p/discover: add hexFile helper
fjl Sep 17, 2020
eccc0e6
p2p/discover: add packetNonce type, implement random+counter nonce
fjl Sep 17, 2020
691c3c7
p2p/discover: add test vector tests
fjl Sep 17, 2020
e12144b
p2p/discover: make test vectors deterministic
fjl Sep 17, 2020
0237023
p2p/discover: add ephemeral-pubkey to test vector output
fjl Sep 17, 2020
e4d93f2
p2p/discover: move v5 encoding to new sub-package v5wire
fjl Sep 22, 2020
38091a9
p2p/discover/v5wire: regenerate handshake test vectors
fjl Sep 22, 2020
d5d2f36
p2p/discover/v5wire: remove unused code
fjl Sep 22, 2020
4f1a20c
p2p/discover: remove unused method in test
fjl Sep 22, 2020
f88786f
p2p/discover: fix some issues in lookup tests
fjl Sep 22, 2020
90443f3
cmd/devp2p: add basic interactive discv5 test suite
fjl Sep 22, 2020
0e120a6
cmd/devp2p: improve usage of discv5 test
fjl Sep 22, 2020
f337817
p2p/discover/v5wire: check flag values better
fjl Sep 22, 2020
e166b4c
p2p/discover/v5wire: move low-level crypto to its own file
fjl Sep 22, 2020
ef46177
p2p/discover/v5wire: avoid ENR pubkey decompression for signature ver…
fjl Sep 22, 2020
c82c795
p2p/discover/v5wire: implement id-sig-input change
fjl Sep 28, 2020
c812ec3
p2p/discover/v5wire: implement version in protocol-id
fjl Sep 29, 2020
13bcca6
p2p/discover/v5wire: use WHOAREYOU IV in ID proof
fjl Sep 30, 2020
93ea479
p2p/discover: authTag -> nonce
fjl Sep 30, 2020
bf8c20d
p2p/discover/v5wire: avoid logging challenge encoding in test vector …
fjl Sep 30, 2020
3897ca5
p2p/discover/v5wire: feed IV into KDF as well
fjl Sep 30, 2020
15e295e
p2p/discover/v5wire: update id signature text
fjl Sep 30, 2020
1fdee23
p2p/discover/v5wire: clean up packet version check
fjl Sep 30, 2020
07e5435
p2p/discover/v5wire: check minimum packet size
fjl Sep 30, 2020
cafd22d
p2p/discover/v5wire: move src-id into authdata
fjl Sep 30, 2020
f288f6c
p2p/discover/v5wire: lower min message size
fjl Sep 30, 2020
b84d7d3
p2p/discover/v5wire: add low-level crypto test vector code
fjl Sep 30, 2020
4eabc4a
p2p/discover: implement simplified challenge data handling
fjl Oct 2, 2020
fbb61f1
p2p/discover/v5wire: add note about concurrency
fjl Oct 5, 2020
7a1bcf4
p2p/discover/v5wire: avoid copying header data when encoding
fjl Oct 5, 2020
80e6ee4
p2p/discover/v5wire: add reusable message ciphertext buffer
fjl Oct 5, 2020
545cf22
p2p/discover/v5wire: remove some TODOs
fjl Oct 5, 2020
2c7ae84
p2p/discover: clean up challenge nonce handling
fjl Oct 5, 2020
354f19c
p2p/discover/v5wire: remove unused argument
fjl Oct 5, 2020
1e3a650
cmd/devp2p: use listening address in local ENR
fjl Oct 5, 2020
282d1a7
cmd/devp2p/internal/v5test: rename testenv -> conn
fjl Oct 6, 2020
7c099cd
internal/utesting: print live output
fjl Oct 7, 2020
15d1a03
cmd/devp2p/internal/v5test: implement FINDNODE test
fjl Oct 7, 2020
104dd5c
cmd/devp2p/internal/v5test: fix shutdown
fjl Oct 7, 2020
91d22af
cmd/devp2p/internal/v5test: fix various things
fjl Oct 7, 2020
08e27fe
cmd/devp2p/internal/v5test: fix handshake issue
fjl Oct 7, 2020
30510f3
p2p/netutil: add IsTimeout
fjl Oct 7, 2020
32f0cac
cmd/devp2p/internal/v5test: add TestPingLargeRequestID
fjl Oct 7, 2020
922ffa8
cmd/devp2p/internal/v5test: use conn.findnode in TestFindnodeZeroDist…
fjl Oct 7, 2020
2f6d273
p2p/discover/v5wire: add check for oversized request ID
fjl Oct 7, 2020
cfdd78e
p2p/discover/v5wire: export request ID error
fjl Oct 7, 2020
9e2214a
cmd/devp2p/internal/v5test: fix new test
fjl Oct 7, 2020
80571fc
cmd/devp2p/internal/v5test: add multi IP test
fjl Oct 7, 2020
456cb2a
cmd/devp2p/internal/v5test: fix multi-IP test
fjl Oct 7, 2020
4c97a48
cmd/devp2p/internal/v5test: fix multi-IP test
fjl Oct 7, 2020
1f40ffc
cmd/devp2p/internal/v5test: add test for interrupted handshake
fjl Oct 7, 2020
9f97dd0
p2p/discover/v5wire: decode ephkey after signature check
fjl Oct 8, 2020
05c03ab
p2p/discover/v5wire: remove sha256reset
fjl Oct 8, 2020
409d121
p2p/discover/v5wire: reduce allocations in handshake decoder
fjl Oct 8, 2020
1eb2e24
cmd/devp2p/internal/v5test: improve listener handling
fjl Oct 8, 2020
b7d8c57
p2p/discover/v5wire: fix style thing
fjl Oct 8, 2020
59b11fa
p2p/discover/v5wire: add bad handshake test and fix
fjl Oct 14, 2020
a44e2f9
p2p/discover/v5wire: remove remaining 'v5' in identifiers
fjl Oct 14, 2020
fd5507d
cmd/devp2p/internal/v5test: fix lint issue
fjl Oct 14, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions cmd/devp2p/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# The devp2p command

The devp2p command line tool is a utility for low-level peer-to-peer debugging and
protocol development purposes. It can do many things.

### ENR Decoding

Use `devp2p enrdump <base64>` to verify and display an Ethereum Node Record.

### Node Key Management

The `devp2p key ...` command family deals with node key files.

Run `devp2p key generate mynode.key` to create a new node key in the `mynode.key` file.

Run `devp2p key to-enode mynode.key -ip 127.0.0.1 -tcp 30303` to create an enode:// URL
corresponding to the given node key and address information.

### Maintaining DNS Discovery Node Lists

The devp2p command can create and publish DNS discovery node lists.

Run `devp2p dns sign <directory>` to update the signature of a DNS discovery tree.

Run `devp2p dns sync <enrtree-URL>` to download a complete DNS discovery tree.

Run `devp2p dns to-cloudflare <directory>` to publish a tree to CloudFlare DNS.

Run `devp2p dns to-route53 <directory>` to publish a tree to Amazon Route53.

You can find more information about these commands in the [DNS Discovery Setup Guide][dns-tutorial].

### Discovery v4 Utilities

The `devp2p discv4 ...` command family deals with the [Node Discovery v4][discv4]
protocol.

Run `devp2p discv4 ping <enode/ENR>` to ping a node.

Run `devp2p discv4 resolve <enode/ENR>` to find the most recent node record of a node in
the DHT.

Run `devp2p discv4 crawl <nodes.json path>` to create or update a JSON node set.

### Discovery v5 Utilities

The `devp2p discv5 ...` command family deals with the [Node Discovery v5][discv5]
protocol. This protocol is currently under active development.

Run `devp2p discv5 ping <ENR>` to ping a node.

Run `devp2p discv5 resolve <ENR>` to find the most recent node record of a node in
the discv5 DHT.

Run `devp2p discv5 listen` to run a Discovery v5 node.

Run `devp2p discv5 crawl <nodes.json path>` to create or update a JSON node set containing
discv5 nodes.

### Discovery Test Suites

The devp2p command also contains interactive test suites for Discovery v4 and Discovery
v5.

To run these tests against your implementation, you need to set up a networking
environment where two separate UDP listening addresses are available on the same machine.
The two listening addresses must also be routed such that they are able to reach the node
you want to test.

For example, if you want to run the test on your local host, and the node under test is
also on the local host, you need to assign two IP addresses (or a larger range) to your
loopback interface. On macOS, this can be done by executing the following command:

sudo ifconfig lo0 add 127.0.0.2

You can now run either test suite as follows: Start the node under test first, ensuring
that it won't talk to the Internet (i.e. disable bootstrapping). An easy way to prevent
unintended connections to the global DHT is listening on `127.0.0.1`.

Now get the ENR of your node and store it in the `NODE` environment variable.

Start the test by running `devp2p discv5 test -listen1 127.0.0.1 -listen2 127.0.0.2 $NODE`.

[dns-tutorial]: https://geth.ethereum.org/docs/developers/dns-discovery-setup
[discv4]: https://github.com/ethereum/devp2p/tree/master/discv4.md
[discv5]: https://github.com/ethereum/devp2p/tree/master/discv5/discv5.md
12 changes: 10 additions & 2 deletions cmd/devp2p/discv4cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -286,15 +286,23 @@ func listen(ln *enode.LocalNode, addr string) *net.UDPConn {
}
usocket := socket.(*net.UDPConn)
uaddr := socket.LocalAddr().(*net.UDPAddr)
ln.SetFallbackIP(net.IP{127, 0, 0, 1})
if uaddr.IP.IsUnspecified() {
ln.SetFallbackIP(net.IP{127, 0, 0, 1})
} else {
ln.SetFallbackIP(uaddr.IP)
}
ln.SetFallbackUDP(uaddr.Port)
return usocket
}

func parseBootnodes(ctx *cli.Context) ([]*enode.Node, error) {
s := params.RinkebyBootnodes
if ctx.IsSet(bootnodesFlag.Name) {
s = strings.Split(ctx.String(bootnodesFlag.Name), ",")
input := ctx.String(bootnodesFlag.Name)
if input == "" {
return nil, nil
}
s = strings.Split(input, ",")
}
nodes := make([]*enode.Node, len(s))
var err error
Expand Down
35 changes: 35 additions & 0 deletions cmd/devp2p/discv5cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,13 @@ package main

import (
"fmt"
"os"
"time"

"github.com/ethereum/go-ethereum/cmd/devp2p/internal/v5test"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/internal/utesting"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/p2p/discover"
"gopkg.in/urfave/cli.v1"
)
Expand All @@ -33,6 +37,7 @@ var (
discv5PingCommand,
discv5ResolveCommand,
discv5CrawlCommand,
discv5TestCommand,
discv5ListenCommand,
},
}
Expand All @@ -53,6 +58,12 @@ var (
Action: discv5Crawl,
Flags: []cli.Flag{bootnodesFlag, crawlTimeoutFlag},
}
discv5TestCommand = cli.Command{
Name: "test",
Usage: "Runs protocol tests against a node",
Action: discv5Test,
Flags: []cli.Flag{testPatternFlag, testListen1Flag, testListen2Flag},
}
discv5ListenCommand = cli.Command{
Name: "listen",
Usage: "Runs a node",
Expand Down Expand Up @@ -103,6 +114,30 @@ func discv5Crawl(ctx *cli.Context) error {
return nil
}

func discv5Test(ctx *cli.Context) error {
// Disable logging unless explicitly enabled.
if !ctx.GlobalIsSet("verbosity") && !ctx.GlobalIsSet("vmodule") {
log.Root().SetHandler(log.DiscardHandler())
}

// Filter and run test cases.
suite := &v5test.Suite{
Dest: getNodeArg(ctx),
Listen1: ctx.String(testListen1Flag.Name),
Listen2: ctx.String(testListen2Flag.Name),
}
tests := suite.AllTests()
if ctx.IsSet(testPatternFlag.Name) {
tests = utesting.MatchTests(tests, ctx.String(testPatternFlag.Name))
}
results := utesting.RunTests(tests, os.Stdout)
if fails := utesting.CountFailures(results); fails > 0 {
return fmt.Errorf("%v/%v tests passed.", len(tests)-fails, len(tests))
}
fmt.Printf("%v/%v passed\n", len(tests), len(tests))
return nil
}

func discv5Listen(ctx *cli.Context) error {
disc := startV5(ctx)
defer disc.Close()
Expand Down
Loading