Co-authored-by: Alexey Sharp <[email protected]>
Co-authored-by: Alex Sharp <[email protected]>

* Fix some cli flag descriptions

* add node about verbosity

* save

* save

* somewhat there but not yet

* lol

* more efficient ethstats

* lint

* not die on no wifi

Observer crawls the Ethereum network and collects information about the nodes.

* save

* save

* [erigon2] Fuzz tests for commitment

* Cleanup

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <[email protected]>

* Introduce unlimited download rate

* More generous burst

Co-authored-by: Alexey Sharp <[email protected]>

* Sentry GRPC: rename Peers to PeerEvents

see erigontech/interfaces#101

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <[email protected]>

* got rid of the automatic usage of net api

* less confusing comment

* ops

* ops2

* important

* ops

* RPC: admin.peers()

This RPC method returns information about the connected remote nodes.
https://geth.ethereum.org/docs/rpc/ns-admin#admin_peers

The peers are collected from all configured sentries.
See: erigontech/interfaces#102

Test with:
curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc": "2.0", "method": "admin_peers", "params": [], "id":1}' localhost:8545

* save

* liner fix

Co-authored-by: alex.sharov <[email protected]>

* Experiment files 1 by 1

* Remove check

* sort preverified snapshots

* docs: docker permissions

* sort preverified snapshots

* sort preverified snapshots

* sort preverified snapshots

* sort preverified snapshots

* sort preverified snapshots

* sort preverified snapshots

* save

* Fix speed log, remove file name

* Move timer out of the loop

* Calculate total size of downloaded files

* Fixes

* Fix

* Fix

* Fix

* Move downloadData

* Fix

* Revert "Fix"

This reverts commit 038e02b.

* Revert "Move downloadData"

This reverts commit 8130a4d.

* Revert "Fix"

This reverts commit 1dca25b.

* Revert "Fix"

This reverts commit ee5a1e8.

* Revert "Fix"

This reverts commit 8af7be7.

* Revert "Fixes"

This reverts commit 50509af.

* Revert "Calculate total size of downloaded files"

This reverts commit 64a26df.

* Remove progress

* Remove progress

Co-authored-by: Alexey Sharp <[email protected]>
Co-authored-by: alex.sharov <[email protected]>

* Snapshots: open torrent udp in docker-compose.yml

* Snapshots: open torrent udp in docker-compose.yml

…3971)

* save

* save

* save

This adds support for a new struct tag "optional". Using this tag, structs used
for RLP encoding/decoding can be extended in a backwards-compatible way,
by adding new fields at the end.

see geth commit ethereum/go-ethereum@700df14

Co-authored-by: Felix Lange <[email protected]>

* Forgot to check err status

* Invalid header shouldn't fail the entire stage

* Potential fix for verification

* multi verify

Co-authored-by: Alexey Sharp <[email protected]>
Co-authored-by: Alex Sharp <[email protected]>

This changes the definitions of Ping and Pong, adding an optional field
for the sequence number. This field was previously encoded/decoded using
the "tail" struct tag, but using "optional" is much nicer.

see ethereum/go-ethereum#22842

Co-authored-by: Felix Lange <[email protected]>

Co-authored-by: Alex Sharp <[email protected]>

* save

* save

* tolerate some fails

* tolerate some fails

Co-authored-by: Alexey Sharp <[email protected]>

…#3997)

* Place finishHandlingForkChoice after startHandlingForkChoice

* forkChoiceMessage -> forkChoice

* Check that safe & finalized blocks are canonical for no-op forkChoice

* Re-introduced cleanup of temporary table

* Fix sign

* Fix lint

* Fix lint

* Revert

Co-authored-by: Alex Sharp <[email protected]>

…downloads inside (#3986)

* save

* save

* save

* save

* save

* save

* save

* save

* save

The test was slow, because it was trying to find
predefined nodeIDs (lookupTestnet) by generating random keys
and trying to find their neighbours
until it hits all nodes of the lookupTestnet.
In addition each FindNode response was waited for 0.5 sec (respTimeout).
This could take up to 30 sec and fail the test suite.

A fake random key generator is now used during the test.
It issues the expected keys, and the lookup converges quickly.
The reply timeout is reduced for the test.
Now it normally takes less than.1 sec.

* use semaphore instead of a chan struct{}
* move MaxPendingPeers default value to DefaultConfig.P2P
* log Error if Accept fails
* replace quit channel with context

* save

* save

* save

* save

* save

* save

* save

* save

* reduce downloader deps

* reduce downloader deps

* reduce downloader deps

* reduce downloader deps

* save

* reduce downloader deps

* Update reset_state.go

* Update reset_state.go

* save

* save

* save

* save

* save

Problem: (nil, nil) from CountPingErrors was not handled.
This happens if the node is not in the db (a bootstrap node),
and was never crawled before.

* cmd/utils: initialize f.Value before setting variable

* override.terminaltotaldifficulty flag

* Add OverrideTerminalTotalDifficulty to default_flags

The test was flaky, because of the "endpoint prediction".
The test starts 5 nodes one by one.
Node 0 is used as a bootstrap node for nodes 1-4.
When it is about to add, say, node 3, nodes 0 and 1 might already have had a chance to communicate,
and updateEndpoints() deletes the node 0 UDP port, because fallbackUDP port was not configured.

In this case node 3 would get a bootstrap node 0 without a port and lead to an error:

    v5_udp_test.go:110: bad bootstrap node "enr:...": missing UDP port

The problem was reproducible by this command:

    go test ./p2p/discover -run TestUDPv5_lookupE2E -count 500

* exchange RLPx Hello even when maxpeers limit is reached
* bump MaxPendingPeers to increase the default handshake queue
  (and the likelyhood of Hello exchange)

…ts are used (#3998)

* Update stageloop.go

* Print

* Consider snapshot headers as parlia checkpoints

* Not fail after not loading snapshot

* Lazy snapshots

* Print number of validators

* More printing

* Use epoch instead of checkpoint interval

* Reduce logging

* Fix compilation

* Remove trace jump dest

* Fix lint

* Not store snapshots every epoch

* Separate snapshot for verification and finalisation

Co-authored-by: Alex Sharp <[email protected]>
Co-authored-by: Alexey Sharp <[email protected]>

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

Co-authored-by: Alexey Sharp <[email protected]>

…4043

* print branchHash

* Print state changes

* Print val

* Fix for author

* Remove prints

Co-authored-by: Alexey Sharp <[email protected]>

Co-authored-by: Alexey Sharp <[email protected]>

* speed up docker image build by use layer cache

* rearrenge Dockerfile

* enable docker layer cache in github action

`cfg` is not a pointer

* refactor test
* add a fast fixed examples test for the main suite
* split slow test for the integration suite

* More relax inclusion of headers in the downloader

* Fix

Co-authored-by: Alexey Sharp <[email protected]>

This reverts commit e758fb8.

* Update node.go

* Update node.go