Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[p2p] Improve p2p #3409

Merged
merged 3 commits into from
Jul 6, 2022
Merged

[p2p] Improve p2p #3409

merged 3 commits into from
Jul 6, 2022

Conversation

Liuhaai
Copy link
Member

@Liuhaai Liuhaai commented May 24, 2022

Description

  • Call adjustment due to the introduction of peerManager in go-pkg package

  • Neighbors in agent class is replaced with ConnectedPeers which is introduced in peerManager

  • MaxPeers parameter is added into p2p config

  • RequestBlocks handler in blocksync is replaced with independent handlers Neighbors, UniCastOutbound, and BlockPeer

fix #3234 #3412 #3448

Type of change

Please delete options that are not relevant.

  • [] Bug fix (non-breaking change which fixes an issue)
  • [] New feature (non-breaking change which adds functionality)
  • [] Code refactor or improvement
  • Breaking change (fix or feature that would cause a new or changed behavior of existing functionality)
  • [] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • [] make test
  • [] fullsync
  • [] Other test (please specify)

Test Configuration:

  • Firmware version:
  • Hardware:
  • Toolchain:
  • SDK:

Checklist:

  • [] My code follows the style guidelines of this project
  • [] I have performed a self-review of my code
  • [] I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • [] My changes generate no new warnings
  • [] I have added tests that prove my fix is effective or that my feature works
  • [] New and existing unit tests pass locally with my changes
  • [] Any dependent changes have been merged and published in downstream modules

@Liuhaai Liuhaai marked this pull request as ready for review May 24, 2022 06:43
@Liuhaai Liuhaai requested a review from a team as a code owner May 24, 2022 06:43
@codecov
Copy link

codecov bot commented May 24, 2022

Codecov Report

Merging #3409 (d987bfa) into master (4e397d3) will decrease coverage by 0.09%.
The diff coverage is 51.47%.

@@            Coverage Diff             @@
##           master    #3409      +/-   ##
==========================================
- Coverage   75.41%   75.32%   -0.10%     
==========================================
  Files         245      245              
  Lines       22699    22715      +16     
==========================================
- Hits        17119    17109      -10     
- Misses       4655     4682      +27     
+ Partials      925      924       -1     
Impacted Files Coverage Δ
blocksync/blocksync.go 72.90% <39.28%> (-8.72%) ⬇️
p2p/agent.go 73.60% <57.14%> (-4.26%) ⬇️
server/itx/heartbeat.go 78.21% <75.00%> (-0.22%) ⬇️
blocksync/buffer.go 86.76% <100.00%> (ø)
api/http.go 88.57% <0.00%> (+5.71%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e397d3...d987bfa. Read the comment docs.

p2p/agent.go Outdated
}
p.host.FindPeers(context.Background())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is also a context.Background() at line 589, use one ctx

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not an issue. the func is directly returned in L590

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the code can work fine. But programming-wise, feels better to define a common ctx = context.Background() ahead, and use it at 2 places, rather than declare a new context wherever needed.

@@ -42,6 +44,7 @@ func TestDummyAgent(t *testing.T) {
}

func TestBroadcast(t *testing.T) {
t.Skip()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why skip? test would fail?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -494,93 +503,13 @@ func newTestConfig() (config.Config, error) {
return cfg, nil
}

func TestBlockSyncerPeerBlockList(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this is removed?

r.NoError(bootnode.Start(ctx))

addrs, err := bootnode.Self()
bootnode, err := p2p.NewHost(context.Background(), p2p.DHTProtocolID(2), p2p.Port(testutil.RandomPort()), p2p.SecureIO(), p2p.MasterKey("bootnode"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change to this? can we still use NewAgent()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bootnode shouldn't JoinOverlay()

for i := 0; i < n; i++ {
port := bootnodePort + i + 1
port := testutil.RandomPort()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+ i + 1 would make sure a diff port for each node, we don't need to change this, unless there is a strong reason?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right.

@dustinxie dustinxie linked an issue May 26, 2022 that may be closed by this pull request
@Liuhaai Liuhaai force-pushed the improveP2p branch 2 times, most recently from 9d686ab to 700d1a0 Compare June 23, 2022 19:15
@dustinxie dustinxie linked an issue Jun 24, 2022 that may be closed by this pull request
@Liuhaai Liuhaai removed the pending label Jun 27, 2022
@dustinxie dustinxie linked an issue Jun 28, 2022 that may be closed by this pull request
Copy link
Collaborator

@CoderZhi CoderZhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

summarize changes and fixes in description, link to related issue or task.

blocksync/blocksync.go Outdated Show resolved Hide resolved
blocksync/blocksync.go Outdated Show resolved Hide resolved
@@ -71,7 +71,7 @@ func (b *blockBuffer) AddBlock(tipHeight uint64, blk *peerBlock) (bool, uint64)
defer b.mu.Unlock()
blkHeight := blk.block.Height()
if blkHeight <= tipHeight {
return false, 0
return false, blkHeight
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no impact to current logic

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep. The change is helpful to log the change of target height by the block of higher height.

blocksync/buffer.go Outdated Show resolved Hide resolved
p2p/agent.go Outdated Show resolved Hide resolved
p2p/agent.go Outdated Show resolved Hide resolved
p2p/agent.go Outdated Show resolved Hide resolved
p2p/agent.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@CoderZhi CoderZhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add description for future reference, and link to related issues

go.mod Outdated
@@ -15,7 +15,7 @@ require (
github.com/grpc-ecosystem/go-grpc-middleware v1.2.0
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/iotexproject/go-fsm v1.0.0
github.com/iotexproject/go-p2p v0.3.4-0.20220624220702-2aa2dd39e7a6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge p2p PR31, make a release, use new version here

@Liuhaai Liuhaai merged commit 2a70177 into iotexproject:master Jul 6, 2022
Liuhaai added a commit that referenced this pull request Jul 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

p2p improvement (using DHT for peer finding) add MaxPeerNumber() option to p2p Fix peerBlockList logic
3 participants