Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spw/refactor br #3469

Merged
merged 3 commits into from
Dec 23, 2021
Merged

Spw/refactor br #3469

merged 3 commits into from
Dec 23, 2021

Conversation

pengweisong
Copy link
Contributor

@pengweisong pengweisong commented Dec 14, 2021

What type of PR is this?

  • bug
  • feature
  • enhancement

What does this PR do?

The original br tools require password-free ssh to each machine which add extra work for users.
This pr's main goal is to remove ssh dependency. To do this, we introduce an agent in each machine.
Then br can handle machines' data through agent's service.
Agent communicate with the meta service through heartbeat. By heartbeat, agent register itself
to meta service and pull the services it should supervise in its host.

The agent: vesoft-inc/nebula-agent#1
The br: vesoft-inc/nebula-br#22

This pr includes:

  1. refactor br related code, including renaming and adjust code structure.
  2. batch the snapshot rpc by spaces
  3. add agent heartbeat
  4. report data/root path in the storaged/graphd heartbeat

Which issue(s)/PR(s) this PR relates to?

The agent: vesoft-inc/nebula-agent#1
The br: vesoft-inc/nebula-br#22

Special notes for your reviewer, ex. impact of this fix, etc:

Additional context:

Checklist:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatible (If it is incompatible, please describe it and add corresponding label.)
  • Need to cherry-pick (If need to cherry-pick to some branches, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to reflect in release notes and how to describe:

                                                            `

@CLAassistant
Copy link

CLAassistant commented Dec 14, 2021

CLA assistant check
All committers have signed the CLA.

@pengweisong pengweisong added the ready-for-testing PR: ready for the CI test label Dec 15, 2021
@codecov-commenter
Copy link

codecov-commenter commented Dec 16, 2021

Codecov Report

Merging #3469 (7599967) into master (9a0fbb6) will decrease coverage by 0.20%.
The diff coverage is 81.67%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3469      +/-   ##
==========================================
- Coverage   85.40%   85.20%   -0.21%     
==========================================
  Files        1289     1303      +14     
  Lines      119798   121492    +1694     
==========================================
+ Hits       102317   103515    +1198     
- Misses      17481    17977     +496     
Impacted Files Coverage Δ
src/common/graph/Response.h 45.66% <ø> (ø)
src/common/time/TimezoneInfo.cpp 50.00% <ø> (+4.54%) ⬆️
src/graph/executor/admin/AddHostsExecutor.cpp 0.00% <0.00%> (ø)
src/graph/executor/admin/AddHostsExecutor.h 0.00% <0.00%> (ø)
src/graph/executor/admin/DropHostsExecutor.cpp 0.00% <0.00%> (ø)
src/graph/executor/admin/DropHostsExecutor.h 0.00% <0.00%> (ø)
src/graph/executor/admin/ShowHostsExecutor.h 100.00% <ø> (ø)
src/graph/planner/plan/Admin.cpp 52.45% <0.00%> (-3.69%) ⬇️
src/graph/planner/plan/PlanNode.h 81.25% <ø> (ø)
src/graph/service/PermissionCheck.cpp 78.72% <ø> (-2.13%) ⬇️
... and 192 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b7560b...7599967. Read the comment docs.

@pengweisong pengweisong marked this pull request as ready for review December 16, 2021 08:58
@pengweisong pengweisong requested a review from a team December 16, 2021 08:59
Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job~ Generally LGTM

src/interface/meta.thrift Show resolved Hide resolved
src/meta/ActiveHostsMan.cpp Outdated Show resolved Hide resolved
src/meta/processors/admin/ListClusterInfoProcessor.cpp Outdated Show resolved Hide resolved
src/meta/processors/admin/ListClusterInfoProcessor.cpp Outdated Show resolved Hide resolved
critical27
critical27 previously approved these changes Dec 22, 2021
Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job~

Nivras
Nivras previously approved these changes Dec 22, 2021
Copy link
Contributor

@Nivras Nivras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job! LGTM

stash

add datapath/rootpath in heartbeat && modify list cluster processor

store dir info by address

group snapshot rpc call in one host

fix snapshot parameter bug

fix canocial path bug

decrease lastUpdateTime not found info log level

stats init && options copy

rename ndoe to service

rename create backup fields

first runnable version

rename include system spaces

add balance thrift

resolve

resolve conflicts

fix test

modify tests

remove unrelated lines

remove unralted files

fix create backup test

add some comments & fix list cluster test

only report dir info once

improve code

add agent hb processor test && fix comments

fix copy right and log

add copyright

tidy the create checkpoint rpc

stash

add datapath/rootpath in heartbeat && modify list cluster processor

store dir info by address

group snapshot rpc call in one host

fix snapshot parameter bug

fix canocial path bug

decrease lastUpdateTime not found info log level

stats init && options copy

rename ndoe to service

rename create backup fields

first runnable version

rename include system spaces

add balance thrift

resolve

resolve conflicts

fix test

modify tests

remove unrelated lines

remove unralted files

fix create backup test

add some comments & fix list cluster test

only report dir info once

improve code

add agent hb processor test && fix comments

fix copy right and log

add copyright
@critical27 critical27 merged commit f670c68 into vesoft-inc:master Dec 23, 2021
@Sophie-Xie Sophie-Xie added the doc affected PR: improvements or additions to documentation label Jan 4, 2022
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Mar 21, 2022
stash

add datapath/rootpath in heartbeat && modify list cluster processor

store dir info by address

group snapshot rpc call in one host

fix snapshot parameter bug

fix canocial path bug

decrease lastUpdateTime not found info log level

stats init && options copy

rename ndoe to service

rename create backup fields

first runnable version

rename include system spaces

add balance thrift

resolve

resolve conflicts

fix test

modify tests

remove unrelated lines

remove unralted files

fix create backup test

add some comments & fix list cluster test

only report dir info once

improve code

add agent hb processor test && fix comments

fix copy right and log

add copyright

tidy the create checkpoint rpc

stash

add datapath/rootpath in heartbeat && modify list cluster processor

store dir info by address

group snapshot rpc call in one host

fix snapshot parameter bug

fix canocial path bug

decrease lastUpdateTime not found info log level

stats init && options copy

rename ndoe to service

rename create backup fields

first runnable version

rename include system spaces

add balance thrift

resolve

resolve conflicts

fix test

modify tests

remove unrelated lines

remove unralted files

fix create backup test

add some comments & fix list cluster test

only report dir info once

improve code

add agent hb processor test && fix comments

fix copy right and log

add copyright

Co-authored-by: Sophie <[email protected]>

Co-authored-by: pengwei.song <[email protected]>
Co-authored-by: Sophie <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc affected PR: improvements or additions to documentation ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants