Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store/gc_worker: Use UnsafeDestroyRange instead of DeleteRange in GC #7560

Merged
merged 16 commits into from
Sep 14, 2018

Conversation

MyonKeminta
Copy link
Contributor

@MyonKeminta MyonKeminta commented Aug 30, 2018

What problem does this PR solve?

This PR tries to solve the problem that the third step of GC, DeleteRanges, is too slow.

What is changed and how it works?

  • Add GetAllStores function to PdClient
  • Add PdClient as a member of GCWorker.
  • In gc_worker, the logic of DeleteRanges is changed. Previously it calls DeleteRanges on each region, now it calls DestroyRange on each TiKV.
  • Check delete_range_done table and pick ranges deleted 24 hours again, and call DestroyRange again

The idea is detailed recorded in tikv/rfcs#2

This PR should not be merged before tikv/tikv#3560

Check List

Tests

Code changes

  • Has interface methods change
    • Constructor of GCWorker needs a PdClient now.

Side effects

  • Increased code complexity
  • Breaking backward compatibility

@jackysp
Copy link
Member

jackysp commented Aug 31, 2018

It's better to separate the modification of vendor to another pr.

@MyonKeminta
Copy link
Contributor Author

Oh, yes. It may be separated out from this PR later.

@MyonKeminta MyonKeminta changed the title [WIP] store/gc_worker: Use DestroyRange instead of DeleteRange in GC [DNM] store/gc_worker: Use UnsafeDestroyRange instead of DeleteRange in GC Sep 6, 2018
zhangjinpeng87
zhangjinpeng87 previously approved these changes Sep 14, 2018
Copy link
Contributor

@zhangjinpeng87 zhangjinpeng87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

zhangjinpeng87
zhangjinpeng87 previously approved these changes Sep 14, 2018
Copy link
Contributor

@zhangjinpeng87 zhangjinpeng87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MyonKeminta
Copy link
Contributor Author

@tiancaiamao PTAL thanks!

@MyonKeminta MyonKeminta changed the title [DNM] store/gc_worker: Use UnsafeDestroyRange instead of DeleteRange in GC store/gc_worker: Use UnsafeDestroyRange instead of DeleteRange in GC Sep 14, 2018
@MyonKeminta
Copy link
Contributor Author

/run-integration-tests

metrics.GCWorkerCounter.WithLabelValues("re_delete_range").Inc()

// We check delete range records that are deleted about 24 hours ago.
safePointTime := oracle.PhysicalToTime(oracle.ExtractPhysical(safePoint))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think calculation on the TS directly is simpler, may need to extract the constant.

reDeleteTs := safepoint + oracle.ComposePhysical(24 * 60 * 60 * 1000), 0)

Signed-off-by: MyonKeminta <[email protected]>
Copy link
Member

@coocood coocood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coocood coocood added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 14, 2018
@coocood
Copy link
Member

coocood commented Sep 14, 2018

/run-all-tests

@zhangjinpeng87
Copy link
Contributor

LGTM

@coocood
Copy link
Member

coocood commented Sep 14, 2018

/run-unit-test

@zhangjinpeng87 zhangjinpeng87 merged commit a7f4687 into pingcap:master Sep 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants