feat(hotkey): build a fundamental framework of hotkey detection #603

Smityz · 2020-09-16T12:17:40Z

Hotkey detection process

+--------------------+  +----------------------------------------------------+
|   Replcia server   |  | Hotkey collector                                   |
|                    |  | +-----------------------------------------------+  |
| +----------------+ |  | | Corase capture                                |  |
| |                | |--> |                 +----------+                  |  |
| |  RPC received  | || | |                 |   Data   |                  |  |
| |                | || | |                 +-----+----+                  |  |
| +-------+--------+ || | |                       |                       |  |
|         |          || | |  +---------------+----v--+-------+---------+  |  |
|         v          || | |  |       |Hot    |       |       |         |  |  |
| +-------+--------+ || | |  |Bucket |Bucket |Bucket |Bucket |Bucket   |  |  |
| |   Replication  | || | |  +-----------+-----------------------------+  |  |
| | (only on the   | || | |              |                                |  |
| |  write path))  | || | +--------------|--------------------------------+  |
| +-------+--------+ || |             +--v---+                               |
|         |          || |             | Data |                               |
|         v          || |             +------+                               |
| +-------+--------+ || |          +-----|-------+-------------+             |
| |                | || |   +------|-------------|-------------|---------+   |
| |  Capture data  ---| |   | Fine |capture      |             |         |   |
| |                | |  |   |      |             |             |         |   |
| +-------+--------+ |  |   | +----v----+   +----v----+   +----v----+    |   |
|         |          |  |   | |  queue  |   |  queue  |   |  queue  |    |   |
|         v          |  |   | +----+----+   +----+----+   +----+----+    |   |
| +-------+--------+ |  |   |      |             |             |         |   |
| |                | |  |   | +----v-------------v-------------v------+  |   |
| |   Place data   | |  |   | |             Analsis pool              |  |   |
| |   to the disk  | |  |   | +-----------------|---------------------+  |   |
| |                | |  |   +-------------------|------------------------+   |
| +----------------+ |  |                       v                            |
|                    |  |                     Hotkey                         |
+--------------------+  +----------------------------------------------------+

What problem does this PR solve?

Rely PR: XiaoMi/rdsn#634
In this PR, I define two interfaces of hotkey_collector.
capture_raw_key is to capture data in capacity_unit_calculator, in the future PR, I will analyze these data to find the hotkey.
handle_operation is to receive the control RPC of hotkey detection.
capture_raw_key is executed in capacity_unit_calculator, I will implentment it in the next PR

src/server/hotkey_coarse_data_collector.h

src/server/hotkey_collector.h

src/server/hotkey_fine_data_collector.h

src/server/hotkey_coarse_data_collector.h

src/server/hotkey_collector.h

src/server/hotkey_fine_data_collector.h

src/server/hotkey_coarse_data_collector.h

src/server/hotkey_coarse_data_collector.cpp

src/server/hotkey_coarse_data_collector.h

src/server/hotkey_collector.h

src/server/hotkey_fine_data_collector.h

…ey-platform

src/server/hotkey_collector.h

acelyc111 · 2020-10-04T06:34:33Z

src/server/capacity_unit_calculator.cpp

+    _read_hotkey_collector->capture_raw_key(key, key.size() + value.size());
+


I think it's not needed to add capture_raw_key in this PR, since only this is just a 'fundamental framework' PR, and some othe read/write paths like add_write_cu missing call it.

capture_raw_key is a fundamental interface of hotkey_collector, I want to show how to use this function

src/server/pegasus_server_impl.cpp

Co-authored-by: Yingchun Lai <[email protected]>

src/server/pegasus_server_impl.cpp

src/server/hotkey_collector.h

src/server/capacity_unit_calculator.cpp

src/server/hotkey_collector.h

levy5307 · 2020-10-09T11:22:15Z

src/server/capacity_unit_calculator.cpp

@@ -122,21 +134,25 @@ void capacity_unit_calculator::add_multi_get_cu(int32_t status,
    }

    if (status == rocksdb::Status::kNotFound) {
+        _read_hotkey_collector->capture_hash_key(hash_key, 1);


maybe you can move this into func add_read_cu
same with add_write_cu

capture_hash_key and add_read_cu are two independent function. It will destroy the original semantics.

hycdong · 2020-10-10T02:42:46Z

src/server/capacity_unit_calculator.cpp

@@ -122,21 +134,25 @@ void capacity_unit_calculator::add_multi_get_cu(int32_t status,
    }

    if (status == rocksdb::Status::kNotFound) {
+        _read_hotkey_collector->capture_hash_key(hash_key, 1);


I notice that you try to capture not-found key for multi_get, but not for single get, could you please explain why you design it?

hycdong · 2020-10-10T02:44:07Z

src/server/capacity_unit_calculator.h

-    void add_scan_cu(int32_t status, const std::vector<::dsn::apps::key_value> &kvs);
+    void add_scan_cu(int32_t status,
+                     const std::vector<::dsn::apps::key_value> &kvs,
+                     const dsn::blob &hash_key_filter_pattern = dsn::blob());


What is the usage of hash_key_filter_pattern?

hycdong · 2020-10-12T01:45:02Z

src/server/capacity_unit_calculator.h

@@ -55,6 +64,9 @@ class capacity_unit_calculator : public dsn::replication::replica_base
 #endif

 private:
+    void count_read_data(const dsn::blob &key, key_type type, int64_t size);
+    void count_write_data(const dsn::blob &key, key_type type, int64_t size);


Is function count_wirte_data used for add cu and capture key for hotkey detection?

Smityz changed the title ~~feat(hotkey detection): add a fundamental construct of hotkey detection~~ feat(hotkey detection): build a fundamental framework of hotkey detection Sep 16, 2020

acelyc111 reviewed Sep 16, 2020

View reviewed changes

src/server/hotkey_coarse_data_collector.h Outdated Show resolved Hide resolved

src/server/hotkey_collector.h Outdated Show resolved Hide resolved

src/server/hotkey_collector.h Outdated Show resolved Hide resolved

src/server/hotkey_fine_data_collector.h Outdated Show resolved Hide resolved

Smityz mentioned this pull request Sep 17, 2020

The way to split PR 502 #588

Closed

acelyc111 reviewed Sep 17, 2020

View reviewed changes

levy5307 reviewed Sep 18, 2020

View reviewed changes

src/server/hotkey_fine_data_collector.h Outdated Show resolved Hide resolved

levy5307 reviewed Sep 18, 2020

View reviewed changes

src/server/hotkey_fine_data_collector.h Outdated Show resolved Hide resolved

neverchanje reviewed Sep 18, 2020

View reviewed changes

src/server/hotkey_coarse_data_collector.h Outdated Show resolved Hide resolved

hycdong reviewed Sep 18, 2020

View reviewed changes

Smityz marked this pull request as draft September 18, 2020 10:39

refactor

6986e58

Smityz marked this pull request as ready for review September 30, 2020 06:03

Smityz force-pushed the hotkey-platform branch from b0db511 to 6986e58 Compare September 30, 2020 06:03

Update hotkey_collector.cpp

8c83ad5

Smityz changed the title ~~feat(hotkey detection): build a fundamental framework of hotkey detection~~ feat(hotkey): build a fundamental framework of hotkey detection Sep 30, 2020

Smityz added 3 commits September 30, 2020 17:24

rdsn

c8de699

Merge branch 'hotkey-platform' of github.com:Smityz/pegasus into hotk…

5b89aa1

…ey-platform

fix

a8b2486

acelyc111 reviewed Oct 4, 2020

View reviewed changes

Smityz and others added 3 commits October 5, 2020 00:41

Update src/server/hotkey_collector.h

27e2f54

Co-authored-by: Yingchun Lai <[email protected]>

update

a271b71

Merge branch 'master' into hotkey-platform

5494345

acelyc111 previously approved these changes Oct 9, 2020

View reviewed changes

hycdong reviewed Oct 9, 2020

View reviewed changes

src/server/pegasus_server_impl.cpp Show resolved Hide resolved

change err

fb2c7b6

Smityz dismissed acelyc111’s stale review via fb2c7b6 October 9, 2020 03:50

levy5307 reviewed Oct 9, 2020

View reviewed changes

src/server/hotkey_collector.h Outdated Show resolved Hide resolved

levy5307 reviewed Oct 9, 2020

View reviewed changes

src/server/capacity_unit_calculator.cpp Outdated Show resolved Hide resolved

levy5307 reviewed Oct 9, 2020

View reviewed changes

src/server/hotkey_collector.h Outdated Show resolved Hide resolved

Smityz added 2 commits October 9, 2020 16:41

add more inpl

c4a7c27

change name

ecdfe37

Smityz added 2 commits October 9, 2020 16:50

update

b40f209

add comments

3730a0b

Smityz requested review from levy5307, acelyc111 and hycdong October 9, 2020 08:59

acelyc111 previously approved these changes Oct 9, 2020

View reviewed changes

levy5307 reviewed Oct 9, 2020

View reviewed changes

Smityz added 2 commits October 10, 2020 10:24

fix

f73d921

fix

179c3a8

Smityz dismissed acelyc111’s stale review via 179c3a8 October 10, 2020 02:25

hycdong reviewed Oct 10, 2020

View reviewed changes

Smityz added 3 commits October 10, 2020 12:20

refactor

5020921

fix name

294e216

fix

21f2230

Smityz requested review from hycdong, levy5307 and acelyc111 October 10, 2020 06:30

Smityz added 2 commits October 10, 2020 16:04

fix add_scan_cu

031b5bd

fix

582ab4a

hycdong reviewed Oct 12, 2020

View reviewed changes

Smityz and others added 3 commits October 12, 2020 11:13

split pr

6b8b9d5

change comment

afd6924

Merge branch 'master' into hotkey-platform

a6440a6

hycdong approved these changes Oct 12, 2020

View reviewed changes

acelyc111 approved these changes Oct 12, 2020

View reviewed changes

hycdong merged commit 3b96b26 into apache:master Oct 12, 2020

Smityz deleted the hotkey-platform branch October 12, 2020 08:04

Smityz mentioned this pull request Oct 29, 2020

feat(hotkey): capture data part3 - declare fine collector #631

Merged

neverchanje mentioned this pull request Mar 1, 2021

Release 2.2.0 #696

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(hotkey): build a fundamental framework of hotkey detection #603

feat(hotkey): build a fundamental framework of hotkey detection #603

Smityz commented Sep 16, 2020 •

edited

Loading

acelyc111 Oct 4, 2020

Smityz Oct 4, 2020

levy5307 Oct 9, 2020

Smityz Oct 10, 2020

hycdong Oct 10, 2020

hycdong Oct 10, 2020 •

edited

Loading

hycdong Oct 12, 2020

		_read_hotkey_collector->capture_raw_key(key, key.size() + value.size());

feat(hotkey): build a fundamental framework of hotkey detection #603

feat(hotkey): build a fundamental framework of hotkey detection #603

Conversation

Smityz commented Sep 16, 2020 • edited Loading

Hotkey detection process

What problem does this PR solve?

acelyc111 Oct 4, 2020

Choose a reason for hiding this comment

Smityz Oct 4, 2020

Choose a reason for hiding this comment

levy5307 Oct 9, 2020

Choose a reason for hiding this comment

Smityz Oct 10, 2020

Choose a reason for hiding this comment

hycdong Oct 10, 2020

Choose a reason for hiding this comment

hycdong Oct 10, 2020 • edited Loading

Choose a reason for hiding this comment

hycdong Oct 12, 2020

Choose a reason for hiding this comment

Smityz commented Sep 16, 2020 •

edited

Loading

hycdong Oct 10, 2020 •

edited

Loading