region_cache: support buckets #439
Conversation
youjiali1995
force-pushed
the
region-cache-with-bucket
branch
2 times, most recently
from
7839d1c to
423dc8b
Compare
Signed-off-by: youjiali1995 <[email protected]>
youjiali1995
force-pushed
the
region-cache-with-bucket
branch
from
423dc8b to
e0ebd26
Compare
…n-cache-with-bucket Signed-off-by: youjiali1995 <[email protected]>
youjiali1995
force-pushed
the
region-cache-with-bucket
branch
3 times, most recently
from
e88ef0f to
30cc87e
Compare
Signed-off-by: youjiali1995 <[email protected]>
youjiali1995
force-pushed
the
region-cache-with-bucket
branch
from
30cc87e to
66ae954
Compare
|
@cfzjywxk PTAL too, if you have time. |
| @@ -1027,6 +1084,8 @@ func (c *RegionCache) BatchLoadRegionsWithKeyRange(bo *retry.Backoffer, startKey | |||
| c.mu.Lock() | |||
| defer c.mu.Unlock() | |||
|
|
|||
| // TODO(youjiali1995): scanRegions always fetch regions from PD and these regions don't contain buckets information | |||
There was a problem hiding this comment.
Does it mean that we are going to add option to the ScanRegions interface, so it can return buckets in the same response?
There was a problem hiding this comment.
No, too much network traffic. A 10GB region can contains 100 buckets, so the payload of a single region can be several KBs. I will make BatchLoadRegionsWithKeyRange() use scanRegionsFromCache() or not insert regions to cache.
| if meta, err = decodeRegionMetaKeyWithShallowCopy(meta); err != nil { | ||
| return false, errors.Errorf("newRegion's range key is not encoded: %v, %v", meta, err) | ||
| } | ||
| } | ||
| region, err := newRegion(bo, c, &pd.Region{Meta: meta}) | ||
| // TODO(youjiali1995): new regions inherit old region's buckets now. Maybe we should make EpochNotMatch error | ||
| // carry buckets information. Can it bring much overhead? |
There was a problem hiding this comment.
Is is true that bucket information usually doesn't change greatly just after a region merge or split? Then, an asynchronous UpdateBucketsIfNeeded from PD may be ok.
There was a problem hiding this comment.
Agree. Buckets is generated asynchronously after region leader or range changed. Querying PD immediately may not get the newer buckets, so not handling it and depending on https://github.com/pingcap/tidb/pull/32818/files#diff-23eb1043649369d5956b123b173b799ded8c6b7cbf4519bb32cb3caa01928d75R936 is also a choice.
Signed-off-by: youjiali1995 <[email protected]>
Signed-off-by: youjiali1995 [email protected]
Region cache now saves buckets information which is fetched from PD by
GetRegion()orGetRegionByID()(tikv/pd#4699):RegionStorebecause it can change without region change.There are some TODOs left, I will file PRs for them:
BatchLoadRegionsWithKeyRange()which is used by GC worker populates the region cache now.GetRegion()traffic flow bysingleflight.