Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat(bulk_load): add bulk load max rollback count #827

Merged
merged 4 commits into from
May 6, 2021

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented Apr 28, 2021

During bulk load process, meta server will control the bulk load status switch: downloading -> downloaded -> ingesting -> succeed, if replica configuration changed or network failed, meta server will rollback status into downloading. In pervious implementation, it is possible for bulk load process rollback for long time. This pull request adds bulk_load_max_rollback_times, if the table has been rollback to downloading for too many times, it will turn status into failed to stop the process.

configuration update:

[meta_server]
+bulk_load_max_rollback_times = 10

@hycdong hycdong added component/bulk-load type/config-change PR that made modification on configs, which should be noted in release note. labels Apr 28, 2021
levy5307
levy5307 previously approved these changes Apr 29, 2021
@Smityz
Copy link
Contributor

Smityz commented Apr 29, 2021

LGTM

@hycdong hycdong marked this pull request as draft April 30, 2021 02:45
@hycdong hycdong marked this pull request as ready for review April 30, 2021 03:35
@hycdong hycdong changed the title feat(bulk_load): meta server adds bulk_load max failover count restriction feat(bulk_load): add bulk load max rollback count Apr 30, 2021
@hycdong hycdong closed this Apr 30, 2021
@hycdong hycdong reopened this Apr 30, 2021
@hycdong hycdong closed this May 6, 2021
@hycdong hycdong reopened this May 6, 2021
@levy5307 levy5307 merged commit 788f4ea into XiaoMi:master May 6, 2021
@hycdong hycdong deleted the bulk_load_rollback_count branch May 6, 2021 10:37
@hycdong hycdong added the 2.2.0 label Aug 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
2.2.0 component/bulk-load type/config-change PR that made modification on configs, which should be noted in release note.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants