Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add backoff mechanism to the retention process #14182

Merged
merged 1 commit into from
Sep 19, 2024

Conversation

vlad-diachenko
Copy link
Contributor

What this PR does / why we need it:
added backoff mechanism to slow down and retry delete requests on retention.
It should help avoid the case when the cloud provider rate limits the compactor due to the huge number of requests fired to delete the chunks.

@vlad-diachenko vlad-diachenko requested a review from a team as a code owner September 19, 2024 13:10
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Sep 19, 2024
@vlad-diachenko vlad-diachenko force-pushed the vlad.diachenko/add-backoff-config-to-retention branch 2 times, most recently from 4542749 to a3cb045 Compare September 19, 2024 13:34
@vlad-diachenko vlad-diachenko force-pushed the vlad.diachenko/add-backoff-config-to-retention branch from a3cb045 to 3f3ccd1 Compare September 19, 2024 13:38
Copy link
Contributor

@benclive benclive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I have a very minor thought about whether -compactor.retention-backoff-config.backoff-min-period is a bit of a long flag/config name but I don't think its a big deal.

@vlad-diachenko vlad-diachenko merged commit 3136880 into main Sep 19, 2024
62 checks passed
@vlad-diachenko vlad-diachenko deleted the vlad.diachenko/add-backoff-config-to-retention branch September 19, 2024 14:34
loki-gh-app bot pushed a commit that referenced this pull request Sep 19, 2024
Signed-off-by: Vladyslav Diachenko <[email protected]>
(cherry picked from commit 3136880)
loki-gh-app bot pushed a commit that referenced this pull request Sep 19, 2024
Signed-off-by: Vladyslav Diachenko <[email protected]>
(cherry picked from commit 3136880)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport k219 backport k220 size/L type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants