Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if backup name exists and add retry on ClickhouseError for queries #173

Merged
merged 2 commits into from
Jul 22, 2024
+9 −5
Merged

Check if backup name exists and add retry on ClickhouseError for queries #173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e6124e8
Check if backup with given name already exists
kirillgarbar Jul 16, 2024
84888b1
Add retry for query on ClickhouseError
kirillgarbar Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions ch_backup/ch_backup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,14 @@ def backup(
"""
# pylint: disable=too-many-branches
# pylint: disable=too-many-locals
backups_with_light_meta = self._context.backup_layout.get_backups(
use_light_meta=True
)

for backup in backups_with_light_meta:
if name == backup.name:
raise ClickhouseBackupError(f"Backup with name {name} already exists")

logging.info(f"Backup sources: {sources}")
assert not (db_names and tables)

Expand All @@ -127,10 +135,6 @@ def backup(
if db_names is not None:
databases = [db for db in databases if db.name in db_names]

backups_with_light_meta = self._context.backup_layout.get_backups(
use_light_meta=True
)

last_backup = next(iter(backups_with_light_meta), None)
if last_backup and not self._check_min_interval(last_backup, force):
msg = "Backup is skipped per backup.min_interval config option."
Expand Down
2 changes: 1 addition & 1 deletion ch_backup/clickhouse/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def settings(self):
"""
return self._session.params

@retry(requests.exceptions.ConnectionError)
@retry((requests.exceptions.ConnectionError, ClickhouseError))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retry of a failed non-SELECT queries will cause unnecessary logs and longer fail time, in that case a check if query is a SELECT query can be added with retries on ClickhouseSelectError or something similar

def query(
self,
query: str,
Expand Down