Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checking backup #287

Closed
justewg opened this issue Oct 24, 2021 · 11 comments · Fixed by #288
Closed

Checking backup #287

justewg opened this issue Oct 24, 2021 · 11 comments · Fixed by #288

Comments

@justewg
Copy link

justewg commented Oct 24, 2021

How can I check my backup creation?
I've run clickhouse-backup as root user using script, that was written in your readme.
Although clickhouse-backup list shows all tables that I want to backup and backup has been created - It's just empty.

root@db-2:/home/ewg# clickhouse-backup list
Local backups:
- 'my_backup_2021-10-24T09-46-50'	(created at 24-10-2021 09:46:50)
Remote backups:
2021/10/24 09:57:35 storage type 'none' not supported

root@db-2:/home/ewg# ls -al /var/lib/clickhouse/store/backup/my_backup_2021-10-24T09-46-50/
total 12
drwxr-xr-x 3 root root 4096 Oct 24 09:46 .
drwxr-xr-x 3 ewg  ewg  4096 Oct 24 09:46 ..
drwxr-xr-x 2 root root 4096 Oct 24 09:46 shadow

root@db-2:/home/ewg# ls -al /var/lib/clickhouse/store/backup/my_backup_2021-10-24T09-46-50/shadow/
total 8
drwxr-xr-x 2 root root 4096 Oct 24 09:46 .
drwxr-xr-x 3 root root 4096 Oct 24 09:46 ..
@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

@justewg did you run clickhouse-backup at the same server where clickhouse-server run?

@justewg
Copy link
Author

justewg commented Oct 24, 2021

А можно по-русски поизъясняться?

@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

вы запустили clickhouse-backup create на том же сервере где запущен clickhouse-server?

ls -la /var/lib/clickhouse/store/backup/my_backup_2021-10-24T09-46-50/shadow
тут тоже пусто?

@justewg
Copy link
Author

justewg commented Oct 24, 2021

Да, тут тоже пусто, вероятно именно потому что я не запускал докер с сервисом, поскольку решил что можно запускать либо clickhouse-backup, скаченный и разархивированный, либо установленный через докер. А докер нужен для работы API, если есть потребность в использовании именно API.
Похоже я плохо осознал документацию в README.
То есть для работы бэкапа необходим докер-сервер ?

@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

Нет, докер сервер для бекапа не нужен, перечитайте пожалуйста мой вопрос

Давайте разбираться
как у вас запущен clickhouse-server? В докере или просто как systemd service?
Как вы запускаете создание backup просто clickhouse-backup create?

Можете запустить
LOG_LEVEL=debug clickhouse-backup create и показать вывод?

@justewg
Copy link
Author

justewg commented Oct 24, 2021

Да. КХ запущен обычным системным демоном.

root@db-2:/home/ewg# LOG_LEVEL=debug clickhouse-backup create
2021/10/24 13:49:38 Create backup '2021-10-24T13-49-38'
2021/10/24 13:49:38 Freeze `testdb`.`users`
2021/10/24 13:49:39 Freeze `testdb`.`test`
2021/10/24 13:49:39 Freeze `testdb`.`address`
2021/10/24 13:49:39 Freeze `testdb`.`city`
2021/10/24 13:49:39 Skip `system`.`asynchronous_metric_log`
2021/10/24 13:49:39 Skip `system`.`metric_log`
2021/10/24 13:49:39 Skip `system`.`part_log`
2021/10/24 13:49:39 Skip `system`.`query_log`
2021/10/24 13:49:39 Skip `system`.`query_thread_log`
2021/10/24 13:49:39 Skip `system`.`trace_log`
2021/10/24 13:49:39 Copy metadata
2021/10/24 13:49:39   Done.
2021/10/24 13:49:39 Move shadow
2021/10/24 13:49:39   Done.
root@db-2:/home/ewg#

@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

что показывает
clickhouse-backup version ?

@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

что показывает
clickhouse-client -q "SELECT version()"

@justewg
Copy link
Author

justewg commented Oct 24, 2021

clickhouse-backup version ничего не показывает,

root@db-2:/home/ewg# clickhouse-backup --version
Version:	 0.5.2
Git Commit:	 f7975491a6c117f4ce03dc30975bce16fe619da6
Build Date:	 2020-05-19
root@db-2:/home/ewg# clickhouse-client -q "SELECT version()"
"SELECT version()"
21.9.4.35
root@db-2:/home/ewg#

@Slach Slach mentioned this issue Oct 24, 2021
@Slach
Copy link
Collaborator

Slach commented Oct 24, 2021

У вас очень старая версия clickhouse-backup, и очень новая версия clickhouse-server в ней движок базы данных Atomic, который не поддерживается clickhouse-backup 0.5.2

обновите clickhouse-backup до 1.1.1

@justewg
Copy link
Author

justewg commented Oct 24, 2021

Благодарю, похоже что теперь сбэкапилось.

root@db-2:/home/ewg# clickhouse-backup list
2021/10/24 15:29:02  info SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
2021/10/24 15:29:02  info SELECT * FROM system.disks;
my_backup_2021-10-24T15-28-40   5.16GiB   24/10/2021 15:28:40   local

@Slach Slach closed this as completed Oct 24, 2021
Slach added a commit that referenced this issue Oct 24, 2021
# v1.2.0

INCOMPATIBLE CHANGES
- REST API `/backup/status` now return only latest executed command with status and error message

IMPROVEMENTS
- Added REST API `/backup/list/local` and `/backup/list/remote` to allow list backup types separately
- Decreased background backup creation time via REST API `/backup/create`, during avoid list remote backups for update metrics value 
- Decreased backup creation time, during avoid scan whole `system.tables` when set `table` query string parameter or `--tables` cli parameter     
- Added `last` and `filter` query string parameters to REST API `/backup/actions`, to avoid pass to client long JSON documents
- Improved `FTP` remote storage parallel upload / download
- Added `FTP_CONCURRENCY` to allow, by default MAX_CPU / 2 
- Added `FTP_DEBUG` setting, to allow debug FTP commands
- Added `FTP` to CI/CD on any commit
- Added race condition check to CI/CD

BUG FIXES
- environment variable `LOG_LEVEL` now apply to `clickhouse-backup server` properly
- fix #280, incorrect prometheus metrics measurement for `/backup/create`, `/backup/upload`, `/backup/download`
- fix #273, return `S3_PART_SIZE` back, but calculates it smartly
- fix #252, now you can pass `last` and `filter` query string parameters
- fix #246, incorrect error messages when use `REMOTE_STORAGE=none`
- fix #283, properly handle error message from `FTP` server 
- fix #268, properly restore legacy backup for schema without database name
- fix #287
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants