Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backup: Use the default zulip backup system #262

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

backup: Use the default zulip backup system #262

wants to merge 1 commit into from

Conversation

hackerkid
Copy link
Member

@hackerkid hackerkid commented Jul 3, 2020 •
edited
Loading

Zulip backup/restore system now works fine in a docker environment. So we don't need this custom system anymore.

Related https://chat.zulip.org/#narrow/stream/31-production-help/topic/docker.20zulip

@hackerkid hackerkid force-pushed the backup-restore branch 4 times, most recently from 0d630db to 4af9c79 Compare August 26, 2020 11:19
@hackerkid hackerkid changed the title [WIP] backup: Use the default zulip backup system backup: Use the default zulip backup system Aug 26, 2020
@hackerkid
Copy link
Member Author

@timabbott This is ready for a review.

@timabbott
Copy link
Member

I think we should probably still add some documentation with the precise commands needed to use ./manage.py backup and scripts/setup/restore-backup, perhaps in a new section in README.md? Spelling out details like how to get a backup tarball (which you may have created on another machine) in place to be accessible inside the container etc. seem important.

@hackerkid hackerkid force-pushed the backup-restore branch 3 times, most recently from 9686261 to 9cad654 Compare August 28, 2020 18:48
@hackerkid
backup: Remove the custom backup and restore system.
00dcae7 
Zulip backup/restore system now works in a docker
environment. So we don't need this anymore.
@hackerkid
Copy link
Member Author

@timabbott I have added the instructions. Also verified that the instructions work correctly.

@hackerkid
Copy link
Member Author

One thing I noticed is that after the restore is completed, the log is filled with events like this and they keep on going and going. I had to ctrl + c and do docker-compose up again for them to stop.

ulip_1      | 2020-08-28 18:57:07,879 INFO spawned: 'zulip_events_outgoing_webhooks' with pid 1003
zulip_1      | 2020-08-28 18:57:07,880 INFO success: zulip_events_email_senders entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,106 INFO success: zulip_events_user_activity_interval entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,106 INFO success: zulip_events_digest_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,107 INFO exited: zulip_events_embed_links (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:08,151 INFO spawned: 'zulip_events_embed_links' with pid 1005
zulip_1      | 2020-08-28 18:57:08,311 INFO success: zulip_events_missedmessage_mobile_notifications entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,311 INFO exited: zulip_events_error_reports (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:08,317 INFO spawned: 'zulip_events_error_reports' with pid 1006
zulip_1      | 2020-08-28 18:57:08,318 INFO exited: zulip_deliver_enqueued_emails (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:08,681 INFO spawned: 'zulip_deliver_enqueued_emails' with pid 1009
zulip_1      | 2020-08-28 18:57:08,682 INFO success: zulip_events_email_mirror entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,682 INFO success: zulip_events_missedmessage_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,835 INFO exited: zulip_events_invites (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:08,842 INFO spawned: 'zulip_events_invites' with pid 1012
zulip_1      | 2020-08-28 18:57:08,848 INFO exited: zulip_events_email_senders (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:08,902 INFO success: zulip_events_outgoing_webhooks entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:08,904 INFO spawned: 'zulip_events_email_senders' with pid 1013
zulip_1      | 2020-08-28 18:57:09,092 INFO exited: zulip_events_user_presence (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,132 INFO spawned: 'zulip_events_user_presence' with pid 1014
zulip_1      | 2020-08-28 18:57:09,134 INFO exited: zulip_events_user_activity (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,163 INFO spawned: 'zulip_events_user_activity' with pid 1015
zulip_1      | 2020-08-28 18:57:09,166 INFO success: zulip_events_embed_links entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:09,295 INFO exited: zulip_events_digest_emails (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,298 INFO spawned: 'zulip_events_digest_emails' with pid 1017
zulip_1      | 2020-08-28 18:57:09,299 INFO exited: zulip_events_deferred_work (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,527 INFO spawned: 'zulip_events_deferred_work' with pid 1019
zulip_1      | 2020-08-28 18:57:09,528 INFO success: zulip_events_error_reports entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:09,538 INFO exited: zulip_events_embedded_bots (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,589 INFO spawned: 'zulip_events_embedded_bots' with pid 1020
zulip_1      | 2020-08-28 18:57:09,590 INFO exited: zulip_events_signups (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,595 INFO spawned: 'zulip_events_signups' with pid 1021
zulip_1      | 2020-08-28 18:57:09,789 INFO success: zulip_deliver_enqueued_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:09,904 INFO success: zulip_events_invites entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:09,904 INFO success: zulip_events_email_senders entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:09,905 INFO exited: zulip_events_email_mirror (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:09,963 INFO spawned: 'zulip_events_email_mirror' with pid 1024
zulip_1      | 2020-08-28 18:57:09,965 INFO exited: zulip_events_user_activity_interval (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:10,052 INFO spawned: 'zulip_events_user_activity_interval' with pid 1027
zulip_1      | 2020-08-28 18:57:10,052 INFO exited: zulip_events_missedmessage_mobile_notifications (exit status 1; not expected)
zulip_1      | 2020-08-28 18:57:10,178 INFO success: zulip_events_user_presence entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:10,178 INFO success: zulip_events_user_activity entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
zulip_1      | 2020-08-28 18:57:10,180 INFO spawned: 'zulip_events_missedmessage_mobile_notifications' with pid 1029
zulip_1      | 2020-08-28 18:57:10,342 INFO success: zulip_events_digest_emails entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
^CGracefully stopping... (press Ctrl+C again to force)

@timabbott
Copy link
Member

That needs investigation in ls -ltr /var/log/zulip/* -- sounds like those queue workers are constantly crashing.

@timabbott
Copy link
Member

@hackerkid any luck investigating the errors seen here?

@hackerkid
Copy link
Member Author

hackerkid commented Feb 18, 2021 •
edited
Loading

I will look into this again. I didn't spend much time investigating last time.

@maxxer maxxer mentioned this pull request Nov 26, 2021
Closed
@maxxer
Copy link
Contributor

maxxer commented Nov 26, 2021

The standard backup procedure is storing the default uploads directory. Since in Docker this is a link it will just backup the link, without the actual data. Is this intended?

I know the uploads dir is already on the host filesystem, but if backups are meant to be backups they should include it. Filing a new issue

@hackerkid
Copy link
Member Author

Closing since this has became quite stale.

@hackerkid hackerkid closed this Sep 22, 2024
@timabbott
Copy link
Member

I'm going to reopen this, since it's one of the larger pieces of unmerged code that we want to integrate, and I don't want to lose track of it. Probably someone else will make a new PR off it soon.

@timabbott timabbott reopened this Sep 26, 2024
@hackerkid
Copy link
Member Author

hackerkid commented Sep 28, 2024 •
edited
Loading

Sounds good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hackerkid @timabbott @maxxer