Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'foreman-maintain service stop' causes Pulpcore core dump when database connection lost #824

Closed
ehelms opened this issue Apr 9, 2024 · 2 comments

Comments

@ehelms
Copy link
Member

ehelms commented Apr 9, 2024

Migrated from https://projects.theforeman.org/issues/37182

Submitted based on community 36512: https://community.theforeman.org/t/pulpcore-coredumps-during-stopping-services/36512

A user is doing some sort of "offline backup" operation which shuts down the database. Afterwards, the user calls 'foreman-maintain service stop', which expects the db endpoint to still be online; this causes a core dump in Pulpcore. Ideally, 'service stop' should be able to handle this edge case without core dumping, whether that is through handling a bad connection or via other means.

Excerpt from messages.log file on community post:

Jan 16 04:00:18 hostname pulpcore-content[210923]:  File "/usr/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 275, in get_new_connection
Jan 16 04:00:18 hostname pulpcore-content[210923]:    connection = self.Database.connect(**conn_params)
Jan 16 04:00:18 hostname pulpcore-content[210923]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 16 04:00:18 hostname pulpcore-content[210923]:  File "/usr/lib/python3.11/site-packages/psycopg/connection.py", line 728, in connect
Jan 16 04:00:18 hostname pulpcore-content[210923]:    raise ex.with_traceback(None)
Jan 16 04:00:18 hostname pulpcore-content[210923]: django.db.utils.OperationalError: connection failed: server closed the connection unexpectedly
Jan 16 04:00:18 hostname pulpcore-content[210923]: #011This probably means the server terminated abnormally
Jan 16 04:00:18 hostname pulpcore-content[210923]: #011before or while processing the request.
@ekohl
Copy link
Member

ekohl commented Sep 12, 2024

I think we should add a Wants=postgresql.service to pulpcore-content.service so systemd does the right thing. theforeman/puppet-pulpcore#359 should do so.

@ekohl
Copy link
Member

ekohl commented Oct 11, 2024

https://community.theforeman.org/t/pulpcore-coredumps-during-stopping-services/36512/8 is the root cause. theforeman/pulpcore-packaging#1361 (and theforeman/pulpcore-packaging#1363 for 3.49) should resolve this. Because it's not in foreman_maintain, I'm closing this issue.

@ekohl ekohl closed this as completed Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants