manageiq-pod stays unhealthy

[balpert89]

Hi there,

i try to deploy manage-iq on OpenShift (https://github.com/ManageIQ/manageiq-pods) but it seems the application does not start up.

running OSE 3.4 (PetSet instead of StatefulSet)
the PVs are in place and bound to the right claims
memcached and postgres are running fine
But the manageiq-0 pod refuses connections:

oc rsh manageiq-0 curl localhost:443
curl: (7) Failed connect to localhost:443; Connection refused

Output from systemctl status:

oc rsh manageiq-0 systemctl status
● manageiq-0
State: running
Jobs: 0 queued
Failed: 0 units
Since: Thu 2017-04-27 10:55:09 UTC; 5min ago
CGroup: /system.slice/docker-7b196f39a1d95e77f71c864eb7785c7268db4245d727612b
├─1 /usr/sbin/init
└─system.slice
├─evmserverd.service
│ └─control
│ ├─701 /bin/bash /bin/evmserver.sh stop
│ └─703 ruby /opt/rubies/ruby-2.3.1/bin/rake evm:stop
├─evm-watchdog.service
│ ├─223 /bin/bash /bin/evm_watchdog
│ └─224 ruby /bin/evm_watchdog.rb
├─dbus.service
│ └─78 /bin/dbus-daemon --system --address=systemd: --nofork --nopi
├─miqvmstat.service
│ └─45 vmstat -a -n 60
├─miqtop.service
│ └─44 top -b -d 60
├─crond.service
│ └─22 /usr/sbin/crond -n
└─systemd-journald.service
└─20 /usr/lib/systemd/systemd-journald

The readiness probe fails and so the pod is frequently restarted.
Here is the petset i am using: https://pastebin.com/UR3WJegN

I tried the Troubleshooting hints on the github but with no success.

the log in evm.log says following:

/var/www/miq/vmdb/lib/workers/evm_server.rb:86:in `database_application_name': undefined method database_application_name' for nil:NilClass (NoMethodError)                                                                                                                                                                                          
        from /var/www/miq/vmdb/lib/workers/evm_server.rb:82:in `set_database_application_name'                                                                                                                                                                                                                                                        
        from /var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start'                                                                                                                                                                                                                                                                                
        from /var/www/miq/vmdb/lib/workers/evm_server.rb:100:in `start'                                                                                                                                                                                                                                                                               
        from /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'

I also made some connectivity tests from the manageiq-pod to the postgresql-pod and the memcached - no issues, connection is established. The postgresql has the correct database and tables.

Unfortunately I am not familiar with Ruby, can you provide any input?

I hope you can help me.

Best regards

[simaishi]

@balpert89 Looks like you're hitting this issue: ManageIQ/manageiq#14904. Are you running the latest image?

[balpert89]

image: docker.io/manageiq/manageiq-pods:app-latest

i suppose :) so how can i work out this issue? or do i have to wait for a new version?

[simaishi]

app-latest always points to the latest image at the time of pull. If you pulled the image today, the image has ManageIQ/manageiq#14904 included as that was merged 2 days ago. And if you have that change, there is no line 100 in /var/www/miq/vmdb/lib/workers/evm_server.rb. Can you double check what image you have?

[balpert89]

hi @simaishi, i pulled all the images today as i deployed the project on another cluster.

Here is the output of the file you mentioned:
https://gist.github.com/balpert89/077385496ce95492e9cba13a19ac385a

[fbladilo]

@balpert89 Seems like the bug @simaishi noted, please ensure your evm_server.rb and miq_server.rb look patched as in :
https://github.com/ManageIQ/manageiq/pull/14904/files#diff-c6351ff454ebabe0a2a4a762d1edcfdeR246

If you want to force your petset to fetch latest image, please adjust imagePullPolicy :

1. oc edit petset manageiq
2. imagePullPolicy: Always
3. Scale down petset and scale back-up

[balpert89]

That was it :-) @fbladilo thx for your help!

Another quick question, does the PODified version of miq supports openid authentication? I found this guide for saml2 http://manageiq.org/docs/reference/latest/auth/saml but couldn't follow because there is no directory /usr/libexec/mod_auth_mellon in the manageiq-pod.

Should I create another issue?

Best regards

[fbladilo]

@balpert89 Feel free to create a separate issue for the openid auth, I'm probably not the best to answer in that regard but we can try to find the correct contact. I will close this one as the original issue has been resolved. Thanks.