Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nginx out of memory crash on startup #321

Closed
ms32035 opened this issue Feb 17, 2024 · 9 comments
Closed

Nginx out of memory crash on startup #321

ms32035 opened this issue Feb 17, 2024 · 9 comments
Labels
stale There has not been activity on this issue or PR for quite some time.

Comments

@ms32035
Copy link

ms32035 commented Feb 17, 2024

Problem/Motivation

The addon crashed on startup with:

2024/02/17 20:26:45 [crit] 403#403: ngx_slab_alloc() failed: no memory

Running this on Rasbperry Pi 5 8GB, and RAM usage peaks at 2GB

Expected behavior

Plugin starts correctly

Actual behavior

-----------------------------------------------------------
 Add-on: InfluxDB
 Scalable datastore for metrics, events, and real-time analytics
-----------------------------------------------------------
 Add-on version: 5.0.0
 You are running the latest version of this add-on.
 System: Debian GNU/Linux 12 (bookworm)  (aarch64 / raspberrypi5-64)
 Home Assistant Core: 2024.2.2
 Home Assistant Supervisor: 2024.01.1
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-timezone: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
[20:26:29] INFO: Configuring timezone (Europe/London)...
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service base-addon-timezone successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/create-users.sh
cont-init: info: /etc/cont-init.d/create-users.sh exited 0
cont-init: info: running /etc/cont-init.d/influxdb.sh
cont-init: info: /etc/cont-init.d/influxdb.sh exited 0
cont-init: info: running /etc/cont-init.d/kapacitor.sh
cont-init: info: /etc/cont-init.d/kapacitor.sh exited 0
cont-init: info: running /etc/cont-init.d/nginx.sh
cont-init: info: /etc/cont-init.d/nginx.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun chronograf (no readiness notification)
services-up: info: copying legacy longrun influxdb (no readiness notification)
services-up: info: copying legacy longrun kapacitor (no readiness notification)
services-up: info: copying legacy longrun nginx (no readiness notification)
[20:26:30] INFO: Chronograf is waiting until InfluxDB is available...
[20:26:30] INFO: Kapacitor is waiting until InfluxDB is available...
s6-rc: info: service legacy-services successfully started
[20:26:31] INFO: Starting the InfluxDB...
[20:26:40] INFO: Starting Chronograf...
[20:26:40] INFO: Starting the Kapacitor

'##:::'##::::'###::::'########:::::'###:::::'######::'####:'########::'#######::'########::
 ##::'##::::'## ##::: ##.... ##:::'## ##:::'##... ##:. ##::... ##..::'##.... ##: ##.... ##:
 ##:'##::::'##:. ##:: ##:::: ##::'##:. ##:: ##:::..::: ##::::: ##:::: ##:::: ##: ##:::: ##:
 #####::::'##:::. ##: ########::'##:::. ##: ##:::::::: ##::::: ##:::: ##:::: ##: ########::
 ##. ##::: #########: ##.....::: #########: ##:::::::: ##::::: ##:::: ##:::: ##: ##.. ##:::
 ##:. ##:: ##.... ##: ##:::::::: ##.... ##: ##::: ##:: ##::::: ##:::: ##:::: ##: ##::. ##::
 ##::. ##: ##:::: ##: ##:::::::: ##:::: ##:. ######::'####:::: ##::::. #######:: ##:::. ##:
..::::..::..:::::..::..:::::::::..:::::..:::......:::....:::::..::::::.......:::..:::::..::

2024/02/17 20:26:40 Using configuration at: /etc/kapacitor/kapacitor.conf
ts=2024-02-17T20:26:40.358Z lvl=error msg="error while sending usage report on startup" service=reporting err="Post \"https://usage.influxdata.com/api/v1/usage/kapacitor\": dial tcp 0.0.0.0:443: connect: connection refused"
time="2024-02-17T20:26:45Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime"
time="2024-02-17T20:26:45Z" level=info msg="Serving chronograf at http://127.0.0.1:8889" component=server
[20:26:45] INFO: Starting NGINX...
2024/02/17 20:26:45 [crit] 403#403: ngx_slab_alloc() failed: no memory
[20:26:45] WARNING: NGINX crashed, halting add-on
[20:26:45] INFO: NGINX stopped, restarting...
s6-rc: info: service legacy-services: stopping
[tcp] 2024/02/17 20:26:45 tcp.Mux: Listener at 127.0.0.1:8088 failed failed to accept a connection, closing all listeners - accept tcp 127.0.0.1:8088: use of closed network connection
[20:26:45] INFO: Chronograf stopped, restarting...
[20:26:45] INFO: Kapacitor stopped, restarting...
[20:26:45] INFO: InfluxDB stopped, restarting...
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service base-addon-timezone: stopping
s6-rc: info: service base-addon-log-level: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service base-addon-timezone successfully stopped
s6-rc: info: service base-addon-log-level successfully stopped
s6-rc: info: service base-addon-banner: stopping
s6-rc: info: service base-addon-banner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
@sinclairpaul
Copy link
Member

In all honesty this is likely a challenge with Bookworm on a PI5, can you duplicate it under HAOS?

@ms32035
Copy link
Author

ms32035 commented Feb 17, 2024

Sadly don't have access to HAOS

@ms32035
Copy link
Author

ms32035 commented Feb 17, 2024

@sinclairpaul would you be able to bump the nginx version to minimise that risk? nginx=1.22.1-9 looks quite dated

@sinclairpaul
Copy link
Member

It's the latest on the Debian stable release....

As we haven't had other reports, it is very likely to do with the install method, and is likely not much that can be done from the addon perspective.

@kloppy1984
Copy link

kloppy1984 commented Feb 26, 2024

I have the same Problem:


Add-on: InfluxDB
Scalable datastore for metrics, events, and real-time analytics

Add-on version: 5.0.0
You are running the latest version of this add-on.
System: Debian GNU/Linux 12 (bookworm) (aarch64 / raspberrypi5-64)
Home Assistant Core: 2024.2.3
Home Assistant Supervisor: 2024.02.0

Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.

s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-timezone: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
[13:22:54] INFO: Configuring timezone (Europe/Paris)...
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service base-addon-timezone successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/create-users.sh
[13:22:58] INFO: InfluxDB init process in progress...
cont-init: info: /etc/cont-init.d/create-users.sh exited 0
cont-init: info: running /etc/cont-init.d/influxdb.sh
[tcp] 2024/02/26 13:23:03 tcp.Mux: Listener at 127.0.0.1:8088 failed failed to accept a connection, closing all listeners - accept tcp 127.0.0.1:8088: use of closed network connection
cont-init: info: /etc/cont-init.d/influxdb.sh exited 0
cont-init: info: running /etc/cont-init.d/kapacitor.sh
cont-init: info: /etc/cont-init.d/kapacitor.sh exited 0
cont-init: info: running /etc/cont-init.d/nginx.sh
cont-init: info: /etc/cont-init.d/nginx.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun chronograf (no readiness notification)
services-up: info: copying legacy longrun influxdb (no readiness notification)
services-up: info: copying legacy longrun kapacitor (no readiness notification)
services-up: info: copying legacy longrun nginx (no readiness notification)
[13:23:04] INFO: Kapacitor is waiting until InfluxDB is available...
[13:23:04] INFO: Chronograf is waiting until InfluxDB is available...
s6-rc: info: service legacy-services successfully started
[13:23:04] INFO: Starting the InfluxDB...
[13:23:14] INFO: Starting the Kapacitor
[13:23:14] INFO: Starting Chronograf...

'##:::'##::::'###::::'########:::::'###:::::'######::'####:'########::'#######::'########::
##::'##::::'## ##::: ##.... ##:::'## ##:::'##... ##:. ##::... ##..::'##.... ##: ##.... ##:
##:'##::::'##:. ##:: ##:::: ##::'##:. ##:: ##:::..::: ##::::: ##:::: ##:::: ##: ##:::: ##:
#####::::'##:::. ##: ########::'##:::. ##: ##:::::::: ##::::: ##:::: ##:::: ##: ########::
##. ##::: #########: ##.....::: #########: ##:::::::: ##::::: ##:::: ##:::: ##: ##.. ##:::
##:. ##:: ##.... ##: ##:::::::: ##.... ##: ##::: ##:: ##::::: ##:::: ##:::: ##: ##::. ##::
##::. ##: ##:::: ##: ##:::::::: ##:::: ##:. ######::'####:::: ##::::. #######:: ##:::. ##:
..::::..::..:::::..::..:::::::::..:::::..:::......:::....:::::..::::::.......:::..:::::..::

2024/02/26 13:23:14 Using configuration at: /etc/kapacitor/kapacitor.conf
time="2024-02-26T13:23:20+01:00" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime"
time="2024-02-26T13:23:20+01:00" level=info msg="Serving chronograf at http://127.0.0.1:8889" component=server
[13:23:20] INFO: Starting NGINX...
2024/02/26 13:23:20 [crit] 482#482: ngx_slab_alloc() failed: no memory
[13:23:20] WARNING: NGINX crashed, halting add-on
[13:23:20] INFO: NGINX stopped, restarting...
s6-rc: info: service legacy-services: stopping
[tcp] 2024/02/26 13:23:20 tcp.Mux: Listener at 127.0.0.1:8088 failed failed to accept a connection, closing all listeners - accept tcp 127.0.0.1:8088: use of closed network connection
[13:23:20] INFO: Chronograf stopped, restarting...
[13:23:20] INFO: InfluxDB stopped, restarting...
[13:23:20] INFO: Kapacitor stopped, restarting...
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service base-addon-timezone: stopping
s6-rc: info: service base-addon-log-level: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service base-addon-timezone successfully stopped
s6-rc: info: service base-addon-log-level successfully stopped
s6-rc: info: service base-addon-banner: stopping
s6-rc: info: service base-addon-banner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

@michaelbenoit
Copy link

Same problem for me

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting

-----------------------------------------------------------
 Add-on: InfluxDB
 Scalable datastore for metrics, events, and real-time analytics
-----------------------------------------------------------
 Add-on version: 5.0.0
 You are running the latest version of this add-on.
 System: Debian GNU/Linux 12 (bookworm)  (aarch64 / raspberrypi5-64)
 Home Assistant Core: 2024.2.4
 Home Assistant Supervisor: 2024.02.0
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-timezone: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
[20:00:59] INFO: Configuring timezone (Europe/Berlin)...
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service base-addon-timezone successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/create-users.sh
cont-init: info: /etc/cont-init.d/create-users.sh exited 0
cont-init: info: running /etc/cont-init.d/influxdb.sh
cont-init: info: /etc/cont-init.d/influxdb.sh exited 0
cont-init: info: running /etc/cont-init.d/kapacitor.sh
cont-init: info: /etc/cont-init.d/kapacitor.sh exited 0
cont-init: info: running /etc/cont-init.d/nginx.sh
cont-init: info: /etc/cont-init.d/nginx.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun chronograf (no readiness notification)
services-up: info: copying legacy longrun influxdb (no readiness notification)
services-up: info: copying legacy longrun kapacitor (no readiness notification)
services-up: info: copying legacy longrun nginx (no readiness notification)
[20:01:00] INFO: Kapacitor is waiting until InfluxDB is available...
[20:01:00] INFO: Chronograf is waiting until InfluxDB is available...
s6-rc: info: service legacy-services successfully started
[20:01:00] INFO: Starting the InfluxDB...
[20:01:11] INFO: Starting the Kapacitor
[20:01:11] INFO: Starting Chronograf...

'##:::'##::::'###::::'########:::::'###:::::'######::'####:'########::'#######::'########::
 ##::'##::::'## ##::: ##.... ##:::'## ##:::'##... ##:. ##::... ##..::'##.... ##: ##.... ##:
 ##:'##::::'##:. ##:: ##:::: ##::'##:. ##:: ##:::..::: ##::::: ##:::: ##:::: ##: ##:::: ##:
 #####::::'##:::. ##: ########::'##:::. ##: ##:::::::: ##::::: ##:::: ##:::: ##: ########::
 ##. ##::: #########: ##.....::: #########: ##:::::::: ##::::: ##:::: ##:::: ##: ##.. ##:::
 ##:. ##:: ##.... ##: ##:::::::: ##.... ##: ##::: ##:: ##::::: ##:::: ##:::: ##: ##::. ##::
 ##::. ##: ##:::: ##: ##:::::::: ##:::: ##:. ######::'####:::: ##::::. #######:: ##:::. ##:
..::::..::..:::::..::..:::::::::..:::::..:::......:::....:::::..::::::.......:::..:::::..::

2024/02/27 20:01:12 Using configuration at: /etc/kapacitor/kapacitor.conf
time="2024-02-27T20:01:17+01:00" level=info msg="Serving chronograf at http://127.0.0.1:8889" component=server
time="2024-02-27T20:01:17+01:00" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime"
[20:01:17] INFO: Starting NGINX...
2024/02/27 20:01:17 [crit] 405#405: ngx_slab_alloc() failed: no memory
[20:01:17] WARNING: NGINX crashed, halting add-on
[20:01:17] INFO: NGINX stopped, restarting...
s6-rc: info: service legacy-services: stopping
[tcp] 2024/02/27 20:01:17 tcp.Mux: Listener at 127.0.0.1:8088 failed failed to accept a connection, closing all listeners - accept tcp 127.0.0.1:8088: use of closed network connection
[20:01:17] INFO: Chronograf stopped, restarting...
[20:01:17] INFO: InfluxDB stopped, restarting...
[20:01:17] INFO: Kapacitor stopped, restarting...
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service base-addon-timezone: stopping
s6-rc: info: service base-addon-log-level: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service base-addon-timezone successfully stopped
s6-rc: info: service base-addon-log-level successfully stopped
s6-rc: info: service base-addon-banner: stopping
s6-rc: info: service base-addon-banner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

@sinclairpaul
Copy link
Member

I would suggest you look at issues against the RPI Bookworm repo, as all of these reports are the same OS/Hardware.

In particular you may want to try the suggestion here raspberrypi/bookworm-feedback#107 (comment)

@kloppy1984
Copy link

I try the suggestion

With Put "kernel=kernel8.img" in config.txt...
The Same Problem."ngx_slab_alloc() failed: no memory"

Copy link

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thanks!

@github-actions github-actions bot added the stale There has not been activity on this issue or PR for quite some time. label Mar 30, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Apr 7, 2024
@github-actions github-actions bot locked and limited conversation to collaborators May 8, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
stale There has not been activity on this issue or PR for quite some time.
Projects
None yet
Development

No branches or pull requests

4 participants