Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hang with Erlang Runtime in Kubernetes Pod #7523

Closed
iutx opened this issue Aug 2, 2023 · 2 comments
Closed

Hang with Erlang Runtime in Kubernetes Pod #7523

iutx opened this issue Aug 2, 2023 · 2 comments
Labels
bug Issue is reported as a bug

Comments

@iutx
Copy link

iutx commented Aug 2, 2023

Describe the bug

We have created a container image with the Erlang environment, and when running it with nerdctl run, we can successfully start it using containerd and execute erl commands smoothly.
However, when we deploy the same image using Kubernetes and execute the erl command inside the pod, it hangs indefinitely. Additionally, when attempting commands involving Erlang, such as rabbitmq-plugins list, we receive an error message similar to erl_child_setup failed with error 32 on line 240.

So far, we have tried enabling privileged containers by using securityContext in the pod specification, but unfortunately, this did not resolve the issue.

Any help or suggestions you can provide would be greatly appreciated. Thank you!

To Reproduce
Dockerfile keywords (From rabbitmq Dockerfile)

Unfortunately, this is a considerably outdated version, and we are actively working towards upgrading it.

FROM centos:7
...
COPY ./erlang-20.1.7.1-1.el7.centos.x86_64.rpm /
RUN yum install -y gcc glibc-devel make ncurses-devel openssl-devel xmlto perl wget xz lsof socat logrotate net-tools
RUN rpm -ivh /erlang-20.1.7.1-1.el7.centos.x86_64.rpm
...
RUN ln -sf /var/lib/rabbitmq/.erlang.cookie /root/
...

Expected behavior
Use erl in Pod.

Affected versions
The OTP versions that are affected by this bug.

Additional context
Add any other context about the problem here. If you wish to attach Erlang code you can either write it directly in the post using code tags, create a gist, or attach it as a zip file to this post.

@iutx iutx added the bug Issue is reported as a bug label Aug 2, 2023
@garazdawi
Copy link
Contributor

Getting erl_child_setup failed with error 32 on line 240. (i.e. EPIPE) suggests that Erlang has terminated abruptly for some reason. Are any linux core files generated? If not, can you generate either an erl_crash.dump or a linux core file and share its contents?

@iutx
Copy link
Author

iutx commented Aug 3, 2023

Getting erl_child_setup failed with error 32 on line 240. (i.e. EPIPE) suggests that Erlang has terminated abruptly for some reason. Are any linux core files generated? If not, can you generate either an erl_crash.dump or a linux core file and share its contents?

Thank you😄, we have identified the issue. It is not directly related to Erlang or RabbitMQ but rather specific to the configuration in containerd. In the Kylin V10 operating system, configuring containerd service with 'infinity' causes problems with the containers.
e.g.

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity

related issues:
systemd/systemd#6559
docker/for-linux#73
docker-library/mysql#579

Once again, thank you for your prompt response to the issue!

@iutx iutx closed this as completed Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue is reported as a bug
Projects
None yet
Development

No branches or pull requests

2 participants