Skip to content
This repository has been archived by the owner on Nov 27, 2023. It is now read-only.

Not able to stop or restart the containers present in same network after dnsname plugin installation #67

Closed
snd94 opened this issue May 12, 2021 · 7 comments · Fixed by #73

Comments

@snd94
Copy link

snd94 commented May 12, 2021

After installing the dnsname plugin built manually from the github(https://github.com/containers/dnsname) facing issue in podman stop or restart.
Machine: RHEL beta 8.4
ERROR:
ERRO[0000] unable to cleanup network for container 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129: "error tearing down CNI namespace configuration for container 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129: Error while removing pod from CNI network "dockerartifacts_service": stat /run/user/0/containers/cni/dnsname/dockerartifacts_service: no such file or directory"
ERRO[0000] Error adding network: failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed
ERRO[0000] Error while adding pod to CNI network "dockerartifacts_service": failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed
Error: error configuring network namespace for container 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129: failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed
[root@INBLRGHMAN136LN ~]# podman restart 6505310e93d4
ERRO[0000] Error adding network: failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed
ERRO[0000] Error while adding pod to CNI network "dockerartifacts_service": failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed
Error: error configuring network namespace for container 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129: failed to allocate for range 0: 10.89.0.3 has been allocated to 6505310e93d4d4587d779fcb882a6266098d39a96ed6052b42a146ac0b11c129, duplicate allocation is not allowed

Looks like network is not cleaned up properly /var/lib/cni/networks/. Without the dnsname plugin ip present under this location gets deleted and new one will be getting created, but with this plugin clean up has some issue.

Need some work around for this issue.
Note: Without the dnsname plugin podman start or restart/stop works fine but inter container communication with the continainer works only when dnsname plugin in installed.

@baude
Copy link
Member

baude commented May 12, 2021

@Luap99 have you ever used dnsname with rootless? i dont think it will work but I have been completely swamped in podman machine these days. What say you.

@snd94
Copy link
Author

snd94 commented May 12, 2021

@baude I am currently using podman version 3.0.2-dev and it is rootfull. Could you please suggest me the work around to fix this issue if there are any... it will be very helpful??

@Luap99
Copy link
Member

Luap99 commented May 12, 2021

@snd94 Do you have the XDG_RUNTIME_DIR envar set to /run/user/0 but started the container without this envar? I think dnsname looks at the wrong location because it tries to use XDG_RUNTIME_DIR when present and /run if not.

@Luap99
Copy link
Member

Luap99 commented May 12, 2021

You can remove the offending ip files under /var/lib/cni/networks/<netname>/... if you are sure that no other container is using this ip at the moment.

@snd94
Copy link
Author

snd94 commented May 12, 2021

@Luap99 I could see ENV variable XDG_RUNTIME_DIR set to /run/user/0
Yes if i remove those IP's from the dir /var/lib/cni/networks//... it works for the next stop/restart, this is kind of manual process. Is there any way to do the network cleanup instead of manually deleting the IPs

@Luap99
Copy link
Member

Luap99 commented May 12, 2021

@snd94 Does the cleanup fail every time? Can you please provide the exact commands to reproduce.

@snd94
Copy link
Author

snd94 commented May 13, 2021

@Luap99 It is sporadic issue after deleting the ip file for some time, since i have 2 containers running on the same network. Based on the error that contains the Ip address need to delete the file from the network path. Only deletion of the file manually is the only solution for this right?

My doubt is is it fine to delete the all the IP's from /var/lib/cni/networks//... before doing restart or stop operation on any of the container, does it have any impact on the other container which uses the Ip address present over there?

Either with podman restart containerid or podman stop containerid will throw the same error.

Luap99 added a commit to Luap99/dnsname that referenced this issue Aug 11, 2021
According to the cni spec[1], a plugin should not return an error on
del. If a plugin returns an error cni will not call the following
plugins. Since the plugins are invoked in reverse order on del, the
portmapping and bridge plugins won't get invoked and therefore leave
iptables rules around.

Fixes containers#67
Fixes containers/podman#10806
Fixes containers/podman#10745

[1] https://github.com/containernetworking/cni/blob/master/SPEC.md#del-remove-container-from-network-or-un-apply-modifications

Signed-off-by: Paul Holzinger <[email protected]>
Luap99 added a commit to Luap99/dnsname that referenced this issue Aug 11, 2021
According to the cni spec[1], a plugin should not return an error on
del. If a plugin returns an error cni will not call the following
plugins. Since the plugins are invoked in reverse order on del, the
portmapping and bridge plugins won't get invoked and therefore leave
iptables rules around.

Fixes containers#67
Fixes containers/podman#10806
Fixes containers/podman#10745

[1] https://github.com/containernetworking/cni/blob/master/SPEC.md#del-remove-container-from-network-or-un-apply-modifications

Signed-off-by: Paul Holzinger <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants