Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pod doesn't create #379

Closed
dubrsl opened this issue Dec 22, 2021 · 61 comments
Closed

Pod doesn't create #379

dubrsl opened this issue Dec 22, 2021 · 61 comments
Labels
bug Something isn't working

Comments

@dubrsl
Copy link
Contributor

dubrsl commented Dec 22, 2021

New version doesn't create pod. As a result, communication between containers within the same pod does not work.

To Reproduce
Steps to reproduce the behavior:

  1. Use folder https://github.com/containers/podman-compose/tree/devel/tests/nets_test1/
  2. podman-compose up -d

Expected behavior
On podman-compose-0.1.7-2.git20201120.el8.noarch

$ podman-compose up -d
using podman version: podman version 3.3.1
podman pod create --name=test --share net -p 8002:8001 -p 8001:8001
a39316fd5c054554053a08fa0e8c6de3a017207fbe82e405209bcc5c89370560
0
podman run --name=test_web1_1 -d --pod=test --label io.podman.compose.config-hash=123 --label io.podman.compose.project=test --label io.podman.compose.version=0.0.1 --label com.docker.compose.container-number=1 --label com.docker.compose.service=web1 -v /jenkins/test/./test1.txt:/var/www/html/index.txt:z,ro --add-host web1:127.0.0.1 --add-host test_web1_1:127.0.0.1 --add-host web2:127.0.0.1 --add-host test_web2_1:127.0.0.1 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
4fa6235e5e7d3a96763927d6edc7d3b9b60728213b455f421609cf1cffff156d
0
podman run --name=test_web2_1 -d --pod=test --label io.podman.compose.config-hash=123 --label io.podman.compose.project=test --label io.podman.compose.version=0.0.1 --label com.docker.compose.container-number=1 --label com.docker.compose.service=web2 -v /jenkins/test/./test2.txt:/var/www/html/index.txt:z,ro --add-host web1:127.0.0.1 --add-host test_web1_1:127.0.0.1 --add-host web2:127.0.0.1 --add-host test_web2_1:127.0.0.1 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
deda03442e957d2687cba6c7309c52c26e4842f3d1b0d65391d9d44617811a08
0

Actual behavior
On new version podman-composer
podman-composer version 1.0.3
podman version 3.4.2

$ podman-compose up -d
['podman', '--version', '']
using podman version: 3.4.2
** excluding:  set()
['podman', 'network', 'exists', 'nets_test1_default']
podman run --name=nets_test1_web1_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test1 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test1 --label com.docker.compose.project.working_dir=/home/slava/work/_GIT/podman-compose/tests/nets_test1 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web1 -v /home/slava/work/_GIT/podman-compose/tests/nets_test1/test1.txt:/var/www/html/index.txt:z,ro --net nets_test1_default --network-alias web1 -p 8001:8001 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
8b92a0e9a5f42261af182ce9e7a6864eaa8dacd36c70abfc834b7f2357644630
exit code: 0
['podman', 'network', 'exists', 'nets_test1_default']
podman run --name=nets_test1_web2_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test1 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test1 --label com.docker.compose.project.working_dir=/home/slava/work/_GIT/podman-compose/tests/nets_test1 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web2 -v /home/slava/work/_GIT/podman-compose/tests/nets_test1/test2.txt:/var/www/html/index.txt:z,ro --net nets_test1_default --network-alias web2 -p 8002:8001 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
44a3d9f6737629f863b4feaa5971c18df5ea5d83db334622ba45d23328487e09
exit code: 0

Environment:

  • OS: Linux Fedora 34

Please return the previous behavior.

@dubrsl dubrsl added the bug Something isn't working label Dec 22, 2021
@muayyad-alsadi
Copy link
Collaborator

I'm using fedora 34, and things work perfectly.

you can use wget to verify that.

@dubrsl
Copy link
Contributor Author

dubrsl commented Dec 22, 2021

I'm using fedora 34, and things work perfectly.

do you use podman-composer version 1.0.3 on fedora34?

@muayyad-alsadi
Copy link
Collaborator

do you use podman-composer version 1.0.3 on fedora34?

yes, here it's working fine, except that this test docker-compose.yml has a typo in

2c60516

connectivity check reaching web2 from web1:

$ podman-compose version
using podman version: 3.4.2
podman-composer version  1.0.3
...
$ podman-compose up -d
...
$ podman-compose exec web1  wget -q -O - http://web2:8001/index.txt
podman exec --interactive --tty nets_test1_web1_1 wget -q -O - http://web2:8001/index.txt
test2

and vice versa, reaching web1 from web2

$ podman-compose exec web2  wget -q -O - http://web1:8001/index.txt
podman exec --interactive --tty nets_test1_web2_1 wget -q -O - http://web1:8001/index.txt
test1

@pixelblut
Copy link

pixelblut commented Dec 23, 2021

I can confirm this behaviour on ArchLinux with podman-compose-git from the AUR.
Edit: It's the same behaviour regardless if i run podman-compose with sudo or not.

└─[$] podman-compose -v
['podman', '--version', '']
using podman version: 3.4.4
podman-composer version 1.0.3
podman --version
podman version 3.4.4
exit code: 0

I've tested it with the same testing docker-compose.yml you updated. But there is no pod being created, only the containers get deployed. In Previous Versions also a Pod was deployed, which provided the seperate Namespace for the Containers.

└─[$] podman-compose -f docker-compose.yml -p busybox_test up -d
['podman', '--version', '']
using podman version: 3.4.4
** excluding: set()
['podman', 'network', 'exists', 'busybox_test_default']
podman run --name=busybox_test_web1_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=busybox_test --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=busybox_test --label com.docker.compose.project.working_dir=/redacted/tests --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web1 -v /redacted/tests/test1.txt:/var/www/html/index.txt:z,ro --net busybox_test_default --network-alias web1 -p 8001:8001 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
c00134b66e8fd9e037ffbc17cdabef6e26a8cab524e3a046277d2bba056a82bf
exit code: 0
['podman', 'network', 'exists', 'busybox_test_default']
podman run --name=busybox_test_web2_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=busybox_test --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=busybox_test --label com.docker.compose.project.working_dir=/redacted/tests --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web2 -v /redacted/tests/test2.txt:/var/www/html/index.txt:z,ro --net busybox_test_default --network-alias web2 -p 8002:8001 -w /var/www/html --hostname web2 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
0f2652483b831ab128fdcc663b6f05cb732222dfba3ccc1f2a76e373c83dcfcd
exit code: 0

└─[$] podman pod ls
POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS

└─[$] podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c00134b66e8f docker.io/library/busybox:latest /bin/busybox http... 4 minutes ago Up 4 minutes ago 0.0.0.0:8001->8001/tcp busybox_test_web1_1
0f2652483b83 docker.io/library/busybox:latest /bin/busybox http... 4 minutes ago Up 4 minutes ago 0.0.0.0:8002->8001/tcp busybox_test_web2_1

Albeit the podman-compose exec command throws an KeyError on my end.

└─[$] podman-compose exec busybox_test_web1_1 wget -q -O - http://busybox_test_web2_1:8001/index.txt
['podman', '--version', '']
using podman version: 3.4.4
Traceback (most recent call last):
File "/usr/bin/podman-compose", line 33, in
sys.exit(load_entry_point('podman-compose==1.0.3', 'console_scripts', 'podman-compose')())
File "/usr/lib/python3.10/site-packages/podman_compose.py", line 1797, in main
podman_compose.run()
File "/usr/lib/python3.10/site-packages/podman_compose.py", line 1024, in run
cmd(self, args)
File "/usr/lib/python3.10/site-packages/podman_compose.py", line 1248, in wrapped
return func(*args, **kw)
File "/usr/lib/python3.10/site-packages/podman_compose.py", line 1554, in compose_exec
container_names=compose.container_names_by_service[args.service]
KeyError: 'busybox_test_web1_1'

@muayyad-alsadi
Copy link
Collaborator

@pixelblut yes, that's normal, why you need it to create a pod!
the question does it communicate.

Albeit the podman-compose exec command throws an KeyError on my end.
KeyError: 'busybox_test_web1_1'

exec takes service name not container name

podman-compose exec busybox_test_web1_1 wget -q -O - http://busybox_test_web2_1:8001/index.txt

should be

 podman-compose exec web1 wget -q -O - http://busybox_test_web2_1:8001/index.txt

In Previous Versions also a Pod was deployed, which provided the seperate Namespace for the Containers.

that was a workaround because legacy podman (slirp4netns in deed) do not have inter-container communication
we put all containers in a single put that shares the network, then make them communicate via localhost

if you need this behavior you can use 0.1.x

pip3 install 'podman-compose<1.0'

but this has many limitations

  • no control of order
  • no multiple ports (ex. a stack of mysql and mysql replica both running on same port)

@dubrsl
Copy link
Contributor Author

dubrsl commented Dec 23, 2021

In my case new version without pod doesn't work and I get the error: wget: bad address 'web2:8001'

$ podman-compose --version
['podman', '--version', '']
using podman version: 3.4.2
podman-composer version  1.0.3
podman --version 
podman version 3.4.2
exit code: 0
$ podman-compose up -d
['podman', '--version', '']
using podman version: 3.4.2
** excluding:  set()
['podman', 'network', 'exists', 'nets_test1_default']
['podman', 'network', 'create', '--label', 'io.podman.compose.project=nets_test1', '--label', 'com.docker.compose.project=nets_test1', 'nets_test1_default']
['podman', 'network', 'exists', 'nets_test1_default']
podman run --name=nets_test1_web1_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test1 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test1 --label com.docker.compose.project.working_dir=/home/slava/work/_GIT/podman-compose/tests/nets_test1 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web1 -v /home/slava/work/_GIT/podman-compose/tests/nets_test1/test1.txt:/var/www/html/index.txt:z,ro --net nets_test1_default --network-alias web1 -p 8001:8001 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
9324d10cbba791ea8451adf4dcf573f3a6279cc6bcb5610b501d2e7ae5d8bb2d
exit code: 0
['podman', 'network', 'exists', 'nets_test1_default']
podman run --name=nets_test1_web2_1 -d --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test1 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test1 --label com.docker.compose.project.working_dir=/home/slava/work/_GIT/podman-compose/tests/nets_test1 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web2 -v /home/slava/work/_GIT/podman-compose/tests/nets_test1/test2.txt:/var/www/html/index.txt:z,ro --net nets_test1_default --network-alias web2 -p 8002:8001 -w /var/www/html --hostname web2 busybox /bin/busybox httpd -f -h /var/www/html -p 8001
a3b93bf846f59688d067d0c5e043c6e8e096b354302b242da284169029770882
exit code: 0
$ podman-compose exec web1  wget -q -O - http://web2:8001/index.txt
['podman', '--version', '']
using podman version: 3.4.2
podman exec --interactive --tty nets_test1_web1_1 wget -q -O - http://web2:8001/index.txt
wget: bad address 'web2:8001'
exit code: 1

Seems it can't resolve the hostname of other container in one project.
@muayyad-alsadi Can you suggest the reasons? Thank you!

@muayyad-alsadi
Copy link
Collaborator

@dubrsl your setup is exactly the same as mine, but mine works.

do you use cni networks or slirp4netns?
did you customize anything of podman settings?
do you run as root or rootless?
did you carry old configs in your home or /etc rpmsave ..etc.

@dubrsl
Copy link
Contributor Author

dubrsl commented Dec 23, 2021

@muayyad-alsadi I found the problem. The package podman-plugins was not installed.
I was wrong and the absence of a pod seemed to me to be the reason for the lack of connection between containers.
Thank you for your work!

@muayyad-alsadi
Copy link
Collaborator

we need to update README.md

@dubrsl
Copy link
Contributor Author

dubrsl commented Dec 24, 2021

#382

@developedsoftware
Copy link

that was a workaround because legacy podman (slirp4netns in deed) do not have inter-container communication
we put all containers in a single put that shares the network, then make them communicate via localhost
if you need this behavior you can use 0.1.x

So how we do tell podman-compose to create the pod? (in the event where we do want all the containers in one pod and communicating via localhost). Surely downgrading to podman-compose<1.0 isnt the only solution?

@muayyad-alsadi
Copy link
Collaborator

the question why do you need to put all containers in a single pod? it was a workaround, because when we started podman-compose we could not use podman network create

there are many downsides of putting containers in a single pod with a shared network (ex. hostnames, could not run two mysql daemon on same port ...etc.)

what is the benefit of maintaining the workaround? it was a workaround, until we get podman network create and when we got it we dropped the work around.

@developedsoftware
Copy link

developedsoftware commented Jan 18, 2022

Because inside my containers I want to communicate over localhost. If I am able to specify the network inside the yml file then this should be possible.

But by putting the containers in a single pod, localhost is the default method of communication? So its just simpler.

There may be legitimate reasons to put the containers in a pod? Do you have an alternative suggestion to allow communication between containers over localhost?

@muayyad-alsadi
Copy link
Collaborator

@developedsoftware I've just create a ticket for that. I'll push solution ASAP

@developedsoftware
Copy link

Legend! Thank you!

@developedsoftware
Copy link

developedsoftware commented Jan 18, 2022

Just a small update - resolved the issues with containers not being in a pod by installing the dnsmask plugin. Which is described as being optional. Now I can communicate using the hostnames.

However, I do feel that we should have a way of telling podman-compose to stick all the containers into a pod and allow them to communicate over localhost.

This is described is many podman tutorials as the quickest way to get an environment up and running - and also does not require the dnsmask plugin - so podman-compose should allow for this (in my opinion - thoughts?)

https://www.redhat.com/sysadmin/container-networking-podman

@muayyad-alsadi
Copy link
Collaborator

muayyad-alsadi commented Jan 18, 2022

here in the readme, we don't say it's optional

Screenshot from 2022-01-18 16-22-04

@developedsoftware
Copy link

It says "Optional Requirement" next to podman dnsname plugin

@muayyad-alsadi
Copy link
Collaborator

aha, he meant by optional that it's not pulled by default with podman package in the repos and you need to add it by hand
I'll update the readme to remove the confusion.

@developedsoftware
Copy link

OK, sounds good. Thank you!

@muayyad-alsadi
Copy link
Collaborator

@developedsoftware I've pushed 407, please test

pip3 install --user https://github.com/containers/podman-compose/archive/devel.tar.gz

@developedsoftware
Copy link

developedsoftware commented Jan 18, 2022

Will do, thanks! What does this push do?

Do I need to alter my yml file in any way?

@cognition9144
Copy link

For others who also get confused, the dnsname plugin won't create entries in /etc/hosts. Rather, it resolves other sevices via dnsmasq.

@developedsoftware
Copy link

@developedsoftware you can use this

https://github.com/containers/podman-compose/blob/devel/tests/nets_test3/docker-compose.yml

@xcffl this is covered by the dns plugin as in the readme

Forgive my ignorance, but that compose file has no reference to a pod?

@muayyad-alsadi
Copy link
Collaborator

Forgive my ignorance, but that compose file has no reference to a pod?

can you give me example of a compose that does?

There are some setups whereby you would want all containers to be within 1 pod?
This is useful if I eventually want to convert the pod into a set of systemd services. That requires them to be managed by a pod service.
please test specially with things that does not share non-network spaces ex. mount ns (volumes), user namespaces ...etc

I'll revert that, because I got

Error: invalid config provided: cannot set hostname when joining the pod UTS namespace: invalid configuration
exit code: 125

when doing podman-compose up for nets_test3

podman pod create --name=pod_nets_test3
podman network create nets_test3_net1
podman run --name=nets_test3_web1_1 -d --pod=pod_nets_test3  --net nets_test3_net1 --network-alias web1 --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001

@muayyad-alsadi
Copy link
Collaborator

now nets_test3 worked fine

$ podman_compose.py up -d
podman pod create --name=pod_nets_test3 --share=
podman network create nets_test3_net1
podman network create nets_test3_net2

podman run --name=nets_test3_web1_1 -d --pod=pod_nets_test3 --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test3 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test3 --label com.docker.compose.project.working_dir=/home/alsadi/proj/podman-compose/tests/nets_test3 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web1 -v /home/alsadi/proj/podman-compose/tests/nets_test3/test1.txt:/var/www/html/index.txt:z,ro --net nets_test3_net1 --network-alias web1 -p 8001:8001 -w /var/www/html --hostname web1 busybox /bin/busybox httpd -f -h /var/www/html -p 8001

podman run --name=nets_test3_web2_1 -d --pod=pod_nets_test3 --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test3 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test3 --label com.docker.compose.project.working_dir=/home/alsadi/proj/podman-compose/tests/nets_test3 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web2 -v /home/alsadi/proj/podman-compose/tests/nets_test3/test2.txt:/var/www/html/index.txt:z,ro --net nets_test3_net1,nets_test3_net2 --network-alias web2 -p 8002:8001 -w /var/www/html --hostname web2 busybox /bin/busybox httpd -f -h /var/www/html -p 8001

podman run --name=nets_test3_web3_1 -d --pod=pod_nets_test3 --label io.podman.compose.config-hash=123 --label io.podman.compose.project=nets_test3 --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=nets_test3 --label com.docker.compose.project.working_dir=/home/alsadi/proj/podman-compose/tests/nets_test3 --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=web3 -v /home/alsadi/proj/podman-compose/tests/nets_test3/test2.txt:/var/www/html/index.txt:z,ro --net nets_test3_net1,nets_test3_net2 --network-alias web3,alias11,alias12,alias21 -w /var/www/html busybox /bin/busybox httpd -f -h /var/www/html -p 8001

Forgive my ignorance, but that compose file has no reference to a pod?

as you can see, it's always created

@developedsoftware
Copy link

Apologies - I understand it will not be part of the compose spec. I guess my question should be

"What do you use to decide if 2 containers should be in the same pod?"

Assume its the network namespace?

@muayyad-alsadi
Copy link
Collaborator

muayyad-alsadi commented Feb 17, 2022

"What do you use to decide if 2 containers should be in the same pod?"

the latest devel branch 1.0.4 puts all containers in the compose in one pod (you don't have to specify any thing manually)

so that you can use

podman pod ls
podman pod logs -f <pod_name>
podman generate systemd <pod_name>

muayyad-alsadi added a commit that referenced this issue Feb 17, 2022
@WuSiYu
Copy link

WuSiYu commented Feb 23, 2022

"What do you use to decide if 2 containers should be in the same pod?"

the latest devel branch 1.0.4 puts all containers in the compose in one pod (you don't have to specify any thing manually)

so that you can use

podman pod ls
podman pod logs -f <pod_name>
podman generate systemd <pod_name>

That is my expect way to management the "docker-compose.yaml based" application on my local home server, just like podman-compose <1.0 version.

But now when I using podman-compose 1.0.4 (installed from pip3 install https://github.com/containers/podman-compose/archive/devel.tar.gz), the pod has successful created, however when I run podman generate systemd xxx, it failed with "Error: error generating systemd unit files: Pod "xxxx" has no infra container".

I know the infra or the pod is may unnecessary, but I think it's good to have an option to switch between "create split containers" and "create pod and other staff like podman-compose <1.0".

@muayyad-alsadi
Copy link
Collaborator

"Error: error generating systemd unit files: Pod "xxxx" has no infra container".

the reason there is no infra container, that when I created it with infra but with share nothing, the status of the pod in "podman ps" showed as "degraded". I'm not sure if this is a bug in podman or not.

you can comment line 1477 "--infra=false", and create the infra
and then down and up then see the output of "podman pod ps"

@muayyad-alsadi
Copy link
Collaborator

@WuSiYu please check this

#442 (comment)

podman-compose --no-pod up -d
# or
podman-compose --pod-args='--infra=false --share=""' up -d
# or
podman-compose --pod-args='--infra=true --share=""' up -d

@bertmelis
Copy link

What is the rationale behind not creating an infra container in a pod by default?

@muayyad-alsadi
Copy link
Collaborator

our containers in the pods shares nothing --share=""

  • they can't share the network namespace because two containers might listen to same port
  • they can't share uts because two containers might have different hostnames

if we do --share="" with an infra container it will appear Degraded and it won't start the infra

STRs

$ podman pod create --name=mypod --share=""
93e43419780832f79579ef01f21ac5ba61020bec95db6c3fdf13e2b1b0f5f836
$ podman run -d --name mycontainer1 --pod=mypod busybox httpd -f -p 8123 -h /etc/
243d0b36fb38b8bc55d760ca4756b5ebf29f3a67e561d54042a7ecd99b0f4a09
$ podman pod ls
POD ID        NAME          STATUS      CREATED        INFRA ID      # OF CONTAINERS
93e434197808  mypod         Degraded    2 minutes ago  1e7e9644d101  2
$ podman ps -a -f 'pod=mypod'
CONTAINER ID  IMAGE                                    COMMAND               CREATED        STATUS            PORTS       NAMES
1e7e9644d101  localhost/podman-pause:4.1.0-1653926608                        4 minutes ago  Created                       93e434197808-infra
243d0b36fb38  docker.io/library/busybox:latest         httpd -f -p 8123 ...  3 minutes ago  Up 3 minutes ago              mycontainer1

the reason it's Degraded is that for some reason podman did not start the infra in case of it's share nothing.

@bertmelis
Copy link

bertmelis commented Jul 24, 2022

That leaves the next question: why does it by default share nothing?

https://docs.podman.io/en/latest/markdown/podman-pod-create.1.html#share-namespace

I have the feeling that at some point there has to be a discussing about whether podman-compose has to be 100% compatibel with Docker or it has to shift into the direction of Podman's specific behaviour.

Don't get me wrong, I'm not experiencing this as a real issue as this is all easily solved by some extra command line arguments.

@muayyad-alsadi
Copy link
Collaborator

because most of compose stack expects the network and uts not to be shared, for example

https://github.com/containers/podman-compose/blob/devel/examples/awx3/docker-compose.yml#L27

this stack defines hostname: awxweb for one container and hostname: awx for another container.
even if the compose does set such values, one might use podman-compose run

run                 create a container similar to a service to run a one-off command

the way compose is defined does not allow for shared network nor shared uts and maybe does not allow for uts

@muayyad-alsadi
Copy link
Collaborator

I've reported it upstream

containers/podman#15048

@developedsoftware
Copy link

Looks like we now have a solution via the upstream chat?

@martdj
Copy link

martdj commented Aug 20, 2022

Could it be that if your host is running a DNS server (bind in my case), the podman dns plugin won't work? I can't create a network and 2 podman containers and have the containers find each other by container name. It's simply not working, regardless of the podman-plugins package being installed (CentOS 9 stream). I wonder if the DNS server is the cause.

@leonidas-o
Copy link

Could it be that if your host is running a DNS server (bind in my case), the podman dns plugin won't work? I can't create a network and 2 podman containers and have the containers find each other by container name. It's simply not working, regardless of the podman-plugins package being installed (CentOS 9 stream). I wonder if the DNS server is the cause.

@martdj sounds strange that your bind service on the host should interfere with dns resolution inside podmans networks. Maybe it's just not correctly setup. I posted in here how I use it, maybe this can help you: #541 (comment)

@martdj
Copy link

martdj commented Aug 20, 2022

Could it be that if your host is running a DNS server (bind in my case), the podman dns plugin won't work? I can't create a network and 2 podman containers and have the containers find each other by container name. It's simply not working, regardless of the podman-plugins package being installed (CentOS 9 stream). I wonder if the DNS server is the cause.

@martdj sounds strange that your bind service on the host should interfere with dns resolution inside podmans networks. Maybe it's just not correctly setup. I posted in here how I use it, maybe this can help you: #541 (comment)

Looks clear enough, but somehow it doesn't work for me.

# podman network create somenet
somenet
# podman network inspect somenet
[
     {
          "name": "somenet",
          "id": "cb150480a0cca762c52413c073231b4ecd9a114e0ff0f15c10da8d7d38585d39",
          "driver": "bridge",
          "network_interface": "podman5",
          "created": "2022-08-21T00:59:26.965190847+02:00",
          "subnets": [
               {
                    "subnet": "10.89.4.0/24",
                    "gateway": "10.89.4.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": true,
          "ipam_options": {
               "driver": "host-local"
          }
     }
]
# podman run -it --name alma1 --network somenet almalinux
[root@0a6b9ddf8c5e /]#
# podman run -it --name alma2 --network somenet almalinux
[root@62bb7a8611de /]`# ping alma1
PING myhostname.domain.lan (192.168.7.150) 56(84) bytes of data.
64 bytes from myhostname.domain.lan.7.168.192.in-addr.arpa (192.168.7.150): icmp_seq=1 ttl=64 time=0.035 ms

Am I missing something obvious?

@muayyad-alsadi
Copy link
Collaborator

depending on your podman version you need dns plugin
if you are using podman 3 make sure podman-plugins or podman-dnsname are installed
if you are using podman 4 make sure they are not installed but instead netavark is installed

podman info | grep -i network
  networkBackend: netavark

@martdj
Copy link

martdj commented Aug 21, 2022

netavark and aardvark-dns are installed. The podman-plugins is also installed, but it's version 4.1.1-6.el9. Am I wrong in assuming that seeing the plugins package matches the version of podman, it will install what's necessary for that podman version? If I would uninstall it, it would also uninstall podman-compose as it depends on it

The networkBackend of my installation is indeed netavark btw

@muayyad-alsadi
Copy link
Collaborator

we got reports the dns resolution no longer work in podman 4.x with CNI dns plugins.
if both are installed it will use CNI

it only works with netavark and aardvark-dns, make sure they are installed and properly configured.

you might need to override config to point to netavark

you might need to do "down/up", recreate the containers and the network

you might need to reset your system maybe with

podman system reset --force

@martdj
Copy link

martdj commented Aug 24, 2022

Thanks for the help. I'm not so sure that running your own DNS server isn't an issue for podman and podman network. I exectuted the podman system reset --force, but since then my nextcloud container is giving me trouble. "podman container ls" is hanging if it runs. Also, when I create a net network (podman network create testing), and try to create a pod on that network (podman run -it --network-testing --name=busy1 busybox, the process hangs.
As executed services I see:
root 28237 28196 0 14:02 pts/4 00:00:00 /usr/libexec/podman/netavark --config /run/containers/networks --rootless=false --aardvark-binary=/usr/libexec/podman/aardvark-dns setup /run/netns/netns-697dc5ef-9dc9-dd65-d8df-c89cca2349dd
root 28306 28237 0 14:02 pts/4 00:00:00 /usr/libexec/podman/aardvark-dns --config /run/containers/networks/aardvark-dns -p 53 run

That, to me, looks very much like podman tries to start a service on port 53. A port that's taken when you run your own DNS server...

@martdj
Copy link

martdj commented Aug 24, 2022

I guess this says it all:
[root@zeus ~]# podman run -it --network=testing --name busy2 busybox ping busy1
PING busy1 (192.168.7.150): 56 data bytes
64 bytes from 192.168.7.150: seq=0 ttl=64 time=0.039 ms

[root@zeus ~]# systemctl stop named

[root@zeus ~]# podman run -it --replace --network=testing --name busy2 busybox ping busy1
PING busy1 (10.89.0.2): 56 data bytes
64 bytes from 10.89.0.2: seq=0 ttl=64 time=0.037 ms
64 bytes from 10.89.0.2: seq=1 ttl=64 time=0.051 ms

This is a bug in podman itself guys. I'll create a separate issue

@martdj
Copy link

martdj commented Aug 24, 2022

I got a reply at my bug report. Apparently this is solved in Podman 4.2 and Netavark/Aardvark 1.1.0. For readers on Red Hat 9 / CentOS 9 Stream and a DNS server on their host, I guess it's good to know they can't use podman-compose to connect containers, while they're not yet on those versions (CentOS Stream currently uses 4.1.1)

@leonidas-o
Copy link

@developedsoftware I've pushed 407, please test

pip3 install --user https://github.com/containers/podman-compose/archive/devel.tar.gz

@muayyad-alsadi can you (or someone else as well) provide a next podman-compose release. Right now the behaviour between 1.0.3 and the above mentioned devel package (1.0.4) changed significantly in terms of pod creation. Also Using this approach will install what exactly? The latest sources from dev branch, the package which was provided at that day you posted the link? Don't know what exactly is behind it.

@ivansmm
Copy link

ivansmm commented Jan 24, 2023

Hallo, I tried the package referred to in last message and I observe that pod is created without specifying the network. Network is still specified per each container individually. Normally in case of pods network is attached to pod and necessary ports are published for the pod as a whole. It looks like specifying same network for each container in pod does not create a common localhost.
Also I noticed that network is created even if --dry-run is specified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

10 participants