Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Podman panics when creating network while network with static IP exists #13124

Closed
mfschumann opened this issue Feb 2, 2022 · 9 comments · Fixed by containers/common#965
Closed

Podman panics when creating network while network with static IP exists #13124

mfschumann opened this issue Feb 2, 2022 · 9 comments · Fixed by containers/common#965
Assignees
@Luap99
Labels
In Progress This issue is actively being worked by the assignee, please do not work on this at this time. kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@mfschumann
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I am using two networks:

  1. The default network defined in 87-podman.conflist.
  2. A network with a static address:
"ipam": {
            "type": "static",
            "routes": [
               {
                  "dst": "0.0.0.0/0"
               }
            ],
            "addresses": [
               [
                  {
                     "subnet": "10.89.0.89/16",
                     "gateway": "10.89.0.1"
                  }
               ]
            ]
         }

When trying to create a third network with podman network create net2 podman panics:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x555fc36ac309]

goroutine 1 [running]:
panic(0x555fc422ac00, 0x555fc4fe3a90)
	/usr/lib/golang/src/runtime/panic.go:1065 +0x565 fp=0xc0005e5138 sp=0xc0005e5070 pc=0x555fc29e60a5
runtime.panicmem()
	/usr/lib/golang/src/runtime/panic.go:212 +0x5d fp=0xc0005e5158 sp=0xc0005e5138 pc=0x555fc29e409d
runtime.sigpanic()
	/usr/lib/golang/src/runtime/signal_unix.go:734 +0x185 fp=0xc0005e5190 sp=0xc0005e5158 pc=0x555fc29fdfc5
github.com/containers/podman/libpod/network.allocatorToIPNets(0xc000472160, 0x2, 0x2, 0x39, 0xc0005e5440, 0x1)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/libpod/network/network.go:121 +0x329 fp=0xc0005e5390 sp=0xc0005e5190 pc=0x555fc36ac309
github.com/containers/podman/libpod/network.GetFreeNetwork(0xc0000b4b00, 0x555fc3ddc683, 0x6, 0x0)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/libpod/network/network.go:93 +0x27a fp=0xc0005e5470 sp=0xc0005e5390 pc=0x555fc36abcba
github.com/containers/podman/libpod/network.createBridge(0x7ffe1bc9beb8, 0x4, 0x0, 0x555fc3ddc683, 0x6, 0x0, 0x0, 0x0, 0x0, 0xc00052e900, ...)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/libpod/network/create.go:145 +0x15c5 fp=0xc0005e59a8 sp=0xc0005e5470 pc=0x555fc36a66e5
github.com/containers/podman/libpod/network.Create(0x7ffe1bc9beb8, 0x4, 0x0, 0x555fc3ddc683, 0x6, 0x0, 0x0, 0x0, 0x0, 0xc00052e900, ...)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/libpod/network/create.go:34 +0x2c8 fp=0xc0005e5ad8 sp=0xc0005e59a8 pc=0x555fc36a4868
github.com/containers/podman/pkg/domain/infra/abi.(*ContainerEngine).NetworkCreate(0xc000136168, 0x555fc44bf908, 0xc0002d2180, 0x7ffe1bc9beb8, 0x4, 0x0, 0x555fc3ddc683, 0x6, 0x0, 0x0, ...)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/pkg/domain/infra/abi/network.go:142 +0xe5 fp=0xc0005e5bd8 sp=0xc0005e5ad8 pc=0x555fc3c12245
github.com/containers/podman/cmd/podman/networks.networkCreate(0x555fc500bac0, 0xc000049a70, 0x1, 0x1, 0x0, 0x0)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/cmd/podman/networks/create.go:110 +0x1d9 fp=0xc0005e5d28 sp=0xc0005e5bd8 pc=0x555fc3d9fd59
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).execute(0x555fc500bac0, 0xc00013a030, 0x1, 0x1, 0x555fc500bac0, 0xc00013a030)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:856 +0x472 fp=0xc0005e5de8 sp=0xc0005e5d28 pc=0x555fc30409d2
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x555fc5015ac0, 0xc000134040, 0x555fc4227300, 0x555fc50d8608)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:974 +0x375 fp=0xc0005e5ec8 sp=0xc0005e5de8 pc=0x555fc30415f5
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:902
github.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteContext(...)
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:895
main.Execute()
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/cmd/podman/root.go:91 +0xe9 fp=0xc0005e5f48 sp=0xc0005e5ec8 pc=0x555fc3dcbde9
main.main()
	/builddir/build/BUILD/podman-3.4.4/_build/src/github.com/containers/podman/cmd/podman/main.go:39 +0x94 fp=0xc0005e5f88 sp=0xc0005e5f48 pc=0x555fc3dcb6f4
runtime.main()
	/usr/lib/golang/src/runtime/proc.go:225 +0x263 fp=0xc0005e5fe0 sp=0xc0005e5f88 pc=0x555fc29e8d23
runtime.goexit()
	/usr/lib/golang/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005e5fe8 sp=0xc0005e5fe0 pc=0x555fc2a1ecc1

Steps to reproduce the issue:

  1. podman network create net1

  2. Modify the ipam block in /etc/cni/net.d/net1.conflist to use a static address (see Description)

  3. podman network create net2

Describe the results you received:
Podman panics, no network is created.

Describe the results you expected:
Network net2 is created.

Additional information you deem important (e.g. issue happens only occasionally):
This issue might be related to #12990.

Output of podman version:

Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.16.8
Built:        Wed Dec  8 12:45:07 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: iot
    version: "35"
  eventLogger: journald
  hostname: fiot.fritz.box
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.16-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 2143547392
  memTotal: 3047059456
  ociRuntime:
    name: crun
    package: crun-1.4.1-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.1
      commit: 802613580a3f25a88105ce4b78126202fef51dfb
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 3046109184
  swapTotal: 3046109184
  uptime: 60h 45m 57.53s (Approximately 2.50 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 12:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.4.4-1.fc35.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
I have observed the bug in Fedora 35 Server on bare metal and in Fedora 35 Server and IoT in a VM.
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Feb 2, 2022
@mfschumann
Copy link
Author

I've had a quick shot with podman 4.0.0-rc1: The panic does not occur using this version, but the IPAM static config is rejected with an error: ERRO[0000] CNI config list /etc/cni/net.d/net1.conflist could not be converted to a libpod config, skipping: unsupported ipam plugin static in /etc/cni/net.d/net1.conflist. So the network with static IPs is not actually present when trying to create the third network. Should I file this behavior as a separate issue against 4.0.0-rc1?

@Luap99
Copy link
Member

Luap99 commented Feb 3, 2022

Why do you use the static ipam plugin? With podman 4.0 you should be able to set multiple static ip address per container with the host-local plugin.

Nevertheless we need to fix it, it is basically the same issue as #12990. The goal was to make unknown configs still work so this should work. Apparently I overlooked the case where people set a different ipam plugin.

@mfschumann
Copy link
Author

I am running podman 3.4 and the static ipam plugin was the simplest way to be found for setting a static IP.

I think this issue should be fixed both in podman 3.x and 4.0 (unless the static ipam plugin is going to be deprecated in podman 4.0).

@Luap99
Copy link
Member

Luap99 commented Feb 4, 2022

3.4 will not get further updates

@Luap99
Copy link
Member

Luap99 commented Feb 4, 2022

With podman 3.4 you can set a static address with --ip (only works for one network)
With 4.0 you can use --network <name>:ip=<ip> to set static ips per network. I think this is much better than having to create one cni config per container.

@mfschumann
Copy link
Author

Absolutely, thanks for the tips!

@github-actions
Copy link

github-actions bot commented Mar 9, 2022

A friendly reminder that this issue had no activity for 30 days.

@baude
Copy link
Member

baude commented Mar 9, 2022

@Luap99 is this fixed in 4.x? If so, lets close

@Luap99
Copy link
Member

Luap99 commented Mar 15, 2022

This is not fixed. It needs changes in c/common/libnetwork.

@Luap99 Luap99 self-assigned this Mar 15, 2022
@Luap99 Luap99 mentioned this issue Mar 15, 2022
Closed
@Luap99 Luap99 added the In Progress This issue is actively being worked by the assignee, please do not work on this at this time. label Mar 17, 2022
Luap99 added a commit to Luap99/common that referenced this issue Mar 17, 2022
@Luap99
libnetwork/cni: add support for arbitrary ipam plugins
9afb6a4 
When we parse CNI config files to convert them into a native Network
struct we should not error unsless there is something broken.
The goal is to allow all cni configs to still function but podman
inspect will not be able to show all informations such as subnets about
this network.

Because this is a valid use case we only log this at info level.

Fixes containers/podman#12990
Fixes containers/podman#13124

Signed-off-by: Paul Holzinger <[email protected]>
Luap99 added a commit to Luap99/common that referenced this issue Mar 17, 2022
@Luap99
libnetwork/cni: add support for arbitrary ipam plugins
2d5f753 
When we parse CNI config files to convert them into a native Network
struct we should not error unless there is something broken.
The goal is to allow all cni configs to still function but podman
inspect will not be able to show all informations such as subnets about
this network.

Because this is a valid use case we only log this at info level.

Fixes containers/podman#12990
Fixes containers/podman#13124

Signed-off-by: Paul Holzinger <[email protected]>
@Luap99 Luap99 mentioned this issue Mar 17, 2022
Merged
Luap99 added a commit to Luap99/common that referenced this issue Mar 17, 2022
@Luap99
libnetwork/cni: add support for arbitrary ipam plugins
87ecf67 
When we parse CNI config files to convert them into a native Network
struct we should not error unless there is something broken.
The goal is to allow all cni configs to still function but podman
inspect will not be able to show all informations such as subnets about
this network.

Because this is a valid use case we only log this at info level.

Fixes containers/podman#12990
Fixes containers/podman#13124

Signed-off-by: Paul Holzinger <[email protected]>
Luap99 added a commit to Luap99/common that referenced this issue Mar 17, 2022
@Luap99
libnetwork/cni: add support for arbitrary ipam plugins
24d5ab3 
When we parse CNI config files to convert them into a native Network
struct we should not error unless there is something broken.
The goal is to allow all cni configs to still function but podman
inspect will not be able to show all informations such as subnets about
this network.

Because this is a valid use case we only log this at info level.

Fixes containers/podman#12990
Fixes containers/podman#13124

Signed-off-by: Paul Holzinger <[email protected]>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@Luap99 Luap99

Labels
In Progress This issue is actively being worked by the assignee, please do not work on this at this time. kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

libnetwork/cni: add support for arbitrary ipam plugins Luap99/common
3 participants
@baude @mfschumann @Luap99