When running "podman pod rm -f " an error message: Error: error freeing lock for container xxxx no such file or directory

[MemoryShadow]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When running podman pod rm -f , an error message:

ERRO[0000] Removing container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816: no such file or directory
ERRO[0000] Removing container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603: no such file or directory
ERRO[0000] Removing container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226: no such file or directory
Error: error freeing lock for container 2ab66107ec395038b0d20c6cc9b2a50661c6b2152679c21eaf6fa9c96da348e9: no such file or directory

Steps to reproduce the issue:

1. Write resource inventory

# Save the output of this file and use kubectl create -f to import
# it into Kubernetes.
#
# Created with podman-4.2.0
apiVersion: v1
kind: Pod
metadata:
  name: mczlf.vipvp.top-pod
  labels:
    app: blog
spec:
  type: NodePort
  containers:
  - name: wordpress
    # https://hub.docker.com/_/wordpress
    image: docker.io/library/wordpress:6.0.2-php7.4-apache
    ports:
    - containerPort: 80
      hostPort: 8000
      protocol: TCP
    env:
    - name: WORDPRESS_DB_HOST
      value: "127.0.0.1"
    - name: WORDPRESS_DB_USER
      value: "MySQL_wp"
    - name: WORDPRESS_DB_PASSWORD
      value: "(!MySQL)000000"
    - name: WORDPRESS_DB_NAME
      value: "wp_db"
    - name: WORDPRESS_TABLE_PREFIX
      value: "wp_"
    volumeMounts:
    - name: wp-themes-data
      mountPath: /var/www/html/wp-content/themes/

  - name: mariadb
    # https://hub.docker.com/_/mariadb
    image: docker.io/library/mariadb:10.3.27
    ports:
    - containerPort: 3306
      hostPort: 3306
      protocol: TCP
    env:
    - name: MYSQL_ROOT_PASSWORD
      value: "(!MySQL)000000"
    - name: MYSQL_DATABASE
      value: "wp_db"
    - name: MYSQL_USER
      value: "MySQL_wp"
    - name: MYSQL_PASSWORD
      value: "(!MySQL)000000"
    startupProbe:
      exec:
        command: [
          "mysql","-p${MYSQL_ROOT_PASSWORD}","-e","CREATE DATABASE ${MYSQL_DATABASE}",
          "mysql","-p${MYSQL_ROOT_PASSWORD}","-e","GRANT ALL ON ${MYSQL_DATABASE}.* to '${MYSQL_USER}'@'127.0.0.1' IDENTIFIED BY '${MYSQL_PASSWORD}';",
          "mysql","-p${MYSQL_ROOT_PASSWORD}","-e","FLUSH privileges;"
          ]
    volumeMounts:
    - name: wp-db-data
      mountPath: /var/lib/mysql/

  - name: adminer
    # https://hub.docker.com/_/adminer
    image: docker.io/library/adminer
    ports:
    - containerPort: 8080
      hostPort: 8080
      protocol: TCP

  hostname: mczlf.vipvp.top
  restartPolicy: Never

  volumes:
  - name: wp-themes-data
    hostPath:
      path: /home/memoryshadow/Documents/podman/mczlf.vipvp.top/wpdata/wordpress-themes
  - name: wp-db-data
    hostPath:
      path: /home/memoryshadow/Documents/podman/mczlf.vipvp.top/wpdata/mariadb
status: {}

2. Run "podman play kube mczlf.vipvp.top-pod.yaml"

(I am currently unable to run this command because the Issue mentioned in this issue will occur)

3.Run "podman pod rm -f mczlf.vipvp.top-pod;"

ERRO[0000] Removing container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816: no such file or directory
ERRO[0000] Removing container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603: no such file or directory
ERRO[0000] Removing container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226: no such file or directory
Error: error freeing lock for container 2ab66107ec395038b0d20c6cc9b2a50661c6b2152679c21eaf6fa9c96da348e9: no such file or directory

Describe the results you received:

When you delete the pod instance after creating it with the resource manifest, you get the following result:

ERRO[0000] Removing container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 374979e860ddbcdf4267e6527f15e06db7a499a9a894a0e29a33dd6927a8a816: no such file or directory
ERRO[0000] Removing container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 62996e55bcc8c74c11f652a57f788399ad83a1460ddc8f2380b2a647871da603: no such file or directory
ERRO[0000] Removing container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226 from pod f26dcc556b094811564986ea8e3e3d86b9dd8ddd3a0ee7e18a4b6dacd90b12c7: error freeing lock for container 7f6665826cbb70da5d164b63a2bec9714a8d812fdd6e4450cc9c7445a4086226: no such file or directory
Error: error freeing lock for container 2ab66107ec395038b0d20c6cc9b2a50661c6b2152679c21eaf6fa9c96da348e9: no such file or directory

After executing "podman pod ls", the result is as follows:

POD ID        NAME                 STATUS      CREATED       INFRA ID      # OF CONTAINERS
f26dcc556b09  mczlf.vipvp.top-pod  Created     35 hours ago  2ab66107ec39  4

or

POD ID        NAME                 STATUS      CREATED       INFRA ID      # OF CONTAINERS
f26dcc556b09  mczlf.vipvp.top-pod  Error       36 hours ago  2ab66107ec39  4

Describe the results you expected:

The pod can be removed normally, and it would be better if we could find out the cause of this problem

Additional information you deem important (e.g. issue happens only occasionally):

This resource list has been used many times before in this environment, and it has worked well before. podman has been unable to continue since this problem occurred

Output of podman version:

Client:       Podman Engine
Version:      4.2.1
API Version:  4.2.1
Go Version:   go1.19
Git Commit:   62b324ddf718411b1d4d0ba8117c632f7f984a38-dirty
Built:        Thu Sep  8 14:52:54 2022
OS/Arch:      linux/amd64

Output of podman info:

host:
arch: amd64
buildahVersion: 1.27.0
cgroupControllers:
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: /usr/bin/conmon 由 conmon 1:2.1.4-1 所拥有path: /usr/bin/conmon
version: 'conmon version 2.1.4, commit: bd1459a3ffbb13eb552cc9af213e1f56f31ba2ee'
cpuUtilization:
idlePercent: 57.86
systemPercent: 6.12
userPercent: 36.02
cpus: 4
distribution:
distribution: manjaro
version: unknown
eventLogger: journald
hostname: ThinkPadT470p
idMappings:
gidmap:
- container_id: 0
host_id: 1001
size: 1
- container_id: 1
host_id: 10000
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 10000
size: 65536
kernel: 5.15.71-1-MANJARO
linkmode: dynamic
logDriver: journald
memFree: 16890650624
memTotal: 25079644160
networkBackend: netavark
ociRuntime:
name: crun
package: /usr/bin/crun 由 crun 1.6-1 所拥有path: /usr/bin/crun
version: |-
crun version 1.6
commit: 18cf2efbb8feb2b2f20e316520e0fd0b6c41ef4d
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
remoteSocket:
path: /run/user/1000/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /etc/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: /usr/bin/slirp4netns
package: /usr/bin/slirp4netns 由 slirp4netns 1.2.0-1 所拥有version: |-
slirp4netns version 1.2.0
commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
libslirp: 4.7.0
SLIRP_CONFIG_VERSION_MAX: 4
libseccomp: 2.5.4
swapFree: 0
swapTotal: 0
uptime: 35h 26m 28.00s (Approximately 1.46 days)
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
volume:
- local
registries:
docker.io:
Blocked: false
Insecure: false
Location: xxxxxx.mirror.aliyuncs.com
MirrorByDigestOnly: false
Mirrors: null
Prefix: docker.io
PullFromMirror: ""
search:
- docker.io
store:
configFile: /home/memoryshadow/.config/containers/storage.conf
containerStore:
number: 5
paused: 0
running: 0
stopped: 5
graphDriverName: overlay
graphOptions: {}
graphRoot: /home/memoryshadow/.local/share/containers/storage
graphRootAllocated: 67049664512
graphRootUsed: 45273731072
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "true"
Supports d_type: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 13
runRoot: /run/user/1000/containers
volumePath: /home/memoryshadow/.local/share/containers/storage/volumes
version:
APIVersion: 4.2.1
Built: 1662619974
BuiltTime: Thu Sep  8 14:52:54 2022
GitCommit: 62b324ddf718411b1d4d0ba8117c632f7f984a38-dirty
GoVersion: go1.19
Os: linux
OsArch: linux/amd64
Version: 4.2.1

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

$ pacman -Q | grep podman
podman 4.2.1-1
podman-compose 1.0.3-2
podman-dnsname 1.3.1-2

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

attach my operating system information:

$ neofetch
██████████████████  ████████   memoryshadow@ThinkPadT470p
██████████████████  ████████   --------------------------
██████████████████  ████████   OS: Manjaro Linux x86_64
██████████████████  ████████   Host: ThinkPad T470P
████████            ████████   Kernel: 5.15.71-1-MANJARO
████████  ████████  ████████   Uptime: 1 day, 11 hours, 54 mins
████████  ████████  ████████   Packages: 1315 (pacman)
████████  ████████  ████████   Shell: bash 5.1.16
████████  ████████  ████████   Resolution: 1920x1080
████████  ████████  ████████   DE: Plasma 5.25.5
████████  ████████  ████████   WM: KWin
████████  ████████  ████████   WM Theme: Breeze 微风
████████  ████████  ████████   Theme: Breath Light [Plasma], Breeze [GTK2/3]
████████  ████████  ████████   Icons: Relax-Dark-Icons [Plasma], Relax-Dark-Icons [GTK2/3]
Terminal: konsole
CPU: Intel i5-7300HQ (4) @ 3.500GHz
GPU: Intel HD Graphics 630
GPU: NVIDIA GeForce 940MX
Memory: 3918MiB / 23917MiB

[MemoryShadow]

Seems to be related to #15526 ?

[MemoryShadow]

Recover after executing "podman system reset"