Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rootless docker-compose "error opening rootless-cni.lck" #10986

Closed
hoshsadiq opened this issue Jul 19, 2021 · 3 comments
Closed

rootless docker-compose "error opening rootless-cni.lck" #10986

hoshsadiq opened this issue Jul 19, 2021 · 3 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@hoshsadiq
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I'm trying to run the below docker-compose.yml (actual one is a bit more complex but this reproduces it), but I get multiple errors, and often also this error. Podman service subsequently crashes and must be restarted to respond to API calls.

I've attached docker-compose.yml and related files as an archive. I'm not sure if it can be replicated with less code but in my attempts the issue was intermittent and I struggled to pinpoint what was causing it. The attached files seem to reproduce the issue pretty consistently though it's still not happening always.

Steps to reproduce the issue:

  1. Download attached archive and extract.
  2. mkdir /tmp/testing && cd /tmp/testing && tar -xzvf $FILE
  3. docker-compose up

Describe the results you received:

$ docker-compose up
Creating network "testing_main" with the default driver
Creating testing_mailslurper_1 ... error
Creating testing_postgres_1    ...

ERROR: for testing_mailslurper_1  error opening "/run/user/1000/rootless-cni/rootless-cni.lck": no such file or directory

ERROR: for testing_postgres_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

ERROR: for mailslurper  error opening "/run/user/1000/rootless-cni/rootless-cni.lck": no such file or directory

ERROR: for postgres  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

Subsequently podman socket/service crashed. The journalctl for podman.service contains the following:

Jul 19 22:23:35 pop-os systemd[2073]: Starting Podman API Service...
Jul 19 22:23:35 pop-os systemd[2073]: Started Podman API Service.
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="/usr/bin/podman filtering at log level info"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="Found CNI network <clipped>_default (type=bridge) at /home/hosh/.config/cni/net.d/<clipped>_default.conflist"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="Setting parallel job count to 25"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="using systemd socket activation to determine API endpoint"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="using API endpoint: ''"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="API server listening on \"/run/user/1000/podman/podman.sock\""
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="APIHandler(ffb08c1e-d8d7-48fd-9cc3-b611e491ae36) -- GET /v1.38/networks/testing_main BEGIN"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="Request Failed(Not Found): unable to find network configuration for testing_main: network not found"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="APIHandler(e362fd46-8c09-4b99-9c6c-ed67fa14d0b9) -- GET /v1.38/volumes/testing_mailslurper_db BEGIN"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="APIHandler(61fdfd3a-e583-4257-9d41-cae7a4ce1494) -- GET /v1.38/volumes/testing_postgres_data BEGIN"
Jul 19 22:23:35 pop-os podman[983494]: time="2021-07-19T22:23:35+01:00" level=info msg="APIHandler(59bb3e79-8e0c-418b-93f3-38642972bf3d) -- GET /v1.38/info BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(3ed443fb-5c4b-4e9e-9107-0a3ad705e416) -- GET /v1.38/networks/testing_main BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Request Failed(Not Found): unable to find network configuration for testing_main: network not found"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(48985b6a-a8e7-40e2-b98d-ef2ca6f43cff) -- POST /v1.38/networks/create BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(8ebcfc5d-65fc-4e15-8257-5f1cfb36fec9) -- GET /v1.38/volumes/testing_mailslurper_db BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(0888a6d0-e509-4e10-8ddb-d0d5897cc356) -- GET /v1.38/volumes/testing_mailslurper_db BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(b2de9b23-d57b-4929-930e-9ace28f1b1ac) -- GET /v1.38/volumes/testing_postgres_data BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(efea2115-12bf-4233-b4f5-983fc5b7e9c3) -- GET /v1.38/volumes/testing_postgres_data BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(ea0c08b6-3929-4004-8c0d-7535dfefea0e) -- GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(2f924593-ce3c-46d4-a59c-c6302623d716) -- GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(4e009a26-d7a0-4dfd-a090-07377f578e9a) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(40a9e7c3-ef31-4d5f-8fa3-698accf97765) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(86310254-2496-40db-a0ee-d901d0e3fb1a) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(e6543543-423b-4ed1-85fd-d13e79285635) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(5609dfd9-00b6-43d0-8a68-c6018ecd9c1a) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(836f115e-8d92-47e8-b0d8-b238eb0ebfc9) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(fa88045f-9f59-44fd-844f-ce95052b2861) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(e5db7830-8629-4c46-ae26-f661124bcc3f) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(79cc71be-4fe1-4ea8-877c-9a624fae7f58) -- GET /v1.38/images/testing_mailslurper/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(e008587d-e96a-44c0-8230-4c27077e9931) -- GET /v1.38/images/docker.io/library/postgres:13-alpine/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(c6713a67-5a7a-4ae6-bc82-8b8a9a82a3d1) -- GET /v1.38/images/testing_postgres-ui-adminer/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(a6c93777-5167-44a2-ba85-9a8e8b3b9242) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(795e45ad-49f4-4b24-9e14-03fb854447c8) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(e4050476-dae1-4299-a3aa-cb8f27320f09) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(614c1fef-da64-4e25-8c36-40bdf5836188) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(807eb599-53ca-4389-ad34-157f4509ae12) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22status%22%3A+%5B%22created%22%2C+%22exited%22%5D%2C+%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(3780f739-4a45-4671-a1d8-9b0c7a20ef0f) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22status%22%3A+%5B%22created%22%2C+%22exited%22%5D%2C+%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(381b899f-aa2e-496e-a617-ef46cdfa40b6) -- GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(9c4cb630-0499-4aec-80ee-c6c0b6afd03d) -- GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(e08e8fcb-8a14-4d40-a442-89c74fb911dd) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(c20b1a1c-0fff-4688-88b9-85e7b6131ea9) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres-ui-adminer%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(aaa89876-b84f-49a0-89c2-bdbefc0fabde) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(2e247ea9-2749-4de9-93b3-5c01b60c6ca8) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(7c57191a-a43b-4dad-9a1c-103623813a83) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dmailslurper%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(6919cc0e-a3b7-4803-bd18-01e5d6fa3152) -- GET /v1.38/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dtesting%22%2C+%22com.docker.compose.service%3Dpostgres%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(1492b289-0e15-47bd-8dd0-d923dec60d85) -- GET /v1.38/images/testing_mailslurper/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(01965af3-9f8b-4aad-b2d1-60710b22f00d) -- GET /v1.38/images/docker.io/library/postgres:13-alpine/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(35b4685a-9868-401a-95b4-e31e500a6c06) -- GET /v1.38/images/testing_mailslurper/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(06d21cdc-e0df-4284-a671-d4ba9e4434fe) -- GET /v1.38/images/docker.io/library/postgres:13-alpine/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(b1c1f14d-fb22-4da9-842b-906cc7b7a81d) -- POST /v1.38/containers/create?name=testing_mailslurper_1 BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(803ee53d-d42a-4d89-b66e-c15be764c30d) -- POST /v1.38/containers/create?name=testing_postgres_1 BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: 2021-07-19 22:23:36.566293723 +0100 BST m=+0.920684627 container create 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 (image=localhost/testing_mailslurper:latest, name=testing_mailslurper_1, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.project=testing, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.oneoff=False, com.docker.compose.config-hash=0070ec3cac715bce12b0ee100d75f2ac9133617d7d9109f8134bb9ffe32001ff, io.buildah.version=1.21.0, com.docker.compose.version=1.25.0, com.docker.compose.service=mailslurper, com.docker.compose.container-number=1)
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(35a02632-2f13-42bd-b490-9aad59e4a5ea) -- GET /v1.38/containers/422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: 2021-07-19 22:23:36.582652907 +0100 BST m=+0.937043798 container create 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b (image=docker.io/library/postgres:13-alpine, name=testing_postgres_1, com.docker.compose.project=testing, com.docker.compose.service=postgres, com.docker.compose.oneoff=False, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.container-number=1, com.docker.compose.version=1.25.0, com.docker.compose.config-hash=611c6c394548c0d8281166c5cb1fad24c3e1673e6a65c06fe9be819f2779f5cb)
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(3e7a235f-af97-4243-b7e0-d5c4df55799b) -- GET /v1.38/containers/5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b/json BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(0403933b-22ec-44bb-9930-fc2a59499e3b) -- POST /v1.38/containers/422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149/attach?logs=0&stdout=1&stderr=1&stream=1 BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="APIHandler(ceab6597-659b-438b-b55e-451f2157b3f3) -- POST /v1.38/containers/5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b/attach?logs=0&stdout=1&stderr=1&stream=1 BEGIN"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Got pod network &{Name:testing_mailslurper_1 Namespace:testing_mailslurper_1 ID:422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 NetNS:/run/user/1000/netns/cni-eb842e01-55de-42ea-ba99-43dbab22985b Networks:[{Name:testing_main Ifname:eth0}] RuntimeConfig:map[testing_main:{IP: MAC: PortMappings:[{HostPort:6245 ContainerPort:6245 Protocol:tcp HostIP:127.0.0.1} {HostPort:8085 ContainerPort:8085 Protocol:tcp HostIP:127.0.0.1} {HostPort:2500 ContainerPort:2500 Protocol:tcp HostIP:127.0.0.1}] Bandwidth:<nil> IpRanges:[]}] Aliases:map[testing_main:[mailslurper]]}"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Got pod network &{Name:testing_postgres_1 Namespace:testing_postgres_1 ID:5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b NetNS:/run/user/1000/netns/cni-83875624-30dd-6eba-82ea-4f609f477de0 Networks:[{Name:testing_main Ifname:eth0}] RuntimeConfig:map[testing_main:{IP: MAC: PortMappings:[{HostPort:5432 ContainerPort:5432 Protocol:tcp HostIP:127.0.0.1}] Bandwidth:<nil> IpRanges:[]}] Aliases:map[testing_main:[postgres]]}"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Found CNI network <clipped>_default (type=bridge) at /home/hosh/.config/cni/net.d/<clipped>_default.conflist"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Found CNI network <clipped>_default (type=bridge) at /home/hosh/.config/cni/net.d/<clipped>_default.conflist"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Found CNI network testing_main (type=bridge) at /home/hosh/.config/cni/net.d/testing_main.conflist"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Adding pod testing_mailslurper_1_testing_mailslurper_1 to CNI network \"testing_main\" (type=bridge)"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Found CNI network testing_main (type=bridge) at /home/hosh/.config/cni/net.d/testing_main.conflist"
Jul 19 22:23:36 pop-os podman[983494]: time="2021-07-19T22:23:36+01:00" level=info msg="Adding pod testing_postgres_1_testing_postgres_1 to CNI network \"testing_main\" (type=bridge)"
Jul 19 22:23:36 pop-os dnsmasq[983870]: started, version 2.84rc2 cachesize 150
Jul 19 22:23:36 pop-os dnsmasq[983870]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
Jul 19 22:23:36 pop-os dnsmasq[983870]: using only locally-known addresses for domain dns.podman
Jul 19 22:23:36 pop-os dnsmasq[983870]: reading /etc/resolv.conf
Jul 19 22:23:36 pop-os dnsmasq[983870]: using only locally-known addresses for domain dns.podman
Jul 19 22:23:36 pop-os dnsmasq[983870]: using nameserver 10.0.2.3#53
Jul 19 22:23:36 pop-os dnsmasq[983870]: read /run/user/1000/containers/cni/dnsname/testing_main/addnhosts - 1 addresses
Jul 19 22:23:36 pop-os dnsmasq[983870]: read /run/user/1000/containers/cni/dnsname/testing_main/addnhosts - 2 addresses
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Got Conmon PID as 983931"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Got Conmon PID as 983935"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.135641246 +0100 BST m=+1.490032129 container init 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 (image=localhost/testing_mailslurper:latest, name=testing_mailslurper_1, com.docker.compose.container-number=1, com.docker.compose.service=mailslurper, com.docker.compose.project=testing, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.version=1.25.0, io.buildah.version=1.21.0, com.docker.compose.config-hash=0070ec3cac715bce12b0ee100d75f2ac9133617d7d9109f8134bb9ffe32001ff, com.docker.compose.oneoff=False)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Performing HTTP Hijack attach to container 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.135837129 +0100 BST m=+1.490228009 container attach 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 (image=localhost/testing_mailslurper:latest, name=testing_mailslurper_1, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.config-hash=0070ec3cac715bce12b0ee100d75f2ac9133617d7d9109f8134bb9ffe32001ff, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.version=1.25.0, com.docker.compose.service=mailslurper, com.docker.compose.project=testing, com.docker.compose.project.config_files=docker-compose.yml, io.buildah.version=1.21.0)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(fa4dee95-757b-4b31-b825-9072786b9356) -- GET /v1.38/containers/422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149/json BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(13bfc373-1645-4da3-b86c-e56f13732f98) -- POST /v1.38/networks/testing_main/disconnect BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.145193026 +0100 BST m=+1.499583916 container init 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b (image=docker.io/library/postgres:13-alpine, name=testing_postgres_1, com.docker.compose.config-hash=611c6c394548c0d8281166c5cb1fad24c3e1673e6a65c06fe9be819f2779f5cb, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=testing, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.service=postgres, com.docker.compose.version=1.25.0)
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.145874541 +0100 BST m=+1.500265425 network disconnect 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 (container=422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149, name=testing_main)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Performing HTTP Hijack attach to container 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.146193913 +0100 BST m=+1.500584794 container attach 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b (image=docker.io/library/postgres:13-alpine, name=testing_postgres_1, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.service=postgres, com.docker.compose.version=1.25.0, com.docker.compose.config-hash=611c6c394548c0d8281166c5cb1fad24c3e1673e6a65c06fe9be819f2779f5cb, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=testing, com.docker.compose.project.config_files=docker-compose.yml)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(1f31aad1-cf20-4c0a-9cd2-e41ab477a9b0) -- GET /v1.38/containers/5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b/json BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Got pod network &{Name:testing_mailslurper_1 Namespace:testing_mailslurper_1 ID:422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 NetNS:/run/user/1000/netns/cni-eb842e01-55de-42ea-ba99-43dbab22985b Networks:[{Name:testing_main Ifname:eth0}] RuntimeConfig:map[testing_main:{IP: MAC: PortMappings:[{HostPort:6245 ContainerPort:6245 Protocol:tcp HostIP:127.0.0.1} {HostPort:8085 ContainerPort:8085 Protocol:tcp HostIP:127.0.0.1} {HostPort:2500 ContainerPort:2500 Protocol:tcp HostIP:127.0.0.1}] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Deleting pod testing_mailslurper_1_testing_mailslurper_1 from CNI network \"testing_main\" (type=bridge)"
Jul 19 22:23:37 pop-os dnsmasq[983870]: read /run/user/1000/containers/cni/dnsname/testing_main/addnhosts - 1 addresses
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(a550e0d2-c8a1-4ba8-9bf2-4cca7d9c5ebb) -- POST /v1.38/networks/testing_main/disconnect BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.159506732 +0100 BST m=+1.513897621 network disconnect 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b (container=5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b, name=testing_main)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Got pod network &{Name:testing_postgres_1 Namespace:testing_postgres_1 ID:5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b NetNS:/run/user/1000/netns/cni-83875624-30dd-6eba-82ea-4f609f477de0 Networks:[{Name:testing_main Ifname:eth0}] RuntimeConfig:map[testing_main:{IP: MAC: PortMappings:[{HostPort:5432 ContainerPort:5432 Protocol:tcp HostIP:127.0.0.1}] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Deleting pod testing_postgres_1_testing_postgres_1 from CNI network \"testing_main\" (type=bridge)"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(59c95e07-4aee-4b5f-aa42-55cfa494ec75) -- POST /v1.38/networks/testing_main/connect BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.329811942 +0100 BST m=+1.684202878 network connect 422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149 (container=422f45eba39d978d6b981cc9f5b790cf470e50a82a1921e1b096e72622d71149, name=testing_main)
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=warning msg="Recovering from API handler panic: error opening \"/run/user/1000/rootless-cni/rootless-cni.lck\": no such file or directory, goroutine 354 [running]:\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1.1(0x555f80720400, 0xc000c760e0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:24 +0xb7\npanic(0x555f803d15a0, 0xc000a89710)\n\t/usr/lib/go-1.14/src/runtime/panic.go:969 +0x16a\ngithub.com/containers/podman/vendor/github.com/containers/storage/pkg/lockfile.(*lockfile).lock(0xc00053a550, 0x555f80000001)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/containers/storage/pkg/lockfile/lockfile_unix.go:107 +0x378\ngithub.com/containers/podman/vendor/github.com/containers/storage/pkg/lockfile.(*lockfile).Lock(0xc00053a550)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/containers/storage/pkg/lockfile/lockfile_unix.go:129 +0x3f\ngithub.com/containers/podman/libpod.(*Runtime).GetRootlessCNINetNs(0xc00010ba40, 0xc000a89601, 0x0, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:279 +0x20c\ngithub.com/containers/podman/libpod.(*Runtime).setUpOCICNIPod(0xc00010ba40, 0xc000a81da0, 0x15, 0xc000a81da0, 0x15, 0xc0005d3ec0, 0x40, 0xc0009c84c0, 0x3d, 0xc000270220, ...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:463 +0x50\ngithub.com/containers/podman/libpod.(*Container).NetworkConnect(0xc000323b80, 0xc0005d3840, 0x40, 0xc000882910, 0xc, 0xc0000a9a00, 0x2, 0x4, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:1168 +0x532\ngithub.com/containers/podman/libpod.(*Runtime).ConnectContainerToNetwork(0xc00010ba40, 0xc0005d3840, 0x40, 0xc0005d3495, 0xc, 0xc0000a9a00, 0x2, 0x4, 0xc000e6b6a8, 0x5200555f7eb7e209)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:1228 +0xce\ngithub.com/containers/podman/pkg/api/handlers/compat.Connect(0x555f80720400, 0xc000c760e0, 0xc00064a300)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/handlers/compat/networks.go:353 +0x1df\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1.2(0x555f80720400, 0xc000c760e0, 0xc00064a200)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:66 +0xfb8\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1(0x555f80720400, 0xc000c760e0, 0xc00064a200)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:69 +0xa8\nnet/http.HandlerFunc.ServeHTTP(0xc000715400, 0x555f80720400, 0xc000c760e0, 0xc00064a200)\n\t/usr/lib/go-1.14/src/net/http/server.go:2041 +0x46\ngithub.com/containers/podman/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ea000, 0x555f80720400, 0xc000c760e0, 0xc00064a000)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/gorilla/mux/mux.go:210 +0xe4\nnet/http.serverHandler.ServeHTTP(0xc0002ceb40, 0x555f80720400, 0xc000c760e0, 0xc00064a000)\n\t/usr/lib/go-1.14/src/net/http/server.go:2836 +0xa5\nnet/http.(*conn).serve(0xc000722d20, 0x555f8072bd40, 0xc0000a97c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1924 +0x86e\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 1 [chan receive]:\ngithub.com/containers/podman/pkg/api/server.(*APIServer).Serve(0xc0002ceb40, 0x12a05f200, 0xc0002a2010)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/server.go:227 +0x168\ngithub.com/containers/podman/cmd/podman/system.restService(0x0, 0x0, 0x12a05f200, 0x555f8158cc20, 0xc00039fe00, 0x555f8161b800, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/cmd/podman/system/service_abi.go:85 +0x236\ngithub.com/containers/podman/cmd/podman/system.service(0x555f8158cc20, 0x555f81649970, 0x0, 0x0, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/cmd/podman/system/service.go:98 +0x1af\ngithub.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).execute(0x555f8158cc20, 0xc0000a8070, 0x0, 0x0, 0x555f8158cc20, 0xc0000a8070)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:852 +0x455\ngithub.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x555f815905a0, 0xc0000ce020, 0x555f80470b60, 0x555f81649970)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:960 +0x34b\ngithub.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).Execute(...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:897\ngithub.com/containers/podman/vendor/github.com/spf13/cobra.(*Command).ExecuteContext(...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/spf13/cobra/command.go:890\nmain.Execute()\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/cmd/podman/root.go:90 +0xee\nmain.main()\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/cmd/podman/main.go:38 +0x98\n\ngoroutine 42 [chan receive]:\ngithub.com/containers/podman/vendor/k8s.io/klog/v2.(*loggingT).flushDaemon(0x555f8161bb00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/k8s.io/klog/v2/klog.go:1164 +0x8d\ncreated by github.com/containers/podman/vendor/k8s.io/klog/v2.init.0\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/k8s.io/klog/v2/klog.go:418 +0xdf\n\ngoroutine 4 [syscall]:\nos/signal.signal_recv(0x0)\n\t/usr/lib/go-1.14/src/runtime/sigqueue.go:147 +0x9e\nos/signal.loop()\n\t/usr/lib/go-1.14/src/os/signal/signal_unix.go:23 +0x24\ncreated by os/signal.Notify.func1\n\t/usr/lib/go-1.14/src/os/signal/signal.go:127 +0x46\n\ngoroutine 22 [select]:\ngithub.com/containers/podman/libpod/shutdown.Start.func1()\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/shutdown/handler.go:45 +0xcd\ncreated by github.com/containers/podman/libpod/shutdown.Start\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/shutdown/handler.go:44 +0x116\n\ngoroutine 66 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb030, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0005d6098, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0005d6080, 0xc00062c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0005d6080, 0xc00062c000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0003bc000, 0xc00062c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000626030, 0xc00062c000, 0x1000, 0x1000, 0xc035993896dab613, 0x280062d72, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00062a000)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00062a000, 0x4, 0x280062d72, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0005dc000, 0x555f8072bd40, 0xc0005d0080)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 6 [sleep]:\ntime.Sleep(0x5f5e100)\n\t/usr/lib/go-1.14/src/runtime/time.go:188 +0xbe\ngithub.com/containers/podman/libpod.(*Runtime).libimageEvents.func2(0xc00010ba40, 0x555f806b9ba0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/runtime.go:737 +0x26f\ncreated by github.com/containers/podman/libpod.(*Runtime).libimageEvents\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/runtime.go:712 +0x4d\n\ngoroutine 142 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fd530, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000986118, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000986100, 0xc000c65000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000986100, 0xc000c65000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000c06008, 0xc000c65000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000a92150, 0xc000c65000, 0x1000, 0x1000, 0xc0359938a2bd9997, 0x28be91100, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000372120)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000372120, 0x4, 0x28be91100, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc000486320, 0x555f8072bd40, 0xc000cb2000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 136 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddaaf0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000af7f18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000af7f00, 0xc00096b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000af7f00, 0xc00096b000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007c3f8, 0xc00096b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000aa1860, 0xc00096b000, 0x1000, 0x1000, 0xc03599389e06ba2f, 0x287323192, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000522840)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000522840, 0x4, 0x287323192, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00036ed20, 0x555f8072bd40, 0xc0008c13c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 132 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddad90, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00088da98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00088da80, 0xc000adb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc00088da80, 0xc000adb000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007c1f0, 0xc000adb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00093d080, 0xc000adb000, 0x1000, 0x1000, 0xc03599389c5a4fd6, 0x28585c736, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0000aa720)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0000aa720, 0x4, 0x28585c736, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042bea0, 0x555f8072bd40, 0xc000aa6ac0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 88 [select]:\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach(0xc00031f7c0, 0xc0005c52c0, 0xc0008eae00, 0x555f80720400, 0xc000c768c0, 0xc0004a014b, 0x0, 0x0, 0xc000d6a5b0, 0x1, ...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:697 +0x6cd\ngithub.com/containers/podman/libpod.(*Container).HTTPAttach(0xc0005c52c0, 0xc0008eae00, 0x555f80720400, 0xc000c768c0, 0xc0004a014b, 0x0, 0x0, 0x1, 0xc000d6a5b0, 0x0, ...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_api.go:308 +0x228\ngithub.com/containers/podman/pkg/api/handlers/compat.AttachContainer(0x555f80720400, 0xc000c768c0, 0xc0008eae00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/handlers/compat/containers_attach.go:103 +0x55a\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1.2(0x555f80720400, 0xc000c768c0, 0xc0008ead00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:66 +0xfb8\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1(0x555f80720400, 0xc000c768c0, 0xc0008ead00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:69 +0xa8\nnet/http.HandlerFunc.ServeHTTP(0xc0007150c0, 0x555f80720400, 0xc000c768c0, 0xc0008ead00)\n\t/usr/lib/go-1.14/src/net/http/server.go:2041 +0x46\ngithub.com/containers/podman/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ea000, 0x555f80720400, 0xc000c768c0, 0xc0008eab00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/gorilla/mux/mux.go:210 +0xe4\nnet/http.serverHandler.ServeHTTP(0xc0002ceb40, 0x555f80720400, 0xc000c768c0, 0xc0008eab00)\n\t/usr/lib/go-1.14/src/net/http/server.go:2836 +0xa5\nnet/http.(*conn).serve(0xc0005c5180, 0x555f8072bd40, 0xc000c24280)\n\t/usr/lib/go-1.14/src/net/http/server.go:1924 +0x86e\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 130 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddaf50, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00088cc18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00088cc00, 0xc000745000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc00088cc00, 0xc000745000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007c0b0, 0xc000745000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000899710, 0xc000745000, 0x1000, 0x1000, 0xc0359938976327d5, 0x2808e9f38, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000373b00)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000373b00, 0x4, 0x2808e9f38, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042bb80, 0x555f8072bd40, 0xc0005bb0c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 117 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddacb0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000957e98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000957e80, 0xc000afe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000957e80, 0xc000afe000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0006ac120, 0xc000afe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00093dbc0, 0xc000afe000, 0x1000, 0x1000, 0xc03599389c6f1e99, 0x2859a95f8, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0000aa840)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0000aa840, 0x4, 0x2859a95f8, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0005c4a00, 0x555f8072bd40, 0xc000aa7400)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 164 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fd450, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000970298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000970280, 0xc000e1f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000970280, 0xc000e1f000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d0898, 0xc000e1f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000a200c0, 0xc000e1f000, 0x1000, 0x1000, 0xc0359938c8539408, 0x2ad19d56e, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00079e2a0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00079e2a0, 0x4, 0x2ad19d56e, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0007220a0, 0x555f8072bd40, 0xc00094a000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 24 [chan receive]:\ngithub.com/containers/podman/pkg/domain/infra.StartWatcher.func1(0xc0000aa360, 0xc00010ba40)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/domain/infra/runtime_libpod.go:369 +0x79\ncreated by github.com/containers/podman/pkg/domain/infra.StartWatcher\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/domain/infra/runtime_libpod.go:365 +0xb7\n\ngoroutine 60 [chan receive]:\ngithub.com/containers/podman/pkg/api/server.(*APIServer).Serve.func2(0xc0002ceb40)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/server.go:195 +0x48\ncreated by github.com/containers/podman/pkg/api/server.(*APIServer).Serve\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/server.go:194 +0xea\n\ngoroutine 61 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb110, 0x72, 0x0)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0004ac598, 0x72, 0x0, 0x0, 0x555f7ff29309)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Accept(0xc0004ac580, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:384 +0x1d6\nnet.(*netFD).accept(0xc0004ac580, 0xc0005b70f8, 0x555f803d0920, 0xc0005b7100)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:238 +0x44\nnet.(*UnixListener).accept(0xc000242000, 0xc000073e20, 0x555f7fef5c9a, 0xc000242e40)\n\t/usr/lib/go-1.14/src/net/unixsock_posix.go:162 +0x34\nnet.(*UnixListener).Accept(0xc000242000, 0xc000073e70, 0x18, 0xc000526000, 0x555f7ee6eaee)\n\t/usr/lib/go-1.14/src/net/unixsock.go:260 +0x66\nnet/http.(*Server).Serve(0xc0002ceb40, 0x555f80720180, 0xc000242000, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/http/server.go:2930 +0x25f\ngithub.com/containers/podman/pkg/api/server.(*APIServer).Serve.func4(0xc0002ceb40, 0xc0004fc1e0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/server.go:219 +0x49\ncreated by github.com/containers/podman/pkg/api/server.(*APIServer).Serve\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/server.go:218 +0x148\n\ngoroutine 62 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb2d0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000476118, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000476100, 0xc00011a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000476100, 0xc00011a000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000010dc8, 0xc00011a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00079c030, 0xc00011a000, 0x1000, 0x1000, 0xc035993895e10ee2, 0x27f0c8642, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00079e000)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00079e000, 0x4, 0x27f0c8642, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0003bbcc0, 0x555f8072bd40, 0xc0002f0000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 63 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb3b0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000476418, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000476400, 0xc00052a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000476400, 0xc00052a000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000010dd0, 0xc00052a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc0005c0b40, 0xc00052a000, 0x1000, 0x1000, 0xc035993896a120f6, 0x27fcc9854, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0004fc240)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0004fc240, 0x4, 0x27fcc9854, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0003bbd60, 0x555f8072bd40, 0xc0000a88c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 65 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb1f0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0004ac618, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0004ac600, 0xc000551000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0004ac600, 0xc000551000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d0028, 0xc000551000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc0005c00c0, 0xc000551000, 0x1000, 0x1000, 0xc0359938959e69ff, 0x27ec9e160, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000522060)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000522060, 0x4, 0x27ec9e160, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0005c4000, 0x555f8072bd40, 0xc0002f0100)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 106 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddb490, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0008bd998, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0008bd980, 0xc000ad4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0008bd980, 0xc000ad4000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0003bcb58, 0xc000ad4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00093c630, 0xc000ad4000, 0x1000, 0x1000, 0xc035993897bd3e2e, 0x280e8b58e, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0000aa600)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0000aa600, 0x4, 0x280e8b58e, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00040be00, 0x555f8072bd40, 0xc000aa6200)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 32 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddae70, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00088c298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00088c280, 0xc000784000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc00088c280, 0xc000784000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007c000, 0xc000784000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00088a030, 0xc000784000, 0x1000, 0x1000, 0xc03599389664d856, 0x27f904fb5, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00079e060)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00079e060, 0x4, 0x27f904fb5, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042b4a0, 0x555f8072bd40, 0xc0000a90c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 86 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fd370, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000956118, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000956100, 0xc000c70000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000956100, 0xc000c70000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0003bc010, 0xc000c70000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00038e360, 0xc000c70000, 0x1000, 0x1000, 0xc0359938c924b3ea, 0x2adeaf54f, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000c6e000)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000c6e000, 0x4, 0x2adeaf54f, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0005c4140, 0x555f8072bd40, 0xc0007c40c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 327 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fce30, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0009acc98, 0x72, 0x2000, 0x2000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0009acc80, 0xc0006ec000, 0x2000, 0x2000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0009acc80, 0xc0006ec000, 0x2000, 0x2000, 0xc00018edc8, 0x555f7eb7a2ae, 0x2000)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d1810, 0xc0006ec000, 0x2000, 0x2000, 0x203000, 0x203000, 0x203000)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\ngithub.com/containers/podman/libpod.httpAttachNonTerminalCopy(0xc0000d1810, 0xc000d90ab0, 0xc000631600, 0x40, 0xc000010101, 0x1, 0x1)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:1600 +0x92\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach.func5(0xc00078aa00, 0xc0005c52c0, 0xc000b43301, 0xc0000d1810, 0xc000d90ab0, 0x555f7ff20101, 0xc00078a960)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:682 +0x267\ncreated by github.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:667 +0x5db\n\ngoroutine 118 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddabd0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000986018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000986000, 0xc000988000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000986000, 0xc000988000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0006ac128, 0xc000988000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000736c30, 0xc000988000, 0x1000, 0x1000, 0xc03599389c0acb65, 0x2853642c3, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00054cb40)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00054cb40, 0x4, 0x2853642c3, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0005c4d20, 0x555f8072bd40, 0xc0007b02c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 146 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1ddaa10, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000a84098, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000a84080, 0xc00089e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000a84080, 0xc00089e000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000010018, 0xc00089e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000a920c0, 0xc00089e000, 0x1000, 0x1000, 0xc03599389e29690f, 0x28754e06d, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00079e0c0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00079e0c0, 0x4, 0x28754e06d, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042b860, 0x555f8072bd40, 0xc0009c2000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 147 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1dda930, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000a84298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000a84280, 0xc000744000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000a84280, 0xc000744000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000010060, 0xc000744000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000a20060, 0xc000744000, 0x1000, 0x1000, 0xc03599389a7b8eac, 0x283a7060c, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00054c000)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00054c000, 0x4, 0x283a7060c, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042bd60, 0x555f8072bd40, 0xc0006d2000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 149 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1dda850, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000a84818, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000a84800, 0xc000cea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000a84800, 0xc000cea000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000111f8, 0xc000cea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00093cf00, 0xc000cea000, 0x1000, 0x1000, 0xc03599389b68a1bc, 0x28494191b, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc00054d260)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc00054d260, 0x4, 0x28494191b, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc000486aa0, 0x555f8072bd40, 0xc000cb3b00)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 151 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1dda770, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000a4c518, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000a4c500, 0xc000d1b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000a4c500, 0xc000d1b000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000011280, 0xc000d1b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc000899500, 0xc000d1b000, 0x1000, 0x1000, 0xc03599389bbf11c3, 0x284ea8923, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0003727e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0003727e0, 0x4, 0x284ea8923, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc000487220, 0x555f8072bd40, 0xc000d01880)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 120 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e1dda690, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00095a098, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00095a080, 0xc000d21000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc00095a080, 0xc000d21000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d0030, 0xc000d21000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc0002ae000, 0xc000d21000, 0x1000, 0x1000, 0xc0359938a1c6b059, 0x28af227da, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc0005220c0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc0005220c0, 0x4, 0x28af227da, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc00042ba40, 0x555f8072bd40, 0xc000d00000)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 144 [select]:\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach(0xc00031f7c0, 0xc00040b9a0, 0xc000980300, 0x555f80720400, 0xc0006da380, 0xc0007300a3, 0x0, 0x0, 0xc000d6a2a0, 0x1, ...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:697 +0x6cd\ngithub.com/containers/podman/libpod.(*Container).HTTPAttach(0xc00040b9a0, 0xc000980300, 0x555f80720400, 0xc0006da380, 0xc0007300a3, 0x0, 0x0, 0x1, 0xc000d6a2a0, 0x0, ...)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_api.go:308 +0x228\ngithub.com/containers/podman/pkg/api/handlers/compat.AttachContainer(0x555f80720400, 0xc0006da380, 0xc000980300)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/handlers/compat/containers_attach.go:103 +0x55a\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1.2(0x555f80720400, 0xc0006da380, 0xc000980200)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:66 +0xfb8\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1(0x555f80720400, 0xc0006da380, 0xc000980200)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:69 +0xa8\nnet/http.HandlerFunc.ServeHTTP(0xc0007150c0, 0x555f80720400, 0xc0006da380, 0xc000980200)\n\t/usr/lib/go-1.14/src/net/http/server.go:2041 +0x46\ngithub.com/containers/podman/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ea000, 0x555f80720400, 0xc0006da380, 0xc000980000)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/gorilla/mux/mux.go:210 +0xe4\nnet/http.serverHandler.ServeHTTP(0xc0002ceb40, 0x555f80720400, 0xc0006da380, 0xc000980000)\n\t/usr/lib/go-1.14/src/net/http/server.go:2836 +0xa5\nnet/http.(*conn).serve(0xc00040b360, 0x555f8072bd40, 0xc0009eba00)\n\t/usr/lib/go-1.14/src/net/http/server.go:1924 +0x86e\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 323 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fcc70, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0009ac398, 0x72, 0x2000, 0x2000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0009ac380, 0xc000640000, 0x2000, 0x2000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0009ac380, 0xc000640000, 0x2000, 0x2000, 0xc000069dc8, 0x555f7eb7a2ae, 0x2000)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d1768, 0xc000640000, 0x2000, 0x2000, 0x203000, 0x203000, 0x203000)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\ngithub.com/containers/podman/libpod.httpAttachNonTerminalCopy(0xc0000d1768, 0xc00026beb0, 0xc00053d780, 0x40, 0xc000010101, 0x1, 0x1)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:1600 +0x92\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach.func5(0xc00078a800, 0xc00040b9a0, 0xc000ae5301, 0xc0000d1768, 0xc00026beb0, 0x555f7ff20101, 0xc00078a7e0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:682 +0x267\ncreated by github.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:667 +0x5db\n\ngoroutine 324 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fd290, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000938a98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000938a80, 0xc000e30000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000938a80, 0xc000e30000, 0x8000, 0x8000, 0x555f80356160, 0x7f30e039bf68, 0xc0003dae30)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007c358, 0xc000e30000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc00048ba10, 0xc000e30000, 0x8000, 0x8000, 0x8000, 0x555f803d1820, 0x1)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).Read(0xc000be2f00, 0xc000e30000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:212 +0x143\ngithub.com/containers/podman/utils.CopyDetachable(0x555f806ed480, 0xc0000d1768, 0x555f806ea700, 0xc00026beb0, 0xc000d2ca48, 0x2, 0x8, 0xc000ac1000, 0x1000, 0x1000)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/utils/utils.go:61 +0xa8\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach.func6(0xc0000d1768, 0xc00026beb0, 0xc000d2ca48, 0x2, 0x8, 0xc00078a8a0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:690 +0x88\ncreated by github.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:689 +0x803\n\ngoroutine 239 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e029c410, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000974518, 0x72, 0x0, 0x1, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000974500, 0xc000627121, 0x1, 0x1, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000974500, 0xc000627121, 0x1, 0x1, 0xc0004a6fc8, 0x555f7eea3a86, 0xc0004a6f78)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0000d0598, 0xc000627121, 0x1, 0x1, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).backgroundRead(0xc000627110)\n\t/usr/lib/go-1.14/src/net/http/server.go:689 +0x5a\ncreated by net/http.(*connReader).startBackgroundRead\n\t/usr/lib/go-1.14/src/net/http/server.go:685 +0xd2\n\ngoroutine 298 [runnable, locked to thread]:\nsyscall.Syscall(0x4b, 0x22, 0x0, 0x0, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/syscall/asm_linux_amd64.s:18 +0x5\nsyscall.Fdatasync(0x22, 0x1000, 0x0)\n\t/usr/lib/go-1.14/src/syscall/zsyscall_linux_amd64.go:393 +0x42\ngithub.com/containers/podman/vendor/go.etcd.io/bbolt.fdatasync(0xc000faf600, 0x1000, 0x1000)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/go.etcd.io/bbolt/bolt_linux.go:9 +0x46\ngithub.com/containers/podman/vendor/go.etcd.io/bbolt.(*Tx).writeMeta(0xc000358460, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/go.etcd.io/bbolt/tx.go:596 +0x108\ngithub.com/containers/podman/vendor/go.etcd.io/bbolt.(*Tx).Commit(0xc000358460, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/go.etcd.io/bbolt/tx.go:208 +0x394\ngithub.com/containers/podman/vendor/go.etcd.io/bbolt.(*DB).Update(0xc000faf600, 0xc000ae12a8, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/go.etcd.io/bbolt/db.go:701 +0xf8\ngithub.com/containers/podman/libpod.(*BoltState).SaveContainer(0xc00051c0a0, 0xc00019c460, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/boltdb_state.go:816 +0x2be\ngithub.com/containers/podman/libpod.(*Container).save(0xc00019c460, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_internal.go:786 +0x4c\ngithub.com/containers/podman/libpod.(*Container).NetworkDisconnect(0xc00019c460, 0xc000c2bfc0, 0x40, 0xc000d8e730, 0xc, 0x0, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:1118 +0x63a\ngithub.com/containers/podman/libpod.(*Runtime).DisconnectContainerFromNetwork(0xc00010ba40, 0xc000c2bfc0, 0x40, 0xc000c2bf15, 0xc, 0xc8c18e1017e91b00, 0xc0007c36a8, 0xc800555f7eb7e209)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/networking_linux.go:1219 +0xad\ngithub.com/containers/podman/pkg/api/handlers/compat.Disconnect(0x555f80720400, 0xc0006da000, 0xc000438d00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/handlers/compat/networks.go:380 +0x193\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1.2(0x555f80720400, 0xc0006da000, 0xc000438b00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:66 +0xfb8\ngithub.com/containers/podman/pkg/api/server.(*APIServer).APIHandler.func1(0x555f80720400, 0xc0006da000, 0xc000438b00)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/pkg/api/server/handler_api.go:69 +0xa8\nnet/http.HandlerFunc.ServeHTTP(0xc000715800, 0x555f80720400, 0xc0006da000, 0xc000438b00)\n\t/usr/lib/go-1.14/src/net/http/server.go:2041 +0x46\ngithub.com/containers/podman/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0004ea000, 0x555f80720400, 0xc0006da000, 0xc000438100)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/gorilla/mux/mux.go:210 +0xe4\nnet/http.serverHandler.ServeHTTP(0xc0002ceb40, 0x555f80720400, 0xc0006da000, 0xc000438100)\n\t/usr/lib/go-1.14/src/net/http/server.go:283
Jul 19 22:23:37 pop-os podman[983494]: 6 +0xa5\nnet/http.(*conn).serve(0xc00019c140, 0x555f8072bd40, 0xc000c63bc0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1924 +0x86e\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 310 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fcff0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00094f698, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00094f680, 0xc000eb4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc00094f680, 0xc000eb4000, 0x1000, 0x1000, 0x555f7ebf5b4c, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc00007cab0, 0xc000eb4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc0002b3560, 0xc000eb4000, 0x1000, 0x1000, 0xc0359938d274e3ef, 0x2b73b25ae, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).fill(0xc000c6e780)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:100 +0x105\nbufio.(*Reader).Peek(0xc000c6e780, 0x4, 0x2b73b25ae, 0x555f8161a7e0, 0x0, 0x0, 0x555f8161a7e0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:138 +0x51\nnet/http.(*conn).serve(0xc0004f4b40, 0x555f8072bd40, 0xc00003c7c0)\n\t/usr/lib/go-1.14/src/net/http/server.go:1949 +0x9d9\ncreated by net/http.(*Server).Serve\n\t/usr/lib/go-1.14/src/net/http/server.go:2962 +0x35e\n\ngoroutine 272 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fcab0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00054cfd8, 0x72, 0x8001, 0x8000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00054cfc0, 0xc000614000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nos.(*File).read(...)\n\t/usr/lib/go-1.14/src/os/file_unix.go:263\nos.(*File).Read(0xc00007c5d8, 0xc000614000, 0x8000, 0x8000, 0x52, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/os/file.go:116 +0x73\nio.copyBuffer(0x555f806ed2c0, 0xc0007cccc0, 0x555f806ed820, 0xc00007c5d8, 0xc000614000, 0x8000, 0x8000, 0x555f7eb32fc5, 0xc000be2cc0, 0xc0003e0fb0)\n\t/usr/lib/go-1.14/src/io/io.go:405 +0x124\nio.Copy(...)\n\t/usr/lib/go-1.14/src/io/io.go:364\nos/exec.(*Cmd).writerDescriptor.func1(0xc000be2cc0, 0xc0003e0fb0)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:311 +0x65\nos/exec.(*Cmd).Start.func1(0xc000df49a0, 0xc0007ccd60)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:441 +0x29\ncreated by os/exec.(*Cmd).Start\n\t/usr/lib/go-1.14/src/os/exec/exec.go:440 +0x634\n\ngoroutine 299 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fcb90, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0008fe298, 0x72, 0x0, 0x1, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0008fe280, 0xc0007d4041, 0x1, 0x1, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc0008fe280, 0xc0007d4041, 0x1, 0x1, 0x3d41544144475020, 0x62696c2f7261762f, 0x65726774736f702f)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc000c06000, 0xc0007d4041, 0x1, 0x1, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).backgroundRead(0xc0007d4030)\n\t/usr/lib/go-1.14/src/net/http/server.go:689 +0x5a\ncreated by net/http.(*connReader).startBackgroundRead\n\t/usr/lib/go-1.14/src/net/http/server.go:685 +0xd2\n\ngoroutine 271 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fcf10, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc00054cf18, 0x72, 0x201, 0x200, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc00054cf00, 0xc000b75600, 0x200, 0x200, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nos.(*File).read(...)\n\t/usr/lib/go-1.14/src/os/file_unix.go:263\nos.(*File).Read(0xc00007c5c8, 0xc000b75600, 0x200, 0x200, 0xc0003dce90, 0x555f7eb75934, 0xc0003dcea0)\n\t/usr/lib/go-1.14/src/os/file.go:116 +0x73\nbytes.(*Buffer).ReadFrom(0xc00063df50, 0x555f806ed820, 0xc00007c5c8, 0x7f30e039b028, 0xc00063df50, 0xc0003dcf01)\n\t/usr/lib/go-1.14/src/bytes/buffer.go:204 +0xb3\nio.copyBuffer(0x555f806ea800, 0xc00063df50, 0x555f806ed820, 0xc00007c5c8, 0x0, 0x0, 0x0, 0x555f7eb32fc5, 0xc0003726c0, 0xc0003dcfb0)\n\t/usr/lib/go-1.14/src/io/io.go:391 +0x2fe\nio.Copy(...)\n\t/usr/lib/go-1.14/src/io/io.go:364\nos/exec.(*Cmd).writerDescriptor.func1(0xc0003726c0, 0xc0003dcfb0)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:311 +0x65\nos/exec.(*Cmd).Start.func1(0xc000df49a0, 0xc0007ccd40)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:441 +0x29\ncreated by os/exec.(*Cmd).Start\n\t/usr/lib/go-1.14/src/os/exec/exec.go:440 +0x634\n\ngoroutine 282 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fc650, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0004fd518, 0x72, 0x201, 0x200, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0004fd500, 0xc000d19c00, 0x200, 0x200, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nos.(*File).read(...)\n\t/usr/lib/go-1.14/src/os/file_unix.go:263\nos.(*File).Read(0xc000c06128, 0xc000d19c00, 0x200, 0x200, 0xc000020690, 0x555f7eb75934, 0xc0000206a0)\n\t/usr/lib/go-1.14/src/os/file.go:116 +0x73\nbytes.(*Buffer).ReadFrom(0xc0005c1dd0, 0x555f806ed820, 0xc000c06128, 0x7f30e039b028, 0xc0005c1dd0, 0xc000020701)\n\t/usr/lib/go-1.14/src/bytes/buffer.go:204 +0xb3\nio.copyBuffer(0x555f806ea800, 0xc0005c1dd0, 0x555f806ed820, 0xc000c06128, 0x0, 0x0, 0x0, 0x555f7eb32fc5, 0xc0003730e0, 0xc0000207b0)\n\t/usr/lib/go-1.14/src/io/io.go:391 +0x2fe\nio.Copy(...)\n\t/usr/lib/go-1.14/src/io/io.go:364\nos/exec.(*Cmd).writerDescriptor.func1(0xc0003730e0, 0xc0000207b0)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:311 +0x65\nos/exec.(*Cmd).Start.func1(0xc000c26f20, 0xc00024b620)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:441 +0x29\ncreated by os/exec.(*Cmd).Start\n\t/usr/lib/go-1.14/src/os/exec/exec.go:440 +0x634\n\ngoroutine 283 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e029c5d0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc0004fd698, 0x72, 0x8001, 0x8000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc0004fd680, 0xc0006a0000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nos.(*File).read(...)\n\t/usr/lib/go-1.14/src/os/file_unix.go:263\nos.(*File).Read(0xc000c06138, 0xc0006a0000, 0x8000, 0x8000, 0x52, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/os/file.go:116 +0x73\nio.copyBuffer(0x555f806ed2c0, 0xc00024b560, 0x555f806ed820, 0xc000c06138, 0xc0006a0000, 0x8000, 0x8000, 0x555f7eb32fc5, 0xc0003730e0, 0xc00056d7b0)\n\t/usr/lib/go-1.14/src/io/io.go:405 +0x124\nio.Copy(...)\n\t/usr/lib/go-1.14/src/io/io.go:364\nos/exec.(*Cmd).writerDescriptor.func1(0xc0003730e0, 0xc00056d7b0)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:311 +0x65\nos/exec.(*Cmd).Start.func1(0xc000c26f20, 0xc00024b640)\n\t/usr/lib/go-1.14/src/os/exec/exec.go:441 +0x29\ncreated by os/exec.(*Cmd).Start\n\t/usr/lib/go-1.14/src/os/exec/exec.go:440 +0x634\n\ngoroutine 328 [IO wait]:\ninternal/poll.runtime_pollWait(0x7f30e02fd0d0, 0x72, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/runtime/netpoll.go:203 +0x57\ninternal/poll.(*pollDesc).wait(0xc000ad1698, 0x72, 0x8000, 0x8000, 0xffffffffffffffff)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:87 +0x47\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/lib/go-1.14/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Read(0xc000ad1680, 0xc000f08000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/internal/poll/fd_unix.go:169 +0x19d\nnet.(*netFD).Read(0xc000ad1680, 0xc000f08000, 0x8000, 0x8000, 0x28, 0x7f30e039ba18, 0xc000023e30)\n\t/usr/lib/go-1.14/src/net/fd_unix.go:202 +0x51\nnet.(*conn).Read(0xc0003bcad0, 0xc000f08000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/net/net.go:184 +0x90\nnet/http.(*connReader).Read(0xc0008f9d10, 0xc000f08000, 0x8000, 0x8000, 0x8000, 0x555f803d1820, 0x1)\n\t/usr/lib/go-1.14/src/net/http/server.go:797 +0xf6\nbufio.(*Reader).Read(0xc000c6f560, 0xc000f08000, 0x8000, 0x8000, 0x0, 0x0, 0x0)\n\t/usr/lib/go-1.14/src/bufio/bufio.go:212 +0x143\ngithub.com/containers/podman/utils.CopyDetachable(0x555f806ed480, 0xc0000d1810, 0x555f806ea700, 0xc000d90ab0, 0xc000d2cf28, 0x2, 0x8, 0x0, 0x0, 0x0)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/utils/utils.go:61 +0xa8\ngithub.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach.func6(0xc0000d1810, 0xc000d90ab0, 0xc000d2cf28, 0x2, 0x8, 0xc00078aa20)\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:690 +0x88\ncreated by github.com/containers/podman/libpod.(*ConmonOCIRuntime).HTTPAttach\n\t/usr/src/packages/BUILD/src/github.com/containers/podman/libpod/oci_conmon_linux.go:689 +0x803\n"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="Request Failed(Internal Server Error): error opening \"/run/user/1000/rootless-cni/rootless-cni.lck\": no such file or directory"
Jul 19 22:23:37 pop-os podman[983494]: time="2021-07-19T22:23:37+01:00" level=info msg="APIHandler(152c65d6-d56a-4d00-8474-96026d263351) -- POST /v1.38/networks/testing_main/connect BEGIN"
Jul 19 22:23:37 pop-os podman[983494]: 2021-07-19 22:23:37.363111149 +0100 BST m=+1.717502041 network connect 5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b (container=5e8f0c5a50c2c676ed04380fda757266896ad8504e8c767aef35cb6f9e55da3b, name=testing_main)

Note the <clipped> bits are unrelated network that I censored out.

Now once this error has been show, podman responds to nothing any more. Podman, docker and docker-compose just sit there and hang.

$ time podman ps
^Cpodman ps  0.15s user 0.06s system 2% cpu 7.041 total

$ time docker-compose down
^CERROR: Aborting.
docker-compose down  0.30s user 0.04s system 1% cpu 18.229 total

$ time docker ps
^C
docker ps  0.02s user 0.02s system 0% cpu 27.373 total

docker ps and docker-compose down produces a some log entries, whereas podman does not:

# single log entry for docker-compose down:
Jul 19 22:41:23 pop-os podman[983494]: time="2021-07-19T22:41:23+01:00" level=info msg="APIHandler(839b16aa-d697-4bc3-ba48-df097f023cd9) -- GET /v1.38/networks/testing_main BEGIN"

# two log entries for docker ps:
Jul 19 22:45:29 pop-os podman[983494]: time="2021-07-19T22:45:29+01:00" level=info msg="APIHandler(6928eeb0-d5da-4c30-bb9c-522ce3533c57) -- HEAD /_ping BEGIN"
Jul 19 22:45:29 pop-os podman[983494]: time="2021-07-19T22:45:29+01:00" level=info msg="APIHandler(ec3c7136-184a-4da7-908c-7cd576467fa6) -- GET /v1.40/containers/json BEGIN"

Finally, restarting podman.socket makes podman responsive again, but the same issue occurs if docker-compose up is run.

Lastly, I also tend to these errors when running compose up (in no particular order):

$ docker-compose up
Creating network "testing_main" with the default driver
Creating testing_postgres_1 ...
Creating testing_mailslurper_1 ... error

ERROR: for testing_mailslurper_1  error configuring network namespace for container 5252dd3e500bc71ab4fbccff5573fab5f4cc2c540f46537b9d73148558e45ea9: error adding pod testing_mailslurper_1_tesCreating testing_postgres_1    ... done
-4782-094d-9974b53f083e": no such file or directory
Creating testing_postgres-ui-adminer_1 ... done

ERROR: for mailslurper  error configuring network namespace for container 5252dd3e500bc71ab4fbccff5573fab5f4cc2c540f46537b9d73148558e45ea9: error adding pod testing_mailslurper_1_testing_mailslurper_1 to CNI network "testing_main": failed to open netns "/run/user/1000/netns/cni-2ac5680f-ae1c-4782-094d-9974b53f083e": failed to Statfs "/run/user/1000/netns/cni-2ac5680f-ae1c-4782-094d-9974b53f083e": no such file or directory
ERROR: Encountered errors while bringing up the project.

$ docker-compose up
Creating network "testing_main" with the default driver
Creating testing_postgres_1 ...
Creating testing_mailslurper_1 ...

ERROR: for testing_postgres_1  ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

ERROR: for testing_mailslurper_1  ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

ERROR: for postgres  ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

ERROR: for mailslurper  ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.

That latter is accompanied with a invalid nil pointer dereference stack trace and also a crash:

Jul 19 22:52:05 pop-os podman[989442]: 2021-07-19 22:52:05.275920112 +0100 BST m=+0.987522743 container create 30b08eab0cc1af234e760861aa7459bfe4bbe5a0572403eaa599036390cfd8c2 (image=docker.io/library/postgres:13-alpine, name=testing_postgres_1, com.docker.compose.config-hash=611c6c394548c0d8281166c5cb1fad24c3e1673e6a65c06fe9be819f2779f5cb, com.docker.compose.project=testing, com.docker.compose.service=postgres, com.docker.compose.oneoff=False, com.docker.compose.project.working_dir=/home/hosh/tmp/testing, com.docker.compose.project.config_files=docker-compose.yml, com.docker.compose.container-number=1, com.docker.compose.version=1.25.0)
Jul 19 22:52:05 pop-os podman[989442]: time="2021-07-19T22:52:05+01:00" level=info msg="APIHandler(f9d53c8d-43a5-4dc7-94b1-ba3013ca65fe) -- GET /v1.38/containers/30b08eab0cc1af234e760861aa7459bfe4bbe5a0572403eaa599036390cfd8c2/json BEGIN"
Jul 19 22:52:05 pop-os podman[989442]: time="2021-07-19T22:52:05+01:00" level=info msg="APIHandler(e9b991f3-8b13-47ef-9f81-c18ab27567f3) -- POST /v1.38/containers/7e27812e9b0b3bcb28a1c18acea41f3634d6d23e076ab1b24cbf803fae431167/attach?logs=0&stdout=1&stderr=1&stream=1 BEGIN"
Jul 19 22:52:05 pop-os podman[989442]: time="2021-07-19T22:52:05+01:00" level=info msg="APIHandler(70e4b931-a0b3-422b-aff1-77f85ce450d9) -- POST /v1.38/containers/30b08eab0cc1af234e760861aa7459bfe4bbe5a0572403eaa599036390cfd8c2/attach?logs=0&stdout=1&stderr=1&stream=1 BEGIN"
Jul 19 22:52:05 pop-os podman[989442]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 19 22:52:05 pop-os podman[989442]: [signal SIGSEGV: segmentation violation code=0x1 addr=0xa0 pc=0x55913c2634f6]
Jul 19 22:52:05 pop-os podman[989442]: goroutine 89 [running]:
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/vendor/github.com/containers/storage.(*store).canUseShifting(0xc00059e000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x5)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/containers/storage/store.go:940 +0x36
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/vendor/github.com/containers/storage.(*store).Mount(0xc00059e000, 0xc000a44340, 0x40, 0x0, 0x0, 0xc0009687d0, 0x0, 0x0, 0x0)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/vendor/github.com/containers/storage/store.go:2711 +0x25e
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/libpod.(*storageService).MountContainerImage(0xc00032c590, 0xc000b449c0, 0x40, 0xffffffffffffff9c, 0xc000d64480, 0x0, 0x0)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/libpod/storage.go:226 +0x22d
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/libpod.(*Container).mount(0xc0001bb9a0, 0x8b, 0x0, 0x0, 0x0)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_internal.go:2037 +0x70
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/libpod.(*Container).mountStorage(0xc0001bb9a0, 0x0, 0x0, 0xc0004d6d18, 0xc0004a6f68)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_internal.go:1521 +0x61d
Jul 19 22:52:05 pop-os podman[989442]: github.com/containers/podman/libpod.(*Container).prepare.func2(0xc00044fea0, 0xc0001bb9a0, 0xc000827de0, 0xc000827dd0, 0xc00044fee0)
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_internal_linux.go:122 +0x75
Jul 19 22:52:05 pop-os podman[989442]: created by github.com/containers/podman/libpod.(*Container).prepare
Jul 19 22:52:05 pop-os podman[989442]:         /usr/src/packages/BUILD/src/github.com/containers/podman/libpod/container_internal_linux.go:120 +0x1d8
Jul 19 22:52:05 pop-os systemd[2073]: podman.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 19 22:52:05 pop-os systemd[2073]: podman.service: Failed with result 'exit-code'.

Describe the results you expected:
It should work, and not crash.

Additional information you deem important (e.g. issue happens only occasionally):

$ docker-compose version
docker-compose version 1.25.0, build unknown
docker-py version: 4.1.0
CPython version: 3.9.5
OpenSSL version: OpenSSL 1.1.1j  16 Feb 2021

$ docker version
Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:41 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: linux/amd64/pop-21.04
 Podman Engine:
  Version:          3.2.2
  APIVersion:       3.2.2
  Arch:             amd64
  BuildTime:        1970-01-01T01:00:00+01:00
  Experimental:     true
  GitCommit:
  GoVersion:        go1.14.7
  KernelVersion:    5.11.0-7620-generic
  MinAPIVersion:    3.1.0
  Os:               linux
 Engine:
  Version:          3.2.2
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.14.7
  Git commit:
  Built:            Thu Jan  1 01:00:00 1970
  OS/Arch:          linux/amd64
  Experimental:     true

Output of podman version:

$ podman version
Alias tip: pm version
Version:      3.2.2
API Version:  3.2.2
Go Version:   go1.14.7
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

Note this does not run after above issue, until podman.socket is restarted.

host:
  arch: amd64
  buildahVersion: 1.21.0
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 8
  distribution:
    distribution: pop
    version: "21.04"
  eventLogger: journald
  hostname: pop-os
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.11.0-7620-generic
  linkmode: dynamic
  memFree: 620052480
  memTotal: 16491663360
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1.5-925d-dirty
      commit: 0d42f1109fd73548f44b01b3e84d04a279e99d2e
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.1.8
      commit: unknown
      libslirp: 4.3.1-git
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 1265242112
  swapTotal: 4294434816
  uptime: 264h 49m 23.45s (Approximately 11.00 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/hosh/.config/containers/storage.conf
  containerStore:
    number: 3
    paused: 0
    running: 0
    stopped: 3
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
      Version: |-
        fusermount3 version: 3.10.2
        fuse-overlayfs: version 1.5
        FUSE library version 3.10.2
        using FUSE kernel interface version 7.31
  graphRoot: /home/hosh/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 57
  runRoot: /run/user/1000/containers
  volumePath: /home/hosh/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.2
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.14.7
  OsArch: linux/amd64
  Version: 3.2.2


Package info (e.g. output of rpm -q podman or apt list podman):

$ apt list podman
Listing... Done
podman/unknown,now 100:3.2.2-1 amd64 [installed]

Happy to provide additional details if necessary. Note, the docker-compsoe file used to work fine in rootful mode, though I haven't tested it with rootful since switching to rootless.

Sorry for the TMI, but these all seemed related to the same issue to me.

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jul 19, 2021
@hoshsadiq
Copy link
Author

Just realised I didn't include the archive.

testing.tar.gz

@Luap99
Copy link
Member

Luap99 commented Jul 20, 2021

Thanks for the detailed report @hoshsadiq. I think this is a duplicate of #10922 which is fixed in podman v3.2.3.

@Luap99 Luap99 closed this as completed Jul 20, 2021
@hoshsadiq
Copy link
Author

Aah! Pardon me. I did look but clearly missed it somehow. Thanks

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

2 participants