pasting large amount of text into vim running inside of podman hangs container

[wdouglascampbell]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When pasting a large amount of text into a container environment (usually while editing a file in vim but also to a /bin/bash and /bin/sh shell prompt) the session hangs and there is no way to cancel, quit, exit, etc.

Steps to reproduce the issue:

1. run a busybox container

podman run --rm -it busybox /bin/sh

2. edit test file

vi test

3. enter insert mode and paste large amount of text

Describe the results you received:

Terminal hangs. I have to open new session and stop the podman container and sometimes that doesn't stop the process so I have to search for it and kill by the process id.

Describe the results you expected:

All content pasted is now in file being edited in vim.

Additional information you deem important (e.g. issue happens only occasionally):

This happens when running PuTTY 0.74. It doesn't happen if running SSH from Windows 10 command line.

It also doesn't happen if I run the container using docker run --rm -it busybox /bin/sh instead of using podman.

Output of podman version:

Version:            1.9.3
RemoteAPI Version:  1
Go Version:         go1.14.2
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  gitCommit: ""
  goVersion: go1.14.2
  podmanVersion: 1.9.3
host:
  arch: amd64
  buildahVersion: 1.14.9
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.18-1.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.18, commit: 6e8799f576f11f902cd8a8d8b45b2b2caf636a85'
  cpus: 12
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: file
  hostname: coreos.
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.7.8-200.fc32.x86_64
  memFree: 6526885888
  memTotal: 67493203968
  ociRuntime:
    name: runc
    package: runc-1.0.0-144.dev.gite6555cc.fc32.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10+dev
      commit: fbdbaf85ecbc0e077f336c03062710435607dbf1
      spec: 1.0.1-dev
  os: linux
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.1-1.fc32.x86_64
    version: |-
      slirp4netns version 1.1.1
      commit: bbf27c5acd4356edb97fa639b4e15e0cd56a39d5
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 2
  swapFree: 0
  swapTotal: 0
  uptime: 168h 33m 41.65s (Approximately 7.00 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 19
    paused: 0
    running: 19
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.1.2-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 74
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.9.3-1.fc32.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):

Fedora CoreOS
PuTTY 0.74

[wdouglascampbell]

Just to be clear. I don't have to be editing a file in vim. I could just the large amount of text at the shell prompt and things stop processing after a second.

[mheon]

Can you attach to the same container again, in a different terminal, after the hang? Does that work?

How much text are we talking about? A few hundred lines, or tens of thousands?

[wdouglascampbell]

In this specific case I am pasting 2500 lines of text.

I will try to re-attach to the container from a different terminal after the hang and report back.

[wdouglascampbell]

Okay. I attached to the still running container that is hung pasting into vim.

I ran ps aux inside the container and noticed that there is not a running vim process anymore eventhough the other terminal still is hung.

[Luap99]

I can reproduce this on current master with something like 230 lines.

[wdouglascampbell]

@Luap99 Thanks. Now at least I know I am not going crazy :) Yes, I have had it happen with less lines but the number of lines doesn't seem consistent but it has always failed for 2500 lines.

[Luap99]

It looks like the -t (tty) option triggers that state, it works when running the container without it.

[wdouglascampbell]

but I cannot interact with the container without the -t option, right? At least it doesn't output anything back to me.

[Luap99]

You can still type the commands in there is just no shell prompt. I know it is not very useful without the tty.

[mheon]

I'm trying to reproduce this locally, without success. Can you provide further details - is this a copy from another OS (over SSH) into a Podman container on a remote system? Anything unusual about the container?

[mheon]

The terminal does seem to go a little haywire for me after a large copy - I'm seeing some unusual behavior trying to scroll with the arrow keys, for example. Definitely not hanging though.

[mheon]

Nevermind - I managed to reproduce. Needed a significantly larger file than what I was using.

[Luap99]

@mheon I'm just running bin/podman run -it busybox vi test then go to https://podman.io and do a ctrl-a ctrl-c then paste this in.

[mheon]

podman attach to the container also seems hung. I'm betting this is Conmon.

[wdouglascampbell]

cool. thanks. was just about to respond.

[wdouglascampbell]

oh. I didn't do podman attach, I was using podman exec from the other terminal.

[mheon]

Conmon is definitely frozen - I'm stracing it, and it's hung on a write call (presumably to the container's stdin?)

[mheon]

Very interesting: when I ran strace a massive block of text popped up in the container - it seems like strace attaching to Conmon momentarily unblocked things?

[TomSweeneyRedHat]

@haircommander PING!

[haircommander]

I believe there's a decent amount of copying data to and from pipes with tty. it's possible there's something bigger than pipes buffering and having trouble catching up. I think moving this to the conmon repo makes sense and we can pick it up there

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[vrothberg]

Now moved over to conmon.

@haircommander, do you have cycles to tackle the issue? More users seem to hit it.

[haircommander]

I am not sure I have the cycles soon, so if y'all have anyone who can look at it, that would get this fixed faster. I will try to take a look in the coming weeks if not.

[giuseppe]

opened a PR: #223

[justinkb]

this broke conmon for me: see here containers/podman#9183