Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

podman-remote exec -i, with stdin redirection, hangs #7360

Closed
edsantiago opened this issue Aug 18, 2020 · 27 comments · Fixed by #10916
Closed

podman-remote exec -i, with stdin redirection, hangs #7360

edsantiago opened this issue Aug 18, 2020 · 27 comments · Fixed by #10916
Assignees
@mheon
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. remote Problem is in podman-remote

Comments

@edsantiago
Copy link
Member

In window 1:

$ ./bin/podman system service --timeout=0

In window 2:

$ ./bin/podman-remote run -d --name foo alpine sleep 20
60a88de15c87d44872b20f2716cd529de4ca0471527151456049dbc3dfeaa927

$ ./bin/podman-remote exec -i foo cat <<<"hi there"
hi there
<<<--- hangs here (no prompt, no nothing) until container sleep finishes. ^D does nothing.

Without -i there is no hang, but there is no output either.

Regular (non-remote) podman works fine: output is as expected, and the exec returns immediately.

master @ 748e882, root & rootless
@edsantiago edsantiago added kind/bug Categorizes issue or PR as related to a bug. remote Problem is in podman-remote labels Aug 18, 2020
@edsantiago edsantiago mentioned this issue Aug 18, 2020
Merged
edsantiago added a commit to edsantiago/libpod that referenced this issue Aug 19, 2020
@edsantiago
system tests: enable more remote tests; cleanup
d254fa4 
info, images, run, networking tests: remove some skip_if_remote()s
that were added in the varlink days. All of these tests now seem
to work with APIv2.

help test: check that first output line from 'podman --help'
is the program description (regression check for containers#7273).

load test: clean up stray images, rewrite test to make it conform
to existing convention. In the process, discover and file containers#7337

exec test (and networking): file containers#7360, and add FIXME comment
to skip()s suggesting evaluating those tests once that is fixed.

pod test: now that containers#6328 is fixed, use 'podman pod inspect --format'
instead of relying on jq

Various other tests: add an explanation of why test is disabled
so we can more easily distinguish "this will never be meaningful
under remote" vs "hey, doesn't work for now, but maybe someday".

Signed-off-by: Ed Santiago <[email protected]>
@vrothberg
Copy link
Member

@mheon, do you have cycles to tackle it? My guts tell me you'll know where to punch :)

@mheon
Copy link
Member

mheon commented Sep 8, 2020

Maybe? I'll self-assign, and we'll see how much I can get done this week.

@mheon mheon self-assigned this Sep 8, 2020
@rhatdan
Copy link
Member

rhatdan commented Oct 5, 2020

@mheon Any movement on this?

@mheon
Copy link
Member

mheon commented Oct 5, 2020

Not yet.

@github-actions
Copy link

github-actions bot commented Nov 5, 2020

A friendly reminder that this issue had no activity for 30 days.

@edsantiago
Copy link
Member Author

Still present in ab1d3a5; removing stale-issue label.

@github-actions
Copy link

github-actions bot commented Dec 6, 2020

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Dec 7, 2020

@mheon Reminder.

@mheon
Copy link
Member

mheon commented Dec 7, 2020

I will try and get to this in January.

@github-actions
Copy link

github-actions bot commented Jan 7, 2021

A friendly reminder that this issue had no activity for 30 days.

@edsantiago
Copy link
Member Author

Issue still present in master @ a475150

@edsantiago edsantiago mentioned this issue Feb 1, 2021
Closed
@mheon
Copy link
Member

mheon commented Feb 5, 2021

I wonder if Conmon 2.0.26 has resolved this - will re-test with it.

@github-actions
Copy link

github-actions bot commented Mar 8, 2021

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Mar 8, 2021

I am assuming this is fixed in Main, reopen if I am mistaken.

@rhatdan rhatdan closed this as completed Mar 8, 2021
@edsantiago edsantiago reopened this Mar 8, 2021
@christian-korneck christian-korneck mentioned this issue Mar 19, 2021
Closed
@christian-korneck
Copy link

christian-korneck commented Mar 22, 2021 •
edited
Loading

also seems to happen when using the docker cli over ssh to a podman server (3.0.1)

$ echo uname | docker -H ssh://<server> exec -i <container> sh
Linux
[hangs until aborted...]

docker cli: 20.10.5, build 55c4c88, macOS 11.2.3 x86_64
podman: 3.0.1, fedora 33 x86_64

(server symlinks /var/run/docker.sock -> /var/run/podman/podman.sock)

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@edsantiago
Copy link
Member Author

Still present in master @ a67aec7, conmon-2.0.27-2.fc33

One more data point I hadn't noticed before: the podman exec cat process exits 137 (kill -9). This is quite unexpected.

@github-actions
Copy link

github-actions bot commented Jun 2, 2021

A friendly reminder that this issue had no activity for 30 days.

@edsantiago
Copy link
Member Author

Still present in 6df37ab

@github-actions
Copy link

github-actions bot commented Jul 5, 2021

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Jul 5, 2021

@mheon This is the great albatross around podman's neck. ...

@christian-korneck christian-korneck mentioned this issue Jul 8, 2021
Closed
@mheon
Copy link
Member

mheon commented Jul 8, 2021

The core problem: EOF on the client isn't being handled properly. We need to get an EOF to the server, without actually closing the connection - we're not done until the server finished streaming output.

@rhatdan
Copy link
Member

rhatdan commented Jul 9, 2021

Seeing this is blocking vscode, it becomes more important.

@mheon
Copy link
Member

mheon commented Jul 12, 2021

Alright, I know generally what needs to be done here. We need to do a one-sided close on the hijacked connection (client needs to close the write side of the conn, while the read side is left open). Unfortunately, that's not enough - the backend is still not terminating, so I need to get into the HTTP attach code and figure out why it's not terminating.

@mheon
Copy link
Member

mheon commented Jul 12, 2021

Apparently it was a very simple fix. PR in progress.

@mheon mheon mentioned this issue Jul 12, 2021
Merged
mheon added a commit to mheon/libpod that referenced this issue Jul 13, 2021
@mheon
Perform a one-sided close of HTTP attach conn on EOF
cbbb1a8 
On EOF of STDIN, we need to perform a one-sided close of the
attach connection on the client side, to ensure that STDIN
finishing will also cause the exec session to terminate, instead
of hang.

Fixes containers#7360

Signed-off-by: Matthew Heon <[email protected]>
@edsantiago edsantiago mentioned this issue Jul 13, 2021
Open
@baude
Copy link
Member

baude commented Jul 14, 2021

@mheon beast!

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@mheon mheon

Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. remote Problem is in podman-remote
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Perform a one-sided close of HTTP attach conn on EOF mheon/libpod
6 participants
@edsantiago @rhatdan @christian-korneck @mheon @baude @vrothberg