podman-remote setup for lima-ubuntu vm #11533

tk185141 · 2021-09-11T11:58:57Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

No.. This is more of HOWTO or question
Description
Unable to find the documentation for connecting to the lima-ubuntu VM. There is documentation for podman but not for the ubuntu (https://github.com/lima-vm/lima/blob/master/examples/podman.yaml#L5-L9)

Steps to reproduce the issue:

Setup a lima ubuntu vm
Apt install podman and buildah (did this separately for testing purposes_
Use both podman and buildah to build OCI images
Validated I can see the image from podman in podman images command
Use podman-remote from the host to try to connect to lima ubuntu vm

Describe the results you received:
podman-remote is returning the podman images from the lima-podman VM instead of lima-ubuntu vm

Describe the results you expected:
podman-remote can be configured to ssh to the lima ubuntu vm

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:
Host

WUSTK185141-CV3:test tk185141$ podman-remote --version
podman-remote version 3.3.1

Guest

tk185141@lima-ubuntu:/Users/tk185141/test$ podman --version
podman version 3.0.1

Output of podman info --debug:

tk185141@lima-ubuntu:/Users/tk185141/test$ podman info --debug
host:
  arch: amd64
  buildahVersion: 1.19.6
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 4
  distribution:
    distribution: ubuntu
    version: "21.04"
  eventLogger: journald
  hostname: lima-ubuntu
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1640274643
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.11.0-31-generic
  linkmode: dynamic
  memFree: 1504284672
  memTotal: 4111011840
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17
      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1640274643/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: /usr/local/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.3.3
  swapFree: 0
  swapTotal: 0
  uptime: 5h 15m 12.83s (Approximately 0.21 days)
registries: {}
store:
  configFile: /home/tk185141.linux/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/local/bin/fuse-overlayfs
      Package: Unknown
      Version: |-
        fusermount3 version: 3.10.2
        fuse-overlayfs: version 1.7.1
        FUSE library version 3.4.1
        using FUSE kernel interface version 7.27
  graphRoot: /home/tk185141.linux/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 7
  runRoot: /run/user/1640274643/containers
  volumePath: /home/tk185141.linux/.local/share/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.16
  OsArch: linux/amd64
  Version: 3.0.1

Package info (e.g. output of rpm -q podman or apt list podman):

tk185141@lima-ubuntu:/Users/tk185141/test$ apt list podman
Listing... Done
podman/hirsute,now 3.0.1+dfsg1-1ubuntu1 amd64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Found this article multiple times and decided that even though it says will not work for mac, I figured to give it a shot. This entire process was run inside of the guest lima VM
Start a ubuntu shell inside of lima
WUSTK185141-CV3:~ tk185141$ limactl start ubuntu
Install buildah
tk185141@lima-ubuntu:/Users/tk185141$ sudo apt install buildah
Build an OCI image using buildah
tk185141@lima-ubuntu:/Users/tk185141/test$ buildah bud -f ./Dockerfile -t my-app-buildah .
Install podman
tk185141@lima-ubuntu:/Users/tk185141/test$ sudo apt install podman
Run podman build
tk185141@lima-ubuntu:/Users/tk185141/test$ podman build -t testimage .
Run podman images

tk185141@lima-ubuntu:/Users/tk185141/test$ podman images
REPOSITORY        TAG   IMAGE ID   CREATED     SIZE
localhost/testimage    latest d6e5f702489e 20 minutes ago 167 MB
localhost/my-app-buildah latest 1d486b95d2b5 22 minutes ago 167 MB
docker.io/library/ubuntu latest fb52e22af1b0 10 days ago   75.2 MB

The text was updated successfully, but these errors were encountered:

afbjorklund · 2021-09-11T14:07:09Z

You can use podman-remote system connection ls to see the various connections, and which is the default (marked with *).

Then you can change it with podman-remote system connection default, or use --connection / $CONTAINER_CONNECTION

Note: this is instead of / in addition to the legacy CONTAINER_HOST and CONTAINER_SSHKEY environment variables

(Which are in turn instead of the PODMAN_USER/PODMAN_HOST/PODMAN_PORT/PODMAN_IDENTITY_FILE env var)

afbjorklund · 2021-09-11T14:36:58Z

limactl start ubuntu.yaml
limactl shell ubuntu

sudo apt update
sudo apt install -y podman

You should be able to start the socket listener with:

$ limactl shell ubuntu systemctl --user start podman.socket

And then see the connection details with limactl ls:

NAME      STATUS     SSH                ARCH      DIR
ubuntu    Running    127.0.0.1:60022    x86_64    /home/anders/.lima/ubuntu

Set up a connection:

$ podman-remote system connection add --default ubuntu ssh://127.0.0.1:60022

And run it:

$ podman-remote version
Error: cannot connect to the Podman socket, please verify that Podman REST API service is running: server API version is too old. Client "3.1.0" server "3.0.0"
$ podman-remote --version
podman version 3.2.3
$ limactl shell ubuntu podman --version
podman version 3.0.1

The error is normal, since podman version 3.2.3 can't talk to podman version 3.0.1.

If installing a matching older version on the host, then it works fine to connect to it.

https://packages.ubuntu.com/hirsute/podman

Alternatively one could install a newer ubuntu podman from an alternative source.

There are some packages available from the Kubic project, see podman: Get Started

https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable/podman

A temporary workaround is also to run the server podman binary remotely over ssh:

# better living through environment
$ export LIMA_INSTANCE=ubuntu
$ lima podman version
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.16
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64
$ lima podman --remote version
Client:
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.16
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

Server:
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.16
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

tk185141 · 2021-09-11T21:43:08Z

Great. This is extremely helpful and for now, I will leverage the workaround. Can we update the docs for this scenario or point me to where it is in case I missed it.

mheon · 2021-09-12T06:13:12Z

Don't think we have any documentation on using Lima with Podman at the moment (our focus primarily being on podman machine). If someone wants to write some, we'd be glad to merge it - PRs always welcome!

afbjorklund · 2021-09-12T07:08:07Z

Currently all documentation and config files are on the Lima side. "(Other container engines can be used too)"

https://github.com/lima-vm/lima#lima-linux-virtual-machines-on-macos-in-most-cases

Lima launches Linux virtual machines with automatic file sharing, port forwarding, and containerd.
Lima can be considered as a some sort of unofficial "macOS subsystem for Linux", or "containerd for Mac".

Eventually this will be alternative for Podman on Mac (and Linux), just like running under WSL is on Windows:

https://podman.io/blogs/2021/09/06/podman-on-macs.html

We also have user reports that running Podman in WSL is quite tenable.
Consider the WSL option if you do not have available Linux servers with Podman installed.

afbjorklund · 2021-09-12T07:44:45Z

But lima should work "out of the box" if using the provided "podman" image, the problem was with the "ubuntu" image.

ubuntu.yaml
- ubuntu 21.04: https://wiki.ubuntu.com/HirsuteHippo
- podman 3.0.1: https://packages.ubuntu.com/hirsute/podman
podman.yaml
- ubuntu 21.10: https://wiki.ubuntu.com/ImpishIndri
- podman 3.2.1: https://packages.ubuntu.com/impish/podman

It should be two lines:

$ limactl start podman.yaml
? Creating an instance "podman" Proceed with the default configuration
INFO[0001] Attempting to download the image from "https://cloud-images.ubuntu.com/impish/current/impish-server-cloudimg-amd64.img" 
...
INFO[0098] READY. Run `limactl shell podman` to open the shell. 
$ LIMA_INSTANCE=podman lima podman version
Version:      3.2.1
API Version:  3.2.1
Go Version:   go1.16.2
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

Or some more, for remote.

# Hint: To allow `podman` CLI on the host to connect to the Podman daemon running inside the guest,
# add `NoHostAuthenticationForLocalhost yes` in ~/.ssh/config , and then run the following commands:
# $ export CONTAINER_HOST=ssh://$(id -un)@localhost:60906/run/user/$(id -u)/podman/podman.sock
# $ export CONTAINER_SSHKEY=$HOME/.lima/_config/user
# $ podman ...

# Hint: To allow `docker` CLI on the host to connect to the Podman daemon running inside the guest, run the following commands:
# $ ssh -f -N -p 60906 -i ~/.lima/_config/user -o NoHostAuthenticationForLocalhost=yes -L $HOME/podman.sock:/run/user/$(id -u)/podman/podman.sock 127.0.0.1
# $ export DOCKER_HOST=unix://$HOME/podman.sock
# $ docker ...

You would probably have similar issues with setting it up, if trying to run Podman Machine with say an Ubuntu image ?

The user experience should be better, after fixing some issues with displaying the connections and creating a socket :

After those, setting up the remote environment would also be simple one-liners. Well, except for some other issues...

afbjorklund · 2021-09-12T08:03:08Z

@tk185141

Can we update the docs for this scenario or point me to where it is in case I missed it.

As always, the websites and the documentation are behind the code... "Patches accepted" ?

afbjorklund · 2021-09-12T19:43:41Z

You can find older podman client binaries under the GitHub releases, such as v3.0.1

$ podman version
Version:      3.2.3
API Version:  3.2.3
Go Version:   go1.15.2
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64
$ podman-remote version
Error: cannot connect to the Podman socket, please verify that Podman REST API service is running: server API version is too old. Client "3.1.0" server "3.0.0"
$ podman-remote-static version
Client:
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.15.8
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Server:
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.16
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

rhatdan · 2021-09-17T13:59:53Z

@tk185141 are you planning on opening a PR to update the documentation on how to do this?

afbjorklund · 2021-09-17T14:12:20Z

Would you need some external link similar to WSL*, or what is needed for Podman on lima ?

* https://www.redhat.com/sysadmin/podman-windows-wsl2

Seems to be the same basic approach, install the Linux distribution of your choice and then add Podman to it.
The rest seemed to be the same as with any podman-remote installation, setting up the the ssh "connection" ?

rhatdan · 2021-09-17T14:15:33Z

Sure a Blog on this would be great.

afbjorklund · 2021-09-17T14:49:35Z

Just couldn't find any info on either (on podman.io), to PR against. I described my experience when running on Linux (above), but I think it would be more interesting to hear from a Mac user - similar to the article that @TomSweeneyRedHat wrote for Win 10.

https://www.redhat.com/sysadmin/podman-windows-wsl

And probably Fedora, rather than Ubuntu - right ? 😉

afbjorklund · 2021-10-11T17:52:19Z

Here is such a YAML file, for running podman with fedora instead: https://github.com/afbjorklund/fedora-lima

$ brew install lima

limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/fedora-podman.yaml

$ export LIMA_INSTANCE=fedora-podman
$ lima podman version
Version:      3.4.0
API Version:  3.4.0
Go Version:   go1.16.8
Built:        Thu Sep 30 19:40:21 2021
OS/Arch:      linux/amd64

github-actions · 2021-11-11T00:03:58Z

A friendly reminder that this issue had no activity for 30 days.

rhatdan · 2021-11-11T13:48:56Z

@tk185141 @afbjorklund Is this something you are willing to write up a document on to explain how to set this up, then we could add it to the installation guide.

vrothberg · 2021-11-17T12:57:21Z

Friendly ping

afbjorklund · 2021-11-17T13:02:20Z

I think it was waiting for another magazine article, probably best if written by a Mac user (instead of Linux) ?

https://www.redhat.com/sysadmin/topics/containers

afbjorklund · 2021-11-17T21:05:17Z

Hopefully it was fairly straight-forward, how to install lima and how to make it run podman/fedora instead of nerdctl/ubuntu.

brew install lima

ubuntu-nerdctl

[macOS]$ limactl start
...
INFO[0029] READY. Run `lima` to open the shell.

[macOS]$ lima uname
Linux

fedora-podman

[macOS]$ limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/fedora-podman.yaml
...
INFO[0108] READY. Run `limactl shell fedora-podman` to open the shell. 

[macOS]$ export LIMA_INSTANCE=fedora-podman
[macOS]$ lima uname
Linux

Maybe @ashley-cui have time to give it a try ?

The GUI is available from here: lima-gui.app (zip)

There has been some improvements since, so now the unix socket is directly exposed on the Mac machine (through tunnel).

$ export CONTAINER_HOST=unix://$HOME/podman.sock
$ podman --remote version
Client:
Version:      3.3.1
API Version:  3.3.1
Go Version:   go1.16.6
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Server:
Version:      3.4.1
API Version:  3.4.1
Go Version:   go1.16.8
Built:        Wed Oct 20 16:35:28 2021
OS/Arch:      linux/amd64

There is a fair share of overlap between podman machine and limactl (such as QEMU), it runs cloud-init instead of ignition.

$ limactl list
NAME             STATUS     SSH                ARCH      CPUS    MEMORY    DISK      DIR
default          Running    127.0.0.1:60022    x86_64    4       4GiB      100GiB    /home/anders/.lima/default
fedora-podman    Running    127.0.0.1:36745    x86_64    2       2GiB      20GiB     /home/anders/.lima/fedora-podman

afbjorklund · 2021-12-15T19:40:09Z

The lima messages have been improved a bit in lima version 0.8.0, so that it now gives actionable output:

INFO[0025] READY. Run `limactl shell podman` to open the shell. 
INFO[0025] To run `podman` on the host (assumes podman-remote is installed): 
INFO[0025] $ export CONTAINER_HOST=unix:///home/anders/.lima/podman/sock/podman.sock 
INFO[0025] $ podman-remote ...

The location of the socket and the podman binary, depends on the current user and operating system.

# To run `podman` on the host (assumes podman-remote is installed):
# $ export CONTAINER_HOST=$(limactl list podman --format 'unix://{{.Dir}}/sock/podman.sock')
# $ podman --remote ...

# To run `docker` on the host (assumes docker-cli is installed):
# $ export DOCKER_HOST=$(limactl list podman --format 'unix://{{.Dir}}/sock/podman.sock')
# $ docker ...

See https://github.com/lima-vm/lima/blob/v0.8.0/examples/podman.yaml (currently: podman 3.2.1)

https://packages.ubuntu.com/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

Instead of using the remote client, it is also possible to run podman in the instance using the shell command:

limactl shell podman podman

For newer podman versions, see the alternative lima yaml files using fedora instead of the default ubuntu:

limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/fedora-podman.yaml

github-actions · 2022-01-15T00:05:03Z

A friendly reminder that this issue had no activity for 30 days.

rhatdan · 2022-01-15T15:03:14Z

This seems to be more of a discussion then an issue. Moving to discussion.

AkihiroSuda added Question Issue is a question about Podman remote Problem is in podman-remote labels Sep 11, 2021

afbjorklund mentioned this issue Sep 15, 2021

Support native source folder for volume mount in remote model #8016

Closed

This comment has been minimized.

Sign in to view

afbjorklund mentioned this issue Sep 19, 2021

Add installation instructions link to running podman with lima containers/podman.io_old#458

Open

afbjorklund mentioned this issue Oct 18, 2021

Support additional linux distro when a podman machine is created #12011

Closed

github-actions bot added the stale-issue label Nov 11, 2021

rhatdan removed the stale-issue label Nov 11, 2021

github-actions bot added the stale-issue label Jan 15, 2022

rhatdan removed the stale-issue label Jan 15, 2022

containers locked and limited conversation to collaborators Jan 15, 2022

rhatdan converted this issue into discussion #12878 Jan 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

This issue was moved to a discussion.

podman-remote setup for lima-ubuntu vm #11533

podman-remote setup for lima-ubuntu vm #11533

tk185141 commented Sep 11, 2021

afbjorklund commented Sep 11, 2021 •

edited

Loading

afbjorklund commented Sep 11, 2021 •

edited

Loading

tk185141 commented Sep 11, 2021

mheon commented Sep 12, 2021

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

rhatdan commented Sep 17, 2021

afbjorklund commented Sep 17, 2021

rhatdan commented Sep 17, 2021

afbjorklund commented Sep 17, 2021 •

edited

Loading

This comment has been minimized.

afbjorklund commented Oct 11, 2021 •

edited

Loading

github-actions bot commented Nov 11, 2021

rhatdan commented Nov 11, 2021

vrothberg commented Nov 17, 2021

afbjorklund commented Nov 17, 2021 •

edited

Loading

afbjorklund commented Nov 17, 2021

afbjorklund commented Dec 15, 2021 •

edited

Loading

github-actions bot commented Jan 15, 2022

rhatdan commented Jan 15, 2022

This issue was moved to a discussion.

This issue was moved to a discussion.

podman-remote setup for lima-ubuntu vm #11533

podman-remote setup for lima-ubuntu vm #11533

Comments

tk185141 commented Sep 11, 2021

afbjorklund commented Sep 11, 2021 • edited Loading

afbjorklund commented Sep 11, 2021 • edited Loading

tk185141 commented Sep 11, 2021

mheon commented Sep 12, 2021

afbjorklund commented Sep 12, 2021 • edited Loading

afbjorklund commented Sep 12, 2021 • edited Loading

afbjorklund commented Sep 12, 2021 • edited Loading

afbjorklund commented Sep 12, 2021 • edited Loading

rhatdan commented Sep 17, 2021

afbjorklund commented Sep 17, 2021

rhatdan commented Sep 17, 2021

afbjorklund commented Sep 17, 2021 • edited Loading

This comment has been minimized.

afbjorklund commented Oct 11, 2021 • edited Loading

github-actions bot commented Nov 11, 2021

rhatdan commented Nov 11, 2021

vrothberg commented Nov 17, 2021

afbjorklund commented Nov 17, 2021 • edited Loading

afbjorklund commented Nov 17, 2021

ubuntu-nerdctl

fedora-podman

afbjorklund commented Dec 15, 2021 • edited Loading

github-actions bot commented Jan 15, 2022

rhatdan commented Jan 15, 2022

This issue was moved to a discussion.

afbjorklund commented Sep 11, 2021 •

edited

Loading

afbjorklund commented Sep 11, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 12, 2021 •

edited

Loading

afbjorklund commented Sep 17, 2021 •

edited

Loading

afbjorklund commented Oct 11, 2021 •

edited

Loading

afbjorklund commented Nov 17, 2021 •

edited

Loading

afbjorklund commented Dec 15, 2021 •

edited

Loading