Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support additional linux distro when a podman machine is created #12011

Closed
cmoulliard opened this issue Oct 18, 2021 · 19 comments
Closed

Support additional linux distro when a podman machine is created #12011

cmoulliard opened this issue Oct 18, 2021 · 19 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@cmoulliard
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

Suggestion: Support additional linux distro when a podman machine is created.

I downloaded a qcow2 image of Centos8 to create a podman VM but the VM do not boot.

Questions:

  • Is it possible to create a podman VM using different Linux OS/Distro (Cento7, Centos8) ?
  • If we can use a different distro, could it be possible document what we should acheive to let podman machine to boot it
    as currently that fails
podman machine init --image-path ~/Images/CentOS-8-GenericCloud-8.4.2105.x86_64.qcow2 centos8

podman machine list
NAME                     VM TYPE     CREATED         LAST UP         CPUS        MEMORY      DISK SIZE
centos8                  qemu        26 seconds ago  26 seconds ago  1           2.147GB     10.74GB
podman-machine-default*  qemu        5 weeks ago     30 minutes ago  1           2.147GB     0B

podman machine start centos8
0000] podman filtering at log level debug
DEBU[0000] [/usr/local/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/cmoullia/.config/containers/podman/machine/qemu/centos8.ign -qmp unix://var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/qmp_centos8.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/centos8_ready.sock,server=on,wait=off,id=centos8_ready -device virtserialport,chardev=centos8_ready,name=org.fedoraproject.port.0 -machine q35,accel=hvf:tcg -cpu host -drive if=virtio,file=/Users/cmoullia/.local/share/containers/podman/machine/qemu/centos8_CentOS-8-GenericCloud-8.4.2105.x86_64.qcow2]
[/usr/local/opt/podman/libexec/gvproxy -listen tcp://0.0.0.0:7777 -listen-qemu unix:///var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/qemu_centos8.sock -pid-file /var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/centos8.pid -ssh-port 59432 --debug]
DEBU[0000] Using search domains: [redhat.com win.redhat.com]
INFO[0000] waiting for clients...
INFO[0000] listening tcp://0.0.0.0:7777
DEBU[0000] [/usr/local/bin/qemu-system-x86_64 -m 2048 -smp 1 -fw_cfg name=opt/com.coreos/config,file=/Users/cmoullia/.config/containers/podman/machine/qemu/centos8.ign -qmp unix://var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/qmp_centos8.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/centos8_ready.sock,server=on,wait=off,id=centos8_ready -device virtserialport,chardev=centos8_ready,name=org.fedoraproject.port.0 -machine q35,accel=hvf:tcg -cpu host -drive if=virtio,file=/Users/cmoullia/.local/share/containers/podman/machine/qemu/centos8_CentOS-8-GenericCloud-8.4.2105.x86_64.qcow2]
INFO[0000] new connection from  to /var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/podman/qemu_centos8.sock
Waiting for VM ...
0 B sent to the VM, 0 B received from the VM
...
- Layer 1 (14 bytes) = Ethernet	{Contents=[..14..] Payload=[..28..] SrcMAC=5a:94:ef:e4:0c:dd DstMAC=5a:94:ef:e4:0c:ee EthernetType=ARP Length=0}
- Layer 2 (28 bytes) = ARP	{Contents=[..28..] Payload=[] AddrType=Ethernet Protocol=IPv4 HwAddressSize=6 ProtAddressSize=4 Operation=2 SourceHwAddress=[..6..] SourceProtAddress=[192, 168, 127, 1] DstHwAddress=[..6..] DstProtAddress=[192, 168, 127, 2]}
DEBU[0103] Stopping udp proxy (read udp 129.250.35.250:123: i/o timeout)
DEBU[0104] Stopping udp proxy (read udp 83.98.201.134:123: i/o timeout)
DEBU[0104] Stopping udp proxy (read udp 87.233.197.123:123: i/o timeout)
DEBU[0104] Stopping udp proxy (read udp 45.159.204.28:123: i/o timeout)
2.8 kB sent to the VM, 3.4 kB received from the VM

Steps to reproduce the issue:

  1. Download a qcow2 file : https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.4.2105-20210603.0.x86_64.qcow2

  2. podman machine init --image-path ~/Images/CentOS-8-GenericCloud-8.4.2105.x86_64.qcow2 centos8

  3. podman machine start centos8

Describe the results you received:

VM do not boot

Output of podman version:

podman -v
podman version 3.4.0

OS`:

MacOS:  BigSur - 11.6
@openshift-ci openshift-ci bot added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 18, 2021
@baude
Copy link
Member

baude commented Oct 18, 2021

it is possible to boot another distribution but that distribution must use ignition during boot and have a recent podman inside it. The doc request is perfectly valid.

@cmoulliard
Copy link
Author

use ignition during boot and have a recent podman inside it

Is it documented how we could achieve that ?

@baude
Copy link
Member

baude commented Oct 18, 2021

ignition is not part of podman but a separate package -> https://coreos.github.io/ignition/

@afbjorklund
Copy link
Contributor

afbjorklund commented Oct 18, 2021

You can use lima instead of podman, if you want to use cloud-init instead of ignition.

Something like https://github.com/afbjorklund/fedora-lima/blob/main/fedora-podman.yaml

Support for CentOS is left as an exercise, but I think it is soon going to be EOL anyway ?


And here it is: https://github.com/afbjorklund/fedora-lima/blob/main/centos-podman.yaml

$ export LIMA_INSTANCE=centos-podman
$ lima podman version
bash: line 0: cd: /home/anders/fedora-lima: No such file or directory
Version:      3.2.3
API Version:  3.2.3
Go Version:   go1.15.14
Built:        Thu Sep 23 21:22:19 2021
OS/Arch:      linux/amd64

Unfortunately sshfs is broken, so it doesn't really work out-of-the-box. Probably a simple fix...

[   38.495119] cloud-init[1376]: + dnf install -y fuse-sshfs
[   39.845344] cloud-init[1376]: CentOS Linux 8 - AppStream                       13 MB/s | 9.3 MB     00:00
[   42.306361] cloud-init[1376]: CentOS Linux 8 - BaseOS                         9.6 MB/s | 7.5 MB     00:00
[   44.128504] cloud-init[1376]: CentOS Linux 8 - Extras                          21 kB/s |  10 kB     00:00
[   44.950542] cloud-init[1376]: No match for argument: fuse-sshfs
[   44.973079] cloud-init[1376]: Error: Unable to find a match: fuse-sshfs

Fixed in:

yum install --enablerepo powertools -y fuse-sshfs

@afbjorklund
Copy link
Contributor

afbjorklund commented Dec 15, 2021

Updated the lima examples to Fedora 35 and CentOS 8-Stream, since CentOS Linux is soon EOL...

limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/fedora-podman.yaml

limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/centos-podman.yaml

They use the "Cloud" versions (with cloud-init), rather than the "CoreOS" versions (with ignition).


$ export LIMA_INSTANCE=centos-podman
$ lima podman version
Version:      3.4.1-dev
API Version:  3.4.1-dev
Go Version:   go1.16.7
Built:        Tue Oct 19 16:11:42 2021
OS/Arch:      linux/amd64
$ lima sh -c 'rpm -q $(rpm -q --whatprovides system-release)'
centos-stream-release-8.5-3.el8.noarch
$ lima rpm -q podman
podman-3.4.1-3.module_el8.6.0+954+963caf36.x86_64
fedora-release-cloud-35-33.noarch
podman-3.4.2-1.fc35.x86_64

See https://github.com/afbjorklund/fedora-lima for details

@cmoulliard
Copy link
Author

I did a test successfully on my mac using limactl start https://raw.githubusercontent.com/afbjorklund/fedora-lima/main/centos-podman.yaml

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 5, 2022

The new lima version has automatic mounting of the podman.sock, haven't ported that from ubuntu over to fedora yet.

limactl start https://raw.githubusercontent.com/lima-vm/lima/v0.8.0/examples/podman.yaml

INFO[0067] READY. Run `limactl shell podman` to open the shell. 
INFO[0067] To run `podman` on the host (assumes podman-remote is installed): 
INFO[0067] $ export CONTAINER_HOST=unix:///home/anders/.lima/podman/sock/podman.sock 
INFO[0067] $ podman-remote ...      

(requires 0.8.0)


EDIT: updated the example now

portForwards:
  - guestSocket: "/run/user/{{.UID}}/podman/podman.sock"
    hostSocket: "{{.Dir}}/sock/podman.sock"

message: |
  To run `podman` on the host (assumes podman-remote is installed):
  $ export CONTAINER_HOST=unix://{{.Dir}}/sock/podman.sock
  $ podman{{if eq .HostOS "linux"}}-remote{{end}} ...

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 5, 2022

@cmoulliard

Suggestion: Support additional linux distro when a podman machine is created.

I'm not sure it is worth the effort, to support cloud-init in addition to ignition ?

@baude ?

Before that has been done, Podman will only be able to support CoreOS for the VM.

It is easier to just use Vagrant* or Lima, if all you want to do is run Podman in a VM ?

* Something like: https://boot2podman.github.io/2020/07/22/machine-replacement.html

@cmoulliard
Copy link
Author

It is easier to just use Vagrant* or Lima, if all you want to do is run Podman in a VM ?

if using lima will allow on macos to do what we could do using docker desktop, then I'm fine

@afbjorklund
Copy link
Contributor

if using lima will allow on macos to do what we could do using docker desktop, then I'm fine

If you are happy with Docker Desktop, then by all means do continue to use Docker Desktop...
But if there is something missing in Podman or in Lima, then do open issues about it explicitely.

As far as I know, Docker Desktop allows you to run the LinuxKit distribution - and nothing else ?
So I don't see how Podman Machine would be different in this regard, you get Fedora CoreOS.

@cmoulliard
Copy link
Author

If you are happy with Docker Desktop, then by all means do continue to use Docker Desktop...

Not really as it only allows to boot a VM running k8s or docker and not buildah and/or podman ;-) and the VM cannot realy be used as a VM to ssh, install software, .... as we can do with vagrant or lima

@cmoulliard
Copy link
Author

As far as I know, Docker Desktop allows you to run the LinuxKit distribution - and nothing else ?

Yes but we cannot really use locally the VM to ssh, install software and compile code by example

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 6, 2022

I think you were supposed to do that in containers, not on the VM.. But anyway, seems off topic ?

The Podman Machine does one thing: run the Podman daemon. For the moment, not even buildah...
It makes it really hard to install software (without rebooting) and not very good to compile code on either.
Normally you would run podman-remote, and do your installation and compilation with run and build.
Using Fedora Cloud can be useful to bring the footprint down, but other than that it would work the same.

360M	Fedora-Cloud-Base-35-1.2.x86_64.qcow2

1,5G	fedora-coreos-35.20211215.3.0-qemu.x86_64.qcow2

There used to be support for other distributions, but it was taken out in favor of using Fedora CoreOS.

It seems unlikely that it would be added back in, since there is a general trend going back to OpenShift 4...
And that is to use CoreOS as the base for running containers on, even if possible to also use the other OS.
With the push towards Podman Desktop, the OS and SSH is likely to get less and less "visible" to the user.
Ultimately it will just be a background service on macOS, similar to the podman systemd service on Linux.

@afbjorklund
Copy link
Contributor

On the off chance that someone would like to make a custom ISO for Fedora, I added some documentation.
Otherwise it will use the regular Fedora Cloud (for OpenStack) image, when running podman with Lima...

But that brings in a lot of overhead, so it is possible to do a more customized "spin" running as a "Live CD".
There is a similar project from Rancher, running containerd with Alpine. But that is enough off-topic for now.

@afbjorklund
Copy link
Contributor

@cmoulliard : time to close ?

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 18, 2022

I leave the old Fedora Linux docs up there in case anyone wants to re-open that thread in the future.

Otherwise it should be plenty with Fedora CoreOS (podman) and Fedora Cloud (lima), meanwhile.
Since podman requires ignition and lima requires cloud-init, those are more or less necessary anyway.

The alternative would be to use Alpine Linux, which already has a podman package in the aports...

lima-alpine:/home/anders/lima$ more /etc/os-release 
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.14.3
PRETTY_NAME="Alpine Linux v3.14"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
BUILD_ID="v0.2.2"
VARIANT_ID="std"
lima-alpine:/home/anders/lima$ podman version
Version:      3.2.3
API Version:  3.2.3
Go Version:   go1.16.8
Git Commit:   b7fc5b608b641cd9b9aec2647f9236e31f8f3b27
Built:        Sat Sep 11 14:33:25 2021
OS/Arch:      linux/amd64

@cmoulliard
Copy link
Author

The content discussed here could become part of a FAQ detailing the alternatives which exist to play with podman using distro A,B, C and tools. WDYT ? @afbjorklund

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 19, 2022

There are no alternatives, afaik. Could add more documentation to Lima, but it doesn't have a proper website or manual yet.
(It also defaults to running ubuntu and nerdctl, so changing the tool over to fedora and podman will always be a bit awkward.)

Documentation for using Podman (and Podman Machine) is at https://docs.podman.io/

It should cover how to install and run podman, on a selection of (but not all) distributions ?

https://podman.io/getting-started/installation


Theoretically you could change Podman to use RHCOS too, but for now it is: FCOS.

Any customer can have a car painted any color that he wants so long as it is black.

You can change the image stream: https://getfedora.org/en/coreos/download

  • Stable
  • Testing
  • Next

There are plenty of other VM tools (like Vagrant), if you need more than "just Podman".

@cmoulliard
Copy link
Author

Alternative was not perhaps the appropriate word. Sorry.
What I'm proposing is to list under podman web site the option which exists to launch on a local machine a VM with different flavors (lima, vagrant) and to mention that podman supports only ignition vs lima cloud-init ... That will help the developers to figure what they will be able to do or not such as add package, ssh to the VM, ...

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

3 participants