Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please provide repo for docker-ce on Fedora 32 #955

Closed
2 of 3 tasks
hhlp opened this issue Mar 16, 2020 · 140 comments
Closed
2 of 3 tasks

Please provide repo for docker-ce on Fedora 32 #955

hhlp opened this issue Mar 16, 2020 · 140 comments

Comments

@hhlp
Copy link

hhlp commented Mar 16, 2020

  • This is a bug report
  • This is a feature request
  • I searched existing issues before opening this one

The Docker-CE Fedora repository is the recommend way to install recommended way. It has many benefits over installing from a package or any other method.

Fedora also releases a new version every six months. The Fedora repository also takes a very long time to become ready after the release of a new version, with many issues being opened after each recent Fedora is release.

Fedora has officially branched Version 32 February 11/2020 and might have a beta release on March 17/2020.

I'm filing this issue to hopefully put this task on the radar, so that there might be a repository set up in time for the Fedora 32 release cycle....

NOTE

Some of us migrate when the beta version is ready so we have to disable the repository and wait for it to be ready

Release Schedule: https://fedorapeople.org/groups/schedule/f-32/f-32-key-tasks.html

Regards.,

@vincent-olivert-riera
Copy link

It seems this won't happen on time for tomorrow's release. Perhaps the focus has been put on getting cgroupsv2 working? opencontainers/runc#2315

@kiview
Copy link
Member

kiview commented Apr 28, 2020

Just noticed that Docker container have no internet connectivity after upgrade to Fedora 32. User needs to add docker network interface to trusted firewall zone. I hope official package will keep this in mind and configure firewalld accordingly.

@christian-korneck
Copy link

christian-korneck commented Apr 29, 2020

I have upgraded my Fedora 31 workstation to Fedora 32. Here are some issues I've experienced with Docker (most of them have been reported by other people above, but I'm trying to give some details).

1.) Since Fedora 31 cgroups v2 is enabled by default. To be able to run Docker you need to revert to cgroups v1 by setting a parameter in the kernel cmdline. (Not a bug, just mentioning it for completeness).

grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
reboot

2.) There is no Fedora 32 repo for Docker, so updates fail. (I had Docker already installed on Fedora 31).

$ dnf repolist docker-ce-stable --verbose
error: Status code: 404 for http://download.docker.com/linux/fedora/32/repodata/repomd.xml (IP: 13.35.253.112) (http://download.docker.com/linux/fedora/32/repodata/repomd.xml).

3.) there's an iptables issue. From inside a Docker container pinging any destination works. However, tcp/udp connections don't work (i.e. the local DNS server set by Docker in the container can only be pinged, but not reached on the DNS service).

journalctl -t firewalld shows these errors after booting:

firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -D FORWARD -i docker0 -o docker0 -j DROP' failed: iptables: Bad rule (does a matching rule exist in that chain?).
firewalld[854]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -D FORWARD -i docker0 -o docker0 -j DROP' failed: iptables: Bad rule (does a matching rule exist in that chain?).

As someone mentioned above, adding the Docker interface to the trusted firewall zone is a quick workaround:

firewall-cmd --permanent --zone=trusted --add-interface=docker0
firewall-cmd --reload

@davidkna
Copy link

The networking issues appear to be caused by firewalld switching to nftables. The changeset also has some recommendations for solvings this issue which to also appear to amount to making docker0 part of a trusted zone.

@Bouhnosaure
Copy link

Bouhnosaure commented Apr 29, 2020

I had this same issue and i wasn't able to ping anything within a container, I've fixed it with :

firewall-cmd --get-zone-of-interface=enp1s0
   FedoraWorkstation

firewall-cmd --zone=FedoraWorkstation --add-masquerade --permanent

firewall-cmd --reload

@nlzet
Copy link

nlzet commented Apr 29, 2020

I had this same issue and i wasn't able to ping anything within a container, I've fixed it with :

firewall-cmd --get-zone-of-interface=enp1s0

   FedoraWorkstation



firewall-cmd --zone=FedoraWorkstation --add-masquerade --permanent



firewall-cmd --reload

another solution for now is changing the firewalld.conf “FirewallBackend” from nftables to iptables

@quackerd
Copy link

Changing FirewallBackend to iptables worked for me on CentOS 8 which defaults to nftables like Fedora 32 does. Docker or libnetwork need to support nftables natively.

@KevinAtSesam
Copy link

KevinAtSesam commented Apr 30, 2020

I had trouble connecting to my local Kafka instance after upgrading to Fedora 32. Something with the routing messed up. I could connect to my local webserver, but the webserver could not communicate with Kafka.

I've corrected the issue, in two steps:

@yrsurya
Copy link

yrsurya commented May 7, 2020

while running docker run with --network=host is also worked for me as I don't want to update those firewalls manually. will wait to hear if this can be reolved soon in latest repos which supports fedora32.

@tymokvo
Copy link

tymokvo commented May 10, 2020

⬆️ same, and can be achieved in a docker-compose.yml with:

    build:
      network: host

under the appropriate service entries.

@christian-korneck
Copy link

christian-korneck commented May 10, 2020

running docker run with --network=host is also worked for me

I don't think that's what you usually want. With --network=host you loose container network isolation and your containers have the same interfaces and IPs as the host. (This is useful when you i.e. want to run a packet capture program in a container that should have access to the host's network interface).

@Kidlike
Copy link

Kidlike commented May 12, 2020

How are we supposed to install docker on a fresh fedora system?
The official instructions (https://docs.docker.com/engine/install/fedora/) don't work...

@vincent-olivert-riera
Copy link

How are we supposed to install docker on a fresh fedora system?
The official instructions (https://docs.docker.com/engine/install/fedora/) don't work...

I think Fedora doesn't care about Docker any more and they just ship podman instead.

You need to wait until the Docker rpm packages for Fedora 32 are built. Or, you can just install the ones for Fedora 31, which should also work.

@jpopelka
Copy link

dnf install moby-engine works for me (after tinkering with cgroups and firewall)

@Frosthage
Copy link

Frosthage commented May 12, 2020

dnf install moby-engine works for me (after tinkering with cgroups and firewall)

Have you managed to make it run containers from a docker-compose file?

@quackerd
Copy link

quackerd commented May 12, 2020

I think Fedora doesn't care about Docker any more and they just ship podman instead.

Docker needs to support nftables and cgroups v2. Debian Buster uses nftables backend by default and had problems with docker too. The workaround is basically - 1. tell the kernel to use cgroup v1. 2. set firewalld backend to iptables or disable firewalld.

@vincent-olivert-riera
Copy link

Docker needs to support nftables and cgroups v2. Debian Buster uses nftables backend by default and had problems with docker too. The workaround is basically - 1. tell the kernel to use cgroup v1. 2. set firewalld backend to iptables or disable firewalld.

I know, I know:

$ cat /proc/cmdline 
BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.6.8-200.fc31.x86_64 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet resume=/dev/mapper/fedora-swap usbcore.autosuspend=-1 systemd.unified_cgroup_hierarchy=0

$ systemctl is-active firewalld
inactive

$ systemctl is-active docker
active

@jpopelka
Copy link

Have you managed to make it run containers from a docker-compose file?

yes

@Kidlike
Copy link

Kidlike commented May 13, 2020

Actually when I ignored the official instructions and just blindly tried dnf install docker, it worked because Fedora has an alias for moby-engine. it's actually docker version 19.03.8.

The rest is the same old story - cgroups, selinux, user groups, etc...

@Frosthage
Copy link

Have you managed to make it run containers from a docker-compose file?

yes

How? My google fu is failing me. :/

@lexfrei
Copy link

lexfrei commented May 13, 2020

@Frosthage use docker stack:

# install
dnf install moby-engine
# remove live-resotore line
vim /etc/sysconfig/docker
# enable and start docker
systemctl enable --now docker
# create one node swarm cluster
docker swarm init
# delpoy
docker stack deploy -c docker-compose.yml redis

Tested on clean f32 just right now, for you <3

@jpopelka
Copy link

jpopelka commented May 14, 2020

How? My google fu is failing me. :/

I don't remember any specific docker-compose related hacks. I think all I had to do (F32) was:

  • grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
  • firewall-cmd --permanent --zone=FedoraWorkstation --add-masquerade
  • dnf install moby-engine docker-compose
  • systemctl enable docker
  • reboot

And docker-compose build & docker-compose up work with my docker-compose.yml

@Frosthage
Copy link

@lexfrei @jpopelka
Thanks heaps!

@sparish-xevo
Copy link

In addition to the steps people have listed above, I also had a br-<ugly-hash> interface that I had to set as trusted before my docker containers were able to talk to each other. Found it by running firewall-config.

@navjotjsingh
Copy link

Is there any ETA on an official Fedora 32 repo?

@rhubarbselleven
Copy link

After reading this and the commentary here I read the following: https://developers.redhat.com/blog/2019/02/21/podman-and-buildah-for-docker-users/

Installed podman and all issues disapeared. Really don't want to be maintaining kernel options - otherwise I'd be running Gentoo again

@agalazis
Copy link

use mobi:
https://fedoramagazine.org/docker-and-fedora-32/

@zdm
Copy link

zdm commented Aug 12, 2020

Users just wants to run containers, they don't wants to build its own docker executable. Moby is not for end-users.

@ghost
Copy link

ghost commented Aug 12, 2020

Users just wants to run containers, they don't want to build its own docker executable. Moby is not for end-users.

Moby allows you to run containers just fine using the exact same commands as those provided by docker-ce. The only problem you might face is SELinux.

@zdm
Copy link

zdm commented Aug 12, 2020 via email

@junaruga
Copy link

junaruga commented Aug 12, 2020

Though I am not sure someone has already reported this, I succeeded to install Docker CE binary RPMs for Fedora 31 to Fedora 32 by the following commands. It works on the basic use cases on my environment.

$ cat /etc/fedora-release 
Fedora release 32 (Thirty Two)

$ curl -fsSL https://get.docker.com -o get-docker.sh

$ sudo sh get-docker.sh
...
Errors during downloading metadata for repository 'docker-ce-stable':
  - Status code: 404 for https://download.docker.com/linux/fedora/32/x86_64/stable/repodata/repomd.xml (IP: 99.86.243.99)
$ sudo sed -i.bak 's/$releasever/31/g' /etc/yum.repos.d/docker-ce.repo

$ sudo dnf install docker-ce docker-ce-cli containerd.io

@junaruga
Copy link

Why not just make new docker release and close this discussion?
As I understand, docker already can work under FC32 without problems.

I think that is because the docker-ce's official RPM repository is not provided from Docker. And Fedora 32 is not mentioned in the official document. After those are fixed, I think we can close this ticket.

https://docs.docker.com/engine/install/fedora/

OS requirements
To install Docker Engine, you need the 64-bit version of one of these Fedora versions:
Fedora 30
Fedora 31

Following the Fedora Magazine's document, we can install moby-engine RPM from Fedora official repository. And it seems it works. But it's not about docker-ce from the docker's repository.

@christian-korneck
Copy link

christian-korneck commented Aug 14, 2020

in case this helps anyone, I've made a quick fork of the Docker install script (the one from https://get.docker.com) with some of the workarounds for Fedora 32 mentioned in this thread. [link]

@zdm
Copy link

zdm commented Sep 9, 2020

seems, that fedora 32 is completely new os, so docker is unable to support it officially yet.

@infraengineer-sre
Copy link

infraengineer-sre commented Sep 10, 2020

There is NO [32](https://download.docker.com/linux/fedora/32/) folder at
download.docker.com/linux/fedora/
so you will get
404 HTTP Code Error with
download.docker.com/linux/fedora/32/x86_64/stable/repodata/repomd.xml

You can ither

download and install

fc31 versions

https://download.docker.com/linux/fedora/31/x86_64/stable/Packages/docker-ce-cli-19.03.12-3.fc31.x86_64.rpm
https://download.docker.com/linux/fedora/31/x86_64/stable/Packages/docker-ce-19.03.12-3.fc31.x86_64.rpm
https://download.docker.com/linux/fedora/31/x86_64/stable/Packages/containerd.io-1.2.13-3.2.fc31.x86_64.rpm

Delete moby if you alredy have
https://linuxconfig.org/how-to-install-and-configure-docker-ce-moby-engine-on-fedora-32
to avoid

Error: 
 Problem: problem with installed package moby-engine-19.03.11-1.ce.git42e35e6.fc32.x86_64
  - package moby-engine-19.03.11-1.ce.git42e35e6.fc32.x86_64 conflicts with docker-ce-cli provided by docker-ce-cli-1:19.03.12-3.fc31.x86_64
  - package docker-ce-cli-1:19.03.12-3.fc31.x86_64 conflicts with docker provided by moby-engine-19.03.11-1.ce.git42e35e6.fc32.x86_64
  - package moby-engine-19.03.8-1.ce.gitafacb8b.fc32.x86_64 conflicts with docker-ce-cli provided by docker-ce-cli-1:19.03.12-3.fc31.x86_64
  - package docker-ce-cli-1:19.03.12-3.fc31.x86_64 conflicts with docker provided by moby-engine-19.03.8-1.ce.gitafacb8b.fc32.x86_64
  - conflicting requests

or

install from official Fedora 32 Fedora Updates x86_64 repo

download-ib01.fedoraproject.org/pub/fedora/linux/updates/32/Everything/x86_64/Packages/m/moby-engine-19.03.11-1.ce.git42e35e6.fc32.x86_64.rpm

if you have not

@junaruga
Copy link

seems, that fedora 32 is completely new os, so docker is unable to support it officially yet.

Yes, I think so.
Fedora 32 has a new version of firewalld where the background is not iptables but nftables.
But firewalld upstream project is already changing the default setting to nftables.
So, this situation will happen on other Linux distributions in the future too.

@infraengineer-sre
Copy link

infraengineer-sre commented Sep 10, 2020

@gunnarvelle
Copy link

Finally!!

https://download.docker.com/linux/fedora/32/

@zdm
Copy link

zdm commented Sep 18, 2020

docker: Error response from daemon: cgroups: cgroup mountpoint does not exist: unknown.

I still need to exec this commands manually after each reboot:

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd 

@thaJeztah
Copy link
Member

Sorry for the long delay; our initial plan was to include Fedora 32 for our upcoming 20.x release, which will include preliminary support for both cgroupsv2 and has changes to automatically reconfigure firewalld for container networking to work.

Work on cgroupsv2 in containerd and runc was still ongoing, and we wanted to wait for those changes to arrive before we started building beta releases of Docker 20.xx.

Given the delay, we decided to start shipping containerd 1.3 as an intermediate step with docker 19.03.13 and up, and to build packages for Fedora 32 with the 19.03.13 patch release.

Due to the changes in Fedora 32, the install experience is slightly more involved than usual, and currently requires some extra manual steps to be performed, depending on your machine's configuration.

We will update our documentation to outline those steps (and possibly add some information to the convenience install script at https://get.docker.com).

Here's an example of the installation procedure on a Fedora 32 machine on DigitalOcean:

Configure your machine to disable cgroups v2

First, make sure that cgroups v2 are disabled (cgroups v2 are not yet supported on containerd 1.3 and lower). This requires a restart of your machine:

sudo grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"
reboot

On the DigitalOcean machines, firewalld is not installed by default:

firewall-cmd --help
-bash: firewall-cmd: command not found

systemctl start firewalld
Failed to start firewalld.service: Unit firewalld.service not found.

(Interestingly /etc/firewalld/firewalld.conf exists, but is a broken symlink to firewalld-standard.conf (which doesn't exist)):

ls -la /etc/firewalld/firewalld.conf
lrwxrwxrwx. 1 root root 23 Apr 22 22:31 /etc/firewalld/firewalld.conf -> firewalld-standard.conf

Install docker

This uses the convenience installation script, but you can also follow the manual install in https://docs.docker.com/engine/install/fedora/#install-using-the-repository

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Docker is not automatically started on RPM-based distros, so start docker, and enable it to start automatically on restart:

sudo systemctl start docker
sudo systemctl enable docker
# Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

Check that docker is running (docker version, docker info);

sudo docker version

Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:03:54 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:49 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

sudo docker info

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.6.6-300.fc32.x86_64
 Operating System: Fedora 32 (Cloud Edition)
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.933GiB
 Name: fedora-s-1vcpu-2gb-ams3-01
 ID: 4XCZ:2SPK:VLLW:HXXC:RZ22:4YQH:EVE6:NQGU:IFBN:625J:OFBV:YLWJ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Run a container and make an internet connection to verify the installation works:

sudo docker run --rm alpine sh -c 'wget -q -O- http://docs.docker.com | grep "<title"'
#  <title>Docker Documentation | Docker Documentation</title>

If your machine has firewalld installed

sudo dnf install -y firewalld

sudo systemctl start firewalld
sudo systemctl enable firewalld

After installing, firewalld blocks networking for containers:

sudo docker run --rm alpine sh -c 'wget -q -O- http://docs.docker.com | grep "<title"'
# wget: bad address 'docs.docker.com'

Configure firewalld to add docker0 to the trusted zone

This currently needs to be done manually. The upcoming Docker 20.x release will perform this automatically if firewalld is detected (see moby/libnetwork#2548). There's also a backport open, but needs to be discussed if we want to include the patch in a patch-release for 19.03 (moby/libnetwork#2575).

Adding docker0 to the trusted zone and reload firewalld configuration (steps taken from moby/libnetwork#2548):

sudo firewall-cmd --permanent --zone=trusted --add-interface=docker0; firewall-cmd --reload
# success
# success

Docker needs to be restarted after this, otherwise containers won't be able to make internet connections:

sudo docker run --rm alpine sh -c 'wget -q -O- http://docs.docker.com | grep "<title"'
# wget: bad address 'docs.docker.com'

Restart the docker daemon:

sudo systemctl restart docker

(note: I did systemctl stop docker and systemctl start docker, butrestart should work)

After this networking works:

sudo docker run --rm alpine sh -c 'wget -q -O- http://docs.docker.com | grep "<title"'
  <title>Docker Documentation | Docker Documentation</title>

@AkihiroSuda
Copy link

This issue can be closed?

@thaJeztah
Copy link
Member

Yes, I think we can close; feel free to continue the conversation

@thaJeztah thaJeztah mentioned this issue Sep 18, 2020
3 tasks
@BBQigniter
Copy link

thanks @thaJeztah worked like a charm on Fedora Server 32

still had to execute
firewall-cmd --permanent --zone=FedoraServer --add-masquerade
so that containers in their assigned network really can talk to each other. Adding docker0 only to the trusted zone seems not to be enough

@thaJeztah
Copy link
Member

@BBQigniter ah, interesting. Could you perhaps open a ticket in https://github.com/moby/moby/issues for that? Wondering if that can/should also be set automatically. Better to have a new ticket for it to track 👍

@junaruga
Copy link

According to this comment: moby/moby#40360 (comment) , the coming Docker version 20 will support cgroups v2.

g-chauvel pushed a commit to g-chauvel/zuul-jobs that referenced this issue Sep 29, 2020
We have Fedora 32 nodes now, retire the Fedora 31 testing and replace
with 32.

Docker isn't supported on Fedora 32.  There's a problem with cgroups
v2 and you can read all about it at [1].  Drop the install test.

Haskell upstream has dropped F32 from the COPR repo as it is now
packaged; a new package with the correct dependencies has been pushed
but isn't available yet (see the inline github issue), so workaround
the missing package.

[1] docker/for-linux#955

Change-Id: I025613bee8c0a8513afa90331431d2c18b616464
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests