Fedora 31 and Container (Docker-ce, Moby-engine) Start Systemd Problem

Upgraded from Fedora 30 to Fedora 31 via DNF, in previous release works fine… the upgrade was sucessfull…

systemctl status docker.service                                                                                
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─docker-storage.conf
   Active: failed (Result: exit-code) since Wed 2019-09-25 11:13:09 CEST; 16min ago
     Docs: https://docs.docker.com
  Process: 116527 ExecStart=/usr/bin/dockerd -H fd:// --data-root=/run/media/hhlp/samsung/docker (code=exited, status=1/FAILURE)
 Main PID: 116527 (code=exited, status=1/FAILURE)
      CPU: 137ms

sep 25 11:13:07 dockerd[116527]: failed to start daemon: Devices cgroup isn't mounted
sep 25 11:13:07 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
sep 25 11:13:07 systemd[1]: docker.service: Failed with result 'exit-code'.
sep 25 11:13:07 systemd[1]: Failed to start Docker Application Container Engine.
sep 25 11:13:09 systemd[1]: docker.service: Service RestartSec=2s expired, scheduling restart.
sep 25 11:13:09 systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
sep 25 11:13:09 systemd[1]: Stopped Docker Application Container Engine.
sep 25 11:13:09 systemd[1]: docker.service: Start request repeated too quickly.
sep 25 11:13:09 systemd[1]: docker.service: Failed with result 'exit-code'.
sep 25 11:13:09 systemd[1]: Failed to start Docker Application Container Engine.
sudo systemctl start docker                                                           
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.

-- Subject: A start job for unit docker.socket has begun execution
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- A start job for unit docker.socket has begun execution.
-- 
-- The job identifier is 9699.
sep 25 11:37:03 hhlp systemd[1]: Listening on Docker Socket for the API.
-- Subject: A start job for unit docker.socket has finished successfully
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- A start job for unit docker.socket has finished successfully.
-- 
-- The job identifier is 9699.
sep 25 11:37:03 hhlp systemd[1]: docker.service: Start request repeated too quickly.
sep 25 11:37:03 hhlp systemd[1]: docker.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- The unit docker.service has entered the 'failed' state with result 'exit-code'.
sep 25 11:37:03 hhlp systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: A start job for unit docker.service has failed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- A start job for unit docker.service has finished with a failure.
-- 
-- The job identifier is 9577 and the job result is failed.
sep 25 11:37:03 hhlp systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- The unit docker.socket has entered the 'failed' state with result 'service-start-limit-hit'.

ref: https://github.com/docker/for-linux/issues/792

  1. The Docker repo for fedora 31 is ready:
  1. :fedora: 31 migrate from CgroupsV1 to CgroupsV2 version, that is the reason the service doesn’t start successfully:

  2. What is CgroupsV2?

Is a control and maximize resource utilization, A Linux kernel mechanism to group and structure workloads, and control the amount of system resources assigned to each group.

WORKAROUND


  1. open /etc/default/grub as admin
  2. Append value of GRUB_CMDLINE_LINUX with systemd.unified_cgroup_hierarchy=0
    sudo grub2-mkconfig > /boot/efi/EFI/fedora/grub.cfg or
    sudo grub2-mkconfig > /boot/grub2/grub.cfg
  1. reboot

NOTE


You can also switch to Podman, which is compatible with Docker. Also, it supports CgroupsV2.

Explanation


The major difference between Docker and Podman is that there is no daemon in Podman. It uses container runtimes as well for example runc but the launched containers are direct descendants of the Podman process. This kind of architecture has its advantages such as the following:

Applied Cgroups or security constraints still control the container: Whatever cgroup constraints you apply on the podman command, the containers launched will receive those same constraints directly.

Advanced features of systemd can be utilized using this model: This can be done by placing podman into a systemd unit file and hence achieving more.

Reference :

Documentation :

What is Podman? Simply put: alias docker=podman

Regards.,

4 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

I have run hello-world successfully on Fedora 29 but failed on Fedora 31 (Linux Kernel 5.3.9-300.fc31.x86_64) with the error below:

➜  ~ docker run -d --name hello hello-world
a74986afe5d3298fa263dd5457c3eaac324a3d2c3a1a3dfe1d2c2c5467e47f12
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"open /sys/fs/cgroup/docker/cpuset.cpus.effective: no such file or directory\"": unknown.

I have spent 5 days to investigate and think that the issue might be because of something wrong with the cgroup and virtualization. The virt-host-validate pass when I boot into rescue mode but fail in normal mode.

➜  ~ virt-host-validate 
QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : WARN (Enable 'cpu' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : WARN (Enable 'cpuset' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : WARN (Enable 'devices' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for cgroup 'blkio' controller support                       : WARN (Enable 'blkio' in kernel Kconfig file or mount/enable cgroup controller in your system)
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : PASS
   LXC: Checking for Linux >= 2.6.26                                         : PASS
   LXC: Checking for namespace ipc                                           : PASS
   LXC: Checking for namespace mnt                                           : PASS
   LXC: Checking for namespace pid                                           : PASS
   LXC: Checking for namespace uts                                           : PASS
   LXC: Checking for namespace net                                           : PASS
   LXC: Checking for namespace user                                          : PASS
   LXC: Checking for cgroup 'cpu' controller support                         : FAIL (Enable 'cpu' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'cpuacct' controller support                     : PASS
   LXC: Checking for cgroup 'cpuset' controller support                      : FAIL (Enable 'cpuset' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'memory' controller support                      : PASS
   LXC: Checking for cgroup 'devices' controller support                     : FAIL (Enable 'devices' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'freezer' controller support                     : FAIL (Enable 'freezer' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking for cgroup 'blkio' controller support                       : FAIL (Enable 'blkio' in kernel Kconfig file or mount/enable cgroup controller in your system)
   LXC: Checking if device /sys/fs/fuse/connections exists                   : PASS

➜  ~ docker version
Client: Docker Engine - Community
 Version:           19.03.4
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        9013bf583a
 Built:             Fri Oct 18 16:05:52 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Anyone has expertise in this please help me. Thank you.

Hello @min2244 …! Welcome to the community! Please do take a few minutes to go over the introductory posts in #start-here when you have the time. They contain lots of useful information.

Please review this POST, in it I explain very well what is this problem about?

Regards.,

@hhlp Thank you so much.

It works like a charm although my docker daemon can start but fail to run a container.
By the way, thank you for recommend us Podman. I started to read about Podman and think it’s very promising.

Read more about this migration at related Wiki’s page:

1 Like