BUG? docker stats don't show the mem usage? #1390

kevinshane · 2018-01-13T02:16:45Z

Creating a bug report/issue:

Required Information:

DietPi Version | v159
SBC Device Rpi3 and 2
Power supply used 5v 2.5a
SD card used ssd root
Distro (EG: Jessie) | Linux rpi2 4.9.62-v7+ Fix for dumb or unset TERM #2 SMP Fri Nov 17 23:52:26 GMT 2017 armv7l GNU/Linux

Additional Information (if applicable):

Software title? Docker
Can this issue be replicated on a fresh installation of DietPi?
after install docker, just pull anything and run docker stats, it won't show mem usage and I/O

Expected behaviour:

should show the memory usage for each container
also I've tested the vmware dietpi os(amd64), it shows

Actual behaviour:

Steps to reproduce:

Did you submit a `dietpi-bugreport`?

Extra details:

The text was updated successfully, but these errors were encountered:

Fourdee · 2018-01-13T12:36:31Z

@kevinshane

Thanks for the report 👍

May be a limitation with RPi (kernel) and/or issue with Docker binary itself. As I have zero experience with Docker, I'll open up for investigations.

Fourdee · 2018-01-13T12:39:57Z

@kevinshane
May also be a issue limited to Jessie, might be worth testing with Debian Stretch (our default RPi image):
http://dietpi.com/downloads/images/DietPi_RPi-armv6-(Stretch).7z

kevinshane · 2018-01-13T13:16:58Z

@Fourdee Thanks man, also I try to install the raspbian lite and it reports correctly, the kernel is
Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux
edit: the raspbian lite is jessie
for both rpi2 and 3

oh btw, I do tested with Stretch, it won't shows the mem usage either unfortantly :(

dietpi@rpi2:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 9.3 (stretch)
Release:	9.3
Codename:	stretch

kevinshane · 2018-01-31T07:25:04Z

just want to report that this issue still remains the same in v6.1

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
cf7ac37ccf62        portainer           0.00%               0B / 0B             0.00%               0B / 0B             0B / 90.1kB         0

the MEM% and MEM USAGE and NET I/O always says 0
please if someone knows how to deal with this issue, please share it! Thanks

MichaIng · 2018-07-27T13:36:01Z

@kevinshane
Sorry that this issue was forgotten, are the mem stats still missing with current DietPi + Docker?

@Fourdee
Docker online docs are still missing: https://dietpi.com/phpbb/viewtopic.php?f=8&t=5&start=110#p9120

MichaIng · 2018-09-25T13:50:29Z

@Fourdee @kevinshane
confirmed issue on Jessie VBox system, while it works fine on Stretch. Log gives:

Sep 25 15:34:01 VM-Jessie dockerd[3965]: time="2018-09-25T15:34:01+02:00" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/fb59c58e24d8c476d85c9e5e13337786129e5d5174172e3cfb72c2dae3e7ae27/shim.sock" deb
ug=false pid=8622
Sep 25 15:34:01 VM-Jessie dockerd[3965]: time="2018-09-25T15:34:01+02:00" level=warning msg="file does not exist: /proc/sys/net/ipv6/conf/all/disable_ipv6 : stat /proc/sys/net/ipv6/conf/all/disable_ipv6: no such file or directory Has IPv
6 been disabled in this node's kernel?"
Sep 25 15:34:01 VM-Jessie kernel: docker0: port 1(vetha67daef) entered forwarding state
Sep 25 15:34:01 VM-Jessie kernel: docker0: port 1(vetha67daef) entered forwarding state
Sep 25 15:34:01 VM-Jessie dockerd[3965]: time="2018-09-25T15:34:01+02:00" level=warning msg="OOM monitoring failed" error="cgroups: memory cgroup not supported on this system"
Sep 25 15:34:10 VM-Jessie dropbear[8685]: Child connection from 192.168.178.21:52627
Sep 25 15:34:12 VM-Jessie dropbear[8685]: Password auth succeeded for 'root' from 192.168.178.21:52627
Sep 25 15:34:16 VM-Jessie kernel: docker0: port 1(vetha67daef) entered forwarding state
Sep 25 15:35:34 VM-Jessie dockerd[3965]: time="2018-09-25T15:35:34+02:00" level=info msg="shim reaped" id=fb59c58e24d8c476d85c9e5e13337786129e5d5174172e3cfb72c2dae3e7ae27
Sep 25 15:35:34 VM-Jessie dockerd[3965]: time="2018-09-25T15:35:34.333723379+02:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 25 15:35:34 VM-Jessie kernel: docker0: port 1(vetha67daef) entered disabled state
Sep 25 15:35:34 VM-Jessie kernel: docker0: port 1(vetha67daef) entered disabled state
Sep 25 15:35:34 VM-Jessie kernel: device vetha67daef left promiscuous mode
Sep 25 15:35:34 VM-Jessie kernel: docker0: port 1(vetha67daef) entered disabled state
Sep 25 15:35:34 VM-Jessie dockerd[3965]: time="2018-09-25T15:35:34.645548595+02:00" level=warning msg="Couldn't run auplink before unmount /mnt/dietpi_userdata/docker-data/aufs/mnt/66d143339d633f7a31225591ae634866c2e49845d24ca1ffcaf03322
b4e71f2f" error="exec: \"auplink\": executable file not found in $PATH" storage-driver=aufs

Indeed, no memory cgroup on Jessie:

root@VM-Jessie:~# ls /sys/fs/cgroup/
blkio  cpu  cpuacct  cpu,cpuacct  cpuset  devices  freezer  net_cls  net_cls,net_prio  net_prio  perf_event  systemd

On Stretch:

root@VM-Stretch:~# ls /sys/fs/cgroup/
blkio  cpu  cpuacct  cpu,cpuacct  cpuset  devices  freezer  memory  net_cls  net_cls,net_prio  net_prio  perf_event  pids  systemd

Further check, the cgroup exists on Jessie, but is actively disabled:

root@VM-Jessie:~# cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  2       2       1
cpu     3       2       1
cpuacct 3       2       1
memory  0       1       0
devices 4       41      1
freezer 5       2       1
net_cls 6       2       1
blkio   7       2       1
perf_event      8       2       1
net_prio        6       2       1

This can be set via boot cmd line parameter. Nothing found in /etc/default/grub, but:

root@VM-Jessie:~# cat /boot/config-3.16.0-6-amd64 | grep MEMCG
CONFIG_MEMCG=y
CONFIG_MEMCG_DISABLED=y
CONFIG_MEMCG_SWAP=y
# CONFIG_MEMCG_SWAP_ENABLED is not set
# CONFIG_MEMCG_KMEM is not set

CONFIG_MEMCG_DISABLED=y sounds at least like memory cgroup actively disabled. Default on Debian Jessie? Why?

On Stretch this is not the case:

root@VM-Stretch:~# cat /boot/config-4.9.0-8-amd64 | grep MEMCG
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
# CONFIG_MEMCG_SWAP_ENABLED is not set

... https://serverfault.com/a/801259

🈯️ Add cgroup_enable=memory to GRUB_CMDLINE_LINUX, update-grub, reboot and:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
f3811f6528a7        dazzling_bhaskara   0.00%               752KiB / 999.8MiB   0.07%               1.3kB / 0B          5.38MB / 0B         0

So works, but I am wondering what the memory cgroup might have for side effects. Why is it disabled by default on Jessie images?

Fourdee · 2018-09-30T17:51:53Z

@MichaIng

So works, but I am wondering what the memory cgroup might have for side effects. Why is it disabled by default on Jessie images?

Could be a limitation with 3.x kernel, or, it was simply a new addition that wasn't set by default due a decision by Debian.

Either way. As this is an external issue, with workaround available, and, Stretch is our current supported platform, we should mark this as closed.

Docker online docs are still missing: https://dietpi.com/phpbb/viewtopic.php?f=8&t=5&start=110#p9120

Yep, will move it to https://github.com/Fourdee/DietPi/issues/1928

Fourdee added the Investigating 🤔 label Jan 13, 2018

Fourdee added the Waiting for user reply ⏳ label Jul 28, 2018

MichaIng added Testing/testers required 🔽 Solution available 🥂 Definite solution has been done and removed Waiting for user reply ⏳ labels Sep 25, 2018

MichaIng added this to the v6.17 milestone Sep 25, 2018

MichaIng added the External bug 🐞 For bugs which are not caused by DietPi. label Sep 25, 2018

Fourdee closed this as completed Sep 30, 2018

Fourdee removed Investigating 🤔 Testing/testers required 🔽 labels Sep 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG? docker stats don't show the mem usage? #1390

BUG? docker stats don't show the mem usage? #1390

kevinshane commented Jan 13, 2018

Fourdee commented Jan 13, 2018 •

edited

Loading

Fourdee commented Jan 13, 2018 •

edited

Loading

kevinshane commented Jan 13, 2018 •

edited

Loading

kevinshane commented Jan 31, 2018

MichaIng commented Jul 27, 2018 •

edited

Loading

MichaIng commented Sep 25, 2018 •

edited

Loading

Fourdee commented Sep 30, 2018

BUG? docker stats don't show the mem usage? #1390

BUG? docker stats don't show the mem usage? #1390

Comments

kevinshane commented Jan 13, 2018

Creating a bug report/issue:

Required Information:

Additional Information (if applicable):

Expected behaviour:

Actual behaviour:

Steps to reproduce:

Did you submit a dietpi-bugreport?

Extra details:

Fourdee commented Jan 13, 2018 • edited Loading

Fourdee commented Jan 13, 2018 • edited Loading

kevinshane commented Jan 13, 2018 • edited Loading

kevinshane commented Jan 31, 2018

MichaIng commented Jul 27, 2018 • edited Loading

MichaIng commented Sep 25, 2018 • edited Loading

Fourdee commented Sep 30, 2018

Did you submit a `dietpi-bugreport`?

Fourdee commented Jan 13, 2018 •

edited

Loading

Fourdee commented Jan 13, 2018 •

edited

Loading

kevinshane commented Jan 13, 2018 •

edited

Loading

MichaIng commented Jul 27, 2018 •

edited

Loading

MichaIng commented Sep 25, 2018 •

edited

Loading