coreos-printk-quiet.service: Lower kernel log level to 4

Closes: coreos/fedora-coreos-tracker#1244

A lot going on for this simple service.  See the tracker issue above
for more info, but briefly:

Anaconda has historically injected `quiet` into the kernel command
line in many cases, and this suppresses *both* kernel and systemd
output.  On computers in general, but particularly many bare metal
servers, the serial console can be slow.  This can cause reliability
issues.

However for servers, we usually *do* want to see informational
output when they boot.  For example, today the kernel "mitigations"
information for hardware vulnerabilities is output.

This change is a compromise; we boot up at the kernel's default
verbosity level (which for Fedora and derivatives is the upstream 7),
but switch to 4 very early on in the real root.  At that point, we've
gotten most of the boot time output, and our initramfs is not
extremely performance sensitive right now.

Also, we explicitly only *lower* the output log level, and only if
there isn't explicitly `debug` on the kernel command line.

overlay.d/05core/usr/lib/systemd/system-preset/40-coreos.preset

@@ -5,6 +5,8 @@ enable console-login-helper-messages-gensnippet-os-release.service
 enable console-login-helper-messages-gensnippet-ssh-keys.service
 # CA certs (probably to add to base fedora eventually)
 enable coreos-update-ca-trust.service
+# Our printk bits
+enable coreos-printk-quiet.service
 # https://github.com/coreos/ignition/issues/1125
 enable coreos-ignition-firstboot-complete.service
 # Delete Ignition config from provider on platforms where it's possible

overlay.d/05core/usr/lib/systemd/system/coreos-printk-quiet.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=CoreOS: Set printk to level 4 (warn)
+Documentation=https://github.com/coreos/fedora-coreos-tracker/issues/1244
+# We can run right after `/proc` being mounted at least
+DefaultDependencies=no
+# We run as early as possible; the only dependency we have really
+# is the implicit After=systemd-journald.socket injected by the
+# default of our stdout writing to the journal.
+Conflicts=shutdown.target
+Before=sysinit.target shutdown.target
+# We don't want to overwrite what a user may have configured via sysctl.
+Before=systemd-sysctl.service
+# Relatedly, we don't want to override an explicitly specified kernel argument
+ConditionKernelCommandLine=!debug
+ConditionKernelCommandLine=!quiet
+ConditionKernelCommandLine=!loglevel
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/bin/bash -euo pipefail -c 'target=4; current=$(cat /proc/sys/kernel/printk | cut -f 1); if test $current -gt $target; then echo "Updating printk to $target"; echo $target > /proc/sys/kernel/printk; fi'
+
+[Install]
+WantedBy=sysinit.target

tests/kola/logging/data/commonlib.sh

@@ -0,0 +1 @@
+../../data/commonlib.sh

tests/kola/logging/printk.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+# kola: { "exclusive": false }
+
+# Verify fix for https://github.com/coreos/fedora-coreos-tracker/issues/1244
+
+set -xeuo pipefail
+
+. $KOLA_EXT_DATA/commonlib.sh
+
+printk=$(cat /proc/sys/kernel/printk)
+
+if ! [[ "$printk" =~ ^4 ]]; then
+    fatal "printk: expected console log level 4, found ${printk}"
+fi
+
+ok "Found expected printk value "