Add page for enabling kdump through ignition #388

mike-nguyen · 2022-04-05T19:55:52Z

As a part of coreos/fedora-coreos-tracker#1147, add a page for enabling kdump through ignition.

dustymabe · 2022-04-07T01:49:19Z

Thanks for writing this up @mike-nguyen.

It seems to me like this page has a lot of overlap with the other page. Maybe we should put the meat of the steps in this new page you've created (i.e. in the last section we show the rpm-ostree kargs that's currently documented on the other page. Then we could link to it from the other page and also maybe make the other page more about what you do with a vmcore once you have it.

dustymabe · 2022-04-07T01:53:40Z

modules/ROOT/pages/sysconfig-configure-kdump.adoc

+
+[NOTE]
+====
+The reboot in the systemd unit is required for kdump to use the correct kernel arguments when it kexecs into the crash kernel.


Suggested change

The reboot in the systemd unit is required for kdump to use the correct kernel arguments when it kexecs into the crash kernel.

The reboot in the systemd unit is required for kdump to use the correct kernel arguments. When kexec runs it re-uses the kernel arguments from the current boot and this has the undesired side effect of re-running Ignition if the machine had never rebooted.

I'm thinking we either give a little more context here, or we link to an issue where this is explained a little further.

I think this note should be at the top before the Ignition config.

dustymabe · 2022-04-07T01:54:44Z

modules/ROOT/pages/sysconfig-configure-kdump.adoc

+        WantedBy=multi-user.target
+----
+
+[NOTE]


might be nice to mention somewhere in here the info from the other page about /etc/kdump.conf

dustymabe · 2022-04-07T01:55:44Z

It seems to me like this page has a lot of overlap with the other page. Maybe we should put the meat of the steps in this new page you've created (i.e. in the last section we show the rpm-ostree kargs that's currently documented on the other page. Then we could link to it from the other page and also maybe make the other page more about what you do with a vmcore once you have it.

I guess another option is we don't create a new page and just add the "how to configure this via Ignition" to the existing page.

travier

Agree we should probably merge this with https://docs.fedoraproject.org/en-US/fedora-coreos/debugging-kernel-crashes/

travier · 2022-04-07T16:13:41Z

modules/ROOT/pages/sysconfig-configure-kdump.adoc

@@ -0,0 +1,51 @@
+:experimental:


Not needed anymore

travier · 2022-04-07T16:16:37Z

modules/ROOT/pages/sysconfig-configure-kdump.adoc

+
+[NOTE]
+====
+The reboot in the systemd unit is required for kdump to use the correct kernel arguments when it kexecs into the crash kernel.


I think this note should be at the top before the Ignition config.

bgilbert · 2022-04-08T02:23:57Z

This is incurring two reboots during first boot. Don't we want something more like this:

variant: fcos
version: 1.4.0
kernel_arguments:
  should_exist:
  - 'crashkernel=300M'
systemd:
  units:
    - name: kdump.service
      enabled: true

coupled with a postprocess script in the fedora-coreos-config user-experience.yaml that updates KDUMP_COMMANDLINE_REMOVE in /etc/sysconfig/kdump?

dustymabe · 2022-04-08T03:30:45Z

This is incurring two reboots during first boot. Don't we want something more like this:
variant: fcos
version: 1.4.0
kernel_arguments:
  should_exist:
  - 'crashkernel=300M'
systemd:
  units:
    - name: kdump.service
      enabled: true
coupled with a postprocess script in the fedora-coreos-config user-experience.yaml that updates KDUMP_COMMANDLINE_REMOVE in /etc/sysconfig/kdump?

I know @mike-nguyen had looked into trying to get something like this to work but I don't know if he ever quite got it. @mike-nguyen do you remember what the complications were?

travier · 2022-04-12T18:03:56Z

It's #388 (comment). We can not do it first boot and need an additional reboot. The note needs to be front and center to prevent folks from falling into this trap.

bgilbert · 2022-04-12T20:54:39Z

It's #388 (comment). We can not do it first boot and need an additional reboot. The note needs to be front and center to prevent folks from falling into this trap.

What's wrong with the approach in #388 (comment) though? Ignition will reboot to add the crashkernel karg, and an appropriate value for KDUMP_COMMANDLINE_REMOVE should ensure the crash kernel doesn't inherit ignition.firstboot.

travier · 2022-04-13T09:42:07Z

What's wrong with the approach in #388 (comment) though? Ignition will reboot to add the crashkernel karg, and an appropriate value for KDUMP_COMMANDLINE_REMOVE should ensure the crash kernel doesn't inherit ignition.firstboot.

Whoops sorry, indeed, I had missed that.

mike-nguyen · 2022-04-13T12:46:12Z

It's #388 (comment). We can not do it first boot and need an additional reboot. The note needs to be front and center to prevent folks from falling into this trap.

What's wrong with the approach in #388 (comment) though? Ignition will reboot to add the crashkernel karg, and an appropriate value for KDUMP_COMMANDLINE_REMOVE should ensure the crash kernel doesn't inherit ignition.firstboot.

I know I tried adding ignition.firstboot to KDUMP_COMMANDLINE_REMOVE in /etc/kdump.conf--this was in the context of after the system already being provisioned through ignition. Its been a while and I forgot why that didn't work. I'll also try making it a part of the post process scripts and will report back.

dustymabe · 2022-04-13T13:26:10Z

Thank you @mike-nguyen!

dustymabe · 2022-05-02T03:17:42Z

I know I tried adding ignition.firstboot to KDUMP_COMMANDLINE_REMOVE in /etc/kdump.conf--this was in the context of after the system already being provisioned through ignition. Its been a while and I forgot why that didn't work. I'll also try making it a part of the post process scripts and will report back.

Seems to work from what I can tell: coreos/fedora-coreos-config#1708

mike-nguyen · 2022-05-02T16:30:29Z

I know I tried adding ignition.firstboot to KDUMP_COMMANDLINE_REMOVE in /etc/kdump.conf--this was in the context of after the system already being provisioned through ignition. Its been a while and I forgot why that didn't work. I'll also try making it a part of the post process scripts and will report back.

Seems to work from what I can tell: coreos/fedora-coreos-config#1708

Sorry this fell off my radar. I tested your PR manually and it worked!

mike-nguyen · 2022-05-02T21:12:45Z

Consolidated everything into the original debugging kdump page and added steps to enable via ignition. This doc is dependent on coreos/fedora-coreos-config#1708 (thanks @dustymabe!) which will remove ignition.firstboot from the kernel arguments for the crash kernel and no longer require a reboot.

modules/ROOT/pages/debugging-kernel-crashes.adoc

bgilbert

Looks good, thanks!

Disable automatic configuration on ethernet devices (DHCP/SLAAT) with no other matching connections Update modules/ROOT/pages/sysconfig-network-configuration.adoc Co-authored-by: Dusty Mabe <[email protected]> Update modules/ROOT/pages/sysconfig-network-configuration.adoc Co-authored-by: Dusty Mabe <[email protected]> Update modules/ROOT/pages/sysconfig-network-configuration.adoc Co-authored-by: Dusty Mabe <[email protected]> Update modules/ROOT/pages/sysconfig-network-configuration.adoc Co-authored-by: Dusty Mabe <[email protected]> provisioning/exoscale: Add note/warning about CLI compression Recommend users to use v1.54.0 and above and to never use the `--cloud-init-compress` argument. Closes coreos/fedora-coreos-tracker#1160 Add page for enabling kdump through ignition (coreos#388) sysconfig-network-configuration: Improve readability and look Improve readability and look of sysconfig-network Final commit

dustymabe reviewed Apr 7, 2022

View reviewed changes

travier reviewed Apr 7, 2022

View reviewed changes

dustymabe mentioned this pull request Apr 14, 2022

F36: Fedora CoreOS Test Week coreos/fedora-coreos-tracker#1123

Closed

mike-nguyen force-pushed the kdump_page branch 3 times, most recently from 6876636 to 4619ec7 Compare May 2, 2022 21:10

mike-nguyen force-pushed the kdump_page branch from 4619ec7 to 3450c7c Compare May 2, 2022 21:20

bgilbert reviewed May 2, 2022

View reviewed changes

modules/ROOT/pages/debugging-kernel-crashes.adoc Outdated Show resolved Hide resolved

modules/ROOT/pages/debugging-kernel-crashes.adoc Outdated Show resolved Hide resolved

Add page for enabling kdump through ignition

59db4c8

mike-nguyen force-pushed the kdump_page branch from 3450c7c to 59db4c8 Compare May 3, 2022 13:16

bgilbert approved these changes May 6, 2022

View reviewed changes

mike-nguyen merged commit d654c92 into coreos:main May 6, 2022

mohelt mentioned this pull request May 9, 2022

sysconfig-network-configuration: Disable Auto Config Ethernet Devices mohelt/fedora-coreos-docs#1

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add page for enabling kdump through ignition #388

Add page for enabling kdump through ignition #388

mike-nguyen commented Apr 5, 2022

dustymabe commented Apr 7, 2022

dustymabe Apr 7, 2022

travier Apr 7, 2022

dustymabe Apr 7, 2022

dustymabe commented Apr 7, 2022

travier left a comment

travier Apr 7, 2022

travier Apr 7, 2022

bgilbert commented Apr 8, 2022

dustymabe commented Apr 8, 2022

travier commented Apr 12, 2022 •

edited

Loading

bgilbert commented Apr 12, 2022

travier commented Apr 13, 2022

mike-nguyen commented Apr 13, 2022 •

edited

Loading

dustymabe commented Apr 13, 2022

dustymabe commented May 2, 2022

mike-nguyen commented May 2, 2022

mike-nguyen commented May 2, 2022 •

edited

Loading

bgilbert left a comment

	The reboot in the systemd unit is required for kdump to use the correct kernel arguments when it kexecs into the crash kernel.
	The reboot in the systemd unit is required for kdump to use the correct kernel arguments. When kexec runs it re-uses the kernel arguments from the current boot and this has the undesired side effect of re-running Ignition if the machine had never rebooted.

Add page for enabling kdump through ignition #388

Add page for enabling kdump through ignition #388

Conversation

mike-nguyen commented Apr 5, 2022

dustymabe commented Apr 7, 2022

dustymabe Apr 7, 2022

Choose a reason for hiding this comment

travier Apr 7, 2022

Choose a reason for hiding this comment

dustymabe Apr 7, 2022

Choose a reason for hiding this comment

dustymabe commented Apr 7, 2022

travier left a comment

Choose a reason for hiding this comment

travier Apr 7, 2022

Choose a reason for hiding this comment

travier Apr 7, 2022

Choose a reason for hiding this comment

bgilbert commented Apr 8, 2022

dustymabe commented Apr 8, 2022

travier commented Apr 12, 2022 • edited Loading

bgilbert commented Apr 12, 2022

travier commented Apr 13, 2022

mike-nguyen commented Apr 13, 2022 • edited Loading

dustymabe commented Apr 13, 2022

dustymabe commented May 2, 2022

mike-nguyen commented May 2, 2022

mike-nguyen commented May 2, 2022 • edited Loading

bgilbert left a comment

Choose a reason for hiding this comment

travier commented Apr 12, 2022 •

edited

Loading

mike-nguyen commented Apr 13, 2022 •

edited

Loading

mike-nguyen commented May 2, 2022 •

edited

Loading