LUKS device reported as wrong type even though type is set correctly

[rugk]

Bug

Background information:
https://discussion.fedoraproject.org/t/how-to-mount-re-use-an-existing-luks-volume/29767?u=rugk

Operating System Version

FCOS 34
fedora-coreos-34.20210503.1.1-live.x86_64

Ignition Version

2.9.0

Environment

bare-metal

Expected Behavior

Do not report this error.

Or report a more detailed error, at least?

Actual Behavior

I get the error that the volume is not of the correct type;

ignition/internal/exec/stages/disks/luks.go

Lines 203 to 204 in 0cbe33f

	s.Logger.Err("volume at %q is not of the correct type, label, or UUID (found %s, %q, %s) and a volume wipe was not requested", *luks.Device, info.Type, info.Label, info.UUID)
	return ErrBadVolume

volume at "" is not of correct type, label, or UUID (found crypto_LUKS, "luks-external", [some uuid]) and a volume wipe was not requested

I guess this is wrong, see below.

Reproduction Steps

variant: fcos
version: 1.3.0
storage:
  luks:
    # external disk
    - name: external
      label: luks-external
      device: /dev/disk/by-id/[…]:0-part1
      clevis:
        tpm2: false
        threshold: 1
        tang:
          - url: # ....
      wipe_volume: false

Other Information

I explicitly set the correct LUKS label, as you can see when I luksDump the resource:

sudo cryptsetup luksDump /dev/sdc1                    
LUKS header information
Version:       	2
Epoch:         	6
Metadata area: 	16384 [bytes]
Keyslots area: 	16744448 [bytes]
UUID:          	46*******d1f
Label:         	luks-external
Subsystem:     	(no subsystem)
Flags:       	(no flags)

AFAIK the code goes in here from here, in here and gets all information.

Given everything looks correct AFAIK, I guess checkLuksDeviceExists somehow works improperly…

The only way, where it could fail may be when opening it with a keyfile – that does not exist.

However, if it cannot open the existing LUKS partition (why does not it try to use Clevis BTW? It is already configured but well…), then it should report this as an error and not complain about the type of the partition.

Whether the device can be opened has nothing to do with what "type" (in whatever meaning) it is.

Edit: I should note the string opening luks device does not appear in my journalctl logs, so after all maybe my speculation at the end is wrong.

[arithx]

So the specific issue here is that Ignition cannot validate that a given clevis backed LUKS device matches the config. Purely being able to unlock a device is not sufficient to say that it's configured exactly as specified in the config.

I believe the current resolution for this bug will be either the modification of that error string or another error path with a more exact error string as the actual behavior here is expected.

[rugk]

Okay, so let's discuss that general thing in #1210.

However, I now tried specifying a keyfile that I know is sure unlockable for Ignition and get the same error – so maybe my speculation about the keyfile being the fault, is wrong after all?

variant: fcos
version: 1.3.0
storage:
  luks:
    # external disk
    - name: external
      label: luks-external
      device: /dev/disk/by-id/[…]:0-part1
      # disabled, because MBR disk is not properly configured yet as this
      # device: /dev/disk/by-partlabel/external
      clevis:
        tpm2: false
        threshold: 1
        tang:
          - url: # […]
      # temporary key"file" i.e. password to unlock disk at provisioning step as clevis unlocking at provisioning is not yet supported
      # see https://github.com/coreos/ignition/issues/1210
      key_file:
        inline: |-
          ******************
      wipe_volume: false

I know the password as a key-file I put into ****************** is a password I can enter to unlock the device (just added via cryptsetup luksAddKey). And I double-checked in the log output on the device, it appears correctly in the Ignition config (as "{keyFile": {source: "data:,******************", "verification": {}}}).
I even gave care not to include a trailing newline there by using |- instead of |, see https://yaml-multiline.info/.

[rugk]

Wait, now I'm confused when comparing the source code.

I added this to my OP:

should note the string opening luks device does not appear in my journalctl logs, so after all maybe my speculation at the end is wrong.

However even the first command is not logged that checks whether that is a LUKS device? I.e. checking if %v is a luks device. Or is that another log level / only logged when a failure appears?

Note: Also witched to corect branch/tag now, it's of course Ignition v2.9.0, but as far as I see no big changes were made there.

[rugk]

Also tried it without having a label specified, failed with the same error.
I now totally wonder what that might be… is is clearly correct, I mean the UUID should be ignored, the label is the same or should also be ignored in my last test and yes, it's also a LUKS2 device, I did confirm that by running sudo cryptsetup isLuks --type luks2 manually. checkLuksDeviceExists does not do anything more…

Uhm… I guess improving the error messages here (so that they show what property is actually missing why it claims the type is wrong) would be useful.

Edit: Now also manually verified that cryptsetup luksOpen works, when it is supplied with a keyfile with a text without a line ending.
So indeed I see nothing which should go wrong here.

Or does Ignition maybe improperly create key files with a line ending?

[bgilbert]

Ignition is not currently designed to reuse existing Clevis-backed LUKS volumes; see #1210 (comment) for some additional context. Since you don't actually want Ignition to modify the device, you don't need to specify the storage.luks section at all; just have Ignition write a line into /etc/crypttab with the _netdev flag set.

[rugk]

Okay fine, but anyway, the approach here should also work, should not it? After all reusing LUKS volumes (with a keyfile) is implemented.

So, to eliminate this:

Or does Ignition maybe improperly create key files with a line ending?

I now attached a keyfile to the LUKS volume with a line ending and it still does not work. So that's not the issue.

Still, something in checkLuksDeviceExists has to be wrong.

Also how do I get these detailed command logs that appear in the source code (.Logger.LogCmd)? As explained, I don't see them in journalctl…

[bgilbert]

Okay fine, but anyway, the approach here should also work, should not it? After all reusing LUKS volumes (with a keyfile) is implemented.

If your existing LUKS volume is only configured with a keyfile, it should work. If it's also configured with Clevis unlocking, it won't work, since Ignition can't verify that the config matches the volume.

[rugk]

If your existing LUKS volume is only configured with a keyfile, it should work.

It is. As explained I explicitly added a keyslot. And it does not work…

If it's also configured with Clevis unlocking, it won't work

well… it is too, but that should not matter, should it?
It's just another keyslot.

I did not see it anywhere in the source code that Ignition would compare keyslots or so.

[rugk]

Okay, so I've removed all key slots, and kept only one remaining keyslot with a keyfile (respectively passphrase) that Ignition can unlock because it is given to it…

It still does not work. The LUKS device is now exactly as specified (no Clevis included anymore, I clevis luks unbind'ed it), but it does not work.
And as said, from reading the source code I doubt Ignition actually verifies the LUKS keyslots.
And I verified in the emergency shell that /run/ignition/luks-keyfiles/external contains the passphrase I have set, correctly without any line-ending.

I still get the "wrong type" error.
And I still get the correct found %s at %q with uuid %q and label %q message, so the device is found.

$ sudo cryptsetup luksDump /dev/sdd1       
LUKS header information
Version:       	2
Epoch:         	14
Metadata area: 	16384 [bytes]
Keyslots area: 	16744448 [bytes]
UUID:          	464[…]ded1f
Label:         	luks-external
Subsystem:     	(no subsystem)
Flags:       	(no flags)

Data segments:
  0: crypt
	offset: […] [bytes]
	length: (whole device)
	cipher: aes-xts-plain64
	sector: 512 [bytes]

Keyslots:
  0: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      argon2i
	Time cost:  6
	Memory:     ******
	Threads:    4
	Salt:       ******
	            ******
	AF stripes: ******
	AF hash:    sha256
	Area offset:32768 [bytes]
	Area length:****** [bytes]
	Digest ID:  0
Tokens:
Digests:
  0: pbkdf2
	Hash:       sha512
	Iterations: ******
	Salt:       ****** 
	            ****** 
	Digest:    ******
	            ****** 
	            ****** 
	            ****** 

variant: fcos
version: 1.3.0
storage:
  luks:
    # external disk
    - name: external
      label: luks-external
      device: /dev/disk/by-id/[…]:0-part1
      # disabled, because MBR disk is not properly configured yet as this
      # device: /dev/disk/by-partlabel/external
      clevis:
        tpm2: false
        threshold: 1
        tang:
          - url: # …
            thumbprint: # …
      # temporary key"file" i.e. password to unlock disk at provisioning step as clevis unlocking at provisioning is not yet supported
      # see https://github.com/coreos/ignition/issues/1210
      key_file:
        inline: |-
          ******
      wipe_volume: false

[bgilbert]

My phrasing was a bit unclear in #1209 (comment). Stating this as clearly as possible: you need to remove the clevis section from your config. Ignition will not reuse existing volumes if it sees that.

[bgilbert]

Error reporting improvements in #1211.