Install cloud images from oscontainer (or at least pretend)

[jlebon]

Right now, the oscontainer and cloud images are completely separate. In theory, the latest container pushed and the latest cloud image should contain the same commit. Though it's not as direct as installing the OS from the oscontainer. The reason for this is that at the cluster level, all it cares about is the oscontainer. It doesn't know anything about OSTree commits or OSTree versions.

One hacky but easiest path for now would be to "fake it" in %post by fiddling with the origin file. Though we'd have to check first that the cloud image we just built corresponds to the latest oscontainer in the registry, and if not... jump ship? Yuck.

[jlebon]

Alternatively, change the status quo and make it care about OSTree commits/versions? Basically, broaden the MachineConfig spec to also include not just the OSImageURL but also the matching OSTree commit/version it expects. That just seems like a hack too though and it's just more fields that could be out of sync.

[ashcrow]

Versioning should be handled at the Version Control Operator (or similar named) which handles preparing manifest for each release. Those then reference the proper OSImageURL among other things which are to be used for a release, update, change, etc..

[ashcrow]

/cc @crawford

[cgwalters]

I was hoping to add support for oscontainers into coreos-assembler. This would follow up from coreos/coreos-assembler#90

[jlebon]

I was hoping to add support for oscontainers into coreos-assembler.

That's just to teach c-a to output oscontainers, right? It would help with the binding problem if the oscontainer and the cloud image is built in one shot, but we still need that cloud image to have the right custom-url.

So one thing we could do here:

1. in the cloud pipeline, key off of the latest pushed container to determine whether there are changes
2. on changes, use the commit from the io.openshift.os-commit label as the arg to ostreesetup
3. tweak origin file in %post to add custom-url

?

[jlebon]

Related: #281

It sounds like we need to weigh whether this needs to be done now, or wait until we pivot to the latest c-a and implement it there. The way the MCD is set up, things would still work OK without this (it'll just force an update to whatever oscontainer it was handed). Though for our developers, it can get annoying to always endure a reboot even if using the latest private AMI. (On the public side, the AMIs are outdated anyway, so rebooting is expected).

One thing we could do to alleviate this until then is to teach pivot to modify the origin in place without rebooting if the commit matches and tell the MCD there's no need to reboot.

[cgwalters]

I roughly agree with #307 (comment) - I am indirectly working on this via taking on coreos-assembler+ootpa and that's mostly hitting 1+2. Then I'll look at step 3, shouldn't be too hard to inject some bits into %post.

[cgwalters]

This is done now.