upgrade: Add support for --pull-only and --deploy-only #642
Conversation
|
I didn't test this locally yet, and would definitely need test suite additions. |
|
One major piece this is also missing is that |
|
I'd say that |
cgwalters
force-pushed
the
upgrade-split-pull-deploy
branch
from
47cfd4a to
22810f4
Compare
|
Okay, took a crack at fixing the gap by adding a "synthetic" pull flag that's used with (I still didn't test this much sorry, typing some code here while waiting for some tests to run) |
This makes it easier to script downloading updates in the background, and only do deployments just before rebooting. Partially addresses ostreedev#640
cgwalters
force-pushed
the
upgrade-split-pull-deploy
branch
from
00aa835 to
47f08ff
Compare
|
Rebased 🏄, now with docs 📚 and tests ✅. |
|
Removing WIP. |
| @@ -85,7 +85,8 @@ gboolean ostree_sysroot_upgrader_check_timestamps (OstreeRepo *repo, | |||
|
|
|||
| typedef enum { | |||
| OSTREE_SYSROOT_UPGRADER_PULL_FLAGS_NONE = 0, | |||
| OSTREE_SYSROOT_UPGRADER_PULL_FLAGS_ALLOW_OLDER = (1 << 0) | |||
| OSTREE_SYSROOT_UPGRADER_PULL_FLAGS_ALLOW_OLDER = (1 << 0), | |||
| OSTREE_SYSROOT_UPGRADER_PULL_FLAGS_SYNTHETIC = (1 << 1) /* Don't actually do a pull, just check timestamps/changed */ | |||
There was a problem hiding this comment.
Do we need this flag though? Seems like it would be cleaner to just seed new_revision with the refspec on upgrader init (similar to what we do in rpm-ostree) and then just not call ostree_sysroot_upgrader_pull() at all, no?
There was a problem hiding this comment.
The equivalent of RPMOSTREE_TRANSACTION_DEPLOY_FLAG_NO_PULL_BASE? So...yeah, I guess we could do that, but I like keeping the timestamp check in there, and I'd argue we should probably do the same in rpm-ostree.
What if e.g. someone did a raw ostree pull --commit-metadata-only or something, which won't do timestamp checks and hence could be more easily MITM'd - then later an admin does ostree admin upgrade --deploy-only, we'd be skipping the timestamp check.
| * compatiblity, we only do cleanup if a user specifies the new --pull-only | ||
| * option. Otherwise, we would break the case of trying to deploy a commit | ||
| * that isn't directly referenced. | ||
| */ |
There was a problem hiding this comment.
I'm still not sure I understand this. Is there any risk to leaving the intermediate data until the cleanup that happens after deployment? Isn't this making us lose potential static delta jumps between consecutive --pull-only operations?
There was a problem hiding this comment.
If a pull is successful, it will set the ref. Let's call this the "new pulled base", as distinct from creating a new deployment. ostree_sysroot_cleanup won't reset the ref.
Hence we should (though this isn't explicitly tested right now) keep deltas, since we still have the ref. Now, that's all the argument for why this won't break anything.
Why I wanted to do this is - think about a scenario where due to a flaky internet connection we do a pull, it gets interrupted, do another pull, it gets interrupted, repeat. If this happens a lot, we will endlessly accumulate partial new commits.
With this change, we'll trigger the one-day-timeout on staged commit data. Now...perhaps we should really do the cleanup after an unsuccessful pull, to ensure that we will always reuse downloaded-but-not-consumed deltas? Will look at that.
There was a problem hiding this comment.
Moved some of that to #713
|
Looks good! |
|
☀️ Test successful - status-atomicjenkins |
As @cgwalters mentioned in coreos#1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642
As Colin mentioned in coreos#1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642
As Colin mentioned in coreos#1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642
As Colin mentioned in coreos#1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642 Closes: coreos#687
As Colin mentioned in coreos#1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642 Closes: coreos#687
As Colin mentioned in #1035, the new `--cache-only` implemented only the rpmmd half of the story. Here we complete that story by also ensuring that when in cache-only mode, we don't download new ostree data nor new packages. We try to complete the requested operation with what we have. To do this, we add support for the same `SYNTHETIC` pull that was added in ostree[1] so that we don't actually pull, but still perform timestamp checking. On the pkgcache side, we disable all remote repos and instead insert all our cached RPMs into the `DnfSack`. Care is taken to still perform SHA256 verification for local pkg installs/replacements. [1] ostreedev/ostree#642 Closes: #687 Closes: #1049 Approved by: cgwalters
This makes it easier to script downloading updates in the background,
and only do deployments just before rebooting.
Partially addresses #640