improve process for migrating from older customized TemplateVM to newer (e.g. Debian 10->11, Fedora 33->34)

[mfc]

The problem you're addressing (if any)

The process for upgrading customized templates in Qubes is very manual with no help from the OS. i'm sure there are ways of making it smoother for the user. i use "customized" to mean "the user has installed additional packages beyond the default installed packages in Qubes, and/or has changed other settings of the Template".

The solution you'd like

some things that would be helpful:

• to be able to get a list what additional packages were installed in a customized template
• to be able to get a list of what other settings changes were implemented (compared to the non-customized template)
• some prompt to try installing those same packages or implement those same changes in another (newer OS) template

The value to a user, and who that user might be

when the user has a template like Debian 10 which has a life of a couple years, they will invariably install some packages or otherwise make some customizations. when, years later, it is time to upgrade the dependent qubes to the next version of the TemplateVM OS, the user will likely not remember all of these changes.

currently it is a tiring trial-and-error post-upgrade process, where the user tries to run some program in a dependent qube (as the shortcuts to apps in dependent qubes will still display to the user even tho the program is no longer installed in the TemplateVM -- the user must refresh the app listing of the qube to get accurate reflection of what is installed -- but these "ghost apps" are currently the only indication that the user needs to install some application still in the TemplateVM!), nothing happens, they think it might be a problem with the app, then remember that the app may not be installed in the TemplateVM, checks & confirms, installs, restarts, etc.

[andrewdavidwong]

currently it is a tiring trial-and-error post-upgrade process

Currently, what we recommend in the documentation is that the user write down the packages and customizations in a separate list somewhere. Of course, this is also far from ideal, and presumably most users don't do this, as it requires a certain discipline most people lack. However, if one does manage to do it, it works much better than trial and error.

[andrewdavidwong]

This is technically a duplicate of part of #3040, but see #3040 (comment). (Also see #2477, which covered only Fedora.)

[andrewdavidwong]

Also note that you can get a history of package manager actions in Fedora with dnf history and some analogous log file in Debian. One can manually reconstruct a list of packages from these histories, and there's probably some more complex way to get the system to generate such a list automatically (like the ones mentioned in #2477). Arguably, the lack of a better or more automated method is a missing feature in the upstream OSes rather than a missing feature in Qubes.

[ninavizz]

I only learned about all of this on Monday, when I finally asked the person who configured my machine why I have 4 versions of Fedora. Mac and Windows OS just don't EOL as Linux ones do; so even knowing this may be a thing, is limited to Linux native users.

EOL notifications would be an easy starting point, but I agree with you both: we totally need a smoother UI-based somethingerother, to help users do this more intuitively.

[andrewdavidwong]

Mac and Windows OS just don't EOL as Linux ones do

That's not really true. For example, the EOL of Windows 7 was a big deal, and it's still a major problem that so many systems continue to run it without any security updates.

EOL notifications would be an easy starting point

Of course, we already have these outside of the OS. See https://www.qubes-os.org/news/categories/#announcements for in-OS notifications.

[ninavizz]

Of course, we already have these outside of the OS. See https://www.qubes-os.org/news/categories/#announcements for in-OS notifications.

The "Hey, this happened!" notification part of the rich info in the articles need to happen within the OS, is the problem. Many users (myself included; and most non-technical users, I'd expect) never think to look at news articles about their computer. Why should I—most of it makes no sense to me, and feels not relevant to me (speaking only for myself, as I have not formally researched this).

I understand that sending an OS level notification is a heavier lift than simply writing the articles, Tweeting/Tooting them, etc., and I also do not say this to put-down the effort that goes into the articles. But, I feel that's the best solution.

Doesn't have to be ultra-elegant/fancy, but a simple notification pointing to these external articles (akin to what I'd drafted in #6905) would go a long way. As would a more holistic Updater experience that would include these notifications.

[andrewdavidwong]

• to be able to get a list what additional packages were installed in a customized template
• to be able to get a list of what other settings changes were implemented (compared to the non-customized template)
• some prompt to try installing those same packages or implement those same changes in another (newer OS) template

Why is any of this necessary when in-place upgrades are possible?

In other words, why the focus on trying to help the user create a separate, new template with the desired programs and customizations rather than trying to help the user upgrade the original template (which already has all the user's desired programs and customizations)?

[marmarta]

Why is any of this necessary when in-place upgrades are possible?

In other words, why the focus on trying to help the user create a separate, new template with the desired programs and customizations rather than trying to help the user upgrade the original template (which already has all the user's desired programs and customizations)?

I think this is a very good point. I think - correct me if I'm wrong, someone-with-more-knowledge - the point is "things can go wrong when updating a template" and also "a clean new template has lower risk of carrying with it old baggage". All reasonable, sure, but I agree that from usability perspective, updating a template is much better than migrating to a new one.

[DemiMarie]

Why is any of this necessary when in-place upgrades are possible?
In other words, why the focus on trying to help the user create a separate, new template with the desired programs and customizations rather than trying to help the user upgrade the original template (which already has all the user's desired programs and customizations)?

I think this is a very good point. I think - correct me if I'm wrong, someone-with-more-knowledge - the point is "things can go wrong when updating a template" and also "a clean new template has lower risk of carrying with it old baggage".

Those are two of the reasons. A third is that all automated testing (openQA, etc) uses fresh templates IIUC.

The “proper” way to manage this is to use Salt to manage everything, but this of course is only feasible for advanced users.

All reasonable, sure, but I agree that from usability perspective, updating a template is much better than migrating to a new one.

Indeed so, especially for the majority who are not using Salt.