Updating MSYS2 unattended is broken (MSYS2 locks up on full system upgrade)

[Ede123]

Hi all,

the latest update seems to have made it impossible to do a full system upgrade of MSYS2 unattendedly (i.e. via CI on AppVeyor).

1. The removal of catgets makes it hard to even have pacman run an update (MSYS2's own CI is affected) for which I found a workaround, though (yes | pacman).
2. Even with the workaround it's still not possible to complete the update as MSYS2 seems to lock up when warning about
   warning: terminate MSYS2 without returning to shell and check for updates again
warning: for example close your terminal window instead of calling exit

Does anybody know a way how to work around this issue?

Also I seem to recall that it worked previously (MSYS2 did not lock up and CI could continue). This seems not to be true any longer. :-/

[Alexpux]

@Ede123 there are no solution for it. This warning is always when upgrade core packages.
You must close MSYS2 console, reopen it and continue upgrade packages. There are no other way

[Ede123]

Yes, I know the message, but previously the console could be closed and there was no locked process reported by mintty. Nowadays a process remains that needs to be killed.

Also note that for this specific reason I chose to run pacman once directly on Windows console before starting a bash and continuing the update. This still allowed to workaround the issue previously, allowing CI to succeed despite upgrades of msys2-runtime.

[Ede123]

@Alexpux I found the issue:

• if pacman is run with --noconfirm it does not wait but return immediately after full system upgrade, see here
• however if pacman is run with --noconfirm it assumes "no" for the question $1 and $2 are in conflict. Remove $2? [y/N]
• in my last change I removed to --noconfirm to account for Drop the catgets library and facility from packages being offered. m… #1139 which in turn caused the blocking behavior

I think we simply need to either
a) change the default answer to the "remove?" question
b) make pacman return in this case

[Ede123]

Is there any reason for wait_indefinitely(); (even when run interactively?)

[Ede123]

Code introduced in Alexpux/MSYS2-pacman@6e0ff83

[Ede123]

Even found a workaround for it that does not require a code change:

Pacman has an undocumented option --ask that takes a number, which is then converted into a bitmask and is used to override the default answer to questions (so we can use --noconfirm again).

The bit mask can be derived from the question type as defined in the enum _alpm_question_type_t, the default answer is overridden here.

It's certainly ugly and the option was even removed at one point for being "non-user friendly" but added back later (mainly for internal usage, though) but it allows CI to work again!

Maybe removing the blocking behavior is still the easiest solution (probably better than using an undocumented option?). Otherwise we could still attempt to change the defaults, but there does not seem to be a straightforward way to do it (at least I did not find the defaults collected in a central place, it seems they're defined at multiple locations in the code...)

[lazka]

@Ede123 nice find! As far as I can see it just reverses the default answer, and only two commands are defaulting to NO using noyes(), ALPM_QUESTION_CONFLICT_PKG and ALPM_QUESTION_REMOVE_PKGS, so "--ask 20" should make it default to yes for everything.

[Ede123]

Ah, yes, I was looking in the wrong place yesterday and assumed "0" was the default answer to all questions, but it is overwritten in cb_question, so --ask=20 should be the proper value.

What about patching MSYS2 pacman to use yesno() everywhere and therefore default to yes? Or do we find that too "risy"? It would certainly be easier than to require everybody who wants to do unattended updates to use an undocumented option...

[lazka]

Sounds good to me, although in this case it will probably not help appveyor users as the current error will prevent pacman updates.

[lazka]

We should also ask appveyor to update the base install after that.

[Ede123]

I created a PR for the change, unfortunately pacman does not build for me right now :-/

[tico-tico]

@Ede123, will --ask 20 still work with your changes? Imagine that i have an old msys2 installation, to update from the old pacman i can use the workaround but after the update to the new version what will happen? Can i add --noconfirm --ask 20 and forget forever?

[Ede123]

@tico-tico: No, with the changes you should only use --noconfirm.

[tico-tico]

Since i can never be sure what version of pacman i have on remote host your change will actually complicate things for me 😕 .
It means i can't write a script to unattendedly update from unknown version to the last one.

[jtanx]

Yeah, to be honest, diverging behaviour on the same flag as arch pacman sounds bad. The patch would be better if the new behaviour was another option, or better yet, get this change (in some form) upstreamed. Or just stick with --ask 20...

[tico-tico]

I too think the better option is to somehow "document" undocumented --ask and return to the original behavior. I doubt that people expect such change in the pacman habits and will be really surprised. Not everyone follows msys2 commits.
it's up to you of course.

[Ede123]

Unfortunate that you comment now that the change was merged...

I still believe this is the best approach (otherwise I would not have proposed it):

• --ask is not documented, i.e. we can assume people don't know about it and don't use it (especially when they do not follows msys2 commits).
• Without this change there is (except the undocumented internal flag) no possibility to run pacman really unattended.

If upstream follows this would obviously be the ideal case, otherwise I can not see how this can be unexpected or surprising to people:

• When running unattended it's the behavior they most likely want
• When running interactive it's obvious what the defaults are and I believe people are intelligent enough to make the proper choice (I doubt muscle memory is a factor for these rare queries)

Personally I don't care what solution we choose for allowing unattended full system upgrades as long as it's reasonable - --ask is not that solution for me (remember it was specifically removed by pacman developers for not being user friendly and I agree - it should not be an option used be end users).

[jtanx]

I agree that --ask 20 is not ideal, but given that the use case is CI, you would only have to set it once and forget about it, making it more or less a non-issue.

I also agree that having some way to confirm with yes (and without using an undocumented flag) is a needed feature - but I disagree with changing the default behaviour of the flag (and which I infer now also breaks the --ask flag). I disagree because it's a breaking change in the subtlest of ways - sure, it may not be what you would want the flag to do as it stands, but there may already be things depending on this behaviour of not overwriting/upgrading what exists. It's also another behavioural difference to vanilla pacman, which may surprise anyone used to that instead of this msys2 flavour.

As mentioned before, given that there is a workaround in vanilla pacman (--ask 20), this should have just been proposed upstream, or if you really must, patched as a separate flag (like --noconfirm-yes).

[tico-tico]

The main thing that really bothers me is that now it's impossible to update from an unknown version in easy way. Maybe you can just add hardcode --ask 20 to always yes (actually just ignore the switch if parameter == 20)?

[eli-schwartz]

FWIW, while pacman itself is not really targeted at fully noninteractive scripting use, the pacutils collection of libalpm frontends (created by one of the lead pacman developers) has a pacinstall/pacremove/pactrans program which contains a series of options under the heading Prompt Disposition Options.

There is a catchall option too:

--yolo
    Set all prompt disposition options to their all or yes value and set --no-confirm.

So for completely unattended use, pacsync && pacinstall --sysupgrade --yolo will automatically install all upgrades without user interaction, while automatically answering yes to all questions or otherwise optimistically attempting to resolve any prompt option.

Maybe you'd be interested in packaging pacutils?

[flaviojs]

Could pacman kill MSYS2 by itself by sending a signal to the parent? (based on a command line flag?)

[goyalyashpal]

where does this issue stand as of now?