Changed solver behaviour in release 1.4.2

[jaimergp]

Troubleshooting docs

• My problem is not solved in the Troubleshooting docs

How did you install Mamba?

Other (please describe)

Search tried in issue tracker

N/A

Latest version of Mamba

• My problem is not solved with the latest version

Tried in Conda?

Not applicable

Describe your issue

The new 1.4.2 release broke some conda integration tests that used to work with all previous versions. I took a look at the changelog and I guess the breaking change must be in one of these three PRs:

• Small libsolv improvments #2399
• Use custom function to properly parse matchspec #2433
• Fix segfault in add_pin/all_problems_structured #2428

We are keeping a list of broken tests at conda/conda-libmamba-solver#186. I'll add more details as we investigate, but the key point is that downgrading to 1.4.1 makes everything pass.

mamba info / micromamba info

N/A

Logs

N/A

environment.yml

N/A

~/.condarc

N/A

[jaimergp]

@costrouc has identified the PR that introduced the change. It's #2433.

I am not sure what part of that PR affects the solver, but we do know that these are the conda/conda tests that have started failing:

• tests/core/test_solve.py::test_force_remove_1
• tests/core/test_solve.py::test_aggressive_update_packages
• tests/core/test_solve.py::test_update_deps_2
• tests/test_modified_upstream.py::test_freeze_deps_1
• tests/test_modified_upstream.py::test_conda_downgrade
• tests/test_modified_upstream.py::test_fast_update_with_update_modifier_not_set

[AntoinePrv]

Hi @jaimergp, thanks for reporting this!

Do you know what is the changed behavior? For instance "when updating with a channel-specific ms, previous libmamba would get [...], now it gets [...] / report an error".

If the change is arbitrary, would you consider that this could be an acceptable break for a libmamba 2.0?

[costrouc]

@AntoinePrv so I did a deep dive into this issue and have all the details in conda/conda-libmamba-solver#186 (comment) onwards.

I have three branches

Good branch

costrouc@68c19e5 which just adds some printing of the libsolv state m_flags and m_jobs and what the matchspec <-> id is and is based on the commit just prior to 0345b9d (the commit which "breaks" a few conda-libmamba-solver tests.

Bad branch

costrouc@99ac04e which again just adds some printing of the libsolve state m_flags and m_jobs and what the matchspec <-> id is and is based on the "bad" commit 0345b9d

"Fix" Branch

This is a branch which has a single commit which "fixes" the bad branch costrouc@f72a63d. Essentially this change throws away the conda channel in the matchspec when SOLVER_INSTALL is issued with https://github.com/costrouc/mamba/blob/issue-186-hack-fix/libmamba/src/core/solver.cpp#L308-L320 for the update condition?

This comment https://github.com/costrouc/mamba/blob/issue-186-hack-fix/libmamba/src/core/pool.cpp#L227-L229 does make me think those that this is all related.

The comments in issue conda/conda-libmamba-solver#186 (comment) dive deep into the libsolve state m_job and the integers.

cc @jaimergp since I now have a "fix" but it does feel wrong to throw away the channel.

[costrouc]

Also wanted to note here the scope of the issue. Looking at costrouc@f72a63d which "fixes" these failing tests in conda-libmamba-solver.

This breakage only occurs when performing an update and "complex" matchspecs are specified e.g. not just package name and version constraint (example conda-forge::numpy would be complex). So the scope is not too large.

From this work though it is clear that the mamba behavior did change for this condition (update + complex matchspecs).

[AntoinePrv]

@costrouc thanks for the details. #2433 did make a common entry point for resolving all matchspecs, which sparsingly done before.

if I understand correctly the behavior is with update commands only? I do remember changing the code used by update, but we did not have a clear semantic of what a channel-specific update means.
For instance:

micromamba create -n foo defaults::numpy     # 1
micromamba update -n foo conda-forge::numpy  # 2

Here what is expected of # 2?

• A/ Update numpy to instead use the conda-forge channel?
• B/ Update numpy only if it comes from the conda-forge channel?

The simple thing to do is B because it is simply evaluating the matchspec.

[jaimergp]

@AntoinePrv I would expect more of A than B, but the difference is subtle depending on what versions are available on each channel.

• Let's say we have defaults::numpy=1.10 and conda-forge::numpy=1.12, then the behaviour is obvious: use numpy=1.12 from conda-forge
• If we have defaults::numpy=1.10 and conda-forge::numpy=1.10, I'd still say we should use conda-forge's numpy
• However, if what we have is defaults::numpy=1.10 and conda-forge::numpy=1.9, we would need to consider what the user is expecting. A forced downgrade via changing channels? I think the channel::pkg_name syntax is really trying to say "I want this package from this channel" more strongly than "with the latest version". Otherwise the user would use conda update -c conda-forge numpy maybe? It's not clear and also not standardized. I am not sure either what conda classic is doing in these cases. It looks like channel has higher precedence than version? In that case, superseding the package from the new channel might be right thing to do?

[AntoinePrv]

With more time to think about this, I think B/ is wrong. B/ implies that spec the user provided is applied to the installed package. However we already have update numpy=1.0 meaning update (or downgrade) to a numpy 1.0.
So the correct way to think about it is that the new package needs to match the spec (maybe that was clear for everyone else ^^).

[AntoinePrv]

Would reverting the behavior back to A/ fix this issue?

[AntoinePrv]

Closing as stale, especially with all the changes coming in 2.0. Feel free to reopen a new issue as needed.