Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use mamba 2 for the base installation of Miniforge #715

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

jjerphan
Copy link
Member

Follow-up of #694 after the merge of conda/constructor#914.

I am looking for ways to test this. For now, the Miniforge example in conda/constructor is the only one element which allows testing it locally.

@jaimergp, @hmaarrfk: do you have any recommendations for better testing this? 🙏

@jjerphan jjerphan force-pushed the mamba-2 branch 2 times, most recently from c3c4c99 to 5c99450 Compare January 20, 2025 13:37
Signed-off-by: Julien Jerphanion <[email protected]>
Signed-off-by: Julien Jerphanion <[email protected]>
@hmaarrfk
Copy link
Contributor

@jaimergp, @hmaarrfk: do you have any recommendations for better testing this? 🙏

can you remind me what the issue was and how it manifested itself?

I had tests that ran the installer itself, maybe we need to install a new package in the base environment? test creating a new environment?

@jjerphan
Copy link
Member Author

Yes, I am adding a few command to create an environment, install and uninstall package to test.

I want to make sure that it is not brittle.

@hmaarrfk
Copy link
Contributor

I want to make sure that it is not brittle.

Great thanks!

I just merged a few using github's interface to resolve some merge conflicts. Got one last mamba1 version out ;)

@jjerphan
Copy link
Member Author

I guess we could have conda 25.1.0 part of this once conda-forge/conda-feedstock#256 is merged.

@jjerphan
Copy link
Member Author

Currently:

error libmamba No CA certificates found on system, aborting
critical libmamba No CA certificates found on system, aborting

I am trying to see what's the best way to install CA certificates on the machines.

@hmaarrfk
Copy link
Contributor

I am trying to see what's the best way to install CA certificates on the machines.

If i recall correctly i just run the base docker images, so however yo uwould do it for the respective docker images (ubuntu based vs fedora based)

Signed-off-by: Julien Jerphanion <[email protected]>
@jaimergp
Copy link
Member

jaimergp commented Jan 20, 2025

A similar error was reported in mamba-org/mamba#3570

@isuruf
Copy link
Member

isuruf commented Jan 22, 2025

Miniforge/mamba needs to work even if the OS has outdated certificates. Trying to fix the docker image is not the solution here.

@jjerphan
Copy link
Member Author

@jaimergp: IIRC, you told me once that you observe this exact problem, and we were wondering whether this was due to ca-certificates or certifi being present or absent (mamba 1.x depends on them transitively), but mamba 2.x does not anymore.

Signed-off-by: Julien Jerphanion <[email protected]>
@jjerphan jjerphan marked this pull request as ready for review January 22, 2025 09:38
@jjerphan jjerphan requested a review from a team as a code owner January 22, 2025 09:38
Copy link
Member Author

@jjerphan jjerphan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments and questions for reviews.

@@ -84,3 +107,44 @@ python -c "import platform; print(platform.machine())"
python -c "import platform; print(platform.release())"

echo "***** Done: Testing installer *****"

echo "***** Testing the usage of mamba main commands *****"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for reviewers: are there other commands or behaviors one could test?

Comment on lines +113 to +115
echo "***** Initialize the current session for mamba *****"
export MAMBA_ROOT_PREFIX="/root/miniforge"
eval "$(mamba shell hook --shell bash)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for reviewers: MAMBA_ROOT_PREFIX and the shell hook will be set in the shell profile file by the installation script if users choose to have it run during the installation, which is not the case here.

I would also like to test when this is the case if possible.

@@ -37,5 +37,6 @@ for TEST_IMAGE_NAME in ${TEST_IMAGE_NAMES}; do
echo "============= Test installer on ${TEST_IMAGE_NAME} ============="
docker run --rm \
-v "$(pwd):${CONSTRUCT_ROOT}" -e CONSTRUCT_ROOT \
--platform "linux/${DOCKER_ARCH}" "${DOCKER_ARCH/\//}/${TEST_IMAGE_NAME}" /construct/scripts/test.sh
-v /etc/ssl/certs:/etc/ssl/certs:ro \
Copy link
Member Author

@jjerphan jjerphan Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side-note: I am not sure about the source of CA certificates libmamba 1 uses, but libmamba 2 uses the system's CA certificates by default now, so the trust store storing them now needs to be mounted.

Are there other scripts or docker image usage in miniforge or conda-forge for which the trust store will need to be mounted?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there other scripts or docker image usage in miniforge or conda-forge for which the trust store will need to be mounted?

No, everything depends on openssl package from conda-forge which uses ca-certificates package in conda-forge. So, libmamba=2 behaviour is wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so this might explain issues like mamba-org/mamba#3570.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once downloaded, micromamba should just work "offline" no internet or certificates required.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mamba-org/mamba#3765 aims at resolving it.

@wolfv
Copy link
Member

wolfv commented Jan 22, 2025

I think we should double check the serialization of URLs in prefix records to make sure that they conform to the "conda standard" (which sadly doesn't exist) - but it should look similar to mamba 1.x / conda / pixi (ie. not URL encoded).

@jjerphan
Copy link
Member Author

jjerphan commented Jan 22, 2025

I think we should double check the serialization of URLs in prefix records to make sure that they conform to the "conda standard" (which sadly doesn't exist) - but it should look similar to mamba 1.x / conda / pixi (ie. not URL encoded).

I can continue mamba-org/mamba#3745 and so that we can have it part of 2.0.6 (cc @maresb), and then we can update this PR to use 2.0.6. Would this work?

@SylvainCorlay
Copy link
Member

Are there other scripts or docker image usage in miniforge or conda-forge for which the trust store will need to be mounted?

No, everything depends on openssl package from conda-forge which uses ca-certificates package in conda-forge. So, libmamba=2 behaviour is wrong.

This is a place where micromamba should differ from mamba again then.

Miniforge3/construct.yaml Outdated Show resolved Hide resolved
Co-authored-by: Jannis Leidel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

7 participants