Support for GCS private conda registry.

[clement-chaneching]

Problem description

Hello,

I would like Pixi to support private conda registry hosted on GCS buckets.
I had actually pushed a PR to rattler a few months ago to have this GCS Middleware support.

I used to test with something like this but never merged it.

Because of some changes to the rattler_repodata_gateway/gateway crate, I now have:

ERROR resolve_conda{group=default platform=linux-64}:get_or_create_subdir{channel=Channel { platforms: None, base_url: Url { scheme: "gcs", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("blunomy-conda-registry")), port: None, path: "/", query: None, fragment: None }, name: None } platform=Linux64}: rattler_repodata_gateway::gateway: error='gcs' is not a supported scheme
  × 'gcs' is not a supported scheme

And I m not sure how to make it work anymore.
Could you have a look and try to support :

[tool.pixi.project]
channels = ["conda-forge", "gcs://<private-registry>"]
platforms = ["linux-64"]

[tool.pixi.dependencies]
package= { version = "==1.0.0", channel = "gcs://<private-registry>"}

Thanks for your help!

[tdejager]

I think this makes sense, we should do the changes in the gateway I think. Is it normat to use the gcs:// url scheme for this :)?

[clement-chaneching]

I think so for google-cloud-storage :)
I have hardcoded this in crates/rattler_networking/src/gcs_middleware.rs:
if req.url().scheme() == "gcs" {

But feel free to change if needed!

[baszalmstra]

We will have to add this to the gateway indeed! Ill try to take a look at that this week!

[baszalmstra]

Docker support is not yet completely finalized in the conda ecosystem. There is an active Conda Enhancement Proposal that covers this use case. We might want to wait until that has been finalized.

[wolfv]

We do have or had support for the oci:// scheme though. It's even documented :)

[baszalmstra]

I read this completely wrong, I was under the impression that you wanted to use the docker registry API.

I drafted a fix in conda/rattler#845 that hopefully works for your case.

[clement-chaneching]

Hello @baszalmstra

This issue should be reopened because it doesnt work as expected:

So now the "gcs" scheme is supported (not getting × 'gcs' is not a supported scheme anymore, but i m getting URL scheme is not allowed.

The gcs URL should be converted later in the gcs_middleware.rs to https://storage.googleapis.com to handle the auth.

I have also pushed a PR to add the GCS middleware to pixi (cannot test it yet)
Could you check that?