split outputs, build with rattler-build

[minrk]

use rattler because this is a lot of outputs and it saves a lot of time for the many env setup/teardown transactions over conda-build

adds suitesparse-mongoose as an additional component and unskips spex on windows, now that the required gmp is available there.

Further splitting not done, that could be, and I think could be done later without major consequence:

• suitesparse-devel to contain headers/cmake/pkg-config files (I think this needs the approach in Rattler single build, separate suitesparse-devel #99, which conda-smithy can't handle yet)

• mongoose is both a lib and executable so could be split further, but since it's one package and doesn't support component install, I did not make libsuitesparse-mongoose, only suitesparse-mongoose with bin and lib.

• needs prepare to split suitesparse outputs admin-requests#1193 before the new outputs can be published

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint --conda-forge . from the recipe directory. You can also examine the workflow logs for more detail.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12031395698. Examine the logs at this URL for more detail.}

[minrk]

lint error is conda-forge/conda-smithy#2165 already fixed by conda-forge/conda-smithy#2166

[minrk]

This actually runs all the builds independently in sequence. A faster approach would be to keep the single 'build and install everything' and use files. Downside of files: we need to keep track of which files go with which package, and know about every added file for each release. The approach in this PR is a lot easier to keep correct, at the expense of some time (still only 5 minutes on linux-64 thanks to the switch to rattler-build).

[xhochy]

At this point you could also think of splitting it into libpkg and libpkg-devel. If you do it at a later point, you can no longer use libpkg as the package containing only the runtime library as this would probably break existing packages (too much).

[minrk]

@xhochy that's a good point. I did think of that and figured the cost of the dev files wasn't big enough to justify the recipe complexity. It's a little hard with this multi-package layout where multiple builds have to run unless I go with a single build and pure files splitting since I don't think I can run a bunch of builds and do files splitting from several different build steps.

One way to do the splitting that I think would be maintainable is to switch to a single build and do files splitting where:

• lib* packages include only libs (a simple glob for each)
• a single suitesparse-devel includes all the headers/cmake files, making the file-include patterns tractable (include/, lib/cmake/ etc.)

so that a package depending on e.g. umfpack would have host:

- libumfpack # lib, run_exports
- suitesparse-devel # headers, etc., no run_exports

Do you think that's worth it?

Some numbers for the fully installed suitesparse to help inform the choice, which should be unchanged (plus the addition of suitesparse-mongoose):

• total instal size: 5.4MB, 159 files
• libs: 4.1MB, 51 files (2/3 are symlinks)
• include: 800kB, 19 files
• cmake, pkg-config: 500kB, 89 files

so the devel package would reduce the number of files for a full suitesparse install by 2/3 and the size by only 20%.

The single-build would also have to wait for conda-forge/conda-smithy#2057 to resolve because rattler-build currently has the build cache behind an --experimental flag we can't yet pass.

[xhochy]

@xhochy that's a good point. I did think of that and figured the cost of the dev files wasn't big enough to justify the recipe complexity.

That is something where I don't have an opinion on as I am not a user of this package, but I stumbled on this case of libpkg vs libpkg-devel several times in the last days and wanted to raise it again, so that if you would have long-term plans for it, that you don't "fall into the same trap":

[minrk]

Since most packages don't split -devel on conda-forge, I've mostly been doing it considering the size/quantity of devel files compared to both the library itself, and the size/frequency of a typical env that depends on it. 1MB of headers in an env that's unlikely to be less that 300MB due to other dependencies doesn't seem entirely worth it.

I also don't think splitting further libumfpack into libumfpack-devel down the line is likely to break anything, as it would only mean the next build depending on libumfpack would need to add -devel, but runtime would only see an install-size savings without any breakage. Or is there something I'm missing there?

[minrk]

@conda-forge-admin please lint

[wolfv]

nice 👀

[wolfv]

I am sure you know, but inline jinja should also work (${{ "ninja" if win else "make" }})

[minrk]

I had the vague feeling that using logic expressed in static yaml would be a little less fiddly for tooling like conda-smithy than using a single string template. Do you think that's true? I'm sure it doesn't really matter for this particular dependency, but I'd love to know more about best practices.

[wolfv]

For rattler-build it doesn't really matter as we have to parse the jinja AST in both cases anyways (the if: <cond> is also evaluated with minijinja, and we extract any used variables from there before rendering the recipe).

[minrk]

Yeah, I had the model in mind that when conda-smithy might be collecting the list of dependencies, it can concatenate all the lists and ignore the conditionals, assuming every branch will be taken by some build. But that may not have anything to do with what actually happens. And presumably it still needs to evaluate jinja for every build matrix entry anyway for completeness, so it probably doesn't make a difference.

I just get the feeling that it's nice to avoid templates where possible in the new format, even if it doesn't actually matter in practice.

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.