Iris Core + Plugins

[trexfeathers]

Separating Iris API into 'core' API and multiple 'plugins' is a popular ambition offline, so deserves to be recorded publicly, especially for voting purposes 😉 .

Benefits:

• Smaller package(s)
• Quicker to install - fewer core dependencies
• Can run on Iris on smaller machines e.g. AWS lambda
• Faster testing - plugins (with accompanying dependencies) get their own separate testing routines
• Clearer development philosophy

Examples of elements that could be plugins:

• Plotting
• Regridding
• File loaders/savers

[tkknight]

Having some specific user requirements/scenarios for this would help.

[bjlittle]

Regardless, it begs the fundamental Iris developer question: What is the core functionality of Iris?

[rcomer]

There are various small packages out there that do specific things that are useful for scientists. For example, I am currently having a go with xeofs which currently has interfaces for pandas, xarray and numpy types. If one were to think about adding an Iris interface to such a package, it would be useful (for testing) to have an Iris flavour that knows about the cube class and probably not much else.

[pp-mo]

@HGWright pointed out : The big cost of this, not just the effort of doing it but the ongoing extra repo maintenance (e.g. multiple dependabot hits)

[trexfeathers]

I am still in favour of establishing a core 'concept' - defining Iris core determines Iris' core dependencies - but I don't think plugins would be worth the hassle. We can instead cut the Conda recipe down to the core dependencies and introduce errors/warnings for when a user needs a core dependency but has not yet installed it.

We can also use the core concept to structure our tests such that we don't have enormous environments everywhere. Perhaps only our core is tested against multiple Python versions, while anything involving optional dependencies is only tested against latest Python.

[trexfeathers]

An important consideration:

• Downstream testing #5936

[trexfeathers]

Something not well captured above:

Iris-grib and iris-esmf-regrid have benefitted from being separated from the release cadence of Iris itself. This may be an opportunity for other areas of Iris too.

If any part of Iris is unlikely to reap this benefit (e.g. our plotting capabilities do not evolve very frequently) then I am still keen to explore optional Conda dependencies rather than full blown repo separation.