Enforcing clarity about plugin inputs and outputs

[heaversm]

One question I've encountered in building pipeline is that deciphering what each plugin does, what inputs it requires (and in what units), and what outputs it produces (and in what units) is varied amongst different plugins, and not always clear.

I'm wondering if either there is a place where this can always clearly be found, or whether you've thought at all about how to enforce that clarity ; universality within the manifest file and / or the plugin itself.

Here's an example for clarity (working off this repo)

The "operational carbon" module, nicely, has within its global config (in initialize), this format:

where it's clear it takes in cpu/energy and grid carbon intensity, and outputs carbon. Yet even here we would still need to do some digging to find out in what functional units these inputs and outputs occur, and, as newbies, what these things even are, and from which plugin you might be able to get those inputs.

Other plugins, such as "group-by" - have their config not within the initialization, but within the "config" root node - which seems a bit inconsistent - it means you have to look two places for potential configuration.

Lastly, in examining the output file, we have something like this:

which is full of rich data, but again, in what units, from which plugin were the numbers derived, and what does each number tell us?

I imagine eventually someone who incorporates this deeply into their pipeline will either already be, or become, an expert in this - but I'm wondering if it's worth spending some of our team's time on providing some guidance for a more clear way this could be organized. And if not, perhaps you can provide the best way, given the plugins being used in a particular pipeline, to quickly find these things.

[josh-swerdlow]

Hi! I'm sure you would like one of the core contributors to respond to this, but I have some thoughts so I put them together. I don't have an answer to would interact with the core framework, but share your concerns and have ideas for balms.

I think this #52 has a lot of really good points already on consistency of units and clarity on input/output if you haven't read it already.

A colleague and I are testing an mvp visual studio code extension that could allow for auto-complete/suggestion of manifest files and hover over expansion for plugins (to show descriptions/README/something) and parameters (to show relevant metadata like units).

This is a band-aid and supposing it works and we finish it, depends on plugin owners to add their new parameters to the manifest as specified https://if.greensoftware.foundation/major-concepts/parameters.

That said, I thought parameters should always been defined in params.ts or an associated params field of the manifest per the doc above. So parameters should always be well known and consistent or else not run.

I've been thinking of the following perspective a lot. Imaginge someone who receives a manifest and is asked to understand it. This could be because you're a new developer who wants to fiddle with it or a colleague who is auditing their colleagues work. In both cases, you want to easily be able to figure out what they did and how it happened. An idea my hackacthon partner and I had was to create a more verbose output from an exporter plugin (human-readable-manifest) that could annotate the operations that occurred or provide direct references (to files) to find more information. I've always been dubious about sending someone a manifest file for an audit. Unless they are re-running it themselves or triaging via fiddling, I would rather audit someones work with human text. What do you think?

[heaversm]

Sorry - just getting around to responding to this. Love the idea of a vscode extension. I do think a web interface would still be a viable and useful way to accomplish this too. Happy to help in either regard if it's useful. In general, think it's way too open ended to rely on plugin creators right now to build plugins in a way in which they are compatible with the framework as a whole and include the necessary information in the necessary formats to facilitate accomplishing general system objectives given expected next or previous step actions. I like the idea of a params.ts - but even more ideal is that the manifest.yml just requires all of the things you might find there (thus being language agnostic). A human readable output would be a good step regardless.

…

On Thu, Apr 18, 2024 at 9:41 PM Josh Swerdlow ***@***.***> wrote: Hi! I'm sure you would like one of the core contributors to respond to this, but I have some thoughts so I put them together. I don't have an answer to would interact with the core framework, but share your concerns and have ideas for balms. I think this #52 <#52> has a lot of really good points already on consistency of units and clarity on input/output if you haven't read it already. A colleague and I are testing an mvp visual studio code extension that could allow for auto-complete/suggestion of manifest files and hover over expansion for plugins (to show descriptions/README/something) and parameters (to show relevant metadata like units). This is a band-aid and supposing it works and we finish it, depends on plugin owners to add their new parameters to the manifest as specified https://if.greensoftware.foundation/major-concepts/parameters. That said, I thought parameters should always been defined in params.ts or an associated params field of the manifest per the doc above. So parameters should always be well known and consistent or else not run. I've been thinking of the following perspective a lot. Imaginge someone who receives a manifest and is asked to understand it. This could be because you're a new developer who wants to fiddle with it or a colleague who is auditing their colleagues work. In both cases, you want to easily be able to figure out what they did and how it happened. An idea my hackacthon partner and I had was to create a more verbose output from an exporter plugin (human-readable-manifest) that could annotate the operations that occurred or provide direct references (to files) to find more information. I've always been dubious about sending someone a manifest file for an audit. Unless they are re-running it themselves or triaging via fiddling, I would rather audit someones work with human text. What do you think? — Reply to this email directly, view it on GitHub <#136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD7CXONIESY4A2HLVDQUGLY6CN75AVCNFSM6AAAAABFR7BLW6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TCNRSGA2DI> . You are receiving this because you authored the thread.Message ID: <Green-Software-Foundation/hack/repo-discussions/136/comments/9162044@ github.com>

-- Mike Heavers Digital Futures Told <https://digitalfuturestold.com/>.

[josh-swerdlow]

Thanks, Mike! I'm always watching the other thread where others are discussions the units problem which includes our discussion here. @atwoosnam is working on the vscode extension. He's working on a mvp to demo and get feedback. your thoughts when that is posted would be great. Will tag you!

[heaversm]

Please do ! Mike Heavers Digital Futures Told <https://digitalfuturestold.com/>.

…

On Wed, May 15, 2024 at 9:33 AM Josh Swerdlow ***@***.***> wrote: Thanks, Mike! I'm always watching the other thread where others are discussions the units problem which includes our discussion here. @atwoosnam <https://github.com/atwoosnam> is working on the vscode extension. He's working on a mvp to demo and get feedback. your thoughts when that is posted would be great. Will tag you! — Reply to this email directly, view it on GitHub <#136 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD7CXPMVPZHQZVSKL7YOL3ZCOE53AVCNFSM6AAAAABFR7BLW6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TINBYGM4DK> . You are receiving this because you authored the thread.Message ID: <Green-Software-Foundation/hack/repo-discussions/136/comments/9448385@ github.com>