Impact Framework Plugin Library & Pipeline Helper

[mr-loop-1]

Prize category

Best Contribution to the Framework

Overview

The webapp is supposed to be the centralised landing page for the upcoming users of the impact framework. Named 'if ecosystem', it provides a common place for the green community to search for plugins and parameters in the impact framework ecosystem. Those contributors who develop their own version of some plugin or extend the functionality altogether to a different horizon, can list their work here for everyone to find and use.

It has a pipeline builder which integrates with the library of plugins and parameters and enables anyone to test out their ideas or to search for best compatible options according to what they have and what they need to say - "now it would make an impact".

Questions to be answered

I divided the plugin scope to standard, unofficial and community, where first two are already in if repositories while community is those plugins that are in other repositories. Can the terminology be improved?

Have you got a project team yet?

I am working on this project alone

Project team

@mr-loop-1

Terms of Participation

• I agree to the hackathon Rules & Terms and Code of Conduct

---

Submission

Project Submission

Summary

The webapp is supposed to be the centralised landing page for the upcoming users of the impact framework. Named 'if ecosystem', it provides a common place for the green community to search for plugins and parameters in the impact framework ecosystem. Those contributors who develop their own version of some plugin or extend the functionality altogether to a different horizon, can list their work here for everyone to find and use.

It has a pipeline builder which integrates with the library of plugins and parameters and enables anyone to test out their ideas or to search for best compatible options according to what they have and what they need to say - "now it would make an impact".

Problem

As the development of framework progresses, more and more people would like to create custom plugins or tweak existing ones or maybe define parameters personal to their project. The possibilities are endless and I believe that's why 'extend the framework' is a category.
There would be users that would like to include an already made plugin according to their project needs and would look at the community to help them find.
My app will act as the common place for both kind of users. It can be used by developers to register their plugin and specify what it takes and what it does. It can be used by consumers to search and find and test which plugin best suits their situation.
The pipeline builder can act as a tool for both new and already familiar user to carve out a compatible solution or see where a new plugin or parameter can fit in.

Application

There are two broad sections of the application.

Pipeline builder:
It is very dynamic in nature and provides informational messages on how to achieve compatibility. Some general cases can be -

• A plugin is added to the pipe, whose required input param is not added then there would be suggestion for that.
• When a output param is desired explicitly by the user, it checks whether some plugin to give that already exists and missing some input param or not. If not, it provides the best* plugin to add to the pipeline based on the current situation of plugins and inputs.
• The pipeline's order can be changed by using the arrow icons, which results in different combination of outputs to be passed down the order
• If a particular input is not given by user, but a plugin down the order is able to get it through a plugin before it, then the builder detects it and gives an informational message for the same.

Plugin & Param Library (npm of impact framework):
This is where accepted* plugins and params from the community can be searched.
I noticed the possibility that there could be multiple variants of the same plugin being deployed in community. Hence, I introduced a scope identifier with @ symbol to specify the repo/org/maintainer of that plugin.

Plus, I also classified them as standard (for official), unofficial and also community (just like universe of apt packages). I think params would be accepted on the list in a more strict way to avoid overpopulation, and hence I only gave the scope identifier assuming all to be official.

Prize category

Best Contribution

Judging Criteria

• The application has the potential to become the go-to page for majority of users who want to use the impact framework. It can be the npm of impact pluigins and pipeline builder can progress into a full mainfest builder
• The idea is to converge the community-driven contributions. As I discuss later, the application is inspired and inline with the design philosophy of framework.
• It can very easily include any plugin built according to the guidelines.

Video

https://www.youtube.com/watch?v=zum1iuZn_dk

Artefacts

Deployed - https://impact-plugins-eco.vercel.app
Github - https://github.com/mr-loop-1/impact-plugins-eco

Usage

Any of input parameters, plugins and their order and output parameters can be explicitly added and the algorithm behind will give the feedback dynamically. Apart from that, most of the things are currently very simple to use.

To run the app, currently the server part is inactive and everything can be done through frontend folder.

git clone ...
cd frontend
npm run dev

Process

Getting the idea just 2 days before the deadline meant working at the edge non stop. I first started with the tech stack and what to handle in server and what in client. I actually used the initial chunk of time on server (express, sql, knex) and schema only, whereas as time ran out I decided to do everything on the frontend (Nextjs, tailwind).

Right from idea, I knew the two most complex challenges - handling the dynamic nature of pipeline builder where there are multiple sources and destinations of info messages & building a good frontend being a backend person.

The algorithm that decides the pipeline builder checks and messages was redesigned and developed several times as some bugs made the then existing implementation a bad choice.

I used error ids to decide the message builder fn to call at a final level. I also used a large amount of state listeners of react to handle how errors would flush out & update when user changes something on frontend.

Inspiration

The idea I came up originally was only for the pipeline builder and also a manifest file builder.

As I read the design philosophy at https://if.greensoftware.foundation/major-concepts/design-philosophy, especially how building plugins for own usage was expected, and how there are not a lot of official/unofficial plugins currently, I decided on the ecosystem library part. This thing is supposed to be similar to the npm website or apt packages of linux.

The biggest inspiration was still pipeline builder as a utility. If the no. of plugins to choose from go even 100, listing all their inputs and outputs, selecting the plugins manually and sorting out their order would be very inefficient. My utility does all in a automated way. I believe my idea to be a winner and that was an inspiration.

Challenges

There was the challenge of handling how to appropriately display errors and suggestions in all three components - input, plugin, output. There was this task of deciding the trigger that send the algorithm function call and I decided that a fully dynamic one would be where every user change is itself a trigger instead of having to manually press a button.

The errors part was problematic because all three components had different error messages for different types of conflicts. Plus, how to propagate these down to the appropriate element (such as input parameter where error occurred) was brain teaser. I implemented a flushing error system with error type, where every user change triggers error updates and type ids define which message get constructed.

Accomplishments

The biggest accomplishment is to be able to complete this to visible finish in 3 days right from thinking the idea. Even though there was not much time, the reflexes still didn't allowed me to deviate much from good coding practices like folder structure and modularity.

The way I quickly went back forth from keyboard and paper to decide on what to take next or to debug the code or deciding the flow of components, was self-motivating and the progress made in the initial day and half kept my stake high.

Learnings

The easiest thing I learnt is something personal which is that how dedication and passion can combine to do the impossible, from giving it a pass because I was not having good ideas to day-night code.

My biggest learning was how to respect and follow the philosophy. I tried to build in the direction that impact framework is thought to be built. Whenever I had to decide on something - I tried to include where impact framework will probably progress and then decided according to that.

Hence, some features that I left out, such as advanced details and filtering in the library (like filter by domain, filter by scope) are because I am not sure whether my approach to these is also the way forward for impact framework.

What's next?

If the solution is accepted, it could be specified officially and further work can be undertaken such as including the docs and making rules etc for plugins and params.

Since the framework team is best equipped with such decisions, they can become the maintainers.

As I dicussed here, such cases need to first be included in guidelines and then they can be included in the application.

Code:

• build the server, currently bypassed
• ability to propose a plugin to be included to the library, currently manually done.
• dashboard to see all the proposals and approve/reject them, currently manually done.
• advanced search and information on library list page, like npm website
• ability to directly copy the pipeline and mainfest config right from the website
• full fledged manifest file builder

where the harmony between available inputs, applied plugins (including order in pipe) and computed/desired outputs is tested out.

[russelltrow]

Hi @mr-loop-1 thank you so much for raising your project, please don't forget to register your project: https://hack.greensoftware.foundation/register/

You must register your project before you can submit your solution for judging. The deadline for submissions is 23:59 UTC on 8th April 2024 (today)