Ability to run the models that are part of the Impact framework in parallel

[srini1978]

Prize category

Help the participant understand which of the prize categories this idea would best fit

• Best Model
• Best Content
• Beyond Carbon
• Best contribution to the framework

User Story

As a developer I want to provide the capability in the framework to be able to run the models in the computation pipeline in parallel so as to improve the response time for calculation of the impact metric.

Rationale

An Impact graph consists of multiple nodes and each node corresponds to a software component. The current way of working of the impact framework is that the impl files from each of the nodes are fed through the computation pipeline serially. In case there is no dependency between the nodes in the graph, there is an opportunity to run the impact framework in parallel so that the computation happens in parallel as well. In case parallel processing is difficult, we can explore asynchronous computation as well.

The aggregation pipeline could be a pipeline that can wait for values from all the nodes to be complete.

Impact

If this idea is successful implemented the computation time of the impact metric will be reduced.

Implementation guidelines

Difficulty

2/5

Scope Of Work

List some of the tasks that will be required to implement this idea

• A new model plugin
• A new feature in an existing model plugin
• A manifest file will have to be written
• Implement test cases
• A new feature to the framework itself (the cli tool)
• It will require changes to the documentation

Examples and resources

IF docs
Model building tutorial

[jawache]

Hey @srini1978, this is an interesting approach but can have some very complex interactions; I was thinking for "contribution to the framework" issues, let's create issues in the IF repo and fully spec them out, and then we can just link to them here in these ideas issues.

So for "contribution to the framework," they will mostly just be taking a fully clear and specced out issue from the backlog and executing it. I'm scared that they are going to have to think through complex arch issues; what if they make the wrong choices and submit a solution that we can't use because it will clash with something else, or have we discounted that approach already?