Impact Framework for LLMs: Injecting transparency into the rise of GenAI

[carolineclivio]

Prize category

Best Content

Overview

Impact of GenAI and LLMs on our Environment

Our project focuses on importance of evaluating the environmental footprint of Large Language Models (LLMs) that power GenAI. Through a short educational video we want to help the software community quantify and understand LLMs' impact on water usage, energy consumption, and carbon emissions. We will be providing actionable insights and recommendations for reducing LLMs' environmental footprint, with the goal of creating some collaboration among stakeholders for a more sustainable AI future.

Questions to be answered

No response

Have you got a project team yet?

Yes and we aren't recruiting

Project team

@beccastanley
@ghrrngtn
@carolineclivio

Terms of Participation

• I agree to the hackathon Rules & Terms and Code of Conduct

Section	Answers
Summary	An educational piece aiming to provide background on the rise of Generative AI (GenAI)and Large Language Models (LLM) and the environmental impacts this growth is causing. We will detail how the Impact Framework can be used to bring transparency to this space and allow organisations to implement better AI Governance and empower end users to make better decisions regarding when to use Generative AI based on the environmental impacts incurred.
Problems	The lack of transparency for the consumer [business or individual] in knowing the environmental impacts caused by GenAI and LLMs usage. Currently, high level data is published by some of the hyperscalers and AI players regarding the training and, more rarely, the usage of these tools. But, in order to understand the actual impact associated with an  individual or organisation's usage, much more detail and data is required.
Application	Our piece outlines the different lifecycle stages of a Generative AI model and highlights the carbon/energy, water and waste impacts of each. We then detail how the impact framework can be used to calculate the specific carbon and water impacts both whilst planning out the architecture of a model and during the use phase by the end user. We use the analogy of GenAI being a "silent employee" and the embodied carbon emissions of an office building being akin to the model training emissions whilst the emissions associated with employee building use are similar to those associated with inference. Whilst we have smart meters and other sensors to monitor building emissions, the Impact Framework is key to providing the granular data required to give this insight for digital systems.
Prize category	Best Content
Judging criteria	The potential impacts of this piece could be:   Increased awareness of the environmental impact of Generative AI usage Implementation of the Impact Framework to monitor Generative AI impacts associated with organisational usage Rollout of AI governance programmes with measures for financial cost, productivity/business gains AND environmental impact. This could include targets for each where a cost/benefit analysis must be undertaken with clear business aims, rather than a blanket roll out of a new AI tool.   For this impact to occur, we would envision a general awareness campaign and/or a company specific version of this piece being delivered to the key decision makers in the technology/digital space to kick-start a workstream of calculating the impact of Generative AI implementation. This would form a pillar of a company’s AI Governance Programme and we suggest it should link in to their Sustainability strategy.
Video	https://youtu.be/T0ZXH7mVrPw
Artefacts	https://www.canva.com/design/DAGAKPC3vR0/9CuIGZQ2RMr3zD6kQYI9Mg/view
Usage	N/A
Process	We wanted to address the lack of data and transparency in the field of AI and decided upon a visual story telling methodology to be as approachable and engaging as possible. We worked alongside our technical colleagues to understand the potential that the Impact Framework has to offer in this space, reviewing the number of data points and impact measures that were currently available. This was tested on a live Large Language Model that we are utilising in the business to confirm the feasibility. We then created the story around this technical possibility.
Inspiration	We all work for a technology company and have a deep passion for Sustainability. We believe that digital technology should be created just like any physical asset a company purchases and uses in terms of how we think about its environmental impacts. However, this impact is often invisible to end users and difficult for them to grasp beyond the energy required to power their device through which they interact with the technology. We wanted to shine a light on these impacts in an industrialised way so that this type of carbon accounting can become as mainstream as we see with physical assets and employee behaviours (commuting etc.). Only once you understand the impact, will it be possible to reduce and optimise usage as we do in the other GHG categories.
Challenges	One of the main challenges was being realistic on the scope of the impact framework against the large number of impacts that the wider Generative AI ecosystem has on the environment. We were very optimistic and wide ranging in our initial storyboard but has to liaise with our technical colleagues to understand how this would be implemented and scaled in practice.
Accomplishments	We are proud of how, as a non-technical team, we researched the area and worked with our technical colleagues to understand the problem and potential solution, and then translate this into an engaging output. We really enjoyed discussing how we could best communicate a concept to others that don’t have a background in software, and which metaphors and analogies would be the easiest to understand. It was difficult to manage the research and presentation creation alongside our day jobs and holidays, but we are proud of the team effort we were able to give in the short time period.
Learnings	We unfortunately learned a lot about the negative impacts the AI is having on our environment and the strains it is putting on already stressed communities. Going into the detail of the life stages of a Generative AI model taught us techniques to optimize the model we have and how we can influence user behaviour to minimize the negative impacts. However, it was great to learn about the potential for the Impact Framework and understand what sort of information is required from a piece of software to understand the water usage and carbon emissions. Through doing this hackathon it has made the thought of footprinting our projects and technology implementation on behalf of our clients seem a lot more realistic and approachable.
What's next?	We hope that this piece provides a way of engaging with and introducing the Impact Framework to non-technical audiences. We aimed to show the potential of the framework to solving the data transparency issue for one of the most talked about technology trends. By engaging on a topic that everyone has heard of, we envision this piece opening the door to conversations within organisations about their software emissions. Generative AI is currently probably a small part of an organisation’s footprint, but if they can implement a robust measurement and governance framework around a newly implemented tool, we hope that this would be used as a template to measure and monitor the vast number of systems that are utilized and truly understand and reduce the footprint of their digital systems.

[carolineclivio]

1. Hyperlink to the Canva: https://www.canva.com/design/DAGAKPC3vR0/9CuIGZQ2RMr3zD6kQYI9Mg/edit?utm_content=DAGAKPC3vR0&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton
   Click the "Present" button at the top right of the screen, and choose "autoplay"
2. Short video: https://youtu.be/T0ZXH7mVrPw
3. Content Submission as a video link: https://youtu.be/wTcSBj5d5jo