[Context] Choice of signal for I #357
Conversation
|
|
||
| For marginal (short- and long-term) it is very difficult to develop standards, since they are both based on assumptions and statistical models that approximate reality. Contrary to the average, it is not possible to compare marginals to what happened in reality and thus it is quite difficult (I assume not impossible) to develop a standard. For instance, the average can easily be verified based on the hourly power mix that is published by TSOs. | ||
|
|
||
| ## Comments / Perspectives |
There was a problem hiding this comment.
Copying my comment to this pull request.
One suggestion would be to list use cases where various signals apply. This would be helpful for partitioners to make decisions. Ideally, it should be a hybrid signal
For instance -
An application would like to calculate SCI daily end of the day, which signal should be used.
An application would like to calculate SCI trends over the last 15 or 30 days.
An application would like to schedule jobs in the next 24-48 hrs and would like to run when carbon intensity is lowest.
An application would like to calculate SCI every X minutes (near real-time) and make certain decisions based on utilization and carbon intensity
An application would like to calculate SCI before and after deployment (i.e., previous and new releases)
The grid data is only available or reported every X intervals (i.e., 6 months, yearly), which signals to use
There was a problem hiding this comment.
This might be a good fit for the guide?
| For marginal (short- and long-term) it is very difficult to develop standards, since they are both based on assumptions and statistical models that approximate reality. Contrary to the average, it is not possible to compare marginals to what happened in reality and thus it is quite difficult (I assume not impossible) to develop a standard. For instance, the average can easily be verified based on the hourly power mix that is published by TSOs. | ||
|
|
||
| ## Comments / Perspectives | ||
|
|
There was a problem hiding this comment.
Some comments from me.
If we put ourselves in the shoes of the intended audience of this spec, I dare say the vast majority will get the emission factors from a third party. Very few, if any, will attempt to calculate this data on their own. Since our goal is to have the SCI be easy to implement, I feel that neither AER, SRMER or LRMER is bad to use, as long as you continuously use the same signal in a proper scientific method and report which one you actually used.
I would dislike if the standard dictated to only use one of these, if it meant that everyone looking to calculate a SCI score were locked to one 3rd party provider of this data or if they suddenly became unable to produce a score.
In the end, for many software practitioners, carbon aware actions might not be possible due to the nature of their software or it might be low on their list of potential impactful changes. Having any signal for I is better than not being able to produce a SCI score and take action.
There was a problem hiding this comment.
Very interesting take on this issue and I love your comment on the lock in risk. Overall, I very much agree with you that this specification should not dictate to use only one of these, but allow the usage of all of them. As outlined in the PR I opened, it would facilitate the adoption of the SCI if we don't mandate only one signal, but open it up for other, equally relevant signals.
|
Some thoughts from my side: The overall goal of this document is to provide a concise, yet comprehensive overview of the discussion and the important aspects of it. As far as I have understood, this document should inform the discussion around what carbon intensities to use as signals in the SCI. To this end, I believe it would be helpful to clearly specify why the discussion was started and what the implications of such changes would be. To facilitate this, I suggest the following:
This topic is vast and complex for everyone to keep an overview and I believe it would be helpful to focus on information that is helpful in this specific discussion. |
|
I do think that the precise topic of discussion needs to be clearly stated at the top of this document. My suggestion for an addition is: Topic of DiscussionThe reason why this discussion was opened is to have a conversation around which emission factors should be permitted for use in the SCI. As PR #353 raises, it is unnecessary to restrict the SCI to a single carbon intensity metric and it therefore recommends to open up the SCI to allow for multiple, equally relevant signals. Removing this restriction enables the application of the SCI to more use cases and thus supports the adoption of the SCI. |
There was a problem hiding this comment.
The suggested changes are following from the discussion with the working group as well as a separate discussion with Henry. We have worked together on a document (which can be shared upon request) and the currently agreed on changes have been provided here.
@Henry-WattTime please add your comments in case I missed something.
| * A discussion has arisen regarding the choice of emissions factor for **I** in the SCI equation. | ||
| * The choice of what to use for I acts as a signal. | ||
| * The specification currently insists on marginal emission factors for **I**, and there is interest in understanding that original choice and exploring other emissions factors options. | ||
|
|
There was a problem hiding this comment.
| ## Topic of Discussion | |
| The reason why this discussion was opened is to have a conversation around which emission factors should be permitted for use in the SCI. Currently, the SCI restricts the use of signals to marginal emission factors, precluding other signals. The outcome of this discussion is a decision on whether the SCI should only allow marginal emission factors or whether the SCI should also allow other signals. | |
| ## Vocabulary | ||
|
|
||
| * **LRMER**: Long-run marginal emission rates | ||
| * **SRMER**: Short-run marginal emission rates | ||
| * **AER**: Average emission rates |
There was a problem hiding this comment.
As discussed with the working group and agreed with Henry, it would be helpful to move clarifying information to the beginning of the document.
| ## Vocabulary | |
| * **LRMER**: Long-run marginal emission rates | |
| * **SRMER**: Short-run marginal emission rates | |
| * **AER**: Average emission rates | |
| ## Vocabulary and Context on Emission Factors | |
| * There are different ways of calculating the carbon intensity of electricity per kWh of electricity. We are discussing three here: | |
| * ** AER (average emission rates)**: Total emissions divided by total electricity consumption, representing the carbon intensity of the entire electricity mix currently on the grid. | |
| * ** SRMER (short-run marginal emission rates)**: The emissions per unit change in electricity consumption, where the structure of the electrical grid (e.g., the generation, transmission, and distribution assets) is considered fixed. This represents the short-term change of system level emissions for a change in load. | |
| * **LRMER (long-run marginal emission rates)**: The emissions per unit change in electricity consumption, where the influence of the change in demand on both the operation and structure of the grid is taken into account. This represents the long-term change of system level emissions for a change of load. | |
| * What average and short-run marginal indicate: | |
| * Average: The average quantifies the carbon intensity of electricity that is currently on the grid by taking into account all generators that produce electricity. The higher the share of green electricity on the grid, the lower the average carbon intensity and vice versa. The average does not provide information on which generator needs to turn on/off as a result of an action. | |
| * Short-run Marginal: The marginal carbon intensity quantifies the carbon intensity of the power plant that increases its generation as a result of an action. The higher the marginal, the dirtier the next generator. It does not provide any information of power plants currently producing electricity and thus cannot inform users whether renewables are generating electricity or not. However, in rare cases of curtailment marginal emission factors are able to indicate renewable generation on the margin. | |
| * Generally speaking, whenever one refers to marginal emission rates, they mean short-run marginal emission rates. Only recently, the distinction between short run and long run marginal has been made. | |
| * Both short-run marginal and average are available at an hourly, monthly and annual level in different regions or zones globally. | |
| * Both short-run marginal and average can be used for decision-making/load-shifting, but they each might signal a different time to consume electricity. This means that they can be used to shift load in time (e.g. different hour) and/or space (e.g. different location). | |
| * So the choice of which signal to use has a significant impact on an SCI score as well as the impact of the action taken based on the SCI. Since the SCI is an action oriented spec the choice of one over the other can impact the actions that people take. | |
| * Short run marginal is the best signal to use if people want to make decisions to optimise the short term. | |
| * Long run marginal is a better signal if you want actions people take to have an impact on a longer time horizon, for example to encourage more renewable plants to be built. It has far fewer benefits when viewed in the shorter term. According to NREL, it is impossible to calculate LRMER on an hourly basis in real-time and thus, proxies are needed. | |
| * [Average is considered to be a proxy for long run marginal](https://github.com/Green-Software-Foundation/sci/pull/353#issuecomment-1432307783), it’s easier to calculate and is correlated with long run marginal. Furthermore, AER strongly correlates with curtailment of renewables. Average emission factors perform poorly for estimating the short-term impact of interventions. |
There was a problem hiding this comment.
I think curtailment is less 'rare' than indicated and an ample opportunity for emissions reductions.
There was a problem hiding this comment.
Indeed, curtailment provides ample opportunity for emissions reductions, though even in the sun-heavy California, combined wind and solar curtailment in 2022 only amounted to around 4.3% (own calculation) of total volume. I am not sure how "rare" is defined, though on an energy volume basis, curtailment is playing a small role.
The above number is based on using official data from CAISO.
Curtailment data from here: http://www.caiso.com/informed/Pages/ManagingOversupply.aspx
Power generation data from here: http://www.caiso.com/Documents/ProductionAndCurtailmentsData_2022.xlsx
There was a problem hiding this comment.
If we could reduce electricity sector emissions in California by nearly 5% by taking advantage of already existing renewable generation, that would be a massive success. That is zero emissions power that is being wasted today. 5% is a massive amount of energy, last month alone California threw away nearly more than 600 GWh of clean energy.
| ### What are short run marginal, long run marginal and average emissions factors for electricity? | ||
|
|
||
| * These are 3 different ways of calculating the carbon intensity of electricity per kWh of electricity. | ||
| * AER: represents the carbon intensity of the electricity mix currently on the grid. It is measured as gCO2 per kWh of electricity. | ||
| * SRMER: represents the short-term change of system-level carbon emissions for a change in load. It is measured as gCO2 per each **_additional_** kWh of electricity. | ||
| * LRMER: represents the change of system-level carbon emissions for a change in load in consideration both the operational and structural implication of changes in electricity demand.It is measured as gCO2 per each **_additional_** kWh of electricity. | ||
| * They each might signal a different time to consume electricity. | ||
| * So the choice of which signal to use has a significant impact on an SCI score as well as the impact of the action taken based on the SCI.Since the SCI is an action oriented spec the choice of one over the other can impact the actions that people take. | ||
| * Short run marginal is the best signal to use if people want to make decisions to optimise the short term. | ||
| * Long run marginal is a better signal if you want actions people take to have an impact on a longer time horizon, for example to encourage more renewable plants to be built. It has far fewer benefits when viewed in the shorter term. According to NREL, it is impossible to calculate LRMER on an hourly basis in real-time and thus, proxies are needed. | ||
| * **[Average is considered to be a proxy for long run marginal](https://github.com/Green-Software-Foundation/sci/pull/353#issuecomment-1432307783)**, it’s easier to calculate and is correlated with long run marginal. Furthermore, AER strongly correlates with curtailment of renewables. Average emission factors perform poorly for estimating the short-term impact of interventions. |
There was a problem hiding this comment.
Remove this section, as it is incorporated above.
| ### What are short run marginal, long run marginal and average emissions factors for electricity? | |
| * These are 3 different ways of calculating the carbon intensity of electricity per kWh of electricity. | |
| * AER: represents the carbon intensity of the electricity mix currently on the grid. It is measured as gCO2 per kWh of electricity. | |
| * SRMER: represents the short-term change of system-level carbon emissions for a change in load. It is measured as gCO2 per each **_additional_** kWh of electricity. | |
| * LRMER: represents the change of system-level carbon emissions for a change in load in consideration both the operational and structural implication of changes in electricity demand.It is measured as gCO2 per each **_additional_** kWh of electricity. | |
| * They each might signal a different time to consume electricity. | |
| * So the choice of which signal to use has a significant impact on an SCI score as well as the impact of the action taken based on the SCI.Since the SCI is an action oriented spec the choice of one over the other can impact the actions that people take. | |
| * Short run marginal is the best signal to use if people want to make decisions to optimise the short term. | |
| * Long run marginal is a better signal if you want actions people take to have an impact on a longer time horizon, for example to encourage more renewable plants to be built. It has far fewer benefits when viewed in the shorter term. According to NREL, it is impossible to calculate LRMER on an hourly basis in real-time and thus, proxies are needed. | |
| * **[Average is considered to be a proxy for long run marginal](https://github.com/Green-Software-Foundation/sci/pull/353#issuecomment-1432307783)**, it’s easier to calculate and is correlated with long run marginal. Furthermore, AER strongly correlates with curtailment of renewables. Average emission factors perform poorly for estimating the short-term impact of interventions. |
| #### Drawbacks of the GHG Protocol when applied to software | ||
|
|
||
| * The way a cloud provider might attribute energy to a customer via the GHG protocol is to divide energy bought by a Data Center by the number of servers in the Data Center. So every server, regardless of how much it's being used, would report that same energy consumption. | ||
| * There is little effort made to try to attribute energy consumption per server never mind per actual process/application. | ||
| * Therefore if a customer made some of their processes more energy efficient, it wouldn't be reflected in the final calcs, every server is said to consume the same energy, we call that the **top down approach** and it's one of the only approaches auditors accept for verifying the reported GHG number for a cloud provider. | ||
| * The alternative way which is more useful if your intention is action vs reporting is to calculate **bottom up**, so measure the actual energy consumption of each of your processes individually and sum them all up to get your total energy consumption of your software application. That approach isn't accurate, often has to use estimated models, has double counting effects, but does give you a relative figure for energy which will go down if you make your application more energy efficient. | ||
| * The same is said from carbon awareness. The way energy consumption is typically calculated (in scope 2 and scope 3) for orgs means it's incredibly hard to prioritise any carbon aware work. Yearly averages mean that if you build in time shifting in your app it won't reduce the reported scope 2/3 figures for your org which is one of the reasons it's been so challenging to get carbon aware computing onto the agenda for most companies. | ||
| * The GHGP Scope 2 Guidance includes two accounting methods: location-based method and market-based method. | ||
| * Location-based Accounting: A location-based method reflects the average emissions intensity of grids on which energy consumption occurs. This allows organisations to report their physical account of their carbon emissions as a result of their electricity consumption. | ||
| * Market-based Accounting: | ||
| * This allows organisations to report lower _gross_ carbon emissions by purchase of energy efforts (market measures) like RECs and GOs. | ||
| * Before market based reporting organisations had to report carbon emissions from energy consumption as say 10 KT and then separately they could _choose _to neutralise those 10 KT with the purchase of renewable energy certificates (which are financial instruments you can purchase that act like offset to neutralise your energy consumption). | ||
| * After market based reporting organisations can report carbon emissions from energy consumption as 0 KT (They can report one figure which already takes into account the energy certificates) | ||
| * Market based reporting has the impact of **deprioritising actions to make software more energy efficient** and **prioritising the purchase of energy certificates**. | ||
| * With market based reporting organisations now have two ways to report a lower scope 2 emissions figure to the public.(1) Consume less energy by making your processes more efficient. (2) Consume the same energy and purchase certificates. (2) is the easier solution for an organisation to prioritise than (1) which takes significant investment and effort. | ||
| * Renewable energy purchasing is often managed by a different team than the teams that develop software, which means organisations can reduce their emissions through market based measures without engaging in any efficiency improvement activity in the software products. | ||
| * In addition if an organisation is already reporting 0 KT in terms of Scope 2 because of their purchase of energy certificates, and did more work to make their processes more energy efficient, they would still report 0 KT, a metric that stays at 0 regardless of what you do, does not encourage action.. | ||
| * Market based reporting doesn’t incentivize making applications more energy efficient. |
There was a problem hiding this comment.
The discussion seems to be around which signals should be allowed in the SCI. In the interest of a more concise and clear document, I believe it would be helpful to remove / shorten content that is not relevant to the discussion. This is a large chunk that creates a lot of noise which can be shortened, or completely removed.
| #### Drawbacks of the GHG Protocol when applied to software | |
| * The way a cloud provider might attribute energy to a customer via the GHG protocol is to divide energy bought by a Data Center by the number of servers in the Data Center. So every server, regardless of how much it's being used, would report that same energy consumption. | |
| * There is little effort made to try to attribute energy consumption per server never mind per actual process/application. | |
| * Therefore if a customer made some of their processes more energy efficient, it wouldn't be reflected in the final calcs, every server is said to consume the same energy, we call that the **top down approach** and it's one of the only approaches auditors accept for verifying the reported GHG number for a cloud provider. | |
| * The alternative way which is more useful if your intention is action vs reporting is to calculate **bottom up**, so measure the actual energy consumption of each of your processes individually and sum them all up to get your total energy consumption of your software application. That approach isn't accurate, often has to use estimated models, has double counting effects, but does give you a relative figure for energy which will go down if you make your application more energy efficient. | |
| * The same is said from carbon awareness. The way energy consumption is typically calculated (in scope 2 and scope 3) for orgs means it's incredibly hard to prioritise any carbon aware work. Yearly averages mean that if you build in time shifting in your app it won't reduce the reported scope 2/3 figures for your org which is one of the reasons it's been so challenging to get carbon aware computing onto the agenda for most companies. | |
| * The GHGP Scope 2 Guidance includes two accounting methods: location-based method and market-based method. | |
| * Location-based Accounting: A location-based method reflects the average emissions intensity of grids on which energy consumption occurs. This allows organisations to report their physical account of their carbon emissions as a result of their electricity consumption. | |
| * Market-based Accounting: | |
| * This allows organisations to report lower _gross_ carbon emissions by purchase of energy efforts (market measures) like RECs and GOs. | |
| * Before market based reporting organisations had to report carbon emissions from energy consumption as say 10 KT and then separately they could _choose _to neutralise those 10 KT with the purchase of renewable energy certificates (which are financial instruments you can purchase that act like offset to neutralise your energy consumption). | |
| * After market based reporting organisations can report carbon emissions from energy consumption as 0 KT (They can report one figure which already takes into account the energy certificates) | |
| * Market based reporting has the impact of **deprioritising actions to make software more energy efficient** and **prioritising the purchase of energy certificates**. | |
| * With market based reporting organisations now have two ways to report a lower scope 2 emissions figure to the public.(1) Consume less energy by making your processes more efficient. (2) Consume the same energy and purchase certificates. (2) is the easier solution for an organisation to prioritise than (1) which takes significant investment and effort. | |
| * Renewable energy purchasing is often managed by a different team than the teams that develop software, which means organisations can reduce their emissions through market based measures without engaging in any efficiency improvement activity in the software products. | |
| * In addition if an organisation is already reporting 0 KT in terms of Scope 2 because of their purchase of energy certificates, and did more work to make their processes more energy efficient, they would still report 0 KT, a metric that stays at 0 regardless of what you do, does not encourage action.. | |
| * Market based reporting doesn’t incentivize making applications more energy efficient. |
There was a problem hiding this comment.
I think we should keep this, one of the key aspects of why this spec exists.
There was a problem hiding this comment.
I also vote for keeping, this is a key point for a lot of people who are only familiar with carbon reporting and the GHG protocol
| * LRMER are very strong predictors of long-term impacts of interventions. AER are a strong proxy for LRMER, while SRMER does not capture long-term impacts of interventions | ||
| * [Planning for the evolution of the electric grid with a long-run marginal emission rate](https://www.cell.com/iscience/fulltext/S2589-0042(22)00185-7?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2589004222001857%3Fshowall%3Dtrue) | ||
| * [Short-run marginal emission rates omit important impacts of electric-sector interventions](https://www.pnas.org/doi/10.1073/pnas.2211624119) | ||
| * Marginal emission factors are based on estimation models and their value cannot be verified. This is because there is no “ground truth” on the true value as the specific marginal generator cannot be verified. But there is good academic research showing that marginal emissions can be calculated and are a meaningful signal. |
There was a problem hiding this comment.
I think this considerably overstates the uncertainly around marginal emissions and how to calculate them. There's further consensus in the academic literature that marginal emissions are the correct metric for measuring the impact of interventions.
There was a problem hiding this comment.
The above statements are based on the insights provided by the cited reports and discussions with US and EU TSOs. Moreover, the fact that SRMER have not been verified has also been raised in today's session by Pieter Gagnon.
Inside the original context document, right after this line there are many sources. For convenience, I have provided them here again. As a reminder, these are cited statements, i.e. these are exact comments from these reports:
- The US RTO PJM states that “marginal units – and the marginal emissions rates based on them –, cannot provide any prediction of the results of an action. "
- “All marginal emissions rate estimation methodologies are imperfect for one reason or another”
- “There is tremendous variation in the way that MEFs can be estimated. Depending on the specific methodology, data sources, and assumptions used, no one estimate is likely to match another. Even two approaches using the same methodology could use different input data or assumptions to estimate marginal emissions impact, leading to two different and sometimes inconsistent estimates. This means that there is no single source of “truth” for marginal emissions impact.”
There was a problem hiding this comment.
Here are a few published peer reviewed academic papers that discuss marginal emissions as being strongly supported science:
“We compare marginal and average emissions factors, finding that AEFs may grossly misestimate the avoided emissions resulting from an intervention… Both supply- and demand-side interventions will displace energy and emissions from conventional generators. MEFs give a consistent metric for assessing the avoided emissions resulting from such interventions.” — Stanford’s Azevedo et al. in Environmental Science & Technology (2012)
“These [marginal emissions] estimates have important implications for understanding the environmental consequences of many electricity-shifting policies… [they] are also relevant for understanding the impacts of activities and policies that increase electricity demand.” — Yale’s Kotchen et al. in the Journal of Economic Behavior & Organization (2014)
“If the load under analysis represents an incremental increase or decrease, marginal ERs can provide an emissions measurement specific to the change. Thus, for consequential LCAs of new loads, marginal factors are recommended.” — University of Michigan’s Ryan et al. in Environmental Science & Technology (2016)
“...the use of AEFs, which reflect grid-average situations, to estimate the effect of an intervention may be problematic, because not all generating technologies would respond to changes in demand proportionally… AEFs could significantly misestimate the amount of emissions avoided by an intervention. By contrast, MEFs estimate the emission intensity of marginal power generation that responds to a change in demand, and are a more appropriate metric to assess emission implications of policy and technology interventions.” — University of Minnesota’s Smith et al. in Environmental Science & Technology (2017)
“Increasing (or decreasing) demand at a given time affects the operation of only the marginal units in the dispatch order, and not the infra-marginal units. Therefore, emission implications of energy storage, or any other incremental demand- or supply-side resource that leads to load-shifting, depend on marginal operating emission rates, based on the emission intensity of the marginal units, and not the average operating emission rates, based on the average emission intensity of the entire grid.” — Columbia’s Shrader et al. in SSRN: Social Science Research Network (2020)
“Marginal Emission Rate provides a mathematically sound and transparent way to quantify the carbon footprint of electricity consumption and production. Mathematically, the MER calculation is similar to the calculation of Locational Marginal Prices (LMP) and system lambda, which are used for economic dispatch of power systems across North America.” — Tabors Caramanis Rudkevich’s Hua He et al. in The Electricity Journal (2021)
|
Comments from me:
|
| **Ciril Wakounig**: Overall, out of the three emission factors, average is very trivial to calculate, as it is a weighted average of generation and related emission factors. Most policy/regulatory/governmental communication and regulation is based on average emission factors, without a general standard (I guess because it is implied). | ||
|
|
||
| For marginal (short- and long-term) it is very difficult to develop standards, since they are both based on assumptions and statistical models that approximate reality. Contrary to the average, it is not possible to compare marginals to what happened in reality and thus it is quite difficult (I assume not impossible) to develop a standard. For instance, the average can easily be verified based on the hourly power mix that is published by TSOs. | ||
|
|
There was a problem hiding this comment.
I added a short summary section with the two points of view which emerged during the last two weeks of discussions on the topic.
| ## Summary | |
| The foundation's members are debating which signal is best to use when calculating carbon intensity. Two main perspectives are being taken into account. The first point of view encourages the use of two signals, one for reporting (e.g., AER) and the other for real-time operational decision-making (e.g., SMER or energy fraction). This approach has the advantage of simplicity and comparability between different projects. The second point of view advises leaving the freedom to employ various signals in accordance with the particular use cases and considering the advantages and disadvantages of each choice. These conversations demonstrate a careful examination of the intricacies involved in determining carbon intensity and emphasize the significance of selecting the most appropriate strategy for each particular circumstance. |
|
'i' shall be the carbon intensity of the grid that evaluates long and short term change in emissions. 'i' shall either be the short run marginal, long run marginal, or average emissions grid intensity. 'i' shall be the carbon intensity of the grid. Leave it as it is (i shall be the marginal grid emissions intensity) Abstain |
|
Straw poll results |
|
Action item for @ciril-emaps and @Henry-WattTime to integrate the change for option 2 |
This resolves the discussion in the working group and issue #357 and clarifies the definition of I. Coauthored by @Henry-WattTime and https://github.com/ciril-emaps This supersedes #352 and #353 and #337. Signed-off-by: Henry-WattTime <[email protected]>
This resolves the discussion in the working group and issue #357 and clarifies the definition of I. Coauthored by @Henry-WattTime and https://github.com/ciril-emaps This supersedes #352 and #353 and #337. Signed-off-by: Henry-WattTime <[email protected]>
|
WG approved pull request #358 which addresses this. |
Created a context document to provide a central place to put background, context, motivations, research, lore - all the contextual information which is needed to properly discuss a topic.
This is not intended to form part of the spec.