Expanding the data format structure of region to allow specifying three regions

[MohElabbas]

Updated the README.md of region to describe the expansion we need to implement to the region definition so that we can integrate the output of InfraCost.
Please check the proposed expansion of the data format structure of region and advise on how to implement it.
With due thanks.

[sandrinecharousset]

Dear @MohElabbas could you please include some more details, and some examples with figures so that I can understand what you really need. i am sorry but with only this description it is not clear to me what you want to do

[MohElabbas]

@sandrinecharousset, Let me first explain the context and then let's see if the description is clear or not. Let's say the interconnection between France and Spain has a flow of 500 MW. InfraCost will determine the agents responsible for this flow and how much each agent is contributing. For example, let's say the generators in Paris are responsible for 400 MW on this line and the generators in Berlin are responsible for 100 MW on this line. Then, the results will include:

| model | scenario | region | variable | unit | subannual | 2015 |
| InfraCost | Scenario Name | Paris:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|
|InfraCost | Scenario Name | Berlin:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|

Based on this flow contribution of each region to the different interconnections, the model will also give the cost contribution of this region to the different interconnections.
I hope with this the context is clear. Let me know otherwise.

[MohElabbas]

I have added four new variables to represent the output of InfraCost. These four variables will require specifying three regions, according to the proposed suggestion above or any other way. Please check the consistency of the variables' names and descriptions.

[sandrinecharousset]

@sandrinecharousset, Let me first explain the context and then let's see if the description is clear or not. Let's say the interconnection between France and Spain has a flow of 500 MW. InfraCost will determine the agents responsible for this flow and how much each agent is contributing. For example, let's say the generators in Paris are responsible for 400 MW on this line and the generators in Berlin are responsible for 100 MW on this line. Then, the results will include:

| model | scenario | region | variable | unit | subannual | 2015 | | InfraCost | Scenario Name | Paris:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400| |InfraCost | Scenario Name | Berlin:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|

Based on this flow contribution of each region to the different interconnections, the model will also give the cost contribution of this region to the different interconnections. I hope with this the context is clear. Let me know otherwise.

Many thanks @MohElabbas this is clear now :-)

[sandrinecharousset]

Is it really possible to identify a specific generator is contributing? I understand from your explanation that this is the share of power flow in the interconnection which is coming from a specific region (maybe produced in yet another one, bought and then sold through this interconnector?) I would remove the word Generation and keep only Active Power flox contribution. (if I have understood well your explanation), and write in the description sth like 'share of the flow in the interconnection region1>region2 which is sent from region3

[sandrinecharousset]

is this the share of the flow in interconnector region1>region2 which is caused by power demand in region3?

[sandrinecharousset]

Who is really going to pay for this? I wouldn't think it is the generator, no? More the entity who is using the interco to send electricity from region 3 to somewhere via region1>region2 interco?

[sandrinecharousset]

the readme should be updated with the explanations in the comments.

[sandrinecharousset]

@sandrinecharousset, Let me first explain the context and then let's see if the description is clear or not. Let's say the interconnection between France and Spain has a flow of 500 MW. InfraCost will determine the agents responsible for this flow and how much each agent is contributing. For example, let's say the generators in Paris are responsible for 400 MW on this line and the generators in Berlin are responsible for 100 MW on this line. Then, the results will include:
| model | scenario | region | variable | unit | subannual | 2015 | | InfraCost | Scenario Name | Paris:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400| |InfraCost | Scenario Name | Berlin:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|
Based on this flow contribution of each region to the different interconnections, the model will also give the cost contribution of this region to the different interconnections. I hope with this the context is clear. Let me know otherwise.

Many thanks @MohElabbas this is clear now :-)

After looking at the variables I am not sure ... I understand that what you want to report is who are the responsible for the flow in an interco (and thus ask them to pay for it). The 'region' field is then region3:region1>region2 ; second part is clear, it is describing the interco frm region1 to region2. Region3 is not clear from the different variables and comments. Is it the region where the energy is produced?
Then about the second variable which is about the capital cost, do you need to affect this share of cost to a region? (which could include different companies..... ); Moreover for me the responsible of the flow is not always the generator but the guy who sells the energy and then for this needs using the interco.
Maybe this would be more efficient with a specific meeting as it is quite a major evolution in the nomenclature, so needs to take time and think of it....

[MohElabbas]

@sandrinecharousset, Let me first explain the context and then let's see if the description is clear or not. Let's say the interconnection between France and Spain has a flow of 500 MW. InfraCost will determine the agents responsible for this flow and how much each agent is contributing. For example, let's say the generators in Paris are responsible for 400 MW on this line and the generators in Berlin are responsible for 100 MW on this line. Then, the results will include:
| model | scenario | region | variable | unit | subannual | 2015 | | InfraCost | Scenario Name | Paris:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400| |InfraCost | Scenario Name | Berlin:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|
Based on this flow contribution of each region to the different interconnections, the model will also give the cost contribution of this region to the different interconnections. I hope with this the context is clear. Let me know otherwise.

Many thanks @MohElabbas this is clear now :-)

After looking at the variables I am not sure ... I understand that what you want to report is who are the responsible for the flow in an interco (and thus ask them to pay for it). The 'region' field is then region3:region1>region2 ; second part is clear, it is describing the interco frm region1 to region2. Region3 is not clear from the different variables and comments. Is it the region where the energy is produced? Then about the second variable which is about the capital cost, do you need to affect this share of cost to a region? (which could include different companies..... ); Moreover for me the responsible of the flow is not always the generator but the guy who sells the energy and then for this needs using the interco. Maybe this would be more efficient with a specific meeting as it is quite a major evolution in the nomenclature, so needs to take time and think of it....

@sandrinecharousset Let me explain more about what the model does so that you can make sense of the variables (please also refer to the model description in the model catalogue if you want more details and see a simple example of the model application). The model traces all the physical flows in the network from where they start, i.e., their source/generation, to where they end, i.e., their sink/demand. Therefore, the flow on a line (not necessarily an interconnection but any line) will be attributed to both demand and generation. Now, I used the word "region" because my understanding is that this is the terminology for describing the spatial dimension, however, the model, in fact, works at the nodal or agent level, and then you can do whatever aggregation you want (region-wise, zone-wise, country-wise, TSO-wise...etc) but you still need to indicate whether this aggregation is an aggregation of demand nodes/agents or generation nodes/agents. That's why the variables (be it flow or cost) need to specify whether it is a demand or generation.
Note that here we are dealing with the physical dispatched flow (or simulated ones of a future scenario), so we don't talk about who sells or buys energy because this is irrelevant to the physical flow, and this is the main idea of the model, that cost should be allocated according to benefits, and here the electrical usage is used as a proxy of benefit. Keep in mind again that when I say cost allocated to a region, I am only referring to the spatial granularity used to represent the electrical network, i.e., whether the lines are connecting regions per se or nodes within the regions. The spatial granularity of the network doesn't really matter for the model as it will track the flow and allocate the cost according to it, but the higher the granularity, the better, so that you can track flow and allocate cost at the agent level.
I hope this clarifies things, but happy to have a quick meeting any time during the week.

[sandrinecharousset]

@sandrinecharousset, Let me first explain the context and then let's see if the description is clear or not. Let's say the interconnection between France and Spain has a flow of 500 MW. InfraCost will determine the agents responsible for this flow and how much each agent is contributing. For example, let's say the generators in Paris are responsible for 400 MW on this line and the generators in Berlin are responsible for 100 MW on this line. Then, the results will include:
| model | scenario | region | variable | unit | subannual | 2015 | | InfraCost | Scenario Name | Paris:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400| |InfraCost | Scenario Name | Berlin:France>Spain | Network|Electricity|Active Power Flow Generation Contribution | MW | Year | 400|
Based on this flow contribution of each region to the different interconnections, the model will also give the cost contribution of this region to the different interconnections. I hope with this the context is clear. Let me know otherwise.

Many thanks @MohElabbas this is clear now :-)

After looking at the variables I am not sure ... I understand that what you want to report is who are the responsible for the flow in an interco (and thus ask them to pay for it). The 'region' field is then region3:region1>region2 ; second part is clear, it is describing the interco frm region1 to region2. Region3 is not clear from the different variables and comments. Is it the region where the energy is produced? Then about the second variable which is about the capital cost, do you need to affect this share of cost to a region? (which could include different companies..... ); Moreover for me the responsible of the flow is not always the generator but the guy who sells the energy and then for this needs using the interco. Maybe this would be more efficient with a specific meeting as it is quite a major evolution in the nomenclature, so needs to take time and think of it....

@sandrinecharousset Let me explain more about what the model does so that you can make sense of the variables (please also refer to the model description in the model catalogue if you want more details and see a simple example of the model application). The model traces all the physical flows in the network from where they start, i.e., their source/generation, to where they end, i.e., their sink/demand. Therefore, the flow on a line (not necessarily an interconnection but any line) will be attributed to both demand and generation. Now, I used the word "region" because my understanding is that this is the terminology for describing the spatial dimension, however, the model, in fact, works at the nodal or agent level, and then you can do whatever aggregation you want (region-wise, zone-wise, country-wise, TSO-wise...etc) but you still need to indicate whether this aggregation is an aggregation of demand nodes/agents or generation nodes/agents. That's why the variables (be it flow or cost) need to specify whether it is a demand or generation. Note that here we are dealing with the physical dispatched flow (or simulated ones of a future scenario), so we don't talk about who sells or buys energy because this is irrelevant to the physical flow, and this is the main idea of the model, that cost should be allocated according to benefits, and here the electrical usage is used as a proxy of benefit. Keep in mind again that when I say cost allocated to a region, I am only referring to the spatial granularity used to represent the electrical network, i.e., whether the lines are connecting regions per se or nodes within the regions. The spatial granularity of the network doesn't really matter for the model as it will track the flow and allocate the cost according to it, but the higher the granularity, the better, so that you can track flow and allocate cost at the agent level. I hope this clarifies things, but happy to have a quick meeting any time during the week.

Dear @MohElabbas now it is completely clear
But then there may be an inconsistency between generator/region and same for the demand side and perhaps expanding the region name this way is not the perfect solution
We should discuss that also with @phackstock
Best

[danielhuppmann]

I suggest to change the variable-name to better reflect the hierarchical nature of the variables, and to use an already-existing region-name in the example.

[MohElabbas]

I have updated the PR according to the received comments. Please let me know if you see anything else that needs changing. Many thanks.

[danielhuppmann]

Thanks, this looks good to me from a variable-naming structure and we can add the three-part-region modifications for your use case.

@sandrinecharousset, are you satisfied with the documentation and description of variables?

[MohElabbas]

@danielhuppmann I have made some slight modifications to the variable structure after discussing with Luis. Could you please re-check them? Thank you.

[sandrinecharousset]

Thank you Mohamed for all the explanations and interesting discussion
Now it is all clear to me

[danielhuppmann]

Thanks @MohElabbas

[erikfilias]

Suggested change

	To represent data that refers to a flow or cost contribution of a generator
	To represent data that refers to a flow or cost allocated to a generator

[erikfilias]

If I understood well, there is an allocation of responsibilities in the flow of a particular transmission line here. Then, the word cost allocated is more precise.