Secondary Data Holder Planned Outages and Status

[CDR-API-Stream]

Description

ADRs will need to be aware of outages and status of the AEMO platform that supports multiple data sets in the Energy sector as a Shared Responsibility Data Holder. A solution on how status and outages should be communicated needs to be determined

Area Affected

The Common APIs section, specifically 'Get Status' and 'Get Outages' with specific reference to Shared Responsibility data clusters

Change Proposed

There are multiple options. Two of these are outlined below but there may be others.

Option 1: AEMO publishes 'Get Status' and 'Get Outages' publicly and ADRs can access these to determine if there is a blanket outage for the Shared Responsibility data clusters.

Option 2: AEMO publishes status and planned outages via some mechanism and every Retailer would then surface this information in their own 'Get Status' and 'Get Outages' APIs

DSB Proposed Solution

The current DSB proposal for this issue is in this comment

[OpalRussAEMO]

For background information, Option 2 is currently supported by AEMO through the below.

• Service status portal: https://aemo.service-now.com/participantportal?id=pp_services_status
• Detailed notice of planned updates and outages are published here: https://aemo.com.au/en/market-notices
• Ability to subscribe to receive market notices via email when published: https://aemo.com.au/en/subscribe

[perlboy]

As discussed in the call I'm supportive of an option involving a Holder obligation to republish AEMO outages/status merged with their own outages/status using an AEMO supplied and CDR formatted Outages/Status API. Making the AEMO endpoint public so others can access it directly sounds good too.

Utilising existing email/website based notices and effectively screen-scraping as per AEMO suggestion above will be non-functional and is strongly opposed. I could list a long list of reasons why this is broken but it's probably best not to insult peoples intelligence.

[OpalRussAEMO]

Utilising existing email/website based notices and effectively screen-scraping as per AEMO suggestion above will be non-functional and is strongly opposed. I could list a long list of reasons why this is broken but it's probably best not to insult peoples intelligence.

Just to clarify, I am in no way suggesting screen-scraping. Just explaining the current process for notifying details of planned outages and that market participants currently use these notices to plan for market system outages.

[perlboy]

Just to clarify, I am in no way suggesting screen-scraping. Just explaining the current process for notifying details of planned outages and that market participants currently use these notices to plan for market system outages.

Well the wording suggested differently: "Option 2 is currently supported by AEMO through the below". The current method is incompatible for CDR and would violate a number of CDR principles including Principle 1, 2, 3, 6 & 7.

[CDR-API-Stream]

After further discussion and analysis, the DSB would like to note the following to help facilitate decision:

• There is preference from community to have publicly hosted AEMO endpoints for status and outage but to also allow DH’s to leverage and expose AEMO status/outage via their endpoints
• A point to consider is how would an ADR know if an outage is due to issues due to DH or SDH (i.e. AEMO)? An option could be a specific error code that enables a DH to notify an ADR about SDH payload not being available due to SDH outage
• AEMO has delivery constraints preventing from delivery of a new solution by 15th November of this year. Taking this into account, the DSB would apply a future dated obligation (FDO) for the implementation of the agreed solution

[CDR-API-Stream]

As per the action captured during MI 10 meeting on 30th March, below is the recommended position for this change for further comments:

• AEMO to host publicly available 'Get Status' and 'Get Outages' endpoints for secondary data
• Retailers must consume and publish AEMO outage and status information via their own 'Get Status' and 'Get Outages' endpoints
• Both of the above will be at a future dated obligation which is yet to be determined

If there are no objections the change will be recommended to the DSB Chair.

[joe-aer]

As the AER will be sharing generic PRD on behalf of those retailers that choose not to self-host, it follows that ADRs would also need to know the outages and status for the AER-hosted PRD endpoints for those retailers.

The EME team would therefore anticipate that those retailers would need to perform a three-way merge when producing responses from their Get Status and Get Outages endpoints; merging data from their own system, data from AEMO, and data from the AER.

Note that the AER's Get Status and Get Outages endpoints will already be hosted publicly as per the standards.

[perlboy]

As the AER will be sharing generic PRD on behalf of those retailers that choose not to self-host, it follows that ADRs would also need to know the outages and status for the AER-hosted PRD endpoints for those retailers.

This seems related to feedback given to the Noting paper: ConsumerDataStandardsAustralia/standards#248 (comment)

The EME team would therefore anticipate that those retailers would need to perform a three-way merge when producing responses from their Get Status and Get Outages endpoints; merging data from their own system, data from AEMO, and data from the AER.

I get the suggestion here but AER isn't actually a declared Secondary Data Holder so I'm not sure of how the handling of this situation is best managed. Architecturally for every leg added there's latency and potential error states that could affect the suitability of the Holders outages/status delivery (ie. if it takes too long to get this information then it isn't really a solution).

Note that the AER's Get Status and Get Outages endpoints will already be hosted publicly as per the standards.

As will AEMO's. I'm wondering if, rather than a 3-way live merge situation, the DSB might be better to declare some sort of "maximum delay" timeline (within 3-5 mins seems fine). That would allow implementers to batch download from AEMO/AER and synchronise to their internal data sources thereby ensuring that their endpoints are responsive rather than live calling multiple APIs at the time of the request.

[CDR-API-Stream]

Thank you for your comments @joe-aer and @perlboy.

In summary, there are two parts to this CR:

1. AEMO must host publicly available 'Get Status' and 'Get Outages' endpoints for secondary data. There is general agreement on this position which should occur at a yet to be determined FDO.
2. Retailers should/must expose AEMO (and potentially AER) outages and status by some mechanism. The inclusion of AER status and outage endpoints and the latency concerns brought up by @perlboy warrant further discussion.

Given this changes in this CR are not specifically targeting November energy deadline, the DSB recommends carrying it over to MI11 as more consultation would allow for a better result, specifically on how to address point 2 above.

[CDR-API-Stream]

Having discussed this further internally, the DSB recommends the following:

1. As agreed in this consultation, AEMO should publicly host Get Status and Get Outages endpoints for secondary data. We will work with AEMO and publish a date by which they can provide the endpoints but it will not be an FDO as the obligation on which end points a data holder should implement is not normally driven from the standards but from the rules.
2. The DSB recommends not mandating the retailers to expose the AEMO status and outage endpoints. The rationale for this is primarily driven from the feedback on issue Energy error codes for issues in data received by DH from SDH #506 (error propagation) which is also valid here. Passing on the outage details from AEMO makes it look like the Primary Data Holder is accountable for those outages. Data Holders can still do this if they wish but it will not be mandated.

If the above is agreed upon, no changes to the standards would be required.

[benkolera]

Yes, I think that passing the statuses through is a good idea to let the ADRs know, but there needs to be careful consideration of this.

I know that Dr G refuses to call an ADR for any call even if they have only a partial outage which has caused some issues for us in the past, and adding new means of communicating partial outages (i.e if aemo is down completely that would only be a partial outage for the data holder) without careful thought about how ADRs should treat partial outages (or adding in extra information to the partial outages to say which endpoints are down or flakey, perhaps) might exacerbate that issue.

I hate to comment on here with just more problems, but yeah I agree that we want to do this but should be careful with introducing it!