Decision Proposal 208 - Binding NFRs

[CDR-API-Stream]

This decision proposal contains the recommendation to make the existing CDR non-functional requirements binding. Feedback is request on the proposed approach, along with any feedback on the suitability of the non-functional requirements as published.

The proposal for consultation is attached below:
Decision Proposal 208 - Binding NFRs.pdf

Feedback is now open for this proposal. Feedback is planned to be closed on Friday 10th September 2021.

[biza-io]

Biza.io intends to provide feedback on this proposal. One of the primary reasons that feedback or suggestions for change has not been forthcoming to date has been the lack of published information with respect to actual utilisation with which to determine the efficacy of the proposed NFRs, suitability relative to build cost, and scale-up scope requirements to hit peak requirements.

Annecdotal evidence (https://frollo.com.au/blog/whats-new-in-the-consumer-data-right-may-2021-update/) for utilisation currently available states:

With over 8 Million Open Banking API calls, we’re responsible for over 95% of Open Banking activity to date.

Assuming this was the case and compressing this even to a 180 day (~6 month) period the average transactions per second of the entire ecosystem is ~0.5 TPS. On this basis the statement within the decision proposal of "Existing implementations should already be built to accommodate these non-functional requirements" would represent the potential for a 100x overbuild of actual demand (with reference to peak recipient use of 50 TPS).

With this in mind, we request the DSB and/or the ACCC provide:

• Aggregated details from (at least) the last 2 reporting periods for the CDR notably the aggregated values for the CDR reporting form item 10. Number of consumer data requests received from accredited persons on behalf of eligible CDR consumers so that responses can be considered in context of actual demand
• Observed access behaviours of recipient applications in as much detail as possible, preferably per endpoint. Biza.io, with the permission of its customers, is willing to assist in providing input here while noting we are a minority of utilisation which is mostly related to the Big 4
• Clarification of the following sentence within the DP "If this recommendation is accepted, then it would be effective immediately once the decision is made.". Historically obligations have been future dated and with a vast number of holders not yet active Biza.io is wary that mandating these NFRs with immediate effect (rather than an FDO with sufficient notice) may delay further the delivery of any data from non-compliant holders at this stage

[RobHale-RAB]

Although it is well over a year since CDR was launched, throughout most of this time we have had just 5 DHs and 2 ADRs actively participating. With limited consumer awareness and published use cases, the volumes of data have been low and, until https://mycdrdata.regionalaustraliabank.com.au/ was launched in May this year, many endpoints (such as payees, direct debits, scheduled payments, and customer details) had not been used at all.

It might therefore be premature to lock in NFRs if the rationale for this move is that the ecosystem is now stable and current performance is representative of future performance.

The points made by @biza-io in their response above are valid. It will be important to ultimately enforce NFRs, but if this is done now, new DHs may place too much emphasis on performance, potentially to the detriment of data quality or accuracy (which are not assessed through NFRs), and delaying DH participation still further.

It is also important when assessing API volumes to date, that we recognise that Frollo and Regional Australia Bank have very different ADR use cases, with very different profiles and impacts on DH endpoint ingestion.

• Both of Regional Australia Bank's use cases adopt one time consent - a single consent authentication and authorisation resulting in a single point in time collection request made of a DH. There is no subsequent or ongoing collection of data.
• Frollo's PFM use case uses time-bound ongoing consent of up to 12 months with each DH. Once consumer consent is given, calls are made to DHs many times per day, potentially for multiple accounts for multiple banks. This has resulted in millions of API calls for a subset of endpoints.
• When looking at performance metrics, it could be argued that the initial data requests made immediately upon consent are most time-critical, because the customer is 'present' at this time - applying for a loan, downloading their CDR data or wanting to analyse their transaction history for budgeting purposes.
• Thereafter, with ongoing consent, endpoint requests are largely done 'behind the scenes' and perhaps not as time critical - although again, this may vary by use case once we have more participation and data.
• Performance of individual endpoints varies significantly, even with the same DH. Banking systems weren't designed or optimised to provide this data. Some endpoints will take longer than others. This information is only visible to ADRs at present.
• Regional Australia Bank has noticed that for many of the current DH brands, response times for calls to Get Account Details are slower than those for Get Transactions For Account. This might be contrary to NFR expectations given the payload for transactions would typically be much greater.
• Many banking systems are optimised to retrieve transaction data very quickly as it is required for current digital banking services. Optimising for CDR will take time.
• While Regional Australia Bank does provide information on endpoint performance to the ACCC, Treasury and the DSB, it is not a formal requirement.

Ultimately, if we want to encourage participation and ensure as many DHs as possible are publishing high quality consumer data to allow ADR use cases to be reliably and effectively developed, NFRs will be necessary. The low participation of DHs to date, despite it being a legislative obligation, indicates many DHs are struggling to get the basics right just now.

On balance, it seems that for the time being, it would be more beneficial to place emphasis on participation, data quality and accuracy than on high endpoint performance

[CDR-API-Stream]

In response to the comments from @biza-io

With this in mind, we request the DSB and/or the ACCC provide:

• Aggregated details from (at least) the last 2 reporting periods for the CDR notably the aggregated values for the CDR reporting form item 10. Number of consumer data requests received from accredited persons on behalf of eligible CDR consumers so that responses can be considered in context of actual demand
• Observed access behaviours of recipient applications in as much detail as possible, preferably per endpoint. Biza.io, with the permission of its customers, is willing to assist in providing input here while noting we are a minority of utilisation which is mostly related to the Big 4

The DSB is, unfortunately, unable to provide this data. We will refer the request to the ACCC. In the meantime, and anticipating that this not be held or may not be publishable, we are asking for feedback from participants based on their own experience as implementors.

• Clarification of the following sentence within the DP "If this recommendation is accepted, then it would be effective immediately once the decision is made.". Historically obligations have been future dated and with a vast number of holders not yet active Biza.io is wary that mandating these NFRs with immediate effect (rather than an FDO with sufficient notice) may delay further the delivery of any data from non-compliant holders at this stage

The proposal is not seeking to change the NFRs, only to change the status of the existing NFRs to binding. This is a technical change only and will have no implementation impacts and there will be no need for a technical transition. This is why it is proposed that the status change would take immediate effect.

[spikejump]

Both Xero and Intuit had given feedback on NFR previously here. We request the review of comments there.
To make existing published NFR binding would make the ecosystem challenging for ADRs who need to refresh data for a large number of consumers on a daily basis.

[DimitriSty]

As per Intuits comments above, please refer to the previously posted feedback on NFRs. The current state of the NFRs do not support he Cloud Accounting use case.

[EnergyAustraliaBA]

Thank you for the opportunity to provide feedback on the binding NFRs. Having reviewed them we have the following clarifications.

1. Since the Banking sector had about 12 months of lead time on implementing CDRs along with NFRs, will the energy sector have a similar lead time to refine the NFRs during maintenance iterations.
2. How recent should the CDR data such as (Customer Data, Billing Data, Tailored Tariff, NMI Standing Data, Metering Data and DER with ADR) be per data set? - can some of the data be offline (hourly, 24 hours) accessed or should all they be real-time? Can this be met overtime progressively rather then real-time from day 1, if so what time frames are we looking at?
3. Correctness at point in time - Some customer data may need to go through internal processes before they become accurate. How will this be reflected from Data quality perspective? What solution or options will be there for DHs to handle this situation.
4. AEMO performance requirements - since tiered response includes the e2e response time, how are AEMO bound by these responses as the response on overall DH will be dependent on performance from AEMO.
5. Clear definition of distribution transaction start and end time. i.e. does the starting time of the transaction time from ADR or when the consumer actually start interaction.
6. Inactive customers with EA - historical data for how long to share with ADR

[AusBanking2]

The ABA thanks the DSB for the opportunity to comment on this decision proposal relating to NFRs.
The full ABA position is attached.

Uploading 210910 - ABA submission DSB CDR DP 208 - NFR.pdf…

[PratibhaOrigin]

Thanks for the opportunity to provide feedback on this topic.
We have reviewed the paper for DP 208 and the original paper DP 21. Few queries and comments for clarification below –

1. What is the requirement around data status, i.e., how current the data needs to be? –if there are processes in progress, do we need to consider the in-flight processes during data sharing.
   For example – If there is a credit card update in progress or a meter change or transfer out in progress, do we need to consider these facts while sharing the data.
   In terms of data latency, few data sets are not real time available on the digital platforms and may take 24-48 hours like re-bill scenarios.

2. Regarding the NFR requirements for bulk data and big payloads - we have raised some concerns considering the 5ms data or C&I data , the payload will be enormous and the NFR threshold seems bit ambitious for that. [https://github.com/Decision Proposal 193 - Energy Non-functional Requirements #193]

3. As the paper suggests - “the DSB has received limited feedback on the non-functional requirements themselves indicating that change is required.”, Origin would request DSB and Treasury to share the data if available regarding the NFRs –

• What % of CDR requests were able to comply and be completed within the current NFR requirements?
• What % of CDR requests were NOT able to comply and be completed within the current NFR requirements?
• If the above requested data is not allowed to share, will the DSB be able to share the “limited feedback” provided to them regarding NFRs and if it has been considered in the NFRs?

4. Making the current NFRs binding – We request Treasury and DSB to consider at least a 12 month grace period for energy participants entering the CDR eco-system fresh. Without real world implementation experience for energy sector, we should be using similar justification that the non-functional requirements could not be fully confirmed as being appropriate by the Energy CDR participants. Having the thresholds there will help us build to those NFR requirements but making them binding may risk shifting focus from data quality requirements to time thresholds.

[johnAEMO]

Thank you for the opportunity to respond to the above Decision Proposal.

AEMO supports the comments made by Energy Australia, Origin and others above and concur that it is too early to make the existing NFRs binding.

Historically the intent was that these “be made non-binding until a broader implementation of both Data Holders and Accredited Data Recipients was in place. Without real world implementation experience it was perceived that the non-functional requirements could not be fully confirmed as being appropriate by the CDR community.”

Banking has had 12 months of experience and still not reached a significant customer base to confirm that these NFRs are achievable under load. Energy has yet to go live and has no indication that these NFRs are achievable. While at this stage in Energy the NFRs remain aspirational, AEMO supports them as valid targets for development of APIs.

[commbankoss]

CBA appreciates the opportunity to provide feedback on this issue and is supportive of the ABA’s submission. In line with previous discussions on the NFRs (see Decision Proposal 21 - Non-Functional Requirements #21 ), CBA is of the view that the NFRs should be tested with a sufficient number of participants in the ecosystem and a review of the NFRs carried out prior to becoming binding. CBA agrees with the ABA that the NFRs should not become binding until after ‘read access’ has been fully implemented by all data holders. CBA also supports introducing a transition period and recommends that implementation of mandatory NFRs is phased over 12 months.

[l-sateesh]

Thank you for the opportunity to provide feedback.

Although the NFRs have been published for some time, implementations for the majority of Data Holders are still in the process of coming on-line. With a large number of Data Holders experiencing delays in their implementations, their focus is more likely to be on being functionally compliant. To enable meaningful participation in this consultation, these organisations should be given sufficient time to understand the characteristics of production load and consider appropriate performance enhancements.

With the November 2021 compliance timeframes there will be an increase in the number of active data holders in the ecosystem. We suggest moving this consultation timeframe to March/April 2022 to give sufficient time for new entrants in the ecosystem to optimize their solution and provide feedback on the non-functional requirements

Functional compliance for tier 2 and 3 data holders was, by convention, required 12 months after tier 1 compliance. This should also be considered as an approach for non-functional compliance.

We also note that further guidance is required for platforms that support multiple data holders. For eg: is it expected that all tenants of a provider must be able to meet the target simultaneously, or one tenant at a time, or is there an assumption of baseload per other tenant, whilst the target tenant is measured?

[biza-io]

Please find attached our response to Decision Proposal 208: Binding NFRs:
DP208 - Binding NFRs.pdf

While the reasons for recommendations are outlined within the document we restate them here as follows:

1. Adopt the following proposed areas of non-functional requirements immediately:
   a. Session Requirements
   b. Availability Requirements
   c. Performance Requirements
   d. Reporting Requirements
   e. Data Latency
   f. Data Quality

2. Consider adding a “lower bound” component to the Traffic Thresholds that is commensurate with ecosystem load. While there are various methods of approaching this as a starting point Biza.io proposes Average TPS per Arrangement with an upper bound at the traffic thresholds. By way of example if the Average TPS per Arrangement was set at 0.1 TPS and an organisation expected the total number of concurrent arrangements being accessed at 100 the testable threshold could be set to 10 TPS thereby allowing pre-Production verification with a realistic number informed by, for instance, internet banking utilisation.

3. As an alternative to (2) consider explicit implementation guidance that reflects the current Live CDR Ecosystem load so that Holders can make informed decisions with regards to capacity and scale in the context of the ecosystem.

4. Stage the introduction of NFRs by making the NFRs binding for Majors first and allow for a 12 month delay for Non-Majors. This would allow Non-Majors to continue onboarding their infrastructure with a focus on quality while being able to consider the architectural choices associated with the delivering the required level of service with real world utilisation statistics for their institution to go by.

[NationalAustraliaBank]

NAB believes in creating and supporting performant Open Banking solutions, essential to building trust, adoption and ultimately customer value provided through the CDR.
Noting the ABA’s response, we would like to provide the following additional feedback:

• NAB considers that all changes to standards should include a notice or transition period. We believe a transition period is important when changing status from voluntary to mandatory even where there are no changes anticipated. This is to give participants sufficient time to undertake testing and refinement of their solutions, especially those data holders who are anticipating a slower performance when more ADRs join the ecosystem.

• We suggest a notice period of at least 6 months be provided before the NFR’s become binding.

• Regarding the Dynamic Client Registration API, the response time currently set in the Data Standards is 1000ms. NAB would like to confirm whether this API is included in the binding NFRs and if yes, how external dependencies outside of the Data Holders sphere of control (eg. time spent calling Registry API) are factored in the overall response time of this API?

• NAB would like further information around the enforcement approach and enforcement consequences relating to the NFRs when they become binding. In particular, we request consideration be given to the variable nature of a Data Holder’s performance against the NFR standards eg. unplanned incident which can adversely impact NFRs for a short period of time.

[gurchan-AGL]

AGL appreciates the opportunity to provide feedback on this decision proposal.

1. Staged Introduction.
   AGL strongly recommend a staged introduction of NFRs to reduce the upfront capital burden of over provisioning of resources such as non-elastic infrastructure and/or software whilst adoption is still in its infancy. An appropriate just in time, right-sizing and data driven approach will help new sectors such as energy better understand and more effectively respond to the required service obligation.

2. Non-binding period.
   Ideally for the Energy sector, AGL recommends a 12-month non-binding grace period. In addition to the benefits described above, the 12-month non-binding grace period provides broad coverage of actual peak volumes typically associated to heighten consumer engagement such as seasonal movers and regulatory pricing events. The captured insights can better inform and test the ability of the data holder of meeting its NFR obligation.

3. Clarity of enforcement approach and consequences.
   AGL would like to understand the enforcement approach and associated consequences relating to the NFRs when they become binding.

[CDR-API-Stream]

Thanks for all of the feedback. There has been quite a bit provided and it will take a day or two to review and process.

This consultation will now be closed.

[CDR-API-Stream]

Please find attached the final decision of the Chair:
Decision 208 - Binding NFRs.pdf