Decision Proposal 119 - Enhanced Error Handling Payload Conventions

[CDR-API-Stream]

This issue is for consultation on Error payload structure and payload conventions to cater for Enhanced Error Handling.

Specifically, this decision proposal is seeking to address the taxonomy and structural elements for enhanced error handling currently represented in the ResponseErrorList.

The consultation draft Decision Proposal has been attached below:
Decision Proposal 119 - Enhanced Error Handling - Data Structure.pdf

Feedback is currently closed for this proposal. Feedback is being considered in an updated decision proposal.

---

As part of the Enhanced Error Handling consultation a Working Group workshop series has now been set up. To register for the first workshop, please register your interest here: https://dsb_enhancederrors_workshop01.eventbrite.com.au

[CDR-API-Stream]

The draft decision proposal for this issue has been posted in the issue description.

A working group meeting series to cover this and the other aspects of Enhanced Error Handling will be set up to walk through the decision proposals. Details will be posted in this issue thread when they are available.

[ryderwj]

After review of the Error Handling – Data Structure proposal, the following feedback is offered:

Error Codes:

Generally support the intent behind Option 3 (Enumerated error codes), with the following recommendations:

1. A period, or some other character, must be reserved to act as the delimiter between the namespace, error category and error code sections. Since the “AU.CDR” namespace itself contains a period, then the period character is not suitable to use as a delimiter. A colon may be a better character to use for this purpose.
2. The CDS.AU namespace may require further specialisation to segment codes across the broader standard. For example a CDS error code relating to bank accounts will be of no relevance to a data recipient in the telco segment.
3. Participant error codes (those not in the CDS.AU namespace) should not be permitted, or be discouraged. Allowing them:
   o adds an unbounded number of potential codes a data recipient has to handle
   o means additional services may be needed to discover those codes and handle them correctly
   o suggests organisations have genuinely unique error conditions which is unlikely – at least at the level of abstraction the standard should be aiming for.
4. Participant error codes, if permitted, should map onto a standard taxonomy otherwise it will be difficult, if not practically impossible, for these errors to be useful to data recipients at the time the error is raised. Alternatively they could be provided as “reason codes” to a standardised error, meaning data recipients can choose to handle it or not based on the top-level error code

A URN structure not unlike the one already used for the ACR in the identity token may be another format to consider.

Error Data Model:

Supportive of Option 1, or a simplified implementation of Option 2 that is not tightly integrated to the request structure. Whilst recognising the better developer experience that error pointers provide, there is concern that the investment required to develop and test this feature for each API now and into the future will be substantial. It is questionable whether the proposal for a source and particularly a JSON Path pointer will be used by data recipients to respond and self-heal, or whether it will simply be logged for triage at a later point. If the latter, then a good error standard will not require this in the response.

Error Code Discovery:

The preference is that participant specific error codes are not part of the specification at all, thus eliminating the need for such a service because all error codes are defined.

Option 2 calls for several infrastructure pieces – a way to publish, a way to consume and presumably a way to classify error codes in some way for each data holder. If error codes need to be published, then Option 2 offers a superior solution to Option 3 but several details would need to be worked through to ensure a satisfactory outcome for all parties, including the change management approach.

[CDR-API-Stream]

Hi @ryderwj, thanks for the feedback. Comments below:

Error Codes

1. This was the intent although it may not have been articulated very well. The intention was to use a period / decimal point delimiter for tokenising the error code. This was in line with the UK OBIE approach. That said, we did look at the hyphenated segmentation of the error code. We'd be happy to hear feedback on what people prefer on this one but the general approach was to have some sort of delimited error code structure.

2. This is true - we have identified the "Register" as one such example. Looking at industry specific categorisation, we're open to feedback. The approach you suggest is possibly worthwhile. The other approach, which we are considering, is to describe sector specific errors like AU.CDR.Entitlements.BankingInvalidAccount by clearly labelling the sector they apply to. The benefit of this is we can still classify / group errors within a broader error category. Bearing in mind the majority of error codes should be industry agnostic (e.g. a malformed or invalid field applies to all sectors). Really what we need is a way to differentiate sector-specific error codes, that's correct.

3. Thanks for the feedback. There is an assumption that some degree of flexibility is required for participants to define different errors and aide migration. This is especially true in the competitive space where DHs may build adjacent APIs. It is also why the discovery service concept was raised.

4. We'd be happy to see examples and hear more views on this.

Error Data Model
This is where data recipient viewpoints will be helpful to shape this discussion.

Error Code Discovery
Thanks. If errors can be appropriately covered by a set of CDR Error Codes that provide this coverage (e.g. using more generic error codes) then the benefit of this discovery service is certainly less warranted.

[CDR-API-Stream]

As part of the Enhanced Error Handling consultation a Working Group workshop series has now been set up. To register for the first workshop, please register your interest here: https://dsb_enhancederrors_workshop01.eventbrite.com.au

[CDR-API-Stream]

The first in a series of workshops covering enhanced error handling was held on Tuesday 09/06/2020 covering the options proposed in this Decision Proposal.

A link to the public workshop whiteboard and collaboration artefacts is available here.

The workshop was well attended and included over 30 participants. Based on the discussion there were some new options identified and good consensus on many aspects. This is summarised below. The next workshop is being held on Tuesday 16/06/2020 and is free to register.

Error Code Structure

Feedback

• A well-defined structure for error codes was well supported
• Good support for a namespaces enumeration approach
• Avoid case sensitivity with error codes - uppercase as one option
• Desire to have code error codes be applicable to all error responses where a custom error code may arise - in other words, a participant's customer error code should be a specialisation of the CDR error code
• Key objective for participants is that the structure can be parseable. Hence, preference to have aspects in the DP that were optional to be mandatory and define a structure that can be easily parsed in a known way
• Issues were raised with the use of a period '.' delimiter for structuring the error codes
• JSON $ref was also proposed.
• Participant Specific Codes (customer error codes):
  Two options were proposed:
  • Use two code attributes. One for the core code code, eg. code and the second is a participant specific custom code e.g. participant_code
  • Use the URN structure to allow participants to append custom codes as a specialisation of an applicable CDR error code
  • The latter option is preferred

Outcomes

1. Agreement to define a URN schema for error codes.
2. Define custom error codes as an extension within the URN format (e.g. using fragments or other approach) so there is a "catch all" CDR error code they fit within.
3. Define a generic error code that allows for participant error codes to be created underneath it when there is no other CDR Error Code to map to.

Error Data Model

Feedback

• source/pointer may be useful but there is a brittleness to using it that must be understood because changes to the standards will change the pointer
• it may also be expensive to maintain in the long term
• difficult to determine if data model changes are required without starting from error scenarios to scope the problem and requirements before considering technical options

Outcomes

1. No data model changes were agreed
2. Agreed to review this DP once error scenarios had been considered under DP Decision Proposal 120 - CDR Error Codes for Enhanced Error Handling #120

Discovery service

Feedback

• General consensus is that discovery services aren't needed if DHs have to conform to the core CDR Error Codes and they are representative of all of the error scenarios that will be encountered
• Agreement that for extensions to the CDR (e.g. commercial APIs exposed by a DH that are publicly available to the ADR market) there is a clear need to offer discoverability and have the error codes published
• Strong agreement not to use a DH hosted API for discovery
• Publishing the error catalogue on the CDR Register has drawbacks: it is only discoverable by data recipients that become accredited - prospective ADRs cannot see and build to the error catalogue ahead of time so it defeats the purpose
• Fourth option identified:
  • Participants (primarily Data Holders) must publish their custom error catalogue on their Developer Portal (or equivalent developer website)
  • Data standards define the structure and format for publishing the catalogue
  • Suggestion that the standards define the JSON schema for the error catalogue
  • Data Recipients can then obtain the custom list of error codes via the DHs developer portal

Outcomes

1. Recommended that Option 4 be adopted
2. DP should be updated to define the JSON schema for Option 4

[commbankoss]

Commonwealth Bank requests that the decision proposal remain open for feedback until July 7th.

Given our engineers are busy with go-live activities, we have had little time to review the proposal and cannot provide feedback at this point.

[CDR-API-Stream]

In light of go-live activities in preparation for July 1st CDR obligations and community feedback this decision proposal has been extended until COB July 7th

[NationalAustraliaBank]

NAB supports the above outcomes in relation to Error Code Structure and the Discovery Service (ie. option 4).

[mannharleen]

Feedback from Origin Energy on DP-119

Section on DP	Comment
None - General comments	Error related to specific endpoints: Focus on endpoint specific error types is required. e.g. For DP109 for NMI Standing Data, there are no discussion around what specific errors are possible and what error codes would look like. e.g. how is DH expected to respond to an incorrect servicePointId
Error codes	We are comfortable with the recommendation to go with the URN naming format. The separator could be a “/”, “.” or “:” Further, the hierarchy needs to be defined such that it is clear and uniform to create newer codes: e.g. AU.CDR.Missing.Header vs AU.CDR.Header.Missing
Error code discovery	We are comfortable with the 4th option that was discussed during the workshops. The 4th options states: Fourth option identified: - Participants (primarily Data Holders) must publish their custom error catalogue on their Developer Portal (or equivalent developer website) - Data standards define the structure and format for publishing the catalogue - Suggestion that the standards define the JSON schema for the error catalogue - Data Recipients can then obtain the custom list of error codes via the DHs developer portal However, we would like to suggest that the DH only host a subset of all the (common/standard and custom) error codes that are applicable to the DH i.e. errors that could be generated by the DH’s responses, i.e. the DH be expected to host: A subset of common error codes that are defined by the Standard and may be returned via DH’s response All custom error codes (specific to the DH)

[commbankoss]

Commonwealth Bank's response to the decision proposal is as follows.

Error Codes
We support the current recommendation, option 3 - Error Codes Enumeration. This option creates a better developer experience, and the detailed enumerations mean that errors don't need to be looked up immediately.

Data Model
We prefer the adoption of a third option: instead of introducing the "source" field, which we believe is excessive, the source value can be placed in the title. For example:

If this option is not accepted by other participants, we support the current recommendation, option 1 - No change.

Error Code Discovery
We support the current recommendation, option 4 - Custom error catalogue on Data Holder's Developer Portal, in combination with option. This would allow each data holder to detail their specific scenarios the errors expected to be returned for them.

We note that while this option provides flexibility to fit error to their custom use-cases, having uniformity for common errors is necessary for consistent error handling, and as such we also recommend a discovery document for the non-custom list of error codes.

[WestpacOpenBanking]

Westpac welcomes the current consultations on error handling. We think that it is important to articulate a set of scenarios where further specification of errors will result in improved customer experiences. Doing this collectively before finalisation of decisions will help to ensure that we take an approach which is fit for the intended purpose.

Consideration will need to be given in terms of the transition to any new error handling approach. In particular, Data Recipients may have already implemented error handling for existing data holder implementations. Specification of new error structures will be a breaking change across the standard holistically and may require a new major version number (changes are also likely to be breaking for each endpoint).

Flexibility should be given to allow for differences in vendor implementations, particularly where conformity would require customisation of certified vendor implementations against the normative references in the information security profile. Customisation of these products creates delivery risk and cost and security risks. An approach to allow this flexibility where needed would be to make use of RFC2119 in the specification of error scenarios. For example, (e.g. “In the case of xyz error scenario, then the server MUST return a 4xx HTTP response code and SHOULD return a 415 HTTP response code…”).

We also remark that providing too much information to customers through error codes to data recipients may in itself create a risk of harm in some instances. Care should be taken to allow data recipients to provide our common customers with good experiences without creating these risks.

Error codes

• We support the use of a well-defined structure for error codes.
• The primary aim of the error code structure should be ease string parsing for developers. In particular error codes should be no longer than necessary and provide useful context for these string comparisons. Inclusion of a reserved namespace, category code and code seem to meet these aims.
• We agree that it would be better to avoid case sensitivity.
• We are supportive of the use of a URN structure and to allow appending of custom codes as a specialisation of an applicable CDR error code.
• We suggest that there is a need for generic error code categories for cases where the source of error is not identifiable or contained within the specified category.
• The initial list of error scenarios to be addressed should be defined before work to determine which categories of error will be included and overall suitability.
• As above, significant provision should be made to allow for generic errors where greater specificity may create a risk of harm.
• An alternative to the inclusion of namespace and error category codes in the code format may be to include additional fields in the error object.

Error data model

• We are in principle supportive of source/pointer. Currently we provide a similar level of granularity in text through the “detail” field. This approach may be brittle, especially in the context of endpoint versioning, and so will incur additional maintenance costs for participants.
• The initial list of error scenarios to be addressed should be defined to ensure that the data model is fit for purpose.
• Thought should be given as to the amount of information present in the string error code vs as additional fields in the error data model. We suggest retaining alignment with the jsonapi.org specifications as is currently the case.

Error discovery

• Error discovery is essentially a documentation issue. Data recipients need to be able to identify error scenarios that will provide for a better customer experience so that they can develop against them.
• A significant need for data holder specific documentation would indicate that the standards are deficient in terms of defining common error scenarios.
• The likely outcome of per-participant error discovery either by developer portal or by API where there are significant volumes of errors of interest to data recipients would be that data recipients would only implement improved experiences for the largest data holders. This is because there are more than 100 ADIs in Australia. This issue will be compounded as other sectors are added to the regime.
• The standards should document errors in both the swagger and the HTML version.

[CDR-API-Stream]

Hi @mannharleen thanks for your feedback. Regarding error code discovery, can you please confirm your expectations.

i.e. the DH be expected to host:

• A subset of common error codes that are defined by the Standard and may be returned via DH’s response

Are you suggesting that DHs should only host the set of custom error codes not defined by the Consumer Data Standards? In other words, DHs would not publish the core CDR error codes they support? If so, this would seem reasonable.

[CDR-API-Stream]

Hi @mannharleen

Further, the hierarchy needs to be defined such that it is clear and uniform to create newer codes: e.g. AU.CDR.Missing.Header vs AU.CDR.Header.Missing

This feedback was received in the workshop as well: to specify the hierarchy consistently. The primary intention is that errors can be conveyed with increasing granularity and there is a deterministic and parseable way for API clients to interpret both CDR error codes and custom error codes across the CDR (including any sector or DH specific nuances).

[CDR-API-Stream]

Hi @commbankoss thank you for the confirmation of your positions.

Re: Data Model

We prefer the adoption of a third option: instead of introducing the "source" field, which we believe is excessive, the source value can be placed in the title.

Can you explain how this is different to applying a SHOULD or MAY to the use of the source parameter? The advantage of the source parameter is that it would be more easily parseable by API clients.

[CDR-API-Stream]

Hi @commbankoss

Re: Error Code Discovery
Can you elaborate on the use of the discovery document for core CDR error codes?

1. Do you see this being implemented using Option 4 as well (i.e. published on the DH developer portal)?
2. Would this be the catalogue of core CDR error codes limited to what the DH knowingly implements?

[CDR-API-Stream]

Hi @WestpacOpenBanking thanks for you feedback and comments.

Consideration will need to be given in terms of the transition to any new error handling approach

Are there any recommendations you have to support the transition? One option would be transitionary fields in the error payload where they are currently used (e.g. x-cdr-code, x-cds-title, x-cds-description). These may be included in the meta object of the error payload so JSONAPI support is preserved.

These could be provided for a period of time before retirement of the transition fields. However this would not accomodate changes in the HTTP status codes if DHs have currently chosen to use a different status code.

[CDR-API-Stream]

Hi @WestpacOpenBanking

A significant need for data holder specific documentation would indicate that the standards are deficient in terms of defining common error scenarios.

This may also indicate that the DH has built competitive extensions to the CDR. It is envisaged that core CDR error codes should be representative of the CDR APIs defined in the Consumer Data Standards.

That said, feedback on the current proposed catalogue #120 is welcomed to understand whether there are scenarios not covered which ADRs and DHs deem important.

[CDR-API-Stream]

This consultation is now being closed. Feedback received will be considered and an initial position will be drafted and published in due course.