CTS incorrectly implements Data Holder Initiated Revocation

[biza-io]

Description

As this issue has legal implications we request it is prioritised as a matter of urgency.

We have opened this change request here because:

1. The previously separate ACCC call and personnel associated with the call have now moved into the DSB
2. "Minor errata and documentation fixes" in 1.11.1 resulted in this issue
3. While stated as minor errata the change itself did not appear to have an associated Decision Proposal (or at least one is not linked in the Changelog)

Within Standards 1.11.1 the definition of "resource path" was "clarified" with the Self-signed JWT Authentication definition of aud specified from:

Contents MUST be the base URI for the end point being accessed.

to:

Contents MUST be the "resource path" for the end point being accessed.

This had the effect of "clarifying" that the aud for calls to the Data Recipient Revocation endpoint must be set to <RecipientBaseUri>/arrangements/revoke however:

1. The current ACCC Conformance Test Suite within the Data Holder Initiated Revocation test successfully passes implementations with the aud set to <RecipientBaseUri>
2. The non-normative examples specify a syntactically incorrect aud (despite noting the audience is the revocation endpoint) of https://data.recipient.com.au/revocation (the prescribed path is <RecipientBaseUri>/arrangements/revoke)
3. Data Recipients are inconsistently accepting or rejecting one or accepting both aud values

Certain Data Recipients are actively (but correctly) rejecting Data Holder initiated revocation requests with an aud of <RecipientBaseUri>. This has resulted in a situation where the legal requirement for notification of revocation to Data Recipients is not being successfully achieved and Data Holders are queuing, essentially forever (in accordance with ACCC/DSB guidance), these requests which are never successfully accepted.

The number of Data Recipients (technically correctly) rejecting requests with an aud of <RecipientBaseUri>/arrangements/revoke are far outnumbered by the Data Holders issuing them. In addition the number of Data Recipients (perhaps incorrectly) accepting requests with aud set to <RecipientBaseUri> significantly outnumber the Data Recipients rejecting them.

Area Affected

• Data Recipient hosted CDR Arrangement Revocation End Point

Change Proposed

We propose the following changes with an intent to have the lowest impact possible to the live ecosystem coming into one of the busiest periods in financial services coupled with likely change freezes approaching or already in place within Holder environments. Consequently we lean, at least initially, towards the modification of Data Recipient behaviour.

1. We propose, similar to the wording used for aud in Private Key JWT Client Authentication, that the aud wording used for Self-signed JWT Client Authentication be altered to:

   REQUIRED. Audience(s) that the JWT is intended for. The Data Holder or Data Recipient Software Product MUST verify that it is an intended audience for the token. The "resource path" for the end point being accessed SHOULD be used. In order to facilitate interoperability and for Data Recipient hosted endpoints only, the endpoint MUST also accept the <RecipientBaseUri> as a value identifying the intended audience.

2. Introduce a future dated obligation to phase out the use of <RecipientBaseUri> as the audience reverting the wording either to its original form or an explicit MUST for <RecipientBaseUri>/arrangements/revoke

3. Align the CTS behaviour to the Data Standards in alignment with the proposed FDO outlined in (2)

4. Align the non-normative example with the expected behaviour

[CDR-API-Stream]

Hi @biza-io this is a reasonable issue to raise and thank you for requesting it be treated as urgent. We have labelled it accordingly and it will be brought into the current maintenance iteration to discuss at next Wednesday's maintenance call.

[ghost]

This is a very sensible proposal. There is currently much confusion and resulting real-world inoperability between DH and ADR implementations. The proposed change would address this issue and clear up confusion.

As an ADR, this will result in some effort from us to implement. However, it is not a great deal of effort and well worth the outcome of interoperability as well as the support and debug time saved.

[CDR-API-Stream]

After discussion with the community throughout the maintenance iteration meetings on this issue, the proposed solution for feedback and discussion in the call today, is as follows:

1. Adopt the change proposed by @biza-io to allow DHs to continue issuing arrangement revocation notifications
2. Support an immediate requirement that ADRs SHOULD support this change
3. Support a future dated obligation that ADRs MUST support this change
4. Support a future dated obligation that DHs and ADRs deprecate the conditional audience claim support (such that the "resource path" for the end point being accessed MUST be used)

For ADRs and DHs performing conformance testing, a conversation with the regulator would be needed for CTS and onboarding support. That would be the appropriate avenue to discuss point 3 in the original change proposed.

A strategic long-term consultation on secure event messaging and notifications will be consulted on in 2022.

[perlboy]

While it's great that the issue has a path way forward is there a reason that the ADR requirement is being marked as a SHOULD in the first place? It seems to me that the DSB enacted a change that was effectively an immediate MUST on Data Holders and the only course correction available is to enact a change that is an immediate MUST on Data Recipients - regardless of what the "process" is meant to be the DSB didn't follow it to begin with. In the absence of such a rectification path what do Data Holders do with revocation requests that are being rejected between now and the FDO?

Regarding fixing the CTS behaviour - I'm confused why this would be an ADR/DH responsibility when the bug was introduced by the DSB. Is there a reason why the DSB can't interact with the Regulator directly, potentially with a matching incident within the CDR Service Management portal? This way the DSB maintains ownership of the bug it created within a changelog described as "Minor Errata" that is effectively resulting in non-compliance of regulated parties.

Finally, I continue to have personal concerns around the lack of governance regarding how the Standards are released and the transparency of the changes being made. This issue was present in the published versions of the Standards from 5 October 2021 and started occurring approximately a month ago as some Data Recipients updated their software, leading Biza.io to open this ticket 3 weeks ago. Once round-tripped the resolution is likely to be nearly 3 months - a long time for what is essentially a legal requirement.

It seems likely, in the absence of a stronger governance framework combined with the lack of a best practice release approach (as indicated in numerous threads before), that this issue will merely be a historical example of an underlying issue that the DSB does not wish to acknowledge and resolve appropriately.

[CDR-API-Stream]

This change has been staged for review: ConsumerDataStandardsAustralia/standards-staging@release/1.15.0...maintenance/428

[perlboy]

Data Recipients MUST only accept the "resource path" for the end point being accessed MUST be used from July 31st 2022.

This doesn't seem to read well and also doesn't specify an obligation on the Data Holder, perhaps:

Until July 31st, Data Recipients MUST accept the "resource path" for the endpoint and the <RecipientBaseURI> as a valid audience value. From July 31st, Data Holders MUST use an audience value matching the "resource path" for the endpoint and the Data Recipient MUST verify the audience matches the "resource path" for the endpoint.

The "resource path" for the end point being accessed SHOULD be used.
In order to facilitate interoperability and for Data Recipient hosted endpoints only, the endpoint MUST also accept the <RecipientBaseUri> as a value identifying the intended audience.
From July 31st 2022: The "resource path" for the end point being accessed MUST be used.

This covers the use case above however it may introduce ambiguity for other self-signed authentication uses (notably metrics endpoint which I note has been modified). With this change the MUST wouldn't apply but the SHOULD would resulting in the Register being able to use any endpoint and the Holder having no explicit guideline. I'm unclear if this impacts installations as we don't use self-signed authentication for metrics ourselves.

Finally, going to flag that the use of "resource path" as a reference is a bit clunky. Resource Path is defined in the Standards as a term now so perhaps that could be used?

[CDR-API-Stream]

Hi @perlboy, thanks for the feedback.

This doesn't seem to read well and also doesn't specify an obligation on the Data Holder, perhaps:

Until July 31st, Data Recipients MUST accept the "resource path" for the endpoint and the as a valid audience value. From July 31st, Data Holders MUST use an audience value matching the "resource path" for the endpoint and the Data Recipient MUST verify the audience matches the "resource path" for the endpoint.

Thanks. The wording of the future obligation has been revised to be clearer and reflect the language above.

This covers the use case above however it may introduce ambiguity for other self-signed authentication uses (notably metrics endpoint which I note has been modified). With this change the MUST wouldn't apply but the SHOULD would resulting in the Register being able to use any endpoint and the Holder having no explicit guideline. I'm unclear if this impacts installations as we don't use self-signed authentication for metrics ourselves.

This is a valid point. The change has been revised in line with this feedback and in line with the change agreed with the community in the maintenance iteration.

Finally, going to flag that the use of "resource path" as a reference is a bit clunky. Resource Path is defined in the Standards as a term now so perhaps that could be used?

Agreed this makes sense. This change has been reflected.

[CDR-API-Stream]

This change was incorporated into release v1.15.0. Refer to Decision 212 for further details.

[perlboy]

Note: As of 17 March 2022 certain plans, accepted by ACCC, within the CTS are enforcing the audience without the /arrangements/revoke part added so the test now fails on CTS for a valid audience value (one including the /arrangements/revoke) for the Data Holder Initiated Revocation test.

This demonstrates the impact of unstructured change management on participants.