Transition of required parameters in the CDR Arrangement JWT

[CDR-API-Stream]

Description

Version 1.17.0 clarified that the CDR Arrangement JWT presented by the Data Holder to the Data Recipient must include

All parameters in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

Some Data Holders have indicated that:

• they are currently constructing a JWT with only the cdr_arrangement_id.
• some Data Recipients are accepting this JWT without rejection.
• they were not able to meet July 31st 2022 timeframes to provide a valid JWT in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication as specified above.
• they supported an end state where the JWT is constructed in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.
• they were not aware of the scale of affected Data Holders.

Area Affected

CDR Arrangement Revocation End Point - CDR Arrangement JWT method.

Change Proposed

Option 1: No change - transition requirements as described are unchanged. In this situation,

• Data Holders present a full and complete JWT to the Data Recipient from July 31st 2022.
• Data Recipients validate the JWT is correct and includes all required parameters in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

Option 2: New transition arrangements - new transition arrangements are introduced to allow Data Holders to present a JWT with either (a) only the cdr_arrangement_id, or (b) a complete JWT. In this situation,

• Data Holders have additional time to transition their implementation to correctly implement the CDR Arrangement JWT
• Data Recipients must support both styles of JWT for a period of time
• This will create a breaking change on Data Recipients and implementation impacts before July 31st 2022
• After the transition period, Data Holders must present a complete JWT and Data Recipients must reject an invalid / incomplete JWT

Option 3: Extend existing transition arrangements - the current transition arrangements are extended to allow Data Holders to present a JWT a complete JWT with a longer lead time. In this situation,

• Data Holders can continue to present the cdr_arrangement_id using the "CDR Arrangement Form Parameter" method
• Data Holders have additional time to transition their implementation to correctly implement the "CDR Arrangement JWT" method and must do so in accordance with the Self-Signed JWT Client Authentication requirements.
• Data Recipients must support both "CDR Arrangement Form Parameter" method and "CDR Arrangement JWT" method for a further six months
• This represents the least change option by simply extending the current transition arrangements
• Data Recipients accepting the cdr_arrangement_jwt in accordance to the "CDR Arrangement JWT" method must validate that all claims including those in accordance with the Self-Signed JWT Client Authentication requirements.
• After the transition period, Data Holders must present a complete JWT and Data Recipients must reject an invalid / incomplete JWT

Feedback is sought from Data Holders that are affected and what impact Option 2 and Option 3 would impose on Data Recipients.

Option 3 is preferred. This will extend existing obligation dates. The proposed obligation date would change from July 31st 2022 to FDO Y22 #3 (August 31st 2022) or FDO Y22 #4 (November 15th 2022). Feedback is sought on the existing obligation dates and the desired transition period.

[anzbankau]

Thank you for raising this change. We are a Data Holder impacted by this issue and are supportive of Option 2. A obligation date of 31/08/2022 would be appropriate in our view.

[CDR-API-Stream]

Hi all / @anzbankau

This change request has been updated with a third option. Given the timeframes of the existing obligations and issues identified to Data Holders this change can be treated as urgent if community input requests it to be.

[anzbankau]

Following on from the addition of option 3 - ANZ maintain support of option 2 as it honours any existing implementation that has been complete during the transition period from 31st March to 31st July. That is, the prior FDO in 1.15 should remain intact for 31st July. A new FDO for the addition of all parameters to JWT should assigned to allow DH and ADR to implement required changes. We would support a FDO for this new aspect of either 31/08/2022 or 15/11/2022, with the later date potentially preferable to give ADRs a longer lead time to implement.

It could also be clarified for Option 2 that although the ADRs will need to accept both style of JWT, they will not need to validate that all parameters are present. The addition of validation will then be added in the new FDO date.

We request this change to be treated urgently so there is predictability about the required changes by DH and ADR for 31st July.

[perlboy]

Given the change wasn't properly published in the Standards until 1.17.0 (May 2022) it seems appropriate that the FDO now move so Option 3 is supported.

Requesting Data Recipients to change their implementation to work around a documentation defect introduced by the DSB is not appropriate. Holders should reverse their broken implementation and the DSB should facilitate a mechanism for handling backward compatible.

As a result of the above it seems appropriate to introduce some FAPI like wording to support sending duplicate parameter (notably cdr_arrangement_id) that must match allowing a fall back strategy for ADRs for non-compliant JWTs:

shall additionally send duplicates of the cdr_arrangement_id as form parameters

This matters because a Data Holder can't transition to only cdr_arrangement_jwt as existing Recipient implementations may not be uplifted yet. At least one ADR (the major one) has confirmed they currently handle a fallback approach and I can confirm that Biza holders are currently sending both cdr_arrangement_id and cdr_arrangement_jwt in it's revocation requests.

Finally, this issue (and it's precursors) seems to be highlighting the brokenness of the DSBs merge and release strategy.

[CDR-API-Stream]

This change request has been marked as urgent with the approval of the Data Standards Chair to progress a solution to provide transition certainly prior to the current obligation date of July 31st 2022.

[commbankoss]

CBA proposes a fourth option, which combines aspects of option two and three to allow for a broader scope of current implementations to be supported until participants are able to implement the required solution.

From a Data Holder perspective, extend the timeframe that both methods are accepted until 15 November 2022 (CDR Arrangement Form Parameter method and the CDR Arrangement JWT method), and where the CDR Arrangement JWT method is used, allow the Data Holder to present a JWT with either (a) only the cdr_arrangement_id or (b) a complete JWT, at which point they will be required to use only the CDR Arrangement JWT methods, constructing the JWT with all required parameters.

From a Data Recipient perspective, extend support of both methods, and where the CDR Arrangement JWT method is used, only validate the cdr_arrangement ID, until the new obligation date (proposed as 15 November above), at which point only the CDR Arrangement JWT method should be accepted, and all required parameters should be validated.

CBA would also like to note that https://github.com/ConsumerDataStandardsAustralia/standards-maintenance/issues/247 was previously raised to propose alternative solutions to implement the functionality currently managed by the ADR revocation endpoint, and that the options presented in that CR would reduce complexity and increase security and interoperability. In addition, it is noted that the current ADR revocation endpoint is completely custom designed and not protected by any security profile (such as FAPI), and requires proper threat and security analysis to be done to make sure this doesn’t cause ecosystem problems in the future.

[CDR-API-Stream]

Feedback and discussion

Based on discussion in the maintenance iteration call, it was supported that

• ADRs should not reject requests presenting the cdr_arrangement_id as a form parameter. However, if present, it must match the same value presented in the CDR Arrangement JWT.
• In other words, from July 31st, Data Holders may continue to send the cdr_arrangement_id as a form parameter in conjunction with the CDR Arrangement JWT method without being rejected.
• ADRs should validate all parameters within the JWT but must not reject a JWT that doesn't include the claims defined in the Self-Signed JWT section. Instead, this requirement is imposed on Data Holders as a SHOULD (recommended) not a MUST.
• In other words, Data Holders should present a CDR Arrangement JWT including all claims in accordance with the Self-Signed JWT section however this is not mandatory. Presenting a JWT with only the cdr_arrangement_id would be valid.

For Data Holders

This would retain the 31st July 2022 transition but allow Data Holders to ship a CDR Arrangement JWT including only the cdr_arrangement_id. It would be highly recommended that Data Holders ship all Self-Signed JWT claims as soon as is practical for integrity protection.

Noting that feedback has indicated that for the options presented above, it was requested that transition be extended to November 15th 2022. It is unclear whether this would be needed now that support of the JWT excluding additional Self-Signed JWT claims is not required. Data Holders may send the revocation request using the CDR Arrangement JWT using only the cdr_arrangement_id from July 31st 2022.

For Data Recipients

This would require them to continue to accept the cdr_arrangement_id as a form parameter, but additionally validate that if present, it would match the value included in the CDR Arrangement JWT.

This proposal would also require Data Recipients to validate all parameters, where presented within the JWT, are correct.

Data Recipients would have additional build requirements to not reject the cdr_arrangement_id as a form parameter from July 31st and validate the value is the same as presented in the JWT where it is supplied in both formats. Data Recipients would also need to support validating all claims in the JWT but not require the Self-Signed JWT claims to be present.

Feedback indicated that Data Recipients are currently supporting both being presented together and they are validating JWTs presented with only the cdr_arrangement_id and as well as all claims required by the Self-Signed JWT method.

Proposal

---

Data Recipient hosted endpoint

The location of the Data Recipient Software Product CDR Arrangement Revocation End Point is determined by the RecipientBaseURI provided by the Data Recipient Software Product in the client Software Statement Assertion (SSA).

This end point will be implemented according to the following:

• Data Recipient Software Products MUST expose their CDR Arrangement Revocation End Point under their recipient_base_uri published in their Software Statement Assertion.
• Data Holders must be authenticated when they call this end point according to the guidance in the Client Authentication section.
• If the cdr_arrangement_id is not related to the client making the call it MUST be rejected.
• From March 31st 2022, Data Recipients MUST support the "CDR Arrangement JWT" method.
• From July 31st 2022, Data Holders MUST send the cdr_arrangement_id using the "CDR Arrangement JWT" method.
• Data Holders MAY additionally send a duplicate of the cdr_arrangement_id as a form parameter.
• Data Recipient Software Products MUST NOT reject requests including the cdr_arragement_id as a form parameter.
• If the cdr_arrangement_id is presented as a form parameter, Data Recipient Software Products MUST validate it is identical to the cdr_arrangement_id presented in the CDR Arrangement JWT.

CDR Arrangement JWT method

The request MUST include the following parameters using the application/x-www-form-urlencoded format in the HTTP request entity-body:

• cdr_arrangement_jwt: A signed JWT that includes the cdr_arrangement_id.
• cdr_arrangement_jwt: A newly signed JWT with the following parameters in accordance with [JWT]:
  • cdr_arrangement_id: The ID of the arrangement that the client wants to revoke.

The cdr_arrangement_jwt SHOULD include all parameters in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

---

Further discussion

This change was discussed as a transition state towards a longer-term target state where a more secure solution is adopted for secure event notifications to Data Recipients. This was discussed as part of the FAPI 2.0 transition for consultation later in 2022.

Discussion also considered the need for explicit standards defining how ADRs verify that a Data Holder is still a valid active participant.

[spikejump]

It seems the original issue is really some Data Holders not able to meet the July 31st 2022 obligation date.

The way the options are proposed are very one-sided with Data Holders having the flexibility to support the new obligation when they're ready. But the Data Recipients are required to support new obligation with added functional requirements. i.e. support both formats concurrently as well as validating cdr_arrangement_id to match when both forms are present in the same request. This seems to be penalizing the ADRs for a delay that's completely out of their control.

[CDR-API-Stream]

Hi @spikejump, the purpose is multi-faceted. This proposal seeks to support a transition where Data Recipients continue to receive a revocation request they can handle whilst minimising impact for both Data Recipients and Data Holders.

We have feedback that some ADRs can meet the current obligation date and all Data Recipients should currently support JWT validation and validation of the cdr_arrangement_id as a form parameter. With this change, ADRs would continue to support this but require the JWT to be present from 31st July: as per the current standards.

The only changes after 31st of July that weren't previously articulated are:

• the validation of Self-Signed JWT claims and validating that they are the same as what is sent in the bearer token.
• allowance for both form parameter and JWT to be shipped together and not be rejected. Data Holders have confirmed they are doing this now and not being rejected which indicates ADRs are currently supporting this.

For the inclusion of Self-Signed JWT claims, rather than making this a MUST for Data Holders it was agreed on the maintenance iteration call that applying a SHOULD requirement for Data Holders to send a JWT including the cdr_arrangement_id and all other claims in the Self-Signed JWT section is preferred because it will reduce the chance of any issues come 31st of July and leave this change to be done within the timeframes of the Data Holder thereafter.

If you have other views you'd like to progress please include them in the conversation.

[spikejump]

Hi @CDR-API-Stream,

Perhaps the ADRs can have relaxation on compliance on the new requirement to match cdr_arrangement_id when both forms are present in the same request? July 31st is only a month away. Or extend the compliance date to the typical 6-month out?

It is most unlikely that DHs will be requesting revocation with two different cdr_arrangement_id in the same request. So may be the matching requirement is not required?

May be one of the following options will be acceptable:

1. DHs are not allowed to send different cdr_arrangement_id in the same request.
2. ADRs' choice as to which of cdr_arrangement_id or cdr_arrangement_jwt to use without ensuring they match.
3. Precedence is with cdr_arrangement_id. If it is in the message then it will be used by the ADR (with cdr_arrangement_jwt ignored); if not then cdr_arrangement_jwt will be used; if both not present then an error is returned. i.e. There's no requirement to do the matching when both are present.
4. Precedence is with cdr_arrangement_jwt. Similar to above, if it is in the message then it will be used with cdr_arrangement_id ignored. Again, not requiring to ensure the two values match.

Option 1 is the best and would be surprising it will be an issue with the DHs. In this case, no change for ADRs.
Option 2 is most likely preferred option for ADRs as there's no uplift required for compliant ADRs.
Options 3 & 4 do require uplift if ADRs' existing implementation does not use one of the precedence rules. In which case, the compliance date should be pushed out.

May be a combination of Options 1 & 2 are suitable.

Thanks for the consideration.

[CDR-API-Stream]

Hi @spikejump,

Perhaps the ADRs can have relaxation on compliance on the new requirement to match cdr_arrangement_id when both forms are present in the same request? July 31st is only a month away. Or extend the compliance date to the typical 6-month out?

Feedback from ADRs on the maintenance iteration call last week indicated that some are already correctly handling this scenario. Given we are currently within the transition phase, ADRs should be handling both the form parameter and JWT methods presently.

1. DHs are not allowed to send different cdr_arrangement_id in the same request.

That is certainly the expectation. However given the DH is acting as the client in this request it is important for the ADR to correctly handle situations where both values are different.

2. ADRs' choice as to which of cdr_arrangement_id or cdr_arrangement_jwt to use without ensuring they match.
3. Precedence is with cdr_arrangement_id. If it is in the message then it will be used by the ADR (with cdr_arrangement_jwt ignored); if not then cdr_arrangement_jwt will be used; if both not present then an error is returned. i.e. There's no requirement to do the matching when both are present.
4. Precedence is with cdr_arrangement_jwt. Similar to above, if it is in the message then it will be used with cdr_arrangement_id ignored. Again, not requiring to ensure the two values match.

Perhaps in this situation additional statements to the affect of Data Holders should not send or must not send a different cdr_arrangement_id if sending both from parameter and JWT could be included however as part of any server side validation, ADRs shouldn't be relying on clients doing the right thing.

This issue is on the agenda for the Maintenance Iteration call tomorrow. If you can attend that would be helpful to further this discussion. Given this issue is marked as urgent, the intention is to arrive at a resolution at the conclusion of the call.

[spikejump]

Hi @CDR-API-Stream,
It may be our misreading of the specification. However, we are not aware at any point the specification mentioned both forms can co-exist at the same time. Our reading had always being support both formats but it is one or the other. This issue is the first time we're exposed to the "co-existence" possibility.

Given the expectation is no DHs will send two different cdr_arrangement_id in the same request then may be the specification can relax the matching of cdr_arrangement_id to be a SHOULD instead of a MUST.

In addition, what should ADRs return in terms of error code when cdr_arrangement_id mismatch? 400 or existing 422?

Thanks for the consideration.

p.s. Apologies I won't be able to make the call today as I've been sick. I only happened to see this when dealing with something critical.

[commbankoss]

CBA agrees with Intuit, that if an additional requirement is being placed on ADRs to match the cdr_arrangement_id when both forms are present in the same request, a July 31st obligation date does not allow for enough time to support this change. CBA recommends extending the compliance date in line with the typical 6 month allowance.

The alternative options proposed by Intuit, whereby matching is not be required, is preferable. CBA supports either Option 1 or Option 2, as both options remove the requirement for and ADR to build additional selection logic if both options are present.

[CDR-API-Stream]

Hi @spikejump,

However, we are not aware at any point the specification mentioned both forms can co-exist at the same time.

The standards currently state that both methods are to be supported:

Until July 31st 2022, Data Recipients MUST support both "CDR Arrangement Form Parameter" method and "CDR Arrangement JWT" method presented to their CDR Arrangement Revocation Endpoint.

[alvin-frollo]

@CDR-API-Stream
Apologies for coming in late to this discussion but would you mind elaborating the thought process of this decision?

Version 1.17.0 clarified that the CDR Arrangement JWT presented by the Data Holder to the Data Recipient must include all parameters in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

This seems to contradict the design of the original proposed solution whose example can still be found in the current version of CDS (screenshot is attached).

In https://consumerdatastandardsaustralia.github.io/standards/#client-authentication under Self-signed JWT Client Authentication, I'd like to highlight the last requirement of that section:

The JWT MUST be accepted from the client at the requested endpoint using the "Authorization Request Header Field" mechanism as described in section 2.1 of [RFC6750].

• Is there any specific reason why we're adding additional parameters into the content of cdr_arrangement_jwt which are already available in the "Authorization Request Header Field"?
• Will the ADRs be expected to perform 2 separate validations for each CDR arrangement revocation request coming from the DH (this seems superfluous?):
  • a validation on the JWT in the "Authorization Request Header Field"
  • an extra validation on the cdr_arrangement_jwt

[spikejump]

Hi @CDR-API-Stream,

Hi @spikejump,

However, we are not aware at any point the specification mentioned both forms can co-exist at the same time.

The standards currently state that both methods are to be supported:

Until July 31st 2022, Data Recipients MUST support both "CDR Arrangement Form Parameter" method and "CDR Arrangement JWT" method presented to their CDR Arrangement Revocation Endpoint.

We've read above quoted part of the specification multiple times. We've never read it as supporting both methods in the same request. We'd say this is a typical interpretation issue for specifications.

Our interpretation of the above is that ADRs are to support both formats during the transition period to facilitate DHs sending old format as well as those DHs (that have uplifted) sending new format but not both formats together in the same request. We don't believe we're alone in this interpretation.

To be frank, it is unexpected for a DH to send both formats in the same request given the old format will need to be removed in the near future. It will be another uplift for the DHs later to remove redundant code in which the exercise will incur additional (unnecessary) costs. The costs could have being avoided if the old format was removed when new format adopted.

Given where we are, we think our proposed Option 1, 2 are quite reasonable.

[CDR-API-Stream]

Thanks all for the additional feedback.

Option 1 would be hard to enforce and would be an anti-pattern for request validation.

For Option 2, if I understand your proposal correctly @spikejump, where both JWT and form parameter are sent in the same request, an additional transition is required (15/11/2022) for ADRs to validate a cdr_arrangement_id presented in the JWT as well as a form parameter. Given the move is towards support for the JWT from July 31st presently, Option 4 seems to be the most reasonable extension of Option 2. In other words, precedence is deterministically set for receiving the cdr_arrangement_id in the JWT. After November 2022, then ADRs must validate the value in the form parameter (if present) is the same as presented in the JWT.

This aligns to the intent of introducing the JWT in the first place.

The updated proposal incorporating @spikejump and @commbankoss feedback is as follows (note 🟧 indicates the changed statements):

---

Data Recipient hosted endpoint

The location of the Data Recipient Software Product CDR Arrangement Revocation End Point is determined by the RecipientBaseURI provided by the Data Recipient Software Product in the client Software Statement Assertion (SSA).

This end point will be implemented according to the following:

• Data Recipient Software Products MUST expose their CDR Arrangement Revocation End Point under their recipient_base_uri published in their Software Statement Assertion.
• Data Holders must be authenticated when they call this end point according to the guidance in the Client Authentication section.
• If the cdr_arrangement_id is not related to the client making the call it MUST be rejected.
• From March 31st 2022, Data Recipients MUST support the "CDR Arrangement JWT" method.
• From July 31st 2022, Data Holders MUST send the cdr_arrangement_id using the "CDR Arrangement JWT" method.
• Data Holders MAY additionally send a duplicate of the cdr_arrangement_id as a form parameter.
• Data Recipient Software Products MUST NOT reject requests including the cdr_arragement_id as a form parameter.
• 🟧 If the cdr_arrangement_id is presented as a form parameter, Data Recipient Software Products SHOULD validate it is identical to the cdr_arrangement_id presented in the CDR Arrangement JWT.
• 🟧 From November 15th 2022, if the cdr_arrangement_id is presented as a form parameter, Data Recipient Software Products MUST validate it is identical to the cdr_arrangement_id presented in the CDR Arrangement JWT.

CDR Arrangement JWT method

The request MUST include the following parameters using the application/x-www-form-urlencoded format in the HTTP request entity-body:

• 🟧 cdr_arrangement_jwt: A signed JWT that includes the cdr_arrangement_id. This is a newly signed JWT with the following parameters in accordance with [JWT]:
  • cdr_arrangement_id: The ID of the arrangement that the client wants to revoke.

The cdr_arrangement_jwt SHOULD include all parameters in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

---

[CDR-API-Stream]

This change as presented above is staged for review:
ConsumerDataStandardsAustralia/standards-staging@release/1.18.0...maintenance/521

[spikejump]

One of the staged change says:

The cdr_arrangement_jwt SHOULD include all parameters in accordance with Data Holders calling Data Recipients using [Self-Signed JWT Client Authentication]

Should this not say MUST? The SHOULD implies not all required parameters need to be supplied which contradicts the MUST wording in Self-Signed JWT Client Authentication.

Perhaps something like the below was intended?

The cdr_arrangement_jwt MUST follow what's prescribed in [Self-Signed JWT Client Authentication] section

[CDR-API-Stream]

In the staged change above, an additional clarifying statement has been proposed for Data Recipient software product validation to deal with Data Holders sending all Self-Signed JWT claims:

• From November 15th 2022, if the Self-Signed JWT claims are presented in the "CDR Arrangement JWT", Data Recipient Software Products MUST validate in accordance with Data Holders calling Data Recipients using Self-Signed JWT Client Authentication.

[CDR-API-Stream]

Hi @spikejump,

Should this not say MUST?

The feedback during the maintenance iteration calls supported a SHOULD (strong recommendation) to permit Data Holders to continue to send the cdr_arrangement_jwt as is during the active transition period but upgrade to supporting all Self-Signed JWT claims as soon as practicable.

[spikejump]

Hi @CDR-API-Stream,

Thanks for the clarification. Where the statement of

The cdr_arrangement_jwt SHOULD include all parameters in accordance with Data Holders calling Data Recipients using [Self-Signed JWT Client Authentication]

appears in the spec in the staged doc does not imply it is only for transition. It sits in "CDR Arrangement JWT method" section and before "Data Holder hosted endpoint" section?? or have we misread the staged information.

It would be clearer if the SHOULD is called out as only applicable during transition period. Thanks.

[CDR-API-Stream]

Hi @spikejump thanks for your feedback on this. The retention of SHOULD was discussed and considered acceptable. The implementation of including all parameters is highly recommended and many DHs indicated their security posture was to do so.

[spikejump]

Hi @CDR-API-Stream,
Apologies for continuing on this. What does "highly recommended" mean? Are the MUST claims in cdr_arrangement_jwt no longer a MUST? e.g. are 'iss, aud etc not MUST? May be we have misunderstood the reference to "all parameters" in the SHOULD sentence. The only OPTIONAL claim/parameter under the Self-signed JWT Client Authentication section is iat.

We are concerned that without clarification of the SHOULD being transitional only, then post Nov when an (existing/upcoming) DH does not send all the MUST claims/parameters (eg. missing aud) in the cdr_arrangement_jwt and is rejected by an ADR, the DH can reference the SHOULD sentence and claim it is conforming to the specification and that the ADR needs to relax its validation.

What may be the concern of stating the SHOULD sentence being transitional only?