Clarity on whether or not padding is required while Base64URL Encoding

[MichaelJBRichards]

I suggest that we should add the following wording to ensure that the requirement for padding is clear.

In section 3 (the API Signature Definition) of the Signature document:
"For the avoidance of doubt: whenever reference is made to Base64URL encoding in the signature definition or in the examples included in this document, then the encoding MUST include appropriate pad characters as specified in Section 3.2 of RFC 4648 (https://tools.ietf.org/html/rfc4648#section-3.2)"

In section 7.2.17 (the definition of a BinaryString) of the Open API specification:
"For the avoidance of doubt: the Base64URL encoding for a BinaryString MUST include appropriate pad characters as specified in Section 3.2 of RFC 4648 (https://tools.ietf.org/html/rfc4648#section-3.2)"

[henrka]

This issue is dependant on #13 and #14 as the BinaryString data type is only used for the IlpPacket today. If IlpPacket is replaced with the actual Transaction (#13) and the BinaryString is not used for the echo data (#14), then this issue is not relevant anymore.

[MichaelJBRichards]

Possibly not for the IlpPacket; but it will still be required for the JWS (first example,) I think

[henrka]

Ah yes you are entirely correct, sorry!

[henrka]

RFC 7515 which is used for JWS states that the padding characters should be omitted though (https://tools.ietf.org/html/rfc7515#page-6):

Base64url Encoding
Base64 encoding using the URL- and filename-safe character set
defined in Section 5 of RFC 4648 [RFC4648], with all trailing '='
characters omitted (as permitted by Section 3.2) and without the
inclusion of any line breaks, whitespace, or other additional
characters. Note that the base64url encoding of the empty octet
sequence is the empty string. (See Appendix C for notes on
implementing base64url encoding without padding.)

[MichaelJBRichards]

I kiss the rod. So should we withdraw?

[MichaelJBRichards]

Alternative for section 3 of the Signature document:

"For the avoidance of doubt: whenever reference is made to Base64URL encoding in the signature definition or in the examples included in this document, then the encoding MUST NOT include pad characters. This is in alignment with RFC 7515 (https://tools.ietf.org/html/rfc7515#page-6)."

[henrka]

To me at least it is already pretty clear that we use BASE64URL as defined in RFC 7515 by the following statement in Section 1.1:

This document uses the notational conventions for BASE64URL(OCTETS), UTF8(STRING), ASCII(STRING), and || defined in RFC 7515.

Section 1.1 in RFC 7515 (Notational Conventions) says:

BASE64URL(OCTETS) denotes the base64url encoding of OCTETS, per Section 2.

Section 2 in RFC 7515 says the following regarding base64url:

Base64url Encoding
Base64 encoding using the URL- and filename-safe character set
defined in Section 5 of RFC 4648 [RFC4648], with all trailing '='
characters omitted (as permitted by Section 3.2) and without the
inclusion of any line breaks, whitespace, or other additional
characters. Note that the base64url encoding of the empty octet
sequence is the empty string. (See Appendix C for notes on
implementing base64url encoding without padding.)

In my opionion, we must draw the line somewhere of what could easily be found in the RFC that we say we follow. There are a lot of other things that we do not mention from the RFC.

[MichaelJBRichards]

Well, I think we already had this conversation. We know that there are implementers who have misinterpreted this, so it seems to me sensible simply to make the requirement more explicit. I don't see the point in drawing this line in advance, since I think that the purpose of these documents is to help people implement applications which meet the requirements, not to waste resources by helping them to fix their mistakes. I don't understand what the downside is...

[henrka]

So for every mistake that an implementer does, we should add a note or text in the specification even if it is already part of the reference to avoid someone else making the same mistake? The specification will then end up being more of an implementation guide than a specification. Maybe there should be a separate document in a FAQ style which shows earlier issues which could help other implementers if they run into problems.

But if the consensus in the CCB is that this should be added in the document, then that is fine by me. I just think this one would be unnecessary as showed in my previous comment.

[millerabel]

@ehenrka @MichaelJBRichards I think Henrik's suggestion for a living FAQ-like document. It can guide future implementors by providing clarity when errors in implementation are discovered. In my view, this document is not "normative" — that is, it does not substitute for the specification. And implementing something incorrectly and failing validation tests can not be excused by "I read it in the 'tips' document so it must be right. The specification is still the prevailing truth.

But it would be a "tips and pratfalls" type of document that helps newcomers avoid the travails of prior travelers. The padding example is quite right. We could end up with an unreadable specification if we repeated all the choices other RFC writers made about each others' specifications.

[henrka]

This issue is included as part of the API version 1.1, but there is no actual change in any of the existing documents as the suggestion was to include the information in a separate FAQ document?

[elnyry-sam-k]

Hi Henrik, that is correct.

The outcome here is that we'll have a new document - "Implementation Guide" which will be part of the Supporting document set.

[henrka]

Hi @elnyry-sam-k, do you know if anyone is working on the Implementation Guide?

[elnyry-sam-k]

Thanks Henrik; as we discussed on our CCB calls, will follow this up