Clarify access token grammar

[IvoWingelaar]

Currently access tokens are specified according to section 5, which is very similar to RFC6750.
A note clarifies that clients should treat the access token as an opaque byte sequence, but leaves ambiguity with regards to what bytes are actually valid.

Confusingly, RFC6750 specifies that the token should follow the b64token grammar, which does not require base64 encoding, but instead is a set of characters defined identically as token68 specified in RFC7235. By design, token68 includes all base64 encoded strings and URI safe strings.

Am I correct in assuming that currently homeservers and clients implicitly follow RFC6750? And if so, I would suggest to make this requirement explicit in the specification.

[turt2live]

Clients are expected to take the access token from a login response and echo it back to the server on subsequent requests. They are not meant to care what format the token is in.

The spec is also already limited to UTF-8 encoding on JSON responses, which essentially means that so long as the server can send and receive the token anything is valid. Servers obviously have an interest in not making it complicated for encoding reasons though, which is why the specification recommends a particular format/encoding.

https://github.com/matrix-org/matrix-doc/issues/666 is the general problem of not specifying what "opaque" actually means.