Federation request signing is easily broken by proxies

[SISheogorath]

I recently came across this issue with synapse: matrix-org/synapse#3294 (comment)

It basically comes down to how reverse-proxies in front of synapse handle things. So far, I checked the spec which doesn't really specify whenever or things have to be url-encoded or not, but all example url-encode strings.

So far one could make assumptions on this, but HTTP explicitly states that URL encoded URLs, should be decoded before using them and make no difference.

Characters other
than those in the "reserved" set are equivalent to their
percent-encoded octets: the normal form is to not encode them (see
Sections 2.1 and 2.2 of [RFC3986]).

For example, the following three URIs are equivalent:

 http://example.com:80/~smith/home.html
 http://EXAMPLE.com/%7Esmith/home.html
 http://EXAMPLE.com:/%7esmith/home.html

https://tools.ietf.org/html/rfc7230#section-2.7.3

When we use HTTP as a standard, we should make sure we don't contradict it. Therefore we should fix the examples and implementations in this perspective.

For some cases, where things might contain special characters, that interfere with the HTTP standard like / we might want to explicitly disallow them in the spec. Because if we don't do this, it'll cause harm to the protocol in the long term perspective (as it already does with various WAF solutions in front of Synapse, when they canonicalize URLs).

[turt2live]

I'm not sure there's much we can do to fix this considering our sigil for aliases is # and event IDs can have / in them.

[uhoreg]

I don't think that we do contradict the RFC here. Is says that "Characters other than those in the "reserved" set are equivalant to their percent-encoded octets" (emphasis mine). However, "/" and "#" are both in the reserved set, so the server does not have to treat "/" and %2F as equal. "~" is not in the reserved set, so "~" and %7E are equivalent.

(Edited due to markdown fail)

[richvdh]

@uhoreg: The problem, I think, is that a proxy is within its right to switch between %-encoding and unencoded text, for unreserved characters. In other words, a reverse-proxy might switch %3A to : or vice-versa, which then breaks the Authorization on the header.

One solution to this would be for us to specify that all unreserved characters must be unencoded when the signature is made, or something - and for the receiving server to unencode any encoded reserved characters before it checks the signature.

[timokoesters]

I am currently running into this problem in Conduit. I think the only real solution for me is to do what richvdh hinted at and convert all unreserved characters to the percent-encoded version, because that seems to work in my tests