Failed handle request via 'PublicRoomListRestServlet'

[typokign]

First of all, thanks a bunch to @spantaleev for his help with my last two issues :D

I'm now running into an inconsistent issue trying to list rooms on the matrix.org room list via Riot. When selecting the matrix.org server from the Room directory, I get a "Riot failed to get the public room list. Internal server error" error in Riot. I also get the following error message in Synapse:

Aug 12 01:17:38 casa matrix-synapse[3271]: 2019-08-12 01:17:38,086 - synapse.http.server - 108 - ERROR - POST-351 - Failed handle request via 'PublicRoomListRestServlet': <XForwardedForRequest at 0x7f6a467dddd0 method='POST' uri='/_matrix/client/r0/publicRooms?server=matrix.org' clientproto='HTTP/1.0' site=8008>
Aug 12 01:17:38 casa matrix-synapse[3271]: Traceback (most recent call last):
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/http/server.py", line 76, in wrapped_request_handler
Aug 12 01:17:38 casa matrix-synapse[3271]:     await h(self, request)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/http/server.py", line 315, in _async_render
Aug 12 01:17:38 casa matrix-synapse[3271]:     callback_return = await callback_return
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/logging/opentracing.py", line 480, in _trace_servlet_inner
Aug 12 01:17:38 casa matrix-synapse[3271]:     result = yield defer.maybeDeferred(func, request, *args, **kwargs)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/rest/client/v1/room.py", line 398, in on_POST
Aug 12 01:17:38 casa matrix-synapse[3271]:     third_party_instance_id=third_party_instance_id,
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/handlers/room_list.py", line 498, in get_remote_public_room_list
Aug 12 01:17:38 casa matrix-synapse[3271]:     third_party_instance_id=third_party_instance_id,
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
Aug 12 01:17:38 casa matrix-synapse[3271]:     result = result.throwExceptionIntoGenerator(g)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
Aug 12 01:17:38 casa matrix-synapse[3271]:     return g.throw(self.type, self.value, self.tb)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/federation/transport/client.py", line 339, in get_public_rooms
Aug 12 01:17:38 casa matrix-synapse[3271]:     destination=remote_server, path=path, args=args, ignore_backoff=True
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
Aug 12 01:17:38 casa matrix-synapse[3271]:     result = result.throwExceptionIntoGenerator(g)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
Aug 12 01:17:38 casa matrix-synapse[3271]:     return g.throw(self.type, self.value, self.tb)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 774, in get_json
Aug 12 01:17:38 casa matrix-synapse[3271]:     timeout=timeout,
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
Aug 12 01:17:38 casa matrix-synapse[3271]:     result = result.throwExceptionIntoGenerator(g)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
Aug 12 01:17:38 casa matrix-synapse[3271]:     return g.throw(self.type, self.value, self.tb)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 241, in _send_request_with_optional_trailing_slash
Aug 12 01:17:38 casa matrix-synapse[3271]:     response = yield self._send_request(request, **send_request_args)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
Aug 12 01:17:38 casa matrix-synapse[3271]:     result = g.send(result)
Aug 12 01:17:38 casa matrix-synapse[3271]:   File "/usr/local/lib/python3.7/site-packages/synapse/http/matrixfederationclient.py", line 475, in _send_request
Aug 12 01:17:38 casa matrix-synapse[3271]:     raise e
Aug 12 01:17:38 casa matrix-synapse[3271]: synapse.api.errors.HttpResponseException: 500: b'Internal Server Error'

This isn't happening consistently, sometimes I'm able to list the rooms on matrix.org, but it definitely fails around 75% of the time.

Thanks!

[spantaleev]

I think I can reproduce it as well.

I couldn't find a bug report at the Synapse bugtracker, so we should probably create a new issue there.

[typokign]

Ah, I did a search and found the solution. In order to use room discovery on other matrix homeservers, the _matrix._tcp.<host> SRV record is required, even if the federation information is being served at <host>/.well-known/matrix. This is contrary to what the DNS docs suggest. Shall I submit a pull request to update the docs?

[spantaleev]

Where did you read this? Perhaps it's a bug, which should be addressed, rather than working around it by adding extra SRV records.

…

On 12 Aug 2019, at 06:42, David Cruz ***@***.***> wrote: Ah, I did a search and found the solution. In order to use room discovery on other matrix homeservers, the _matrix._tcp.<host> SRV record is required, which was not part of the DNS docs. Shall I submit a pull request to update the docs? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[spantaleev]

Closing this, as it's tracked in the upstream bug over at the Synapse repository.