Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issues with synapse and synapse-coturn services not starting #458

Open
wants to merge 3 commits into
base: testing
Choose a base branch
from

Conversation

Lab-8916100448256
Copy link

@Lab-8916100448256 Lab-8916100448256 commented May 3, 2024

Problem

  • synapse-coturn service does not start at boot
  • synapse service does not start on ipv4 only server

synapse-coturn error (this error is happening only at boot time. if the service is manually started later on, there is no error) :

May 02 22:22:31 domain.tld turnserver[1003]: 0: : NO EXPLICIT LISTENER ADDRESS(ES) ARE CONFIGURED
May 02 22:22:31 domain.tld turnserver[1003]: 0: : ===========Discovering listener addresses: =========
May 02 22:22:31 domain.tld turnserver[1003]: 0: : Listener address to use: 127.0.0.1
May 02 22:22:31 domain.tld turnserver[1003]: 0: : ERROR: main: Cannot configure any meaningful IP listener address

synapse error (this error is happening on servers that do not have ipv6 enabled :

May 03 14:08:34 domain.tld python[285652]: twisted.internet.error.CannotListenError: Couldn't listen on ::1:8008: [Errno 99] Cannot assign requested address.

Solution

  • for synapse-coturn service error, added the option -L 0.0.0.0 on ExecStart line in systemd unit file
  • for synapse service error, changed the listening address to localhost instead of ::1 and 127.0.0.1

PR Status

  • [x ] Code finished and ready to be reviewed/tested
  • [x ] The fix/enhancement were manually tested (if applicable)

Automatic tests

Automatic tests can be triggered on https://ci-apps-dev.yunohost.org/ after creating the PR, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)

@Lab-8916100448256 Lab-8916100448256 changed the title Testing Fix issues with synapse and synapse-coturn services sot starting May 3, 2024
@Lab-8916100448256 Lab-8916100448256 changed the title Fix issues with synapse and synapse-coturn services sot starting Fix issues with synapse and synapse-coturn services not starting May 3, 2024
@Salamandar
Copy link
Member

Can you link to the doc showing that localhost is a valid value ?
Also https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/reverse_proxy.md?plain=1#L43
It looks like we could just put 127.0.0.1 because we always have local ipv4

@Salamandar
Copy link
Member

Also according to https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/usage/configuration/config_documentation.md?plain=1#L457

the default is sane and we could just remove the line altogether.

Also it's for http-not-s traffic, it shouldn't even be used because yunohost enforces https…

@Lab-8916100448256
Copy link
Author

Can you link to the doc showing that localhost is a valid value ? Also https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/reverse_proxy.md?plain=1#L43 It looks like we could just put 127.0.0.1 because we always have local ipv4

I have not read any doc about that. I just tried it and it was working.
I have seen references to computers having only ipv6 somewhere. I don't remember where and I don't know if that could happen with a yunohost server

@Lab-8916100448256
Copy link
Author

Also according to https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/usage/configuration/config_documentation.md?plain=1#L457

the default is sane and we could just remove the line altogether.

Also it's for http-not-s traffic, it shouldn't even be used because yunohost enforces https…

It's used by nginx that is reverse proxing it to https

@Josue-T
Copy link

Josue-T commented May 3, 2024

Hello,

Maybe it look that you have an issue about pour network stack. Are you sure that ipv4 and ipv6 are enabled on your server ?

@@ -8,7 +8,7 @@ User=turnserver
Group=turnserver
Type=notify
EnvironmentFile=/etc/matrix-__APP__/coturn_env
ExecStart=/usr/bin/turnserver -c /etc/matrix-__APP__/coturn.conf $EXTRA_OPTIONS --pidfile=
ExecStart=/usr/bin/turnserver -c /etc/matrix-__APP__/coturn.conf $EXTRA_OPTIONS -L 0.0.0.0 --pidfile=
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it really needed to bind to any public addresses?
this is not reverse proxied?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not look like this is proxied. I do not find any reference to the port that is used (5351 in my case) in nginx config and the port is open on the firewall.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it looks like this is publicly available on purpose.

@Lab-8916100448256
Copy link
Author

Hello,

Maybe it look that you have an issue about pour network stack. Are you sure that ipv4 and ipv6 are enabled on your server ?

ipv6 is disabled. And that is on purpose. There are other things that break in my case if I enable ipv6 : https://forum.yunohost.org/t/fixed-installing-letsecrypt-certificate-is-taking-a-very-long-time/28645

@Lab-8916100448256
Copy link
Author

Lab-8916100448256 commented May 4, 2024

Can you link to the doc showing that localhost is a valid value ? Also https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/docs/reverse_proxy.md?plain=1#L43 It looks like we could just put 127.0.0.1 because we always have local ipv4

I though about this during the night. And I kind of remember cases (no in synapse) where using localhost for a bind address was not working but using 127.0.0.1 was working. So yes it is probably better to use 127.0.0.1

@Lab-8916100448256
Copy link
Author

@Salamandar , I have changed localhost to 127.0.0.1

@Josue-T
Copy link

Josue-T commented May 4, 2024

Hello,
Maybe it look that you have an issue about pour network stack. Are you sure that ipv4 and ipv6 are enabled on your server ?

ipv6 is disabled. And that is on purpose. There are other things that break in my case if I enable ipv6 : https://forum.yunohost.org/t/fixed-installing-letsecrypt-certificate-is-taking-a-very-long-time/28645

Hello,

Honestly I'm not sure that it's a good idea to drop ipv6 into the config. The future is to move to Ipv6.

For your specific case I think you don't need to completely disable ipv6. Since you don't have any default ipv6 route your issue should be solved.

Also as on nginx config side it's configured to reverse proxy to localhost so your change will break instance with ipv6 enabled.

@Salamandar
Copy link
Member

Honestly I'm not sure that it's a good idea to drop ipv6 into the config. The future is to move to Ipv6.

Well yes but actually no, here we are talking exclusively about internal IP, and most apps are curently only listening on ipv4 127.0.0.1.

Copy link

@Josue-T Josue-T left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said before to me the best way to fix your issue is to enable again ipv6 and mange correctly the ipv6 network stack. Normally on Linux from the point that you don't have any default ipv6 route you shouldn't have any timeout issue.

Note that probably some other apps expect to have ipv6 enabled, so disabling completely ipv6 is not a good solution to me.

@@ -304,7 +304,7 @@ listeners:
tls: false
type: http
x_forwarded: true
bind_addresses: ['::1', '127.0.0.1']
bind_addresses: [127.0.0.1]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of you remove ipv6 listening you also need to avoid that nginx try a ipv6 connection. So you also need explicitly set proxy pass to 127.0.0.1.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially set this to localhost instead of 127.0.0.1, which would not break nginx config.
I need to find back the issue I'm kind of remembering about 127.0.0.1 working and localhost failing. Maybe it was not about listening on localhost but it was about connecting to localhost to another app that was listening on 127.0.0.1. Basically another app that is broken by the same thing you are telling that should not be done here.

@@ -8,7 +8,7 @@ User=turnserver
Group=turnserver
Type=notify
EnvironmentFile=/etc/matrix-__APP__/coturn_env
ExecStart=/usr/bin/turnserver -c /etc/matrix-__APP__/coturn.conf $EXTRA_OPTIONS --pidfile=
ExecStart=/usr/bin/turnserver -c /etc/matrix-__APP__/coturn.conf $EXTRA_OPTIONS -L 0.0.0.0 --pidfile=
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By this you remove the default value to listening on ipv4 and ipv6. So this will break instances with working ipv6.

@Lab-8916100448256
Copy link
Author

Lab-8916100448256 commented May 7, 2024

I'll do some more tests to see what is strictly necessary to do in all that I have disabled regarding IPV6 to fix my issue of let's encrypt timeouts and try to find a better fix of my IP stack.
However note that this will affect only the synapse error. If I remember correctly, the coturn error is also happening on IPV6 enabled host. I'll test that again to confirm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants