WWSympa is not always respecting wwsympa_url

[mivance-atvi]

We recently upgraded our VMs from Ubuntu Xenial LTS to current Ubuntu Bionic LTS. Our Sympa VM was one of these and this had a concomitant upgrade in Sympa. We have two problems with the current release:

1. After login we are redirected a naked URL that does not respect wwsympa_url
2. Archive links generate a naked URL that does not respect wwsympa_url.

For (2), imagine we have a proxy/forwarding server at acme.activision.com that is internally public facing, and a private VM at acme-sympa.local that runs the Sympa instance. We forward acme.activision.com/wws to acme-sympa. Thus, if we visit:

https://acme.activision.com/wws/info/ai

we get a proper page. If we then attempt to access the archive at

https://acme.activision.com/wws/arc/ai

we instead get a redirect to

https://acme-sympa/wws/arc/ai/2017-03/

which does not exist as something that can be resolved.

We had a similar problem to this last year when we upgraded from 14.x to 16.x, see the email "generation of incorrect links in archive pages only" on sympa-users. In that case we discovered this patch which fixed the issue:

https://sourcesup.renater.fr/scm/viewvc.php/branches/sympa-6.2-branch/src/cgi/wwsympa.fcgi.in?root=sympa&r1=12687&r2=12686&pathrev=12687

However our new version of Sympa already contains this patch.

Note that if I manually navigate to

https://acme.activision.com/wws/arc/ai/2017-03/

And then click on archive links these generate correctly loaded URLS such as

https://acme.activision.com/wws/arc/ai/2017-03/msg00001.html

which load just fine.

Version

Version: 6.2.24~dfsg-1

Installation method

Ubuntu packages.

Expected behavior

Redirect URLs are generated correctly that respect wwsympa_url.

Actual behavior

Incorrect URL redirects are generated.

Additional information

You can see our configuration bits here:

foo@acme-sympa:/etc/sympa/sympa$ grep acme *
sympa.conf:domain       acme.activision.com
sympa.conf:wwsympa_url  https://acme.activision.com/wws
sympa.conf:soap_url     http://acme.activision.com/sympasoap

I've also set log_level in sympa.conf and see the direct generation but nothing else particularly interesting. E.g.

Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Scenario::new() Unable to read file /etc/sympa/acme.activision.com/scenari/include.shared_doc.d_read.header
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Scenario::new() Unable to read file /etc/sympa/scenari/include.shared_doc.d_read.header
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Scenario::new() Unable to read file /usr/share/sympa/default/scenari/include.shared_doc.d_read.header
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug2 Sympa::Scenario::verify(HASH, is_subscriber([listname],[sender]), HASH, )
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug2 Sympa::Scenario::verify(HASH, is_editor([listname],[sender]), HASH, )
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug2 Sympa::Scenario::verify(HASH, is_owner([listname],[sender]), HASH, )
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug2 Sympa::Scenario::verify(HASH, true(), HASH, )
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Session::store()
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Session::renew(id_session=54877109468092)
Jan  7 13:54:45 acme-sympa wwsympa[4618]: debug Sympa::Session::set_cookie(localhost, session, secure= )
Jan  7 13:54:45 acme-sympa wwsympa[4618]: notice main:: Redirecting to https://acme-sympa/wws/arc/ai/2017-03/

Here is our minimal nginx setup for the proxy passthrough:

    location /wws {
        proxy_pass http://acme-sympa/wws;
    }
    location /static-sympa {
        proxy_pass http://acme-sympa/static-sympa;
    }
    location /wwsicons {
        proxy_pass http://acme-sympa/wwsicons;
    }
    location /wwsarchive {
        proxy_pass http://acme-sympa/wwsarchive;
    }
    location /icons {
        proxy_pass http://acme-sympa/icons;
    }

[mivance-atvi]

This seems to be down to the authority handling in Sympa::get_url. For instance if I modify wwsympa.fcgi@1512 to pass default this resolves the redirect on login:

    unless ($temporary_actions{$action} or $ENV{'REQUEST_METHOD'} ne 'GET') {
        my $redirect_url =
            Sympa::Tools::WWW::get_my_url($robot, authority => 'default');
        $redirect_url =~ s/[?].*\z//;
        $session->{'redirect_url'} = $redirect_url;
    }

Similarly if I modify do_arc:

    # Given partial URI, redirect to base.
    unless ($in{'month'}) {
        $param->{'redirect_to'} = Sympa::get_url(
            $list, 'arc',
            nomenu    => $param->{'nomenu'},
            paths     => [$latest, ''],        # Ends with '/'.
            authority => 'default'
        );
        return 1;
    }

this resolves the redirect setup for accessing the archives.

[ikedas]

Hi @mivance-atvi, does this help? A change on 6.2.15:

sympa/ONEWS

Lines 154 to 167 in 5951669

	***** - Now wwsympa_url parameter defines the URL publicly accessed by users.
	***** - Contrarily http_host and cookie_domain parameters indicate the URI
	***** and/or domain which are locally recognized by wwsympa, i.e taken form
	***** HTTP_HOST ("Host:" filed) and REQUEST_URI CGI environments.
	***** - "Location:" response field sent by wwsympa to cause redirection also
	***** gives local URL.
	***** If requests sent from users to wwsympa are rewritten by intermediate
	***** server(s), you should make sure that wwsympa server and rewriting
	***** server(s) are appropriately configured. For example when the server is
	***** placed behind some sort of reverse-proxy including Apache mod_proxy
	***** (with ProxyPass), nginx (using proxy_pass) and squid (in accelerator
	***** mode). Note that not all reverse-proxies rewrite requests. For
	***** example, Pound does never rewrite requests by default.
	***** If it is not the case, existing configuration need not changing.

[mivance-atvi]

Hi @ikedas , it's not clear to me what to do with this, nor what effect it should have from looking at the Sympa source.

I tried setting 'http_host' in sympa.conf to be similar to the originating server e.g. 'acme.activision.com' which is what I see in the HTTP 'Host:' field on successful page loads, but it did not change the behavior. Nor did having the '/wws' base path appended.

In my tests, on a successful load 'Referer:' will look like 'https://acme.activision.com/wws/lists'. On an unsuccessful load the bad redirect will show a 302 with 'Location:' set to 'https://acme-sympa/wws/arc/acme/2018-11/', and then this will be a failed connection.

Looking at the source it's not clear to me what functionality http_host provides beyond identifying a robot config (not an issue for us as we only have a single "domain") and then overriding the cookie_domain in wwsympa.fcgi.

[ikedas]

Hi @mivance-atvi ,

There are two ideas to tweak WWSympa with proxy. (A) rewriting all and (B) passing through all.

For generalization, these host names (domain) are used in below:

• ext.example.org for host part of publicly accessed web URL.
• int.example.org for host name of WWSympa accessed by proxy.
• mail.example.org for mail domain.

Note: Following settings have not been verified.

(A) Rewrite "Host:", "Location:" and cookie domain

• Users recognize WWSympa as "ext.example.org".
• WWSympa resognizes itself as "int.example.org".

sympa.conf:

domain          mail.example.org
wwsympa_url     https://ext.example.org/wws
http_host       http://int.example.org/wws
cookie_domain   int.example.org

Note that http_host and cookie_domain should have the host name which WWSympa recognizes itself as, instead of what users recognize.

nginx conf:

server {
    server_name ext.example.org;
    ...
    location /wws {
        proxy_pass http://int.example.org/wws;
        proxy_cookie_domain int.example.org ext.example.org;
    }
    ...
}

proxy_pass rewrites "Host:" request field, and it implies proxy_redirect setting that rewrites "Location:" response field. proxy_cookie_domain rewrites "domain" attribute of "Set-Cookie:" response field.

(B) Pass through host names

• Users recognize WWSympa as "ext.example.org".
• WWSympa resognizes itself as "ext.example.org".

sympa.conf:

domain          mail.example.org
wwsympa_url     https://ext.example.org/wws

Note that http_host and cookie_domain have default values so that they have the same domain as wwsympa_url.

nginx conf:

server {
    server_name ext.example.org;
    ...
    location /wws {
        proxy_pass http://int.example.org/wws;
        proxy_redirect off;
        proxy_set_header Host $host;
    }
    ...
}

"Host:", "Location:" and domain of "Set-Cookie:" won't be rewritten.

[mivance-atvi]

Hi @ikedas thanks for the extra info. FWIW setup (A) does not work. However setup (B) does. It's not clear to me if the former is a legitimate bug. Regardless we'll go forward with (B) for now, thank you again.

[ikedas]

@mivance-atvi, thanks for confirming. I'll investigate (A) later, and will add instruction on both (A) and (B) to documentation.

[ikedas]

Let's continue on sympa-community/sympa-community.github.io#35. This issue is closed.