handle 401 on websocket connection

[h3ndrik]

Since the websocket update, I have difficulties logging in from remote.

Home Assistant release (hass --version): 0.38.3
Python release (python3 --version): Python 3.5.3
Component/platform: webui

Description of problem:

My (slightly more complex) setup contains a nginx proxy which does https and enforces http basic auth if you're outside my home network. This results in the following behaviour:

1. everything works fine at home
2. when i'm away from home, the browser got the UI already in cache and only requests /api/websocket and receives a http 401, tries again /api/websocket, receives 401...
   2.1. this is repeated several times and then the browser shows the cached version of the login screen, but again - there is no new request to the server.
   2.2. the 401 responses to the websocket connection don't trigger any new username/password prompt

The home-assistant authentication is disabled: # api_password:

I'm not exactly an expert on web-development... I think the problem is the web-ui (javascript) not handling the http status 401 correctly. Is it possible to disregard the browser cache and reload "/" or something so the browser will show a new authentication popup? I'd like to use the http basic auth from nginx so i can have multiple logins/accounts.

Probably(?) related issue: #5954

nginx configuration:

map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
}

server {
        listen 443 ssl;
        listen [::]:443 ssl;

        # SSL configuration
        ssl_certificate /etc/letsencrypt/live/<...>/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/<...>/privkey.pem;

        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";

        access_log      /var/log/nginx/smarthome_access.log;
        error_log       /var/log/nginx/smarthome_error.log;

        charset                 utf-8;

        server_name <...>;

        location / {
                satisfy                 any;
                allow                   192.168.0.0/24;
                auth_pam                "Smarthome";
                auth_pam_service_name   "nginx";

                proxy_pass      http://192.168.0.10:8123;
                proxy_buffering         off;
                proxy_set_header        Host            $http_host;
                proxy_set_header        X-Real-IP       $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        X-Forwarded-Proto       $scheme;
                proxy_redirect http:// https://;
                proxy_http_version      1.1;
                proxy_set_header        Upgrade         $http_upgrade;
                proxy_set_header        Connection      $connection_upgrade;
        }
}

[camrun91]

the server name line <...> I assume you have the actual DNS name there? I know I was having issues with this until I completely matched the example config here https://home-assistant.io/docs/ecosystem/nginx/ at the bottom of the page. There are a few differences between that config and your attached config.

[andriej]

@camrun91 config is not the issue, as I have it set like on example page. Additional headers which @h3ndrik have shouldn't affect the proxy.
Everything was working ok before 0.38

[hessu]

Hi, it appears that some browsers do not pass over the authentication information from plain HTTP requests session to the WebSocket requests. The 401 response is coming from nginx, not hass.

To support those browsers with WebSocket API, the auth_pam directives would need to be excluded for /api/websocket URL as described in #5954, comment by #5954 (comment) – the example of adding a separate "location /api/websocket" block without auth_* directives works, although the description around it does not describe why it's needed.

[h3ndrik]

@camrun91 Yes, the server name line contains the actual DNS name.

If i exclude the /api/websocket URL from the authentication as @hessu pointed out, wouldn't that leave me with a security issue there? Maybe i have to dig into the code and learn how websockets work.

But after reading #5954 (comment) again... it could well be the browser that is using some cached, pre 0.38-code?
I purged the cache and -for now- it is working. But i'm not sure if it will keep working after the session times out again and nginx will send a 401. I'll try that next.

[demonspork]

This issue is in polymer see the issue here :
home-assistant/frontend#110

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[KlaasH]

Running 0.44.2, no change.

[abmantis]

Also having this issue on 0.45.

[cmsimike]

Shot in the dark here - what browser/os?
Safari/MacOS and iOS do not pass client certificates when making a websocket connection. If you are using similar os/browser, maybe that's it? And if so, can you try in Chrome?

[abmantis]

@cmsimike I've tried with both Chrome and Firefox on desktop and Chrome and Opera on Android.

[elupus]

Was just hit with this issue too. Clearing cache resolves is for one session.

I think a workaround would be to let nginx change cache headers on the main screen, so it gets loaded an re-authenticated.

Still a better solution would be preferable

[elupus]

This seem eerie similar to: #1303

[elupus]

Also i just noticed that it's not just cache going one here. Home Assistant register a service_worker.js which seems to handle requests without them hitting server.

[elupus]

An ugly workaround to temporarily restore connectivity without clearing cache is to have nginx serve a special logout url that resides at an url location that is whitelisted in service_worker.js (/api, /local, ...).

This logout url will de-register the service_worker and clear authentication, allowing you to re-login.

In your nginx server block add an include for the below logout snippet.

/etc/nginx/snippets/logout.conf

location /local/logout {
        return 401;
}

error_page 401 /local/errors/401.html;

location /local/errors {
        auth_basic off;
        ssi        on;
        ssi_types  text/html;
        alias /var/www/html/errors;
}

/var/www/html/errors/401.html

<!DOCTYPE html>
<script>
navigator.serviceWorker.getRegistrations().then(function(registrations) {
         for(let registration of registrations) {
                   registration.unregister()
         } })
</script>
<p>You're not authorised. <a href="<!--# echo var="scheme" -->://<!--# echo var="host" -->/">Login</a>.</p>

[andkit]

Thx for the tip with the whitelisted uris (google lead me here after I made out the service worker as the culprit responsible for eating away my requests)

For anyone else with this issue, you probably don't even have to deregister the service worker, you just need an url that will trigger the authentication dialog (the websocket requests don't and everything else comes out of the cache)

I'm going to try this setup for a while (the same with a 200 static response worked for me on desktop and android, the redirect would just be added convenience but I'm almost certain it should work too)

    location = /local/auth_me {
        add_header Cache-control private;
        add_header Cache-control no-cache;
        add_header Cache-control no-store;
        if ($remote_user != "") {
            return 302 /;
        }
    }

[aderusha]

I'm seeing this same problem using a Netscaler for advanced authentication.

[vondruska]

At least using Chrome, another workaround is to prevent the service worker from loading through nginx. You'll lose things that depend on the worker like HTML5 Push Notifications, but I don't use those features in my environment. Here's the steps I used:

1. Add the deny to service_worker.js in your nginx config:

location = /service_worker.js {
  deny all;
}

2. Save and apply/reload/restart nginx config
3. In Chrome, visit chrome://serviceworker-internals.
4. Find the domain of your Home Assistant instance, click the Unregister button
5. Clear the browser cache on the domain
6. Revisit Home Assistant

And you should be good to go. You'll need to do step 3-5 on each device that you've visited HA on. Clearing the cache could be enough but I've had mixed results here.

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[demonspork]

I saw another issue related to this get closed on its own citing "we won't degrade everyone's performance for an edge case" but I did not understand.

Why can't the service worker or something see a 301 response... and then just follow it to authenticate?

If it gets an access denied error of some sort it should still ask for login, but I don't see how allowing it to follow a redirect could be harmful to anyone's experience while at the same time will allow these "edge cases" to work better.

[andriej]

I have just moved and got my HA server out of the box - just moving all configs to newest version.
Will see if problem still occures, but still I can't imagine just opening port straight for service withoit nginx in between. Especially that I use it together as a shared port with VPN.

To the point - I've re-registered notifications with HASS and don't want to get this ticket closed without being resolved...

[xentac]

I was able to use the hints in this issue that /local isn't cached and put my auth urls under there (so they always hit the server). Then I wrote some javascript using extra_html_url (from #9150) to request the auth url on load. If it comes back as 401, I change window.location to my auth url, bypassing all of the websocket retrying and the broken user/pass auth of home assistant.

Testing the html was the hardest because the service worker caches so aggressively, I had to clear site data every time I changed something so that the changes would be picked up.

It's still not ideal that the code assumes any non-successful websocket connection means we need to pass a username and password.

[andriej]

Just checked; The problem still persists in latest version.

[jurriaan]

Also having this problem, which is probably a bug in Chrome (at least I am having this issue in Chrome) where a 401 does not trigger the authentication. See https://bugs.chromium.org/p/chromium/issues/detail?id=623464

Unfortunately they say it won't be fixed for some time, so I'm also going to disable the service worker.

[andriej]

Now the problem is even worse - translations are getting 401 even with proper authentication...

[avocadosalsa]

I am seeing the same issue @andriej mentioned about the translations triggering a 401 response even with proper authentication.

[linvinus]

hi guys!
maybe you have the same problem? home-assistant/frontend#667

[iliketoprogram14]

I gave up and made the switch to SSL+NGINX+oauth2_proxy+notifications. Once I got that working, I haven't had any problems since.

[andriej]

Can you share config(s) except the oauth's secrets?
or perfectly make them as a tutorial on website (I can help after get it to work)

[andriej]

or it's rather question to @xentac especially for the nginx file.

-- EDIT:
I've managed to do some working config with notifications (will see if they last few hours)
Although I can write how-to, but last file I need is something you've described:

I have configured /api/__checkauth to return a 200 or a 404 if authenticated and
a 401 if not authenticated.

in gist. currently I've got 404 with it, but solution works.

[andriej]

@xentac - any chance on getting the file/config in nginx from you?

[dgomes]

Can any of you update the documentation with a solution for this ? Then we could close this issue :)

[andriej]

Well, there's option to handle this error by running in example the oAuth way, but still there's problem with basic-auth method and I didn't find any 'ugly workaround' for this.

[iliketoprogram14]

@dgomes this issue is still technically unsolved with no documented workaround, so it should stay open.

[dgomes]

What about @xentac solution ?

[andriej]

It's totally different, as BASIC AUTH means username and password, while oAuth is logging with external authorization platform (gmail, facebook etc.)

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[abmantis]

Still an issue.

[ulope]

Same issue here.

[swvajanyatek]

same here :(

[abmantis]

@elupus workaround seems to have stop working lately. I'm not sure if it is related to lovelace.

[elupus]

I switched to an oauth based solution inspired by a previous poster, but i think it can be generalized/simplified to basic auth.

But a more complete solution would probably be a patch so that login screen of hass is alwayed fetched from server and not in service worker. That said auth stuff in hass seem to be in flux right now with user support added so maybe not the best time to work on a real patch.

[pgross41]

I also have this issue. The only workaround I have found on mobile is to load the site in incognito mode.

[abmantis]

This issue just got worse with the new auth system :(

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[abmantis]

Still an issue

[mlebrun]

Also experiencing this

[stale]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.