Web socket fails to connect through Apache #4723

lplassman · 2022-01-11T04:02:17Z

OS/Web Information

Web Browser: Chrome
Local OS: MacOS 12.2
Remote OS: Ubuntu 20.04 LTS
Remote Architecture: amd64
code-server --version: 4.0.1

Logs

code-server --log debug
[2022-01-11T03:38:43.612Z] info code-server 4.0.1 735c6da
[2022-01-11T03:38:43.613Z] info Using user-data-dir ~/.local/share/code-server
[2022-01-11T03:38:43.628Z] info Using config file ~/.config/code-server/config.yaml
[2022-01-11T03:38:43.628Z] info HTTP server listening on http://0.0.0.0:8080/
[2022-01-11T03:38:43.628Z] info - Authentication is enabled
[2022-01-11T03:38:43.628Z] info - Using password from ~/.config/code-server/config.yaml
[2022-01-11T03:38:43.628Z] info - Not serving HTTPS
[22:38:51] Extension host agent started.
[22:38:51] [Theme Service] Scanning for theme extension...
[22:38:51] [Theme Service] theme-abyss
[22:38:51] [Theme Service] theme-defaults
[22:38:51] [Theme Service] theme-kimbie-dark
[22:38:51] [Theme Service] theme-monokai
[22:38:51] [Theme Service] theme-monokai-dimmed
[22:38:51] [Theme Service] theme-quietlight
[22:38:51] [Theme Service] theme-red
[22:38:51] [Theme Service] theme-solarized-dark
[22:38:51] [Theme Service] theme-solarized-light
[22:38:51] [Theme Service] theme-tomorrow-night-blue
[22:38:51] [Theme Service] Attempting to find user's active theme: Default Dark+
[2022-01-11T03:38:54.949Z] debug got latest version {"latest":"4.0.1"}
[2022-01-11T03:38:54.949Z] debug comparing versions {"current":"4.0.1","latest":"4.0.1"}
[2022-01-11T03:39:51.007Z] debug 0 active connections

Screenshot

Notes

I currently only experience this error on code-server 4.0.1. code-server 3.12.0 works without issue.
I am currently using apache2 with a reverse proxy. The current config for the apache2 virtual host is below:

<Location "/">
ProxyPass http://127.0.0.1:8080/
ProxyPassReverse http://127.0.0.1:8080/
ProxyPassReverse /
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule ^/?(.*) "ws://127.0.0.1:8080/" [P,L]

I also made sure all the websockets modules are loaded in apache2 and working without problems. No errors occur on the Apache web server. Any suggestions on how to get this working with Apache2?

This issue can be reproduced in VS Code: No

The text was updated successfully, but these errors were encountered:

JimmyNi · 2022-01-12T03:04:21Z

Same issue here. Using apache2 with reverse proxy. Only occur with 4.0.1, code-server 3.12.0 works fine for me.

jsjoeio · 2022-01-13T16:54:01Z

@code-asher any ideas?

code-asher · 2022-01-13T18:55:22Z

If the exact same config works with 3.12 that does seem like it could be an issue with 4.0 but I have no idea how; so far all the 1006 errors I have seen have been caused by misconfiguration. I am not very familiar with Apache but maybe this comment has a clue: #2104 (comment)

konni · 2022-01-17T15:56:16Z

Same issue here using apache2 with reverse proxy

jamieshaw · 2022-01-17T19:35:39Z

Ditto.

3.12.0 works fine. 4.0.1 returns 1006.
apache2 reverse proxy.

wombatinua · 2022-01-19T12:44:38Z

Same issue.

gmiscione · 2022-01-19T14:53:56Z

Same issue, 3.12.0 works fine and 4.0.1 fails with this error. I'm using apache2 as a reverse proxy. Trying to access the service directly without the proxy seems to work

code-asher · 2022-01-19T16:40:28Z

Sounds like we need to spin up Apache and try to reproduce.

dokastho · 2022-01-19T19:17:57Z

I found a fix on nginx #4763 not sure if there’s an Apache analog for those settings

siklosi · 2022-01-24T09:06:39Z

I'm guessing that line from release log "Web socket compression has been made the default (when supported). This means
the --enable flag will no longer take permessage-deflate as an option" is causing problem. I tried finding analog to "proxy_set_header Accept-Encoding gzip" nginx but I have not yet figured way to make it work.

KaKi87 · 2022-01-24T09:59:49Z

not sure if there’s an Apache analog for those settings

Me neither.

the --enable flag will no longer take permessage-deflate

I cannot find any documentation on said flag, how does it work ?
Would there be a --disable flag too, which would allow apache users to upgrade to 4.x ?

code-asher · 2022-01-26T16:58:47Z

There is no way to disable compression support. It is only used if the browser tells the server that it supports compression. I am not sure issues with compression could cause a 1006 but you could could try stripping `permessage-deflate` from the `Sec-WebSocket-Extensions` header to see if that does anything (this header is how the browser informs the server it supports compression)

code-asher · 2022-01-26T17:52:55Z

Confirming I was able to reproduce:

$ sudo apt install apache2
$ sudo a2enmod rewrite proxy proxy_http proxy_wstunnel
$ sudo vim /etc/apache2/sites-available/000-default.conf # See below
$ sudo apache2
$ code-server --version
4.0.1 735c6da829535969ff7193c79379299e4a1cb9bc
$ code-server --port 8080

<VirtualHost *:80>
  ProxyPass / http://localhost:8080/
  RewriteEngine on
  RewriteCond %{HTTP:Upgrade} websocket [NC]
  RewriteCond %{HTTP:Connection} upgrade [NC]
  RewriteRule ^/?(.*) "ws://localhost:8080/$1" [P,L]
</VirtualHost>

Open localhost in browser
Get 1006 errors

Then to confirm it is a 4.0.1 issue I killed code-server with C-c and installed 3.12.0 with yarn:

$ yarn global add [email protected]
$ ~/.yarn/bin/code-server --version # .yarn/bin is not in my path
3.12.0 4cd55f94c0a72f05c18cea070e10b969996614d2
$ ~/.yarn/bin/code-server --port 8080

Open localhost in browser (Apache is still running from earlier)
Everything works

spenceradolph · 2022-02-01T19:29:33Z

I'm also seeing this bug, with NGINX (using nginx proxy manager). I'm currently trying to fix with #4763 but I'm not having any luck. Latest docker version of code-server.

KaKi87 · 2022-02-01T20:00:11Z

So, will there be a way to disable compression ?

spenceradolph · 2022-02-02T16:51:44Z

Update! The problem was resolved when I tried it from an outside network. So it may have something to do with local routing / proxying differences. Unsure if this is directly related, but my websocket 1006 has been resolved. If I'm local and want to test through the domain I have it on, I'll have to figure out something else.

Websocket 1006 error occurred under these conditions.

nginxproxymanager to code-server (all in docker-compose) for my subdomain.
also exposed code-server to local machine
localhost:8080 to code-server works perfectly
code.mydomain.com gives 1006 error (tried from same host)
code.mydomain.com works perfectly (tried from host outside network)

KaKi87 · 2022-02-02T17:04:52Z

I'm hosting code-server in a dedicated server so I'm definitely in a different network.

chrirauch · 2022-02-03T13:26:09Z

same here. Reverse Proxy with apache. 3.X Versions running fine

jsjoeio · 2022-02-03T19:47:39Z

Thanks for the comments all! We've recognized this is affecting a lot of people. Instead of commenting, please give the initial issue description a "thumbs-up". Thanks!

predkambrij · 2022-03-05T20:14:01Z

FYI: I also see some content security policy complaints in console log. There are also some 1006 errors, that are resolved in newer versions microsoft/vscode-remote-release#6031
Although code server works without apache reverse proxy (http, not https), that's not really clever to run though the internet :)

khanakia · 2022-03-29T06:17:31Z

I am having the same issue with code-server 4.2.0 and Server version: Apache/2.4.18 (Ubuntu)

piit79 · 2022-03-29T06:32:17Z

Yes, same here. Was hoping 4.2.0 would fix the issue but alas.

Is there no workaround at all? My Code Server in Home Assistant has been unusable for many weeks now :(

KaKi87 · 2022-04-03T11:06:47Z

I have 35 enabled apache configurations and am not going to migrate to nginx for this.

However, I am stuck to version 3.x until this bug is solved.

leuit · 2022-04-21T14:46:24Z

For those affected, what version of Apache are you running?

I was able to resolve the issue by updating apache2 to Apache/2.4.53 — however, I believe any version >= Apache/2.4.47 may work.

I was experiencing websocket 1006 errors on Apache/2.4.41

Apache's mod_proxy_wstunnel documentation mention this:

Since httpd 2.4.47, mod_proxy_http can handle WebSocket upgrading and tunneling in accordance to RFC 7230, this directive controls whether mod_proxy_wstunnel should hand over to mod_proxy_http to this, which is the case by default.

Setting to Off lets mod_proxy_wstunnel handle WebSocket requests as in httpd 2.4.46 and earlier.

The grammar is a bit off, but I just assumed Apache was handling the websocket proxy incorrectly and I updated apache.

Hope this helps!

gmiscione · 2022-04-21T15:27:48Z

Thanks @leuit, upgrading to Apache/2.4.53 solved the issue

lplassman · 2022-04-21T15:30:12Z

Thank you @leuit. Upgrading apache2 to 2.4.53 solved the issue for me as well.

lplassman · 2022-04-21T15:31:18Z

Closing this since the issue seems fixed

SCratORS · 2022-04-21T15:35:10Z

@leuit Thanks!

KaKi87 · 2022-04-21T15:35:23Z

I am running apache2 v2.4.38, and can't seem to be able to get a newer version.
apt policy apache2 outouts the following :

apache2:
  Installed: 2.4.38-3+deb10u7
  Candidate: 2.4.38-3+deb10u7

lplassman · 2022-04-21T15:38:34Z

@KaKi87 I had to add this repository, and then reinstall apache to upgrade beyond 2.4.41

sudo add-apt-repository ppa:ondrej/apache2

KaKi87 · 2022-04-21T15:44:19Z

I am running Debian (10), not Ubuntu.

leuit · 2022-04-21T15:53:58Z

@KaKi87

I am running Debian (10), not Ubuntu.

Try using this to add the repository:
curl -sSL https://packages.sury.org/apache2/README.txt | sudo bash -x

Then proceed with upgrading apache2.

Disclaimer: I'm not familiar with Debian. I'm suggesting this based on this guide

KaKi87 · 2022-04-21T16:12:09Z

I successfully upgraded both apache2 and code-server, thanks !

ihowell · 2022-05-05T18:49:05Z

I am using Apache2 version 2.4.53 and cannot use the workaround from @leuit , as disabling mod_proxy_http breaks other services I run on my server. Is there another fix?

KaKi87 · 2022-05-05T19:17:09Z

You don't need to disable that. Once upgraded, it just works.

ihowell · 2022-05-05T19:26:44Z

@KaKi87 Well, it isn't working for me, and neither is v3.12. I am however, running code-server on a separate machine than the apache server and just trying to forward the site. This is the current config I'm using:

<VirtualHost *:80>
	     ServerName vscode.mydomain.com
	     UseCanonicalName on
	     Redirect / https://vscode.mydomain.com
</VirtualHost>

<VirtualHost *:443>
	     ServerName vscode.mydomain.com
	     
	     # setup the proxy
	     ProxyRequests off
	     <Proxy *>
	     	    Order deny,allow
		    Allow from all
	     </Proxy>

	     ProxyErrorOverride On

	     Header set Connection "Upgrade"
	     RequestHeader setifempty Connection "Upgrade"
	     Header set Upgrade "websocket"
	     RequestHeader setifempty Upgrade "websocket"

	     ProxyPass / http://192.168.50.60:8081/
	     ProxyPassReverse / http://192.168.50.60:8081/

	     <Location />
	     	       Order allow,deny
		       Allow from all
	     </Location>

	     SSLEngine on
	     Include /etc/letsencrypt/options-ssl-apache.conf
	     ErrorLog "/var/log/apache2/error.vscode.mydomain.dev.log"
	     SSLCertificateFile /etc/letsencrypt/live/mydomain/fullchain.pem
     	     SSLCertificateKeyFile /etc/letsencrypt/live/mydomain/privkey.pem
</VirtualHost>

And on the other machine I'm using code-server --bind-addr 0.0.0.0:8081 to start it. Any ideas?

leuit · 2022-05-05T19:52:51Z

@ihowell

Are you receiving the same error as OP? Is it not upgrading the websocket connection properly?

You say you're running code-server on a separate machine from apache; can you directly access code-server via the port it's listening on? Depending on the error you're receiving, I think it may just be a communication issue.

If you are indeed receiving the error on the websocket upgrade and can access the frontend webpage of code-server; you'll simply just need to upgrade apache2 as the version does not handle websocket upgrades according to RFC 7230

ihowell · 2022-05-05T19:56:49Z

I am receiving the same 1006 error as op, however I don't know how to tell if the websocket is being upgraded properly. I can access code-server from http://192.168.50.60:8081 in my browser and everything works fine. I just followed instructions and upgraded my server to Apache2 version 2.4.53, but it still isn't working.

Edit: When I run code-server with --log debug, I don't see any errors in the logs.

Edit 2: I now have it working by using the rewriting rules above instead of using my configuration for proxy passing and reverse proxy passing.

leuit · 2022-05-06T00:21:56Z

@ihowell Wonderful! I was going to suggest changing the apache config next; glad you got it resolved!

KaKi87 · 2022-05-06T07:36:01Z

Here's my working config using apache2 v2.4.52 & code-server v4.3.0, using ProxyPass (and RewriteEngine only for WebSocket).

<VirtualHost *:80>
    ServerName code.kaki87.net
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

<VirtualHost *:443>
    ServerName code.kaki87.net
    ProxyRequests Off
    ProxyPass / http://localhost:3090/
    ProxyPassReverse / http://localhost:3090/
    RewriteEngine On
    RewriteCond %{HTTP:Upgrade} =websocket [NC]
    RewriteRule /(.*) ws://localhost:3090/$1 [P,L]
    SSLCertificateFile /home/kaki/.acme.sh/kaki87.net/fullchain.cer
    SSLCertificateKeyFile /home/kaki/.acme.sh/kaki87.net/kaki87.net.key
</VirtualHost>

I always try to make those as light as possible.
I don't think proxying to the local host or to another host on the local network changes anything.
Also, @ihowell, when making future reports about something that isn't working, please document the unexpected behavior as much as you can, because it is as important as documenting your configuration. :)

Drealine · 2022-05-21T21:12:38Z

Upgrade Apache2 to 2.4.52 also solved the issue for me without change anythingelse in apache configuration.

maverjk · 2022-05-30T12:09:52Z

Hi @ihowell can you please share your solution? I have your same issue running last Apache version (2.4.6)

lachlansimpson · 2022-09-16T05:32:51Z

Upgrading apache2 worked for me.

ZhengHui-Z · 2023-09-03T15:42:01Z

This issue has not been resolved in version 4.

mashroomxl · 2023-11-12T09:15:25Z

try nginx config as follows:

location ~ ${your frontend url pattern} {
    # add it if your url is dynamic
    resolver {your DNS server}
    
    # trick here, consult here for Nginx official doc
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection upgrade;
    # trick here, if absent, the server will block the websocket request if the origin is NOT equal to the host
    proxy_set_header Origin http://${your backend host}/;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    
    proxy_set_header Accept-Encoding gzip;
    proxy_cookie_domain ${your backend host} ${your frontend host};
    
    proxy_pass ${your backend url pattern};
}

config above working for me. my scene：

code-server: 4.18.0, running in an container of Kubernetes
nginx:1.9.7, running in an container of another Kubernetes.

jsjoeio added the needs-investigation This issue needs to be further investigated label Jan 13, 2022

bakerkj mentioned this issue Jan 16, 2022

New 4.0 does not work when accessing HA through reverse proxy hassio-addons/addon-vscode#371

Closed

jsjoeio added this to the On Deck milestone Jan 19, 2022

jsjoeio added bug Something isn't working and removed needs-investigation This issue needs to be further investigated labels Jan 26, 2022

jsjoeio modified the milestones: On Deck, On Deck - High Priority Jan 26, 2022

code-asher changed the title ~~Error: Websocket close with status code 1006~~ Web socket fails to connect through Apache Feb 16, 2022

lplassman closed this as completed Apr 21, 2022

leuit mentioned this issue Apr 21, 2022

The workbench failed to connect to the server (Error: WebSocket close with status code 1006) #4443

Closed

haruzept mentioned this issue May 1, 2022

The workbench failed to connect to the server (Error: Time limit reached) hassio-addons/addon-vscode#442

Closed

robinkar mentioned this issue Jun 1, 2022

Accept capitalization in HTTP upgrade header in web microsoft/vscode#150961

Merged

mattmix mentioned this issue Jun 14, 2022

[Bug]: Websocket closed on non-lowercase Upgrade #5264

Closed

4 tasks

jneuhauser mentioned this issue Oct 28, 2022

Extensions using web views not working with self signed certificate #5671

Closed

4 tasks

Lvshujun0918 mentioned this issue Aug 10, 2023

OpenLiteSpeed Websocket & Chatwoot litespeedtech/openlitespeed#352

Open

Web socket fails to connect through Apache #4723

Web socket fails to connect through Apache #4723

Comments

lplassman commented Jan 11, 2022

OS/Web Information

Logs

Screenshot

Notes

JimmyNi commented Jan 12, 2022

jsjoeio commented Jan 13, 2022

code-asher commented Jan 13, 2022 via email

konni commented Jan 17, 2022

jamieshaw commented Jan 17, 2022

wombatinua commented Jan 19, 2022

gmiscione commented Jan 19, 2022 • edited Loading

code-asher commented Jan 19, 2022

dokastho commented Jan 19, 2022

siklosi commented Jan 24, 2022

KaKi87 commented Jan 24, 2022

code-asher commented Jan 26, 2022 via email

code-asher commented Jan 26, 2022 • edited Loading

spenceradolph commented Feb 1, 2022 • edited Loading

KaKi87 commented Feb 1, 2022

spenceradolph commented Feb 2, 2022

KaKi87 commented Feb 2, 2022

chrirauch commented Feb 3, 2022

jsjoeio commented Feb 3, 2022

predkambrij commented Mar 5, 2022

khanakia commented Mar 29, 2022

piit79 commented Mar 29, 2022

KaKi87 commented Apr 3, 2022

leuit commented Apr 21, 2022

gmiscione commented Apr 21, 2022

lplassman commented Apr 21, 2022

lplassman commented Apr 21, 2022

SCratORS commented Apr 21, 2022

KaKi87 commented Apr 21, 2022

lplassman commented Apr 21, 2022

KaKi87 commented Apr 21, 2022

leuit commented Apr 21, 2022

KaKi87 commented Apr 21, 2022

ihowell commented May 5, 2022

KaKi87 commented May 5, 2022

ihowell commented May 5, 2022 • edited Loading

leuit commented May 5, 2022 • edited Loading

ihowell commented May 5, 2022 • edited Loading

leuit commented May 6, 2022

KaKi87 commented May 6, 2022

Drealine commented May 21, 2022

maverjk commented May 30, 2022

lachlansimpson commented Sep 16, 2022

ZhengHui-Z commented Sep 3, 2023

mashroomxl commented Nov 12, 2023

gmiscione commented Jan 19, 2022 •

edited

Loading

code-asher commented Jan 26, 2022 •

edited

Loading

spenceradolph commented Feb 1, 2022 •

edited

Loading

ihowell commented May 5, 2022 •

edited

Loading

leuit commented May 5, 2022 •

edited

Loading

ihowell commented May 5, 2022 •

edited

Loading