Handle slow networks more gracefully

[qian2501]

• Web Browser: Chrome 87.0.4280.88
• Local OS: Windows 10 build 1709 16299.1087
• Remote OS: CentOS 8.2.2004
• Remote Architecture: x86_64
• code-server --version: 3.7.4 11f5378

[Problem]
Extension does not activate, even after “active extension...” showed up at status bar. This include git extension shipped with vscode, which cause source control panel says “No source control providers registered” and “git.path” setting grayed out.

[Log]
No content in code-server-stderr.log

code-server-stdout.log

[2020-12-05T07:43:31.503Z] info  code-server 3.7.4 11f53784c58f68e7f4c5b3b8dae9407caa41725b
[2020-12-05T07:43:31.504Z] info  Using user-data-dir ~/.local/share/code-server
[2020-12-05T07:43:31.512Z] info  Using config file ~/.config/code-server/config.yaml
[2020-12-05T07:43:31.512Z] info  HTTP server listening on http://127.0.0.1:8080 
[2020-12-05T07:43:31.512Z] info    - Authentication is enabled
[2020-12-05T07:43:31.512Z] info      - Using password from ~/.config/code-server/config.yaml
[2020-12-05T07:43:31.512Z] info    - Not serving HTTPS
[2020-12-05T07:43:38.334Z] debug forking vs code...
[2020-12-05T07:43:38.536Z] debug setting up vs code...
debug vscode got message from code-server {"type":"init"}
debug vscode got message from code-server {"type":"socket"}
debug vscode New connection {"token":"dd205da5-c5fa-41cc-8576-de1f56b0b552"}
[2020-12-05T07:43:42.893Z] debug got latest version {"latest":"3.7.4"}
[2020-12-05T07:43:42.893Z] debug comparing versions {"current":"3.7.4","latest":"3.7.4"}
debug Creating terminal {"id":0,"terminals":0}
debug Resolved shell launch configuration {"id":0}
debug Resolved terminal environment {"id":0}
debug Created terminal {"id":0}
debug vscode got message from code-server {"type":"socket"}
debug vscode New connection {"token":"8b7f3062-ff75-4b27-a080-a0ccac3ce0a0"}
debug Terminal bound {"id":0}
debug Terminal replaying {"id":0}
debug vscode got message from code-server {"type":"socket"}

remoteagent.log

[2020-12-05 15:43:55.114] [remoteagent] [error] [File Watcher (chokidar)] Inotify limit reached (ENOSPC)
[2020-12-05 15:43:55.114] [remoteagent] [error] [File Watcher (chokidar)] Inotify limit reached (ENOSPC)

telemetry.log

[2020-12-05 15:43:38.579] [telemetry] [info] The below are logs for every telemetry event sent from VS Code once the log level is set to trace.
[2020-12-05 15:43:38.579] [telemetry] [info] ===========================================================

[code-asher]

Hmm the only thing that looks a bit suspect is the inotify limit but I'm not sure if it could cause this issue. It might be worth raising the limit to check. Could you post the logs from the browser console as well? Thank you!

[qian2501]

Hmm the only thing that looks a bit suspect is the inotify limit but I'm not sure if it could cause this issue. It might be worth raising the limit to check. Could you post the logs from the browser console as well? Thank you!

Hi, I doubled checked in my browser, due to sec policy I will not be able to post them. But I saw something is stuck. There is a request using wss protocol keeps pending, that request has type=Management, and there is another repeating wss request with type=ExtensionHost, both of them have a 101 status code. And nginx log reveals the same.

Oh, one more thing, it loaded once on my iPad with Safari, so it’s not a stable repeating issue any more, but issue probability is still very high.

[nhooyr]

Might be related to #2388

[qian2501]

I found something more in nginx log.
The pending request:

[10/Dec/2020:11:32:03 +0800] "GET /?type=Management&reconnectionToken=6d8536ff-9ad7-4ccf-be10-115919d8178d&reconnection=true&skipWebSocketFrames=false HTTP/1.1" 101 4455470 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" "-"

The repeating request:

[10/Dec/2020:11:36:01 +0800] "GET /?type=ExtensionHost&reconnectionToken=3ba472ef-3afb-408b-b807-8923800ff51d&reconnection=true&skipWebSocketFrames=false HTTP/1.1" 101 199 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" "-"

And a lot entry in error.log with something like this:

connect() failed (111: Connection refused) while connecting to upstream... upstream: "http://[::1]:8080/...

Basically every request have an entry like this.

Could this have something to do with privilege? I installed code-server using root while running it with another sudo-able user.

[qian2501]

Well, I have it loaded on my windows machine today. It’s getting confusing.

[ghost]

I have the same issue and found the following situation:

Local environment:
Windows 10 Pro 2004 Build 19041.685
Docker desktop 3.0.0
Docker engine 20.10.0
WSL2 Ubuntu 18.04LTS
codercom/code-server:3.8.0 > code-server v3.8.0

Server environments AWS:
Ubuntu 18.04 and 20.04
Docker engine 20.10.0
codercom/code-server:3.8.0 > code-server v3.8.0

Server environments Oracle cloud:
Ubuntu 18.04 and 20.04 also with Oracle linux 7.9
Docker engine 20.10.0
codercom/code-server:3.8.0 > code-server v3.8.0

Internet connections:
ADSL - 10Mbps/1Mbps
LTE - 25Mbps/30Mbps

Browsers:
Google Chrome 87.0.4280.88 (64-bit)
Firefox 84.0.1 (64-bit)

Running coder locally:
everything works fine with both browsers, extensions load as expected and git integration works.

Running coder in AWS/Oracle environments with LTE:
everything works fine with both browsers, extensions load as expected and git integration works.

Running coder in AWS/Oracle environments with ADSL:
Problem happened here with both browsers, in my case this internet connection is slow most of the time sometimes with high latency, extension didn't load and for that reason git integration shows "No source control providers registered".

For me the situation is only happening with ADSL connection, so I can assume is something related to the speed/latency/stability of the connection.

Error in browsers log:

[Service Worker] registered register.ts:16:14
The web worker extension host is started without an iframe sandbox! webWorkerExtensionHost.ts:113:13
unreachable code after return statement client.js:23255:2
 WARN Workbench did not finish loading in 10 seconds, that might be a problem that should be reported. log.ts:191:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] starting reconnecting loop. You can get more information with the trace log level. log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] waiting for 5 seconds before reconnecting... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] resolving connection... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] connecting to testcoder.robertos.page:80... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] reconnected! log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] starting reconnecting loop. You can get more information with the trace log level. log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] waiting for 5 seconds before reconnecting... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] resolving connection... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] connecting to testcoder.robertos.page:80... log.ts:185:11
 INFO [remote-connection][ExtensionHost][eb254…][reconnect] reconnected! log.ts:185:11
Error received from starting extension host extensionHostManager.ts:73:12
timeout extensionHostManager.ts:74:12
  ERR ENOPRO: No file system provider found for resource 'git:/home/coder/projects/Test/public/sh/ubuntu_install.sh?%7B%22path%22%3A%22%2Fhome%2Fcoder%2Fprojects%2FTest%2Fpublic%2Fsh%2Fubuntu_install.sh%22%2C%22ref%22%3A%22~%22%7D'
    withProvider fileService.ts:122```

[code-asher]

Hmm yeah it does look like it's timing out. Maybe we should remove the timeout or make it larger.

[qian2501]

Well, it is confirmed a network issue. By the time I submitted this issue I was on a biz trip in some less advanced city. LTE was not that good there. Everything works fine back home.

[imjakey]

I have the same issue, I have tried deploy in aliyun and qcloud，both of them have a fast ping，but the wss requests like
"/?type=Management&reconnectionToken=6d8536ff-9ad7-4ccf-be10-115919d8178d&reconnection=true&skipWebSocketFrames=false" timeout almost everytime，but everything works fine when it run on another host in my local area network。

[SBBTD]

I have the same issue. 1Mbps
timeout 60 seconds

log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] resolving connection...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] connecting to [hideUrl]...
log.ts:284  INFO [remote-connection][Management   ][ba346…][reconnect] reconnected!

[lettore]

I'm on satellite connection at sea and I have the same problem, it should be nice if it would be possible to manage degraded networks. Not always it's possible to connect to the server on a lan.

[jsjoeio]

I'm on satellite connection at sea

Probably one of the coolest comments I've seen on GitHub. I'm sure there's a good story there.

Anyways, yes, it would be nice! We have some other priorities but would love to improve this UX!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no activity occurs in the next 5 days.

[code-asher]

Unfortunately I believe this will need to be fixed upstream, unless we want to try patching their module loader.