Issues reaching api.teletype.atom.io behind a proxy

[as-cii]

Originally reported by @jrschumacher on Nov 15th 2017

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Followed all applicable steps in the debugging guide
  • Checked the FAQs on the message board for common solutions
  • Checked that your issue isn't already filed?

Description

When trying to start teletype in an environment which sits behind a firewall I get the following error:

Failed to initialize the teletype package
Establishing a teletype connection failed with error: Failed to fetch 

This is a result of this query:

Request URL:https://api.teletype.atom.io/protocol-version
Request Method:GET
Status Code:407 Proxy Authentication Required (from disk cache)

Note: I am running CNTLM which interfaces with the proxy. I have PROXY, HTTP_PROXY, HTTPS_PROXY, proxy, http_proxy, https_proxy configured. My local network settings are all configured to work with the proxy. Additionally apm is configured to work with my proxy settings.

Steps to Reproduce

1. Enable the teletype plugin behind a proxy

Expected behavior:

I expect the plug to enable without an error message

Actual behavior:

I receive the error message described above.

Reproduces how often:

100%

Versions

Atom    : 1.22.0
Electron: 1.6.15
Chrome  : 56.0.2924.87
Node    : 7.4.0
apm  1.18.8
npm  3.10.10
node 6.9.5 x64
python 2.7.10
git 2.14.1

macos 10.12.6 (16G29)

Additional Information

> apm config list
; cli configs
globalconfig = "/Users/rschumacher/.atom/.apm/.apmrc"
user-agent = "npm/3.10.10 node/v6.9.5 darwin x64"
userconfig = "/Users/rschumacher/.atom/.apmrc"

; environment configs
proxy = "http://localhost:3128/"

; project config /Users/rschumacher/.npmrc
https-proxy = "http://localhost:3128/"
proxy = "http://localhost:3128/"
strict-ssl = false

[jrschumacher]

@as-cii thanks for creating this

[jrschumacher]

@as-cii is there any other information or tests I could provide you?

[as-cii]

You are very welcome, @jrschumacher. Thank you for helping us investigate this problem.

So my current thinking is that this might be a problem with the proxy credentials, as the 407 error is indicating. Perhaps Electron is trying to show an authentication dialog but Atom is preventing that? Electron exposes an app.on('login') API that we may use for that.

A few things that might be worth trying to get a sanity check:

• Can you reach https://api.teletype.atom.io via curl? e.g. curl https://api.teletype.atom.io/protocol-version
• If you open the DevTools on Atom, what happens if you call fetch("https://api.teletype.atom.io/protocol-version")?

Depending on what the output of the above commands is, we should maybe experiment with the aforementioned Electron API. Thanks for your continued help on this, @jrschumacher! ⚡️

[bac]

1. Yes:

curl https://api.teletype.atom.io/protocol-version
{"version":4}

fetch("https://api.teletype.atom.io/protocol-version")
Promise {[[PromiseStatus]]: "pending", [[PromiseValue]]: undefined}
VM1285:1 GET https://api.teletype.atom.io/protocol-version net::ERR_INSECURE_RESPONSE
(anonymous) @ VM1285:1
index.html:1 Uncaught (in promise) TypeError: Failed to fetch

So teletype does not appear to be using the cert I've installed for our proxy. For pip to work, for example, I have to add configuration like:

cert = /etc/ssl/certs/ca-certificates.crt

Does teletype need something similar?

[as-cii]

Thanks for the feedback, @bac! Yes, it is possible that Electron needs to be configured to read that particular certificate. I think there are some APIs to allow that (https://electronjs.org/docs/api/session#sessetcertificateverifyprocproc), but we would need to experiment and see if they offer what we need for this issue.

Have you got Chrome installed on your machine? If so, do you remember configuring the certificate in Chrome too?

[krzkowalczyk]

I got the same issue, but I've managed to resolve it.
Electron uses chromium ca trust store which uses NSS Shared DB. Once I've added certificates to this store teletype succesfully connected to api.

More infor here : https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_cert_management.md

[jasonrudolph]

@krzkowalczyk: Thanks for sharing this solution! 🙇

@bac @jrschumacher: Can you give this approach a try and let us know if it resolves this issue for you?

[kevteljeur]

Hi, I'm still having trouble with this. I tried making my own proxy (to... proxy through the proxy, if you follow, with a valid user/pass) which worked for talking to Teletype server, but not for Pusher (it didn't work entirely, only partially). I was curious more than anything. Anyway, now I'm experimenting with adding a proxy option to the Teletype config UI, and getting Teletype and Pusher to use that.

I think that fetch will not work, since that will be stopped by the proxy, but maybe there's another way around it.

[jasonrudolph]

@kevteljeur: Thanks for looking into this. If you find a solution that resolves the issue for you, or if you know of a way for us to reproduce the issue, please let us know.

[kevteljeur]

Actually, on closer inspection, I've hit a wall. I figured out how I could work with fetch to basically shim proxy support into the fetch call, so that it would function through a proxy (untested) but... Pusher is the web version, which doesn't have proxy support, and not the Node version, which does. Not sure where I can go from here.

Teletype would be a great tool for what I'm doing with other devs, but that damn proxy issue.

Updated to add what I'm trying to achieve: An additional field for specifying a proxy to use for Teletype (I looked at reading it from process.env, but that would be an assumption, especially if the developer had set up a local instance of Teletype server to use).

[kevteljeur]

So I've been trying to look at how a request via a proxy is structured; I don't know a lot about this, so I can't make effective use of the time I have available for it (I spent some time studying the npm Request module yesterday). Does anyone know if a request can be restructured for sending though a proxy in clientside JavaScript? If so, then it is possible, by modifying the fetch request (to Teletype server) and setting headers and URLs for Pusher.js. Combined with my suggested proxy field above, this is doable. I just don't know what modifications to make to the requests for that to be tested.

Also, I see that Slack (using Electronjs for their app) is able to do it. But their app is not open.

Does anyone know enough about proxies to be able to assist? I'd be happy to code up and field test any proposals. Teletype is a great feature, and I can't imagine that my team will be the only one benefitting from using Teletype behind an authenticated proxy.

[jrschumacher]

@jasonrudolph This only works for Linux it seems. I'm running on OSX High Sierra.

[jrschumacher]

Have you got Chrome installed on your machine? If so, do you remember configuring the certificate in Chrome too?

@as-cii For Chrome I didn't need to add a custom cert. It uses the cert in the Keychain Access.app. In Firefox it is required to install the custom cert as they don't use the system certs.

[kevteljeur]

I've not gotten any further, partly due the inherent madness of trying to write a proxy to make this work. Is there no way to get Atom to play nicely with a proxy? I see that Slack (also an Electron app) does it...

Hypothesis: When using window.fetch and they get a 407 response, they then ask for the proxy details, and store them.

If Atom did this, or took them from the apm configuration, then it could handle this too. Does this sound reasonable to anyone?

[lihe6666]

I met the same problem.

mac os v10.13.1
atom v1.23.3
teletype v0.6.0

solution:

shutdown dnsmasq, shutdown shadowsocks, modify dns, 127.0.0.1 this row delete. reinstall OK.

I hope could help you.

[jasonrudolph]

I see that Slack (also an Electron app) does it...

Hypothesis: When using window.fetch and they get a 407 response, they then ask for the proxy details, and store them.

@kevteljeur: Thanks for mentioning that. 🙇 That definitely sounds like it's worth investigating.

To investigate that approach, we'll first need to find a way to reproduce the issue (#236 (comment)). So far, we haven't had any luck reproducing it. 😥 With that in mind, I'll make a brief plea for help on that front. 😇

---

📣🆘 Request for help reproducing this issue 🙏😅

To help us diagnose the issue, if anyone here is able to provide a Dockerfile or a Vagrant image that we can then use to reproduce this locally, that would be a huge help.

Alternatively, if anyone can provide detailed instructions for reproducing this issue by starting with a clean installation of a particular operating system and then setting up a proxy step-by-step, that would also be a big help.

Once we have a way to reproduce the issue, we should have a much better chance of resolving it. 🤞

[kevteljeur]

Funny enough, I was just think that I should check in and see if anything made it in to 0.7.0 on this. I should be able to help you there; I've tested window.fetch and Atom cannot handle a proxy, the console is full of failed requests (HTTP 407 status). So it isn't particularly a Teletype issue, but Teletype needs to be able to get past a proxy and Atom cannot help it.

Basically, the headers for a window.fetch request (and presumably for Pusher also) need to be set correctly to work. window.fetch doesn't respect browser settings the way that, for example, XMLHttpRequest does. It's a lot closer to the metal so you have to detect and handle that sort of stuff. I don't know how browsers do that, but they've clearly replicated it in Slack (even with a cute Slack-branded UI for proxy username/password). Atom could read the APMRC settings and get the proxy settings from that. I've had limited success in moving beyond a 407 response but I'm not so experienced with it and I have to avoid my timesheet filling up with 'circumventing the proxy' ;-)

Maybe if you set up a proxy on your development machine (Node.js, for example) with a password and then set your system to use it - if your browser or Slack challenge you for the credentials, then you're in business.

[jasonrudolph]

Maybe if you set up a proxy on your development machine (Node.js, for example) ... then you're in business.

You might be overestimating my proxy skills. 🙈:wink:

Can you point me toward some detailed instructions for setting up a proxy in a way that will reliably reproduce this issue?

[kevteljeur]

Good question. I'll see tomorrow. I'm not very knowledgeable about proxies myself, but something like this: https://github.com/nodejitsu/node-http-proxy using auth (Basic auth) would do it. I'll see tomorrow or Thursday if I can reproduce the effect using that Node library and if so then we'll be on the way.

[jasonrudolph]

It looks like some of Atom's core functionality (e.g., checking for software updates) is also facing problems when using Atom behind a proxy:

• Problems installing packages, themes, and updates behind a proxy (Problem installing Packages/ Themes and Updates (Corporate SSL Proxies) atom#8465, Cannot install packages behind proxy: "tunneling socket could not be established, cause=socket hang up" apm#611)
• Problems checking for Atom updates behind a proxy (Check for atom update behind proxy atom#16240)
• Problems fetching package details behind a proxy (Fix package detail view when behind a proxy settings-view#1010)

With that in mind, I suspect that resolving these proxy issues for Teletype users will likely require an update in atom/atom.

[kevteljeur]

Well, it's important to separate between the apm use of the proxy (which is probably not going to change, because it works like npm) and the UI layer. But, that being said, the UI layer is obviously a problem too. Since I've just squeaked past a particular deadline, I can spare a minute to test this.

[kevteljeur]

When you ask the right question of Google, you may receive: https://discuss.atom.io/t/window-fetch-not-following-system-network-configuration/39139

[Jargon4072]

"I got the same issue, but I've managed to resolve it.
Electron uses chromium ca trust store which uses NSS Shared DB. Once I've added certificates to this store teletype succesfully connected to api.

More infor here : https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_cert_management.md"

@krzkowalczyk can you please explain how u done it( adding certificates) and what certs are required?

[Jargon4072]

I am getting this error "net::ERR_TUNNEL_CONNECTION_FAILED". I have configured proxy everywhere and apm install and curl is working fine for teletype.

[jasonrudolph]

/xref microsoft/vscode#22369, which added proxy support in VS Code

[kevteljeur]

Well then.

[kevteljeur]

atom/atom#14866 - I tried this but it made no difference for me.

[kevteljeur]

Hi, any news on this, or still dependent on core?

[jasonrudolph]

Hi, any news on this, or still dependent on core?

@kevteljeur: It's probably dependent on core. It's definitely dependent on having detailed steps for reproducing the issue. 😇

[jrschumacher]

@jasonrudolph do you have access to adding an event listener (to the app resource) within a plugin? In other electron apps I have to prompt the user for login and password when the login event is called.

Ref: https://github.com/egoist/devdocs-desktop/pull/72/files#diff-2018087f584c4398b5c3a23fc0e5f9dbR83

You'll also see this behavior in the the latest VSCode.

[jasonrudolph]

@jrschumacher: Thanks for the idea. That might do the trick, but we'd still need a way to reliably reproduce this issue so that we can verify the fix.

[jrschumacher]

Yes I understand. Unfortunately in my corporate network I’m not involved with its configuration, but I’ll do some research and see if I can find a open source vpn which simulates this.

On Thu, Apr 12, 2018 at 14:23 Jason Rudolph ***@***.***> wrote: @jrschumacher <https://github.com/jrschumacher>: Thanks for the idea. That might do the trick, but we'd still need a way to reliably reproduce this issue <#236 (comment)> so that we can verify the fix. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#236 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC11WyDse8xON28KNz7mgOVHSyBElzhks5tn6m_gaJpZM4QiWQv> .

-- *Ryan Schumacher* software development [image: tep.io] <http://tep.io/> [image: tep.io on Facebook] facebook <https://www.facebook.com/tepdotio> [image: tep.io on LinkedIn] linkedin <http://www.linkedin.com/company/38pages> cell 205.515.0911 office 205.291.2520

[jdbellamy]

I was able to get teletype working by starting atom with the --proxy-server argument.

Trying to pass my proxy credentials via the URL (e.g. http://user:pass@proxy-server:3128) was resulting in net::ERR_NO_SUPPORTED_PROXIES errors, so I stood up a local squid container as an intermediary pass-through proxy.

I was then able to start atom with the following command:

atom . --proxy-server=http://localhost:3128

[kevteljeur]

Checking in. I haven't looked at this in a while but my need for it is no less. Has anyone had any success? Or are we still tied to getting core Atom to play nicely with network proxies?

I did take a look at the Code changes (referenced above) but I have to say that I didn't understand them well enough.

[ghost]

I can reproduce this; I have env variables set:

http_proxy=http://localhost:8888
https_proxy=http://localhost:8888

As well as .apmrc:

proxy=http://localhost:8888/
https-proxy=http://localhost:8888/
strict-ssl=false

Atom seems to not recognize these.

Starting atom with "--proxy-server=http://localhost:8888" works.

apm 1.19.0
npm 3.10.10
node 6.9.5 x64
atom 1.28.2
python 3.6.3
git 2.18.0.windows.1
visual studio
Microsoft Windows [Version 10.0.14393]

I'm behind a corporate proxy, that I forward with cntlm.

[kevteljeur]

Any word on this?