Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

undici with ProxyAgent fails with 400 where curl and node-fetch with https-proxy-agent succeeds #2542

Closed
nugmanoff opened this issue Dec 25, 2023 · 8 comments · Fixed by #2556
Closed

undici with ProxyAgent fails with 400 where curl and node-fetch with https-proxy-agent succeeds #2542

nugmanoff opened this issue Dec 25, 2023 · 8 comments · Fixed by #2556
Labels
bug Something isn't working

Comments

@nugmanoff
Copy link

nugmanoff commented Dec 25, 2023 •
edited
Loading

Bug Description

My proxy setup is not working with undici, but it succeeds with node-fetch + https-proxy-agent and it also succeeds with plain curl.

Reproducible By

Here is the reproduction repo: https://github.com/nugmanoff/undici-proxy-reproduction/tree/main

In the given repo, if you run node node-fetch.js it will fail with 407, which is expected, because I have emptied the strings in credentials.js (I can give some credentials in DM if someone from the maintainers would be willing to test with real credentials, so that request succeeds with 200).

But if you run node undici.js it will fail with [AbortError]: Proxy response (400) !== 200 when HTTP Tunneling

Even if you run it like curl -x https://:@us.smartproxy.com:10001 -L https://ip.smartproxy.com (notice the empty username and password combo) it will fail with curl: (56) CONNECT tunnel failed, response 407 which matches with node-fetch behavior

Expected Behavior

Expected behavior is that proxy setup works with undici given that it works in node-fetch and curl.

In my understanding receiving 407 on request via proxy with empty credentials is expected behavior here, because it signifies that the request is actually made through proxy. But with undici it gives 400, which is really strange and to me looks like request is not being sent properly in the first place (?).

Maybe I am missing out on some 'standard' which node-fetch and curl doesn't comply with, but undici does – so it fails (deliberately) where others don't. But anyway it is not expected given that it is pretty basic setup. (Btw it also works with axios)

Logs & Screenshots

Full undici failed log

/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/index.js:115
      Error.captureStackTrace(err, this)
            ^

TypeError: fetch failed
    at fetch (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/index.js:115:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///Users/nugmanoff/Desktop/undici-proxy/undici.js:12:20 {
  cause: DOMException [Error]: Request was cancelled.
      at new DOMException (node:internal/per_context/domexception:53:5)
      at makeAppropriateNetworkError (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/response.js:448:38)
      at httpNetworkFetch (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/index.js:1907:14)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async httpNetworkOrCacheFetch (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/index.js:1594:29)
      at async httpFetch (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/index.js:1168:33)
      at async /Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/index.js:601:16
      at async mainFetch (/Users/nugmanoff/Desktop/undici-proxy/node_modules/undici/lib/fetch/index.js:585:16) {
    cause: RequestAbortedError [AbortError]: Proxy response (400) !== 200 when HTTP Tunneling

Environment

Node v18.18.0
Undici v6.2.1
Node-fetch v3.3.2
Https-proxy-agent v7.0.2
@nugmanoff nugmanoff added the bug Something isn't working label Dec 25, 2023
@KhafraDev
Copy link
Member

cc @RafaelGSS

@mcollina
Copy link
Member

A PR that fixes this would be highly welcomed, thanks!

@nugmanoff
Copy link
Author

@mcollina @KhafraDev is it safe to assume that it is a confirmed [by you] bug? I am double-checking so that I am sure that I didn't miss anything in proxy setup and my reproduction is valid.

@mcollina
Copy link
Member

At a first look, it seems valid.

@metcoder95
Copy link
Member

metcoder95 commented Dec 26, 2023 •
edited
Loading

I've been playing with it and it seems the request is badly formatted, the server is replying with 400 + HTML content saying Your browser sent an invalid request.

I'm pretty unsure what would be failing as we attach to the common Basic Auth spec.

Output:

Status code: 400 <html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>

@RafaelGSS
Copy link
Member

I'm looking to it.

@RafaelGSS
Copy link
Member

Found the bug. PR is coming

@RafaelGSS RafaelGSS mentioned this issue Dec 28, 2023
Merged
@RafaelGSS
Copy link
Member

#2556

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

lib: fix Host header when CONNECT ProxyAgent nodejs/undici
5 participants
@mcollina @metcoder95 @nugmanoff @RafaelGSS @KhafraDev