-
Notifications
You must be signed in to change notification settings - Fork 7.3k
HTTP request ends too early #6143
Comments
How is the connection terminated at the TCP level? (tcpdump or wireshark will tell you.) As you're not getting an error, I suspect that it's a regular FIN/ACK sequence? |
@bnoordhuis I don't have the time to debug it as of now. If really necessary, I will do it in the weekend. |
'If really necessary' depends on how important you think it is that this gets looked into. I have no easy way to replicate your set-up. I can set up a netem qdisc but that probably doesn't work quite like Apple's NLC so then we won't be testing the same thing. On the subject of FIN/ACK: if that's indeed what gets sent over the wire (be it a virtual or a physical wire), then there is no bug, just business as usual. |
Indeed there's a FIN sent over the wire, with a fatal error:
Wireshark detected that the FIN packet occurred because of an error, could node do the same and fire an |
It's a protocol error but not necessarily a fatal one. Corrupted packets and retransmits happen all the time. Apart from the PSH flag, it looks like regular connection termination. Not that the PSH flag is harmful, it's just mildly uncommon.
Not if the operating system doesn't report it. Which it probably won't because, as mentioned, it's not a fatal error. Sorry, but I don't see anything here that suggests that it's a node.js issue. Thanks for following up though. |
@bnoordhuis I've tried to download with curl https://codeload.github.com/jashkenas/backbone/tar.gz/1.0.0 > bleh.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
12 8302k 12 1003k 0 0 14293 0 0:09:54 0:01:11 0:08:43 25325
curl: (18) transfer closed with 7474745 bytes remaining to read I know that some servers might not send the content-length header, but for those that send we could add the same check and fire an |
I think so. The core http library is deliberately low-level. Not as low-level as I would like but that's another story. Throwing errors on bad Content-Length headers would also be quite disruptive because it's not that uncommon. I think most browsers regard it as a size hint, nothing more. |
@bnoordhuis after talking with an engineer of GitHub, I think that nodejs should be smarter in the way he handles the transfer:
Their servers are indeed sending chunked responses. Could node detect if the connection was closed in the middle of a chunk and fire an |
It does, it emits an 'aborted' event on the response object. |
@bnoordhuis I've search the document for the |
You're right, sorry. 'aborted' is an internal-ish event, you should also get the documented 'close' event. |
If the aborted event is not a public API, then how is it possible to distinguish a truncated chunked download from a successful one? Won't both emit a close event? |
@jfirebaugh @satazor @bnoordhuis I know it's been a while, but to answer your questions and anyone else who finds this ticket via Google:
So no, there's no reliable way to protect against truncation of plain-old HTTP. But on the other hand, HTTPS is on the rise and has two mechanisms that protect very well against truncation. And things like chunked transfers can be used on top of HTTPS to get even more security. Reference: https://tools.ietf.org/html/rfc5246#section-7.2.1 |
It looks like the request is firing the
end
event too early, even before the whole download ends. This happens only on unstable connections. I've detected this issue while investigating the causes of bower/bower#824.This simple script, without 3rd party code, is able to replicate the issue with the following NLC (network link conditioner) settings:
Output of the program with those NLC settings:
So basically,
end
event gets fired too early. If I switch off NLC everything is fine:I was expecting the
end
event to be fired only when the download completes, or at very last, anerror
event.The text was updated successfully, but these errors were encountered: