get blocks forever with Transfer-Encoding: chunked and stream=true

[samoconnor]

I have a simple Node.js server that streams data forever:

var http = require('http');

http.createServer(function(req, res) {

  setInterval(function() {
    res.write('Hello\n');
  }, 1000);

}).listen(8000);

Connect from REPL to see what headers are returned...

julia> r = HTTP.get("http://127.0.0.1:8000"; stream = true)
HTTP.Response:
"""
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked
Date: Wed, 30 Aug 2017 00:34:25 GMT

Note that because of #85, this blocks forever, but we can see that the server has set Transfer-Encoding: chunked.

Working around #85, we can read the response one line at a time...

julia> r = HTTP.get("http://127.0.0.1:8000"; stream = true) ; nothing
julia> for l in eachline(HTTP.body(r))
           println(l)
       end
Hello
Hello
Hello
Hello
...

However, this only works with the following patch applied:

--- a/src/parser.jl
+++ b/src/parser.jl
@@ -1300,7 +1300,7 @@ function parse!(r::Union{Request, Response}, parser, bytes, len=length(bytes);
     @debug(PARSING_DEBUG, @__LINE__, "exiting maybe unfinished...")
     @debug(PARSING_DEBUG, @__LINE__, ParsingStateCode(p_state))
     b = p_state == start_state || p_state == s_dead
-    he = b | (p_state >= s_headers_done)
+    he = b | (p_state >= s_chunk_size_start)
     m = b | (p_state >= s_body_identity_eof)
     return errno, he, m, String(bytes[p:end])

Without this patch, the call to get never returns, because headerscomplete is never true here:
https://github.com/JuliaWeb/HTTP.jl/blob/master/src/client2.jl#L257

[quinnj]

should be closed on master now; thanks for the patch.