Response is being truncated when it includes multibyte chars

[ryan-plated]

it appears bytesize returns the wrong length

em-http-request/lib/em-http/client.rb

Line 185 in bcba7b6

head['content-length'] = body.bytesize

oesmith/puffing-billy#233

[igrigorik]

Thanks for reporting this. Do you have an example where this fails for em-http? The thread you linked to talks about Chrome receiving the response.. it's unclear to where if/where em-http is involved there.

[nicedawg]

I found this reported issue while looking into oesmith/puffing-billy#193 -- specifically the problem the described in this comment, and the subsequent discussion.

My primitive understanding of the puffing-billy code is that it makes a request (acting as a proxy on behalf of a browser like chrome), ultimately using EventMachine::HttpRequest to send the request and getting the response from EventMachine::HttpClient. (I could be misunderstanding it, though.)

In this comment somebody created a simple reproduction of the issue I'm interested in.

Of course, this may not help with oesmith/puffing-billy#233, but I hope it at least sheds (correct) light on how puffing-billy uses em-http-request.

[igrigorik]

@nicedawg hmm, thanks for the link. On first pass, I don't think these two issues are related. It's also not clear to me if em-http is the culprit in that thread, but that's a different discussion.

@ryan-plated per your earlier comment...

require 'json'

hash = { localidade: 'Niterói', bairro: 'Icaraí' }
json = JSON.generate(hash)

puts json
puts json.bytesize

ruby test.rb
{"localidade":"Niterói","bairro":"Icaraí"}
44

That looks correct to me. Assuming the payload is not being converted to some other encoding before it's passed to em-http, bytesize should do return the right value?

[hawkins]

The original puffing-billy issue which sparked this issue has since been closed with a workaround that doesn't even point to a change in this repo. Is there still an underlying issue with EM HTTP and multibyte character codes?

[ryan-plated]

I think it is fine to close the comment now. I haven't run in to it recently, Thanks for following up.