net/http: Content-Length is not set in outgoing request when using ioutil.NopCloser #34295
Comments
|
I'm not sure if the |
|
Hello there @johansja, thank you for filing this request and welcome to the Go project! So, ioutil.NopCloser swallows its underlying reader and we can no longer get access to it directly as per even its signature https://golang.org/pkg/io/ioutil/#NopCloser and thus we can't guess its ContentLength currently, unfortunately! Therefore it turns into the case of streaming content that is continuously flowing in until it decides to send EOF on its own. that's why you see that If you pass in reqBody as bytes.BufferString(
Here is a tip to make your test self contained and even speed up your development making it self contained and more fun, Go's net/http/httptest has a package we can use to create servers and echo our responses to us or even simpler to not have to spin up a server you can pass in an http.RoundTripper that mocks that echo backend and just dumps out the response for inspection and here is the code With httptest.Serverhttps://play.golang.org/p/RghMmV5dhNKor inlined below: package main
import (
"bytes"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"net/http/httputil"
)
func main() {
cst := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
reqBlob, _ := httputil.DumpRequest(r, true)
w.Write(reqBlob)
}))
defer cst.Close()
reqBody := ioutil.NopCloser(bytes.NewBufferString(`{}`))
req, err := http.NewRequest("POST", cst.URL, reqBody)
if err != nil {
log.Fatalf("Cannot create request: %v", err)
}
res, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatalf("Cannot do: %v", err)
}
defer res.Body.Close()
resBody, err := ioutil.ReadAll(res.Body)
if err != nil {
log.Fatalf("Cannot read body: %v", err)
}
log.Printf("Response Body:\n%s", resBody)
}With a custom roundtripper aka doesn't make an RPChttps://play.golang.org/p/aUCgAHYWbRS or inlined below: package main
import (
"bytes"
"io/ioutil"
"log"
"net/http"
"net/http/httputil"
)
type echoRoundTripper int
func (ert *echoRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
reqBlob, _ := httputil.DumpRequestOut(req, true)
res := &http.Response{
Status: "200 OK",
StatusCode: 200,
Body: ioutil.NopCloser(bytes.NewReader(reqBlob)),
}
return res, nil
}
func main() {
reqBody := ioutil.NopCloser(bytes.NewBufferString(`{}`))
req, err := http.NewRequest("POST", "http://localhost:80/post", reqBody)
if err != nil {
log.Fatalf("Cannot create request: %v", err)
}
client := &http.Client{Transport: new(echoRoundTripper)}
res, err := client.Do(req)
if err != nil {
log.Fatalf("Cannot do: %v", err)
}
defer res.Body.Close()
resBody, err := ioutil.ReadAll(res.Body)
if err != nil {
log.Fatalf("Cannot read body: %v", err)
}
log.Printf("Response Body:\n%s", resBody)
}I shall close this issue as working as intended, but perhaps we could examine documenting the cases for which ContentLength cannot be inferred directly. ** disclaimer: if we come to a consensus, we can try hard to inspect the ioutil.NopCloser by an unsafe cast, and then with that reader's length if inspectable but that might violate user expectations and previous older programs, and also creates brittle code and importing unsafe in net/http might be frowned on ** |
odeke-em
changed the title
|
In the docs for net/http.NewRequestWithContext https://golang.org/pkg/net/http/#NewRequestWithContext which just got added in Go1.13 and by following net/http.NewRequest https://golang.org/pkg/net/http/#NewRequest which links back to NewRequestWithContext, we already document this so nothing left to do here. |
|
Thanks for looking into this, @odeke-em . The explanation is giving me more knowledge on Go as well. |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
Start a httpbin server locally.
Run the following program
What did you expect to see?
Content-Length header is set when it is received by the server.
What did you see instead?
Content-Length header is missing when it is received by the server.
2019/09/14 12:55:00 Response Body: { "args": {}, "data": "{}", "files": {}, "form": {}, "headers": { "Accept-Encoding": "gzip", "Host": "localhost:80", "Transfer-Encoding": "chunked", "User-Agent": "Go-http-client/1.1" }, "json": {}, "origin": "172.17.0.1", "url": "http://localhost:80/post" }Versus what I would receive if I use
2019/09/14 12:55:22 Response Body: { "args": {}, "data": "{}", "files": {}, "form": {}, "headers": { "Accept-Encoding": "gzip", "Content-Length": "2", "Host": "localhost:80", "User-Agent": "Go-http-client/1.1" }, "json": {}, "origin": "172.17.0.1", "url": "http://localhost:80/post" }The text was updated successfully, but these errors were encountered: