PANIC: pkg: runtime error: slice bounds out of range goroutine 836

[yuantianwen]

2017/06/15 14:39:33 [HTTPClient] Failed to send request: GET /folder/app-shelves?sign=61624000000001@EB-hEQoinZMubkHYS_VEsiOnTy7MF4iyvExTnVbX1v8&deviceName=FRD-AL00&language=zh&cat=HITOP00000162&mcc=460&mnc= HTTP/1.1
Connection: keep-alive
Resource: 0
Content-Type: application/json
Snapshot: t45.inf
Mode: HTTP
User-Agent: Java/1.8.0_92
Host: 10.26.0.254:8092
Accept: text/html, image/gif, image/jpeg, *; q=.2, /; q=.2

2017/06/15 14:39:33 [HTTPClient] Response:
2017/06/15 14:39:33 PANIC: pkg: runtime error: slice bounds out of range goroutine 836 [running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
/usr/local/go/src/runtime/debug/stack.go:24 +0x80
main.(*HTTPClient).Send.func1(0xc826519c78, 0xc826519c90)
/go/src/github.com/buger/gor/http_client.go:147 +0x52d
panic(0xa56e00, 0xc820010170)
/usr/local/go/src/runtime/panic.go:443 +0x4e9
github.com/buger/gor/proto.Path(0xc82823e000, 0x5, 0x19000, 0x0, 0x0, 0x0)
/go/src/github.com/buger/gor/proto/proto.go:348 +0xf8
github.com/buger/gor/proto.Status(0xc82823e000, 0x5, 0x19000, 0x0, 0x0, 0x0)
/go/src/github.com/buger/gor/proto/proto.go:448 +0x46
main.(*HTTPClient).Send(0xc827c7dc00, 0xc82314137f, 0x178, 0x178, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/buger/gor/http_client.go:209 +0x274c
main.(*HTTPOutput).sendRequest(0xc8200945a0, 0xc827c7dc00, 0xc823141340, 0x1b7, 0x1b7)
/go/src/github.com/buger/gor/output_http.go:205 +0x2b6
main.(*HTTPOutput).startWorker(0xc8200945a0)
/go/src/github.com/buger/gor/output_http.go:125 +0x1c7
created by main.(*HTTPOutput).workerMaster
/go/src/github.com/buger/gor/output_http.go:99 +0x85

[buger]

Thank you, will check!

[yuantianwen]

hello ,mr buger, i read your code,i found you use pcap api to catch http request package,but i found sometime pcap have package lose problem. my test case is below:

1. use jmeter as web press test tool
2. i prepare two tomcat environment,one as product env,other as test env.
3. run gor: gor --input-raw :8080 --output-file requests.gor
4. i check product env tomcat log:access_2017-06-15.log and count the http request numbers，for example :1000 times get request.
5. then i check requests.gor,i count the http request numbers, in theory the number must be same with pre-step number, i test more time ,but here number always lower the product env . i doubt if we lost package when using pcap api.
   this issue is very serious, can you help me figure out this issue

[buger]

Due to nature of libpcap, there is no guarantee of intercepting 100% of requests. And loss itself heavily depend on traffic type and its amount, for example, chances of missing large payload (file upload) is higher than simple GET requests.

You can try increasing system buffers to reduce the loss:

 sysctl -w net.core.rmem_default=65536
 sysctl -w net.core.wmem_default=65536
 sysctl -w net.core.rmem_max=16777216
 sysctl -w net.core.wmem_max=16777216