buffer_filter_integration_test flake

[mattklein123]

[ RUN      ] Protocols/BufferIntegrationTest.RouterRequestPopulateContentLength/IPv4_Http2Downstream_HttpUpstream
[2019-08-08 19:51:10.086][33931][critical][assert] [test/integration/http_integration.cc:297] assert failure: result. Details: The connection disconnected unexpectedly, and allow_unexpected_disconnects_ is false.
 See https://github.com/envoyproxy/envoy/blob/master/test/integration/README.md#unexpected-disconnects
[2019-08-08 19:51:10.086][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:81] Caught Aborted, suspect faulting address 0x848b
[2019-08-08 19:51:10.086][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:69] Backtrace (use tools/stack_decode.py to get line numbers):
[2019-08-08 19:51:10.086][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #0: __restore_rt [0x7f31a0fc2390]
[2019-08-08 19:51:10.099][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #1: Envoy::HttpIntegrationTest::~HttpIntegrationTest() [0x6f125a]
[2019-08-08 19:51:10.103][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #2: Envoy::(anonymous namespace)::BufferIntegrationTest_RouterRequestPopulateContentLength_Test::~BufferIntegrationTest_RouterRequestPopulateContentLength_Test() [0x6d64c2]
[2019-08-08 19:51:10.108][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #3: testing::internal::HandleExceptionsInMethodIfSupported<>() [0x1104f24]
[2019-08-08 19:51:10.113][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #4: testing::TestInfo::Run() [0x1105ed9]
[2019-08-08 19:51:10.117][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #5: testing::TestSuite::Run() [0x11067f7]
[2019-08-08 19:51:10.122][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #6: testing::internal::UnitTestImpl::RunAllTests() [0x11135f7]
[2019-08-08 19:51:10.126][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #7: testing::internal::HandleExceptionsInMethodIfSupported<>() [0x1112e64]
[2019-08-08 19:51:10.130][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #8: testing::UnitTest::Run() [0x1112cfe]
[2019-08-08 19:51:10.134][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #9: Envoy::TestRunner::RunTests() [0xbc3289]
[2019-08-08 19:51:10.139][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #10: main [0xbc28f2]
[2019-08-08 19:51:10.139][33931][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:73] #11: __libc_start_main [0x7f31a0c07830]
external/bazel_tools/tools/test/test-setup.sh: line 310: 33931 Aborted                 (core dumped) "${TEST_PATH}" "$@" 2>&1

@euroelessar I think this a flake from a recent test you added. Could you run the tests in a --runs_per_test loop and PTAL?

[euroelessar]

Sure, will take a look at it shortly.

[jmarantz]

Per discussion on the merged PR this test is also quite slow (15s * 8 = 120s) and it is not at all CPU bound; just sitting waiting for events on all threads.

Maybe use SimTime for this?

[euroelessar]

I've likely misused the api for making requests, checking it now and will send a pull request once understand what's wrong. Test itself is supposed to be fairly simple (just make single request & check what headers are observed by upstream), so shouldn't take any noticeable time by itself.