Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ThreadSanitizer: data race in HeaderMapImpl with libc++ #7927

Closed
PiotrSikora opened this issue Aug 14, 2019 · 2 comments · Fixed by #7929
Closed

ThreadSanitizer: data race in HeaderMapImpl with libc++ #7927

PiotrSikora opened this issue Aug 14, 2019 · 2 comments · Fixed by #7929
Assignees
@PiotrSikora
Labels
area/test flakes

Comments

@PiotrSikora
Copy link
Contributor

PiotrSikora commented Aug 14, 2019 •
edited
Loading

Description:

There is a data race in HeaderMapImpl that's reported only when linking TSan with libc++.

Repro steps:

bazel test --config=clang-tsan --config=libc++ //test/...

Note that the data race is not reported when compiled with -c opt.

Logs:

The test results are a bit flaky, so this varies from run to run, but this is a sample list of tests that fail because of this failure:

//test/extensions/filters/http/cors:cors_filter_integration_test         FAILED in 29.7s
//test/extensions/filters/http/csrf:csrf_filter_integration_test         FAILED in 124.2s
//test/extensions/filters/http/lua:lua_integration_test                  FAILED in 35.5s
//test/extensions/filters/http/rbac:rbac_filter_integration_test         FAILED in 90.0s
//test/extensions/filters/http/tap:tap_filter_integration_test           FAILED in 33.9s
//test/integration:ads_integration_test                                  FAILED in 123.3s
//test/integration:cds_integration_test                                  FAILED in 29.0s
//test/integration:eds_integration_test                                  FAILED in 35.4s
//test/integration:hds_integration_test                                  FAILED in 14.9s
//test/integration:header_integration_test                               FAILED in 124.5s
//test/integration:integration_test                                      FAILED in 189.4s
//test/integration:load_stats_integration_test                           FAILED in 52.1s
//test/integration:overload_integration_test                             FAILED in 59.7s
//test/integration:websocket_integration_test                            FAILED in 135.7s
//test/integration:idle_timeout_integration_test                         FAILED in 2 out of 2 in 171.8s
//test/integration:protocol_integration_test                             FAILED in 3 out of 3 in 188.8s

Call Stack:

[ RUN      ] Protocols/ProtocolIntegrationTest.HittingEncoderFilterLimit/IPv6_HttpDownstream_HttpUpstream
==================
WARNING: ThreadSanitizer: data race (pid=2820)
  Write of size 8 at 0x7b5800004170 by main thread:
    #0 std::__1::__list_imp<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::__unlink_nodes(std::__1::__list_node_base<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, void*>*, std::__1::__list_node_base<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, void*>*) <null> (protocol_integration_test+0x1aeef28)
    #1 std::__1::__list_imp<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::clear() <null> (protocol_integration_test+0x1aeecd4)
    #2 std::__1::__list_imp<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::~__list_imp() <null> (protocol_integration_test+0x1aeec28)
    #3 std::__1::list<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::~list() <null> (protocol_integration_test+0x1aeebe8)
    #4 Envoy::Http::HeaderMapImpl::HeaderList::~HeaderList() <null> (protocol_integration_test+0x1aeeb98)
    #5 Envoy::Http::HeaderMapImpl::~HeaderMapImpl() <null> (protocol_integration_test+0x1aeeb51)
    #6 Envoy::Http::HeaderMapImpl::~HeaderMapImpl() <null> (protocol_integration_test+0x4e8c31c)
    #7 std::__1::default_delete<Envoy::Http::HeaderMapImpl>::operator()(Envoy::Http::HeaderMapImpl*) const <null> (protocol_integration_test+0x1c3ca9e)
    #8 std::__1::__shared_ptr_pointer<Envoy::Http::HeaderMapImpl*, std::__1::default_delete<Envoy::Http::HeaderMapImpl>, std::__1::allocator<Envoy::Http::HeaderMapImpl> >::__on_zero_shared() <null> (protocol_integration_test+0x1c3c719)
    #9 std::__1::__shared_weak_count::__release_shared() <null> (protocol_integration_test+0x505a70d)
    #10 Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool) <null> (protocol_integration_test+0x1bfd6e2)
    #11 Envoy::ProtocolIntegrationTest_HittingEncoderFilterLimit_Test::TestBody() <null> (protocol_integration_test+0x1aafc1f)
    #12 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x50045b6)
    #13 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x4fea30e)
    #14 testing::Test::Run() <null> (protocol_integration_test+0x4fcf53b)
    #15 testing::TestInfo::Run() <null> (protocol_integration_test+0x4fd045e)
    #16 testing::TestSuite::Run() <null> (protocol_integration_test+0x4fd0e84)
    #17 testing::internal::UnitTestImpl::RunAllTests() <null> (protocol_integration_test+0x4fe02ea)
    #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x500a806)
    #19 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x4fedf2e)
    #20 testing::UnitTest::Run() <null> (protocol_integration_test+0x4fdfb9b)
    #21 RUN_ALL_TESTS() <null> (protocol_integration_test+0x3a85097)
    #22 Envoy::TestRunner::RunTests(int, char**) <null> (protocol_integration_test+0x3a84365)
    #23 main <null> (protocol_integration_test+0x3a8302b)

  Previous read of size 8 at 0x7b5800004170 by thread T11:
    #0 std::__1::__list_imp<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::begin() const <null> (protocol_integration_test+0x4e8c886)
    #1 std::__1::list<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::begin() const <null> (protocol_integration_test+0x4e8c828)
    #2 Envoy::Http::HeaderMapImpl::HeaderList::begin() const <null> (protocol_integration_test+0x4e8b418)
    #3 Envoy::Http::HeaderMapImpl::iterate(Envoy::Http::HeaderMap::Iterate (*)(Envoy::Http::HeaderEntry const&, void*), void*) const <null> (protocol_integration_test+0x4e8999f)
    #4 Envoy::Http::Http1::StreamEncoderImpl::encodeHeaders(Envoy::Http::HeaderMap const&, bool) <null> (protocol_integration_test+0x3563f43)
    #5 Envoy::Http::Http1::ResponseStreamEncoderImpl::encodeHeaders(Envoy::Http::HeaderMap const&, bool) <null> (protocol_integration_test+0x356582e)
    #6 Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1::operator()() const <null> (protocol_integration_test+0x1c0b7e1)
    #7 decltype(std::__1::forward<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1&>(fp)()) std::__1::__invoke<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1&>(Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1&) <null> (protocol_integration_test+0x1c0b700)
    #8 void std::__1::__invoke_void_return_wrapper<void>::__call<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1&>(Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1&) <null> (protocol_integration_test+0x1c0b660)
    #9 std::__1::__function::__alloc_func<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1, std::__1::allocator<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1>, void ()>::operator()() <null> (protocol_integration_test+0x1c0b600)
    #10 std::__1::__function::__func<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1, std::__1::allocator<Envoy::FakeStream::encodeHeaders(Envoy::Http::HeaderMapImpl const&, bool)::$_1>, void ()>::operator()() <null> (protocol_integration_test+0x1c09c9c)
    #11 std::__1::__function::__value_func<void ()>::operator()() const <null> (protocol_integration_test+0x1bed988)
    #12 std::__1::function<void ()>::operator()() const <null> (protocol_integration_test+0x1bed678)
    #13 Envoy::Event::DispatcherImpl::runPostCallbacks() <null> (protocol_integration_test+0x3eab831)
    #14 Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1::operator()() const <null> (protocol_integration_test+0x3eafcd8)
    #15 decltype(std::__1::forward<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1&>(fp)()) std::__1::__invoke<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1&>(Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1&) <null> (protocol_integration_test+0x3eafc60)
    #16 void std::__1::__invoke_void_return_wrapper<void>::__call<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1&>(Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1&) <null> (protocol_integration_test+0x3eafbc0)
    #17 std::__1::__function::__alloc_func<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1, std::__1::allocator<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1>, void ()>::operator()() <null> (protocol_integration_test+0x3eafb60)
    #18 std::__1::__function::__func<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1, std::__1::allocator<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::unique_ptr<Envoy::Buffer::WatermarkFactory, std::__1::default_delete<Envoy::Buffer::WatermarkFactory> >&&, Envoy::Api::Api&, Envoy::Event::TimeSystem&)::$_1>, void ()>::operator()() <null> (protocol_integration_test+0x3eae1bc)
    #19 std::__1::__function::__value_func<void ()>::operator()() const <null> (protocol_integration_test+0x1bed988)
    #20 std::__1::function<void ()>::operator()() const <null> (protocol_integration_test+0x1bed678)
    #21 Envoy::Event::TimerImpl::TimerImpl(Envoy::CSmartPtr<event_base, &(event_base_free)>&, std::__1::function<void ()>)::$_0::operator()(int, short, void*) const <null> (protocol_integration_test+0x3fd3c63)
    #22 Envoy::Event::TimerImpl::TimerImpl(Envoy::CSmartPtr<event_base, &(event_base_free)>&, std::__1::function<void ()>)::$_0::__invoke(int, short, void*) <null> (protocol_integration_test+0x3fd3be7)
    #23 event_process_active_single_queue <null> (protocol_integration_test+0x4e7629c)
    #24 event_base_loop <null> (protocol_integration_test+0x4e70f48)
    #25 Envoy::Event::LibeventScheduler::run(Envoy::Event::Dispatcher::RunType) <null> (protocol_integration_test+0x3fd2295)
    #26 Envoy::Event::DispatcherImpl::run(Envoy::Event::Dispatcher::RunType) <null> (protocol_integration_test+0x3eab6f1)
    #27 Envoy::FakeUpstream::threadRoutine() <null> (protocol_integration_test+0x1c0423f)
    #28 Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11::operator()() const <null> (protocol_integration_test+0x1c21ee8)
    #29 decltype(std::__1::forward<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11&>(fp)()) std::__1::__invoke<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11&>(Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11&) <null> (protocol_integration_test+0x1c21e70)
    #30 void std::__1::__invoke_void_return_wrapper<void>::__call<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11&>(Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11&) <null> (protocol_integration_test+0x1c21dd0)
    #31 std::__1::__function::__alloc_func<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11, std::__1::allocator<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11>, void ()>::operator()() <null> (protocol_integration_test+0x1c21d70)
    #32 std::__1::__function::__func<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11, std::__1::allocator<Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool)::$_11>, void ()>::operator()() <null> (protocol_integration_test+0x1c203cc)
    #33 std::__1::__function::__value_func<void ()>::operator()() const <null> (protocol_integration_test+0x1bed988)
    #34 std::__1::function<void ()>::operator()() const <null> (protocol_integration_test+0x1bed678)
    #35 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>)::$_0::operator()(void*) const <null> (protocol_integration_test+0x4e4bf18)
    #36 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>)::$_0::__invoke(void*) <null> (protocol_integration_test+0x4e4bea8)

  Location is heap block of size 648 at 0x7b5800003f00 allocated by main thread:
    #0 malloc <null> (protocol_integration_test+0x1a15fc7)
    #1 operator new(unsigned long) <null> (protocol_integration_test+0x505b2a9)
    #2 Envoy::ProtocolIntegrationTest_HittingEncoderFilterLimit_Test::TestBody() <null> (protocol_integration_test+0x1aafc1f)
    #3 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x50045b6)
    #4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x4fea30e)
    #5 testing::Test::Run() <null> (protocol_integration_test+0x4fcf53b)
    #6 testing::TestInfo::Run() <null> (protocol_integration_test+0x4fd045e)
    #7 testing::TestSuite::Run() <null> (protocol_integration_test+0x4fd0e84)
    #8 testing::internal::UnitTestImpl::RunAllTests() <null> (protocol_integration_test+0x4fe02ea)
    #9 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x500a806)
    #10 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x4fedf2e)
    #11 testing::UnitTest::Run() <null> (protocol_integration_test+0x4fdfb9b)
    #12 RUN_ALL_TESTS() <null> (protocol_integration_test+0x3a85097)
    #13 Envoy::TestRunner::RunTests(int, char**) <null> (protocol_integration_test+0x3a84365)
    #14 main <null> (protocol_integration_test+0x3a8302b)

  Thread T11 (tid=14848, running) created by main thread at:
    #0 pthread_create <null> (protocol_integration_test+0x1a174b5)
    #1 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>) <null> (protocol_integration_test+0x4e4ba35)
    #2 std::__1::__unique_if<Envoy::Thread::ThreadImplPosix>::__unique_single std::__1::make_unique<Envoy::Thread::ThreadImplPosix, std::__1::function<void ()>&>(std::__1::function<void ()>&) <null> (protocol_integration_test+0x4e4c200)
    #3 Envoy::Thread::ThreadFactoryImplPosix::createThread(std::__1::function<void ()>) <null> (protocol_integration_test+0x4e4bdbb)
    #4 Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool) <null> (protocol_integration_test+0x1c02b2c)
    #5 Envoy::FakeUpstream::FakeUpstream(std::__1::shared_ptr<Envoy::Network::Address::Instance const> const&, Envoy::FakeHttpConnection::Type, Envoy::Event::TestTimeSystem&, bool) <null> (protocol_integration_test+0x1c030a7)
    #6 Envoy::BaseIntegrationTest::createUpstreams() <null> (protocol_integration_test+0x1c54913)
    #7 Envoy::BaseIntegrationTest::initialize() <null> (protocol_integration_test+0x1c53ea4)
    #8 Envoy::ProtocolIntegrationTest_HittingEncoderFilterLimit_Test::TestBody() <null> (protocol_integration_test+0x1aafa62)
    #9 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x50045b6)
    #10 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (protocol_integration_test+0x4fea30e)
    #11 testing::Test::Run() <null> (protocol_integration_test+0x4fcf53b)
    #12 testing::TestInfo::Run() <null> (protocol_integration_test+0x4fd045e)
    #13 testing::TestSuite::Run() <null> (protocol_integration_test+0x4fd0e84)
    #14 testing::internal::UnitTestImpl::RunAllTests() <null> (protocol_integration_test+0x4fe02ea)
    #15 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x500a806)
    #16 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) <null> (protocol_integration_test+0x4fedf2e)
    #17 testing::UnitTest::Run() <null> (protocol_integration_test+0x4fdfb9b)
    #18 RUN_ALL_TESTS() <null> (protocol_integration_test+0x3a85097)
    #19 Envoy::TestRunner::RunTests(int, char**) <null> (protocol_integration_test+0x3a84365)
    #20 main <null> (protocol_integration_test+0x3a8302b)

SUMMARY: ThreadSanitizer: data race (/ramdisk/piotrsikora/cache/bazel/_bazel_piotrsikora/052b53ce070b0dcea90180b82120ed3b/execroot/envoy/bazel-out/k8-fastbuild/bin/test/integration/protocol_integration_test+0x1aeef28) in std::__1::__list_imp<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, std::__1::allocator<Envoy::Http::HeaderMapImpl::HeaderEntryImpl> >::__unlink_nodes(std::__1::__list_node_base<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, void*>*, std::__1::__list_node_base<Envoy::Http::HeaderMapImpl::HeaderEntryImpl, void*>*)
==================

cc @lizan
@lizan
Copy link
Member

lizan commented Aug 14, 2019

Looks like a FakeStream issue, for decodeHeaders it seems gone with following change

diff --git a/test/integration/fake_upstream.cc b/test/integration/fake_upstream.cc
index fd970b6de..21e77b3e9 100644
--- a/test/integration/fake_upstream.cc
+++ b/test/integration/fake_upstream.cc
@@ -80,14 +80,15 @@ void FakeStream::encode100ContinueHeaders(const Http::HeaderMapImpl& headers) {
 }

 void FakeStream::encodeHeaders(const Http::HeaderMapImpl& headers, bool end_stream) {
-  std::shared_ptr<Http::HeaderMapImpl> headers_copy(
+  std::unique_ptr<Http::HeaderMapImpl> headers_copy(
       new Http::HeaderMapImpl(static_cast<const Http::HeaderMap&>(headers)));
   if (add_served_by_header_) {
     headers_copy->addCopy(Http::LowerCaseString("x-served-by"),
                           parent_.connection().localAddress()->asString());
   }
-  parent_.connection().dispatcher().post([this, headers_copy, end_stream]() -> void {
-    encoder_.encodeHeaders(*headers_copy, end_stream);
+  parent_.connection().dispatcher().post([this, headers = headers_copy.release(), end_stream]() -> void {
+    encoder_.encodeHeaders(*headers, end_stream);
+    delete headers;
   });
 }

I guess apply this similar to other places should fix them, and guessing #7928 is similar too.

@lizan
Copy link
Member

lizan commented Aug 14, 2019

@PiotrSikora can you take a stab on this? I don't have bandwidth to do a full fix this week.

@PiotrSikora PiotrSikora mentioned this issue Aug 14, 2019
Merged
PiotrSikora added a commit to PiotrSikora/envoy that referenced this issue Aug 15, 2019
@PiotrSikora
test: fix data races in FakeStream.
555b7a7 
Fixes envoyproxy#7927.

Signed-off-by: Piotr Sikora <[email protected]>
lizan pushed a commit that referenced this issue Aug 19, 2019
@PiotrSikora @lizan
test: fix data races in FakeStream. (#7929)
869981f 
Fixes #7927.

Risk Level: low (test only)

Signed-off-by: Piotr Sikora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PiotrSikora PiotrSikora

Labels
area/test flakes
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test: fix data races in FakeStream. PiotrSikora/envoy
3 participants
@PiotrSikora @lizan @mattklein123