Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<vector>: Add ASan annotations. #2071

Merged
merged 68 commits into from
Dec 11, 2021
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
46a84b6
Initial asan annotation commit
cbezault Jul 1, 2021
b6cf70c
Some testing fixups
cbezault Jul 1, 2021
daa4db9
First working pass at asan annotations
cbezault Jul 21, 2021
72f27a9
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Jul 21, 2021
67b9a50
Fixup merge error and env.lst
cbezault Jul 21, 2021
a50a9e8
clang-format
cbezault Jul 21, 2021
ecca61c
More clang-format
cbezault Jul 21, 2021
9b3bad0
Remove unused variable
cbezault Jul 26, 2021
ae3ecd1
Add ASan to the CI image
cbezault Jul 28, 2021
7a43662
Update stl/inc/vector
cbezault Jul 28, 2021
295cb82
Apply suggestions from code review
cbezault Jul 28, 2021
4bc1d3e
Update VMSS to add ASAN.
StephanTLavavej Jul 28, 2021
201e707
Resolve most of Misco's comments
cbezault Jul 28, 2021
32ece64
Address Alex's comments
cbezault Jul 28, 2021
0e05186
Require x86 or x64
cbezault Jul 28, 2021
4001372
Apply suggestions from code review
cbezault Jul 29, 2021
84bcc99
Update stl/inc/vector
cbezault Jul 30, 2021
8540313
Merge branch 'main' into gh2071
StephanTLavavej Aug 4, 2021
521b8fe
Merge branch 'main' into gh2071
StephanTLavavej Aug 4, 2021
a1a004c
clang-format.
StephanTLavavej Aug 4, 2021
c9c77f9
Merge branch 'main' of https://github.com/cbezault/STL
cbezault Aug 10, 2021
632566b
Fixup tests and annotations
cbezault Aug 12, 2021
3a09cd9
Remove unused types
cbezault Aug 12, 2021
138752a
Resolve PR comments
cbezault Aug 17, 2021
9977914
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Aug 17, 2021
d977cf9
sigh, missed a file
cbezault Aug 17, 2021
9dd9d6c
Remove shouty banners from bad merge
cbezault Aug 17, 2021
3d43d92
Resolve comments
cbezault Aug 18, 2021
0a6d34f
Resolve more comments
cbezault Aug 18, 2021
56a3d83
Resolve PR comments
cbezault Aug 18, 2021
873dd34
Expand the allocator aware tests to also run at compile time and fix …
miscco Aug 26, 2021
fe7e654
Cleanup the test a bit
miscco Aug 27, 2021
513f645
Merge branch 'main' into fix_constexpr_vector_again
miscco Sep 2, 2021
f97defa
Add Bug citation for EDG
miscco Sep 2, 2021
4fc4765
Typo in tests/std/tests/VSO_0000000_allocator_propagation
CaseyCarter Sep 2, 2021
74cd839
Make the test classier, like a tuxedo cat!
StephanTLavavej Sep 3, 2021
35c0a06
Cleanup move assignment
miscco Sep 7, 2021
102e358
Clean up the different assignment helpers
miscco Sep 7, 2021
dc735af
Asan implementation for vector
miscco Sep 7, 2021
07431d2
Merge Misco's changes
cbezault Sep 8, 2021
6179818
Only link in stl_asan.lib when __SANITIZE_ADDRESS__ is defined
cbezault Sep 9, 2021
f2c0f11
Cleanup whitespace changes
cbezault Sep 10, 2021
14364c9
Add pragma detect mismatch
cbezault Sep 13, 2021
14fbbb2
Add asan_noop file
cbezault Sep 15, 2021
422930a
Some more cleanups
cbezault Sep 16, 2021
afdaa81
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Sep 16, 2021
705a00e
Many of Casey's review comments
CaseyCarter Sep 23, 2021
0e49ba2
Cleanup types of arguments to the _ASAN_VECTOR_MEOW macros
CaseyCarter Sep 23, 2021
159ac35
More msbuild stuff + Casey's comments
cbezault Sep 28, 2021
2559230
Add alignment to std::allocator
cbezault Sep 28, 2021
c4fb4af
Update alignment check variable
cbezault Sep 28, 2021
318d472
clang-format
cbezault Sep 28, 2021
336b40c
Try catch when filling a new buffer
cbezault Sep 29, 2021
ebea23a
Make a new type to guard creating a new asan annotated vector with po…
cbezault Sep 30, 2021
d55025b
Add tests with noexcept non-trivial type and exceptable trivial type
cbezault Sep 30, 2021
cdd62a8
More msbuild changes
cbezault Sep 30, 2021
aa67435
Address review comments
cbezault Nov 8, 2021
33e22ad
Turn on vector annotation and tests for clang-cl
cbezault Nov 11, 2021
e5ced0f
x86 needs an additional leading underscore
cbezault Nov 15, 2021
4fe40a3
Resolve merge conflict
cbezault Nov 15, 2021
4094aa6
Fixup bad find/replace
cbezault Nov 22, 2021
934985d
Code review feedback.
StephanTLavavej Nov 23, 2021
32702bf
Access control for guard data members.
StephanTLavavej Nov 30, 2021
159f370
Don't include for CLR
cbezault Dec 10, 2021
032983d
Fixup x86 symbols
cbezault Dec 10, 2021
a8dae00
Set _ANNOTATE_VECTOR in internal msbuild
cbezault Dec 10, 2021
9596451
Update tests so they have some more coverage and should work in both …
cbezault Dec 10, 2021
5dff266
Test fixup
cbezault Dec 10, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions stl/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ endforeach()

# Objs that exist in both libcpmt[d][01].lib and msvcprt[d].lib.
set(IMPLIB_SOURCES
${CMAKE_CURRENT_LIST_DIR}/src/asan_noop.cpp
${CMAKE_CURRENT_LIST_DIR}/src/filesystem.cpp
${CMAKE_CURRENT_LIST_DIR}/src/format.cpp
${CMAKE_CURRENT_LIST_DIR}/src/locale0_implib.cpp
Expand Down Expand Up @@ -392,6 +393,10 @@ set(SOURCES
${CMAKE_CURRENT_LIST_DIR}/src/xwstoxfl.cpp
)

set(ASAN_SOURCES
${CMAKE_CURRENT_LIST_DIR}/src/asan.cpp
)

set(EHA_SOURCES
${CMAKE_CURRENT_LIST_DIR}/src/excptptr.cpp
)
Expand Down Expand Up @@ -543,11 +548,11 @@ add_stl_dlls("d" "_DEBUG" "${VCLIBS_DEBUG_OPTIONS}" "" "/opt:ref,noicf")

function(add_stl_statics FLAVOR_SUFFIX THIS_CONFIG_DEFINITIONS THIS_CONFIG_COMPILE_OPTIONS)
add_library(libcpmt${FLAVOR_SUFFIX}_eha OBJECT ${EHA_SOURCES})
target_compile_definitions(libcpmt${FLAVOR_SUFFIX}_eha PRIVATE "${THIS_CONFIG_DEFINITIONS}")
target_compile_definitions(libcpmt${FLAVOR_SUFFIX}_eha PRIVATE "${THIS_CONFIG_DEFINITIONS};_ANNOTATE_VECTOR")
target_compile_options(libcpmt${FLAVOR_SUFFIX}_eha PRIVATE "${THIS_CONFIG_COMPILE_OPTIONS};/EHa")

add_library(libcpmt${FLAVOR_SUFFIX} STATIC ${HEADERS} ${IMPLIB_SOURCES} ${SOURCES} ${INITIALIZER_SOURCES} ${STATIC_SOURCES})
target_compile_definitions(libcpmt${FLAVOR_SUFFIX} PRIVATE "${THIS_CONFIG_DEFINITIONS}")
target_compile_definitions(libcpmt${FLAVOR_SUFFIX} PRIVATE "${THIS_CONFIG_DEFINITIONS};_ANNOTATE_VECTOR")
target_compile_options(libcpmt${FLAVOR_SUFFIX} PRIVATE "${THIS_CONFIG_COMPILE_OPTIONS};/EHsc")
target_link_libraries(libcpmt${FLAVOR_SUFFIX} PRIVATE Boost::math libcpmt${FLAVOR_SUFFIX}_eha std_init_once_begin_initialize std_init_once_complete)
endfunction()
Expand All @@ -557,3 +562,5 @@ add_stl_statics("1" "_ITERATOR_DEBUG_LEVEL=1" "${VCLIBS_RELEASE_OPTIONS}")
add_stl_statics("d" "_DEBUG;_ITERATOR_DEBUG_LEVEL=2" "${VCLIBS_DEBUG_OPTIONS}")
add_stl_statics("d1" "_DEBUG;_ITERATOR_DEBUG_LEVEL=1" "${VCLIBS_DEBUG_OPTIONS}")
add_stl_statics("d0" "_DEBUG;_ITERATOR_DEBUG_LEVEL=0" "${VCLIBS_DEBUG_OPTIONS}")

add_library(stl_asan STATIC ${ASAN_SOURCES})
363 changes: 323 additions & 40 deletions stl/inc/vector

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions stl/inc/xmemory
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,9 @@ _NODISCARD constexpr allocation_result<typename allocator_traits<_Alloc>::pointe
#endif // __cpp_lib_concepts
#endif // _HAS_CXX23

// The number of user bytes a single byte of ASAN shadow memory can track.
_INLINE_VAR constexpr size_t _Asan_granularity = 8;

template <class _Ty>
class allocator {
public:
Expand Down Expand Up @@ -857,6 +860,8 @@ public:
return static_cast<size_t>(-1) / sizeof(_Ty);
}
#endif // _HAS_DEPRECATED_ALLOCATOR_MEMBERS

static constexpr size_t _Minimum_allocation_alignment = _Asan_granularity;
};

template <>
Expand Down
14 changes: 14 additions & 0 deletions stl/msbuild/stl_asan/dirs.proj
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!--
Copyright (c) Microsoft Corporation.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->

<Import Project="$(MSBuildThisFileDirectory)..\..\..\..\crt-common.settings.targets" />

<ItemGroup Condition="'$(BuildExePhase)' == '1'">
<ProjectFile Include="stl_asan.nativeproj" />
</ItemGroup>

<Import Project="$(_NTDRIVE)$(_NTROOT)\tools\Microsoft.DevDiv.Traversal.targets" />
</Project>
16 changes: 16 additions & 0 deletions stl/msbuild/stl_asan/stl_asan.files.settings.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="Dogfood">
<!--
Copyright (c) Microsoft Corporation.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->

<ItemGroup>
<BuildFiles Include="
$(CrtRoot)\github\stl\src\asan.cpp;
">
<BuildAs>nativecpp</BuildAs>
</BuildFiles>
</ItemGroup>

</Project>
14 changes: 14 additions & 0 deletions stl/msbuild/stl_asan/stl_asan.nativeproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="Dogfood">
<!--
Copyright (c) Microsoft Corporation.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->

<PropertyGroup>
<Arm64X>true</Arm64X>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)stl_asan.settings.targets"/>

</Project>
29 changes: 29 additions & 0 deletions stl/msbuild/stl_asan/stl_asan.settings.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="Dogfood">
<!--
Copyright (c) Microsoft Corporation.
SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-->

<PropertyGroup>
<FinalBinary>p_stl_asan</FinalBinary>
<TargetType>LIBRARY</TargetType>
<Arm64CombinedPdb>true</Arm64CombinedPdb>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)..\..\..\..\crt_build.settings.targets"/>

<PropertyGroup>
<OutputName>stl_asan</OutputName>
</PropertyGroup>

<PropertyGroup>
<ClProgramDataBaseFileName>$(OutputLibPdbPath)$(OutputName)$(PdbVerName).pdb</ClProgramDataBaseFileName>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)stl_asan.files.settings.targets"/>

<Import Project="$(VCToolsRootPath)\crt\crt_build.targets"/>
<Target Name="GetBaseAddress"/>

</Project>
1 change: 1 addition & 0 deletions stl/msbuild/stl_base/stl.files.settings.targets
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
<!-- Objs that exist in both libcpmt[d][01].lib and msvcprt[d].lib
(controlled by IncludeInLink and IncludeInImportLib). -->
<BuildFiles Include="
$(CrtRoot)\github\stl\src\asan_noop.cpp;
cbezault marked this conversation as resolved.
Show resolved Hide resolved
$(CrtRoot)\github\stl\src\filesystem.cpp;
$(CrtRoot)\github\stl\src\format.cpp;
$(CrtRoot)\github\stl\src\locale0_implib.cpp;
Expand Down
8 changes: 8 additions & 0 deletions stl/src/asan.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// Copyright (c) Microsoft Corporation.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

namespace std {
extern "C" {
extern const bool _Asan_vector_should_annotate = true;
}
} // namespace std
9 changes: 9 additions & 0 deletions stl/src/asan_noop.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
// Copyright (c) Microsoft Corporation.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

extern "C" {
extern const bool _Asan_vector_should_annotate_default = false;

void __cdecl __sanitizer_annotate_contiguous_container_default(
const void*, const void*, const void*, const void*) noexcept {}
}
1 change: 1 addition & 0 deletions tests/std/test.lst
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ tests\GH_001638_dllexport_derived_classes
tests\GH_001850_clog_tied_to_cout
tests\GH_001858_iostream_exception
CaseyCarter marked this conversation as resolved.
Show resolved Hide resolved
tests\GH_001914_cached_position
tests\GH_002030_asan_annotate_vector
tests\GH_002039_byte_is_not_trivially_swappable
tests\GH_002058_debug_iterator_race
tests\GH_002120_streambuf_seekpos_and_seekoff
Expand Down
37 changes: 37 additions & 0 deletions tests/std/tests/GH_002030_asan_annotate_vector/env.lst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

# This test matrix is the usual test matrix, with all currently unsupported options removed, crossed with the ASan flags.
# TRANSITION, VSO-1350252
# Due to a bug in the ASan libs using ASan with /MD or /MT requires IDL==0 and using /MDd or /MTd requires IDL==2.
cbezault marked this conversation as resolved.
Show resolved Hide resolved
# clang-cl does not currently support targeting /MDd or /MTd.
RUNALL_INCLUDE ..\prefix.lst
RUNALL_CROSSLIST
PM_CL="-fsanitize=address /Zi /wd4611 /w14640 /Zc:threadSafeInit-" PM_LINK="/debug"
RUNALL_CROSSLIST
PM_CL="/BE /c /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MT /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++17 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++20 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++latest /permissive- /Zc:char8_t- /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++latest /permissive- /Zc:noexceptTypes- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++14 /fp:except /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++latest /permissive- /Zc:wchar_t- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MT /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MT /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /fp:strict /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/Za /EHsc /MD /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/Za /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++14"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++17"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive-"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive- /D_HAS_CXX23 /fp:strict"
Loading