Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<vector>: Add ASan annotations. #2071

Merged
merged 68 commits into from
Dec 11, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
46a84b6
Initial asan annotation commit
cbezault Jul 1, 2021
b6cf70c
Some testing fixups
cbezault Jul 1, 2021
daa4db9
First working pass at asan annotations
cbezault Jul 21, 2021
72f27a9
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Jul 21, 2021
67b9a50
Fixup merge error and env.lst
cbezault Jul 21, 2021
a50a9e8
clang-format
cbezault Jul 21, 2021
ecca61c
More clang-format
cbezault Jul 21, 2021
9b3bad0
Remove unused variable
cbezault Jul 26, 2021
ae3ecd1
Add ASan to the CI image
cbezault Jul 28, 2021
7a43662
Update stl/inc/vector
cbezault Jul 28, 2021
295cb82
Apply suggestions from code review
cbezault Jul 28, 2021
4bc1d3e
Update VMSS to add ASAN.
StephanTLavavej Jul 28, 2021
201e707
Resolve most of Misco's comments
cbezault Jul 28, 2021
32ece64
Address Alex's comments
cbezault Jul 28, 2021
0e05186
Require x86 or x64
cbezault Jul 28, 2021
4001372
Apply suggestions from code review
cbezault Jul 29, 2021
84bcc99
Update stl/inc/vector
cbezault Jul 30, 2021
8540313
Merge branch 'main' into gh2071
StephanTLavavej Aug 4, 2021
521b8fe
Merge branch 'main' into gh2071
StephanTLavavej Aug 4, 2021
a1a004c
clang-format.
StephanTLavavej Aug 4, 2021
c9c77f9
Merge branch 'main' of https://github.com/cbezault/STL
cbezault Aug 10, 2021
632566b
Fixup tests and annotations
cbezault Aug 12, 2021
3a09cd9
Remove unused types
cbezault Aug 12, 2021
138752a
Resolve PR comments
cbezault Aug 17, 2021
9977914
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Aug 17, 2021
d977cf9
sigh, missed a file
cbezault Aug 17, 2021
9dd9d6c
Remove shouty banners from bad merge
cbezault Aug 17, 2021
3d43d92
Resolve comments
cbezault Aug 18, 2021
0a6d34f
Resolve more comments
cbezault Aug 18, 2021
56a3d83
Resolve PR comments
cbezault Aug 18, 2021
873dd34
Expand the allocator aware tests to also run at compile time and fix …
miscco Aug 26, 2021
fe7e654
Cleanup the test a bit
miscco Aug 27, 2021
513f645
Merge branch 'main' into fix_constexpr_vector_again
miscco Sep 2, 2021
f97defa
Add Bug citation for EDG
miscco Sep 2, 2021
4fc4765
Typo in tests/std/tests/VSO_0000000_allocator_propagation
CaseyCarter Sep 2, 2021
74cd839
Make the test classier, like a tuxedo cat!
StephanTLavavej Sep 3, 2021
35c0a06
Cleanup move assignment
miscco Sep 7, 2021
102e358
Clean up the different assignment helpers
miscco Sep 7, 2021
dc735af
Asan implementation for vector
miscco Sep 7, 2021
07431d2
Merge Misco's changes
cbezault Sep 8, 2021
6179818
Only link in stl_asan.lib when __SANITIZE_ADDRESS__ is defined
cbezault Sep 9, 2021
f2c0f11
Cleanup whitespace changes
cbezault Sep 10, 2021
14364c9
Add pragma detect mismatch
cbezault Sep 13, 2021
14fbbb2
Add asan_noop file
cbezault Sep 15, 2021
422930a
Some more cleanups
cbezault Sep 16, 2021
afdaa81
Merge branch 'main' of https://github.com/Microsoft/STL
cbezault Sep 16, 2021
705a00e
Many of Casey's review comments
CaseyCarter Sep 23, 2021
0e49ba2
Cleanup types of arguments to the _ASAN_VECTOR_MEOW macros
CaseyCarter Sep 23, 2021
159ac35
More msbuild stuff + Casey's comments
cbezault Sep 28, 2021
2559230
Add alignment to std::allocator
cbezault Sep 28, 2021
c4fb4af
Update alignment check variable
cbezault Sep 28, 2021
318d472
clang-format
cbezault Sep 28, 2021
336b40c
Try catch when filling a new buffer
cbezault Sep 29, 2021
ebea23a
Make a new type to guard creating a new asan annotated vector with po…
cbezault Sep 30, 2021
d55025b
Add tests with noexcept non-trivial type and exceptable trivial type
cbezault Sep 30, 2021
cdd62a8
More msbuild changes
cbezault Sep 30, 2021
aa67435
Address review comments
cbezault Nov 8, 2021
33e22ad
Turn on vector annotation and tests for clang-cl
cbezault Nov 11, 2021
e5ced0f
x86 needs an additional leading underscore
cbezault Nov 15, 2021
4fe40a3
Resolve merge conflict
cbezault Nov 15, 2021
4094aa6
Fixup bad find/replace
cbezault Nov 22, 2021
934985d
Code review feedback.
StephanTLavavej Nov 23, 2021
32702bf
Access control for guard data members.
StephanTLavavej Nov 30, 2021
159f370
Don't include for CLR
cbezault Dec 10, 2021
032983d
Fixup x86 symbols
cbezault Dec 10, 2021
a8dae00
Set _ANNOTATE_VECTOR in internal msbuild
cbezault Dec 10, 2021
9596451
Update tests so they have some more coverage and should work in both …
cbezault Dec 10, 2021
5dff266
Test fixup
cbezault Dec 10, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions stl/inc/vector
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,7 @@ constexpr _Ty* _Unfancy_maybe_null(_Ty* _Ptr) noexcept { // do nothing for plain
#if __has_feature(address_sanitizer)
#define _ACTIVATE_VECTOR_ANNOTATION
#define _INSERT_VECTOR_ANNOTATION
#pragma comment(linker, "/INFERASANLIBS")
#endif // __has_feature(address_sanitizer)
#elif defined(_ANNOTATE_VECTOR) // ^^^ __clang__ ^^^ // vvv _ANNOTATE_VECTOR vvv
#define _INSERT_VECTOR_ANNOTATION
Expand Down Expand Up @@ -3429,6 +3430,8 @@ _NODISCARD _CONSTEXPR20 _Iter_diff_t<_VbIt> _Count_vbool(_VbIt _First, const _Vb
#undef _ASAN_VECTOR_CREATE_GUARD
#undef _ASAN_VECTOR_EXTEND_GUARD
#undef _ASAN_VECTOR_RELEASE_GUARD
#undef _ACTIVATE_VECTOR_ANNOTATION
#undef _INSERT_VECTOR_ANNOTATION
_STD_END

#pragma pop_macro("new")
Expand Down
2 changes: 1 addition & 1 deletion stl/msbuild/stl_base/libcp.settings.targets
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
<PropertyGroup>
<ClProgramDataBaseFileName>$(OutputLibPdbPath)$(OutputName)$(PdbVerName).pdb</ClProgramDataBaseFileName>
<ClDefines Condition="'$(DependsOnConcRT)' == 'true'">$(ClDefines);_STL_CONCRT_SUPPORT</ClDefines>
<ClDefines>$(ClDefines);_VCRT_ALLOW_INTERNALS</ClDefines>
<ClDefines>$(ClDefines);_VCRT_ALLOW_INTERNALS;_ANNOTATE_VECTOR</ClDefines>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)\stl.files.settings.targets"/>
Expand Down
77 changes: 50 additions & 27 deletions tests/std/tests/GH_002030_asan_annotate_vector/env.lst
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,54 @@
# clang-cl does not currently support targeting /MDd or /MTd.
RUNALL_INCLUDE ..\prefix.lst
RUNALL_CROSSLIST
PM_CL="-fsanitize=address /Zi /wd4611 /w14640 /Zc:threadSafeInit-" PM_LINK="/debug"
PM_CL="/Zi /wd4611 /w14640 /Zc:threadSafeInit-" PM_LINK="/debug"
RUNALL_CROSSLIST
PM_CL="/BE /c /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MT /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/BE /c /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++17 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++20 /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++latest /permissive- /Zc:char8_t- /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MD /std:c++latest /permissive- /Zc:noexceptTypes- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++14 /fp:except /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++latest /permissive- /Zc:wchar_t- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MT /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MT /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /fp:strict /fno-sanitize-address-vcasan-lib"
PM_CL="/EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/Za /EHsc /MD /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/Za /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++14"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++17"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive-"
PM_COMPILER="clang-cl" PM_CL="-fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive- /D_HAS_CXX23 /fp:strict"
PM_CL="-fsanitize=address /BE /c /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /BE /c /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /BE /c /EHsc /MT /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /BE /c /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MD /std:c++17 /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MD /std:c++20 /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MD /std:c++latest /permissive- /Zc:char8_t- /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MD /std:c++latest /permissive- /Zc:noexceptTypes- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MDd /std:c++14 /fp:except /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MDd /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MDd /std:c++latest /permissive- /Zc:wchar_t- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MT /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MT /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MTd /std:c++latest /permissive /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MTd /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MTd /std:c++latest /permissive- /fp:strict /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /Za /EHsc /MD /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="-fsanitize=address /Za /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /BE /c /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /BE /c /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /BE /c /EHsc /MT /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /BE /c /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MD /std:c++14 /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MD /std:c++17 /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MD /std:c++20 /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MD /std:c++latest /permissive- /Zc:char8_t- /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MD /std:c++latest /permissive- /Zc:noexceptTypes- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MDd /std:c++14 /fp:except /Zc:preprocessor /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MDd /std:c++17 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MDd /std:c++20 /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MDd /std:c++latest /permissive- /Zc:wchar_t- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MT /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MT /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MTd /std:c++latest /permissive /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MTd /std:c++latest /permissive- /analyze:only /analyze:autolog- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MTd /std:c++latest /permissive- /fp:strict /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /EHsc /MTd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /Za /EHsc /MD /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
PM_CL="/D_ANNOTATE_VECTOR /Za /EHsc /MDd /std:c++latest /permissive- /fno-sanitize-address-vcasan-lib"
# TRANSITION, clang-cl does not support /alternatename so we cannot test /D_ANNOTATE_VECTOR without -fsanitize=address
PM_COMPILER="clang-cl" PM_CL="-fsanitize=address -fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++14"
PM_COMPILER="clang-cl" PM_CL="-fsanitize=address -fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MD /std:c++17"
PM_COMPILER="clang-cl" PM_CL="-fsanitize=address -fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive-"
PM_COMPILER="clang-cl" PM_CL="-fsanitize=address -fno-ms-compatibility -fno-delayed-template-parsing /EHsc /MT /std:c++latest /permissive- /D_HAS_CXX23 /fp:strict"
20 changes: 19 additions & 1 deletion tests/std/tests/GH_002030_asan_annotate_vector/test.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft Corporation.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

// REQUIRES: asan, x64 || x86
// REQUIRES: x64 || x86

#include <cassert>
#include <cstddef>
Expand All @@ -14,10 +14,20 @@

using namespace std;

#ifndef __SANITIZE_ADDRESS__
#if defined(__clang__) && defined(__has_feature)
#if __has_feature(address_sanitizer)
#define __SANITIZE_ADDRESS__
#endif
#endif
#endif

#ifdef __SANITIZE_ADDRESS__
extern "C" {
void* __sanitizer_contiguous_container_find_bad_address(const void* beg, const void* mid, const void* end) noexcept;
void __asan_describe_address(void*) noexcept;
}
#endif // ASan instrumentation enabled

struct non_trivial_can_throw {
non_trivial_can_throw() {}
Expand Down Expand Up @@ -115,6 +125,7 @@ class input_iterator_tester {

template <class T, class Alloc>
bool verify_vector(vector<T, Alloc>& vec) {
#ifdef __SANITIZE_ADDRESS__
size_t buffer_bytes = vec.capacity() * sizeof(T);
void* buffer = vec.data();
void* aligned_start = align(8, 1, buffer, buffer_bytes);
Expand Down Expand Up @@ -145,6 +156,10 @@ bool verify_vector(vector<T, Alloc>& vec) {
__asan_describe_address(bad_address);

return false;
#else // ^^^ ASan instrumentation enabled ^^^ // vvv ASan instrumentation disabled vvv
(void) vec;
return true;
#endif // Asan instrumentation disabled
}

// Note: This class does not satisfy all the allocator requirements but is sufficient for this test.
Expand Down Expand Up @@ -880,7 +895,9 @@ void run_allocator_matrix() {
}

int main() {
// Do some work even when we aren't instrumented
run_allocator_matrix<char>();
#ifdef __SANITIZE_ADDRESS__
run_allocator_matrix<int>();
run_allocator_matrix<double>();
run_allocator_matrix<non_trivial_can_throw>();
Expand All @@ -896,4 +913,5 @@ int main() {
test_resize_throw();
test_insert_n_throw();
#endif // !__clang__
#endif // ASan instrumentation enabled
}
26 changes: 1 addition & 25 deletions tests/utils/stl/test/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,31 +290,7 @@ def _parseFlags(self, litConfig):

# clang doesn't know how to link in the VS version of the asan runtime automatically
if 'asan' in self.config.available_features and 'clang' in self.config.available_features:
targetArch = litConfig.target_arch.casefold()
archString = str()
if targetArch == 'x86'.casefold():
archString = 'i386'
elif targetArch == 'x64'.casefold():
archString = 'x86_64'

dbgString = str()
if 'debug_CRT' in self.config.available_features:
dbgString = '_dbg'

if 'dynamic_CRT' in self.config.available_features:
self.linkFlags.append('/wholearchive:clang_rt.asan' + dbgString + '_dynamic_runtime_thunk-' +
archString + '.lib')
self.linkFlags.append('clang_rt.asan' + dbgString + '_dynamic-' + archString + '.lib')

if targetArch == 'x86'.casefold():
self.linkFlags.append('/include:___asan_seh_interceptor')
elif targetArch == 'x64'.casefold():
self.linkFlags.append('/include:__asan_seh_interceptor')
else:
self.linkFlags.append('/wholearchive:clang_rt.asan' + dbgString + '-' + archString + '.lib')
self.linkFlags.append('clang_rt.asan_cxx' + dbgString + '-' + archString + '.lib')

self.linkFlags.append('/incremental:no')
self.linkFlags.append("/INFERASANLIBS")
StephanTLavavej marked this conversation as resolved.
Show resolved Hide resolved


class LibcxxTest(STLTest):
Expand Down