CMake code to run ASAN for Windows tests

[JoergAtGithub]

This PR extends the CMake lists to run ASAN address sanitzer for tests under Windows (requires Win10 or 11).

• It builds a standalone (no dependencies to local VisualStudio installation pathes) executable with ASAN code integrated.
• In case of an address sanitizer failure, an ASAN Dump file is generated under out\build\x64-Debug\src\windows with a name like AsanDump_046D_C52F_0001_000C.dmp

[Youw]

BTW: in my other project(s) I didn't it. for ASAN

[JoergAtGithub]

Static linking is not needed in general, you could als add the ASAN DLL, that comes in various flavors with VisualStudio, to your DLL search path.
But it's placed in a directory with a name like this:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x86\clang_rt.asan_dbg_dynamic-i386.dll
I'm not able to determine this path using CMake. It depends on many things like VisualStudio installation path, Visual Studio version and the build type.

[Youw]

It is rather easy to get this path:

• it is MSVC-specitic (so unser if(MSVC));
• as per documentation - on MSVC the support is limited to x86 and x64 on Windows 10, so only one of two possible suffixes: i386 or x86_64;
• the location is documented and it matches the path to CMAKE_C_COMPILER directory;

---

But in any case - I don't believe we need to handle that in our CMake. ASAN runs are designed to be executed only in development builds (e.g. no "production" scenarios).
The runtime is always available from MSVC development sonsole, so I believe that is just enough.

[JoergAtGithub]

This setup works for me on my local system. If I inject an memory access error, it reports it, otherwise every test is "Passed":

But on GitHub CI, at least the --output-on-error functionality of ctest seems not to work. But this seems to be a general ctest issue, not related to this ASAN PR.

[Youw]

But on GitHub CI, at least the --output-on-error functionality of ctest seems not to work. But this seems to be a general ctest issue, not related to this ASAN PR.

Agree

[Youw]

warning C5072: ASAN enabled without debug information emission. Enable debug info for better ASAN error reporting

So we better make RelWithDebInfo builds on CI for ASAN.

---

I'm working in this right now. Will post an update shortly.

[JoergAtGithub]

This looks much better than my attemp! Works well for me on local VS2022 as on CI.
But I get Linker warnings:

Run cmake --build . --config RelWithDebInfo --target install
MSBuild version 17.4.1+9a89d02ff for .NET Framework
  Checking Build System
  Building Custom Rule D:/a/hidapi/hidapi/hidapisrc/windows/CMakeLists.txt
  hid.c
  hidapi_descriptor_reconstruct.c
  Generating Code...
LINK : warning LNK4044: unrecognized option '/fsanitize=address'; ignored [D:\a\hidapi\hidapi\build\msvc\src\windows\hidapi_winapi.vcxproj]
LINK : warning LNK4300: ignoring '/INCREMENTAL' because input module contains ASAN metadata [D:\a\hidapi\hidapi\build\msvc\src\windows\hidapi_winapi.vcxproj]
LINK : warning LNK4044: unrecognized option '/fsanitize=address'; ignored [D:\a\hidapi\hidapi\build\msvc\src\windows\hidapi_winapi.vcxproj]
     Creating library D:/a/hidapi/hidapi/build/msvc/src/windows/RelWithDebInfo/hidapi.lib and object D:/a/hidapi/hidapi/build/msvc/src/windows/RelWithDebInfo/hidapi.exp

[Youw]

Hm, interesting. I haven't seen this warning locally. But I use MSVC 2019 though.

With gcc this linker option even required (otherwise the ASAN library is not linked).
I guess we need another MSVC workaround.

[Youw]

As for:

warning LNK4300: ignoring '/INCREMENTAL' because input module contains ASAN metadata

I don't have a quick solution for it right now, and I'll probably keep it as is.

[JoergAtGithub]

Why not setting /INCREMENTAL:NO ?

[Youw]

AFAIK MSVC compiler/linker usually complains if the same option is passed multiple times, and /INCREMENTAL is already present in the link option list, so it is not about setting it, but rather replacing which is not as trivial as one might think

and I like simple/elegant solutions

but OK, I'll try

[Youw]

probably time to merge it, before it gets ugly again