Meta issue to track changes for adding plugin and LLVM_BUILD_LLVM_DYLIB support for Windows

[fsfod]

This is meta issue to track adding support for building LLVM and Clang as a shared library on windows and to also allow the use on plugins for Support clang plugins on Windows GSOC project. This will mostly be achieved by adding explicit visibility macros to public API surface that will dllexport'ed. These changes will also benefit non windows platforms by reducing the numbers of exported symbols when the default symbol visibility is set to hidden at a latter time.
Some rational for these changes is explained in this old LLVM discourse post Supporting LLVM_BUILD_LLVM_DYLIB on Windows.

With the current code merged in it is possible build a LLVM DLL using clang-cl, but not all the required symbols exported yet so tools aren't buildable. A branch with most of the changes from the open PR's merged in can be found here along with full visibility macros to create a working windows build with working plugins for LLVM and Clang. The CMake options to use are -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_BUILD_LLVM_DYLIB_VIS=ON -DLLVM_LINK_LLVM_DYLIB=ON -DCLANG_LINK_CLANG_DYLIB=ON

LLVM

• Initial changes for llvm shared library build using explicit visibility annotations #96630
• [TableGen] Add explicit symbol visibility macros to code generated #107873
• [llvm] Support llvm::Any across shared libraries on windows #108051
• Reland 'Update llvm::Registry to work for LLVM shared library builds on windows' (#109024) #112640
• [Object][NFC] Add extern template declarations needed by llvm-objdump #109156
• [X86] Switch to the new symbol visibility macros #109982
• Add visibility macros for Target initialize function declarations both c and cpp versions needed by llvm-c tests.
• Add symbol visibility macros to abi-breaking.h.cmake #110898
• Add explicit symbol visibility macros to InstrProfData.inc #110732
• [Support] Include Support/thread.h before api implementations #111175.
• [llvm] Add explicit visibility macros to YAMLTraits classes #111484
• [SandboxIR] Add extern templates for GlobalWithNodeAPI #111940
• [llvm] Enable building Analysis plugins on windows #112303
• [Minidump] Add extern template declarations for MinidumpFile::getListStream #112568
• [Support] Add clang tooling generated explicit visibility macros #113097
• [ORC] Add visibility macros to functions needed by lli and jitlink #113271
• [Remarks] Fix missing include in HotnessThresholdParser for Expected #113755
• [ORC] Switch to new visibility macros for JIT debug symbols #113848
• Export LLVM plugin entrypoint for windows shared library builds #115431
• [JITLink][AArch32] Add explicit visibility macros to functions needed by unittests #116557
• Manual visibility macros for explicit function template instantiations in LLVM
• Clang tooling generated visibility macros for LLVM.
• clang-cl template dllexport workaround.
• Switch LLVM visibility macros to be fully driven by CMake like Clang, gn build system will have to be updated as well
• Default symbol visibility on non windows platforms set to hidden.
• MSVC deleted and defaulted class members changes.
• MSVC dllexport ilist fix or default template argument workaround.
• MSVC dllexport cl::opt fix or workaround.

Clang

• [Clang] Add explicit visibility symbol macros #108276
• [Clang][TableGen] Add explicit symbol visibility macros to code generated #109362
• [clang-tools-extra] Fix add_clang_library usage #109321
• [Clang] Change Attribute plugin to link clang-cpp like other examples #109319
• [Clang] Update Interpreter tests to use clang_target_link_libraries #110154
• [Clang][ASTMatchers] Add visibility macros to variables declared by macros #110206
• [clangd] Update clangDaemonTweaks to set symbol visibility macros #112304
• Sketch of clang tooling generated visibility macros for Clang Clang tooling generated visibility macros for Clang  #109702
• MSVC deleted and defaulted class members changes.

Clang Tooling Fixes and improvements

• Update Clang to store source location for explicit function template instantiation definitions and declarations so tool can annotate them and not guess
• Make FrontendAction::BeginSourceFile virtual so proxy Actions work correctly
• Fix non usable API in FixItRewriter to access computed changes without writing a file

[annamthomas]

hi @fsfod , I'm having trouble building my local downstream fork of LLVM which uses dynamic build and link (i.e. these options are present: -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON).

If I revert back to before the first change landed in this series, it builds completely fine.
The reason is that we have some dynamic dependencies between our downstream library and LLVM and when building those files, it complains of same issue as reported here: #107687

i.e.

[02:35:10] #include "llvm/CodeGen/GenVT.inc"
[02:35:10]          ^~~~~~~~~~~~~~~~~~~~~~~~
[02:35:10] 1 error generated.

Is there any workarounds you can suggest for this? I've tried adding the -DLLVM_BUILD_LLVM_DYLIB_VIS=ON but it does not help.

[fsfod]

I'm not sure why my changes would cause that file not to be generated. I did notice there was a merged PR thats looks similar to your issue #109306

[annamthomas]

I'm on truck but that didn't fix it.

After multiple iterations, what I see finally helped is reverting the change:
283c2c8: "[TableGen] Add explicit symbol visibility macros to code generated (#107873)"

And I can build on trunk. How does the visibility macros work actually?

[fsfod]

For non windows and i think mingw it would be turned in to __attribute__((visibility("default"))) which is redundant atm because the default symbols visibility has not yet been changed to hidden. On windows platforms it should be empty unless LLVM_BUILD_LLVM_DYLIB_VIS is used.