build: add option to enable clang-cl on Windows

[targos]

The goal here is to do the right changes with lessons learned from #35433.

[nodejs-github-bot]

Review requested:

• @nodejs/actions
• @nodejs/gyp

[targos]

Stopping for now. Locally, current error is:

  ..\..\out\Release\\obj\global_intermediate\icudt75l_dat.obj: unknown machine: 0
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(1599,5): error MSB6006: Arrêt de "llvm-lib.exe" avec le code 1. [D:\Git\nodejs\node\tools\icu\icudata.vcxproj]

Which is #34201 (hack: 711a822)

[targos]

Rebased on #52873 so I can use what it does.
I added a better version of the ICU hack based on the suggestion from #34201 (comment). I say we wait for nodejs/build#3709 and test what happens with the cross-compiler in CI.

[StefanStojanovic]

Rebased on #52873 so I can use what it does. I added a better version of the ICU hack based on the suggestion from #34201 (comment). I say we wait for nodejs/build#3709 and test what happens with the cross-compiler in CI.

First of all, I'm very glad to see this new PR that aims to remove hardcoded stuff. Second, about the ICU hack, please see my comment from the original PR. Since that project is compiled for host architecture, for cross-compilation we cannot use _M_AMD64/_M_ARM64

[targos]

I have no simple idea to fix it then. If we look at Chromium, I think they just prebuild the data files and add them to source control:
https://chromium.googlesource.com/chromium/deps/icu/+/refs/heads/main/README.chromium https://chromium.googlesource.com/chromium/deps/icu/+/refs/heads/main/common/

[targos]

This feature interests me. Looking at the code, this looks like a PoC, which is completely fine for this stage, but I was just wondering what you see as the end result here @targos?

The way I see it, clang and MSVC would both have to be supported in parallel for some time, before potentially moving to Clang completely. In that time, we'd need a way to choose either of them, eg. vcbuild.bat would still use MSVC, while vcbuild.bat clang would use clang.

Luckily we could leverage __clang__ to have separate code for the 2 compilers, but we'd probably need either to push some deps changes upstream, or leave them as floating patches. Overall, it would be good to know the scope of changes and affected dependencies once PoC is done so productization can be planned well.

Originally posted by @StefanStojanovic in #35433 (review)

[targos]

I ported everything from the previous PR. Build works on my Windows PC.

[targos]

When building locally, I noticed a lot of warnings on the OpenSSL side. Should we care about them?

@nodejs/crypto

Unknown escape sequence:

openssl\crypto\ct\ct_log.c(173,15): warning : unknown escape sequence '\ ' [-Wunknown-escape-sequence] [D:\a\node\node\deps\openssl\openssl.vcxproj]
openssl\include\internal/cryptlib.h(71,35): message : expanded from macro 'CTLOG_FILE' [D:\a\node\node\deps\openssl\openssl.vcxproj]
<command line>(48,32): message : expanded from macro 'OPENSSLDIR' [D:\a\node\node\deps\openssl\openssl.vcxproj]
openssl\crypto\ct\ct_log.c(173,15): warning : unknown escape sequence '\ ' [-Wunknown-escape-sequence] [D:\a\node\node\deps\openssl\openssl.vcxproj]
openssl\include\internal/cryptlib.h(71,35): message : expanded from macro 'CTLOG_FILE' [D:\a\node\node\deps\openssl\openssl.vcxproj]
<command line>(48,47): message : expanded from macro 'OPENSSLDIR' [D:\a\node\node\deps\openssl\openssl.vcxproj]

Unused function:

openssl\ssl\ssl_sess.c(27,1): warning : unused function 'sk_SSL_SESSION_value' [-Wunused-function] [D:\a\node\node\deps\openssl\openssl.vcxproj]
openssl\include\openssl/../../../config/././archs/VC-WIN64A/asm/include/openssl/safestack.h(175,29): message : expanded from macro 'DEFINE_STACK_OF' [D:\a\node\node\deps\openssl\openssl.vcxproj]
openssl\include\openssl/../../../config/././archs/VC-WIN64A/asm/include/openssl/safestack.h(73,40): message : expanded from macro 'SKM_DEFINE_STACK_OF' [D:\a\node\node\deps\openssl\openssl.vcxproj]

You can see them all here: https://github.com/nodejs/node/actions/runs/8989345480/job/24692243725

[lemire]

This switches the builds to ClangCL, correct? Meaning it is not an option, it just does it... right?

(Not an objection, I just want to clarify the intention.)

[targos]

It's what it does for now, but my intention is to refactor to add an option.

[StefanStojanovic]

I have no simple idea to fix it then. If we look at Chromium, I think they just prebuild the data files and add them to source control:
https://chromium.googlesource.com/chromium/deps/icu/+/refs/heads/main/README.chromium https://chromium.googlesource.com/chromium/deps/icu/+/refs/heads/main/common/

I'll look into these links and the ICU issue generally in much more detail, but for us .dat files are not an issue. We have a problem in the next step when making a .obj file out of it. I'll get back to you once I have a better understanding of this.

It's what it does for now, but my intention is to refactor to add an option.

This should be the way to go. Eventually, we can move to using Clang in the CI, but this should be done in phases and while maintaining MSVC.

[targos]

Added an option in 58a7bf7

[targos]

Rebased, squashed into a single commit, removed the ICU hack.
Now MSVC should be the default and I think we can land this PR.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/59167/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/59199/

[StefanStojanovic]

Just a note on this, Most likely, after landing V8 v12.4 (last change to V8) there is an error cross-compiling to ARM64 with ClangCL. I'm certain that previously I could build it if I just changed *pCPU value.

I'm currently working on that ICU issue, so I'll also investigate this as part of that. The error I'm getting btw is this:

  In file included from ..\..\deps\v8\src\objects\elements.cc:28:
  In file included from ..\..\deps\v8\third_party/fp16/src/include/fp16.h:5:
  In file included from ..\..\deps\v8\third_party\fp16\src\include\fp16/fp16.h:17:
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(28,9): error : use of undeclared identifier '_CopyFloatFromInt32' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(46,20): error : use of undeclared identifier '_CopyInt32FromFloat' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(64,9): error : use of undeclared identifier '_CopyDoubleFromInt64' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(82,20): error : use of undeclared identifier '_CopyInt64FromDouble' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]

[targos]

This needs a new approval after my force-push

[StefanStojanovic]

Let's give it a try, although my approval is gray so I doubt it will help you...

[targos]

@lemire Can you give me another green tick?

[targos]

Just a note on this, Most likely, after landing V8 v12.4 (last change to V8) there is an error cross-compiling to ARM64 with ClangCL. I'm certain that previously I could build it if I just changed *pCPU value.

I'm currently working on that ICU issue, so I'll also investigate this as part of that. The error I'm getting btw is this:

  In file included from ..\..\deps\v8\src\objects\elements.cc:28:
  In file included from ..\..\deps\v8\third_party/fp16/src/include/fp16.h:5:
  In file included from ..\..\deps\v8\third_party\fp16\src\include\fp16/fp16.h:17:
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(28,9): error : use of undeclared identifier '_CopyFloatFromInt32' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(46,20): error : use of undeclared identifier '_CopyInt32FromFloat' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(64,9): error : use of undeclared identifier '_CopyDoubleFromInt64' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]
..\..\deps\v8\third_party\fp16\src\include\fp16/bitcasts.h(82,20): error : use of undeclared identifier '_CopyInt64FromDouble' [E:\work\node\tools\v8_gypfiles\v8_base_without_compiler.vcxproj]

FWIW I found that support for _Copy* intrinsics was added to LLVM 18.1.0 (llvm/llvm-project@03c698a)

[lemire]

@targos

Why is this not using std::bit_cast?

https://en.cppreference.com/w/cpp/numeric/bit_cast

[nodejs-github-bot]

Landed in efa63f3

[StefanStojanovic]

FWIW I found that support for _Copy* intrinsics was added to LLVM 18.1.0 (llvm/llvm-project@03c698a)

I already have a PR in FP16 not to use _Copy* for ClangCL. As there is no LLVM 18.0.0 from what I see in their releases, I could potentially use __clang_major__ to restrict it to valid ClangCL versions only. Although I couldn't test it with ClangCL installed from Visual Studio, I'll try downloading LLVM and checking if it works correctly with v18.x

Why is this not using std::bit_cast?

The FP16 is a header-only library and states Compatible with C99 and C++11. Since std::bit_cast is a C++20 feature it cannot be used there.