Fix macOS compilation #5325

vit9696 · 2022-02-03T11:24:23Z

Tested on macOS 12.2 with the following compilation commands:

cmake -G "Unix Makefiles" -DCMAKE_CPP=$(which gcc)
make -j16

Dependencies like snappy were installed with Homebrew.

vit9696 · 2022-02-03T12:26:37Z

To be clear, the cpp on macOS is rather fragile, so one should preprocess with the compiler instead. It is a surprise for me that it worked as late as 10.15. Current error log:

CMake Error at CMakeLists.txt:1382 (message):
  ***
  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cpp
  failed: ***

  In file included from
  /Users/user/AppleDeveloper/dynamorio/build/configure_temp.h:146:

  In file included from
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/types.h:184:


  In file included from
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_types/_fd_def.h:32:


  In file included from
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/Availability.h:136:


  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:142:2:
  error: #endif without #if

  #endif /* #if defined(__has_builtin) */

   ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:150:2:
  error: #endif without #if

  #endif /* __ENABLE_LEGACY_IPHONE_AVAILABILITY */

   ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:153:32:
  error: missing '(' after '__has_attribute'

      #if defined(__has_attribute) && defined(__has_feature)
                                 ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:153:58:
  error: missing '(' after '__has_feature'

      #if defined(__has_attribute) && defined(__has_feature)
                                                           ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4432:6:
  error: #endif without #if

      #endif
       ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4433:2:
  error: #endif without #if

  #endif /* __ENABLE_LEGACY_MAC_AVAILABILITY */

   ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4484:41:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN2(a,b) __API_A_BEGIN(a) __API_A_BEGIN(b)
                                          ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4484:58:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN2(a,b) __API_A_BEGIN(a) __API_A_BEGIN(b)
                                                           ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4486:45:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN4(a,b,c,d) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d)
                                              ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4486:62:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN4(a,b,c,d) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d)
                                                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4486:79:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN4(a,b,c,d) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d)
                                                                                ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4486:96:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN4(a,b,c,d) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d)
                                                                                                 ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:49:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                  ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:66:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                                   ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:83:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                                                    ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:100:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                                                                     ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:117:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                                                                                      ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4488:134:
  error: _Pragma takes a parenthesized string literal

      #define __API_AVAILABLE_BEGIN6(a,b,c,d,e,f) __API_A_BEGIN(a) __API_A_BEGIN(b) __API_A_BEGIN(c) __API_A_BEGIN(d) __API_A_BEGIN(e) __API_A_BEGIN(f)
                                                                                                                                       ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4481:30:
  note: expanded from macro '__API_A_BEGIN'

      #define __API_A_BEGIN(x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_AVAILABLE_PLATFORM_##x))), apply_to = __API_APPLY_TO)))
                               ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4528:50:
  error: _Pragma takes a parenthesized string literal

      #define __API_DEPRECATED_BEGIN_MSG3(msg,a,b) __API_D_BEGIN(msg,a) __API_D_BEGIN(msg,b)
                                                   ^

  
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:4525:35:
  note: expanded from macro '__API_D_BEGIN'

      #define __API_D_BEGIN(msg, x) _Pragma(__API_RANGE_STRINGIFY (clang attribute (__attribute__((availability(__API_DEPRECATED_PLATFORM_##x,message=msg))), apply_to = __API_APPLY_TO)))
                                    ^

  fatal error: too many errors emitted, stopping now [-ferror-limit=]

  20 errors generated.

By the way, could you provide a link or an update to clarify what is the state of DynamoRIO on macOS? Thank you!

An extra indirection layer is necessary to force preprocessor expansion in cases where DECLARE_FUNC argument is itself a macro.

vit9696 · 2022-02-03T13:51:24Z

Ready for merging on my side. Test failure on ci-windows is apparently caused by a timeout (lag?) on the CI side and not by the changes.

derekbruening · 2022-02-03T16:13:32Z

Thank for the contribution.

For future updates please avoid force-pushing (as noted in the docs https://dynamorio.org/page_contributing.html, https://dynamorio.org/page_code_reviews.html) as it ruins multi-step reviews (can't see just new changes since review; messes up comment history). The final squash-and-merge into master will clean up all the commits.

derekbruening · 2022-02-03T16:21:11Z

I was going to say, please update our automated tests in .github/workflows/ci-osx.yml to use 12 along with your fixes here, but it looks like Github only goes to macos-11 today: https://github.com/actions/virtual-environments

We should still go to 11: if you update the test does it work with your changes?

vit9696 · 2022-02-03T18:01:07Z

It does compile with the new Xcode, but the test suite is indeed partly broken. Would you like me to disable the deprecation warning with a pragma?

vit9696 · 2022-02-03T18:14:17Z

Also, I failed to find this issue earlier, but now I see that this pull request resolves #4344.

derekbruening · 2022-02-03T23:19:37Z

It does compile with the new Xcode, but the test suite is indeed partly broken.

OK, so maybe we need to separate that out since it likely takes some work, and just put in the compilation fix in this PR.

Would you like me to disable the deprecation warning with a pragma?

Sorry, what warning is this?

vit9696 · 2022-02-04T01:25:42Z

Tests fail, because vfork is deprecated on macOS. Can either silence the warning or disable the test.

OK, so maybe we need to separate that out since it likely takes some work, and just put in the compilation fix in this PR.

Can do, but maybe first see how bad it is. Fixing the vfork one should be very straightforward.

derekbruening · 2022-02-04T01:35:20Z

Tests fail, because vfork is deprecated on macOS. Can either silence the warning or disable the test.

Disabling the test on mac seems reasonable.

OK, so maybe we need to separate that out since it likely takes some work, and just put in the compilation fix in this PR.

Can do, but maybe first see how bad it is. Fixing the vfork one should be very straightforward.

Sure, sounds good. Just didn't want to pile too much work at once here.

vit9696 · 2022-02-04T12:37:50Z

I had to additionally fix syscall-mod test, which had a crash with modern clang. There are two issues there:

syscall in Sys-V ABI has a defined list of clobbered registers, which is not respected (see x86-64-psABI-draft.pdf from https://gitlab.com/x86-psABIs/i386-ABI/-/issues/2). DR may have replaced syscall with something like mov -7, %eax during the test, but I should say it must not be relied on, as this code yields UB when DR fails to instrument it and the result of the test would be thus unreliable.
Overwritten %eax (by the first mov) is also not present in the clobber list, actually leading to clang assuming %eax is unchanged before and after the inline asm block, and thus causing the crash observed in the test failure.

Since one needs to define a complete list of clobbered registers, i.e. modified implicitly or explicitly aside the outputs, in the inline asm, I also tried to fix the code for other architectures. I do not have the AArch64 ABI nearby to be sure, and the i386 ABI does not define the int 0x80 ABI, so that would be Linux-specific, but the changes for the other architectures are good to cover at least case (2).

Hopefully we will be good now :-)

vit9696 · 2022-02-04T12:48:33Z

Yes, looks good now. code_api|client.winxfer failed on windows 32-bit, but I have not changed anything related, so it must be a sporadic failure.

derekbruening · 2022-02-04T20:03:34Z

code_api|client.winxfer failed on windows 32-bit, but I have not changed anything related, so it must be a sporadic failure.

Yes known flake #4732

By the way, could you provide a link or an update to clarify what is the state of DynamoRIO on macOS? Thank you!

Biggest missing thing on Mac is a private loader. Without it, large tools like Dr. Memory have their resources and libraries collide with large applications. So right now only small/isolated tools or small apps work. Another significant missing piece is Mac on AArch64 (e.g., M1) but it may not be too much work since Linux support on AArch64 is pretty good: probably mostly signal state and other things. At a high level, what's missing are new developers to own Mac support and spend time on it.

derekbruening

Just some nits and minor suggestions

core/arch/asm_defines.asm

suite/tests/CMakeLists.txt

make/cpp2asm_support.cmake

suite/tests/client-interface/syscall-mod.c

derekbruening

Looks good. Thank you for the patch.

derekbruening · 2022-02-05T17:06:29Z

add to allowlist

derekbruening · 2022-02-05T17:06:36Z

run arm tests

vit9696 added 3 commits February 3, 2022 16:24

Fix label concatenation with _ on macOS

69d9613

An extra indirection layer is necessary to force preprocessor expansion in cases where DECLARE_FUNC argument is itself a macro.

Conditionally use SYS_waitevent syscall removed in macOS 11.x

46d5105

Workaround snappy being found but remain unusable on macOS

a87f0fb

vit9696 added 2 commits February 3, 2022 20:42

Update macOS runner to macOS 11

f1791d4

Update macOS Xcode version to 13.2.1 (latest available on GitHub)

e376267

vit9696 added 4 commits February 4, 2022 14:01

Disable vfork test on macOS as this function is deprecated

ee8be86

Add clobber list to syscall-mod test

7a6f04a

Also add clobber list to other architectures in syscall-mod

1da7d9c

Fix accidental commit of the wrong code

19c2afc

derekbruening approved these changes Feb 4, 2022

View reviewed changes

Address review comments

3ff5a5e

derekbruening approved these changes Feb 5, 2022

View reviewed changes

Merge branch 'master' into master

db20a76

derekbruening merged commit 9696bb2 into DynamoRIO:master Feb 5, 2022

vit9696 mentioned this pull request Feb 5, 2022

Implement MacOS private loader #1285

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix macOS compilation #5325

Fix macOS compilation #5325

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

derekbruening commented Feb 3, 2022

derekbruening commented Feb 3, 2022

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

derekbruening commented Feb 3, 2022

vit9696 commented Feb 4, 2022 •

edited

Loading

derekbruening commented Feb 4, 2022

vit9696 commented Feb 4, 2022

vit9696 commented Feb 4, 2022

derekbruening commented Feb 4, 2022

derekbruening left a comment

derekbruening left a comment

derekbruening commented Feb 5, 2022

derekbruening commented Feb 5, 2022

Fix macOS compilation #5325

Fix macOS compilation #5325

Conversation

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

derekbruening commented Feb 3, 2022

derekbruening commented Feb 3, 2022

vit9696 commented Feb 3, 2022

vit9696 commented Feb 3, 2022

derekbruening commented Feb 3, 2022

vit9696 commented Feb 4, 2022 • edited Loading

derekbruening commented Feb 4, 2022

vit9696 commented Feb 4, 2022

vit9696 commented Feb 4, 2022

derekbruening commented Feb 4, 2022

derekbruening left a comment

Choose a reason for hiding this comment

derekbruening left a comment

Choose a reason for hiding this comment

derekbruening commented Feb 5, 2022

derekbruening commented Feb 5, 2022

vit9696 commented Feb 4, 2022 •

edited

Loading