From 5b09c005e5fe958715e715cb79b807f2ade1c285 Mon Sep 17 00:00:00 2001 From: Philippe Blain Date: Tue, 26 May 2020 17:11:27 -0400 Subject: [PATCH] enable macOS build Add support for building GDB on macOS. On macOS, debuggers need to be codesigned with a codesigning certificate located in the System keychain to be able to control other processes. The script `recipe/macos-codesign/macos-setup-codesign.sh` sets up this certificate. It is taken from the LLDB repository [1] and only slightly modified to use "gdb_codesign" as the certificate name. Using a script seems more robust than the manual method mentioned in the GDB wiki [2]. This script requires 'sudo', so it is run automatically on the CI but not in a user installation. It is copied to the installation prefix so that users can run it after installing GDB. The script `recipe/macos-codesign/macos-codesign-gdb.sh` actually calls 'macos-setup-codesign.sh' and then signs the GDB executable using the just created certificate. A post-link script runs the `recipe/macos-codesign/macos-codesign-gdb.sh` is passwordless sudo is available (in the CI), and if it's not, it writes to $PREFIX/.messages.txt such that users are notified that they need to sign GDB in order to use it, using the provided script. An activate script is also added to make sure that GDB is correctly codesigned upon environment activation. If it's not codesigned, users are instructed to run the provided script. Since the Python executable from the conda-forge python package does not include debugging symbols on macOS (see [3],[4]), we skip the test for the GDB "libpython" integration and add a simple "Hello World" C program, just to test that the debugger can actually run a program. We use Travis CI for the macOS build since the conda-forge Travis macOS image use macOS 10.13 ('xcode9.4') [5]. When tested under Azure and Travis under macOS 10.14 and 10.15, the build phase succeeds but when running the executable in GDB, GDB hangs. This is probably due to system security settings that need tweaking (I'm suspecting a graphical authentification popup since that's what I witnessed during local testing). We also add a README in the recipe dir to warn maintainers and contributors that it's possible that the automated testing starts failing when the Travis CI macOS 10.13 image is decomissioned (by Travis or conda-forge). [1]: https://github.com/llvm/llvm-project/blob/master/lldb/scripts/macos-setup-codesign.sh [2]: https://sourceware.org/gdb/wiki/PermissionsDarwin [3]: https://github.com/conda-forge/gdb-feedstock/pull/23/#issuecomment-643008755 [4]: https://github.com/conda-forge/python-feedstock/issues/354 [5]: https://docs.travis-ci.com/user/reference/osx/ --- conda-forge.yml | 2 + recipe/README.md | 11 ++++ recipe/activate.sh | 9 +++ recipe/build.sh | 25 +++++++- recipe/macos-codesign/gdb-entitlement.xml | 9 +++ recipe/macos-codesign/macos-codesign-gdb.sh | 14 +++++ recipe/macos-codesign/macos-setup-codesign.sh | 59 +++++++++++++++++++ .../macos-setup-codesign.sh.patch | 25 ++++++++ recipe/meta.yaml | 13 +++- recipe/post-link.sh | 40 +++++++++++++ recipe/run_test.sh | 20 +++++++ recipe/testing/hello.c | 5 ++ 12 files changed, 229 insertions(+), 3 deletions(-) create mode 100644 recipe/README.md create mode 100644 recipe/activate.sh create mode 100644 recipe/macos-codesign/gdb-entitlement.xml create mode 100755 recipe/macos-codesign/macos-codesign-gdb.sh create mode 100755 recipe/macos-codesign/macos-setup-codesign.sh create mode 100644 recipe/macos-codesign/macos-setup-codesign.sh.patch create mode 100644 recipe/post-link.sh create mode 100644 recipe/testing/hello.c diff --git a/conda-forge.yml b/conda-forge.yml index 403cb0b..20bc654 100644 --- a/conda-forge.yml +++ b/conda-forge.yml @@ -1,3 +1,5 @@ +provider: + osx: travis appveyor: secure: {BINSTAR_TOKEN: tumuXLL8PU75WMnRDemRy02ruEq2RpNxeK3dz0MjFssnosPm2v4EFjfNB4PTotA1} compiler_stack: comp7 diff --git a/recipe/README.md b/recipe/README.md new file mode 100644 index 0000000..c38e6b8 --- /dev/null +++ b/recipe/README.md @@ -0,0 +1,11 @@ +Note: the macOS build uses Travis because the conda-forge Travis config uses macOS 10.13, +and running an executable in GDB under SSH on macOS 10.14+ (including the Travis and Azure images) makes GDB hang. + +This is probably related to the fact that some system security settings need to be adjusted +for the debugger to be able to run unimpeded (I'm guessing that it's waiting for a graphical +authentication popup window somehow.) + +[This person][1] suggests using an entitlement file with additional entitlements compared to the [official recommendation][2], so it could be worth investigating adding these entitlements when the 10.13 Travis image is decomissioned. + +[1]: https://timnash.co.uk/getting-gdb-to-semi-reliably-work-on-mojave-macos/ +[2]: https://sourceware.org/gdb/wiki/PermissionsDarwin#Sign_and_entitle_the_gdb_binary diff --git a/recipe/activate.sh b/recipe/activate.sh new file mode 100644 index 0000000..aab0870 --- /dev/null +++ b/recipe/activate.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +# Check gdb is codesigned +if [[ $(uname) == "Darwin" ]]; then + if ! codesign -vv $CONDA_PREFIX/bin/gdb; then + echo "Warning: GDB is not codesigned." + cat $CONDA_PREFIX/etc/gdb/.messages.txt + fi +fi diff --git a/recipe/build.sh b/recipe/build.sh index f345e54..3deb363 100644 --- a/recipe/build.sh +++ b/recipe/build.sh @@ -1,5 +1,7 @@ #!/bin/bash +set -eu + # Download the right script to debug python processes curl -SL https://raw.githubusercontent.com/python/cpython/$PY_VER/Tools/gdb/libpython.py \ > "$SP_DIR/libpython.py" @@ -17,6 +19,23 @@ gdb.events.new_objfile.connect(setup_python) end ' >> "$PREFIX/etc/gdbinit" +# macOS specificities +if [[ $target_platform == "osx-64" ]]; then + # prevent a VERSION file being confused by clang++ with $CONDA_PREFIX/include/c++/v1/version + mv intl/VERSION intl/VERSION.txt + # install needed scripts to generate a codesigning certificate and sign the gdb executable + cp $RECIPE_DIR/macos-codesign/macos-setup-codesign.sh $PREFIX/bin/ + cp $RECIPE_DIR/macos-codesign/macos-codesign-gdb.sh $PREFIX/bin/ + # copy the entitlement file + mkdir -p $PREFIX/etc/gdb + cp $RECIPE_DIR/macos-codesign/gdb-entitlement.xml $PREFIX/etc/gdb/ + # add libiconv and expat flags + libiconv_flag="--with-libiconv-prefix=$PREFIX" + expat_flag="--with-libexpat-prefix=$PREFIX" + # Setup the necessary GDB startup command for macOS Sierra and later + echo "set startup-with-shell off" >> "$PREFIX/etc/gdbinit" +fi + export CPPFLAGS="$CPPFLAGS -I$PREFIX/include" # Setting /usr/lib/debug as debug dir makes it possible to debug the system's # python on most Linux distributions @@ -24,6 +43,10 @@ export CPPFLAGS="$CPPFLAGS -I$PREFIX/include" --prefix="$PREFIX" \ --with-separate-debug-dir="$PREFIX/lib/debug:/usr/lib/debug" \ --with-python \ - --with-system-gdbinit="$PREFIX/etc/gdbinit" || (cat config.log && exit 1) + --with-system-gdbinit="$PREFIX/etc/gdbinit" \ + ${libiconv_flag:-} \ + ${expat_flag:-} \ + || (cat config.log && exit 1) make -j${CPU_COUNT} make install + diff --git a/recipe/macos-codesign/gdb-entitlement.xml b/recipe/macos-codesign/gdb-entitlement.xml new file mode 100644 index 0000000..bfce04f --- /dev/null +++ b/recipe/macos-codesign/gdb-entitlement.xml @@ -0,0 +1,9 @@ + + + + + com.apple.security.cs.debugger + + + + diff --git a/recipe/macos-codesign/macos-codesign-gdb.sh b/recipe/macos-codesign/macos-codesign-gdb.sh new file mode 100755 index 0000000..644174c --- /dev/null +++ b/recipe/macos-codesign/macos-codesign-gdb.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +set -eu + +# macOS specificities: codesign the GDB executable +# Generate code-signing certificate (needs `sudo`) +$CONDA_PREFIX/bin/macos-setup-codesign.sh +# unset a variable set by old versions of the clang activation script that prevents using `/usr/bin/codesign` +# (in case old builds of conda-forge compilers are installed in the installation environment) +# see https://github.com/conda-forge/clang-compiler-activation-feedstock/issues/18 +# and https://github.com/conda-forge/clang-compiler-activation-feedstock/pull/19 +unset CODESIGN_ALLOCATE +# Sign the GDB binary +codesign --entitlements $CONDA_PREFIX/etc/gdb/gdb-entitlement.xml --force --sign gdb_codesign $CONDA_PREFIX/bin/gdb diff --git a/recipe/macos-codesign/macos-setup-codesign.sh b/recipe/macos-codesign/macos-setup-codesign.sh new file mode 100755 index 0000000..6f592c5 --- /dev/null +++ b/recipe/macos-codesign/macos-setup-codesign.sh @@ -0,0 +1,59 @@ +#!/bin/bash + +# This script is copied from https://github.com/llvm/llvm-project/blob/master/lldb/scripts/macos-setup-codesign.sh + +CERT="gdb_codesign" + +function error() { + echo error: "$@" 1>&2 + exit 1 +} + +function cleanup { + # Remove generated files + rm -f "$TMPDIR/$CERT.tmpl" "$TMPDIR/$CERT.cer" "$TMPDIR/$CERT.key" > /dev/null 2>&1 +} + +trap cleanup EXIT + +# Check if the certificate is already present in the system keychain +security find-certificate -Z -p -c "$CERT" /Library/Keychains/System.keychain > /dev/null 2>&1 +if [ $? -eq 0 ]; then + echo Certificate has already been generated and installed + exit 0 +fi + +# Create the certificate template +cat <$TMPDIR/$CERT.tmpl +[ req ] +default_bits = 2048 # RSA key size +encrypt_key = no # Protect private key +default_md = sha512 # MD to use +prompt = no # Prompt for DN +distinguished_name = codesign_dn # DN template +[ codesign_dn ] +commonName = "$CERT" +[ codesign_reqext ] +keyUsage = critical,digitalSignature +extendedKeyUsage = critical,codeSigning +EOF + +echo Generating and installing gdb_codesign certificate + +# Generate a new certificate +openssl req -new -newkey rsa:2048 -x509 -days 3650 -nodes -config "$TMPDIR/$CERT.tmpl" -extensions codesign_reqext -batch -out "$TMPDIR/$CERT.cer" -keyout "$TMPDIR/$CERT.key" > /dev/null 2>&1 +[ $? -eq 0 ] || error Something went wrong when generating the certificate + +# Install the certificate in the system keychain +sudo security add-trusted-cert -d -r trustRoot -p codeSign -k /Library/Keychains/System.keychain "$TMPDIR/$CERT.cer" > /dev/null 2>&1 +[ $? -eq 0 ] || error Something went wrong when installing the certificate + +# Install the key for the certificate in the system keychain +sudo security import "$TMPDIR/$CERT.key" -A -k /Library/Keychains/System.keychain > /dev/null 2>&1 +[ $? -eq 0 ] || error Something went wrong when installing the key + +# Kill task_for_pid access control daemon +sudo pkill -f /usr/libexec/taskgated > /dev/null 2>&1 + +# Exit indicating the certificate is now generated and installed +exit 0 diff --git a/recipe/macos-codesign/macos-setup-codesign.sh.patch b/recipe/macos-codesign/macos-setup-codesign.sh.patch new file mode 100644 index 0000000..72cf479 --- /dev/null +++ b/recipe/macos-codesign/macos-setup-codesign.sh.patch @@ -0,0 +1,25 @@ +--- macos-setup-codesign.sh 2020-06-13 20:43:24.000000000 -0400 ++++ recipe/macos-codesign/macos-setup-codesign.sh 2020-06-13 20:47:52.000000000 -0400 +@@ -1,9 +1,11 @@ + #!/bin/bash + +-CERT="lldb_codesign" ++# This script is copied from https://github.com/llvm/llvm-project/blob/master/lldb/scripts/macos-setup-codesign.sh ++ ++CERT="gdb_codesign" + + function error() { +- echo error: "$@" ++ echo error: "$@" 1>&2 + exit 1 + } + +@@ -36,7 +38,7 @@ + extendedKeyUsage = critical,codeSigning + EOF + +-echo Generating and installing lldb_codesign certificate ++echo Generating and installing gdb_codesign certificate + + # Generate a new certificate + openssl req -new -newkey rsa:2048 -x509 -days 3650 -nodes -config "$TMPDIR/$CERT.tmpl" -extensions codesign_reqext -batch -out "$TMPDIR/$CERT.cer" -keyout "$TMPDIR/$CERT.key" > /dev/null 2>&1 diff --git a/recipe/meta.yaml b/recipe/meta.yaml index 2c7a829..4584a46 100644 --- a/recipe/meta.yaml +++ b/recipe/meta.yaml @@ -10,8 +10,11 @@ source: sha256: 1e55b4d7cdca7b34be12f4ceae651623aa73b2fd640152313f9f66a7149757c4 build: - number: 1 - skip: True # [win or osx] + number: 2 + skip: True # [win] + # needed by macOS codesigning script + script_env: + - TMPDIR requirements: build: @@ -24,15 +27,21 @@ requirements: - texinfo - xz - zlib + - libiconv + - expat run: - python - ncurses - xz - zlib + - libiconv + - expat test: commands: - gdb --version + requires: + - {{ compiler('c') }} about: home: https://www.gnu.org/software/gdb/ diff --git a/recipe/post-link.sh b/recipe/post-link.sh new file mode 100644 index 0000000..062d56f --- /dev/null +++ b/recipe/post-link.sh @@ -0,0 +1,40 @@ +#!/bin/bash + +set -eu + +# Returns 0 if current user is in the sudoers file +# and sudo-ing does not require a password. +can_sudo_without_password () { +sudo -ln | \grep -q '(ALL) NOPASSWD: ALL' +} + +# macOS specificities: codesign the GDB executable +if [[ $(uname) == "Darwin" ]]; then + # On CI, sign the executable (for the tests) + if can_sudo_without_password; then + $PREFIX/bin/macos-codesign-gdb.sh + else + # Create the message shown at the end of installation + cat <<-EOF > $PREFIX/.messages.txt + Due to macOS security restrictions, the GDB executable + needs to be codesigned to be able to control other processes. + + This process requires administrative permissions + (your user must be able to run \`sudo\`). + + To codesign GDB, simply run the included script: + + macos-codesign-gdb.sh + + and enter your password. + + Make sure this environment, "$(basename $PREFIX)", is activated + so that "macos-codesign-gdb.sh" is found in your \$PATH. + + For more information, see: https://sourceware.org/gdb/wiki/PermissionsDarwin + EOF + # Copy the message file since we might need to show it in the activate script + # and conda deletes it after displaying it + cp $PREFIX/.messages.txt $PREFIX/etc/gdb/.messages.txt + fi +fi diff --git a/recipe/run_test.sh b/recipe/run_test.sh index 4830af0..4b297b6 100644 --- a/recipe/run_test.sh +++ b/recipe/run_test.sh @@ -5,7 +5,27 @@ echo "CONDA_PY:$CONDA_PY" export CONDA_PY=`python -c "import sys;print('%s%s'%sys.version_info[:2])"` echo "CONDA_PY:$CONDA_PY" +if [[ $(uname) == "Darwin" ]]; then + sudo /usr/sbin/DevToolsSecurity -enable + sudo security authorizationdb write system.privilege.taskport allow +fi + +# Show GDB startup files +gdb --help + +# Run hello world test +$CC -o hello -g "$RECIPE_DIR/testing/hello.c" +gdb -batch -ex "run" --args hello + +# Run python test +if [[ $(uname) == "Darwin" ]]; then + # Skip python test on macOS, since the Python executable is missing debug symbols. + # see https://github.com/conda-forge/gdb-feedstock/pull/23/#issuecomment-643008755 + # and https://github.com/conda-forge/python-feedstock/issues/354 + exit 0 +fi gdb -batch -ex "run" -ex "py-bt" --args python "$RECIPE_DIR/testing/process_to_debug.py" | tee gdb_output + if [[ "$CONDA_PY" != "27" ]]; then grep "built-in method kill" gdb_output fi diff --git a/recipe/testing/hello.c b/recipe/testing/hello.c new file mode 100644 index 0000000..03bde36 --- /dev/null +++ b/recipe/testing/hello.c @@ -0,0 +1,5 @@ +#include +int main() { + printf("Hello, World!\n"); + return 0; +}