pivy-tool segfaults when initializing Yubikey on MacOS M1

[arjenroodselaar]

I am trying to run pivy-tool on MacOS on an M1 SoC and am running into the following segfault:

arjen@mbp pivy % lldb ./pivy-tool
(lldb) target create "./pivy-tool"
Current executable set to '/Users/arjen/oxidecomputer/pivy/pivy-tool' (arm64).
(lldb) run init
Process 87019 launched: '/Users/arjen/oxidecomputer/pivy/pivy-tool' (arm64)
Process 87019 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3f)
    frame #0: 0x0000000181caea9c libsystem_platform.dylib`_platform_strcmp + 12
libsystem_platform.dylib`:
->  0x181caea9c <+12>: ldrb   w5, [x1], #0x1
    0x181caeaa0 <+16>: subs   x3, x4, x5
    0x181caeaa4 <+20>: ccmp   w4, #0x0, #0x4, eq
    0x181caeaa8 <+24>: b.ne   0x181caea90               ; <+0>
Target 0: (pivy-tool) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3f)
  * frame #0: 0x0000000181caea9c libsystem_platform.dylib`_platform_strcmp + 12
    frame #1: 0x000000010002917c pivy-tool`errf_caused_by(e=0x0000000000000033, name="DeviceOutOfMemoryError") at errf.c:265:7 [opt]
    frame #2: 0x0000000100006680 pivy-tool`cmd_init at pivy-tool.c:1088:6 [opt]
    frame #3: 0x0000000100004d5c pivy-tool`main(argc=2, argv=0x000000016fdff460) at pivy-tool.c:3044:9 [opt]
    frame #4: 0x00000001819010e0 dyld`start + 2360
(lldb) register read
General Purpose Registers:
        x0 = 0x00000001001efce2  "eviceOutOfMemoryError"
        x1 = 0x000000000000003f
        x2 = 0x0000000000000002
        x3 = 0x0000000000000044
        x4 = 0x0000000000000044
        x5 = 0x0000000000000000
        x6 = 0x0000000004000100
        x7 = 0x0000000000000000
        x8 = 0xc3a89c20274c002b
        x9 = 0xc3a89c20274c002b
       x10 = 0x0100000100000000 (0x0000000100000000) pivy-tool`_mh_execute_header
       x11 = 0x000000100003fa23
       x12 = 0x708c3d9f02040204
       x13 = 0x0002000110000400
       x14 = 0x0000000200001000
       x15 = 0x00020000100003d8
       x16 = 0x0000000181caea90  libsystem_platform.dylib`_platform_strcmp
       x17 = 0x00000001dc301e38  (void *)0x0000000181cb1650: _platform_memmove
       x18 = 0x0000000000000000
       x19 = 0x00000001001efce1  "DeviceOutOfMemoryError"
       x20 = 0x0000000000000033
       x21 = 0x0000600002dfc000
       x22 = 0x000000016fdfeff8
       x23 = 0x00000001001efab7  "PermissionError"
       x24 = 0x00000001001efcd3  "ArgumentError"
       x25 = 0x000000010022e000  X509V3_str_reasons + 216
       x26 = 0x0000000100228000  optstring
       x27 = 0x0000000100228000  optstring
       x28 = 0x0000000100228000  optstring
        fp = 0x000000016fdff030
        lr = 0x000000010002917c  pivy-tool`errf_caused_by + 56 at errf.c:265:7
        sp = 0x000000016fdff010
        pc = 0x0000000181caea9c  libsystem_platform.dylib`_platform_strcmp + 12
      cpsr = 0x00001000

Looking at the pc this seems to be an invalid data fetch.

I tried running the x86_64 version of the binary and get further into unwinding the exception, but it seems to print the some memory, the contents of the PATH variable and then some more (stack or heap?) memory (censored on purpose) and then segfaults as well:

arjen@mbp ~ % env /usr/bin/arch -x86_64 /opt/pivy/bin/pivy-tool init
pivy-tool: error occurred while executing 'init'
  Caused by cmd_init: failed to write to card
    in cmd_init() at pivy-tool.c:1097
  Caused by :
    in () at :2
  Caused by <dirty memory from what looks a prior process here>
    in :<PATH variable contents>:<partial data from something else>
zsh: segmentation fault  env /usr/bin/arch -x86_64 /opt/pivy/bin/pivy-tool init

Anything I can/should try to provide more debug info?

Edit: this behavior happens with both the 0.11.2 release and main.

[arjenroodselaar]

Poking at this a bit more, I realized that release 0.11.1 worked. Manually bisecting the commit tree things seem to start failing for me with commit d28025a and onwards. During this build clang gives a warning about a codepath where an errf_t struct is not initialized:

arjen@mbp pivy % git checkout d28025a15dcc7bf9b783eca2a0026323634a70de
Previous HEAD position was 12f970b libpivy: add useful support functions
HEAD is now at d28025a piv: abstract out cardcap parsing/generating from pivy-tool
arjen@mbp pivy % make -j4
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o pivy-tool.o -c pivy-tool.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o piv.o -c piv.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o tlv.o -c tlv.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o debug.o -c debug.c
In file included from pivy-tool.c:59:
./piv-ca.h:341:42: warning: declaration of 'struct json_tokener' will not be visible outside of this function [-Wvisibility]
size_t json_tokener_get_parse_end(struct json_tokener *tok);
                                         ^
piv.c:8748:6: warning: variable 'err' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
        if (*out == NULL) {
            ^~~~~~~~~~~~
piv.c:8757:10: note: uninitialized use occurs here
        return (err);
                ^~~
piv.c:8748:2: note: remove the 'if' if its condition is always true
        if (*out == NULL) {
        ^~~~~~~~~~~~~~~~~~
piv.c:8684:13: note: initialize the variable 'err' to silence this warning
        errf_t *err;
                   ^
                    = NULL
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o bunyan.o -c bunyan.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o errf.o -c errf.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o utils.o -c utils.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o readpassphrase.o -c readpassphrase.c
1 warning generated.
In file included from pivy-tool.c:59:
./piv-ca.h:341:42: warning: declaration of 'struct json_tokener' will not be visible outside of this function [-Wvisibility]
size_t json_tokener_get_parse_end(struct json_tokener *tok);
                                         ^
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o piv-certs.o -c piv-certs.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64 -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"'  -O2 -g -D_GNU_SOURCE -DPIVY_VERSION='"0.11.1"' -o pkinit_asn1.o -c pkinit_asn1.c
In file included from piv-certs.c:62:
./piv-ca.h:341:42: warning: declaration of 'struct json_tokener' will not be visible outside of this function [-Wvisibility]
size_t json_tokener_get_parse_end(struct json_tokener *tok);
                                         ^
piv-certs.c:1379:37: warning: format specifies type 'unsigned long' but the argument has type 'uint64_t' (aka 'unsigned long long') [-Wformat]
                rc = sshbuf_putf(sbuf, "S-1-%lu", idauth);
                                            ~~~   ^~~~~~
                                            %llu
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include  -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -o pivy-agent.o -c pivy-agent.c
1 warning generated.
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include   -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -std=gnu99 -o pivy-box.o -c pivy-box.c
1 warning generated.
2 warnings generated.
In file included from piv-certs.c:62:
./piv-ca.h:341:42: warning: declaration of 'struct json_tokener' will not be visible outside of this function [-Wvisibility]
size_t json_tokener_get_parse_end(struct json_tokener *tok);
                                         ^
piv-certs.c:1379:37: warning: format specifies type 'unsigned long' but the argument has type 'uint64_t' (aka 'unsigned long long') [-Wformat]
                rc = sshbuf_putf(sbuf, "S-1-%lu", idauth);
                                            ~~~   ^~~~~~
                                            %llu
piv.c:8748:6: warning: variable 'err' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
        if (*out == NULL) {
            ^~~~~~~~~~~~
piv.c:8757:10: note: uninitialized use occurs here
        return (err);
                ^~~
piv.c:8748:2: note: remove the 'if' if its condition is always true
        if (*out == NULL) {
        ^~~~~~~~~~~~~~~~~~
piv.c:8684:13: note: initialize the variable 'err' to silence this warning
        errf_t *err;
                   ^
                    = NULL
2 warnings generated.
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include   -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -std=gnu99 -o ebox.o -c ebox.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include   -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -std=gnu99 -o ebox-cmd.o -c ebox-cmd.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include   -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -std=gnu99 -o sss/hazmat.o -c sss/hazmat.c
cc -I/System/Library/Frameworks/PCSC.framework/Headers/ -I/Users/arjen/oxidecomputer/pivy/libressl/include   -arch x86_64 -arch arm64  -DEBOX_USER_TPL_PATH='"$HOME/Library/Preferences/pivy/tpl/$TPL"' -DEBOX_SYSTEM_TPL_PATH='"/Library/Preferences/pivy/tpl/$TPL"' -fstack-protector-all -fwrapv -fPIC -D_FORTIFY_SOURCE=2 -Wall -g -O2 -gdwarf-2 -O2 -g -D_GNU_SOURCE -std=gnu99 -o sss/randombytes.o -c sss/randombytes.c
sss/randombytes.c:186:10: warning: Using arc4random system call [-W#pragma-messages]
# pragma message("Using arc4random system call")
         ^
1 warning generated.
1 warning generated.
cc -arch x86_64 -arch arm64  -o pivy-tool pivy-tool.o piv.o tlv.o debug.o bunyan.o errf.o utils.o readpassphrase.o piv-certs.o pkinit_asn1.o /Users/arjen/oxidecomputer/pivy/openssh/libssh.a -L/Users/arjen/oxidecomputer/pivy/libressl/crypto/.libs -lcrypto -framework PCSC -lz -lproc
sss/randombytes.c:186:10: warning: Using arc4random system call [-W#pragma-messages]
# pragma message("Using arc4random system call")
         ^
1 warning generated.
cc -arch x86_64 -arch arm64  -o pivy-agent pivy-agent.o piv.o tlv.o debug.o bunyan.o errf.o utils.o readpassphrase.o /Users/arjen/oxidecomputer/pivy/openssh/libssh.a -L/Users/arjen/oxidecomputer/pivy/libressl/crypto/.libs -lcrypto -framework PCSC -lz -lproc
cc -arch x86_64 -arch arm64  -o pivy-box pivy-box.o ebox.o ebox-cmd.o piv.o tlv.o debug.o bunyan.o errf.o utils.o readpassphrase.o sss/hazmat.o sss/randombytes.o /Users/arjen/oxidecomputer/pivy/openssh/libssh.a -L/Users/arjen/oxidecomputer/pivy/libressl/crypto/.libs -lcrypto -framework PCSC -lz -ledit -lproc
arjen@mbp pivy % ./pivy-tool init
zsh: segmentation fault  ./pivy-tool init

[arekinath]

Ah, good catch. Apparently I haven't been building with clang and reading the warning output recently, which is a mistake.