Building on mac os issue

[willm]

Hi,

I thought I'd start a separate issue for building on mac os. It obviously seems to be a problem with building the juice dependency, but I'm unsure of how to resolve it. I've just noticed that this actually seems to be written by you as well! cmake seems to run without errors, but when I run the generated makefile, I get:

Scanning dependencies of target juice-static
[  3%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/addr.c.o
[  3%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/agent.c.o
[  6%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/crc32.c.o
[  9%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/hmac.c.o
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/hmac.c:22:10: fatal error: 'nettle/hmac.h' file not found
#include <nettle/hmac.h>
         ^~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [deps/libjuice/CMakeFiles/juice-static.dir/src/hmac.c.o] Error 1
make[1]: *** [deps/libjuice/CMakeFiles/juice-static.dir/all] Error 2
make: *** [all] Error 2

[paullouisageneau]

This is because libjuice requires libnettle as dependency. It should often be present as GnuTLS crypto backend, but apparently not in your case (I guess for you GnuTLS is linked against gcrypt instead). You have a choice here: you need either to install libnettle, or to switch the TLS backend to OpenSSL by changing the cmake command to cmake -B build -DUSE_JUICE=1 -DUSE_GNUTLS=0.

[willm]

Thanks for your feedback, so I looked into it and I actually do have gnu tls installed including nettle, see this snippet of output from cmake:

-- Performing Test has_wno_address_of_packet_member
-- Performing Test has_wno_address_of_packet_member - Success
-- Found GnuTLS: /usr/local/Cellar/gnutls/3.6.11.1/lib/libgnutls.dylib (found version "3.6.11")
-- Found NETTLE: /usr/local/lib/libnettle.dylib
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/will.munn/code/experiments/libdatachannel/build

I checked and both libnettle.dylib and libgnutls.dylib are at those locations. Is it expecting a different version?

I also tried with open ssl but then ran into this issue:

Scanning dependencies of target juice-static
[  3%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/addr.c.o
[  3%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/agent.c.o
[  6%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/crc32.c.o
[  9%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/hmac.c.o
[  9%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/ice.c.o
[ 12%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/juice.c.o
[ 12%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/log.c.o
[ 15%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/random.c.o
[ 18%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/stun.c.o
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/stun.c:34:9: warning: 'htonll' macro redefined [-Wmacro-redefined]
#define htonll(x)                                                                                  \
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_endian.h:141:9: note: previous definition is here
#define htonll(x)       __DARWIN_OSSwapInt64(x)
        ^
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/stun.c:36:9: warning: 'ntohll' macro redefined [-Wmacro-redefined]
#define ntohll(x) htonll(x)
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/sys/_endian.h:140:9: note: previous definition is here
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
        ^
2 warnings generated.
[ 18%] Building C object deps/libjuice/CMakeFiles/juice-static.dir/src/udp.c.o
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/udp.c:70:12: error: use of undeclared identifier 'IP_PMTUDISC_DO'
        int val = IP_PMTUDISC_DO;
                  ^
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/udp.c:71:31: error: use of undeclared identifier 'IP_MTU_DISCOVER'
        setsockopt(sock, IPPROTO_IP, IP_MTU_DISCOVER, &val, sizeof(val));
                                     ^
2 errors generated.
make[2]: *** [deps/libjuice/CMakeFiles/juice-static.dir/src/udp.c.o] Error 1
make[1]: *** [deps/libjuice/CMakeFiles/juice-static.dir/all] Error 2
make: *** [all] Error 2

[paullouisageneau]

Oh OK, path MTU discovery is only available on Linux, it needs to be disabled otherwise.
This should be fixed by 6f39994

The missing header for libnettle is weird then, it should work fine since you seem to have everything installed. Do you have the header file /usr/local/include/nettle/hmac.h?

[willm]

Yes I have that header file.

with open ssl I'm still seeing this issue after pulling your latest changes.

/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/udp.c:70:12: error: use of undeclared identifier 'IP_PMTUDISC_DO'
        int val = IP_PMTUDISC_DO;
                  ^
/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/udp.c:71:31: error: use of undeclared identifier 'IP_MTU_DISCOVER'
        setsockopt(sock, IPPROTO_IP, IP_MTU_DISCOVER, &val, sizeof(val));
                                     ^
2 errors generated.

[paullouisageneau]

OK, then it means you need to add /usr/local/include to the system C/C++ header search paths before compiling. I'm not very familiar with MacOS, but just running xcode-select --install might do the trick.

Hum, did you do git submodule update after pulling the changes?

[willm]

Ah no I didn't update submodules, now I get

/Users/will.munn/code/experiments/libdatachannel/deps/libjuice/src/udp.c:74:31: error: use of undeclared identifier 'IP_DONTFRAG'
        setsockopt(sock, IPPROTO_IP, IP_DONTFRAG, (char *)&enabled, sizeof(enabled));
                                     ^
1 error generated.
make[2]: *** [deps/libjuice/CMakeFiles/juice-static.dir/src/udp.c.o] Error 1
make[1]: *** [deps/libjuice/CMakeFiles/juice-static.dir/all] Error 2
make: *** [all] Error 2

with open ssl

[willm]

regarding /usr/local/include, I do see it listed when I run gcc -x c++ -v -E /dev/null which I believe lists the include paths. should /usr/local/include/nettle be listed there separately?

[paullouisageneau]

OK, the IP_DONTFRAG option does not exist on Mac OS, and there seems to be no other way to disable fragmentation. So at least for now I bypassed the MTU discovery on Mac OS to get around that issue: 7af3da7
Note you'll need a submodule update on this one also.

If /usr/src/include is in the compiler include paths it should work fine, the nettle subdirectory should not be listed. Maybe it's set for gcc but you use clang to compile? You can list clang include paths similarly with clang -v -E - < /dev/null.

[willm]

Sorry I've been afk for a few days. I ran the clang command, /usr/local/include is listed there so it's not that.

[paullouisageneau]

That's puzzling. And what if you specify it manually when you run cmake, like this?

CPPFLAGS=-I/usr/local/include cmake ..

[willm]

I get the same thing complaining about fatal error: 'nettle/hmac.h' file not found

[willm]

I've just noticed, the nettle directory is a symlink

ls -al /usr/local/include/nettle
lrwxr-xr-x  1 will.munn  admin  37 27 Jan 21:09 /usr/local/include/nettle -> ../Cellar/nettle/3.4.1/include/nettle

however, then I tried

cmake CPPFLAGS=-I/usr/local/Cellar/nettle/3.4.1/include  -DUSE_JUICE=1 -DUSE_GNUTLS=1 ..

but got the same error when I ran make

[paullouisageneau]

That's a good point about the symlink, it could cause the problem somehow, however CPPFLAGS should be passed as a shell variable to cmake otherwise I'm pretty sure it'll be ignored:

CPPFLAGS=-I/usr/local/Cellar/nettle/3.4.1/include cmake  -DUSE_JUICE=1 -DUSE_GNUTLS=1 ..

[willm]

Ah thanks sorry about that, I tried putting the cppflags at the beginning, still no luck

[paullouisageneau]

Hum, I'm a bit stuck here... Did you have more luck with OpenSSL?

[willm]

I've set up a travis build running on macos. I've got to the same point, maybe it will help? can you see this? https://travis-ci.org/willm/libdatachannel/builds/657518778

[willm]

interestingly, if I make a file test.cpp

#include <nettle/hmac.h>

int main() {}

then run clang++ ./test.cpp it seems to compile fine.

[paullouisageneau]

Yes I have acces to the travis build. Thanks, I'll try to investigate a bit.

[paullouisageneau]

I didn't find the actual reason why /usr/local/include is not searched, but it works when specified explicitly in CMakeLists.txt so that's good enough. The compilation should now work properly on MacOS with the changes in #40. Could you confirm please?

[paullouisageneau]

I'm merging the PR, please re-open the issue if you encounter another compilation issue on your side.