Issue running tests on MacBook Pro M2 Max

[JosephBoyd]

I get the following when running (ql:quickload :coalton/tests) after brew installing sbcl 2.3.2 (only version available) on Coalton ([main] Add CCL support)

Backtrace:
0: (ERROR "SB-MPFR failed to load, for some reason. ~ ..)
1: ((SB-C::TOP-LEVEL-FORM (COMMON-LISP:SETQ COMMON-LISP:PACKAGE (SB-INT:FIND-UNDELETED-PACKAGE-OR-LOSE "COALTON-LIBRARY/BIG-FLOAT")))) [toplevel]
2: (SB-FASL::LOAD-FASL-GROUP #S(SB-FASL::FASL-INPUT :STREAM #<SB-SYS:FD-STREAM for "file /Users/josephboyd/.cache/common-lisp/sbcl-2.3.2-macosx-arm64/Users/josephboyd/quicklisp/local-projects/coalton/lib..
3: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SB-FASL::LOAD-AS-FASL))
4: (SB-IMPL::CALL-WITH-LOADER-PACKAGE-NAMES #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SB-FASL::LOAD-AS-FASL) {70063781FB}>)
5: (SB-FASL::LOAD-AS-FASL #<SB-SYS:FD-STREAM for "file /Users/josephboyd/.cache/common-lisp/sbcl-2.3.2-macosx-arm64/Users/josephboyd/quicklisp/local-projects/coalton/library/big-float/impl-sbcl.fasl" {70..
6: ((COMMON-LISP:LABELS SB-FASL::LOAD-STREAM-1 :IN COMMON-LISP:LOAD) #<SB-SYS:FD-STREAM for "file /Users/josephboyd/.cache/common-lisp/sbcl-2.3.2-macosx-arm64/Users/josephboyd/quicklisp/local-projects/co..
7: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (COMMON-LISP:LABELS SB-FASL::LOAD-STREAM-1 :IN COMMON-LISP:LOAD) {10854294B}> #<SB-SYS:FD-STREAM for "file /Users/josephboyd/.cache/common-lisp/sbcl-2.3.2-..
8: (COMMON-LISP:LOAD #P"/Users/josephboyd/.cache/common-lisp/sbcl-2.3.2-macosx-arm64/Users/josephboyd/quicklisp/local-projects/coalton/library/big-float/impl-sbcl.fasl" :VERBOSE COMMON-LISP:NIL :PRINT CO..
9: (UIOP/UTILITY:CALL-WITH-MUFFLED-CONDITIONS #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN UIOP/LISP-BUILD:LOAD*) {7006373E2B}> ("Overwriting already existing readtable ~S." #(#:FINALIZERS-OFF-WARN..
10: ((SB-PCL::EMF ASDF/ACTION:PERFORM) # # #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "coalton/library/big-float" "impl-sbcl">)
11: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
12: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS (ASDF/LISP-ACTION:LOAD-OP ASDF/LISP-ACTION:CL-SOURCE-FILE)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "coalton/library/big-float" "impl..
13: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (COMMON-LISP:T COMMON-LISP:T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "coalton/library/big-float" "impl-sbcl">) [fast-method..
14: ((:METHOD ASDF/PLAN:PERFORM-PLAN (COMMON-LISP:T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {7005BF14B3}>) [fast-method]
15: ((COMMON-LISP:FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
16: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (COMMON-LISP:T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {7005BF14B3}>) [fast-method]
17: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "coalton/tests"> :PLAN-CLASS COMMON-LISP:NIL :PLAN-OPTIONS COMMON-..
18: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) # # #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "coalton/tests"> :VERBOSE COMMON-LISP:NIL)
19: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/OPERATE:OPERATE))
20: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (COMMON-LISP:T COMMON-LISP:T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "coalton/tests"> :VERBOSE COMMON-LISP:NIL) [fast-method]
21: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) # # ASDF/LISP-ACTION:LOAD-OP "coalton/tests" :VERBOSE COMMON-LISP:NIL)
22: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/OPERATE:OPERATE))
23: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (COMMON-LISP:T COMMON-LISP:T)) ASDF/LISP-ACTION:LOAD-OP "coalton/tests" :VERBOSE COMMON-LISP:NIL) [fast-method]
24: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/OPERATE:OPERATE) {7005BE416B}> :OVERRIDE COMMON-LISP:T :KEY COMMON-LISP:NIL :OVERRIDE-CACHE COMMON-LISP:T :..
25: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/OPERATE:OPERATE))
26: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN ASDF/OPERATE:OPERATE) {7005BD91CB}> :OVERRIDE COMMON-LISP:NIL :KEY COMMON-LISP:NIL :OVERRIDE-CACHE COMMON-LISP:N..
27: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (COMMON-LISP:T COMMON-LISP:T)) ASDF/LISP-ACTION:LOAD-OP "coalton/tests" :VERBOSE COMMON-LISP:NIL) [fast-method]
28: (ASDF/OPERATE:LOAD-SYSTEM "coalton/tests" :VERBOSE COMMON-LISP:NIL)
29: (QUICKLISP-CLIENT::CALL-WITH-MACROEXPAND-PROGRESS #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY) {7005BD913B}>)
30: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "coalton/tests" :PROMPT COMMON-LISP:NIL)
31: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (COMMON-LISP:T COMMON-LISP:T)) # #<FUNCTION (COMMON-LISP:FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {7005BB8D9B}>) ..
32: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL COMMON-LISP:T)) #<QL-IMPL:SBCL {700D610643}> #<FUNCTION (COMMON-LISP:FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD..
33: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (COMMON-LISP:T)) :COALTON/TESTS :PROMPT COMMON-LISP:NIL :SILENT COMMON-LISP:NIL :VERBOSE COMMON-LISP:NIL) [fast-method]
34: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {7005BB3A8B}>)
35: (SB-INT:SIMPLE-EVAL-IN-LEXENV (QUICKLISP-CLIENT:QUICKLOAD :COALTON/TESTS) #)
36: (COMMON-LISP:EVAL (QUICKLISP-CLIENT:QUICKLOAD :COALTON/TESTS))
37: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
38: (SLYNK::CALL-WITH-RETRY-RESTART "Retry SLY mREPL evaluation request." #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK-MREPL::MREPL-EVAL-1) {7005BB302B}>)
39: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
40: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::CALL-WITH-LISTENER))
41: (SLYNK::CALL-WITH-BINDINGS ((COMMON-LISP:PACKAGE . #<COMMON-LISP:PACKAGE "COMMON-LISP-USER">) (COMMON-LISP:DEFAULT-PATHNAME-DEFAULTS . #P"/Users/josephboyd/") (COMMON-LISP:* . #1=(:COALTON)) (COMM..
42: (SLYNK-MREPL::MREPL-EVAL-1 #<SLYNK-MREPL::MREPL mrepl-1-1> "(ql:quickload :coalton/tests)")
43: (SLYNK-MREPL::MREPL-EVAL #<SLYNK-MREPL::MREPL mrepl-1-1> "(ql:quickload :coalton/tests)")
44: (SLYNK:PROCESS-REQUESTS COMMON-LISP:NIL)
45: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
46: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
47: (SLYNK-SBCL::CALL-WITH-BREAK-HOOK # #<FUNCTION (COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::SPAWN-CHANNEL-THREAD) {700D61000B}>)
48: ((COMMON-LISP:FLET SLYNK-BACKEND:CALL-WITH-DEBUGGER-HOOK :IN "/Users/josephboyd/.emacs.d/elpa/sly-20230224.911/slynk/backend/sbcl.lisp") # #<FUNCTION (COMMON-LISP:L..
49: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::CALL-WITH-LISTENER))
50: (SLYNK::CALL-WITH-BINDINGS ((COMMON-LISP:PACKAGE . #<COMMON-LISP:PACKAGE "COMMON-LISP-USER">) (COMMON-LISP:DEFAULT-PATHNAME-DEFAULTS . #P"/Users/josephboyd/") (COMMON-LISP:* . #1=(:COALTON)) (COMM..
51: ((COMMON-LISP:LAMBDA COMMON-LISP:NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
52: ((COMMON-LISP:FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
53: ((COMMON-LISP:FLET "WITHOUT-INTERRUPTS-BODY-151" :IN SB-THREAD::RUN))
54: ((COMMON-LISP:FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
55: ((COMMON-LISP:FLET "WITHOUT-INTERRUPTS-BODY-144" :IN SB-THREAD::RUN))
56: (SB-THREAD::RUN)

[stylewarning]

Hello @JosephBoyd! This is happening because it can't find the MPFR library. Any one of these three ideas should fix your issue.

1. Install MPFR with brew: brew install mpfr

2. Disable MPFR by setting the environment variable COALTON_PORTABLE_BIGFLOAT=1 before compiling Coalton.

3. Disable MPFR by pushing :coalton-portable-bigfloat onto your *features* (possibly in your ~/.sbclrc file) before compiling Coalton.

You may need to delete your FASL cache prior to rebuilding.

[eliaslfox]

Installing mpfr on M1/M2 is a little more involved. The default install location for Homebrew on M1/M2 is /opt/homebrew which isn't in the default dlopen search path. This can be fixed by setting the env var DYLD_LIBRARY_PATH to /opt/homebrew/lib before launching emacs or sbcl.

[JosephBoyd]

I confirmed that mpfr was installed via brew previously and tried setting DYLD_LIBRARY_PATH as @eliaslfox suggested but came up with the same results. Setting COALTON_PORTABLE_BIGFLOAT=1 does allow the tests to complete. I will continue to poke around to see how I can get mpfr discoverable.

Thanks for the feedback!

[eliaslfox]

MacOS is kinda perticular with that env var. It can't be set after a process is started, and it isn't always inherited. The easiest way to set it is to launch emacs from a shell:

$ export DYLD_LIBRARY_PATH=/opt/homebrew/lib
$ emacs

or to make your inferior lisp program a shell script that does it:

#!/bin/sh
export DYLD_LIBRARY_PATH=/opt/homebrew/lib
exec sbcl $@

[JosephBoyd]

Tried the suggestions by @eliaslfox, but still had problems. I'm new to Macs and in my research there is an indication that SIP is more aggressive in removing DYLD prefixed variables. I ended up building SBCL from source to specify an rpath value which required me to temporarily disable SIP in order for the build process to find the necessary dylib files for mpfr and gmp. Not sure if the homebrew folks would be willing to modify their build for extensions to include paths as they do for the zstd compression library which is looking in the homebrew directories. In conclusion, my custom build of SBCL has been working fine with sb-mpfr. Hats off to all of you for your work on Coalton!

[eliaslfox]

This should be fixed in a future version of sbcl https://groups.google.com/g/sbcl-devel/c/onxqb4vxDDM