-
Notifications
You must be signed in to change notification settings - Fork 868
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Apple linker does not accept -commons use_dylibs
flag anymore
#12427
Comments
I have reported the regression (compared to Xcode 14 and earlier linkers) to Apple as FB13194355. |
similar problem I have encountered when I was trying to install super-dist package using spack and using [email protected]
Does this warning treated differently in earlier versions of apple-clang? |
It might be worth trying Not sure if this is related to this issue thougth If this fixes the issue, all the credit should go to @jeffhammond |
Is Open MPI getting these flags from GNU Libtool? I.e., is this actually a Libtool issue? |
No x 2: Line 236 in 984944d
|
FYI I have verified the following works with XCode 15.3 on Sonoma 14.4, which is the workaround Apple gave me. I also confirmed it works when
// extern2.c
#include <stdio.h>
#include <stdint.h>
int MPIR_F08_MPI_IN_PLACE;
void p(void)
{
printf("MPIR_F08_MPI_IN_PLACE=%d &MPIR_F08_MPI_IN_PLACE=%p &MPIR_F08_MPI_IN_PLACE=%zu\n",
MPIR_F08_MPI_IN_PLACE, &MPIR_F08_MPI_IN_PLACE, (intptr_t)&MPIR_F08_MPI_IN_PLACE);
}
void MPI_Allreduce(void ** sendbuf, void ** recvbuf,
int * count, int * datatype,
int * op, int * comm, int * ierror)
{
printf("sendbuf=%p, sendbuf=%zu\n", sendbuf, (intptr_t)sendbuf);
printf("sendbuf is MPI_IN_PLACE? %s\n",
(intptr_t)sendbuf==(intptr_t)&MPIR_F08_MPI_IN_PLACE ? "yes" : "no");
printf("recvbuf=%p, recvbuf=%zu\n", recvbuf, (intptr_t)recvbuf);
printf("*count=%d, *datatype=%d, *op=%d, *comm=%d\n",
*count, *datatype, *op, *comm);
*ierror = 911;
} ! extern.F90
module mpi
use iso_c_binding
!type(c_ptr), bind(C,name="MPI_F_IN_PLACE") :: MPI_IN_PLACE
integer(c_int), bind(C, name="MPIR_F08_MPI_IN_PLACE"), target :: MPI_IN_PLACE
interface
subroutine p() bind(C,name="p")
end subroutine
end interface
interface
SUBROUTINE MPI_ALLREDUCE(SENDBUF, RECVBUF, COUNT, DATATYPE, OP, COMM, IERROR) &
bind(C,name="MPI_Allreduce")
use iso_c_binding
import :: MPI_IN_PLACE
!DEC$ ATTRIBUTES NO_ARG_CHECK :: sendbuf,recvbuf
!GCC$ ATTRIBUTES NO_ARG_CHECK :: sendbuf,recvbuf
!$PRAGMA IGNORE_TKR sendbuf,recvbuf
!DIR$ IGNORE_TKR sendbuf,recvbuf
!IBM* IGNORE_TKR sendbuf,recvbuf
INTEGER(kind=c_int) :: SENDBUF(*), RECVBUF(*)
INTEGER(kind=c_int) :: COUNT, DATATYPE, OP, COMM, IERROR
END SUBROUTINE MPI_ALLREDUCE
end interface
end module mpi
program main
use mpi
implicit none
real :: buf(100)
integer :: ierror
call p
buf = 17
print*,'LOC(MPI_IN_PLACE)=',LOC(MPI_IN_PLACE)
print*,'LOC(buf)=',LOC(buf)
call MPI_ALLREDUCE(MPI_IN_PLACE,buf,1,2,3,4,ierror)
print*,ierror
end program main |
I am trying to do the same for Homebrew: Homebrew/homebrew-core#166807 My original analysis was that |
this worked for me:
thanks @jeffhammond @ggouaillardet |
Oddly enough this worked for me: brew install gcc-13
../configure --prefix=/opt/extlib/openmpi/5.0.2/gcc/13.2.0 \
--with-libevent=internal \
--enable-mpi1-compatibility \
--enable-static \
--enable-pmix-timing \
CC=gcc-13 CXX=g++-13 FC=gfortran-13
make clean
make -j 8
make check
sudo make install I was unable to install open-mpi 5.0.3 with the same method. |
I'm sorry for the huge delay here. Thanks for the citation of I have Sonoma 14.4.1 with XCode 15.3, and Homebrew gfortran
But I don't see these warnings when I compile with the homebrew gfortran.
What is different between my setup and yours? |
We have re-enabled the "classic linker" in Homebrew gfortran at some point. |
Ah, gotcha. Is this issue moot, then? Or do we still need to investigate the use of I see some comments in our code that these flags were necessary at some point, but I'm afraid I don't know/remember why they were necessary (i.e., to know if they are still necessary). |
They are still necessary. They are incompatible with Apple's new linker, hence we (for now) rely on the old linker. |
Ok. Given that homebrew gfortran has updated, should we close this issue? |
Well, it's a workaround, not a proper fix: at some point the "classic linker" might not be supported by Apple anymore. Maybe an alternative implementation is possible? |
Let me make sure I'm parsing your reply correctly:
Is that correct? If so, can you explain / remind me why we need |
All of that is correct.
The Fortran part of open-mpi uses them for common blocks. I haven't dug more on how and why. |
Ah, yes, we do use some common blocks for sentinel values (i.e., they really have to be global so that we can look for them by address, not by value): ompi/ompi/include/mpif-sentinels.h Lines 60 to 68 in ce3742c
|
Also, I have tested and even if MPI wasn't using |
@fxcoudert MPI implementations have to use It might be possible to define |
Hello I am running MacOS 14.5 on Apple M1, Xcode 15.4, gcc-14, g++-14 and fortran-14 I compiled open-mpi-5.0.3 :
Then when I compile a program I face similar problem:
|
For anyone else running into this, I found the following ways to all work around this:
|
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is supported. This is needed to support the -Wl,-commons,use_dylibs linker flag, which is needed for common Fortran symbols on MaxOS. Signed-off-by: Jeff Squyres <[email protected]>
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is needed. Specifically, Open MPI has used -Wl,-commons,use_dylibs for decades to support common symbols (e.g., MPI_BOTTOM) in the Fortran bindings. There is a window of Xcode versions where this switch was effectively disabled; it effectively required the additional -Wl,-ld_classic switch to force the use of the "old" Apple linker (that still supported -Wl,-commons,use_dylibs). Update the configury to test whether we need -Wl,-ld_classic or not. Signed-off-by: Jeff Squyres <[email protected]>
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is needed. Specifically, Open MPI has used -Wl,-commons,use_dylibs for decades to support common symbols (e.g., MPI_BOTTOM) in the Fortran bindings. There is a window of Xcode versions where this switch was effectively disabled; it effectively required the additional -Wl,-ld_classic switch to force the use of the "old" Apple linker (that still supported -Wl,-commons,use_dylibs). Update the configury to test whether we need -Wl,-ld_classic or not. Signed-off-by: Jeff Squyres <[email protected]>
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is needed. Specifically, Open MPI has used -Wl,-commons,use_dylibs for decades to support common symbols (e.g., MPI_BOTTOM) in the Fortran bindings. There is a window of Xcode versions where this switch was effectively disabled; it effectively required the additional -Wl,-ld_classic switch to force the use of the "old" Apple linker (that still supported -Wl,-commons,use_dylibs). Update the configury to test whether we need -Wl,-ld_classic or not. Signed-off-by: Jeff Squyres <[email protected]>
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is needed. Specifically, Open MPI has used -Wl,-commons,use_dylibs for decades to support common symbols (e.g., MPI_BOTTOM) in the Fortran bindings. There is a window of Xcode versions where this switch was effectively disabled; it effectively required the additional -Wl,-ld_classic switch to force the use of the "old" Apple linker (that still supported -Wl,-commons,use_dylibs). Update the configury to test whether we need -Wl,-ld_classic or not. Signed-off-by: Jeff Squyres <[email protected]> (cherry picked from commit 84555f0)
Per open-mpi#12427, on MacOS, add -Wl,-ld_classic to the Fortran wrapper compiler if that flag is needed. Specifically, Open MPI has used -Wl,-commons,use_dylibs for decades to support common symbols (e.g., MPI_BOTTOM) in the Fortran bindings. There is a window of Xcode versions where this switch was effectively disabled; it effectively required the additional -Wl,-ld_classic switch to force the use of the "old" Apple linker (that still supported -Wl,-commons,use_dylibs). Update the configury to test whether we need -Wl,-ld_classic or not. Signed-off-by: Jeff Squyres <[email protected]> (cherry picked from commit 84555f0)
The fix for this has been merged into main, v4.1.x, and v5.0.x. It will be included in the next releases of v4.1 and v5.0. Thank you! |
Great news @jsquyres! Just to be clear, this will be in |
@Chrismarsh This will be available in 5.0.4 |
It's going to be in v4.1.7, too. We keep promising to get v4.1.7 out "someday", but there hasn't been an urgent need yet. It will not be in any v4.0.x release -- that series is dead. |
Background information
What version of Open MPI are you using? 5.0.2
Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)
Installed from released sources as part of Homebrew build (Homebrew/homebrew-core#166807)
Please describe the system on which you are running
Details of the problem
Compiling any Fortran MPI code with
mpifort hellof.f90 -o hellof
with Xcode 15.3 gives:That is because
-commons use_dylibs
is now ignored (giving the warning), which leads to the symbol being rejected as defined twice.The text was updated successfully, but these errors were encountered: