Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Illegal instruction using openblas 0.3.13+4 on 1.7-dev #834

Closed
sethaxen opened this issue Apr 5, 2021 · 21 comments · Fixed by JuliaPackaging/Yggdrasil#2782
Closed

Illegal instruction using openblas 0.3.13+4 on 1.7-dev #834

sethaxen opened this issue Apr 5, 2021 · 21 comments · Fixed by JuliaPackaging/Yggdrasil#2782
Labels
regression Regression in behavior compared to a previous version

Comments

@sethaxen
Copy link
Contributor

sethaxen commented Apr 5, 2021

After JuliaLang/julia#40343 I am unable to call certain LinearAlgebra functions anymore. An example:

julia> using LinearAlgebra

julia> A = randn(4, 4)
4×4 Matrix{Float64}:
  1.19616   -0.832824  -0.779283   2.62967
 -0.520257  -1.05399    0.282232  -0.0441448
 -1.38003   -0.580062  -2.14204    0.224513
 -0.837389   0.238154  -0.107394  -2.57343

julia> eigen(A)

signal (4): Illegal instruction: 4
in expression starting at REPL[3]:1
dlapy2_64_ at /Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib (unknown line)
Allocations: 2231132 (Pool: 2230671; Big: 461); GC: 3
Illegal instruction: 4

If I check out the previous commit on master and build Julia, everything works fine.

I tried to generate an RR bug report but got the error:

ERROR: RR not available on this platform

My version info:

julia> versioninfo()
Julia Version 1.7.0-DEV.845
Commit 58fba2b356 (2021-04-04 23:29 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_EDITOR = code
This was referenced Apr 6, 2021
Merged
Merged
@pablosanjose
Copy link
Contributor

pablosanjose commented Apr 6, 2021 •
edited
Loading

Uhm, looks bad. Unfortunately I cannot confirm on my machine (also a mac). I get no error with latest nightly.

julia> versioninfo()
Julia Version 1.7.0-DEV.847
Commit fedefe913a (2021-04-06 03:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code

I also tried with the vs/40279 branch without error.

@KristofferC
Copy link
Member

If I check out the previous commit on master and build Julia, everything works fine.

Can you try do make clean; rm -rf usr between the builds. Can you also try downloading the nightly build and see if it repros there.

@sethaxen
Copy link
Contributor Author

sethaxen commented Apr 6, 2021 •
edited
Loading

If I check out the previous commit on master and build Julia, everything works fine.

Can you try do make clean; rm -rf usr between the builds. Can you also try downloading the nightly build and see if it repros there.

I cleaned several times and still saw the error, but just to be safe, I made a fresh clone of the repo and am makeing it now. EDIT: Yes, same error on a fresh build.

I see the same error on the nightly build:

julia> using LinearAlgebra

julia> eigen(randn(10, 10))

signal (4): Illegal instruction: 4
in expression starting at REPL[2]:1
dlapy2_64_ at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libopenblas64_.0.3.13.dylib (unknown line)
Allocations: 2245082 (Pool: 2244614; Big: 468); GC: 3
Illegal instruction: 4
julia> versioninfo()
Julia Version 1.7.0-DEV.847
Commit fedefe913a (2021-04-06 03:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_EDITOR = code

@jaakkor2
Copy link

jaakkor2 commented Apr 6, 2021

I can't even start Julia-1.7 nightly on my Mac

$ exec '/Applications/Julia-1.7.app/Contents/Resources/julia/bin/julia'
ERROR: Unable to load dependent library /Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib
Message: dlopen(/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib, 6): Symbol not found: _clock_gettime
  Referenced from: /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia//libgfortran.5.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia//libgfortran.5.dylib
Unable to load "/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/julia/libopenblas64_.dylib"
┌ Error: No loaded BLAS libraries were built with ILP64 support
└ @ LinearAlgebra.BLAS /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.7/LinearAlgebra/src/blas.jl:154
Quitting.

[Process completed]

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM)2 Duo CPU     P7350  @ 2.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, penryn)

Julia-1.6.0 is fine.

@giordano
Copy link
Contributor

giordano commented Apr 6, 2021

@jaakkor2 can you also please try on the commit before JuliaLang/julia#40343?

@jaakkor2
Copy link

jaakkor2 commented Apr 6, 2021

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343, as my Mac is from 2009 and I cannot likely compile anything in a reasonable time in it.

@giordano
Copy link
Contributor

giordano commented Apr 6, 2021

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343

There you go: https://julialangnightlies-s3.julialang.org/bin/mac/x64/1.7/julia-89fff18d59-mac64.dmg

@jaakkor2
Copy link

jaakkor2 commented Apr 6, 2021

@giordano , only if I can get a download link to a nightly before JuliaLang/julia#40343

There you go: https://julialangnightlies-s3.julialang.org/bin/mac/x64/1.7/julia-89fff18d59-mac64.dmg

Same error with julia-89fff18d59-mac64.dmg as with latest nightly (creation time difference between the two is only 6 hours). More nightlies to test? From mid-February before libblastrampoline went in?

@giordano
Copy link
Contributor

giordano commented Apr 6, 2021

Find a commit and replace the first 10 characters of the hash in the URL. I think these tarballs are available for 60 or 90 days.

@ViralBShah
Copy link
Member

For the folks who are seeing this issue, can you confirm that 1.6 works fine (I think it should be but just checking).

@giordano
Copy link
Contributor

giordano commented Apr 6, 2021

Determing which is the illegal instruction would give us some more clues of what's going on. Look for example in dmesg log if there is a line referencing it

@jaakkor2 jaakkor2 mentioned this issue Nov 26, 2024
Closed
@sethaxen
Copy link
Contributor Author

sethaxen commented Apr 6, 2021

For the folks who are seeing this issue, can you confirm that 1.6 works fine (I think it should be but just checking).

Yes, Julia 1.6 works fine for me. As does the commit immediately before JuliaLang/julia#40343.

Determing which is the illegal instruction would give us some more clues of what's going on. Look for example in dmesg log if there is a line referencing it

Here's the output of sudo dmesg that looks relevant:

Waking up reference: 20605Thread waiting on reference 20605 woke upevaluation result: 20605, allowed, cache,  1617663216Waking up reference: 20606Thread waiting on reference 20606 woke upevaluation result: 20606, allowed, cache,  1617663217Waking up reference: 20607Thread waiting on reference 20607 woke upevaluation result: 20607, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libopenlibm.3.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libopenlibm.3.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20608Thread waiting on reference 20608 woke upevaluation result: 20608, allowed, cache,  1617663217Waking up reference: 20609Thread waiting on reference 20609 woke upevaluation result: 20609, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libunwind.1.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libunwind.1.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20610Thread waiting on reference 20610 woke upevaluation result: 20610, allowed, cache,  1617663217AMFI: '/Users/saxen/projects/julia/usr/lib/libLLVM.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libLLVM.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20611Thread waiting on reference 20611 woke upevaluation result: 20611, allowed, cache,  1617662974Waking up reference: 20612Thread waiting on reference 20612 woke upevaluation result: 20612, allowed, cache,  1617663808Waking up reference: 20613Thread waiting on reference 20613 woke upevaluation result: 20613, denied, skip-cache,  0Library load (/private/var/folders/5v/8c378lxx4ddg21zlmzy88dgm0000gp/T/tmp.KMeniZ) rejected: library load disallowed by system policy
purgatory_cleaner_cb:19279: remove_purgatory_entry failed on 0x254c940-dead for file-id 39110976 err 2
Waking up reference: 20614Thread waiting on reference 20614 woke upevaluation result: 20614, allowed, cache,  1617730927AMFI: '/Users/saxen/projects/julia/usr/lib/libpcre2-8.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libpcre2-8.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20615Thread waiting on reference 20615 woke upevaluation result: 20615, allowed, cache,  1617663352AMFI: '/Users/saxen/projects/julia/usr/lib/libgmp.10.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libgmp.10.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20616Thread waiting on reference 20616 woke upevaluation result: 20616, allowed, cache,  1617663368AMFI: '/Users/saxen/projects/julia/usr/lib/libmpfr.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmpfr.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20617Thread waiting on reference 20617 woke upevaluation result: 20617, allowed, cache,  1617663372AMFI: '/Users/saxen/projects/julia/usr/lib/libblastrampoline.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libblastrampoline.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20618Thread waiting on reference 20618 woke upevaluation result: 20618, allowed, cache,  1617663427AMFI: '/Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libopenblas64_.0.3.13.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20619Thread waiting on reference 20619 woke upevaluation result: 20619, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libgfortran.5.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libgfortran.5.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20620Thread waiting on reference 20620 woke upevaluation result: 20620, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libquadmath.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libquadmath.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20621Thread waiting on reference 20621 woke upevaluation result: 20621, allowed, cache,  1617663485AMFI: '/Users/saxen/projects/julia/usr/lib/libcholmod.3.0.14.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcholmod.3.0.14.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20622Thread waiting on reference 20622 woke upevaluation result: 20622, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libamd.2.4.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libamd.2.4.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20623Thread waiting on reference 20623 woke upevaluation result: 20623, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libcolamd.2.9.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcolamd.2.9.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20624Thread waiting on reference 20624 woke upevaluation result: 20624, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparseconfig.5.8.1.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparseconfig.5.8.1.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20625Thread waiting on reference 20625 woke upevaluation result: 20625, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libccolamd.2.9.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libccolamd.2.9.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20626Thread waiting on reference 20626 woke upevaluation result: 20626, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libcamd.2.4.6.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcamd.2.4.6.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20627Thread waiting on reference 20627 woke upevaluation result: 20627, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparse_wrapper.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libsuitesparse_wrapper.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20628Thread waiting on reference 20628 woke upevaluation result: 20628, allowed, cache,  1617663426AMFI: '/Users/saxen/projects/julia/usr/lib/libnghttp2.14.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libnghttp2.14.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20629Thread waiting on reference 20629 woke upevaluation result: 20629, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libcurl.4.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libcurl.4.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20630Thread waiting on reference 20630 woke upevaluation result: 20630, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libssh2.1.0.1.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libssh2.1.0.1.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20631Thread waiting on reference 20631 woke upevaluation result: 20631, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libz.1.2.11.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libz.1.2.11.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20632Thread waiting on reference 20632 woke upevaluation result: 20632, allowed, cache,  1617663429AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedtls.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedtls.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20633Thread waiting on reference 20633 woke upevaluation result: 20633, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedx509.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedx509.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20634Thread waiting on reference 20634 woke upevaluation result: 20634, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedcrypto.2.24.0.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libmbedcrypto.2.24.0.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20635Thread waiting on reference 20635 woke upevaluation result: 20635, allowed, cache,  1617663413AMFI: '/Users/saxen/projects/julia/usr/lib/libdSFMT.dylib' has no CMS blob?
AMFI: '/Users/saxen/projects/julia/usr/lib/libdSFMT.dylib': Unrecoverable CT signature issue, bailing out.
AMFI: code signature validation failed.
Waking up reference: 20636Thread waiting on reference 20636 woke upevaluation result: 20636, allowed, cache,  1617663407AMFI: Process does not have a code signature, allowing core dump for pid 73884

@JeffBezanson JeffBezanson added the regression Regression in behavior compared to a previous version label Apr 6, 2021
@ViralBShah
Copy link
Member

ViralBShah commented Apr 6, 2021 •
edited
Loading

The only thing that changed was to add a 2 line LAPACK patch, and then building new binaries. I can't imagine the LAPACK patch causing any of this - which leads me to think that it has to be the toolchain. No other changes in openblas either.

@ViralBShah ViralBShah changed the title Illegal instruction using openblas Illegal instruction using openblas 0.3.13+4 on 1.7-dev Apr 6, 2021
@thofma
Copy link

thofma commented Apr 7, 2021

Just want to mention that this started appearing on GitHub Actions CI, e.g. https://github.com/thofma/Hecke.jl/runs/2279959685?check_suite_focus=true.

@ViralBShah
Copy link
Member

Should we temporarily go back to 0.3.13+3?

@ViralBShah
Copy link
Member

@staticfloat Do you have any thoughts on what might be going on?

@giordano
Copy link
Contributor

giordano commented Apr 7, 2021

Should we temporarily go back to 0.3.13+3?

The fact that it's reproducible on GitHub Actions now makes it slightly easier to debug (at least for someone who's more familiar with macOS than me 🙂)

@staticfloat
Copy link
Member

@sethaxen I have been unable to reproduce this, but my guess is that's because the oldest mac processor I have access to is a Haswell chip (which has AVX2) while yours is an Ivy Bridge (which does not).

Can you run julia inside of lldb and get a backtrace for me? If you can write the reproducer as a script, it should be pretty easy to run lldb -- julia ./test.jl, then type r to run the program, and finally once it crashes, show me the output of bt to get a backtrace.

@sethaxen
Copy link
Contributor Author

sethaxen commented Apr 7, 2021

@staticfloat here you go:

(base) iMacs-iMac:julia saxen$ lldb -- ./julia ./test.jl
(lldb) target create "./julia"
Current executable set to '/Users/saxen/projects/julia/julia' (x86_64).
(lldb) settings set -- target.run-args  "./test.jl"
(lldb) r
Process 94550 launched: '/Users/saxen/projects/julia/julia' (x86_64)
Process 94550 stopped
* thread JuliaLang/julia#1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x0000000118b299f5 libopenblas64_.0.3.13.dylib`dlapy2_64_ + 149
libopenblas64_.0.3.13.dylib`dlapy2_64_:
->  0x118b299f5 <+149>: vfmadd213sd 0x1e111a(%rip), %xmm1, %xmm1 ; xmm1 = (xmm1 * xmm1) + mem
    0x118b299fe <+158>: vsqrtsd %xmm1, %xmm1, %xmm1
    0x118b29a02 <+162>: vmulsd %xmm1, %xmm0, %xmm0
    0x118b29a06 <+166>: retq
Target 0: (julia) stopped.
(lldb) bt
* thread JuliaLang/julia#1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x0000000118b299f5 libopenblas64_.0.3.13.dylib`dlapy2_64_ + 149
    frame JuliaLang/julia#1: 0x000000011873ab00 libopenblas64_.0.3.13.dylib`dlarfg_64_ + 160
    frame JuliaLang/julia#2: 0x00000001186e0cdb libopenblas64_.0.3.13.dylib`dgehd2_64_ + 459
    frame JuliaLang/julia#3: 0x00000001186e156b libopenblas64_.0.3.13.dylib`dgehrd_64_ + 1947
    frame JuliaLang/julia#4: 0x00000001186df6f7 libopenblas64_.0.3.13.dylib`dgeevx_64_ + 1687
    frame JuliaLang/julia#5: 0x00000001364571fe
    frame JuliaLang/julia#6: 0x00000001364589c1
    frame JuliaLang/julia#7: 0x000000013645a240
    frame JuliaLang/julia#8: 0x000000013645a371
    frame JuliaLang/julia#9: 0x00000001018a0fde libjulia-internal.1.7.dylib`do_call [inlined] jl_apply(args=<unavailable>, nargs=<unavailable>) at julia.h:1764:12 [opt]
    frame JuliaLang/julia#10: 0x00000001018a0fcd libjulia-internal.1.7.dylib`do_call(args=0x0000000111190ff0, nargs=<unavailable>, s=0x0000000000000001) at interpreter.c:117 [opt]
    frame JuliaLang/julia#11: 0x000000010189f5f4 libjulia-internal.1.7.dylib`eval_body(stmts=0x0000000110dbbeb0, s=0x00007ffeefbfd490, ip=<unavailable>, toplevel=1) at interpreter.c:0 [opt]
    frame JuliaLang/julia#12: 0x000000010189fbbb libjulia-internal.1.7.dylib`jl_interpret_toplevel_thunk(m=0x00000001077e9ee0, src=0x0000000110d3b810) at interpreter.c:718:21 [opt]
    frame JuliaLang/julia#13: 0x00000001018ba8e7 libjulia-internal.1.7.dylib`jl_toplevel_eval_flex(m=<unavailable>, e=<unavailable>, fast=<unavailable>, expanded=<unavailable>) at toplevel.c:884:18 [opt]
    frame JuliaLang/julia#14: 0x00000001018ba685 libjulia-internal.1.7.dylib`jl_toplevel_eval_flex(m=<unavailable>, e=<unavailable>, fast=<unavailable>, expanded=125834048) at toplevel.c:829:19 [opt]
    frame JuliaLang/julia#15: 0x00000001018bb1d5 libjulia-internal.1.7.dylib`jl_toplevel_eval_in [inlined] jl_toplevel_eval(m=0x00000001077e9ee0, v=0x000000010fef0810) at toplevel.c:893:12 [opt]
    frame JuliaLang/julia#16: 0x00000001018bb1c3 libjulia-internal.1.7.dylib`jl_toplevel_eval_in(m=0x00000001077e9ee0, ex=0x000000010fef0810) at toplevel.c:936 [opt]
    frame JuliaLang/julia#17: 0x000000010757c0b8 sys.dylib`japi1_include_string_49633 at boot.jl:369
    frame JuliaLang/julia#18: 0x000000010188953d libjulia-internal.1.7.dylib`jl_apply_generic [inlined] _jl_invoke(F=0x0000000108d83620, args=0x00007ffeefbfdb68, nargs=4, mfunc=0x0000000108d82930, world=<unavailable>) at gf.c:0 [opt]
    frame JuliaLang/julia#19: 0x00000001018894ce libjulia-internal.1.7.dylib`jl_apply_generic(F=0x0000000108d83620, args=0x00007ffeefbfdb68, nargs=4) at gf.c:2425 [opt]
    frame JuliaLang/julia#20: 0x00000001073899e3 sys.dylib`japi1__include_31233 at loading.jl:1154
    frame JuliaLang/julia#21: 0x00000001075e6405 sys.dylib`japi1_include_36935 at Base.jl:387
    frame JuliaLang/julia#22: 0x000000010188953d libjulia-internal.1.7.dylib`jl_apply_generic [inlined] _jl_invoke(F=0x000000010a3ab440, args=0x00007ffeefbfe210, nargs=2, mfunc=0x0000000107d977c0, world=<unavailable>) at gf.c:0 [opt]
    frame JuliaLang/julia#23: 0x00000001018894ce libjulia-internal.1.7.dylib`jl_apply_generic(F=0x000000010a3ab440, args=0x00007ffeefbfe210, nargs=2) at gf.c:2425 [opt]
    frame JuliaLang/julia#24: 0x0000000107025fd3 sys.dylib`julia_exec_options_50382 at client.jl:292
    frame JuliaLang/julia#25: 0x0000000107027918 sys.dylib`julia__start_40839 at client.jl:495
    frame JuliaLang/julia#26: 0x0000000107027a79 sys.dylib`jfptr__start_40840 + 9
    frame JuliaLang/julia#27: 0x000000010188953d libjulia-internal.1.7.dylib`jl_apply_generic [inlined] _jl_invoke(F=0x0000000108a158a0, args=0x00007ffeefbff660, nargs=0, mfunc=0x0000000108a15410, world=<unavailable>) at gf.c:0 [opt]
    frame JuliaLang/julia#28: 0x00000001018894ce libjulia-internal.1.7.dylib`jl_apply_generic(F=0x0000000108a158a0, args=0x00007ffeefbff660, nargs=0) at gf.c:2425 [opt]
    frame JuliaLang/julia#29: 0x00000001018da8e3 libjulia-internal.1.7.dylib`true_main [inlined] jl_apply(args=<unavailable>, nargs=1) at julia.h:1764:12 [opt]
    frame JuliaLang/julia#30: 0x00000001018da8d5 libjulia-internal.1.7.dylib`true_main(argc=<unavailable>, argv=<unavailable>) at jlapi.c:560 [opt]
    frame JuliaLang/julia#31: 0x00000001018da7d4 libjulia-internal.1.7.dylib`repl_entrypoint(argc=<unavailable>, argv=<unavailable>) at jlapi.c:702:15 [opt]
    frame JuliaLang/julia#32: 0x0000000100000f79 julia`main(argc=<unavailable>, argv=<unavailable>) at loader_exe.c:51:15 [opt]
    frame JuliaLang/julia#33: 0x00007fff6c483cc9 libdyld.dylib`start + 1

@chriselrod
Copy link
Contributor

chriselrod commented Apr 7, 2021 •
edited
Loading 

libopenblas64_.0.3.13.dylib`dlapy2_64_:
->  0x118b299f5 <+149>: vfmadd213sd 0x1e111a(%rip), %xmm1, %xmm1 ; xmm1 = (xmm1 * xmm1) + mem

vfmadd213sd is a problem if you're on ivybridge (no fma).

Unless I'm missing something, dlapy2 is implemented in fortran rather than asm.

@staticfloat
Copy link
Member

staticfloat commented Apr 7, 2021 •
edited
Loading

Indeed that is a problem. I found the issue; we were adding the platform targeting makevars to FLAGS (an undefined variable) instead of flags, so the x86_64 builds were targeting the same architecture as the machine they were getting built on, which does have FMA instructions. BB's audit pass didn't catch this because it sees that OpenBLAS has CPUID instructions inside of it, so it assumes it knows what it's doing. :P

I've got a new build of OpenBLAS making its way through the Yggdrasil pipeline, so once that's done, we need to bump the version in stdlib/OpenBLAS_jll/Project.toml and we should be good to go.

Julia PR here: JuliaLang/julia#40396

@sethaxen sethaxen mentioned this issue Apr 7, 2021
Merged
@ViralBShah ViralBShah mentioned this issue Apr 8, 2021
Merged
@ViralBShah ViralBShah mentioned this issue Apr 6, 2021
Closed
@KristofferC KristofferC transferred this issue from JuliaLang/julia Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
regression Regression in behavior compared to a previous version
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix OpenBLAS using incorrect instructions JuliaPackaging/Yggdrasil
10 participants
@staticfloat @ViralBShah @JeffBezanson @giordano @KristofferC @pablosanjose @jaakkor2 @chriselrod @sethaxen @thofma