Fix macosx runpath dynamic library issue freetype lib

[neomatrix369]

Following the issue reported on #202, applied the suggestions from @mbvreddy to see if this helps resolve the test execution issues @smlambert and her testing/JCK testing teams are facing.

At the moment limiting the fix to openjdk/installedfreetype/lib/libfreetype.6.dylib only.

[neomatrix369]

Testing hypotheis via jenkins job - https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk9_build_x86-64_macos-libfreetype/

@smlambert The artifact JDK that comes out of this build can be something you can take over and run, and see if it helps solve the issue, let me know the outcome.

Would also help if you record your test steps on this PR, so I can run the tests myself against the same JDK and see whats happening.

I'll just need the command-line commands that you run to reproduce your issue. Thanks.

[neomatrix369]

Some activities around runpath dependency on the libfreetype... library, snippet from the build logs:

...
/Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/freetype-2.4.0/objs/.libs/psnames.o   -lz    -install_name  /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib/libfreetype.6.dylib -compatibility_version 12 -current_version 12.0 -Wl,-single_module
libtool: link: dsymutil /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/freetype-2.4.0/objs/.libs/libfreetype.6.dylib || :
libtool: link: (cd "/Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/freetype-2.4.0/objs/.libs" && rm -f "libfreetype.dylib" && ln -s "libfreetype.6.dylib" "libfreetype.dylib")
libtool: link: ar cru /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/freetype-2.4.0/objs/.libs/libfreetype.a  
...

[neomatrix369]

A bit disappointing (snippets from the logs):

...
Releasing the runpath dependency of the dynamic library /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib/libfreetype.6.dylib
+ install_name_tool -id @rpath/libfreetype.6.dylib /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib/libfreetype.6.dylib
error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: can't open file: /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib/libfreetype.6.dylib (No such file or directory)
...

Back to investigating why it says this?

The file does exist in that directory and I have checked it and also run the command manually (auto-complete on the CLI), and it says the same error.

[neomatrix369]

#216 (comment) has now been fixed, so we should get messages like the below in the logs:

...
Releasing the runpath dependency of the dynamic library /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib//libfreetype.6.dylib
+ install_name_tool -id @rpath/libfreetype.6.dylib /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib//libfreetype.6.dylib
+ set +x
Successfully released the runpath dependency of the dynamic library /Users/jenkins/workspace/openjdk9_build_x86-64_macos-libfreetype/openjdk/installedfreetype/lib//libfreetype.6.dylib
...

[smlambert]

I am actually using the Jenkins test_personal build to rerun the tests that were excluded due to this problem (instructions on how to run the test_personal build are here: https://github.com/AdoptOpenJDK/openjdk-tests/wiki/How-to-Run-a-Personal-Test-Build-on-Jenkins).

I re-enabled them in my branch smlambert:reincludeIssue136Tests. Here are the values I used to test this:

[smlambert]

Link to the personal build: https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/162/

If they all pass, I will ask for a review and merge this PR: adoptium/aqa-tests#238

[neomatrix369]

If they all pass feel free to LGTM the PR, I'll add you to the reviewers list, mostly if you don't get any of the dynamic library related MacOSX issues - as the change is very specific to them, other issues may not get resolved with this fix

[smlambert]

Still see the errors:
https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/162/testReport/junit/

It was your build number 6 that I should have used?

[neomatrix369]

@smlambert Yes build 6, basically this JDK https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk9_build_x86-64_macos-libfreetype/lastSuccessfulBuild/artifact/OpenJDK9_x64_Mac_201801021417.tar.gz.

I see a different error in https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/162/testReport/junit/java_beans_EventHandler_Test6179222/java/Test6179222/, not libfreetype.6.dylib related, can you pls point me to the tests that complain about libfreetype.6.dylib with the old error message as you raised on the issue.

[smlambert]

https://ci.adoptopenjdk.net/view/work%20in%20progress/job/test_personal/162/testReport/junit/java_beans_EventHandler_Test6179222/java/Test6179222 - contains the same message as we saw before:

java.lang.UnsatisfiedLinkError: /Users/jenkins/workspace/test_personal/openjdkbinary/j2sdk-image/lib/libfontmanager.dylib: dlopen(/Users/jenkins/workspace/test_personal/openjdkbinary/j2sdk-image/lib/libfontmanager.dylib, 1): Library not loaded: @ rpath/libfreetype.6.dylib

[neomatrix369]

@smlambert I have a temporary solution to the UnsatisfiedLinkError, see https://examples.javacodegeeks.com/java-basics/exceptions/java-lang-unsatisfiedlinkerror-how-to-handle-unsatisfied-link-error/, can you try this and see if it works, we can then see how we can fix this on the build itself. At buildtime I'm unable to determine what the runpath should be.

[mbvreddy]

@rpath is required. Reason it failed even with it is because library name in jdk/lib/libfreetype.dylib.6 is incorrect on Mac. I tested locally and it worked.

[smlambert]

Yes, I can confirm what @mbvreddy has already stated.

The 2 problems:

1. the misnamed libfreetype.dylib.6 in the lib folder

2. libfontmanager.dylib should have @ rpath / libfreetype.6.dylib

DXU751:tmp jenkins$ otool -L /Users/jenkins/workspace/tmp/jdk-9+181/lib/libfontmanager.dylib
/Users/jenkins/workspace/tmp/jdk-9+181/lib/libfontmanager.dylib:
@ rpath /libfontmanager.dylib (compatibility version 1.0.0, current version 1.0.0)
libfreetype.6.dylib (compatibility version 12.0.0, current version 12.0.0)
@ rpath /libawt.dylib (compatibility version 1.0.0, current version 1.0.0)
@ rpath /libjava.dylib (compatibility version 1.0.0, current version 1.0.0)
@ rpath /libjvm.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 104.1.0)

-Djava.library.path workaround suggestion does not work. Checked against tests, and against a smaller standalone test that calls System.load("/Users/jenkins/workspace/tmp/jdk-9+181/lib/libfontmanager.dylib")

A hackety-hack to force 'success' was
cp libfreetype.dylib.6 libfreetype.6.dylib
and
ln -s ln -s /Users/jenkins/workspace/tmp/jdk-9+181/lib/libfreetype.6.dylib libfreetype.6.dylib

but I will wait for the actual fix before reincluding tests.

[neomatrix369]

@mbvreddy After all the conversations on issue #202, here's our current solution, I have reverted the change to include @rpath in the parameter.

You mentioned it worked for you, could you please share your command history with us, so I can apply them here.

I have seen @smlambert's workaround, unsure if I can include them in the build script.

[neomatrix369]

@smlambert @mbvreddy the answers might appear obvious to you but I'm hesitating to make a change, as, from past experience, I have seen, one fix resolves one issue and creates a new one. I don't follow dynamic libraries on the MacOSX any more than what I have read about them in this issue.

I could apply the copy file approach like how you have done and see how far it goes, can't create a symlink as the change must be compatible with others user environemnts.

[smlambert]

Tested https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk9_build_x86-64_macos-libfreetype/12/artifact/OpenJDK9_x64_Mac_201805021710.tar.gz, and the 40 failures relating to missing library are resolved.

https://ci.adoptopenjdk.net/view/work%20in%20progress/job/TestBuild_Sandbox/103/testReport/
We have 10 remaining failures, that are unrelated and will be addressed by a separate test issue that relate to awt test failures.

[karianna]

LGTM - tests passing depending

[neomatrix369]

@smlambert @gdams Can you please review, and approve so I can merge the PR - tests related to this fix have passed as per @smlambert's test Jenkins job.

[neomatrix369]

@smlambert Please raise the other failing tests as a separate issue on git and we can have a look at it

[gdams]

The code looks good!

[smlambert]

Thanks @neomatrix369 !

The remaining test failures are very likely due to the x11 server setup issue, which I will look at next.

[smlambert]

Confirmed the test failures related to this issue have been fixed, rerun against:

Java8: https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk8_build_x86-64_macos_libfreefont_fix/lastSuccessfulBuild/artifact/OpenJDK8_x64_Mac_201817022035.tar.gz

Java9: https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk9_build_x86-64_macos-libfreetype/lastSuccessfulBuild/artifact/OpenJDK9_x64_Mac_201817021647.tar.gz

Java10: https://ci.adoptopenjdk.net/view/work%20in%20progress/job/openjdk10_build_x86-64_macos-libfreetype/lastSuccessfulBuild/artifact/OpenJDK10_x64_Mac_201818020046.tar.gz