Need to sign jnilibs for macOS notarization #7
Comments
|
FYI: @akurtakov @merks |
|
I assume, but wanted to confirm, that if/when Orbit provides a version with signed libraries, that you will switch linuxtools to use that instead. |
|
Sure will do |
|
Looking in Orbit, I see these
I assume that this version, which is in the CDT, package is not also a problem:
|
|
I don't know why at the moment Apple isn't complaining about JNA's jnilibs. It has in the past, and may again in the future. |
|
Maybe the jna ones are signed? Is there a way to tell? These the the exact jars: https://repo1.maven.org/maven2/com/github/jnr/jffi/1.3.11/jffi-1.3.11-native.jar If the mac libraries aren't signed probably the Windows *.dll aren't signed as well, but maybe no one notices nor complains? |
|
If they are signed I would be surprised because the JNA maintainers in the past said they are not interested in providing signed jnilibs in their release. I don't know how to test on a non-mac machine if they are signed, and I don't have a mac machine to check with.
Correct :-) The Apple notarization process makes us do it. |
|
So it's probably better to plan ahead to avoid risk? If notarization suddenly fails late in a release cycle, that could put a damper on things. I think it will be especially important for these bundles not to be included in any features. Also, if we are going to be modifying the jar's content, which is the entire point/goal here, I think we need to be careful not to use exactly the same (unqualified) version (or we need to use a different bundle symbolic name). |
|
https://openjdk.org/jeps/442 - as soon as it's no longer preview in some JVM version (hopefully 22) we should start using it and stop dealing with jni/jna/jnr . |
Yes I agree. We have had some late failures in the past too since Apple is a moving target, e.g. m2e has (or had) an embedded jnilib that Hannes signed in the past eclipse-m2e/m2e-core#591 Notarization does not run on EPP for each build, but I (or you) can manually trigger it at anytime to see if there is a problem. Run https://ci.eclipse.org/packaging/job/notarize-downloads/ (but probably best disable https://ci.eclipse.org/packaging/job/simrel.epp-tycho-build/ so that the staging repo isn't changed while notarization is running). IIRC Notarization runs on Platform on each build. |
It might be a case of getting everyone else to start using it too. |
|
Well, from platform side doing core.net and equinox.security should free us from jna dependency. Which would be good first step. |
Add maven-sign to the orbit-aggregation #7
|
The bundles with signed native libraries, including signed *.dll, are available here: https://download.eclipse.org/tools/orbit/simrel/maven-sign/ They are also available in the aggregation: https://download.eclipse.org/tools/orbit/simrel/orbit-aggregation/nightly/latest In the current SimRel, these are the strict requirements on these two things.
Directly reusing com.sun.jna 5.13.0 as a Maven dependency is quite popular:
Only passage @ruspl-afed locks in that specific exact version. How best do we ensure that we don't end up with the unsigned versions in the aggregation? I could add a rule to specifically exclude them, but we probably ought to get the contributing projects to use these signed versions instead... It is possible to create an exclusion rule, though it's marked as an error when placed on a repository that doesn't contain a matching IU:
It would fail like this:
|
|
@merks Is everything ready to switch Linux Tools to signed jffi ? |
|
Yes, both nightly builds (maven-sign and orbit-aggregation) contain the new IU with a qualifier:
|
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details.
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details.
@merks I assume with @akurtakov updating Linux Tools and you adding exclusion rule we'll have the correct version in SimRel soon? If so, we can try to notarize the nightly build of EPP. |
|
With this change https://git.eclipse.org/r/c/simrel/org.eclipse.simrel.build/+/203578 I've contributed (temporarily) both new bundles to SimRel aggregation. The new jffi.native version is the only one in the aggregation:
So once the aggregation completes https://ci.eclipse.org/simrel/job/simrel.runaggregator.pipeline/ you can test if notarization works. Until passage doesn't lock in the unqualified version of com.sun.jna, we can't do much about that, but the newer version is there:
|
|
Thanks Ed. EPP build is running, when done I will try to notarize it and report back, |
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
|
Notarized successfully. |
|
The new jffi exposes an issue. External Plug-in Libraries project that is auto-managed by JDT(?) has an error:
For some reason jffi appears as an unpacked bundle, perhaps due to |
|
Steps to reproduce last comment:
Good thing is I found an easy workaround for now, right-click on the bundle in the plug-ins view and choose remove from java workspace scope:
|
|
I think what causes the issue is having Bundle-ClassPath: .,jniIs this a JDT bug, or a JFFI bug? |
|
The ebr version looked like this: The new version like this: The ebr versions exported packages look kind of bogus, given these can't even be package names. The only classpath difference is the / on the jni. I think the MANIFEST.MF must also be on the classpath, so I don't think it's wrong the way it is. But I could modify the recipe to add the /? |
|
Oh, no, I can't change the recipe because this doesn't use a recipe. It's exactly like this in Maven central: https://repo1.maven.org/maven2/com/github/jnr/jffi/1.3.11/jffi-1.3.11.jar |
|
Perhaps I could change it because there is a post-processing step when the libraries are being signed, but it's no clear why the / would make JDT happy/happier... |
|
Is this one done or there is still some problem that I have to look at ? |
eclipse-platform#1197 eclipse-platform#1199 Use version of com.sun.jna with signed libraries: eclipse-orbit/orbit-simrel#7 Update net.bytebuddy to 1.14.6
eclipse-platform#1197 eclipse-platform#1199 Use version of com.sun.jna with signed libraries: eclipse-orbit/orbit-simrel#7 Update net.bytebuddy to 1.14.6
#1197 #1199 Use version of com.sun.jna with signed libraries: eclipse-orbit/orbit-simrel#7 Update net.bytebuddy to 1.14.6
|
I think we are done here now. |
Orbit signs the native libs to get MacOS notarization succeed. See eclipse-orbit/orbit-simrel#7 for details. Adjustments for everything to resolve properly with latest Orbit.
|
I am having the same problem with a non sandboxed java app that is using JNA 5.10. after i run the notarize command:
that fails with this output: |
|
@DanielZanchi macOS notarization can be a bit of a whack-a-mole game as items that used to notarize suddenly stop as Apple adds more and more checks. Thanks for the heads-up. Eclipse signing and notarization is working fine at the moment (from Jan 30): (full log) |
The new version of JFFI contributed here and used by linuxtools (which is included in CPP and Embed CPP package) here causes notarization to fail because the jnilibs are not signed.
In EBR based Orbit we do sign the jnilibs as part of the Orbit process. See this bugzilla for reference the last time JFFI failed and this part of pom.xml for how we sign the jnilibs.
Note that JNA needed signing in the past, but Apple's notarization processes sometimes look arbitrarily deep in jars and provides a moving target that we need to chase regularly. Also, the JNA maintainers are not interested in providing signed jnilibs in their release.
full json of error
{ "logFormatVersion": 1, "jobId": "b0fe30eb-7a90-48a0-9b26-de50880fcd81", "status": "Invalid", "statusSummary": "Archive contains critical validation errors", "statusCode": 4000, "archiveFilename": "eclipse-cpp-2023-09-M2-macosx-cocoa-aarch64-6370600730443560226.dmg", "uploadDate": "2023-08-03T18:24:52Z", "sha256": "320d3da6e1e89a841ba4e796f083ddec1c66e81bb11a1c6c9ab444443b2bdf84", "ticketContents": null, "issues": [ { "severity": "error", "code": null, "path": "eclipse-cpp-2023-09-M2-macosx-cocoa-aarch64-6370600730443560226.dmg/Eclipse.app/Contents/Eclipse/plugins/com.github.jnr.jffi.native_1.3.11.jar/jni/Darwin/libjffi-1.2.jnilib", "message": "The binary is not signed with a valid Developer ID certificate.", "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721", "architecture": "x86_64" }, { "severity": "error", "code": null, "path": "eclipse-cpp-2023-09-M2-macosx-cocoa-aarch64-6370600730443560226.dmg/Eclipse.app/Contents/Eclipse/plugins/com.github.jnr.jffi.native_1.3.11.jar/jni/Darwin/libjffi-1.2.jnilib", "message": "The signature does not include a secure timestamp.", "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733", "architecture": "x86_64" }, { "severity": "error", "code": null, "path": "eclipse-cpp-2023-09-M2-macosx-cocoa-aarch64-6370600730443560226.dmg/Eclipse.app/Contents/Eclipse/plugins/com.github.jnr.jffi.native_1.3.11.jar/jni/Darwin/libjffi-1.2.jnilib", "message": "The binary is not signed with a valid Developer ID certificate.", "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087721", "architecture": "arm64" }, { "severity": "error", "code": null, "path": "eclipse-cpp-2023-09-M2-macosx-cocoa-aarch64-6370600730443560226.dmg/Eclipse.app/Contents/Eclipse/plugins/com.github.jnr.jffi.native_1.3.11.jar/jni/Darwin/libjffi-1.2.jnilib", "message": "The signature does not include a secure timestamp.", "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087733", "architecture": "arm64" } ] }The text was updated successfully, but these errors were encountered: