Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[xmppclient] Update smack to 4.4.4 and fix dependencies #11670

Closed
wants to merge 3 commits into from

Conversation

fwolter
Copy link
Member

@fwolter fwolter commented Nov 30, 2021

I updated smack to 4.4.4 and updated also all transitive dependencies. A connection to a XMPP server is working when letting it run with eclipse/bndtools and the karaf feature verification succeeds, too. But when building a kar and deploying it in an openHAB instance, I get the following error:

org.apache.felix.resolver.reason.ReasonException: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=org.openhab.binding.xmppclient; type=karaf.feature; version="[3.2.0.SNAPSHOT,3.2.0.SNAPSHOT]"; filter:="(&(osgi.identity=org.openhab.binding.xmppclient)(type=karaf.feature)(version>=3.2.0.SNAPSHOT)(version<=3.2.0.SNAPSHOT))" [caused by: Unable to resolve org.openhab.binding.xmppclient/3.2.0.SNAPSHOT: missing requirement [org.openhab.binding.xmppclient/3.2.0.SNAPSHOT] osgi.identity; osgi.identity=minidns-client; type=osgi.bundle; version="[1.0.2,1.0.2]"; resolution:=mandatory [caused by: Unable to resolve minidns-client/1.0.2: missing requirement [minidns-client/1.0.2] osgi.wiring.package; filter:="(osgi.wiring.package=android.os)"]]

The transitive dependency minidns-client imports android.os. I tried to wrap the minidns-client bundle and add !android.* to Import-Package, but that doesn't seem to have any effect to the manifest file in the minidns-client JAR in the KAR file.

<bundle dependency="true">wrap:mvn:org.minidns/minidns-client/1.0.2$Import-Package=!android.*&amp;overwrite=merge</bundle>

I think I came to an end with my OSGi/karaf knowledge. Any help appreciated. @wborn @Flowdalic @ibauersachs

Fixes #11560

@fwolter fwolter added bug An unexpected problem or unintended behavior of an add-on help wanted work in progress A PR that is not yet ready to be merged labels Nov 30, 2021
@fwolter fwolter requested a review from a team November 30, 2021 10:50
@Flowdalic
Copy link

Flowdalic commented Nov 30, 2021

I fear that I can not help much with OSGi and OpenHAB specifics, but I want to make one remark:

The transitive dependency minidns-client imports android.os.

This is not correct: minidns-client does not import android.os, it just tries to perform a reflective access to android.os.SystemProperties. I hope that this information helps to fix the issue.

@ibauersachs
Copy link

ibauersachs commented Nov 30, 2021

Do you actually need minidns-client? Smack can use various resolvers, and I'd suggest using the one from the JVM since that is available anyway (and you're referencing smack-java8 anyway). I.e. remove smack-resolver-minidns

You also don't want xpp3:xpp3_min and org.apache.servicemix.bundles:org.apache.servicemix.bundles.xmlpull. The former is the original (without OSGi metadata), the latter is a wrapper to include OSGi metadata. Pick one.

The transitive dependency minidns-client imports android.os.

This is not correct: minidns-client does not import android.os, it just tries to perform a reflective access to android.os.SystemProperties. I hope that this information helps to fix the issue.

That's not correct: bnd tries to be 'smart' by default and scans for Class.forName usages, resulting in an OSGi import for android.os. To remove that, the next minidns version should probably either exclude android.os or add the bnd instruction -noclassforname.
I don't use minidns in Jitsi, that's why I didn't notice this import.

Import-Package: android.os,org.minidns,org.minidns.cache,org.minidns.d
 nsmessage;version="[1.0,2)",org.minidns.dnsname;version="[1.0,2)",org
 .minidns.dnsqueryresult,org.minidns.dnsserverlookup,org.minidns.edns;
 version="[1.0,2)",org.minidns.record;version="[1.0,2)",org.minidns.so
 urce,org.minidns.util;version="[1.0,2)"

Signed-off-by: Fabian Wolter <[email protected]>
@fwolter
Copy link
Member Author

fwolter commented Nov 30, 2021

I removed smack-resolver-minidns and its transitive dependencies, which seems to work. Thanks for the hint! However, minidns-core seems to be necessary during compile time: https://github.com/igniterealtime/Smack/blob/48f5e349b9a318ba2a1d82aef9fa069e62da10bb/smack-core/src/main/java/org/jivesoftware/smack/ConnectionConfiguration.java#L83

Now, the same problem with smack-core and sun.security.pkcs11:

org.apache.felix.resolver.reason.ReasonException: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=org.openhab.binding.xmppclient; type=karaf.feature; version="[3.2.0.SNAPSHOT,3.2.0.SNAPSHOT]"; filter:="(&(osgi.identity=org.openhab.binding.xmppclient)(type=karaf.feature)(version>=3.2.0.SNAPSHOT)(version<=3.2.0.SNAPSHOT))" [caused by: Unable to resolve org.openhab.binding.xmppclient/3.2.0.SNAPSHOT: missing requirement [org.openhab.binding.xmppclient/3.2.0.SNAPSHOT] osgi.identity; osgi.identity=smack-streammanagement; type=osgi.bundle; version="[4.4.4,4.4.4]"; resolution:=mandatory [caused by: Unable to resolve smack-streammanagement/4.4.4: missing requirement [smack-streammanagement/4.4.4] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.jivesoftware.smack)(version>=4.4.0)(!(version>=5.0.0)))" [caused by: Unable to resolve smack-core/4.4.4: missing requirement [smack-core/4.4.4] osgi.wiring.package; filter:="(osgi.wiring.package=sun.security.pkcs11)"]]]

@ibauersachs
Copy link

minidns-core is required both at compile and at runtime. But that's really the only minidns dependency.

The last resolving issue boils down to: sun.security.pkcs11. That package is part of the JVM, add it to the system packags (via the property org.osgi.framework.system.packages.extra). See the doc for details about that property.
Arguably, sun.security.* could be made optional in smack-core's imports.

@Flowdalic
Copy link

Do you actually need minidns-client? Smack can use various resolvers, and I'd suggest using the one from the JVM since that is available anyway (and you're referencing smack-java8 anyway). I.e. remove smack-resolver-minidns

The advantage of minidns-client is that it allows Smack to use DANE to authenticate the server certificate. But for now, OpenHAB could probably live well without MiniDNS and instead using smack-resolver-javax.

That's not correct: bnd tries to be 'smart' by default and scans for Class.forName usages, resulting in an OSGi import for android.os.

I see. I wonder if there is a way to disable this for certain Java packages, e.g. org.minidns, besides the artifact declaring -noclassforname?

@Flowdalic
Copy link

For the paper trail: MiniDNS/minidns#123 should allow you to use minidns-client in an OSGi environment (assuming that I did everything correctly and this works).

@ibauersachs
Copy link

That's not correct: bnd tries to be 'smart' by default and scans for Class.forName usages, resulting in an OSGi import for android.os.

I see. I wonder if there is a way to disable this for certain Java packages, e.g. org.minidns, besides the artifact declaring -noclassforname?

I don't know. I discovered that "feature" the hard way when upgrading maven-bundle-plugin. IMO the exclusion you mention should be android, i.e. never declare an import for this package, not disabled it for a project. But in any case, the default should be to import such packages only optional.

@fwolter
Copy link
Member Author

fwolter commented Nov 30, 2021

Adding sun.security.pkcs11 to org.osgi.framework.system.packages.extra works, but the karaf feature verification during build isn't aware of the package, yet. But I managed to wrap smack-core successfully to overwrite Import-Package. Propably not the finest way, but we don't need to change the framework settings. BTW there are plenty of other addons, masking sun.security. Which feature isn't working in smack, if we'd keep it removed?

When installing the xmppclient feature, the entire OSGi environment needs to be refreshed due to org.apache.aries.spifly.dynamic.framework.extension:

openhab> feature:install -t -v openhab-binding-xmppclient
Adding features: openhab-binding-xmppclient/[3.2.0.SNAPSHOT,3.2.0.SNAPSHOT]
Changes to perform:
  Region: root
    Bundles to install:
      mvn:org.jxmpp/jxmpp-core/1.0.3
      mvn:org.jxmpp/jxmpp-jid/1.0.3
      mvn:org.jxmpp/jxmpp-util-cache/1.0.3
      mvn:org.minidns/minidns-core/1.0.2
      mvn:org.apache.aries.spifly/org.apache.aries.spifly.dynamic.framework.extension/1.3.2
      mvn:org.openhab.addons.bundles/org.openhab.binding.xmppclient/3.2.0-SNAPSHOT
      wrap:mvn:org.igniterealtime.smack/smack-core/4.4.4$Import-Package=!sun.security.pkcs11,org.jivesoftware.*&overwrite=merge
      mvn:org.igniterealtime.smack/smack-experimental/4.4.4
      mvn:org.igniterealtime.smack/smack-extensions/4.4.4
      mvn:org.igniterealtime.smack/smack-im/4.4.4
      mvn:org.igniterealtime.smack/smack-java8/4.4.4
      mvn:org.igniterealtime.smack/smack-resolver-javax/4.4.4
      mvn:org.igniterealtime.smack/smack-sasl-javax/4.4.4
      mvn:org.igniterealtime.smack/smack-streammanagement/4.4.4
      mvn:org.igniterealtime.smack/smack-tcp/4.4.4
      mvn:org.igniterealtime.smack/smack-xmlparser/4.4.4
      mvn:org.igniterealtime.smack/smack-xmlparser-xpp3/4.4.4
      wrap:mvn:org.hsluv/hsluv/0.2
      wrap:mvn:xpp3/xpp3/1.1.4c
  Bundles to refresh:
[... lots of bundles ...]
    org.eclipse.osgi/3.16.300.v20210525-1715 (Attached fragments changed: [osgi.identity; type="osgi.fragment"; version:Version="4.3.3"; osgi.identity="org.apache.karaf.features.extension", org.apache.aries.spifly.dynamic.framework.extension/1.3.2])
[... lots of bundles ...]

That leads eventually to:

The system bundle needs to be refreshed, restarting Karaf...

After a restart, strange things happen like NPEs deep in Jetty and freezing karaf console.

When I remove the org.apache.aries.spifly.dynamic.framework.extension dependency and keep only org.apache.aries.spifly.dynamic.bundle, the first installation of the xmppclient feature works and the addon is working as expected. But after uninstalling and installing the xmppclient feature again, the famous "Xpp3XmlPullParserFactory not a subtype" exception arises. So, I guess it's only a fluke that it works during the first installation.

I couldn't find any documentation what exactly org.apache.aries.spifly.dynamic.framework.extension is for. The SPI Fly docs mention only org.apache.aries.spifly.dynamic.bundle.

@ibauersachs
Copy link

Adding sun.security.pkcs11 to org.osgi.framework.system.packages.extra works, but the karaf feature verification during build isn't aware of the package, yet. But I managed to wrap smack-core successfully to overwrite Import-Package. Propably not the finest way, but we don't need to change the framework settings. BTW there are plenty of other addons, masking sun.security. Which feature isn't working in smack, if we'd keep it removed?

The PKCS11 usage is about smartcard authentication to an XMPP server, so most likely not anything that's used in OpenHAB. I already have sun.security.pkcs in .extra in Jitsi, so again something I didn't notice and which should become an optional import in Smack.

When installing the xmppclient feature, the entire OSGi environment needs to be refreshed due to org.apache.aries.spifly.dynamic.framework.extension:

Well, SpiFly is a framework extension, so I'm not suprised by that. Can't you add SpiFly as a system package (instead of inside this plugin)?

That leads eventually to:

The system bundle needs to be refreshed, restarting Karaf...

After a restart, strange things happen like NPEs deep in Jetty and freezing karaf console.

When I remove the org.apache.aries.spifly.dynamic.framework.extension dependency and keep only org.apache.aries.spifly.dynamic.bundle, the first installation of the xmppclient feature works and the addon is working as expected. But after uninstalling and installing the xmppclient feature again, the famous "Xpp3XmlPullParserFactory not a subtype" exception arises. So, I guess it's only a fluke that it works during the first installation.

I couldn't find any documentation what exactly org.apache.aries.spifly.dynamic.framework.extension is for. The SPI Fly docs mention only org.apache.aries.spifly.dynamic.bundle.

Hmm, good question, I think the framework extension used to be mentioned on that doc page.

Copy link

@Flowdalic Flowdalic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You added a lot of dependencies that are either unnecessary or transitive dependency of the main entry dependencies of Smack. In OpenHAB case, such a main entry point would be smack-java8, or maybe even smack-java8-full if you don't care about fine grained dependency control.

In a normal pom.xml, you would not add things like smack-core, as it is such a transitive dependency. However, this is maybe the right thing to do due to a particularity of OpenHABs build system, which I am unfamiliar with.

</dependency>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-xmlparser</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</properties>

<dependencies>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-java7</artifactId>
<artifactId>smack-core</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</dependency>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-xmlparser-xpp3</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

<version>${smack.version}</version>
</dependency>
<dependency>
<groupId>xpp3</groupId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smack on Java SE does not use xpp3, so it should be possible to drop this.

</dependency>
<dependency>
<groupId>org.jxmpp</groupId>
<artifactId>jxmpp-core</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</dependency>
<dependency>
<groupId>org.jxmpp</groupId>
<artifactId>jxmpp-jid</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</dependency>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-sasl-javax</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</dependency>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-resolver-javax</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You want to add this dependency explicitly if you want to use the javax resolver instead of MiniDNS.

</dependency>
<dependency>
<groupId>org.igniterealtime.smack</groupId>
<artifactId>smack-streammanagement</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

</dependency>
<dependency>
<groupId>org.hsluv</groupId>
<artifactId>hsluv</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You usually don't want to add this dependency explicitly.

@fwolter
Copy link
Member Author

fwolter commented Dec 21, 2021

Thanks for your review! Indeed, openHAB needs all transitive dependencies specified in the pom.xml. I'll probably continue to work on this after the holidays.

@maniac103
Copy link
Contributor

While you're at it, would it make sense to move the XMPP dependencies to -core, managed as transport feature (openhab.tp-xmppclient) there similarly to MQTT, HTTP and the like?
The reason for my question is me having a binding (Ecovacs vacuum cleaners) in the works that would also benefit from that as I need a XMPP client there as well.

@fwolter
Copy link
Member Author

fwolter commented Feb 20, 2022

Seems like my priorities shifted a bit. I played with the xmppclient upgrade to learn more about OSGi and karaf (which worked), but I don't use xmppclient in my openHAB setup. I'd be happy if someone could pick up this PR.

Before adding the latest xmppclient library version to OH, org.apache.aries.spifly.dynamic.framework.extension propably needs to be loaded as a system package in the official OH distribution. But I don't want to trigger this process. I don't have a clue what the impact to the entire system would be.

would it make sense to move the XMPP dependencies to -core, managed as transport feature (openhab.tp-xmppclient) there similarly to MQTT, HTTP and the like?

Sounds like a reasonable approach.

@fwolter fwolter closed this Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An unexpected problem or unintended behavior of an add-on help wanted work in progress A PR that is not yet ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[xmppclient] Binding depends on smack-experimental, which is not in OH repository
4 participants