Allow generic participant scheme and id

[mhvdboog]

Hi,

I have a request to make the SMP implementation more generic.

A participant identifier scheme must have a structure following the PEPPOL. The Australian eInvoicing project would like to use your SMP implementation but the participant has a different format, based on ebCore Party Id. The scheme therefore has colons in it and the value is a string without colons.

Possibly make it configurable?

E.g
<smp:ParticipantIdentifier scheme=”urn:oasis:names:tc:ebcore:partyid-type:iso6523:0151”> 23601120601 </smp:ParticipantIdentifier>

I think I traced it to the following code.

peppol-commons:

com.helger.peppol.identifier.CIdentifier;

MAX_IDENTIFIER_SCHEME_LENGTH = 25;
PARTICIPANT_IDENTIFIER_SCHEME_REGEX = "[a-z0-9]+-[a-z0-9]+-[a-z0-9]+";

com.helger.peppol.identifier.IdentifierHelper;

PATTERN_PARTICIPANT_ID = "^([^:]*):(.*)$";

Regards,
Martijn.

[phax]

Hi Martin!
This is definitively a tricky one. The things you are pointing to are the starting points.
Additionally there are a lot of other places where this stuff is (implicitly) used.

I need to perform a separation between generic and PEPPOL identifiers.
Do you use your own scheme for document and process identifiers as well?
// Philip

[mhvdboog]

Hi Philip,

The document and process identifiers seem to be fine. Possible examples are:

Document identifier

<DocumentIdentifier scheme="dbc-docid"> urn:www.digitalbusinesscouncil.com.au:dbc:einvoicing:doctype:core-invoice:xsd::core-invoice-1##urn:www.digitalbusinesscouncil.com.au:dbc:einvoicing:process:einvoicing01:ver1.0 </DocumentIdentifier>

Process identifier

<ProcessIdentifier scheme="dbc-procid">urn:www.digitalbusinesscouncil.com.au:dbc:einvoicing:ver1.0 </ProcessIdentifier>

I tested the current version of SMP and it accepts the values fine.

I don't know your code very well but I was thinking (very simplistically) to initialise the length values and regex values from environment properties for a quick fix. Default values are the current ones. Override with -D arguments ;-)

[phax]

Well, I'm more the generic type :)
I introduced a so called "Identifier factory" that lets you choose what to do. This is something in peppol-commons project-
I need to test everything through and will let you know by tomorrow.

[phax]

Hi Martin!
I think I found a proper solution now, but I'm not quite happy with the custom identifier validation.
There is now a Simple...Identifier that simply takes all values and a special Peppol...Identifier that handles the PEPPOL specific requirements.

I though about making special checks based on the scheme (like iso6523-actorid-upis) but I'm not quite sure whether this is appropriate or not - any thoughts on this one?

// Philip

[mhvdboog]

Hello Philip,

The scheme we are proposing to use is based on ebCore Party Id. The scheme identifier is a URN so doesn’t resemble the scheme identifiers from PEPPOL. As long as the scheme and id are separated by a double colon (::) it shouldn't matter what the structure of the scheme is. To be honest I'm not sure if there is a requirement to test for correctedness of the scheme or id. Maybe for the Simple...Identifier just keep it generic and allow any value?

Kind regards,

Martijn.

[phax]

Hi Martin!
i know that you want to use ebCore Party Id.
I was thinking about some SPI interface that allows to plugin custom rules for validating the values for certain scheme IDs.
E.g. for iso6523-actorid-upis I would check if the value contains a ':' and that the first part is a 4-digit numerical value.
Would that be of help to you?

Can you please corss-check if the current SNAPSHOT works for you - thanks.

Other question: you are using the E-SENSE scheme for DNS name creation (base32 (sha256 (value.lowercase()))) which is not yet supported by the respective SMPClient. If you need support for this one, please file an issue as well. Thanks.

// Philip

[mhvdboog]

Hi Philip,

I'll check by Wednesday, is the Snapshot available from the maven repository by any chance, or do I need to build from source?

I'm not familiar with SPI. Anything to make it configurable and allow implementers to configure the behaviour of the application would benefit users I would think.

We decided to stick with the BDX location specification and encode the identifier with MD5, so no change needed, unless eSENS will become a user ;-)

Thank you so much for all your effort Philip. I understand this is all done in your own time?

Regards,

Martijn.

[phax]

Hi Martin!
I will configure the SMP to push SNAPSHOTs to Maven central - I will keep you posted when it is online.
I also need to update the WIKI so that the new property for the identifier factory is documented.
// Philip

[phax]

The current SNAPSHOT can now always be found here: https://oss.sonatype.org/content/repositories/snapshots/com/helger/peppol-smp-server-webapp/5.0.0-SNAPSHOT/

[mhvdboog]

Hi Philip,

Found the artefacts! Two questions:

1. I use the maven dependency plugin for copying the artifact from the repository. The snapshot artifacts seems to use some other naming convention which is not compatible with the plugin.
   For the release version I use this configuration:

   <artifactItem>
<groupId>com.helger</groupId>
<artifactId>peppol-smp-server-webapp</artifactId>
<version>4.1.2</version>
<type>war</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}</outputDirectory>
<destFileName>peppol-smp-server.war</destFileName>
</artifactItem>

2. I manually downloaded the webapp war from the repository;

https://oss.sonatype.org/content/repositories/snapshots/com/helger/peppol-smp-server-webapp/5.0.0-SNAPSHOT/peppol-smp-server-webapp-5.0.0-20160607.170614-5.war

Then I build a docker image to deploy to google containers. When I run the image with the release version everything is fine. When I run the image with the snapshot version I get the following error:
Caused by: java.lang.UnsupportedClassVersionError: com/helger/photon/bootstrap3/stub/PhotonStubServletContainerInitializer : Unsupported major.minor version 52.0 (unable to load class com.helger.photon.bootstrap3.stub.PhotonStubServletContainerInitializer)

Probably something obvious I missed?

[phax]

I think if you modify the configuration to 5.0.0-SNAPSHOT this should do the trick for you.
Additionally version 5 requires JDK 8!

[mhvdboog]

Hi Philip,

You're getting there!

Changed to JKD 8, the server now starts 👍
Changed the configuration for smp.identifiertype=simple 👍
Entered a ebCore Party Id kinda like identifier 👍
So all looks good so far.

However, querying the SMP with the ID fails:
http://192.168.99.100/urn%3Aoasis%3Anames%3Atc%3Aebcore%3Apartyid-type%3Aiso6523%3A0151%3A%3A12345678901
Results in:
Not found: Failed to parse serviceGroup 'urn:oasis:names:tc:ebcore:partyid-type:iso6523:0151::12345678901' at http://192.168.99.100/urn%3Aoasis%3Anames%3Atc%3Aebcore%3Apartyid-type%3Aiso6523%3A0151%3A%3A12345678901

Also an error appears when starting the server but I assume this is not a critical error because the Web UI works as expected (as far as my limited test goes):
[2016-06-08T06:11:53,241] [SMP-SERVER] [ERROR] [localhost-startStop-1] Unable to load an SPI implementation of interface com.helger.peppol.smpserver.backend.ISMPBackendRegistrarSPI -- com.helger.commons.lang.ServiceLoaderHelper.getAllSPIImplementations(ServiceLoaderHelper.java:151) java.util.ServiceConfigurationError: com.helger.peppol.smpserver.backend.ISMPBackendRegistrarSPI: Provider com.helger.peppol.smpserver.mock.MockSMPBackendRegistrarSPI not found

Haven't tested the maven dependency setting but I suspect it won't work because the files are appended with build numbers, but I'll let you know.

[phax]

I fixed both issues - the error does not occur any longer (was a configuration problem on my side with test resource filtering) and the REST APIs now use the identifier factory for parsing (tested and working).

The new SNAPSHOT is available at peppol-smp-server-webapp-5.0.0-20160608.082845-7.war (when using SNAPSHOTs, use mvn -U to force update of SNAPSHOTs)

[mhvdboog]

Awesome!

<ns2:ServiceGroup xmlns="http://busdox.org/transport/identifiers/1.0/" xmlns:ns2="http://busdox.org/serviceMetadata/publishing/1.0/" xmlns:ns3="http://www.w3.org/2005/08/addressing" xmlns:ns4="http://www.w3.org/2000/09/xmldsig#">
  <ParticipantIdentifier scheme="urn:oasis:names:tc:ebcore:partyid-type:iso6523:0151">12345678901</ParticipantIdentifier>
  <ns2:ServiceMetadataReferenceCollection/>
</ns2:ServiceGroup>

Signed service metadata seems to work as well.

I'll test some more tomorrow but looks fine so far. Thank you so much Philip!!

maven dependency plugin still isn't happy but manual download works!

 <artifactItem>
   <groupId>com.helger</groupId>
   <artifactId>peppol-smp-server-webapp</artifactId>
   <version>5.0.0-SNAPSHOTS</version>
   <classifier>20160608.082845-7</classifier>
   <type>war</type>
   <overWrite>false</overWrite>
   <outputDirectory>${project.build.directory}</outputDirectory>
   <destFileName>peppol-smp-server.war</destFileName>
</artifactItem>

turns into
Downloading: https://oss.sonatype.org/content/repositories/snapshots/com/helger/peppol-smp-server-webapp/5.0.0-SNAPSHOTS/peppol-smp-server-webapp-5.0.0-SNAPSHOTS-20160608.082845-7.war

[phax]

Change SNAPSHOTS to SNAPSHOT and try without the classifier element please.
I think I have an issue with SML communication but I need to check this first.

[phax]

The truststore issue was resolved in the meantime. If you grab a new SNAPSHOT you should be fine. Additionally I added the "register SMP to SML" from peppol.helger.com into the tool.
And here is my next question: do you use the PEPPOL SML, or are you setting up a different one? If so, that means that I need to make the SML locations customizable as well.

[mhvdboog]

We'll be using a SML different from PEPPOL. Our model for updating the SML might be slightly different so hold off on that one if you like.
I think you can close this issue :-)

[phax]

Okay, so basically you need a custom SMLClient. Do you have a similiar interface compared to ManageParticipantIdentifierServiceCaller and ManageServiceMetadataServiceCaller in the peppol-commons/peppol-sml-client project? If so, we can potentially define common base interfaces and you will get a possibility to plugin a separate implementation. But this is definitively a separate issue :)
From my point of view you can close this one.