Deploying Maven SNAPSHOT version for testing

[reckart]

I can apparently register a UIMA/Maven component which is a SNAPSHOT version via its Maven coordinates. The registry obtains the artifact and offers me a choice of the components inside it.

However, SNAPSHOT versions change all the time. I am fixing things, update documentation, etc. etc.

My feeling is that the registry only obtains the SNAPSHOT artifact once and then uses a cached version, because I do not see the latest changes to my artifact when I try to register it.

How can I force the registry to use the clear its local cache and use the latest version of the SNAPSHOT artifact (and also of any SNAPSHOT dependencies that artifact might have)?

It would be good to to have access to a list of the resolved artifacts for a component and their timestamps/SHA1 hashes so one can check which specific file versions are being used by the registry.

[greenwoodma]

Yes, I believe once a component is registered the docker image is created and that's that. So registering a -SNAPSHOT version will act as you have observed; i.e. it will never get updated.

Having said that I think the real fix here is that we should prevent the platform from allowing you to register a SNAPSHOT version. One of the main aims of the platform is to ensure reproducability. If we allow SNAPSHOT versions of components then it becomes impossible to grantee any sense of reproducability as the code could easily change. In fact if we limit to pulling from Maven central then that should ensure reproducability as you can't push SNAPSHOT releases (or an artifact that depends on a SNAPSHOT release) to central.

[reckart]

@greenwoodma if the system creates a docker image from the SNAPSHOT at the time it is registered, it should be ok. What I am worried about is that the system might cache Maven artifacts across registrations.

So if I delete a component and re-register it, will it re-obtain the SNAPSHOT and all its dependencies freshly from their respective repositories and not use any cached artifacts?

[reckart]

@greenwoodma IMHO for development, it is important to be able to test SNAPSHOTs on the platform. If one has to do full releases to Maven Central every time a problem is fixed just in order to hit the next problem, IMHO that is a problem.

[reckart]

I'm talking about private deployment and testing on the platform. Preventing the sharing of SNAPSHOT versions is very reasonable.

[galanisd]

Yes, I believe once a component is registered the docker image is created and that's that. So registering a -SNAPSHOT version will act as you have observed; i.e. it will never get updated.

For UIMA/Maven components we have a generic built-in Docker image that each time...
in every workflow execution.

• Downloads the component and its dependencies from Maven Central.
• Starts the component with a UIMA generic executor.

In the OMTD workfow execution part there is no caching and the latest (SNAPSHOT) artifact is downloaded. As far as I understand the issue seems to be in the Registry side (@antleb).

Unassigned myself.

[reckart]

In the OMTD workfow execution part there is no caching and the latest (SNAPSHOT) artifact is downloaded. As far as I understand the issue seems to be in the Registry side (@antleb).

Note that I am not entirely sure there actually is an issue. But there are indicators such as that the registry offered me the choice between 6 components while I only generated descriptors for one.

Of course that could also be caused by a different issue, namely the registry could be scanning the classes for component information even if the JAR already includes a META-INF/eu.openminted.share/descriptors.txt. As a user, it is hard to tell what actually happens because the system logs no information of the individual steps it is performing to the UI. (I opened a new issue for this question: #9).

[reckart]

FYI @azielinskiACC

[reckart]

Hm, there is also some issue with the Maven plugin... I just tried downloading a JAR from the OMTD repo... it includes the uima descriptors, but not the OMTD descriptors:

https://repo.openminted.eu/content/repositories/snapshots/eu/openminted/uc-tdm-socialsciences/ss-variable-detection/1.0.1-SNAPSHOT/ss-variable-detection-1.0.1-20180408.232950-48.jar

Need to check what is happening here.

[reckart]

Tried re-registering with a tweaked version string, but doesn't seem to work: openminted/omtd-registry#21

[reckart]

Ok, I can confirm that the registry does no caching. I have rebuilt the UC SSH SNAPSHOTs (this time with proper descriptors) and resolved them again: the registry correctly showed me only the single component that I wished to expose from this artifact and also picked up the correct XML descriptor.

So I'll close this issue. Should I see other hints at something being cached, I may reopen it.

[reckart]

I think I am having some issues here again...

On services.openminted.eu, I am unable to register the latest builds of DKPro Core 1.9.2-SNAPSHOTs. I have made some modifications to the metadata (e.g. of the PdfReader component), but when I try to register it, I am always confronted with previous metadata which does not include my changes. The up-to-date JARs with the proper metadata are present in the OMTD SNAPSHOT Maven repo as well as in the UKP SNAPSHOT Maven repo.

Seems to work on test.openminted.eu though.

@antleb