More flexibility in ID Generation

[koenedaele]

Use Case

Currently Atramhasis has three ways of generating a concept id:

• a numeric id generated from the database
• a GUID
• manual entry

Some users would like to have more options. Things like noid (https://metacpan.org/dist/Noid/view/noid) have been mentioned, but there are also usecases where an organisation has a central service for distributing identifiers. Therefore, the best option might be to allow users to specify a custom id generator for a certain conceptscheme. A second option is including the noid algorithm as another option similar to GUID.

It might be possible to have the system call a URI generator, but Atramhasis normally generates the URI by combining the UriGenerator and the local identifier. This would introduce extra complexities though, so for now is considered out of scope as this propasal will make it possible to generate URI's with custom local identifiers.

Implementation

Allow configuring a noid minter

Generating a noid requires some configuration such as a template, scheme and naa. These will need to be entered when configuring the provider. There's a also a Python implementation: https://pypi.org/project/noid/

IdGenerationStrategy.NOID

Allow configuring a custom id generator

By allowing a user to call a custom id generation function, we could provide a lot of flexibility (at the cost of a little bit of coding).

IdGenerationStrategy.CUSTOM

When a provider is configured with this strategy, the SkosManager.get_next_cid method calls a custom method that was passed to the provider in dotted syntax, eg. mythes.idgenerators.spam. This custom method should expect to be passed an environment that can be used for making outbound calls. Ideally this would be the pyramid request, but right now the DataManagers only have access to the SQLAlchemy session. Not sure if this is enough, or if more is needed. If we ever want to have other DataManagers that connect to eg. a SPARQL store as opposed to a SQL database, having the DataManagers receive the request seems like a step in the right direction.

TODO:

• @mielvds : Could you detail how noids are configured and what configuration options need to be present to make this work?
• @Wim-De-Clercq : Could you check the implementation for custom functions and provide feedback on this?

[mielvds]

A NOID uses a template and is reproducible via a numeric index. Some things wrt config come to mind:

• setting a start index
• choose between a random index or sequential index
• an exclusion list to prevent conflicts
• a system that prevents conflicts in general, for instance with imported URIs that use the same namespace.
  Other configuration option are (parts of) the namespace, but I'm assuming atramhasis has this already.

[Wim-De-Clercq]

While I am generally a little bit wary of passing the request object around because in my eyes it's a bit of a God object containing settings, the json, the registry which holds all services and the like, the logged in user, database session and more...

I still think it's probably the best thing to do to allow users the most liberty in how they chose to implement the logic.
We can't know what a implementor might want, so I see the request as the only decent solution to pass along.

The implementation can be quite simple I think. Pyramid has a utility method to resolve strings to callables.
So something along the lines of this would probably do the trick.

from pyramid.util import DottedNameResolver

user_setting = "my_package:concept.id_generator"
resolver = DottedNameResolver()
id_generator = resolver.resolve(user_setting)
concept_id = id_generator(request)

And since we currently call get_next_cid from inside a pyramid view, the request is already available.
We just need to expose a certain setting, read that setting, and execute the above when necessary.

[koenedaele]

@Wim-De-Clercq Sweet. This looks like an easy fix that provides a lot of extra flexibility. I'll turn this into an Issue and add it to the backlog.

@mielvds Should we add NOID as a first-class option or are we counting on people reimplementing this through the custom code option? I'm fine either way, but it's not something we have active plans to use. Although they are a lot prettier and easier to use than the standard GUIDs, so I see the appeal

[koenedaele]

Created #925 to allow configuring a custom id generator.

[koenedaele]

TODO: create issue for generating NOID