ObO needs sea-map to support datasets with disparate fields / vocabs

[wu-lee]

No description provided.

[wu-lee]

The various datasets use these fields:

ObO:

id, name, homepage, description, street_address, postcode, latitude, longitude

Oxford:

id, name, homepage, description, street_address, locality, postcode, latitude, longitude, twitter, facebook, org_structure, primary_activity, activities, 

Co-Ops UK:

id, name, homepage, description, street_address, locality, region, postcode, countryId, latitude, longitude

DotCoop:

id, name, homepage, description, street_address, locality, region, postcode, country_name,  latitude, longitude

What they have in common are:

id, name, homepage, description, <various address fields>, latitude, longitude

Plus some have twitter, facebook and email fields. Solidarity Oxford has the vocabs org_structure, primary_activity and secondary_activities.

Currently this works out with the current code for ObO if we ignore the vocabs, because they're all somewhat normalised to the standard.csv schema.

We can't easily add the vocabs if only one dataset has them, or if they are present but different. How would we combine them in the drop down? And does it make sense to have such a visible feature which only really applies to a small fraction of the data being shown?

More generally, if the datasets were not normalised to standard.csv (as might become more the case in future), we would probably need to go the route discussed in the field review discussion: provide some number of supported field types, which imply a semantics and a common display style in the directory/pop ups, and allow datasets to be configurable lists of names, associated with one of these types.

However, I don't think this is needed for ObO yet. Do you @ColmMassey ?

[ColmMassey]

I think we agreed that for this next iteration we would create a field 'short postcode', which is the first half of the postcode. It would be temporarily created within the javascript rather than a real field being created. It would only be useful for UK data, but valuable when there are no other common fields that make sense to be used as the primary field used for the Directory.

[wu-lee]

I've done this, and deployed a development version on the dev-1 server.

I'm thinking that the next (or at least impending) sea-map development step is to move all the field definitions out of the core and make them configurable.

[ColmMassey]

Does this warrant a new ticket. Title still seems appropriate.

[wu-lee]

The aim is something we want in general, yes - but not just for ObO. So perhaps it could be moved to sea-map, or superceded by one there? I can't find the issue about field reviews offhand, but that seems related.

[wu-lee]

The work on custom fields (which now uses TypeScript) is nominally working, and I am going to build on it to better support multiple datasets. For instance, discrepancies in the data fields can be ironed out by creating derived fields which normalise the discrepancies.

The main map site I've been working on this with is the Mutual Aid map:

https://dev.mutual-aid.solidarityeconomy.coop/

You should see an alphabetic directory, a trivial custom field which just takes the first letter of the initiative name,

The configuration is done in code now. The source code defining the above content is here:

https://github.com/SolidarityEconomyAssociation/mutual-aid-project/blob/d632d4d9af4000ee6e212ce1c32906fc3da22334/src/index.ts#L45

I've tested the code on the ICA map, and this seems to be working too.

What I want to do next is allow the vocab fields to define their own default-graph-uri. One problem with the existing config, is that it allowed only one to be set for all vocabs. With no default-graph-uri setting, the vocabs query goes extremely slowly. But when there are multiple datasets, only one can have the right default-graph setting.

(This is why the public ObyO map was so slow, until I added the default graph setting back; the private one is unavoidably slow.)

It might also be possible to add a custom field which reads vocabs from Airtable, if Airtable's API allows cross-origin requests. Although in the long term this is not a great solution - since the browser does the reading, if there are lots of visitors to the map, there is a lot of traffic to Airtable.

[ColmMassey]

The work on custom fields (which now uses TypeScript) is nominally working,

Great.

and I am going to build on it to better support multiple datasets. For instance, discrepancies in the data fields can be ironed out by creating derived fields which normalise the discrepancies.

But that will need some explaining... I propose you present this at the next All-Hands.

[wu-lee]

Owned by Oxford have a development deployment of sea-map which hsa configurable initiative attributes, and, by bypassing the SPARQL pipeline, can obtain extra fields not in standard.csv, such as the "Relationship" field.

This was completed for a demo, and is currently hardwired to their case - it needs to be made configurable.

Possibly later, the SPARQL pipeline needs to be made more flexible such that it can process these extra fields. At the moment it just does what it did, processing the standard.csv fields into linked data, ignoring any extras.

[wu-lee]

Let's move this to a new ticket

[ColmMassey]

Does https://github.com/SolidarityEconomyAssociation/technology-and-infrastructure/issues/80 capture what is left to do in this ticket @wu-lee ?

[wu-lee]

Yes, I think so - unless you want to include doing this with SPARQL on the roadmap?