Create a JSON-LD context file that can be reused across NIDM-Results documents

[cmaumet]

As suggested by @satra, it would be nice to create a JSON-LD context file listing our preferred prefixes available in https://github.com/incf-nidash/nidm/blob/master/nidm/nidm-results/terms/prefixes.csv.

This context file could then be imported by any NIDM-Results JSON-LD document.

We would need:

• to create a function that reads the preferred prefixes CSV file and transform each line to a JSON-LD context object specification.
• Update scripts/refresh.py to trigger this function as part of the tests to make sure it is run after each update.

Here is an example of JSON-LD context file: https://json-ld.org/contexts/person.jsonld

[khelm]

@dbkeator, @cmaumet, @satra : There now seems to be a problem accessing the NIDM-Results JSON-LD context file as an error is now appearing when I run refresh.py that references that file. The error seems to have something to do with the Name on the SSL certificate, not matching. The salient part is : (Caused by SSLError(CertificateError("hostname 'raw.githubusercontent.com' doesn't match u'www.github.com) but here is the full error traceback:
/home/karl/Work/packages/anaconda3/envs/nidm/lib/python2.7/site-packages/urllib3/connection.py:395: SubjectAltNameWarning: Certificate for purl.org has nosubjectAltName, falling back to check for a commonNamefor now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) SubjectAltNameWarning, /home/karl/Work/packages/anaconda3/envs/nidm/lib/python2.7/site-packages/urllib3/connection.py:395: SubjectAltNameWarning: Certificate for raw.githubusercontent.com has nosubjectAltName, falling back to check for a commonNamefor now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.) SubjectAltNameWarning, Traceback (most recent call last): File "scripts/refresh.py", line 55, in <module> main() File "scripts/refresh.py", line 40, in main recompute_all_ex.main() File "/home/karl/Work/INCF/nidm/nidm/scripts/../nidm/nidm-results/scripts/recompute_all_ex.py", line 29, in main create_spm_example.main() File "/home/karl/Work/INCF/nidm/nidm/scripts/../nidm/nidm-results/scripts/create_spm_example.py", line 649, in main example.create_example() File "/home/karl/Work/INCF/nidm/nidm/scripts/../nidm/nidm-results/scripts/create_example_from_templates.py", line 161, in create_example 'http://purl.org/nidash/context') File "/home/karl/Work/packages/anaconda3/envs/nidm/lib/python2.7/site-packages/pyld/jsonld.py", line 145, in compact return JsonLdProcessor().compact(input_, ctx, options) File "/home/karl/Work/packages/anaconda3/envs/nidm/lib/python2.7/site-packages/pyld/jsonld.py", line 690, in compact 'jsonld.CompactError', cause=cause) pyld.jsonld.JsonLdError: ('Could not process context before compaction.',) Type: jsonld.CompactError Cause: ('Could not process JSON-LD context.',) Type: jsonld.ContextError Cause: ('Dereferencing a URL did not result in a valid JSON-LD context.',) Type: jsonld.ContextUrlError Code: loading remote context failed Details: {'url': 'http://purl.org/nidash/context'} Cause: ('Could not retrieve a JSON-LD document from the URL.',) Type: jsonld.LoadDocumentError Code: loading document failed Cause: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /incf-nidash/nidm/master/nidm/nidm-results/terms/nidmr.json (Caused by SSLError(CertificateError("hostname 'raw.githubusercontent.com' doesn't match u'www.github.com'",),)) File "/home/karl/Work/packages/anaconda3/envs/nidm/lib/python2.7/site-packages/pyld/documentloader/requests.py", line 63, in loader <more stuff>

This error seems to be solved by reissuing the SSL cert, but that doesn't look like an option here. I don't know what the purl redirect looks like for this file, but does anyone have the Purl login info so I can check? @satra IIRC you had it at one time?

[khelm]

Interesting. The context file URL resolves to the correct file, so it looks like it really is an error related to the cert.

[cmaumet]

@khelm : I've shared the details for the purl login with you by email.

[khelm]

@Maumet I originally thought that the link you sent me was the issue since purl does use Let's Encrypt for it's certificates. I downloaded an updated cacert.pem file and pointed to that in my .bashrc file, but still got the same error. However, I now think it has to do with the version of requests or urllib3 that is called by jsonld.py. It's a little hard to tell which is the problem - or both - since the warning at the top of the error is from urllib3 and the error at the bottom is from requests. Unfortunately, due to the way that they are called - from within jsonld.py I can't add the flag to not check the certificate without modifying the actual module code. Upgrading urllib3 doesn't fix the error, but does raise a warning saying that the version is past what nidmresults has specified and upgrading requests gives a different error, again in a package that shouldn't be modified. I also created a new conda environment and installed nidmresults into that, but got different errors. Part of the issue, I think, is the use of python 2.7.
Commented out the lines in refresh.py to resolve this and will check for any errors on the website that this causes.
Since this python 2.7 code is getting harder to work with as these errors occur, I will pursue splitting off the new work we're doing with NIDM-Experiment into a separate repo, since we now have new terms webpages and will soon have a term graph map.