Adding app_context from SenNet

[gesinaphillips]

This deviates from the SenNet in a few necessary and one maybe-tryhard way.

• Necessary: I need to enhance the update in SenNet to allow for passing the Ingest API URL as well. This led to the changes in validation_utils.
• Am I doing too much here?: I added Upload.get_app_context to clean the submitted app_context data to ensure that all necessary keys/values are present, but to prioritize any user-submitted data.

Note the TODO in upload.py regarding the hardcoded defaults.

    # TODO: this would be better in a config so it could be shared with
    # validation_utils.get_assaytype_data / would not conflict with SenNet values
    # For the moment, ingest_url is just hard-coded in get_assaytype_data
    # to allow validate_tsv.py to run

What do you think?

[jpuerto-psc]

@gesinaphillips - so the hard-coding is setting the ingest-api value when the validate_tsv function is used outside of the context of an upload? I think this will cause issues in SenNet (not sure) because I believe that they do exactly that. They call that function without constructing an upload, when someone uploads a metadata.tsv file. So I think we would have to hard-code that value in the SenNet IVT. Everything else looks fine!

[gesinaphillips]

@jpuerto-psc validation_utils.get_tsv_errors actually ends up creating an Upload (kind of a strange pattern, still trying to think of an alternative). It's actually the validation_utils.get_schema_version call in validate_tsv.py where the ingest_url is required, which is why it's hard-coded directly into validation_utils.get_assaytype_data (which is called by get_schema_version).

But either way, if we have the HuBMAP URLs hard-coded in upload.py and validation_utils.py, they will have to make sure to change those to add their own SenNet URLs.

[sunset666]

@jpuerto-psc, Updated: Not a major issue with the update on SenNet since the fork will allow the hardcoded url to be updated, we just need to inform about this.

[jpuerto-psc]

@jpuerto-psc validation_utils.get_tsv_errors actually ends up creating an Upload (kind of a strange pattern, still trying to think of an alternative). It's actually the validation_utils.get_schema_version call in validate_tsv.py where the ingest_url is required, which is why it's hard-coded directly into validation_utils.get_assaytype_data (which is called by get_schema_version).

But either way, if we have the HuBMAP URLs hard-coded in upload.py and validation_utils.py, they will have to make sure to change those to add their own SenNet URLs.

Fantastic, then we're good here I think!