Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IQSS/8337_add metadatalanguage to export/import/migrate #8437

Conversation

qqmyers
Copy link
Member

@qqmyers qqmyers commented Feb 17, 2022

What this PR does / why we need it: This PR adds support for the new metadatalanguage value to the OAI_ORE export, and the json import/json-ld migrate API calls. This makes the metadata language used for the dataset visible in one more export and allows sending that value in when importing /migrating from prior exports/other systems.
Which issue(s) this PR closes:

Closes #8337

Special notes for your reviewer: FWIW: Since the metadatalanguage setting constrains which languages are allowed, the import/migrate api calls return a Bad Request error if you send one that isn't allowed. (Also assumes the default if you don't send one.)

Suggestions on how to test this: Configure with metadata languages and verify that the OAI-ORE export contains an https://schema.org/inLanguage value. The import and migrate APIs can also be tested by adding this value. (I'll look at the guide examples - this is optional so I don't want to add it to all examples but I can see about showing it somewhere.)

Does this PR introduce a user interface change? If mockups are available, please link/include them here: export and api effects only. (FWIW: This is one of several updates to the OAI_ORE that will be happening - DataCommons is making several more so we may have an overall release note on the package of changes.)

Is there a release notes update needed for this change?:

Additional documentation:

@coveralls
Copy link

coveralls commented Mar 7, 2022

Coverage Status

Coverage increased (+0.1%) to 18.994% when pulling 9f651c0 on GlobalDataverseCommunityConsortium:IQSS/8337-handle_metadatalanguage_in_import/migrate into 3669666 on IQSS:develop.

@landreev landreev self-assigned this Mar 14, 2022
String expectedString = getData(response.getBody().asString());

// Delete the dataset via native API
Response deleteDatasetResponse = UtilIT.deleteDatasetViaNativeApi(datasetId, apiToken);
deleteDatasetResponse.prettyPrint();
assertEquals(200, deleteDatasetResponse.getStatusCode());


logger.info("SENDING to reCreate Dataset: " + expectedString);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to assume this info log message was left here on purpose.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~yes - we seem to have many tests that write useful intermediate results to the log (and seeing the exact json-ld sent to this API is useful). That said, nominally this and others could be changed to fine if we're concerned about the size of the IT test logs but we might also then want an easy way to up the log level when IT tests run.

Copy link
Contributor

@landreev landreev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

@landreev landreev removed their assignment Mar 14, 2022
@kcondon kcondon self-assigned this Mar 14, 2022
@kcondon
Copy link
Contributor

kcondon commented Mar 29, 2022

Retesting:

  1. With no languages or metadatalanguages defined, json import and dataset migration endpoints fail with Specified metadatalanguage ( metadataLanguage, http://schema.org/inLanguage) not allowed in this collection. DDI import works.
    [update] A recent change made metadata language required. I was using the prior example files without such language. So, a question might be why does ddi import continue to work and I'll need to retest while specifying language. A second question might be should language be required if metadata language is not enabled?

@kcondon
Copy link
Contributor

kcondon commented Apr 11, 2022

Thanks, @qqmyers !

@qqmyers qqmyers force-pushed the IQSS/8337-handle_metadatalanguage_in_import/migrate branch from 9f651c0 to 67de235 Compare April 11, 2022 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request/Idea: Support metadatalanguage in imports
5 participants