-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AG-1385] Fixes JSON Schema expectation passing when it should fail #131
Conversation
Quality Gate passedIssues Measures |
"type": "string" | ||
}, | ||
"maxItems": 100, | ||
"minItems": 1, | ||
"type": "array" | ||
"type": "array", | ||
"uniqueItems": true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👀 tricky...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔥 LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch! Looks good to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sometimes it's the little things... ;)
Problem:
The
expect_column_values_to_match_json_schema
expectation was not correctly failing when members of thego_terms
list in thegene_biodomains
field were not all unique. After investigation, I determined that this was due to theuniqueItems
JSON Schema property not being applied correctly in ourgene_biodomains
JSON Schema file.Solution:
Fix the application of the property so that it works as intended.
Testing:
In order to ensure that the expectation and therefore the data pipeline is failing when we expect, I created an invalid
genes_biodomains.json
file with non-uniquego_terms
and ran the pipeline with these changes applied. The processing error was raised as expected.Bad JSON I used for testing:
Error received: