[ancestral, translate] node data validation improvements #1440
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1440 +/- ##
==========================================
+ Coverage 68.70% 68.82% +0.11%
==========================================
Files 69 69
Lines 7574 7595 +21
Branches 1858 1860 +2
==========================================
+ Hits 5204 5227 +23
+ Misses 2088 2086 -2
Partials 282 282 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thanks for following up here @jameshadfield!
I left some comments and came away from this PR wondering if we should take the opportunity to redesign how node data is handled within Augur. Maybe it'd be better to discuss through a synchronized meeting?
jameshadfield
force-pushed
the
validation-improvements
branch
from
34b0787
to
54bde7f
Compare
'q' is not a valid formatting type for python (it's a snakemake feature)
NodeDataFile (called by NodeDataReader, which is called by read_node_data) no longer attempts JSON validation when the requested validation mode is SKIP.
By leveraging the existing validation designed for reading node-data files we not only validate against the annotation schema but also perform other sanity checks on the node-data structure for which a schema does not (yet) exist. As part of this we add a `--validation-mode` argument following `augur export v2` which allows users to skip validation of the output node-data and (translate only) skip validation of the input node-data.
jameshadfield
force-pushed
the
validation-improvements
branch
from
54bde7f
to
1a9cc7f
Compare
The help message for `--validation-mode` is very specific: > 'error' mode causes a non-zero exit status if any validation checks > failed, while 'warn' does not. In the actual implementation however we would print an error and exit with code 2 for any validation check _except_ the augur version check. Now, for `ValidationMode.WARN` we catch these errors and turn them into warnings such that augur can exit code 0 as specified. Note that during validation, any non-ValidationError exception will be be fatal even in `WARN`. An example of this would be errors loading the schema itself. See the following review comment for more: <#1440 (comment)>
jameshadfield
force-pushed
the
validation-improvements
branch
from
1a9cc7f
to
a43536c
Compare
There was a problem hiding this comment.
Thanks for the validation improvements! Looks good to me 👍
This seems like a good direction and in the future we should consider adding validations to any augur commands that produce node-data JSONs. Personally, I'd like to see these validation failures early in the pipeline before the final augur export step.
See commit messages for details. Motivated by #1438 (review)