feat: add performance assessment to acceptance tests #1771
Conversation
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit 838fa46 📊 Notices ComparisonNew Errors (1 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
Dropped Errors (0 out of 1519 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (2 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
Dropped Warnings (1 out of 1519 datasets, ~0%) ✅Details of new errors due to code change, which is less than the provided threshold of 1%.
🛡️ Corruption Check1 out of 1520 sources (~0 %) are corrupted.
⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit ead9f46 📊 Notices ComparisonNew Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1523 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
|
@cka-y |
|
@emmambd exactly -- this follows the naming convention of the reports i.e. |
.../main/java/org/mobilitydata/gtfsvalidator/outputcomparator/io/CorruptedSourcesCollector.java
Show resolved
Hide resolved
main/src/main/java/org/mobilitydata/gtfsvalidator/runner/ValidationRunner.java
Show resolved
Hide resolved
📝 Acceptance Test Report📋 Summary✅ The rule acceptance has passed for commit ec6ba68 📊 Notices ComparisonNew Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Errors (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. New Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. Dropped Warnings (0 out of 1523 datasets, ~0%) ✅No changes were detected due to the code change. 🛡️ Corruption Check0 out of 1523 sources (~0 %) are corrupted. ⏱️ Performance Assessment📈 Validation TimeAssess the performance in terms of seconds taken for the validation process.
|
Summary:
This PR introduces several key improvements and changes:
latestandreferencereports.latestandreferenceare included in the JSON acceptance report. Each element in the list is formatted as follows:{ "sourceId": "<datasetId>", "referenceValidationTimeSeconds": 15.0, "latestValidationTimeSeconds": 0.924875546, "differenceSeconds": -14.075124454000001 }Expected Behavior:
Example of the markdown report for modified reports (posted in the comments of the PR here)
Example of the JSON acceptance test report
Please make sure these boxes are checked before submitting your pull request - thanks!
gradle testto make sure you didn't break anything