Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mutation test should be explicit about the expected error #705

Closed
pgrange opened this issue Feb 6, 2023 · 0 comments · Fixed by #719
Closed

Mutation test should be explicit about the expected error #705

pgrange opened this issue Feb 6, 2023 · 0 comments · Fixed by #719
Assignees

Comments

@pgrange
Copy link
Contributor

pgrange commented Feb 6, 2023

Why

Our mutation tests check that, given an invalid transaction, the validator would fail. But if we just make a totally wrong mutation that creates a transaction which wouldn't make any sense, then the validator would fail but for the wrong reason and this limit the values of our tests.

We'd like to have each mutation test be explicit about the expected error from the validation script so that we are confident that our validator failed for the write reason.

What

Now we can associate, with any mutation, an optional expected error message. So let's update all the mutation to set an expected error message and ensure they're all failing for the good reason.

If it makes sense, we might start introducing some error messages systematic management, like error code or something. At least something which reduce plain string messages duplication between tests and validator code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants