Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mutation testing #104

Merged
merged 8 commits into from
Nov 9, 2024
Merged

Mutation testing #104

merged 8 commits into from
Nov 9, 2024

Conversation

see-quick
Copy link
Member

@see-quick see-quick commented Nov 5, 2024

This PR adds to the test container mutation testing. Specifically [1]. During adoption, I saw that there are a lot of paths in the Strimzi* classes, which are not tested. So I have also added a few tests to solve it.

You can read about the benefits of mutation testing here [2].

Moreover, I have updated Azure-CI with mutation testing within the pipeline. Also if someone writes a new code then the build may fail because I have set the following thresholds. So there will be a need to add UTs to cover such features and be at least on such threshold as we have currently.

<mutationThreshold>80</mutationThreshold>
<coverageThreshold>71</coverageThreshold>

[1] - https://pitest.org/
[2] - https://www.arcmutate.com/benefits/

Signed-off-by: see-quick <[email protected]>
Signed-off-by: see-quick <[email protected]>
@see-quick see-quick added the enhancement New feature or request label Nov 5, 2024
@see-quick see-quick added this to the 0.109.0 milestone Nov 5, 2024
@see-quick see-quick requested a review from a team November 5, 2024 15:33
@see-quick see-quick self-assigned this Nov 5, 2024
Copy link
Member

@Frawless Frawless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with it, but using @DoNotMutate to quite a lot of methods within the class looks a little bit strange.

pom.xml Show resolved Hide resolved
@see-quick
Copy link
Member Author

I am okay with it, but using @DoNotMutate to quite a lot of methods within the class looks a little bit strange.

Yeah, but those methods are mainly used in Integration tests or cause a timeout of Mutation minions (e.g., randomUuid). In general, you don't want to include those because I would need to add these tests to UTs and that's not a good approach IMO (f.e., image pulling). Because now they run around 50s and with those tests, it would need like much more...

Signed-off-by: see-quick <[email protected]>
Signed-off-by: see-quick <[email protected]>
@see-quick see-quick merged commit d779f77 into main Nov 9, 2024
7 checks passed
@see-quick see-quick deleted the mutation branch November 14, 2024 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants