Not all relevent tests run for mutation in 1.1.1-2

[krzs]

Hello,

I think I've found a bug in pitest 1.1.1 and 1.1.2 with Void Method Call Mutator.
I have a simple class:

package test;

import java.io.Serializable;
import com.google.common.base.Preconditions;

public class MyRequest implements Serializable {

   private static final long serialVersionUID = -3548858114709541512L;
   private Long userId;

   public Long getUserId() {
      return userId;
   }

   public void setUserId(Long userId) {
      this.userId = userId;
   }

   public void validate() throws IllegalStateException {
      Preconditions.checkState(userId != null);
   }

}

and a test:

package test;

import static org.junit.Assert.assertEquals;
import org.junit.Test;

public class MyRequestTest {

   @Test(expected = IllegalStateException.class)
   public void testValidateEmpty() {
      MyRequest req = new MyRequest();
      req.validate();
   }

   @Test
   public void testValidateOk() {
      final Long userId = 99L;

      MyRequest req = new MyRequest();
      req.setUserId(userId);
      req.validate();

      assertEquals(userId, req.getUserId());
   }
}

All possible cases and mutations are covered. However pitest says:

removed call to com/google/common/base/Preconditions::checkState → SURVIVED

for line:

      Preconditions.checkState(userId != null);

which is not true, because if I remove this line the first test testValidateEmpty does not throw IllegalStateException and, of course, fails.

If I change the version of pitest to 1.1.0 it works fine, so it must have been broken somewhere in 1.1.1.

[hcoles]

Thanks for reporting this. I've just managed to reproduce it locally.

I don't yet understand the issue, but it most likely relates to the block based coverage system that was introduced 1.1.1.

[hcoles]

Commenting out testValidateOk causes the mutation to be killed. Hmm.

[hcoles]

Ok found the issue.

When a line is contained within more than 1 block and is covered by multiple tests, if some of those tests hit only 1 of those blocks the full set of covering tests is not seen as only the tests for one of those blocks is used.

I'd rank that as very serious - I'll get a fix out ASAP.

Thanks again for reporting it.

[krzs]

Ok, thanks.
I'm waiting for the new version. It's a really great tool!

[hcoles]

Should be syncing across to maven central now after a battle with the gpg plugin.