Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Commands not fully cloned during the shrinking process #270

Closed
dubzzz opened this issue Dec 19, 2018 · 3 comments
Closed

Commands not fully cloned during the shrinking process #270

dubzzz opened this issue Dec 19, 2018 · 3 comments

Comments

@dubzzz
Copy link
Owner

dubzzz commented Dec 19, 2018

Travis detected the following corner cases:

FAIL test/e2e/model/CommandsArbitrary.spec.ts (12.755s)
  ● CommandsArbitrary (seed: 1545186172739) › commands › Should shrink towards minimal case even with other arbitraries
    expect(received).toEqual(expected)
    Expected value to equal:
      "failure"
    Received:
      "success,failure"
      134 |       );
      135 |       expect(out.failed).toBe(true);
    > 136 |       expect(out.counterexample![1].toString()).toEqual('failure');
          |                                                 ^
      137 |     });
      138 |   });
      139 | });
      at Object.<anonymous> (test/e2e/model/CommandsArbitrary.spec.ts:136:49)
@dubzzz dubzzz added the bug label Dec 19, 2018
@dubzzz dubzzz pinned this issue Dec 19, 2018
@dubzzz
Copy link
Owner Author

dubzzz commented Dec 20, 2018

Current investigation proved that the issue is related to the fact that integer arbitraries memoize whenever it shrinks once to zero, after that it will not shrink to zero anymore.

√ [[7,7],,[]]
√ [[],,[2,3]]
√ [[3,6],success,success,[]]
√ [[3,8],,[8,1]]
× [[6,0,5],success,success,failure,[]]
. √ [[],,[]]
. √ [[0,5],success,success,[]]
. √ [[0,0,5],success,success,[]]
. × [[3,0,5],success,failure,[]]
. . √ [[0,5],success,success,[]]
. . × [[2,0,5],success,failure,[]]
. . . √ [[0,5],success,success,[]]
. . . × [[1,0,5],success,failure,[]]
. . . . √ [[0,5],success,success,[]]
. . . . × [[1],failure,[]]
. . . . . √ [[1],,[]]
. . . . . √ [[1],failure,[]]
. . . . . × [[1],success,failure,[]]
. . . . . . √ [[1],failure,[]]

For the moment, I see no easy fix to have it working while keeping an efficient shrinker for integers and by transitivity (all arbitraries derive from this one) all others.

@dubzzz
Copy link
Owner Author

dubzzz commented Dec 20, 2018

PR #271 is supposed to help troubleshooting this kind of issue

@dubzzz
Copy link
Owner Author

dubzzz commented Dec 20, 2018

The last line of the execution dump - ie √ [[1],failure,[]] - shows that failure command has been executed while the run has been considered to be a green one.

As commands highly depends on cloneable capabilities it might be related to an issue while cloning them: failure one might not have been cloned.

@dubzzz dubzzz changed the title Potential bug in commands shrinker Potential bug in the way to clone commands during shrinking Dec 20, 2018
@dubzzz dubzzz changed the title Potential bug in the way to clone commands during shrinking Commands are not fully cloned during the shrinking process Dec 21, 2018
@dubzzz dubzzz changed the title Commands are not fully cloned during the shrinking process Commands not fully cloned during the shrinking process Dec 21, 2018
@dubzzz dubzzz closed this as completed in 14222aa Dec 21, 2018
@dubzzz dubzzz unpinned this issue Dec 21, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant