Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strengthen FilterRoutingTests #33149

Conversation

DaveCTurner
Copy link
Contributor

Today the FilterRoutingTests take the belt-and-braces approach of excluding
some node attribute values and including some others. This means that we don't
really test that both inclusion and exclusion work correctly: as long as one of
them works as expected then the test will pass. This change improves these
tests by only using one approach at once, demonstrating that both do indeed
work.

Today the FilterRoutingTests take the belt-and-braces approach of excluding
some node attribute values and including some others. This means that we don't
really test that both inclusion and exclusion work correctly: as long as one of
them works as expected then the test will pass. This change improves these
tests by only using one approach at once, demonstrating that both do indeed
work.
@DaveCTurner DaveCTurner added >non-issue v7.0.0 :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) v6.5.0 labels Aug 25, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

Copy link
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am in favor of the change, it's a good change. But I do have a difference in philosophy on using randomness for branch coverage. I would like to see the codebase move more and more away from this.

.build()))
.build();
final Settings.Builder settingsBuilder = settings(Version.CURRENT).put("index.number_of_shards", 2).put("index.number_of_replicas", 1);
if (randomBoolean()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here to improve these tests. However, I have been moving away from using randomness as a poor man's way to get branch coverage. I think that randomness is good for testing edge cases by randomly varying input data, but I don't like seeing it to half of the time cover one branch and half of the time cover another. I would prefer to see two tests here.

@DaveCTurner
Copy link
Contributor Author

Seems reasonable. This kinda snowballed as I dug into the implementation and realised there were quite a lot of things that aren't very well covered at the moment:

  • filters on multiple attributes
  • combinations of filters of different senses
  • require filters
  • wildcards

Some of these are somewhat covered at a lower level by DiscoveryNodeFiltersTests, but this doesn't properly verify the plumbing in FilterAllocationDecider, and I thought that it'd be good to add a few tests to show how this all fits together too.

@DaveCTurner DaveCTurner requested a review from jasontedor August 26, 2018 12:55
@DaveCTurner
Copy link
Contributor Author

Build failed due to issue described in #29140. @elasticmachine retest this please.

Copy link
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@DaveCTurner DaveCTurner merged commit 284c45a into elastic:master Sep 10, 2018
@DaveCTurner DaveCTurner deleted the 2018-08-24-strengthen-filter-routing-tests branch September 10, 2018 09:23
DaveCTurner added a commit that referenced this pull request Sep 10, 2018
Today the FilterRoutingTests take the belt-and-braces approach of excluding
some node attribute values and including some others. This means that we don't
really test that both inclusion and exclusion work correctly: as long as one of
them works as expected then the test will pass. This change improves these
tests by only using one approach at once, demonstrating that both do indeed
work, and adds tests for various other scenarios too.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Sep 11, 2018
* master:
  Add full cluster restart base class (elastic#33577)
  Validate list values for settings (elastic#33503)
  Copy and validatie soft-deletes setting on resize (elastic#33517)
  Test: Fix package name
  SQL: Fix result column names for arithmetic functions (elastic#33500)
  Upgrade to latest Lucene snapshot (elastic#33505)
  Enable not wiping cluster settings after REST test (elastic#33575)
  MINOR: Remove Dead Code in SearchScript (elastic#33569)
  [Test] Remove duplicate method in TestShardRouting (elastic#32815)
  mute test on windows
  Update beats template to include apm-server metrics (elastic#33286)
  Fix typos (elastic#33499)
  [CCR] Delay auto follow license check (elastic#33557)
  [CCR] Add create_follow_index privilege (elastic#33559)
  Strengthen FilterRoutingTests (elastic#33149)
  Correctly handle PKCS#11 tokens for system keystore (elastic#33460)
  Remove some duplicate request conversion methods. (elastic#33538)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >non-issue v6.5.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants