Add Wildcard ("Contains") filter

[shaharmor]

Hi,

This PR adds a new filter to the filter bar, for the Wildcard query.

The way it works is as follow:

1. If the input contains a * char, it will just use the input as is.
2. If it doesn't contain a * char, it will wrap the input string with * to get a true "contains" filter of *input*.

Update:
New logic only wraps input with * from both sides. See #16357 (comment)

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[Bargs]

Hi @shaharmor, thanks for submitting this. I'll try to give it a look as soon as I get some time.

One thing we'll have to consider before merging any sort of wildcard filter is how we can prevent it from taking down anyone's cluster. Leading wildcards can be extremely heavy and dangerous, which is why the query_string query allows admins to turn them off. It might be as simple as making wildcard queries an opt-in feature in the advanced settings. We might also consider offering prefix queries by default so users can get some of the benefit of a wildcard query without opting in to everything.

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[matthew-hickok]

This would be great. Would love to see it happen.

[rashmivkulkarni]

jenkins, test this

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[matthew-hickok]

So close!

[timroes]

Hi @shaharmor,

I am sorry this PR was forgotten about and I apologize for that. If you still want to continue working on this, please merge master into your PR, and I'll make sure we're reviewing and continuing merging this time. Otherwise please feel free to close this.

Cheers,
Tim

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[shaharmor]

@timroes I will update it for master in the next couple of days

[shaharmor]

Hey @timroes

The "name" of the filter is "contains", which implies that any value entered will be wrapped with * from both sides, converting a filter term of value to *value*.

• Should this same filter also support "starts with" & "ends with"?
  This means that if a user enters the filter term of value* or *value, should we still make sure the value is wrapped with * from both sides, or just leave the term as is?

• If we decide to always wrap the value from both sides, should I also create two additional filters, one for "starts with" and one for "ends with"?

Note that when the new filter is active, its "title" becomes field contains value.
If we'll decide to support starts with and ends with, it will have to show the * characters like this:

• `"field" contains "*value*"
• `"field" contains "value*"
• `"field" contains "*value"

Which might not be what you want.

Please let me know how you'd like me to proceed.

[Bargs]

Hey @shaharmor, sorry again for how inconsistent we've been with feedback on this PR. We'd really like to push it forward if you have the time. I've taken a deeper look at the current code and left some feedback. In addition to the inline comments, I have a couple other comments:

• To answer your previous question, I think it would be best to create separate filter types for "Starts with" and "Ends with".
• We should add an advanced setting similar to the existing query:allowLeadingWildcards to prevent the creation and execution of "contains" and "ends with" filters since they could be dangerous to a cluster.

[Bargs]

One thing we learned when we were working on adding wildcard support to KQL is that text fields don't work well with simple wildcard queries. It's actually best to use a query_string query for wildcard queries on text fields because it does some special handling that makes it work how most users would expect. I'd suggest updating the query building logic to look like this:

  // Text fields should be treated in a special manner because their values are analyzed.
  // For example, a field configured with the standard analyzer with a value of "Foo Bar" would not
  // match with a simple wildcard query on the value "Foo Ba*" because the analyzer tokenized and
  // lowercased the text at index time. The actual values stored in the inverted index are "foo" and
  // "bar" but the wildcard query is literally searching for a token that starts with "Foo Ba".
  // The query_string query attempts to make wildcards more useful with text fields. It will do
  // tokenization and some simple normalization on the term that contains the wildcard, so that the
  // query "Foo Ba*" would actually match. This functionality is not exposed in any other query type
  // so we have to sort of abuse the query_string query here if we're using a text field
  if (field.esTypes && field.esTypes.includes('text')) {
    filter.query = {
      query_string: {
        fields: [field.name],
        query: `*${escapeQueryString(value)}*`,
      }
    };
  }
  else {
    filter.query = {
      wildcard: {
        [field.name]: {
          value: `*${value}*`,
        },
      },
    };
  }

[Bargs]

We need to take into account scripted fields, similar to how other filters builders do.

[Bargs]

Could you also add some unit tests for this function?

[timroes]

I will close this PR for now as stalled, since there hasn't been any activity for quiet some time. Please feel free to reopen this if we want to continue with it. Thanks a lot!