Warn user if regular expression is used when match_pattern defaults to simple in dynamic_template mappings

[quux00]

Description

This was flagged by @romseygeek in #95558 (comment) to be targeted for a new issue.

In dynamic_template mappings, 'simple' match types (which is the default setting) will happily accept regexes (rather than simple wildcards) without error or warning, but then they will not work as the user likely intended. We should emit a warning if the user provides a pattern that 'looks like' a regex, but haven't explicitly set match_type.

---

Deeper context:

The validate method in DynamicTemplate.parse tries to catch these, but it only catches the invalid pattern when match_pattern=regex. For example, see the test: https://github.com/elastic/elasticsearch/blob/main/server/src/test/java/org/elasticsearch/index/mapper/DynamicTemplateParseTests.java#L132

Current behavior is:

  "mappings": {
    "dynamic_templates": [
      {
        "mytemplate": {
          "match_pattern": "regex",
          "match": "*name",
          "unmatch": "*middle",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  }

fails with:

    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping: Pattern [*name] of type [regex] is invalid. Cannot create dynamic template [mytemplate].",\

but:

      {
        "mytemplate": {
          "match": ".*name$",
          "unmatch": "^.*middle.*",
          "mapping": {
            "type": "keyword"
          }
        }
      }

(where "match_pattern": "simple" is implied) works with no problems (until you try to actually put documents in with the mapping, of course).

We could probably add a method to Regex.java. There is already this function

   public static boolean isSimpleMatchPattern(String str) {
        return str.indexOf('*') != -1;
    }

but that's not enough for this scenario.

This code indicates that the only allowed patterns for "simple" are "xxx*", "xxx", "xxx" and "xxxyyy", so we could check for things like .* and ^xxx and yyy$ and chars like [, ] and | and maybe parens and then run it through REGEX.matches() and if that does NOT fail, then it appears to be a valid actual regex, not a simple pattern and then give a warning to users.

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[quux00]

Under what conditions should we put a warning header about the pattern_match="simple" case.

I can think of two cases:

1. the pattern contains no wildcard (*) at all (e.g., "match":"first_name". Is this likely an error or is this is a valid use of a dynamic_template match?

2. the pattern appears to be a full regular expression.

The latter is the goal of this ticket. Should we also warn on case 1?

[quux00]

Additional problem. ES seems to allow most regex "meta-characters" in field names.

Based on my testing, these characters are allowed in ES field names and can be queried.

^
$
?
+
|
[]
()
{}
\d
\D
\w
\W
\s
\S

And .* is allowed in a simple pattern match anywhere except at the start of a field name, due to dotted field names, such as "user.name" - the simple pattern for it would "user.*".

So the only thing we can test for that is definitely not allowed is a match pattern that starts with .*. All the rest appear to be valid in ES field names.

Example:

PUT my-index-000001
{
  "mappings": {
    "dynamic_templates": [
      {
        "mytemplate": {
          "match_pattern": "simple",
          "match": "*[foo]",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  }
}


GET my-index-000001/_mapping

PUT my-index-000001/_doc/1
{
  "employee": "fred",
  "my[foo]": "mymetadata goes here"
}

GET my-index-000001/_search 
{
  "query": {
    "match": {
      "my[foo]": "mymetadata goes here"
    }
  }
}

// the _search query above works:
{
  "took": 4,
  ...
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 0.2876821,
    "hits": [
      {
        "_index": "my-index-000001",
        "_id": "1",
        "_score": 0.2876821,
        "_source": {
          "employee": "fred",
          "my[foo]": "mymetadata goes here"
        }
      }
    ]
  }
}