[Fleet] Namespace section in Fleet Agent Policies should disallow special characters

[kevinlog]

Kibana version:
7.11

Elasticsearch version:
7.11

Describe the bug:
While triaging a customer issue, we discovered that they were trying to stream Security Endpoint data to an index with a namespace that contained special characters, i.e. a name like endpoint+security. When the Endpoint tried to stream data to ES, ES rejected the index name due to the namespace with special characters.

Steps to reproduce:

1. Create an Agent Policy with a namespace with special characters, i.e. endpoint+security
2. Deploy an Agent with Endpoint (or presumably other beats) with that Agent Policy
3. See that ES rejects data streamed to indices with the namespace section that contains special characters

Expected behavior:
Creating the indices should work correctly and data should stream correctly to ES.

Since special characters aren't allowed by ES in the index names, we should restrict them during Agent Policy creation in Fleet.

Screenshots (if relevant):
This should be disallowed:

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

Any additional context:

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[elasticmachine]

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[jen-huang]

I am surprised that + is an issue, it is not a restricted character according to ES docs, only that index names cannot start with +, which is not the case with namespaces: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params

Do we have access to the exact error reported?

For reference, we implemented namespace validation in #75381 and #78522.

[zez3]

Does this help?

[elastic_agent.endpoint_security][notice] Client.cpp:458 Failed to create in index (2) json ({"@timestamp":"2021-03-22T15:48:08.1507223Z","agent":{"id":"24a9e8dd-d275-dc73-0e55-b6f4bce4794d","type":"endpoint","version":"7.11.2"},"data_stream":{"dataset":"endpoint.events.file","namespace":"hg+av","type":"logs"},"ecs":{"version":"1.6.0"},"elastic":{"agent":{"id":"2613b330-7490-11eb-b379-67cb6102d4e9"}},"event":{"action":"creation","category":["file"],"created":"2021-03-22T15:48:08.1507223Z","dataset":"endpoint.events.file","id":"M3SB2RLn8Uhyxdq1++++0wPc","kind":"event","module":"endpoint","sequence":1017699,"type":["creation"]},"file":{"Ext":{"windows":{"zone_identifier":-1}},"extension":"dat-journal","name":"netDB.dat-journal","path":"C:\\Program Files\\Fortinet\\FortiClient\\logs\\netDB.dat-journal"},"host":{"architecture":"x86_64","hostname":"DESKTOP-BP8AL9Q","id":"6d4b6b37-5fcd-4de2-8f7c-d18fcaee15c2","ip":["169.254.217.190","fe80::11ed:b7d5:d8ee:d9be","10.0.2.15","fe80::a5e5:7f08:2dc4:32a6","127.0.0.1","::1"],"mac":["00:09:0f:fe:00:01","08:00:27:46:5e:b2"],"name":"DESKTOP-BP8AL9Q","os":{"Ext":{"variant":"Windows 10 Home Single Language"},"family":"windows","full":"Windows 10 Home Single Language 1909 (10.0.18363.1441)","kernel":"1909 (10.0.18363.1441)","name":"Windows","platform":"windows","version":"1909 (10.0.18363.1441)"}},"message":"Endpoint file event","process":{"Ext":{"ancestry":["MjRhOWU4ZGQtZDI3NS1kYzczLTBlNTUtYjZmNGJjZTQ3OTRkLTMwMjQtMTMyNjA4MjQwODAuNTA2MTg3MDAw","MjRhOWU4ZGQtZDI3NS1kYzczLTBlNTUtYjZmNGJjZTQ3OTRkLTY5Ni0xMzI2MDgyNDA3OC44MjExODg5MDA=","MjRhOWU4ZGQtZDI3NS1kYzczLTBlNTUtYjZmNGJjZTQ3OTRkLTU1Mi0xMzI2MDgyNDA3OC43MjQzMDIxMDA="]},"entity_id":"MjRhOWU4ZGQtZDI3NS1kYzczLTBlNTUtYjZmNGJjZTQ3OTRkLTUwNDgtMTMyNjA4MjQwODMuNjEzMjI3NjAw","executable":"C:\\Program Files\\Fortinet\\FortiClient\\FCDBLog.exe","name":"FCDBLog.exe","pid":5048},"user":{"domain":"NT AUTHORITY","id":"S-1-5-18","name":"SYSTEM"}}) reason (failed to parse field [data_stream.namespace] of type [constant_keyword] in document with id '82WgWngBVg1FH-G0gP1X'. Preview of field's value: 'hg+av') status (400)

[jen-huang]

@michalpristas Can you chime in here from the agent side? I am surprised that + in namespace is giving a parsing error, it's not a restricted character based on the rules we implemented (same rules as ES index name restrictions). Does namespace have additional restrictions?

[michalpristas]

i checked the code and i see that + is forbidden for dataset as a prefix, otherwise allowed
but this error is coming from endpoint.

[kevinlog]

@michalpristas understood that the error is coming from Endpoint, but it seems ES is sending the error to Endpoint which it is reflecting back.

@ferullo is the above your understanding?

[michalpristas]

as for this code it should be ok https://github.com/elastic/elasticsearch/blob/9d681937d5ad2afe94291ca3f5af43c1cb01e1ec/server/src/main/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexService.java#L215

[michalpristas]

can this be a mapping issue? @ruflin could you chime in here?

[kevinlog]

good point @michalpristas

@jen-huang mentioned something similar yesterday.

I was thinking that it could be the same mapping issue as this: https://github.com/elastic/endpoint-dev/issues/8640

[ferullo]

@kevinlog that is right. That log message is Endpoint logging a response from Elasticsearch when Endpoint tried to write data.

[EricDavisX]

Hi, can I get a re-review on the priority of this? @jen-huang @ruflin @kevinlog - shall we target 7.14 for it?

[amolnater-qasource]

Hi @EricDavisX
We have revalidated this issue on 7.14.0 Snapshot Kibana Cloud environment.

Build details:

 Artifact Link: https://snapshots.elastic.co/7.14.0-bdd01eb6/downloads/beats/elastic-agent/elastic-agent-7.14.0-SNAPSHOT-windows-x86_64.zip
 Build: 41361
 Commit: 66d63f384e775fe02799bd9a64b5f6353f123f30

Steps followed:

1. Created a policy with namespace "endpoint+security" and added Endpoint Security integration to it.
2. Installed Agent with that policy.
3. Observed data for System and Endpoint Security under Data Streams tab with "endpoint+security" namespace.

Screenshots:

Data Streams:

Indicies:

Thanks
QAS

[jen-huang]

It sounds like this is working now? @kevinlog or @ferullo do you recall anything special about the original report, for example should we validate Endpoint using default for namespace initially, and then changing it to endpoint+security? Otherwise we can close this out.

[kevinlog]

Based on @EricDavisX test, looks like it's working now. I actually this this problem was related to an earlier issue with namespaces and wasn't directly related to the special characters.

I am OK to close this one out @jen-huang

thanks!

[muskangulati-qasource]

Bug Conversion

• Test-Case not required as this particular checkpoint is already covered in the following test case:
  • https://elastic.testrail.io/index.php?/cases/view/15361

Thanks!