Sidecar collector backend configuration name cannot contain "." #4885
Assignees
Milestone
Comments
bernd
added
bug
blocker
bernd
added a commit
that referenced
this issue
Jul 6, 2018
This fixes a problem when using a "." character in a collector backend name. It also makes it easier to run MongoDB queries against collector status objects. Fixes #4885
bernd
added a commit
to Graylog2/collector-sidecar
that referenced
this issue
Jul 6, 2018
The collector backend status data structure changed from a map to a set of objects. Refs Graylog2/graylog2-server#4885 Refs Graylog2/graylog2-server#4886
mariussturm
pushed a commit
that referenced
this issue
Jul 9, 2018
This fixes a problem when using a "." character in a collector backend name. It also makes it easier to run MongoDB queries against collector status objects. Fixes #4885
mariussturm
pushed a commit
to Graylog2/collector-sidecar
that referenced
this issue
Jul 9, 2018
The collector backend status data structure changed from a map to a set of objects. Refs Graylog2/graylog2-server#4885 Refs Graylog2/graylog2-server#4886
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Expected Behavior
Sidecar collector backend name accepts arbitrary values.
Current Behavior
I was using
packetbeat-6.3as a name for a newly configured sidecar collector backend. Creating the backend config worked fine as well as assigning the backend config to a sidecar.After assigning the configuration to a sidecar, I saw the following error in the server log:
The problem is that we are using a map for the collector backend status where the key is the backend name. Since MongoDB doesn't allow
.characters in key names, we are getting this error.Example:
In this case we only have a
filebeatbackend in thenode_details.status.collectorsstructure.{ "_id" : "...", "node_id" : "...", "node_name" : "...", "node_details" : { "operating_system" : "Linux", "ip" : "...", "metrics" : {}, "status" : { "status" : 0, "message" : "1 collectors running", "collectors" : { "filebeat" : { "status" : 0, "message" : "" } } } }, "assignments" : [], "sidecar_version" : "1.0.0", "last_seen" : "..." }Possible Solution
1. Use a set of collector objects
We could convert the
node_details.status.collectorsmap to a set of objects and add the name as a field in the collector status object. That way we don't run into problems with invalid characters in MongoDB field names.{ "node_details" : { "status" : { "collectors" : [ { "name": "filebeat", "status": 0, "message": "..." } ] } } }2. Index collector status objects by collector ID
We could also use the collector backend config ID as a key name in the map. That would also avoid problems with MongoDB field names.
{ "node_details" : { "status" : { "collectors" : { "<unique-collector-id>" : { "name": "filebeat", "status" : 0, "message" : "" } } } } }3. Don't allow certain characters in collector backend names
We could also restrict certain characters in the backend names. That means we would need to validate the user input and show an error in the UI.
That said, I would not do this because it would be inconsistent with the other "name" values in the sidecar configuration. Also, implementation details of the database shouldn't be a constraint for naming configurations in my opinion.
Steps to Reproduce (for bugs)
Context
I am in favor of the first possible solution (set of collector objects) because that also makes it easier for MongoDB queries. Using dynamic fields in MongoDB, and thus a dynamic schema, can make it hard to run some queries. For example "return all status objects where
status != 0" is hard to express as a MongoDB query with a dynamic document schema.Your Environment
The text was updated successfully, but these errors were encountered: