sidecar: auto assign configuration based on meta information

[jalogisch]

When having a fleet of sidecars it would be helpful to assign configuration based on information that are available.

That could be a configuration that is assigned because the OS is Linux, or the IP is in some kind of range. Or the hostname fits into some kind of regex.

If meta information are available - like requested in Graylog2/collector-sidecar#280 - assign configuration based on that.

Other possible option would be to make use of a lookup table where the result are the configurations that should be assigned to the sidecar.

All of the above would help to gain a high level of automation and reduce the need of manually assignment of configuration to sidecars.

[J-Camping]

If a baseline config for Windows and Linux could be set for all new sidecars, that would be very useful.

I used to use the tags functionality in the old sidecar yaml file, but it seems like there isn't a way to automatically assign sidecars to a configuration with the new version.

[ion-storm]

How about per domain? Pull windows domain attributes from wmic or ipconfig to implement with this feature. Since the removal of tags in initial configurations with Graylog 3.0 its a real PITA to manage thousands of endpoints.

[lonewaffle]

Definitely support this. I'd like to install the sidecar during OS deployment, but can't find a solution that automatically assigns a basic config. The way it is now seems like I'll have to periodically go into Graylog and look for sidecars without configs and assign them which is a task that's frankly easy to forget to do.

[skuzbucket1]

Whatever can be done here would be helpful. We are moving from Splunk to Graylog and this is the only manual process that we have encountered that is slowing down the process.

[cisco1115]

+1 on this, sorely missing for us as well.

[mlahaye811]

The capability to auto-assign a configuration based on some information / tag is a missing key feature that would help to efficiently deploy on many endpoints in an automated way. It would be awesome if this feature could be prioritized soon.

[zyxep]

Technically it was possible with the old sidecar collector, it used tags.
But they decided to remove that feature in the new sidecar.

I would also love this, because I also have a fleet of linux servers.

[cmiscloni]

+1

[markusg80]

+1

[Luubosi]

+1

[ObiWanCanOweMe]

+1

[devatnull]

Guys, this is such an important feature that all your competitors includes in their platform without exception.

Why is it delayed for 3 years? What is the technical reason for this?

Can't this be done via a new function in pipeline for example using CIDR mask to interact with sidecar policy?

Why not bring back metadata sensitive sensors on graylog that auto assigns this?

What could you be prioritizing so much more important than this?

[cisco1115]

Guys, this is such an important feature that all your competitors includes in their platform without exception.

Why is it delayed for 3 years? What is the technical reason for this?

Can't this be done via a new function in pipeline for example using CIDR mask to interact with sidecar policy?

Why not bring back metadata sensitive sensors on graylog that auto assigns this?

What could you be prioritizing so much more important than this?

+100

[matt-fairway]

+1000

[CPLT-C1]

Has anyone found a work-around? Reliably deploying GrayLog to monitor endpoints seems to be not possible if default configs can't be automatically applied.

[ObiWanCanOweMe]

Is this something locked behind an enterprise license perhaps?

[H2Cyber]

Is this something locked behind an enterprise license perhaps?

No, the feature is not there even in the enterprise version.

Here is the latest feedback that I have seen on this from Graylog staff : #10454 (comment)

[l0calhost]

We created a way that works for us by programming against the API. This way, you can get a list of all sidecars, identify the ones without an assigned config and assign a default one based on the sidecars os.

Sadly, graylog left a lot of strings unattached by going from collector-sidecar to graylog-sidecar and missing out the tags feature. At least they provide APIs to connect your own log management solution with the software. I'm afraid this will be the only possible approach if you want it done anywhere short-term: DIY.

[boosty]

Hi everyone, we understand that managing a larger fleet of Sidecars using different configurations is currently quite cumbersome. The team is now working on a related improvement, which we can hopefully share soon.

[boosty]

Hi everyone,

with Graylog 5.0 we released the possibility to automatically assign Sidecar configurations, based on tags.

A list of tags can be assigned in a Sidecar's YAML configuration. Collector configuration matching any of these tags will then automatically be applied to the Sidecar.

Details:

• Sidecar support for tags and collectors with multiple configurations #13433
• https://go2docs.graylog.org/5-0/getting_in_log_data/graylog_sidecar.html?Highlight=sidecar#AssigningTags

Recently a related change was added to the Sidecar installer on Windows to accept tags at installation time, for convenience:

• Provide -TAGS= support for windows silent installer collector-sidecar#464

Please note that tags have to be set explicitly. We are not generating tags based on meta information, as originally requested here. Therefore I will keep this issue open for now.

[boosty]

We are closing this now, because we believe that the recently added support for tags should solve the main issue when managing a larger fleet of Sidecars.

The possibility of auto-generating tags can be considered in a separate issue if there is enough demand for it.