drivers/docker: labels containers with task info

[notnoop]

Label docker containers with task info (i.e. job/alloc/task id).

Using com.hashicorp.nomad. namespace to avoid name clashes with user
labels.

Closes #4781. This is a less ambitious scope of #4995 - we intend to add a more flexible configurable option in the future, but this seems like a nice change on its own.

[nickethier]

Can you add a small section on the driver docs with this. Perhapse a section under the Container Name section thats something like Default Labels

[jippi]

Can we get a toggle for this, please? it will incur a cost for services like DataDog (per tag billing)

Also there are likely people like me out there, who already inject these our self, and do not want them duplicated into DataDog / Influx docker monitoring

[notnoop]

@jippi Good catch! Thanks for raising this. I'd be more inclined to hold off supporting this until we have a configuration option for default labels, rather than supporting a boolean toggle to be deprecated soon; but I'd wait for rest of team input.

But I'm curious, for DataDog cases, how do folks manage tag integrations? Do you use whitelists?
Container Labels can be set in myriads of ways (e.g. operator set, inherited from image labels, set from scheduler), and managing costs by restricting labels use from all sources seems a bit odd for me; and would love to have a guiding principle for ourselves as we think of labels options.

[notnoop]

we have decided to punt on this feature until we have a configurable option for setting user-defined labels.

@jippi Looking at the DataDog case, it seems that datadog agent uses a whitelist for labels[1], so I assume adding new tags shouldn't be a problem, I assume. But Influx by default includes all labels as tags indeed, but that's configurable[2]. Does this match your understanding?

[1] https://github.com/DataDog/dd-agent/blob/73f11cd003f487a1f0fd7ded7c147dfa6ed17064/datadog.conf.example#L164-L173
[2] https://github.com/influxdata/telegraf/tree/master/plugins/inputs/docker

[jippi]

@notnoop yep, it does :)

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.