Enrich logs with metadata

[mr-karan]

Proposal

Nomad logs which are placed inside the allocs directory capture the application logs from stdout/stderr directly. There are ways to get metadata using the docker driver, by using logging.config.labels property, thanks to #9885 which was merged recently. However, I am not sure if there's a way to get this metadata for other task drivers and enrich the logs in the same way. Currently Nomad is basically outsourcing the job of enriching the logs to Docker driver's capabilities. Does it make sense for Nomad to have this feature built in?

Use-cases

Can get logs with additional context like Job ID, Name, Namespace etc for all jobs started with any task execution driver.

[tgross]

Hi @mr-karan! There's been a few high-level requests around improving the logging story (ref). It's definitely something we want to do and we want to make sure we're tackling the problem holistically. Thanks for opening this issue! I'll mark it for roadmapping.

[mr-karan]

Bump.

Would love to see logging get some more ❤️ for non docker drivers 😄

[mr-karan]

In case anyone's interested in a stop gap solution, I made a small program which I plan to use with vector https://github.com/mr-karan/nomad-vector-logger