Add parent.id to the tracing field set

[sgryphon]

This is discussed in issue #998 . There is a separate pull request to address the main clarity issues; this PR is for adding the parent.id field.

This change needs a minor version bump, e.g. 1.8 (or higher), as it adds a new field.

The trace.id correlates all messages for a logical operation, span.id correlates all the messages within one processing section (e.g. one request to a component), and parent.id shows the parent-child relationships between processing sections.

trace.id is the most important, and the others can be somewhat inferred from timing and other details, span.id the next most, and parent.id the least, but still useful.

Some of the information may be redundant, e.g. parent.id can be merged in from an Application Performance Monitoring (APM) or other tracing system, and will be repeated the same for each span.id (the parent doesn't change).

Having the information locally can, however, make processing easier. Rather than refer to another system, the parent-child hierarchy can be determined directly from the logs. e.g. the user may not even be running an APM solution, and may only have basic logs, but still want to determine the hierarchy.

It also allows the hierarchy to be determined with only a subset of messages, because the information is repeated in each.

Note that there are many other similar details in ECS, where fields are repeated (denormalised) for ease of access. For example once you know a user id, the name, email, full_name, domain, etc will all be repeated the same every time for that user. The same with machine fields, operating system fields, etc.

Ease of use, having the information directly in the record, is higher value than just having a reference (saving space) and needing to pull that data in from elsewhere, so even if parent.id is available from elsewhere (e.g. an APM system), it is still very useful to have it as part of the ECS messages.

NOTE: I have just done this as a PR, as it is only a minor change to the specification, however if the full RFC process is needed, I am also happy to do that.

[alankis]

I am literary dying to have this field. What are missing in this PR? How can this be resolved and introduced in ECS 1.8 or smaller minor version?

[sgryphon]

Updated for merge from latest master.

@alankis it needs someone with permissions to review and approve (I don't have them).

Note that ECS is just a specification, and it does allow you to add your own custom fields, so there is nothing stopping you from adding a parent.id field, as a custom field, in your logging. There is some discussion of this in the Custom Fields section of the reference guide.

If you want to be super-safe, in case the final implementation ends up different, then you could also use Parent.id, as suggested, which would avoid conflict with parent.id if/when it is added. The downside would be that you would then need to do a migration from your different custom field to the official one.

If you use parent.id, then it avoids having to do a migration, with the minor risk that the standard could end up with a different field using the name parent for something else (very unlikely).

Meanwhile, we can hope that the powers that be review this!

[ebeahan]

Thanks, @sgryphon, for opening this!

Let's focus on reaching a decision on parent.id in #998 before moving forward. I've dropped a note to hopefully revive that discussion.

[ebeahan]

Note that ECS is just a specification, and it does allow you to add your own custom fields, so there is nothing stopping you from adding a parent.id field, as a custom field, in your logging. There is some discussion of this in the Custom Fields section of the reference guide.

@alankis Yes, as @sgryphon advised, you're welcome to add custom fields. If you have any use cases or general thoughts around parent.id to share, feel free to join into the discussion in #998.

[ebeahan]

Per #998 (comment), we're not likely to add establish a parent.id field in ECS in the near-term.