[RFC] Stage 0: Introduce Entity Field Set into ECS

[tinnytintin10]

Overview

An entity represents a discrete, identifiable component within an IT environment that can be described by a set of attributes and maintains its identity over time. Entities can be physical (like hosts or devices), logical (like containers or processes), or abstract (like applications or services).

Currently, ECS provides specific field sets for certain categories of entities (e.g., host, user, cloud, orchestrator) to capture their metadata. However, as IT infrastructure continues to evolve, we encounter an increasing number of entity types that don't cleanly fit into existing field sets – for example, storage services like S3, database instances like DynamoDB, or various other cloud services and IT-related infrastructure components (both digital and physical).

This RFC proposes a new entity fieldset that aims to solve this and several other challenges. Currently at Stage 0 (strawperson), seeking initial feedback on the approach and concept. See /rfcs/text/0049-entity-fields.md for more details.

• Sponsor: @MikePaquette & @YulNaumenko
• Author: @tinnytintin10

PR Guidelines

• Have you signed the contributor license agreement? ✅
• Have you followed the contributor guidelines? ✅
• For proposing substantial changes or additions to the schema, have you reviewed the [RFC process] (https://github.com/elastic/ecs/blob/main/rfcs/README.md)? ✅
• If submitting code/script changes, have you verified all tests pass locally using make test? N/A
• If submitting schema/fields updates, have you generated new artifacts by running make and committed those changes? N/A
• Is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed. ✅
• Have you added an entry to the CHANGELOG.next.md? N/A

[tinnytintin10]

@MikePaquette @YulNaumenko, I've drafted the RFC to introduce the entity field set into ECS like we talked about. Before taking it out of draft, I wanted to check with you both to see if there's anything you think should be included or addressed as part of this stage. Lmk 🙏🏾

[oren-zohar]

@tinnytintin10 would we still need it?

[tinnytintin10]

@oren-zohar Can you elaborate? Are you referring to entity.metadat (i.e, do we still need metadata when we have attribute)? Also, see the updated name and description (now entity.raw instead of metadata)

[maxcold]

do we want to solve the problem of different id/name sources for entities? eg. EC2 instance id vs arn?

[tinnytintin10]

I've updated the entity.id field description to provide guidance for choosing between multiple identifiers. To be transparent - there will always be some ambiguity in this selection that we can't fully resolve yet as it depends on specific use cases and contexts. However, we can provide these basic criteria for selecting the primary identifier:

1. Persists across the entity's lifecycle
2. Ensures uniqueness within its scope
3. Is commonly used for queries and correlation
4. Is readily available in most observations (events/logs, etc.,)

Alternative identifiers are preserved in entity.raw. Wdyt?

[maxcold]

yeah, I don't have a good way to solve it tbh. What I can only think of is another field, smth like id type which could be smth like arn or instance-id but I'm not sure it's a good idea tbh

[romulets]

The issue with ids is something beyond the scope of this RFC, in my opinion. More important than knowing what type of id is being persisted, is having a consistent id of that entity when ingested from different sources. I don't think adding a id type field would ensure consistency, it only gives a context in which I'm not sure it's needed.

We need consistent ids and entity resolution when needed. Neither of these solutions touch the RFC right now, as far as I understand.

[maxcold]

I'm guessing category and type are chosen to be consistent with event.* , but should we be consistent here? I personally always found type and category confusing, especially when it comes to understanding what is higher in the hierarchy. Wouldn't it be simpler to have category and sub-category or smth like that?

[JordanSh]

I've had some offline discussions with tin on this subject, my suggestion is type as the higher classification and subtype (or anything else) for secondary. This will better align with Observability Inventory which uses entity.type to search, group and filter:

And will also align better with our codebase which already mentions and uses entity.type quite a bit. having two different fields will require us to "juggle" between them depending on the part of the code we are working in.

[tinnytintin10]

I like the suggestion of using more explicitly hierarchical terms. Something like type/sub_type would make the relationship self-evident without needing to know any conventions.

However, I deliberately aligned with ECS's categorization hierarchy (where event.category is higher level than event.type) when coming up with this. I'd love to get input from the ECS team on whether maintaining this consistency is warranted here. While consistency with existing patterns has value, entities serve a different purpose and might benefit from more intuitive naming.

cc @MikePaquette

[opauloh]

There will be an allowed set of values maintained for this field to ensure consistency.

In that case, there will be an allowed_values property to entity.category, similarly as for the event.category? That would be nice, as it allows us to list the expected values and include a description for every category, such as the example below:

    - name: category
      allowed_values:
        - name: host
          description: >
            Entities in this category represent computing devices such as physical machines, virtual machines, or cloud instances. This includes hosts in on-premise data centers, cloud providers (AWS EC2, GCP Compute Engine, Azure VM), and edge devices.
Events in this category may relate to system health, performance metrics, security monitoring, and configuration changes.
        - name: user
          description: >
            This category represents human or service identities that interact with systems and resources. Users can be defined in directories such as Active Directory, IAM roles in cloud providers, or application-specific accounts.
Events in this category include authentication attempts, role changes, permissions updates, and identity-related security incidents.
...

[cla-checker-service]

❌ Author of the following commits did not sign a Contributor Agreement:
c20956e, 8d583c0, 347c3d3

Please, read and sign the above mentioned agreement if you want to contribute to this project

[tinnytintin10]

Reviewed this with @MikePaquette and are good to go for boarder reviews 🚀

cc @tehilashn @oren-zohar @YulNaumenko

[opauloh]

There will be an allowed set of values maintained for this field to ensure consistency.

In that case, there will be an allowed_values property to entity.category, similarly as for the event.category? That would be nice, as it allows us to list the expected values and include a description for every category, such as the example below:

    - name: category
      allowed_values:
        - name: host
          description: >
            Entities in this category represent computing devices such as physical machines, virtual machines, or cloud instances. This includes hosts in on-premise data centers, cloud providers (AWS EC2, GCP Compute Engine, Azure VM), and edge devices.
Events in this category may relate to system health, performance metrics, security monitoring, and configuration changes.
        - name: user
          description: >
            This category represents human or service identities that interact with systems and resources. Users can be defined in directories such as Active Directory, IAM roles in cloud providers, or application-specific accounts.
Events in this category include authentication attempts, role changes, permissions updates, and identity-related security incidents.
...

[opauloh]

Should criticality be included in entity as well? i.e entity.criticality

[JordanSh]

shouldn't tags and labels also be included in entity?