diff --git a/CHANGELOG.next.md b/CHANGELOG.next.md index be936302d8..8e9cddd41e 100644 --- a/CHANGELOG.next.md +++ b/CHANGELOG.next.md @@ -8,28 +8,40 @@ Thanks, you're awesome :-) --> ## Unreleased -### Schema Changes +### Breaking Changes -#### Breaking changes +* Removing deprecated --oss from generator #1404 +* Removing use-cases directory #1405 +* Remove `host.user.*` field reuse. #1439 + +### Schema Changes #### Bugfixes #### Added +* `elf.*` field set added as beta. #1410 +* Remove `beta` from `orchestrator` field set. #1417 +* Extend `threat.*` field set beta. #1438 + #### Improvements * Fix ecs GitHub repo link source branch #1393 +* Add --exclude flag to Generator to support field removal testing #1411 +* Explicitly include user identifiers in `relater.user` description. #1420 #### Deprecated -### Tooling and Artifact Changes +* Note deprecation of the `host.user.*` field reuse. #1422 -#### Breaking changes +### Tooling and Artifact Changes #### Bugfixes #### Added +* Support `match_only_text` data type in Go code generator. #1418 + #### Improvements #### Deprecated diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 766f1a1bc2..4af171b7c5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -61,6 +61,7 @@ You need these tools to contribute to the ECS repo: * Request feedback about your changes. - Create a [Pull Request](https://help.github.com/articles/creating-a-pull-request/) against the ECS repo. - (Look for the `Compare & pull request` button on your branch in github.com.) + - Add an entry to [CHANGELOG.next.md](CHANGELOG.next.md). - Wait for reviews on your PR. - Incorporate review comments and push updates if needed. * Thank you for your contribution! @@ -96,15 +97,26 @@ Please follow these guidelines when submitting Issues: * Click `New issue`. Provide as many details as possible to help reviewers and other contributors understand your proposal. * Add your text, and click `Submit new issue`. -### Backports +### Branching -ECS maintains multiple release branches in the repo. The `master` branch is where all new contributions should be submitted, and features and bug fixes will be backported into other branches when appropriate. Any backporting needs will be handled by the ECS team. +ECS follows this branching strategy: + +* The `master` is the next major version. It is where all new contributions are first merged. This includes new features and bug fixes, and it may also include breaking changes. +* The `.x` is the next minor version and gets backports of most non-breaking features and fixes. +* The `.` is the next release of a minor version, including patch releases. + +### Changelog + +ECS maintains two changelog files: -#### Branching +* [CHANGELOG.md](CHANGELOG.md) contains a list of notable changes for each released version of ECS. +* [CHANGELOG.next.md](CHANGELOG.next.md) contains a list of unreleased ECS changes. -* The `master` branch is where all new contributions are merged. This includes new features and bug fixes, and it may also include breaking changes. -* The `1.x` branch gets backports of most non-breaking features and fixes. This branch represents the next `major.minor` release. -* The `major.minor` branches (e.g. `1.6`, `1.5`, `1.4`, etc.) contain the latest released version of those releases. +Breaking changes intended for the next major version should be included underneath the `Breaking changes` sections in `CHANGELOG.next.md`. + +### Backports + +ECS maintains multiple release branches in the repo. The `master` branch is where all new contributions should be submitted, and features and bug fixes will be backported into other branches when appropriate. Any backporting needs will be handled by the ECS team. #### Tooling diff --git a/Makefile b/Makefile index 327f64b49f..2076cfa82b 100644 --- a/Makefile +++ b/Makefile @@ -35,8 +35,6 @@ check-license-headers: .PHONY: clean clean: rm -rf build generated/elasticsearch/component experimental/generated/elasticsearch/component - # Clean all markdown files for use-cases - find ./use-cases -type f -name '*.md' -not -name 'README.md' -print0 | xargs -0 rm -- # Alias to generate source code for all languages. .PHONY: codegen @@ -64,7 +62,7 @@ fmt: ve # Alias to generate everything. .PHONY: generate -generate: generator legacy_use_cases codegen +generate: generator codegen $(PYTHON) --version # Run the new generator @@ -82,11 +80,6 @@ gocodegen: -schema=../schemas \ -out=../code/go/ecs -# Generate the Use Cases -.PHONY: legacy_use_cases -legacy_use_cases: ve - $(PYTHON) scripts/use-cases.py --stdout=true >> /dev/null - # Check Makefile format. .PHONY: makelint makelint: SHELL:=/bin/bash diff --git a/USAGE.md b/USAGE.md index 3c84f113dc..5ba60b306f 100644 --- a/USAGE.md +++ b/USAGE.md @@ -27,10 +27,10 @@ relevant artifacts for their unique set of data sources. * [Generator Options](#generator-options) + [Out](#out) + [Include](#include) + + [Exclude](#exclude) + [Subset](#subset) + [Ref](#ref) + [Mapping & Template Settings](#mapping--template-settings) - + [OSS (Deprecated)](#oss-deprecated) + [Strict Mode](#strict-mode) + [Intermediate-Only](#intermediate-only) @@ -232,6 +232,41 @@ Include can be used together with the `--ref` flag to merge custom fields into a > NOTE: The `--include` mechanism will not validate custom YAML files prior to merging. This allows for modifying existing ECS fields in a custom schema without having to redefine all the mandatory field attributes. +#### Exclude + +Use the `--exclude` flag to generate ephemeral ECS artifacts based on the current ECS schema field definitions minus fields considered for removal, e.g. to assess impact of removing these. Warning! This is not the recommended route to remove a field permanently as it is not intentended to be invoked during the build process. Definitive field removal should be implemented using a custom [Subset](#subset) or via the [RFC process](https://github.com/elastic/ecs/tree/master/rfcs/README.md). Example: + +``` +$ python scripts/generator.py --exclude=../my-project/my-exclude-file.yml +$ python scripts/generator.py --exclude="../my-project/schemas/a*.yml" +``` + +The `--exclude` flag expects a path to one or more YAML files using the same [file format](https://github.com/elastic/ecs/tree/master/schemas#fields-supported-in-schemasyml) as the ECS schema files. You can also use a subset, provided that relevant `name` and `fields` fields are preserved. + +``` +--- +- name: log + fields: + - name: original +``` + +The root Field Set `name` must always be present and specified with no dots `.`. Subfields may be specified using dot notation, for example: + +``` +--- +- name: log + fields: + - name: syslog.severity.name +``` + +Generate artifacts using `--exclude` to load our custom definitions in addition to `--out` to place them in the desired output directory: + +``` +$ python scripts/generator.py --exclude ../myproject/exclude-set.yml/ --out ../myproject/out/ +Loading schemas from local files +Running generator. ECS version 1.11.0 +``` + #### Subset If your indices will never populate particular ECS fields, there's no need to include those field definitions in your index mappings. The `--subset` argument allows for passing a subset definition YAML file which indicates which field sets or specific fields to include in the generated artifacts. @@ -352,30 +387,6 @@ The `--template-settings` argument defines [index level settings](https://www.el For `template.json`, the `mappings` object is left empty: `{}`. Likewise the `properties` object remains empty in the `mapping.json` example. This will be filled in automatically by the script. -#### OSS (Deprecated) - -**Important**: This feature has been deprecated. An Apache 2.0 distribution of Elasticsearch will not be -produced starting with the 7.11 release. Learn more about these changes [here](https://www.elastic.co/pricing/faq/licensing). - -Users that want to use an Apache 2.0 distribution of Elasticsearch do not have access to the basic data types. -However some of these types have an OSS replacement that can be used instead, without too much loss of functionality. - -This flag performs a best effort fallback, replacing basic data types with their OSS replacement. - -Indices using purely OSS types will benefit from the normalization of ECS, but may be missing on some of the added functionality of these basic types. - -Current fallbacks applied by this flag are: - -- `constant_keyword` => `keyword` -- `wildcard` => `keyword` -- `version` => `keyword` - -Usage: - -``` -$ python scripts/generator.py --oss -``` - #### Strict Mode The `--strict` argument enables "strict mode". Strict mode performs a stricter validation step against the schema's contents. diff --git a/code/go/ecs/elf.go b/code/go/ecs/elf.go new file mode 100644 index 0000000000..ee17c2870f --- /dev/null +++ b/code/go/ecs/elf.go @@ -0,0 +1,123 @@ +// Licensed to Elasticsearch B.V. under one or more contributor +// license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright +// ownership. Elasticsearch B.V. licenses this file to you under +// the Apache License, Version 2.0 (the "License"); you may +// not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +// Code generated by scripts/gocodegen.go - DO NOT EDIT. + +package ecs + +import ( + "time" +) + +// These fields contain Linux Executable Linkable Format (ELF) metadata. +type Elf struct { + // Extracted when possible from the file's metadata. Indicates when it was + // built or compiled. It can also be faked by malware creators. + CreationDate time.Time `ecs:"creation_date"` + + // Machine architecture of the ELF file. + Architecture string `ecs:"architecture"` + + // Byte sequence of ELF file. + ByteOrder string `ecs:"byte_order"` + + // CPU type of the ELF file. + CpuType string `ecs:"cpu_type"` + + // Header class of the ELF file. + HeaderClass string `ecs:"header.class"` + + // Data table of the ELF header. + HeaderData string `ecs:"header.data"` + + // Application Binary Interface (ABI) of the Linux OS. + HeaderOsAbi string `ecs:"header.os_abi"` + + // Header type of the ELF file. + HeaderType string `ecs:"header.type"` + + // Version of the ELF header. + HeaderVersion string `ecs:"header.version"` + + // Version of the ELF Application Binary Interface (ABI). + HeaderAbiVersion string `ecs:"header.abi_version"` + + // Header entrypoint of the ELF file. + HeaderEntrypoint int64 `ecs:"header.entrypoint"` + + // "0x1" for original ELF files. + HeaderObjectVersion string `ecs:"header.object_version"` + + // An array containing an object for each section of the ELF file. + // The keys that should be present in these objects are defined by + // sub-fields underneath `elf.sections.*`. + Sections []Sections `ecs:"sections"` + + // List of exported element names and types. + Exports map[string]interface{} `ecs:"exports"` + + // List of imported element names and types. + Imports map[string]interface{} `ecs:"imports"` + + // List of shared libraries used by this ELF object. + SharedLibraries string `ecs:"shared_libraries"` + + // telfhash symbol hash for ELF file. + Telfhash string `ecs:"telfhash"` + + // An array containing an object for each segment of the ELF file. + // The keys that should be present in these objects are defined by + // sub-fields underneath `elf.segments.*`. + Segments []Segments `ecs:"segments"` +} + +type Sections struct { + // ELF Section List flags. + Flags string `ecs:"flags"` + + // ELF Section List name. + Name string `ecs:"name"` + + // ELF Section List offset. + PhysicalOffset string `ecs:"physical_offset"` + + // ELF Section List type. + Type string `ecs:"type"` + + // ELF Section List physical size. + PhysicalSize int64 `ecs:"physical_size"` + + // ELF Section List virtual address. + VirtualAddress int64 `ecs:"virtual_address"` + + // ELF Section List virtual size. + VirtualSize int64 `ecs:"virtual_size"` + + // Shannon entropy calculation from the section. + Entropy int64 `ecs:"entropy"` + + // Chi-square probability distribution of the section. + Chi2 int64 `ecs:"chi2"` +} + +type Segments struct { + // ELF object segment type. + Type string `ecs:"type"` + + // ELF object segment sections. + Sections string `ecs:"sections"` +} diff --git a/code/go/ecs/related.go b/code/go/ecs/related.go index 22acb9fee2..b32da1647c 100644 --- a/code/go/ecs/related.go +++ b/code/go/ecs/related.go @@ -31,7 +31,7 @@ type Related struct { // All of the IPs seen on your event. IP string `ecs:"ip"` - // All the user names seen on your event. + // All the user names or other user identifiers seen on the event. User string `ecs:"user"` // All the hashes seen on your event. Populating this field, then using it diff --git a/code/go/ecs/threat.go b/code/go/ecs/threat.go index 0df5e08049..ae3ef6f0b4 100644 --- a/code/go/ecs/threat.go +++ b/code/go/ecs/threat.go @@ -34,6 +34,65 @@ type Threat struct { // retrospectively tagged to events. Framework string `ecs:"framework"` + // The alias(es) of the group for a set of related intrusion activity that + // are tracked by a common name in the security community. While not + // required, you can use a MITRE ATT&CK® group alias(es). + GroupAlias string `ecs:"group.alias"` + + // The id of the group for a set of related intrusion activity that are + // tracked by a common name in the security community. While not required, + // you can use a MITRE ATT&CK® group id. + GroupID string `ecs:"group.id"` + + // The name of the group for a set of related intrusion activity that are + // tracked by a common name in the security community. While not required, + // you can use a MITRE ATT&CK® group name. + GroupName string `ecs:"group.name"` + + // The reference URL of the group for a set of related intrusion activity + // that are tracked by a common name in the security community. While not + // required, you can use a MITRE ATT&CK® group reference URL. + GroupReference string `ecs:"group.reference"` + + // The id of the software used by this threat to conduct behavior commonly + // modeled using MITRE ATT&CK®. While not required, you can use a MITRE + // ATT&CK® software id. + SoftwareID string `ecs:"software.id"` + + // The name of the software used by this threat to conduct behavior + // commonly modeled using MITRE ATT&CK®. While not required, you can use a + // MITRE ATT&CK® software name. + SoftwareName string `ecs:"software.name"` + + // The platforms of the software used by this threat to conduct behavior + // commonly modeled using MITRE ATT&CK®. While not required, you can use a + // MITRE ATT&CK® software platforms. + // Recommended Values: + // * AWS + // * Azure + // * Azure AD + // * GCP + // * Linux + // * macOS + // * Network + // * Office 365 + // * SaaS + // * Windows + SoftwarePlatforms string `ecs:"software.platforms"` + + // The reference URL of the software used by this threat to conduct + // behavior commonly modeled using MITRE ATT&CK®. While not required, you + // can use a MITRE ATT&CK® software reference URL. + SoftwareReference string `ecs:"software.reference"` + + // The type of software used by this threat to conduct behavior commonly + // modeled using MITRE ATT&CK®. While not required, you can use a MITRE + // ATT&CK® software type. + // Recommended values + // * Malware + // * Tool + SoftwareType string `ecs:"software.type"` + // The id of tactic used by this threat. You can use a MITRE ATT&CK® // tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/ ) TacticID string `ecs:"tactic.id"` diff --git a/docs/field-details.asciidoc b/docs/field-details.asciidoc index f6d20c5893..11a568fd67 100644 --- a/docs/field-details.asciidoc +++ b/docs/field-details.asciidoc @@ -1849,6 +1849,517 @@ example: `1.0.0` |===== +[[ecs-elf]] +=== ELF Header Fields + +These fields contain Linux Executable Linkable Format (ELF) metadata. + +beta::[ These fields are in beta and are subject to change.] + +[discrete] +==== ELF Header Field Details + +[options="header"] +|===== +| Field | Description | Level + +// =============================================================== + +| +[[field-elf-architecture]] +<> + +| Machine architecture of the ELF file. + +type: keyword + + + +example: `x86-64` + +| extended + +// =============================================================== + +| +[[field-elf-byte-order]] +<> + +| Byte sequence of ELF file. + +type: keyword + + + +example: `Little Endian` + +| extended + +// =============================================================== + +| +[[field-elf-cpu-type]] +<> + +| CPU type of the ELF file. + +type: keyword + + + +example: `Intel` + +| extended + +// =============================================================== + +| +[[field-elf-creation-date]] +<> + +| Extracted when possible from the file's metadata. Indicates when it was built or compiled. It can also be faked by malware creators. + +type: date + + + + + +| extended + +// =============================================================== + +| +[[field-elf-exports]] +<> + +| List of exported element names and types. + +type: flattened + + +Note: this field should contain an array of values. + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-abi-version]] +<> + +| Version of the ELF Application Binary Interface (ABI). + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-class]] +<> + +| Header class of the ELF file. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-data]] +<> + +| Data table of the ELF header. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-entrypoint]] +<> + +| Header entrypoint of the ELF file. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-object-version]] +<> + +| "0x1" for original ELF files. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-os-abi]] +<> + +| Application Binary Interface (ABI) of the Linux OS. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-type]] +<> + +| Header type of the ELF file. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-header-version]] +<> + +| Version of the ELF header. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-imports]] +<> + +| List of imported element names and types. + +type: flattened + + +Note: this field should contain an array of values. + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections]] +<> + +| An array containing an object for each section of the ELF file. + +The keys that should be present in these objects are defined by sub-fields underneath `elf.sections.*`. + +type: nested + + +Note: this field should contain an array of values. + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-chi2]] +<> + +| Chi-square probability distribution of the section. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-entropy]] +<> + +| Shannon entropy calculation from the section. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-flags]] +<> + +| ELF Section List flags. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-name]] +<> + +| ELF Section List name. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-physical-offset]] +<> + +| ELF Section List offset. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-physical-size]] +<> + +| ELF Section List physical size. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-type]] +<> + +| ELF Section List type. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-virtual-address]] +<> + +| ELF Section List virtual address. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-sections-virtual-size]] +<> + +| ELF Section List virtual size. + +type: long + + + + + +| extended + +// =============================================================== + +| +[[field-elf-segments]] +<> + +| An array containing an object for each segment of the ELF file. + +The keys that should be present in these objects are defined by sub-fields underneath `elf.segments.*`. + +type: nested + + +Note: this field should contain an array of values. + + + + + +| extended + +// =============================================================== + +| +[[field-elf-segments-sections]] +<> + +| ELF object segment sections. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-segments-type]] +<> + +| ELF object segment type. + +type: keyword + + + + + +| extended + +// =============================================================== + +| +[[field-elf-shared-libraries]] +<> + +| List of shared libraries used by this ELF object. + +type: keyword + + +Note: this field should contain an array of values. + + + + + +| extended + +// =============================================================== + +| +[[field-elf-telfhash]] +<> + +| telfhash symbol hash for ELF file. + +type: keyword + + + + + +| extended + +// =============================================================== + +|===== + +[discrete] +==== Field Reuse + +The `elf` fields are expected to be nested at: `file.elf`, `process.elf`. + +Note also that the `elf` fields are not expected to be used directly at the root of the events. + + + + [[ecs-error]] === Error Fields @@ -2874,6 +3385,14 @@ example: `1001` // =============================================================== +| <> +| `file.elf.*`| beta:[ This field reuse is beta and subject to change.] + +These fields contain Linux Executable Linkable Format (ELF) metadata. + +// =============================================================== + + | <> | `file.hash.*` | Hashes, usually file hashes. @@ -3622,13 +4141,6 @@ example: `1325` // =============================================================== -| <> -| `host.user.*` -| Fields to describe the user relevant to the event. - -// =============================================================== - - |===== [[ecs-http]] @@ -4788,8 +5300,6 @@ type: keyword Fields that describe the resources which container orchestrators manage or act upon. -beta::[ These fields are in beta and are subject to change.] - [discrete] ==== Orchestrator Field Details @@ -5876,6 +6386,14 @@ Note also that the `process` fields may be used directly at the root of the even // =============================================================== +| <> +| `process.elf.*`| beta:[ This field reuse is beta and subject to change.] + +These fields contain Linux Executable Linkable Format (ELF) metadata. + +// =============================================================== + + | <> | `process.hash.*` | Hashes, usually file hashes. @@ -6113,7 +6631,7 @@ Note: this field should contain an array of values. [[field-related-user]] <> -| All the user names seen on your event. +| All the user names or other user identifiers seen on the event. type: keyword @@ -7029,6 +7547,202 @@ example: `MITRE ATT&CK` // =============================================================== +| +[[field-threat-group-alias]] +<> + +| beta:[ This field is beta and subject to change. ] + +The alias(es) of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group alias(es). + +type: keyword + + +Note: this field should contain an array of values. + + + +example: `[ "Magecart Group 6" ]` + +| extended + +// =============================================================== + +| +[[field-threat-group-id]] +<> + +| beta:[ This field is beta and subject to change. ] + +The id of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group id. + +type: keyword + + + +example: `G0037` + +| extended + +// =============================================================== + +| +[[field-threat-group-name]] +<> + +| beta:[ This field is beta and subject to change. ] + +The name of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group name. + +type: keyword + + + +example: `FIN6` + +| extended + +// =============================================================== + +| +[[field-threat-group-reference]] +<> + +| beta:[ This field is beta and subject to change. ] + +The reference URL of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group reference URL. + +type: keyword + + + +example: `https://attack.mitre.org/groups/G0037/` + +| extended + +// =============================================================== + +| +[[field-threat-software-id]] +<> + +| beta:[ This field is beta and subject to change. ] + +The id of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software id. + +type: keyword + + + +example: `S0552` + +| extended + +// =============================================================== + +| +[[field-threat-software-name]] +<> + +| beta:[ This field is beta and subject to change. ] + +The name of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software name. + +type: keyword + + + +example: `AdFind` + +| extended + +// =============================================================== + +| +[[field-threat-software-platforms]] +<> + +| beta:[ This field is beta and subject to change. ] + +The platforms of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software platforms. + +Recommended Values: + + * AWS + + * Azure + + * Azure AD + + * GCP + + * Linux + + * macOS + + * Network + + * Office 365 + + * SaaS + + * Windows + +type: keyword + + +Note: this field should contain an array of values. + + + +example: `[ "Windows" ]` + +| extended + +// =============================================================== + +| +[[field-threat-software-reference]] +<> + +| beta:[ This field is beta and subject to change. ] + +The reference URL of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software reference URL. + +type: keyword + + + +example: `https://attack.mitre.org/software/S0552/` + +| extended + +// =============================================================== + +| +[[field-threat-software-type]] +<> + +| beta:[ This field is beta and subject to change. ] + +The type of software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software type. + +Recommended values + + * Malware + + * Tool + +type: keyword + + + +example: `Tool` + +| extended + +// =============================================================== + | [[field-threat-tactic-id]] <> @@ -8240,7 +8954,7 @@ example: `["kibana_admin", "reporting_user"]` [discrete] ==== Field Reuse -The `user` fields are expected to be nested at: `client.user`, `destination.user`, `host.user`, `server.user`, `source.user`, `user.changes`, `user.effective`, `user.target`. +The `user` fields are expected to be nested at: `client.user`, `destination.user`, `server.user`, `source.user`, `user.changes`, `user.effective`, `user.target`. Note also that the `user` fields may be used directly at the root of the events. diff --git a/docs/fields.asciidoc b/docs/fields.asciidoc index a9adf96872..5da8f2d9d1 100644 --- a/docs/fields.asciidoc +++ b/docs/fields.asciidoc @@ -42,6 +42,8 @@ all fields are defined. | <> | Meta-information specific to ECS. +| <> | These fields contain Linux Executable Linkable Format (ELF) metadata. + | <> | Fields about errors of any kind. | <> | Fields breaking down the event details. diff --git a/docs/usage/user.asciidoc b/docs/usage/user.asciidoc index 64fa215a04..6951723edb 100644 --- a/docs/usage/user.asciidoc +++ b/docs/usage/user.asciidoc @@ -18,8 +18,6 @@ Here are the subjects covered in this page. * <> -* <> - [discrete] [[ecs-user-usage-categorization]] ===== Categorization @@ -113,7 +111,6 @@ Here's the full list of places where the user fields can appear: * `destination.user.*` * `client.user.*` * `server.user.*` -* `host.user.*` (<>) Let's go over the meaning of each. @@ -421,10 +418,3 @@ it's the creation / rename of the user, or events where this user was active in For examples of mapping events from various sources, you can look at https://github.com/elastic/ecs/blob/master/rfcs/text/0007-multiple-users.md#source-data[RFC 0007 in section Source Data]. - -[discrete] -[[ecs-user-usage-deprecations]] -===== Deprecations - -As of ECS 1.8, `host.user.*` fields are deprecated and will be removed at the next -major version of ECS. diff --git a/docs/use-cases.asciidoc.disabled b/docs/use-cases.asciidoc.disabled deleted file mode 100644 index 687f8dea53..0000000000 --- a/docs/use-cases.asciidoc.disabled +++ /dev/null @@ -1,22 +0,0 @@ -[[ecs-use-cases]] -== Use Cases - -The power and versatility of {ecs} is best illustrated through use cases. - -NOTE: Some use cases contain both ECS fields and additional fields which are not -in ECS to describe the full use case. Non-ECS fields are in italic. - - * https://github.com/elastic/ecs/blob/master/use-cases/apm.md[APM] - * https://github.com/elastic/ecs/blob/master/use-cases/auditbeat.md[Auditbeat] - * https://github.com/elastic/ecs/blob/master/use-cases/beats.md[Beats] - * https://github.com/elastic/ecs/blob/master/use-cases/filebeat-apache-access.md[Filebeat Apache] - * https://github.com/elastic/ecs/blob/master/use-cases/kubernetes.md[Kubernetes] - * https://github.com/elastic/ecs/blob/master/use-cases/logging.md[Logging] - * https://github.com/elastic/ecs/blob/master/use-cases/metricbeat.md[Metricbeat] - * https://github.com/elastic/ecs/blob/master/use-cases/tls.md[TLS] - * https://github.com/elastic/ecs/blob/master/use-cases/web-logs.md[Parsing web server logs] - -We welcome https://github.com/elastic/ecs/blob/master/CONTRIBUTING.md[contributions] of additional ECS uses cases. - - - diff --git a/docs/using-mapping-network-events.asciidoc b/docs/using-mapping-network-events.asciidoc index 90986f6a2f..d98e9197c2 100644 --- a/docs/using-mapping-network-events.asciidoc +++ b/docs/using-mapping-network-events.asciidoc @@ -79,7 +79,7 @@ Looking back at the original event, it shows the source device is the DNS client ---- "client": { "ip": "192.168.86.222", - "port": 64734 + "port": 54162 } ---- diff --git a/experimental/generated/beats/fields.ecs.yml b/experimental/generated/beats/fields.ecs.yml index f2c68b783f..cfa8add373 100644 --- a/experimental/generated/beats/fields.ecs.yml +++ b/experimental/generated/beats/fields.ecs.yml @@ -1538,7 +1538,7 @@ type: keyword ignore_above: 1024 description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian default_field: false - name: cpu_type level: extended @@ -1614,7 +1614,10 @@ - name: sections level: extended type: nested - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' default_field: false - name: sections.chi2 level: extended @@ -1673,7 +1676,10 @@ - name: segments level: extended type: nested - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' default_field: false - name: segments.sections level: extended @@ -1691,14 +1697,13 @@ level: extended type: keyword ignore_above: 1024 - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. default_field: false - name: telfhash level: extended type: keyword ignore_above: 1024 - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. default_field: false - name: error title: Error @@ -2169,7 +2174,7 @@ type: keyword ignore_above: 1024 description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian default_field: false - name: elf.cpu_type level: extended @@ -2245,7 +2250,10 @@ - name: elf.sections level: extended type: nested - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' default_field: false - name: elf.sections.chi2 level: extended @@ -2304,7 +2312,10 @@ - name: elf.segments level: extended type: nested - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' default_field: false - name: elf.segments.sections level: extended @@ -2322,14 +2333,13 @@ level: extended type: keyword ignore_above: 1024 - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. default_field: false - name: elf.telfhash level: extended type: keyword ignore_above: 1024 - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. default_field: false - name: extension level: extended @@ -3299,75 +3309,6 @@ type: long description: Seconds the host has been up. example: 1325 - - name: user.domain - level: extended - type: keyword - ignore_above: 1024 - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - - name: user.email - level: extended - type: wildcard - description: User email address. - - name: user.full_name - level: extended - type: wildcard - multi_fields: - - name: text - type: text - norms: false - default_field: false - description: User's full name, if available. - example: Albert Einstein - - name: user.group.domain - level: extended - type: keyword - ignore_above: 1024 - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - - name: user.group.id - level: extended - type: keyword - ignore_above: 1024 - description: Unique identifier for the group on the system/platform. - - name: user.group.name - level: extended - type: keyword - ignore_above: 1024 - description: Name of the group. - - name: user.hash - level: extended - type: keyword - ignore_above: 1024 - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - - name: user.id - level: core - type: keyword - ignore_above: 1024 - description: Unique identifier of the user. - - name: user.name - level: core - type: wildcard - multi_fields: - - name: text - type: text - norms: false - default_field: false - description: Short name or login of the user. - example: albert - - name: user.roles - level: extended - type: keyword - ignore_above: 1024 - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - default_field: false - name: http title: HTTP group: 2 @@ -4759,7 +4700,7 @@ type: keyword ignore_above: 1024 description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian default_field: false - name: elf.cpu_type level: extended @@ -4835,7 +4776,10 @@ - name: elf.sections level: extended type: nested - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' default_field: false - name: elf.sections.chi2 level: extended @@ -4894,7 +4838,10 @@ - name: elf.segments level: extended type: nested - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' default_field: false - name: elf.segments.sections level: extended @@ -4912,14 +4859,13 @@ level: extended type: keyword ignore_above: 1024 - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. default_field: false - name: elf.telfhash level: extended type: keyword ignore_above: 1024 - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. default_field: false - name: entity_id level: extended @@ -5100,7 +5046,7 @@ type: keyword ignore_above: 1024 description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian default_field: false - name: parent.elf.cpu_type level: extended @@ -5176,7 +5122,10 @@ - name: parent.elf.sections level: extended type: nested - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' default_field: false - name: parent.elf.sections.chi2 level: extended @@ -5235,7 +5184,10 @@ - name: parent.elf.segments level: extended type: nested - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' default_field: false - name: parent.elf.segments.sections level: extended @@ -5253,14 +5205,13 @@ level: extended type: keyword ignore_above: 1024 - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. default_field: false - name: parent.elf.telfhash level: extended type: keyword ignore_above: 1024 - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. default_field: false - name: parent.entity_id level: extended @@ -6091,7 +6042,7 @@ level: extended type: keyword ignore_above: 1024 - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. default_field: false - name: rule title: Rule @@ -7055,6 +7006,185 @@ The value should be uppercase, and not include the colon.' example: C default_field: false + - name: indicator.file.elf.architecture + level: extended + type: keyword + ignore_above: 1024 + description: Machine architecture of the ELF file. + example: x86-64 + default_field: false + - name: indicator.file.elf.byte_order + level: extended + type: keyword + ignore_above: 1024 + description: Byte sequence of ELF file. + example: Little Endian + default_field: false + - name: indicator.file.elf.cpu_type + level: extended + type: keyword + ignore_above: 1024 + description: CPU type of the ELF file. + example: Intel + default_field: false + - name: indicator.file.elf.creation_date + level: extended + type: date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + default_field: false + - name: indicator.file.elf.exports + level: extended + type: flattened + description: List of exported element names and types. + default_field: false + - name: indicator.file.elf.header.abi_version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF Application Binary Interface (ABI). + default_field: false + - name: indicator.file.elf.header.class + level: extended + type: keyword + ignore_above: 1024 + description: Header class of the ELF file. + default_field: false + - name: indicator.file.elf.header.data + level: extended + type: keyword + ignore_above: 1024 + description: Data table of the ELF header. + default_field: false + - name: indicator.file.elf.header.entrypoint + level: extended + type: long + format: string + description: Header entrypoint of the ELF file. + default_field: false + - name: indicator.file.elf.header.object_version + level: extended + type: keyword + ignore_above: 1024 + description: '"0x1" for original ELF files.' + default_field: false + - name: indicator.file.elf.header.os_abi + level: extended + type: keyword + ignore_above: 1024 + description: Application Binary Interface (ABI) of the Linux OS. + default_field: false + - name: indicator.file.elf.header.type + level: extended + type: keyword + ignore_above: 1024 + description: Header type of the ELF file. + default_field: false + - name: indicator.file.elf.header.version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF header. + default_field: false + - name: indicator.file.elf.imports + level: extended + type: flattened + description: List of imported element names and types. + default_field: false + - name: indicator.file.elf.sections + level: extended + type: nested + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + default_field: false + - name: indicator.file.elf.sections.chi2 + level: extended + type: long + format: number + description: Chi-square probability distribution of the section. + default_field: false + - name: indicator.file.elf.sections.entropy + level: extended + type: long + format: number + description: Shannon entropy calculation from the section. + default_field: false + - name: indicator.file.elf.sections.flags + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List flags. + default_field: false + - name: indicator.file.elf.sections.name + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List name. + default_field: false + - name: indicator.file.elf.sections.physical_offset + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List offset. + default_field: false + - name: indicator.file.elf.sections.physical_size + level: extended + type: long + format: bytes + description: ELF Section List physical size. + default_field: false + - name: indicator.file.elf.sections.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List type. + default_field: false + - name: indicator.file.elf.sections.virtual_address + level: extended + type: long + format: string + description: ELF Section List virtual address. + default_field: false + - name: indicator.file.elf.sections.virtual_size + level: extended + type: long + format: string + description: ELF Section List virtual size. + default_field: false + - name: indicator.file.elf.segments + level: extended + type: nested + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + default_field: false + - name: indicator.file.elf.segments.sections + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment sections. + default_field: false + - name: indicator.file.elf.segments.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment type. + default_field: false + - name: indicator.file.elf.shared_libraries + level: extended + type: keyword + ignore_above: 1024 + description: List of shared libraries used by this ELF object. + default_field: false + - name: indicator.file.elf.telfhash + level: extended + type: keyword + ignore_above: 1024 + description: telfhash symbol hash for ELF file. + default_field: false - name: indicator.file.extension level: extended type: keyword @@ -7716,12 +7846,12 @@ level: extended type: keyword ignore_above: 1024 - description: "The platform of the software used by this threat to conduct behavior\ + description: "The platforms of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ - \ a MITRE ATT&CK\xAE software platform.\nExpected values\n * AWS\n * Azure\n\ - \ * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office 365\n\ - \ * PRE\n * SaaS\n * Windows" - example: Windows + \ a MITRE ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n \ + \ * Azure\n * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office\ + \ 365\n * SaaS\n * Windows" + example: '[ "Windows" ]' default_field: false - name: software.reference level: extended @@ -7738,7 +7868,7 @@ ignore_above: 1024 description: "The type of software used by this threat to conduct behavior commonly\ \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ - \ ATT&CK\xAE software type.\nExpected values\n * Malware\n * Tool" + \ ATT&CK\xAE software type.\nRecommended values\n * Malware\n * Tool" example: Tool default_field: false - name: tactic.id diff --git a/experimental/generated/csv/fields.csv b/experimental/generated/csv/fields.csv index e606d2dc41..3f094eda81 100644 --- a/experimental/generated/csv/fields.csv +++ b/experimental/generated/csv/fields.csv @@ -222,10 +222,10 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,file,file.directory,wildcard,extended,,/home/alice,Directory where the file is located. 2.0.0-dev+exp,true,file,file.drive_letter,keyword,extended,,C,Drive letter where the file is located. 2.0.0-dev+exp,true,file,file.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. -2.0.0-dev+exp,true,file,file.elf.byte_order,keyword,extended,,"Little Endian, Big Endian",Byte sequence of ELF file. +2.0.0-dev+exp,true,file,file.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. 2.0.0-dev+exp,true,file,file.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. 2.0.0-dev+exp,true,file,file.elf.creation_date,date,extended,,,Build or compile date. -2.0.0-dev+exp,true,file,file.elf.exports,flattened,extended,,,List of exported element names and types. +2.0.0-dev+exp,true,file,file.elf.exports,flattened,extended,array,,List of exported element names and types. 2.0.0-dev+exp,true,file,file.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). 2.0.0-dev+exp,true,file,file.elf.header.class,keyword,extended,,,Header class of the ELF file. 2.0.0-dev+exp,true,file,file.elf.header.data,keyword,extended,,,Data table of the ELF header. @@ -234,8 +234,8 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,file,file.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. 2.0.0-dev+exp,true,file,file.elf.header.type,keyword,extended,,,Header type of the ELF file. 2.0.0-dev+exp,true,file,file.elf.header.version,keyword,extended,,,Version of the ELF header. -2.0.0-dev+exp,true,file,file.elf.imports,flattened,extended,,,List of imported element names and types. -2.0.0-dev+exp,true,file,file.elf.sections,nested,extended,,,Section information of the ELF file. +2.0.0-dev+exp,true,file,file.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev+exp,true,file,file.elf.sections,nested,extended,array,,Section information of the ELF file. 2.0.0-dev+exp,true,file,file.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. 2.0.0-dev+exp,true,file,file.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. 2.0.0-dev+exp,true,file,file.elf.sections.flags,keyword,extended,,,ELF Section List flags. @@ -245,11 +245,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,file,file.elf.sections.type,keyword,extended,,,ELF Section List type. 2.0.0-dev+exp,true,file,file.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. 2.0.0-dev+exp,true,file,file.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. -2.0.0-dev+exp,true,file,file.elf.segments,nested,extended,,,ELF object segment list. +2.0.0-dev+exp,true,file,file.elf.segments,nested,extended,array,,ELF object segment list. 2.0.0-dev+exp,true,file,file.elf.segments.sections,keyword,extended,,,ELF object segment sections. 2.0.0-dev+exp,true,file,file.elf.segments.type,keyword,extended,,,ELF object segment type. -2.0.0-dev+exp,true,file,file.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object -2.0.0-dev+exp,true,file,file.elf.telfhash,keyword,extended,,,telfhash hash for ELF files +2.0.0-dev+exp,true,file,file.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev+exp,true,file,file.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev+exp,true,file,file.extension,keyword,extended,,png,"File extension, excluding the leading dot." 2.0.0-dev+exp,true,file,file.gid,keyword,extended,,1001,Primary group ID (GID) of the file. 2.0.0-dev+exp,true,file,file.group,keyword,extended,,alice,Primary group name of the file. @@ -372,18 +372,6 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,host,host.os.version,keyword,extended,,10.14.1,Operating system version as a raw string. 2.0.0-dev+exp,true,host,host.type,keyword,core,,,Type of host. 2.0.0-dev+exp,true,host,host.uptime,long,extended,,1325,Seconds the host has been up. -2.0.0-dev+exp,true,host,host.user.domain,keyword,extended,,,Name of the directory the user is a member of. -2.0.0-dev+exp,true,host,host.user.email,wildcard,extended,,,User email address. -2.0.0-dev+exp,true,host,host.user.full_name,wildcard,extended,,Albert Einstein,"User's full name, if available." -2.0.0-dev+exp,true,host,host.user.full_name.text,text,extended,,Albert Einstein,"User's full name, if available." -2.0.0-dev+exp,true,host,host.user.group.domain,keyword,extended,,,Name of the directory the group is a member of. -2.0.0-dev+exp,true,host,host.user.group.id,keyword,extended,,,Unique identifier for the group on the system/platform. -2.0.0-dev+exp,true,host,host.user.group.name,keyword,extended,,,Name of the group. -2.0.0-dev+exp,true,host,host.user.hash,keyword,extended,,,Unique user hash to correlate information for a user in anonymized form. -2.0.0-dev+exp,true,host,host.user.id,keyword,core,,,Unique identifier of the user. -2.0.0-dev+exp,true,host,host.user.name,wildcard,core,,albert,Short name or login of the user. -2.0.0-dev+exp,true,host,host.user.name.text,text,core,,albert,Short name or login of the user. -2.0.0-dev+exp,true,host,host.user.roles,keyword,extended,array,"[""kibana_admin"", ""reporting_user""]",Array of user roles at the time of the event. 2.0.0-dev+exp,true,http,http.request.body.bytes,long,extended,,887,Size in bytes of the request body. 2.0.0-dev+exp,true,http,http.request.body.content,wildcard,extended,,Hello world,The full HTTP request body. 2.0.0-dev+exp,true,http,http.request.body.content.text,text,extended,,Hello world,The full HTTP request body. @@ -508,10 +496,10 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.command_line,wildcard,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev+exp,true,process,process.command_line.text,text,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev+exp,true,process,process.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. -2.0.0-dev+exp,true,process,process.elf.byte_order,keyword,extended,,"Little Endian, Big Endian",Byte sequence of ELF file. +2.0.0-dev+exp,true,process,process.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. 2.0.0-dev+exp,true,process,process.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. 2.0.0-dev+exp,true,process,process.elf.creation_date,date,extended,,,Build or compile date. -2.0.0-dev+exp,true,process,process.elf.exports,flattened,extended,,,List of exported element names and types. +2.0.0-dev+exp,true,process,process.elf.exports,flattened,extended,array,,List of exported element names and types. 2.0.0-dev+exp,true,process,process.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). 2.0.0-dev+exp,true,process,process.elf.header.class,keyword,extended,,,Header class of the ELF file. 2.0.0-dev+exp,true,process,process.elf.header.data,keyword,extended,,,Data table of the ELF header. @@ -520,8 +508,8 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. 2.0.0-dev+exp,true,process,process.elf.header.type,keyword,extended,,,Header type of the ELF file. 2.0.0-dev+exp,true,process,process.elf.header.version,keyword,extended,,,Version of the ELF header. -2.0.0-dev+exp,true,process,process.elf.imports,flattened,extended,,,List of imported element names and types. -2.0.0-dev+exp,true,process,process.elf.sections,nested,extended,,,Section information of the ELF file. +2.0.0-dev+exp,true,process,process.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev+exp,true,process,process.elf.sections,nested,extended,array,,Section information of the ELF file. 2.0.0-dev+exp,true,process,process.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. 2.0.0-dev+exp,true,process,process.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. 2.0.0-dev+exp,true,process,process.elf.sections.flags,keyword,extended,,,ELF Section List flags. @@ -531,11 +519,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.elf.sections.type,keyword,extended,,,ELF Section List type. 2.0.0-dev+exp,true,process,process.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. 2.0.0-dev+exp,true,process,process.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. -2.0.0-dev+exp,true,process,process.elf.segments,nested,extended,,,ELF object segment list. +2.0.0-dev+exp,true,process,process.elf.segments,nested,extended,array,,ELF object segment list. 2.0.0-dev+exp,true,process,process.elf.segments.sections,keyword,extended,,,ELF object segment sections. 2.0.0-dev+exp,true,process,process.elf.segments.type,keyword,extended,,,ELF object segment type. -2.0.0-dev+exp,true,process,process.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object -2.0.0-dev+exp,true,process,process.elf.telfhash,keyword,extended,,,telfhash hash for ELF files +2.0.0-dev+exp,true,process,process.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev+exp,true,process,process.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev+exp,true,process,process.entity_id,keyword,extended,,c2c455d9f99375d,Unique identifier for the process. 2.0.0-dev+exp,true,process,process.executable,wildcard,extended,,/usr/bin/ssh,Absolute path to the process executable. 2.0.0-dev+exp,true,process,process.executable.text,text,extended,,/usr/bin/ssh,Absolute path to the process executable. @@ -559,10 +547,10 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.parent.command_line,wildcard,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev+exp,true,process,process.parent.command_line.text,text,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev+exp,true,process,process.parent.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. -2.0.0-dev+exp,true,process,process.parent.elf.byte_order,keyword,extended,,"Little Endian, Big Endian",Byte sequence of ELF file. +2.0.0-dev+exp,true,process,process.parent.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. 2.0.0-dev+exp,true,process,process.parent.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. 2.0.0-dev+exp,true,process,process.parent.elf.creation_date,date,extended,,,Build or compile date. -2.0.0-dev+exp,true,process,process.parent.elf.exports,flattened,extended,,,List of exported element names and types. +2.0.0-dev+exp,true,process,process.parent.elf.exports,flattened,extended,array,,List of exported element names and types. 2.0.0-dev+exp,true,process,process.parent.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). 2.0.0-dev+exp,true,process,process.parent.elf.header.class,keyword,extended,,,Header class of the ELF file. 2.0.0-dev+exp,true,process,process.parent.elf.header.data,keyword,extended,,,Data table of the ELF header. @@ -571,8 +559,8 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.parent.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. 2.0.0-dev+exp,true,process,process.parent.elf.header.type,keyword,extended,,,Header type of the ELF file. 2.0.0-dev+exp,true,process,process.parent.elf.header.version,keyword,extended,,,Version of the ELF header. -2.0.0-dev+exp,true,process,process.parent.elf.imports,flattened,extended,,,List of imported element names and types. -2.0.0-dev+exp,true,process,process.parent.elf.sections,nested,extended,,,Section information of the ELF file. +2.0.0-dev+exp,true,process,process.parent.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev+exp,true,process,process.parent.elf.sections,nested,extended,array,,Section information of the ELF file. 2.0.0-dev+exp,true,process,process.parent.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. 2.0.0-dev+exp,true,process,process.parent.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. 2.0.0-dev+exp,true,process,process.parent.elf.sections.flags,keyword,extended,,,ELF Section List flags. @@ -582,11 +570,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,process,process.parent.elf.sections.type,keyword,extended,,,ELF Section List type. 2.0.0-dev+exp,true,process,process.parent.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. 2.0.0-dev+exp,true,process,process.parent.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. -2.0.0-dev+exp,true,process,process.parent.elf.segments,nested,extended,,,ELF object segment list. +2.0.0-dev+exp,true,process,process.parent.elf.segments,nested,extended,array,,ELF object segment list. 2.0.0-dev+exp,true,process,process.parent.elf.segments.sections,keyword,extended,,,ELF object segment sections. 2.0.0-dev+exp,true,process,process.parent.elf.segments.type,keyword,extended,,,ELF object segment type. -2.0.0-dev+exp,true,process,process.parent.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object -2.0.0-dev+exp,true,process,process.parent.elf.telfhash,keyword,extended,,,telfhash hash for ELF files +2.0.0-dev+exp,true,process,process.parent.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev+exp,true,process,process.parent.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev+exp,true,process,process.parent.entity_id,keyword,extended,,c2c455d9f99375d,Unique identifier for the process. 2.0.0-dev+exp,true,process,process.parent.executable,wildcard,extended,,/usr/bin/ssh,Absolute path to the process executable. 2.0.0-dev+exp,true,process,process.parent.executable.text,text,extended,,/usr/bin/ssh,Absolute path to the process executable. @@ -706,7 +694,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,related,related.hash,keyword,extended,array,,All the hashes seen on your event. 2.0.0-dev+exp,true,related,related.hosts,keyword,extended,array,,All the host identifiers seen on your event. 2.0.0-dev+exp,true,related,related.ip,ip,extended,array,,All of the IPs seen on your event. -2.0.0-dev+exp,true,related,related.user,keyword,extended,array,,All the user names seen on your event. +2.0.0-dev+exp,true,related,related.user,keyword,extended,array,,All the user names or other user identifiers seen on the event. 2.0.0-dev+exp,true,rule,rule.author,keyword,extended,array,"[""Star-Lord""]",Rule author 2.0.0-dev+exp,true,rule,rule.category,keyword,extended,,Attempted Information Leak,Rule category 2.0.0-dev+exp,true,rule,rule.description,keyword,extended,,Block requests to public DNS over HTTPS / TLS protocols,Rule description @@ -828,6 +816,35 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,threat,threat.indicator.file.device,keyword,extended,,sda,Device that is the source of the file. 2.0.0-dev+exp,true,threat,threat.indicator.file.directory,wildcard,extended,,/home/alice,Directory where the file is located. 2.0.0-dev+exp,true,threat,threat.indicator.file.drive_letter,keyword,extended,,C,Drive letter where the file is located. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.creation_date,date,extended,,,Build or compile date. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.exports,flattened,extended,array,,List of exported element names and types. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.class,keyword,extended,,,Header class of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.data,keyword,extended,,,Data table of the ELF header. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.entrypoint,long,extended,,,Header entrypoint of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.object_version,keyword,extended,,,"""0x1"" for original ELF files." +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.type,keyword,extended,,,Header type of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.header.version,keyword,extended,,,Version of the ELF header. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections,nested,extended,array,,Section information of the ELF file. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.flags,keyword,extended,,,ELF Section List flags. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.name,keyword,extended,,,ELF Section List name. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.physical_offset,keyword,extended,,,ELF Section List offset. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.physical_size,long,extended,,,ELF Section List physical size. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.type,keyword,extended,,,ELF Section List type. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.segments,nested,extended,array,,ELF object segment list. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.segments.sections,keyword,extended,,,ELF object segment sections. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.segments.type,keyword,extended,,,ELF object segment type. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev+exp,true,threat,threat.indicator.file.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev+exp,true,threat,threat.indicator.file.extension,keyword,extended,,png,"File extension, excluding the leading dot." 2.0.0-dev+exp,true,threat,threat.indicator.file.gid,keyword,extended,,1001,Primary group ID (GID) of the file. 2.0.0-dev+exp,true,threat,threat.indicator.file.group,keyword,extended,,alice,Primary group name of the file. @@ -920,7 +937,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev+exp,true,threat,threat.indicator.type,keyword,extended,,ipv4-addr,Type of indicator 2.0.0-dev+exp,true,threat,threat.software.id,keyword,extended,,S0552,ID of the software 2.0.0-dev+exp,true,threat,threat.software.name,keyword,extended,,AdFind,Name of the software. -2.0.0-dev+exp,true,threat,threat.software.platforms,keyword,extended,,Windows,Platform of the software. +2.0.0-dev+exp,true,threat,threat.software.platforms,keyword,extended,array,"[ ""Windows"" ]",Platforms of the software. 2.0.0-dev+exp,true,threat,threat.software.reference,keyword,extended,,https://attack.mitre.org/software/S0552/,Software reference URL. 2.0.0-dev+exp,true,threat,threat.software.type,keyword,extended,,Tool,Software type. 2.0.0-dev+exp,true,threat,threat.tactic.id,keyword,extended,array,TA0002,Threat tactic id. diff --git a/experimental/generated/ecs/ecs_flat.yml b/experimental/generated/ecs/ecs_flat.yml index 47e425e7c7..4420384ea2 100644 --- a/experimental/generated/ecs/ecs_flat.yml +++ b/experimental/generated/ecs/ecs_flat.yml @@ -3145,7 +3145,7 @@ file.elf.architecture: file.elf.byte_order: dashed_name: file-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: file.elf.byte_order ignore_above: 1024 level: extended @@ -3183,7 +3183,8 @@ file.elf.exports: flat_name: file.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -3281,17 +3282,22 @@ file.elf.imports: flat_name: file.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened file.elf.sections: dashed_name: file-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' flat_name: file.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -3396,11 +3402,15 @@ file.elf.sections.virtual_size: type: long file.elf.segments: dashed_name: file-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' flat_name: file.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -3428,7 +3438,7 @@ file.elf.segments.type: type: keyword file.elf.shared_libraries: dashed_name: file-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: file.elf.shared_libraries ignore_above: 1024 level: extended @@ -3436,19 +3446,18 @@ file.elf.shared_libraries: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword file.elf.telfhash: dashed_name: file-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: file.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword file.extension: dashed_name: file-extension @@ -4891,135 +4900,6 @@ host.uptime: normalize: [] short: Seconds the host has been up. type: long -host.user.domain: - dashed_name: host-user-domain - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: user - short: Name of the directory the user is a member of. - type: keyword -host.user.email: - dashed_name: host-user-email - description: User email address. - flat_name: host.user.email - level: extended - name: email - normalize: [] - original_fieldset: user - short: User email address. - type: wildcard -host.user.full_name: - dashed_name: host-user-full-name - description: User's full name, if available. - example: Albert Einstein - flat_name: host.user.full_name - level: extended - multi_fields: - - flat_name: host.user.full_name.text - name: text - norms: false - type: text - name: full_name - normalize: [] - original_fieldset: user - short: User's full name, if available. - type: wildcard -host.user.group.domain: - dashed_name: host-user-group-domain - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.group.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: group - short: Name of the directory the group is a member of. - type: keyword -host.user.group.id: - dashed_name: host-user-group-id - description: Unique identifier for the group on the system/platform. - flat_name: host.user.group.id - ignore_above: 1024 - level: extended - name: id - normalize: [] - original_fieldset: group - short: Unique identifier for the group on the system/platform. - type: keyword -host.user.group.name: - dashed_name: host-user-group-name - description: Name of the group. - flat_name: host.user.group.name - ignore_above: 1024 - level: extended - name: name - normalize: [] - original_fieldset: group - short: Name of the group. - type: keyword -host.user.hash: - dashed_name: host-user-hash - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - flat_name: host.user.hash - ignore_above: 1024 - level: extended - name: hash - normalize: [] - original_fieldset: user - short: Unique user hash to correlate information for a user in anonymized form. - type: keyword -host.user.id: - dashed_name: host-user-id - description: Unique identifier of the user. - flat_name: host.user.id - ignore_above: 1024 - level: core - name: id - normalize: [] - original_fieldset: user - short: Unique identifier of the user. - type: keyword -host.user.name: - dashed_name: host-user-name - description: Short name or login of the user. - example: albert - flat_name: host.user.name - level: core - multi_fields: - - flat_name: host.user.name.text - name: text - norms: false - type: text - name: name - normalize: [] - original_fieldset: user - short: Short name or login of the user. - type: wildcard -host.user.roles: - dashed_name: host-user-roles - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - flat_name: host.user.roles - ignore_above: 1024 - level: extended - name: roles - normalize: - - array - original_fieldset: user - short: Array of user roles at the time of the event. - type: keyword http.request.body.bytes: dashed_name: http-request-body-bytes description: Size in bytes of the request body. @@ -6566,7 +6446,7 @@ process.elf.architecture: process.elf.byte_order: dashed_name: process-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: process.elf.byte_order ignore_above: 1024 level: extended @@ -6604,7 +6484,8 @@ process.elf.exports: flat_name: process.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -6702,17 +6583,22 @@ process.elf.imports: flat_name: process.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened process.elf.sections: dashed_name: process-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' flat_name: process.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -6817,11 +6703,15 @@ process.elf.sections.virtual_size: type: long process.elf.segments: dashed_name: process-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' flat_name: process.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -6849,7 +6739,7 @@ process.elf.segments.type: type: keyword process.elf.shared_libraries: dashed_name: process-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: process.elf.shared_libraries ignore_above: 1024 level: extended @@ -6857,19 +6747,18 @@ process.elf.shared_libraries: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword process.elf.telfhash: dashed_name: process-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: process.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword process.entity_id: dashed_name: process-entity-id @@ -7153,7 +7042,7 @@ process.parent.elf.architecture: process.parent.elf.byte_order: dashed_name: process-parent-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: process.parent.elf.byte_order ignore_above: 1024 level: extended @@ -7191,7 +7080,8 @@ process.parent.elf.exports: flat_name: process.parent.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -7289,17 +7179,22 @@ process.parent.elf.imports: flat_name: process.parent.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened process.parent.elf.sections: dashed_name: process-parent-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' flat_name: process.parent.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -7404,11 +7299,15 @@ process.parent.elf.sections.virtual_size: type: long process.parent.elf.segments: dashed_name: process-parent-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' flat_name: process.parent.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -7436,7 +7335,7 @@ process.parent.elf.segments.type: type: keyword process.parent.elf.shared_libraries: dashed_name: process-parent-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: process.parent.elf.shared_libraries ignore_above: 1024 level: extended @@ -7444,19 +7343,18 @@ process.parent.elf.shared_libraries: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword process.parent.elf.telfhash: dashed_name: process-parent-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: process.parent.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword process.parent.entity_id: dashed_name: process-parent-entity-id @@ -8839,14 +8737,14 @@ related.ip: type: ip related.user: dashed_name: related-user - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. flat_name: related.user ignore_above: 1024 level: extended name: user normalize: - array - short: All the user names seen on your event. + short: All the user names or other user identifiers seen on the event. type: keyword rule.author: dashed_name: rule-author @@ -10025,6 +9923,7 @@ threat.framework: short: Threat classification framework. type: keyword threat.group.alias: + beta: This field is beta and subject to change. dashed_name: threat-group-alias description: "The alias(es) of the group for a set of related intrusion activity\ \ that are tracked by a common name in the security community. While not required,\ @@ -10039,6 +9938,7 @@ threat.group.alias: short: Alias of the group. type: keyword threat.group.id: + beta: This field is beta and subject to change. dashed_name: threat-group-id description: "The id of the group for a set of related intrusion activity that are\ \ tracked by a common name in the security community. While not required, you\ @@ -10052,6 +9952,7 @@ threat.group.id: short: ID of the group. type: keyword threat.group.name: + beta: This field is beta and subject to change. dashed_name: threat-group-name description: "The name of the group for a set of related intrusion activity that\ \ are tracked by a common name in the security community. While not required,\ @@ -10065,6 +9966,7 @@ threat.group.name: short: Name of the group. type: keyword threat.group.reference: + beta: This field is beta and subject to change. dashed_name: threat-group-reference description: "The reference URL of the group for a set of related intrusion activity\ \ that are tracked by a common name in the security community. While not required,\ @@ -10354,6 +10256,335 @@ threat.indicator.file.drive_letter: original_fieldset: file short: Drive letter where the file is located. type: keyword +threat.indicator.file.elf.architecture: + dashed_name: threat-indicator-file-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: threat.indicator.file.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword +threat.indicator.file.elf.byte_order: + dashed_name: threat-indicator-file-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: threat.indicator.file.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword +threat.indicator.file.elf.cpu_type: + dashed_name: threat-indicator-file-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: threat.indicator.file.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword +threat.indicator.file.elf.creation_date: + dashed_name: threat-indicator-file-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when it + was built or compiled. It can also be faked by malware creators. + flat_name: threat.indicator.file.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date +threat.indicator.file.elf.exports: + dashed_name: threat-indicator-file-elf-exports + description: List of exported element names and types. + flat_name: threat.indicator.file.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened +threat.indicator.file.elf.header.abi_version: + dashed_name: threat-indicator-file-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: threat.indicator.file.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword +threat.indicator.file.elf.header.class: + dashed_name: threat-indicator-file-elf-header-class + description: Header class of the ELF file. + flat_name: threat.indicator.file.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword +threat.indicator.file.elf.header.data: + dashed_name: threat-indicator-file-elf-header-data + description: Data table of the ELF header. + flat_name: threat.indicator.file.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword +threat.indicator.file.elf.header.entrypoint: + dashed_name: threat-indicator-file-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: threat.indicator.file.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long +threat.indicator.file.elf.header.object_version: + dashed_name: threat-indicator-file-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: threat.indicator.file.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword +threat.indicator.file.elf.header.os_abi: + dashed_name: threat-indicator-file-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: threat.indicator.file.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword +threat.indicator.file.elf.header.type: + dashed_name: threat-indicator-file-elf-header-type + description: Header type of the ELF file. + flat_name: threat.indicator.file.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword +threat.indicator.file.elf.header.version: + dashed_name: threat-indicator-file-elf-header-version + description: Version of the ELF header. + flat_name: threat.indicator.file.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword +threat.indicator.file.elf.imports: + dashed_name: threat-indicator-file-elf-imports + description: List of imported element names and types. + flat_name: threat.indicator.file.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened +threat.indicator.file.elf.sections: + dashed_name: threat-indicator-file-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' + flat_name: threat.indicator.file.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested +threat.indicator.file.elf.sections.chi2: + dashed_name: threat-indicator-file-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: threat.indicator.file.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long +threat.indicator.file.elf.sections.entropy: + dashed_name: threat-indicator-file-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: threat.indicator.file.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long +threat.indicator.file.elf.sections.flags: + dashed_name: threat-indicator-file-elf-sections-flags + description: ELF Section List flags. + flat_name: threat.indicator.file.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword +threat.indicator.file.elf.sections.name: + dashed_name: threat-indicator-file-elf-sections-name + description: ELF Section List name. + flat_name: threat.indicator.file.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword +threat.indicator.file.elf.sections.physical_offset: + dashed_name: threat-indicator-file-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: threat.indicator.file.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword +threat.indicator.file.elf.sections.physical_size: + dashed_name: threat-indicator-file-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: threat.indicator.file.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long +threat.indicator.file.elf.sections.type: + dashed_name: threat-indicator-file-elf-sections-type + description: ELF Section List type. + flat_name: threat.indicator.file.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword +threat.indicator.file.elf.sections.virtual_address: + dashed_name: threat-indicator-file-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: threat.indicator.file.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long +threat.indicator.file.elf.sections.virtual_size: + dashed_name: threat-indicator-file-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: threat.indicator.file.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long +threat.indicator.file.elf.segments: + dashed_name: threat-indicator-file-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' + flat_name: threat.indicator.file.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested +threat.indicator.file.elf.segments.sections: + dashed_name: threat-indicator-file-elf-segments-sections + description: ELF object segment sections. + flat_name: threat.indicator.file.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword +threat.indicator.file.elf.segments.type: + dashed_name: threat-indicator-file-elf-segments-type + description: ELF object segment type. + flat_name: threat.indicator.file.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword +threat.indicator.file.elf.shared_libraries: + dashed_name: threat-indicator-file-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: threat.indicator.file.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword +threat.indicator.file.elf.telfhash: + dashed_name: threat-indicator-file-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: threat.indicator.file.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword threat.indicator.file.extension: dashed_name: threat-indicator-file-extension description: 'File extension, excluding the leading dot. @@ -11428,6 +11659,7 @@ threat.indicator.type: short: Type of indicator type: keyword threat.software.id: + beta: This field is beta and subject to change. dashed_name: threat-software-id description: "The id of the software used by this threat to conduct behavior commonly\ \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ @@ -11441,6 +11673,7 @@ threat.software.id: short: ID of the software type: keyword threat.software.name: + beta: This field is beta and subject to change. dashed_name: threat-software-name description: "The name of the software used by this threat to conduct behavior commonly\ \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ @@ -11454,21 +11687,24 @@ threat.software.name: short: Name of the software. type: keyword threat.software.platforms: + beta: This field is beta and subject to change. dashed_name: threat-software-platforms - description: "The platform of the software used by this threat to conduct behavior\ + description: "The platforms of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ - \ ATT&CK\xAE software platform.\nExpected values\n * AWS\n * Azure\n * Azure\ - \ AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office 365\n * PRE\n *\ - \ SaaS\n * Windows" - example: Windows + \ ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n * Azure\n *\ + \ Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office 365\n * SaaS\n\ + \ * Windows" + example: '[ "Windows" ]' flat_name: threat.software.platforms ignore_above: 1024 level: extended name: software.platforms - normalize: [] - short: Platform of the software. + normalize: + - array + short: Platforms of the software. type: keyword threat.software.reference: + beta: This field is beta and subject to change. dashed_name: threat-software-reference description: "The reference URL of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ @@ -11482,10 +11718,11 @@ threat.software.reference: short: Software reference URL. type: keyword threat.software.type: + beta: This field is beta and subject to change. dashed_name: threat-software-type description: "The type of software used by this threat to conduct behavior commonly\ \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ - \ software type.\nExpected values\n * Malware\n * Tool" + \ software type.\nRecommended values\n * Malware\n * Tool" example: Tool flat_name: threat.software.type ignore_above: 1024 diff --git a/experimental/generated/ecs/ecs_nested.yml b/experimental/generated/ecs/ecs_nested.yml index 43661a6843..eac8912ca8 100644 --- a/experimental/generated/ecs/ecs_nested.yml +++ b/experimental/generated/ecs/ecs_nested.yml @@ -2570,6 +2570,7 @@ ecs: title: ECS type: group elf: + beta: These fields are in beta and are subject to change. description: These fields contain Linux Executable Linkable Format (ELF) metadata. fields: elf.architecture: @@ -2586,7 +2587,7 @@ elf: elf.byte_order: dashed_name: elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: elf.byte_order ignore_above: 1024 level: extended @@ -2621,7 +2622,8 @@ elf: flat_name: elf.exports level: extended name: exports - normalize: [] + normalize: + - array short: List of exported element names and types. type: flattened elf.header.abi_version: @@ -2710,16 +2712,21 @@ elf: flat_name: elf.imports level: extended name: imports - normalize: [] + normalize: + - array short: List of imported element names and types. type: flattened elf.sections: dashed_name: elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' flat_name: elf.sections level: extended name: sections - normalize: [] + normalize: + - array short: Section information of the ELF file. type: nested elf.sections.chi2: @@ -2814,11 +2821,15 @@ elf: type: long elf.segments: dashed_name: elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' flat_name: elf.segments level: extended name: segments - normalize: [] + normalize: + - array short: ELF object segment list. type: nested elf.segments.sections: @@ -2843,25 +2854,24 @@ elf: type: keyword elf.shared_libraries: dashed_name: elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: elf.shared_libraries ignore_above: 1024 level: extended name: shared_libraries normalize: - array - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword elf.telfhash: dashed_name: elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword group: 2 name: elf @@ -2870,9 +2880,11 @@ elf: expected: - as: elf at: file + beta: This field reuse is beta and subject to change. full: file.elf - as: elf at: process + beta: This field reuse is beta and subject to change. full: process.elf top_level: false short: These fields contain Linux Executable Linkable Format (ELF) metadata. @@ -3935,7 +3947,7 @@ file: file.elf.byte_order: dashed_name: file-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: file.elf.byte_order ignore_above: 1024 level: extended @@ -3973,7 +3985,8 @@ file: flat_name: file.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -4071,17 +4084,22 @@ file: flat_name: file.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened file.elf.sections: dashed_name: file-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' flat_name: file.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -4186,11 +4204,15 @@ file: type: long file.elf.segments: dashed_name: file-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' flat_name: file.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -4218,7 +4240,7 @@ file: type: keyword file.elf.shared_libraries: dashed_name: file-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: file.elf.shared_libraries ignore_above: 1024 level: extended @@ -4226,19 +4248,18 @@ file: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword file.elf.telfhash: dashed_name: file-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: file.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword file.extension: dashed_name: file-extension @@ -5243,6 +5264,10 @@ file: - full: file.code_signature schema_name: code_signature short: These fields contain information about binary code signatures. + - beta: This field reuse is beta and subject to change. + full: file.elf + schema_name: elf + short: These fields contain Linux Executable Linkable Format (ELF) metadata. - full: file.hash schema_name: hash short: Hashes, usually file hashes. @@ -5252,9 +5277,6 @@ file: - full: file.x509 schema_name: x509 short: These fields contain x509 certificate metadata. - - full: file.elf - schema_name: elf - short: These fields contain Linux Executable Linkable Format (ELF) metadata. short: Fields describing files. title: File type: group @@ -5992,141 +6014,11 @@ host: normalize: [] short: Seconds the host has been up. type: long - host.user.domain: - dashed_name: host-user-domain - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: user - short: Name of the directory the user is a member of. - type: keyword - host.user.email: - dashed_name: host-user-email - description: User email address. - flat_name: host.user.email - level: extended - name: email - normalize: [] - original_fieldset: user - short: User email address. - type: wildcard - host.user.full_name: - dashed_name: host-user-full-name - description: User's full name, if available. - example: Albert Einstein - flat_name: host.user.full_name - level: extended - multi_fields: - - flat_name: host.user.full_name.text - name: text - norms: false - type: text - name: full_name - normalize: [] - original_fieldset: user - short: User's full name, if available. - type: wildcard - host.user.group.domain: - dashed_name: host-user-group-domain - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.group.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: group - short: Name of the directory the group is a member of. - type: keyword - host.user.group.id: - dashed_name: host-user-group-id - description: Unique identifier for the group on the system/platform. - flat_name: host.user.group.id - ignore_above: 1024 - level: extended - name: id - normalize: [] - original_fieldset: group - short: Unique identifier for the group on the system/platform. - type: keyword - host.user.group.name: - dashed_name: host-user-group-name - description: Name of the group. - flat_name: host.user.group.name - ignore_above: 1024 - level: extended - name: name - normalize: [] - original_fieldset: group - short: Name of the group. - type: keyword - host.user.hash: - dashed_name: host-user-hash - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - flat_name: host.user.hash - ignore_above: 1024 - level: extended - name: hash - normalize: [] - original_fieldset: user - short: Unique user hash to correlate information for a user in anonymized form. - type: keyword - host.user.id: - dashed_name: host-user-id - description: Unique identifier of the user. - flat_name: host.user.id - ignore_above: 1024 - level: core - name: id - normalize: [] - original_fieldset: user - short: Unique identifier of the user. - type: keyword - host.user.name: - dashed_name: host-user-name - description: Short name or login of the user. - example: albert - flat_name: host.user.name - level: core - multi_fields: - - flat_name: host.user.name.text - name: text - norms: false - type: text - name: name - normalize: [] - original_fieldset: user - short: Short name or login of the user. - type: wildcard - host.user.roles: - dashed_name: host-user-roles - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - flat_name: host.user.roles - ignore_above: 1024 - level: extended - name: roles - normalize: - - array - original_fieldset: user - short: Array of user roles at the time of the event. - type: keyword group: 2 name: host nestings: - host.geo - host.os - - host.user prefix: host. reused_here: - full: host.geo @@ -6135,9 +6027,6 @@ host: - full: host.os schema_name: os short: OS fields contain information about the operating system. - - full: host.user - schema_name: user - short: Fields to describe the user relevant to the event. short: Fields describing the relevant computing instance. title: Host type: group @@ -7390,7 +7279,6 @@ observer: title: Observer type: group orchestrator: - beta: These fields are in beta and are subject to change. description: Fields that describe the resources which container orchestrators manage or act upon. fields: @@ -8423,7 +8311,7 @@ process: process.elf.byte_order: dashed_name: process-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: process.elf.byte_order ignore_above: 1024 level: extended @@ -8461,7 +8349,8 @@ process: flat_name: process.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -8559,17 +8448,22 @@ process: flat_name: process.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened process.elf.sections: dashed_name: process-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' flat_name: process.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -8674,11 +8568,15 @@ process: type: long process.elf.segments: dashed_name: process-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' flat_name: process.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -8706,7 +8604,7 @@ process: type: keyword process.elf.shared_libraries: dashed_name: process-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: process.elf.shared_libraries ignore_above: 1024 level: extended @@ -8714,19 +8612,18 @@ process: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword process.elf.telfhash: dashed_name: process-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: process.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword process.entity_id: dashed_name: process-entity-id @@ -9010,7 +8907,7 @@ process: process.parent.elf.byte_order: dashed_name: process-parent-elf-byte-order description: Byte sequence of ELF file. - example: Little Endian, Big Endian + example: Little Endian flat_name: process.parent.elf.byte_order ignore_above: 1024 level: extended @@ -9048,7 +8945,8 @@ process: flat_name: process.parent.elf.exports level: extended name: exports - normalize: [] + normalize: + - array original_fieldset: elf short: List of exported element names and types. type: flattened @@ -9146,17 +9044,22 @@ process: flat_name: process.parent.elf.imports level: extended name: imports - normalize: [] + normalize: + - array original_fieldset: elf short: List of imported element names and types. type: flattened process.parent.elf.sections: dashed_name: process-parent-elf-sections - description: Section information of the ELF file. + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' flat_name: process.parent.elf.sections level: extended name: sections - normalize: [] + normalize: + - array original_fieldset: elf short: Section information of the ELF file. type: nested @@ -9261,11 +9164,15 @@ process: type: long process.parent.elf.segments: dashed_name: process-parent-elf-segments - description: ELF object segment list. + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' flat_name: process.parent.elf.segments level: extended name: segments - normalize: [] + normalize: + - array original_fieldset: elf short: ELF object segment list. type: nested @@ -9293,7 +9200,7 @@ process: type: keyword process.parent.elf.shared_libraries: dashed_name: process-parent-elf-shared-libraries - description: List of shared libraries used by this ELF object + description: List of shared libraries used by this ELF object. flat_name: process.parent.elf.shared_libraries ignore_above: 1024 level: extended @@ -9301,19 +9208,18 @@ process: normalize: - array original_fieldset: elf - short: List of shared libraries used by this ELF object + short: List of shared libraries used by this ELF object. type: keyword process.parent.elf.telfhash: dashed_name: process-parent-elf-telfhash - description: telfhash is symbol hash for ELF files, just like imphash is imports - hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + description: telfhash symbol hash for ELF file. flat_name: process.parent.elf.telfhash ignore_above: 1024 level: extended name: telfhash normalize: [] original_fieldset: elf - short: telfhash hash for ELF files + short: telfhash hash for ELF file. type: keyword process.parent.entity_id: dashed_name: process-parent-entity-id @@ -10595,15 +10501,16 @@ process: - full: process.code_signature schema_name: code_signature short: These fields contain information about binary code signatures. + - beta: This field reuse is beta and subject to change. + full: process.elf + schema_name: elf + short: These fields contain Linux Executable Linkable Format (ELF) metadata. - full: process.hash schema_name: hash short: Hashes, usually file hashes. - full: process.pe schema_name: pe short: These fields contain Windows Portable Executable (PE) metadata. - - full: process.elf - schema_name: elf - short: These fields contain Linux Executable Linkable Format (ELF) metadata. - full: process.parent schema_name: process short: Information about the parent process. @@ -10760,14 +10667,14 @@ related: type: ip related.user: dashed_name: related-user - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. flat_name: related.user ignore_above: 1024 level: extended name: user normalize: - array - short: All the user names seen on your event. + short: All the user names or other user identifiers seen on the event. type: keyword group: 2 name: related @@ -12034,6 +11941,7 @@ threat: short: Threat classification framework. type: keyword threat.group.alias: + beta: This field is beta and subject to change. dashed_name: threat-group-alias description: "The alias(es) of the group for a set of related intrusion activity\ \ that are tracked by a common name in the security community. While not required,\ @@ -12048,6 +11956,7 @@ threat: short: Alias of the group. type: keyword threat.group.id: + beta: This field is beta and subject to change. dashed_name: threat-group-id description: "The id of the group for a set of related intrusion activity that\ \ are tracked by a common name in the security community. While not required,\ @@ -12061,6 +11970,7 @@ threat: short: ID of the group. type: keyword threat.group.name: + beta: This field is beta and subject to change. dashed_name: threat-group-name description: "The name of the group for a set of related intrusion activity\ \ that are tracked by a common name in the security community. While not required,\ @@ -12074,6 +11984,7 @@ threat: short: Name of the group. type: keyword threat.group.reference: + beta: This field is beta and subject to change. dashed_name: threat-group-reference description: "The reference URL of the group for a set of related intrusion\ \ activity that are tracked by a common name in the security community. While\ @@ -12363,6 +12274,335 @@ threat: original_fieldset: file short: Drive letter where the file is located. type: keyword + threat.indicator.file.elf.architecture: + dashed_name: threat-indicator-file-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: threat.indicator.file.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword + threat.indicator.file.elf.byte_order: + dashed_name: threat-indicator-file-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: threat.indicator.file.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword + threat.indicator.file.elf.cpu_type: + dashed_name: threat-indicator-file-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: threat.indicator.file.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword + threat.indicator.file.elf.creation_date: + dashed_name: threat-indicator-file-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + flat_name: threat.indicator.file.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date + threat.indicator.file.elf.exports: + dashed_name: threat-indicator-file-elf-exports + description: List of exported element names and types. + flat_name: threat.indicator.file.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened + threat.indicator.file.elf.header.abi_version: + dashed_name: threat-indicator-file-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: threat.indicator.file.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword + threat.indicator.file.elf.header.class: + dashed_name: threat-indicator-file-elf-header-class + description: Header class of the ELF file. + flat_name: threat.indicator.file.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword + threat.indicator.file.elf.header.data: + dashed_name: threat-indicator-file-elf-header-data + description: Data table of the ELF header. + flat_name: threat.indicator.file.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword + threat.indicator.file.elf.header.entrypoint: + dashed_name: threat-indicator-file-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: threat.indicator.file.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long + threat.indicator.file.elf.header.object_version: + dashed_name: threat-indicator-file-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: threat.indicator.file.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword + threat.indicator.file.elf.header.os_abi: + dashed_name: threat-indicator-file-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: threat.indicator.file.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword + threat.indicator.file.elf.header.type: + dashed_name: threat-indicator-file-elf-header-type + description: Header type of the ELF file. + flat_name: threat.indicator.file.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword + threat.indicator.file.elf.header.version: + dashed_name: threat-indicator-file-elf-header-version + description: Version of the ELF header. + flat_name: threat.indicator.file.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword + threat.indicator.file.elf.imports: + dashed_name: threat-indicator-file-elf-imports + description: List of imported element names and types. + flat_name: threat.indicator.file.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened + threat.indicator.file.elf.sections: + dashed_name: threat-indicator-file-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + flat_name: threat.indicator.file.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested + threat.indicator.file.elf.sections.chi2: + dashed_name: threat-indicator-file-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: threat.indicator.file.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long + threat.indicator.file.elf.sections.entropy: + dashed_name: threat-indicator-file-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: threat.indicator.file.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long + threat.indicator.file.elf.sections.flags: + dashed_name: threat-indicator-file-elf-sections-flags + description: ELF Section List flags. + flat_name: threat.indicator.file.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword + threat.indicator.file.elf.sections.name: + dashed_name: threat-indicator-file-elf-sections-name + description: ELF Section List name. + flat_name: threat.indicator.file.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword + threat.indicator.file.elf.sections.physical_offset: + dashed_name: threat-indicator-file-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: threat.indicator.file.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword + threat.indicator.file.elf.sections.physical_size: + dashed_name: threat-indicator-file-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: threat.indicator.file.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long + threat.indicator.file.elf.sections.type: + dashed_name: threat-indicator-file-elf-sections-type + description: ELF Section List type. + flat_name: threat.indicator.file.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword + threat.indicator.file.elf.sections.virtual_address: + dashed_name: threat-indicator-file-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: threat.indicator.file.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long + threat.indicator.file.elf.sections.virtual_size: + dashed_name: threat-indicator-file-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: threat.indicator.file.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long + threat.indicator.file.elf.segments: + dashed_name: threat-indicator-file-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + flat_name: threat.indicator.file.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested + threat.indicator.file.elf.segments.sections: + dashed_name: threat-indicator-file-elf-segments-sections + description: ELF object segment sections. + flat_name: threat.indicator.file.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword + threat.indicator.file.elf.segments.type: + dashed_name: threat-indicator-file-elf-segments-type + description: ELF object segment type. + flat_name: threat.indicator.file.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword + threat.indicator.file.elf.shared_libraries: + dashed_name: threat-indicator-file-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: threat.indicator.file.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword + threat.indicator.file.elf.telfhash: + dashed_name: threat-indicator-file-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: threat.indicator.file.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword threat.indicator.file.extension: dashed_name: threat-indicator-file-extension description: 'File extension, excluding the leading dot. @@ -13440,6 +13680,7 @@ threat: short: Type of indicator type: keyword threat.software.id: + beta: This field is beta and subject to change. dashed_name: threat-software-id description: "The id of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ @@ -13453,6 +13694,7 @@ threat: short: ID of the software type: keyword threat.software.name: + beta: This field is beta and subject to change. dashed_name: threat-software-name description: "The name of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ @@ -13466,21 +13708,24 @@ threat: short: Name of the software. type: keyword threat.software.platforms: + beta: This field is beta and subject to change. dashed_name: threat-software-platforms - description: "The platform of the software used by this threat to conduct behavior\ + description: "The platforms of the software used by this threat to conduct behavior\ \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ - \ a MITRE ATT&CK\xAE software platform.\nExpected values\n * AWS\n * Azure\n\ - \ * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office 365\n\ - \ * PRE\n * SaaS\n * Windows" - example: Windows + \ a MITRE ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n \ + \ * Azure\n * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office\ + \ 365\n * SaaS\n * Windows" + example: '[ "Windows" ]' flat_name: threat.software.platforms ignore_above: 1024 level: extended name: software.platforms - normalize: [] - short: Platform of the software. + normalize: + - array + short: Platforms of the software. type: keyword threat.software.reference: + beta: This field is beta and subject to change. dashed_name: threat-software-reference description: "The reference URL of the software used by this threat to conduct\ \ behavior commonly modeled using MITRE ATT&CK\xAE. While not required, you\ @@ -13494,10 +13739,11 @@ threat: short: Software reference URL. type: keyword threat.software.type: + beta: This field is beta and subject to change. dashed_name: threat-software-type description: "The type of software used by this threat to conduct behavior commonly\ \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ - \ ATT&CK\xAE software type.\nExpected values\n * Malware\n * Tool" + \ ATT&CK\xAE software type.\nRecommended values\n * Malware\n * Tool" example: Tool flat_name: threat.software.type ignore_above: 1024 @@ -15462,9 +15708,6 @@ user: - as: user at: destination full: destination.user - - as: user - at: host - full: host.user - as: user at: server full: server.user diff --git a/experimental/generated/elasticsearch/7/template.json b/experimental/generated/elasticsearch/7/template.json index e95282fb92..d9c85007fd 100644 --- a/experimental/generated/elasticsearch/7/template.json +++ b/experimental/generated/elasticsearch/7/template.json @@ -1670,63 +1670,6 @@ }, "uptime": { "type": "long" - }, - "user": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "email": { - "type": "wildcard" - }, - "full_name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "type": "wildcard" - }, - "group": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "hash": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "type": "wildcard" - }, - "roles": { - "ignore_above": 1024, - "type": "keyword" - } - } } } }, @@ -3742,6 +3685,123 @@ "ignore_above": 1, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "extension": { "ignore_above": 1024, "type": "keyword" diff --git a/experimental/generated/elasticsearch/component/host.json b/experimental/generated/elasticsearch/component/host.json index 6100963315..b7eeb41cbe 100644 --- a/experimental/generated/elasticsearch/component/host.json +++ b/experimental/generated/elasticsearch/component/host.json @@ -178,63 +178,6 @@ }, "uptime": { "type": "long" - }, - "user": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "email": { - "type": "wildcard" - }, - "full_name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "type": "wildcard" - }, - "group": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "hash": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "type": "wildcard" - }, - "roles": { - "ignore_above": 1024, - "type": "keyword" - } - } } } } diff --git a/experimental/generated/elasticsearch/component/threat.json b/experimental/generated/elasticsearch/component/threat.json index c4a3056201..123ee68bdc 100644 --- a/experimental/generated/elasticsearch/component/threat.json +++ b/experimental/generated/elasticsearch/component/threat.json @@ -132,6 +132,123 @@ "ignore_above": 1, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "extension": { "ignore_above": 1024, "type": "keyword" diff --git a/experimental/schemas/threat.yml b/experimental/schemas/threat.yml index dac8061c52..523f909f06 100644 --- a/experimental/schemas/threat.yml +++ b/experimental/schemas/threat.yml @@ -194,103 +194,3 @@ Identifies the type of the atomic indicator that matched a local environment endpoint or network event. example: domain-name - - - name: software.id - level: extended - type: keyword - short: ID of the software - description: > - The id of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software id. - - example: "S0552" - - - name: software.name - level: extended - type: keyword - short: Name of the software. - description: > - The name of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software name. - - example: "AdFind" - - - name: software.platforms - level: extended - type: keyword - short: Platform of the software. - description: > - The platform of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software platform. - - Expected values - * AWS - * Azure - * Azure AD - * GCP - * Linux - * macOS - * Network - * Office 365 - * PRE - * SaaS - * Windows - - example: "Windows" - - - name: software.reference - level: extended - type: keyword - short: Software reference URL. - description: > - The reference URL of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software reference URL. - - example: "https://attack.mitre.org/software/S0552/" - - - name: software.type - level: extended - type: keyword - short: Software type. - description: > - The type of software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software type. - - Expected values - * Malware - * Tool - - example: "Tool" - - - name: group.alias - level: extended - type: keyword - short: Alias of the group. - description: > - The alias(es) of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group alias(es). - - example: '[ "Magecart Group 6" ]' - normalize: - - array - - - name: group.id - level: extended - type: keyword - short: ID of the group. - description: > - The id of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group id. - - example: "G0037" - - - name: group.name - level: extended - type: keyword - short: Name of the group. - description: > - The name of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group name. - - example: "FIN6" - - - name: group.reference - level: extended - type: keyword - short: Reference URL of the group. - description: > - The reference URL of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group reference URL. - - example: "https://attack.mitre.org/groups/G0037/" diff --git a/generated/beats/fields.ecs.yml b/generated/beats/fields.ecs.yml index 3067eab765..34f3e59cf5 100644 --- a/generated/beats/fields.ecs.yml +++ b/generated/beats/fields.ecs.yml @@ -1329,6 +1329,191 @@ ECS versions -- this field lets integrations adjust to the schema version of the events.' example: 1.0.0 + - name: elf + title: ELF Header + group: 2 + description: These fields contain Linux Executable Linkable Format (ELF) metadata. + type: group + fields: + - name: architecture + level: extended + type: keyword + ignore_above: 1024 + description: Machine architecture of the ELF file. + example: x86-64 + default_field: false + - name: byte_order + level: extended + type: keyword + ignore_above: 1024 + description: Byte sequence of ELF file. + example: Little Endian + default_field: false + - name: cpu_type + level: extended + type: keyword + ignore_above: 1024 + description: CPU type of the ELF file. + example: Intel + default_field: false + - name: creation_date + level: extended + type: date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + default_field: false + - name: exports + level: extended + type: flattened + description: List of exported element names and types. + default_field: false + - name: header.abi_version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF Application Binary Interface (ABI). + default_field: false + - name: header.class + level: extended + type: keyword + ignore_above: 1024 + description: Header class of the ELF file. + default_field: false + - name: header.data + level: extended + type: keyword + ignore_above: 1024 + description: Data table of the ELF header. + default_field: false + - name: header.entrypoint + level: extended + type: long + format: string + description: Header entrypoint of the ELF file. + default_field: false + - name: header.object_version + level: extended + type: keyword + ignore_above: 1024 + description: '"0x1" for original ELF files.' + default_field: false + - name: header.os_abi + level: extended + type: keyword + ignore_above: 1024 + description: Application Binary Interface (ABI) of the Linux OS. + default_field: false + - name: header.type + level: extended + type: keyword + ignore_above: 1024 + description: Header type of the ELF file. + default_field: false + - name: header.version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF header. + default_field: false + - name: imports + level: extended + type: flattened + description: List of imported element names and types. + default_field: false + - name: sections + level: extended + type: nested + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + default_field: false + - name: sections.chi2 + level: extended + type: long + format: number + description: Chi-square probability distribution of the section. + default_field: false + - name: sections.entropy + level: extended + type: long + format: number + description: Shannon entropy calculation from the section. + default_field: false + - name: sections.flags + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List flags. + default_field: false + - name: sections.name + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List name. + default_field: false + - name: sections.physical_offset + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List offset. + default_field: false + - name: sections.physical_size + level: extended + type: long + format: bytes + description: ELF Section List physical size. + default_field: false + - name: sections.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List type. + default_field: false + - name: sections.virtual_address + level: extended + type: long + format: string + description: ELF Section List virtual address. + default_field: false + - name: sections.virtual_size + level: extended + type: long + format: string + description: ELF Section List virtual size. + default_field: false + - name: segments + level: extended + type: nested + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + default_field: false + - name: segments.sections + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment sections. + default_field: false + - name: segments.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment type. + default_field: false + - name: shared_libraries + level: extended + type: keyword + ignore_above: 1024 + description: List of shared libraries used by this ELF object. + default_field: false + - name: telfhash + level: extended + type: keyword + ignore_above: 1024 + description: telfhash symbol hash for ELF file. + default_field: false - name: error title: Error group: 2 @@ -1790,6 +1975,185 @@ The value should be uppercase, and not include the colon.' example: C default_field: false + - name: elf.architecture + level: extended + type: keyword + ignore_above: 1024 + description: Machine architecture of the ELF file. + example: x86-64 + default_field: false + - name: elf.byte_order + level: extended + type: keyword + ignore_above: 1024 + description: Byte sequence of ELF file. + example: Little Endian + default_field: false + - name: elf.cpu_type + level: extended + type: keyword + ignore_above: 1024 + description: CPU type of the ELF file. + example: Intel + default_field: false + - name: elf.creation_date + level: extended + type: date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + default_field: false + - name: elf.exports + level: extended + type: flattened + description: List of exported element names and types. + default_field: false + - name: elf.header.abi_version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF Application Binary Interface (ABI). + default_field: false + - name: elf.header.class + level: extended + type: keyword + ignore_above: 1024 + description: Header class of the ELF file. + default_field: false + - name: elf.header.data + level: extended + type: keyword + ignore_above: 1024 + description: Data table of the ELF header. + default_field: false + - name: elf.header.entrypoint + level: extended + type: long + format: string + description: Header entrypoint of the ELF file. + default_field: false + - name: elf.header.object_version + level: extended + type: keyword + ignore_above: 1024 + description: '"0x1" for original ELF files.' + default_field: false + - name: elf.header.os_abi + level: extended + type: keyword + ignore_above: 1024 + description: Application Binary Interface (ABI) of the Linux OS. + default_field: false + - name: elf.header.type + level: extended + type: keyword + ignore_above: 1024 + description: Header type of the ELF file. + default_field: false + - name: elf.header.version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF header. + default_field: false + - name: elf.imports + level: extended + type: flattened + description: List of imported element names and types. + default_field: false + - name: elf.sections + level: extended + type: nested + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + default_field: false + - name: elf.sections.chi2 + level: extended + type: long + format: number + description: Chi-square probability distribution of the section. + default_field: false + - name: elf.sections.entropy + level: extended + type: long + format: number + description: Shannon entropy calculation from the section. + default_field: false + - name: elf.sections.flags + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List flags. + default_field: false + - name: elf.sections.name + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List name. + default_field: false + - name: elf.sections.physical_offset + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List offset. + default_field: false + - name: elf.sections.physical_size + level: extended + type: long + format: bytes + description: ELF Section List physical size. + default_field: false + - name: elf.sections.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List type. + default_field: false + - name: elf.sections.virtual_address + level: extended + type: long + format: string + description: ELF Section List virtual address. + default_field: false + - name: elf.sections.virtual_size + level: extended + type: long + format: string + description: ELF Section List virtual size. + default_field: false + - name: elf.segments + level: extended + type: nested + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + default_field: false + - name: elf.segments.sections + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment sections. + default_field: false + - name: elf.segments.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment type. + default_field: false + - name: elf.shared_libraries + level: extended + type: keyword + ignore_above: 1024 + description: List of shared libraries used by this ELF object. + default_field: false + - name: elf.telfhash + level: extended + type: keyword + ignore_above: 1024 + description: telfhash symbol hash for ELF file. + default_field: false - name: extension level: extended type: keyword @@ -2554,82 +2918,10 @@ this could be the container, for example, or other information meaningful in your environment.' - name: uptime - level: extended - type: long - description: Seconds the host has been up. - example: 1325 - - name: user.domain - level: extended - type: keyword - ignore_above: 1024 - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - - name: user.email - level: extended - type: keyword - ignore_above: 1024 - description: User email address. - - name: user.full_name - level: extended - type: keyword - ignore_above: 1024 - multi_fields: - - name: text - type: text - norms: false - default_field: false - description: User's full name, if available. - example: Albert Einstein - - name: user.group.domain - level: extended - type: keyword - ignore_above: 1024 - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - - name: user.group.id - level: extended - type: keyword - ignore_above: 1024 - description: Unique identifier for the group on the system/platform. - - name: user.group.name - level: extended - type: keyword - ignore_above: 1024 - description: Name of the group. - - name: user.hash - level: extended - type: keyword - ignore_above: 1024 - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - - name: user.id - level: core - type: keyword - ignore_above: 1024 - description: Unique identifier of the user. - - name: user.name - level: core - type: keyword - ignore_above: 1024 - multi_fields: - - name: text - type: text - norms: false - default_field: false - description: Short name or login of the user. - example: albert - - name: user.roles - level: extended - type: keyword - ignore_above: 1024 - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - default_field: false + level: extended + type: long + description: Seconds the host has been up. + example: 1325 - name: http title: HTTP group: 2 @@ -3812,6 +4104,185 @@ Some arguments may be filtered to protect sensitive information.' example: /usr/bin/ssh -l user 10.0.0.16 default_field: false + - name: elf.architecture + level: extended + type: keyword + ignore_above: 1024 + description: Machine architecture of the ELF file. + example: x86-64 + default_field: false + - name: elf.byte_order + level: extended + type: keyword + ignore_above: 1024 + description: Byte sequence of ELF file. + example: Little Endian + default_field: false + - name: elf.cpu_type + level: extended + type: keyword + ignore_above: 1024 + description: CPU type of the ELF file. + example: Intel + default_field: false + - name: elf.creation_date + level: extended + type: date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + default_field: false + - name: elf.exports + level: extended + type: flattened + description: List of exported element names and types. + default_field: false + - name: elf.header.abi_version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF Application Binary Interface (ABI). + default_field: false + - name: elf.header.class + level: extended + type: keyword + ignore_above: 1024 + description: Header class of the ELF file. + default_field: false + - name: elf.header.data + level: extended + type: keyword + ignore_above: 1024 + description: Data table of the ELF header. + default_field: false + - name: elf.header.entrypoint + level: extended + type: long + format: string + description: Header entrypoint of the ELF file. + default_field: false + - name: elf.header.object_version + level: extended + type: keyword + ignore_above: 1024 + description: '"0x1" for original ELF files.' + default_field: false + - name: elf.header.os_abi + level: extended + type: keyword + ignore_above: 1024 + description: Application Binary Interface (ABI) of the Linux OS. + default_field: false + - name: elf.header.type + level: extended + type: keyword + ignore_above: 1024 + description: Header type of the ELF file. + default_field: false + - name: elf.header.version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF header. + default_field: false + - name: elf.imports + level: extended + type: flattened + description: List of imported element names and types. + default_field: false + - name: elf.sections + level: extended + type: nested + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + default_field: false + - name: elf.sections.chi2 + level: extended + type: long + format: number + description: Chi-square probability distribution of the section. + default_field: false + - name: elf.sections.entropy + level: extended + type: long + format: number + description: Shannon entropy calculation from the section. + default_field: false + - name: elf.sections.flags + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List flags. + default_field: false + - name: elf.sections.name + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List name. + default_field: false + - name: elf.sections.physical_offset + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List offset. + default_field: false + - name: elf.sections.physical_size + level: extended + type: long + format: bytes + description: ELF Section List physical size. + default_field: false + - name: elf.sections.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List type. + default_field: false + - name: elf.sections.virtual_address + level: extended + type: long + format: string + description: ELF Section List virtual address. + default_field: false + - name: elf.sections.virtual_size + level: extended + type: long + format: string + description: ELF Section List virtual size. + default_field: false + - name: elf.segments + level: extended + type: nested + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + default_field: false + - name: elf.segments.sections + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment sections. + default_field: false + - name: elf.segments.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment type. + default_field: false + - name: elf.shared_libraries + level: extended + type: keyword + ignore_above: 1024 + description: List of shared libraries used by this ELF object. + default_field: false + - name: elf.telfhash + level: extended + type: keyword + ignore_above: 1024 + description: telfhash symbol hash for ELF file. + default_field: false - name: entity_id level: extended type: keyword @@ -3982,6 +4453,185 @@ Some arguments may be filtered to protect sensitive information.' example: /usr/bin/ssh -l user 10.0.0.16 default_field: false + - name: parent.elf.architecture + level: extended + type: keyword + ignore_above: 1024 + description: Machine architecture of the ELF file. + example: x86-64 + default_field: false + - name: parent.elf.byte_order + level: extended + type: keyword + ignore_above: 1024 + description: Byte sequence of ELF file. + example: Little Endian + default_field: false + - name: parent.elf.cpu_type + level: extended + type: keyword + ignore_above: 1024 + description: CPU type of the ELF file. + example: Intel + default_field: false + - name: parent.elf.creation_date + level: extended + type: date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + default_field: false + - name: parent.elf.exports + level: extended + type: flattened + description: List of exported element names and types. + default_field: false + - name: parent.elf.header.abi_version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF Application Binary Interface (ABI). + default_field: false + - name: parent.elf.header.class + level: extended + type: keyword + ignore_above: 1024 + description: Header class of the ELF file. + default_field: false + - name: parent.elf.header.data + level: extended + type: keyword + ignore_above: 1024 + description: Data table of the ELF header. + default_field: false + - name: parent.elf.header.entrypoint + level: extended + type: long + format: string + description: Header entrypoint of the ELF file. + default_field: false + - name: parent.elf.header.object_version + level: extended + type: keyword + ignore_above: 1024 + description: '"0x1" for original ELF files.' + default_field: false + - name: parent.elf.header.os_abi + level: extended + type: keyword + ignore_above: 1024 + description: Application Binary Interface (ABI) of the Linux OS. + default_field: false + - name: parent.elf.header.type + level: extended + type: keyword + ignore_above: 1024 + description: Header type of the ELF file. + default_field: false + - name: parent.elf.header.version + level: extended + type: keyword + ignore_above: 1024 + description: Version of the ELF header. + default_field: false + - name: parent.elf.imports + level: extended + type: flattened + description: List of imported element names and types. + default_field: false + - name: parent.elf.sections + level: extended + type: nested + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + default_field: false + - name: parent.elf.sections.chi2 + level: extended + type: long + format: number + description: Chi-square probability distribution of the section. + default_field: false + - name: parent.elf.sections.entropy + level: extended + type: long + format: number + description: Shannon entropy calculation from the section. + default_field: false + - name: parent.elf.sections.flags + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List flags. + default_field: false + - name: parent.elf.sections.name + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List name. + default_field: false + - name: parent.elf.sections.physical_offset + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List offset. + default_field: false + - name: parent.elf.sections.physical_size + level: extended + type: long + format: bytes + description: ELF Section List physical size. + default_field: false + - name: parent.elf.sections.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF Section List type. + default_field: false + - name: parent.elf.sections.virtual_address + level: extended + type: long + format: string + description: ELF Section List virtual address. + default_field: false + - name: parent.elf.sections.virtual_size + level: extended + type: long + format: string + description: ELF Section List virtual size. + default_field: false + - name: parent.elf.segments + level: extended + type: nested + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + default_field: false + - name: parent.elf.segments.sections + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment sections. + default_field: false + - name: parent.elf.segments.type + level: extended + type: keyword + ignore_above: 1024 + description: ELF object segment type. + default_field: false + - name: parent.elf.shared_libraries + level: extended + type: keyword + ignore_above: 1024 + description: List of shared libraries used by this ELF object. + default_field: false + - name: parent.elf.telfhash + level: extended + type: keyword + ignore_above: 1024 + description: telfhash symbol hash for ELF file. + default_field: false - name: parent.entity_id level: extended type: keyword @@ -4404,7 +5054,7 @@ level: extended type: keyword ignore_above: 1024 - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. default_field: false - name: rule title: Rule @@ -5172,6 +5822,89 @@ can be provided by detecting systems, evaluated at ingest time, or retrospectively tagged to events. example: MITRE ATT&CK + - name: group.alias + level: extended + type: keyword + ignore_above: 1024 + description: "The alias(es) of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group alias(es)." + example: '[ "Magecart Group 6" ]' + default_field: false + - name: group.id + level: extended + type: keyword + ignore_above: 1024 + description: "The id of the group for a set of related intrusion activity that\ + \ are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group id." + example: G0037 + default_field: false + - name: group.name + level: extended + type: keyword + ignore_above: 1024 + description: "The name of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group name." + example: FIN6 + default_field: false + - name: group.reference + level: extended + type: keyword + ignore_above: 1024 + description: "The reference URL of the group for a set of related intrusion\ + \ activity that are tracked by a common name in the security community. While\ + \ not required, you can use a MITRE ATT&CK\xAE group reference URL." + example: https://attack.mitre.org/groups/G0037/ + default_field: false + - name: software.id + level: extended + type: keyword + ignore_above: 1024 + description: "The id of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software id." + example: S0552 + default_field: false + - name: software.name + level: extended + type: keyword + ignore_above: 1024 + description: "The name of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software name." + example: AdFind + default_field: false + - name: software.platforms + level: extended + type: keyword + ignore_above: 1024 + description: "The platforms of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n \ + \ * Azure\n * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office\ + \ 365\n * SaaS\n * Windows" + example: '[ "Windows" ]' + default_field: false + - name: software.reference + level: extended + type: keyword + ignore_above: 1024 + description: "The reference URL of the software used by this threat to conduct\ + \ behavior commonly modeled using MITRE ATT&CK\xAE. While not required, you\ + \ can use a MITRE ATT&CK\xAE software reference URL." + example: https://attack.mitre.org/software/S0552/ + default_field: false + - name: software.type + level: extended + type: keyword + ignore_above: 1024 + description: "The type of software used by this threat to conduct behavior commonly\ + \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ + \ ATT&CK\xAE software type.\nRecommended values\n * Malware\n * Tool" + example: Tool + default_field: false - name: tactic.id level: extended type: keyword diff --git a/generated/csv/fields.csv b/generated/csv/fields.csv index 2ba05d75be..b2cbafd36b 100644 --- a/generated/csv/fields.csv +++ b/generated/csv/fields.csv @@ -190,6 +190,35 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,file,file.device,keyword,extended,,sda,Device that is the source of the file. 2.0.0-dev,true,file,file.directory,keyword,extended,,/home/alice,Directory where the file is located. 2.0.0-dev,true,file,file.drive_letter,keyword,extended,,C,Drive letter where the file is located. +2.0.0-dev,true,file,file.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. +2.0.0-dev,true,file,file.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. +2.0.0-dev,true,file,file.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. +2.0.0-dev,true,file,file.elf.creation_date,date,extended,,,Build or compile date. +2.0.0-dev,true,file,file.elf.exports,flattened,extended,array,,List of exported element names and types. +2.0.0-dev,true,file,file.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). +2.0.0-dev,true,file,file.elf.header.class,keyword,extended,,,Header class of the ELF file. +2.0.0-dev,true,file,file.elf.header.data,keyword,extended,,,Data table of the ELF header. +2.0.0-dev,true,file,file.elf.header.entrypoint,long,extended,,,Header entrypoint of the ELF file. +2.0.0-dev,true,file,file.elf.header.object_version,keyword,extended,,,"""0x1"" for original ELF files." +2.0.0-dev,true,file,file.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. +2.0.0-dev,true,file,file.elf.header.type,keyword,extended,,,Header type of the ELF file. +2.0.0-dev,true,file,file.elf.header.version,keyword,extended,,,Version of the ELF header. +2.0.0-dev,true,file,file.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev,true,file,file.elf.sections,nested,extended,array,,Section information of the ELF file. +2.0.0-dev,true,file,file.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. +2.0.0-dev,true,file,file.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. +2.0.0-dev,true,file,file.elf.sections.flags,keyword,extended,,,ELF Section List flags. +2.0.0-dev,true,file,file.elf.sections.name,keyword,extended,,,ELF Section List name. +2.0.0-dev,true,file,file.elf.sections.physical_offset,keyword,extended,,,ELF Section List offset. +2.0.0-dev,true,file,file.elf.sections.physical_size,long,extended,,,ELF Section List physical size. +2.0.0-dev,true,file,file.elf.sections.type,keyword,extended,,,ELF Section List type. +2.0.0-dev,true,file,file.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. +2.0.0-dev,true,file,file.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. +2.0.0-dev,true,file,file.elf.segments,nested,extended,array,,ELF object segment list. +2.0.0-dev,true,file,file.elf.segments.sections,keyword,extended,,,ELF object segment sections. +2.0.0-dev,true,file,file.elf.segments.type,keyword,extended,,,ELF object segment type. +2.0.0-dev,true,file,file.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev,true,file,file.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev,true,file,file.extension,keyword,extended,,png,"File extension, excluding the leading dot." 2.0.0-dev,true,file,file.gid,keyword,extended,,1001,Primary group ID (GID) of the file. 2.0.0-dev,true,file,file.group,keyword,extended,,alice,Primary group name of the file. @@ -281,18 +310,6 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,host,host.os.version,keyword,extended,,10.14.1,Operating system version as a raw string. 2.0.0-dev,true,host,host.type,keyword,core,,,Type of host. 2.0.0-dev,true,host,host.uptime,long,extended,,1325,Seconds the host has been up. -2.0.0-dev,true,host,host.user.domain,keyword,extended,,,Name of the directory the user is a member of. -2.0.0-dev,true,host,host.user.email,keyword,extended,,,User email address. -2.0.0-dev,true,host,host.user.full_name,keyword,extended,,Albert Einstein,"User's full name, if available." -2.0.0-dev,true,host,host.user.full_name.text,text,extended,,Albert Einstein,"User's full name, if available." -2.0.0-dev,true,host,host.user.group.domain,keyword,extended,,,Name of the directory the group is a member of. -2.0.0-dev,true,host,host.user.group.id,keyword,extended,,,Unique identifier for the group on the system/platform. -2.0.0-dev,true,host,host.user.group.name,keyword,extended,,,Name of the group. -2.0.0-dev,true,host,host.user.hash,keyword,extended,,,Unique user hash to correlate information for a user in anonymized form. -2.0.0-dev,true,host,host.user.id,keyword,core,,,Unique identifier of the user. -2.0.0-dev,true,host,host.user.name,keyword,core,,albert,Short name or login of the user. -2.0.0-dev,true,host,host.user.name.text,text,core,,albert,Short name or login of the user. -2.0.0-dev,true,host,host.user.roles,keyword,extended,array,"[""kibana_admin"", ""reporting_user""]",Array of user roles at the time of the event. 2.0.0-dev,true,http,http.request.body.bytes,long,extended,,887,Size in bytes of the request body. 2.0.0-dev,true,http,http.request.body.content,keyword,extended,,Hello world,The full HTTP request body. 2.0.0-dev,true,http,http.request.body.content.text,text,extended,,Hello world,The full HTTP request body. @@ -416,6 +433,35 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,process,process.code_signature.valid,boolean,extended,,true,Boolean to capture if the digital signature is verified against the binary content. 2.0.0-dev,true,process,process.command_line,keyword,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev,true,process,process.command_line.text,text,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. +2.0.0-dev,true,process,process.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. +2.0.0-dev,true,process,process.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. +2.0.0-dev,true,process,process.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. +2.0.0-dev,true,process,process.elf.creation_date,date,extended,,,Build or compile date. +2.0.0-dev,true,process,process.elf.exports,flattened,extended,array,,List of exported element names and types. +2.0.0-dev,true,process,process.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). +2.0.0-dev,true,process,process.elf.header.class,keyword,extended,,,Header class of the ELF file. +2.0.0-dev,true,process,process.elf.header.data,keyword,extended,,,Data table of the ELF header. +2.0.0-dev,true,process,process.elf.header.entrypoint,long,extended,,,Header entrypoint of the ELF file. +2.0.0-dev,true,process,process.elf.header.object_version,keyword,extended,,,"""0x1"" for original ELF files." +2.0.0-dev,true,process,process.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. +2.0.0-dev,true,process,process.elf.header.type,keyword,extended,,,Header type of the ELF file. +2.0.0-dev,true,process,process.elf.header.version,keyword,extended,,,Version of the ELF header. +2.0.0-dev,true,process,process.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev,true,process,process.elf.sections,nested,extended,array,,Section information of the ELF file. +2.0.0-dev,true,process,process.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. +2.0.0-dev,true,process,process.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. +2.0.0-dev,true,process,process.elf.sections.flags,keyword,extended,,,ELF Section List flags. +2.0.0-dev,true,process,process.elf.sections.name,keyword,extended,,,ELF Section List name. +2.0.0-dev,true,process,process.elf.sections.physical_offset,keyword,extended,,,ELF Section List offset. +2.0.0-dev,true,process,process.elf.sections.physical_size,long,extended,,,ELF Section List physical size. +2.0.0-dev,true,process,process.elf.sections.type,keyword,extended,,,ELF Section List type. +2.0.0-dev,true,process,process.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. +2.0.0-dev,true,process,process.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. +2.0.0-dev,true,process,process.elf.segments,nested,extended,array,,ELF object segment list. +2.0.0-dev,true,process,process.elf.segments.sections,keyword,extended,,,ELF object segment sections. +2.0.0-dev,true,process,process.elf.segments.type,keyword,extended,,,ELF object segment type. +2.0.0-dev,true,process,process.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev,true,process,process.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev,true,process,process.entity_id,keyword,extended,,c2c455d9f99375d,Unique identifier for the process. 2.0.0-dev,true,process,process.executable,keyword,extended,,/usr/bin/ssh,Absolute path to the process executable. 2.0.0-dev,true,process,process.executable.text,text,extended,,/usr/bin/ssh,Absolute path to the process executable. @@ -438,6 +484,35 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,process,process.parent.code_signature.valid,boolean,extended,,true,Boolean to capture if the digital signature is verified against the binary content. 2.0.0-dev,true,process,process.parent.command_line,keyword,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. 2.0.0-dev,true,process,process.parent.command_line.text,text,extended,,/usr/bin/ssh -l user 10.0.0.16,Full command line that started the process. +2.0.0-dev,true,process,process.parent.elf.architecture,keyword,extended,,x86-64,Machine architecture of the ELF file. +2.0.0-dev,true,process,process.parent.elf.byte_order,keyword,extended,,Little Endian,Byte sequence of ELF file. +2.0.0-dev,true,process,process.parent.elf.cpu_type,keyword,extended,,Intel,CPU type of the ELF file. +2.0.0-dev,true,process,process.parent.elf.creation_date,date,extended,,,Build or compile date. +2.0.0-dev,true,process,process.parent.elf.exports,flattened,extended,array,,List of exported element names and types. +2.0.0-dev,true,process,process.parent.elf.header.abi_version,keyword,extended,,,Version of the ELF Application Binary Interface (ABI). +2.0.0-dev,true,process,process.parent.elf.header.class,keyword,extended,,,Header class of the ELF file. +2.0.0-dev,true,process,process.parent.elf.header.data,keyword,extended,,,Data table of the ELF header. +2.0.0-dev,true,process,process.parent.elf.header.entrypoint,long,extended,,,Header entrypoint of the ELF file. +2.0.0-dev,true,process,process.parent.elf.header.object_version,keyword,extended,,,"""0x1"" for original ELF files." +2.0.0-dev,true,process,process.parent.elf.header.os_abi,keyword,extended,,,Application Binary Interface (ABI) of the Linux OS. +2.0.0-dev,true,process,process.parent.elf.header.type,keyword,extended,,,Header type of the ELF file. +2.0.0-dev,true,process,process.parent.elf.header.version,keyword,extended,,,Version of the ELF header. +2.0.0-dev,true,process,process.parent.elf.imports,flattened,extended,array,,List of imported element names and types. +2.0.0-dev,true,process,process.parent.elf.sections,nested,extended,array,,Section information of the ELF file. +2.0.0-dev,true,process,process.parent.elf.sections.chi2,long,extended,,,Chi-square probability distribution of the section. +2.0.0-dev,true,process,process.parent.elf.sections.entropy,long,extended,,,Shannon entropy calculation from the section. +2.0.0-dev,true,process,process.parent.elf.sections.flags,keyword,extended,,,ELF Section List flags. +2.0.0-dev,true,process,process.parent.elf.sections.name,keyword,extended,,,ELF Section List name. +2.0.0-dev,true,process,process.parent.elf.sections.physical_offset,keyword,extended,,,ELF Section List offset. +2.0.0-dev,true,process,process.parent.elf.sections.physical_size,long,extended,,,ELF Section List physical size. +2.0.0-dev,true,process,process.parent.elf.sections.type,keyword,extended,,,ELF Section List type. +2.0.0-dev,true,process,process.parent.elf.sections.virtual_address,long,extended,,,ELF Section List virtual address. +2.0.0-dev,true,process,process.parent.elf.sections.virtual_size,long,extended,,,ELF Section List virtual size. +2.0.0-dev,true,process,process.parent.elf.segments,nested,extended,array,,ELF object segment list. +2.0.0-dev,true,process,process.parent.elf.segments.sections,keyword,extended,,,ELF object segment sections. +2.0.0-dev,true,process,process.parent.elf.segments.type,keyword,extended,,,ELF object segment type. +2.0.0-dev,true,process,process.parent.elf.shared_libraries,keyword,extended,array,,List of shared libraries used by this ELF object. +2.0.0-dev,true,process,process.parent.elf.telfhash,keyword,extended,,,telfhash hash for ELF file. 2.0.0-dev,true,process,process.parent.entity_id,keyword,extended,,c2c455d9f99375d,Unique identifier for the process. 2.0.0-dev,true,process,process.parent.executable,keyword,extended,,/usr/bin/ssh,Absolute path to the process executable. 2.0.0-dev,true,process,process.parent.executable.text,text,extended,,/usr/bin/ssh,Absolute path to the process executable. @@ -495,7 +570,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,related,related.hash,keyword,extended,array,,All the hashes seen on your event. 2.0.0-dev,true,related,related.hosts,keyword,extended,array,,All the host identifiers seen on your event. 2.0.0-dev,true,related,related.ip,ip,extended,array,,All of the IPs seen on your event. -2.0.0-dev,true,related,related.user,keyword,extended,array,,All the user names seen on your event. +2.0.0-dev,true,related,related.user,keyword,extended,array,,All the user names or other user identifiers seen on the event. 2.0.0-dev,true,rule,rule.author,keyword,extended,array,"[""Star-Lord""]",Rule author 2.0.0-dev,true,rule,rule.category,keyword,extended,,Attempted Information Leak,Rule category 2.0.0-dev,true,rule,rule.description,keyword,extended,,Block requests to public DNS over HTTPS / TLS protocols,Rule description @@ -591,6 +666,15 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 2.0.0-dev,true,source,source.user.roles,keyword,extended,array,"[""kibana_admin"", ""reporting_user""]",Array of user roles at the time of the event. 2.0.0-dev,true,span,span.id,keyword,extended,,3ff9a8981b7ccd5a,Unique identifier of the span within the scope of its trace. 2.0.0-dev,true,threat,threat.framework,keyword,extended,,MITRE ATT&CK,Threat classification framework. +2.0.0-dev,true,threat,threat.group.alias,keyword,extended,array,"[ ""Magecart Group 6"" ]",Alias of the group. +2.0.0-dev,true,threat,threat.group.id,keyword,extended,,G0037,ID of the group. +2.0.0-dev,true,threat,threat.group.name,keyword,extended,,FIN6,Name of the group. +2.0.0-dev,true,threat,threat.group.reference,keyword,extended,,https://attack.mitre.org/groups/G0037/,Reference URL of the group. +2.0.0-dev,true,threat,threat.software.id,keyword,extended,,S0552,ID of the software +2.0.0-dev,true,threat,threat.software.name,keyword,extended,,AdFind,Name of the software. +2.0.0-dev,true,threat,threat.software.platforms,keyword,extended,array,"[ ""Windows"" ]",Platforms of the software. +2.0.0-dev,true,threat,threat.software.reference,keyword,extended,,https://attack.mitre.org/software/S0552/,Software reference URL. +2.0.0-dev,true,threat,threat.software.type,keyword,extended,,Tool,Software type. 2.0.0-dev,true,threat,threat.tactic.id,keyword,extended,array,TA0002,Threat tactic id. 2.0.0-dev,true,threat,threat.tactic.name,keyword,extended,array,Execution,Threat tactic. 2.0.0-dev,true,threat,threat.tactic.reference,keyword,extended,array,https://attack.mitre.org/tactics/TA0002/,Threat tactic URL reference. diff --git a/generated/ecs/ecs_flat.yml b/generated/ecs/ecs_flat.yml index 3e8772e2b0..9e609566c9 100644 --- a/generated/ecs/ecs_flat.yml +++ b/generated/ecs/ecs_flat.yml @@ -2782,6 +2782,335 @@ file.drive_letter: normalize: [] short: Drive letter where the file is located. type: keyword +file.elf.architecture: + dashed_name: file-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: file.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword +file.elf.byte_order: + dashed_name: file-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: file.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword +file.elf.cpu_type: + dashed_name: file-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: file.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword +file.elf.creation_date: + dashed_name: file-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when it + was built or compiled. It can also be faked by malware creators. + flat_name: file.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date +file.elf.exports: + dashed_name: file-elf-exports + description: List of exported element names and types. + flat_name: file.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened +file.elf.header.abi_version: + dashed_name: file-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: file.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword +file.elf.header.class: + dashed_name: file-elf-header-class + description: Header class of the ELF file. + flat_name: file.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword +file.elf.header.data: + dashed_name: file-elf-header-data + description: Data table of the ELF header. + flat_name: file.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword +file.elf.header.entrypoint: + dashed_name: file-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: file.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long +file.elf.header.object_version: + dashed_name: file-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: file.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword +file.elf.header.os_abi: + dashed_name: file-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: file.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword +file.elf.header.type: + dashed_name: file-elf-header-type + description: Header type of the ELF file. + flat_name: file.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword +file.elf.header.version: + dashed_name: file-elf-header-version + description: Version of the ELF header. + flat_name: file.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword +file.elf.imports: + dashed_name: file-elf-imports + description: List of imported element names and types. + flat_name: file.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened +file.elf.sections: + dashed_name: file-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' + flat_name: file.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested +file.elf.sections.chi2: + dashed_name: file-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: file.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long +file.elf.sections.entropy: + dashed_name: file-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: file.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long +file.elf.sections.flags: + dashed_name: file-elf-sections-flags + description: ELF Section List flags. + flat_name: file.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword +file.elf.sections.name: + dashed_name: file-elf-sections-name + description: ELF Section List name. + flat_name: file.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword +file.elf.sections.physical_offset: + dashed_name: file-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: file.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword +file.elf.sections.physical_size: + dashed_name: file-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: file.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long +file.elf.sections.type: + dashed_name: file-elf-sections-type + description: ELF Section List type. + flat_name: file.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword +file.elf.sections.virtual_address: + dashed_name: file-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: file.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long +file.elf.sections.virtual_size: + dashed_name: file-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: file.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long +file.elf.segments: + dashed_name: file-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' + flat_name: file.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested +file.elf.segments.sections: + dashed_name: file-elf-segments-sections + description: ELF object segment sections. + flat_name: file.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword +file.elf.segments.type: + dashed_name: file-elf-segments-type + description: ELF object segment type. + flat_name: file.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword +file.elf.shared_libraries: + dashed_name: file-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: file.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword +file.elf.telfhash: + dashed_name: file-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: file.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword file.extension: dashed_name: file-extension description: 'File extension, excluding the leading dot. @@ -3861,138 +4190,6 @@ host.uptime: normalize: [] short: Seconds the host has been up. type: long -host.user.domain: - dashed_name: host-user-domain - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: user - short: Name of the directory the user is a member of. - type: keyword -host.user.email: - dashed_name: host-user-email - description: User email address. - flat_name: host.user.email - ignore_above: 1024 - level: extended - name: email - normalize: [] - original_fieldset: user - short: User email address. - type: keyword -host.user.full_name: - dashed_name: host-user-full-name - description: User's full name, if available. - example: Albert Einstein - flat_name: host.user.full_name - ignore_above: 1024 - level: extended - multi_fields: - - flat_name: host.user.full_name.text - name: text - norms: false - type: text - name: full_name - normalize: [] - original_fieldset: user - short: User's full name, if available. - type: keyword -host.user.group.domain: - dashed_name: host-user-group-domain - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.group.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: group - short: Name of the directory the group is a member of. - type: keyword -host.user.group.id: - dashed_name: host-user-group-id - description: Unique identifier for the group on the system/platform. - flat_name: host.user.group.id - ignore_above: 1024 - level: extended - name: id - normalize: [] - original_fieldset: group - short: Unique identifier for the group on the system/platform. - type: keyword -host.user.group.name: - dashed_name: host-user-group-name - description: Name of the group. - flat_name: host.user.group.name - ignore_above: 1024 - level: extended - name: name - normalize: [] - original_fieldset: group - short: Name of the group. - type: keyword -host.user.hash: - dashed_name: host-user-hash - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - flat_name: host.user.hash - ignore_above: 1024 - level: extended - name: hash - normalize: [] - original_fieldset: user - short: Unique user hash to correlate information for a user in anonymized form. - type: keyword -host.user.id: - dashed_name: host-user-id - description: Unique identifier of the user. - flat_name: host.user.id - ignore_above: 1024 - level: core - name: id - normalize: [] - original_fieldset: user - short: Unique identifier of the user. - type: keyword -host.user.name: - dashed_name: host-user-name - description: Short name or login of the user. - example: albert - flat_name: host.user.name - ignore_above: 1024 - level: core - multi_fields: - - flat_name: host.user.name.text - name: text - norms: false - type: text - name: name - normalize: [] - original_fieldset: user - short: Short name or login of the user. - type: keyword -host.user.roles: - dashed_name: host-user-roles - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - flat_name: host.user.roles - ignore_above: 1024 - level: extended - name: roles - normalize: - - array - original_fieldset: user - short: Array of user roles at the time of the event. - type: keyword http.request.body.bytes: dashed_name: http-request-body-bytes description: Size in bytes of the request body. @@ -5534,6 +5731,335 @@ process.command_line: normalize: [] short: Full command line that started the process. type: keyword +process.elf.architecture: + dashed_name: process-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: process.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword +process.elf.byte_order: + dashed_name: process-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: process.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword +process.elf.cpu_type: + dashed_name: process-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: process.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword +process.elf.creation_date: + dashed_name: process-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when it + was built or compiled. It can also be faked by malware creators. + flat_name: process.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date +process.elf.exports: + dashed_name: process-elf-exports + description: List of exported element names and types. + flat_name: process.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened +process.elf.header.abi_version: + dashed_name: process-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: process.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword +process.elf.header.class: + dashed_name: process-elf-header-class + description: Header class of the ELF file. + flat_name: process.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword +process.elf.header.data: + dashed_name: process-elf-header-data + description: Data table of the ELF header. + flat_name: process.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword +process.elf.header.entrypoint: + dashed_name: process-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: process.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long +process.elf.header.object_version: + dashed_name: process-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: process.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword +process.elf.header.os_abi: + dashed_name: process-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: process.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword +process.elf.header.type: + dashed_name: process-elf-header-type + description: Header type of the ELF file. + flat_name: process.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword +process.elf.header.version: + dashed_name: process-elf-header-version + description: Version of the ELF header. + flat_name: process.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword +process.elf.imports: + dashed_name: process-elf-imports + description: List of imported element names and types. + flat_name: process.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened +process.elf.sections: + dashed_name: process-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' + flat_name: process.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested +process.elf.sections.chi2: + dashed_name: process-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: process.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long +process.elf.sections.entropy: + dashed_name: process-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: process.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long +process.elf.sections.flags: + dashed_name: process-elf-sections-flags + description: ELF Section List flags. + flat_name: process.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword +process.elf.sections.name: + dashed_name: process-elf-sections-name + description: ELF Section List name. + flat_name: process.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword +process.elf.sections.physical_offset: + dashed_name: process-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: process.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword +process.elf.sections.physical_size: + dashed_name: process-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: process.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long +process.elf.sections.type: + dashed_name: process-elf-sections-type + description: ELF Section List type. + flat_name: process.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword +process.elf.sections.virtual_address: + dashed_name: process-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: process.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long +process.elf.sections.virtual_size: + dashed_name: process-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: process.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long +process.elf.segments: + dashed_name: process-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' + flat_name: process.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested +process.elf.segments.sections: + dashed_name: process-elf-segments-sections + description: ELF object segment sections. + flat_name: process.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword +process.elf.segments.type: + dashed_name: process-elf-segments-type + description: ELF object segment type. + flat_name: process.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword +process.elf.shared_libraries: + dashed_name: process-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: process.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword +process.elf.telfhash: + dashed_name: process-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: process.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword process.entity_id: dashed_name: process-entity-id description: 'Unique identifier for the process. @@ -5804,6 +6330,335 @@ process.parent.command_line: original_fieldset: process short: Full command line that started the process. type: keyword +process.parent.elf.architecture: + dashed_name: process-parent-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: process.parent.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword +process.parent.elf.byte_order: + dashed_name: process-parent-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: process.parent.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword +process.parent.elf.cpu_type: + dashed_name: process-parent-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: process.parent.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword +process.parent.elf.creation_date: + dashed_name: process-parent-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when it + was built or compiled. It can also be faked by malware creators. + flat_name: process.parent.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date +process.parent.elf.exports: + dashed_name: process-parent-elf-exports + description: List of exported element names and types. + flat_name: process.parent.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened +process.parent.elf.header.abi_version: + dashed_name: process-parent-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: process.parent.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword +process.parent.elf.header.class: + dashed_name: process-parent-elf-header-class + description: Header class of the ELF file. + flat_name: process.parent.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword +process.parent.elf.header.data: + dashed_name: process-parent-elf-header-data + description: Data table of the ELF header. + flat_name: process.parent.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword +process.parent.elf.header.entrypoint: + dashed_name: process-parent-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: process.parent.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long +process.parent.elf.header.object_version: + dashed_name: process-parent-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: process.parent.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword +process.parent.elf.header.os_abi: + dashed_name: process-parent-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: process.parent.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword +process.parent.elf.header.type: + dashed_name: process-parent-elf-header-type + description: Header type of the ELF file. + flat_name: process.parent.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword +process.parent.elf.header.version: + dashed_name: process-parent-elf-header-version + description: Version of the ELF header. + flat_name: process.parent.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword +process.parent.elf.imports: + dashed_name: process-parent-elf-imports + description: List of imported element names and types. + flat_name: process.parent.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened +process.parent.elf.sections: + dashed_name: process-parent-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.sections.*`.' + flat_name: process.parent.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested +process.parent.elf.sections.chi2: + dashed_name: process-parent-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: process.parent.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long +process.parent.elf.sections.entropy: + dashed_name: process-parent-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: process.parent.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long +process.parent.elf.sections.flags: + dashed_name: process-parent-elf-sections-flags + description: ELF Section List flags. + flat_name: process.parent.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword +process.parent.elf.sections.name: + dashed_name: process-parent-elf-sections-name + description: ELF Section List name. + flat_name: process.parent.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword +process.parent.elf.sections.physical_offset: + dashed_name: process-parent-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: process.parent.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword +process.parent.elf.sections.physical_size: + dashed_name: process-parent-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: process.parent.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long +process.parent.elf.sections.type: + dashed_name: process-parent-elf-sections-type + description: ELF Section List type. + flat_name: process.parent.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword +process.parent.elf.sections.virtual_address: + dashed_name: process-parent-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: process.parent.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long +process.parent.elf.sections.virtual_size: + dashed_name: process-parent-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: process.parent.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long +process.parent.elf.segments: + dashed_name: process-parent-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields underneath + `elf.segments.*`.' + flat_name: process.parent.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested +process.parent.elf.segments.sections: + dashed_name: process-parent-elf-segments-sections + description: ELF object segment sections. + flat_name: process.parent.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword +process.parent.elf.segments.type: + dashed_name: process-parent-elf-segments-type + description: ELF object segment type. + flat_name: process.parent.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword +process.parent.elf.shared_libraries: + dashed_name: process-parent-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: process.parent.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword +process.parent.elf.telfhash: + dashed_name: process-parent-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: process.parent.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword process.parent.entity_id: dashed_name: process-parent-entity-id description: 'Unique identifier for the process. @@ -6456,14 +7311,14 @@ related.ip: type: ip related.user: dashed_name: related-user - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. flat_name: related.user ignore_above: 1024 level: extended name: user normalize: - array - short: All the user names seen on your event. + short: All the user names or other user identifiers seen on the event. type: keyword rule.author: dashed_name: rule-author @@ -7655,6 +8510,136 @@ threat.framework: normalize: [] short: Threat classification framework. type: keyword +threat.group.alias: + beta: This field is beta and subject to change. + dashed_name: threat-group-alias + description: "The alias(es) of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group alias(es)." + example: '[ "Magecart Group 6" ]' + flat_name: threat.group.alias + ignore_above: 1024 + level: extended + name: group.alias + normalize: + - array + short: Alias of the group. + type: keyword +threat.group.id: + beta: This field is beta and subject to change. + dashed_name: threat-group-id + description: "The id of the group for a set of related intrusion activity that are\ + \ tracked by a common name in the security community. While not required, you\ + \ can use a MITRE ATT&CK\xAE group id." + example: G0037 + flat_name: threat.group.id + ignore_above: 1024 + level: extended + name: group.id + normalize: [] + short: ID of the group. + type: keyword +threat.group.name: + beta: This field is beta and subject to change. + dashed_name: threat-group-name + description: "The name of the group for a set of related intrusion activity that\ + \ are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group name." + example: FIN6 + flat_name: threat.group.name + ignore_above: 1024 + level: extended + name: group.name + normalize: [] + short: Name of the group. + type: keyword +threat.group.reference: + beta: This field is beta and subject to change. + dashed_name: threat-group-reference + description: "The reference URL of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group reference URL." + example: https://attack.mitre.org/groups/G0037/ + flat_name: threat.group.reference + ignore_above: 1024 + level: extended + name: group.reference + normalize: [] + short: Reference URL of the group. + type: keyword +threat.software.id: + beta: This field is beta and subject to change. + dashed_name: threat-software-id + description: "The id of the software used by this threat to conduct behavior commonly\ + \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ + \ software id." + example: S0552 + flat_name: threat.software.id + ignore_above: 1024 + level: extended + name: software.id + normalize: [] + short: ID of the software + type: keyword +threat.software.name: + beta: This field is beta and subject to change. + dashed_name: threat-software-name + description: "The name of the software used by this threat to conduct behavior commonly\ + \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ + \ software name." + example: AdFind + flat_name: threat.software.name + ignore_above: 1024 + level: extended + name: software.name + normalize: [] + short: Name of the software. + type: keyword +threat.software.platforms: + beta: This field is beta and subject to change. + dashed_name: threat-software-platforms + description: "The platforms of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ + \ ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n * Azure\n *\ + \ Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office 365\n * SaaS\n\ + \ * Windows" + example: '[ "Windows" ]' + flat_name: threat.software.platforms + ignore_above: 1024 + level: extended + name: software.platforms + normalize: + - array + short: Platforms of the software. + type: keyword +threat.software.reference: + beta: This field is beta and subject to change. + dashed_name: threat-software-reference + description: "The reference URL of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ + \ ATT&CK\xAE software reference URL." + example: https://attack.mitre.org/software/S0552/ + flat_name: threat.software.reference + ignore_above: 1024 + level: extended + name: software.reference + normalize: [] + short: Software reference URL. + type: keyword +threat.software.type: + beta: This field is beta and subject to change. + dashed_name: threat-software-type + description: "The type of software used by this threat to conduct behavior commonly\ + \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE ATT&CK\xAE\ + \ software type.\nRecommended values\n * Malware\n * Tool" + example: Tool + flat_name: threat.software.type + ignore_above: 1024 + level: extended + name: software.type + normalize: [] + short: Software type. + type: keyword threat.tactic.id: dashed_name: threat-tactic-id description: "The id of tactic used by this threat. You can use a MITRE ATT&CK\xAE\ diff --git a/generated/ecs/ecs_nested.yml b/generated/ecs/ecs_nested.yml index 5c3f2fac5a..b973815a12 100644 --- a/generated/ecs/ecs_nested.yml +++ b/generated/ecs/ecs_nested.yml @@ -2213,6 +2213,327 @@ ecs: short: Meta-information specific to ECS. title: ECS type: group +elf: + beta: These fields are in beta and are subject to change. + description: These fields contain Linux Executable Linkable Format (ELF) metadata. + fields: + elf.architecture: + dashed_name: elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + short: Machine architecture of the ELF file. + type: keyword + elf.byte_order: + dashed_name: elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + short: Byte sequence of ELF file. + type: keyword + elf.cpu_type: + dashed_name: elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + short: CPU type of the ELF file. + type: keyword + elf.creation_date: + dashed_name: elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + flat_name: elf.creation_date + level: extended + name: creation_date + normalize: [] + short: Build or compile date. + type: date + elf.exports: + dashed_name: elf-exports + description: List of exported element names and types. + flat_name: elf.exports + level: extended + name: exports + normalize: + - array + short: List of exported element names and types. + type: flattened + elf.header.abi_version: + dashed_name: elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + short: Version of the ELF Application Binary Interface (ABI). + type: keyword + elf.header.class: + dashed_name: elf-header-class + description: Header class of the ELF file. + flat_name: elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + short: Header class of the ELF file. + type: keyword + elf.header.data: + dashed_name: elf-header-data + description: Data table of the ELF header. + flat_name: elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + short: Data table of the ELF header. + type: keyword + elf.header.entrypoint: + dashed_name: elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + short: Header entrypoint of the ELF file. + type: long + elf.header.object_version: + dashed_name: elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + short: '"0x1" for original ELF files.' + type: keyword + elf.header.os_abi: + dashed_name: elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword + elf.header.type: + dashed_name: elf-header-type + description: Header type of the ELF file. + flat_name: elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + short: Header type of the ELF file. + type: keyword + elf.header.version: + dashed_name: elf-header-version + description: Version of the ELF header. + flat_name: elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + short: Version of the ELF header. + type: keyword + elf.imports: + dashed_name: elf-imports + description: List of imported element names and types. + flat_name: elf.imports + level: extended + name: imports + normalize: + - array + short: List of imported element names and types. + type: flattened + elf.sections: + dashed_name: elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + flat_name: elf.sections + level: extended + name: sections + normalize: + - array + short: Section information of the ELF file. + type: nested + elf.sections.chi2: + dashed_name: elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + short: Chi-square probability distribution of the section. + type: long + elf.sections.entropy: + dashed_name: elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + short: Shannon entropy calculation from the section. + type: long + elf.sections.flags: + dashed_name: elf-sections-flags + description: ELF Section List flags. + flat_name: elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + short: ELF Section List flags. + type: keyword + elf.sections.name: + dashed_name: elf-sections-name + description: ELF Section List name. + flat_name: elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + short: ELF Section List name. + type: keyword + elf.sections.physical_offset: + dashed_name: elf-sections-physical-offset + description: ELF Section List offset. + flat_name: elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + short: ELF Section List offset. + type: keyword + elf.sections.physical_size: + dashed_name: elf-sections-physical-size + description: ELF Section List physical size. + flat_name: elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + short: ELF Section List physical size. + type: long + elf.sections.type: + dashed_name: elf-sections-type + description: ELF Section List type. + flat_name: elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + short: ELF Section List type. + type: keyword + elf.sections.virtual_address: + dashed_name: elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + short: ELF Section List virtual address. + type: long + elf.sections.virtual_size: + dashed_name: elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + short: ELF Section List virtual size. + type: long + elf.segments: + dashed_name: elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + flat_name: elf.segments + level: extended + name: segments + normalize: + - array + short: ELF object segment list. + type: nested + elf.segments.sections: + dashed_name: elf-segments-sections + description: ELF object segment sections. + flat_name: elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + short: ELF object segment sections. + type: keyword + elf.segments.type: + dashed_name: elf-segments-type + description: ELF object segment type. + flat_name: elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + short: ELF object segment type. + type: keyword + elf.shared_libraries: + dashed_name: elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + short: List of shared libraries used by this ELF object. + type: keyword + elf.telfhash: + dashed_name: elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + short: telfhash hash for ELF file. + type: keyword + group: 2 + name: elf + prefix: elf. + reusable: + expected: + - as: elf + at: file + beta: This field reuse is beta and subject to change. + full: file.elf + - as: elf + at: process + beta: This field reuse is beta and subject to change. + full: process.elf + top_level: false + short: These fields contain Linux Executable Linkable Format (ELF) metadata. + title: ELF Header + type: group error: description: 'These fields can represent errors of any kind. @@ -3260,6 +3581,335 @@ file: normalize: [] short: Drive letter where the file is located. type: keyword + file.elf.architecture: + dashed_name: file-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: file.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword + file.elf.byte_order: + dashed_name: file-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: file.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword + file.elf.cpu_type: + dashed_name: file-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: file.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword + file.elf.creation_date: + dashed_name: file-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + flat_name: file.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date + file.elf.exports: + dashed_name: file-elf-exports + description: List of exported element names and types. + flat_name: file.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened + file.elf.header.abi_version: + dashed_name: file-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: file.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword + file.elf.header.class: + dashed_name: file-elf-header-class + description: Header class of the ELF file. + flat_name: file.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword + file.elf.header.data: + dashed_name: file-elf-header-data + description: Data table of the ELF header. + flat_name: file.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword + file.elf.header.entrypoint: + dashed_name: file-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: file.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long + file.elf.header.object_version: + dashed_name: file-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: file.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword + file.elf.header.os_abi: + dashed_name: file-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: file.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword + file.elf.header.type: + dashed_name: file-elf-header-type + description: Header type of the ELF file. + flat_name: file.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword + file.elf.header.version: + dashed_name: file-elf-header-version + description: Version of the ELF header. + flat_name: file.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword + file.elf.imports: + dashed_name: file-elf-imports + description: List of imported element names and types. + flat_name: file.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened + file.elf.sections: + dashed_name: file-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + flat_name: file.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested + file.elf.sections.chi2: + dashed_name: file-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: file.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long + file.elf.sections.entropy: + dashed_name: file-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: file.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long + file.elf.sections.flags: + dashed_name: file-elf-sections-flags + description: ELF Section List flags. + flat_name: file.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword + file.elf.sections.name: + dashed_name: file-elf-sections-name + description: ELF Section List name. + flat_name: file.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword + file.elf.sections.physical_offset: + dashed_name: file-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: file.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword + file.elf.sections.physical_size: + dashed_name: file-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: file.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long + file.elf.sections.type: + dashed_name: file-elf-sections-type + description: ELF Section List type. + flat_name: file.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword + file.elf.sections.virtual_address: + dashed_name: file-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: file.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long + file.elf.sections.virtual_size: + dashed_name: file-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: file.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long + file.elf.segments: + dashed_name: file-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + flat_name: file.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested + file.elf.segments.sections: + dashed_name: file-elf-segments-sections + description: ELF object segment sections. + flat_name: file.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword + file.elf.segments.type: + dashed_name: file-elf-segments-type + description: ELF object segment type. + flat_name: file.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword + file.elf.shared_libraries: + dashed_name: file-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: file.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword + file.elf.telfhash: + dashed_name: file-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: file.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword file.extension: dashed_name: file-extension description: 'File extension, excluding the leading dot. @@ -3881,6 +4531,7 @@ file: name: file nestings: - file.code_signature + - file.elf - file.hash - file.pe - file.x509 @@ -3889,6 +4540,10 @@ file: - full: file.code_signature schema_name: code_signature short: These fields contain information about binary code signatures. + - beta: This field reuse is beta and subject to change. + full: file.elf + schema_name: elf + short: These fields contain Linux Executable Linkable Format (ELF) metadata. - full: file.hash schema_name: hash short: Hashes, usually file hashes. @@ -4634,144 +5289,11 @@ host: normalize: [] short: Seconds the host has been up. type: long - host.user.domain: - dashed_name: host-user-domain - description: 'Name of the directory the user is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: user - short: Name of the directory the user is a member of. - type: keyword - host.user.email: - dashed_name: host-user-email - description: User email address. - flat_name: host.user.email - ignore_above: 1024 - level: extended - name: email - normalize: [] - original_fieldset: user - short: User email address. - type: keyword - host.user.full_name: - dashed_name: host-user-full-name - description: User's full name, if available. - example: Albert Einstein - flat_name: host.user.full_name - ignore_above: 1024 - level: extended - multi_fields: - - flat_name: host.user.full_name.text - name: text - norms: false - type: text - name: full_name - normalize: [] - original_fieldset: user - short: User's full name, if available. - type: keyword - host.user.group.domain: - dashed_name: host-user-group-domain - description: 'Name of the directory the group is a member of. - - For example, an LDAP or Active Directory domain name.' - flat_name: host.user.group.domain - ignore_above: 1024 - level: extended - name: domain - normalize: [] - original_fieldset: group - short: Name of the directory the group is a member of. - type: keyword - host.user.group.id: - dashed_name: host-user-group-id - description: Unique identifier for the group on the system/platform. - flat_name: host.user.group.id - ignore_above: 1024 - level: extended - name: id - normalize: [] - original_fieldset: group - short: Unique identifier for the group on the system/platform. - type: keyword - host.user.group.name: - dashed_name: host-user-group-name - description: Name of the group. - flat_name: host.user.group.name - ignore_above: 1024 - level: extended - name: name - normalize: [] - original_fieldset: group - short: Name of the group. - type: keyword - host.user.hash: - dashed_name: host-user-hash - description: 'Unique user hash to correlate information for a user in anonymized - form. - - Useful if `user.id` or `user.name` contain confidential information and cannot - be used.' - flat_name: host.user.hash - ignore_above: 1024 - level: extended - name: hash - normalize: [] - original_fieldset: user - short: Unique user hash to correlate information for a user in anonymized form. - type: keyword - host.user.id: - dashed_name: host-user-id - description: Unique identifier of the user. - flat_name: host.user.id - ignore_above: 1024 - level: core - name: id - normalize: [] - original_fieldset: user - short: Unique identifier of the user. - type: keyword - host.user.name: - dashed_name: host-user-name - description: Short name or login of the user. - example: albert - flat_name: host.user.name - ignore_above: 1024 - level: core - multi_fields: - - flat_name: host.user.name.text - name: text - norms: false - type: text - name: name - normalize: [] - original_fieldset: user - short: Short name or login of the user. - type: keyword - host.user.roles: - dashed_name: host-user-roles - description: Array of user roles at the time of the event. - example: '["kibana_admin", "reporting_user"]' - flat_name: host.user.roles - ignore_above: 1024 - level: extended - name: roles - normalize: - - array - original_fieldset: user - short: Array of user roles at the time of the event. - type: keyword group: 2 name: host nestings: - host.geo - host.os - - host.user prefix: host. reused_here: - full: host.geo @@ -4780,9 +5302,6 @@ host: - full: host.os schema_name: os short: OS fields contain information about the operating system. - - full: host.user - schema_name: user - short: Fields to describe the user relevant to the event. short: Fields describing the relevant computing instance. title: Host type: group @@ -6043,7 +6562,6 @@ observer: title: Observer type: group orchestrator: - beta: These fields are in beta and are subject to change. description: Fields that describe the resources which container orchestrators manage or act upon. fields: @@ -6626,101 +7144,430 @@ process: flat_name: process.code_signature.signing_id ignore_above: 1024 level: extended - name: signing_id + name: signing_id + normalize: [] + original_fieldset: code_signature + short: The identifier used to sign the process. + type: keyword + process.code_signature.status: + dashed_name: process-code-signature-status + description: 'Additional information about the certificate status. + + This is useful for logging cryptographic errors with the certificate validity + or trust status. Leave unpopulated if the validity or trust of the certificate + was unchecked.' + example: ERROR_UNTRUSTED_ROOT + flat_name: process.code_signature.status + ignore_above: 1024 + level: extended + name: status + normalize: [] + original_fieldset: code_signature + short: Additional information about the certificate status. + type: keyword + process.code_signature.subject_name: + dashed_name: process-code-signature-subject-name + description: Subject name of the code signer + example: Microsoft Corporation + flat_name: process.code_signature.subject_name + ignore_above: 1024 + level: core + name: subject_name + normalize: [] + original_fieldset: code_signature + short: Subject name of the code signer + type: keyword + process.code_signature.team_id: + dashed_name: process-code-signature-team-id + description: 'The team identifier used to sign the process. + + This is used to identify the team or vendor of a software product. The field + is relevant to Apple *OS only.' + example: EQHXZ8M8AV + flat_name: process.code_signature.team_id + ignore_above: 1024 + level: extended + name: team_id + normalize: [] + original_fieldset: code_signature + short: The team identifier used to sign the process. + type: keyword + process.code_signature.trusted: + dashed_name: process-code-signature-trusted + description: 'Stores the trust status of the certificate chain. + + Validating the trust of the certificate chain may be complicated, and this + field should only be populated by tools that actively check the status.' + example: 'true' + flat_name: process.code_signature.trusted + level: extended + name: trusted + normalize: [] + original_fieldset: code_signature + short: Stores the trust status of the certificate chain. + type: boolean + process.code_signature.valid: + dashed_name: process-code-signature-valid + description: 'Boolean to capture if the digital signature is verified against + the binary content. + + Leave unpopulated if a certificate was unchecked.' + example: 'true' + flat_name: process.code_signature.valid + level: extended + name: valid + normalize: [] + original_fieldset: code_signature + short: Boolean to capture if the digital signature is verified against the binary + content. + type: boolean + process.command_line: + dashed_name: process-command-line + description: 'Full command line that started the process, including the absolute + path to the executable, and all arguments. + + Some arguments may be filtered to protect sensitive information.' + example: /usr/bin/ssh -l user 10.0.0.16 + flat_name: process.command_line + ignore_above: 1024 + level: extended + multi_fields: + - flat_name: process.command_line.text + name: text + norms: false + type: text + name: command_line + normalize: [] + short: Full command line that started the process. + type: keyword + process.elf.architecture: + dashed_name: process-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: process.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword + process.elf.byte_order: + dashed_name: process-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: process.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword + process.elf.cpu_type: + dashed_name: process-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: process.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword + process.elf.creation_date: + dashed_name: process-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + flat_name: process.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date + process.elf.exports: + dashed_name: process-elf-exports + description: List of exported element names and types. + flat_name: process.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened + process.elf.header.abi_version: + dashed_name: process-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: process.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword + process.elf.header.class: + dashed_name: process-elf-header-class + description: Header class of the ELF file. + flat_name: process.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword + process.elf.header.data: + dashed_name: process-elf-header-data + description: Data table of the ELF header. + flat_name: process.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword + process.elf.header.entrypoint: + dashed_name: process-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: process.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long + process.elf.header.object_version: + dashed_name: process-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: process.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword + process.elf.header.os_abi: + dashed_name: process-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: process.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword + process.elf.header.type: + dashed_name: process-elf-header-type + description: Header type of the ELF file. + flat_name: process.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword + process.elf.header.version: + dashed_name: process-elf-header-version + description: Version of the ELF header. + flat_name: process.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword + process.elf.imports: + dashed_name: process-elf-imports + description: List of imported element names and types. + flat_name: process.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened + process.elf.sections: + dashed_name: process-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + flat_name: process.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested + process.elf.sections.chi2: + dashed_name: process-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: process.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long + process.elf.sections.entropy: + dashed_name: process-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: process.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long + process.elf.sections.flags: + dashed_name: process-elf-sections-flags + description: ELF Section List flags. + flat_name: process.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags normalize: [] - original_fieldset: code_signature - short: The identifier used to sign the process. + original_fieldset: elf + short: ELF Section List flags. type: keyword - process.code_signature.status: - dashed_name: process-code-signature-status - description: 'Additional information about the certificate status. - - This is useful for logging cryptographic errors with the certificate validity - or trust status. Leave unpopulated if the validity or trust of the certificate - was unchecked.' - example: ERROR_UNTRUSTED_ROOT - flat_name: process.code_signature.status + process.elf.sections.name: + dashed_name: process-elf-sections-name + description: ELF Section List name. + flat_name: process.elf.sections.name ignore_above: 1024 level: extended - name: status + name: sections.name normalize: [] - original_fieldset: code_signature - short: Additional information about the certificate status. + original_fieldset: elf + short: ELF Section List name. type: keyword - process.code_signature.subject_name: - dashed_name: process-code-signature-subject-name - description: Subject name of the code signer - example: Microsoft Corporation - flat_name: process.code_signature.subject_name + process.elf.sections.physical_offset: + dashed_name: process-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: process.elf.sections.physical_offset ignore_above: 1024 - level: core - name: subject_name + level: extended + name: sections.physical_offset normalize: [] - original_fieldset: code_signature - short: Subject name of the code signer + original_fieldset: elf + short: ELF Section List offset. type: keyword - process.code_signature.team_id: - dashed_name: process-code-signature-team-id - description: 'The team identifier used to sign the process. - - This is used to identify the team or vendor of a software product. The field - is relevant to Apple *OS only.' - example: EQHXZ8M8AV - flat_name: process.code_signature.team_id + process.elf.sections.physical_size: + dashed_name: process-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: process.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long + process.elf.sections.type: + dashed_name: process-elf-sections-type + description: ELF Section List type. + flat_name: process.elf.sections.type ignore_above: 1024 level: extended - name: team_id + name: sections.type normalize: [] - original_fieldset: code_signature - short: The team identifier used to sign the process. + original_fieldset: elf + short: ELF Section List type. type: keyword - process.code_signature.trusted: - dashed_name: process-code-signature-trusted - description: 'Stores the trust status of the certificate chain. - - Validating the trust of the certificate chain may be complicated, and this - field should only be populated by tools that actively check the status.' - example: 'true' - flat_name: process.code_signature.trusted + process.elf.sections.virtual_address: + dashed_name: process-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: process.elf.sections.virtual_address + format: string level: extended - name: trusted + name: sections.virtual_address normalize: [] - original_fieldset: code_signature - short: Stores the trust status of the certificate chain. - type: boolean - process.code_signature.valid: - dashed_name: process-code-signature-valid - description: 'Boolean to capture if the digital signature is verified against - the binary content. - - Leave unpopulated if a certificate was unchecked.' - example: 'true' - flat_name: process.code_signature.valid + original_fieldset: elf + short: ELF Section List virtual address. + type: long + process.elf.sections.virtual_size: + dashed_name: process-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: process.elf.sections.virtual_size + format: string level: extended - name: valid + name: sections.virtual_size normalize: [] - original_fieldset: code_signature - short: Boolean to capture if the digital signature is verified against the binary - content. - type: boolean - process.command_line: - dashed_name: process-command-line - description: 'Full command line that started the process, including the absolute - path to the executable, and all arguments. + original_fieldset: elf + short: ELF Section List virtual size. + type: long + process.elf.segments: + dashed_name: process-elf-segments + description: 'An array containing an object for each segment of the ELF file. - Some arguments may be filtered to protect sensitive information.' - example: /usr/bin/ssh -l user 10.0.0.16 - flat_name: process.command_line + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + flat_name: process.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested + process.elf.segments.sections: + dashed_name: process-elf-segments-sections + description: ELF object segment sections. + flat_name: process.elf.segments.sections ignore_above: 1024 level: extended - multi_fields: - - flat_name: process.command_line.text - name: text - norms: false - type: text - name: command_line + name: segments.sections normalize: [] - short: Full command line that started the process. + original_fieldset: elf + short: ELF object segment sections. + type: keyword + process.elf.segments.type: + dashed_name: process-elf-segments-type + description: ELF object segment type. + flat_name: process.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword + process.elf.shared_libraries: + dashed_name: process-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: process.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword + process.elf.telfhash: + dashed_name: process-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: process.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. type: keyword process.entity_id: dashed_name: process-entity-id @@ -6992,6 +7839,335 @@ process: original_fieldset: process short: Full command line that started the process. type: keyword + process.parent.elf.architecture: + dashed_name: process-parent-elf-architecture + description: Machine architecture of the ELF file. + example: x86-64 + flat_name: process.parent.elf.architecture + ignore_above: 1024 + level: extended + name: architecture + normalize: [] + original_fieldset: elf + short: Machine architecture of the ELF file. + type: keyword + process.parent.elf.byte_order: + dashed_name: process-parent-elf-byte-order + description: Byte sequence of ELF file. + example: Little Endian + flat_name: process.parent.elf.byte_order + ignore_above: 1024 + level: extended + name: byte_order + normalize: [] + original_fieldset: elf + short: Byte sequence of ELF file. + type: keyword + process.parent.elf.cpu_type: + dashed_name: process-parent-elf-cpu-type + description: CPU type of the ELF file. + example: Intel + flat_name: process.parent.elf.cpu_type + ignore_above: 1024 + level: extended + name: cpu_type + normalize: [] + original_fieldset: elf + short: CPU type of the ELF file. + type: keyword + process.parent.elf.creation_date: + dashed_name: process-parent-elf-creation-date + description: Extracted when possible from the file's metadata. Indicates when + it was built or compiled. It can also be faked by malware creators. + flat_name: process.parent.elf.creation_date + level: extended + name: creation_date + normalize: [] + original_fieldset: elf + short: Build or compile date. + type: date + process.parent.elf.exports: + dashed_name: process-parent-elf-exports + description: List of exported element names and types. + flat_name: process.parent.elf.exports + level: extended + name: exports + normalize: + - array + original_fieldset: elf + short: List of exported element names and types. + type: flattened + process.parent.elf.header.abi_version: + dashed_name: process-parent-elf-header-abi-version + description: Version of the ELF Application Binary Interface (ABI). + flat_name: process.parent.elf.header.abi_version + ignore_above: 1024 + level: extended + name: header.abi_version + normalize: [] + original_fieldset: elf + short: Version of the ELF Application Binary Interface (ABI). + type: keyword + process.parent.elf.header.class: + dashed_name: process-parent-elf-header-class + description: Header class of the ELF file. + flat_name: process.parent.elf.header.class + ignore_above: 1024 + level: extended + name: header.class + normalize: [] + original_fieldset: elf + short: Header class of the ELF file. + type: keyword + process.parent.elf.header.data: + dashed_name: process-parent-elf-header-data + description: Data table of the ELF header. + flat_name: process.parent.elf.header.data + ignore_above: 1024 + level: extended + name: header.data + normalize: [] + original_fieldset: elf + short: Data table of the ELF header. + type: keyword + process.parent.elf.header.entrypoint: + dashed_name: process-parent-elf-header-entrypoint + description: Header entrypoint of the ELF file. + flat_name: process.parent.elf.header.entrypoint + format: string + level: extended + name: header.entrypoint + normalize: [] + original_fieldset: elf + short: Header entrypoint of the ELF file. + type: long + process.parent.elf.header.object_version: + dashed_name: process-parent-elf-header-object-version + description: '"0x1" for original ELF files.' + flat_name: process.parent.elf.header.object_version + ignore_above: 1024 + level: extended + name: header.object_version + normalize: [] + original_fieldset: elf + short: '"0x1" for original ELF files.' + type: keyword + process.parent.elf.header.os_abi: + dashed_name: process-parent-elf-header-os-abi + description: Application Binary Interface (ABI) of the Linux OS. + flat_name: process.parent.elf.header.os_abi + ignore_above: 1024 + level: extended + name: header.os_abi + normalize: [] + original_fieldset: elf + short: Application Binary Interface (ABI) of the Linux OS. + type: keyword + process.parent.elf.header.type: + dashed_name: process-parent-elf-header-type + description: Header type of the ELF file. + flat_name: process.parent.elf.header.type + ignore_above: 1024 + level: extended + name: header.type + normalize: [] + original_fieldset: elf + short: Header type of the ELF file. + type: keyword + process.parent.elf.header.version: + dashed_name: process-parent-elf-header-version + description: Version of the ELF header. + flat_name: process.parent.elf.header.version + ignore_above: 1024 + level: extended + name: header.version + normalize: [] + original_fieldset: elf + short: Version of the ELF header. + type: keyword + process.parent.elf.imports: + dashed_name: process-parent-elf-imports + description: List of imported element names and types. + flat_name: process.parent.elf.imports + level: extended + name: imports + normalize: + - array + original_fieldset: elf + short: List of imported element names and types. + type: flattened + process.parent.elf.sections: + dashed_name: process-parent-elf-sections + description: 'An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`.' + flat_name: process.parent.elf.sections + level: extended + name: sections + normalize: + - array + original_fieldset: elf + short: Section information of the ELF file. + type: nested + process.parent.elf.sections.chi2: + dashed_name: process-parent-elf-sections-chi2 + description: Chi-square probability distribution of the section. + flat_name: process.parent.elf.sections.chi2 + format: number + level: extended + name: sections.chi2 + normalize: [] + original_fieldset: elf + short: Chi-square probability distribution of the section. + type: long + process.parent.elf.sections.entropy: + dashed_name: process-parent-elf-sections-entropy + description: Shannon entropy calculation from the section. + flat_name: process.parent.elf.sections.entropy + format: number + level: extended + name: sections.entropy + normalize: [] + original_fieldset: elf + short: Shannon entropy calculation from the section. + type: long + process.parent.elf.sections.flags: + dashed_name: process-parent-elf-sections-flags + description: ELF Section List flags. + flat_name: process.parent.elf.sections.flags + ignore_above: 1024 + level: extended + name: sections.flags + normalize: [] + original_fieldset: elf + short: ELF Section List flags. + type: keyword + process.parent.elf.sections.name: + dashed_name: process-parent-elf-sections-name + description: ELF Section List name. + flat_name: process.parent.elf.sections.name + ignore_above: 1024 + level: extended + name: sections.name + normalize: [] + original_fieldset: elf + short: ELF Section List name. + type: keyword + process.parent.elf.sections.physical_offset: + dashed_name: process-parent-elf-sections-physical-offset + description: ELF Section List offset. + flat_name: process.parent.elf.sections.physical_offset + ignore_above: 1024 + level: extended + name: sections.physical_offset + normalize: [] + original_fieldset: elf + short: ELF Section List offset. + type: keyword + process.parent.elf.sections.physical_size: + dashed_name: process-parent-elf-sections-physical-size + description: ELF Section List physical size. + flat_name: process.parent.elf.sections.physical_size + format: bytes + level: extended + name: sections.physical_size + normalize: [] + original_fieldset: elf + short: ELF Section List physical size. + type: long + process.parent.elf.sections.type: + dashed_name: process-parent-elf-sections-type + description: ELF Section List type. + flat_name: process.parent.elf.sections.type + ignore_above: 1024 + level: extended + name: sections.type + normalize: [] + original_fieldset: elf + short: ELF Section List type. + type: keyword + process.parent.elf.sections.virtual_address: + dashed_name: process-parent-elf-sections-virtual-address + description: ELF Section List virtual address. + flat_name: process.parent.elf.sections.virtual_address + format: string + level: extended + name: sections.virtual_address + normalize: [] + original_fieldset: elf + short: ELF Section List virtual address. + type: long + process.parent.elf.sections.virtual_size: + dashed_name: process-parent-elf-sections-virtual-size + description: ELF Section List virtual size. + flat_name: process.parent.elf.sections.virtual_size + format: string + level: extended + name: sections.virtual_size + normalize: [] + original_fieldset: elf + short: ELF Section List virtual size. + type: long + process.parent.elf.segments: + dashed_name: process-parent-elf-segments + description: 'An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`.' + flat_name: process.parent.elf.segments + level: extended + name: segments + normalize: + - array + original_fieldset: elf + short: ELF object segment list. + type: nested + process.parent.elf.segments.sections: + dashed_name: process-parent-elf-segments-sections + description: ELF object segment sections. + flat_name: process.parent.elf.segments.sections + ignore_above: 1024 + level: extended + name: segments.sections + normalize: [] + original_fieldset: elf + short: ELF object segment sections. + type: keyword + process.parent.elf.segments.type: + dashed_name: process-parent-elf-segments-type + description: ELF object segment type. + flat_name: process.parent.elf.segments.type + ignore_above: 1024 + level: extended + name: segments.type + normalize: [] + original_fieldset: elf + short: ELF object segment type. + type: keyword + process.parent.elf.shared_libraries: + dashed_name: process-parent-elf-shared-libraries + description: List of shared libraries used by this ELF object. + flat_name: process.parent.elf.shared_libraries + ignore_above: 1024 + level: extended + name: shared_libraries + normalize: + - array + original_fieldset: elf + short: List of shared libraries used by this ELF object. + type: keyword + process.parent.elf.telfhash: + dashed_name: process-parent-elf-telfhash + description: telfhash symbol hash for ELF file. + flat_name: process.parent.elf.telfhash + ignore_above: 1024 + level: extended + name: telfhash + normalize: [] + original_fieldset: elf + short: telfhash hash for ELF file. + type: keyword process.parent.entity_id: dashed_name: process-parent-entity-id description: 'Unique identifier for the process. @@ -7522,6 +8698,7 @@ process: name: process nestings: - process.code_signature + - process.elf - process.hash - process.parent - process.pe @@ -7537,6 +8714,10 @@ process: - full: process.code_signature schema_name: code_signature short: These fields contain information about binary code signatures. + - beta: This field reuse is beta and subject to change. + full: process.elf + schema_name: elf + short: These fields contain Linux Executable Linkable Format (ELF) metadata. - full: process.hash schema_name: hash short: Hashes, usually file hashes. @@ -7696,14 +8877,14 @@ related: type: ip related.user: dashed_name: related-user - description: All the user names seen on your event. + description: All the user names or other user identifiers seen on the event. flat_name: related.user ignore_above: 1024 level: extended name: user normalize: - array - short: All the user names seen on your event. + short: All the user names or other user identifiers seen on the event. type: keyword group: 2 name: related @@ -8983,6 +10164,136 @@ threat: normalize: [] short: Threat classification framework. type: keyword + threat.group.alias: + beta: This field is beta and subject to change. + dashed_name: threat-group-alias + description: "The alias(es) of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group alias(es)." + example: '[ "Magecart Group 6" ]' + flat_name: threat.group.alias + ignore_above: 1024 + level: extended + name: group.alias + normalize: + - array + short: Alias of the group. + type: keyword + threat.group.id: + beta: This field is beta and subject to change. + dashed_name: threat-group-id + description: "The id of the group for a set of related intrusion activity that\ + \ are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group id." + example: G0037 + flat_name: threat.group.id + ignore_above: 1024 + level: extended + name: group.id + normalize: [] + short: ID of the group. + type: keyword + threat.group.name: + beta: This field is beta and subject to change. + dashed_name: threat-group-name + description: "The name of the group for a set of related intrusion activity\ + \ that are tracked by a common name in the security community. While not required,\ + \ you can use a MITRE ATT&CK\xAE group name." + example: FIN6 + flat_name: threat.group.name + ignore_above: 1024 + level: extended + name: group.name + normalize: [] + short: Name of the group. + type: keyword + threat.group.reference: + beta: This field is beta and subject to change. + dashed_name: threat-group-reference + description: "The reference URL of the group for a set of related intrusion\ + \ activity that are tracked by a common name in the security community. While\ + \ not required, you can use a MITRE ATT&CK\xAE group reference URL." + example: https://attack.mitre.org/groups/G0037/ + flat_name: threat.group.reference + ignore_above: 1024 + level: extended + name: group.reference + normalize: [] + short: Reference URL of the group. + type: keyword + threat.software.id: + beta: This field is beta and subject to change. + dashed_name: threat-software-id + description: "The id of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software id." + example: S0552 + flat_name: threat.software.id + ignore_above: 1024 + level: extended + name: software.id + normalize: [] + short: ID of the software + type: keyword + threat.software.name: + beta: This field is beta and subject to change. + dashed_name: threat-software-name + description: "The name of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software name." + example: AdFind + flat_name: threat.software.name + ignore_above: 1024 + level: extended + name: software.name + normalize: [] + short: Name of the software. + type: keyword + threat.software.platforms: + beta: This field is beta and subject to change. + dashed_name: threat-software-platforms + description: "The platforms of the software used by this threat to conduct behavior\ + \ commonly modeled using MITRE ATT&CK\xAE. While not required, you can use\ + \ a MITRE ATT&CK\xAE software platforms.\nRecommended Values:\n * AWS\n \ + \ * Azure\n * Azure AD\n * GCP\n * Linux\n * macOS\n * Network\n * Office\ + \ 365\n * SaaS\n * Windows" + example: '[ "Windows" ]' + flat_name: threat.software.platforms + ignore_above: 1024 + level: extended + name: software.platforms + normalize: + - array + short: Platforms of the software. + type: keyword + threat.software.reference: + beta: This field is beta and subject to change. + dashed_name: threat-software-reference + description: "The reference URL of the software used by this threat to conduct\ + \ behavior commonly modeled using MITRE ATT&CK\xAE. While not required, you\ + \ can use a MITRE ATT&CK\xAE software reference URL." + example: https://attack.mitre.org/software/S0552/ + flat_name: threat.software.reference + ignore_above: 1024 + level: extended + name: software.reference + normalize: [] + short: Software reference URL. + type: keyword + threat.software.type: + beta: This field is beta and subject to change. + dashed_name: threat-software-type + description: "The type of software used by this threat to conduct behavior commonly\ + \ modeled using MITRE ATT&CK\xAE. While not required, you can use a MITRE\ + \ ATT&CK\xAE software type.\nRecommended values\n * Malware\n * Tool" + example: Tool + flat_name: threat.software.type + ignore_above: 1024 + level: extended + name: software.type + normalize: [] + short: Software type. + type: keyword threat.tactic.id: dashed_name: threat-tactic-id description: "The id of tactic used by this threat. You can use a MITRE ATT&CK\xAE\ @@ -10938,9 +12249,6 @@ user: - as: user at: destination full: destination.user - - as: user - at: host - full: host.user - as: user at: server full: server.user diff --git a/generated/elasticsearch/6/template.json b/generated/elasticsearch/6/template.json index 2a964ae302..77852349de 100644 --- a/generated/elasticsearch/6/template.json +++ b/generated/elasticsearch/6/template.json @@ -895,6 +895,123 @@ "ignore_above": 1, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "extension": { "ignore_above": 1024, "type": "keyword" @@ -1317,66 +1434,6 @@ }, "uptime": { "type": "long" - }, - "user": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "email": { - "ignore_above": 1024, - "type": "keyword" - }, - "full_name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "group": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "hash": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "roles": { - "ignore_above": 1024, - "type": "keyword" - } - } } } }, @@ -1987,6 +2044,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" @@ -2086,6 +2260,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" @@ -2806,6 +3097,50 @@ "ignore_above": 1024, "type": "keyword" }, + "group": { + "properties": { + "alias": { + "ignore_above": 1024, + "type": "keyword" + }, + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "software": { + "properties": { + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "platforms": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "tactic": { "properties": { "id": { diff --git a/generated/elasticsearch/7/template.json b/generated/elasticsearch/7/template.json index fea66d43bb..25aa491dcc 100644 --- a/generated/elasticsearch/7/template.json +++ b/generated/elasticsearch/7/template.json @@ -891,6 +891,123 @@ "ignore_above": 1, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "extension": { "ignore_above": 1024, "type": "keyword" @@ -1313,66 +1430,6 @@ }, "uptime": { "type": "long" - }, - "user": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "email": { - "ignore_above": 1024, - "type": "keyword" - }, - "full_name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "group": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "hash": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "roles": { - "ignore_above": 1024, - "type": "keyword" - } - } } } }, @@ -1983,6 +2040,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" @@ -2082,6 +2256,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" @@ -2802,6 +3093,50 @@ "ignore_above": 1024, "type": "keyword" }, + "group": { + "properties": { + "alias": { + "ignore_above": 1024, + "type": "keyword" + }, + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "software": { + "properties": { + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "platforms": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "tactic": { "properties": { "id": { diff --git a/generated/elasticsearch/component/file.json b/generated/elasticsearch/component/file.json index ea4c4a4a1d..464fb1e9eb 100644 --- a/generated/elasticsearch/component/file.json +++ b/generated/elasticsearch/component/file.json @@ -62,6 +62,123 @@ "ignore_above": 1, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "extension": { "ignore_above": 1024, "type": "keyword" diff --git a/generated/elasticsearch/component/host.json b/generated/elasticsearch/component/host.json index 018c9f58ac..d8667a9f3e 100644 --- a/generated/elasticsearch/component/host.json +++ b/generated/elasticsearch/component/host.json @@ -182,66 +182,6 @@ }, "uptime": { "type": "long" - }, - "user": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "email": { - "ignore_above": 1024, - "type": "keyword" - }, - "full_name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "group": { - "properties": { - "domain": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "ignore_above": 1024, - "type": "keyword" - } - } - }, - "hash": { - "ignore_above": 1024, - "type": "keyword" - }, - "id": { - "ignore_above": 1024, - "type": "keyword" - }, - "name": { - "fields": { - "text": { - "norms": false, - "type": "text" - } - }, - "ignore_above": 1024, - "type": "keyword" - }, - "roles": { - "ignore_above": 1024, - "type": "keyword" - } - } } } } diff --git a/generated/elasticsearch/component/process.json b/generated/elasticsearch/component/process.json index 5bbf33b350..e1ab458007 100644 --- a/generated/elasticsearch/component/process.json +++ b/generated/elasticsearch/component/process.json @@ -54,6 +54,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" @@ -153,6 +270,123 @@ "ignore_above": 1024, "type": "keyword" }, + "elf": { + "properties": { + "architecture": { + "ignore_above": 1024, + "type": "keyword" + }, + "byte_order": { + "ignore_above": 1024, + "type": "keyword" + }, + "cpu_type": { + "ignore_above": 1024, + "type": "keyword" + }, + "creation_date": { + "type": "date" + }, + "exports": { + "type": "flattened" + }, + "header": { + "properties": { + "abi_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "class": { + "ignore_above": 1024, + "type": "keyword" + }, + "data": { + "ignore_above": 1024, + "type": "keyword" + }, + "entrypoint": { + "type": "long" + }, + "object_version": { + "ignore_above": 1024, + "type": "keyword" + }, + "os_abi": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "imports": { + "type": "flattened" + }, + "sections": { + "properties": { + "chi2": { + "type": "long" + }, + "entropy": { + "type": "long" + }, + "flags": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_offset": { + "ignore_above": 1024, + "type": "keyword" + }, + "physical_size": { + "type": "long" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + }, + "virtual_address": { + "type": "long" + }, + "virtual_size": { + "type": "long" + } + }, + "type": "nested" + }, + "segments": { + "properties": { + "sections": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + }, + "type": "nested" + }, + "shared_libraries": { + "ignore_above": 1024, + "type": "keyword" + }, + "telfhash": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "entity_id": { "ignore_above": 1024, "type": "keyword" diff --git a/generated/elasticsearch/component/threat.json b/generated/elasticsearch/component/threat.json index 4f3a4a36c8..e976bd78fa 100644 --- a/generated/elasticsearch/component/threat.json +++ b/generated/elasticsearch/component/threat.json @@ -12,6 +12,50 @@ "ignore_above": 1024, "type": "keyword" }, + "group": { + "properties": { + "alias": { + "ignore_above": 1024, + "type": "keyword" + }, + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "software": { + "properties": { + "id": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "platforms": { + "ignore_above": 1024, + "type": "keyword" + }, + "reference": { + "ignore_above": 1024, + "type": "keyword" + }, + "type": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, "tactic": { "properties": { "id": { diff --git a/rfcs/text/0012-orchestrator-field-set.md b/rfcs/text/0012-orchestrator-field-set.md index c99da3a159..3796a34908 100644 --- a/rfcs/text/0012-orchestrator-field-set.md +++ b/rfcs/text/0012-orchestrator-field-set.md @@ -1,7 +1,7 @@ # 0012: Orchestrator field set creation -- Stage: **2 (candidate)** -- Date: **2021-03-26** +- Stage: **3 (finished)** +- Date: **2021-05-14** There is currently no ECS field set for container orchestration engines. There is an example of an ECS [use-case][0] for Kubernetes, but it largely relies on other ECS field sets, and doesn't cover all of the @@ -93,10 +93,6 @@ The proposed change adds nine fields, as described below: API version being used to carry out the action ``` - - ## Usage The `orchestrator` field set will be used to capture typical concepts employed @@ -198,10 +194,6 @@ Examples of source data include: } ``` - - ## Scope of impact As this RFC involves the creation of an entirely new fieldset, no breaking @@ -239,19 +231,8 @@ cover all the logical primitives of popular orchestrators. Input from contributo who have experience with the various alternative orchestration providers would be particularly valuable. - - - - -## Real-world implementations - - +*Resolution*: Input from various orchestrators has been considered to ensure this field +set remains as generic as possible. ## People @@ -275,6 +256,7 @@ The following are the people that consulted on the contents of this RFC. * Stage 0: https://github.com/elastic/ecs/pull/1209 * Stage 1: https://github.com/elastic/ecs/pull/1230 * Stage 2: https://github.com/elastic/ecs/pull/1299 +* Stage 3: https://github.com/elastic/ecs/pull/1343 [0]: https://github.com/elastic/ecs/blob/master/use-cases/kubernetes.yml [1]: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/ diff --git a/rfcs/text/0015-create-file-elf.md b/rfcs/text/0015-create-file-elf.md index e6bb06dc54..e683d272fb 100644 --- a/rfcs/text/0015-create-file-elf.md +++ b/rfcs/text/0015-create-file-elf.md @@ -1,7 +1,7 @@ # 0015: Create the ELF sub-field of the File fieldset - Stage: **2 (candidate)** -- Date: **2021-02-10** +- Date: **2021-05-04** Create the Executable Linkable Format (ELF) sub-field, of the `file` top-level fieldset. This document metadata can be used for malware research, as well as coding and other application development efforts. @@ -235,6 +235,7 @@ The following are the people that consulted on the contents of this RFC. * Stage 1: https://github.com/elastic/ecs/pull/1077 * Stage 2: https://github.com/elastic/ecs/pull/1294 + * Stage 2 advancement date correction: https://github.com/elastic/ecs/pull/1409 -- Stage: **1 (draft)** -- Date: **2021-04-06** +- Stage: **2 (candidate)** +- Date: **2021-05-25** + +- Stage: **0 (strawperson)** +- Date: **2021-05-11** + + + + + +Indexing `message` fields as the `text` type in security and application logs consumes significant disk space. Part of the disk space spent is on indexing to support scoring and phrase queries, which aren't often used in logging use cases. Elasticsearch 7.14 introduces a new field type called `match_only_text` which is a more space-efficient variant of the `text` field type for this logging-focused use cases. + +This RFC proposes migrating existing ECS `text` fields to `match_only_text`. Most current ECS datasets are focused heavily on logging use cases, and we can pass this disk space savings onto users by migrating `text` fields to `match_only_text` by default in ECS. Upcoming changes in Elasticsearch will default to indexing the `message` field as `match_only_text`, and this change in ECS will also align better with this new stack default. + + + +## Fields + +The following fields are currently indexed as `text` and are candidates to migrate to `match_only_text`: + +* `message` +* `error.message` + + + + + + +## Usage + +Data is indexed the same as a `text` field that has: + +* `index_options: docs` +* `norms: false` + +`match_only_text` uses the `_source` for positional queries like `match_phrase` + +The `match_only_text` type supports the same feature set as `text`, except the following: + +* No support for scoring: queries ignore index statistics and produce constant scores. +* Span queries are unsupported. If a span query is run, then shards where the field is mapped as match_only_text will be returned as failed in the search response and their hits will be ignored. +* Phrase and intervals queries run slower. + +Like `text`, `match_only_text` fields do not support aggregations. + +This new field is part of the text family, so it is returned as a text field in the `_field_caps` output. Being a member of the `text` field family means migrating fields from `text` to `match_only_text` is a non-breaking change and the fields of `text` and `match_only_text` can be queried alongside each other. + + + +## Source data + + + + + + + +## Scope of impact + + + +## Concerns + + + + + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @ebeahan | author +* @jpountz | subject matter expert + + + + +## References + + + +* https://www.elastic.co/guide/en/elasticsearch/reference/master/text.html#match-only-text-field-type +* https://github.com/elastic/elasticsearch/pull/66172 +* https://github.com/elastic/ecs/issues/1377 +* https://github.com/elastic/elasticsearch/issues/64467 +* https://github.com/elastic/elasticsearch/blob/7.x/x-pack/plugin/core/src/main/resources/data-streams-mappings.json#L14-L22 + +### RFC Pull Requests + + + +* Stage 0: https://github.com/elastic/ecs/pull/1396 + + diff --git a/rfcs/text/0024-ticket-fields.md b/rfcs/text/0024-ticket-fields.md new file mode 100644 index 0000000000..d2d4b9842a --- /dev/null +++ b/rfcs/text/0024-ticket-fields.md @@ -0,0 +1,110 @@ +# 0024: Ticket fields + + +- Stage: **0 (strawperson)** +- Date: **2021-05-11** + + + + +Stage 0: Provide a schema definition for fields related to tickets. Tickets, include, but are not limited to, Change Requests, +Incidents, User Stories, Hardware/Software Requests, etc. This enables storing information about tickets in Elasticsearch +in a common format enabling Elasticsearch to function as a backing DB for ticketing systems, it also enables storing ticket +meta-data for use in Enterprise Search, and it plays a role in security & observability with relating log and metric data to open +Event, Incident or Security Vulnerability tickets. [Sample schema](https://github.com/elastic/ecs/compare/master...kc-comcast:ticket-block) + + +## Fields + + + + + +## Usage + + + +## Source data + + + + + + + +## Scope of impact + + + +## Concerns + + + + + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @kc-comcast | author + + + + +## References + + + +### RFC Pull Requests + + + +* Stage 0: https://github.com/elastic/ecs/pull/1383 + + diff --git a/experimental/schemas/elf.yml b/schemas/elf.yml similarity index 80% rename from experimental/schemas/elf.yml rename to schemas/elf.yml index 82b17da920..525b155414 100644 --- a/experimental/schemas/elf.yml +++ b/schemas/elf.yml @@ -4,12 +4,18 @@ group: 2 description: > These fields contain Linux Executable Linkable Format (ELF) metadata. + beta: > + These fields are in beta and are subject to change. type: group reusable: top_level: false expected: - - file - - process + - at: file + as: elf + beta: This field reuse is beta and subject to change. + - at: process + as: elf + beta: This field reuse is beta and subject to change. fields: - name: creation_date short: Build or compile date. @@ -31,7 +37,7 @@ Byte sequence of ELF file. type: keyword level: extended - example: Little Endian, Big Endian + example: Little Endian - name: cpu_type description: > @@ -90,10 +96,16 @@ "0x1" for original ELF files. - name: sections + short: Section information of the ELF file. description: > - Section information of the ELF file. + An array containing an object for each section of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.sections.*`. type: nested level: extended + normalize: + - "array" - name: sections.flags description: > @@ -159,33 +171,43 @@ List of exported element names and types. level: extended type: flattened + normalize: + - array - name: imports description: > List of imported element names and types. type: flattened level: extended + normalize: + - array - name: shared_libraries description: > - List of shared libraries used by this ELF object + List of shared libraries used by this ELF object. type: keyword level: extended normalize: - array - name: telfhash - short: telfhash hash for ELF files + short: telfhash hash for ELF file. description: > - telfhash is symbol hash for ELF files, just like imphash is imports hash for PE files. Learn more at https://github.com/trendmicro/telfhash. + telfhash symbol hash for ELF file. type: keyword level: extended - name: segments + short: ELF object segment list. description: > - ELF object segment list. + An array containing an object for each segment of the ELF file. + + The keys that should be present in these objects are defined by sub-fields + underneath `elf.segments.*`. type: nested level: extended + normalize: + - array - name: segments.type description: ELF object segment type. diff --git a/schemas/orchestrator.yml b/schemas/orchestrator.yml index e63105e3ce..e90577ca70 100644 --- a/schemas/orchestrator.yml +++ b/schemas/orchestrator.yml @@ -3,8 +3,6 @@ title: Orchestrator group: 2 short: Fields relevant to container orchestrators. - beta: > - These fields are in beta and are subject to change. description: > Fields that describe the resources which container orchestrators manage or act upon. diff --git a/schemas/related.yml b/schemas/related.yml index 5e53009475..c40e339e7e 100644 --- a/schemas/related.yml +++ b/schemas/related.yml @@ -29,7 +29,8 @@ level: extended type: keyword description: > - All the user names seen on your event. + All the user names or other user identifiers seen on the event. + normalize: - array diff --git a/schemas/threat.yml b/schemas/threat.yml index 62477b28a1..acb36a2115 100644 --- a/schemas/threat.yml +++ b/schemas/threat.yml @@ -24,6 +24,116 @@ example: MITRE ATT&CK + - name: group.alias + level: extended + type: keyword + short: Alias of the group. + beta: This field is beta and subject to change. + description: > + The alias(es) of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group alias(es). + + example: '[ "Magecart Group 6" ]' + normalize: + - array + + - name: group.id + level: extended + type: keyword + short: ID of the group. + beta: This field is beta and subject to change. + description: > + The id of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group id. + + example: "G0037" + + - name: group.name + level: extended + type: keyword + short: Name of the group. + beta: This field is beta and subject to change. + description: > + The name of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group name. + + example: "FIN6" + + - name: group.reference + level: extended + type: keyword + short: Reference URL of the group. + beta: This field is beta and subject to change. + description: > + The reference URL of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group reference URL. + + example: "https://attack.mitre.org/groups/G0037/" + + - name: software.id + level: extended + type: keyword + short: ID of the software + beta: This field is beta and subject to change. + description: > + The id of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software id. + + example: "S0552" + + - name: software.name + level: extended + type: keyword + short: Name of the software. + beta: This field is beta and subject to change. + description: > + The name of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software name. + + example: "AdFind" + + - name: software.platforms + level: extended + type: keyword + short: Platforms of the software. + beta: This field is beta and subject to change. + description: > + The platforms of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software platforms. + + Recommended Values: + * AWS + * Azure + * Azure AD + * GCP + * Linux + * macOS + * Network + * Office 365 + * SaaS + * Windows + + example: '[ "Windows" ]' + normalize: + - array + + - name: software.reference + level: extended + type: keyword + short: Software reference URL. + beta: This field is beta and subject to change. + description: > + The reference URL of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software reference URL. + + example: "https://attack.mitre.org/software/S0552/" + + - name: software.type + level: extended + type: keyword + short: Software type. + beta: This field is beta and subject to change. + description: > + The type of software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. While not required, you can use a MITRE ATT&CK® software type. + + Recommended values + * Malware + * Tool + + example: "Tool" + - name: tactic.id level: extended type: keyword diff --git a/schemas/user.yml b/schemas/user.yml index 501c5f9a5a..8066e06cfc 100644 --- a/schemas/user.yml +++ b/schemas/user.yml @@ -15,7 +15,6 @@ expected: - client - destination - - host - server - source - at: user diff --git a/scripts/cmd/gocodegen/gocodegen.go b/scripts/cmd/gocodegen/gocodegen.go index 726be3e436..303e8992f9 100644 --- a/scripts/cmd/gocodegen/gocodegen.go +++ b/scripts/cmd/gocodegen/gocodegen.go @@ -321,7 +321,7 @@ func goDataType(fieldName, elasticsearchDataType string) string { } switch elasticsearchDataType { - case "keyword", "wildcard", "version", "constant_keyword", "text", "ip", "geo_point": + case "keyword", "wildcard", "version", "constant_keyword", "text", "match_only_text", "ip", "geo_point": return "string" case "long": return "int64" diff --git a/scripts/generator.py b/scripts/generator.py index c099ee54ff..31772f3c79 100644 --- a/scripts/generator.py +++ b/scripts/generator.py @@ -12,10 +12,10 @@ from generators import intermediate_files from schema import loader -from schema import oss from schema import cleaner from schema import finalizer from schema import subset_filter +from schema import exclude_filter def main(): @@ -46,11 +46,10 @@ def main(): print('Experimental ECS version ' + ecs_generated_version) fields = loader.load_schemas(ref=args.ref, included_files=args.include) - if args.oss: - oss.fallback(fields) cleaner.clean(fields, strict=args.strict) finalizer.finalize(fields) fields = subset_filter.filter(fields, args.subset, out_dir) + fields = exclude_filter.exclude(fields, args.exclude) nested, flat = intermediate_files.generate(fields, os.path.join(out_dir, 'ecs'), default_dirs) if args.intermediate_only: @@ -60,7 +59,7 @@ def main(): es_template.generate(nested, ecs_generated_version, out_dir, args.mapping_settings) es_template.generate_legacy(flat, ecs_generated_version, out_dir, args.template_settings, args.mapping_settings) beats.generate(nested, ecs_generated_version, out_dir) - if args.include or args.subset: + if args.include or args.subset or args.exclude: exit() ecs_helpers.make_dirs(docs_dir) @@ -73,6 +72,8 @@ def argument_parser(): Note that "--include experimental/schemas" will also respect this git ref.') parser.add_argument('--include', nargs='+', help='include user specified directory of custom field definitions') + parser.add_argument('--exclude', nargs='+', + help='exclude user specified subset of the schema') parser.add_argument('--subset', nargs='+', help='render a subset of the schema') parser.add_argument('--out', action='store', help='directory to output the generated files') @@ -80,7 +81,6 @@ def argument_parser(): help='index template settings to use when generating elasticsearch template') parser.add_argument('--mapping-settings', action='store', help='mapping settings to use when generating elasticsearch template') - parser.add_argument('--oss', action='store_true', help='replace basic data types with oss ones where possible') parser.add_argument('--strict', action='store_true', help='enforce strict checking at schema cleanup') parser.add_argument('--intermediate-only', action='store_true', diff --git a/scripts/generators/beats.py b/scripts/generators/beats.py index 6f70921f67..5708a03555 100644 --- a/scripts/generators/beats.py +++ b/scripts/generators/beats.py @@ -83,9 +83,9 @@ def write_beats_yaml(beats_file, ecs_version, out_dir): def file_header(): - return ''' + return """ # WARNING! Do not edit this file directly, it was generated by the ECS project, # based on ECS version {version}. # Please visit https://github.com/elastic/ecs to suggest changes to ECS fields. -'''.lstrip() +""".lstrip() diff --git a/scripts/generators/ecs_helpers.py b/scripts/generators/ecs_helpers.py index 086f4d592d..fbf7f4a2a1 100644 --- a/scripts/generators/ecs_helpers.py +++ b/scripts/generators/ecs_helpers.py @@ -159,7 +159,7 @@ def yaml_load(filename): def list_subtract(original, subtracted): - '''Subtract two lists. original = subtracted''' + """Subtract two lists. original = subtracted""" return [item for item in original if item not in subtracted] @@ -175,7 +175,7 @@ def list_extract_keys(lst, key_name): def is_intermediate(field): - '''Encapsulates the check to see if a field is an intermediate field or a "real" field.''' + """Encapsulates the check to see if a field is an intermediate field or a "real" field.""" return ('intermediate' in field['field_details'] and field['field_details']['intermediate']) diff --git a/scripts/generators/es_template.py b/scripts/generators/es_template.py index 83ae6fb61a..46d529963e 100644 --- a/scripts/generators/es_template.py +++ b/scripts/generators/es_template.py @@ -6,11 +6,17 @@ from generators import ecs_helpers from schema.cleaner import field_or_multi_field_datatype_defaults -from schema.oss import TYPE_FALLBACKS +TYPE_FALLBACKS = { + 'constant_keyword': 'keyword', + 'wildcard': 'keyword', + 'version': 'keyword' +} + # Composable Template + def generate(ecs_nested, ecs_version, out_dir, mapping_settings_file): """This generates all artifacts for the composable template approach""" all_component_templates(ecs_nested, ecs_version, out_dir) @@ -263,13 +269,13 @@ def default_mapping_settings(): def es6_type_fallback(mappings): - ''' + """ Visits each leaf in mappings object and fallback to an Elasticsearch 6.x supported type. Since a field like `wildcard` won't have the same defaults as a `keyword` field, we must add any missing defaults. - ''' + """ for (name, details) in mappings.items(): if 'type' in details: diff --git a/scripts/generators/intermediate_files.py b/scripts/generators/intermediate_files.py index d21800936f..c085039b62 100644 --- a/scripts/generators/intermediate_files.py +++ b/scripts/generators/intermediate_files.py @@ -20,7 +20,7 @@ def generate(fields, out_dir, default_dirs): def generate_flat_fields(fields): - '''Generate ecs_flat.yml''' + """Generate ecs_flat.yml""" filtered = remove_non_root_reusables(fields) flattened = {} visitor.visit_fields_with_memo(filtered, accumulate_field, flattened) @@ -28,7 +28,7 @@ def generate_flat_fields(fields): def accumulate_field(details, memo): - '''Visitor function that accumulates all field details in the memo dict''' + """Visitor function that accumulates all field details in the memo dict""" if 'schema_details' in details or ecs_helpers.is_intermediate(details): return field_details = copy.deepcopy(details['field_details']) @@ -39,7 +39,7 @@ def accumulate_field(details, memo): def generate_nested_fields(fields): - '''Generate ecs_nested.yml''' + """Generate ecs_nested.yml""" nested = {} # Flatten each field set, but keep all resulting fields nested under their # parent/host field set. @@ -71,13 +71,13 @@ def generate_nested_fields(fields): def remove_internal_attributes(field_details): - '''Remove attributes only relevant to the deeply nested structure, but not to ecs_flat/nested.yml.''' + """Remove attributes only relevant to the deeply nested structure, but not to ecs_flat/nested.yml.""" field_details.pop('node_name', None) field_details.pop('intermediate', None) def remove_non_root_reusables(fields_nested): - ''' + """ Remove field sets that have top_level=false from the root of the field definitions. This attribute means they're only meant to be in the "reusable/expected" locations @@ -87,7 +87,7 @@ def remove_non_root_reusables(fields_nested): still needs to keep all field sets at the root of the YAML file, as it the official information about each field set. It's the responsibility of users consuming ecs_nested.yml to skip the field sets with top_level=false. - ''' + """ fields = {} for (name, field) in fields_nested.items(): if 'reusable' not in field['schema_details'] or field['schema_details']['reusable']['top_level']: diff --git a/scripts/helper.py b/scripts/helper.py deleted file mode 100644 index f9bfa05756..0000000000 --- a/scripts/helper.py +++ /dev/null @@ -1,179 +0,0 @@ -import yaml -import glob - - -def read_schema_file(path): - """Read a schema.yml file and cleans up the fields - """ - fields = [] - with open(path) as f: - fields = yaml.safe_load(f.read()) - - clean_namespace_fields(fields) - return fields - - -def read_use_case_file(path): - """Read a use-case.yml file and cleans up the fields - """ - with open(path) as f: - use_case = yaml.safe_load(f.read()) - - fields = use_case["fields"] - clean_namespace_fields(fields) - use_case["fields"] = fields - return use_case - - -def clean_namespace_fields(fields): - """Cleans up all fields to set defaults - """ - for namespace in fields: - - # For now set the default group to 2 - if "group" not in namespace: - namespace["group"] = 2 - - prefix = "" - # Prefix if not base namespace - if namespace["name"] != "base": - prefix = namespace["name"] - - clean_fields(namespace["fields"], prefix, namespace["group"]) - - -def clean_fields(fields, prefix, group): - for field in fields: - clean_string_field(field, "description") - clean_string_field(field, "footnote") - clean_string_field(field, "example") - clean_string_field(field, "type") - - # Add prefix if needed - if prefix != "": - field["name"] = prefix + "." + field["name"] - - if 'level' not in field.keys(): - field["level"] = '(use case)' - - if 'group' not in field.keys(): - # If no group set, set parent group - field["group"] = group - - # if "multi_fields" in field: - # for f in field["multi_fields"]: - # clean_string_field(f, "description") - # clean_string_field(f, "example") - # clean_string_field(f, "type") - - # # multi fields always have a prefix - # f["name"] = field["name"] + "." + f["name"] - - # if 'group' not in f.keys(): - # # If no group set, set parent group - # f["group"] = group - - -def clean_string_field(field, key): - """Cleans a string field and creates an empty string for the field in case it does not exist - """ - if key in field.keys(): - # Remove all spaces and newlines from beginning and end - field[key] = str(field[key]).strip() - else: - field[key] = "" - - if "index" in field and field["index"] == False: - field["type"] = "(not indexed)" - - -def get_markdown_row(field, link, multi_field): - """Creates a markdown table for the given fields - """ - - # Replace newlines with HTML representation as otherwise newlines don't work in Markdown - description = field["description"].replace("\n", "
") - - show_name = field["name"] - - ecs = True - if 'ecs' in field.keys(): - ecs = field["ecs"] - - # non ecs fields are in italic - if not ecs: - show_name = "*" + field["name"] + "*" - description = "*" + description + "*" - - example = "" - if field["example"] != "": - # Add ticks around examples to not break table - example = "`{}`".format(field["example"]) - - # If link is true, it link to the anchor is provided. This is used for the use-cases - if link and ecs: - return '| [{}]({}#{}) | {} | {} | {} | {} |\n'.format(show_name, link, field["name"], description, field["level"], field["type"], example) - - # By default a anchor is attached to the name - return '| {} | {} | {} | {} | {} |\n'.format(field["name"], show_name, description, field["level"], field["type"], example) - - -def get_schema(): - fields = [] - for file in sorted(glob.glob("schemas/*.yml")): - fields = fields + read_schema_file(file) - return fields - - -def get_markdown_section(namespace, title_prefix="##", link=False): - section_name = namespace["name"] - - # Title - output = '{} {} fields\n\n'.format(title_prefix, section_name, namespace["title"]) - - # Description - # Replaces one newlines with two as otherwise double newlines do not show up in markdown - output += namespace["description"].replace("\n", "\n\n") + "\n" - - # Reusable object details - if "reusable" in namespace and "expected" in namespace["reusable"]: - sorted_fields = sorted(namespace["reusable"]["expected"]) - rendered_fields = map(lambda f: "`{}.{}`".format(f, section_name), sorted_fields) - output += "The `{}` fields are expected to be nested at: {}.\n\n".format( - section_name, ', '.join(rendered_fields)) - - if "top_level" in namespace["reusable"] and namespace["reusable"]["top_level"]: - template = "Note also that the `{}` fields may be used directly at the top level.\n\n" - else: - template = "Note also that the `{}` fields are not expected to " + \ - "be used directly at the top level.\n\n" - output += template.format(section_name) - - # Table - titles = ["Field", "Description", "Level", "Type", "Example"] - - for title in titles: - output += "| {} ".format(title) - output += "|\n" - - for title in titles: - output += "|---" - output += "|\n" - - # Sort fields for easier readability - namespaceFields = sorted(namespace["fields"], key=lambda field: field["name"]) - - # Print fields into a table - for field in namespace["fields"]: - output += get_markdown_row(field, link, False) - if "multi_fields" in field: - for f in field["multi_fields"]: - output += get_markdown_row(f, link, True) - - output += "\n\n" - - # Footnote - if "footnote" in namespace: - output += namespace["footnote"].replace("\n", "\n\n") + "\n" - - return output diff --git a/scripts/schema/cleaner.py b/scripts/schema/cleaner.py index b316ef7298..efcc1d08a2 100644 --- a/scripts/schema/cleaner.py +++ b/scripts/schema/cleaner.py @@ -56,7 +56,7 @@ def schema_cleanup(schema): def schema_mandatory_attributes(schema): - '''Ensures for the presence of the mandatory schema attributes and raises if any are missing''' + """Ensures for the presence of the mandatory schema attributes and raises if any are missing""" current_schema_attributes = sorted(list(schema['field_details'].keys()) + list(schema['schema_details'].keys())) missing_attributes = ecs_helpers.list_subtract(SCHEMA_MANDATORY_ATTRIBUTES, current_schema_attributes) @@ -74,7 +74,7 @@ def schema_mandatory_attributes(schema): def schema_assertions_and_warnings(schema): - '''Additional checks on a fleshed out schema''' + """Additional checks on a fleshed out schema""" single_line_short_description(schema, strict=strict_mode) if 'beta' in schema['field_details']: single_line_beta_description(schema, strict=strict_mode) @@ -143,7 +143,7 @@ def field_defaults(field): def field_or_multi_field_datatype_defaults(field_details): - '''Sets datatype-related defaults on a canonical field or multi-field entries.''' + """Sets datatype-related defaults on a canonical field or multi-field entries.""" if field_details['type'] == 'keyword': field_details.setdefault('ignore_above', 1024) if field_details['type'] == 'text': @@ -160,7 +160,7 @@ def field_or_multi_field_datatype_defaults(field_details): def field_mandatory_attributes(field): - '''Ensures for the presence of the mandatory field attributes and raises if any are missing''' + """Ensures for the presence of the mandatory field attributes and raises if any are missing""" if ecs_helpers.is_intermediate(field): return current_field_attributes = sorted(field['field_details'].keys()) @@ -180,7 +180,7 @@ def field_mandatory_attributes(field): def field_assertions_and_warnings(field): - '''Additional checks on a fleshed out field''' + """Additional checks on a fleshed out field""" if not ecs_helpers.is_intermediate(field): # check short description length if in strict mode single_line_short_description(field, strict=strict_mode) diff --git a/scripts/schema/exclude_filter.py b/scripts/schema/exclude_filter.py new file mode 100644 index 0000000000..5717ecfb6f --- /dev/null +++ b/scripts/schema/exclude_filter.py @@ -0,0 +1,78 @@ +from schema import loader + +# This script should be run downstream of the subset filters - it takes +# all ECS and custom fields already loaded by the latter and explicitly +# removes a subset, for example, to simulate impact of future removals + + +def exclude(fields, exclude_file_globs): + excludes = load_exclude_definitions(exclude_file_globs) + + if excludes: + fields = exclude_fields(fields, excludes) + + return fields + + +def long_path(path_as_list): + return '.'.join([e for e in path_as_list]) + + +def pop_field(fields, node_path, path, removed): + """pops a field from yaml derived dict using path derived from ordered list of nodes""" + if node_path[0] in fields: + if len(node_path) == 1: + flat_name = long_path(path) + fields.pop(node_path[0]) + return flat_name + else: + inner_field = node_path.pop(0) + if 'fields' in fields[inner_field]: + popped = pop_field(fields[inner_field]['fields'], node_path, path, removed) + # if object field with no remaining fields and not 'base', pop it + if fields[inner_field]['fields'] == {} and inner_field != 'base': + fields.pop(inner_field) + return popped + else: + raise ValueError( + '--exclude specified, but no path to field {} found'.format(long_path(path))) + else: + this_long_path = long_path(path) + # Check in case already removed parent + if not any([this_long_path.startswith(long_path) for long_path in removed if long_path != None]): + raise ValueError('--exclude specified, but no field {} found'.format(this_long_path)) + + +def exclude_trace_path(fields, item, path, removed): + """traverses paths to one or more nodes in a yaml derived dict""" + for list_item in item: + node_path = path.copy() + # cater for name.with.dots + for name in list_item['name'].split('.'): + node_path.append(name) + if not 'fields' in list_item: + parent = node_path[0] + removed.append(pop_field(fields, node_path, node_path.copy(), removed)) + # if parent field has no remaining fields and not 'base', pop it + if parent != 'base' and parent in fields and len(fields[parent]['fields']) == 0: + fields.pop(parent) + else: + raise ValueError('--exclude specified, can\'t parse fields in file {}'.format(item)) + + +def exclude_fields(fields, excludes): + """Traverses fields and eliminates any field which matches the excludes""" + if excludes: + for ex_list in excludes: + for item in ex_list: + exclude_trace_path(fields, item['fields'], [item['name']], []) + return fields + + +def load_exclude_definitions(file_globs): + if not file_globs: + return [] + excludes = loader.load_definitions(file_globs) + if not excludes: + raise ValueError('--exclude specified, but no exclusions found in {}'.format(file_globs)) + return excludes diff --git a/scripts/schema/finalizer.py b/scripts/schema/finalizer.py index 84b4c21062..648349b8ef 100644 --- a/scripts/schema/finalizer.py +++ b/scripts/schema/finalizer.py @@ -19,7 +19,7 @@ def finalize(fields): - '''Intended entrypoint of the finalizer.''' + """Intended entrypoint of the finalizer.""" perform_reuse(fields) calculate_final_values(fields) @@ -46,7 +46,7 @@ def order_reuses(fields): def perform_reuse(fields): - '''Performs field reuse in two phases''' + """Performs field reuse in two phases""" foreign_reuses, self_nestings = order_reuses(fields) # Phase 1: foreign reuse @@ -99,11 +99,11 @@ def perform_reuse(fields): def ensure_valid_reuse(reused_schema, destination_schema=None): - ''' + """ Raise if either the reused schema or destination schema have root=true. Second param is optional, if testing for a self-nesting (where source=destination). - ''' + """ if reused_schema['schema_details']['root']: msg = "Schema {} has attribute root=true and therefore cannot be reused.".format( reused_schema['field_details']['name']) @@ -115,7 +115,7 @@ def ensure_valid_reuse(reused_schema, destination_schema=None): def append_reused_here(reused_schema, reuse_entry, destination_schema): - '''Captures two ways of denoting what field sets are reused under a given field set''' + """Captures two ways of denoting what field sets are reused under a given field set""" # Legacy, too limited destination_schema['schema_details'].setdefault('nestings', []) destination_schema['schema_details']['nestings'] = sorted( @@ -136,7 +136,7 @@ def append_reused_here(reused_schema, reuse_entry, destination_schema): def set_original_fieldset(fields, original_fieldset): - '''Recursively set the 'original_fieldset' attribute for all fields in a group of fields''' + """Recursively set the 'original_fieldset' attribute for all fields in a group of fields""" def func(details): # Don't override if already set (e.g. 'group' for user.group.* fields) details['field_details'].setdefault('original_fieldset', original_fieldset) @@ -144,7 +144,7 @@ def func(details): def field_group_at_path(dotted_path, fields): - '''Returns the ['fields'] hash at the dotted_path.''' + """Returns the ['fields'] hash at the dotted_path.""" path = dotted_path.split('.') nesting = fields for next_field in path: @@ -163,17 +163,17 @@ def field_group_at_path(dotted_path, fields): def calculate_final_values(fields): - ''' + """ This function navigates all fields recursively. It populates a few more values for the fields, especially path-based values like flat_name. - ''' + """ visitor.visit_fields_with_path(fields, field_finalizer) def field_finalizer(details, path): - '''This is the function called by the visitor to perform the work of calculate_final_values''' + """This is the function called by the visitor to perform the work of calculate_final_values""" name_array = path + [details['field_details']['node_name']] flat_name = '.'.join(name_array) details['field_details']['flat_name'] = flat_name diff --git a/scripts/schema/loader.py b/scripts/schema/loader.py index 04f3218ae4..a662622274 100644 --- a/scripts/schema/loader.py +++ b/scripts/schema/loader.py @@ -1,6 +1,5 @@ import copy import glob -import os import yaml from generators import ecs_helpers @@ -109,12 +108,12 @@ def read_schema_blob(blob, ref): def nest_schema(raw, file_name): - ''' + """ Raw schema files are an array of schema details: [{'name': 'base', ...}] This function loops over the array (usually 1 schema per file) and turns it into a dict with the schema name as the key: { 'base': { 'name': 'base', ...}} - ''' + """ fields = {} for schema in raw: if 'name' not in schema: @@ -251,3 +250,33 @@ def merge_fields(a, b): a[key].setdefault('fields', {}) a[key]['fields'] = merge_fields(a[key]['fields'], b[key]['fields']) return a + + +def load_yaml_file(file_name): + with open(file_name) as f: + return yaml.safe_load(f.read()) + + +# You know, for silent tests +def warn(message): + print(message) + + +def eval_globs(globs): + """Accepts an array of glob patterns or file names, returns the array of actual files""" + all_files = [] + for g in globs: + new_files = glob.glob(g) + if len(new_files) == 0: + warn("{} did not match any files".format(g)) + else: + all_files.extend(new_files) + return all_files + + +def load_definitions(file_globs): + sets = [] + for f in eval_globs(file_globs): + raw = load_yaml_file(f) + sets.append(raw) + return sets diff --git a/scripts/schema/oss.py b/scripts/schema/oss.py deleted file mode 100644 index bfc07bb071..0000000000 --- a/scripts/schema/oss.py +++ /dev/null @@ -1,31 +0,0 @@ -# This script performs a best effort fallback of basic data types to equivalent -# OSS data types. -# Note however that not all basic data types have an OSS replacement. -# -# The way this script is currently written, it has to be run on the fields *before* -# the cleaner script applies defaults, as there's no concept of defaults here. -# But since it navigates using the visitor script, it can easily be moved around -# in the chain, provided we add support for defaults as well. -# -# For now, no warning is output on basic fields that don't have a fallback. -# This could be improved if ECS starts using such types. - -from schema import visitor - -TYPE_FALLBACKS = { - 'constant_keyword': 'keyword', - 'wildcard': 'keyword', - 'version': 'keyword' -} - - -def fallback(fields): - """Verify all fields for basic data type usage, and fallback to an OSS equivalent if appropriate.""" - visitor.visit_fields(fields, field_func=perform_fallback) - - -def perform_fallback(field): - """Performs a best effort fallback of basic data types to equivalent OSS data types.""" - fallback_type = TYPE_FALLBACKS.get(field['field_details']['type']) - if fallback_type: - field['field_details']['type'] = fallback_type diff --git a/scripts/schema/subset_filter.py b/scripts/schema/subset_filter.py index 791d6d3cc7..8c91929f0d 100644 --- a/scripts/schema/subset_filter.py +++ b/scripts/schema/subset_filter.py @@ -1,8 +1,6 @@ -import glob -import yaml import os from generators import intermediate_files -from schema import cleaner +from schema import cleaner, loader # This script takes all ECS and custom fields already loaded, and lets users # filter out the ones they don't need. @@ -22,7 +20,7 @@ def filter(fields, subset_file_globs, out_dir): def combine_all_subsets(subsets): - '''Merges N subsets into one. Strips top level 'name' and 'fields' keys as well as non-ECS field options since we can't know how to merge those.''' + """Merges N subsets into one. Strips top level 'name' and 'fields' keys as well as non-ECS field options since we can't know how to merge those.""" merged_subset = {} for subset in subsets: strip_non_ecs_options(subset['fields']) @@ -33,37 +31,12 @@ def combine_all_subsets(subsets): def load_subset_definitions(file_globs): if not file_globs: return [] - subsets = [] - for f in eval_globs(file_globs): - raw = load_yaml_file(f) - subsets.append(raw) + subsets = loader.load_definitions(file_globs) if not subsets: raise ValueError('--subset specified, but no subsets found in {}'.format(file_globs)) return subsets -def load_yaml_file(file_name): - with open(file_name) as f: - return yaml.safe_load(f.read()) - - -def eval_globs(globs): - '''Accepts an array of glob patterns or file names, returns the array of actual files''' - all_files = [] - for g in globs: - new_files = glob.glob(g) - if len(new_files) == 0: - warn("{} did not match any files".format(g)) - else: - all_files.extend(new_files) - return all_files - - -# You know, for silent tests -def warn(message): - print(message) - - ecs_options = ['fields', 'enabled', 'index'] @@ -75,7 +48,7 @@ def strip_non_ecs_options(subset): def merge_subsets(a, b): - '''Merges field subset definitions together. The b subset is merged into the a subset. Assumes that subsets have been stripped of non-ecs options.''' + """Merges field subset definitions together. The b subset is merged into the a subset. Assumes that subsets have been stripped of non-ecs options.""" for key in b: if key not in a: a[key] = b[key] @@ -96,7 +69,7 @@ def merge_subsets(a, b): def extract_matching_fields(fields, subset_definitions): - '''Removes fields that are not in the subset definition. Returns a copy without modifying the input fields dict.''' + """Removes fields that are not in the subset definition. Returns a copy without modifying the input fields dict.""" retained_fields = {x: fields[x].copy() for x in subset_definitions} for key, val in subset_definitions.items(): retained_fields[key]['field_details'] = fields[key]['field_details'].copy() diff --git a/scripts/schema/visitor.py b/scripts/schema/visitor.py index 5c2e030da5..3c3d762bad 100644 --- a/scripts/schema/visitor.py +++ b/scripts/schema/visitor.py @@ -1,5 +1,5 @@ def visit_fields(fields, fieldset_func=None, field_func=None): - ''' + """ This function navigates the deeply nested tree structure and runs provided functions on each fieldset or field encountered (both optional). @@ -11,7 +11,7 @@ def visit_fields(fields, fieldset_func=None, field_func=None): The 'field_func(details)' provided will be called for each field, with the dictionary containing the field's details ({'field_details': {}, 'fields': {}). - ''' + """ for (name, details) in fields.items(): if fieldset_func and 'schema_details' in details: fieldset_func(details) @@ -24,14 +24,14 @@ def visit_fields(fields, fieldset_func=None, field_func=None): def visit_fields_with_path(fields, func, path=[]): - ''' + """ This function navigates the deeply nested tree structure and runs the provided function on all fields and field sets. The 'func' provided will be called for each field, with the dictionary containing their details ({'field_details': {}, 'fields': {}) as well as the path array leading to the location of the field in question. - ''' + """ for (name, details) in fields.items(): if 'field_details' in details: func(details, path) @@ -44,14 +44,14 @@ def visit_fields_with_path(fields, func, path=[]): def visit_fields_with_memo(fields, func, memo=None): - ''' + """ This function navigates the deeply nested tree structure and runs the provided function on all fields and field sets. The 'func' provided will be called for each field, with the dictionary containing their details ({'field_details': {}, 'fields': {}) as well as the 'memo' you pass in. - ''' + """ for (name, details) in fields.items(): if 'field_details' in details: func(details, memo) diff --git a/scripts/tests/unit/test_schema_cleaner.py b/scripts/tests/unit/test_schema_cleaner.py index 3a6a7830c7..a6c8fd7284 100644 --- a/scripts/tests/unit/test_schema_cleaner.py +++ b/scripts/tests/unit/test_schema_cleaner.py @@ -425,7 +425,7 @@ def test_multiline_short_override_description_warns_strict_disabled(self): self.fail("cleaner.single_line_short_override_description() raised Exception unexpectedly.") def test_clean(self): - '''A high level sanity test''' + """A high level sanity test""" fields = self.schema_process() cleaner.clean(fields) # schemas are processed diff --git a/scripts/tests/unit/test_schema_exclude_filter.py b/scripts/tests/unit/test_schema_exclude_filter.py new file mode 100644 index 0000000000..5b6cb5d6ad --- /dev/null +++ b/scripts/tests/unit/test_schema_exclude_filter.py @@ -0,0 +1,108 @@ +from schema import exclude_filter +import mock +import os +import sys +import unittest + +sys.path.append(os.path.join(os.path.dirname(__file__), '../..')) + + +class TestSchemaExcludeFilter(unittest.TestCase): + + def setUp(self): + self.maxDiff = None + + @mock.patch('schema.loader.warn') + def test_load_exclude_definitions_raises_when_no_exclude_found(self, mock_warn): + with self.assertRaisesRegex(ValueError, + "--exclude specified, but no exclusions found in \['foo\*.yml'\]"): + exclude_filter.load_exclude_definitions(['foo*.yml']) + + def test_exclude_field(self): + fields = {'my_field_set': {'fields': { + 'my_field_exclude': {'field_details': {'flat_name': 'my_field_set.my_field_exclude'}}, + 'my_field_persist': {'field_details': {'flat_name': 'my_field_set.my_field_persist'}}}}} + excludes = [ + [{'name': 'my_field_set', 'fields': [{'name': 'my_field_exclude'}]}]] + fields = exclude_filter.exclude_fields(fields, excludes) + expect_persisted = {'my_field_set': {'fields': { + 'my_field_persist': {'field_details': {'flat_name': 'my_field_set.my_field_persist'}}}}} + self.assertEqual(fields, expect_persisted) + + def test_exclude_field_deep_path(self): + fields = {'d0': {'fields': { + 'd1': {'field_details': {'flat_name': 'd0.d1'}, 'fields': { + 'd2': {'field_details': {'flat_name': 'd0.d1.d2'}, 'fields': { + 'd3': {'field_details': {'flat_name': 'd0.d1.d2.d3'}, 'fields': { + 'd4': {'field_details': {'flat_name': 'd0.d1.d2.d3.d4'}, 'fields': { + 'd5': {'field_details': {'flat_name': 'd0.d1.d2.d3.d4.d5'}}}}}}}}}}}}} + excludes = [[{'name': 'd0', 'fields': [{ + 'name': 'd1.d2.d3.d4.d5'}]}]] + fields = exclude_filter.exclude_fields(fields, excludes) + expect_persisted = {} + self.assertEqual(fields, expect_persisted) + + def test_exclude_field_dot_path(self): + fields = {'d0': {'fields': { + 'd1': {'field_details': {'flat_name': 'd0.d1'}, 'fields': { + 'd2': {'field_details': {'flat_name': 'd0.d1.d2'}, 'fields': { + 'd3': {'field_details': {'flat_name': 'd0.d1.d2.d3'}, 'fields': { + 'd4': {'field_details': {'flat_name': 'd0.d1.d2.d3.d4'}, 'fields': { + 'd5': {'field_details': {'flat_name': 'd0.d1.d2.d3.d4.d5'}}}}}}}}}}}}} + excludes = [[{'name': 'd0', 'fields': [{ + 'name': 'd1.d2.d3.d4.d5'}]}]] + fields = exclude_filter.exclude_fields(fields, excludes) + expect_persisted = {} + self.assertEqual(fields, expect_persisted) + + def test_exclude_field_base_always_persists(self): + fields = {'base': {'fields': { + 'd1': {'field_details': {'flat_name': 'base.d1'}, 'fields': { + 'd2': {'field_details': {'flat_name': 'base.d1.d2'}, 'fields': { + 'd3': {'field_details': {'flat_name': 'base.d1.d2.d3'}, 'fields': { + 'd4': {'field_details': {'flat_name': 'base.d1.d2.d3.d4'}, 'fields': { + 'd5': {'field_details': {'flat_name': 'base.d1.d2.d3.d4.d5'}}}}}}}}}}}}} + excludes = [[{'name': 'base', 'fields': [{ + 'name': 'd1.d2.d3.d4.d5'}]}]] + fields = exclude_filter.exclude_fields(fields, excludes) + expect_persisted = {'base': {'fields': {}}} + self.assertEqual(fields, expect_persisted) + + def test_exclude_fields(self): + fields = {'my_field_set': {'fields': { + 'my_field_exclude_1': {'field_details': {'flat_name': 'my_field_set.my_field_exclude_1'}}, + 'my_field_exclude_2': {'field_details': {'flat_name': 'my_field_set.my_field_exclude_2'}}}}} + excludes = [[{'name': 'my_field_set', 'fields': [ + {'name': 'my_field_exclude_1'}, {'name': 'my_field_exclude_2'}]}]] + fields = exclude_filter.exclude_fields(fields, excludes) + expect_persisted = {} + self.assertEqual(fields, expect_persisted) + + def test_exclude_non_existing_field_set(self): + fields = {'my_field_set': {'fields': { + 'my_field': {'field_details': {'flat_name': 'my_field_set.my_field'}}}}} + excludes = [[{'name': 'my_non_existing_field_set', 'fields': [ + {'name': 'my_field_exclude'}]}]] + with self.assertRaisesRegex(ValueError, + "--exclude specified, but no field my_non_existing_field_set.my_field_exclude found"): + exclude_filter.exclude_fields(fields, excludes) + + def test_exclude_non_existing_field(self): + fields = {'my_field_set': {'fields': { + 'my_field': {'field_details': {'flat_name': 'my_field_set.my_field'}}}}} + excludes = [[{'name': 'my_field_set', 'fields': [ + {'name': 'my_non_existing_field'}]}]] + with self.assertRaisesRegex(ValueError, + "--exclude specified, but no field my_field_set.my_non_existing_field found"): + exclude_filter.exclude_fields(fields, excludes) + + def test_exclude_non_existing_field_deep_path(self): + fields = {'d0': {'fields': { + 'd1': {'field_details': {'flat_name': 'd0.d1'}}, 'fields': { + 'd2': {'field_details': {'flat_name': 'd0.d1.d2'}}, 'fields': { + 'd3': {'field_details': {'flat_name': 'd0.d1.d2.d3'}}}}}}} + excludes = [[{'name': 'd0', 'fields': [{ + 'name': 'd1.d2.d3.d4.d5'}]}]] + with self.assertRaisesRegex(ValueError, + "--exclude specified, but no path to field d0.d1.d2.d3.d4.d5 found"): + exclude_filter.exclude_fields(fields, excludes) diff --git a/scripts/tests/unit/test_schema_loader.py b/scripts/tests/unit/test_schema_loader.py index fde33e0a1c..b9b263f5df 100644 --- a/scripts/tests/unit/test_schema_loader.py +++ b/scripts/tests/unit/test_schema_loader.py @@ -14,6 +14,14 @@ class TestSchemaLoader(unittest.TestCase): def setUp(self): self.maxDiff = None + @mock.patch('schema.loader.warn') + def test_eval_globs(self, mock_warn): + files = loader.eval_globs(['schemas/*.yml', 'missing*']) + self.assertTrue(mock_warn.called, "a warning should have been printed for missing*") + self.assertIn('schemas/base.yml', files) + self.assertEqual(list(filter(lambda f: f.startswith('missing'), files)), [], + "The 'missing*' pattern should not show up in the resulting files") + # Pseudo-fixtures def schema_base(self): diff --git a/scripts/tests/unit/test_schema_oss.py b/scripts/tests/unit/test_schema_oss.py deleted file mode 100644 index 910b7959ca..0000000000 --- a/scripts/tests/unit/test_schema_oss.py +++ /dev/null @@ -1,44 +0,0 @@ -import os -import pprint -import sys -import unittest - -sys.path.append(os.path.join(os.path.dirname(__file__), '../..')) - -from schema import oss -from schema import visitor - - -class TestSchemaOss(unittest.TestCase): - - def setUp(self): - self.maxDiff = None - - # Fallbacks - - def test_wildcard_fallback(self): - field = {'field_details': {'name': 'myfield', 'type': 'wildcard'}} - oss.perform_fallback(field) - self.assertEqual('keyword', field['field_details']['type']) - - def test_version_fallback(self): - field = {'field_details': {'name': 'myfield', 'type': 'version'}} - oss.perform_fallback(field) - self.assertEqual('keyword', field['field_details']['type']) - - def test_constant_keyword_fallback(self): - field = {'field_details': {'name': 'myfield', 'type': 'constant_keyword'}} - oss.perform_fallback(field) - self.assertEqual('keyword', field['field_details']['type']) - - # Not falling back - - def test_basic_without_fallback(self): - field = {'field_details': {'name': 'myfield', 'type': 'histogram'}} - oss.perform_fallback(field) - self.assertEqual('histogram', field['field_details']['type']) - - def test_oss_no_fallback_needed(self): - field = {'field_details': {'name': 'myfield', 'type': 'keyword'}} - oss.perform_fallback(field) - self.assertEqual('keyword', field['field_details']['type']) diff --git a/scripts/tests/unit/test_schema_subset_filter.py b/scripts/tests/unit/test_schema_subset_filter.py index f108dba4bc..e7ae5fd211 100644 --- a/scripts/tests/unit/test_schema_subset_filter.py +++ b/scripts/tests/unit/test_schema_subset_filter.py @@ -14,15 +14,7 @@ class TestSchemaSubsetFilter(unittest.TestCase): def setUp(self): self.maxDiff = None - @mock.patch('schema.subset_filter.warn') - def test_eval_globs(self, mock_warn): - files = subset_filter.eval_globs(['schemas/*.yml', 'missing*']) - self.assertTrue(mock_warn.called, "a warning should have been printed for missing*") - self.assertIn('schemas/base.yml', files) - self.assertEqual(list(filter(lambda f: f.startswith('missing'), files)), [], - "The 'missing*' pattern should not show up in the resulting files") - - @mock.patch('schema.subset_filter.warn') + @mock.patch('schema.loader.warn') def test_load_subset_definitions_raises_when_no_subset_found(self, mock_warn): with self.assertRaisesRegex(ValueError, "--subset specified, but no subsets found in \['foo\*.yml'\]"): diff --git a/scripts/use-cases.py b/scripts/use-cases.py deleted file mode 100644 index 99df56a8aa..0000000000 --- a/scripts/use-cases.py +++ /dev/null @@ -1,95 +0,0 @@ -import yaml -import os -import argparse -from helper import * -import os.path - - -def write_stdout(): - - schema = get_schema() - flat_schema = create_flat_schema(schema) - - links = "" - for file in sorted(os.listdir("./use-cases")): - - output = "" - - if not file.endswith(".yml"): - continue - - use_case = read_use_case_file("./use-cases/" + file) - - # Intentionally a relative link, to avoid leaving forked repo or branch - schema_link = "use-cases/" - # Link list to field prefixes - links += " * [{}]({}{}.md)\n".format(use_case["title"], schema_link, use_case["name"]) - - output += "## {} use case\n\n".format(use_case["title"]) - output += "{}\n\n".format(use_case["description"]) - - fields = [] - for use_case_section in use_case["fields"]: - # In case a description exists for a prefix, add it as field with .* - if "description" in use_case_section and use_case_section["description"] != "": - fields.append({ - "name": use_case_section["name"] + ".*", - "description": use_case_section["description"], - "type": "", - "level": "", - "example": "", - "ecs": False, - }) - - for section_fields in use_case_section["fields"]: - # Complete ECS fields with ECS information if not set - if section_fields["name"] in flat_schema: - section_fields["ecs"] = True - section_fields["type"] = flat_schema[section_fields["name"]]["type"] - section_fields["level"] = flat_schema[section_fields["name"]]["level"] - if section_fields["description"] == "": - section_fields["description"] = flat_schema[section_fields["name"]]["description"] - if section_fields["example"] == "": - section_fields["example"] = flat_schema[section_fields["name"]]["example"] - else: - section_fields["ecs"] = False - section_fields["level"] = "(use case)" - - fields.append(section_fields) - - global_fields = {"name": use_case["name"], "title": use_case["title"], "description": "", "fields": fields} - # Generate use cases with a relative link to access field definitions - output += get_markdown_section(global_fields, "###", "../README.md") + "\n" - - # Write output to /use-cases/use_case["name"].md file - # Adjust links - - with open("./use-cases/" + use_case["name"] + ".md", "w") as f: - f.write(output) - - print("\n" + links + "\n\n") - - -def create_flat_schema(schema): - fields = {} - - for namespace in schema: - if len(namespace["fields"]) == 0: - continue - - for f in namespace["fields"]: - fields[f["name"]] = f - - return fields - - -if __name__ == "__main__": - - parser = argparse.ArgumentParser() - parser.add_argument('--stdout', help='output to stdout instead of files') - args = parser.parse_args() - - # Outputs html of links to each use case (for the readme) - # and generates an html file per use case besides their each yaml file. - if args.stdout == "true": - write_stdout() diff --git a/use-cases/README.md b/use-cases/README.md deleted file mode 100644 index cb658c9432..0000000000 --- a/use-cases/README.md +++ /dev/null @@ -1,10 +0,0 @@ -# Use cases - -The use cases directory is used to define the fields for some more specific use -cases. All the fields used here are inherited from ECS but are referenced here -to have more details on it. - -## Generate - -Execute `make` in the project's root directory after modifying any of the `.yml` -files in this directory to rebuild the `.md` Markdown files. diff --git a/use-cases/apm.md b/use-cases/apm.md deleted file mode 100644 index 8a63ae0aa3..0000000000 --- a/use-cases/apm.md +++ /dev/null @@ -1,21 +0,0 @@ -## APM use case - -ECS usage for the APM data. - -### APM fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| *id* | *Unique id to describe the event.* | (use case) | keyword | `8a4f500d` | -| [@timestamp](../README.md#@timestamp) | Timestamp when the event was created in the app / service. | core | date | `2016-05-23T08:05:34.853Z` | -| *agent.** | *The agent fields are used to describe which agent did send the information.
* | | | | -| [agent.version](../README.md#agent.version) | APM Agent version. | core | keyword | `3.14.0` | -| [agent.name](../README.md#agent.name) | APM agent name. | core | keyword | `elastic-node` | -| *service.** | *The service fields describe the service inside which the APM agent is running.
* | | | | -| [service.id](../README.md#service.id) | Unique identifier of the running service. | core | keyword | `d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6` | -| [service.name](../README.md#service.name) | Name of the service the agent is running in. This is normally a user defined name. | core | keyword | `user-service` | -| [service.version](../README.md#service.version) | Version of the service the agent is running in. This depends on if the service is given a version. | core | keyword | `3.2.4` | - - - diff --git a/use-cases/apm.yml b/use-cases/apm.yml deleted file mode 100644 index 677ce3c996..0000000000 --- a/use-cases/apm.yml +++ /dev/null @@ -1,58 +0,0 @@ -title: APM -name: apm -description: - ECS usage for the APM data. -fields: -- name: base - fields: - - name: id - type: keyword - description: > - Unique id to describe the event. - example: 8a4f500d - - name: "@timestamp" - type: date - phase: 1 - example: "2016-05-23T08:05:34.853Z" - description: > - Timestamp when the event was created in the app / service. - -- name: agent - description: > - The agent fields are used to describe which agent did send the information. - fields: - - name: version - type: keyword - description: > - APM Agent version. - example: 3.14.0 - - name: name - type: keyword - description: > - APM agent name. - example: elastic-node - -- name: service - description: > - The service fields describe the service inside which the APM agent is running. - fields: - - name: id - type: keyword - description: > - Unique identifier of the running service. - example: d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6 - - - name: name - type: keyword - example: "user-service" - description: > - Name of the service the agent is running in. This is normally a - user defined name. - - - name: version - type: keyword - example: "3.2.4" - description: > - Version of the service the agent is running in. This depends - on if the service is given a version. - diff --git a/use-cases/auditbeat.md b/use-cases/auditbeat.md deleted file mode 100644 index dff825a597..0000000000 --- a/use-cases/auditbeat.md +++ /dev/null @@ -1,44 +0,0 @@ -## Auditbeat use case - -ECS usage in Auditbeat. - -### Auditbeat fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| [event.module](../README.md#event.module) | Auditbeat module name. | core | keyword | `apache` | -| *file.** | *File attributes.
* | | | | -| [file.path](../README.md#file.path) | The path to the file. | extended | keyword | `/home/alice/example.png` | -| [file.target_path](../README.md#file.target_path) | The target path for symlinks. | extended | keyword | | -| [file.type](../README.md#file.type) | The file type (file, dir, or symlink). | extended | keyword | `file` | -| [file.device](../README.md#file.device) | The device. | extended | keyword | `sda` | -| [file.inode](../README.md#file.inode) | The inode representing the file in the filesystem. | extended | keyword | `256383` | -| [file.uid](../README.md#file.uid) | The user ID (UID) or security identifier (SID) of the file owner. | extended | keyword | `1001` | -| [file.owner](../README.md#file.owner) | The file owner's username. | extended | keyword | `alice` | -| [file.gid](../README.md#file.gid) | The primary group ID (GID) of the file. | extended | keyword | `1001` | -| [file.group](../README.md#file.group) | The primary group name of the file. | extended | keyword | `alice` | -| [file.mode](../README.md#file.mode) | The mode of the file in octal representation. | extended | keyword | `416` | -| [file.size](../README.md#file.size) | The file size in bytes (field is only added when `type` is `file`). | extended | long | `16384` | -| [file.mtime](../README.md#file.mtime) | The last modified time of the file (time when content was modified). | extended | date | | -| [file.ctime](../README.md#file.ctime) | The last change time of the file (time when metadata was changed). | extended | date | | -| *hash.** | *Hash fields used in Auditbeat.
The hash field contains cryptographic hashes of data associated with the event (such as a file). The keys are names of cryptographic algorithms. The values are encoded as hexidecimal (lower-case).
All fields in user can have one or multiple entries.
* | | | | -| *hash.blake2b_256* | *BLAKE2b-256 hash of the file.* | (use case) | keyword | | -| *hash.blake2b_384* | *BLAKE2b-384 hash of the file.* | (use case) | keyword | | -| *hash.blake2b_512* | *BLAKE2b-512 hash of the file.* | (use case) | keyword | | -| [hash.md5](../README.md#hash.md5) | MD5 hash. | extended | keyword | | -| [hash.sha1](../README.md#hash.sha1) | SHA-1 hash. | extended | keyword | | -| *hash.sha224* | *SHA-224 hash (SHA-2 family).* | (use case) | keyword | | -| [hash.sha256](../README.md#hash.sha256) | SHA-256 hash (SHA-2 family). | extended | keyword | | -| *hash.sha384* | *SHA-384 hash (SHA-2 family).* | (use case) | keyword | | -| [hash.sha512](../README.md#hash.sha512) | SHA-512 hash (SHA-2 family). | extended | keyword | | -| *hash.sha512_224* | *SHA-512/224 hash (SHA-2 family).* | (use case) | keyword | | -| *hash.sha512_256* | *SHA-512/256 hash (SHA-2 family).* | (use case) | keyword | | -| *hash.sha3_224* | *SHA3-224 hash (SHA-3 family).* | (use case) | keyword | | -| *hash.sha3_256* | *SHA3-256 hash (SHA-3 family).* | (use case) | keyword | | -| *hash.sha3_384* | *SHA3-384 hash (SHA-3 family).* | (use case) | keyword | | -| *hash.sha3_512* | *SHA3-512 hash (SHA-3 family).* | (use case) | keyword | | -| *hash.xxh64* | *XX64 hash of the file.* | (use case) | keyword | | - - - diff --git a/use-cases/auditbeat.yml b/use-cases/auditbeat.yml deleted file mode 100644 index b7b25fcb48..0000000000 --- a/use-cases/auditbeat.yml +++ /dev/null @@ -1,156 +0,0 @@ -title: Auditbeat -name: auditbeat -description: - ECS usage in Auditbeat. -fields: -- name: event - fields: - - name: module - description: > - Auditbeat module name. -- name: file - title: File - description: > - File attributes. - fields: - - name: path - type: keyword - description: The path to the file. - - - name: target_path - type: keyword - description: The target path for symlinks. - - - name: type - type: keyword - description: The file type (file, dir, or symlink). - - - name: device - type: keyword - description: The device. - - - name: inode - type: keyword - description: The inode representing the file in the filesystem. - - - name: uid - type: keyword - description: > - The user ID (UID) or security identifier (SID) of the file owner. - - - name: owner - type: keyword - description: The file owner's username. - - - name: gid - type: keyword - description: The primary group ID (GID) of the file. - - - name: group - type: keyword - description: The primary group name of the file. - - - name: mode - type: keyword - example: 0640 - description: The mode of the file in octal representation. - - - name: size - type: long - description: The file size in bytes (field is only added when `type` is `file`). - - - name: mtime - type: date - description: The last modified time of the file (time when content was modified). - - - name: ctime - type: date - description: The last change time of the file (time when metadata was changed). - - -- name: hash - group: 3 - description: > - Hash fields used in Auditbeat. - - The hash field contains cryptographic hashes of data associated with the event - (such as a file). The keys are names of cryptographic algorithms. The values - are encoded as hexidecimal (lower-case). - - All fields in user can have one or multiple entries. - fields: - - name: blake2b_256 - type: keyword - description: BLAKE2b-256 hash of the file. - - - name: blake2b_384 - type: keyword - description: BLAKE2b-384 hash of the file. - - - name: blake2b_512 - type: keyword - description: BLAKE2b-512 hash of the file. - - - name: md5 - type: keyword - description: > - MD5 hash. - - - name: sha1 - type: keyword - description: > - SHA-1 hash. - - - name: sha224 - type: keyword - description: > - SHA-224 hash (SHA-2 family). - - - name: sha256 - type: keyword - description: > - SHA-256 hash (SHA-2 family). - - - name: sha384 - type: keyword - description: > - SHA-384 hash (SHA-2 family). - - - name: sha512 - type: keyword - description: > - SHA-512 hash (SHA-2 family). - - - name: sha512_224 - type: keyword - description: > - SHA-512/224 hash (SHA-2 family). - - - name: sha512_256 - type: keyword - description: > - SHA-512/256 hash (SHA-2 family). - - - name: sha3_224 - type: keyword - description: > - SHA3-224 hash (SHA-3 family). - - - name: sha3_256 - type: keyword - description: > - SHA3-256 hash (SHA-3 family). - - - name: sha3_384 - type: keyword - description: > - SHA3-384 hash (SHA-3 family). - - - name: sha3_512 - type: keyword - description: > - SHA3-512 hash (SHA-3 family). - - - name: xxh64 - type: keyword - description: XX64 hash of the file. diff --git a/use-cases/beats.md b/use-cases/beats.md deleted file mode 100644 index c96e994b2d..0000000000 --- a/use-cases/beats.md +++ /dev/null @@ -1,18 +0,0 @@ -## Beats use case - -ECS fields used in Beats. - -### Beats fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| *id* | *Unique id to describe the event.* | (use case) | keyword | `8a4f500d` | -| *timestamp* | *Timestamp when the event was created.* | (use case) | date | `2016-05-23T08:05:34.853Z` | -| *agent.** | *The agent fields are used to describe by which beat the information was collected.
* | | | | -| [agent.version](../README.md#agent.version) | Beat version. | core | keyword | `6.0.0-rc2` | -| [agent.name](../README.md#agent.name) | Beat name. | core | keyword | `filebeat` | -| [agent.id](../README.md#agent.id) | Unique beat identifier. | core | keyword | `8a4f500d` | - - - diff --git a/use-cases/beats.yml b/use-cases/beats.yml deleted file mode 100644 index 92911bb4dd..0000000000 --- a/use-cases/beats.yml +++ /dev/null @@ -1,38 +0,0 @@ -title: Beats -name: beats -description: - ECS fields used in Beats. -fields: -- name: base - fields: - - name: id - type: keyword - description: > - Unique id to describe the event. - example: 8a4f500d - - name: timestamp - type: date - phase: 1 - example: "2016-05-23T08:05:34.853Z" - description: > - Timestamp when the event was created. - -- name: agent - description: > - The agent fields are used to describe by which beat the information was collected. - fields: - - name: version - type: keyword - description: > - Beat version. - example: 6.0.0-rc2 - - name: name - type: keyword - description: > - Beat name. - example: filebeat - - name: id - type: keyword - description: > - Unique beat identifier. - example: 8a4f500d diff --git a/use-cases/filebeat-apache-access.md b/use-cases/filebeat-apache-access.md deleted file mode 100644 index a9ef41840f..0000000000 --- a/use-cases/filebeat-apache-access.md +++ /dev/null @@ -1,29 +0,0 @@ -## Filebeat Apache use case - -ECS fields used in Filebeat for the apache module. - -### Filebeat Apache fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| *id* | *Unique id to describe the event.* | (use case) | keyword | `8a4f500d` | -| [@timestamp](../README.md#@timestamp) | Timestamp of the log line after processing. | core | date | `2016-05-23T08:05:34.853Z` | -| [message](../README.md#message) | Log message of the event | core | text | `Hello World` | -| [event.module](../README.md#event.module) | Currently fileset.module | core | keyword | `apache` | -| [event.dataset](../README.md#event.dataset) | Currenly fileset.name | core | keyword | `access` | -| [source.ip](../README.md#source.ip) | Source ip of the request. Currently apache.access.remote_ip | core | ip | `192.168.1.1` | -| [user.name](../README.md#user.name) | User name in the request. Currently apache.access.user_name | core | keyword | `ruflin` | -| *http.method* | *Http method, currently apache.access.method* | (use case) | keyword | `GET` | -| *http.url* | *Http url, currently apache.access.url* | (use case) | keyword | `http://elastic.co/` | -| [http.version](../README.md#http.version) | Http version, currently apache.access.http_version | extended | keyword | `1.1` | -| *http.response.code* | *Http response code, currently apache.access.response_code* | (use case) | keyword | `404` | -| *http.response.body_sent.bytes* | *Http response body bytes sent, currently apache.access.body_sent.bytes* | (use case) | long | `117` | -| *http.referer* | *Http referrer code, currently apache.access.referrer
NOTE: In the RFC its misspell as referer and has become accepted standard* | (use case) | keyword | `http://elastic.co/` | -| *user_agent.** | *User agent fields as in schema. Currently under apache.access.user_agent.*
* | | | | -| [user_agent.original](../README.md#user_agent.original) | Original user agent. Currently apache.access.agent | extended | keyword | `http://elastic.co/` | -| *geoip.** | *User agent fields as in schema. Currently under apache.access.geoip.*
These are extracted from source.ip
Should they be under source.geoip?
* | | | | -| *geoip....* | *All geoip fields.* | (use case) | keyword | | - - - diff --git a/use-cases/filebeat-apache-access.yml b/use-cases/filebeat-apache-access.yml deleted file mode 100644 index ac8c004095..0000000000 --- a/use-cases/filebeat-apache-access.yml +++ /dev/null @@ -1,111 +0,0 @@ -title: Filebeat Apache -name: filebeat-apache-access -description: - ECS fields used in Filebeat for the apache module. -fields: -- name: base - fields: - - name: id - type: keyword - description: > - Unique id to describe the event. - example: 8a4f500d - - name: "@timestamp" - type: date - example: "2016-05-23T08:05:34.853Z" - description: > - Timestamp of the log line after processing. - - name: message - type: date - example: "Hello World" - description: > - Log message of the event - -- name: event - fields: - - name: module - type: keyword - description: > - Currently fileset.module - example: apache - - name: dataset - type: keyword - example: access - description: > - Currenly fileset.name - -- name: source - fields: - - name: ip - type: ip - description: > - Source ip of the request. Currently apache.access.remote_ip - example: 192.168.1.1 - -- name: user - fields: - - name: name - type: keyword - description: > - User name in the request. Currently apache.access.user_name - example: ruflin - -- name: http - fields: - - name: method - type: keyword - description: > - Http method, currently apache.access.method - example: GET - - name: url - type: keyword - description: > - Http url, currently apache.access.url - example: "http://elastic.co/" - - name: version - type: keyword - description: > - Http version, currently apache.access.http_version - example: 1.1 - - name: response.code - type: keyword - description: > - Http response code, currently apache.access.response_code - example: 404 - - name: response.body_sent.bytes - type: long - description: > - Http response body bytes sent, currently apache.access.body_sent.bytes - example: 117 - - name: referer - type: keyword - description: > - Http referrer code, currently apache.access.referrer - - NOTE: In the RFC its misspell as referer and has become accepted standard - example: http://elastic.co/ - -- name: user_agent - title: User Agent - description: > - User agent fields as in schema. Currently under apache.access.user_agent.* - fields: - - name: original - type: keyword - description: > - Original user agent. Currently apache.access.agent - example: http://elastic.co/ - -- name: geoip - title: Geoip - description: > - User agent fields as in schema. Currently under apache.access.geoip.* - - These are extracted from source.ip - - Should they be under source.geoip? - fields: - - name: ... - type: keyword - description: > - All geoip fields. diff --git a/use-cases/kubernetes.md b/use-cases/kubernetes.md deleted file mode 100644 index 5588da6060..0000000000 --- a/use-cases/kubernetes.md +++ /dev/null @@ -1,21 +0,0 @@ -## Kubernetes use case - -You can monitor containers running in a Kubernetes cluster by adding Kubernetes-specific information under `kubernetes.` - - -### Kubernetes fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| [container.id](../README.md#container.id) | Unique container id. | core | keyword | `fdbef803fa2b` | -| [container.name](../README.md#container.name) | Container name. | extended | keyword | | -| [host.hostname](../README.md#host.hostname) | Hostname of the host.
It normally contains what the `hostname` command returns on the host machine. | core | keyword | `kube-high-cpu-42` | -| *kubernetes.pod.name* | *Kubernetes pod name* | (use case) | keyword | `foo-webserver` | -| *kubernetes.namespace* | *Kubernetes namespace* | (use case) | keyword | `foo-team` | -| *kubernetes.labels* | *Kubernetes labels map* | (use case) | object | | -| *kubernetes.annotations* | *Kubernetes annotations map* | (use case) | object | | -| *kubernetes.container.name* | *Kubernetes container name. This name is unique within the pod only. It is different from the `container.name` field.* | (use case) | keyword | | - - - diff --git a/use-cases/kubernetes.yml b/use-cases/kubernetes.yml deleted file mode 100644 index f4c46317f8..0000000000 --- a/use-cases/kubernetes.yml +++ /dev/null @@ -1,51 +0,0 @@ -name: kubernetes -title: Kubernetes -description: > - You can monitor containers running in a Kubernetes cluster by adding - Kubernetes-specific information under `kubernetes.` - -fields: -- name: container - fields: - - - name: id - example: fdbef803fa2b - - - name: name - -- name: host - fields: - - - name: hostname - example: kube-high-cpu-42 - -- name: kubernetes - fields: - - - name: pod.name - type: keyword - description: > - Kubernetes pod name - example: foo-webserver - - - name: namespace - type: keyword - description: > - Kubernetes namespace - example: foo-team - - - name: labels - type: object - description: > - Kubernetes labels map - - - name: annotations - type: object - description: > - Kubernetes annotations map - - - name: container.name - type: keyword - description: > - Kubernetes container name. This name is unique within the pod only. - It is different from the `container.name` field. diff --git a/use-cases/logging.md b/use-cases/logging.md deleted file mode 100644 index be7efd0b6b..0000000000 --- a/use-cases/logging.md +++ /dev/null @@ -1,22 +0,0 @@ -## Logging use case - -ECS fields used in logging use cases. - -### Logging fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| *id* | *Unique id of the log entry.* | (use case) | keyword | `8a4f500d` | -| *timestamp* | *Timestamp of the log line.* | (use case) | date | `2016-05-23T08:05:34.853Z` | -| [message](../README.md#message) | The log message.
This can contain the full log line or based on the processing only the extracted message part. This is expected to be human readable. | core | text | `Hello World` | -| *hostname* | *Hostname extracted from the log line.* | (use case) | keyword | `www.example.com` | -| *ip* | *IP Address extracted from the log line. Can be IPv4 or IPv6.* | (use case) | ip | `192.168.1.12` | -| [log.level](../README.md#log.level) | Log level field. Is expected to be `WARN`, `ERR`, `INFO` etc. | core | keyword | `ERR` | -| *log.line* | *Line number the log event was collected from.* | (use case) | long | `18` | -| *log.offset* | *Offset of the log event.* | (use case) | long | `12` | -| *source.** | *Describes from where the log entries come from.
* | | | | -| *source.path* | *File path of the file the data is harvested from.* | (use case) | keyword | `/var/log/test.log` | - - - diff --git a/use-cases/logging.yml b/use-cases/logging.yml deleted file mode 100644 index f307ced148..0000000000 --- a/use-cases/logging.yml +++ /dev/null @@ -1,67 +0,0 @@ -title: Logging -name: logging -description: - ECS fields used in logging use cases. -fields: -- name: base - fields: - - name: id - type: keyword - description: > - Unique id of the log entry. - example: 8a4f500d - - name: timestamp - type: date - example: "2016-05-23T08:05:34.853Z" - description: > - Timestamp of the log line. - - name: message - type: text - required: true - example: "Hello World" - description: > - The log message. - - This can contain the full log line or based on the processing - only the extracted message part. This is expected to be human readable. - - - name: hostname - type: keyword - example: "www.example.com" - description: > - Hostname extracted from the log line. - - name: ip - type: ip - example: "192.168.1.12" - description: > - IP Address extracted from the log line. Can be IPv4 or IPv6. - - -- name: log - fields: - - name: level - type: keyword - description: > - Log level field. Is expected to be `WARN`, `ERR`, `INFO` etc. - example: ERR - - name: line - type: long - description: > - Line number the log event was collected from. - example: 18 - - name: offset - type: long - description: > - Offset of the log event. - example: 12 - - -- name: source - description: > - Describes from where the log entries come from. - fields: - - name: path - type: keyword - description: > - File path of the file the data is harvested from. - example: /var/log/test.log diff --git a/use-cases/metricbeat.md b/use-cases/metricbeat.md deleted file mode 100644 index c573a7897e..0000000000 --- a/use-cases/metricbeat.md +++ /dev/null @@ -1,31 +0,0 @@ -## Metricbeat use case - -ECS fields used Metricbeat. - -### Metricbeat fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| *id* | *Unique id to describe the event.* | (use case) | keyword | `8a4f500d` | -| *timestamp* | *Timestamp when the event was created.* | (use case) | date | `2016-05-23T08:05:34.853Z` | -| [agent.version](../README.md#agent.version) | Beat version. | core | keyword | `6.0.0-rc2` | -| [agent.name](../README.md#agent.name) | Beat name. | core | keyword | `filebeat` | -| [agent.id](../README.md#agent.id) | Unique beat identifier. | core | keyword | `8a4f500d` | -| *service.** | *The service fields describe the service for / from which the data was collected.
If logs or metrics are collected from Redis, `service.name` would be `redis`. This allows to find and correlate logs for a specicic service or even version with `service.version`.
* | | | | -| [service.id](../README.md#service.id) | Unique identifier of the running service.
This id should uniquely identify this service. This makes it possible to correlate logs and metrics for one specific service. For example in case of issues with one redis instance, it's possible to filter on the id to see metrics and logs for this single instance. | core | keyword | `d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6` | -| [service.name](../README.md#service.name) | Name of the service data is collected from.
The name is normally the same as the module name. | core | keyword | `elasticsearch` | -| [service.version](../README.md#service.version) | Version of the service the data was collected from.
This allows to look at a data set only for a specific version of a service. | core | keyword | `3.2.4` | -| *service.host* | *Host address that is used to connect to the service.
This normally contains hostname + port.
REVIEW: Should this be service.uri instead, sometimes it's more then just the host? It could also include a path or the protocol.* | (use case) | keyword | `elasticsearch:9200` | -| *request.rtt* | *Request round trip time.
How long did the request take to fetch metrics from the service.
REVIEW: THIS DOES NOT EXIST YET IN ECS.* | (use case) | long | `115` | -| *error.** | *Error namespace
Use for errors which can happen during fetching information for a service.
* | | | | -| [error.message](../README.md#error.message) | Error message returned by the service during fetching metrics. | core | text | | -| [error.code](../README.md#error.code) | Error code returned by the service during fetching metrics. | core | keyword | | -| [host.hostname](../README.md#host.hostname) | Hostname of the system metricbeat is running on or user defined name. | core | keyword | | -| *host.timezone.offset.sec* | *Timezone offset of the host in seconds.* | (use case) | long | | -| [host.id](../README.md#host.id) | Unique host id. | core | keyword | | -| [event.module](../README.md#event.module) | Name of the module this data is coming from. | core | keyword | `mysql` | -| [event.dataset](../README.md#event.dataset) | Name of the dataset.
This contains the information which is currently stored in metricset.name and metricset.module. | core | keyword | `stats` | - - - diff --git a/use-cases/metricbeat.yml b/use-cases/metricbeat.yml deleted file mode 100644 index 74b8217fd4..0000000000 --- a/use-cases/metricbeat.yml +++ /dev/null @@ -1,146 +0,0 @@ -title: Metricbeat -name: metricbeat -description: - ECS fields used Metricbeat. -fields: -- name: base - fields: - - name: id - type: keyword - description: > - Unique id to describe the event. - example: 8a4f500d - - name: timestamp - type: date - phase: 1 - example: "2016-05-23T08:05:34.853Z" - description: > - Timestamp when the event was created. - -- name: agent - fields: - - name: version - type: keyword - description: > - Beat version. - example: 6.0.0-rc2 - - name: name - type: keyword - description: > - Beat name. - example: filebeat - - name: id - type: keyword - description: > - Unique beat identifier. - example: 8a4f500d - -- name: service - description: > - The service fields describe the service for / from which the data was collected. - - If logs or metrics are collected from Redis, `service.name` would be `redis`. This allows - to find and correlate logs for a specicic service or even version with `service.version`. - - fields: - - name: id - type: keyword - description: > - Unique identifier of the running service. - - This id should uniquely identify this service. This makes it possible - to correlate logs and metrics for one specific service. For example - in case of issues with one redis instance, it's possible to filter on the id - to see metrics and logs for this single instance. - - example: d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6 - - - name: name - type: keyword - example: "elasticsearch" - description: > - Name of the service data is collected from. - - The name is normally the same as the module name. - - - name: version - type: keyword - example: "3.2.4" - description: > - Version of the service the data was collected from. - - This allows to look at a data set only for a specific version of a service. - - - name: host - type: keyword - example: "elasticsearch:9200" - description: > - Host address that is used to connect to the service. - - This normally contains hostname + port. - - REVIEW: Should this be service.uri instead, sometimes it's more then just the host? - It could also include a path or the protocol. - -- name: request - fields: - - name: rtt - type: long - description: > - Request round trip time. - - How long did the request take to fetch metrics from the service. - - REVIEW: THIS DOES NOT EXIST YET IN ECS. - - example: 115 - -- name: error - description: > - Error namespace - - Use for errors which can happen during fetching information for a service. - fields: - - name: message - type: text - description: > - Error message returned by the service during fetching metrics. - - - name: code - type: long - description: > - Error code returned by the service during fetching metrics. - -- name: host - fields: - - name: hostname - type: keyword - description: > - Hostname of the system metricbeat is running on or user defined name. - - - name: timezone.offset.sec - type: long - description: > - Timezone offset of the host in seconds. - - - name: id - type: keyword - description: > - Unique host id. - -- name: event - fields: - - name: module - type: keyword - description: > - Name of the module this data is coming from. - example: mysql - - name: dataset - type: keyword - description: > - Name of the dataset. - - This contains the information which is currently stored in metricset.name - and metricset.module. - - example: stats diff --git a/use-cases/web-logs.md b/use-cases/web-logs.md deleted file mode 100644 index 57f9a96062..0000000000 --- a/use-cases/web-logs.md +++ /dev/null @@ -1,29 +0,0 @@ -## Parsing web server logs use case - -Representing web server access logs in ECS. -This use case uses previous definitions for `http` and `user_agent` fields sets, which were taken out of ECS temporarily for Beta1. Their official definition in ECS is expected to change slightly. -Using the fields as represented here is not expected to conflict with ECS, but may require a transition, when they are re-introduced officially. - -### Parsing web server logs fields - - -| Field | Description | Level | Type | Example | -|---|---|---|---|---| -| [@timestamp](../README.md#@timestamp) | Time at which the response was sent, and the web server log created. | core | date | `2016-05-23T08:05:34.853Z` | -| *http.** | *Fields related to HTTP requests and responses.
* | | | | -| [http.request.method](../README.md#http.request.method) | Http request method. | extended | keyword | `GET, POST, PUT` | -| [http.request.referrer](../README.md#http.request.referrer) | Referrer for this HTTP request. | extended | keyword | `https://blog.example.com/` | -| [http.response.status_code](../README.md#http.response.status_code) | Http response status code. | extended | long | `404` | -| [http.response.body.content](../README.md#http.response.body.content) | The full http response body. | extended | keyword | `Hello world` | -| [http.version](../README.md#http.version) | Http version. | extended | keyword | `1.1` | -| *user_agent.** | *The user_agent fields normally come from a browser request. They often show up in web service logs coming from the parsed user agent string.
* | | | | -| [user_agent.original](../README.md#user_agent.original) | Unparsed version of the user_agent. | extended | keyword | `Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1` | -| *user_agent.device* | *Name of the physical device.* | (use case) | keyword | | -| [user_agent.version](../README.md#user_agent.version) | Version of the physical device. | extended | keyword | `12.0` | -| *user_agent.major* | *Major version of the user agent.* | (use case) | long | | -| *user_agent.minor* | *Minor version of the user agent.* | (use case) | long | | -| *user_agent.patch* | *Patch version of the user agent.* | (use case) | keyword | | -| [user_agent.name](../README.md#user_agent.name) | Name of the user agent. | extended | keyword | `Chrome` | - - - diff --git a/use-cases/web-logs.yml b/use-cases/web-logs.yml deleted file mode 100644 index 06a8f47d9e..0000000000 --- a/use-cases/web-logs.yml +++ /dev/null @@ -1,111 +0,0 @@ -title: Parsing web server logs -name: web-logs -description: - Representing web server access logs in ECS. - - This use case uses previous definitions for `http` and `user_agent` fields sets, - which were taken out of ECS temporarily for Beta1. Their official definition - in ECS is expected to change slightly. - - Using the fields as represented here is not expected to conflict with ECS, - but may require a transition, when they are re-introduced officially. -fields: - -- name: base - fields: - - name: "@timestamp" - type: date - phase: 1 - example: "2016-05-23T08:05:34.853Z" - description: > - Time at which the response was sent, and the web server log created. - -- name: http - title: HTTP - group: 2 - description: > - Fields related to HTTP requests and responses. - type: group - fields: - - - name: request.method - type: keyword - description: > - Http request method. - example: GET, POST, PUT - - - name: request.referrer - type: keyword - description: > - Referrer for this HTTP request. - example: https://blog.example.com/ - - - name: response.status_code - type: long - description: > - Http response status code. - example: 404 - - - name: response.body.content - type: keyword - description: > - The full http response body. - example: Hello world - - - name: version - type: keyword - description: > - Http version. - example: 1.1 - -- name: user_agent - title: User agent - group: 2 - description: > - The user_agent fields normally come from a browser request. They often - show up in web service logs coming from the parsed user agent string. - type: group - fields: - - - name: original - level: extended - type: keyword - description: > - Unparsed version of the user_agent. - - - name: device - level: extended - type: keyword - description: > - Name of the physical device. - - - name: version - level: extended - type: keyword - description: > - Version of the physical device. - - - name: major - level: extended - type: long - description: > - Major version of the user agent. - - - name: minor - level: extended - type: long - description: > - Minor version of the user agent. - - - name: patch - level: extended - type: keyword - description: > - Patch version of the user agent. - - - name: name - level: extended - type: keyword - example: Chrome - description: > - Name of the user agent.