Set agent.name to hostname by default #18000

andrewkroh · 2020-04-26T23:11:31Z

What does this PR do?

Since ECS does not define agent.hostname it will be removed in a future release.
In order to always have a field available to identify the agent by name we will
set the agent.name to hostname unless the user has provided a custom name.

Why is it important?

This puts the hostname into events by default. This will be important for cases where events may not contain the forwarding machine's hostname. For example when forwarding syslog data we may not have the host or observer namespaces populated.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
~~- [ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Relates #16377

Logs

Event output when name is unset.

  "agent": {
    "name": "mac.example.com",
    "type": "filebeat",
    "version": "8.0.0",
    "hostname": "mac.example.com",
    "ephemeral_id": "a690e034-28ac-413b-8bad-b618b57a2d79",
    "id": "fb930b06-9125-40b5-902c-bf0b0f077592"
  },

Event when name: my_custom_name is set:

  "agent": {
    "type": "filebeat",
    "version": "8.0.0",
    "hostname": "mac.example.com",
    "ephemeral_id": "c421fd5a-613b-4c40-aaa3-128c7e84a47c",
    "id": "b7e68e5b-cc5b-43b3-9e6b-0e74d2abde4d",
    "name": "my_custom_name"
  }

elasticmachine · 2020-04-30T00:40:03Z

Pinging @elastic/siem (Team:SIEM)

exekias · 2020-04-30T14:16:49Z

libbeat/publisher/processing/default.go

+			"type":         info.Beat,                 // Per ECS this is not a valid type value.
+			"ephemeral_id": info.EphemeralID.String(), // Not in ECS.
+			"hostname":     info.Hostname,
+			"id":           info.ID.String(), // Not in ECS.


should "name" also be here?

nevermind, I see this has not changed

Correct, I didn't change observer. I had to separate them to avoid introducing changes to it. observer does need some changes, but that's another task - probably for 8.0.

elasticmachine · 2020-04-30T15:31:17Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [andrewkroh commented: run tests]
Start Time: 2020-05-02T02:53:47.881+0000
Duration: 85 min 25 sec (5125116)
Commit: db5eb1c

Test stats 🧪

Test	Results
Failed	0
Passed	7688
Skipped	1210
Total	8898

Steps errors

Expand to view the steps failures

Name: Make -C generator/_templates/metricbeat test
- Description: make -C generator/_templates/metricbeat test
- Result: FAILURE
- Duration: 2 min 38 sec<
- Start Time: 2020-05-02T03:24:58.364+0000
Name: Make -C generator/_templates/beat test
- Description: make -C generator/_templates/beat test
- Result: FAILURE
- Duration: 1 min 50 sec<
- Start Time: 2020-05-02T03:27:26.755+0000
Name: Make -C generator/_templates/metricbeat test
- Description: make -C generator/_templates/metricbeat test
- Result: FAILURE
- Duration: 1 min 24 sec<
- Start Time: 2020-05-02T03:29:54.114+0000
Name: Make -C generator/_templates/beat test
- Description: make -C generator/_templates/beat test
- Result: FAILURE
- Duration: 6 min 14 sec<
- Start Time: 2020-05-02T03:39:45.712+0000
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Result: FAILURE
- Duration: 1 min 27 sec<
- Start Time: 2020-05-02T03:46:02.118+0000

Since ECS does not define agent.hostname it will be removed in a future release. In order to always have a field available to identify the agent by name we will set the agent.name to hostname unless the user has provided a custom name. Relates elastic#16377

andrewkroh · 2020-05-02T02:53:25Z

run tests

Since ECS does not define agent.hostname it will be removed in a future release. In order to always have a field available to identify the agent by name we will set the agent.name to hostname unless the user has provided a custom name. Relates elastic#16377 (cherry picked from commit 3711ee6)

Since ECS does not define agent.hostname it will be removed in a future release. In order to always have a field available to identify the agent by name we will set the agent.name to hostname unless the user has provided a custom name. Relates #16377 (cherry picked from commit 3711ee6)

andrewkroh added review libbeat needs_backport PR is waiting to be backported to other branches. ecs labels Apr 26, 2020

andrewkroh force-pushed the feature/libbeat/default-agent-name-hostname branch from e0905d7 to 612a17a Compare April 26, 2020 23:12

andrewkroh mentioned this pull request Apr 26, 2020

Beats use of non-ECS field agent.hostname causing user confusion #16377

Closed

4 tasks

andrewkroh marked this pull request as ready for review April 26, 2020 23:22

andrewkroh requested a review from a team as a code owner April 26, 2020 23:22

andresrc added the [zube]: In Review label Apr 27, 2020

andrewkroh force-pushed the feature/libbeat/default-agent-name-hostname branch from c00f9fb to c0e59e7 Compare April 28, 2020 01:16

andrewkroh added the Team:SIEM label Apr 30, 2020

exekias reviewed Apr 30, 2020

View reviewed changes

exekias approved these changes Apr 30, 2020

View reviewed changes

andrewkroh added 4 commits May 1, 2020 20:32

Update field docs for agent.hostname

545f5b7

Update tests

c7a75fa

Update metricbeat field docs

db5eb1c

andrewkroh force-pushed the feature/libbeat/default-agent-name-hostname branch from 3d845bb to db5eb1c Compare May 2, 2020 00:34

andrewkroh merged commit 3711ee6 into elastic:master May 2, 2020

zube bot added [zube]: Done and removed [zube]: In Review labels May 2, 2020

andrewkroh mentioned this pull request May 2, 2020

Cherry-pick #18000 to 7.x: Set agent.name to hostname by default #18155

Merged

5 tasks

andrewkroh added v7.8.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 2, 2020

andresrc removed the [zube]: Done label May 4, 2020

ebeahan mentioned this pull request Oct 29, 2021

[RFC] Host and Hostname fields - Stage 0 elastic/ecs#1512

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set agent.name to hostname by default #18000

Set agent.name to hostname by default #18000

andrewkroh commented Apr 26, 2020 •

edited by andresrc

Loading

elasticmachine commented Apr 30, 2020

exekias Apr 30, 2020

exekias Apr 30, 2020

andrewkroh Apr 30, 2020

elasticmachine commented Apr 30, 2020 •

edited

Loading

Build stats

Test stats 🧪

andrewkroh commented May 2, 2020

Set agent.name to hostname by default #18000

Set agent.name to hostname by default #18000

Conversation

andrewkroh commented Apr 26, 2020 • edited by andresrc Loading

What does this PR do?

Why is it important?

Checklist

Related issues

Logs

elasticmachine commented Apr 30, 2020

exekias Apr 30, 2020

Choose a reason for hiding this comment

exekias Apr 30, 2020

Choose a reason for hiding this comment

andrewkroh Apr 30, 2020

Choose a reason for hiding this comment

elasticmachine commented Apr 30, 2020 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Steps errors

andrewkroh commented May 2, 2020

andrewkroh commented Apr 26, 2020 •

edited by andresrc

Loading

elasticmachine commented Apr 30, 2020 •

edited

Loading