Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aws] Remove hardcoded event.dataset field and use ECS field instead #8811

Merged
merged 9 commits into from
Jan 11, 2024
Merged

[aws] Remove hardcoded event.dataset field and use ECS field instead #8811

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
490cf29
Remove hardcoded event.dataset field from AWS integration
kaiyan-sheng Dec 28, 2023
1bf9ca7
remove hardcoded event.dataset from aws_logs
kaiyan-sheng Dec 28, 2023
4ef8376
add changelog
kaiyan-sheng Dec 28, 2023
47b795f
change cloudwatch_logs default dataset to aws.cloudwatch_logs
kaiyan-sheng Jan 2, 2024
40ce525
add extra line
kaiyan-sheng Jan 2, 2024
964e2cd
undo change for default dataset name and make event.dataset a constan…
kaiyan-sheng Jan 4, 2024
ec61c83
remove type constant_keyword
kaiyan-sheng Jan 4, 2024
3dd3741
force event.dataset type to constant_keyword in aws integration besid…
kaiyan-sheng Jan 8, 2024
4ba50a0
Merge remote-tracking branch 'upstream/main' into remove_event_dataset
kaiyan-sheng Jan 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/aws/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "2.11.2"
changes:
- description: Remove hardcoded event.dataset field and use ecs instead.
type: bugfix
link: https://github.com/elastic/integrations/pull/8811
- version: "2.11.1"
changes:
- description: Improve wording on milliseconds.
Expand Down
4 changes: 0 additions & 4 deletions packages/aws/data_stream/apigateway_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.apigateway_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/apigateway_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,6 @@
name: container.name
- external: ecs
name: log.file.path
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/apigateway_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.apigateway_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/apigateway_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/billing/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.billing
3 changes: 3 additions & 0 deletions packages/aws/data_stream/billing/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/cloudfront_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.cloudfront_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/cloudfront_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,3 +154,6 @@
name: container.labels
- external: ecs
name: container.name
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/cloudtrail/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.cloudtrail
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/cloudtrail/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,6 @@
name: tls.cipher
- external: ecs
name: tls.client.server_name
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/cloudwatch_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.cloudwatch_logs
2 changes: 2 additions & 0 deletions packages/aws/data_stream/cloudwatch_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,5 @@
name: container.labels
- external: ecs
name: container.name
- external: ecs
name: event.dataset
6 changes: 1 addition & 5 deletions packages/aws/data_stream/cloudwatch_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,4 @@
- name: event.module
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.cloudwatch_metrics
value: aws
3 changes: 3 additions & 0 deletions packages/aws/data_stream/cloudwatch_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/dynamodb/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.dynamodb
3 changes: 3 additions & 0 deletions packages/aws/data_stream/dynamodb/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/ebs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.ebs
3 changes: 3 additions & 0 deletions packages/aws/data_stream/ebs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/ec2_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.ec2_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/ec2_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,6 @@
name: container.name
- external: ecs
name: log.file.path
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/ec2_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.ec2_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/ec2_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/ecs_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.ecs_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/ecs_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/elb_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.elb_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/elb_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,6 @@
name: container.name
- external: ecs
name: log.file.path
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/elb_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.elb_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/elb_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/emr_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.emr_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/emr_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,6 @@
name: log.level
- external: ecs
name: log.file.path
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/emr_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.emr_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/emr_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/firewall_logs/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.firewall_logs
- name: input.type
type: keyword
description: Input type
3 changes: 3 additions & 0 deletions packages/aws/data_stream/firewall_logs/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,6 @@
name: container.name
- external: ecs
name: log.file.path
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/firewall_metrics/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.firewall_metrics
3 changes: 3 additions & 0 deletions packages/aws/data_stream/firewall_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/guardduty/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@
type: constant_keyword
description: Event module.
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: aws.guardduty
- name: '@timestamp'
type: date
description: Event timestamp.
3 changes: 3 additions & 0 deletions packages/aws/data_stream/guardduty/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,6 @@
name: user.roles
- external: ecs
name: container.security_context.privileged
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/inspector/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@
type: constant_keyword
description: Event module.
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset.
value: aws.inspector
- name: '@timestamp'
type: date
description: Event timestamp.
3 changes: 3 additions & 0 deletions packages/aws/data_stream/inspector/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,6 @@
name: vulnerability.score.version
- external: ecs
name: vulnerability.severity
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/kinesis/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.kinesis
3 changes: 3 additions & 0 deletions packages/aws/data_stream/kinesis/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/lambda/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.lambda
3 changes: 3 additions & 0 deletions packages/aws/data_stream/lambda/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/natgateway/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.natgateway
3 changes: 3 additions & 0 deletions packages/aws/data_stream/natgateway/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
4 changes: 0 additions & 4 deletions packages/aws/data_stream/rds/fields/base-fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@
type: constant_keyword
description: Event module
value: aws
- name: event.dataset
type: constant_keyword
description: Event dataset
value: aws.rds
3 changes: 3 additions & 0 deletions packages/aws/data_stream/rds/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@
- name: agent.id
external: ecs
dimension: true
- external: ecs
name: event.dataset
type: constant_keyword
Loading