Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Rename awscloudwatch input to aws-cloudwatch input #22228

Merged
merged 5 commits into from
Nov 10, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -707,6 +707,7 @@ from being added to events by default. {pull}18159[18159]
- Add SSL option to checkpoint module {pull}19560[19560]
- Add max_number_of_messages config into s3 input. {pull}21993[21993]
- Update Okta documentation for new stateful restarts. {pull}22091[22091]
- Rename awscloudwatch input to aws-cloudwatch. {pull}22228[22228]
- Rename google-pubsub input to gcp-pubsub. {pull}22213[22213]
- Copy tag names from MISP data into events. {pull}21664[21664]
- Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291]
Expand Down
14 changes: 7 additions & 7 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ grouped in the following categories:
* <<exported-fields-apache>>
* <<exported-fields-auditd>>
* <<exported-fields-aws>>
* <<exported-fields-awscloudwatch>>
* <<exported-fields-aws-cloudwatch>>
* <<exported-fields-azure>>
* <<exported-fields-barracuda>>
* <<exported-fields-beat-common>>
Expand Down Expand Up @@ -2292,21 +2292,21 @@ type: keyword

--

[[exported-fields-awscloudwatch]]
== awscloudwatch fields
[[exported-fields-aws-cloudwatch]]
== aws-cloudwatch fields

Fields from AWS CloudWatch logs.



[float]
=== awscloudwatch
=== aws-cloudwatch

Fields from AWS CloudWatch logs.



*`awscloudwatch.log_group`*::
*`aws-cloudwatch.log_group`*::
+
--
The name of the log group to which this event belongs.
Expand All @@ -2315,7 +2315,7 @@ type: keyword

--

*`awscloudwatch.log_stream`*::
*`aws-cloudwatch.log_stream`*::
+
--
The name of the log stream to which this event belongs.
Expand All @@ -2324,7 +2324,7 @@ type: keyword

--

*`awscloudwatch.ingestion_time`*::
*`aws-cloudwatch.ingestion_time`*::
+
--
The time the event was ingested in AWS CloudWatch.
Expand Down
3 changes: 2 additions & 1 deletion filebeat/docs/filebeat-options.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ subdirectories of a directory.

You can configure {beatname_uc} to use the following inputs:

* <<{beatname_lc}-input-aws-cloudwatch>>
* <<{beatname_lc}-input-azure-eventhub>>
* <<{beatname_lc}-input-cloudfoundry>>
* <<{beatname_lc}-input-container>>
Expand All @@ -84,7 +85,7 @@ You can configure {beatname_uc} to use the following inputs:

include::multiline.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-awscloudwatch.asciidoc[]
include::../../x-pack/filebeat/docs/inputs/input-aws-cloudwatch.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-azure-eventhub.asciidoc[]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

:libbeat-xpack-dir: ../../../../x-pack/libbeat

:type: awscloudwatch
:type: aws-cloudwatch

[id="{beatname_lc}-input-{type}"]
=== AWS CloudWatch input
Expand All @@ -13,7 +13,7 @@

beta[]

`awscloudwatch` input can be used to retrieve all logs from all log streams in a
`aws-cloudwatch` input can be used to retrieve all logs from all log streams in a
specific log group. `filterLogEvents` AWS API is used to list log events from
the specified log group. Amazon CloudWatch Logs can be used to store log files
from Amazon Elastic Compute Cloud(EC2), AWS CloudTrail, Route53, and other sources.
Expand All @@ -29,14 +29,14 @@ separate source of logs in CloudWatch Logs makes up a separate log stream.
["source","yaml",subs="attributes"]
----
{beatname_lc}.inputs:
- type: awscloudwatch
- type: aws-cloudwatch
log_group_arn: arn:aws:logs:us-east-1:428152502467:log-group:test:*
scan_frequency: 1m
credential_profile_name: elastic-beats
start_position: beginning
----

The `awscloudwatch` input supports the following configuration options plus the
The `aws-cloudwatch` input supports the following configuration options plus the
<<{beatname_lc}-input-{type}-common-options>> described later.

[float]
Expand Down Expand Up @@ -100,12 +100,12 @@ collecting logs from the same region and AWS account.

[float]
==== `aws credentials`
In order to make AWS API calls, `awscloudwatch` input requires AWS credentials.
In order to make AWS API calls, `aws-cloudwatch` input requires AWS credentials.
Please see <<aws-credentials-config,AWS credentials options>> for more details.

[float]
=== AWS Permissions
Specific AWS permissions are required for IAM user to access awscloudwatch:
Specific AWS permissions are required for IAM user to access aws-cloudwatch:
----
logs:FilterLogEvents
----
Expand Down
6 changes: 3 additions & 3 deletions x-pack/filebeat/input/awscloudwatch/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
- key: awscloudwatch
title: "awscloudwatch"
- key: aws-cloudwatch
title: "aws-cloudwatch"
description: >
Fields from AWS CloudWatch logs.
fields:
- name: awscloudwatch
- name: aws-cloudwatch
type: group
default_field: false
description: >
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/input/awscloudwatch/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ type config struct {
func defaultConfig() config {
return config{
ForwarderConfig: harvester.ForwarderConfig{
Type: "awscloudwatch",
Type: "aws-cloudwatch",
},
StartPosition: "beginning",
ScanFrequency: 10 * time.Second,
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/input/awscloudwatch/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

32 changes: 22 additions & 10 deletions x-pack/filebeat/input/awscloudwatch/input.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,21 @@ import (
awscommon "github.com/elastic/beats/v7/x-pack/libbeat/common/aws"
)

const inputName = "awscloudwatch"
const (
inputName = "aws-cloudwatch"
oldInputName = "awscloudwatch"
)

func init() {
err := input.Register(inputName, NewInput)
if err != nil {
panic(errors.Wrapf(err, "failed to register %v input", inputName))
}

err = input.Register(oldInputName, NewInput)
ChrsMark marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
panic(errors.Wrapf(err, "failed to register %v input", oldInputName))
}
}

// awsCloudWatchInput is a input for AWS CloudWatch logs
Expand All @@ -41,11 +49,11 @@ type awsCloudWatchInput struct {
awsConfig awssdk.Config

logger *logp.Logger
outlet channel.Outleter // Output of received awscloudwatch logs.
outlet channel.Outleter // Output of received aws-cloudwatch logs.
inputCtx *channelContext

workerOnce sync.Once // Guarantees that the worker goroutine is only started once.
workerWg sync.WaitGroup // Waits on awscloudwatch worker goroutine.
workerWg sync.WaitGroup // Waits on aws-cloudwatch worker goroutine.
stopOnce sync.Once
close chan struct{}

Expand All @@ -69,17 +77,21 @@ func (c *channelContext) Err() error {
}
func (c *channelContext) Value(key interface{}) interface{} { return nil }

// NewInput creates a new awscloudwatch input
// NewInput creates a new aws-cloudwatch input
func NewInput(cfg *common.Config, connector channel.Connector, context input.Context) (input.Input, error) {
cfgwarn.Beta("awsclouwatch input type is used")
cfgwarn.Beta("aws-clouwatch input type is used")
logger := logp.NewLogger(inputName)

// Extract and validate the input's configuration.
config := defaultConfig()
if err := cfg.Unpack(&config); err != nil {
return nil, errors.Wrap(err, "failed unpacking config")
}
logger.Debug("awscloudwatch input config = ", config)
logger.Debug("aws-cloudwatch input config = ", config)

if config.Type == oldInputName {
logger.Warnf("%s input name is deprecated, please use %s instead", oldInputName, inputName)
}

if config.LogGroupARN != "" {
logGroupName, regionName, err := parseARN(config.LogGroupARN)
Expand Down Expand Up @@ -122,8 +134,8 @@ func (in *awsCloudWatchInput) Run() {
in.workerOnce.Do(func() {
in.workerWg.Add(1)
go func() {
in.logger.Infof("awscloudwatch input worker for log group: '%v' has started", in.config.LogGroupName)
defer in.logger.Infof("awscloudwatch input worker for log group '%v' has stopped.", in.config.LogGroupName)
in.logger.Infof("aws-cloudwatch input worker for log group: '%v' has started", in.config.LogGroupName)
defer in.logger.Infof("aws-cloudwatch input worker for log group '%v' has stopped.", in.config.LogGroupName)
defer in.workerWg.Done()
in.run()
}()
Expand Down Expand Up @@ -300,12 +312,12 @@ func (in *awsCloudWatchInput) forwardEvent(event beat.Event) error {
return nil
}

// Stop stops the awscloudwatch input
// Stop stops the aws-cloudwatch input
func (in *awsCloudWatchInput) Stop() {
in.stopOnce.Do(func() {
defer in.outlet.Close()
close(in.close)
in.logger.Info("Stopping awscloudwatch input")
in.logger.Info("Stopping aws-cloudwatch input")
})
}

Expand Down