Skip to content

Commit

Permalink
[Filebeat] Rename awscloudwatch input to aws-cloudwatch input (#22228) (
Browse files Browse the repository at this point in the history
#22524)

* Rename awscloudwatch input to aws-cloudwatch input

* add deprecation warning message

(cherry picked from commit 28ad00b)
  • Loading branch information
kaiyan-sheng authored Nov 11, 2020
1 parent bcbef06 commit d242a6d
Show file tree
Hide file tree
Showing 8 changed files with 43 additions and 29 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,7 @@ from being added to events by default. {pull}18159[18159]
- Add max_number_of_messages config into s3 input. {pull}21993[21993]
- Update Okta documentation for new stateful restarts. {pull}22091[22091]
- Add SSL option to checkpoint module {pull}19560[19560]
- Rename awscloudwatch input to aws-cloudwatch. {pull}22228[22228]
- Rename google-pubsub input to gcp-pubsub. {pull}22213[22213]
- Copy tag names from MISP data into events. {pull}21664[21664]
- Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291]
Expand Down
14 changes: 7 additions & 7 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ grouped in the following categories:
* <<exported-fields-apache>>
* <<exported-fields-auditd>>
* <<exported-fields-aws>>
* <<exported-fields-awscloudwatch>>
* <<exported-fields-aws-cloudwatch>>
* <<exported-fields-azure>>
* <<exported-fields-barracuda>>
* <<exported-fields-beat-common>>
Expand Down Expand Up @@ -2292,21 +2292,21 @@ type: keyword

--

[[exported-fields-awscloudwatch]]
== awscloudwatch fields
[[exported-fields-aws-cloudwatch]]
== aws-cloudwatch fields

Fields from AWS CloudWatch logs.



[float]
=== awscloudwatch
=== aws-cloudwatch

Fields from AWS CloudWatch logs.



*`awscloudwatch.log_group`*::
*`aws-cloudwatch.log_group`*::
+
--
The name of the log group to which this event belongs.
Expand All @@ -2315,7 +2315,7 @@ type: keyword

--

*`awscloudwatch.log_stream`*::
*`aws-cloudwatch.log_stream`*::
+
--
The name of the log stream to which this event belongs.
Expand All @@ -2324,7 +2324,7 @@ type: keyword

--

*`awscloudwatch.ingestion_time`*::
*`aws-cloudwatch.ingestion_time`*::
+
--
The time the event was ingested in AWS CloudWatch.
Expand Down
3 changes: 2 additions & 1 deletion filebeat/docs/filebeat-options.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ subdirectories of a directory.

You can configure {beatname_uc} to use the following inputs:

* <<{beatname_lc}-input-aws-cloudwatch>>
* <<{beatname_lc}-input-azure-eventhub>>
* <<{beatname_lc}-input-cloudfoundry>>
* <<{beatname_lc}-input-container>>
Expand All @@ -84,7 +85,7 @@ You can configure {beatname_uc} to use the following inputs:

include::multiline.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-awscloudwatch.asciidoc[]
include::../../x-pack/filebeat/docs/inputs/input-aws-cloudwatch.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-azure-eventhub.asciidoc[]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

:libbeat-xpack-dir: ../../../../x-pack/libbeat

:type: awscloudwatch
:type: aws-cloudwatch

[id="{beatname_lc}-input-{type}"]
=== AWS CloudWatch input
Expand All @@ -13,7 +13,7 @@

beta[]

`awscloudwatch` input can be used to retrieve all logs from all log streams in a
`aws-cloudwatch` input can be used to retrieve all logs from all log streams in a
specific log group. `filterLogEvents` AWS API is used to list log events from
the specified log group. Amazon CloudWatch Logs can be used to store log files
from Amazon Elastic Compute Cloud(EC2), AWS CloudTrail, Route53, and other sources.
Expand All @@ -29,14 +29,14 @@ separate source of logs in CloudWatch Logs makes up a separate log stream.
["source","yaml",subs="attributes"]
----
{beatname_lc}.inputs:
- type: awscloudwatch
- type: aws-cloudwatch
log_group_arn: arn:aws:logs:us-east-1:428152502467:log-group:test:*
scan_frequency: 1m
credential_profile_name: elastic-beats
start_position: beginning
----

The `awscloudwatch` input supports the following configuration options plus the
The `aws-cloudwatch` input supports the following configuration options plus the
<<{beatname_lc}-input-{type}-common-options>> described later.

[float]
Expand Down Expand Up @@ -100,12 +100,12 @@ collecting logs from the same region and AWS account.

[float]
==== `aws credentials`
In order to make AWS API calls, `awscloudwatch` input requires AWS credentials.
In order to make AWS API calls, `aws-cloudwatch` input requires AWS credentials.
Please see <<aws-credentials-config,AWS credentials options>> for more details.

[float]
=== AWS Permissions
Specific AWS permissions are required for IAM user to access awscloudwatch:
Specific AWS permissions are required for IAM user to access aws-cloudwatch:
----
logs:FilterLogEvents
----
Expand Down
6 changes: 3 additions & 3 deletions x-pack/filebeat/input/awscloudwatch/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
- key: awscloudwatch
title: "awscloudwatch"
- key: aws-cloudwatch
title: "aws-cloudwatch"
description: >
Fields from AWS CloudWatch logs.
fields:
- name: awscloudwatch
- name: aws-cloudwatch
type: group
default_field: false
description: >
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/input/awscloudwatch/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ type config struct {
func defaultConfig() config {
return config{
ForwarderConfig: harvester.ForwarderConfig{
Type: "awscloudwatch",
Type: "aws-cloudwatch",
},
StartPosition: "beginning",
ScanFrequency: 10 * time.Second,
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/input/awscloudwatch/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

32 changes: 22 additions & 10 deletions x-pack/filebeat/input/awscloudwatch/input.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,21 @@ import (
awscommon "github.com/elastic/beats/v7/x-pack/libbeat/common/aws"
)

const inputName = "awscloudwatch"
const (
inputName = "aws-cloudwatch"
oldInputName = "awscloudwatch"
)

func init() {
err := input.Register(inputName, NewInput)
if err != nil {
panic(errors.Wrapf(err, "failed to register %v input", inputName))
}

err = input.Register(oldInputName, NewInput)
if err != nil {
panic(errors.Wrapf(err, "failed to register %v input", oldInputName))
}
}

// awsCloudWatchInput is a input for AWS CloudWatch logs
Expand All @@ -41,11 +49,11 @@ type awsCloudWatchInput struct {
awsConfig awssdk.Config

logger *logp.Logger
outlet channel.Outleter // Output of received awscloudwatch logs.
outlet channel.Outleter // Output of received aws-cloudwatch logs.
inputCtx *channelContext

workerOnce sync.Once // Guarantees that the worker goroutine is only started once.
workerWg sync.WaitGroup // Waits on awscloudwatch worker goroutine.
workerWg sync.WaitGroup // Waits on aws-cloudwatch worker goroutine.
stopOnce sync.Once
close chan struct{}

Expand All @@ -69,17 +77,21 @@ func (c *channelContext) Err() error {
}
func (c *channelContext) Value(key interface{}) interface{} { return nil }

// NewInput creates a new awscloudwatch input
// NewInput creates a new aws-cloudwatch input
func NewInput(cfg *common.Config, connector channel.Connector, context input.Context) (input.Input, error) {
cfgwarn.Beta("awsclouwatch input type is used")
cfgwarn.Beta("aws-clouwatch input type is used")
logger := logp.NewLogger(inputName)

// Extract and validate the input's configuration.
config := defaultConfig()
if err := cfg.Unpack(&config); err != nil {
return nil, errors.Wrap(err, "failed unpacking config")
}
logger.Debug("awscloudwatch input config = ", config)
logger.Debug("aws-cloudwatch input config = ", config)

if config.Type == oldInputName {
logger.Warnf("%s input name is deprecated, please use %s instead", oldInputName, inputName)
}

if config.LogGroupARN != "" {
logGroupName, regionName, err := parseARN(config.LogGroupARN)
Expand Down Expand Up @@ -122,8 +134,8 @@ func (in *awsCloudWatchInput) Run() {
in.workerOnce.Do(func() {
in.workerWg.Add(1)
go func() {
in.logger.Infof("awscloudwatch input worker for log group: '%v' has started", in.config.LogGroupName)
defer in.logger.Infof("awscloudwatch input worker for log group '%v' has stopped.", in.config.LogGroupName)
in.logger.Infof("aws-cloudwatch input worker for log group: '%v' has started", in.config.LogGroupName)
defer in.logger.Infof("aws-cloudwatch input worker for log group '%v' has stopped.", in.config.LogGroupName)
defer in.workerWg.Done()
in.run()
}()
Expand Down Expand Up @@ -300,12 +312,12 @@ func (in *awsCloudWatchInput) forwardEvent(event beat.Event) error {
return nil
}

// Stop stops the awscloudwatch input
// Stop stops the aws-cloudwatch input
func (in *awsCloudWatchInput) Stop() {
in.stopOnce.Do(func() {
defer in.outlet.Close()
close(in.close)
in.logger.Info("Stopping awscloudwatch input")
in.logger.Info("Stopping aws-cloudwatch input")
})
}

Expand Down

0 comments on commit d242a6d

Please sign in to comment.