Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Fix hardcoded amazonaws.com endpoint #24861

Merged
merged 4 commits into from
Apr 9, 2021
Merged

[Filebeat] Fix hardcoded amazonaws.com endpoint #24861

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8c8f721
#24420: Fix hardcoded amazonaws.com endpoint
legoguy1000 Mar 30, 2021
d375c63
Update unit tests
legoguy1000 Mar 31, 2021
5569cf0
Update tests per review
legoguy1000 Apr 2, 2021
85d5b0f
update error output and add test
legoguy1000 Apr 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Improve Cisco ASA/FTD parsing of messages - better support for identity FW messages. Change network.bytes, source.bytes, and destination.bytes to long from integer since value can exceed integer capacity. Add descriptions for various processors for easier pipeline editing in Kibana UI. {pull}23766[23766]
- Updating Oauth2 flow for m365_defender fileset. {pull}24829[24829]
- Improve PanOS parsing and ingest pipeline. {issue}22413[22413] {issue}22748[22748] {pull}24799[24799]
- Fix S3 input validation for non amazonaws.com domains. {issue}24420[24420] {pull}24861[24861]

*Heartbeat*

Expand Down
6 changes: 3 additions & 3 deletions x-pack/filebeat/input/awss3/collector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,14 +226,14 @@ func (c *s3Collector) changeVisibilityTimeout(queueURL string, visibilityTimeout
return err
}

func getRegionFromQueueURL(queueURL string) (string, error) {
func getRegionFromQueueURL(queueURL string, endpoint string) (string, error) {
// get region from queueURL
// Example: https://sqs.us-east-1.amazonaws.com/627959692251/test-s3-logs
queueURLSplit := strings.Split(queueURL, ".")
if queueURLSplit[0] == "https://sqs" && queueURLSplit[2] == "amazonaws" {
if queueURLSplit[0] == "https://sqs" && ((endpoint != "" && strings.Join(queueURLSplit[2:], ".") == endpoint) || queueURLSplit[2] == "amazonaws") {
return queueURLSplit[1], nil
}
return "", fmt.Errorf("queueURL is not in format: https://sqs.{REGION_ENDPOINT}.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME}")
return "", fmt.Errorf("queueURL is not in format: https://sqs.{REGION_ENDPOINT}.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME} or https://sqs.{REGION_ENDPOINT}.{ENDPOINT}/{ACCOUNT_NUMBER}/{QUEUE_NAME}")
legoguy1000 marked this conversation as resolved.
Show resolved Hide resolved
}

// handle message
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/input/awss3/input.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func (in *s3Input) createCollector(ctx v2.Context, pipeline beat.Pipeline) (*s3C
return nil, err
}

regionName, err := getRegionFromQueueURL(in.config.QueueURL)
regionName, err := getRegionFromQueueURL(in.config.QueueURL, in.config.AwsConfig.Endpoint)
if err != nil {
err := fmt.Errorf("getRegionFromQueueURL failed: %w", err)
log.Error(err)
Expand Down