Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename s3 input to aws-s3 input #23469

Merged
merged 8 commits into from
Jan 19, 2021
Merged

Rename s3 input to aws-s3 input #23469

merged 8 commits into from
Jan 19, 2021

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Jan 12, 2021

What does this PR do?

This PR is to rename Filebeat s3 input to aws-s3 input.
Closes elastic/integrations#488

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  1. Test the Filebeat aws-s3 input:
filebeat.inputs:
- type: aws-s3
  enabled: true
  queue_url: https://sqs.eu-central-1.amazonaws.com/428152502467/filebeat-aws-elb-test
  credential_profile_name: elastic-beats

This should start aws-s3 input with no error.

  1. Test Filebeat filesets that use aws-s3 input:
./filebeat modules enable aws

and change modules.d/aws.yml to enable a fileset in aws module. For example:

- module: aws
  cloudtrail:
    enabled: true
    var.queue_url: https://sqs.us-east-1.amazonaws.com/428152502467/test-fb-ks
    var.credential_profile_name: elastic-beats

Then run Filebeat and you should see no error. If there are new logs going into the S3 which generates new SQS messages in test-fb-ks, you should also see the events in Elasticsearch.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 12, 2021
@kaiyan-sheng kaiyan-sheng self-assigned this Jan 12, 2021
@kaiyan-sheng kaiyan-sheng added in progress Pull request is currently in progress. Team:Platforms Label for the Integrations - Platforms team labels Jan 12, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Platforms)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 12, 2021
@kaiyan-sheng kaiyan-sheng added the needs_backport PR is waiting to be backported to other branches. label Jan 12, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 12, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #23469 updated

    • Start Time: 2021-01-15T16:37:02.285+0000
  • Duration: 51 min 49 sec

  • Commit: e3a2beb

Test stats 🧪

Test Results
Failed 0
Passed 6555
Skipped 578
Total 7133

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6555
Skipped 578
Total 7133

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@bmorelli25
Copy link
Member

For the doc build, change this line:

https://github.com/elastic/beats/blame/18fb50493f6cd60eed1661caa3c4939de59db20e/filebeat/docs/filebeat-options.asciidoc#L79

It should be:

* <<{beatname_lc}-input-aws-s3>>

When I make this change the PR builds locally 🎉

@kaiyan-sheng kaiyan-sheng removed the in progress Pull request is currently in progress. label Jan 14, 2021
Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think filebeat/docs/aws-credentials-examples.asciidoc and x-pack/filebeat/_meta/config/filebeat.inputs.reference.xpack.yml.tmpl need updating too.

otherwise looks great.

@kaiyan-sheng
Copy link
Contributor Author

@leehinman Good catch! It's fixed now, thank you!

@kaiyan-sheng kaiyan-sheng merged commit 616266f into elastic:master Jan 19, 2021
@kaiyan-sheng kaiyan-sheng deleted the rename_s3 branch January 19, 2021 17:05
@kaiyan-sheng kaiyan-sheng added v7.12.0 and removed needs_backport PR is waiting to be backported to other branches. labels Jan 19, 2021
kaiyan-sheng added a commit that referenced this pull request Jan 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Platforms Label for the Integrations - Platforms team v7.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rename Filebeat s3 input to aws-s3 input
4 participants