Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 input: Add fallback configuration for content type so that events can be decoded #25697

Closed
ppf2 opened this issue May 12, 2021 · 1 comment · Fixed by #25772
Closed

S3 input: Add fallback configuration for content type so that events can be decoded #25697

ppf2 opened this issue May 12, 2021 · 1 comment · Fixed by #25772
Assignees

Comments

@ppf2
Copy link
Member

ppf2 commented May 12, 2021

If the S3 object has a Content-Type of application/json, we will decode it as JSON in the input. Sometimes, the uploaded S3 objects do not have content type set. For these cases, it will be helpful to have an optional configuration to tell the S3 input to fallback to a specified content type of choice when no Content-Type is specified so that we will still attempt to decode them.

@andrewkroh

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 12, 2021
@ppf2 ppf2 changed the title S3 input: Add fallback to application/json so that events can be decoded S3 input: Add fallback configuration for content type so that events can be decoded May 12, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 12, 2021
@andrewkroh andrewkroh added the Filebeat Filebeat label May 12, 2021
@leehinman leehinman self-assigned this May 14, 2021
leehinman added a commit to leehinman/beats that referenced this issue May 18, 2021
- new option `content_type`
- can be set at input or file selector level
- overrides Content-Type that was given to the S3 object when it was
  uploaded.

Closes elastic#25697
leehinman added a commit that referenced this issue May 19, 2021
* aws-s3 input default content-type

- new option `content_type`
- can be set at input or file selector level
- overrides Content-Type that was given to the S3 object when it was
  uploaded.

Closes #25697

Co-authored-by: kaiyan-sheng <[email protected]>
mergify bot pushed a commit that referenced this issue May 19, 2021
* aws-s3 input default content-type

- new option `content_type`
- can be set at input or file selector level
- overrides Content-Type that was given to the S3 object when it was
  uploaded.

Closes #25697

Co-authored-by: kaiyan-sheng <[email protected]>
(cherry picked from commit d62b1be)
leehinman added a commit that referenced this issue May 19, 2021
* aws-s3 input default content-type

- new option `content_type`
- can be set at input or file selector level
- overrides Content-Type that was given to the S3 object when it was
  uploaded.

Closes #25697

Co-authored-by: kaiyan-sheng <[email protected]>
(cherry picked from commit d62b1be)

Co-authored-by: Lee Hinman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants