Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.0] [s3] Change default for preserve_data_ordering to On #508

Open
PettitWesley opened this issue Dec 23, 2022 · 4 comments
Open

[2.0] [s3] Change default for preserve_data_ordering to On #508

PettitWesley opened this issue Dec 23, 2022 · 4 comments

Comments

@PettitWesley
Copy link
Contributor

Currently this field defaults to Off: https://docs.fluentbit.io/manual/pipeline/outputs/s3

However, realistically, most S3 users should want to use that setting. There's little real downside to it, it won't realistically impact performance, since S3 will now all be on sync IO with my recent bug fix (fluent/fluent-bit#6573).

2.0 is an opportunity to "break" backwards compatibility in small ways. While upstream already has 2.0 out, this distro has not released it yet (which will be our 3.0).

Our first release should include changing the default to on for Preserve_Data_Ordering

*If you have any comments or concerns on this plan, please note them in this issue.` This distro is still testing Fluent Bit 2.0 and so we will not release it until next year.

@PettitWesley
Copy link
Contributor Author

fluent/fluent-bit#6617

@dshackith
Copy link

If this issue is not fixed, setting this to default on will cause problems.

@PettitWesley
Copy link
Contributor Author

@dshackith Can you please open an issue on the AWS repo for this. https://github.com/aws/aws-for-fluent-bit

The preserve_data_ordering feature doesn't impact the S3 code path IIRC, so the behavior shouldn't change due to that.

@dshackith
Copy link

Opened #541

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants