Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs for MAX_FILE_SIZE_PUT_OBJECT. #839

Closed
wants to merge 1 commit into from
Closed

Update docs for MAX_FILE_SIZE_PUT_OBJECT. #839

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pipeline/outputs/s3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ See [here](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e706b
| json_date_format | Specify the format of the date. Supported formats are _double_, _epoch_, _iso8601_ (eg: _2018-05-30T09:39:52.000681Z_) and _java_sql_timestamp_ (eg: _2018-05-30 09:39:52.000681_) | iso8601 |
| total_file_size | Specifies the size of files in S3. Maximum size is 50G, minimim is 1M. | 100M |
| upload_chunk_size | The size of each 'part' for multipart uploads. Max: 50M | 5,242,880 bytes |
| MAX_FILE_SIZE_PUT_OBJECT | Max file size allowed to publish to S3 MAX: 1GB. | 1,073,741,824 bytes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't a new user config option. Its a different max size for total_file_size for the use_put_object case

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we may dont need the MAX_FILE_SIZE_PUT_OBJECT to be list in the documentation .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea just note the max allowed value in the total_file_size row a little farther up, that's what I mean

| upload_timeout | Whenever this amount of time has elapsed, Fluent Bit will complete an upload and create a new file in S3. For example, set this value to 60m and you will get a new file every hour. | 10m |
| store_dir | Directory to locally buffer data before sending. When multipart uploads are used, data will only be buffered until the `upload_chunk_size` is reached. | /tmp/fluent-bit/s3 |
| s3\_key_format | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the rewrite_tag filter. Add $UUID in the format string to insert a random string. Add $INDEX in the format string to insert an integer that increments each upload. Add $TAG in the format string to insert the full log tag; add $TAG\[0] to insert the first part of the tag in the s3 key. The tag is split into “parts” using the characters specified with the `s3_key_format_tag_delimiters` option. Add extension directly after the last piece of the format string to insert a key suffix. If you want to specify a key suffix and you are in `use_put_object` mode, you must specify $UUID as well. More explanations can be found in `use_put_object` option. See the in depth examples and tutorial in the documentation. | /fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S |
Expand Down