Support authentication in the fluent source #7532
Comments
jszwedko
added
the
type: enhancement
New source for consuming data from fluentd and fluent-bit via their native [forward protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1). This implementation supports the various formats that the specification allows for but does not implement the authentication protocol (#7532) or the acknowledgment (#7533) parts of the protocol. I left these for follow-up based on user demand given it is a significant divergence from the existing TCP source which just consumes data rather than request/response. Aside from the source implementation, the other significant change here is extraction of some bits of the `docker_logs` source into a shared `docker` module to use it in the `fluent` source integration tests. The integration tests are more similar to the `docker_logs` integration tests than the other component integration tests in that they each run a separate docker container rather than just standing up one long running one before running the tests as these tests need to run fluentd and fluent-bit with different configurations. Apologies, I didn't break up the early commits into logical changes, but hopefully the PR itself is reviewable. Closes #419 Signed-off-by: Jesse Szwedko <[email protected]>
|
@jszwedko Hi! Also, what about compression btw? Is it supported, will be supported? Cause sending side supports it - https://docs.fluentd.org/output/forward#how-to-enable-gzip-compression Update: |
Hey! We support https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1#compressedpackedforward-mode See: https://github.com/timberio/vector/blob/master/src/sources/fluent.rs#L346-L354 |
|
@jszwedko Thanks a lot for supporting the |
|
No estimate on timeline unfortunately. We'd be happy to see a contribution though if someone is motivated. |
As part of #419 we are adding a
fluentsource, but chose not to include authentication in the initial implementation due to the additional work it will require. This issue represents that work.Please leave a 👍 or comment below if you would find this support useful.
The text was updated successfully, but these errors were encountered: