Reorganize the file source/server defaults for easier internal use #4048
Labels
meta: idea
Anything in the idea phase. Needs further discussion and consensus before work can begin.
needs: approval
Needs review & approval before work can begin.
source: file
Anything `file` source related
source: kubernetes_logs
Anything `kubernetes_logs` source related
type: task
Generic non-code related tasks
As mentioned at #4043 (comment), it would be great to have a way to opt-out of thinking about some of the configuration parameters for the
file_source::FileServer
. However, it currently doesn't implement theDefault
trait.However, there is a default implementation at the
vector::sources::file::FileConfig
with a lot of useful defaults.There are some usability problems we'd like to solve here:
Default
vector::sources::file::FileConfig
and take some value from it, but it's suboptimal: in advanced use cases,vector::sources::file::FileConfig
parameters don't directly translate to thefile_source::FileServer
(like when using a custompath_provider
);vector::sources::file::FileConfig
is scoped to thevector::sources::file
, while we might want to use it in other source, i.e.vector::sources::kubernetes_logs
;file_source::FileServer
initialization is problematic because then there's no distinction between the parameters that are set to their respective values explicitly or just set to whatever the defaults are presumed to be; as already mentioned, we'd like to have a way to just opt-out of specifying the default values, omit the fields and rely on default-initialization.To achieve it, the proposed solution is to alter the
file_source::FileServer
in such a way that it is constructible via some other struct, that has theDefault
-initializer (with all the "primitve" or trivial values in it), and a set of other parameters that are hard to provide defaults for (generic or otherwise abstracted):This is just a sample code to illustrate the idea, the exact structure is worth proper discussion.
@lukesteensen, do you like this approach?
The text was updated successfully, but these errors were encountered: