Add filelog receiver to the core distribution

[swiatekm]

The core distribution is missing a way to read logs from a file on disk, a very basic functionality for a telemetry collector agent. The log data model is now stable, filelog receiver itself is in Beta, and gets a reasonable amount of real-world testing in vendor distributions.

The binary size increases from 91MB to 94MB on Linux amd64, which is acceptable.

[jpkrohling]

I think we don't have a precedent for this: who should make the final call? @open-telemetry/collector-maintainers?

[swiatekm]

Core seems a bit behind contrib in general, probably because there isn't a process for including components in core. I'd have expected transform processor to have been included as well, for example.

[atoulme]

I would be interested to understand the rationale and distinction between the core and contribs distros as well.

[swiatekm]

My assumption was that core was the minimum functionality needed for a telemetry agent. Good for simple use cases, tests, PoCs, for people new to Otel to play around with, etc. Whereas contrib is a kitchen sink of everything maintained by the Otel org. No clue if this is actually the case, or why core has the exact components it does.

[jpkrohling]

@bogdandrutu (or was it @tigrannajaryan) mentioned a few years ago on a SIG Collector that the core contains what we (otelcol contributors) expect most users to use daily. This hasn't been formalized, and we all deserve a formal definition or guideline, perhaps based on the statement from the past.

[atoulme]

I am not sure. I am also not sure how stability correlates to inclusion. Plenty of components in contrib are also not listed in contrib.
I think we need to work on those definitions before we move forward.

[jpkrohling]

Contrib is everything that is in both core and the contrib repositories, minus:

• components that are still under development
• components that we have problems with their security model (we might have one or two of those)

Anything else is likely an omission from the original code owner, possibly because they might not know the component needed to be added to the manifest.

I am also not sure how stability correlates to inclusion.

Stability should not play a role in the inclusion. I would argue that components still in alpha might not be relevant enough to the broader audience to be included in the core distribution.

[bogdandrutu]

I think we should discuss tomorrow in the SIG meeting about this, and if we want at all to have a core distribution.

[codeboten]

Added to the agenda

[codeboten]

I would suggest that core should only include components that are in the core repository. This may require us to move existing components from contrib into core.

This would make it easy to point to the repo and say here are all the components. Contrib would be everything in core + contrib

[Aneurysm9]

I would suggest that core should only include components that are in the core repository. This may require us to move existing components from contrib into core.

We already did this once in the opposite direction, moving components from core to contrib. It was a PITA and required we stop the world in both repos to migrate the code without losing git history. I don't think we should do this again.

[swiatekm]

As this question was essentially superseded by the distros discussion in #360, I'm going to close it.