Fix race condition in otelcol component wrappers

[thampiotr]

PR Description

We have a general issue with OTel components where consumers may be used before the Start functions in OTel have finished running. This is because in OTel Start functions are non-blocking and sometimes do work to set things up, like it was the case for batch_processor. In Alloy, however we have Run function that is blocking for the lifetime of the component. As soon as it's called, we consider the component Running. In OTel, however, the Start function should be called and exit to consider a component running.

The solution here is to pause the Consumer until we are sure that the OTel component scheduler has called Start on all OTel components. The consumer will block any attempts to feed data to it.

Which issue(s) this PR fixes

Notes to the Reviewer

thampiotr@MacWork ~/w/alloy (update-otel-112-fix-race)> go test ./internal/component/otelcol/internal/scheduler/ -count 50 -race
ok  	github.com/grafana/alloy/internal/component/otelcol/internal/scheduler	3.543s
thampiotr@MacWork ~/w/alloy (update-otel-112-fix-race)> go test ./internal/component/otelcol/internal/lazyconsumer/ -count 50 -race
ok  	github.com/grafana/alloy/internal/component/otelcol/internal/lazyconsumer	17.495s

PR Checklist

• CHANGELOG.md updated
• Documentation added
• Tests updated
• Config converters updated

[wildum]

Thanks for working on this, I think that's a step in the right direction.
The logic was not super clear to me at first in the scheduler, the idea is that the components are created in the paused state so they are not paused on the first run but are always resumed after being started because they are paused when the scheduler stops running, right? Maybe we could have this explanation in the scheduler code

[clayton-cornell]

Just some edits to align with the same changes in #2012 Might also be "fixed" up when the PR conflicts are resolved?

[wildum]

LGTM, did you test it with the version of the batch processor that was causing the flaky tests?

[thampiotr]

LGTM, did you test it with the version of the batch processor that was causing the flaky tests?

Yes. There is a test added in batch_processor that fails if the changes here are removed.