[processor/k8sattributesprocessor] Leaking goroutine caused by invalid kube client init

[crobert-1]

Component(s)

processor/k8sattributes

Describe the issue you're reporting

Problem:
Relevant code:

opentelemetry-collector-contrib/processor/k8sattributesprocessor/internal/kube/client.go

Line 93 in 09884bd

go c.deleteLoop(time.Second*30, defaultPodDeleteGracePeriod)

The k8sattributesprocessor starts a goroutine when creating a new k8s client that is used for keeping the pod cache up to date. However, if the client initialization fails, the New method returns an error without a reference to the client. This results in a leaked goroutine as there's no way to stop the started goroutine.

Proposed solution:

From my perspective, goroutines should not be started in init or New, but Start instead. I think this is a much more accurate representation of behavior to users.

I've implemented my proposed solution in #30842. Essentially, start the same goroutine inside Start, then use the existing passthroughMode configuration option to immediately return if the rest of the kube client's functionality is not necessary. This means the kubeclient should always be started and stopped, rather than conditionally. I think this is a cleaner UX for the kube client.

Alternatives:

1. Handle stopping the goroutine in New. This would mean calling c.Stop() for every possible error in the New method. This seems error prone as it would be easy to forget adding the Stop call when adding a new error condition. I also don't believe it's idiomatic to create the goroutine in New as it's returning a Client object, but the goroutine is running on a WatchClient-specific method.

The limiting factors here are that we want to start this goroutine for every kube client, and the New method returns nil for the Client object if an error occurs.

[github-actions]

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

Initializing handling the goroutine in Start/Stop certainly seems like the best option. Good job catching this!

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.