[FEATURE REQ] Spring - Add health indicator for Event Hub stream binder

[saragluna]

Goal

We need to add a health indicator for azure-spring-cloud-stream-binder-eventhubs, this will show the status of the EH binder in /actuator/health endpoint:

Counterparts

Kafka binder

The Kafka binder's health indicator creates a metadataConsumer and uses this consumer to check whether a partition is alive:

https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/blob/e2eca34e4b8f7fc0dab2f19a19094cce0dfda2e4/spring-cloud-stream-binder-kafka/src/main/java/org/springframework/cloud/stream/binder/kafka/KafkaBinderHealthIndicator.java#L115-L167

AWS Kinesis binder

Similiar to the Kafka implementation:

https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/blob/cd64d8918b073b3b3ddfbf4b8747c16fcffc231d/spring-cloud-stream-binder-kinesis/src/main/java/org/springframework/cloud/stream/binder/kinesis/KinesisBinderHealthIndicator.java#L44-L70

Rabbit binder

https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/blob/e39487a4ce6c121d74c0e2778393d827cfc509bb/spring-cloud-stream-binder-rabbit/src/main/java/org/springframework/cloud/stream/binder/rabbit/config/RabbitServiceAutoConfiguration.java#L206-L215

Conclusion

They all use the underlying SDK to test whether the connection to the broker is alive or whether the broker is in a healthy state.

How to implement

The ideal way to implement this is to:

1. Get the instance of the EH client used by a binder.
2. Perform a low-cost operation on the client and try to determine the liveness of the EH client

However, our EH SDK doesn't provide such functions, take the EventProcessorClient for example, it only has an isRunning() method. We can't get the raw EH client via an EventProcessorClient object.

azure-sdk-for-java/sdk/eventhubs/azure-messaging-eventhubs/src/main/java/com/azure/messaging/eventhubs/EventProcessorClient.java

Lines 173 to 175 in 5672b86

	public synchronized boolean isRunning() {
	return isRunning.get();
	}

So the first step to implement the health indicator is to call the isRunning method, and for the next step, we need to work with the EH sdk to see whether we could get more info through the EventProcessorClient.

Spring Cloud Stream's default implementation

https://github.com/spring-cloud/spring-cloud-stream/blob/93acdcf85ad3d47936d71cd5bcc38eefb0d4290a/spring-cloud-stream/src/main/java/org/springframework/cloud/stream/config/BindersHealthIndicatorAutoConfiguration.java

[saragluna]

Hi @conniey, @srnagar, @YijunXieMS

As we're going to add the health indicator for our EH binder, which uses the EventProcessorClient as the consumer, but we find out that we can't get sufficient information from the EventProcessorClient, such as whether the client is alive.

Is it possible that the EventProcessorClient could expose an API to get the status of the underlying eventHubAsyncClient and perform some low-cost operations on it?

[conniey]

This sounds similar to #19391.

Can you give a more concrete scenario of the metrics you want? (ie. I want to know how many partitions I own")

[saragluna]

Adding more related issues: microsoft/spring-cloud-azure#521

[saragluna]

It's a little different from the metrics, the health indicator is used to indicate the health condition of the application components. So for the event hubs stream binder, the health indicator should indicate these:

• The connectivity of the event hub client
• The health conditions of each partition

[conniey]

@saragluna
As far as I know, Event Hubs does not expose any "health condition" of each partition information. @JamesBirdsall may know more.

We can say if the Event Hub client is connected or not though.

Is that the information you need?

[saragluna]

@conniey, I think for the moment to tell whether the EH client is connected is what we need.

[saragluna]

Closing this issue now.