Configurable Health Indicator to observe HTTP Endpoints

[obfischer]

I would like to have a generic health indicator to observe the availability of an HTTP endpoint, provided by another service.

Why do I see the need for such an implementation? At my clients I see a wide use of Spring Boot Admin as central service to monitor Spring Boot based applications. Often a service is shown there as healthy, but it isn't healthy (I am aware of the difference of liveness and readyness) as an REST-Endpoint or any other HTTP endpoint is not available or reachable. Although they have also other monitoring solutions running, mostly it is much easier to add an health indicator in a Spring Boot application than to get another department to do it.

I have written such an health indicator over the last years many times and I would be happy to contribute it, if I get some advices how to implement it in such a way, that it fits at best to the actuator framework in Spring Boot under the aspects of architecture and maintainability.

[mhalbritter]

We talked about that in our last meeting, and we're not against it. Can you provide some more details how that http check would work? We worried a bit about configuration vs. flexibility. E.g. for a HTTP healthcheck, one should be able to configure the URL, timeouts and (I guess) headers.

[jonatan-ivanov]

I did this a few times in the past and I was always missing this feature, big +1 from me. :)
Let me also throw in my two cents:

I think there are two main "flavors" of such a HealthIndicator:

1. Generic
   This type of HealthIndicator should be able to work with any HTTP endpoints and users should be able to specify the request (RequestBuilder/RequestFactory), the response/outcome -> Health mapping, the Exception -> Health mapping (if the client signals errors that way) and the client(builder) to set timeouts/etc.

2. Spring-Boot specific
   If we know that the app we call is also a Spring Boot app with a health endpoint, we know how to create a request (only need the url), how to map the response (since that is also a Health instance) and we can have an opinionated error mapping too. This can lead to nice things like registering health checks automatically for example if you use an "interface client" (like feign, retrofit, and the new interface clients in Framework) and you add a method that returns Health (or signal it via an interface).
   (For this, I would also consider making Health de-serializable or providing and registering a deserializer.)

Something like this. The adapter and other components are here, the client is here.

I think both would be nice (not necessarily in one issue).

[obfischer]

Hi @mhalbritter,

let me first describe the current implementation. At the moment is there a default configuration, but a connection and read timeout are configurable. In case of an negative result, the health check provides an error description, either the returned status code or the message from a caught exception.

A generic implementation, suitable for the actuator framework needs in my opionion these features and the following additional features:

• configurable expected HTTP status code
• HTTP method to be used OPTIONS or GET
• matcher for the returned body
• matcher for expected HTTP headers

Having this features, the health indicator should be usefull for a wide range of scenarios.

[obfischer]

@jonatan-ivanov I will incoporate your suggestions, if I will implement it.

[chicobento]

This would help to mitigate the issue when actuator endpoints are deployed on a separate management context:

If your Actuator endpoints are deployed on a separate management context, the endpoints do not use the same web infrastructure (port, connection pools, framework components) as the main application.
In this case, a probe check could be successful even if the main application does not work properly (for example, it cannot accept new connections).
For this reason, is it a good idea to make the liveness and readiness health groups available on the main server port.

Related to #25471

[bclozel]

Is this a duplicate of #25151 ?

[obfischer]

@bclozel, I would say no, as my proposal is more about the tooling and how to do it (how to do it), while #25151 is about what to do (what to monitor).

[mhalbritter]

Hey @obfischer,

matcher for the returned body
matcher for expected HTTP headers

Are we talking about a ready-made class in Spring Boot which you can extend and add it to your context or are we talking about some properties which would enable that feature?

I'm asking because I'm not keen on the idea of somehow specifying the matchers in the properties.

[obfischer]

Hi @mhalbritter,

I think both should be possible.

I'm asking because I'm not keen on the idea of somehow specifying the matchers in the properties.

[obfischer]

I will start with a implementation in my own repository for now. If anyone has feature requirements, feel free to add them here as comment.

[mentallurg]

@obfischer: How far are you with implementation?