Ability to alert when there's been no data for x amount of time

[mikecote]

There's been conversations about the ability to fire specific alert actions when an alert hasn't seen data for a certain period of time.

There's questions around if this is an alert level and/or alert instance level thing. We will eventually have an alert level status of "no data".

The "no data for certain period of time" portion seems to be a common use case. They can always set it to 0 to fire actions immediately.

In regards to the above, @pmuellr also mentioned the idea of adding a debounce feature for the alert actions.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

Ya, my thinking was that if we have a "status" of no-data, then we should also allow a customer to set an alert when the alert is in that status.

I think the original thinking was that this would be at the alert level, and not instance alert level. For alerts that do some kind of query and get instance ids from the data in the query, alert level seems like it makes sense. Would certainly make sense for index threshold. There may be some alerts in the future that have instance id's which are more "fixed", or not determined by the query, and are more well-known in advance. An no-data at the alert instance level might make sense there.

Right now, we don't have a way for an alert executor to inform the framework about a no-data condition at the alert level. Seems like we'd have to add a new function to the services we pass into the executor:

kibana/x-pack/plugins/alerting/server/types.ts

Lines 46 to 48 in c9c37f7

	export interface AlertServices extends Services {
	alertInstanceFactory: (id: string) => AlertInstance;
	}

Presumably, if that function was called, no other actions should be scheduled for that turn, before or after the function is called.

[pmuellr]

The "no data for certain period of time" portion seems to be a common use case. They can always set it to 0 to fire actions immediately.

More knobs and dials! :-) The existing throttle should work fine with this alerting, but can't handle "ignoring" the condition if it was only for a short time frame - you'd always have the alert fire, at least once. Same with debounce - but presumably debounce would fire later than w/ a throttle.

Throttle / debounce don't feel right to me, a separate "no-data for XXX duration" does kinda feel right, just adds more complexity all around.

[XavierM]

we will fix this issue with this one -> #115973