[Actions] Ability to test executing an action from UI

[peterschretlen]

Setting up an action may involve configuring a variety of parameters including credentials, notification channels, and message formatting.

To have confidence that an action has been setup correctly, it will help to have a way to execute the action with sample/test data.

Configuring aspects of an action like the message body and message markdown formatting can be iterative, so the test should be quick and easy to perform. Ideally any missing data can be automatically filled.

From discussion with the team here's a proposal for how this would work:

Test button when creating an alert

• In the UI, there is a test button for each action (for example if there are action groups, with an action in each).
• The alert type provides a static sample data, that is representative of the data it sends to an action.
• When the test button for an action is clicked, it will
  • use the sample data to render any templates and show a preview with all the action parameters displayed (this helps to catch issues like realizing you’re using the wrong connector)
  • From the preview window, lets you execute the action for an end-to-end test.

Test button when creating a connector (moved to #75467)

• You will have to save the connector settings first, then a test button will appear
• The test button will work in the same way as when creating an alert:
  • sample/dummy values will be used to fill any missing parameters
  • a preview will be shown
  • from the preview, the action can be executed

[elasticmachine]

Pinging @elastic/kibana-stack-services (Team:Stack Services)

[sorantis]

FYI, we're testing alert triggers in Metrics Alerts : #67684

[mikecote]

A note that these executions should probably be audit logged as well.

[mikecote]

I'm thinking a new API would be needed for this. The API would take config, secrets, params and execute it. This will handle the scenario of testing the configuration before creating a connector.

We can consider splitting the issue and defer Test button when creating an alert use case for now.

[mikecote]

From #134380

cc @mcpate

Describe the feature:
In particular, when working with a Slack connector, it would be great to be able to preview (or test) what the Slack notification will look like. I could imagine this also being useful with indexing data, sending an email, etc. Part of the reason of the need for this is that it isn't always clear what the templated, built-in variables will look like.

Describe a specific use case for the feature:
In Watcher, part of our final development process for a new Watch was to execute the Action (which would send a message to Slack) so that we could make sure the format of the notification looked correct (inc. testing any links contained therein). We are migrating our Watches to Kibana Alerting (with a Slack connector), and as we're re-creating them in Rules, and we are trying to use some of templated variables, we don't have a good way to test what the notifications will look like.