[UII] Support integration policies without agent policy references (aka orphaned integration policies)

[jen-huang]

Summary

Resolves #182220.

This PR allows integration polices to be saved without being added to any agent policies. These integration policies can be considered "orphaned." Through the API, an empty policy_ids: [] array can be passed during create operations to add it to no agent policies. The same empty array can be passed during update operations to clear it from all previous agent policies. Clearing agent policies references this way will also set the deprecated policy_id field to null.

Clearing agent policy references requires the same licensing as the general reusable integration policies feature.

I spotted a bug where removing one or more agent policy references does not bump their revision. Revision was only bumped for only newly added references This has been fixed in this PR as well.

On the UI side, orphaned integration policies can only be discovered on the Integrations details > Policies table:

Agent policies can be unattached in the manage agent policies modal:

Integration policy create/edit form allows agent policies field to be cleared, with a confirmation modal as a heads up to the user. If the user previously had agent policies attached, and is now clearing it, the normal This action will update the selected agent policies will be shown:

Checklist

• Consider interaction with space aware policies Will be done in [Fleet] Make reusable integration policies and space awareness work together #190727
• Test with integrations that register custom policy editors
• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
  • Will be done in Add docs for 'orphaned' integration policies ingest-docs#1261
• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[obltmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[jen-huang]

@kilfoyle I'm not sure what the overall state of docs for reusable integration policies is, but wanted to ping you about this change in case we should document this ability.

[kilfoyle]

Thanks for the ping, @jen-huang! Docs issue is here and I should get to both this and the reusable integrations policies docs this week (or early next).

Btw, thanks a lot for providing the screencaps and so much detail in this and your other PRs! 🙏 It helps me so much not having to track this stuff down.

[gergoabraham]

played around with Elastic Defend integration, adding/removing integrations to agent policies and checked how the @elastic/security-defend-workflows related pages work.

thanks for the modifications, everything looks good! found some weird stuff on our onboarding page, but looks like it wasn't perfect before as well with multiple policy ids, so i created a follow up issue for @elastic/security-defend-workflows team: https://github.com/elastic/security-team/issues/10362

cc @dasansol92 @caitlinbetz

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: e54c2d9

Failed CI Steps

• Jest Tests #16
• FTR Configs #67

Test Failures

• [job] [logs] FTR Configs #67 / Cloud Security Posture Test adding Cloud Security Posture Integrations CSPM AZURE Azure Single Manual Service Principle with Client Secret Azure Single Manual Service Principle with Client Secret Workflow
• [job] [logs] Jest Tests #16 / useSetupTechnology should update new agent policy and selected policy tab when setup technology is agent-based

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
fleet	1.8MB	1.8MB	+1.5KB
securitySolution	18.0MB	18.0MB	+25.0B
total			+1.5KB

Unknown metric groups

References to deprecated APIs

id	before	after	diff
securitySolution	474	476	+2

History

• 💛 Build #229890 was flaky 453bb1a
• 💛 Build #229579 was flaky 11e970d
• 💔 Build #229536 failed abbcc3a
• 💔 Build #229326 failed a6d1b82
• 💔 Build #229279 failed 8c81fe2
• 💔 Build #228999 failed cee20bb

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jen-huang

[juliaElastic]

LGTM

[kilfoyle]

LGTM for the docs! 👍