[Security Solution] Related Integration shows as not installed even when it is

[spong]

First reported in 8.6.1, but also reproduced in 8.5.0, the Elastic Defend or Endpoint integration will show as Not installed when it is in fact installed.

Debugging this, we can work our way back from the client since it appears the detection_engine/fleet/integrations/installed API is returning back that it's actually installed.

Here we determine if the integration is installed by seeing if the related integration from the Rule is within the list of installed packages returned by fleet. The package name check ends up being truthy, however the integration name check fails since the rule doesn't provide an related integration name (since there isn't one for Elastic Defend), yet the our data model is including integration details with an '' integration_name.

kibana/x-pack/plugins/security_solution/public/detections/components/rules/related_integrations/integration_details.ts

Lines 81 to 84 in 21de750

	const match = installedIntegrations.find(
	(ii: InstalledIntegration) =>
	ii.package_name === ri.package && ii?.integration_name === ri?.integration
	);

Now heading to the server, we set the integration_name from the policy.input[].policy_template, however this is undefined for the Elastic Defend integration, and so ends up as emptystring ''.

kibana/x-pack/plugins/security_solution/server/lib/detection_engine/fleet_integrations/api/get_installed_integrations/installed_integration_set.ts

Line 129 in 21de750

const integrationName = normalizeString(input.policy_template); // e.g. 'cloudtrail'

As a result, when determining if we need to augment the package with integration details, we now have a mismatch between packageInfo.package_name & integrationInfo.integration_name, which results in us adding integration details, but with an invalid integration_name.

kibana/x-pack/plugins/security_solution/server/lib/detection_engine/fleet_integrations/api/get_installed_integrations/installed_integration_set.ts

Line 88 in 21de750

return packageInfo.package_name === integrationInfo.integration_name

which is why the client is determining that the integration is not installed.

So looks like the root cause here is that there's no policy_template being returned on the packagePolicies' input object for the Elastic Defend integration:

We can see this directly with the /kbn/api/fleet/package_policies fleet API as well:

Elastic Defend integration on the left, Fleet Server integration on the right:

---

I haven't found any specific fleet/package-registry changes yet that would explain why the Elastic Defend packagePolicies don't include a policy_template (maybe it happened as part of the refactor from Elastic Security -> Elastic Defend?), so if not a change introduced in the interim, perhaps we missed this in our initial testing @banderror?

Either way, looks like we can either use the root input.type or the nested config.integration_config.type off of the packagePolicy if the policy_template isn't present. Will need to test with lone packages and packages+integrations to see the behavior here.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[banderror]

@MadameSheema Can folks please verify that #149646 fixes this bug? The fix should be available in 8.7 BC.

@spong I also noticed that the fix has been backported to 8.6 and targets 8.6.3, but I couldn't find the 8.6.3 version in the release schedule. Is the label wrong or the spreadsheet? :)

[MadameSheema]

@karanbirsingh-qasource @sukhwindersingh-qasource may you please help to validate this issue?

[spong]

@spong I also noticed that the fix has been backported to 8.6 and targets 8.6.3, but I couldn't find the 8.6.3 version in the release schedule. Is the label wrong or the spreadsheet? :)

No, you're correct that there's no scheduled 8.6.3, but I wanted this fix to be there just in case they decide to cut another patch release (as there was an 8.5.3) since this issue has been around since at least 8.5.x.

@karanbirsingh-qasource @sukhwindersingh-qasource may you please help to validate this issue?

Just a heads up that there are still these issues (#150968 & #149644) with related integrations, so not everything is resolved here just yet. I'm working these issues for the next BC, but let me know if you come across anything extra in testing and I can try to address along with those fixes.

[essodjolo]

FYI: I have a user who encountered this issue on version 8.4.3.

[sukhwindersingh-qasource]

Hi @MadameSheema

We have validated this issue on 8.7.0 BC4 build and observed that issue is not occurring, It is Fixed. ✔️

Please find the below Testing Details:

Build info

VERSION: 8.7.0
BUILD: 60949
COMMIT: de22cd9361a0dbf429f9648d3c7b7c45aa862e90

Screen-Recording

Rules.-.Kibana.Mozilla.Firefox.2023-02-23.16-29-32.mp4

Hence, We are marking it as QA Validated!!

Thanks!!