[Cloud Security] posture_type Backward compatibility changes

[JordanSh]

Resolves #150608

Summary

This PR adds BC to the newly added posture_type field. Since this field only exists in findings from our latest integration version, older versions of findings won't have it. Thus some of our code on Kibana's side needs to be able to accomplish certain functionalities even without it.

ESQuery filtering

We used the posture_type field to filter the different dashboard tabs. In order to keep the same filtering functionalities, a new runtime field was added.

export const getSafePostureTypeRuntimeMapping = (): MappingRuntimeFields => ({
  safe_posture_type: {
    type: 'keyword',
    script: {
      source: `
        if (!doc.containsKey('rule.benchmark.posture_type'))
          {
            def safe_posture_type = 'kspm';
            emit(safe_posture_type);
            return
          }
        else
        {
            def safe_posture_type = doc["rule.benchmark.posture_type"].value;
            emit(safe_posture_type);
            return
        }
      `,
    },
  },
});

This runtime field creates a safe to use field which will be equal to the posture type even for older findings. we will then use this field instead in order to query over the findings in all of our getter functions

const runtimeMappings: MappingRuntimeFields = getSafePostureTypeRuntimeMapping();
const query: QueryDslQueryContainer = {
  bool: {
    filter: [{ term: { safe_posture_type: policyTemplate } }],
  },
};

const [stats, groupedFindingsEvaluation, clustersWithoutTrends, trends] = await Promise.all(
  [
    getStats(esClient, query, pitId, runtimeMappings),
    getGroupedFindingsEvaluation(esClient, query, pitId, runtimeMappings),
    getClusters(esClient, query, pitId, runtimeMappings),
    getTrends(esClient, policyTemplate),
  ]
);

Dashboard navigation query

Since we don't always have posture_type, we need to have a solution for dashboard navigations for those cases as well.
In order to do that, the navigation option of using negated filters was added to the useNavigate hook. it can be used like so

export const getPolicyTemplateQuery = (policyTemplate: PosturePolicyTemplate): NavFilter => {
  if (policyTemplate === CSPM_POLICY_TEMPLATE) {
    return { 'rule.benchmark.posture_type': CSPM_POLICY_TEMPLATE };
  }

  return { 'rule.benchmark.posture_type': { value: CSPM_POLICY_TEMPLATE, negate: true } };
};

will result in

Removal of direct posture_type === 'kspm' checks

We no longer check if kspm is the posture type directly, all checks are now for posture_type === 'cspm' and if not, resulting in a default value. For example:

Before

export const getClusterIdQuery = (cluster: Cluster) => {
  if (cluster.meta.benchmark.posture_type === CSPM_POLICY_TEMPLATE) {
    return { 'cloud.account.name': cluster.meta.cloud?.account.name };
  }
  if (cluster.meta.benchmark.posture_type === 'kspm') {
    return { cluster_id: cluster.meta.assetIdentifierId };
  }

  return {};
};

After

export const getClusterIdQuery = (cluster: Cluster) => {
  if (cluster.meta.benchmark.posture_type === CSPM_POLICY_TEMPLATE) {
    return { 'cloud.account.name': cluster.meta.cloud?.account.name };
  }
  return { cluster_id: cluster.meta.assetIdentifierId };
};

More of those were already done in #151265 as well

How to check

You will need to have findings without the posture_type field and run on Kibana 8.7. this can be done either with mocks or persistent ES, from there all dashboard functionalities should work as expected.

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[JordanSh]

unrelated but this fixes problems of unmatching summary sizes between dashboard tabs when there are not enough items in the compliance by score table

[opauloh]

Looks great 🚀

[opauloh]

nice, usually I'm against runtime mappings, but since it's on the latest index, performance won't be an issue, great BC solution

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 99e98dd

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
cloudSecurityPosture	145.3KB	145.3KB	+12.0B

History

• 💔 Build #109299 failed 68d4b95
• 💚 Build #109032 succeeded 7c99597

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @JordanSh

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.7	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 151647

Questions ?

Please refer to the Backport tool documentation

[opauloh]

💔 All backports failed

Status Branch Result
❌ 8.7 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 151647

Questions ?

Please refer to the Backport tool documentation

I am going to manually backport this PR, and that should fix this issue