[SIEM] Update TLS tables for ECS 1.4+

[andrewkroh]

The TLS tables in the SIEM UI we first created before there were any TLS fields defined in Elastic Common Schema (ECS). Since ECS 1.4 the TLS fields have been added. The Beat data sources are being updated to produce ECS conforming TLS events. Now the queries executed by the UI need to be updated.

One open question is whether the tables should remain backward-compatible with the earlier Packetbeat format for some time period.

See also:

• [SIEM] Non-ECS fields in SIEM #43649
• [Packetbeat] Output ECS-compatible TLS fields beats#15497 (in v7.6.0)
• [Filebeat] Add ECS TLS fields to existing filesets beats#15757 (in v7.7.0)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[XavierM]

One open question is whether the tables should remain backward-compatible with the earlier Packetbeat format for some time period.

We need to discuss that with @MikePaquette

[MikePaquette]

One open question is whether the tables should remain backward-compatible with the earlier Packetbeat format for some time period.

If possible, yes, please.

[antcodd]

The fields used for drag and drop don't seem to have been updated so return no results when dragged to a timeline, see #67299.

It would also be good to update the fields used in timeline row renderers, such as the certificate fingerprint.

[andrewkroh]

I re-reviewed the TLS table in 8.13.2 and the query from Inspect shows it only uses fields present in ECS 8.11. Closing.

{
  "aggs": {
    "count": {
      "cardinality": {
        "field": "tls.server.hash.sha1"
      }
    },
    "sha1": {
      "terms": {
        "field": "tls.server.hash.sha1",
        "size": 10,
        "order": {
          "_key": "desc"
        }
      },
      "aggs": {
        "issuers": {
          "terms": {
            "field": "tls.server.issuer"
          }
        },
        "subjects": {
          "terms": {
            "field": "tls.server.subject"
          }
        },
        "not_after": {
          "terms": {
            "field": "tls.server.not_after"
          }
        },
        "ja3": {
          "terms": {
            "field": "tls.client.ja3"
          }
        }
      }
    }
  },
  "query": {
    "bool": {
      "filter": [
        {
          "bool": {
            "must": [],
            "filter": [],
            "should": [],
            "must_not": []
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2024-04-16T04:00:00.000Z",
              "lte": "2024-04-17T03:59:59.999Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ]
    }
  },
  "size": 0
}