[FEATURE] Add Integrations for Observability

[YANG-DB]

Is your feature request related to a problem?
Today in OpenSearch, Observability and its dashboards are only partially aware (traces only) of the schematic structure of Observability signals types. In addition the actual schema index mapping is not present internally in the Observability plugin and has to be imported externally.

Integrating different (Observability) data producers and correlating different signals is non standardized and very customer tailored. Most of the time this is due to missing schema definition and correlated field names.

Integration of a NEW Observability data sources (such as NGINX / Tomcat / postgres) includes complicated configuration of both the ingestion process and the actual index store, manually discovery of the specific format of that new datasource and the actual crafting of the dedicated dashboards with its proprietary fields.

This process of is not transferable between difference customers since we currently can’t share customer proprietary configurations and dashboards.

What solution would you like?
Our goal is creating a consolidated sharable Observability solution which allows customers to ingest any type of telemetry data from any type of provider and have the capability of displaying and analysis of this data in a common and unified way.

Customers using Observability are expecting our solution to allow simple and out of the box integration and configuration.

Using a unified schema that models all the Observability components, and allowing customers to easily add integrations that would simplify their daily monitoring and incidents investigations with pre-build dashboards and pre-defined correlation and alerts.

As an example for the importance of a common schema - think about a multi-layered application that produces multiple log and trace signals from different software and Network components. If we could address all these signals using a common vocabulary we could simplify the correlating queries using common fields such as “process.args”, “host.domain”, “observer.os.name”

What alternatives have you considered?
N/A

Do you have any additional context?

• [o11y] Support Simple Schema for Observability  #1398
• Support OTEL based Trace Index Mapping in place #1388
• Support OTEL based Metrics Index Mapping in place #1389