Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for Simple Schema #2940

Merged
merged 11 commits into from
Feb 22, 2023
40 changes: 40 additions & 0 deletions _observing-your-data/sso.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
layout: default
title: Simple Schema for Observability
nav_order: 120
---

# Simple Schema for Observability
Introduced 2.6
{: .label .label-purple }

Starting with OpenSearch 2.6, a standardization for conforming to a common and unified observability schema has been introduced: Simple Schema for Observability (SSO). [Observability]({{site.url}}{{site.baseurl}}/observing-your-data/index/) is a collection of plugins and applications that let you visualize data-driven events by using Piped Processing Language to explore, discover, and query data stored in OpenSearch.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

With the schema in place, observability tools can ingest, automatically extract, aggregate data, and create custom common dashboards, making it easier to understand the system at a higher level.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

SSO is inspired by both [OTEL](https://opentelemetry.io/docs/) and Elastic Common Schema (ECS) and uses a variety of events that ECS has to offer for logs with the OTEL metadata.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

Alerts will be supported in a future release.
{: .note }

### Use cases
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

Use cases for SSO include:

* Ingest observability data from different data types.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
* Move from proprietary configurations that are non-transferable to a consolidated sharable Observability solution that allows users to ingest any type of telemetry data from any type of provider and have the capability of displaying an analysis of this data.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
* Data Prepper conforms to the SSO schema for metrics and will gradually support traces and logs.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

[Data Prepper's]({{site.url}}{{site.baseurl}}/data-prepper/index/) existing traces mapping has enriched data in the form of serviceMap, which differs from SSO traces. This will consolidate with Observability by using the SSO traces schema and introduce serviceMap as an enriched calculated field.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
{: .note }

## Traces and metrics

Schema definitions for traces and metrics are defined and supported by the Observability plugin.

These schema definitions include:

- Index structure (mapping).
- [Index naming conventions](https://github.com/opensearch-project/observability/issues/1405).
- JSON schema for enforcement and validation of the structure.
- [Integration](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3412) feature for adding pre-canned dashboards and assets.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved