From 9196ab85bfda5b2f17a60e6c4f457012d05cc69f Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Tue, 6 Oct 2020 13:37:50 -0700 Subject: [PATCH 01/23] Add database metric semantic conventions --- .../metrics/semantic_conventions/database.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 specification/metrics/semantic_conventions/database.md diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md new file mode 100644 index 00000000000..93ddcde614d --- /dev/null +++ b/specification/metrics/semantic_conventions/database.md @@ -0,0 +1,77 @@ +# Semantic Conventions for Database Metrics + +This document contains semantic conventions for database client metrics in +OpenTelemetry. When instrumenting database clients, also consider the +[general metric semantic conventions](README.md#general-metric-semantic-conventions). + +## Common + +The following labels **SHOULD** be applied to all database metric instruments. + +| Attribute | Description | Example | Required | +|------------------------|--------------|----------|----------| +| `db.system` | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | Yes | +| `db.connection_string` | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | No | +| `db.user` | Username for accessing the database. | `readonly_user`
`reporting_user` | No | +| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP` | Conditional [1] | + +**[1]:** Recommended in general, required for in-process databases (`"inproc"`). + +## Call-level Metric Instruments + +The following metric instruments **SHOULD** be iterated for every database operation. + +| Name | Instrument | Units | Description | +|----------------------|---------------|--------------|-------------| +| `db.client.duration` | ValueRecorder | milliseconds | measures the duration of the database operation | + +Database operations **SHOULD** include execution of queries, including DDL, DML, +DCL, and TCL SQL statements (and the corresponding operations in non-SQL +databases), as well as connect operations. + +### Labels + +In addition to the [common](#common) labels, the following labels **SHOULD** be +applied to all database call-level metric instruments. + +| Attribute | Type | Description | Example | Required | +|------------|------|--------------|----------|----------| +| `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Conditional [2] | +| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional
Required if applicable. | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET` | Conditional
Required, if `db.statement` is not applicable. | + +**[1]:** In some SQL databases, the database name to be used is called "schema name". + +**[2]:** Required, if applicable and no more-specific attribute is defined. + +**[3]:** The value may be sanitized to exclude sensitive information. + +**[4]:** While it would semantically make sense to set this, e.g., to a SQL keyword like `SELECT` or `INSERT`, it is not recommended to attempt any client-side parsing of `db.statement` just to get this property (the back end can do that if required). + +**[5]:** To reduce cardinality, the values for `db.statement` and `db.operation` +should have parameters removed or substituted. The resulting value should be a +low-cardinality value represeting the statement or operation being executed on +the database. It may be a stored procedure name (without arguments), SQL +statement without variable arguments, operation name, etc. + +### Call-level labels for specific technologies + +| Attribute | Description | Example | Required | +|---------------------------|--------------|----------|----------| +| `db.cassandra.keyspace` | The name of the keyspace being accessed. To be used instead of the generic `db.name` attribute. | `mykeyspace` | Yes | +| `db.hbase.namespace` | The [HBase namespace](https://hbase.apache.org/book.html#_namespace) being accessed. To be used instead of the generic `db.name` attribute. | `default` | Yes | +| `db.redis.database_index` | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` label. | `0`
`1`
`15` | Conditional [1] | +| `db.mongodb.collection` | The collection being accessed within the database stated in `db.name`. | `customers`
`products` | Yes | + +**[1]:** Required, if other than the default database (`0`). + + +## Connection-level Metric Instruments + +The following metric instruments should be collected every harvest interval. They should have all [common](#common) labels applied to them. + +| Name | Instrument | Units | Description | +|---------------------------|---------------|--------------|-------------| +| `db.connectionPool.limit` | ValueObserver | {connections} | measures the total number of database connections available in the connection pool | +| `db.connectionPool.usage` | ValueObserver | {connections} | measures the number of database connections _in use_. | + From 330e728aedd82fb617c8b38b579e067b29508a9d Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Wed, 7 Oct 2020 15:51:09 -0700 Subject: [PATCH 02/23] Add detailed connection pooling semantic conventions for metrics --- .../metrics/semantic_conventions/database.md | 20 +++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 93ddcde614d..3c423f77a2a 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -65,10 +65,22 @@ statement without variable arguments, operation name, etc. **[1]:** Required, if other than the default database (`0`). - -## Connection-level Metric Instruments - -The following metric instruments should be collected every harvest interval. They should have all [common](#common) labels applied to them. +## Connection Pooling Metric Instruments + +If possible, instrumentation should collect the following. They should have +all [common](#common) labels applied to them. + +| Name | Instrument | Units | Description | +|---------------------------|------------|---------------|-------------| +| `db.connections.new` | Counter | {connections} | The number of new connections created. | +| `db.connections.taken` | Counter | {connections} | The number of connections taken from the connection pool. | +| `db.connections.returned` | Counter | {connections} | The number of connections returned to the connection pool. | +| `db.connections.reused` | Counter | {connections} | The number of connections reused. | +| `db.connections.closed` | Counter | {connections} | The number of connections closed. | + + +Otherwise, the following metric instruments should be collected. They should +have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | |---------------------------|---------------|--------------|-------------| From 4fbd26f18edc02b4f70cd441b7a9d8be109ca429 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Tue, 6 Oct 2020 13:52:54 -0700 Subject: [PATCH 03/23] Add exception.type to database metrics semantic conventions --- .../metrics/semantic_conventions/database.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 3c423f77a2a..8b6aa48148a 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -34,11 +34,12 @@ databases), as well as connect operations. In addition to the [common](#common) labels, the following labels **SHOULD** be applied to all database call-level metric instruments. -| Attribute | Type | Description | Example | Required | -|------------|------|--------------|----------|----------| -| `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Conditional [2] | -| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional
Required if applicable. | -| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET` | Conditional
Required, if `db.statement` is not applicable. | +| Attribute | Type | Description | Example | Required | +|------------------|--------|--------------|----------|----------| +| `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Conditional [2] | +| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional
Required if applicable. | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET` | Conditional
Required, if `db.statement` is not applicable. | +| `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Conditional
Required if applicable. | **[1]:** In some SQL databases, the database name to be used is called "schema name". From 2ae2bbbd5171421dec25bfcad199e79007eb79c6 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Wed, 7 Oct 2020 15:49:01 -0700 Subject: [PATCH 04/23] Clean up grammar in database semantic conventions --- .../metrics/semantic_conventions/database.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 8b6aa48148a..cfb793e8989 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -6,7 +6,7 @@ OpenTelemetry. When instrumenting database clients, also consider the ## Common -The following labels **SHOULD** be applied to all database metric instruments. +The following labels SHOULD be applied to all database metric instruments. | Attribute | Description | Example | Required | |------------------------|--------------|----------|----------| @@ -19,27 +19,27 @@ The following labels **SHOULD** be applied to all database metric instruments. ## Call-level Metric Instruments -The following metric instruments **SHOULD** be iterated for every database operation. +The following metric instruments SHOULD be iterated for every database operation. | Name | Instrument | Units | Description | |----------------------|---------------|--------------|-------------| -| `db.client.duration` | ValueRecorder | milliseconds | measures the duration of the database operation | +| `db.client.duration` | ValueRecorder | milliseconds | The duration of the database operation. | -Database operations **SHOULD** include execution of queries, including DDL, DML, +Database operations SHOULD include execution of queries, including DDL, DML, DCL, and TCL SQL statements (and the corresponding operations in non-SQL databases), as well as connect operations. ### Labels -In addition to the [common](#common) labels, the following labels **SHOULD** be +In addition to the [common](#common) labels, the following labels SHOULD be applied to all database call-level metric instruments. | Attribute | Type | Description | Example | Required | |------------------|--------|--------------|----------|----------| | `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Conditional [2] | -| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional
Required if applicable. | +| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional.
Required if applicable. | | `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET` | Conditional
Required, if `db.statement` is not applicable. | -| `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Conditional
Required if applicable. | +| `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Conditional.
Required if applicable. | **[1]:** In some SQL databases, the database name to be used is called "schema name". @@ -68,7 +68,7 @@ statement without variable arguments, operation name, etc. ## Connection Pooling Metric Instruments -If possible, instrumentation should collect the following. They should have +If possible, instrumentation SHOULD collect the following. They SHOULD have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | @@ -80,11 +80,11 @@ all [common](#common) labels applied to them. | `db.connections.closed` | Counter | {connections} | The number of connections closed. | -Otherwise, the following metric instruments should be collected. They should +Otherwise, the following metric instruments SHOULD be collected. They SHOULD have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | |---------------------------|---------------|--------------|-------------| -| `db.connectionPool.limit` | ValueObserver | {connections} | measures the total number of database connections available in the connection pool | -| `db.connectionPool.usage` | ValueObserver | {connections} | measures the number of database connections _in use_. | +| `db.connectionPool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | +| `db.connectionPool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | From 1c2b9e9ab58685c18235dacad7a5a1049513b946 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Wed, 7 Oct 2020 15:47:40 -0700 Subject: [PATCH 05/23] Add net.peer labels to database metrics semantic conventions --- specification/metrics/semantic_conventions/database.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index cfb793e8989..632a2559d83 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -13,7 +13,11 @@ The following labels SHOULD be applied to all database metric instruments. | `db.system` | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | Yes | | `db.connection_string` | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | No | | `db.user` | Username for accessing the database. | `readonly_user`
`reporting_user` | No | -| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP` | Conditional [1] | +| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP`
`Unix` | Conditional [1] | +| `net.peer.ip` | Remote address of the peer (dotted decimal for IPv4 or [RFC5952](https://tools.ietf.org/html/rfc5952) for IPv6) | `127.0.0.1` | No | +| `net.peer.port` | Remote port number. | `80`
`8080`
`443` | No | +| `net.peer.name` | Remote hostname or similar, see note below. | `example.com` | No | + **[1]:** Recommended in general, required for in-process databases (`"inproc"`). From c220dfd98968e25dc82564bbe3c274f618bcab41 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 8 Oct 2020 12:49:04 -0700 Subject: [PATCH 06/23] Link to trace semantic conventions list of known db systems --- specification/metrics/semantic_conventions/database.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 632a2559d83..ecc6f570055 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -10,16 +10,19 @@ The following labels SHOULD be applied to all database metric instruments. | Attribute | Description | Example | Required | |------------------------|--------------|----------|----------| -| `db.system` | An identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers. | `other_sql` | Yes | +| `db.system` | An identifier for the database management system (DBMS) product being used. [1] | `other_sql` | Yes | | `db.connection_string` | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | No | | `db.user` | Username for accessing the database. | `readonly_user`
`reporting_user` | No | -| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP`
`Unix` | Conditional [1] | +| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP`
`Unix` | Conditional [2] | | `net.peer.ip` | Remote address of the peer (dotted decimal for IPv4 or [RFC5952](https://tools.ietf.org/html/rfc5952) for IPv6) | `127.0.0.1` | No | | `net.peer.port` | Remote port number. | `80`
`8080`
`443` | No | | `net.peer.name` | Remote hostname or similar, see note below. | `example.com` | No | -**[1]:** Recommended in general, required for in-process databases (`"inproc"`). +**[1]:** See the [database trace semantic conventions](../../trace/semantic_conventions/database.md#connection-level-attributes) +for the list of well-known database system values. + +**[2]:** Recommended in general, required for in-process databases (`"inproc"`). ## Call-level Metric Instruments From 9924cff0f40443cffa65903e07bccf68331a47f4 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Wed, 7 Oct 2020 15:53:04 -0700 Subject: [PATCH 07/23] Replace database statement with operation and table name in metric semantic conventions --- .../metrics/semantic_conventions/database.md | 23 ++++++++----------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index ecc6f570055..2d2471ab419 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -43,24 +43,19 @@ applied to all database call-level metric instruments. | Attribute | Type | Description | Example | Required | |------------------|--------|--------------|----------|----------| -| `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Conditional [2] | -| `db.statement` | string | The database statement being executed. [3][5] | `SELECT * FROM wuser_table`
`SET mykey "WuValue"` | Conditional.
Required if applicable. | -| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET` | Conditional
Required, if `db.statement` is not applicable. | -| `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Conditional.
Required if applicable. | +| `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Required if applicable. | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET`
`SELECT` | Required if applicable. | +| `db.table` | string | The name of the primary table, collection, segment, etc... that the operation is acting upon. | `user_table` | Required if applicable. | +| `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Required if applicable. | **[1]:** In some SQL databases, the database name to be used is called "schema name". -**[2]:** Required, if applicable and no more-specific attribute is defined. +**[4]:** For SQL operations, this should be set to the SQL keyword (example: `SELECT` or `INSERT`). -**[3]:** The value may be sanitized to exclude sensitive information. - -**[4]:** While it would semantically make sense to set this, e.g., to a SQL keyword like `SELECT` or `INSERT`, it is not recommended to attempt any client-side parsing of `db.statement` just to get this property (the back end can do that if required). - -**[5]:** To reduce cardinality, the values for `db.statement` and `db.operation` -should have parameters removed or substituted. The resulting value should be a -low-cardinality value represeting the statement or operation being executed on -the database. It may be a stored procedure name (without arguments), SQL -statement without variable arguments, operation name, etc. +**[5]:** To reduce cardinality, the value for `db.operation` should have parameters +removed or substituted. The resulting value should be a low-cardinality value +represeting the statement or operation being executed on the database. It may be +a stored procedure name (without arguments), operation name, etc. ### Call-level labels for specific technologies From 6b66fd30d78dd1647402158011620f3f02cd1674 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 8 Oct 2020 11:15:58 -0700 Subject: [PATCH 08/23] Replace Attribute name with Label name in database metric semantic conventions --- specification/metrics/semantic_conventions/database.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 2d2471ab419..73b1c10b837 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -8,7 +8,7 @@ OpenTelemetry. When instrumenting database clients, also consider the The following labels SHOULD be applied to all database metric instruments. -| Attribute | Description | Example | Required | +| Label Name | Description | Example | Required | |------------------------|--------------|----------|----------| | `db.system` | An identifier for the database management system (DBMS) product being used. [1] | `other_sql` | Yes | | `db.connection_string` | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | No | @@ -41,10 +41,10 @@ databases), as well as connect operations. In addition to the [common](#common) labels, the following labels SHOULD be applied to all database call-level metric instruments. -| Attribute | Type | Description | Example | Required | +| Label Name | Type | Description | Example | Required | |------------------|--------|--------------|----------|----------| | `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Required if applicable. | -| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET`
`SELECT` | Required if applicable. | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET`
`SELECT`
`CONNECT` | Required if applicable. | | `db.table` | string | The name of the primary table, collection, segment, etc... that the operation is acting upon. | `user_table` | Required if applicable. | | `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Required if applicable. | @@ -59,7 +59,7 @@ a stored procedure name (without arguments), operation name, etc. ### Call-level labels for specific technologies -| Attribute | Description | Example | Required | +| Label Name | Description | Example | Required | |---------------------------|--------------|----------|----------| | `db.cassandra.keyspace` | The name of the keyspace being accessed. To be used instead of the generic `db.name` attribute. | `mykeyspace` | Yes | | `db.hbase.namespace` | The [HBase namespace](https://hbase.apache.org/book.html#_namespace) being accessed. To be used instead of the generic `db.name` attribute. | `default` | Yes | From 10a3ef4b5f6537c298b8a856b14c39dc751f2b5a Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 8 Oct 2020 11:16:24 -0700 Subject: [PATCH 09/23] Reorder connection pooling metric instruments in semantic conventions --- .../metrics/semantic_conventions/database.md | 23 +++++++++---------- 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 73b1c10b837..aa430d32b4a 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -70,8 +70,17 @@ a stored procedure name (without arguments), operation name, etc. ## Connection Pooling Metric Instruments -If possible, instrumentation SHOULD collect the following. They SHOULD have -all [common](#common) labels applied to them. +Otherwise, the following metric instruments SHOULD be collected. They SHOULD +have all [common](#common) labels applied to them. + +| Name | Instrument | Units | Description | +|---------------------------|---------------|--------------|-------------| +| `db.connection_pool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | +| `db.connection_pool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | + +If the following detailed information is available, instrumentation MAY collect +the following metric instruments. They SHOULD have all [common](#common) labels +applied to them. | Name | Instrument | Units | Description | |---------------------------|------------|---------------|-------------| @@ -80,13 +89,3 @@ all [common](#common) labels applied to them. | `db.connections.returned` | Counter | {connections} | The number of connections returned to the connection pool. | | `db.connections.reused` | Counter | {connections} | The number of connections reused. | | `db.connections.closed` | Counter | {connections} | The number of connections closed. | - - -Otherwise, the following metric instruments SHOULD be collected. They SHOULD -have all [common](#common) labels applied to them. - -| Name | Instrument | Units | Description | -|---------------------------|---------------|--------------|-------------| -| `db.connectionPool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | -| `db.connectionPool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | - From e9b0bdb72e54e7f73c5e5c15bb173a328d8a4f3a Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 8 Oct 2020 12:45:55 -0700 Subject: [PATCH 10/23] Add duration database metric examples --- .../metrics/semantic_conventions/database.md | 120 ++++++++++++++++++ 1 file changed, 120 insertions(+) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index aa430d32b4a..e0ae62a1c36 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -68,6 +68,126 @@ a stored procedure name (without arguments), operation name, etc. **[1]:** Required, if other than the default database (`0`). +### Examples + +#### PostgreSQL SELECT Query + +For a client executing a query like this: + +```SQL +SELECT * FROM public.user_table WHERE user_id = 301; +``` + +while connected to a PostgreSQL database named "user_db" running on host +`postgres-server:5432`, the following instrument should result: + +```json +{ + "name": "db.client.duration", + "labels": { + "db.operation": "SELECT", + "db.table": "user_table", + "db.name": "user_db", + "db.system": "postgresql", + "db.connection_string": "postgresql://postgres-server:5432/user_db", + "db.user": "", + "net.peer.ip": "192.0.10.2", + "net.peer.port": 5432, + "net.peer.name": "postgres-server" + } +} +``` + +#### MySQL SELECT Query + +For a client executing a query like this: + +```SQL +SELECT * FROM orders WHERE order_id = 301; +``` + +while connected to a MySQL database named "ShopDb" running on host +`shopdb.example.com`, the following instrument should result: + + +```json +{ + "name": "db.client.duration", + "labels": { + "db.operation": "SELECT", + "db.table": "orders", + "db.name": "ShopDb", + "db.system": "mysql", + "db.connection_string": "Server=shopdb.example.com;Database=ShopDb;Uid=billing_user;TableCache=true;UseCompression=True;MinimumPoolSize=10;MaximumPoolSize=50;", + "db.user": "billing_user", + "net.peer.name": "shopdb.example.com", + "net.peer.ip": "192.0.2.12", + "net.peer.port": "3306", + "net.transport": "IP.TCP" + } +} + +``` + +#### Redis HMSET + +For a client executing a Redis HMSET command like this: + +```redis +HMSET myhash field1 'Hello' field2 'World +``` + +while connecting to a Redis instance over a Unix socket, the following instrument +should result: + +```json +{ + "name": "db.client.duration", + "labels": { + "db.operation": "HMSET", + "db.table": "myhash", + "db.system": "redis", + "db.user": "the_user", + "net.peer.name": "/tmp/redis.sock", + "net.transport": "Unix", + "db.redis.database_index": "15" + } +} +``` + +#### MongoDB findAndModify + +For a mongo client executing the `findAndModify` command like this: + +```javascript +{ + findAndModify: "people", + query: { name: "Tom", state: "active", rating: { $gt: 10 } }, + sort: { rating: 1 }, + update: { $inc: { score: 1 } } +} +``` + +against the database named "userdatabase" while connected to a MongoDB available +at `mongodb.example.com`, the following instrument should result: + +```json +{ + "name": "db.client.duration", + "labels": { + "db.operation": "findAndModify", + "db.table": "people", + "db.name": "userdatabase", + "db.system": "mongodb", + "db.user": "the_user", + "net.peer.name": "mongodb.example.com", + "net.peer.ip": "192.0.2.14", + "net.peer.port": "27017", + "net.transport": "IP.TCP" + } +} +``` + ## Connection Pooling Metric Instruments Otherwise, the following metric instruments SHOULD be collected. They SHOULD From 506fe6e21e71567f5a1b6d92de56fc75322583fe Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 8 Oct 2020 14:57:22 -0700 Subject: [PATCH 11/23] Fix markdown-lint errors --- .../metrics/semantic_conventions/database.md | 68 +++++++++---------- 1 file changed, 33 insertions(+), 35 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index e0ae62a1c36..86234a57f42 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -1,7 +1,7 @@ # Semantic Conventions for Database Metrics This document contains semantic conventions for database client metrics in -OpenTelemetry. When instrumenting database clients, also consider the +OpenTelemetry. When instrumenting database clients, also consider the [general metric semantic conventions](README.md#general-metric-semantic-conventions). ## Common @@ -18,7 +18,6 @@ The following labels SHOULD be applied to all database metric instruments. | `net.peer.port` | Remote port number. | `80`
`8080`
`443` | No | | `net.peer.name` | Remote hostname or similar, see note below. | `example.com` | No | - **[1]:** See the [database trace semantic conventions](../../trace/semantic_conventions/database.md#connection-level-attributes) for the list of well-known database system values. @@ -78,7 +77,7 @@ For a client executing a query like this: SELECT * FROM public.user_table WHERE user_id = 301; ``` -while connected to a PostgreSQL database named "user_db" running on host +while connected to a PostgreSQL database named "user_db" running on host `postgres-server:5432`, the following instrument should result: ```json @@ -106,24 +105,23 @@ For a client executing a query like this: SELECT * FROM orders WHERE order_id = 301; ``` -while connected to a MySQL database named "ShopDb" running on host +while connected to a MySQL database named "ShopDb" running on host `shopdb.example.com`, the following instrument should result: - ```json { "name": "db.client.duration", "labels": { - "db.operation": "SELECT", - "db.table": "orders", - "db.name": "ShopDb", - "db.system": "mysql", - "db.connection_string": "Server=shopdb.example.com;Database=ShopDb;Uid=billing_user;TableCache=true;UseCompression=True;MinimumPoolSize=10;MaximumPoolSize=50;", - "db.user": "billing_user", - "net.peer.name": "shopdb.example.com", - "net.peer.ip": "192.0.2.12", - "net.peer.port": "3306", - "net.transport": "IP.TCP" + "db.operation": "SELECT", + "db.table": "orders", + "db.name": "ShopDb", + "db.system": "mysql", + "db.connection_string": "Server=shopdb.example.com;Database=ShopDb;Uid=billing_user;TableCache=true;UseCompression=True;MinimumPoolSize=10;MaximumPoolSize=50;", + "db.user": "billing_user", + "net.peer.name": "shopdb.example.com", + "net.peer.ip": "192.0.2.12", + "net.peer.port": "3306", + "net.transport": "IP.TCP" } } @@ -144,13 +142,13 @@ should result: { "name": "db.client.duration", "labels": { - "db.operation": "HMSET", - "db.table": "myhash", - "db.system": "redis", - "db.user": "the_user", - "net.peer.name": "/tmp/redis.sock", - "net.transport": "Unix", - "db.redis.database_index": "15" + "db.operation": "HMSET", + "db.table": "myhash", + "db.system": "redis", + "db.user": "the_user", + "net.peer.name": "/tmp/redis.sock", + "net.transport": "Unix", + "db.redis.database_index": "15" } } ``` @@ -175,15 +173,15 @@ at `mongodb.example.com`, the following instrument should result: { "name": "db.client.duration", "labels": { - "db.operation": "findAndModify", - "db.table": "people", - "db.name": "userdatabase", - "db.system": "mongodb", - "db.user": "the_user", - "net.peer.name": "mongodb.example.com", - "net.peer.ip": "192.0.2.14", - "net.peer.port": "27017", - "net.transport": "IP.TCP" + "db.operation": "findAndModify", + "db.table": "people", + "db.name": "userdatabase", + "db.system": "mongodb", + "db.user": "the_user", + "net.peer.name": "mongodb.example.com", + "net.peer.ip": "192.0.2.14", + "net.peer.port": "27017", + "net.transport": "IP.TCP" } } ``` @@ -204,8 +202,8 @@ applied to them. | Name | Instrument | Units | Description | |---------------------------|------------|---------------|-------------| -| `db.connections.new` | Counter | {connections} | The number of new connections created. | -| `db.connections.taken` | Counter | {connections} | The number of connections taken from the connection pool. | -| `db.connections.returned` | Counter | {connections} | The number of connections returned to the connection pool. | -| `db.connections.reused` | Counter | {connections} | The number of connections reused. | +| `db.connections.new` | Counter | {connections} | The number of new connections created. | +| `db.connections.taken` | Counter | {connections} | The number of connections taken from the connection pool. | +| `db.connections.returned` | Counter | {connections} | The number of connections returned to the connection pool. | +| `db.connections.reused` | Counter | {connections} | The number of connections reused. | | `db.connections.closed` | Counter | {connections} | The number of connections closed. | From b16d67d097c011b727102dc786e31f014414b3c2 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Fri, 9 Oct 2020 11:32:20 -0700 Subject: [PATCH 12/23] Add markdown TOC to database metric semantic conventions Co-authored-by: Armin Ruech --- specification/metrics/semantic_conventions/database.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 86234a57f42..36696a0c807 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -1,5 +1,13 @@ # Semantic Conventions for Database Metrics + + + + +Run the tool to have the TOC generated here + + + This document contains semantic conventions for database client metrics in OpenTelemetry. When instrumenting database clients, also consider the [general metric semantic conventions](README.md#general-metric-semantic-conventions). From deea04547bf92336a398f0ee2396c32f27a53f10 Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 9 Oct 2020 11:33:39 -0700 Subject: [PATCH 13/23] Update database semantic convention; respond to PR feedback --- .../metrics/semantic_conventions/database.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 36696a0c807..7a0d2607b97 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -140,7 +140,7 @@ while connected to a MySQL database named "ShopDb" running on host For a client executing a Redis HMSET command like this: ```redis -HMSET myhash field1 'Hello' field2 'World +HMSET myhash field1 'Hello' field2 'World' ``` while connecting to a Redis instance over a Unix socket, the following instrument @@ -196,16 +196,17 @@ at `mongodb.example.com`, the following instrument should result: ## Connection Pooling Metric Instruments -Otherwise, the following metric instruments SHOULD be collected. They SHOULD -have all [common](#common) labels applied to them. +The following metric instruments SHOULD be collected for database connection +pools. They SHOULD have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | |---------------------------|---------------|--------------|-------------| | `db.connection_pool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | | `db.connection_pool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | -If the following detailed information is available, instrumentation MAY collect -the following metric instruments. They SHOULD have all [common](#common) labels + +If the following detailed information is available, the following metric +instruments MAY be collected. They SHOULD have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | From 4d96aab2a2d0afd42ba1ce7a75aea90bd0fb3942 Mon Sep 17 00:00:00 2001 From: Justin Date: Fri, 9 Oct 2020 14:23:27 -0700 Subject: [PATCH 14/23] Update changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b813bfbbb0..c79ae4bd4ad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,7 @@ New: ([#994](https://github.com/open-telemetry/opentelemetry-specification/pull/994)) - Add Metric SDK specification (partial): covering terminology and Accumulator component ([#626](https://github.com/open-telemetry/opentelemetry-specification/pull/626)) +- Add semantic conventions for Database metrics ([#1076](https://github.com/open-telemetry/opentelemetry-specification/pull/1076)) Updates: From dc92381b8631cf778f61053603b89d80965caa5e Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 10 Oct 2020 11:16:21 -0700 Subject: [PATCH 15/23] Fix markdown-lint errors in database metric semantic conventions --- specification/metrics/semantic_conventions/database.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 7a0d2607b97..1eb3ee0b6ba 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -204,7 +204,6 @@ pools. They SHOULD have all [common](#common) labels applied to them. | `db.connection_pool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | | `db.connection_pool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | - If the following detailed information is available, the following metric instruments MAY be collected. They SHOULD have all [common](#common) labels applied to them. From 88c6e1cbe7d14504c33a0575e8587effdc211667 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Fri, 23 Oct 2020 11:49:48 -0700 Subject: [PATCH 16/23] Update CHANGELOG.md Co-authored-by: Armin Ruech --- CHANGELOG.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c79ae4bd4ad..1e4fa7cc9b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,7 +39,8 @@ New: ([#994](https://github.com/open-telemetry/opentelemetry-specification/pull/994)) - Add Metric SDK specification (partial): covering terminology and Accumulator component ([#626](https://github.com/open-telemetry/opentelemetry-specification/pull/626)) -- Add semantic conventions for Database metrics ([#1076](https://github.com/open-telemetry/opentelemetry-specification/pull/1076)) +- Add semantic conventions for Database metrics + ([#1076](https://github.com/open-telemetry/opentelemetry-specification/pull/1076)) Updates: From 7bdc85d9cf011a3bbf89cc53b955801046f16850 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Fri, 23 Oct 2020 12:49:10 -0700 Subject: [PATCH 17/23] Actually add the TOC to database metric semantic conventions --- .../metrics/semantic_conventions/database.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 1eb3ee0b6ba..1238e6fe053 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -4,7 +4,16 @@ -Run the tool to have the TOC generated here +- [Common](#common) +- [Call-level Metric Instruments](#call-level-metric-instruments) + * [Labels](#labels) + * [Call-level labels for specific technologies](#call-level-labels-for-specific-technologies) + * [Examples](#examples) + + [PostgreSQL SELECT Query](#postgresql-select-query) + + [MySQL SELECT Query](#mysql-select-query) + + [Redis HMSET](#redis-hmset) + + [MongoDB findAndModify](#mongodb-findandmodify) +- [Connection Pooling Metric Instruments](#connection-pooling-metric-instruments) From cf830466b9d89d810df08a0e2b129105592c8be7 Mon Sep 17 00:00:00 2001 From: Justin Date: Mon, 2 Nov 2020 11:37:40 -0800 Subject: [PATCH 18/23] Add db.sql.table to db metric semantic conventions --- specification/metrics/semantic_conventions/database.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 1238e6fe053..67d2c684c05 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -60,13 +60,12 @@ applied to all database call-level metric instruments. | Label Name | Type | Description | Example | Required | |------------------|--------|--------------|----------|----------| | `db.name` | string | If no [tech-specific label](#call-level-labels-for-specific-technologies) is defined, this attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). [1] | `customers`
`main` | Required if applicable. | -| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`. [4][5] | `findAndModify`
`HMSET`
`SELECT`
`CONNECT` | Required if applicable. | -| `db.table` | string | The name of the primary table, collection, segment, etc... that the operation is acting upon. | `user_table` | Required if applicable. | +| `db.operation` | string | The name of the operation being executed, e.g. the [MongoDB command name](https://docs.mongodb.com/manual/reference/command/#database-operations) such as `findAndModify`, or the SQL keyword. [4][5] | `findAndModify`
`HMSET`
`SELECT`
`CONNECT` | Required if applicable. | | `exception.type` | string | The type of the exception (its fully-qualified class name, if applicable). The dynamic type of the exception should be preferred over the static type in languages that support it. | `java.sql.SQLException`
`psycopg2.OperationalError` | Required if applicable. | **[1]:** In some SQL databases, the database name to be used is called "schema name". -**[4]:** For SQL operations, this should be set to the SQL keyword (example: `SELECT` or `INSERT`). +**[4]:** When setting this to an SQL keyword, it is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if the operation name is provided by the library being instrumented. If the SQL statement has an ambiguous operation, or performs more than one operation, this value may be omitted. **[5]:** To reduce cardinality, the value for `db.operation` should have parameters removed or substituted. The resulting value should be a low-cardinality value @@ -81,9 +80,12 @@ a stored procedure name (without arguments), operation name, etc. | `db.hbase.namespace` | The [HBase namespace](https://hbase.apache.org/book.html#_namespace) being accessed. To be used instead of the generic `db.name` attribute. | `default` | Yes | | `db.redis.database_index` | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` label. | `0`
`1`
`15` | Conditional [1] | | `db.mongodb.collection` | The collection being accessed within the database stated in `db.name`. | `customers`
`products` | Yes | +| `db.sql.table` | string | The name of the primary table that the operation is acting upon, including the schema name (if applicable). [2] | `public.users`
`customers` | Conditional
Recommended if available. | **[1]:** Required, if other than the default database (`0`). +**[2]:** It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if it is provided by the library being instrumented. If the operation is acting upon an anonymous table, or more than one table, this value may be omitted. + ### Examples #### PostgreSQL SELECT Query From ec58de44c2fa993fc6cf790edb55b364f6894413 Mon Sep 17 00:00:00 2001 From: Justin Date: Mon, 2 Nov 2020 12:06:03 -0800 Subject: [PATCH 19/23] Add more explanation of db connection pool metrics --- .../metrics/semantic_conventions/database.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 67d2c684c05..a77246d99b6 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -221,8 +221,10 @@ applied to them. | Name | Instrument | Units | Description | |---------------------------|------------|---------------|-------------| -| `db.connections.new` | Counter | {connections} | The number of new connections created. | -| `db.connections.taken` | Counter | {connections} | The number of connections taken from the connection pool. | -| `db.connections.returned` | Counter | {connections} | The number of connections returned to the connection pool. | -| `db.connections.reused` | Counter | {connections} | The number of connections reused. | -| `db.connections.closed` | Counter | {connections} | The number of connections closed. | +| `db.connections.taken` | Counter | {connections} | Incremented when a connection is requested and returned from the pool. | +| `db.connections.new` | Counter | {connections} | Incremented when a connection is requested and returned from the pool only if the connection pool returns a newly created connection. | +| `db.connections.reused` | Counter | {connections} | Incremented when a connection is requested and returned from the pool only if the connection pool returns a previously used connection. | +| `db.connections.returned` | Counter | {connections} | Incremented when the application is finished using a connection and returns it to the pool. | +| `db.connections.closed` | Counter | {connections} | Incremented when the pool closes a connection. [1] | + +**[1]:** Connection pools sometimes close connections after some pre-configured time has elapsed, or when the application is shutting down. The `db.connections.closed` instrument should be incremented for each connection closed. It should _eventually_ be true that `db.connections.new - db.connections.closed = 0`. From 9f9545771e705d588fb31d8e1aacc67b86e23298 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 11 Nov 2020 11:05:17 -0800 Subject: [PATCH 20/23] Update database metric semantic convention in response to PR feedback --- .../metrics/semantic_conventions/database.md | 33 +++++++++++-------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index a77246d99b6..7f2d873bfb2 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -30,10 +30,10 @@ The following labels SHOULD be applied to all database metric instruments. | `db.system` | An identifier for the database management system (DBMS) product being used. [1] | `other_sql` | Yes | | `db.connection_string` | The connection string used to connect to the database. It is recommended to remove embedded credentials. | `Server=(localdb)\v11.0;Integrated Security=true;` | No | | `db.user` | Username for accessing the database. | `readonly_user`
`reporting_user` | No | -| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP`
`Unix` | Conditional [2] | | `net.peer.ip` | Remote address of the peer (dotted decimal for IPv4 or [RFC5952](https://tools.ietf.org/html/rfc5952) for IPv6) | `127.0.0.1` | No | -| `net.peer.port` | Remote port number. | `80`
`8080`
`443` | No | | `net.peer.name` | Remote hostname or similar, see note below. | `example.com` | No | +| `net.peer.port` | Remote port number. | `80`
`8080`
`443` | No | +| `net.transport` | Transport protocol used. See note below. See [general network connection attributes](../../trace/semantic_conventions/span-general.md#general-network-connection-attributes). | `IP.TCP`
`Unix` | Conditional [2] | **[1]:** See the [database trace semantic conventions](../../trace/semantic_conventions/database.md#connection-level-attributes) for the list of well-known database system values. @@ -48,8 +48,8 @@ The following metric instruments SHOULD be iterated for every database operation |----------------------|---------------|--------------|-------------| | `db.client.duration` | ValueRecorder | milliseconds | The duration of the database operation. | -Database operations SHOULD include execution of queries, including DDL, DML, -DCL, and TCL SQL statements (and the corresponding operations in non-SQL +Measured database operations SHOULD include execution of queries, including DDL, +DML, DCL, and TCL SQL statements (and the corresponding operations in non-SQL databases), as well as connect operations. ### Labels @@ -65,12 +65,17 @@ applied to all database call-level metric instruments. **[1]:** In some SQL databases, the database name to be used is called "schema name". -**[4]:** When setting this to an SQL keyword, it is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if the operation name is provided by the library being instrumented. If the SQL statement has an ambiguous operation, or performs more than one operation, this value may be omitted. +**[4]:** When setting this to an SQL keyword, it is not recommended to attempt +any client-side parsing of `db.statement` just to get this property, but it +should be set if the operation name is provided by the library being +instrumented. If the SQL statement has an ambiguous operation, or performs more +than one operation, this value may be omitted. -**[5]:** To reduce cardinality, the value for `db.operation` should have parameters -removed or substituted. The resulting value should be a low-cardinality value -represeting the statement or operation being executed on the database. It may be -a stored procedure name (without arguments), operation name, etc. +**[5]:** To reduce cardinality, the value for `db.operation` should include +placeholders instead of actual parameter values. The resulting value should be +low-cardinality and represent the statement or operation being executed on +the database. It may be a stored procedure name (without arguments), +operation name, etc. ### Call-level labels for specific technologies @@ -104,11 +109,11 @@ while connected to a PostgreSQL database named "user_db" running on host "name": "db.client.duration", "labels": { "db.operation": "SELECT", - "db.table": "user_table", "db.name": "user_db", "db.system": "postgresql", "db.connection_string": "postgresql://postgres-server:5432/user_db", "db.user": "", + "db.sql.table": "public.user_table", "net.peer.ip": "192.0.10.2", "net.peer.port": 5432, "net.peer.name": "postgres-server" @@ -132,11 +137,11 @@ while connected to a MySQL database named "ShopDb" running on host "name": "db.client.duration", "labels": { "db.operation": "SELECT", - "db.table": "orders", "db.name": "ShopDb", "db.system": "mysql", "db.connection_string": "Server=shopdb.example.com;Database=ShopDb;Uid=billing_user;TableCache=true;UseCompression=True;MinimumPoolSize=10;MaximumPoolSize=50;", "db.user": "billing_user", + "db.sql.table": "orders", "net.peer.name": "shopdb.example.com", "net.peer.ip": "192.0.2.12", "net.peer.port": "3306", @@ -162,9 +167,9 @@ should result: "name": "db.client.duration", "labels": { "db.operation": "HMSET", - "db.table": "myhash", "db.system": "redis", "db.user": "the_user", + "db.redis.database_index": "0", "net.peer.name": "/tmp/redis.sock", "net.transport": "Unix", "db.redis.database_index": "15" @@ -193,7 +198,7 @@ at `mongodb.example.com`, the following instrument should result: "name": "db.client.duration", "labels": { "db.operation": "findAndModify", - "db.table": "people", + "db.mongo.collection": "people", "db.name": "userdatabase", "db.system": "mongodb", "db.user": "the_user", @@ -207,7 +212,7 @@ at `mongodb.example.com`, the following instrument should result: ## Connection Pooling Metric Instruments -The following metric instruments SHOULD be collected for database connection +The following metric instruments SHOULD be used for database connection pools. They SHOULD have all [common](#common) labels applied to them. | Name | Instrument | Units | Description | From b2bebfc95cbce417683c4a59e82a795bfcf0d6d1 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Thu, 19 Nov 2020 14:00:35 -0800 Subject: [PATCH 21/23] Add technology-specific db metric instruments guidelines --- .../metrics/semantic_conventions/database.md | 33 ++++++++++++------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index 7f2d873bfb2..daf8e2563e5 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -14,6 +14,8 @@ + [Redis HMSET](#redis-hmset) + [MongoDB findAndModify](#mongodb-findandmodify) - [Connection Pooling Metric Instruments](#connection-pooling-metric-instruments) +- [Additional metric instruments for specific technologies](#additional-metric-instruments-for-specific-technologies) + * [Examples](#examples-1) @@ -220,16 +222,25 @@ pools. They SHOULD have all [common](#common) labels applied to them. | `db.connection_pool.limit` | ValueObserver | {connections} | The total number of database connections available in the connection pool. | | `db.connection_pool.usage` | ValueObserver | {connections} | The number of database connections _in use_. | -If the following detailed information is available, the following metric -instruments MAY be collected. They SHOULD have all [common](#common) labels -applied to them. +## Additional metric instruments for specific technologies -| Name | Instrument | Units | Description | -|---------------------------|------------|---------------|-------------| -| `db.connections.taken` | Counter | {connections} | Incremented when a connection is requested and returned from the pool. | -| `db.connections.new` | Counter | {connections} | Incremented when a connection is requested and returned from the pool only if the connection pool returns a newly created connection. | -| `db.connections.reused` | Counter | {connections} | Incremented when a connection is requested and returned from the pool only if the connection pool returns a previously used connection. | -| `db.connections.returned` | Counter | {connections} | Incremented when the application is finished using a connection and returns it to the pool. | -| `db.connections.closed` | Counter | {connections} | Incremented when the pool closes a connection. [1] | +Additional metric instruments MAY be used for technology-specific measurements. Such +instruments SHOULD have all [common](#common) labels applied to them, and their names +should follow the pattern (in which `{db.system}` is one of the allowed values for the +db.system trace attribute): -**[1]:** Connection pools sometimes close connections after some pre-configured time has elapsed, or when the application is shutting down. The `db.connections.closed` instrument should be incremented for each connection closed. It should _eventually_ be true that `db.connections.new - db.connections.closed = 0`. +``` +db.{db.system}.* +``` + +### Examples + +* `db.redis.connections.taken` +* `db.redis.connections.new` +* `db.redis.connections.reused` +* `db.redis.connections.returned` +* `db.redis.connections.closed` +* `db.mongo.average_object_size` +* `db.mongo.write_lock_percentage` +* `db.mongo.object_count` +* `db.postgresql.cache_hits` From e6acc6acfffff9833216f548d70a2b1ed4aaab69 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Sun, 29 Nov 2020 12:41:26 -0800 Subject: [PATCH 22/23] Fix markdown table layout for db metric labels --- specification/metrics/semantic_conventions/database.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index daf8e2563e5..c78ecd2fda8 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -87,7 +87,7 @@ operation name, etc. | `db.hbase.namespace` | The [HBase namespace](https://hbase.apache.org/book.html#_namespace) being accessed. To be used instead of the generic `db.name` attribute. | `default` | Yes | | `db.redis.database_index` | The index of the database being accessed as used in the [`SELECT` command](https://redis.io/commands/select), provided as an integer. To be used instead of the generic `db.name` label. | `0`
`1`
`15` | Conditional [1] | | `db.mongodb.collection` | The collection being accessed within the database stated in `db.name`. | `customers`
`products` | Yes | -| `db.sql.table` | string | The name of the primary table that the operation is acting upon, including the schema name (if applicable). [2] | `public.users`
`customers` | Conditional
Recommended if available. | +| `db.sql.table` | The name of the primary table that the operation is acting upon, including the schema name (if applicable). [2] | `public.users`
`customers` | Conditional
Recommended if available. | **[1]:** Required, if other than the default database (`0`). From efba901d05511a0995cfd02545dd2302185b8b81 Mon Sep 17 00:00:00 2001 From: Justin Foote Date: Sun, 29 Nov 2020 12:46:46 -0800 Subject: [PATCH 23/23] Replace "mongo" with "mongodb" in metrics database semantic conventions --- specification/metrics/semantic_conventions/database.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/metrics/semantic_conventions/database.md b/specification/metrics/semantic_conventions/database.md index c78ecd2fda8..21fd554d48b 100644 --- a/specification/metrics/semantic_conventions/database.md +++ b/specification/metrics/semantic_conventions/database.md @@ -240,7 +240,7 @@ db.{db.system}.* * `db.redis.connections.reused` * `db.redis.connections.returned` * `db.redis.connections.closed` -* `db.mongo.average_object_size` -* `db.mongo.write_lock_percentage` -* `db.mongo.object_count` +* `db.mongodb.average_object_size` +* `db.mongodb.write_lock_percentage` +* `db.mongodb.object_count` * `db.postgresql.cache_hits`