Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation #1141

justinfoote · 2020-10-23T19:27:53Z

Changes

This PR adds a new attribute to the trace semantic conventions for database operations: db.table, and adds guidance that this attribute should be set only if the value is readily available from the library being instrumented.
It also changes the guidance around db.operation to indicate that it should be set to the SQL keyword, but also only if the value is readily available from the library being instrumented.

anuraaga

Table is important, this seems nice

specification/trace/semantic_conventions/database.md

Oberon00 · 2020-10-24T11:00:57Z

semantic_conventions/trace/database.yaml

+            It is not recommended to attempt any client-side parsing of
+            `db.statement` just to get this property, but it should be set if 
+            the operation name is provided by the library being instrumented.
+            If the statement has an ambiguous operation, or performs more than


This seems to imply that operation is always derived from statement, which is not true. In fact, the main purpose of this attribute is to support DBs that do not use a statement/query language as their API, such as CouchDB.

Yeah, I see your point. This certainly was not my intention.

In the main "brief" description of this attribute, we still list the mongo example first. We could add a few more examples there.

Or I wonder if it would be better to include something in the footnote like this:

note: > In the case of SQL databases, the operation should be set to the SQL keyword (example: `SELECT` or `INSERT`). It is not recommended to attempt any client-side parsing of `db.statement` just to get this property, but it should be set if the operation name is provided by the library being instrumented. If the statement has an ambiguous operation, or performs more than one operation, this value may be omitted.

Is this redundant with the note in the 'required' column: "Required, if db.statement is not available."

What do you think?

I suggested a simpler change by emphasizing that that part only applies to SQL in #1141 (review)

semantic_conventions/trace/database.yaml

justinfoote · 2020-10-29T20:36:02Z

@open-telemetry/technical-committee Is this ready to merge?

semantic_conventions/trace/database.yaml

Oberon00 · 2020-10-30T16:31:49Z

semantic_conventions/trace/database.yaml

+            It is not recommended to attempt any client-side parsing of
+            `db.statement` just to get this property, but it should be set if 
+            the operation name is provided by the library being instrumented.
+            If the statement has an ambiguous operation, or performs more than


I suggested a simpler change by emphasizing that that part only applies to SQL in #1141 (review)

Oberon00 · 2020-10-30T16:34:45Z

semantic_conventions/trace/database.yaml

+            If the statement has an ambiguous operation, or performs more than
+            one operation, this value may be omitted.
+          examples: ['findAndModify', 'HMSET', 'SELECT']
+        - id: table


Actually I just realized that the term table is probably specific to SQL. Thus I recommend adding this to a new semantic convention db.sql in this file that extends: db and has this table attribute, so that the full name is db.sql.table.

An alternative might be to have a more generic term, but since a table likely has no 1:1 mapping to other DB concepts (there might be several concepts that match a bit), I think having this in a SQL-specific convention would be easier.

This is a good idea. I'll leave the operation where it is, but move the table to an SQL specific section in Call-level attributes for specific technologies.

Done in 657670c

Oberon00

Requesting changes because I have concerns for adding this SQL-specific attribute to the very general DB conventions.

Oberon00

LGTM! Thanks for responding to my comments so quickly!

justinfoote · 2020-11-02T18:27:29Z

@open-telemetry/technical-committee I think this actually is ready to merge now.

semantic_conventions/trace/database.yaml

Co-authored-by: Armin Ruech <[email protected]>

justinfoote · 2020-11-06T03:15:04Z

@bogdandrutu @jmacd -- this is the PR I mentioned in the metrics SIG today that's holding up the database metrics semantic conventions.

arminru · 2020-11-09T16:53:53Z

@justinfoote Sorry I've missed that before: Could you please add an entry in the changelog to make people aware of the new table and the changed operation attribute? Once that's done we can merge this PR right away.

justinfoote · 2020-11-09T18:07:20Z

@arminru Added to the changelog in 862361f.

jamal · 2020-11-10T16:17:11Z

One question about this change, I noticed that we still have db.mongodb.collection. Is it expected to use both or should the other be deprecated in favor of db.table?

arminru · 2020-11-10T16:27:09Z

@jamal Actually we should have updated the PR name and changelog entry - it is called db.sql.table now and explicitly only applies to SQL-like databases and is thus not a replacement for the dedicated db.mongodb.collection.

* Fix changelog entry for #1141 * Apply suggestions from code review Co-authored-by: Christian Neumüller <[email protected]>

jamal · 2020-11-10T18:12:38Z

@arminru Ah, I missed that, thank you. This definitely solves some of the problems I wanted to solve.

An aside but being new to all of this, I'm a bit curious. Was this discussed in a specification meeting? I don't see any conversations around this in the PR, and I would have loved to be able to provide some feedback.

Having a specific solution for sql makes sense, since table is not a generic name (for example, Mongo). But, there are cases I wanted to cover as well that don't make sense. For example, Redis doesn't have a concept of table / collection. But, in our own implementation we introduced a wrapper over Lettuce to allow us to specify a collection name at the connection level. This allows us to segment metrics / traces, and I was hoping to extend our own instrumentation to tag this in traces. Obviously, this is not standard so it wouldn't be part of the spec.

But, because this change is specific to SQL, vendors / backends may not support this for Redis and allow us to provide this enhancement. I wonder if there are any thoughts on how to support cases like this?

I'll review other cases to see if I can think of other gaps I would want to make sure are covered and will open a separate issue if it comes up.

Thanks so much!

Oberon00 · 2020-11-10T18:17:44Z

Was this discussed in a specification meeting? I don't see any conversations around this in the PR, and I would have loved to be able to provide some feedback.

There is more discussion in the linked issue #1104. I don't remember this being discussed in a meeting.

jamal · 2020-11-10T19:05:09Z

@Oberon00 Oh, sorry, I feel silly. Totally missed that conversation when I checked the Issue again. Thanks! I'll think through this and will start a separate conversation if I have any recommendations on how this could be extended. Thanks so much!

…operation (open-telemetry#1141) * Add db.table to database semantic conventions * Fix spacing in db semantic conventions * Move db.table to SQL-specific call level attributes * Clarify instructions for setting db.operation to a SQL keyword * Specify that db.sql.table MUST NOT be set when not applicable Co-authored-by: Armin Ruech <[email protected]> * Add db.table and db.operation to CHANGELOG Co-authored-by: Armin Ruech <[email protected]>

* Fix changelog entry for open-telemetry#1141 * Apply suggestions from code review Co-authored-by: Christian Neumüller <[email protected]>

Add db.table to database semantic conventions

bcc9ca1

justinfoote requested review from a team October 23, 2020 19:27

github-actions bot assigned tigrannajaryan Oct 23, 2020

justinfoote mentioned this pull request Oct 23, 2020

Database metric semantic conventions #1076

Closed

iNikem approved these changes Oct 24, 2020

View reviewed changes

anuraaga approved these changes Oct 24, 2020

View reviewed changes

specification/trace/semantic_conventions/database.md Outdated Show resolved Hide resolved

Oberon00 reviewed Oct 24, 2020

View reviewed changes

jmacd approved these changes Oct 27, 2020

View reviewed changes

thisthat reviewed Oct 27, 2020

View reviewed changes

semantic_conventions/trace/database.yaml Outdated Show resolved Hide resolved

Fix spacing in db semantic conventions

5c05f92

Oberon00 mentioned this pull request Oct 30, 2020

Is it okay to use SQL statement type (e.g. just "SELECT") as span name? #1139

Closed

Oberon00 reviewed Oct 30, 2020

View reviewed changes

Oberon00 changed the title ~~Add db.table to database semantic conventions~~ Database semantic conventions: Add db.table, allow SQL keyword as db.operation Oct 30, 2020

Oberon00 reviewed Oct 30, 2020

View reviewed changes

Oberon00 requested changes Oct 30, 2020

View reviewed changes

justinfoote added 2 commits October 30, 2020 09:51

Move db.table to SQL-specific call level attributes

657670c

Clarify instructions for setting db.operation to a SQL keyword

ff2e13b

Oberon00 approved these changes Oct 30, 2020

View reviewed changes

arminru approved these changes Nov 3, 2020

View reviewed changes

semantic_conventions/trace/database.yaml Outdated Show resolved Hide resolved

mateuszrzeszutek mentioned this pull request Nov 3, 2020

SQL Normalizer should allow normalizing IN queries open-telemetry/opentelemetry-java-instrumentation#1409

Closed

Specify that db.sql.table MUST NOT be set when not applicable

eb4d3cd

Co-authored-by: Armin Ruech <[email protected]>

justinfoote force-pushed the db_table_semantics branch from 23a919e to eb4d3cd Compare November 3, 2020 16:04

Add db.table and db.operation to CHANGELOG

862361f

Merge branch 'master' into db_table_semantics

f504893

arminru merged commit 1954360 into open-telemetry:master Nov 9, 2020

justinfoote deleted the db_table_semantics branch November 9, 2020 21:33

arminru changed the title ~~Database semantic conventions: Add db.table, allow SQL keyword as db.operation~~ Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation Nov 10, 2020

arminru added a commit to dynatrace-oss-contrib/opentelemetry-specification that referenced this pull request Nov 10, 2020

Fix changelog entry for open-telemetry#1141

b5d71bc

arminru mentioned this pull request Nov 10, 2020

Fix changelog entry for #1141 #1220

Merged

arminru added a commit that referenced this pull request Nov 10, 2020

Fix changelog entry for #1141 (#1220)

2c92d8a

* Fix changelog entry for #1141 * Apply suggestions from code review Co-authored-by: Christian Neumüller <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation #1141

Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation #1141

justinfoote commented Oct 23, 2020

anuraaga left a comment

Oberon00 Oct 24, 2020

justinfoote Oct 25, 2020

Oberon00 Oct 30, 2020 •

edited

Loading

justinfoote commented Oct 29, 2020

Oberon00 Oct 30, 2020 •

edited

Loading

Oberon00 Oct 30, 2020

Oberon00 Oct 30, 2020

justinfoote Oct 30, 2020 •

edited

Loading

justinfoote Oct 30, 2020

Oberon00 left a comment

Oberon00 left a comment

justinfoote commented Nov 2, 2020

justinfoote commented Nov 6, 2020

arminru commented Nov 9, 2020

justinfoote commented Nov 9, 2020

jamal commented Nov 10, 2020

arminru commented Nov 10, 2020

jamal commented Nov 10, 2020

Oberon00 commented Nov 10, 2020 •

edited

Loading

jamal commented Nov 10, 2020

Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation #1141

Database semantic conventions: Add db.sql.table, allow SQL keyword as db.operation #1141

Conversation

justinfoote commented Oct 23, 2020

Changes

anuraaga left a comment

Choose a reason for hiding this comment

Oberon00 Oct 24, 2020

Choose a reason for hiding this comment

justinfoote Oct 25, 2020

Choose a reason for hiding this comment

Oberon00 Oct 30, 2020 • edited Loading

Choose a reason for hiding this comment

justinfoote commented Oct 29, 2020

Oberon00 Oct 30, 2020 • edited Loading

Choose a reason for hiding this comment

Oberon00 Oct 30, 2020

Choose a reason for hiding this comment

Oberon00 Oct 30, 2020

Choose a reason for hiding this comment

justinfoote Oct 30, 2020 • edited Loading

Choose a reason for hiding this comment

justinfoote Oct 30, 2020

Choose a reason for hiding this comment

Oberon00 left a comment

Choose a reason for hiding this comment

Oberon00 left a comment

Choose a reason for hiding this comment

justinfoote commented Nov 2, 2020

justinfoote commented Nov 6, 2020

arminru commented Nov 9, 2020

justinfoote commented Nov 9, 2020

jamal commented Nov 10, 2020

arminru commented Nov 10, 2020

jamal commented Nov 10, 2020

Oberon00 commented Nov 10, 2020 • edited Loading

jamal commented Nov 10, 2020

Oberon00 Oct 30, 2020 •

edited

Loading

Oberon00 Oct 30, 2020 •

edited

Loading

justinfoote Oct 30, 2020 •

edited

Loading

Oberon00 commented Nov 10, 2020 •

edited

Loading