Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: (qe) Telemetry capturing for PDP/Data proxy #3618

Merged
merged 12 commits into from
Jan 30, 2023

Conversation

miguelff
Copy link
Contributor

Extracted from #3505 and to be released after the preparatory refactoring in #3617

Tracked in https://github.com/prisma/client-planning/issues/158
Sibling to prisma/prisma#16748
Closes https://github.com/prisma/client-planning/issues/138
Closes https://github.com/prisma/client-planning/issues/144
Closes https://github.com/prisma/client-planning/issues/145

Implements traces and log capturing in concordance with the (private link) contract specification

The technical design is best explained throw the interaction diagram below:

diagram

In the diagram, you will see objects whose lifetime is static. The boxes for those have a double
width margin. These are:

  • The server itself
  • The global TRACER, which handles log! and span! and uses the global PROCESSOR to
    process the data constituting a trace Spans and log Events
  • The global PROCESSOR, which manages the Storage set of data structures, holding logs,
    traces (and capture settings) per request.

Then, through the request lifecycle, different objects are created and dropped:

  • When a request comes in, its headers are processed and a [Settings] object is built, this
    object determines, for the request, how logging and tracing are going to be captured: if only
    traces, logs, or both, and which log levels are going to be captured.
  • Based on the settings, a new Capturer is created; a capturer is nothing but an exporter
    wrapped to start capturing / fetch the captures for this particular request.

Then the capturing process works in this way:

  • The server receives a query [1]
  • It grabs the HTTP headers and builds a Capture object [2], which is configured with the settings
    denoted by the X-capture-telemetry
  • Now the server tells the Capturer to start capturing all the logs and traces occurring on
    the request [3] (denoted by a trace_id) The trace_id is either carried on the traceparent
    header or implicitly created on the first span of the request. To start capturing implies
    creating for the trace_id in two different data structures: logs and traces; and storing
    the settings for selecting the Spans and Event to capture [4].
  • The server dispatches the request and Somewhere else in the code, it is processed [5].
  • There the code logs events and emits traces asynchronously, as part of the processing [6]
  • Traces and Logs arrive at the TRACER, and get hydrated in the PROCESSOR [7],
    which writes them in the shard corresponding to the current trace_id, into the
    logs and traces storage [8]. The settings previously stored under the trace_id
    key, are used to pick which events and spans are going to be captured based on their level.
  • When the code that dispatches the request is done it returns a PrismaResponse to the
    server [9].
  • Then the server asks the PROCESSOR to fetch the captures [10]
  • And right after that, it fetches the captures from the Storage [11]. At that time, although
    that's not represented in the diagram, the captures are deleted from the storage, thus
    freeing any memory used for capturing during the request
  • Finally, the server sets the logs and traces extensions in the PrismaResponse[12],
    it serializes the extended response in json format and returns it as an HTTP Response
    blob [13].

@miguelff miguelff force-pushed the integration/log-capture-prep-refactoring branch from c144b46 to 4b6c971 Compare January 20, 2023 12:12
@miguelff miguelff force-pushed the integration/log-capture branch from 5fc71ed to 5b4baea Compare January 20, 2023 12:19
@miguelff miguelff marked this pull request as ready for review January 20, 2023 12:20
@miguelff miguelff requested a review from a team January 20, 2023 12:20
@miguelff miguelff added this to the 4.10.0 milestone Jan 20, 2023
@miguelff miguelff force-pushed the integration/log-capture branch from 5b4baea to 1da36a3 Compare January 22, 2023 21:36
@miguelff miguelff force-pushed the integration/log-capture-prep-refactoring branch from fffa84e to 6970951 Compare January 22, 2023 21:36
@miguelff miguelff force-pushed the integration/log-capture branch 3 times, most recently from 4351a6d to ff36a73 Compare January 23, 2023 18:48
@miguelff miguelff force-pushed the integration/log-capture-prep-refactoring branch from ca0e1b0 to d4977fb Compare January 24, 2023 11:07
@miguelff miguelff force-pushed the integration/log-capture branch from ff36a73 to 9a3c4f5 Compare January 24, 2023 11:12
@miguelff miguelff force-pushed the integration/log-capture-prep-refactoring branch from d4977fb to dbd7ac1 Compare January 24, 2023 12:02
@miguelff miguelff force-pushed the integration/log-capture branch from 9a3c4f5 to e8d49a5 Compare January 24, 2023 12:03
Base automatically changed from integration/log-capture-prep-refactoring to main January 24, 2023 14:22
@miguelff miguelff force-pushed the integration/log-capture branch from d393ef3 to 7f149a8 Compare January 24, 2023 14:24
Copy link
Contributor

@tomhoule tomhoule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With all the context and future plans we discussed, let's merge this.

query-engine/core/src/interactive_transactions/mod.rs Outdated Show resolved Hide resolved
query-engine/core/src/telemetry/capturing/settings.rs Outdated Show resolved Hide resolved
query-engine/core/src/telemetry/capturing/settings.rs Outdated Show resolved Hide resolved
query-engine/core/src/telemetry/capturing/capturer.rs Outdated Show resolved Hide resolved
@miguelff
Copy link
Contributor Author

80442d5 fixed the problem with BEGIN statements not being captured at the beginning of the request.

curl -X POST http://localhost:57581/transactions/start -H 'Content-Type: application/json' -H 'X-capture-telemetry: query,tracing' -d '{"max_wait":2000,"timeout":5000}'

{
  "id": "cldensyuh0002rb9nzo10w4iy",
  "extensions": {
    "logs": [
      {
        "span_id": "563b074839c5c57e",
        "level": "query",
        "timestamp": [
          1674832125,
          565891000
        ],
        "attributes": {
          "params": "[]",
          "query": "SELECT 1",
          "duration_ms": "12"
        }
      },
      {
        "span_id": "3b3817ccec15d3e9",
        "level": "query",
        "timestamp": [
          1674832125,
          572383000
        ],
        "attributes": {
          "query": "BEGIN",
          "duration_ms": "4",
          "params": "[]"
        }
      }
    ],
    "traces": [
      {
        "trace_id": "73797568303030327a6f313077346979",
        "span_id": "563b074839c5c57e",
        "parent_span_id": "fc6e12b498c57dce",
        "name": "prisma:engine:db_query",
        "start_time": [
          1674832125,
          553089000
        ],
        "end_time": [
          1674832125,
          566096000
        ],
        "attributes": {
          "db.statement": "SELECT 1"
        }
      },
      {
        "trace_id": "73797568303030327a6f313077346979",
        "span_id": "fc6e12b498c57dce",
        "parent_span_id": "6925d3c29b00efb1",
        "name": "prisma:engine:connection",
        "start_time": [
          1674832125,
          552560000
        ],
        "end_time": [
          1674832125,
          567526000
        ],
        "attributes": {
          "db.type": "postgres"
        }
      },
      {
        "trace_id": "73797568303030327a6f313077346979",
        "span_id": "3b3817ccec15d3e9",
        "parent_span_id": "6925d3c29b00efb1",
        "name": "prisma:engine:db_query",
        "start_time": [
          1674832125,
          567792000
        ],
        "end_time": [
          1674832125,
          572568000
        ],
        "attributes": {
          "db.statement": "BEGIN"
        }
      }
    ]
  }
}

@miguelff miguelff force-pushed the integration/log-capture branch from 8b17658 to 3aa0d7b Compare January 30, 2023 11:06
@miguelff miguelff merged commit dcb909a into main Jan 30, 2023
@miguelff miguelff deleted the integration/log-capture branch January 30, 2023 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants