Skip to content

Commit

Permalink
feat: mysql support (#525)
Browse files Browse the repository at this point in the history
* chore: set up mysql plugin

* feat: wip mysql plugin

* fix: lint

* feat(mysql-plugin): trace cluster pool queries

* chore: update attributes to match spec names

* feat(mysql): add query overloads

* test(mysql): add tests

* test(mysql): run tests in ci

* test(mysql): use environment for test params

* test(mysql): fix environment

* chore(mysql): set supported versions

* test(mysql): fix version number

* ci: fix build and add mysql to cache

* chore: pin @types/node to fix compile error

* ci: add mysql port to env

* test(mysql): increase coverage

* chore: add mysql to supported plugins list

* test: fix typo

* fix: reference to this module

rename plugin to thisPlugin to avoid confusion

* fix: lint

* test: add pool.getConnection tests

* Update examples/mysql/package.json

Co-Authored-By: Mayur Kale <[email protected]>

* docs: update mysql example

* chore: add redis to default plugins list

* chore: remove wip mysql plugin from defaults

* chore: add codecov script

* chore(mysql): add supported version to readme

* fix: lint

* chore: remove ts-ignore, set status ok

* chore: unwrap connections on next call after unwrap

* chore: updates from review comments

* ci: order envs

* chore: add mysql to default plugins

* chore: update README plugin list
  • Loading branch information
dyladan authored and mayurkale22 committed Nov 27, 2019
1 parent f7185ab commit 5a52eda
Show file tree
Hide file tree
Showing 23 changed files with 1,832 additions and 6 deletions.
19 changes: 18 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ version: 2

node_test_env: &node_test_env
RUN_POSTGRES_TESTS: 1
RUN_MYSQL_TESTS: 1
RUN_MONGODB_TESTS: 1
RUN_REDIS_TESTS: 1
POSTGRES_USER: postgres
Expand All @@ -11,8 +12,12 @@ node_test_env: &node_test_env
OPENTELEMETRY_REDIS_HOST: 'localhost'
OPENTELEMETRY_REDIS_PORT: 6379
MONGODB_HOST: localhost
MONGODB_PORT: 27017
MONGODB_DB: opentelemetry-tests
MONGODB_PORT: 27017
MYSQL_USER: otel
MYSQL_PASSWORD: secret
MYSQL_DATABASE: circle_database
MYSQL_PORT: 3306

postgres_service: &postgres_service
image: circleci/postgres:9.6-alpine
Expand All @@ -25,6 +30,14 @@ redis_service: &redis_service
mongo_service: &mongo_service
image: mongo

mysql_service: &mysql_service
image: circleci/mysql:5.7
environment:
MYSQL_USER: otel
MYSQL_PASSWORD: secret
MYSQL_DATABASE: circle_database
MYSQL_ROOT_PASSWORD: rootpw

cache_1: &cache_1
key: npm-cache-01-{{ .Environment.CIRCLE_JOB }}-{{ checksum "/tmp/checksums.txt" }}-1
paths:
Expand Down Expand Up @@ -57,6 +70,7 @@ cache_2: &cache_2
- packages/opentelemetry-plugin-document-load/node_modules
- packages/opentelemetry-plugin-https/node_modules
- packages/opentelemetry-plugin-postgres/opentelemetry-plugin-pg-pool/node_modules
- packages/opentelemetry-plugin-mysql/node_modules
- packages/opentelemetry-exporter-prometheus/node_modules

node_unit_tests: &node_unit_tests
Expand Down Expand Up @@ -157,6 +171,7 @@ jobs:
- image: node:8
environment: *node_test_env
- *postgres_service
- *mysql_service
- *redis_service
- *mongo_service
<<: *node_unit_tests
Expand All @@ -165,6 +180,7 @@ jobs:
- image: node:10
environment: *node_test_env
- *postgres_service
- *mysql_service
- *redis_service
- *mongo_service
<<: *node_unit_tests
Expand All @@ -173,6 +189,7 @@ jobs:
- image: node:12
environment: *node_test_env
- *postgres_service
- *mysql_service
- *redis_service
- *mongo_service
<<: *node_unit_tests
Expand Down
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,16 +109,18 @@ OpenTelemetry is vendor-agnostic and can upload data to any backend with various
OpenTelemetry can collect tracing data automatically using plugins. Vendors/Users can also create and use their own. Currently, OpenTelemetry supports automatic tracing for:

#### Node Plugins
- [@opentelemetry/plugin-http](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-http)
- [@opentelemetry/plugin-dns](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-dns)
- [@opentelemetry/plugin-grpc](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-grpc)
- [@opentelemetry/plugin-http](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-http)
- [@opentelemetry/plugin-https](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-https)
- [@opentelemetry/plugin-dns](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-dns)
- [@opentelemetry/plugin-mongodb-core](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-mongodb-core)
- [@opentelemetry/plugin-mysql](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-mysql)
- [@opentelemetry/plugin-pg](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-postgres/opentelemetry-plugin-pg)
- [@opentelemetry/plugin-redis](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-redis)
- [@opentelemetry/plugin-pg-pool](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-postgres/opentelemetry-plugin-pg-pool) - WIP
- [@opentelemetry/plugin-ioredis](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-ioredis) - WIP
- [@opentelemetry/plugin-mysql](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-mysql) - WIP

##### WIP Plugins
- [@opentelemetry/plugin-pg-pool](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-postgres/opentelemetry-plugin-pg-pool)
- [@opentelemetry/plugin-ioredis](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-ioredis)

#### Web Plugins
- [@opentelemetry/plugin-document-load](https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-plugin-document-load)
Expand Down
74 changes: 74 additions & 0 deletions examples/mysql/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Overview

OpenTelemetry MySQL Instrumentation allows the user to automatically collect trace data and export them to the backend of choice (we can use Zipkin or Jaeger for this example), to give observability to distributed systems.

This is a modification of the HTTP example that executes multiple parallel requests that interact with a MySQL server backend using the `mysql` npm module. The example displays traces using multiple connection methods.
- Direct Connection Query
- Pool Connection Query
- Cluster Pool Connection Query


## Installation

```sh
$ # from this directory
$ npm install
```

Setup [Zipkin Tracing](https://zipkin.io/pages/quickstart.html)
or
Setup [Jaeger Tracing](https://www.jaegertracing.io/docs/latest/getting-started/#all-in-one)

## Run the Application

### Zipkin

- Run the server

```sh
$ # from this directory
$ npm run server
```

- Run the client

```sh
$ # from this directory
$ npm run client
```

#### Zipkin UI
`server` script should output the `traceid` in the terminal (e.g `traceid: 4815c3d576d930189725f1f1d1bdfcc6`).
Go to Zipkin with your browser [http://localhost:9411/zipkin/traces/(your-trace-id)]() (e.g http://localhost:9411/zipkin/traces/4815c3d576d930189725f1f1d1bdfcc6)

<p align="center"><img src="./images/zipkin-ui.png?raw=true"/></p>

### Jaeger

- Run the server

```sh
$ # from this directory
$ npm run server
```

- Run the client

```sh
$ # from this directory
$ npm run client
```
#### Jaeger UI

`server` script should output the `traceid` in the terminal (e.g `traceid: 4815c3d576d930189725f1f1d1bdfcc6`).
Go to Jaeger with your browser [http://localhost:16686/trace/(your-trace-id)]() (e.g http://localhost:16686/trace/4815c3d576d930189725f1f1d1bdfcc6)

<p align="center"><img src="images/jaeger-ui.png?raw=true"/></p>

## Useful links
- For more information on OpenTelemetry, visit: <https://opentelemetry.io/>
- For more information on OpenTelemetry for Node.js, visit: <https://github.com/open-telemetry/opentelemetry-js/tree/master/packages/opentelemetry-node>

## LICENSE

Apache License 2.0
81 changes: 81 additions & 0 deletions examples/mysql/client.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
'use strict';

const opentelemetry = require('@opentelemetry/core');
const config = require('./setup');

/**
* The trace instance needs to be initialized first, if you want to enable
* automatic tracing for built-in plugins (HTTP and MySQL in this case).
*/
config.setupTracerAndExporters('http-client-service');

const http = require('http');
const tracer = opentelemetry.getTracer();

/** A function which makes requests and handles response. */
function makeRequest() {
// span corresponds to outgoing requests. Here, we have manually created
// the span, which is created to track work that happens outside of the
// request lifecycle entirely.
const span = tracer.startSpan('makeRequest');

let queries = 0
let responses = 0;

tracer.withSpan(span, () => {
queries += 1;
http.get({
host: 'localhost',
port: 8080,
path: '/connection/query'
}, (response) => {
let body = [];
response.on('data', chunk => body.push(chunk));
response.on('end', () => {
responses += 1;
console.log(body.toString());
if (responses === queries) span.end();
});
});
});
tracer.withSpan(span, () => {
queries += 1;
http.get({
host: 'localhost',
port: 8080,
path: '/pool/query'
}, (response) => {
let body = [];
response.on('data', chunk => body.push(chunk));
response.on('end', () => {
responses += 1;
console.log(body.toString());
if (responses === queries) span.end();
});
});
});
tracer.withSpan(span, () => {
queries += 1;
http.get({
host: 'localhost',
port: 8080,
path: '/cluster/query'
}, (response) => {
let body = [];
response.on('data', chunk => body.push(chunk));
response.on('end', () => {
responses += 1;
console.log(body.toString());
if (responses === queries) span.end();
});
});
});

// The process must live for at least the interval past any traces that
// must be exported, or some risk being lost if they are recorded after the
// last export.
console.log('Sleeping 5 seconds before shutdown to ensure all records are flushed.')
setTimeout(() => { console.log('Completed.'); }, 5000);
}

makeRequest();
Binary file added examples/mysql/images/jaeger-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added examples/mysql/images/zipkin-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 45 additions & 0 deletions examples/mysql/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"name": "mysql-example",
"private": true,
"version": "0.2.0",
"description": "Example of mysql integration with OpenTelemetry",
"main": "index.js",
"scripts": {
"zipkin:server": "cross-env EXPORTER=zipkin node ./server.js",
"zipkin:client": "cross-env EXPORTER=zipkin node ./client.js",
"jaeger:server": "cross-env EXPORTER=jaeger node ./server.js",
"jaeger:client": "cross-env EXPORTER=jaeger node ./client.js"
},
"repository": {
"type": "git",
"url": "git+ssh://[email protected]/open-telemetry/opentelemetry-js.git"
},
"keywords": [
"opentelemetry",
"mysql",
"tracing"
],
"engines": {
"node": ">=8"
},
"author": "OpenTelemetry Authors",
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/open-telemetry/opentelemetry-js/issues"
},
"dependencies": {
"@opentelemetry/core": "^0.2.0",
"@opentelemetry/exporter-jaeger": "^0.2.0",
"@opentelemetry/exporter-zipkin": "^0.2.0",
"@opentelemetry/node": "^0.2.0",
"@opentelemetry/plugin-http": "^0.2.0",
"@opentelemetry/plugin-mysql": "^0.2.0",
"@opentelemetry/tracing": "^0.2.0",
"mysql": "*"
},
"homepage": "https://github.com/open-telemetry/opentelemetry-js#readme",
"devDependencies": {
"@types/mysql": "*",
"cross-env": "^6.0.0"
}
}
Loading

0 comments on commit 5a52eda

Please sign in to comment.