From c53e44dbcb56b9337505e9fc5ab5e67271db1c8f Mon Sep 17 00:00:00 2001 From: Francis Bogsanyi Date: Thu, 5 Mar 2020 12:15:25 -0500 Subject: [PATCH] Add Rack instrumentation (#166) * rack: Add Gemfile, gemspec, version * rack-test: "simple testing API for Rack apps" * fix: "Could not find rake-12.3.3 in any of the sources" * Add TracerMiddleware * use dup._call env pattern * RE: thread safety: "... easiest and most efficient way to do this is to dup the middleware object that is created in runtime." * do we need 'quantization' now? * Add Adapters::Rack::Adapter * retain middleware names (feature exists in dd-trace-rb) * tests: Add test_helper * Add example: trace_demonstration e.g., $ docker-compose run --rm ex-adapter-rack bash-5.0$ ruby trace_demonstration.rb * Add Rakefile e.g., $ bundle exec rake test * Add QueueTime * from dd-trace-rb * translate spec to minitest * Add Adapters::Rack * Update to 2020 copyright * Initialize 'now' later * Adapt to Instrumentation Auto Installer interface * PR #164 * Fix example to use updated Instrumentation Auto Installer * verified by running via, e.g., $ docker-compose run --rm ex-adapter-rack bash-5.0$ bundle bash-5.0$ ruby trace_demonstration.rb Expected: console output of span data * Handle errors by setting span.status, leave a TODO and rescue clause * Allow config[:quantization] * to allow for lower cardinality span names * Remove optional parent context extraction * defeats purpose of opentelemetry, which *is* distributed tracing * Resolve 'http.base_url' TODO * add 'http.target' * Resolve 'resource name' TODO * it seems that dd-trace-rb's 'span.resource' is the same as 'span.name', at least in this case * Resolve 'http.route' TODO * in sinatra, data is available via env['sinatra.route'].split.last, but otherwise, doesn't seem to be readily available * Note: missing 'span.set_error()' for now * Resolve FrontendSpan TODOs * resolve 'http_server.queue' TODO * span kind of 'proxy' is not defined, yet * Optimize allowed_request_headers * reduce string allocations * TIL: a nested 'before' block doesn't run before *each* test, but the top/first 'before' block does. (weird) * Optimize allowed_response_headers() * prevent unneeded string and object allocation * once a header key is found, add it to the list of response headers to search for * Optimize return of EMPTY_HASH frozen constant * Refactor to avoid using dup._call(env) * avoid using instance variables * Add Appraisals, integrate into circleci * Integrate rubocop, fix violations, add adapters to top-level rake task * per work in #179 * Update example to use new config * per #171, #177 * Rewrite examples * one that is simple, no config options * demonstrate integration via 'Rack::Builder#use' * this is how ddtrace is currently working * demonstrate integration using config[:application] * ultimate goal: 0 config (automagic integration) * Automatically patch Rack::Builder * in tests, keep an unpatched ('untainted') version of Rack::Builder, restore before and after each test * fix: ruby2.4: private method `define_method' called * Port ddtrace Quantization::HTTP * Integrate Util::Quantization * Revert "Automatically patch Rack::Builder" This reverts commit de910256a88ed1d52c6dd430fac3807cfe7be05d. # Conflicts: # adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb * Add missing files needed for Bundler.require * Update Rakefile * Avoid patching config[:application] during installation * config[:application] is patched in retain_middleware_names, in which case it is required if config[:retain_middleware_names] is truthy * goal: leave .use call to the user * Refine/optimize allowed_request_headers * Refine/optimize allowed_response_headers * Avoid circular require * Use SDK.configure to streamline test_helper.rb setup * Revert "Integrate Util::Quantization" This reverts commit 99f44e847beac5c8ed15b0f28e926def81029546. Discussed in SIG: * avoid eager-quantization (heavy, potentially unwanted) * defer to user preferences (via config) * probably better to err on the side of 'too-specific' vs. 'too-general', since 'too-specific' can be made more general, but maybe not vice-versa # Conflicts: # adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb * Revert "Port ddtrace Quantization::HTTP" This reverts commit e9c021b45c3bfba93accb37e5845acdb78c2a065. * cleanup remnants that aren't used for now * Fix example/trace_demonstration2.rb to integrate explicitly, with 'use' * Update example/trace_demonstration2.rb documentation * Optimize allowed_response_headers to avoid using Hash#detect * Simplify allowed_{rack,request}_header_names to inline config * Optimize to return EMPTY_HASH if allowed_{response,rack_request}_headers.empty? * Adjust to context prop changes * Remove unused variables * Use kind: :server for both frontend and request span * Make request_span parented by frontend_span * explicitly manage frontend_span parent context, and prevent automatic span activation * manage frontend_span life-cycle explicitly via a new context, using it as the request_span's parent, if it's available * Implement using helpers to that in_span doesn't have to record and re-raise error * Cleanup some URL wrapper methods * goal: eliminate need for Rack::Request allocation * Optimize: return without assigning local variable * Just use http.{scheme,host,target} (remove url, base_url) * Inline Rack::Request#fullpath * Fix .circleci/config.yml after conflict * Adjust error handling according to #184 * Rewrite to utilize in_span * note that order of finished spans is now swapped * Reduce comments that were more useful in development/review * Update http.host to use HTTP_HOST or 'unknown' Co-Authored-By: Matthew Wear * Update request_start_time to be number, not timestamp Co-Authored-By: Matthew Wear * Remove request_span comment * Remove 'service' attribute when creating frontend span * value is potentially nil, which is not a valid attribute value * also 'service' is not an official semantic convention and will probably come from the application resource in the future * Change frontend_span to 'http_server.proxy', make request_span :internal * request_span.kind is :internal if frontend_span is present * future: change request_span :kind to ':proxy' if/when it gets added to spec Co-authored-by: Matthew Wear --- .circleci/config.yml | 9 + Rakefile | 6 + adapters/rack/.rubocop.yml | 27 +++ adapters/rack/Appraisals | 13 ++ adapters/rack/Gemfile | 15 ++ adapters/rack/Rakefile | 28 +++ adapters/rack/example/Gemfile | 9 + adapters/rack/example/trace_demonstration.rb | 27 +++ adapters/rack/example/trace_demonstration2.rb | 28 +++ adapters/rack/gemfiles/rack_2.0.gemfile | 12 ++ adapters/rack/gemfiles/rack_2.1.gemfile | 12 ++ .../rack/lib/opentelemetry-adapters-rack.rb | 7 + adapters/rack/lib/opentelemetry/adapters.rb | 13 ++ .../lib/opentelemetry/adapters/adapters.rb | 16 ++ .../rack/lib/opentelemetry/adapters/rack.rb | 18 ++ .../opentelemetry/adapters/rack/adapter.rb | 57 ++++++ .../rack/middlewares/tracer_middleware.rb | 191 ++++++++++++++++++ .../adapters/rack/util/queue_time.rb | 48 +++++ .../opentelemetry/adapters/rack/version.rb | 13 ++ .../rack/opentelemetry-adapters-rack.gemspec | 42 ++++ adapters/rack/test/.rubocop.yml | 4 + .../adapters/rack/adapter_test.rb | 76 +++++++ .../middlewares/tracer_middleware_test.rb | 191 ++++++++++++++++++ .../adapters/rack/util/queue_time_test.rb | 90 +++++++++ .../test/opentelemetry/adapters/rack_test.rb | 28 +++ adapters/rack/test/test_helper.rb | 20 ++ docker-compose.yml | 4 + 27 files changed, 1004 insertions(+) create mode 100644 adapters/rack/.rubocop.yml create mode 100644 adapters/rack/Appraisals create mode 100644 adapters/rack/Gemfile create mode 100644 adapters/rack/Rakefile create mode 100644 adapters/rack/example/Gemfile create mode 100644 adapters/rack/example/trace_demonstration.rb create mode 100644 adapters/rack/example/trace_demonstration2.rb create mode 100644 adapters/rack/gemfiles/rack_2.0.gemfile create mode 100644 adapters/rack/gemfiles/rack_2.1.gemfile create mode 100644 adapters/rack/lib/opentelemetry-adapters-rack.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/adapters.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/rack.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/rack/middlewares/tracer_middleware.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/rack/util/queue_time.rb create mode 100644 adapters/rack/lib/opentelemetry/adapters/rack/version.rb create mode 100644 adapters/rack/opentelemetry-adapters-rack.gemspec create mode 100644 adapters/rack/test/.rubocop.yml create mode 100644 adapters/rack/test/opentelemetry/adapters/rack/adapter_test.rb create mode 100644 adapters/rack/test/opentelemetry/adapters/rack/middlewares/tracer_middleware_test.rb create mode 100644 adapters/rack/test/opentelemetry/adapters/rack/util/queue_time_test.rb create mode 100644 adapters/rack/test/opentelemetry/adapters/rack_test.rb create mode 100644 adapters/rack/test/test_helper.rb diff --git a/.circleci/config.yml b/.circleci/config.yml index 0f643a82f..d1f24c9bd 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -76,6 +76,15 @@ commands: gem install --no-document bundler -v '~> 2.0.2' bundle install --jobs=3 --retry=3 bundle exec rake test + - run: + name: Bundle + CI (Adapters - Rack) + command: | + cd adapters/rack + gem uninstall -aIx bundler + gem install --no-document bundler -v '~> 2.0.2' + bundle install --jobs=3 --retry=3 + bundle exec appraisal install + bundle exec appraisal rake test - run: name: Bundle + CI (Adapters - Redis) command: | diff --git a/Rakefile b/Rakefile index a3f524caf..463a37025 100644 --- a/Rakefile +++ b/Rakefile @@ -75,6 +75,12 @@ GEM_INFO = { OpenTelemetry::Adapters::Net::HTTP::VERSION } }, + "opentelemetry-adapters-rack" => { + version_getter: ->() { + require './lib/opentelemetry/adapters/rack/version.rb' + OpenTelemetry::Adapters::Rack::VERSION + } + }, "opentelemetry-adapters-redis" => { version_getter: ->() { require './lib/opentelemetry/adapters/redis/version.rb' diff --git a/adapters/rack/.rubocop.yml b/adapters/rack/.rubocop.yml new file mode 100644 index 000000000..7284b77cd --- /dev/null +++ b/adapters/rack/.rubocop.yml @@ -0,0 +1,27 @@ +AllCops: + TargetRubyVersion: '2.4.0' + +Bundler/OrderedGems: + Exclude: + - gemfiles/**/* +Lint/UnusedMethodArgument: + Enabled: false +Metrics/AbcSize: + Max: 18 +Metrics/LineLength: + Enabled: false +Metrics/MethodLength: + Max: 20 +Metrics/ParameterLists: + Enabled: false +Naming/FileName: + Exclude: + - "lib/opentelemetry-adapters-rack.rb" +Style/FrozenStringLiteralComment: + Exclude: + - gemfiles/**/* +Style/ModuleFunction: + Enabled: false +Style/StringLiterals: + Exclude: + - gemfiles/**/* diff --git a/adapters/rack/Appraisals b/adapters/rack/Appraisals new file mode 100644 index 000000000..afa4b7cab --- /dev/null +++ b/adapters/rack/Appraisals @@ -0,0 +1,13 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +appraise 'rack-2.1' do + gem 'rack', '~> 2.1.2' +end + +appraise 'rack-2.0' do + gem 'rack', '2.0.8' +end diff --git a/adapters/rack/Gemfile b/adapters/rack/Gemfile new file mode 100644 index 000000000..8af0cd4fc --- /dev/null +++ b/adapters/rack/Gemfile @@ -0,0 +1,15 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +source 'https://rubygems.org' + +gemspec + +gem 'opentelemetry-api', path: '../../api' + +group :test do + gem 'opentelemetry-sdk', path: '../../sdk' +end diff --git a/adapters/rack/Rakefile b/adapters/rack/Rakefile new file mode 100644 index 000000000..197514f1d --- /dev/null +++ b/adapters/rack/Rakefile @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'bundler/gem_tasks' +require 'rake/testtask' +require 'yard' +require 'rubocop/rake_task' + +RuboCop::RakeTask.new + +Rake::TestTask.new :test do |t| + t.libs << 'test' + t.libs << 'lib' + t.test_files = FileList['test/**/*_test.rb'] +end + +YARD::Rake::YardocTask.new do |t| + t.stats_options = ['--list-undoc'] +end + +if RUBY_ENGINE == 'truffleruby' + task default: %i[test] +else + task default: %i[test rubocop yard] +end diff --git a/adapters/rack/example/Gemfile b/adapters/rack/example/Gemfile new file mode 100644 index 000000000..251efbc10 --- /dev/null +++ b/adapters/rack/example/Gemfile @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +source 'https://rubygems.org' + +gem 'opentelemetry-adapters-rack', path: '../../../adapters/rack' +gem 'opentelemetry-api', path: '../../../api' +gem 'opentelemetry-sdk', path: '../../../sdk' +gem 'rack' +gem 'rack-test' diff --git a/adapters/rack/example/trace_demonstration.rb b/adapters/rack/example/trace_demonstration.rb new file mode 100644 index 000000000..30fcb1872 --- /dev/null +++ b/adapters/rack/example/trace_demonstration.rb @@ -0,0 +1,27 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'rubygems' +require 'bundler/setup' + +Bundler.require + +OpenTelemetry::SDK.configure do |c| + c.use 'OpenTelemetry::Adapters::Rack' +end + +# setup fake rack application: +builder = Rack::Builder.app do + # integration should be automatic in web frameworks (like rails), + # but for a plain Rack application, enable it in your config.ru, e.g., + use OpenTelemetry::Adapters::Rack::Middlewares::TracerMiddleware + + app = ->(_env) { [200, { 'Content-Type' => 'text/plain' }, ['All responses are OK']] } + run app +end + +# demonstrate tracing (span output to console): +puts Rack::MockRequest.new(builder).get('/') diff --git a/adapters/rack/example/trace_demonstration2.rb b/adapters/rack/example/trace_demonstration2.rb new file mode 100644 index 000000000..8b09ceac7 --- /dev/null +++ b/adapters/rack/example/trace_demonstration2.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'rubygems' +require 'bundler/setup' + +Bundler.require + +# setup fake rack application: +builder = Rack::Builder.new +app = ->(_env) { [200, { 'Content-Type' => 'text/plain' }, ['All responses are OK']] } +builder.run app + +# demonstrate integration using 'retain_middlware_names' and 'application': +OpenTelemetry::SDK.configure do |c| + c.use 'OpenTelemetry::Adapters::Rack', retain_middleware_names: true, + application: builder, + record_frontend_span: true +end + +# integrate instrumentation explicitly: +builder.use OpenTelemetry::Adapters::Rack::Middlewares::TracerMiddleware + +# demonstrate tracing (span output to console): +puts Rack::MockRequest.new(builder).get('/') diff --git a/adapters/rack/gemfiles/rack_2.0.gemfile b/adapters/rack/gemfiles/rack_2.0.gemfile new file mode 100644 index 000000000..19d162252 --- /dev/null +++ b/adapters/rack/gemfiles/rack_2.0.gemfile @@ -0,0 +1,12 @@ +# This file was generated by Appraisal + +source "https://rubygems.org" + +gem "opentelemetry-api", path: "../../../api" +gem "rack", "2.0.8" + +group :test do + gem "opentelemetry-sdk", path: "../../../sdk" +end + +gemspec path: "../" diff --git a/adapters/rack/gemfiles/rack_2.1.gemfile b/adapters/rack/gemfiles/rack_2.1.gemfile new file mode 100644 index 000000000..4a1ed5a7f --- /dev/null +++ b/adapters/rack/gemfiles/rack_2.1.gemfile @@ -0,0 +1,12 @@ +# This file was generated by Appraisal + +source "https://rubygems.org" + +gem "opentelemetry-api", path: "../../../api" +gem "rack", "~> 2.1.2" + +group :test do + gem "opentelemetry-sdk", path: "../../../sdk" +end + +gemspec path: "../" diff --git a/adapters/rack/lib/opentelemetry-adapters-rack.rb b/adapters/rack/lib/opentelemetry-adapters-rack.rb new file mode 100644 index 000000000..a2f3a3444 --- /dev/null +++ b/adapters/rack/lib/opentelemetry-adapters-rack.rb @@ -0,0 +1,7 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require_relative './opentelemetry/adapters' diff --git a/adapters/rack/lib/opentelemetry/adapters.rb b/adapters/rack/lib/opentelemetry/adapters.rb new file mode 100644 index 000000000..aa64071a4 --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters.rb @@ -0,0 +1,13 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +module OpenTelemetry + # Adapters should be able to handle the case when the library is not installed on a user's system. + module Adapters + end +end + +require_relative './adapters/rack' diff --git a/adapters/rack/lib/opentelemetry/adapters/adapters.rb b/adapters/rack/lib/opentelemetry/adapters/adapters.rb new file mode 100644 index 000000000..c9446df58 --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/adapters.rb @@ -0,0 +1,16 @@ +# frozen_string_literal: true + +# Copyright 2019 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +module OpenTelemetry + # "Instrumentation adapters" are specified by + # https://github.com/open-telemetry/opentelemetry-specification/blob/57714f7547fe4dcb342ad0ad10a80d86118431c7/specification/overview.md#instrumentation-adapters + # + # Adapters should be able to handle the case when the library is not installed on a user's system. + module Adapters + end +end + +require_relative './adapters/rack' diff --git a/adapters/rack/lib/opentelemetry/adapters/rack.rb b/adapters/rack/lib/opentelemetry/adapters/rack.rb new file mode 100644 index 000000000..9ef82145a --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/rack.rb @@ -0,0 +1,18 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'opentelemetry' + +module OpenTelemetry + module Adapters + # Contains the OpenTelemetry adapter for the Rack gem + module Rack + end + end +end + +require_relative './rack/adapter' +require_relative './rack/version' diff --git a/adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb b/adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb new file mode 100644 index 000000000..3e6753e96 --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/rack/adapter.rb @@ -0,0 +1,57 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'opentelemetry' + +module OpenTelemetry + module Adapters + module Rack + # The Adapter class contains logic to detect and install the Rack + # instrumentation adapter + class Adapter < OpenTelemetry::Instrumentation::Adapter + install do |config| + require_dependencies + + retain_middleware_names if config[:retain_middleware_names] + end + + present do + defined?(::Rack) + end + + private + + def require_dependencies + require_relative 'middlewares/tracer_middleware' + end + + MissingApplicationError = Class.new(StandardError) + + # intercept all middleware-compatible calls, retain class name + def retain_middleware_names + next_middleware = config[:application] + raise MissingApplicationError unless next_middleware + + while next_middleware + if next_middleware.respond_to?(:call) + next_middleware.singleton_class.class_eval do + alias_method :__call, :call + + def call(env) + env['RESPONSE_MIDDLEWARE'] = self.class.to_s + __call(env) + end + end + end + + next_middleware = next_middleware.instance_variable_defined?('@app') && + next_middleware.instance_variable_get('@app') + end + end + end + end + end +end diff --git a/adapters/rack/lib/opentelemetry/adapters/rack/middlewares/tracer_middleware.rb b/adapters/rack/lib/opentelemetry/adapters/rack/middlewares/tracer_middleware.rb new file mode 100644 index 000000000..b3a4f71fe --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/rack/middlewares/tracer_middleware.rb @@ -0,0 +1,191 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'opentelemetry/trace/status' + +require_relative '../util/queue_time' + +module OpenTelemetry + module Adapters + module Rack + module Middlewares + # TracerMiddleware propagates context and instruments Rack requests + # by way of its middleware system + class TracerMiddleware # rubocop:disable Metrics/ClassLength + class << self + def allowed_rack_request_headers + @allowed_rack_request_headers ||= Array(config[:allowed_request_headers]).each_with_object({}) do |header, memo| + memo["HTTP_#{header.to_s.upcase.gsub(/[-\s]/, '_')}"] = build_attribute_name('http.request.headers.', header) + end + end + + def allowed_response_headers + @allowed_response_headers ||= Array(config[:allowed_response_headers]).each_with_object({}) do |header, memo| + memo[header] = build_attribute_name('http.response.headers.', header) + memo[header.to_s.upcase] = build_attribute_name('http.response.headers.', header) + end + end + + def build_attribute_name(prefix, suffix) + prefix + suffix.to_s.downcase.gsub(/[-\s]/, '_') + end + + def config + Rack::Adapter.instance.config + end + + private + + def clear_cached_config + @allowed_rack_request_headers = nil + @allowed_response_headers = nil + end + end + + EMPTY_HASH = {}.freeze + + def initialize(app) + @app = app + end + + def call(env) + original_env = env.dup + extracted_context = OpenTelemetry.propagation.extract(env) + frontend_context = create_frontend_span(env, extracted_context) + + # restore extracted context in this process: + OpenTelemetry::Context.with_current(frontend_context || extracted_context) do + request_span_name = create_request_span_name(env['REQUEST_URI'] || original_env['PATH_INFO']) + request_span_kind = frontend_context.nil? ? :server : :internal + tracer.in_span(request_span_name, + attributes: request_span_attributes(env: env), + kind: request_span_kind) do |request_span| + @app.call(env).tap do |status, headers, response| + set_attributes_after_request(request_span, status, headers, response) + end + end + end + ensure + finish_span(frontend_context) + end + + private + + # return Context with the frontend span as the current span + def create_frontend_span(env, extracted_context) + request_start_time = OpenTelemetry::Adapters::Rack::Util::QueueTime.get_request_start(env) + + return unless config[:record_frontend_span] && !request_start_time.nil? + + span = tracer.start_span('http_server.proxy', + with_parent_context: extracted_context, + attributes: { + 'component' => 'http', + 'start_time' => request_start_time.to_f + }, + kind: :server) + + extracted_context.set_value(current_span_key, span) + end + + def finish_span(context) + context[current_span_key]&.finish if context + end + + def current_span_key + OpenTelemetry::Trace::Propagation::ContextKeys.current_span_key + end + + def tracer + OpenTelemetry::Adapters::Rack::Adapter.instance.tracer + end + + def request_span_attributes(env:) + { + 'component' => 'http', + 'http.method' => env['REQUEST_METHOD'], + 'http.host' => env['HTTP_HOST'] || 'unknown', + 'http.scheme' => env['rack.url_scheme'], + 'http.target' => fullpath(env) + }.merge(allowed_request_headers(env)) + end + + # e.g., "/webshop/articles/4?s=1": + def fullpath(env) + query_string = env['QUERY_STRING'] + path = env['SCRIPT_NAME'] + env['PATH_INFO'] + + query_string.empty? ? path : "#{path}?#{query_string}" + end + + # https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/data-http.md#name + # + # recommendation: span.name(s) should be low-cardinality (e.g., + # strip off query param value, keep param name) + # + # see http://github.com/open-telemetry/opentelemetry-specification/pull/416/files + def create_request_span_name(request_uri_or_path_info) + # NOTE: dd-trace-rb has implemented 'quantization' (which lowers url cardinality) + # see Datadog::Quantization::HTTP.url + + if (implementation = config[:url_quantization]) + implementation.call(request_uri_or_path_info) + else + request_uri_or_path_info + end + end + + def set_attributes_after_request(span, status, headers, _response) + span.status = OpenTelemetry::Trace::Status.http_to_status(status) + span.set_attribute('http.status_code', status) + + # NOTE: if data is available, it would be good to do this: + # set_attribute('http.route', ... + # e.g., "/users/:userID? + span.set_attribute('http.status_text', ::Rack::Utils::HTTP_STATUS_CODES[status]) + + allowed_response_headers(headers).each { |k, v| span.set_attribute(k, v) } + end + + def allowed_request_headers(env) + return EMPTY_HASH if self.class.allowed_rack_request_headers.empty? + + {}.tap do |result| + self.class.allowed_rack_request_headers.each do |key, value| + result[value] = env[key] if env.key?(key) + end + end + end + + def allowed_response_headers(headers) + return EMPTY_HASH if headers.nil? + return EMPTY_HASH if self.class.allowed_response_headers.empty? + + {}.tap do |result| + self.class.allowed_response_headers.each do |key, value| + if headers.key?(key) + result[value] = headers[key] + else + # do case-insensitive match: + headers.each do |k, v| + if k.upcase == key + result[value] = v + break + end + end + end + end + end + end + + def config + Rack::Adapter.instance.config + end + end + end + end + end +end diff --git a/adapters/rack/lib/opentelemetry/adapters/rack/util/queue_time.rb b/adapters/rack/lib/opentelemetry/adapters/rack/util/queue_time.rb new file mode 100644 index 000000000..e2870a645 --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/rack/util/queue_time.rb @@ -0,0 +1,48 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +module OpenTelemetry + module Adapters + module Rack + module Util + # QueueTime simply... + module QueueTime + REQUEST_START = 'HTTP_X_REQUEST_START' + QUEUE_START = 'HTTP_X_QUEUE_START' + MINIMUM_ACCEPTABLE_TIME_VALUE = 1_000_000_000 + + module_function + + def get_request_start(env, now = nil) # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity + header = env[REQUEST_START] || env[QUEUE_START] + return unless header + + # nginx header is seconds in the format "t=1512379167.574" + # apache header is microseconds in the format "t=1570633834463123" + # heroku header is milliseconds in the format "1570634024294" + time_string = header.to_s.delete('^0-9') + return if time_string.nil? + + # Return nil if the time is clearly invalid + time_value = "#{time_string[0, 10]}.#{time_string[10, 6]}".to_f + return if time_value.zero? || time_value < MINIMUM_ACCEPTABLE_TIME_VALUE + + # return the request_start only if it's lesser than + # current time, to avoid significant clock skew + request_start = Time.at(time_value) + now ||= Time.now.utc + request_start.utc > now ? nil : request_start + rescue StandardError => e + # in case of an Exception we don't create a + # `request.queuing` span + OpenTelemetry.logger.debug("[rack] unable to parse request queue headers: #{e}") + nil + end + end + end + end + end +end diff --git a/adapters/rack/lib/opentelemetry/adapters/rack/version.rb b/adapters/rack/lib/opentelemetry/adapters/rack/version.rb new file mode 100644 index 000000000..89afa11ba --- /dev/null +++ b/adapters/rack/lib/opentelemetry/adapters/rack/version.rb @@ -0,0 +1,13 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +module OpenTelemetry + module Adapters + module Rack + VERSION = '0.0.0' + end + end +end diff --git a/adapters/rack/opentelemetry-adapters-rack.gemspec b/adapters/rack/opentelemetry-adapters-rack.gemspec new file mode 100644 index 000000000..3bec1727c --- /dev/null +++ b/adapters/rack/opentelemetry-adapters-rack.gemspec @@ -0,0 +1,42 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +lib = File.expand_path('lib', __dir__) +$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) +require 'opentelemetry/adapters/rack/version' + +Gem::Specification.new do |spec| + spec.name = 'opentelemetry-adapters-rack' + spec.version = OpenTelemetry::Adapters::Rack::VERSION + spec.authors = ['OpenTelemetry Authors'] + spec.email = ['cncf-opentelemetry-contributors@lists.cncf.io'] + + spec.summary = 'Rack instrumentation adapter for the OpenTelemetry framework' + spec.description = 'Rack instrumentation adapter for the OpenTelemetry framework' + spec.homepage = 'https://github.com/open-telemetry/opentelemetry-ruby' + spec.license = 'Apache-2.0' + + spec.files = ::Dir.glob('lib/**/*.rb') + + ::Dir.glob('*.md') + + ['LICENSE'] + spec.require_paths = ['lib'] + spec.required_ruby_version = '>= 2.4.0' + + spec.add_dependency 'opentelemetry-api', '~> 0.0' + + spec.add_development_dependency 'appraisal', '~> 2.2.0' + spec.add_development_dependency 'bundler', '~> 2.0.2' + spec.add_development_dependency 'minitest', '~> 5.0' + spec.add_development_dependency 'opentelemetry-sdk', '~> 0.0' + spec.add_development_dependency 'rack', '~> 2.0.8' + spec.add_development_dependency 'rack-test', '~> 1.1.0' + spec.add_development_dependency 'rake', '~> 12.3.3' + spec.add_development_dependency 'rubocop', '~> 0.73.0' + spec.add_development_dependency 'simplecov', '~> 0.17.1' + spec.add_development_dependency 'webmock', '~> 3.7.6' + spec.add_development_dependency 'yard', '~> 0.9' + spec.add_development_dependency 'yard-doctest', '~> 0.1.6' +end diff --git a/adapters/rack/test/.rubocop.yml b/adapters/rack/test/.rubocop.yml new file mode 100644 index 000000000..dd9425858 --- /dev/null +++ b/adapters/rack/test/.rubocop.yml @@ -0,0 +1,4 @@ +inherit_from: ../.rubocop.yml + +Metrics/BlockLength: + Enabled: false diff --git a/adapters/rack/test/opentelemetry/adapters/rack/adapter_test.rb b/adapters/rack/test/opentelemetry/adapters/rack/adapter_test.rb new file mode 100644 index 000000000..663611d03 --- /dev/null +++ b/adapters/rack/test/opentelemetry/adapters/rack/adapter_test.rb @@ -0,0 +1,76 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'test_helper' + +require_relative '../../../../lib/opentelemetry/adapters/rack/adapter' + +describe OpenTelemetry::Adapters::Rack::Adapter do + let(:adapter_class) { OpenTelemetry::Adapters::Rack::Adapter } + let(:adapter) { adapter_class.instance } + let(:config) { {} } + + after do + # simulate a fresh install: + adapter.instance_variable_set('@installed', false) + adapter.install({}) + end + + describe 'config[:retain_middleware_names]' do + let(:config) { Hash(retain_middleware_names: true) } + + describe 'without config[:application]' do + it 'raises error' do + # allow for re-installation with new config: + adapter.instance_variable_set('@installed', false) + + assert_raises adapter_class::MissingApplicationError do + adapter.install(config) + end + end + end + + describe 'default' do + class MyAppClass + attr_reader :env + + def use(*); end + + def call(env) + @env = env + [200, { 'Content-Type' => 'text/plain' }, ['OK']] + end + end + + let(:app) { MyAppClass.new } + + describe 'without config' do + it 'does not set RESPONSE_MIDDLEWARE' do + app.call({}) + + _(app.env['RESPONSE_MIDDLEWARE']).must_be_nil + end + end + + describe 'with config[:application]' do + let(:config) do + { retain_middleware_names: true, + application: app } + end + + it 'retains RESPONSE_MIDDLEWARE after .call' do + # allow for re-installation with new config: + adapter.instance_variable_set('@installed', false) + + adapter.install(config) + app.call({}) + + _(app.env['RESPONSE_MIDDLEWARE']).must_equal 'MyAppClass' + end + end + end + end +end diff --git a/adapters/rack/test/opentelemetry/adapters/rack/middlewares/tracer_middleware_test.rb b/adapters/rack/test/opentelemetry/adapters/rack/middlewares/tracer_middleware_test.rb new file mode 100644 index 000000000..f1af14919 --- /dev/null +++ b/adapters/rack/test/opentelemetry/adapters/rack/middlewares/tracer_middleware_test.rb @@ -0,0 +1,191 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'test_helper' + +# require Adapter so .install method is found: +require_relative '../../../../../lib/opentelemetry/adapters/rack' +require_relative '../../../../../lib/opentelemetry/adapters/rack/adapter' +require_relative '../../../../../lib/opentelemetry/adapters/rack/middlewares/tracer_middleware' + +describe OpenTelemetry::Adapters::Rack::Middlewares::TracerMiddleware do + let(:adapter_module) { OpenTelemetry::Adapters::Rack } + let(:adapter_class) { adapter_module::Adapter } + let(:adapter) { adapter_class.instance } + + let(:described_class) { OpenTelemetry::Adapters::Rack::Middlewares::TracerMiddleware } + + let(:app) { ->(_env) { [200, { 'Content-Type' => 'text/plain' }, ['OK']] } } + let(:middleware) { described_class.new(app) } + let(:rack_builder) { Rack::Builder.new } + + let(:exporter) { EXPORTER } + let(:first_span) { exporter.finished_spans.first } + + let(:default_config) { {} } + let(:config) { default_config } + let(:env) { {} } + + before do + # clear captured spans: + exporter.reset + + # simulate a fresh install: + adapter.instance_variable_set('@installed', false) + adapter.install(config) + + # clear out cached config: + described_class.send(:clear_cached_config) + + # integrate tracer middleware: + rack_builder.run app + rack_builder.use described_class + end + + after do + # installation is 'global', so it should be reset: + adapter.instance_variable_set('@installed', false) + adapter.install(default_config) + end + + describe '#call' do + before do + Rack::MockRequest.new(rack_builder).get('/', env) + end + + it 'records attributes' do + _(first_span.attributes['component']).must_equal 'http' + _(first_span.attributes['http.method']).must_equal 'GET' + _(first_span.attributes['http.status_code']).must_equal 200 + _(first_span.attributes['http.status_text']).must_equal 'OK' + _(first_span.attributes['http.target']).must_equal '/' + _(first_span.status.canonical_code).must_equal OpenTelemetry::Trace::Status::OK + _(first_span.attributes['http.url']).must_be_nil + _(first_span.name).must_equal '/' + _(first_span.kind).must_equal :server + end + + it 'has no parent' do + _(first_span.parent_span_id).must_equal '0000000000000000' + end + + describe 'config[:allowed_request_headers]' do + let(:env) { Hash('HTTP_FOO_BAR' => 'http foo bar value') } + + it 'defaults to nil' do + _(first_span.attributes['http.request.headers.foo_bar']).must_be_nil + end + + describe 'when configured' do + let(:config) { default_config.merge(allowed_request_headers: ['foo_BAR']) } + + it 'returns attribute' do + _(first_span.attributes['http.request.headers.foo_bar']).must_equal 'http foo bar value' + end + end + end + + describe 'config[:allowed_response_headers]' do + let(:app) do + ->(_env) { [200, { 'Foo-Bar' => 'foo bar response header' }, ['OK']] } + end + + it 'defaults to nil' do + _(first_span.attributes['http.response.headers.foo_bar']).must_be_nil + end + + describe 'when configured' do + let(:config) { default_config.merge(allowed_response_headers: ['Foo-Bar']) } + + it 'returns attribute' do + _(first_span.attributes['http.response.headers.foo_bar']).must_equal 'foo bar response header' + end + + describe 'case-sensitively' do + let(:config) { default_config.merge(allowed_response_headers: ['fOO-bAR']) } + + it 'returns attribute' do + _(first_span.attributes['http.response.headers.foo_bar']).must_equal 'foo bar response header' + end + end + end + end + + describe 'config[:record_frontend_span]' do + let(:request_span) { exporter.finished_spans.first } + + describe 'default' do + it 'does not record span' do + _(exporter.finished_spans.size).must_equal 1 + end + + it 'does not parent the request_span' do + _(request_span.parent_span_id).must_equal '0000000000000000' + end + end + + describe 'when recordable' do + let(:config) { default_config.merge(record_frontend_span: true) } + let(:env) { Hash('HTTP_X_REQUEST_START' => Time.now.to_i) } + let(:frontend_span) { exporter.finished_spans[1] } + let(:request_span) { exporter.finished_spans[0] } + + it 'records span' do + _(exporter.finished_spans.size).must_equal 2 + _(frontend_span.name).must_equal 'http_server.proxy' + _(frontend_span.attributes['service']).must_be_nil + end + + it 'changes request_span kind' do + _(request_span.kind).must_equal :internal + end + + it 'frontend_span parents request_span' do + _(request_span.parent_span_id).must_equal frontend_span.span_id + end + end + end + end + + describe 'config[:quantization]' do + before do + Rack::MockRequest.new(rack_builder).get('/really_long_url', env) + end + + describe 'without quantization' do + it 'span.name is uri path' do + _(first_span.name).must_equal '/really_long_url' + end + end + + describe 'with quantization' do + let(:quantization_example) do + # demonstrate simple shortening of URL: + ->(url) { url.to_s[0..5] } + end + let(:config) { default_config.merge(url_quantization: quantization_example) } + + it 'mutates url according to url_quantization' do + _(first_span.name).must_equal '/reall' + end + end + end + + describe '#call with error' do + SimulatedError = Class.new(StandardError) + + let(:app) do + ->(_env) { raise SimulatedError } + end + + it 'records error in span and then re-raises' do + assert_raises SimulatedError do + Rack::MockRequest.new(rack_builder).get('/', env) + end + _(first_span.status.canonical_code).must_equal OpenTelemetry::Trace::Status::UNKNOWN_ERROR + end + end +end diff --git a/adapters/rack/test/opentelemetry/adapters/rack/util/queue_time_test.rb b/adapters/rack/test/opentelemetry/adapters/rack/util/queue_time_test.rb new file mode 100644 index 000000000..cedd4bcdb --- /dev/null +++ b/adapters/rack/test/opentelemetry/adapters/rack/util/queue_time_test.rb @@ -0,0 +1,90 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'test_helper' + +require 'rack' +require_relative '../../../../../lib/opentelemetry/adapters/rack/util/queue_time' + +describe OpenTelemetry::Adapters::Rack::Util::QueueTime do + let(:described_class) { OpenTelemetry::Adapters::Rack::Util::QueueTime } + + describe '#get_request_start' do + let(:request_start) { described_class.get_request_start(env) } + + describe 'given a Rack env with' do + describe 'milliseconds' do + describe 'REQUEST_START' do + let(:env) { { described_class::REQUEST_START => "t=#{expected}" } } + let(:expected) { 1_512_379_167.574 } + it { expect(request_start.to_f).must_equal(expected) } + + describe 'but does not start with t=' do + let(:env) { { described_class::REQUEST_START => expected } } + it { expect(request_start.to_f).must_equal(expected) } + end + + describe 'without decimal places' do + let(:env) { { described_class::REQUEST_START => expected } } + let(:expected) { 1_512_379_167_574 } + it { expect(request_start.to_f).must_equal(1_512_379_167.574) } + end + + describe 'but a malformed expected' do + let(:expected) { 'foobar' } + it { _(request_start).must_be_nil } + end + + describe 'before the start of the acceptable time range' do + let(:expected) { 999_999_999.000 } + it { _(request_start).must_be_nil } + end + end + + describe 'QUEUE_START' do + let(:env) { { described_class::QUEUE_START => "t=#{expected}" } } + let(:expected) { 1_512_379_167.574 } + it { expect(request_start.to_f).must_equal(expected) } + end + end + + describe 'microseconds' do + describe 'REQUEST_START' do + let(:env) { { described_class::REQUEST_START => "t=#{expected}" } } + let(:expected) { 1_570_633_834.463123 } + it { expect(request_start.to_f).must_equal(expected) } + + describe 'but does not start with t=' do + let(:env) { { described_class::REQUEST_START => expected } } + it { expect(request_start.to_f).must_equal(expected) } + end + + describe 'without decimal places' do + let(:env) { { described_class::REQUEST_START => expected } } + let(:expected) { 1_570_633_834_463_123 } + it { expect(request_start.to_f).must_equal(1_570_633_834.463123) } + end + + describe 'but a malformed expected' do + let(:expected) { 'foobar' } + it { _(request_start).must_be_nil } + end + end + + describe 'QUEUE_START' do + let(:env) { { described_class::QUEUE_START => "t=#{expected}" } } + let(:expected) { 1_570_633_834.463123 } + it { expect(request_start.to_f).must_equal(expected) } + end + end + + describe 'nothing' do + let(:env) { {} } + it { _(request_start).must_be_nil } + end + end + end +end diff --git a/adapters/rack/test/opentelemetry/adapters/rack_test.rb b/adapters/rack/test/opentelemetry/adapters/rack_test.rb new file mode 100644 index 000000000..284bf606c --- /dev/null +++ b/adapters/rack/test/opentelemetry/adapters/rack_test.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'test_helper' + +require_relative '../../../lib/opentelemetry/adapters/rack' + +describe OpenTelemetry::Adapters::Rack do + let(:adapter) { OpenTelemetry::Adapters::Rack::Adapter.instance } + + it 'has #name' do + _(adapter.name).must_equal 'OpenTelemetry::Adapters::Rack' + end + + it 'has #version' do + _(adapter.version).wont_be_nil + _(adapter.version).wont_be_empty + end + + describe '#install' do + it 'accepts argument' do + adapter.install({}) + end + end +end diff --git a/adapters/rack/test/test_helper.rb b/adapters/rack/test/test_helper.rb new file mode 100644 index 000000000..b13701e24 --- /dev/null +++ b/adapters/rack/test/test_helper.rb @@ -0,0 +1,20 @@ +# frozen_string_literal: true + +# Copyright 2020 OpenTelemetry Authors +# +# SPDX-License-Identifier: Apache-2.0 + +require 'rack' + +require 'opentelemetry/sdk' + +require 'minitest/autorun' +require 'webmock/minitest' + +# global opentelemetry-sdk setup: +EXPORTER = OpenTelemetry::SDK::Trace::Export::InMemorySpanExporter.new +span_processor = OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new(EXPORTER) + +OpenTelemetry::SDK.configure do |c| + c.add_span_processor span_processor +end diff --git a/docker-compose.yml b/docker-compose.yml index 8b0f6e1e3..c48594388 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -36,6 +36,10 @@ services: <<: *base working_dir: /app/adapters/net_http/example + ex-adapter-rack: + <<: *base + working_dir: /app/adapters/rack/example + ex-adapter-redis: <<: *base environment: