Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add elasticsearch instrumentation #1525

Merged
merged 61 commits into from
Oct 14, 2022
Merged

Add elasticsearch instrumentation #1525

Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
12b1357
add elasticsearch instrumentation
tannalynn Sep 28, 2022
d091e38
Update instrumentation_generator
kaylareopelle Oct 3, 2022
e510eb9
Add alias for original method
kaylareopelle Oct 3, 2022
2f1fd24
Use ::Elasticsearch in to_instrument condition
kaylareopelle Oct 3, 2022
f0bc620
New finding
kaylareopelle Oct 4, 2022
b721d22
Create datastore segment on #perform_request call
kaylareopelle Oct 4, 2022
7eadaea
Hard-code cluster name
kaylareopelle Oct 4, 2022
5805d00
added path to datastore segment and updated
tannalynn Oct 4, 2022
0490175
Rename Mongo::Obfuscator to NosqlObfucsator
kaylareopelle Oct 4, 2022
b2c89d3
Add params, body, method as obfuscated attributes
kaylareopelle Oct 4, 2022
58b96d5
Add ability to capture queries and optionally obfuscate
kaylareopelle Oct 4, 2022
9b43317
Replace attributes with notice_nosql_statement
kaylareopelle Oct 5, 2022
00d2d01
add elasticsearch to database multiverse group
tannalynn Oct 5, 2022
d4e3b99
Merge branch 'dev' into add_elasticsearch_instrumentation
tannalynn Oct 5, 2022
f324ed5
alphabetized services in github actions
tannalynn Oct 5, 2022
129bfa8
add elastisearch 7 and 8 services
tannalynn Oct 5, 2022
a5fbe12
add elasticsearch 7 and 8 to multiverse
tannalynn Oct 5, 2022
a0bb6cf
check elasticsearch by gem version
tannalynn Oct 5, 2022
15c19fc
use string for cluster name for elasticsearch
tannalynn Oct 5, 2022
8a5636c
add elasticsearch to pr ci.yml
tannalynn Oct 5, 2022
8176d74
fixed elasticsearch chain
tannalynn Oct 6, 2022
7bccb79
elasticsearch 7 added to docker compose
tannalynn Oct 6, 2022
3f6a169
fixes the alias method for chain in elasticsearch
tannalynn Oct 6, 2022
c8c6eb6
add both elastisearch 7 and 8 to docker compose
tannalynn Oct 6, 2022
604f21b
decrease memory elasticsearch containers can use
tannalynn Oct 6, 2022
7b240a9
Allow truncate_query to handle nil arguments
kaylareopelle Oct 6, 2022
416ef6e
Add elasticsearch instrumentation tests
kaylareopelle Oct 6, 2022
a43327a
Update method name in chain instrumentation
kaylareopelle Oct 6, 2022
2085738
Add Envfile entry for elasticsearch 7
kaylareopelle Oct 6, 2022
514eaf1
Merge branch 'add_elasticsearch_instrumentation' into elasticsearch_t…
kaylareopelle Oct 6, 2022
25ab63e
Fix missing end
kaylareopelle Oct 6, 2022
106331d
Update alias method to include both renames
kaylareopelle Oct 6, 2022
d0590af
Update expectation
kaylareopelle Oct 6, 2022
f6a20d3
Add version conditional to chain
kaylareopelle Oct 6, 2022
12f84c2
Always use path for :port_path_or_id
kaylareopelle Oct 6, 2022
0cde3df
Adapt tests for Docker
kaylareopelle Oct 6, 2022
9d3ce29
Merge pull request #1533 from newrelic/elasticsearch_tests
kaylareopelle Oct 6, 2022
583b61f
Updated cluster_name and hosts for elasticsearch
tannalynn Oct 6, 2022
65828f4
update cluster_name test to use docker-cluster
tannalynn Oct 6, 2022
9442ad3
update cluster_name to use perform_request
tannalynn Oct 6, 2022
e716093
use nil for cluster_name if unable to get it
tannalynn Oct 7, 2022
ae957dc
Merge branch 'dev' into add_elasticsearch_instrumentation
tannalynn Oct 13, 2022
9df3328
add changelog entry
tannalynn Oct 13, 2022
fc55ded
add elasicsearch pr link to changelog
tannalynn Oct 13, 2022
616a16f
Update newrelic.yml
tannalynn Oct 13, 2022
00df722
removed outdated comments
tannalynn Oct 13, 2022
f0a2f5b
Merge branch 'add_elasticsearch_instrumentation' of github.com:newrel…
tannalynn Oct 13, 2022
80ce6f4
Merge branch 'dev' into add_elasticsearch_instrumentation
tannalynn Oct 13, 2022
693490f
add test for nil query
tannalynn Oct 13, 2022
3be723b
Merge branch 'add_elasticsearch_instrumentation' of github.com:newrel…
tannalynn Oct 13, 2022
ea46b81
capture_query will early return if query is nil
tannalynn Oct 13, 2022
7789529
Update lib/new_relic/agent/instrumentation/elasticsearch/instrumentat…
tannalynn Oct 13, 2022
7e943b3
address PR feedback
tannalynn Oct 13, 2022
9922c28
fix alphabetization
tannalynn Oct 14, 2022
e40ef6b
Merge branch 'add_elasticsearch_instrumentation' of github.com:newrel…
tannalynn Oct 14, 2022
42a2275
Merge branch 'dev' into add_elasticsearch_instrumentation
fallwith Oct 14, 2022
7a9a80d
add empty lines after gaurd clause
tannalynn Oct 14, 2022
9b56338
Dynamically set Elasticsearch operation
kaylareopelle Oct 14, 2022
a6f901d
Move instrumentation.elasticsearch
kaylareopelle Oct 14, 2022
c9380e3
Update lib/new_relic/agent/instrumentation/elasticsearch/instrumentat…
kaylareopelle Oct 14, 2022
6e44b20
Merge pull request #1543 from newrelic/set_elasticsearch_operation_dy…
kaylareopelle Oct 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 81 additions & 33 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,43 @@ jobs:
needs: run_rubocop
runs-on: ubuntu-22.04
services:
elasticsearch7:
image: elasticsearch:7.16.2
env:
discovery.type: single-node
ports:
- 9200:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
elasticsearch8:
image: elasticsearch:8.4.2
env:
discovery.type: single-node
xpack.security.enabled: false
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
fallwith marked this conversation as resolved.
Show resolved Hide resolved
ports:
- 9250:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
memcached:
image: memcached:latest
ports:
- 11211:11211
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: ${{ contains(fromJson('["2.2.10", "2.3.8", "2.4.10"]'), matrix.ruby-version) && 'mongo:5.0.11' || 'mongo:latest' }}
ports:
- 27017:27017
mysql:
image: mysql:5.7
env:
Expand All @@ -120,19 +157,6 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: ${{ contains(fromJson('["2.2.10", "2.3.8", "2.4.10"]'), matrix.ruby-version) && 'mongo:5.0.11' || 'mongo:latest' }}
ports:
- 27017:27017
rabbitmq:
image: rabbitmq:latest
ports:
Expand All @@ -142,12 +166,12 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
memcached:
image: memcached:latest
redis:
image: redis
ports:
- 11211:11211
- 6379:6379
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
Expand Down Expand Up @@ -282,30 +306,54 @@ jobs:
if: ${{ needs.check_jruby_multiverse.outputs.run_job == 'true' }}
runs-on: ubuntu-22.04
services:
mysql:
image: mysql:5.7
elasticsearch7:
image: elasticsearch:7.16.2
env:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
discovery.type: single-node
ports:
- "3306:3306"
postgres:
image: postgres:latest
- 9200:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
elasticsearch8:
image: elasticsearch:8.4.2
env:
discovery.type: single-node
xpack.security.enabled: false
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
ports:
- 5432:5432
redis:
image: redis
- 9250:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
memcached:
image: memcached:latest
ports:
- 6379:6379
- 11211:11211
options: >-
--health-cmd "redis-cli ping"
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: mongo:5.0.11
ports:
- 27017:27017
mysql:
image: mysql:5.7
env:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
ports:
- "3306:3306"
postgres:
image: postgres:latest
ports:
- 5432:5432
rabbitmq:
image: rabbitmq:latest
ports:
Expand All @@ -315,12 +363,12 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
memcached:
image: memcached:latest
redis:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the alphabetizing!

image: redis
ports:
- 11211:11211
- 6379:6379
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
Expand Down
114 changes: 81 additions & 33 deletions .github/workflows/ci_cron.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,43 @@ jobs:
needs: run_rubocop
runs-on: ubuntu-22.04
services:
elasticsearch7:
image: elasticsearch:7.16.2
env:
discovery.type: single-node
ports:
- 9200:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
elasticsearch8:
image: elasticsearch:8.4.2
env:
discovery.type: single-node
xpack.security.enabled: false
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
ports:
- 9250:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
memcached:
image: memcached:latest
ports:
- 11211:11211
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: ${{ contains(fromJson('["2.2.10", "2.3.8", "2.4.10"]'), matrix.ruby-version) && 'mongo:5.0.11' || 'mongo:latest' }}
ports:
- 27017:27017
mysql:
image: mysql:5.7
env:
Expand All @@ -142,19 +179,6 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: ${{ contains(fromJson('["2.2.10", "2.3.8", "2.4.10"]'), matrix.ruby-version) && 'mongo:5.0.11' || 'mongo:latest' }}
ports:
- 27017:27017
rabbitmq:
image: rabbitmq:latest
ports:
Expand All @@ -164,12 +188,12 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
memcached:
image: memcached:latest
redis:
image: redis
ports:
- 11211:11211
- 6379:6379
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
Expand Down Expand Up @@ -271,30 +295,54 @@ jobs:
needs: run_rubocop
runs-on: ubuntu-22.04
services:
mysql:
image: mysql:5.7
elasticsearch7:
image: elasticsearch:7.16.2
env:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
discovery.type: single-node
ports:
- "3306:3306"
postgres:
image: postgres:latest
- 9200:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
elasticsearch8:
image: elasticsearch:8.4.2
env:
discovery.type: single-node
xpack.security.enabled: false
ES_JAVA_OPTS: "-Xms1g -Xmx1g"
ports:
- 5432:5432
redis:
image: redis
- 9250:9200
options: >-
--health-cmd "curl http://localhost:9200/_cluster/health"
--health-interval 10s
--health-timeout 5s
--health-retries 10
memcached:
image: memcached:latest
ports:
- 6379:6379
- 11211:11211
options: >-
--health-cmd "redis-cli ping"
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mongodb:
image: mongo:5.0.11
ports:
- 27017:27017
mysql:
image: mysql:5.7
env:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
ports:
- "3306:3306"
postgres:
image: postgres:latest
ports:
- 5432:5432
rabbitmq:
image: rabbitmq:latest
ports:
Expand All @@ -304,12 +352,12 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
memcached:
image: memcached:latest
redis:
image: redis
ports:
- 11211:11211
- 6379:6379
options: >-
--health-cmd "timeout 5 bash -c 'cat < /dev/null > /dev/udp/127.0.0.1/11211'"
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/scripts/rubygems-publish.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# frozen_string_literal: true
gem_name = ARGV[0]
raise "gem name sans version must be supplied" if gem_name.to_s == ""

Expand All @@ -21,7 +22,7 @@
if $?.to_i.zero?
puts "#{gem_filename} successfully pushed to rubygems.org!"
else
if result =~ /Repushing of gem versions is not allowed/
if result.include?('Repushing of gem versions is not allowed')
fallwith marked this conversation as resolved.
Show resolved Hide resolved
puts "Pushing #{gem_filename} skipped because this version is already published to rubygems.org!"
exit 0
else
Expand Down
13 changes: 12 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,17 @@

## v8.12.0

Version 8.12.0 of the agent delivers some valuable code cleanup, and increases the default number of recorded Custom Events.
Version 8.12.0 of the agent delivers new Elasticsearch instrumentation, some valuable code cleanup, and increases the default number of recorded Custom Events.

* **Support for Elasticsearch instrumentation**

This release adds support to automatically instrument the [elasticsearch](https://rubygems.org/gems/elasticsearch) gem. Versions 7.x and 8.x are supported. [PR#1525](https://github.com/newrelic/newrelic-ruby-agent/pull/1525)

| Configuration name | Default | Behavior |
| ----------- | ----------- |----------- |
| `instrumentation.elasticsearch` | auto | Controls auto-instrumentation of the elasticsearch library at start up. May be one of `auto`, `prepend`, `chain`, `disabled`. |
| `elasticsearch.capture_queries` | true | If `true`, the agent captures Elasticsearch queries in transaction traces. |
| `elasticsearch.obfuscate_queries` | true | If `true`, the agent obfuscates Elasticsearch queries in transaction traces. |

* **Cleanup: Remove orphaned code from unit tests**

Expand All @@ -13,6 +23,7 @@
* **Increase default for `custom_insights_events.max_samples_stored`**

New Relic has discovered a large number of [Custom Events](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/report-custom-event-data/) are dropped due to the configured value for `custom_insights_events.max_samples_stored`. In an effort to help customers receive more of their custom events, we're raising the default maximum value for custom events stored per minute from 1,000 events to 3,000 events. The highest possible number of events that can be sent per minute is 100,000.

## v8.11.0

Version 8.11.0 of the agent updates the `newrelic deployments` command to work with API keys issued to newer accounts, fixes a memory leak in the instrumentation of Curb error handling, further preps for Ruby 3.2.0 support, and includes several community member driven cleanup and improvement efforts. Thank you to everyone involved!
Expand Down
Loading