Scrub request body contents to remove illegal UTF-8 characters #832

Anasjabri · 2020-07-14T20:56:11Z

What does this pull request do?

Scrub request body contents to remove illegal UTF-8 characters

Why is it important?

Because workers die with malformed format string

Checklist

I have signed the Contributor License Agreement.
My code follows the style guidelines of this project (See .rubocop.yml)
I have rebased my changes on top of the latest master branch

I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

I have made corresponding changes to the documentation
I have updated CHANGELOG.asciidoc
I have updated supported-technologies.asciidoc
Added an API method or config option? Document in which version this will be introduced

Related issues

cla-checker-service · 2020-07-14T20:56:14Z

💚 CLA has been signed

apmmachine · 2020-07-14T21:00:47Z

💔 Tests Failed

Expand to view the summary

Build stats

Build Cause: [mikker commented: jenkins, please run the tests]
Start Time: 2020-08-04T21:39:52.808+0000
Duration: 31 min 29 sec

Test stats 🧪

Test	Results
Failed	1
Passed	59217
Skipped	0
Total	59218

Test errors

Expand to view the tests failures

Name: Tests / Tests / Ruby:docker.elastic.co/observability-ci/jruby:9.1-7-jdk#rails-5.0 / Rails integration handles multiple threads – Rails integration
- Age: 1
- Duration: 2.870785
- Error Details: failed Rails integration handles multiple threads

Steps errors

Expand to view the steps failures

Name: Shell Script
- Description: [2020-08-04T21:47:49.755Z] + ./spec/scripts/spec.sh docker.elastic.co/observability-ci/jruby:9.2-13-
- Duration: 4 min 47 sec
- Start Time: 2020-08-04T21:47:49.465+0000
- log
Name: Shell Script
- Description: [2020-08-04T21:47:43.723Z] + ./spec/scripts/spec.sh docker.elastic.co/observability-ci/jruby:9.2-13-
- Duration: 4 min 42 sec
- Start Time: 2020-08-04T21:47:43.423+0000
- log
Name: Shell Script
- Description: [2020-08-04T21:47:33.387Z] + ./spec/scripts/spec.sh docker.elastic.co/observability-ci/jruby:9.2-12-
- Duration: 6 min 40 sec
- Start Time: 2020-08-04T21:47:33.094+0000
- log
Name: Shell Script
- Description: [2020-08-04T21:48:13.929Z] + ./spec/scripts/spec.sh docker.elastic.co/observability-ci/jruby:9.1-7-j
- Duration: 11 min 43 sec
- Start Time: 2020-08-04T21:48:13.637+0000
- log
Name: Shell Script
- Description: [2020-08-04T21:59:30.821Z] + ./spec/scripts/spec.sh docker.elastic.co/observability-ci/jruby:9.1-7-j
- Duration: 10 min 50 sec
- Start Time: 2020-08-04T21:59:30.531+0000
- log
Name: Error signal
- Description:
- Duration: 0 min 0 sec
- Start Time: 2020-08-04T22:09:20.637+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-08-04T22:11:10.014Z] Installing webmock 3.8.3
[2020-08-04T22:11:10.274Z] Installing domain_name 0.5.20190701
[2020-08-04T22:11:10.274Z] Fetching http-parser 1.2.1
[2020-08-04T22:11:10.274Z] Fetching elasticsearch-transport 7.8.1
[2020-08-04T22:11:10.274Z] Installing http-parser 1.2.1 with native extensions
[2020-08-04T22:11:10.274Z] Installing elasticsearch-transport 7.8.1
[2020-08-04T22:11:11.216Z] Fetching grpc 1.30.2 (x86_64-linux)
[2020-08-04T22:11:11.216Z] Fetching sinatra 2.0.8.1
[2020-08-04T22:11:11.216Z] Fetching rspec 3.9.0
[2020-08-04T22:11:11.216Z] Installing rspec 3.9.0
[2020-08-04T22:11:11.216Z] Fetching rspec-its 1.3.0
[2020-08-04T22:11:11.477Z] Installing rspec-its 1.3.0
[2020-08-04T22:11:11.477Z] Installing sinatra 2.0.8.1
[2020-08-04T22:11:11.477Z] Fetching sneakers 2.12.0
[2020-08-04T22:11:11.477Z] Fetching simplecov-cobertura 1.3.1
[2020-08-04T22:11:11.477Z] Installing sneakers 2.12.0
[2020-08-04T22:11:11.477Z] Installing simplecov-cobertura 1.3.1
[2020-08-04T22:11:11.477Z] Fetching protobuf-cucumber 3.10.8
[2020-08-04T22:11:11.739Z] Fetching aws-sdk-dynamodb 1.51.0
[2020-08-04T22:11:11.739Z] Installing protobuf-cucumber 3.10.8
[2020-08-04T22:11:11.739Z] Installing aws-sdk-dynamodb 1.51.0
[2020-08-04T22:11:12.000Z] Fetching aws-sdk-sqs 1.30.0
[2020-08-04T22:11:12.000Z] Installing aws-sdk-sqs 1.30.0
[2020-08-04T22:11:12.000Z] Fetching shoryuken 5.0.4
[2020-08-04T22:11:12.261Z] Installing shoryuken 5.0.4
[2020-08-04T22:11:12.261Z] Fetching rubocop 0.88.0
[2020-08-04T22:11:12.521Z] Fetching http-cookie 1.0.3
[2020-08-04T22:11:12.521Z] Installing http-cookie 1.0.3
[2020-08-04T22:11:12.521Z] Installing rubocop 0.88.0
[2020-08-04T22:11:12.521Z] Fetching elasticsearch 7.8.1
[2020-08-04T22:11:12.783Z] Installing elasticsearch 7.8.1
[2020-08-04T22:11:12.783Z] Fetching yarjuf 2.0.0
[2020-08-04T22:11:12.783Z] Installing yarjuf 2.0.0
[2020-08-04T22:11:12.783Z] Fetching resque 2.0.0
[2020-08-04T22:11:13.045Z] Installing resque 2.0.0
[2020-08-04T22:11:13.045Z] Installing grpc 1.30.2 (x86_64-linux)
[2020-08-04T22:11:13.306Z] Fetching cucumber-messages 12.4.0
[2020-08-04T22:11:13.306Z] Installing cucumber-messages 12.4.0
[2020-08-04T22:11:13.306Z] Fetching http 4.4.1
[2020-08-04T22:11:13.567Z] Installing http 4.4.1
[2020-08-04T22:11:13.828Z] Fetching cucumber-gherkin 14.2.0
[2020-08-04T22:11:13.828Z] Installing cucumber-gherkin 14.2.0
[2020-08-04T22:11:14.089Z] Fetching cucumber-create-meta 1.0.0
[2020-08-04T22:11:14.089Z] Installing cucumber-create-meta 1.0.0
[2020-08-04T22:11:14.089Z] Fetching cucumber-html-formatter 7.2.0
[2020-08-04T22:11:14.089Z] Using elastic-apm 3.8.0 from source at `.`
[2020-08-04T22:11:14.089Z] Fetching cucumber-core 7.1.0
[2020-08-04T22:11:14.351Z] Installing cucumber-core 7.1.0
[2020-08-04T22:11:14.351Z] Installing cucumber-html-formatter 7.2.0
[2020-08-04T22:11:14.612Z] Fetching rubocop-performance 1.7.1
[2020-08-04T22:11:14.612Z] Fetching cucumber-wire 3.1.0
[2020-08-04T22:11:14.872Z] Installing cucumber-wire 3.1.0
[2020-08-04T22:11:14.872Z] Installing rubocop-performance 1.7.1
[2020-08-04T22:11:14.872Z] Fetching cucumber 4.1.0
[2020-08-04T22:11:15.133Z] Installing cucumber 4.1.0
[2020-08-04T22:11:15.394Z] Bundle complete! 35 Gemfile dependencies, 117 gems now installed.
[2020-08-04T22:11:15.394Z] Bundled gems are installed into `./vendor/2.7`
[2020-08-04T22:11:15.394Z] Post-install message from i18n:
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] HEADS UP! i18n 1.1 changed fallbacks to exclude default locale.
[2020-08-04T22:11:15.394Z] But that may break your application.
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] If you are upgrading your Rails application from an older version of Rails:
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] Please check your Rails app for 'config.i18n.fallbacks = true'.
[2020-08-04T22:11:15.394Z] If you're using I18n (>= 1.1.0) and Rails (< 5.2.2), this should be
[2020-08-04T22:11:15.394Z] 'config.i18n.fallbacks = [I18n.default_locale]'.
[2020-08-04T22:11:15.394Z] If not, fallbacks will be broken in your app by I18n 1.1.x.
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] If you are starting a NEW Rails application, you can ignore this notice.
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] For more info see:
[2020-08-04T22:11:15.394Z] https://github.com/svenfuchs/i18n/releases/tag/v1.1.0
[2020-08-04T22:11:15.394Z] 
[2020-08-04T22:11:15.394Z] Post-install message from sucker_punch:
[2020-08-04T22:11:15.394Z] Sucker Punch v2.0 introduces backwards-incompatible changes.
[2020-08-04T22:11:15.394Z] Please see https://github.com/brandonhilkert/sucker_punch/blob/master/CHANGES.md#200 for details.
[2020-08-04T22:11:17.948Z] Coverage report generated for RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec, RSpec to /app/coverage/coverage.xml
[2020-08-04T22:11:18.553Z] [Cobertura] Publishing Cobertura coverage report...
[2020-08-04T22:11:18.553Z] 
[2020-08-04T22:11:18.952Z] [Cobertura] Publishing Cobertura coverage results...
[2020-08-04T22:11:18.952Z] 
[2020-08-04T22:11:18.958Z] [Cobertura] Cobertura coverage report found.
[2020-08-04T22:11:18.958Z] 
[2020-08-04T22:11:20.462Z] Failed in branch Tests
[2020-08-04T22:11:20.530Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2020-08-04T22:11:20.549Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2020-08-04T22:11:20.591Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2020-08-04T22:11:20.652Z] Stage "Release" skipped due to earlier failure(s)
[2020-08-04T22:11:20.667Z] Stage "Release" skipped due to earlier failure(s)
[2020-08-04T22:11:20.697Z] Stage "Release" skipped due to earlier failure(s)
[2020-08-04T22:11:20.729Z] Stage "Release" skipped due to earlier failure(s)
[2020-08-04T22:11:21.275Z] Running on Jenkins in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-832
[2020-08-04T22:11:21.317Z] [INFO] getVaultSecret: Getting secrets
[2020-08-04T22:11:21.392Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-08-04T22:11:22.093Z] + chmod 755 generate-build-data.sh
[2020-08-04T22:11:22.093Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-832/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-832/runs/9 FAILURE 1889026
[2020-08-04T22:11:22.643Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-832/runs/9/steps/?limit=10000 -o steps-info.json
[2020-08-04T22:11:25.871Z] (23) Failed writing body
[2020-08-04T22:11:26.782Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-832/runs/9/tests/?status=FAILED -o tests-errors.json

mikker · 2020-07-20T20:59:19Z

Hi @Anasjabri! This is great, would love to include it. Can you sign the CLA? Otherwise I can't merge.

mikker · 2020-07-20T20:59:44Z

jenkins, please run the tests

Anasjabri · 2020-07-27T16:17:01Z

Hi @Anasjabri! This is great, would love to include it. Can you sign the CLA? Otherwise I can't merge.

@mikker I did sign CLA multiple times. below is the pdf.Don't know why bot is not picking it up
Please_DocuSign_Individual_Contributor_Licen (6).pdf

mikker · 2020-08-03T11:36:57Z

The CLA machine sees you signed with yournamemail @ gmail – is that email also registered on you GH profile?

Anasjabri · 2020-08-04T18:35:42Z

The CLA machine sees you signed with yournamemail @ gmail – is that email also registered on you GH profile?

Yes

gtback · 2020-08-04T21:23:51Z

Hi @Anasjabri, thanks for contributing!

It looks like the first commit (0d8513b) was made using a different email address:

Author: Anas AlJabri <[email protected]>

If you can do any of the following:

Sign another CLA with that address,
Rebase that commit and modify the author (this StackOverflow answer has some directions)
Just squash all the commits into a single change with your Gmail address.
That should be enough to make our robot happy!

mikker · 2020-08-04T21:39:25Z

jenkins, please run the tests

mikker · 2020-08-05T09:45:49Z

Test failure is unrelated.

mikker · 2020-08-05T09:46:43Z

Thank you for contributing to the agent, @Anasjabri 💙💚💛💜❤️

Anasjabri force-pushed the 831 branch from af24ab5 to 0d8513b Compare July 20, 2020 16:29

Anasjabri marked this pull request as ready for review July 20, 2020 16:40

Scrub request body content

7900400

Anasjabri force-pushed the 831 branch from 8e6f5de to 7900400 Compare August 4, 2020 21:29

mikker merged commit b79d2d6 into elastic:master Aug 5, 2020

mikker self-assigned this Aug 6, 2020

mikker added the [zube]: Done label Aug 6, 2020

mikker added this to the 7.9 milestone Aug 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scrub request body contents to remove illegal UTF-8 characters #832

Scrub request body contents to remove illegal UTF-8 characters #832

Anasjabri commented Jul 14, 2020 •

edited

Loading

cla-checker-service bot commented Jul 14, 2020 •

edited

Loading

apmmachine commented Jul 14, 2020 •

edited

Loading

Build stats

Test stats 🧪

mikker commented Jul 20, 2020 •

edited

Loading

mikker commented Jul 20, 2020

Anasjabri commented Jul 27, 2020 •

edited

Loading

mikker commented Aug 3, 2020 •

edited

Loading

Anasjabri commented Aug 4, 2020 •

edited

Loading

gtback commented Aug 4, 2020

mikker commented Aug 4, 2020

mikker commented Aug 5, 2020

mikker commented Aug 5, 2020

Scrub request body contents to remove illegal UTF-8 characters #832

Scrub request body contents to remove illegal UTF-8 characters #832

Conversation

Anasjabri commented Jul 14, 2020 • edited Loading

What does this pull request do?

Why is it important?

Checklist

Related issues

cla-checker-service bot commented Jul 14, 2020 • edited Loading

apmmachine commented Jul 14, 2020 • edited Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Steps errors

Log output

mikker commented Jul 20, 2020 • edited Loading

mikker commented Jul 20, 2020

Anasjabri commented Jul 27, 2020 • edited Loading

mikker commented Aug 3, 2020 • edited Loading

Anasjabri commented Aug 4, 2020 • edited Loading

gtback commented Aug 4, 2020

mikker commented Aug 4, 2020

mikker commented Aug 5, 2020

mikker commented Aug 5, 2020

Anasjabri commented Jul 14, 2020 •

edited

Loading

cla-checker-service bot commented Jul 14, 2020 •

edited

Loading

apmmachine commented Jul 14, 2020 •

edited

Loading

mikker commented Jul 20, 2020 •

edited

Loading

Anasjabri commented Jul 27, 2020 •

edited

Loading

mikker commented Aug 3, 2020 •

edited

Loading

Anasjabri commented Aug 4, 2020 •

edited

Loading