Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.14] beater/api/intake: fix rate limiting (backport #5518) #5667

Closed
wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jul 9, 2021

This is an automatic backport of pull request #5518 done by Mergify.
Cherry-pick of 97a6f9b has failed:

On branch mergify/bp/7.14/pr-5518
Your branch is up to date with 'origin/7.14'.

You are currently cherry-picking commit 97a6f9b8.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   beater/api/intake/handler.go
	both modified:   beater/api/intake/handler_test.go
	both modified:   systemtest/rum_test.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

* beater/api/intake: fix rate limiting

Copy the batch processor in the closure before
copying, to avoid wrapping it multiple times.

* systemtest: fix rate limiting test

The test was wrong, and only passed because of
the bug that has been fixed. We should be using
up the per-IP event limits on the first two
requests, but weren't taking into account the
burst multiplier (limit x 3).

(cherry picked from commit 97a6f9b)

# Conflicts:
#	beater/api/intake/handler.go
#	beater/api/intake/handler_test.go
#	systemtest/rum_test.go
@mergify mergify bot added the conflicts There is a conflict in the backported pull request label Jul 9, 2021
@mergify mergify bot assigned axw Jul 9, 2021
@apmmachine
Copy link
Contributor

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-07-09T19:19:23.238+0000

  • Duration: 6 min 38 sec

  • Commit: f96efa4

Trends 🧪

Image of Build Times

Steps errors 1

Expand to view the steps failures

Run intake
  • Took 2 min 0 sec . View more details on here
  • Description: ./.ci/scripts/intake.sh

Log output

Expand to view the last 100 lines of log output

[2021-07-09T19:24:37.910Z] 2021/07/09 19:24:37 internal_metrics
[2021-07-09T19:24:37.910Z] 2021/07/09 19:24:37 profile_metrics
[2021-07-09T19:24:37.910Z] 2021/07/09 19:24:37 traces
[2021-07-09T19:24:37.910Z] 2021/07/09 19:24:37 app_metrics
[2021-07-09T19:24:37.910Z] 2021/07/09 19:24:37 Package fields and docs generated for version 0.3.0
[2021-07-09T19:24:37.910Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667/.gvm/versions/go1.16.5.linux.amd64/bin/go build -o build/linux/elastic-package github.com/elastic/elastic-package
[2021-07-09T19:24:37.910Z] go: downloading github.com/elastic/elastic-package v0.0.0-20210310173719-3b8f76516ae3
[2021-07-09T19:24:37.910Z] go: downloading github.com/AlecAivazis/survey/v2 v2.1.1
[2021-07-09T19:24:37.910Z] go: downloading github.com/elastic/package-spec/code/go v0.0.0-20210310145052-baee4c3da9cc
[2021-07-09T19:24:38.168Z] go: downloading github.com/google/go-github/v32 v32.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/Masterminds/semver v1.5.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/go-git/go-billy/v5 v5.0.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/go-git/go-git/v5 v5.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/jedib0t/go-pretty v4.3.0+incompatible
[2021-07-09T19:24:38.168Z] go: downloading github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
[2021-07-09T19:24:38.168Z] go: downloading github.com/google/go-querystring v1.0.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/mitchellh/go-homedir v1.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/kylelemons/godebug v1.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/aymerick/raymond v2.0.2+incompatible
[2021-07-09T19:24:38.168Z] go: downloading github.com/sergi/go-diff v1.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d
[2021-07-09T19:24:38.168Z] go: downloading github.com/Masterminds/semver/v3 v3.1.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/creasty/defaults v1.5.1
[2021-07-09T19:24:38.168Z] go: downloading github.com/rakyll/statik v0.1.7
[2021-07-09T19:24:38.168Z] go: downloading github.com/xeipuuv/gojsonschema v1.2.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/go-git/gcfg v1.5.0
[2021-07-09T19:24:38.168Z] go: downloading github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
[2021-07-09T19:24:38.168Z] go: downloading github.com/emirpasic/gods v1.12.0
[2021-07-09T19:24:38.427Z] go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
[2021-07-09T19:24:38.427Z] go: downloading gopkg.in/warnings.v0 v0.1.2
[2021-07-09T19:24:38.427Z] go: downloading github.com/kevinburke/ssh_config v0.0.0-20190725054713-01f96b0aa0cd
[2021-07-09T19:24:38.427Z] go: downloading github.com/xanzy/ssh-agent v0.2.1
[2021-07-09T19:24:38.427Z] go: downloading github.com/go-openapi/strfmt v0.20.1
[2021-07-09T19:24:38.427Z] go: downloading github.com/mattn/go-runewidth v0.0.9
[2021-07-09T19:24:38.427Z] go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb
[2021-07-09T19:24:38.427Z] go: downloading github.com/oklog/ulid v1.3.1
[2021-07-09T19:24:38.427Z] go: downloading github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d
[2021-07-09T19:24:38.427Z] go: downloading github.com/go-openapi/errors v0.20.0
[2021-07-09T19:24:38.427Z] go: downloading go.mongodb.org/mongo-driver v1.5.2
[2021-07-09T19:24:38.995Z] go: downloading github.com/go-stack/stack v1.8.0
[2021-07-09T19:24:51.221Z] elastic-package has been installed.
[2021-07-09T19:24:51.221Z] Format the package
[2021-07-09T19:24:51.221Z] Done
[2021-07-09T19:24:51.221Z] Build the package
[2021-07-09T19:24:51.221Z] Package built: /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667/src/github.com/elastic/apm-server/build/integrations/apm/0.3.0
[2021-07-09T19:24:51.221Z] Done
[2021-07-09T19:24:51.221Z] 2021/07/09 19:24:50 exec: go list -m
[2021-07-09T19:24:51.221Z] >> Building apm-server.yml for linux/amd64
[2021-07-09T19:24:51.221Z] >> Building apm-server.docker.yml for linux/amd64
[2021-07-09T19:24:51.221Z] Generated fields.yml for apm-server to /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667/src/github.com/elastic/apm-server/build/fields/fields.yml
[2021-07-09T19:24:51.790Z] Generated fields.yml for apm-server to /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667/src/github.com/elastic/apm-server/build/fields/fields.all.yml
[2021-07-09T19:24:54.324Z] /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667/.gvm/versions/go1.16.5.linux.amd64/bin/go build -o build/linux/goimports golang.org/x/tools/cmd/goimports
[2021-07-09T19:24:54.892Z] beater/api/intake/handler.go:79:1: expected statement, found '<<'
[2021-07-09T19:24:54.893Z] beater/api/intake/handler.go:91:1: expected statement, found '>>'
[2021-07-09T19:24:54.893Z] beater/api/intake/handler.go:91:57: illegal character U+0023 '#'
[2021-07-09T19:24:54.893Z] beater/api/intake/handler_test.go:156:1: expected declaration, found '<<'
[2021-07-09T19:24:54.893Z] beater/api/intake/handler_test.go:165:2: expected declaration, found 'for'
[2021-07-09T19:24:54.893Z] beater/api/intake/handler_test.go:191:4: expected declaration, found tc
[2021-07-09T19:24:54.893Z] beater/api/intake/handler_test.go:245:57: illegal character U+0023 '#'
[2021-07-09T19:24:55.460Z] systemtest/rum_test.go:165:1: expected statement, found '<<'
[2021-07-09T19:24:55.460Z] systemtest/rum_test.go:175:1: expected statement, found '=='
[2021-07-09T19:24:55.460Z] systemtest/rum_test.go:184:1: expected statement, found '>>'
[2021-07-09T19:24:55.460Z] systemtest/rum_test.go:184:57: illegal character U+0023 '#'
[2021-07-09T19:24:55.460Z] systemtest/rum_test.go:190:3: expected '}', found 'EOF'
[2021-07-09T19:24:56.396Z] Makefile:251: recipe for target 'check-gofmt' failed
[2021-07-09T19:24:56.396Z] make: *** [check-gofmt] Error 2
[2021-07-09T19:24:56.396Z] + cleanup
[2021-07-09T19:24:56.396Z] + rm -rf /tmp/tmp.dHwbsSNDMM
[2021-07-09T19:24:57.860Z] Stage "Build and Test" skipped due to earlier failure(s)
[2021-07-09T19:24:58.303Z] Stage "linux build" skipped due to earlier failure(s)
[2021-07-09T19:24:58.319Z] Stage "windows build-test" skipped due to earlier failure(s)
[2021-07-09T19:24:58.334Z] Stage "OSX build-test" skipped due to earlier failure(s)
[2021-07-09T19:24:58.348Z] Stage "Unit Test" skipped due to earlier failure(s)
[2021-07-09T19:24:58.362Z] Stage "System and Environment Tests" skipped due to earlier failure(s)
[2021-07-09T19:24:58.375Z] Stage "Benchmarking" skipped due to earlier failure(s)
[2021-07-09T19:24:58.388Z] Stage "Check kibana Obj. Updated" skipped due to earlier failure(s)
[2021-07-09T19:24:58.401Z] Stage "Hey-Apm" skipped due to earlier failure(s)
[2021-07-09T19:24:58.415Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-09T19:24:58.422Z] Stage "APM Integration Tests" skipped due to earlier failure(s)
[2021-07-09T19:24:58.508Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-09T19:24:58.882Z] Failed in branch linux build
[2021-07-09T19:24:58.893Z] Failed in branch windows build-test
[2021-07-09T19:24:58.903Z] Failed in branch OSX build-test
[2021-07-09T19:24:58.912Z] Failed in branch Unit Test
[2021-07-09T19:24:58.922Z] Failed in branch System and Environment Tests
[2021-07-09T19:24:58.932Z] Failed in branch Benchmarking
[2021-07-09T19:24:58.944Z] Failed in branch Check kibana Obj. Updated
[2021-07-09T19:24:58.955Z] Failed in branch Hey-Apm
[2021-07-09T19:24:58.965Z] Failed in branch APM Integration Tests
[2021-07-09T19:24:59.072Z] Stage "Package" skipped due to earlier failure(s)
[2021-07-09T19:24:59.309Z] Failed in branch Package
[2021-07-09T19:25:00.063Z] Running on Jenkins in /var/lib/jenkins/workspace/pm-server_apm-server-mbp_PR-5667
[2021-07-09T19:25:00.223Z] [INFO] getVaultSecret: Getting secrets
[2021-07-09T19:25:00.289Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-07-09T19:25:01.446Z] + chmod 755 generate-build-data.sh
[2021-07-09T19:25:01.446Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5667/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5667/runs/1 FAILURE 337932
[2021-07-09T19:25:01.446Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5667/runs/1/steps/?limit=10000 -o steps-info.json
[2021-07-09T19:25:01.446Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-server/apm-server-mbp/PR-5667/runs/1/tests/?status=FAILED -o tests-errors.json
[2021-07-09T19:25:01.697Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-07-09T19:25:02.617Z] Retry 2/3 exited 22, retrying in 2 seconds...

@simitt simitt closed this Jul 12, 2021
@mergify mergify bot deleted the mergify/bp/7.14/pr-5518 branch July 12, 2021 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conflicts There is a conflict in the backported pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants