Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[build] add MADVDONTNEED to builds #23001

Merged
merged 3 commits into from
Dec 16, 2020
Merged

[build] add MADVDONTNEED to builds #23001

merged 3 commits into from
Dec 16, 2020

Conversation

fearful-symmetry
Copy link
Contributor

What does this PR do?

This adds GODEBUG="madvdontneed=1" to the docker images and init scripts.
Note that I haven't tested this yet as I'm not sure what to check beyond "the env var exists in the final builds"

Why is it important?

See #22786 From the linked issue:

The go runtime defaults to MADV_FREE by default. In that case pages are still assigned with the Beat, even after the pages have been "returned" to the OS. The kernel will eventually claim these pages, but RSS memory usage is reported as very high (which it is not necessarily). This seems to mess with the OOM killer at times.

Checklist

  • My code follows the style guidelines of this project
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@fearful-symmetry fearful-symmetry added enhancement Team:Integrations Label for the Integrations team labels Dec 8, 2020
@fearful-symmetry fearful-symmetry requested a review from a team December 8, 2020 23:30
@fearful-symmetry fearful-symmetry self-assigned this Dec 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 8, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Dec 9, 2020 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #23001 updated

  • Start Time: 2020-12-15T15:54:27.044+0000

  • Duration: 66 min 9 sec

Test stats 🧪

Test Results
Failed 0
Passed 17500
Skipped 1383
Total 18883

Steps errors 2

Expand to view the steps failures

Terraform Apply on x-pack/metricbeat/module/aws
  • Took 0 min 18 sec . View more details on here
Terraform Apply on x-pack/metricbeat/module/aws
  • Took 0 min 16 sec . View more details on here

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 17500
Skipped 1383
Total 18883

@fearful-symmetry
Copy link
Contributor Author

So, we're building into the docker images:

            "Env": [
                "PATH=/usr/share/metricbeat:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "ELASTIC_CONTAINER=true",
                "GODEBUG=madvdontneed=1"
            ],

@fearful-symmetry fearful-symmetry requested a review from urso December 9, 2020 17:39
jsoriano
jsoriano reviewed Dec 14, 2020
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for adding this! Only a comment about the defaults in debian init files.

dev-tools/packaging/templates/deb/elastic-agent.sh.tmpl Outdated Show resolved Hide resolved
dev-tools/packaging/templates/deb/init.sh.tmpl Outdated Show resolved Hide resolved
@fearful-symmetry
Copy link
Contributor Author

@jsoriano thanks, fixed!

jsoriano
jsoriano approved these changes Dec 15, 2020
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@fearful-symmetry fearful-symmetry merged commit 8a68dc9 into elastic:master Dec 16, 2020
fearful-symmetry added a commit to fearful-symmetry/beats that referenced this pull request Dec 16, 2020
@fearful-symmetry
[build] add MADVDONTNEED to builds (elastic#23001)
8437095 
* add MADVDONTNEED to builds

* delay setting godebug in init scripts

(cherry picked from commit 8a68dc9)
@fearful-symmetry fearful-symmetry mentioned this pull request Dec 16, 2020
Merged
3 tasks
fearful-symmetry added a commit to fearful-symmetry/beats that referenced this pull request Dec 16, 2020
@fearful-symmetry
[build] add MADVDONTNEED to builds (elastic#23001)
9b815b6 
* add MADVDONTNEED to builds

* delay setting godebug in init scripts

(cherry picked from commit 8a68dc9)
@fearful-symmetry fearful-symmetry mentioned this pull request Dec 16, 2020
Merged
3 tasks
fearful-symmetry added a commit that referenced this pull request Dec 17, 2020
@fearful-symmetry
[build] add MADVDONTNEED to builds (#23001) (#23176)
f39e45e 
* add MADVDONTNEED to builds

* delay setting godebug in init scripts

(cherry picked from commit 8a68dc9)
fearful-symmetry added a commit that referenced this pull request Dec 17, 2020
@fearful-symmetry
[build] add MADVDONTNEED to builds (#23001) (#23175)
cb12286 
* add MADVDONTNEED to builds

* delay setting godebug in init scripts

(cherry picked from commit 8a68dc9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@urso urso urso approved these changes

@jsoriano jsoriano jsoriano approved these changes

Assignees

@fearful-symmetry fearful-symmetry

Labels
enhancement Team:Integrations Label for the Integrations team v7.11.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@fearful-symmetry @elasticmachine @jsoriano @urso