Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix self metrics when containerized (backport to 6.2) #6930

Closed
wants to merge 139 commits into from

Conversation

jsoriano
Copy link
Member

Backport of #6641 to 6.2

tsg and others added 30 commits July 24, 2017 22:10
…to_6.x_branch

Automatic merge from master to 6.x branch
…to_6.x_branch

Automatic merge from master to 6.x branch
…er_to_6.x_branch

Automatic merge from master to 6.x branch
This adds the scripts that are executed by Jenkins to this repo. They
need to be added to other branches and kept in sync.

(cherry picked from commit 5ecba0f)
…er_to_6.x_branch

Automatic merge from master to 6.x branch
Automatic merge from master to 6.x branch
…to_6.x_branch

Automatic merge from master to 6.x branch
…to_6.x_branch

Automatic merge from master to 6.x branch
This will be the next release from this branch. Since the branch is not released,
we can already set the docs version as well.
- fix reference config: always_kibana -> setup.dashboards.always_kibana
- pass the beat its hostname instead of configured beat name (which by
  chance is the hostname by default)
- if always_kibana is set to true, don't check elasticsearch, but try to
  install via kibana only
- only reuse elasticsearch username/password if elasticsearch output is
  enabled

(cherry picked from commit 554701f)
There is no need to set explicitly `frequency` and `query_delay`
parameters for ML datafeeds. The current values are the default
values but a change in the way datafeeds with aggregations work
makes those frequencies invalid settings. Also, in the future the
smart defaults might evolve, so using them will leverage such
improvements.
…to_6.x_branch

Automatic merge from master to 6.x branch
…to_6.x_branch

Automatic merge from master to 6.x branch
In elastic#5992 I tried to change back some of the geoip lookup fields. But it seems the changes are also in 6.2 if the cache is invalidated. Trying to only invalidate the cache instead.
kvch and others added 28 commits March 27, 2018 09:06
* libbeat: build internal metrics on windows and linux without cgo

* libbeat: add ephemeral id even if no internal memory metrics is reported

* fix incorrect build tags && create common file for all platforms
Instead of using a single large blob of random characters use a random set of words to be more like a log message.

(cherry picked from commit 197008e)
Remove `auditbeat.modules:` from config reload example file. And change Glob to glob.

(cherry picked from commit 5426852)
Cherry-pick elastic#6690 to 6.2: Use random sentence instead of random characters in test
On Linux, kernel processes don't have an exe on disk so the `/proc/[PID]/exe` symlink is broken. When Metricbeat tries to read the symlink it gets an error and it skips the whole process. This adds a check for ENOENT and ignores the error which allows kernel processes to be reported by Metricbeat. This is the error that it fixes.

    2018-01-30T20:31:02.512Z	DEBUG	[processes]	process/process.go:443	Skip process pid=12113: error getting process exe for pid=12113: readlink /proc/12113/exe: no such file or directory

Fixes elastic#5700

(cherry picked from commit 6bde7d8)
Cherry-pick elastic#6224 to 6.2: Fix system process metricset for kernel processes
Similar to the `dev-toosl/set_version` script. At the moment, it
edits the `version.asciidoc` file and runs `make update`.

(cherry picked from commit 2b24805)
…lastic#6724)

This updates go-libaudit to v0.1.0 which contains a fix for elastic/go-libaudit#20.

(cherry picked from commit 3f3fb57)
User reports some HTTP servers may respond with a broken status line that's
missing a space between the status code and the optional status phrase. HTTP
clients tested already support this behavior. This patch adjusts the http
parser so that this deviation from the standard is accepted.

Fixes elastic#6176
A message with a client header consisting on a partial frame (not
all data received for this frame) could result in a panic.
* Fix out of bounds access to slice in MongoDB parser

Ignore MongoDB message and drop the TCP stream if a malformed
query / response is received, instead of logging a panic.

Closes elastic#5188

* Update CHANGELOG
Fixes packetbeat termination problems with both af_packet and pcap
captures.

Fixes elastic#6535
There was a bounds check error in parsing HTTP responses. A malformed
response line in the form "HTTP/1.1\r\n" would cause a panic when parsed.

Related to elastic#6409
When the log harvester initialisation fails (i.e. due to wrong configuration),
it causes a panic instead of reporting the error.

Fixes elastic#6800
use append operations (StringBuilder pattern) instead of fmt.Sprintf to
reduce allocation overhead when generating the id.

The benchmark generates 2500 ids per run. As the ID is a string, one
at least one allocation per ID is required. Results show the change cuts
allocations by 3.

benchmark                  old ns/op     new ns/op     delta
BenchmarkStateString-4     633983        274193        -56.75%
BenchmarkStateString-4     634008        275098        -56.61%
BenchmarkStateString-4     627028        273854        -56.33%
BenchmarkStateString-4     629635        275542        -56.24%
BenchmarkStateString-4     625108        284221        -54.53%
BenchmarkStateString-4     632816        272703        -56.91%
BenchmarkStateString-4     633756        275215        -56.57%
BenchmarkStateString-4     671575        273220        -59.32%
BenchmarkStateString-4     663687        272034        -59.01%
BenchmarkStateString-4     629798        279793        -55.57%

benchmark                  old allocs     new allocs     delta
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%
BenchmarkStateString-4     7400           2500           -66.22%

benchmark                  old bytes     new bytes     delta
BenchmarkStateString-4     157619        118388        -24.89%
BenchmarkStateString-4     157623        118387        -24.89%
BenchmarkStateString-4     157630        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157631        118387        -24.90%
BenchmarkStateString-4     157630        118387        -24.90%

(cherry picked from commit 42c0ef6)
…ic#6804)

* Filebeat in memory states registry improvements

- make States type easier to find by moving into separate states.go file
- Ensure provides States constructor is actually used
- Add ID->array index, index for faster lookups on update and find
  operations. When updating states in a big registry, the Updates
  converged to quadratic complexity. The index helps in keeping the
  complexity about linear in number of state updates.
- Debug will print number of states subject to future cleanups (if state TTL > 0)
- Add title to states unit tests

* Fix godocs

* update state index in Cleanup

* Refine index handling + reintroduce debug message

* review

(cherry picked from commit 490cbcd)
- Defer registrar state gc until the registry needs to be written.
- when updating regitrar states from batch, ensure all updated states will
have same timestamp.

Depends on elastic#6346

(cherry picked from commit 2d2400b)
* Use specific beat name's instead of 'the Beat'
* Dont show Logstash info unless it's supported.
* Make the index pattern decoupled from the beat name
* Make it possible to skip the pipeline docs in output.elasticsearch
* Introduce beat_default_index_prefix.
* Use name of beat in shared-kibana-config.asciidoc
* Add .\ for PS instruction and make it possible to remove logstash mention.
* Introduce 'has_ml_jobs'
* Added `html_docs` to .gitignore
* Better outputconfig for Elasticsearch only beats.
* Less blamy wording
* Only talk about Filebeat for filebeat docs
* Update the rest of the max_retries sections to only talk about Filebeat for filebeat
* Special case for apm-server as it was only introduced in 6.0
* expand beat_default_index_prefix use (#2)
Minor adaptions for getting-help and
command-reference sections.
@jsoriano
Copy link
Member Author

Wrong pull request

@jsoriano jsoriano closed this Apr 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.