Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Metricbeat default config to reduce disk space #4329

Merged
merged 1 commit into from
May 29, 2017
Merged

Change Metricbeat default config to reduce disk space #4329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ https://github.com/elastic/beats/compare/v6.0.0-alpha1...master[Check the HEAD d
- Add `kube-state-metrics` based metrics to `kubernetes` module {pull}4253[4253]
- Add debug logging to Jolokia JMX metricset. {pull}4341[4341]
- Add events metricset for kubernetes metricbeat module {pull}4315[4315]
- Change Metricbeat default configuration file to be better optimized for most users. {pull}4329[4329]

*Packetbeat*

Expand Down
1 change: 1 addition & 0 deletions metricbeat/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ collect-docs: python-env
configs: python-env
cat ${ES_BEATS}/metricbeat/_meta/common.yml > _meta/beat.yml
. ${PYTHON_ENV}/bin/activate; python ${ES_BEATS}/script/config_collector.py --beat ${BEAT_NAME} $(PWD) >> _meta/beat.yml
cat ${ES_BEATS}/metricbeat/_meta/setup.yml >> _meta/beat.yml
cat ${ES_BEATS}/metricbeat/_meta/common.full.yml > _meta/beat.full.yml
. ${PYTHON_ENV}/bin/activate; python ${ES_BEATS}/script/config_collector.py --beat ${BEAT_NAME} --full $(PWD) >> _meta/beat.full.yml

Expand Down
5 changes: 5 additions & 0 deletions metricbeat/_meta/setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be the damned default in ES to begin with...

index.codec: best_compression
#_source.enabled: false
37 changes: 11 additions & 26 deletions metricbeat/docs/modules/system.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,45 +86,30 @@ in <<configuration-metricbeat>>. Here is an example configuration:
----
metricbeat.modules:
- module: system
enabled: true
period: 10s
metricsets:
# CPU stats
- cpu

# System Load stats
- load

# Per CPU core stats
- memory
#- core

# IO stats
#- diskio

# Per filesystem stats
- filesystem

# File system summary stats
- fsstat

# Memory stats
- memory

# Network stats
- network

# Processes summary
- process_summary

# Per process stats
- process

# Sockets (linux only)
#- socket
enabled: true
period: 10s
processes: ['.*']
process.include_top_n:
by_cpu: 5 # include top 5 processes by CPU
by_memory: 5 # include top 5 processes by memory
- module: system
enabled: true
period: 1m
metricsets:
- filesystem
- fsstat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really rename this to filbeatsystem_summary ;-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you mean filesystem_summary. I agree with you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol, yes :-)

filters:
- drop_event.when.regexp.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
----

[float]
Expand Down
42 changes: 16 additions & 26 deletions metricbeat/metricbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,47 +12,37 @@ metricbeat.modules:

#------------------------------- System Module -------------------------------
- module: system
enabled: true
period: 10s
metricsets:
# CPU stats
- cpu

# System Load stats
- load

# Per CPU core stats
- memory
#- core

# IO stats
#- diskio

# Per filesystem stats
- filesystem

# File system summary stats
- fsstat

# Memory stats
- memory

# Network stats
- network

# Processes summary
- process_summary

# Per process stats
- process

# Sockets (linux only)
#- socket
enabled: true
period: 10s
processes: ['.*']
process.include_top_n:
by_cpu: 5 # include top 5 processes by CPU
by_memory: 5 # include top 5 processes by memory
- module: system
enabled: true
period: 1m
metricsets:
- filesystem
- fsstat
filters:
- drop_event.when.regexp.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'


#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
#_source.enabled: false

#================================ General =====================================

Expand Down
37 changes: 11 additions & 26 deletions metricbeat/module/system/_meta/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,40 +1,25 @@
- module: system
enabled: true
period: 10s
metricsets:
# CPU stats
- cpu

# System Load stats
- load

# Per CPU core stats
- memory
#- core

# IO stats
#- diskio

# Per filesystem stats
- filesystem

# File system summary stats
- fsstat

# Memory stats
- memory

# Network stats
- network

# Processes summary
- process_summary

# Per process stats
- process

# Sockets (linux only)
#- socket
enabled: true
period: 10s
processes: ['.*']
process.include_top_n:
by_cpu: 5 # include top 5 processes by CPU
by_memory: 5 # include top 5 processes by memory
- module: system
enabled: true
period: 1m
metricsets:
- filesystem
- fsstat
filters:
- drop_event.when.regexp.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
1 change: 1 addition & 0 deletions setup.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#======================== Template options ===============================