Affecting all Beats
-
Change Elasticsearch output index configuration to be based on format strings. If index has been configured, no date will be appended anymore to the index name. 2119
-
Replace
output.kafka.use_type
byoutput.kafka.topic
accepting a format string. 2188 -
If the path specified by the
-c
flag is not absolute and-path.config
is not specified, it is considered relative to the current working directory. 2245 -
rename
tls
configurations section tossl
. 2330 -
rename
certificate_key
configuration tokey
. 2330 -
replace
tls.insecure
withssl.verification_mode
setting. 2330 -
replace
tls.min/max_version
withssl.supported_protocols
setting requiring full protocol name. 2330
Metricbeat
-
Change field type system.process.cpu.start_time from keyword to date. 1565
-
redis/info metricset fields were renamed up according to the naming conventions.
Packetbeat
Filebeat
-
Set close_inactive default to 5 minutes (was 1 hour before)
-
Set clean_removed and close_removed to true by default
Affecting all Beats
-
Fix logstash output handles error twice when asynchronous sending fails. 2441
-
Fix Elasticsearch structured error response parsing error. 2229
-
Fixed the run script to allow the overriding of the configuration file. 2171
-
Fix logstash output crash if no hosts are configured. 2325
-
Fix array value support in -E CLI flag. 2521
-
Fix merging array values if -c CLI flag is used multiple times. 2521
-
Fix beats failing to start due to invalid duplicate key error in configuration file. 2521
-
Fix panic on non writable logging directory. 2571
Metricbeat
-
Fix module filters to work properly with drop_event filter. 2249
Packetbeat
Filebeat
Winlogbeat
-
Fix corrupt registry file that occurs on power loss by disabling file write caching. 2313
Affecting all Beats
-
Add script to generate the Kibana index-pattern from fields.yml. 2122
-
Enhance redis output key selection based on format string. 2169
-
Configurable redis
keys
using filters and format strings. 2169 -
Add format string support to
output.kafka.topic
. 2188 -
Add
output.kafka.topics
for more advanced kafka topic selection per event. 2188 -
Add support for kafka 0.10.
-
Add support for kafka 0.10. 2190
-
Add SASL/PLAIN authentication support to kafka output. 2190
-
Make Kafka metadata update configurable. 2190
-
Add kafka version setting (optional) enabling kafka broker version support. 2190
-
Add kafka message timestamp if at least version 0.10 is configured. 2190
-
Add configurable kafka event key setting. 2284
-
Add settings for configuring the kafka partitioning strategy. 2284
-
Add partitioner settings
reachable_only
to ignore partitions not reachable by network. 2284 -
Enhance contains condition to work on fields that are arrays of strings. 2237
-
Lookup the configuration file relative to the
-path.config
CLI flag. 2245 -
Re-write import_dashboards.sh in Golang. 2155
-
Update to Go 1.7. 2306
-
Log total non-zero internal metrics on shutdown. 2349
-
Add support for encrypted private key files by introducing
ssl.key_passphrase
setting. 2330 -
Add experimental symlink support with
symlinks
config 2478
Metricbeat
Packetbeat
Filebeat
Affecting all Beats
-
Rename the
filters
section toprocessors
. 1944 -
Introduce the condition with
when
in the processor configuration. 1949 -
The Elasticsearch template is now loaded by default. 1993
-
The Redis output
index
setting is renamed tokey
.index
still works but it’s deprecated. 2077 -
The undocumented file output
index
setting was removed. Usefilename
instead. 2077
Metricbeat
Packetbeat
-
Set
enabled
` inpacketbeat.protocols.icmp
configuration totrue
by default. 1988
Affecting all Beats
-
Fix sync publisher
PublishEvents
return value if client is closed concurrently. 2046
Metricbeat
-
Do not send zero values when no value was present in the source. 1972
Filebeat
Winlogbeat
-
Fix potential data loss between Winlogbeat restarts, reporting unpublished lines as published. 2041
Affecting all Beats
-
Periodically log internal metrics. 1955
-
Add enabled setting to all output modules. 1987
-
Command line flag
-c
can be used multiple times. 1985 -
Add OR/AND/NOT to the condition associated with the processors. 1983
-
Add
-E
CLI flag for overwriting single config options via command line. 1986 -
Choose the mapping template file based on the Elasticsearch version. 1993
-
Check stdout being available when console output is configured. 2035
Metricbeat
-
Add pgid field to process information. https://github.com/elastic/beats/pull/ 2021[2021]
Packetbeat
Filebeat
Affecting all Beats
-
The topology_expire option of the Elasticserach output was removed. 1907
Filebeat
-
Stop following symlink. Symlinks are now ignored: 1686
Affecting all Beats
-
Reset backoff factor on partial ACK. 1803
-
Fix beats load balancer deadlock if max_retries: -1 or publish_async is enabled in filebeat. 1829
-
Fix logstash output with pipelining mode enabled not reconnecting. 1876
-
Empty configuration sections become merge-able with variables containing full path. 1900
-
Fix error message about required fields missing not printing the missing field name. 1900
Metricbeat
-
Fix the CPU values returned for each core. 1863
Packetbeat
Winlogbeat
-
Fix issue with rendering forwarded event log records. 1891
Affecting all Beats
-
All configuration settings under
shipper:
are moved to be top level configuration settings. I.e.shipper.name:
becomesname:
in the configuration file. 1570
Topbeat
-
Topbeat is replaced by Metricbeat.
Filebeat
-
The state for files which fall under ignore_older is not stored anymore. This has the consequence, that if a file which fell under ignore_older is updated, the whole file will be crawled.
Affecting all Beats
-
Add conditions to generic filtering. 1623
Metricbeat
-
First public release, containing the following modules: apache, mysql, nginx, redis, system, and zookeeper.
Filebeat
-
The registry format was changed to an array instead of dict. The migration to the new format will happen automatically at the first startup. 1703
Affecting all Beats
-
The support for doing GeoIP lookups is deprecated and will be removed in version 6.0. 1601
Affecting all Beats
Packetbeat
Filebeat
-
Default location for the registry file was changed to be
data/registry
from the binary directory, rather than.filebeat
in the current working directory. This affects installations for zip/tar.gz/source, the location for DEB and RPM packages stays the same. 1373
Affecting all Beats
-
Drain response buffers when pipelining is used by Redis output. 1353
-
Unterminated environment variable expressions in config files will now cause an error 1389
-
Fix issue with the automatic template loading when Elasticsearch is not available on Beat start. 1321
-
Fix bug affecting -cpuprofile, -memprofile, and -httpprof CLI flags 1415
-
Fix race when multiple outputs access the same event with logstash output manipulating event 1410 1428
-
Seed random number generator using crypto.rand package. https://github.com/elastic/beats/pull/1503{1503]
-
Fix beats hanging in -configtest 1213
-
Fix kafka log message output 1516
Filebeat
-
Improvements in registrar dealing with file rotation. 1281
-
Fix issue with JSON decoding where
@timestamp
ortype
keys with the wrong type could cause Filebeat to crash. 1378 -
Fix issue with JSON decoding where values having
null
as values could crash Filebeat. 1466 -
Multiline reader normalizing newline to use
\n
. 1552
Winlogbeat
Affecting all Beats
-
Add support for TLS to Redis output. 1353
-
Add SOCKS5 proxy support to Redis output. 1353
-
Failover and load balancing support in redis output. 1353
-
Multiple-worker per host support for redis output. 1353
-
Added ability to escape
${x}
in config files to avoid environment variable expansion 1389 -
Configuration options and CLI flags for setting the home, data and config paths. 1373
-
Configuration options and CLI flags for setting the default logs path. 1437
-
Update to Go 1.6.2 1447
-
Add Elasticsearch template files compatible with Elasticsearch 2.x. 1501
-
Add scripts for managing the dashboards of a single Beat 1359
Packetbeat
-
Fix compile issues for OpenBSD. 1347
Topbeat
-
Updated elastic/gosigar version so Topbeat can compile on OpenBSD. 1403
Affecting all Beats
Topbeat
libbeat
Packetbeat
-
Rename output fields in the dns package. Former flag
recursion_allowed
becomesrecursion_available
. 803 Former SOA fieldttl
becomesminimum
. 803 -
The fully qualified domain names which are part of output fields values of the dns package now terminate with a dot. 803
-
Remove the count field from the exported event 1210
Topbeat
Filebeat
Winlogbeat
Affecting all Beats
-
Logstash output will not retry events that are not JSON-encodable 927
Packetbeat
Topbeat
-
Fix issue with
cpu.system_p
being greater than 1 on Windows 1128
Filebeat
Winlogbeat
Affecting all Beats
-
Update builds to Golang version 1.6
-
Add option to Elasticsearch output to pass http parameters in index operations 805
-
Improve Logstash and Elasticsearch backoff behavior. 927
-
Add experimental Kafka output. 942
-
Add config file option to configure GOMAXPROCS. 969
-
Improve shutdown handling in libbeat. 1075
-
Add
fields
andfields_under_root
options under theshipper
configuration 1092 -
Add the ability to use a SOCKS5 proxy with the Logstash output 823
-
The
-configtest
flag will now print "Config OK" to stdout on success 1249
Packetbeat
Topbeat
-
Add
username
to processes 845
Filebeat
Winlogbeat
-
Add caching of event metadata handles and the system render context for the wineventlog API 888
-
Improve config validation by checking for unknown top-level YAML keys. 1100
-
Add the ability to set tags, fields, and fields_under_root as options for each event log 1092
-
Add additional data to the events published by Winlogbeat. The new fields are
activity_id
,event_data
,keywords
,opcode
,process_id
,provider_guid
,related_activity_id
,task
,thread_id
,user_data
, andversion
. 1053 -
Add
event_id
,level
, andprovider
configuration options for filtering events 1218 -
Add
include_xml
configuration option for including the raw XML with the event 1218
-
All Beats can hang or panic on shutdown if the next server in the pipeline (e.g. Elasticsearch or Logstash) is not reachable. 1319
-
When running the Beats as a service on Windows, you need to manually load the Elasticsearch mapping template. 1315
-
The ES template automatic load doesn’t work if Elasticsearch is not available when the Beat is starting. 1321
Filebeat
-
Default config for ignore_older is now infinite instead of 24h, means ignore_older is disabled by default. Use close_older to only close file handlers.
Packetbeat
-
Split real_ip_header value when it contains multiple IPs 1241
Winlogbeat
-
Fix invalid
event_id
on Windows XP and Windows 2003 1227
Affecting all Beats
Packetbeat
-
Fix setting direction to out and use its value to decide when dropping events if ignore_outgoing is enabled 557
-
Fix logging issue with file-based output where newlines could be misplaced during concurrent logging 650
-
Reduce memory usage by having separate queue sizes for single events and bulk events. 649 516
-
Set default bulk_max_size value to 2048 628
-
Fix logstash window size of 1 not increasing. 598
Packetbeat
Filebeat
-
Set spool_size default value to 2048 628
Affecting all Beats
Packetbeat
Topbeat
-
Group all CPU usage per core statistics and export them optionally if cpu_per_core is configured 496
Filebeat
Winlogbeat
-
First public release of Winlogbeat
Filebeat
-
Fix force_close_files in case renamed file appeared very fast. 302
Packetbeat
-
Improve MongoDB message correlation. 377
-
Improve redis parser performance. 422
-
Fix panic on nil in redis protocol parser. 384
-
Fix errors redis parser when messages are split in multiple TCP segments. 402
-
Fix errors in redis parser when length prefixed strings contain sequences of CRLF. 402
-
Fix errors in redis parser when dealing with nested arrays. 402
Affecting all Beats
-
Fix random panic on shutdown by calling shutdown handler only once. elastic/filebeat#204
-
Fix credentials are not send when pinging an elasticsearch host. elastic/fileabeat#287
Filebeat
-
Fix problem that harvesters stopped reading after some time and filebeat stopped processing events #257
-
Fix line truncating by internal buffers being reused by accident #258
-
Set default ignore_older to 24 hours #282
Affecting all Beats
-
The
shipper
output field is renamed tobeat.name
. #285 -
Use of
enabled
as a configuration option for outputs (elasticsearch, logstash, etc.) has been removed. #264 -
Use of
disabled
as a configuration option for tls has been removed. #264 -
The
-test
command line flag was renamed to-configtest
. #264 -
Disable geoip by default. To enable it uncomment in config file. #305
Filebeat
-
Removed utf-16be-bom encoding support. Support will be added with fix for #205
-
Rename force_close_windows_files to force_close_files and make it available for all platforms.
Affecting all Beats
-
Disable logging to stderr after configuration phase. #276
-
Set the default file logging path when not set in config. #275
-
Fix bug silently dropping records based on current window size. elastic/filebeat#226
-
Fix direction field in published events. #300
-
Fix elasticsearch structured errors breaking error handling. #309
Packetbeat
-
Packetbeat will now exit if a configuration error is detected. #357
-
Fixed an issue handling DNS requests containing no questions. #369
Topbeat
-
Fix leak of Windows handles. #98
-
Fix memory leak of process information. #104
Filebeat
-
Filebeat will now exit if a configuration error is detected. #198
-
Fix to enable prospector to harvest existing files that are modified. #199
-
Improve line reading and encoding to better keep track of file offsets based on encoding. #224
-
Set input_type by default to "log"
Affecting all Beats
-
Rename timestamp field with @timestamp. #237
Packetbeat
-
Rename timestamp field with @timestamp. #343
Topbeat
-
Rename timestamp field with @timestamp for a better integration with Logstash. #80
Filebeat
-
Rename the timestamp field with @timestamp #168
-
Rename tail_on_rotate prospector config to tail_files
-
Removal of line field in event. Line number was not correct and does not add value. #217
Affecting all Beats
-
Use stderr for console log output. #219
-
Handle empty event array in publisher. #207
-
Respect '*' debug selector in IsDebug. #226 (#339)
-
Limit number of workers for Elasticsearch output. #226
-
On Windows, remove service related error message when running in the console. #242
-
Fix waitRetry no configured in single output mode configuration. elastic/filebeat#144
-
Use http as the default scheme in the elasticsearch hosts #253
-
Respect max bulk size if bulk publisher (collector) is disabled or sync flag is set.
-
Always evaluate status code from Elasticsearch responses when indexing events. #192
-
Use bulk_max_size configuration option instead of bulk_size. #256
-
Fix max_retries=0 (no retries) configuration option. #266
-
Filename used for file based logging now defaults to beat name. #267
Packetbeat
-
Close file descriptors used to monitor processes. #337
-
Remove old RPM spec file. It moved to elastic/beats-packer. #334
Topbeat
-
Don’t wait for one period until shutdown #75
Filebeat
-
Omit 'fields' from event JSON when null. #126
-
Make offset and line value of type long in elasticsearch template to prevent overflow. #140
-
Fix locking files for writing behaviour. #156
-
Introduce 'document_type' config option per prospector to define document type for event stored in elasticsearch. #133
-
Add 'input_type' field to published events reporting the prospector type being used. #133
-
Fix high CPU usage when not connected to Elasticsearch or Logstash. #144
-
Fix issue that files were not crawled anymore when encoding was set to something other then plain. #182
Affecting all Beats
-
Add Console output plugin. #218
-
Add timestamp to log messages #245
-
Send @metadata.beat to Logstash instead of @metadata.index to prevent possible name clashes and give user full control over index name used for Elasticsearch
-
Add logging messages for bulk publishing in case of error #229
-
Add option to configure number of parallel workers publishing to Elasticsearch or Logstash.
-
Set default bulk size for Elasticsearch output to 50.
-
Set default http timeout for Elasticsearch to 90s.
-
Improve publish retry if sync flag is set by retrying only up to max bulk size events instead of all events to be published.
Filebeat
-
Introduction of backoff, backoff_factor, max_backoff, partial_line_waiting, force_close_windows_files config variables to make crawling more configurable.
-
All Godeps dependencies were updated to master on 2015-10-21 [#122]
-
Set default value for ignore_older config to 10 minutes. #164
-
Added the fields_under_root setting to optionally store the custom fields top level in the output dictionary. #188
-
Add more encodings by using x/text/encodings/htmlindex package to select encoding by name.
Affecting all Beats
-
Update tls config options naming from dash to underline #162
-
Feature/output modes: Introduction of PublishEvent(s) to be used by beats #118 #115
Packetbeat
-
Renamed http module config file option 'strip_authorization' to 'redact_authorization'
-
Save_topology is set to false by default
-
Rename elasticsearch index to [packetbeat-]YYYY.MM.DD
Topbeat
-
Percentage fields (e.g user_p) are exported as a float between 0 and 1 #34
Affecting all Beats
-
Determine Elasticsearch index for an event based on UTC time #81
-
Fixing ES output’s defaultDeadTimeout so that it is 60 seconds #103
-
ES outputer: fix timestamp conversion #91
-
Fix TLS insecure config option #239
-
ES outputer: check bulk API per item status code for retransmit on failure.
Packetbeat
-
Support for lower-case header names when redacting http authorization headers
-
Redact proxy-authorization if redact-authorization is set
-
Fix some multithreading issues #203
-
Fix negative response time #216
-
Fix memcache TCP connection being nil after dropping stream data. #299
-
Add missing DNS protocol configuration to documentation #269
Topbeat
-
Don’t divide the reported memory by an extra 1024 #60
Affecting all Beats
-
Add logstash output plugin #151
-
Integration tests for Beat → Logstash → Elasticsearch added #195 #188 #168 #137 #128 #112
-
Large updates and improvements to the documentation
-
Add direction field to publisher output to indicate inbound/outbound transactions #150
-
Add tls configuration support to elasticsearch and logstash outputers #139
-
All external dependencies were updated to the latest version. Update to Golang 1.5.1 #162
-
Guarantee ES index is based in UTC time zone #164
-
Cache: optional per element timeout #144
-
Make it possible to set hosts in different ways. #135
-
Expose more TLS config options #124
-
Use the Beat name in the default configuration file path #99
Packetbeat
-
add [.editorconfig file](http://editorconfig.org/)
-
add (experimental/unsupported?) saltstack files
-
Sample config file cleanup
-
Moved common documentation to [libbeat repository](https://github.com/elastic/libbeat)
-
Update build to go 1.5.1
-
Adding device descriptions to the -device output.
-
Generate coverage for system tests
-
Move go-daemon dependency to beats-packer
-
Rename integration tests to system tests
-
Made the
-devices
option more user friendly in casesudo
is not used. Issue #296. -
Publish expired DNS transactions #301
-
Update protocol guide to libbeat changes
-
Add protocol registration to new protocol guide
-
Make transaction timeouts configurable #300
-
Add direction field to the exported fields #317
Topbeat
-
Document fields in a standardized format (etc/fields.yml) #34
-
Updated to use new libbeat Publisher #37 #41
-
Update to go 1.5.1 #43
-
Updated configuration files with comments for all options #65
-
Documentation improvements