Skip to content

Commit

Permalink
Forward port 8.0.0 changelog to main (#30265)
Browse files Browse the repository at this point in the history 
* Forward port 8.0.0 changelog to 8.1 (#30264)

* docs: Prepare Changelog for 8.0.0 (#30215)

* docs: Close changelog for 8.0.0

* Initial cleanup

* We no longer ship journalbeat

* Fix duplicate entry

* Add missing breaking changes

* Review comments

* Remove empty sections

Co-authored-by: Andres Rodriguez <[email protected]>
(cherry picked from commit 545f381)

* Additional cleanup

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 7456bdd)

* Additional cleanup
  • Loading branch information
@andresrc
andresrc authored Feb 8, 2022
1 parent 478a5ce commit 68872a0
Show file tree
Hide file tree
Showing 3 changed files with 114 additions and 91 deletions.
113 changes: 98 additions & 15 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,113 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.0.0-rc2]]
=== Beats version 8.0.0-rc2
[[release-notes-8.0.0]]
=== Beats version 8.0.0
https://github.com/elastic/beats/compare/v7.17.0...v8.0.0[View commits]

Changes will be described in the GA release.
==== Breaking changes

*Affecting all Beats*

- Remove the deprecated `xpack.monitoring.*` settings. Going forward only `monitoring.*` settings may be used. {issue}9424[9424] {pull}18608[18608]
- Remove deprecated/undocumented IncludeCreatorMetadata setting from kubernetes metadata config options {pull}28006[28006]
- Remove deprecated fields from kubernetes module {pull}28046[28046]
- Remove deprecated config option aws_partition. {pull}28120[28120]
- Improve stats API by adding host metadata. {pull}27963[27963]
- Libbeat: logp package forces ECS compliant logs. Logs are JSON formatted. Options to enable ECS/JSON have been removed. {issue}15544[15544] {pull}28573[28573]
- Remove `auto` from the available options of `setup.ilm.enabled` and set the default value to `true`. {pull}28671[28671]
- add_process_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]
- add_docker_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]
- Use data streams instead of indices for storing events from Beats. {pull}28450[28450]
- Remove option `setup.template.type` and always load composable template with data streams. {pull}28450[28450]
- Remove several ILM options (`rollover_alias` and `pattern`) as data streams does not require index aliases. {pull}28450[28450]
- Index template's default_fields setting is only populated with ECS fields. {pull}28596[28596] {issue}28215[28215]
- Remove `auto` from the available options of `setup.ilm.enabled` and set the default value to `true`. {pull}28671[28671]
- Remove deprecated `--template` and `--ilm-policy` flags. Use `--index-management` instead. {pull}28870[28870]
- Remove options `logging.files.suffix` and default to datetime endings in log file names. The format of the new name is `{beatname}-{date}(-n)?.ndjson`. Exmaple log files names from oldest to newest: `filebeat-20200101.ndjson`, `filebeat-20200101-1.ndjson`, `filebeat-20200101-2.ndjson`. {pull}28927[28927]
- Allign kubernetes configuration settings. {pull}29908[29908]
- The extension of the log files of Beats and Elastic Agent is changed to `.ndjson`. If you are collecting the logs, you must change the path configuration to `/path/to/logs/{beatname}*.ndjson` to avoid any issues. {pull}28927[28927]
- Remove legacy support for SSLv3. {pull}30071[30071]

*Filebeat*

- Add `while_pattern` type to multiline reader. {pull}19662[19662]
- auditd dataset: Use process.args to store program arguments instead of auditd.log.aNNN fields. {pull}29601[29601]
- Remove deprecated old awscloudwatch input name. {pull}29844[29844]

*Metricbeat*

- Remove network and diskio metrics from ec2 metricset. {pull}28316[28316]
- Rename read/write_io.ops_per_sec to read/write.iops in rds metricset. {pull}28350[28350]
- system/process metricset: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]

*Packetbeat*

- `event.category` no longer contains the value `network_traffic` because this is not a valid ECS event category value. {pull}20556[20556]
- Remove deprecated TLS fields in favor of tls.server.x509 and tls.client.x509 ECS fields. {pull}28487[28487]
- HTTP: The field `http.request.method` will maintain its original case. {pull}28620[28620]

*Winlogbeat*

- Remove top level `hash` property from sysmon events {pull}20653[20653]
- Move module processing from local Javascript processor to ingest node {issue}29184[29184] {pull}29435[29435]

==== Bugfixes

*Auditbeat*

- libbeat/processors/add_process_metadata: Fix memory leak in process cache. {issue}24890[24890] {pull}29717[29717]

[[release-notes-8.0.0-rc1]]
=== Beats version 8.0.0-rc1
*Filebeat*

- Fix using log_group_name_prefix in aws-cloudwatch input. {pull}29695[29695]

*Heartbeat*

Changes will be described in the GA release.
- Add fonts to support more different types of characters for multiple languages. {pull}29861[29861]

[[release-notes-8.0.0-beta1]]
=== Beats version 8.0.0-beta1
*Metricbeat*

Changes will be described in the GA release.
- Extract correct index property in kibana.stats metricset {pull}29622[29622]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license expiration date correctly. {pull}29711[29711]

[[release-notes-8.0.0-alpha2]]
=== Beats version 8.0.0-alpha2
*Packetbeat*

- Prevent incorrect use of AMQP protocol parsing from causing silent failure. {pull}29017[29017]
- Fix error handling in MongoDB protocol parsing. {pull}29017[29017]
- Redis: fix incorrectly handle with two-words redis command. {issue}14872[14872] {pull}14873[14873]
- Unify gopacket dependencies. {pull}29167[29167]

==== Added

*Affecting all Beats*

Changes will be described in the GA release.
- Add config option `rotate_on_startup` to file output {issue}19150[19150] {pull}19347[19347]
- Update to ECS 8.0 fields. {pull}28620[28620]
- Support custom analyzers in fields.yml. {issue}28540[28540] {pull}28926[28926]
- Support self signed certificates on outputs {pull}29229[29229]
- Add FIPS configuration option for all AWS API calls. {pull}[28899]
- Warn users when connecting to older versions of Elasticsearch instances. {pull}29723[29723]
- `add_fields` processor is now able to set metadata in events {pull}30092[30092]

[[release-notes-8.0.0-alpha1]]
=== Beats version 8.0.0-alpha1
*Auditbeat*

- system/process: Prevent hashing files in other mnt namespaces. {issue}25777[25777] {issue}29678[29678] {pull}29786[29786]

*Metricbeat*

- Preliminary AIX support {pull}27954[27954]
- Add option to skip older k8s events {pull}29396[29396]
- Add `elasticsearch.cluster.id` field to Beat and Kibana modules. {pull}29577[29577]
- Add `elasticsearch.cluster.id` field to Logstash module. {pull}29625[29625]

*Winlogbeat*

- Add support for sysmon event ID 26; FileDeleteDetected. {issue}26280[26280] {pull}29957[29957]

*Elastic Log Driver*

Changes will be described in the GA release.
- Fixed docs for hosts {pull}23644[23644]

[[release-notes-7.17.0]]
=== Beats version 7.17.0
Expand Down
86 changes: 15 additions & 71 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,78 +10,26 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

*Affecting all Beats*

- Remove the deprecated `xpack.monitoring.*` settings. Going forward only `monitoring.*` settings may be used. {issue}9424[9424] {pull}18608[18608]
- Remove deprecated/undocumented IncludeCreatorMetadata setting from kubernetes metadata config options {pull}28006[28006]
- Remove deprecated fields from kubernetes module {pull}28046[28046]
- Remove deprecated config option aws_partition. {pull}28120[28120]
- Improve stats API {pull}27963[27963]
- Libbeat: logp package forces ECS compliant logs. Logs are JSON formatted. Options to enable ECS/JSON have been removed. {issue}15544[15544] {pull}28573[28573]
- Update docker client. {pull}28716[28716]
- Remove `auto` from the available options of `setup.ilm.enabled` and set the default value to `true`. {pull}28671[28671]
- add_process_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]
- add_docker_metadata processor: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]
- Use data streams instead of indices for storing events from Beats. {pull}28450[28450]
- Remove option `setup.template.type` and always load composable template with data streams. {pull}28450[28450]
- Remove several ILM options (`rollover_alias` and `pattern`) as data streams does not require index aliases. {pull}28450[28450]
- Index template's default_fields setting is only populated with ECS fields. {pull}28596[28596] {issue}28215[28215]
- Remove deprecated `--template` and `--ilm-policy` flags. Use `--index-management` instead. {pull}28870[28870]
- Remove options `logging.files.suffix` and default to datetime endings in log file names. The format of the new name is `{beatname}-{date}(-n)?.ndjson`. Exmaple log files names from oldest to newest: `filebeat-20200101.ndjson`, `filebeat-20200101-1.ndjson`, `filebeat-20200101-2.ndjson`. {pull}28927[28927]
- Remove Journalbeat. Use `journald` input of Filebeat instead. {pull}29131[29131]
- `include_matches` option of `journald` input no longer accepts a list of string. {pull}29294[29294]
- Allign kubernetes configuration settings. {pull}29908[29908]
- The extension of the log files of Beats and Elastic Agent is changed to `.ndjson`. If you are collecting the logs, you must change the path configuration to `/path/to/logs/{beatname}*.ndjson` to avoid any issues. {pull}28927[28927]
- Remove legacy support for SSLv3. {pull}30071[30071]
- `add_fields` processor is now able to set metadata in events {pull}30092[30092]
- Add metadata change support for some processors {pull}30183[30183]

*Auditbeat*

- File integrity dataset (macOS): Replace unnecessary `file.origin.raw` (type keyword) with `file.origin.text` (type `text`). {issue}12423[12423] {pull}15630[15630]
- Change event.kind=error to event.kind=event to comply with ECS. {issue}18870[18870] {pull}20685[20685]

*Filebeat*

- Fix parsing of Elasticsearch node name by `elasticsearch/slowlog` fileset. {pull}14547[14547]
- With the default configuration the cloud modules (aws, azure, googlecloud, o365, okta)
- With the default configuration the cef and panw modules will no longer send the `host`
- Add `while_pattern` type to multiline reader. {pull}19662[19662]
- auditd dataset: Use process.args to store program arguments instead of auditd.log.aNNN fields. {pull}29601[29601]
- Remove deprecated old awscloudwatch input name. {pull}29844[29844]

*Heartbeat*

- Add fonts to support more different types of characters for multiple languages. {pull}29606[29861]

*Metricbeat*

- Remove deprecated fields in Docker module. {issue}11835[11835] {pull}27933[27933]
- Remove deprecated fields in Kafka module. {pull}27938[27938]
- Remove deprecated config option default_region from aws module. {pull}28120[28120]
- Remove network and diskio metrics from ec2 metricset. {pull}28316[28316]
- Rename read/write_io.ops_per_sec to read/write.iops in rds metricset. {pull}28350[28350]
- Remove linux-only metrics from diskio, memory {pull}28292[28292]
- Remove deprecated config option perfmon.counters from windows/perfmon metricset. {pull}28282[28282]
- Remove deprecated fields in Redis module. {issue}11835[11835] {pull}28246[28246]
- system/process metricset: Replace usage of deprecated `process.ppid` field with `process.parent.pid`. {pull}28620[28620]

*Packetbeat*
*Packetbeat*

- Redis: fix incorrectly handle with two-words redis command. {issue}14872[14872] {pull}14873[14873]
- `event.category` no longer contains the value `network_traffic` because this is not a valid ECS event category value. {pull}20556[20556]
- Remove deprecated TLS fields in favor of tls.server.x509 and tls.client.x509 ECS fields. {pull}28487[28487]
- HTTP: The field `http.request.method` will maintain its original case. {pull}28620[28620]
- Unify gopacket dependencies. {pull}29167[29167]

*Winlogbeat*

- Add support to Sysmon file delete events (event ID 23). {issue}18094[18094]
- Improve ECS field mappings in Sysmon module. `related.hash`, `related.ip`, and `related.user` are now populated. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. Hashes are now also populated to the corresponding `process.hash`, `process.pe.imphash`, `file.hash`, or `file.pe.imphash`. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. `file.name`, `file.directory`, and `file.extension` are now populated. {issue}18364[18364]
- Improve ECS field mappings in Sysmon module. `rule.name` is populated for all events when present. {issue}18364[18364]
- Remove top level `hash` property from sysmon events {pull}20653[20653]
- Move module processing from local Javascript processor to ingest node {issue}29184[29184] {pull}29435[29435]
- Fix run loop when reading from evtx file {pull}30006[30006]

*Functionbeat*

Expand All @@ -94,7 +42,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

*Auditbeat*

- libbeat/processors/add_process_metadata: Fix memory leak in process cache. {issue}24890[24890] {pull}29717[29717]
- auditd: Add error.message to events when processing fails. {pull}30009[30009]

*Filebeat*
Expand All @@ -105,22 +52,24 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

*Heartbeat*

- Add fonts to support more different types of characters for multiple languages. {pull}29861[29861]

*Filebeat*


*Heartbeat*


*Metricbeat*

- Extract correct index property in kibana.stats metricset {pull}29622[29622]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license expiration date correctly. {pull}29711[29711]
- Enhance metricbeat on openshift documentation {pull}30054[30054]

*Packetbeat*

- Prevent incorrect use of AMQP protocol parsing from causing silent failure. {pull}29017[29017]
- Fix error handling in MongoDB protocol parsing. {pull}29017[29017]

*Winlogbeat*

- Add provider names to Security pipeline conditional check in routing pipeline. {issue}27288[27288] {pull}29781[29781]
- Fix run loop when reading from evtx file {pull}30006[30006]

*Functionbeat*

Expand All @@ -134,24 +83,19 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

*Affecting all Beats*

- Add config option `rotate_on_startup` to file output {issue}19150[19150] {pull}19347[19347]
- Name all k8s workqueue. {pull}28085[28085]
- Update to ECS 8.0 fields. {pull}28620[28620]
- Support custom analyzers in fields.yml. {issue}28540[28540] {pull}28926[28926]
- Discover changes in Kubernetes nodes metadata as soon as they happen. {pull}23139[23139]
- Support self signed certificates on outputs {pull}29229[29229]
- Update k8s library {pull}29394[29394]
- Add FIPS configuration option for all AWS API calls. {pull}28899[28899]
- Add support for latest k8s versions v1.23 and v1.22 {pull}29575[29575]
- Add `script` processor to all beats {issue}29269[29269] {pull}29752[29752]
- Only connect to Elasticsearch instances with the same version or newer. {pull}29683[29683]
- Move umask from code to service files. {pull}29708[29708]
- Add FIPS configuration option for all AWS API calls. {pull}[28899]
- Warn users when connecting to older versions of Elasticsearch instances. {pull}29723[29723]
- Add metadata change support for some processors {pull}30183[30183]

*Auditbeat*

- system/process: Prevent hashing files in other mnt namespaces. {issue}25777[25777] {issue}29678[29678] {pull}29786[29786]
- system/socket: Add process.entity_id capture for socket events. {issue}30230[30230] {pull}30231[30231]

*Filebeat*
Expand All @@ -167,21 +111,23 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif
- threatintel module: Add new Recorded Future integration. {pull}30030[30030]
- Add pipeline in FB's supported hints. {pull}30212[30212]

*Auditbeat*


*Filebeat*


*Heartbeat*



*Metricbeat*

- Preliminary AIX support {pull}27954[27954]
- Add option to skip older k8s events {pull}29396[29396]
- Add `add_resource_metadata` configuration to Kubernetes module. {pull}29133[29133]
- Add `containerd` module with `cpu`, `memory`, `blkio` metricsets. {pull}29247[29247]
- Add `container.id` and `container.runtime` ECS fields in container metricset. {pull}29560[29560]
- Add `memory.workingset.limit.pct` field in Kubernetes container/pod metricset. {pull}29547[29547]
- Add k8s metadata in state_cronjob metricset. {pull}29572[29572]
- Add `elasticsearch.cluster.id` field to Beat and Kibana modules. {pull}29577[29577]
- Add `elasticsearch.cluster.id` field to Logstash module. {pull}29625[29625]
- Add `xpack.enabled` support for Enterprise Search module. {pull}29871[29871]
- Add gcp firestore metricset. {pull}29918[29918]
- Remove strict parsing on RabbitMQ module {pull}30090[30090]
Expand All @@ -196,11 +142,9 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

*Winlogbeat*

- Add support for sysmon event ID 26; FileDeleteDetected. {issue}26280[26280] {pull}29957[29957]

*Elastic Log Driver*

- Fixed docs for hosts {pull}23644[23644]

==== Deprecated

Expand Down
6 changes: 1 addition & 5 deletions libbeat/docs/release.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@ This section summarizes the changes in each release. Also read
<<breaking-changes>> for more detail about changes that affect
upgrade.

* <<release-notes-8.0.0-rc2>>
* <<release-notes-8.0.0-rc1>>
* <<release-notes-8.0.0-beta1>>
* <<release-notes-8.0.0-alpha2>>
* <<release-notes-8.0.0-alpha1>>
* <<release-notes-8.0.0>>
* <<release-notes-7.17.0>>
* <<release-notes-7.16.3>>
* <<release-notes-7.16.2>>
Expand Down

0 comments on commit 68872a0

Please sign in to comment.