Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Rename googlecloud module to gcp module #22214

Merged
merged 18 commits into from
Nov 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -716,6 +716,7 @@ from being added to events by default. {pull}18159[18159]
- Add SSL option to checkpoint module {pull}19560[19560]
- Add max_number_of_messages config into s3 input. {pull}21993[21993]
- Update Okta documentation for new stateful restarts. {pull}22091[22091]
- Rename googlecloud module to gcp module. {pull}22214[22214]
- Rename awscloudwatch input to aws-cloudwatch. {pull}22228[22228]
- Rename google-pubsub input to gcp-pubsub. {pull}22213[22213]
- Copy tag names from MISP data into events. {pull}21664[21664]
Expand Down
6 changes: 3 additions & 3 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ grouped in the following categories:
* <<exported-fields-envoyproxy>>
* <<exported-fields-f5>>
* <<exported-fields-fortinet>>
* <<exported-fields-googlecloud>>
* <<exported-fields-gcp>>
* <<exported-fields-gsuite>>
* <<exported-fields-haproxy>>
* <<exported-fields-host-processor>>
Expand Down Expand Up @@ -68452,8 +68452,8 @@ type: integer

--

[[exported-fields-googlecloud]]
== Google Cloud fields
[[exported-fields-gcp]]
== Google Cloud Platform (GCP) fields

Module for handling logs from Google Cloud.

Expand Down
2 changes: 2 additions & 0 deletions filebeat/docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,6 @@ include::./faq.asciidoc[]

include::{libbeat-dir}/contributing-to-beats.asciidoc[]

include::redirects.asciidoc[]


Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
This file is generated! See scripts/docs_collector.py
////

[[filebeat-module-googlecloud]]
[[filebeat-module-gcp]]
[role="xpack"]

:modulename: googlecloud
:modulename: gcp
:has-dashboards: false

== Google Cloud module
Expand All @@ -29,18 +29,18 @@ include::../include/config-option-intro.asciidoc[]
==== `audit` fileset settings

[role="screenshot"]
image::./images/filebeat-googlecloud-audit.png[]
image::./images/filebeat-gcp-audit.png[]

Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
audit:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-audit
var.subscription_name: filebeat-googlecloud-audit-sub
var.topic: gcp-vpc-audit
var.subscription_name: filebeat-gcp-audit-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -80,12 +80,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
vpcflow:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-flowlogs
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.topic: gcp-vpc-flowlogs
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -125,12 +125,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
firewall:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-firewall
var.subscription_name: filebeat-googlecloud-vpc-firewall-sub
var.topic: gcp-vpc-firewall
var.subscription_name: filebeat-gcp-vpc-firewall-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -170,5 +170,5 @@ field. Defaults to `false`, meaning the original message is not saved.
=== Fields

For a description of each field in the module, see the
<<exported-fields-googlecloud,exported fields>> section.
<<exported-fields-gcp,exported fields>> section.

4 changes: 2 additions & 2 deletions filebeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This file is generated! See scripts/docs_collector.py
* <<filebeat-module-envoyproxy>>
* <<filebeat-module-f5>>
* <<filebeat-module-fortinet>>
* <<filebeat-module-googlecloud>>
* <<filebeat-module-gcp>>
* <<filebeat-module-gsuite>>
* <<filebeat-module-haproxy>>
* <<filebeat-module-ibmmq>>
Expand Down Expand Up @@ -91,7 +91,7 @@ include::modules/elasticsearch.asciidoc[]
include::modules/envoyproxy.asciidoc[]
include::modules/f5.asciidoc[]
include::modules/fortinet.asciidoc[]
include::modules/googlecloud.asciidoc[]
include::modules/gcp.asciidoc[]
include::modules/gsuite.asciidoc[]
include::modules/haproxy.asciidoc[]
include::modules/ibmmq.asciidoc[]
Expand Down
10 changes: 10 additions & 0 deletions filebeat/docs/redirects.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
["appendix",role="exclude",id="redirects"]
= Deleted pages

The following pages have moved or been deleted.

[role="exclude",id="filebeat-module-googlecloud"]
== Google Cloud module

See <<filebeat-module-gcp>>.

73 changes: 65 additions & 8 deletions x-pack/filebeat/filebeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -813,8 +813,8 @@ filebeat.modules:
# "+02:00" for GMT+02:00
# var.tz_offset: local

#----------------------------- Google Cloud Module -----------------------------
- module: googlecloud
#--------------------- Google Cloud Platform (GCP) Module ---------------------
- module: gcp
vpcflow:
enabled: true

Expand All @@ -823,11 +823,11 @@ filebeat.modules:

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: googlecloud-vpc-flowlogs
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
Expand All @@ -841,11 +841,11 @@ filebeat.modules:

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: googlecloud-vpc-firewall
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-firewall-sub
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
Expand All @@ -859,11 +859,68 @@ filebeat.modules:

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: googlecloud-vpc-audit
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-audit
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

#----------------------------- Googlecloud Module -----------------------------
# googlecloud module is deprecated, please use gcp instead
- module: gcp
vpcflow:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

firewall:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

audit:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

54 changes: 54 additions & 0 deletions x-pack/filebeat/module/gcp/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
- module: gcp
kaiyan-sheng marked this conversation as resolved.
Show resolved Hide resolved
vpcflow:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

firewall:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

audit:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]

:modulename: googlecloud
:modulename: gcp
:has-dashboards: false

== Google Cloud module
Expand All @@ -24,18 +24,18 @@ include::../include/config-option-intro.asciidoc[]
==== `audit` fileset settings

[role="screenshot"]
image::./images/filebeat-googlecloud-audit.png[]
image::./images/filebeat-gcp-audit.png[]

Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
audit:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-audit
var.subscription_name: filebeat-googlecloud-audit-sub
var.topic: gcp-vpc-audit
var.subscription_name: filebeat-gcp-audit-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -75,12 +75,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
vpcflow:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-flowlogs
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.topic: gcp-vpc-flowlogs
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -120,12 +120,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
firewall:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-firewall
var.subscription_name: filebeat-googlecloud-vpc-firewall-sub
var.topic: gcp-vpc-firewall
var.subscription_name: filebeat-gcp-vpc-firewall-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- key: googlecloud
title: Google Cloud
- key: gcp
title: Google Cloud Platform (GCP)
description: >
Module for handling logs from Google Cloud.
fields:
Expand Down
Loading