Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #16197 to 7.x: Add ingress nginx controller fileset #16268

Merged
merged 2 commits into from
Feb 13, 2020

Conversation

ChrsMark
Copy link
Member

Cherry-pick of PR #16197 to 7.x branch. Original message:

Resolves: #8451

This PR adds fileset for ingress-nginx controller logs. Log patterns could be found on the controllers' docs.

In order to add the new pipeline a new setting called "version" is being introduced also in this PR. See the sum-up of the discussion on this comment.

Screenshot 2020-02-11 at 16 44 03

How to test this

Filebeat

  1. in /tmp/testnginx one can use the sample logs(see test.log file) used for automated testing.
  2. Enable nginx Filebeat module
  3. Configure the fileset.
- module: nginx
  # Access logs
  ingress_controller:
    enabled: true
    var.paths: ["/tmp/testnginx"]
  1. Setup the pipelines in ES: ./filebeat setup
  2. Start Filebeat and verify that events are properly shown in Kibana. NOTE: set the time window properly in Kibana so as to display the events in the timeline since the events have timestamps which will be in the past.

k8s environment

  1. Deploy an ingress managed sample app on Minikube following the official k8s docs
  2. check that ingress controller logs are being populated when doing curls (like described in k8s docs). Ingress controller pod lies under kube-system namespace and you will be able to check its logs with sth like nginx-ingress-controller-6fc5bcc8c9-zm8zv.
  3. Redirect pods' logs to a temporary file: kubectl -n kube-system logs -f nginx-ingress-controller-6fc5bcc8c9-zm8zv >> /tmp/ingresspod
  4. Inside minikube configure nginx module like:
- module: nginx
  # Ingress-nginx controller logs. This is disabled by default. It could be used in Kubernetes environments to parse ingress-nginx logs
  ingress_controller:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/tmp/ingresspod"]
  1. Run filebeat and check that logs are parsed correctly and shown in Kibana properly.
  2. Visit http://hello-world.info/v2 so as to create new log entries.

@ChrsMark ChrsMark requested a review from a team as a code owner February 12, 2020 12:40
@ChrsMark ChrsMark self-assigned this Feb 12, 2020
Copy link
Contributor

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@ChrsMark ChrsMark merged commit 6464944 into elastic:7.x Feb 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants