Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Kubernetes - Metricbeat] Providing config option to disable Kubeadm config api requests #40086

Merged
merged 29 commits into from
Sep 10, 2024

Conversation

gizas
Copy link
Contributor

@gizas gizas commented Jul 3, 2024

  • Enhancement

NOTE: Dont merge until elastic/elastic-agent-autodiscover#98 is also merged

The elastic-agent-autodoscover library updated to v0.8.2

Proposed commit message

WHAT: Provide a configuration option to disable /api/v1/namespaces/kube-system/configmaps/kubeadm-config . In more details user will disable the api requests to the modules he configures by using configuration use_kubeadm: false

WHY: https://github.com/elastic/enhancements/issues/21172

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  • Install a kubernetes cluster with audit-logs enabled
  • Create a local elastic stack with elastic-package stack up -d -v --version=8.15.0-SNAPSHOT
  • Update the go.mod file of this repo and point to the elastic-agent-autodiscovery folder, where related changes should also exist. Your go.mod file should have the following entry:
replace github.com/elastic/elastic-agent-autodiscover => /Users/andreasgkizas/elastic/elastic-agent-autodiscover
  • Run go mod tidy
  • Install Filebeat and collect audit logs
  • Follow instructions of this page to build the metricbeat binary
 GOOS=linux GOARCH=arm64 go build
 kubectl apply -f module/kubernetes/_meta/test/docs/01_playground/metricbeat.yaml
 kubectl cp metricbeat `kubectl get pod -n kube-system -l k8s-app=metricbeat -o jsonpath='{.items[].metadata.name}'`:/usr/share/metricbeat/ -n kube-system\n
 kubectl exec `kubectl get pod -n kube-system -l k8s-app=metricbeat -o jsonpath='{.items[].metadata.name}'` -n kube-system -- bash -c "metricbeat -e -c /etc/metricbeat.yml"
  • Use use_kubeadm: false flag in your enabled modules
  • Open your local kibana cluster, navigate to Discovery and search inside kibana audit-logs for relevant api requests
Screenshot 2024-07-03 at 12 06 49 PM

Related issues

Screenshots

With 8.15.0 Image no changes:

Screenshot 2024-07-03 at 12 06 49 PM

We enable all metricsets (included events) and we have 85 API requests

With This repo code but the flag is disable_kubeadm: false

Screenshot 2024-07-03 at 12 34 26 PM

We enable all metricsets (included events) and we still have 85 API requests

With This repo code but the flag is disable_kubeadm: true`

Screenshot 2024-07-03 at 1 08 16 PM

We have no API requests after 10:04 that metrics collection established

{"log.level":"info","@timestamp":"2024-07-03T10:04:20.520Z","log.logger":"publisher_pipeline_output","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run","file.name":"pipeline/client_worker.go","file.line":145},"message":"Connection to backoff(elasticsearch(https://elasticsearch:9200)) established","service.name":"metricbeat","ecs.version":"1.6.0"}

@gizas gizas requested a review from a team as a code owner July 3, 2024 10:35
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 3, 2024
@mergify mergify bot assigned gizas Jul 3, 2024
Copy link
Contributor

mergify bot commented Jul 3, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @gizas? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@gizas gizas added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team label Jul 3, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 3, 2024
Copy link
Contributor

mergify bot commented Jul 3, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kubeadm upstream/kubeadm
git merge upstream/main
git push upstream kubeadm

@gizas gizas changed the title [Kubernetes] Providing config option to disable Kubeadm config api requests [Kubernetes - Metricbeat] Providing config option to disable Kubeadm config api requests Jul 17, 2024
@gizas
Copy link
Contributor Author

gizas commented Jul 19, 2024

I have updated description and autodiscovery library to make use of use_kubeadm variable

See comment elastic/elastic-agent-autodiscover#98 (comment)

gizas added a commit to elastic/elastic-agent-autodiscover that referenced this pull request Aug 12, 2024
@gizas gizas requested a review from a team as a code owner August 13, 2024 13:23
Copy link
Contributor

mergify bot commented Aug 13, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kubeadm upstream/kubeadm
git merge upstream/main
git push upstream kubeadm

@gizas gizas added the backport-skip Skip notification from the automated backport with mergify label Aug 13, 2024
Copy link
Contributor

mergify bot commented Aug 14, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kubeadm upstream/kubeadm
git merge upstream/main
git push upstream kubeadm

Copy link
Contributor

mergify bot commented Aug 28, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b kubeadm upstream/kubeadm
git merge upstream/main
git push upstream kubeadm

@gizas
Copy link
Contributor Author

gizas commented Sep 6, 2024

@MichaelKatsoulis I added again the use_kubeadm option here

Reason is that kubelet metricsets, proxy and scheduler make use of AddClusterECSMeta (like here) which is using finally the kubernetes config struct.

I have repeated the tests and now works ok!

Copy link
Contributor

@MichaelKatsoulis MichaelKatsoulis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gizas
Copy link
Contributor Author

gizas commented Sep 9, 2024

@elastic/beats-tech-leads can I have a review here? The pr looks good and tested

Copy link
Member

@cmacknz cmacknz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving go.mod changes.

Copy link
Contributor

@tetianakravchenko tetianakravchenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gizas gizas merged commit 7176eec into main Sep 10, 2024
126 of 129 checks passed
@gizas gizas deleted the kubeadm branch September 10, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-skip Skip notification from the automated backport with mergify Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants