Update openshift documentation, manifests for metricbeat and elastic-agent standalone #30054

tetianakravchenko · 2022-01-27T10:38:22Z

Signed-off-by: Tetiana Kravchenko [email protected]

What does this PR do?

metricbeat documentation: add kubernetes.proxy config for openshift setup
add comments related to the openshift installation for metricbeat and elastic-agent standalone

Why is it important?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify · 2022-01-27T10:38:47Z

This pull request does not have a backport label. Could you fix it @tetianakravchenko? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-01-27T10:56:37Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-01-27T10:38:38.193+0000
Duration: 18 min 51 sec
Commit: eb0b6c62025bc4fde947e6befd6e7e7100c2891b

Test stats 🧪

Test	Results
Failed	0
Passed	3
Skipped	0
Total	3

Steps errors

Expand to view the steps failures

`deploy/kubernetes-lint - make -C deploy/kubernetes all;make check-no-changes;`

Took 0 min 4 sec . View more details here
Description: make -C deploy/kubernetes all;make check-no-changes;

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'hudson.AbortException: script returned exit code 2'

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2022-01-27T12:41:21Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-02-04T15:31:06.209+0000
Duration: 126 min 51 sec

Test stats 🧪

Test	Results
Failed	0
Passed	7028
Skipped	2288
Total	9316

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2022-01-27T12:50:05Z

Pinging @elastic/integrations (Team:Integrations)

ChrsMark · 2022-02-02T16:21:43Z

metricbeat/docs/running-on-kubernetes.asciidoc

@@ -88,16 +88,7 @@ spec:
 ===== Red Hat OpenShift configuration

 If you are using Red Hat OpenShift, you need to specify additional settings in
-the manifest file and enable the container to run as privileged.


Are we removing this?

I've checked it on Openshiftv4 (GCP setup) and with privileged: true and without privileged: true getting the same result:
from the container:

mount | grep /usr/share/metricbeat/data /dev/sda4 on /usr/share/metricbeat/data type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,prjquota)

no errors descibed here: #17516 (comment)

additionally to that, I wanted to keep metricbeat and elastic-agent manifests (beside the actual configuration of modules/datasets) the same - elastic/observability-docs#1498

That's good! However maybe we need to be more explicit here and mention in which versions it is not required to avoid confusion?

Should we rely on openshift version here instead of kubernetes version? according to this doc the last available openshift version 3.11 uses Kubernetes 1.11, which is not supported by us I think

Strictly speaking yes we can claim that older versions are not supported however a small comment like NOTE: for older versions of v4 users need to run the container as privileged so as to provide a good hint to possible users that are still with older versions.

I will add it.
I didn't check the metricbeat on openshift v3, but somehow I think it even might not work (at least some functionality) - issue described in this comment elastic/integrations#2065 (comment)

done - 8d1e5d2

ChrsMark · 2022-02-02T16:22:15Z

metricbeat/docs/running-on-kubernetes.asciidoc

+        - proxy
+      period: 10s
+      host: ${NODE_NAME}
+      hosts: ["localhost:29101"]


Is the only thing needed? Any notes or links to how the proxy should be tuned on k8s' side?

yes, there is no need to change sdn-config configMap, this is the default configuration:

kube-proxy-config.yaml: |- apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 0.0.0.0 bindAddressHardFail: false clientConnection: acceptContentTypes: "" burst: 0 contentType: "" kubeconfig: "" qps: 0 clusterCIDR: 10.128.0.0/14 configSyncPeriod: 0s conntrack: maxPerCore: null min: null tcpCloseWaitTimeout: null tcpEstablishedTimeout: null detectLocalMode: "" enableProfiling: true healthzBindAddress: 0.0.0.0:10256 hostnameOverride: "" iptables: masqueradeAll: false masqueradeBit: 0 minSyncPeriod: 0s syncPeriod: 0s ipvs: excludeCIDRs: null minSyncPeriod: 0s scheduler: "" strictARP: false syncPeriod: 0s tcpFinTimeout: 0s tcpTimeout: 0s udpTimeout: 0s kind: KubeProxyConfiguration metricsBindAddress: 0.0.0.0:29101 mode: unidling+iptables nodePortAddresses: null oomScoreAdj: null portRange: "" showHiddenMetricsForVersion: "" udpIdleTimeout: 0s winkernel: enableDSR: false networkName: "" sourceVip: ""

note here: metricsBindAddress: 0.0.0.0:29101

ChrsMark · 2022-02-04T15:07:43Z

metricbeat/metricbeat.yml

-# In case you specify and additional path, the scheme is required: http://localhost:5601/path
-# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
-#host: "localhost:5601"
+  # Kibana Host


Is this related?

no, it is a result of make update, in other case tests (linting) were failing 🤷‍♀️

I did a rebase - this change already was present in main

Signed-off-by: Tetiana Kravchenko <[email protected]>

…crease mem requests & limits Signed-off-by: Tetiana Kravchenko <[email protected]>

Signed-off-by: Tetiana Kravchenko <[email protected]>

…cs#1498 Signed-off-by: Tetiana Kravchenko <[email protected]>

Signed-off-by: Tetiana Kravchenko <[email protected]>

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 27, 2022

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 27, 2022

mtojek added the Team:Integrations Label for the Integrations team label Jan 27, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 27, 2022

tetianakravchenko changed the title ~~update doc and manifest for metricbeat & elastic-agent standalone~~ Update openshift documentation, manifests for metricbeat and elastic-agent standalone Jan 27, 2022

tetianakravchenko marked this pull request as ready for review January 27, 2022 12:50

tetianakravchenko requested review from ChrsMark and MichaelKatsoulis January 27, 2022 12:50

This was referenced Jan 27, 2022

Test and verify that Elastic-Agent with k8s and system integrations run on Openshift elastic/integrations#2065

Closed

elastic-agent standalone on openshift documentation elastic/observability-docs#1498

Merged

ChrsMark reviewed Feb 2, 2022

View reviewed changes

ChrsMark reviewed Feb 4, 2022

View reviewed changes

tetianakravchenko and others added 9 commits February 4, 2022 16:30

update doc and manifest for metricbeat & elastic-agent standalone

d1b1330

Signed-off-by: Tetiana Kravchenko <[email protected]>

add comment to add bearer_token_file, ssl.certificate_authorities; in…

4fc4dba

…crease mem requests & limits Signed-off-by: Tetiana Kravchenko <[email protected]>

align standalone configs

7579fa7

Signed-off-by: Tetiana Kravchenko <[email protected]>

update comments, metricbeat doc according to elastic/observability-do…

56faebd

…cs#1498 Signed-off-by: Tetiana Kravchenko <[email protected]>

fix linting

104f5f0

Signed-off-by: Tetiana Kravchenko <[email protected]>

add a comment about https for kube-state-metrics

0ed80de

Signed-off-by: Tetiana Kravchenko <[email protected]>

add changelog

7620b5b

Signed-off-by: Tetiana Kravchenko <[email protected]>

Update CHANGELOG.next.asciidoc

2a0c334

add note about older versions; fix typo

5a76f20

Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko force-pushed the openshift-documentation branch from 8d1e5d2 to 5a76f20 Compare February 4, 2022 15:30

ChrsMark approved these changes Feb 7, 2022

View reviewed changes

tetianakravchenko merged commit da8c90c into elastic:main Feb 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update openshift documentation, manifests for metricbeat and elastic-agent standalone #30054

Update openshift documentation, manifests for metricbeat and elastic-agent standalone #30054

tetianakravchenko commented Jan 27, 2022 •

edited

Loading

mergify bot commented Jan 27, 2022

elasticmachine commented Jan 27, 2022

Build stats

Test stats 🧪

`deploy/kubernetes-lint - make -C deploy/kubernetes all;make check-no-changes;`

`Error signal`

elasticmachine commented Jan 27, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

elasticmachine commented Jan 27, 2022

ChrsMark Feb 2, 2022

tetianakravchenko Feb 3, 2022

ChrsMark Feb 4, 2022

tetianakravchenko Feb 4, 2022

ChrsMark Feb 4, 2022

tetianakravchenko Feb 4, 2022

tetianakravchenko Feb 4, 2022

ChrsMark Feb 2, 2022

tetianakravchenko Feb 3, 2022 •

edited

Loading

ChrsMark Feb 4, 2022

ChrsMark Feb 4, 2022

tetianakravchenko Feb 4, 2022 •

edited

Loading

tetianakravchenko Feb 4, 2022

Update openshift documentation, manifests for metricbeat and elastic-agent standalone #30054

Update openshift documentation, manifests for metricbeat and elastic-agent standalone #30054

Conversation

tetianakravchenko commented Jan 27, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Jan 27, 2022

elasticmachine commented Jan 27, 2022

💔 Build Failed

Build stats

Test stats 🧪

Steps errors

deploy/kubernetes-lint - make -C deploy/kubernetes all;make check-no-changes;

Error signal

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 27, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 27, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tetianakravchenko Feb 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tetianakravchenko Feb 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tetianakravchenko commented Jan 27, 2022 •

edited

Loading

`deploy/kubernetes-lint - make -C deploy/kubernetes all;make check-no-changes;`

`Error signal`

elasticmachine commented Jan 27, 2022 •

edited by jenkins-beats-ci bot

Loading

tetianakravchenko Feb 3, 2022 •

edited

Loading

tetianakravchenko Feb 4, 2022 •

edited

Loading