[Filebeat] Fix id for config map

[constanca-m]

This PR is a fix to a bug introduced in #37401.

Defining the id as

id: kubernetes-container-logs-${data.kubernetes.pod.name}-${data.kubernetes.container.id}

Causes the error:

{"log.level":"error","@timestamp":"2024-01-04T13:13:02.933Z","log.origin":{"file.name":"instance/beat.go","file.line":1307},"message":"Exiting: failed to read input configuration: failed to unpack input configuration: missing field accessing 'filebeat.inputs.0.id' (source:'/etc/filebeat.yml')","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: failed to read input configuration: failed to unpack input configuration: missing field accessing 'filebeat.inputs.0.id' (source:'/etc/filebeat.yml')

We need to remove the fields ${data.*} in the id to fix it.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @constanca-m? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

❕ Build Aborted

Either there was a build timeout or someone aborted the build.

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 37 min 41 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 36 min 4 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[gsantoro]

Do you mind updating also:

• beats/dev-tools/kubernetes/filebeat/manifest.run.yaml

  Line 116 in c9f1426

  id: kubernetes-container-logs-${data.kubernetes.pod.name}-${data.kubernetes.container.id}

• beats/dev-tools/kubernetes/filebeat/manifest.debug.yaml

  Line 116 in c9f1426

  id: kubernetes-container-logs-${data.kubernetes.pod.name}-${data.kubernetes.container.id}

to keep dev-tools manifests up to date with deploy manifests

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 135 min 53 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[rdner]

@constanca-m

By default the length for fingerprint is 1024, which is what is causing the error. We can reduce it to 300 (minimum possible is 64), and that solves the problem.

It's not an error, it's a warning and it's totally normal to have it until the file is large enough to be picked up. The warning needs to be there so the user knows why there are no events coming from a file yet.

Reducing the fingerprint might cause collisions in file identity, lowering the value must be done only if the user is 100% sure that the new amount of characters is always unique for each file. Please keep the value as it is now.

[constanca-m]

Thank you for the explanation @rdner , so the warning does not cause unexpected behavior? I will remove that line in that case.

[rdner]

@constanca-m The warning is just to notify the user why the ingestion of the file got delayed. Perhaps we could re-phrase this warning to be more clear and less error-looking. Ideas are welcome.

It does not cause any incorrect/unexpected behaviour, it's a normal state of things.

[rdner]

These substitutions should work only in autodiscover, not in regular input configuration, AFAIK.

I see @gsantoro made this change in #37401
How was it tested?

[constanca-m]

It was not correctly tested in that PR, this is a fix to that

[rdner]

@constanca-m My point is rather if this was not tested, how can we trust the rest of the changes from that PR? Perhaps we need to ask @gsantoro to re-test those changes.

[gsantoro]

hello @rdner , sorry that was my fault. It was an unintended change. I'll do some testing now anyway

[gsantoro]

I have now tested this. it works as expected with this PR changes. thanks @constanca-m for catching this

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 136 min 56 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[rdner]

LGTM in the current state.