otel: add test for document equivalence between agent filestream input and filebeat receiver

[mauri870]

What does this PR do?

This PR adds a test that compares a log line ingested by a filestream agent input and a filebeat OTel receiver with equivalent configuration.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Related issues

• Relates https://github.com/elastic/ingest-dev/issues/4736
• For https://github.com/elastic/ingest-dev/issues/3676

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[leehinman]

I think if you add the default processors listed at https://github.com/elastic/beats/blob/76f40863c9918ee1a691a2bb29cbab98a5d83878/x-pack/filebeat/cmd/root.go#L48 and add the processors listed in the generate.go below you should get a much shorter list of fields to ignore.

tests are failing

[pierrehilbert]

I don't think the failing tests are related to this PR:

=== RUN   TestDownloadFile
    utils_test.go:21: 
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-aws-1738851441003538166/elastic/elastic-agent/dev-tools/mage/downloads/utils_test.go:21
        	Error:      	Expected nil, but got: &fmt.wrapError{msg:"downloading file https://www.elastic.co/robots.txt: GET request failed with 504", err:(*errors.errorString)(0xc00085a970)}
        	Test:       	TestDownloadFile
--- FAIL: TestDownloadFile (20.57s)

Will update the branch to have them running again and hope the gateway failure was temporary

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b hybrid-agent-test upstream/hybrid-agent-test
git merge upstream/main
git push upstream hybrid-agent-test

[leehinman]

LGTM

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube