[fbreceiver] Default Processors are not being applied

[khushijain21]

When fbreceiver is created, it adds default processor to the settings here.

beats/x-pack/filebeat/fbreceiver/factory.go

Line 40 in 95b5dce

globalProcs, err := processors.NewPluginConfigFromList(defaultProcessors())

Expected:
It is expected that all documents ingested have host related fields added to it. According to this ref here https://www.elastic.co/guide/en/beats/filebeat/current/add-host-metadata.html

Got:
Except for host.name, no other fields are part of the ingested doc.

When we explicitly set the default processor in the config file. It correctly adds all host related fields.

How to test this locally

1. cd x-pack/filebeat
2. mage docker:composeUp
3. Change filebeat-otel.yml to following config

filebeat.inputs:
  - type: filestream
    id: filestream-input-id
    enabled: true
    paths:
      - ./test.log

output:
  elasticsearch:
    hosts: ["http://localhost:9200"]
    username: admin
    password: testing

setup.kibana:
  host: http://localhost:5601
  username: admin
  password: testing

4. Start filebeat in otel mode

./filebeat otel 

Check the document ingested

5. Add processors and run with the same command again.

processors:
  - add_host_metadata:
      when:
        not:
          contains:
            tags: forwarded

6. The host related parameters are correctly set

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[mauri870]

This is also happening for fbreceiver running as part of EDOT, in this test we add the processors manually otherwise the fields are not present.

[mauri870]

I looked a bit into this and it seems that the call that instantiate the processors is:

beats/x-pack/filebeat/fbreceiver/factory.go

Line 44 in 090584e

settings.Processing = processing.MakeDefaultSupport(true, globalProcs, processing.WithECS, processing.WithHost, processing.WithAgentMeta())

However it only applies the processors if fleetmode.Enabled() is true, which is neither the case for EDOT nor filebeat otel:

beats/libbeat/publisher/processing/default.go

Lines 118 to 124 in 090584e

	if fleetmode.Enabled() && !beatCfg.HasField("processors") {
	log.Debugf("In fleet mode with no processors specified, defaulting to global processors")
	rawProcessors = fleetDefaultProcessors
	} else {
	rawProcessors = cfg.Processors
	}

This results in none of the add_* processors being applied, the host.name field is only present because of processing.WithHost.

[mauri870]

I'm not entirely sure if this is a bug or just the default behavior.

cc @leehinman