Duplicated log entry fields when using Kubernetes filter with Merge_Log

[fhoy]

With a particular combination of filters, I end up with duplicated fields in the log entry, without understanding how. Detailed, minimal config is attached, but a short description is as follows:

• Read log line 2019-01-18T08:44:02,099 message=Some message using input plugin "dummy".
• Filter log line through filter "parser", using a regex parser extracting fields prefix_timestamp and prefix_message.
• Filter log object through filter "nest", moving extracted fields to be subfields of field log, and removing prefix prefix_.
• Filter log object through filter "kubernetes", using Merge_Log to store Kubernetes fields under the log field.
• Print log object using output plugin "stdout".

The result of this is the following:

{"log"=>{"timestamp"=>"2019-01-18T08:44:02,099", "message"=>"Some message"}, "timestamp"=>"2019-01-18T08:44:02,099", "message"=>"Some message", "kubernetes"=>{"dummy"=>"Wed Jan 23 10:24:00 2019"}}

, i.e. fields timestamp and message occur twice; at the root of the JSON object, and nested within field log. Furthermore, Kubernetes data is not nested within the log field.

Setting the Merge_Log attribute of the Kubernetes filter to Off gives

{"log"=>{"timestamp"=>"2019-01-18T08:44:02,099", "message"=>"Some message"}, "kubernetes"=>{"dummy"=>"Wed Jan 23 10:35:15 2019"}}

, i.e. no duplication.

I haven't been able to figure out what goes wrong in the Fluent Bit code doing the merge, or if I have misunderstood how the filters cooperate.

To Reproduce

• Save attached files to directory ${dir_containing_attached_files}.
• Rename files to remove .txt suffix.
• Run docker run -i -t -v ${dir_containing_attached_files}:/fluent-bit/test fluent/fluent-bit:0.14.9 /fluent-bit/bin/fluent-bit -c /fluent-bit/test/fluent-bit.conf

I've also tested with the latest available image, 1.0.3, with the same results.

Config files:

• fluent-bit.conf.txt
• parsers.conf.txt

[encircled]

Hi,
I've faced similar issue. When a json log entry contains duplicate attributes and Merge_Log is On, fluent_d does:

• An entry with duplicate is completely skipped without any error in fluent d logs
• Sends a few duplicated log entries (other ones)

[edsiper]

FYI:

I've pushed fix 1d14886 to handle duplicated keys in a map. The workaround is basically: when converting the data to JSON, if a duplicated key is found, only pack the last value found for that key.

This will be part of v1.6 release next week.