"message" configuration parameter ignored in Logstash 7.2.0 and up

[TimKri]

• Version: 7.2.0 and up
• Operating System: Reproduced on RHEL 7.7 and Windows 10
• Config File:

input { 
  stdin { } 
}
output {
  syslog {
        host => "127.0.0.1"
        port => "514"
        sourcehost => "test"
        message => "dummy"
    }
}

• Steps to Reproduce:

Expected behaviour: The output will replace the default %{message} field with the text "dummy" and send it to the syslog server running on localhost.

Output on Logstash 7.1.1:
Feb 3 14:21:50 test LOGSTASH[-]: dummy

Output on Logstash >7.2.0:
Feb 3 14:23:48 test LOGSTASH[-]: 2020-02-03T14:23:48.725Z hostname logmessage

Also validated using tcpdump:

# tcpdump -nnAs0 -i lo port 52467

listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes

02:41:27.609504 IP 127.0.0.1.52467 > 127.0.0.1.514: SYSLOG user.notice, length: 82

E..nwy@[email protected]<13>Feb 05 01:41:27 test LOGSTASH[-]: 2020-02-05T01:41:27.504Z hostname logmessage

I'm guessing the change in codec.encode() might be the culprit?
elastic/logstash#10620

[ristov]

Also, according to documentation, the default value for message text should be the value of %{message} field. However, when formatting the message text, the plugin mistakenly prepends a timestamp and hostname in JSON format to the value of the %{message} field, for example:
2020-02-05T15:01:49.973Z {name=localhost.localdomain})

For instance, when I send in a message with the following "message" field

192.168.56.1 - - [05/Feb/2020:17:01:41 +0200] "GET /noindex/css/fonts/Bold/OpenSans-Bold.woff HTTP/1.1" 404 239 "http://192.168.56.102/noindex/css/open-sans.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0"

the plugin creates the following message text from it:

2020-02-05T15:01:49.973Z {name=localhost.localdomain} 192.168.56.1 - - [05/Feb/2020:17:01:41 +0200] "GET /noindex/css/fonts/Bold/OpenSans-Bold.woff HTTP/1.1" 404 239 "http://192.168.56.102/noindex/css/open-sans.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0"

However, appending such arbitrary data should not happen.

[jsvd]

This is due to, since 7.2.0 there's a codec delegator class that wraps the codec, and causes the class identity test to fail. The issue is tracked here: elastic/logstash#11434

The current workaround is to force the codec plain to be used, like so:

input { 
  stdin { } 
}
output {
  syslog {
        host => "127.0.0.1"
        port => "5140"
        sourcehost => "test"
        message => "dummy"
        codec => plain { format => "%{message}" }
    }
}

[user578]

The current workaround is to force the codec plain to be used, like so:

This workaround doesn't work for logstash 7.4.2

[antoinetran]

Omg, we spent so much time trying to use this message field, and after finding the same workaround, I was going to raise an issue. We reproduce the issue with logstash-7.3.2-1.noarch and logstash-output-syslog-3.0.4 . The workaround works.

[guyrdahan]

had similar issue with my configuration, when using input filter to parse the message filed to json, the logstash syslog output was working only after i configured codec => json

[makefu]

I encountered the same issue (logstash 8.4), workaround didn't work.
instead of trying to work around the issue further we will fall back to creating the raw message with the tcp output and codec => plain { message => "our %{message}" }