syslog input plugin's grok semantics do not match RFC 3164; matching valid syslog messages fails (Logstash 1.4.2)

[PumpMagic]

1.4.2's syslog input plugin (lib/logstash/inputs/syslog.rb) uses the grok syntax of "SYSLOGLINE". See line 58:

"match" => { "message" => "<%{POSINT:priority}>%{SYSLOGLINE}" },

As far as I can tell, this "SYSLOGLINE" syntax is undefined in patterns/grok-patterns, much less anywhere. Valid syslog messages passed to the syslog input are tagged with _grokparsefailure and are assigned a default syslog priority.

[jordansissel]

% pwd
/Users/jls/build/logstash-1.4.2/patterns
% grep SYSLOGLINE *
linux-syslog:SYSLOGLINE %{SYSLOGBASE2} %{GREEDYDATA:message}

I see it defined. What are you looking at?

[jordansissel]

Additionally, what problem are you encountering that causes you to believe SYSLOGLINE is not defined?

[PumpMagic]

You're absolutely right, I missed that definition in my hastiness. Before I waste any more of your time, let me debug this further with this knowledge.

[PumpMagic]

Okay, I think I've found the issue. SYSLOGBASE2 ends in %{SYSLOGPROG}:, which, unless my interpretation of RFC 3164 is off, is an optional rather than mandatory field. As a result, the syslog messages sent by my application, like this one, result in grok parse failures.

<134>Feb 9 11:49:07 ryan-centos [Switch] [main] SWITCH IS NOW BOOTED AND FUNCTIONAL

If I remove %{SYSLOGPROG}: from SYSLOGBASE2, everything is parsed fine. Can we make %{SYSLOGPROG} optional?

[kinghrothgar]

I agree with PumpMagic. If you see the 5.4 Examples section of RFC 3164 this is a valid messasge:

<13>Feb  5 17:32:18 10.0.0.99 Use the BFG!

There is also the problem that the colon following SYLOGPROG (this is the TAG section in the RFC) is not required. A space is a valid delimiter to end the TAG section. I am currently having problems with this message being rejected:

<166>Feb 17 18:50:54 net002-re1 fpc1 Next-hop resolution requests from interface 326 throttled

This is a message sent from Junos OS. The problem is that if we make both the colon and SYLOGPROG optional, it can mistakenly match part of the CONTENT as the the program name in the case of the "Use the BFG!". I believe this is better however then failing to match both my valid Junos message and PumpMagic.

EDIT: Also, should this be moved over to https://github.com/logstash-plugins/logstash-patterns-core/issues

[offero]

I encountered the same issue where I could not find the definition of SYSLOGLINE. It is strange for something from the logstash repo to depend on a variable defined in the plugins repo. That was not intuitive for me.

[untergeek]

I believe we should deprecate the syslog input in favor of RFC-based codecs. Too many users see "syslog" and simply presume it will work, and are disappointed when it doesn't. The truth is that many logging implementations sending via syslog do not comply with an RFC.

If the codec is called syslog_rfc3164 (or whatever RFC it matches), then there will be no confusion about what is supported and what is not.

[offero]

That would be most excellent. I too, am having to customize multiple grok patterns to obtain syslog messages correctly for my programs.

[jordansissel]

For Logstash 1.5.0, we've moved all plugins to individual repositories, so I have moved this issue to logstash-plugins/logstash-input-syslog#12. Let's continue the discussion there! :)