Issue with source value "-" for IPv4 strategy?

[knalli]

Hi,

I've found an issue the filter looks to "crash" internally when a field value has an "invalid" value. I'm new in Logstash (config setup) and not yet Ruby aware -- so perhaps I've missed something.

Given a sample config

filter {
      anonymize {
        algorithm => "IPV4_NETWORK"
        fields => ["proxyip"]
        key => "16"
      }
}

my LS daemon (1.5 RC2) was "hanging" after some time. No more logs, no output, no new messages in process. And forwarders (i.e. logstash-forwarder) are noticed this as well and were stucked.

I had spend some time to figure out in some circumstances some of my nginx access logs will produce - as a proxyip value (a local request w/o a proxy). I can fix this with either an additional condition ([proxy] != "-") or even better removing the field matching this condition.

But the question is: Is this behavior as expected? If I'm not missing something, the current specs do not cover any negative tests?

[driskell]

irb reports:

2.0.0-p576 :003 > ip = IPAddr.new("")
IPAddr::InvalidAddressError: invalid address
    from /Users/Jason/.rvm/rubies/ruby-2.0.0-p576/lib/ruby/2.0.0/ipaddr.rb:559:in `in6_addr'
    from /Users/Jason/.rvm/rubies/ruby-2.0.0-p576/lib/ruby/2.0.0/ipaddr.rb:496:in `initialize'
    from (irb):3:in `new'
    from (irb):3
    from /Users/Jason/.rvm/rubies/ruby-2.0.0-p576/bin/irb:12:in `<main>'

So if the field is empty or not a valid address this exception raises.

[driskell]

jirb reports:

2.0.0-p576 :002 > ip = IPAddr.new("")
ArgumentError: invalid address
    from /usr/local/Cellar/jruby/1.7.19/libexec/lib/ruby/1.9/ipaddr.rb:544:in `in6_addr'
    from /usr/local/Cellar/jruby/1.7.19/libexec/lib/ruby/1.9/ipaddr.rb:481:in `initialize'
    from (irb):2:in `evaluate'
    from org/jruby/RubyKernel.java:1107:in `eval'
    from org/jruby/RubyKernel.java:1507:in `loop'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from /usr/local/bin/jirb:13:in `(root)'

Both inherit from StandardError so no idea why the filter worker wasn't logging it as it died (it has a rescue => e - I can only think buffering.

But either way it really needs handling in the filter.

[eht16]

In #8 there is a possible solution by catching the ArgumentError exception, create a log message and tag the event for later reference.

I'm not completely sure if this is the best solution, but we need exactly this now as our Logstash instances crash often because of this error and we are not even able to find the causing log events because they are eaten by the crash.