Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds IPV6_NETWORK method #53

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Adds IPV6_NETWORK method #53

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2f90ce7
adds IPV6_NETWORK method
Oct 2, 2019
5fb56df
updates asciidoc
Oct 2, 2019
bd6b930
removes redundant code
ldormoy May 14, 2021
d0898a5
Merge branch 'master' into master
kaisecheng Oct 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ This plugin supports the following configuration options plus the <<plugins-{typ
| <<plugins-{type}s-{plugin}-concatenate_sources>> |<<boolean,boolean>>|No
| <<plugins-{type}s-{plugin}-concatenate_all_fields>> |<<boolean,boolean>>|No
| <<plugins-{type}s-{plugin}-key>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-method>> |<<string,string>>, one of `["SHA1", "SHA256", "SHA384", "SHA512", "MD5", "MURMUR3", "IPV4_NETWORK", "UUID", "PUNCTUATION"]`|Yes
| <<plugins-{type}s-{plugin}-method>> |<<string,string>>, one of `["SHA1", "SHA256", "SHA384", "SHA512", "MD5", "MURMUR3", "IPV4_NETWORK", "IPV6_NETWORK", "UUID", "PUNCTUATION"]`|Yes
| <<plugins-{type}s-{plugin}-source>> |<<array,array>>|No
| <<plugins-{type}s-{plugin}-target>> |<<string,string>>|No
|=======================================================================
Expand Down Expand Up @@ -95,14 +95,14 @@ source fields given.
* Value type is <<string,string>>
* There is no default value for this setting.

When used with the `IPV4_NETWORK` method fill in the subnet prefix length.
When used with the `IPV4_NETWORK` or `IPV6_NETWORK` method fill in the subnet prefix length.
With other methods, optionally fill in the HMAC key.

[id="plugins-{type}s-{plugin}-method"]
===== `method`

* This is a required setting.
* Value can be any of: `SHA1`, `SHA256`, `SHA384`, `SHA512`, `MD5`, `MURMUR3`, `IPV4_NETWORK`, `UUID`, `PUNCTUATION`
* Value can be any of: `SHA1`, `SHA256`, `SHA384`, `SHA512`, `MD5`, `MURMUR3`, `IPV4_NETWORK`, `IPV6_NETWORK`, `UUID`, `PUNCTUATION`
* Default value is `"SHA1"`

The fingerprint method to use.
Expand All @@ -119,6 +119,11 @@ the hash value will be the masked-out address using the number of bits
specified in the `key` option. For example, with "1.2.3.4" as the input
and `key` set to 16, the hash becomes "1.2.0.0".

If set to `IPV6_NETWORK` the input data needs to be a IPv6 address and
the hash value will be the masked-out address using the number of bits
specified in the `key` option. For example, with "2001:db8:85a3::8a2e:370:7334" as the input
and `key` set to 112, the hash becomes "2001:db8:85a3::8a2e:370:0".

If set to `PUNCTUATION`, all non-punctuation characters will be removed
from the input string.

Expand Down
24 changes: 22 additions & 2 deletions lib/logstash/filters/fingerprint.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ class LogStash::Filters::Fingerprint < LogStash::Filters::Base
# Any current contents of that field will be overwritten.
config :target, :validate => :string, :default => 'fingerprint'

# When used with the `IPV4_NETWORK` method fill in the subnet prefix length.
# When used with the `IPV4_NETWORK` or `IPV6_NETWORK` method fill in the subnet prefix length.
# With other methods, optionally fill in the HMAC key.
config :key, :validate => :string

Expand All @@ -55,13 +55,18 @@ class LogStash::Filters::Fingerprint < LogStash::Filters::Base
# specified in the `key` option. For example, with "1.2.3.4" as the input
# and `key` set to 16, the hash becomes "1.2.0.0".
#
# If set to `IPV6_NETWORK` the input data needs to be a IPv6 address and
# the hash value will be the masked-out address using the number of bits
# specified in the `key` option. For example, with "2001:db8:85a3::8a2e:370:7334" as the input
# and `key` set to 112, the hash becomes "2001:db8:85a3::8a2e:370:0".
#
# If set to `PUNCTUATION`, all non-punctuation characters will be removed
# from the input string.
#
# If set to `UUID`, a
# https://en.wikipedia.org/wiki/Universally_unique_identifier[UUID] will
# be generated. The result will be random and thus not a consistent hash.
config :method, :validate => ['SHA1', 'SHA256', 'SHA384', 'SHA512', 'MD5', "MURMUR3", "IPV4_NETWORK", "UUID", "PUNCTUATION"], :required => true, :default => 'SHA1'
config :method, :validate => ['SHA1', 'SHA256', 'SHA384', 'SHA512', 'MD5', "MURMUR3", "IPV4_NETWORK", "IPV6_NETWORK", "UUID", "PUNCTUATION"], :required => true, :default => 'SHA1'

# When set to `true` and `method` isn't `UUID` or `PUNCTUATION`, the
# plugin concatenates the names and values of all fields given in the
Expand Down Expand Up @@ -92,6 +97,16 @@ def register
)
end
class << self; alias_method :fingerprint, :fingerprint_ipv4_network; end
when :IPV6_NETWORK
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the implementation is the same as IPv4, I suggest to reuse the condition in ipv4 instead of duplicate the code
when :IPV4_NETWORK, :IPV6_NETWORK

if @key.nil?
raise LogStash::ConfigurationError, I18n.t(
"logstash.runner.configuration.invalid_plugin_register",
:plugin => "filter",
:type => "fingerprint",
:error => "Key value is empty. please fill in a subnet prefix length"
)
end
class << self; alias_method :fingerprint, :fingerprint_ipv6_network; end
when :MURMUR3
class << self; alias_method :fingerprint, :fingerprint_murmur3; end
when :UUID
Expand Down Expand Up @@ -151,6 +166,11 @@ def fingerprint_ipv4_network(ip_string)
IPAddr.new(ip_string).mask(@key.to_i).to_s.force_encoding(Encoding::UTF_8)
end

def fingerprint_ipv6_network(ip_string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above, we can remove the duplication

# in JRuby 1.7.11 outputs as US-ASCII
IPAddr.new(ip_string).mask(@key.to_i).to_s.force_encoding(Encoding::UTF_8)
end

def fingerprint_openssl(data)
# since OpenSSL::Digest instances aren't thread safe, we must ensure that
# each pipeline worker thread gets its own instance.
Expand Down
16 changes: 16 additions & 0 deletions spec/filters/fingerprint_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,22 @@
end
end

describe "fingerprint ipaddress with IPV6_NETWORK method" do
config <<-CONFIG
filter {
fingerprint {
source => ["clientip"]
method => "IPV6_NETWORK"
key => 112
}
}
CONFIG

sample("clientip" => "2001:db8:85a3::8a2e:370:7334") do
insist { subject.get("fingerprint") } == "2001:db8:85a3::8a2e:370:0"
end
end

describe "fingerprint string with MURMUR3 method" do
config <<-CONFIG
filter {
Expand Down