Skip to content

Commit

Permalink
work in progress for auto-population, idaholab#135
Browse files Browse the repository at this point in the history
  • Loading branch information
@mmguero
mmguero committed Jul 7, 2023
1 parent a70971b commit 68e8ebc
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 24 deletions.
2 changes: 2 additions & 0 deletions docs/asset-interaction-analysis.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ If the `LOGSTASH_NETBOX_AUTO_POPULATE` [environment variable in `./config/logsta

However, careful consideration should be made before enabling this feature: the purpose of an asset management system is to document the intended state of a network: with Malcolm configured to populate NetBox with the live network state, a network misconfiguration fault could result in an **incorrect documented configuration**.

Devices created using this autopopulate method will have their `status` field set to `inventory`. The device manufacturer will be . It is recommended that users periodically review automatically-created devices for correctness and to fill in known details that couldn't be determined from network traffic. For example, the `manufacturer` field for automatically-created devices will be set based on the organizational unique identifier (OUI) determined from the first three bytes of the observed MAC address, which may not be accurate if the device's traffic was observed across a router. If possible, observed hostnames will be used in the naming of the automatically-created devices, falling back to the device manufacturer otherwise (e.g., `MYHOSTNAME @ 10.10.0.123` vs. `Schweitzer Engineering @ 10.10.0.123`).

See [idaholab/Malcolm#135](https://github.com/idaholab/Malcolm/issues/135) for more information on this feature.

## <a name="NetBoxPopActive"></a>Populate NetBox inventory via active discovery
Expand Down
65 changes: 41 additions & 24 deletions logstash/ruby/netbox_enrich.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ def filter(event)
return [event]
end

_key_ip = IPAddr.new(_key) rescue nil
_url = @netbox_url
_url_base = @netbox_url_base
_url_suffix = @netbox_url_suffix
Expand Down Expand Up @@ -261,6 +262,7 @@ def filter(event)
_autopopulate_site = nil
_vrfs = nil
_devices = nil
_exception_error = false

# handle :ip_device first, because if we're doing autopopulate we're also going to use
# some of the logic from :ip_vrf
Expand Down Expand Up @@ -330,14 +332,16 @@ def filter(event)
break unless (_tmp_ip_addresses.length() >= _page_size)
else
# weird/bad response, bail
_exception_error = true
break
end
end # while true
rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end

if _autopopulate && (_query[:offset] == 0)
if _autopopulate && (_query[:offset] == 0) && !_exception_error

# no results found, autopopulate enabled, let's create an entry for this device

Expand Down Expand Up @@ -384,6 +388,7 @@ def filter(event)
end
rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end
# return the manuf with the highest match
!_manufs&.empty? ? _manufs.max_by{|k| k[:match] } : nil
Expand Down Expand Up @@ -432,6 +437,7 @@ def filter(event)

rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end
_site
}
Expand Down Expand Up @@ -467,6 +473,7 @@ def filter(event)

rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end
_drole
}
Expand Down Expand Up @@ -544,7 +551,7 @@ def filter(event)
if _autopopulate_dtype&.fetch(:id, nil)&.nonzero?

# create the device
_device_name = "#{_key} (#{_autopopulate_manuf[:name]})"
_device_name = _autopopulate_hostname.to_s.empty? ? "#{_autopopulate_manuf[:name]} @ #{_key}" : "#{_autopopulate_hostname} @ #{_key}"
_device_data = { :name => _device_name,
:device_type => _autopopulate_dtype[:id],
:device_role => _autopopulate_drole[:id],
Expand All @@ -567,6 +574,7 @@ def filter(event)

rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end

if !_autopopulate_device.nil?
Expand Down Expand Up @@ -627,6 +635,7 @@ def filter(event)
end
rescue Faraday::Error
# give up aka do nothing
_exception_error = true
end
_vrfs = collect_values(crush(_vrfs))
_lookup_result = _vrfs unless (_lookup_type != :ip_vrf)
Expand All @@ -649,30 +658,38 @@ def filter(event)
then
_autopopulate_interface = _interface_create_reponse
end
end # check if device was created and has ID

if !_autopopulate_interface.nil? && _autopopulate_interface.fetch(:id, nil)&.nonzero?
# interface has been created, we need to create an IP address for it
_ip_data = { :address => "#{_key}/#{(IPAddr.new(_key) rescue nil)&.prefix()}",
:assigned_object_type => _autopopulate_manuf[:vm] ? "virtualization.vminterface" : "dcim.interface",
:assigned_object_id => _autopopulate_interface[:id],
:status => "active" }
if (_vrf = _autopopulate_interface.fetch(:vrf, nil)) &&
(_vrf.has_key?(:id))
then
_ip_data[:vrf] = _vrf[:id]
end
if (_ip_create_reponse = _nb.post('ipam/ip-addresses/', _ip_data.to_json, _nb_headers).body) &&
_ip_create_reponse.is_a?(Hash) &&
_ip_create_reponse.has_key?(:id)
then
_autopopulate_ip = _ip_create_reponse
end
end # check if interface was created and has ID
if !_autopopulate_interface.nil? && _autopopulate_interface.fetch(:id, nil)&.nonzero?
# interface has been created, we need to create an IP address for it
_ip_data = { :address => "#{_key}/#{_key_ip&.prefix()}",
:assigned_object_type => _autopopulate_manuf[:vm] ? "virtualization.vminterface" : "dcim.interface",
:assigned_object_id => _autopopulate_interface[:id],
:status => "active" }
if (_vrf = _autopopulate_interface.fetch(:vrf, nil)) &&
(_vrf.has_key?(:id))
then
_ip_data[:vrf] = _vrf[:id]
end
if (_ip_create_reponse = _nb.post('ipam/ip-addresses/', _ip_data.to_json, _nb_headers).body) &&
_ip_create_reponse.is_a?(Hash) &&
_ip_create_reponse.has_key?(:id)
then
_autopopulate_ip = _ip_create_reponse
end
end # check if interface was created and has ID

if !_autopopulate_ip.nil? && _autopopulate_ip.fetch(:id, nil)&.nonzero?
# IP address was created, need to associate it as the primary IP for the device
_primary_ip_data = { _key_ip&.ipv6? ? :primary_ip6 : :primary_ip4 => _autopopulate_ip[:id] }
if (_ip_primary_reponse = _nb.patch("#{_autopopulate_manuf[:vm] ? 'virtualization/virtual-machines' : 'dcim/devices'}/#{_autopopulate_device[:id]}/", _primary_ip_data.to_json, _nb_headers).body) &&
_ip_primary_reponse.is_a?(Hash) &&
_ip_primary_reponse.has_key?(:id)
then
_autopopulate_device = _ip_create_reponse
end
end # check if the IP address was created and has an ID

if !_autopopulate_ip.nil? && _autopopulate_ip.fetch(:id, nil)&.nonzero?
# TODO: set IP address as primary for device
end # check if IP was created and has ID
end # check if device was created and has ID

# yield return value for cache_hash getset
_lookup_result
Expand Down

0 comments on commit 68e8ebc

Please sign in to comment.