-
Notifications
You must be signed in to change notification settings - Fork 419
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Autonomous System (AS) fields #341
Conversation
fields.yml
Outdated
system number (ASN) uniquely identifies each network on the Internet. | ||
example: 15169 | ||
|
||
- name: organization.name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think organization_name
would be a better name, unless there will be more properties than just name
under the organization
property.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I missed the suggestion of organization.country_code
. Maybe that information is better suited for the geo
field group, though.
edit: and there’s already an organization
field group so that #328 might actually solve this implicitly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking that the source.geo
field might be populated with the location of the IP whereas the details under source.autonomous_system.organization
would be about the organization (like perhaps the country where the organization is headquartered.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could have something like source.autonomous_system.organization.geo
, but maybe that's a bit over the top.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
geo
is a “reusable object” so it could be used under organization
. The current structure in this PR won’t preclude this from specified in the schema at a later time. This PR will get us the bare minimum fields for using the geoip processor with the AS database.
couple of thoughts if its not too late:
I have to consider a bit more to relearn my way down the bgp/routing stack, but I can immediately see NPM desires to track routing changes by AS (particularly in dual homed BGP setups) and usign a single organization header to describe a single AS country code (as opposed to src/dst AS & related info)... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the links, @andrewkroh. I wasn't familiar with these datasets, and the links were extremely useful.
Looking at some of the possibilities of what could go under autonomous systems, the final field names will indeed get very long. I wonder if we should do like with geo
, and contract the name to as
? E.g. source.as.number
, source.as.organization.*
.
After looking at a few records on the ARIN website and looking at the RIR files linked from the team-cymru.com page, I agree we can start with .number
and .organization.name
for now. To continue with your example nested under source
, here's what it could look like:
source.as.number
source.as.organization.name
- This leaves room for other org metadata, such as
organization.handle
- This leaves room for other org metadata, such as
I would defer discussion on the nesting of the country code for now. It's not in the default free Maxmind DB, so I suppose the need is not pressing yet.
For future growth, I think the current proposal is fine. We can discuss other fields when the need arises.
So to get this merged:
- We need to discuss whether we shorten the name to
as
- The PR needs to be rebased on top of master
- We should mark
organization
as reusable underas
, rather than redefining the field explicitly there.
I updated the PR to start address @webmat's actions:
|
@webmat What's the right way to accomplish the last bullet? I tried this after remove the
|
I reverted the change to make |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the adjustments, @andrewkroh. I'm surprised the nesting didn't work, but this is not a big deal. I'll check this out next time I play in the code.
We're good to merge once the changelog is rebased :-)
…tic/ecs#341 - keep backward compatibility though too
This adds fields for Autonomous System (AS) information. It's common to enrich events containing IP addresses with information about the AS controlling that address.
Update module pipelines to enrich events with autonomous system info (number and org name). All of the modules that had a geoip processor now also add autonomous system info. Field naming is based on ECS (elastic/ecs#341).
Update module pipelines to enrich events with autonomous system info (number and org name). All of the modules that had a geoip processor now also add autonomous system info. Field naming is based on ECS (elastic/ecs#341).
This adds fields for Autonomous System (AS) information. It's common to enrich
events containing IP addresses with information about the AS controlling that
address.
It fields are in a reusable object that can be attached to source, destination, client,
and server. For example:
source.as.number
source.as.organization.name
Some future fields we might want in this object are (I'm considering them now just to make sure nothing we do now precludes future growth):
cidr
- the IP range advertised by the AS (e.g. 216.90.108.0/24)organization.country_code_iso
- country code where the org is registeredregistry
- what registry the AS is with (e.g. ARIN)registration_date
- When the AS was first registeredlast_update
- When the AS registration was last updated.Some resources to see what kind of data is available.