Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New network fields #43

Closed
wants to merge 1 commit into from
Closed

New network fields #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,10 +286,14 @@ Fields related to network data.
| <a name="network.protocol"></a>`network.protocol` | Network protocol name. | keyword | | `http` |
| <a name="network.direction"></a>`network.direction` | Direction of the network traffic.<br/>Recommended values are:<br/> * inbound<br/> * outbound<br/> * unknown | keyword | | `inbound` |
| <a name="network.forwarded_ip"></a>`network.forwarded_ip` | Host IP address when the source IP address is the proxy. | ip | | `192.1.1.2` |
| <a name="network.session_id"></a>`network.session_id` | This is the session ID or connection ID, a unique identifier for the session /connection. Bro conn.log uid field gets mapped here. | keyword | | `00beeff00d1234` |
| <a name="network.virtual_ip"></a>`network.virtual_ip` | The virtual IP associated with a session or connection. Useful when load balancers, firewalls, and routers use a virtual IP to "advertise" a service. | ip | | `192.168.1.1` |
| <a name="network.inbound.bytes"></a>`network.inbound.bytes` | Network inbound bytes. | long | | `184` |
| <a name="network.inbound.packets"></a>`network.inbound.packets` | Network inbound packets. | long | | `12` |
| <a name="network.outbound.bytes"></a>`network.outbound.bytes` | Network outbound bytes. | long | | `184` |
| <a name="network.outbound.packets"></a>`network.outbound.packets` | Network outbound packets. | long | | `12` |
| <a name="network.total.bytes"></a>`network.total.bytes` | Network Total bytes: Usually sum (inbound.bytes, outbound.bytes) | long | | `368` |
| <a name="network.total.packets"></a>`network.total.packets` | Network Total packets: Usually sum (inbound.packets, outbound.packets) | long | | `24` |


## <a name="organization"></a> Organization fields
Expand Down
4 changes: 4 additions & 0 deletions schema.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,10 @@ network.inbound.packets,long,0,12
network.outbound.bytes,long,0,184
network.outbound.packets,long,0,12
network.protocol,keyword,0,http
network.session_id,keyword,0,00beeff00d1234
network.total.bytes,long,0,368
network.total.packets,long,0,24
network.virtual_ip,ip,0,192.168.1.1
organization.id,keyword,0,
organization.name,text,0,
os.family,keyword,0,debian
Expand Down
26 changes: 26 additions & 0 deletions schemas/network.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,22 @@
Host IP address when the source IP address is the proxy.
example: 192.1.1.2

- name: session_id
type: keyword
description: >
This is the session ID or connection ID,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading in this field and the next one about connection, I wonder if we should introduce `connection.* as mentioned in an other thread instead of uptting it under network.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @ruflin I forgot to reply before closing this PR out for splitting. Are you just talking about a name change i.e. s/network/connection? The network.* field set is intended to pick up flow and connection-based fields, and also network events that are not flow/connection related. I think this requires a bit more thought before making a decision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually thinking if we could use a network and a connection prefix. I think a big chunk of the info we have right now in network is in the right place, but there a things like forwarded_ip which probably fit better into connection and also these fields here.

Definitively needs more discussions, just an idea.

a unique identifier for the session /connection.
Bro conn.log uid field gets mapped here.
example: 00beeff00d1234

- name: virtual_ip
type: ip
description: >
The virtual IP associated with a session or connection.
Useful when load balancers, firewalls, and routers use a
virtual IP to "advertise" a service.
example: 192.168.1.1

# Metrics
- name: inbound.bytes
type: long
Expand All @@ -48,3 +64,13 @@
description: >
Network outbound packets.
example: 12
- name: total.bytes
type: long
description: >
Network Total bytes: Usually sum (inbound.bytes, outbound.bytes)
example: 368
- name: total.packets
type: long
description: >
Network Total packets: Usually sum (inbound.packets, outbound.packets)
example: 24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would phrase both of these more directly, e.g. "The sum of inbound.packets + outbound.packets", same for bytes.

17 changes: 17 additions & 0 deletions template.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,23 @@
"protocol": {
"ignore_above": 1024,
"type": "keyword"
},
"session_id": {
"ignore_above": 1024,
"type": "keyword"
},
"total": {
"properties": {
"bytes": {
"type": "long"
},
"packets": {
"type": "long"
}
}
},
"virtual_ip": {
"type": "ip"
}
}
},
Expand Down