Haproxy filebeat TCP log format #8526
Conversation
sayden
added
enhancement
in progress
| IP of the destination host | ||
|
|
||
| -- | ||
|
|
There was a problem hiding this comment.
We may want to make use of ECS here https://github.com/elastic/ecs#-destination-fields
There was a problem hiding this comment.
I have given it a try but I got quite a lot of small blocking issues so I'm gonna continue with the old naming yet.
There was a problem hiding this comment.
Could you use ECS names under haproxy namespace in any case? (it'd be haproxy.destination.port and haproxy.destination.ip here)
| description: Total time in milliseconds elapsed between the accept and the last close | ||
| - name: connection_waiting_time_ms | ||
| type: long | ||
| description: Total time in milliseconds elapsed between the accept and the last close |
There was a problem hiding this comment.
Other fields in this file start with the name of the metric, we should probably open another PR to rephrase them for consistency, @sayden can you take care of this?
| description: time_wait_ms is the total time in milliseconds spent waiting for a full HTTP request from the client (not counting body) after the first byte was received. | ||
| type: long | ||
|
|
||
| - name: default |
There was a problem hiding this comment.
I wouldn't namespace these common fields under default.
There was a problem hiding this comment.
Oh, I guess this comes from #8428
| "haproxy.backend_name": "app", | ||
| "haproxy.backend_queue": 0, | ||
| "haproxy.bytes_read": 212, | ||
| "haproxy.client_ip": "127.0.0.1", |
There was a problem hiding this comment.
Client IP could be a public one, so geoip data is filled.
| "haproxy.frontend_name": "main", | ||
| "haproxy.pid": 25457, | ||
| "haproxy.process_name": "haproxy", | ||
| "haproxy.server_name": "<NOSRV>", |
There was a problem hiding this comment.
I guess this is that no server name was provided in the config? I guess that for this case we could leave this field empty, or even remove it.
It'd be nice to have also an example of a log line with a proper server name.
| "haproxy.termination_state": "SC", | ||
| "haproxy.total_waiting_time_ms": -1, | ||
| "input.type": "log", | ||
| "message": "Sep 20 15:44:23 127.0.0.1 haproxy[25457]: 127.0.0.1:40962 [20/Sep/2018:15:44:23.285] main app/<NOSRV> -1/-1/0 212 SC 1/1/0/0/0 0/0", |
There was a problem hiding this comment.
In other modules, message is removed after it is parsed, we may want to do the same on this module.
sayden
force-pushed
the
haproxy-filebeat-tcplog-format
branch
from
5120f9f to
3e82444
Compare
sayden
force-pushed
the
haproxy-filebeat-tcplog-format
branch
from
3e82444 to
98f5f97
Compare
|
Continues here #8637 |
urso
removed
the
needs_backport
Refer to this Issue for more details elastic/integrations#3250
WIP of the TCP log format that requires #8428 to be merged first