Looking for sampling feature as packetbeat is sending too much traffic over network

[harshamadala]

Raising issue here to discuss more about sampling code implementation
Previous discussion in post:
https://discuss.elastic.co/t/is-there-any-way-to-do-data-sampling-at-client-side/50428/8?u=harshafrnd4u

Since packetbeat is doing lot of work to prepare a event with all fields, Is there any way we can design for it to randomly sample or sample based on some frequency like (send one event out of every ten events). May be we can enhance using drop_events but it would be better if we can drop before preparing event with fields.

Please guide me where can I add this kind of functionality in packetbeat.

[ruflin]

@harshamadala Where to add this probably heavily depends on what it does. I would suggest to first have here a discussion on how this should work and behave. There are lots of things to be taken into account, also related to ordering. Just an example: What happens first: Sampling or filtering?

[harshamadala]

@ruflin I suggest it would be better if we can do some sampling based on ip like (send every 10th transaction based on ip) I dont know whether it is possible to sample based on ip before decoding complete transaction, so that we can reduce some cpu utilisation. One obvious possible thing is we can sample and skip further process for few transactions before sending them over network, as it is consuming lot of network bandwidth.

[ruflin]

Pulling in @urso in this conversation as he knows much more about sampling.

[calston]

Probably worth looking at the mechanism SFlow uses for sampling

[ghost]

Rate limiting as goreplay does would be a good starting point. This will optimize the network bandwidth aspects:

https://github.com/buger/goreplay/wiki/Rate-limiting

[ghost]

Actually, this is one of the major reasons I decided to go ahead and use goreplay tool. The free version does only http so we don't have visibility across other protocols :-( but gives us more control over other aspects. In no way comparing these two tools but as can be seen there is a bit of overlap when it comes to monitoring incoming traffic.

Would make packetbeat really useful if we can add this outbound rate-limiting aspects.

[ghost]

Can someone please comment on the idea proposed above? I think this is a much needed feature. Would love to see it happen and help with testing etc.

[lcaballero]

I suggested something like this here: Rate limit published events. I did write something that works for my specific use case. I think something for the library itself would be a great feature. Hence, I started the discussion to cover how it might be implemented.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue doesn't have a Team:<team> label.