Cisco Umbrella Filebeat Module

[Furb13]

Filebeat module

Module: Cisco Umbrella
Documentation: https://docs.umbrella.com/deployment-umbrella/docs/log-formats-and-versioning

Looking to get the DNS, Proxy, IP and Cloud Firewall logs parsed to ECS format.

• Test log files exist for the grok patterns (Exist in documentation link)
• Generated output for at least 1 log file exists (Exist in documentation link)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[ry-wiz]

This would fit very well with the current Cisco ASA and FTD filebeats as most customers that are running ASA/FTD most likely are utilizing Umbrella on the edge.

[jamiehynds]

Thanks for the request. We're currently focused on improving our coverage for ASA/FTD events, but will certainly take the Umbrella request on board.

Umbrella appears to ship events via CSV files to S3 buckets. If anyone is willing to provide some sample events (sanitised is fine) please let us know. Always great to have sample datasets when developing new modules.

[andrewkroh]

@NeilADesai shared his Logstash config for parsing the DNS logs at https://github.com/NeilADesai/CiscoUmbrellaDNS/tree/master.

[jamiehynds]

@P1llus does your DevNet account include Umbrella? Could be an easy way to generate some sample events for Proxy, IP and Cloud Firewall.

[P1llus]

The Umbrella is a cloud based solution so devnet don't have anything for that, but they do have specific logins for trials on Umbrella + a MSP/MSSP trial potential: https://signup.umbrella.com/

Should be pretty quick to get a demo up.

[Furb13]

I can get you some scrubbed DNS and proxy logs, but don't have the IP or Cloud Firewall ones. Let me know if that would be helpful.

Also, the way it logs is Umbrella will send to a self-owned S3 bucket, which will drop .csv.gz files with the csv compressed.
https://docs.umbrella.com/deployment-umbrella/docs/setting-up-an-amazon-s3-bucket

[P1llus]

I think the initial part will have to see if we have beat functionality that can read compressed files on S3, the biggest issue there is that I currently don't have access to a S3 endpoint to test that, however that is something that might be able to be resolved 👍

In terms of parsing and creating a pipeline that can happen before the input has been resolved, so that its ready once we figure out a way to do it.

Indeed test logs would be needed, so if you do have any @Furb13 then for sure it would be helpful to create the pipeline for me. If you want to share it privately feel free to contact me directly.

[P1llus]

Looking at the code for the s3 input on filebeat does have references to unzipping files that are gzipped. @Furb13 maybe you would have the possibility to test it out?
Would be interesting to see how it looks like when its being read. Since its CSV we just need to use a key-value processor to split up the data and map it to ECS :)