-
Notifications
You must be signed in to change notification settings - Fork 189
Logstash
Config File: /etc/logstash/conf.d/logstash.conf
GROK Patterns: /etc/logstash/custom_patterns/
Translate YAML Files: /etc/logstash/translate/
- /opt/nsm/bro/logs/current/*.log
- /var/log/sweetsecurity.log
- /var/log/kern.log
- capture_loss.log
- communication.log
- conn.log
- dhcp.log
- dns.log
- dpd.log
- files.log
- ftp.log
- http.log
- intel.log
- known_services.log
- notice.log
- packet_filter.log
- pe.log
- reporter.log
- signatures.log
- smtp.log
- software.log
- ssh.log
- ssl.log
- stats.log
- tunnel.log
- weird.log
- x509.log
Log Field: orig_h --> geoip Field: geoip_src
Log Field: resp_h --> geoip Field: geoip_dst
The translate plugin is a community maintained plugin. As such, it does not ship with the default Logstash installation and must be installed manually. The installer will do this for you, however to manually install (for documentation purposes), the command is:
sudo /usr/share/logstash/bin/logstash-plugin install logstash-filter-translate
Log Fields orig_h, resp_h --> Destination Fields: tor_ip, malicious_ip
Every 60 minutes, the Sweet Security Client service will pull malicious IP addresses from malwaredomainlist.com and format them into a YAML file.
Destination: /etc/logstash/translate/maliciousIP.yaml
Format: "ipaddress": "YES"
If an IP address in orig_h or resp_h is found in the maliciousIP.yaml file, the word "YES" will be placed into the malicious_ip field when adding the log into Elasticsearch. From Kibana, searching malicious_ip:"YES" will show any matches to this translate plugin.
Every 60 minutes, the Sweet Security Client service will pull Tor Exit Node IP addresses from torproject.org and format them into a YAML file.
Destination: /etc/logstash/translate/torIP.yaml
Format: "ipaddress": "YES"
If an IP address in orig_h or resp_h is found in the torIP.yaml file, the word "YES" will be placed into the tor_ip field when adding the log into Elasticsearch. From Kibana, searching tor_ip:"YES" will show any matches to this translate plugin.
Logstash will write to Elasticsearch over authenticated port 9201, even if everything is on a single box.
hosts => "server_ip:9201"
user => "elasticsearch_user"
password => "elasticsearch_pass"
ssl => true
ssl_certificate_verification => false
Apache is configured to create a self signed certificate. If you choose to replace this with a certifacate signed by a trusted authority, change ssl_certificate_verification to true.
In order to properly run Logstash on the ARM architecture of Raspberry Pi, you need to update the JFFI module with one which supports ARM. This is done automatically but the installer by cloning the https://github.com/jnr/jffi repository. To manually do this (for documentation purposes), follow these steps:
git clone https://github.com/jnr/jffi
cd jffi/
sudo ant jar
cp build/jni/libjffi-1.2.so /usr/share/logstash/vendor/jruby/lib/jni/arm-Linux/libjffi-1.2.so
cd /usr/share/logstash/vendor/jruby/lib
sudo zip -g jruby-complete-1.7.11.jar jni/arm-Linux/libjffi-1.2.so
The installed version of Logstash is missing the email plugin which may prevent it from starting. Install with "/usr/share/logstash/bin/logstash-plugin install logstash-output-email"