Features

This repo contains the following scripts in scripts/ dir consider this branch a different tool.

• a script for extracting the accumulated threat levels from slips alerts.json
• a script for extracting the ground truth labels for each time window given the conn.log.labeled for a given IP
• a script to determine the best threshold for sips based on the extracted threat levels and ground truth

Installation

pip3 install -r requirements.txt

---

Usage

python3 -m pip install -r requirements.txt

command for generating all zeek files in the dataset/

zeek -C -r <pcap> tcp_inactivity_timeout=60mins tcp_attempt_delay=1min

command for labeling conn.log files

python3 netflowlabeler.py -c labels.config -f /path/to/generated/conn.log

Note that the conn.log given to netflowlabeler should be tab-separated, not json. netflow labeler will drop the conn.log.labeled in the same directory of the given conn.log

(optional) To label the rest of the Zeek files using an already labeled conn.log file (conn.log.labeled)

zeek-files-labeler.py -l conn.log.labeled -f folder-with-zeek-log-files

command for extracting max accumulated threat level for all timewindows from an alert.json

python3 -m scripts.max_accumulated_score_extractor_for_slips alerts.json <host_ip> <used_slips_threshold>

command for getting the best slips threshold given the extracted ground truth labels and max accumulated scores

Note: this script assumes the correct ground truth labels are in scripts/extracted_gt_tw_labels.py and the correct max accumulated scores of slips are in scripts/extracted_levels.py

Note: This script completely discards flows and timewindows with any label other than benign or malicious, e.g. background, unknown label, no label etc.

• to print the metrics to cli

python3 -m scripts.slips_metrics_getter 

• to plot the metrics

python3 -m scripts.slips_metrics_getter  -p

Note: To print and plot the metrics, scripts/extracted_scores/extracted_levels.py must t be updated using the max_accumulated_score_extractor_for_slips.py script

command for extracting ground truth labels from a conn.log.labeled file

note: we only extract the labels per timewindow per ip

python3 main.py -gtf conn.log.labeled -i <host_ip>

• To extract the ground truth timewindow labels

  python3 -m scripts.ground_truth_timewindow_labels_extractor -gtf conn.log.labeled -i 147.32.83.234

About

This repo was developed at the Stratosphere Laboratory at the Czech Technical University in Prague.