From eeec1b22cd29e5abfddfad7a436758ce01440726 Mon Sep 17 00:00:00 2001 From: dmachard <5562930+dmachard@users.noreply.github.com> Date: Sat, 6 Jan 2024 09:09:53 +0100 Subject: [PATCH 01/18] Update docs --- README.md | 2 +- docs/collectors.md | 2 +- docs/loggers/logger_kafka.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 48459d79..7b2ff715 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ *NOTE: The code before version 1.x is considered beta quality and is subject to breaking changes.* -`DNS-collector` acts as a passive high speed **ingestor, aggregator and distributor** for your DNS logs with usage indicators and security analysis, written in **Golang**. The DNS traffic can be collected and aggregated from simultaneously [sources](./docs/collectors.md) like DNStap streams, network interface or log files and relays it to multiple other [listeners](./docs/loggers.md) with some [transformations](./docs/transformers.md) on it ([traffic filtering](./docs/transformers.md#dns-filtering), [user privacy](./docs/transformers.md#user-privacy), ...). +`DNS-collector` acts as a passive high speed **ingestor** with **pipelining** support for your DNS logs, written in **Golang**. It allows enhancing your DNS logs by adding metadata, extracting usage patterns, and facilitating security analysis. The DNS traffic can be collected and aggregated from simultaneously [sources](./docs/collectors.md) like DNStap streams, network interface or log files and relays it to multiple other [listeners](./docs/loggers.md) with some [transformations](./docs/transformers.md) on it ([traffic filtering](./docs/transformers.md#dns-filtering), [user privacy](./docs/transformers.md#user-privacy), ...). > Additionally, DNS-collector also support > diff --git a/docs/collectors.md b/docs/collectors.md index 3bd916c2..312c89e2 100644 --- a/docs/collectors.md +++ b/docs/collectors.md @@ -7,4 +7,4 @@ | [Tail](collectors/collector_tail.md) | Tail on plain text file | | [XDP Sniffer](collectors/collector_xdp.md) | Live capture on network interface with XDP | | [AF_PACKET Sniffer](collectors/collector_afpacket.md) | Live capture on network interface with AF_PACKET socket | -| [File Ingestor](collectors/collector_file.md) | File ingestor like pcap | +| [File Ingestor](collectors/collector_fileingestor.md) | File ingestor like pcap | diff --git a/docs/loggers/logger_kafka.md b/docs/loggers/logger_kafka.md index 83895a6d..027fb8a0 100644 --- a/docs/loggers/logger_kafka.md +++ b/docs/loggers/logger_kafka.md @@ -1,6 +1,6 @@ # Logger: Kafka Producer -Kafka producer +Kafka producer, based on [kafka-go](https://github.com/segmentio/kafka-go) library. Options: From 6b6bce94266a6208d71086e58d204946cd1d6341 Mon Sep 17 00:00:00 2001 From: dmachard <5562930+dmachard@users.noreply.github.com> Date: Sat, 6 Jan 2024 12:11:08 +0100 Subject: [PATCH 02/18] update overview diagram --- README.md | 2 +- docs/_images/overview.png | Bin 39023 -> 21778 bytes docs/overview.drawio | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7b2ff715..da20f666 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ Run
-Multiplexer +Pipelining ![overview](./docs/_images/overview.png) diff --git a/docs/_images/overview.png b/docs/_images/overview.png index 158e92c98d11132f8693133f59ae22adfba94ccb..5e5e569e73991586e2d5ed124a9493cb8bc38c1e 100644 GIT binary patch literal 21778 zcmeFYXHZm4*ER}B&S5};hE|P|P{sr) #%v4)O8&VE|Bf!ArzhR=L=6I1A
zQmYU}?C|&d{ie%(jWfD8<-gNJH6qhU5wiZrn%Ak&3b{~BAT@w!&uh)T=I(%A10t11
zg7}-Sx@&{ExL6EeJ=GNqloOcZWN$7h0PB(8#DoydpE%?47@S-52J0Rygy=dEdXh ?z1x+2lgu(
zaQ-;0nxtm^Hu&o0%^e-0|9 HQ>%+dAEnK
z`5U{hXY4@UhU120`@j2X{OwN_3`R`U=vZO5|0+26`tr=~(N@gMQgoD+B=&pURM&P}&+LcvGB4xi%P9
!SoCY4fnGSwZ=nSL$=i17~xNK_KSp*Tp&oY5ulq;v|SJ)
zmXYs
z$GYA%J*V!nt6X0
zm)N+$Dar=N6!z*%kVFMXpHx{Bw=D(-vy4x&r@-B8Pui2!2sA1DxFf
z6JOcd5lK%*5}H^*-T#k0*PqT*A3V0Of2~yY%>PH&Ux!8YzR|