update docs

dmachard · Nov 3, 2024 · 5aa2abe · 5aa2abe
1 parent 40370e9
commit 5aa2abe
Show file tree

Hide file tree

Showing 2 changed files with 74 additions and 77 deletions.
diff --git a/README.md b/README.md
@@ -3,7 +3,7 @@
 <img src="https://img.shields.io/badge/go%20version-min%201.21-green" alt="Go version"/>
 <img src="https://img.shields.io/badge/go%20tests-508-green" alt="Go tests"/>
 <img src="https://img.shields.io/badge/go%20bench-21-green" alt="Go bench"/>
-<img src="https://img.shields.io/badge/go%20lines-31679-green" alt="Go lines"/>
+<img src="https://img.shields.io/badge/go%20lines-31977-green" alt="Go lines"/>
 </p>
 
 <p align="center">
@@ -19,7 +19,7 @@
 > Additionally, DNS-collector also support
 >
 > - [Extended](https://github.com/dmachard/go-dns-collector/blob/main/docs/extended_dnstap.md) DNStap with TLS encryption, compression, and more metadata capabilities
-> - DNS protocol conversions to [plain text, JSON, jinja and more](https://github.com/dmachard/go-dns-collector/blob/main/docs/dnsconversions.md)
+> - DNS protocol conversions to [Plain text, Key/Value JSON, Jinja and more](https://github.com/dmachard/go-dns-collector/blob/main/docs/dnsconversions.md)
 > - DNS parser with [Extension Mechanisms for DNS (EDNS)](https://github.com/dmachard/go-dns-collector/blob/main/docs/dnsparser.md) support
 > - Live capture on a network interface
 > - IPv4/v6 defragmentation and TCP reassembly

diff --git a/docs/dnsconversions.md b/docs/dnsconversions.md
@@ -5,8 +5,7 @@
 All DNStap messages or network-captured data are first transformed into the [DNSMessage](https://pkg.go.dev/github.com/dmachard/go-dnscollector/dnsutils#DNSMessage) structure. Once in this structure, the object can be further converted to text, JSON, or any specific output format (e.g., Kafka, Redis).
 
 - [Text Format Inline](#text-format-inline)
-- [JSON Advanced Structure](#)
-- [Flat JSON (Key/Value)](#flat-json-format)
+- [JSON Enconding](#json-encoding)
 - [Jinja templating](#jinja-templating)
 
 ## Text format inline
@@ -113,7 +112,6 @@ global:
 
 ## JSON encoding
 
-
 The `DNS-collector` enables the transformation of DNS queries or replies into `JSON` format.
 The JSON format contains DNS messages with additionnal metadata added by transformers or collectors.
 
@@ -124,6 +122,77 @@ The default JSON payload parts:
 - `dns`: DNS fields.
 - `edns`: Extended DNS options.
 
+
+The Flat JSON is a simple key-value pair structure that supports customization of the keys in the format.
+
+At times, a single level key-value output in JSON is easier to ingest than multi-level JSON structures.
+Utilizing `flat-json` delivers every output field as its own key/value pair but requires more processing
+on the host running DNS-collector.
+
+This format is recommended because custom relabeling can be applied on it (drop keys or rename it).
+
+Here's a flat JSON output formatted using `jq`:
+
+```json
+{
+  "dns.flags.aa": false,
+  "dns.flags.ad": false,
+  "dns.flags.qr": true,
+  "dns.flags.ra": true,
+  "dns.flags.tc": false,
+  "dns.flags.rd": false,
+  "dns.flags.cd": false,
+  "dns.length": 82,
+  "dns.malformed-packet": false,
+  "dns.id": 34555,
+  "dns.opcode": 0,
+  "dns.qname": "google.nl",
+  "dns.qtype": "A",
+  "dns.rcode": "NOERROR",
+  "dns.qclass": "IN",
+  "dns.qdcount": 0,
+  "dns.ancount": 1,
+  "dns.arcount": 0,
+  "dns.nscount": 0,
+  "dns.resource-records.an.0.name": "google.nl",
+  "dns.resource-records.an.0.rdata": "142.251.39.99",
+  "dns.resource-records.an.0.rdatatype": "A",
+  "dns.resource-records.an.0.ttl": 300,
+  "dns.resource-records.an.0.class": "IN",
+  "dns.resource-records.ar": "-",
+  "dns.resource-records.ns": "-",
+  "dnstap.identity": "foo",
+  "dnstap.peer-name": "172.16.0.2",
+  "dnstap.latency": 0.000000,
+  "dnstap.operation": "CLIENT_RESPONSE",
+  "dnstap.timestamp-rfc3339ns": "2023-03-31T10:14:46.664534902Z",
+  "dnstap.version": "BIND 9.18.13-1+ubuntu20.04.1+isc+1-Ubuntu",
+  "dnstap.extra": "-",
+  "dnstap.policy-rule": "-",
+  "dnstap.policy-type": "-",
+  "dnstap.policy-action": "-",
+  "dnstap.policy-match": "-",
+  "dnstap.policy-value": "-",
+  "dnstap.query-zone": "-",
+  "edns.dnssec-ok": 0,
+  "edns.options.0.code": 10,
+  "edns.options.0.data": "-",
+  "edns.options.0.name": "COOKIE",
+  "edns.rcode": 0,
+  "edns.udp-size": 1232,
+  "edns.version": 0,
+  "network.family": "IPv4",
+  "network.ip-defragmented": false,
+  "network.protocol": "UDP",
+  "network.query-ip": "127.0.0.1",
+  "network.query-port": "36232",
+  "network.response-ip": "127.0.0.1",
+  "network.response-port": "53",
+  "network.tcp-reassembled": false,
+}
+```
+
+
 **Example JSON Output**
 
 ```json
@@ -207,7 +276,6 @@ The default JSON payload parts:
 }
 ```
 
-
 **Extended JSON format**
 
 This JSON message can be extended by collector(s):
@@ -223,74 +291,3 @@ This JSON message can be also extended by transformer(s):
 - [Traffic reducer](transformers/transform_trafficreducer.md)
 - [Traffic filtering](transformers/transformer_trafficfiltering.md)
 *
-
-## Flat JSON format
-
-Flat JSON is a simple key-value pair structure that supports customization of the keys in the format.
-
-At times, a single level key-value output in JSON is easier to ingest than multi-level JSON structures.
-Utilizing `flat-json` delivers every output field as its own key/value pair but requires more processing
-on the host running DNS-collector.
-
-This format is recommended because custom relabeling can be applied on it (drop keys or rename it).
-
-Here's a flat JSON output formatted using `jq`:
-
-```json
-{
-  "dns.flags.aa": false,
-  "dns.flags.ad": false,
-  "dns.flags.qr": true,
-  "dns.flags.ra": true,
-  "dns.flags.tc": false,
-  "dns.flags.rd": false,
-  "dns.flags.cd": false,
-  "dns.length": 82,
-  "dns.malformed-packet": false,
-  "dns.id": 34555,
-  "dns.opcode": 0,
-  "dns.qname": "google.nl",
-  "dns.qtype": "A",
-  "dns.rcode": "NOERROR",
-  "dns.qclass": "IN",
-  "dns.qdcount": 0,
-  "dns.ancount": 1,
-  "dns.arcount": 0,
-  "dns.nscount": 0,
-  "dns.resource-records.an.0.name": "google.nl",
-  "dns.resource-records.an.0.rdata": "142.251.39.99",
-  "dns.resource-records.an.0.rdatatype": "A",
-  "dns.resource-records.an.0.ttl": 300,
-  "dns.resource-records.an.0.class": "IN",
-  "dns.resource-records.ar": "-",
-  "dns.resource-records.ns": "-",
-  "dnstap.identity": "foo",
-  "dnstap.peer-name": "172.16.0.2",
-  "dnstap.latency": 0.000000,
-  "dnstap.operation": "CLIENT_RESPONSE",
-  "dnstap.timestamp-rfc3339ns": "2023-03-31T10:14:46.664534902Z",
-  "dnstap.version": "BIND 9.18.13-1+ubuntu20.04.1+isc+1-Ubuntu",
-  "dnstap.extra": "-",
-  "dnstap.policy-rule": "-",
-  "dnstap.policy-type": "-",
-  "dnstap.policy-action": "-",
-  "dnstap.policy-match": "-",
-  "dnstap.policy-value": "-",
-  "dnstap.query-zone": "-",
-  "edns.dnssec-ok": 0,
-  "edns.options.0.code": 10,
-  "edns.options.0.data": "-",
-  "edns.options.0.name": "COOKIE",
-  "edns.rcode": 0,
-  "edns.udp-size": 1232,
-  "edns.version": 0,
-  "network.family": "IPv4",
-  "network.ip-defragmented": false,
-  "network.protocol": "UDP",
-  "network.query-ip": "127.0.0.1",
-  "network.query-port": "36232",
-  "network.response-ip": "127.0.0.1",
-  "network.response-port": "53",
-  "network.tcp-reassembled": false,
-}
-```