Skip to content

Commit

Permalink
refactor to optimize dnsmessage function (#632)
Browse files Browse the repository at this point in the history
* refactor flatten function
* Update README.md
* reduce op/s for ToTextLine
* clean code
dmachard authored Mar 3, 2024
1 parent e475120 commit 30790c2
Showing 6 changed files with 770 additions and 191 deletions.
2 changes: 1 addition & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
@@ -25,4 +25,4 @@ linters:
- gocritic

# list of linters to use in the future:
#- gosec
#- gosec
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<p align="center">
<img src="https://goreportcard.com/badge/github.com/dmachard/go-dns-collector" alt="Go Report"/>
<img src="https://img.shields.io/badge/go%20version-min%201.20-green" alt="Go version"/>
<img src="https://img.shields.io/badge/go%20tests-417-green" alt="Go tests"/>
<img src="https://img.shields.io/badge/go%20lines-38329-green" alt="Go lines"/>
<img src="https://img.shields.io/badge/go%20tests-429-green" alt="Go tests"/>
<img src="https://img.shields.io/badge/go%20lines-38661-green" alt="Go lines"/>
</p>

<p align="center">
356 changes: 272 additions & 84 deletions dnsutils/message.go

Large diffs are not rendered by default.

563 changes: 472 additions & 91 deletions dnsutils/message_test.go

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -63,6 +63,13 @@ Execute a test for one specific testcase in a package
go test -timeout 10s -cover -v ./loggers -run Test_SyslogRun
```

Run bench

```bash
cd dnsutils/
go test -run=^$ -bench=.
```

## Update Golang version and package dependencies

Update package dependencies
29 changes: 16 additions & 13 deletions docs/dnsjson.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# DNS-collector - DNS JSON encoding
# DNS-collector - JSON encoding

The dns collector enable to transform dns queries or replies in JSON format.
A JSON format contains dns message with additionnal metadata added by transformers or collectors.
The `DNS-collector` enables the transformation of DNS queries or replies into `JSON` format.
The JSON format contains DNS messages with additionnal metadata added by transformers or collectors.

Default JSON payload::
Main default JSON payload parts:

- `network`: query/response ip and port, the protocol and family used
- `dnstap`: message type, arrival packet time, latency.
- `dns`: dns fields
- `edns`: extended dns options
- `network`: Query/response IP and port, the protocol, and family used.
- `dnstap`: Message type, arrival packet time, latency.
- `dns`: DNS fields.
- `edns`: Extended DNS options.

Example:

@@ -87,10 +87,13 @@ Example:
}
```

## Flat JSON export format
## Flat JSON format (recommended)

Sometimes, a single level key-value output in JSON is easier to ingest than multi-level JSON.
Using flat-json requires more processing on the host running go-dnscollector but delivers every output field as its own key/value pair. Here's a flat-json output as formatted by `jq`:
At times, a single level key-value output in JSON is easier to ingest than multi-level JSON structures.
Utilizing `flat-json` delivers every output field as its own key/value pair but requires more processing
on the host running DNS-collector.

Here's a flat JSON output formatted using `jq`:

```json
{
@@ -112,8 +115,8 @@ Using flat-json requires more processing on the host running go-dnscollector but
"dns.resource-records.an.0.rdata": "142.251.39.99",
"dns.resource-records.an.0.rdatatype": "A",
"dns.resource-records.an.0.ttl": 300,
"dns.resource-records.ar": [],
"dns.resource-records.ns": [],
"dns.resource-records.ar": "-",
"dns.resource-records.ns": "-",
"dnstap.identity": "foo",
"dnstap.peer-name": "172.16.0.2",
"dnstap.latency": "0.000000",

0 comments on commit 30790c2

Please sign in to comment.