v1.4.0-beta2

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces a new transformer to reorder logs by timestamp, enhanced support for the PowerDNS protobuf format, and a new OpenTelemetry logger."

What's Changed

• feat: set minimal go version to 1.23
• feat(logger): new opentelemetry logger with tracing support by @dmachard in #912
• feat(transformer): add configurable unique fields to traffic reducer by @dmachard in #905
• feat(powerdns): add support for more fields (MessageId, RequestorId...) by @dmachard in #907 #909
• feat(transformer): new reordering transform to sort logs by timestamp by @dmachard in #910
• fix: add missing telemetry counter in powerdns collector by @dmachard in #911
• chore(deps): bump loki to v3.3.2 by @dmachard in #915
• chore(deps): bump alpine from 3.20.3 to 3.21.0 by @dependabot in #899
• chore(deps): bump golang from 1.23.3-alpine3.19 to 1.23.4-alpine3.19 by @dependabot in #900
• chore(deps): bump google.golang.org/protobuf from 1.35.2 to 1.36.0 by @dependabot in #903
• chore(deps): bump github.com/prometheus/common from 0.60.1 to 0.61.0 by @dependabot in #904
• chore(deps): bump golang.org/x/net from 0.31.0 to 0.33.0 by @dependabot in #913

Full Changelog: v1.3.0...v1.4.0-beta2

v1.3.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces a powerful new transformer designed to track newly observed domains. This release is also the final release for the year. Thank you for your support.

What's Changed

• feat: New domain tracker by @dmachard in #892
• feat: Add RD flag handling in text format directive by @dmachard in #893
• feat: dns parser: set RCODE only in DNS responses by @dmachard in #885
• feat: support direct sniffing on gre interface on packet sniffer by @dmachard in #884
• feat: add support of the gre protocol decoding on packet sniffer by @dmachard in #883
• feat: bpf filter improved for packet sniffer by @dmachard in #883
• fix: correct RCODE name from NOIMP to NOTIMP by @dmachard in #888
• fix: properly set DNStap operation for DNS update based on QR flag by @dmachard in #889
• fix: prevent multiple channel close in Telemetry Stop method by @dmachard in #890
• chore(ci): update testing-dnstap.yml with unbound 1.22 by @dmachard in #868
• chore(ci): update .goreleaser.yml by @dmachard in #869
• chore(doc): update README.md - add docker pull badge by @dmachard in #870
• chore(deps): bump golang from 1.23.2-alpine3.19 to 1.23.3-alpine3.19 by @dependabot in #871
• chore(deps): bump golang.org/x/sys from 0.26.0 to 0.27.0 by @dependabot in #872
• chore(deps): bump golang.org/x/net from 0.30.0 to 0.31.0 by @dependabot in #873
• chore(deps): bump google.golang.org/protobuf from 1.35.1 to 1.35.2 by @dependabot in #880
• chore(deps): bump github.com/tinylib/msgp from 1.2.4 to 1.2.5 by @dependabot in #897
• chore(deps): bump golang.org/x/sys from 0.27.0 to 0.28.0 by @dependabot in #896
• chore(deps): bump golang.org/x/crypto from 0.29.0 to 0.31.0 by @dependabot in #901

Full Changelog: v1.2.0...v1.3.0

v1.2.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

What's Changed

• feat: added ElasticSearch API basic authentication by @dmachard in #866
• feat: add new jinja format settings in console and logfile by @dmachard in #856
• fix: set go version in githubactions properly by @dmachard in #862
• fix: dns parser - TLSA typo by @tomarcade in #860
• fix #837 : file logger: compression is no more blocking by @dmachard in #854
• fix #636 : file logger: no more block postrotate command by @dmachard in #855
• chore(docs): update documentations by @dmachard in #861
• chore(deps): bump github.com/tinylib/msgp from 1.2.2 to 1.2.3 by @dependabot in #858
• chore(deps): bump github.com/prometheus/common from 0.60.0 to 0.60.1 by @dependabot in #857
• chore(deps): bump github.com/tinylib/msgp from 1.2.3 to 1.2.4 by @dependabot in #863
• chore(deps): bump github.com/fsnotify/fsnotify from 1.7.0 to 1.8.0 by @dependabot in #864

Bug reporters

• @yadavp27 #837
• @romainw #636

New Contributors

• @tomarcade made their first contribution in #860

Full Changelog: v1.1.0...v1.2.0

v1.1.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some fixes and increase performance with file logger.

What's Changed

• feat: disable global telemetry by default + channel buffer size up to 8192 by @dmachard in #840
• feat: file logger: reducing the frequency of disk writes by @dmachard in #841
• feat: add arcount, nscount and ancount in json output #828 by @dmachard in #829
• feat: lookup ECS ip on geoip transform #776 by @dmachard in #830
• feat: prometheus logger: add new counter for dns messages operations by @dmachard in #852
• feat: set go minimum version to 1.21 by @dmachard in #851
• fix #849 : hang during shutdown with high load by @dmachard in #850
• fix: invalid ingress counter for dnstap collector by @dmachard in #839
• fix #831 : afpacket sniffer: fix deadlock on stop under load by @dmachard in #842
• fix: error on dns decoder for response without question by @dmachard in #843
• fix(doc): update transform_rewrite.md by @jpmens in #825
• chore(deps): bump github.com/tinylib/msgp from 1.2.1 to 1.2.2 by @dependabot in #834
• chore(deps): bump github.com/klauspost/compress from 1.17.9 to 1.17.11 by @dependabot in #824 #846
• chore(deps): bump github.com/prometheus/client_golang from 1.20.3 to 1.20.5 by @dependabot in #823 #847
• chore(deps): bump golang.org/x/net from 0.29.0 to 0.30.0 by @dependabot in #832
• chore(deps): bump github.com/prometheus/common from 0.59.1 to 0.60.0 by @dependabot in #833
• chore(deps): bump golang from 1.23.1-alpine3.19 to 1.23.2-alpine3.19 by @dependabot in #835
• chore(deps): bump github.com/grafana/loki/v3 from 3.2.0 to 3.2.1 by @dependabot in #848
• chore(deps): bump google.golang.org/protobuf from 1.34.2 to 1.35.1 by @dependabot in #844

Bug reporters

• @BarboraAni #831

New Contributors

• @jpmens made their first contribution in #825

Full Changelog: v1.0.0...v1.1.0

v1.0.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

The release is the first stable release with great new features.

What's Changed

• feat: log malformed packet enabled by default by @dmachard in #819
• feat: build with go1.23 by @dmachard in #818
• feat: add tests for matching feature with pipeline mode by @dmachard in #797
• feat: convert latency field to float + new advanced example by @dmachard in #758
• feat: support round robin partitionner for kafka producer by @dmachard in #759 #809
• feat: update README.md and remove beta code warning by @dmachard in #755
• feat: add rewrite transform + quote some text directives by @dmachard in #756
• feat: update normalize transform to replace non printable characters by @dmachard in #754
• feat: add custom text format with jinja templating by @dmachard in #748
• feat: enhanced text-format with raw text by @peterand-pa in #609
• feat: move docker config to pipelines by @dmachard in #757
• fix: matching with arrays failed with dnsmessage collector by @dmachard in #797
• fix: decode properly dynamic/update dns queries by @dmachard in #819
• chore(deps): bump to loki 3.2 by @dmachard in #742 #768 #782 #817
• chore(docs): update README.md by @dmachard in #760
• chore(deps): bump github.com/Azure/sdk/azidentity from 1.5.1 to 1.6.0 by @dependabot in #743
• chore(deps): update unbound and pdns-recursor version in CI by @dmachard in #765
• chore(deps): bump github.com/prometheus/common from 0.48.0 to 0.59.1 by @dependabot in #726 #762 #806
• chore(deps): bump github.com/miekg/dns from 1.1.59 to 1.1.62 by @dependabot in #744 #788
• chore(deps): bump google.golang.org/protobuf from 1.34.1 to 1.34.2 by @dependabot in #745
• chore(deps): bump github.com/klauspost/compress from 1.17.8 to 1.17.9 by @dependabot in #746
• chore(deps): bump alpine from 3.20.0 to 3.20.3 by @dependabot in #752 #778 #802
• chore(deps): bump prometheus from 0.51.0 to 0.54.1 by @dependabot in #751 #773 #781 #794
• chore(deps): bump github.com/dmachard/go-dnstap-protobuf from 1.0.1 to 1.0.2 by @dependabot in #750
• chore(deps): bump github.com/oschwald/maxminddb-golang from 1.13.0 to 1.13.1 by @dependabot in #761
• chore(deps): bump golang.org/x/net from 0.26.0 to 0.28.0 by @dependabot in #767 #780
• chore(deps): bump golang.org/x/sys from 0.22.0 to 0.25.0 by @dependabot in #779 #783 #803
• chore(deps): bump github.com/IBM/sarama from 1.43.2 to 1.43.3 by @dependabot in #787
• chore(deps): bump github.com/creasty/defaults from 1.7.0 to 1.8.0 by @dependabot in #789
• chore(deps): bump github.com/cilium/ebpf from 0.15.0 to 0.16.0 by @dependabot in #777
• chore(deps): bump github.com/tinylib/msgp from 1.1.9 to 1.2.1 by @dependabot in #766 #804
• chore(deps): bump golang from 1.22.4-alpine3.19 to 1.22.6-alpine3.19 by @dependabot in #770 #784
• chore(deps): bump google.golang.org/grpc from 1.64.0 to 1.64.1 by @dependabot in #772
• chore(deps): bump github.com/prometheus/client_golang from 1.19.1 to 1.20.3 by @dependabot in #805

Bug reporters

• @drenton1 #799

New Contributors

• @peterand-pa made their first contribution in #609

Full Changelog: v0.46.0...v1.0.0

v0.46.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

The release is the last one before the first stable release. In the meantime, this release introduces some great features:

• Some new global metrics to follow your pipeline performance and dropped packets.
• The multiplexer configuration mode is marked as DEPRECATED.
  Please to update your config with pipelines definition.
• The memory usage of the tool has been reduced.

What's Changed

• feat: add integration examples with Loki, Prometheus and InfluxDB by @dmachard in #740
• feat: new global settings for default buffer size set to 4096 instead of 65535 by @dmachard in #739
• feat: add global telemetry #641 #512 by @dmachard in #724
• feat: mark as deprecated the multiplexer mode by @dmachard in #719
• feat: add pid file support by @dmachard in #723
• fix(regression): dnstap files with ingestor not properly processed by @dmachard in #732
• fix: invalid error message with config checker by @dmachard in #733
• chore(deps): bump github.com/oschwald/maxminddb-golang from 1.12.0 to 1.13.0 by @dependabot in #725
• chore(deps): bump golang from 1.22.3-alpine3.19 to 1.22.4-alpine3.19 by @dependabot in #734
• chore(deps): bump golang.org/x/sys from 0.20.0 to 0.21.0 by @dependabot in #735
• chore(deps): bump goreleaser/goreleaser-action from 5 to 6 by @dependabot in #730
• chore(deps): bump golang.org/x/net from 0.25.0 to 0.26.0 by @dependabot in #736
• chore(deps): bump alpine from 3.19.1 to 3.20.0 by @dependabot in #721

Bug reporters

• @romainw #729

Full Changelog: v0.45.0...v0.46.0

v0.45.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

The focus of this update is again bug fixes and code factoring to prepare future stable release.

What's Changed

• fix: support properly yaml tag flow in config checker by @dmachard in #716
• fix: to avoid deadlock on stop with the XDP sniffer by @dmachard in #712
• chore(docs): add missing docs and tests for dnsmessage collector by @dmachard in #717
• chore(refactor): redesign transformers by @dmachard in #715
• chore(refactor): use external default library to set default values in config by @dmachard in #714
• chore(refactor): remove xdp and pkgutils packages by @dmachard in #710
• chore(refactor): use external go-netutils library by @dmachard in #713 @dmachard in #709
• chore(refactor): remove processors package by @dmachard in #704
• chore(deps): bump github.com/prometheus/client_golang from 1.19.0 to 1.19.1 by @dependabot in #706
• chore(deps): bump golang from 1.22.2-alpine3.19 to 1.22.3-alpine3.19 by @dependabot in #707
• chore(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.1 by @dependabot in #697
• chore(deps): bump golangci/golangci-lint-action from 5 to 6 by @dependabot in #700
• chore(deps): bump github.com/cilium/ebpf from 0.14.0 to 0.15.0 by @dependabot in #698
• chore(deps): bump github.com/dmachard/go-clientsyslog from 0.3.0 to 0.4.0 by @dependabot in #705
• chore(deps): bump github.com/IBM/sarama from 1.43.1 to 1.43.2 by @dependabot in #699

Full Changelog: v0.44.0...v0.45.0

v0.44.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

The focus of this update is on maintenance to prepare future stable release

• Fixed support for IP fragments with AFPACKET sniffer.
• Major code refactoring of the base code.
• Completely redesigned the configuration checker.

What's Changed

• feat: support IP defrag on afpacket sniffer by @dmachard in #691
• feat: redesign config checker to support freeform keys by @dmachard in #678
• feat: add Scalyr defaults by @pieterlexis-tomtom in #677
• fix: quick fix to avoid panic with fragmented packet by @dmachard in #684
• fix: kafka - add missing textformat setting by @dmachard in #694
• chore(refactor): major code refactoring and cleanup by @dmachard in #667 #702 #696 #701
• chore(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8 by @dependabot in #680
• chore(deps): bump golangci/golangci-lint-action from 4 to 5 by @dependabot in #692
• chore(deps): bump github.com/miekg/dns from 1.1.58 to 1.1.59 by @dependabot in #687
• chore(deps): bump github.com/dmachard/go-dnstap-protobuf from 1.0.0 to 1.0.1 by @dependabot in #688
• chore(deps): bump github.com/dmachard/go-powerdns-protobuf from 1.1.0 to 1.1.1 by @dependabot in #689

Bug reporters

• @top-secrett #683
• @pieterlexis-tomtom #676

Full Changelog: v0.43.0...v0.44.0

v0.43.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some great features

• Compression support on DNSTap collector and logger
• Custom Flat-JSON relabeling
• New ClickHouse logger thank to @zunnurainbadar

What's Changed

• feat: transform: normalize lowercase RR's by @dmachard in #665
• feat: build with golang 1.22 and 1.20 support removed by @dmachard in #660
• feat: added Clickhouse Logger by @zunnurainbadar in #349
• feat: add compression support on DNSTap client and server by @dmachard in #538
• feat: new relabeling transform by @dmachard in #645
• fix: relabel-config error in loki client by @dmachard in #664
• chore(docs): add more docs for the atags transformer by @dmachard in #656 #659
• chore(deps): bump golang.org/x/sys from 0.17.0 to 0.18.0 by @dependabot in #651
• chore(deps): bump golang.org/x/net from 0.21.0 to 0.22.0 by @dependabot in #652
• chore(deps): bump Loki to v2.9.6
• chore(refactor): code cleanup on transformers by @dmachard in #666

New Contributors

• @zunnurainbadar made their first contribution in #349

Bug reporters

• @rajmohanram #643

Full Changelog: v0.42.0...v0.43.0

v0.42.0

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some important bug fixes (memory leak) and minor features

• Fix memory leak with ElasticSearch logger, thank to @misaki-kawakami to report that.
• Optimizations to reduce CPU usage, more particularly for flat-json
• Add support for query-zone field on DNStap collector

What's Changed

• feat: decode dns class on queries and replies by @dmachard in #635
• feat: more docs for kafka and fluentd loggers, change channel buffer to 4096 by @dmachard in #637 #639
• feat: normalize transformer: add icann managed flag in dnsmessage by @dmachard in #640
• feat: refactor to optimize dnsmessage model by @dmachard in #632
• feat: dnstap collector: decode query zone field by @dmachard in #621
• feat: dnstap collector: ignore empty raw dns payload by @dmachard in #622
• fix: elasticsearch: release buffer properly to avoid memory leaks by @dmachard in #630 #644
• fix: build to support 32b platform by @dmachard in #620
• chore(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7 by @dependabot in #625
• chore(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #633
• chore(deps): bump golang.org/x/sys from 0.16.0 to 0.17.0 by @dependabot in #606
• chore(deps): bump golang.org/x/net from 0.20.0 to 0.21.0 by @dependabot in #607
• chore(deps): bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 by @dependabot in #616
• chore(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 by @dependabot in #647
• chore(deps): bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 by @dependabot in #648
• chore(ci): add timeout in github actions tests by @dmachard in #623
• chore(ci): add bench-go github actions by @dmachard in #638
• chore(ci): bump softprops/action-gh-release from 1 to 2 by @dependabot in #646

Bug reporters

• @mikeS7
• @misaki-kawakami

Full Changelog: v0.41.0...v0.42.0