Add collector for PowerDNS protocol buffers

[sevencastles]

Unfortunately DNSTAP support is quite limited in PowerDNS. Would it be possible to add a collector for PowerDNS protocol buffers ?

Thanks in advance,
Carlo

[dmachard]

external protobuf library for powerdns under development https://github.com/dmachard/go-powerdns-protobuf

[dmachard]

Support for powerdns protobuf is implemented in the new beta release 0.17.0-b3. Documentation is here.
This is a first attempt, any feedbacks will be appreciated.

[sevencastles]

Thanks for your work, first of all. I am testing the release and so far I am seeing only one issue. The field "query-ip" shows nil values, redacted example here:

[
   {
      "network":{
         "family":"INET",
         "protocol":"UDP",
         "query-ip":"\u003cnil\u003e",
         "query-port":"0",
         "response-ip":"10.0.0.1",
         "response-port":"53",
         "as-number":"-",
         "as-owner":"-"
      },
      "dns":{
         "length":54,
         "opcode":0,
         "rcode":"-",
         "qname":"www.apple.com.",
         "qname-public-suffix":"com",
         "qname-effective-tld-plus-one":"apple.com",
         "qtype":"A",
         "flags":{
            "qr":false,
            "tc":false,
            "aa":false,
            "ra":false,
            "ad":false
         },
         "resource-records":{
            "an":[
               
            ],
            "ns":[
               
            ],
            "ar":[
               
            ]
         },
         "malformed-packet":0
      },
      "edns":{
         "udp-size":0,
         "rcode":0,
         "version":0,
         "dnssec-ok":0,
         "options":[
            
         ]
      },
      "dnstap":{
         "operation":"DNSOutgoingQueryType",
         "identity":"my-powerdns-recursor-server",
         "timestamp-rfc3339ns":"2022-03-07T15:14:42.000041676Z",
         "latency":"-"
      },
      "geo":{
         "city":"-",
         "continent":"-",
         "country-isocode":"-"
      }
   },
   {
      "network":{
         "family":"INET",
         "protocol":"UDP",
         "query-ip":"\u003cnil\u003e",
         "query-port":"0",
         "response-ip":"10.0.0.1",
         "response-port":"53",
         "as-number":"-",
         "as-owner":"-"
      },
      "dns":{
         "length":210,
         "opcode":0,
         "rcode":"NOERROR",
         "qname":"www.apple.com.",
         "qname-public-suffix":"com",
         "qname-effective-tld-plus-one":"apple.com",
         "qtype":"A",
         "flags":{
            "qr":false,
            "tc":false,
            "aa":false,
            "ra":false,
            "ad":false
         },
         "resource-records":{
            "an":[
               {
                  "name":"",
                  "rdatatype":"CNAME",
                  "ttl":1473,
                  "rdata":"www.apple.com.edgekey.net."
               },
               {
                  "name":"",
                  "rdatatype":"CNAME",
                  "ttl":11228,
                  "rdata":"www.apple.com.edgekey.net.globalredir.akadns.net."
               },
               {
                  "name":"",
                  "rdatatype":"CNAME",
                  "ttl":926,
                  "rdata":"e6858.dscx.akamaiedge.net."
               },
               {
                  "name":"",
                  "rdatatype":"A",
                  "ttl":20,
                  "rdata":"23.222.19.65"
               }
            ],
            "ns":[
               
            ],
            "ar":[
               
            ]
         },
         "malformed-packet":0
      },
      "edns":{
         "udp-size":0,
         "rcode":0,
         "version":0,
         "dnssec-ok":0,
         "options":[
            
         ]
      },
      "dnstap":{
         "operation":"DNSIncomingResponseType",
         "identity":"my-powerdns-recursor-server",
         "timestamp-rfc3339ns":"2022-03-07T15:14:42.000045485Z",
         "latency":"0.003809"
      },
      "geo":{
         "city":"-",
         "continent":"-",
         "country-isocode":"-"
      }
   }
]

This is with PowerDNS Recursor with outgoingProtobufServer(). protobufServer() doesn't show this behaviour.

[dmachard]

Thank for the feedback,
a new beta 0.17.0-b5 is available with the fix.

[sevencastles]

I can confirm the fix works. For the rest, I've been running the collector for a day, with no issues.

[dmachard]

v0.17.0 release available