Backport of NET-7813 - DNS : SERVFAIL when resolving PTR records into release/1.18.x

[hc-github-team-consul-core]

Backport

This PR is auto-generated from #20679 to be assessed for backporting due to the inclusion of the label backport/1.18.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@jmurret
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.

---

Description

For PTR records, the recursors enabled flag is not obeyed. It will return a SERVFAIL record in this case.

The acceptance criteria is:

• Given recursors are not enabled and regardless of whether the recursors collection is populated, when a PTR record is requested that does not exist, then a response with the following is returned:
  • no answer or extra information
  • SOA record for the NS / NXDOMAIN
  • RCode is Name Error

NOTE: Some reverse lookup tests were moved from dns_test.go and dns_service_lookup_test.go into the new dns_reverse_lookup_test.go file so that we did not have to add the new TestDNS_ReverseLookup_NotFound() test into dns_test.go as it it already huge and has issues displaying in Github's UI

Testing & Reproduction steps

without recursors:

1. consul agent -dev
2. dig -p 8600 -x 127.0.0.3 @localhost
   Expected Result:

; <<>> DiG 9.10.6 <<>> -p 8600 -x 127.0.0.3 @localhost
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 1214
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;3.0.0.127.in-addr.arpa.		IN	PTR

;; AUTHORITY SECTION:
consul.			0	IN	SOA	ns.consul. hostmaster.consul. 1708456317 3600 600 86400 0

;; Query time: 1 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Feb 20 12:11:57 MST 2024
;; MSG SIZE  rcvd: 107

with recursors array defined but without recursors enabled:

1. consul agent -dev -log-level=debug -hcl 'recursors = ["1.1.1.1"]'
2. dig -p 8600-x 1.1.1.1 @localhost +norecurse`
   Expected Result:

; <<>> DiG 9.10.6 <<>> -p 8600 -x 1.1.1.1 @localhost +norecurse
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 7655
;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;1.1.1.1.in-addr.arpa.		IN	PTR

;; AUTHORITY SECTION:
consul.			0	IN	SOA	ns.consul. hostmaster.consul. 1708456427 3600 600 86400 0

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Feb 20 12:13:47 MST 2024
;; MSG SIZE  rcvd: 105

Links

PR Checklist

• updated test coverage
• external facing docs updated
• appropriate backport labels added
• not a security concern

---

Overview of commits

• 6f643c0 - fbe92df - b3faeb8

[hashicorp-cla]

All committers have signed the CLA.

[github-team-consul-core-pr-approver]

Auto approved Consul Bot automated PR