NodeMeta filtering Catalog List Services API is not working

[tommyvicananza]

We are experiencing a incorrect working of filtering in Catalog API, according to documentation https://developer.hashicorp.com/consul/api-docs/catalog#list-services use of node-meta is deprecated and you suggest to use filter instead, using filter for NodeMeta keys are not working with /v1/catalog/services/?filter=NodeMeta.network_segment=="default" its response is empty with a 200 http code, however it works fine with a specific service /v1/catalog/service/<anyservice>/?filter=NodeMeta.network_segment=="default"

[dcanadillas]

I am experiencing this as well:

$ curl -ks -G -H "X-Consul-Token: $TOKEN" https://192.168.2.10/v1/catalog/service/frontend --data-urlencode filter="NodeMeta[\"synthetic-node\"] == true" | jq
[
  {
    "ID": "",
    "Node": "minikube-virtual",
    "Address": "192.168.39.239",
    "Datacenter": "dcanadillas",
    "TaggedAddresses": null,
    "NodeMeta": {
      "synthetic-node": "true"
    },
...
...
]

$ curl -ks -G -H "X-Consul-Token: $TOKEN" https://192.168.2.10/v1/catalog/services --data-urlencode filter="NodeMeta[\"synthetic-node\"] == true" | jq
{}

So, the NodeMeta filter is not working within the API request.

Other filters work as expected:

$ curl -ks -G -H "X-Consul-Token: $TOKEN" https://192.168.2.10/v1/catalog/services --data-urlencode filter="ServiceMeta[\"synthetic-node\"] == true" | jq
{
  "backend": [],
  "backend-sidecar-proxy": [],
  "counting": [],
  "counting-sidecar-proxy": [],
  "dashboard": [],
  "dashboard-sidecar-proxy": [],
  "frontend": [
    "v1.1-amd64",
    "v1.2-amd64"
  ],
  "frontend-sidecar-proxy": [
    "v1.1-amd64",
    "v1.2-amd64"
  ],
  "ingress-gateway": [],
  "mesh-gateway": []
}

$ curl -ks -G -H "X-Consul-Token: $TOKEN" https://192.168.2.10/v1/catalog/services --data-urlencode filter="ServiceKind == \"mesh-gateway\"" | jq
{
  "mesh-gateway": []
}

This is happening on these different Consul versions:

• Consul Ent 1.15.2
• Consul Ent 1.14.4

Expected outcome

The NodeMeta filter to the /v1/catalog/services endpoint should return the filtered response (example below):

{
  "backend": [],
  "backend-sidecar-proxy": [],
  "counting": [],
  "counting-sidecar-proxy": [],
  "dashboard": [],
  "dashboard-sidecar-proxy": [],
  "frontend": [
    "v1.1-amd64",
    "v1.2-amd64"
  ],
  "frontend-sidecar-proxy": [
    "v1.1-amd64",
    "v1.2-amd64"
  ],
  "ingress-gateway": [],
  "mesh-gateway": []
}

[lkysow]

I looked into this a bit. The problem is that here:

consul/agent/consul/catalog_endpoint.go

Lines 596 to 600 in 7f4fd27

	if len(args.NodeMetaFilters) > 0 {
	reply.Index, serviceNodes, err = state.ServicesByNodeMeta(ws, args.NodeMetaFilters, &args.EnterpriseMeta, args.PeerName)
	} else {
	reply.Index, serviceNodes, err = state.Services(ws, &args.EnterpriseMeta, args.PeerName)
	}

We uses state.Services if NodeFilters is empty (which it is when using the generic bexpr filter argument) which doesn't actually populate any NodeMeta fields and so the bexpr filtering doesn't work because all NodeMeta fields are nil.

Workaround for now is to use the deprecated node-meta query param to do the filtering. Proper fix would I think be to populate the NodeMeta fields (i.e. use a different state method), however we'd need to look at if that effects performance at all since it's querying/watching more tables.

[tommyvicananza]

Thank you @lkysow

Good to hear about your discovery, we are already using node-meta like workaround, I hope the fix will be no tricky.

Let us know whether is gonna be implemented in the future