GraphQL: Support for filtering on custom fields

[emil-nasso]

NetBox version

v3.0.3

Feature type

New functionality

Proposed functionality

There doesn't seem to be a way to filter based on custom fields in the GraphQL-api.

The documentation mentions "The GraphQL API employs the same filtering logic as the UI and REST API" so one approach would be to just follow the same pattern where cf_foo_bar etc are exposed as arguments for the list query of an entity where foo_bar is a custom field.

(Reference to discussion about this: #7569 )

Use case

We have the use for this when listing tenants via the graphql API. We put the customers name in the name field but we have a custom field where we have an internal identifier that is used as a reference when linking the data in netbox up with other systems though integrations.

Database changes

No response

External dependencies

No response

[candlerb]

Another way might be via a graphql query on custom_fields. I note that you can ask for custom_fields in a response:

# curl -sS -X GET -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8001/graphql/ \
  --data '{"query": "query {device_list(id:\"5\") {id custom_fields}}"}' | python3 -m json.tool
{
    "data": {
        "device_list": [
            {
                "id": "5",
                "custom_fields": {
                    "snmp_module": [
                        "mikrotik_secret"
                    ]
                }
            }
        ]
    }
}

(There's also custom_field_data which returns a string holding JSON). But as far as I can tell, you can't currently filter on custom_fields in a query:

# curl -sS -X GET -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8001/graphql/ \
  --data '{"query": "query {device_list(custom_fields:{snmp_module:\"mikrotik_secret\"}) {id custom_fields}}"}' | python3 -m json.tool
{
    "errors": [
        {
            "message": "Unknown argument \"custom_fields\" on field \"device_list\" of type \"Query\".",
            "locations": [
                {
                    "line": 1,
                    "column": 20
                }
            ]
        }
    ]
}

It would of course be good if such a query could be pushed down to a SQL condition in postgres, rather than making Netbox retrieve all objects from the database and discard the ones which don't match.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our contributing guide.

[jeremystretch]

Blocked by #9856

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[arthanson]

Blocked on #11949

[arthanson]

Adding blocked until #9856

[sirtux]

@arthanson Since #9856 is done, can you unblock this issue? Any ideas if and when this is now going to happen?

[jeremystretch]

I've marked this as "needs owner" for anyone who would like to take a stab at implementing this for v4.0 (with the new Strawberry GraphQL backend). @arthanson anything in particular you'd like to point out for this?

[christophbeberweil]

I would like to take a stab at it :) Right now I am looking into extending extras.graphql.mixins.CustomFieldsMixin, which seems to be a promising candidate for this change. Does anyone have a concrete preference how this should be implemented? If so feel free to leave any pointers

[jeremystretch]

@christophbeberweil thanks, I've assigned this to you.

[christophbeberweil]

Hey @jeremystretch,

I think I found a way to enable filtering of custom fields in GraphQL. My solution works by extending netbox.graphql.filter_mixins.autotype_decorator, in such a way that create_attribute_and_function is called for each applicable CustomField of the decrated filter.

This works for str fields (foo_txt) but unfortunately, the filter method seems to iterate over the querystring and search for each occurrence of the letter in the custom field. While this does search, it is obviously not usable. Could I use StrFilterLookup instead? I saw its definition in the Graph_i_QL frontend, but not in the source code. Or is there another way to define the actual filter behaviour that I missed?

Speaking of the Graph_i_QL frontend, while the backend detects my new filter cf_foo_txt, but the lint in GraphiQL does not know of cf_foo_txt.

When I create a new custom field a_int for dcim.Device with type integer it is not recognized in the graphql api unless the server is restarted. Do you know of a way how to make graphql aware of changing custom fields at runtime, without the need for a server restart?

After I restart the server, resolve_field raises an exception (AttributeError: 'NoneType' object has no attribute 'model') in netbox.filtersets.BaseFilterSet.get_additional_lookups because the field_name custom_field_data__a_int queryied by get_model_field can not be found on the model Device. Interestingly, exactly the same code works fine for a CustomField of type str. What could be the reason for this?

Since these changes are not ready for merging, I did not open a merge request. Please consider this commit: ad78d04

Thanks :)

[christophbeberweil]

Hi @jeremystretch, did you have a chance to look at my questions yet? Maybe you could indicate if my changes go in the right directopn from your point of view?

Thanks

[jeremystretch]

@christophbeberweil I am not able to spend time on this, hence it being tagged for a volunteer. If you need help, I recommend reaching out to others in the community to see if they're able to assist.