Add filters to the search function

[MatMoore]

Context

We have a UI component for filters from https://design-patterns.service.justice.gov.uk/components/filter/

Currently we are just showing the example and it doesn't do anything. So we should make it possible to apply the filters when calling the search method of the catalogue client.

In scope

• Decide what should be filterable
• Make it possible to pass in filters
• Include the filters in the GraphQL query
• (Optional) The search query can return possible facet information via https://datahubproject.io/docs/graphql/objects#facetmetadata - can we use this to extract the possible filter values and the number of results with each value?

Out of scope for now

• Accessible autocomplete to narrow down the filter options

[MatMoore]

I did some investigation to figure out what names can be passed in as filters

In the default frontend, these are the allowed filter fields:

export const ENTITY_FILTER_NAME = '_entityType';
export const LEGACY_ENTITY_FILTER_NAME = 'entity';
export const ENTITY_INDEX_FILTER_NAME = '_index';
export const ENTITY_SUB_TYPE_FILTER_NAME = '_entityType␞typeNames';
export const TAGS_FILTER_NAME = 'tags';
export const GLOSSARY_TERMS_FILTER_NAME = 'glossaryTerms';
export const CONTAINER_FILTER_NAME = 'container';
export const DOMAINS_FILTER_NAME = 'domains';
export const OWNERS_FILTER_NAME = 'owners';
export const TYPE_NAMES_FILTER_NAME = 'typeNames';
export const PLATFORM_FILTER_NAME = 'platform';
export const FIELD_TAGS_FILTER_NAME = 'fieldTags';
export const EDITED_FIELD_TAGS_FILTER_NAME = 'editedFieldTags';
export const FIELD_GLOSSARY_TERMS_FILTER_NAME = 'fieldGlossaryTerms';
export const EDITED_FIELD_GLOSSARY_TERMS_FILTER_NAME = 'editedFieldGlossaryTerms';
export const FIELD_PATHS_FILTER_NAME = 'fieldPaths';
export const FIELD_DESCRIPTIONS_FILTER_NAME = 'fieldDescriptions';
export const EDITED_FIELD_DESCRIPTIONS_FILTER_NAME = 'editedFieldDescriptions';
export const DESCRIPTION_FILTER_NAME = 'description';
export const REMOVED_FILTER_NAME = 'removed';
export const ORIGIN_FILTER_NAME = 'origin';
export const DEGREE_FILTER_NAME = 'degree';

Note ENTITY_SUB_TYPE_FILTER_NAME is a special case, where two filters on the backend are presented as one on the frontend. In the query parameter the two fields are encoded using 0x1e record separator characters, and then these are converted into two combined filters by the generateInputWithNestedFilters function.

Presumably any field that is part of the elasticsearch document structure can be used.

From slack I got an example of a customProperty filter. This includes the customer property key as part of the value:
orFilters: [{and: [{field: "customProperties", values: ["prop1=value1"]}]}]}

If we just want to filter on the existence of a customProperty, then we can pass the custom property key as the value instead: https://datahubproject.io/docs/how/search#graphql:~:text=Find%20a%20dataset%20with%20a%20property%2C%20encoding

Based on this, I've allowed passing in domains, tags, and customProperties as valid filters. Anything else will be rejected.
We are not yet using containers, so I've skipped that one for now. We could also use browsePaths to filter e.g. tables within a database.

Note that the domains value is a list of IDs, so we need to know the IDs of each domain.