Add support for filtering, sorting, and pagination to Metadata Service

[prodonjs]

/kind feature
Target Q3

Describe the solution you'd like
These features should be added to the API to enable these operations in the UI. Ideally, List requests would have the same shape as requests in the Pipelines API so we could re-purpose most of the code there.

[prodonjs]

/area api

[prodonjs]

/improvement enhancement

[lilida]

Any progress on this?

[jtfogarty]

/kind feature
/priority p2

[apryiomka]

Currently hangs with substantial number of artifacts

[issue-label-bot]

Issue Label Bot is not confident enough to auto-label this issue.
See dashboard for more details.

[jlewi]

/cc @paveldournov @rmgogogo

[jlewi]

In slack @apryiomka reported that this is in connection with KFP.

As the number of KFP runs increases, Artifact store becomes slower and slower. Also it doesn't have a good pagination mechanism and now takes forever to load. Does anyone know if there is any work planned to address this?

[rmgogogo]

for pagination, there is already a ticket in kubeflow/pipelines.
@Bobgy

for the "Artifact store becomes slower and slower",
@jingzhang36 for perf

[Bobgy]

FYI, the tracking issue for pagination is kubeflow/pipelines#3226.

[Bobgy]

@lilida @apryiomka Do you use metadata with Kubeflow Pipelines?
Does kubeflow/pipelines#3457 help you? It shows execution details page when you click on a pipeline run step's Metadata tab. The tab has links to artifacts related to that executions.