[UI] Pagination for artifact and execution list pages

[Bobgy]

Both pages show all content from server in one page.
Without server side pagination, this will be a blocker for frequent usages.

/cc @neuromage
Do you know if MLMD server supports pagination?

/area frontend
/area backend
/kind feature

[Bobgy]

/assign @Bobgy

[neuromage]

Both pages show all content from server in one page.
Without server side pagination, this will be a blocker for frequent usages.

/cc @neuromage
Do you know if MLMD server supports pagination?

It does not. There's another internal bug discussing the need for this, I'll cc you on that one.

/cc @hughmiao

[numerology]

+1. Pagination helps the scalability of MLMD as well.

[Bobgy]

Bumping priority because there are more feedback this is highly needed for scalable usage of the UI.
I can start working on this once the backend and db layer supports pagination.

[Bobgy]

Dropping priority, because I think #3457 will be a better way for navigating MLMD data. We can leverage existing KFP UI for pagination, sorting and filtering support.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Bobgy]

/lifecycle frozen

[jli]

just FYI: I noticed that the "Artifacts" and "Executions" pages are slow on my KFP 1.5.1 cluster. The /ml_metadata.MetadataStoreService/GetExecutions and /ml_metadata.MetadataStoreService/GetArtifacts endpoints both return >200MB of data:

It also seems like the Artifacts page then launches a GetEventByArtifactIDs request for each artifact concurrently, resulting in the browser using 100% CPU to send a huge number of requests, and the tab becoming unresponsive:

My team doesn't use the Artifacts or Executions pages, so this isn't a huge issue for us, but just sharing that these pages seem to become unusable for teams with a large number of runs.

[djwhatle]

I'm seeing every load of the artifacts page fail, seemingly due to the large number of requests (thousands) fired off all at once.

index.js:38 POST /ml_metadata.MetadataStoreService/GetEventsByArtifactIDs net::ERR_INSUFFICIENT_RESOURCES

Any update on current thoughts on how to improve this situation @Bobgy? Wondering if maintainers are leaning towards deprecating the artifacts page in favor of the run details page per #3457, or if pagination of artifacts is still on the roadmap?

[zijianjoy]

@djwhatle Pagination is on our roadmap to allow for better performance on Artifact page. We don't currently plan to deprecate this page. Artifact page is the entry for MLMD Lineage graph.

[thesuperzapper]

@zijianjoy, the current "Artifacts" page really is not acceptable, it is creating an unbelievable number of requests, we either need to remove this page, or prevent it from flooding the user's cluster with requests.

EDIT: I think the quickest "fix" is to limit the number of artifacts returned, at least until we can add pagination.

[Bobgy]

+1, before the page is in a usable state, maybe it's better to hide it first?

[rimolive]

Pagination was implemented in latest versions. Closing this issue.

/close

[google-oss-prow]

@rimolive: Closing this issue.

In response to this:

Pagination was implemented in latest versions. Closing this issue.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.