Abysmal response time when fetching pages to be listed in page attributes. 2nd try

[bobbingwide]

Describe the bug
The time to populate the Parent Page: pull down in the Page Attributes settings box is still abysmal.

I'm raising this since #5376 was closed without being fixed.
I re-discovered the problem while attempting to resolve a similar problem to that reported in #9441.

To Reproduce

1. Edit a hierarchical post type, such as Pages, in a site which has a lot of pages and where the pages contain multiple shortcodes which could take a long time to expand.
2. Measure the time taken to populate the Parent Page: pull down.
3. Tabulate the results and compare with the time to load the classic-editor.

Expected behavior
Page parent to be populated as quickly or faster than in the Classic Editor.

Observed behaviour
Multiple REST requests of the form
https://s.b/oikcom/wp-json/wp/v2/pages?per_page=100&exclude=7413&parent_exclude=7413&orderby=menu_order&order=asc&context=edit&_locale=user
with the second and subsequent requests including &page=n

• Each REST request took an absolute age to respond.
• Every single page was being expanded, and all the shortcodes were being expanded
• For both the_content and get_the_excerpt.
• When the elapsed execution time exceeded 120 seconds my local server terminated the request.
• Changing my code to not expand shortcodes reduced the elapsed time so that I did finally get to see the Parent Page: dropdown.

I notice that that requests do not include the _fields=id,title,parent parameter that was suggested would help with the performance.

When I add this parameter manually the performance is significantly better.

https://s.b/oikcom/wp-json/wp/v2/pages?per_page=100&exclude=7413&parent_exclude=7413&orderby=menu_order&order=asc&_locale=user&_fields=id,parent,title

Server execution reduced from 4.3 secs to 0.9
Note: Tested in browser with &context=edit removed.

Screenshots

If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows
• Browser: Chrome

Additional context

• WordPress 5.0.3 with / without Gutenberg 4.9

[youknowriad]

This is issue could be considered a duplicate of #7384 as we need a paginated select component to solve it. An quick win could be to increase the maximum number of items fetched per page to reduce the number of the requests being performed.

[bobbingwide]

An quick win could be to increase the maximum number of items fetched per page to reduce the number of the requests being performed.

That won't really reduce the amount of time wasted in the server. Each page will still be expanded to return the content and excerpt.

For my CPT called oik_file, with 511 posts, each request of 100 takes approximately 6 seconds.
Adding _fields=id,title,parent reduces the server time to 0.97

[youknowriad]

That's also a good thing to consider _fields but it's specific to this particular dropdown

[aduth]

I'm closing #15115 as a duplicate of this issue, which proposes the same in adding the _fields parameter. There's also #15114 which touches on related issues surrounding reuse of data amongst _fields-qualified requests.

[adamsilverstein]

Thanks for the _fields suggestion @bobbingwide - I added that to #16666 which should also resolve this issue, can you please give that a try?

[bobbingwide]

Thanks for the _fields suggestion @bobbingwide - I added that to #16666 which should also resolve this issue, can you please give that a try?

Hi, I'm using Gutenberg 7.1.0 now and I still have a problem with the solution.
I've realised I'm not actually testing your fix, but I'm going to document it anyway.

The popup list for Parent Page: often appears with duplicate entries.
For example: oik_api custom post type

Chrome inspector: HTML for the <select> controls options

Chrome inspector: Network requests.
Shows 4 requests. Post ID 7689 is only returned on the first response. Parent 7687.

[bobbingwide]

OK. So no progress is being made on this or #9441, which is approaching a year since I documented my findings. I was unable to test #16666. @adamsilverstein what do you suggest? Is there a fork I can easily access to try it?

[bobbingwide]

I don't know how many other defects are associated with this issue but we suffered it again today in a site which only had just over 140 pages, but where each page contained a number of shortcodes ( from tablepress ) that were being unnecessarily expanded.

We also noted another issue where the page list that was returned was almost empty. We didn't complete the investigation of the root cause but had a hunch it was due to all pages having a menu order of 0. Changing the menu order for a couple of pages appeared to resolve the issue.

In my opinion this is not an enhancement - it's a bug.
As previously stated, the Classic editor and Quick edit have no problems obtaining the list of pages for the Parent field.

[bobbingwide]

We also noted another issue where the page list that was returned was almost empty

Related: #20430, #13991

The popup list for Parent Page: often appears with duplicate entries.

Related: #12795

[joyously]

It seems this particular call should not have context=edit.

[bobbingwide]

It seems this particular call should not have context=edit.

I believe context=edit is required to ensure that the author's capabilities are respected.

[bobbingwide]

OK, so as you can see from the above, I've finally developed a fix. Added one line of code!
Now to look at the orderby problem, server side.

[bobbingwide]

I've finally developed a fix. Added one line of code!

Apparently it's been tried and deemed no good. Back to the drawing board?