admin: limit number of bytes an admin point can emit

[jmarantz]

Title: admin endpoints may yield more bytes than browsers can hold in memory, and it should be possible (and maybe default behavior) to limit that.

Description:
See related issue #16139 which is for /stats/prometheus in particular. But the same issue is present for other admin endpoints.

There are several approaches to this, which may need to be combined in some cases.

1. don't buffer up data in the server but flush it out to the admin http stream periodically. this is done for /stats but not for other endpoints that need it. This addresses server memory for the most part. It is also not done yet for /stats/prometheus though. Prom stats perf improvements #24998 attempts to fix; it proved harder than expected.
2. use high/low watermark hooks to pause generating and streaming more content until the client has read it. I implemented this but failed to write a test proving it worked: admin: apply flow control via high/low watermarks #31087
3. use a paginated API to only provide explicitly asked for data. I implemented this in stats but did not fully develop it. (WiP admin: Add paging UI to admin stats page. #19413)
4. use a hierarchical API to segment the data being asked for. I implemented this for stats also, including a UI with an outline-type view like a file-explorer, and ajax requests to fill in scopes as you expand them. This worked really nicely but I never got the PR ready for review. (RFC admin: hierarchical stats viewer #18670)
5. Simply limit the default number of bytes any admin endpoint can emit, and provide a query-param to override the default. The risk is that if the output format is JSON then truncating it would break it. However, https://github.com/envoyproxy/envoy/blob/main/source/common/json/json_streamer.h could easily be enhanced to give it a maximum byte count, after which it will auto-close all hierarchy and suppress further streaming. That API is used only for /stats?format=json (it's significantly faster than populating a protobuf and serializing) but it could be used for other endpoints to safely limit output.

@nezdolik
@rulex123

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.