Implement HttpContent.writeTo() async API

[lorban]

Implementation of the HttpContent.writeTo() async API.

Fixes #8790

[lorban]

@gregw There are two things I take from you previous comment:

• org.eclipse.jetty.server.Components should expose getSizedByteBufferPool() instead of getByteBufferPool() as its implementations should always have a default size. IOResources too should be modified to take a sized buffer pool where it expects the pool + size + directness triplet. This refactoring should be done independently in another PR.

• Making HttpContent.writeTo() take a buffer pool as one of its arguments looks terrible from an API perspective: there is no fundamental reason why writing would require a buffer pool, and most implementations don't: only ResourceHttpContent needs one.
  In my thinking, the sink fundamentally is linked to the channel that does hold a sized pool so we should just expose that by making writeTo() take a subinterface of Content.Sink. This is what I did and it doesn't look too bad, and definitely better than writeTo() taking a sized pool which I tried too.

[gregw]

@lorban I have created #12046 that improves the use of Sized ByteBufferPools.

[lorban]

@gregw I like #12046 but it's not solving where should ResourceHttpContent.writeTo() should find its buffer pool.

Let me try something...

[lorban]

Here is a summary of the status of the HttpContent overhaul:

in progress:

• add sized ByteBufferPools in Components (Improve the usage of Sized ByteBufferPool #12046)
• add HttpContent.writeTo() (the core of this PR)
• no ByteBufferPool in the signature of HttpContent.writeTo() as only ResourceHttpContent needs it (another proposal in this PR)

still needed:

• remove HttpContent.release() (only needed by CachingHttpContentFactory, it's too complex and should be rewritten)
• javadoc HttpContent

optional:

• Replace ByteBufferPool with sized ByteBufferPool everywhere a pool + size + directness is used (e.g.: IOResources)

[gregw]

@gregw I like #12046 but it's not solving where should ResourceHttpContent.writeTo() should find its buffer pool.

@lorban if you want to suck #12046 into this PR and close it, I'm OK with that.

[lorban]

@lorban if you want to suck #12046 into this PR and close it, I'm OK with that.

I prefer to focus solely on HttpContent.writeTo() here as I fear merging the two would turn this PR into an omnibus one. Those two concerns can be worked on separately and in parallel so I've reverted any related changes from this PR.

Answered.

[lorban]

Latest update: after a conversation with @sbordet, he managed to convince me that the HttpConfiguration's outputBufferSize and useOutputDirectByteBuffers are unrelated to the buffer size/directness the ResourceHttpContent should use.

ResourceHttpContent is the only HttpContent implementation that requires a sized buffer pool to use in writeTo(), other HttpContent implementations use buffers with freely decided size/directness.

So I've modified ResourceHttpContentFactory so that it takes a sized buffer pool that can then be used by ResourceHttpContent, the buffer size and directness being configured by the instantiators of ResourceHttpContentFactory, i.e.: ResourceServlet or ResourceService which allow configuring those parameters via some servlet init parameters and setters respectively.

So the final signature if the new async API finally has this elegant form:

void writeTo(Content.Sink sink, long offset, long length, Callback callback);

[gregw]

I'm not entirely convinced that the directness is unrelated. But am happy to have an elegant API and use configuration to align sizes and directness.

[lorban]

Followup PRs:

• Improve the usage of Sized ByteBufferPool #12046
• Use ByteBufferPool.Sized more #12060
• Cleanup and javadoc HttpContent API #12069

Once all those have been merged, the HttpContent rework can be considered done.

[gregw]

Very close now....

Handled.