Dispatcher-based JoinableTaskFactory should post messages more conservatively

[AArnott]

In Visual Studio, we have case where 500MB or more in memory is occupied with already-executed SingleExecuteProtector objects in the WPF Dispatcher's queue. To avoid deadlocks in cases of modal dialogs, we always post every request to the UI thread via the JTF as well as a queue backed by Windows or some GUI framework. This provides at least two avenues for every main thread work item to be completed, often leaving the lower-level queue filled with dead work items.

We want to reduce the strain this puts on memory and CPU to hold and eventually process and dump these dead objects.

The proposal is for the JoinableTaskFactory.PostToUnderlyingSynchronizationContext method to enqueue to its own private queue, from which it can trim dead work items opportunistically. It will necessarily also queue a work item to whatever the lower-level queue is as it does today, except that it will take care to only do this if there isn't already such a work item in that queue. When that work item is serviced, that JTF object will process some or all of its private queue. If it yields control while its queue is non-empty, it will post another item to the lower level queue to request more time later.

The most aggravated case is that of the WPF Dispatcher, which is enqueued here:

vs-threading/src/Microsoft.VisualStudio.Threading/DispatcherExtensions.cs

Line 70 in 9065e6e

this.dispatcher.BeginInvoke(this.priority, callback, state);

We should consider adding the private queue and management functions to the base class, such that all derived classes can opt into using it easily. The default implementation and this WPF override are likely to both benefit. The derived type in Visual Studio that schedule work to TaskSchedulers may want to participate as well.