copy_both: handle Future cancellation case

[petrosagg]

When a user starts a CopyBoth query a state machine is created that is responsible for driving the CopyBoth protocol forward regardless of what the user does with the returned handles. The state machine starts in the Setup phase and only gives control back to the user (in the form of the returned handles) after having finished with the Setup. Therefore the code assumed (incorrectly) that the handles will never be dropped while in Setup mode.

It turns out there is one more way of dropping the handle even while in the Setup phase, and that is to cancel the entire Future object returned by the copy_both method at just the right time. This resulted in the state machine observing that the handles were dropped while still in the Setup phase, which was asserted to never happen.

This PR fixes the problem by handling the case where a handles are dropped while still in the Setup phase.

Fixes: https://github.com/MaterializeInc/database-issues/issues/8604

[def-]

Could we test this deterministically by adding a failpoint with a sleep?
Something like this in workflow_sink_kafka_restart:

Materialized(
            environment_extra=["FAILPOINTS=kafka_sink_commit_transaction=sleep(5000)"]
        )

[petrosagg]

Could we test this deterministically by adding a failpoint with a sleep?

We could but then this makes this PR non-upstreamable. I just had an idea that might work in a unit test though, will report back in a moment

[petrosagg]

@def- added a unit test and confirmed that it fails without this PR