tests.system.TestStreamingPull: test_streaming_pull_max_messages[rest-rest] failed

[flaky-bot]

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.

---

commit: 26d2b39
buildURL: Build Status, Sponge
status: failed

Test output

self = 
publisher = 
topic_path_base = 'projects/precise-truck-742/topics/t-1689361995945'
subscription_path_base = 'projects/precise-truck-742/subscriptions/s-1689361995947'
cleanup = [(>, ()...39ab0>>, (), {'subscription': 'projects/precise-truck-742/subscriptions/s-1689361995947-streaming-pull-max-messages'})]
def test_streaming_pull_max_messages(
    self, publisher, topic_path_base, subscription_path_base, cleanup
):
    subscriber = pubsub_v1.SubscriberClient(transport="grpc")
    custom_str = "-streaming-pull-max-messages"
    topic_path = topic_path_base + custom_str
    subscription_path = subscription_path_base + custom_str
    # Make sure the topic and subscription get deleted.
    cleanup.append((publisher.delete_topic, (), {"topic": topic_path}))
    cleanup.append(
        (subscriber.delete_subscription, (), {"subscription": subscription_path})
    )

    # create a topic and subscribe to it
    publisher.create_topic(name=topic_path)
    subscriber.create_subscription(name=subscription_path, topic=topic_path)

    batch_sizes = (7, 4, 8, 2, 10, 1, 3, 8, 6, 1)  # total: 50
    _publish_messages(publisher, topic_path, batch_sizes=batch_sizes)

    # now subscribe and do the main part, check for max pending messages
    total_messages = sum(batch_sizes)
    flow_control = types.FlowControl(max_messages=5)
    callback = StreamingPullCallback(
        processing_time=1, resolve_at_msg_count=total_messages
    )

    subscription_future = subscriber.subscribe(
        subscription_path, callback, flow_control=flow_control
    )

    # Expected time to process all messages in ideal case:
    #     (total_messages / FlowControl.max_messages) * processing_time
    #
    # With total=50, max messages=5, and processing_time=1 this amounts to
    # 10 seconds (+ overhead), thus a full minute should be more than enough
    # for the processing to complete. If not, fail the test with a timeout.
    try:


      callback.done_future.result(timeout=60)


tests/system.py:654:

self = None, timeout = 60
def result(self, timeout=None):
    """Return the result of the call that the future represents.

    Args:
        timeout: The number of seconds to wait for the result if the future
            isn't done. If None, then there is no limit on the wait time.

    Returns:
        The result of the call that the future represents.

    Raises:
        CancelledError: If the future was cancelled.
        TimeoutError: If the future didn't finish executing before the given
            timeout.
        Exception: If the call raised then that exception will be raised.
    """
    try:
        with self._condition:
            if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:
                raise CancelledError()
            elif self._state == FINISHED:
                return self.__get_result()

            self._condition.wait(timeout)

            if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:
                raise CancelledError()
            elif self._state == FINISHED:
                return self.__get_result()
            else:


              raise TimeoutError()


E                   concurrent.futures._base.TimeoutError
/usr/local/lib/python3.10/concurrent/futures/_base.py:448: TimeoutError
During handling of the above exception, another exception occurred:
self = <tests.system.TestStreamingPull object at 0x7f55475b28f0>

publisher = <google.cloud.pubsub_v1.PublisherClient object at 0x7f5544354a60>

topic_path_base = 'projects/precise-truck-742/topics/t-1689361995945'

subscription_path_base = 'projects/precise-truck-742/subscriptions/s-1689361995947'

cleanup = [(<bound method PublisherClient.delete_topic of <google.cloud.pubsub_v1.PublisherClient object at 0x7f5544354a60>>, ()...39ab0>>, (), {'subscription': 'projects/precise-truck-742/subscriptions/s-1689361995947-streaming-pull-max-messages'})]
def test_streaming_pull_max_messages(
    self, publisher, topic_path_base, subscription_path_base, cleanup
):
    subscriber = pubsub_v1.SubscriberClient(transport="grpc")
    custom_str = "-streaming-pull-max-messages"
    topic_path = topic_path_base + custom_str
    subscription_path = subscription_path_base + custom_str
    # Make sure the topic and subscription get deleted.
    cleanup.append((publisher.delete_topic, (), {"topic": topic_path}))
    cleanup.append(
        (subscriber.delete_subscription, (), {"subscription": subscription_path})
    )

    # create a topic and subscribe to it
    publisher.create_topic(name=topic_path)
    subscriber.create_subscription(name=subscription_path, topic=topic_path)

    batch_sizes = (7, 4, 8, 2, 10, 1, 3, 8, 6, 1)  # total: 50
    _publish_messages(publisher, topic_path, batch_sizes=batch_sizes)

    # now subscribe and do the main part, check for max pending messages
    total_messages = sum(batch_sizes)
    flow_control = types.FlowControl(max_messages=5)
    callback = StreamingPullCallback(
        processing_time=1, resolve_at_msg_count=total_messages
    )

    subscription_future = subscriber.subscribe(
        subscription_path, callback, flow_control=flow_control
    )

    # Expected time to process all messages in ideal case:
    #     (total_messages / FlowControl.max_messages) * processing_time
    #
    # With total=50, max messages=5, and processing_time=1 this amounts to
    # 10 seconds (+ overhead), thus a full minute should be more than enough
    # for the processing to complete. If not, fail the test with a timeout.
    try:
        callback.done_future.result(timeout=60)
    except exceptions.TimeoutError:


      pytest.fail(


            "Timeout: receiving/processing streamed messages took too long."
        )

E           Failed: Timeout: receiving/processing streamed messages took too long.
tests/system.py:656: Failed

[flaky-bot]

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (26d2b39), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

[parthea]

Closing as obsolete. Flaky bot will re-open if the test is still flaky