Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add response sender test base #7254

Merged
merged 8 commits into from
May 31, 2024
Merged

Conversation

kthui
Copy link
Contributor

@kthui kthui commented May 21, 2024

Set up the base for response sender testing, which enables testing varies ways a model may return response(s), including improper use of response sender.
Currently, this test only tests existing behaviors of decoupled and non-decoupled models, before any work is done on their data pipelines or response sender.

The control of testing is from the test script by setting the input tensors:

  • number_of_response_before_return
  • send_complete_final_flag_before_return
  • return_a_response
  • number_of_response_after_return
  • send_complete_final_flag_after_return

There are definite some improvements possible on the test base, i.e. reporting the size of requests, allow the final flag to be sent with the last response or separately, add support for dynamic batching requests. but I think those can be added (or already added) in next PRs when needed.

Next PRs:

@kthui kthui force-pushed the jacky-res-sender-test branch from 27b8509 to 15b5e56 Compare May 22, 2024 07:37
@kthui kthui marked this pull request as ready for review May 22, 2024 07:37
@kthui kthui requested review from Tabrizian and rmccorm4 May 22, 2024 07:43
Copy link
Contributor

@rmccorm4 rmccorm4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High level overview of tests look good to me, left some comments.

@kthui kthui requested a review from rmccorm4 May 25, 2024 02:30
@kthui kthui force-pushed the jacky-res-sender-test branch from bc3442e to 576a381 Compare May 30, 2024 23:20
@kthui kthui force-pushed the jacky-res-sender-test branch from 576a381 to 84d1422 Compare May 30, 2024 23:31
@kthui kthui requested a review from Tabrizian May 31, 2024 17:54
@kthui kthui merged commit 59fc4d4 into jacky-res-sender-main May 31, 2024
3 checks passed
@kthui kthui deleted the jacky-res-sender-test branch May 31, 2024 21:44
@kthui kthui restored the jacky-res-sender-test branch May 31, 2024 21:59
@kthui kthui deleted the jacky-res-sender-test branch May 31, 2024 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants