-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor StreamInferHandler for better readability #6257
Conversation
src/grpc/stream_infer_handler.cc
Outdated
bool finished = false; | ||
|
||
if (state->step_ == Steps::START) { | ||
finished = RequestStartStep(state, rpc_ok); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we pull the step transition from within these helpers to this Process function? It would help in giving a complete picture of the state machine being implemented here.
If not, then may be add a comment describing to which state the state can transition to in these helper functions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be nice, but there's a lot of branching in each function where if (this) state = state1; else state = state2;
It would be non-trivial effort to extract this out I think.
I added some descriptive comments for now.
… rmccormick-stream-order
// Write the next response if it is ready... | ||
state->context_->WriteResponseIfReady(nullptr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tanmayv25 I don't understand this part. Do you know why we try to Write the next response (not the current' state's response) here in the WRITTEN state for a non-decoupled request?
For the non-decoupled case, isn't every response's write handled when calling the response callback or error handling for a given request?
src/grpc/stream_infer_handler.cc
Outdated
state->step_ = Steps::FINISH; | ||
// Finished with the request. | ||
return true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also @tanmayv25 , I don't really understand the point of the COMPLETE state/step. Why don't we directly transition to FINISH state anywhere we would transition to COMPLETE instead? Seems like an unnecessary intermediate step.
…, break out decoupled handling in Process() directly
Closing due to lots of changes around request cancellation. May come back to this later. |
This PR does two things:
step
inStreamInferHandler::Process
from one large ~400 line function into helper functions for each state.if not not_finished
into justif finished
in Common/InferHandler/StreamInferHandlerProcess()
functions. I found the double negative logic confusing that the function did:and then the caller checks the negation of that:
So instead if's more direct to be: