fix: remote engine: handle mistral error on stream mode #1871
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe Your Changes
This pull request includes several changes to the
engine
module, focusing on improving error handling, bypassing model checks, and ensuring proper model path assignment. The most important changes include updates to theStartModel
function inmodels.cc
, theStreamWriteCallback
function inremote_engine.cc
, and theStartModel
method inmodel_service.cc
.Error handling improvements:
engine/extensions/remote-engine/remote_engine.cc
: Updated theStreamWriteCallback
function to useJson::Reader
for parsing JSON strings, improving error handling for JSON parsing.Model path assignment:
engine/controllers/models.cc
: Modified theStartModel
function to assign thellama_model_path
tomodel_path
ifmodel_path
is null, ensuring the correct model path is used.Model check bypass:
engine/services/model_service.cc
: Added a condition to bypass the model check for vision models, allowing theStartModel
method to proceed without checking for model existence in the database.Code formatting:
engine/services/model_service.cc
: Adjusted the formatting of the error message returned when a model fails to start, ensuring better readability.Fixes Issues
Self Checklist