Add an API endpoint to load the last-used model #5516
+21
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds an internal API endpoint to the OpenAI API that allows loading of the last used model.
This new endpoint would be particularly helpful for scenarios where VRAM management is necessary - a third-party application can ask text-generation-webui to vacate VRAM (e.g. with
/v1/internal/model/unload
), then quickly reload the model that was just active once some other task is done (for example, image generation). That technique is employed in thesd_api_pictures
extension with AUTOMATIC1111's Web UI. This PR would allow other applications to perform the same technique with text-generation-webui./v1/internal/model/loadlast
triggers the newmodels.load_last_model()
function if it's POSTed to.As a bonus, this also fixes a bug in
models.reload_model()
- it would fail sinceshared.model_name
was set toNone
bymodels.unload_model()
, meaning thatreload_model()
would then attempt to loadNone
as a result. Setting it to the newly-addedshared.last_model_name
variable should fix that issue.