Azure AI Inference SDK - Beta 2 updates #36163
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main reason for this release, shortly after the first release:
model
as an optional input argument to thecomplete
method ofChatCompletionsClient
. This is required for a high-visiblity project. For this project, developers must setmodel
.Breaking change (noted in CHANGELOG.md):
input_tokens
was removed from classEmbeddingsUsage
, as this was never defined in theREST API and the service never returned this value.
Other changes in this release:
model_extras
for sync and async clients. Make sure the additional parameters appear at the root of the JSON request payload, and make sure theunknown_parameters
HTTP request header was set topass_through
.model
argument, as well as all other arguments defined by the REST API. It will catch any regressions in hand-written code.extras
input argument to chat completions in hand-written code. The name was changed tomodel_extras
before the first release, but looks like we still had some left-over ref-doc comments that describe the no-longer-existing argument.sample_chat_completions_with_image_data.py
. Forgot to do that in the first release.complete
method withstream=True
returnsIterable[StreamingChatCompletionsUpdate]
forthe synchronous
ChatComletionsClient
, andIterable[StreamingChatCompletionsUpdate]
for the asynchronousChatCompletionsClient
. Per feedback from Anna T.