Update RunRequest class to accept new create run parameters #360
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added parameters: - temperature - top_p - max_prompt_tokens - max_completion_tokens - truncation_strategy - response_format
|
Hello @aallam ! :) A small contribution to this amazing library, thank you for creating it! I didn't updated the Cheers! |
aallam
requested changes
Jun 23, 2024
| * What sampling temperature to use, between 0 and 2. | ||
| * Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | ||
| */ | ||
| @SerialName("temperature") val temperature: Int? = null, |
There was a problem hiding this comment.
Suggested change
| @SerialName("temperature") val temperature: Int? = null, | |
| @SerialName("temperature") val temperature: Double? = null, |
I believe this should be a floating number
| * An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. | ||
| * So 0.1 means only the tokens comprising the top 10% probability mass are considered. | ||
| */ | ||
| @SerialName("top_p") val topP: Int? = null, |
There was a problem hiding this comment.
Suggested change
| @SerialName("top_p") val topP: Int? = null, | |
| @SerialName("top_p") val topP: Double? = null, |
Same here :)
| * If the run exceeds the number of prompt tokens specified, the run will end with status `incomplete`. | ||
| * See `incomplete_details` for more info. | ||
| */ | ||
| @SerialName("max_prompt_tokens") val maxPromptTokens: Int? = null, |
There was a problem hiding this comment.
Suggested change
| @SerialName("max_prompt_tokens") val maxPromptTokens: Int? = null, | |
| @SerialName("max_prompt_tokens") val maxPromptTokens: Long? = null, |
I would suggest using Longs here
| * If the run exceeds the number of completion tokens specified, the run will end with status `incomplete`. | ||
| * See `incomplete_details` for more info. | ||
| */ | ||
| @SerialName("max_completion_tokens") val maxCompletionTokens: Int? = null, |
There was a problem hiding this comment.
Suggested change
| @SerialName("max_completion_tokens") val maxCompletionTokens: Int? = null, | |
| @SerialName("max_completion_tokens") val maxCompletionTokens: Long? = null, |
Same suggestion here
Comment on lines
+14
to
+20
| @SerialName("type") val type: String, | ||
|
|
||
| /** | ||
| * The number of most recent messages from the thread when constructing the context for the run. | ||
| */ | ||
| @SerialName("last_messages") val lastMessages: Int? = null | ||
| ) |
There was a problem hiding this comment.
Maybe we can improve a little bit the API here, with something like this:
Suggested change
| @SerialName("type") val type: String, | |
| /** | |
| * The number of most recent messages from the thread when constructing the context for the run. | |
| */ | |
| @SerialName("last_messages") val lastMessages: Int? = null | |
| ) | |
| @SerialName("type") val type: TruncationStrategyType, | |
| /** | |
| * The number of most recent messages from the thread when constructing the context for the run. | |
| */ | |
| @SerialName("last_messages") val lastMessages: Int? = null | |
| ) | |
| /** | |
| * The truncation strategy to use for the thread. | |
| */ | |
| @JvmInline | |
| @Serializable | |
| public value class TruncationStrategyType(public val value: String) { | |
| public companion object { | |
| public val TruncationStrategyType: Auto = TruncationStrategyType("auto") | |
| public val TruncationStrategyType LastMessages = TruncationStrategyType("last_messages") | |
| } | |
| } |
aallam
force-pushed
the
main
branch
2 times, most recently
from
ba88d2a to
91f971a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your change
Added properties to RunRequest class to reflect new parameters introduced in Assistant run API.
See https://platform.openai.com/docs/api-reference/runs/createRun
Added parameters list:
temperaturetop_pmax_prompt_tokensmax_completion_tokenstruncation_strategyresponse_formatWhat problem is this fixing?
It allows to use the new run API parameters.