Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add venice.ai api model provider #1008

Merged
merged 5 commits into from
Dec 13, 2024
Merged

feat: add venice.ai api model provider #1008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ce418ac
add venice api model provider
proteanx Dec 12, 2024
d5a6666
allow users to select models
proteanx Dec 12, 2024
ae47846
temp rm venice image gen model
proteanx Dec 12, 2024
e66242a
rm unused freq/presence penalty
proteanx Dec 12, 2024
d3e0f78
add back lock file
proteanx Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,12 @@ WALLET_SECRET_SALT= # ONLY DEFINE IF YOU WANT TO USE TEE Plugin, otherwise it wi
# Galadriel Configuration
GALADRIEL_API_KEY=gal-* # Get from https://dashboard.galadriel.com/

# Venice Configuration
VENICE_API_KEY= # generate from venice settings
SMALL_VENICE_MODEL= # Default: llama-3.3-70b
MEDIUM_VENICE_MODEL= # Default: llama-3.3-70b
LARGE_VENICE_MODEL= # Default: llama-3.1-405b

# fal.ai Configuration
FAL_API_KEY=
FAL_AI_LORA_PATH=
Expand Down
5 changes: 5 additions & 0 deletions agent/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,11 @@ export function getTokenForProvider(
character.settings?.secrets?.HYPERBOLIC_API_KEY ||
settings.HYPERBOLIC_API_KEY
);
case ModelProviderName.VENICE:
return (
character.settings?.secrets?.VENICE_API_KEY ||
settings.VENICE_API_KEY
);
}
}

Expand Down
23 changes: 23 additions & 0 deletions packages/core/src/generation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,29 @@ export async function generateText({
break;
}

case ModelProviderName.VENICE: {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just add this as a case around line 149, as venice follow the openAI api spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I tested this as well (definitely works) but I plan to add the venice_parameters that are exclusive to venice as they come out.

elizaLogger.debug("Initializing Venice model.");
const venice = createOpenAI({
apiKey: apiKey,
baseURL: endpoint
});

const { text: veniceResponse } = await aiGenerateText({
model: venice.languageModel(model),
prompt: context,
system:
runtime.character.system ??
settings.SYSTEM_PROMPT ??
undefined,
temperature: temperature,
maxTokens: max_response_length,
});

response = veniceResponse;
elizaLogger.debug("Received response from Venice model.");
break;
}

default: {
const errorMessage = `Unsupported provider: ${provider}`;
elizaLogger.error(errorMessage);
Expand Down
14 changes: 14 additions & 0 deletions packages/core/src/models.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,20 @@ export const models: Models = {
[ModelClass.IMAGE]: settings.IMAGE_HYPERBOLIC_MODEL || "FLUX.1-dev",
},
},
[ModelProviderName.VENICE]: {
endpoint: "https://api.venice.ai/api/v1",
settings: {
stop: [],
maxInputTokens: 128000,
maxOutputTokens: 8192,
temperature: 0.6,
},
model: {
[ModelClass.SMALL]: settings.SMALL_VENICE_MODEL || "llama-3.3-70b",
[ModelClass.MEDIUM]: settings.MEDIUM_VENICE_MODEL || "llama-3.3-70b",
[ModelClass.LARGE]: settings.LARGE_VENICE_MODEL || "llama-3.1-405b",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could probably add an image model here like flux-dev-unsencored as the default with the option to control in the .env file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had image generation in here but it needs more work so I figured I'd add it in a later PR. In the first commit here it's included though.

},
},
};

export function getModel(provider: ModelProviderName, type: ModelClass) {
Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ export type Models = {
[ModelProviderName.VOLENGINE]: Model;
[ModelProviderName.NANOGPT]: Model;
[ModelProviderName.HYPERBOLIC]: Model;
[ModelProviderName.VENICE]: Model;
};

/**
Expand Down Expand Up @@ -234,6 +235,7 @@ export enum ModelProviderName {
VOLENGINE = "volengine",
NANOGPT = "nanogpt",
HYPERBOLIC = "hyperbolic",
VENICE = "venice",
}

/**
Expand Down
Loading
Loading