feat: atoma provider #2082
feat: atoma provider #2082
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hi @francis2tm! Welcome to the ai16z community. Thanks for submitting your first pull request; your efforts are helping us accelerate towards AGI. We'll review it shortly. You are now a ai16z contributor!
…eason, usage metrics, and timestamp conversion
… modelClass options
…K version change - Added Atoma API URL and model configurations to .env.example. - Updated pnpm-lock.yaml to reference the latest Atoma SDK version. - Enhanced index.ts to include Atoma as a model provider for token retrieval. - Implemented Atoma model initialization and response handling in generation.ts. - Updated models.ts to utilize dynamic settings for Atoma model classes.
…dels.ts - Deleted EMBEDDING_ATOMA_MODEL and IMAGE_ATOMA_MODEL from .env.example. - Removed corresponding model class settings from models.ts to streamline configuration.
…and consistency - Updated Atoma model settings in models.ts to encapsulate parameters (stop, maxInputTokens, maxOutputTokens, temperature) within each model class. - Reformatted code for better readability and maintainability. - Minor formatting adjustments in types.ts for consistency in enum declaration and interface method signatures.
- Standardized import statements and removed unnecessary line breaks. - Simplified function definitions and improved inline comments for clarity. - Enhanced logging messages for better debugging and error tracking. - Ensured consistent use of single quotes for strings throughout the file.
- Deleted the handleAtoma function, which was responsible for generating objects for Atoma models. - This change simplifies the codebase by removing unused functionality, improving maintainability.
|
@coderabbitai review |
📝 WalkthroughWalkthroughThe pull request introduces integration for the Atoma SDK across multiple configuration and source files. The changes add a new model provider called "Atoma" with support for small, medium, and large model configurations. The implementation includes environment configuration, token retrieval, text generation capabilities, and type definitions to seamlessly incorporate the Atoma model into the existing system architecture. Changes
Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
packages/core/src/models.ts (1)
1001-1032: Consider differentiating models for different size classes.The current configuration uses the same model "meta-llama/Llama-3.3-70B-Instruct" for all size classes. Consider using different models optimized for each class to provide better resource utilization and performance options.
Add frequency and presence penalties for better text generation.
Add
frequency_penaltyandpresence_penaltyparameters to align with other providers and improve text generation quality:[ModelProviderName.ATOMA]: { endpoint: settings.ATOMA_API_URL || "https://api.atoma.network/v1", model: { [ModelClass.SMALL]: { name: settings.SMALL_ATOMA_MODEL || "meta-llama/Llama-3.3-70B-Instruct", stop: [], maxInputTokens: 128000, maxOutputTokens: 8192, temperature: 0.7, + frequency_penalty: 0.4, + presence_penalty: 0.4, }, // Apply similar changes to MEDIUM and LARGE classesConsider adding embedding and image model support.
Many providers support embedding and image models. Consider adding these capabilities if the Atoma provider supports them.
.env.example (1)
89-95: Consider differentiating model tiers and enhancing documentation.
- All model tiers currently default to the same model. Consider using different models for each tier to better balance performance and cost.
- The API URL comment mentions "Default:" but doesn't specify the value.
- The bearer token format and requirements should be documented.
Apply this diff to improve the configuration block:
# Atoma SDK Configuration -ATOMASDK_BEARER_AUTH= # Atoma SDK Bearer Auth token -ATOMA_API_URL= # Default: https://api.atoma.network/v1 -SMALL_ATOMA_MODEL= # Default: meta-llama/Llama-3.3-70B-Instruct -MEDIUM_ATOMA_MODEL= # Default: meta-llama/Llama-3.3-70B-Instruct -LARGE_ATOMA_MODEL= # Default: meta-llama/Llama-3.3-70B-Instruct +ATOMASDK_BEARER_AUTH= # Atoma SDK Bearer Auth token (format: at-xxxxxxxxxxxx) +ATOMA_API_URL=https://api.atoma.network/v1 # Default: https://api.atoma.network/v1 +SMALL_ATOMA_MODEL=meta-llama/Llama-3.3-7B-Instruct # Default: meta-llama/Llama-3.3-7B-Instruct +MEDIUM_ATOMA_MODEL=meta-llama/Llama-3.3-70B-Instruct # Default: meta-llama/Llama-3.3-70B-Instruct +LARGE_ATOMA_MODEL=meta-llama/Llama-3.3-140B-Instruct # Default: meta-llama/Llama-3.3-140B-Instruct
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.env.example(1 hunks)agent/src/index.ts(1 hunks)packages/core/src/generation.ts(2 hunks)packages/core/src/models.ts(1 hunks)packages/core/src/types.ts(3 hunks)
🔇 Additional comments (4)
packages/core/src/types.ts (1)
232-232: LGTM!The ATOMA provider is correctly added to both the Models type and ModelProviderName enum, maintaining type safety and following the established pattern.
Also applies to: 268-268
agent/src/index.ts (1)
476-480: LGTM!The ATOMA case in getTokenForProvider follows the established pattern and correctly handles token retrieval from both character settings and global settings.
packages/core/src/generation.ts (1)
966-994: LGTM!The ATOMA case in generateText is well-implemented, using createOpenAI with proper configuration and following the established pattern for text generation.
.env.example (1)
89-89: LGTM! Proper section placement.The Atoma SDK configuration block is correctly placed in the "Model Provider Configurations" section, maintaining alphabetical ordering.
Background
Atoma is a decentralized private and verifiable AI execution network. It provides the infrastructure for developers and enterprises to use open-source AI models and build AI applications with full privacy.
What does this PR do?
Adds Atoma Network as an inference provider
What kind of change is this?
Features (non-breaking change which adds functionality)
Web3 Agents deserve a decentralized AI cloud.
Documentation changes needed?
Yes, add atoma provider docs
Testing
Ran
pnpm start --character="characters/trump.character.json"with"modelProvider": "atoma"Where should a reviewer start?
Detailed testing steps