Update generation.ts to fix TOGETHER/LLAMACLOUD image generation #786
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lalalune
approved these changes
Dec 2, 2024
dsldsl
pushed a commit
to dsldsl/eliza
that referenced
this pull request
Dec 7, 2024
Update generation.ts to fix TOGETHER/LLAMACLOUD image generation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to:
Risks
Background
For use of LLAMACLOUD/TOGETHER via imageModelProvider: ModelProviderName.TOGETHER the API didn't appear to be correctly implemented.
What does this PR do?
Fixes the implementation of the together api in packages/core/src/generation.ts
What kind of change is this?
Bug fixes (non-breaking change which fixes an issue)
together API call was not fetching the generated image and storing it correctly
Documentation changes needed?
none
Testing
i added " imageModelProvider: ModelProviderName.TOGETHER," to defaultcharacter.ts
added a together API key to the .env file along with openai key
set the following
DISCORD_APPLICATION_ID=
DISCORD_API_TOKEN=
IMAGE_GEN=TRUE
USE_OPENAI_EMBEDDING=TRUE
USE_OLLAMA_EMBEDDING=TRUE
Where should a reviewer start?
Detailed testing steps
then proceeded to go to discord and asked the agent to generate an image of a cat girl
which resulted in the following in the console log
◎ LOGS
Generating image with prompt:
<@&1303451439787544669> draw me some anime cat girls with black hair and sparkly digital eyes
ℹ INFORMATIONS
Generating image with options:
{"imageModelProvider":"black-forest-labs/FLUX.1-schnell"}
◎ LOGS
Image generation successful, number of images:
1
◎ LOGS
Processing image 1:
generated_1733131072769_0
◎ LOGS
Generated caption for image 1: