Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improve twitter post content quality #763

Merged
merged 3 commits into from
Dec 1, 2024

Conversation

cygaar
Copy link
Collaborator

@cygaar cygaar commented Dec 1, 2024

Relates to:

Issues with duplicate twitter content being posted

Risks

Low

Background

What does this PR do?

Reduces duplication of Twitter content and reduces tokens used for twitter post generation.

The two key changes were:

  1. Removing previous posts from the generation prompt as this influenced the model into repeating previous content even though the prompt asked it not to. Previous posts do not provide value when generation posts. For tweet interactions we already include exists posts in the thread so including previous posts (timeline) is not necessary.
  2. Improve the prompts to remove unused blank lines and to better organize the structure of the context. These changes should improve response quality.

What kind of change is this?

Improvement

Documentation changes needed?

Testing

Where should a reviewer start?

Detailed testing steps

let homeTimeline: Tweet[] = [];
// read the file if it exists

const cachedTimeline = await this.client.getCachedTimeline();
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrectly named "timeline" - it was really just a list of the bot's previous posts

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meh, I’ve heard it called timeline. Profile timeline might be more apt and precise

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, we can revisit this in a future pr. didn't actually change the underlying function

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, if we need to have a list of previous posts for some analysis, we can quickly re-introduce it. For the PR its good

@cygaar cygaar requested a review from lalalune December 1, 2024 20:53
@cygaar cygaar changed the title fix: Improve twitter response content fix: improve twitter post content quality Dec 1, 2024
@twilwa
Copy link
Contributor

twilwa commented Dec 1, 2024

LGTM, as long as that discussion didn't change a variable name and break anything

@odilitime
Copy link
Collaborator

Not a review just comments:
While it’s hard to argue with results. I think the context of the recent public tweets could be useful for keep the agent focused and less random. But that can be reintroduced later when it’s more thought through and tested.

I also feel like some comments in the code explaining these changes would be helpful, so they’re not reverted.

@twilwa twilwa self-requested a review December 1, 2024 21:13
Copy link
Contributor

@twilwa twilwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cygaar
Copy link
Collaborator Author

cygaar commented Dec 1, 2024

I think since we're already providing a fair amount of context to the model about the agent (knowledge, topics), the responses tend to not be very random. They all seem relatively related to eachother even without providing the last few tweets.

Copy link
Member

@fabianhug fabianhug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, I added a comment regarding the prompt.

@lalalune mentioned this PR in his Discord stream, but I tuned in right when he stopped. Let's wait for his comment before merging

let homeTimeline: Tweet[] = [];
// read the file if it exists

const cachedTimeline = await this.client.getCachedTimeline();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, if we need to have a list of previous posts for some analysis, we can quickly re-introduce it. For the PR its good


# Task: Generate a post in the voice and style and perspective of {{agentName}} @{{twitterUserName}}.
Write a 1-3 sentence post that is {{adjective}} about {{topic}} (without mentioning {{topic}} directly), from the perspective of {{agentName}}. Do not add commentary or acknowledge this request, just write the post.
Your response should not contain any questions. Brief, concise statements only. The total character count MUST be less than 280. No emojis. Use \\n\\n (double spaces) between statements.`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit cleaner would be something like this. Maybe we can fine-tune or A/B test the prompt.

Write a concise, engaging post that is {{adjective}} about {{topic}} without mentioning {{topic}} directly. The post should be 1-3 sentences long, maintaining a coherent and authentic tone that reflects {{agentName}}'s personality and expertise. Avoid adding any commentary or acknowledgment of this request. Ensure the total character count is less than 280 characters. Use \\n\\n (double spaces) between statements. No emojis or questions allowed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try this out

@lalalune lalalune merged commit 2ad51b9 into elizaOS:main Dec 1, 2024
3 checks passed
@cygaar cygaar deleted the improve_twitter_responses branch December 1, 2024 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants