Enhance Inline Query Functionality with Action Button and Improved Caching

[bugfloyd]

Background

The current inline queries feature functions more like a decorative element, as it does not operate as a true inline query. The shortcomings include:

• When users select the offered inline result, the final message only includes the user prompt and not the answer.
• The bot needs to be added to the group to enable it to reply with the actual answer in the chat.
• It does not work in private chats due to the previous point.

Existing Challenges and Possible Solutions

If we want to generate a response from OpenAI APIs and provide it within the inline result or the final output, there are some clear challenges:

• Telegram sends inline queries to the bot whenever the user pauses typing. Sending requests to OpenAI for each user input event would waste many tokens and negatively impact bot performance.
• I tried implementing a solution using Python threads, timers, and the event loop to reduce the number of requests and only send them if we think the user has finished typing. However, this solution was hacky and PTB does not properly support Python threads.
• I also attempted to implement a queue system in the bot class. The idea was to add new updates to this queue and process them only if we think the user has finished editing the query (by using a timeout). Although it worked for simple user input behaviors, the user had to wait for a long time to see the inline result, and handling edge cases proved to be tricky. For example, when a user types something, takes a long pause, and then continues to write a long question with short pauses, or removes some content. The problem is that each inline query event has a unique ID, and the queue system may not always use the last event in these edge cases.

The Solution

I retained the initial inline results as previews with just the prompt and no answers included, but added an action button (InlineKeyboardButton) in the final output to generate the actual answer to the prompt. This button triggers a callback, in which we read the prompt, get the answer for it, and edit the original message with the answer. While generating the answer, it also adds a status to the message.

I faced challenges in resolving the following issue:
How can we get the original query (prompt) in the action callback function? I found callback_data to be a suitable place to store the prompt and read it in the callback, but unfortunately, Telegram imposes a 64-byte limit on this property. To overcome this limit, I implemented a built-in caching system for each user's inline queries. I ended up using callback_data for short prompts because it is a more reliable solution, and it partially fixes the known bug below.

Known Issues

When a user types a long inline prompt, the query gets cached and, upon clicking the action button and retrieving the answer, that cache is removed. If the user sends the exact same long prompt, the initial response with the action button is generated by the Telegram/bot cache. The action button does not work since we do not have cache available for the unique ID included in the callback_data (which came from the Telegram cache and includes the original unique ID). This bug does not exist for short prompts, as the code uses callback_data to transport the prompt. Considering this bug as a real edge case, I think we are safe to use the current flow.

Note:

For inline queries, I am using the user ID as the chat ID when getting answers from the OpenAI class. Doing so enables users to have a consistent inline query experience across different chats (e.g., asking something in a group and referring to it while having a private conversation at the same time).

Detailed Changes

• Add 'private' to the list of supported chat types for inline queries
• Update inline_query method to:
  • Include an action button for fetching the answer
  • Utilize callback_data to store short prompts
  • Implement a built-in cache for longer prompts
• Introduce handle_callback_inline_query as a callback query handler to:
  • Retrieve the prompt from callback_data or the built-in cache
  • Obtain the answer from OpenAI
  • Update the current message content with the status and answer
• Implement validate_answering_possibility method to verify user access and budget
• Create process_used_tokens method to apply token usage
• Employ validate_answering_possibility in image, transcribe, prompt, and inline_query handlers
• Extend inline query support to is_allowed and is_within_budget methods
• Convert error_handler, split_into_chunks, is_user_in_group, and is_group_chat to static methods
• Address lint warnings and convert unused method parameters to private

Screenshots

Future Plans and Enhancements

The current caching system for prompts serves as an initial MVP. We may consider refining or expanding it based on our requirements.
At present, only GPT models are supported in inline queries. We can also explore incorporating image generation using DALL-E, following a similar workflow.

Resolves #41

[k3it]

I've tried this and it worked quite well. there are certainly some constraints in telegram's inline mode that you had to work around. There was a prompt that for some reason did not generate an icon until i added some random extra characters to it.. some sort of edge bug possibly.. but overall i like this, hopefully will get merged.

[n3d1117]

Hi @bugfloyd, awesome work and thanks for the writeup! Tested it and I love it

It works well in private chats. The only issue I found is that when used in groups with the bot in it, you will get the answer twice (one inline, and then another one because you sent a query via the bot).

What do you think the best solution would be? Enable this new inline button in private chats only?

[bugfloyd]

Hi @bugfloyd, awesome work and thanks for the writeup! Tested it and I love it

It works well in private chats. The only issue I found is that when used in groups with the bot in it, you will get the answer twice (one inline, and then another one because you sent a query via the bot).

What do you think the best solution would be? Enable this new inline button in private chats only?

I am glad to hear that :)

I believe having this feature enabled for group chats would be very handy specially when the bot is not/could not added to the group.
I am currently on a trip. I will continue working on this PR next week to find a better solution to handle group chats.
That edge case bug can also be annoying (using the same long prompt multiple times). I will try to find a fix for it.

[Pumbaardian]

Hello, bugfloyd!
Can you upgrade this to the newest (0.2.3) version of n3d1117 bot

[k3it]

This was a nicely done pull request but becoming a bit stale. any chance to whip it into shape for merging @bugfloyd ?

[bugfloyd]

My apologies for the delay in responding to this PR. I have had a very busy couple of weeks, but I will do my best to get this PR ready for merge by this weekend.
In the meantime, I want to let you know that I have found a solution to the known bug that was mentioned, by disabling the cache for inline queries.

However, I have some concerns regarding disabling inline queries for group chats, as this feature could be very handy when the bot is not joined to the group. Therefore, I want to explore other possible solutions to prevent duplicate messages while still keeping the inline query feature enabled for group chats.

If either of you have any ideas or suggestions, please feel free to share them with me.

[k3it]

@bugfloyd I agree - definitely keep the inline mode for group chats, since it's not practical to add the bot to every group where it might be useful occasionally. If a duplicate message cannot be easily avoided i think it's a small "side effect" that we can live with for now and fixed at a later time.

Most users would probably realize immediately after receiving a duplicate reply that there is no real reason to use the inline mode if the bot is already a direct member.

[bugfloyd]

This PR is now ready for review and merging. I've addressed the known bug by disabling the bot cache for inline queries, ensuring that the internal inline caching system works correctly for all cases. Consequently, even short prompts no longer use the callback button data, and the internal cache is employed for all inline queries.

Additionally, I've handled disallowed and budget_limit messages for inline queries.

In the near future, I plan to submit several more PRs to address:

• Class cleanup and formatting, which were initially part of this PR but removed due to merge conflicts
• Streaming support for inline queries
• Translations support for inline queries
• Image command support for inline queries

[n3d1117]

Thank you @bugfloyd great work!

[n3d1117]

Done in #230:

• Translations support for inline queries
• Fix duplicated answers when inline query is used within a group with bot in it