Agent got stucked randomly after a few conversation when using in Japanese

[kun432]

Tried quick start with Japanese, agent seemed stuck randomly after a few conversation.

$ python 1_basic_chatbot.py

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ # IDENTITY and PURPOSE                                                                                                 │
│ - This is a conversation with a helpful and friendly AI assistant.                                                     │
│                                                                                                                        │
│ # OUTPUT INSTRUCTIONS                                                                                                  │
│ - Always respond using the proper JSON schema.                                                                         │
│ - Always use the available additional information and context to enhance the response.                                 │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Agent: Hello! How can I assist you today?
You: 日本の総理大臣の名前を教えて。
Agent: 日本の総理大臣の名前は岸田文雄（きしだ ふみお）です。彼は2021年10月4日に就任しました。
You: それはもう古いよ。
Agent:

そうですね、彼はもう長い間その役職に就いています。日本の総理大臣は、時代の変化に応じて新しいリーダーシップが求められること
が多いです。最近の政治情勢について何か特に興味があることはありますか？
You: 違う、あなたの情報が古い

(no response...)

waited for 5 mins without any response, so seems stuck.

also I found weird behaviours like these:

• blank response returned.
• infinite blank response without changing turn of conversation.

In English, seems no problems.

How can I debug or trace it?

[KennyVaneetvelde]

Huh, that is very strange. Do you ever have any issues using Japanese with https://github.com/instructor-ai/instructor ?

[kun432]

Do you ever have any issues using Japanese with https://github.com/instructor-ai/instructor ?

I have tried instructor before and am not familiar with, but have never faced that issue.

Anyway, tested again with enabling logging.DEBUG, as I said before, infinite blank response without changing turn of conversation. happened and seems response from LLM is broken and long.

2024-11-26 08:42:33,463 - instructor - DEBUG - process_response.py:134 - Instructor Raw Response: ChatCompletion(id='chatcmpl-XXXXXXXXXXXXXXXXXXXX', choices=[Choice(finish_reason='length', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_XXXXXXXXXXXXXXXXXXXX, function=Function(arguments='{"chat_message":"\\n\\n\\n\\n\\n\\n\\n\(__snip__)\n\\n\\n\\n\\n\\n\\n', name='BaseAgentOutputSchema'), type='function')]))], created=1732577945, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier=None, system_fingerprint='fp_XXXXXXX', usage=CompletionUsage(completion_tokens=16384, prompt_tokens=1128, total_tokens=17512, completion_tokens_details=None, prompt_tokens_details=None))
2024-11-26 08:42:33,466 - instructor - DEBUG - retry.py:136 - Retrying, attempt: 2

and results in exceeding max_tokens

instructor.exceptions.InstructorRetryException: The output is incomplete due to a max_tokens length limit.
2024-11-26 08:49:15,407 - httpcore.connection - DEBUG - _trace.py:47 - close.started
2024-11-26 08:49:15,407 - httpcore.connection - DEBUG - _trace.py:47 - close.complete

changed with gpt-4o, seems stable more than gpt-4o-mini with just small tests. so,model issue? maybe?

In my experience, have never had any problems with combination of gpt-4o-mini + function calling for output schema + Japanese for production. maybe I should test more with instructor itself.