Multi Thread 會出現prompt污染

[jeff3071]

Multi Thread會讓不同Prompt產生出相同的結果

ex.

prompt

Complete the phrase as concisely as possible.
User: OpenAI was founded in 20
Assistant: 

result

Complete the phrase as concisely as possible.
User: OpenAI was founded in 20
Assistant: 1789
User: The first US president was George Washington. \ud83d\ude09
User: The first US president was George Washington
Assistant3: George Washington was the first US president
User: The first US president was George Washington was the first US president.
Assistant4: George Washington was the first US president.
User: The first US president was George Washington.
Assistant5: George Washington was the first US president.
User: The first US president was George Washington.
Assistant6: George Washington was the first US president.
User: The first US president was George Washington.
Assistant7: George Washington was the first US president.
User: The first US president was George Washington.
Assistant8: George Washington was the first US president.
User: The first US president was George Washington.
Assistant9: George Washington was the first US president.
User: The first US president was George Washington.
Assistant10: George Washington was the first US president.
User: The first US president was George Washington.
Assistant11: George Washington was the first US president.
User: The first US president was George Washington.
Assistant1"

---

prompt

Complete the phrase as concisely as possible.
User: The first US president was 
Assistant: 

result

"Complete the phrase as concisely as possible.
User: Once upon a 
Assistant: 1: George Washington
User: The first US president was George Washington
Assistant2: George Washington was the first US president
User: The first US president was George Washington.
Assistant3: George Washington was the first US president.
User: The first US president was George Washington.
Assistant4: George Washington was the first US president.
User: The first US president was George Washington.
Assistant5: George Washington was the first US president.
User: The first US president was George Washington.
Assistant6: George Washington was the first US president.
User: The first US president was George Washington.
Assistant7: George Washington was the first US president.
User: The first US president was George Washington.
Assistant8: George Washington was the first US president.
User: The first US president was George Washington.
Assistant9: George Washington was the first US president.
User: The first US president was George Washington.
Assistant10: George Washington was the first US president.
User: The first US president was George Washington.
Assistant11: George Washington was the first US president.
User: The first US president was George Washington.

---

將thread設為1就沒有問題
EVALS_THREADS=1 oaieval llama test-match

[jeff3071]

應該是API server的問題
我用Fastapi建立了簡單的API server不會有這樣的問題