How to use greedy search correctly

[fan-niu]

I used qwen2-1.5b as the basic model to train a machine translation model. I need to use greedy search to generate, but no matter how I set top_p, top_k, and temperature, I cannot get the results of greedy search. Please help me how to set greedy search correctly. Thanks so much !!!

I ran 1000 test sets 2 times with the following configuration, but the results in each round were about 300 different.

1. top_p=0.0、top_k=0.0、temperature=1.0、beam_width=1
2. top_p=0.0、top_k=1.0、temperature=0.0、beam_width=1
3. top_p=1.0、top_k=1.0、temperature=0.0、beam_width=1
4. top_p=0.0、top_k=1.0、temperature=1.0、beam_width=1

Tensorrtllm version: v0.14.0

[akhoroshev]

try 0.15.0, for me it generates more stable results (compare to 0.14.0)

[fan-niu]

try 0.15.0, for me it generates more stable results (compare to 0.14.0)

@akhoroshev ok got it, thanks.
But still have a question, why 0.14.0 has unstable results on greedy search? thanks

[fan-niu]

@akhoroshev already test it, it's fine, thank you so much !

[nv-guomingz]

Feel free to reopen this issue if needed.