You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
curl --request POST --url http://localhost:8080/completion --header "Content-Type: application/json" --data '{"prompt": "
Write a c++ program printing G'day.","n_predict": 512}'
output:
{"content":"\nHere is a simple C++ program that prints \"Hello, World!\": \n\n```c++ \n#include <iostream> \n#include <iostream> Hello, this code snippet.\n#include hello world program to print \"Hello, using c++ Hello!","generation_settings":{"dynatemp_exponent":1.0,"dynatemp_range":0.0,"frequency_penalty":0.0,"grammar":"","ignore_eos":false,"logit_bias":[],"min_keep":0,"min_p":0.05000000074505806,"mirostat":0,"mirostat_eta":0.10000000149011612,"mirostat_tau":5.0,"model":"../ollama_wks/Orion-14B-Chat-Q2_K.gguf","n_ctx":2048,"n_keep":0,"n_predict":-1,"n_probs":0,"penalize_nl":true,"penalty_prompt_tokens":[],"presence_penalty":0.0,"repeat_last_n":64,"repeat_penalty":1.100000023841858,"samplers":["top_k","tfs_z","typical_p","top_p","min_p","temperature"],"seed":4294967295,"stop":[],"stream":false,"temperature":0.800000011920929,"tfs_z":1.0,"top_k":40,"top_p":0.949999988079071,"typical_p":1.0,"use_penalty_prompt_tokens":false},"id_slot":0,"model":"../ollama_wks/Orion-14B-Chat-Q2_K.gguf","prompt":"Write a c++ program printing hello world.","stop":true,"stopped_eos":true,"stopped_limit":false,"stopped_word":false,"stopping_word":"","timings":{"predicted_ms":25227.774,"predicted_n":69,"predicted_per_second":2.7350807883406594,"predicted_per_token_ms":365.6199130434783,"prompt_ms":1169.981,"prompt_n":9,"prompt_per_second":7.692432612153531,"prompt_per_token_ms":129.9978888888889},"tokens_cached":77,"tokens_evaluated":9,"tokens_predicted":69,"truncated":false}
The chat template for Orion's models is missing, and applying chatml format will give wrong response.
output:
gg model and chat template can be found here.
The text was updated successfully, but these errors were encountered: