Whats the difference between input prefix / suffix (inp_pfx/inp_sfx) and line prefix / suffix (line_pfx/line_sfx)?

[danemadsen]

I'm looking to integrate prompt templates directly into code for my app and I would like to know what each of these variables mean? They both seem to have the same purpose so why have both?

[KerfuffleV2]

Trying to understand the logic in main? Good luck!

inp_pfx and inp_sfx are (as far as I know) only used when instruct mode is turned on.

llama.cpp/examples/main/main.cpp

Lines 321 to 323 in 465219b

	// prefix & suffix for instruct mode
	const auto inp_pfx = ::llama_tokenize(ctx, "\n\n### Instruction:\n\n", add_bos, true);
	const auto inp_sfx = ::llama_tokenize(ctx, "\n\n### Response:\n\n", false, true);

The values are hardcoded to the old Alpaca format.

On the other hand, line_prefix and line_suffix seem to come from the user specified parameters (on the commandline --in-prefix, --in-suffix)

llama.cpp/examples/main/main.cpp

Lines 765 to 768 in 465219b

	const auto line_pfx = ::llama_tokenize(ctx, params.input_prefix, false, true);
	const auto line_inp = ::llama_tokenize(ctx, buffer, false, false);
	const auto line_sfx = ::llama_tokenize(ctx, params.input_suffix, false, true);
	LOG("input tokens: %s\n", LOG_TOKENS_TOSTR_PRETTY(ctx, line_inp).c_str());

If that's not enough information to do what you want then you might also try looking at the log file and even adding a few of your own LOG(...) lines if needed.