Line breaks in free text responses causing answer to be truncated

[rbyh]

Compare answer and generated tokens:
https://www.expectedparrot.com/content/571cc86a-282b-410b-8d21-530b9577a92d

[rbyh]

answer:
White breath in the air, Frozen ground crunches below,

generated_tokens:
White breath in the air, Frozen ground crunches below, Winter's silent sleep.

raw_model_response:
{'candidates': [{'content': {'parts': [{'text': "White breath in the air,\nFrozen ground crunches below,\nWinter's silent sleep. \n"}], 'role': 'model'}, 'finish_reason': 1, 'safety_ratings': [{'category': 8, 'probability': 1, 'blocked': False}, {'category': 10, 'probability': 1, 'blocked': False}, {'category': 7, 'probability': 1, 'blocked': False}, {'category': 9, 'probability': 1, 'blocked': False}], 'avg_logprobs': -0.0750277421691201, 'token_count': 0, 'grounding_attributions': []}], 'usage_metadata': {'prompt_token_count': 9, 'candidates_token_count': 22, 'total_token_count': 31, 'cached_content_token_count': 0}, 'model_version': 'gemini-1.5-flash'}

We can see that the raw response does include \n proper line breaks, so the issue is extracting and formatting the response.

[rbyh]

The issue appears to be that this method is looking for the last \n new line character and treating it as a comment, but free text does not include a comment:

edsl/edsl/language_models/RawResponseHandler.py

Line 73 in 7e64fa3

if last_newline == -1:

[rbyh]

@johnjosephhorton I think it could be safer/cleaner for the instruction to be to use the word 'comment' (or ## would be better?) to mark where the comment starts. This issue seems most likely to occur with free text questions, but you could imagine line breaks being used in question list or dict responses, especially if text is being quoted in the answer. (We have to let LLMs recite poetry!) I don't think we should just "fix" it for free text by having a special prompt to try to handle some unknown number of line breaks.

[rbyh]

Related edge case issue: if the user adds an instruction to not add a comment in the question text instead of modifying answering instructions or using include_comment=False the issue can occur. But I don't think this is going to happen often.

Another option could be to also have a default comment field for free text questions.