if role.upper() == "ASSISTANT": input_messages.append(msg+ OUTPUT_POSTFIX)
should be
if role.upper() == "ASSISTANT": input_messages.append(msg + " " + OUTPUT_POSTFIX)
For example, 3</s> is completely different from 3 </s>. The former string doesn't return the end of sequence token. It's mean the EOS token never appears in training process, so leading to infinitively generation in inference.
if role.upper() == "ASSISTANT": input_messages.append(msg+ OUTPUT_POSTFIX)should be
if role.upper() == "ASSISTANT": input_messages.append(msg + " " + OUTPUT_POSTFIX)For example,
3</s>is completely different from3 </s>. The former string doesn't return the end of sequence token. It's mean the EOS token never appears in training process, so leading to infinitively generation in inference.