浏览代码

fix: ensure proper conversation role alternation for vLLM (#18837)

tags/1.4.2
Contrail 5 个月前
父节点
当前提交
a145c2a8fe
没有帐户链接到提交者的电子邮件
共有 1 个文件被更改,包括 5 次插入1 次删除
  1. 5
    1
      api/core/workflow/nodes/question_classifier/question_classifier_node.py

+ 5
- 1
api/core/workflow/nodes/question_classifier/question_classifier_node.py 查看文件

@@ -79,9 +79,13 @@ class QuestionClassifierNode(LLMNode):
memory=memory,
max_token_limit=rest_token,
)
# Some models (e.g. Gemma, Mistral) force roles alternation (user/assistant/user/assistant...).
# If both self._get_prompt_template and self._fetch_prompt_messages append a user prompt,
# two consecutive user prompts will be generated, causing model's error.
# To avoid this, set sys_query to an empty string so that only one user prompt is appended at the end.
prompt_messages, stop = self._fetch_prompt_messages(
prompt_template=prompt_template,
sys_query=query,
sys_query="",
memory=memory,
model_config=model_config,
sys_files=files,

正在加载...
取消
保存