浏览代码

set ollama keep_alive (#985)

### What problem does this PR solve?

#980 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.7.0
KevinHuSh 1年前
父节点
当前提交
f077b57f8b
没有帐户链接到提交者的电子邮件
共有 2 个文件被更改,包括 6 次插入4 次删除
  1. 2
    2
      api/db/init_data.py
  2. 4
    2
      rag/llm/chat_model.py

+ 2
- 2
api/db/init_data.py 查看文件

{ {
"fid": factory_infos[12]["name"], "fid": factory_infos[12]["name"],
"llm_name": "BAAI/bge-reranker-v2-m3", "llm_name": "BAAI/bge-reranker-v2-m3",
"tags": "LLM,CHAT,",
"max_tokens": 16385,
"tags": "RE-RANK,2k",
"max_tokens": 2048,
"model_type": LLMType.RERANK.value "model_type": LLMType.RERANK.value
}, },
] ]

+ 4
- 2
rag/llm/chat_model.py 查看文件

response = self.client.chat( response = self.client.chat(
model=self.model_name, model=self.model_name,
messages=history, messages=history,
options=options
options=options,
keep_alive=-1
) )
ans = response["message"]["content"].strip() ans = response["message"]["content"].strip()
return ans, response["eval_count"] + response.get("prompt_eval_count", 0) return ans, response["eval_count"] + response.get("prompt_eval_count", 0)
model=self.model_name, model=self.model_name,
messages=history, messages=history,
stream=True, stream=True,
options=options
options=options,
keep_alive=-1
) )
for resp in response: for resp in response:
if resp["done"]: if resp["done"]:

正在加载...
取消
保存