瀏覽代碼

Fix: change ollama default num_ctx. (#6395)

### What problem does this PR solve?

#6163

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.18.0
Kevin Hu 7 月之前
父節點
當前提交
a2a4bfe3e3
No account linked to committer's email address
共有 2 個文件被更改,包括 20 次插入15 次删除
  1. 17
    14
      agent/component/categorize.py
  2. 3
    1
      rag/llm/chat_model.py

+ 17
- 14
agent/component/categorize.py 查看文件

for c, desc in self.category_description.items(): for c, desc in self.category_description.items():
if desc.get("description"): if desc.get("description"):
descriptions.append( descriptions.append(
"--------------------\nCategory: {}\nDescription: {}\n".format(c, desc["description"]))
"\nCategory: {}\nDescription: {}".format(c, desc["description"]))


self.prompt = """ self.prompt = """
You're a text classifier. You need to categorize the user’s questions into {} categories,
namely: {}
Here's description of each category:
{}

You could learn from the following examples:
{}
You could learn from the above examples.
Just mention the category names, no need for any additional words.
---- Real Data ----
{}
Role: You're a text classifier.
Task: You need to categorize the user’s questions into {} categories, namely: {}

Here's description of each category:
{}

You could learn from the following examples:
{}
You could learn from the above examples.

Requirements:
- Just mention the category names, no need for any additional words.

---- Real Data ----
USER: {}\n
""".format( """.format(
len(self.category_description.keys()), len(self.category_description.keys()),
"/".join(list(self.category_description.keys())), "/".join(list(self.category_description.keys())),
"\n".join(descriptions), "\n".join(descriptions),
"- ".join(cate_lines),
"\n\n- ".join(cate_lines),
chat_hist chat_hist
) )
return self.prompt return self.prompt

+ 3
- 1
rag/llm/chat_model.py 查看文件

if "max_tokens" in gen_conf: if "max_tokens" in gen_conf:
del gen_conf["max_tokens"] del gen_conf["max_tokens"]
try: try:
options = {}
options = {
"num_ctx": 32768
}
if "temperature" in gen_conf: if "temperature" in gen_conf:
options["temperature"] = gen_conf["temperature"] options["temperature"] = gen_conf["temperature"]
if "max_tokens" in gen_conf: if "max_tokens" in gen_conf:

Loading…
取消
儲存