浏览代码

refine rerank (#1056)

### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.8.0
KevinHuSh 1年前
父节点
当前提交
b8eedbdd86
没有帐户链接到提交者的电子邮件
共有 1 个文件被更改,包括 5 次插入3 次删除
  1. 5
    3
      rag/llm/rerank_model.py

+ 5
- 3
rag/llm/rerank_model.py 查看文件

token_count = 0 token_count = 0
for _, t in pairs: for _, t in pairs:
token_count += num_tokens_from_string(t) token_count += num_tokens_from_string(t)
batch_size = 32
batch_size = 4096
res = [] res = []
for i in range(0, len(pairs), batch_size): for i in range(0, len(pairs), batch_size):
scores = self._model.compute_score(pairs[i:i + batch_size], max_length=2048) scores = self._model.compute_score(pairs[i:i + batch_size], max_length=2048)
scores = sigmoid(np.array(scores)).tolist()
res.extend(scores)
if isinstance(scores, float): res.append(scores)
else: res.extend(scores)
return np.array(res), token_count return np.array(res), token_count




for i in range(0, len(pairs), batch_size): for i in range(0, len(pairs), batch_size):
scores = self._model.compute_score(pairs[i:i + batch_size], max_length=self._model.max_length) scores = self._model.compute_score(pairs[i:i + batch_size], max_length=self._model.max_length)
scores = sigmoid(np.array(scores)).tolist() scores = sigmoid(np.array(scores)).tolist()
if isinstance(scores, float): res.append(scores)
res.extend(scores) res.extend(scores)
return np.array(res), token_count return np.array(res), token_count




正在加载...
取消
保存