浏览代码

Optimize tencent_vector knowledge base deletion error handling with batch processing support (#22726)

Co-authored-by: liuchen15 <liuchen15@gaotu.cn>
Co-authored-by: crazywoola <427733928@qq.com>
tags/1.7.0
issac2e 3 个月前
父节点
当前提交
58d92970a9
没有帐户链接到提交者的电子邮件
共有 1 个文件被更改,包括 13 次插入3 次删除
  1. 13
    3
      api/core/rag/datasource/vdb/tencent/tencent_vector.py

+ 13
- 3
api/core/rag/datasource/vdb/tencent/tencent_vector.py 查看文件

@@ -206,9 +206,19 @@ class TencentVector(BaseVector):
def delete_by_ids(self, ids: list[str]) -> None:
if not ids:
return
self._client.delete(
database_name=self._client_config.database, collection_name=self.collection_name, document_ids=ids
)

total_count = len(ids)
batch_size = self._client_config.max_upsert_batch_size
batch = math.ceil(total_count / batch_size)

for j in range(batch):
start_idx = j * batch_size
end_idx = min(total_count, (j + 1) * batch_size)
batch_ids = ids[start_idx:end_idx]

self._client.delete(
database_name=self._client_config.database, collection_name=self.collection_name, document_ids=batch_ids
)

def delete_by_metadata_field(self, key: str, value: str) -> None:
self._client.delete(

正在加载...
取消
保存