浏览代码

fix parameter error (#1925)

### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.10.0
Kevin Hu 1年前
父节点
当前提交
77f0fb03e3
没有帐户链接到提交者的电子邮件
共有 1 个文件被更改,包括 2 次插入2 次删除
  1. 2
    2
      deepdoc/parser/txt_parser.py

+ 2
- 2
deepdoc/parser/txt_parser.py 查看文件

@@ -15,7 +15,7 @@ from rag.nlp import find_codec,num_tokens_from_string
import re

class RAGFlowTxtParser:
def __call__(self, fnm, binary=None, chunk_token_num=128):
def __call__(self, fnm, binary=None, chunk_token_num=128, delimiter="\n!?;。;!?"):
txt = ""
if binary:
encoding = find_codec(binary)
@@ -27,7 +27,7 @@ class RAGFlowTxtParser:
if not l:
break
txt += l
return self.parser_txt(txt, chunk_token_num)
return self.parser_txt(txt, chunk_token_num, delimiter)

@classmethod
def parser_txt(cls, txt, chunk_token_num=128, delimiter="\n!?;。;!?"):

正在加载...
取消
保存