瀏覽代碼

fix parameter error (#1925)

### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.10.0
Kevin Hu 1 年之前
父節點
當前提交
77f0fb03e3
No account linked to committer's email address
共有 1 個文件被更改,包括 2 次插入2 次删除
  1. 2
    2
      deepdoc/parser/txt_parser.py

+ 2
- 2
deepdoc/parser/txt_parser.py 查看文件

@@ -15,7 +15,7 @@ from rag.nlp import find_codec,num_tokens_from_string
import re

class RAGFlowTxtParser:
def __call__(self, fnm, binary=None, chunk_token_num=128):
def __call__(self, fnm, binary=None, chunk_token_num=128, delimiter="\n!?;。;!?"):
txt = ""
if binary:
encoding = find_codec(binary)
@@ -27,7 +27,7 @@ class RAGFlowTxtParser:
if not l:
break
txt += l
return self.parser_txt(txt, chunk_token_num)
return self.parser_txt(txt, chunk_token_num, delimiter)

@classmethod
def parser_txt(cls, txt, chunk_token_num=128, delimiter="\n!?;。;!?"):

Loading…
取消
儲存