瀏覽代碼

Fix: encode detect error. (#6006)

### What problem does this PR solve?

#5967

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
tags/v0.17.2
Kevin Hu 7 月之前
父節點
當前提交
e05cdc2f9c
沒有連結到貢獻者的電子郵件帳戶。
共有 1 個檔案被更改,包括 2 行新增1 行删除
  1. 2
    1
      rag/nlp/__init__.py

+ 2
- 1
rag/nlp/__init__.py 查看文件

@@ -53,7 +53,8 @@ all_codecs = [
def find_codec(blob):
detected = chardet.detect(blob[:1024])
if detected['confidence'] > 0.5:
return detected['encoding']
if detected['encoding'] == "ascii":
return "utf-8"

for c in all_codecs:
try:

Loading…
取消
儲存