Fix: Solve the OOM issue when passing large PDF files while using QA chunking method. (#8464)
### What problem does this PR solve?
Using the QA chunking method with a large PDF (e.g., 300+ pages) may
lead to OOM in the ragflow-worker module.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix some bugs in text2sql.(#4279)(#4281)
### What problem does this PR solve?
- The incorrect results in parsing CSV files of the QA knowledge base in
the text2sql scenario. Process CSV files using the csv library. Decouple
CSV parsing from TXT parsing
- Most llm return results in markdown format ```sql query ```, Fix
execution error caused by LLM output SQLmarkdown format.### Type of
change
- [x] Bug Fix (non-breaking change which fixes an issue)
Update progress info and start welcome info (#3768)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Refactoring
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?
Use consistent log file names, introduced initLogger
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
make titles in markdown not be splited with following content (#2971)
### What problem does this PR solve?
#2970
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add get_txt function to reduce duplicate code
### Type of change
- [x] Refactoring
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Format file format from Windows/dos to Unix (#1949)
### What problem does this PR solve?
Related source file is in Windows/DOS format, they are format to Unix
format.
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Support displaying tables in the chunks of pdf file when using QA parser (#1263)
### What problem does this PR solve?
Support displaying tables in the chunks of pdf file when using QA parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Support displaying images in the chunks of docx files when using general parser (#1253)
### What problem does this PR solve?
Support displaying images in chunks of docx files when using general
parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Place pdf's image at the correct position in QA parser (#1235)
### What problem does this PR solve?
Place pdf's image at the correct position in QA parser
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Add docx support for manual parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add docx support for QA parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add markdown support for QA parser
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Support extracting questions and answers from PDF files
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
fix bug about reload knowledgebase configuration reloading (#210)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/209)]
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
liuzhenghua
Kevin Hu
Jin Hai
TeslaZY
Zhichang Yu
yqkcn
cHz
Zhedong Cen