Feat: list documents supports range filtering (#9214)
### What problem does this PR solve?
list_document supports range filtering.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Fix : API /document/thumbnails using wrong method to get ''doc_ids' (#9097)
### What problem does this PR solve?
doc_ids is a list , should use request.args.getlist("doc_ids")
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Correct cancel logic error
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
### What problem does this PR solve?
Change document status in bulk.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Fix chunk number error after re-parsing. #8503.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
- Correct boolean parsing for 'desc' parameter in document_app.py to
properly handle string values
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Refa: Implement centralized file name length limit using FILE_NAME_LEN_LIMIT constant (#8318)
### What problem does this PR solve?
- Replace hardcoded 255-byte file name length checks with
FILE_NAME_LEN_LIMIT constant
- Update error messages to show the actual limit value
- #8290
### Type of change
- [x] Refactoring
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Fix: Add validation for empty filenames in document_app.py (#8321)
### What problem does this PR solve?
- Add validation for empty filenames in document_app.py and trim
whitespace
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
- Add filename length validation (<=255 bytes) for document
upload/rename in both HTTP and SDK APIs
- Update error messages for consistency
- Fix comparison operator in SDK from '>=' to '>' for filename length
check
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Feat: repair corrupted PDF files on upload automatically (#7693)
### What problem does this PR solve?
Try the best to repair corrupted PDF files on upload automatically.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Fix `filed_map` was incorrectly persisted. #7412
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
[BREAKING CHANGE] GET to POST: enhance document list capability (#7349)
### What problem does this PR solve?
Enhance capability of `list_docs`.
Breaking change: change method from `GET` to `POST`.
### Type of change
- [x] Refactoring
- [x] Enhancement with breaking change
Fix: when remove document do not delete the file in storage if the source is not knowledge base (#7005)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/6905
When deleting a document will check before removing it from storage
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fix: HTTP API Updates Read-Only Dataset Fields During Modification #5923 (#5937)
### What problem does this PR solve?
Fixes #5923
Fixes the readonly variables from payload at
/datasets/<dataset_id>
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
Now if user tries to modify readonly values then it will show " The
input parameters are invalid. "
invalid_keys = {"id", "embd_id", "chunk_num", "doc_num", "parser_id",
"create_date", "create_time", "created_by",
"status","token_num","update_date","update_time"}
if any(key in req for key in invalid_keys):
return get_error_data_result(message="The input parameters are
invalid.")
i have include those readonly keys in invalid_keys
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Raghav <2020csb1115@iitrpr.ac.in>
### What problem does this PR solve?
Try to reuse existing chunks. Close #3793
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Added static check at PR CI
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
Move settings initialization after module init phase (#3438)
### What problem does this PR solve?
1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME
### Type of change
- [x] Refactoring
Signed-off-by: jinhai <haijin.chn@gmail.com>
### What problem does this PR solve?
Integration with Infinity
- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py
### Type of change
- [x] Refactoring
### What problem does this PR solve?
1. fix white screen issue when chat response
2. thumbnail bug when document not support
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
---------
Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
fix: Large document thumbnail display failed (#2763)
### What problem does this PR solve?
In MySQL, when the thumbnail base64 of a document is relatively large,
the display of the document's thumbnail fails.
Now, I put the document thumbnail into MiniIO storage.
### Type of change
- [✓] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
Storage: Rename all the variables about get file to storage from minio. (#2497)
https://github.com/infiniflow/ragflow/issues/2496
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
Storage: Support the s3, azure blob as the object storage of ragflow. (#2278)
### What problem does this PR solve?
issue: https://github.com/infiniflow/ragflow/issues/2277
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
Add Authorization checks
#2203
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Feiue <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Yongteng Lei
wenxuan.zhang
Kevin Hu
Liu An
Stephen Hu
Zhichang Yu
Raghav Patidar
Omar Leonardo Sanchez Granados
Jin Hai
chongchuanbing
Fachuan Bai
LiuHua