fix:failed to Create new Chunk in database #1159 (#1214)
### What problem does this PR solve?
fix bug: [1159](https://github.com/infiniflow/ragflow/issues/1159)
using embd which user configured at knowledgebase when create new chunk
in database
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
created get_dataset, update_dataset API and fixed: delete (#1201)
### What problem does this PR solve?
Added get_dataset and update_dataset API.
Fixed delete_dataset.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
added documentation for api and fixed: duplicate get_dataset() (#1190)
### What problem does this PR solve?
Added the documentation for api and fixed duplicate get_dataset()
methods.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
### What problem does this PR solve?
This PR have completed both HTTP API and Python SDK for
'delete_dataset". In addition, there are tests for it.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
This PR have completed both HTTP API and Python SDK for 'list_dataset".
In addition, there are tests for it.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
This PR have finished 'create dataset' of both HTTP API and Python SDK.
HTTP API:
```
curl --request POST --url http://<HOST_ADDRESS>/api/v1/dataset --header 'Content-Type: application/json' --header 'Authorization: <ACCESS_KEY>' --data-binary '{
"name": "<DATASET_NAME>"
}'
```
Python SDK:
```
from ragflow.ragflow import RAGFLow
ragflow = RAGFLow('<ACCESS_KEY>', 'http://127.0.0.1:9380')
ragflow.create_dataset("dataset1")
```
TODO:
- ACCESS_KEY is the login_token when user login RAGFlow, currently.
RAGFlow should have the function that user can add/delete access_key.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Supports obtaining PDF documents from web pages (#1107)
### What problem does this PR solve?
Knowledge base management supports crawling information from web pages
and generating PDF documents
### Type of change
- [x] New Feature (Support document from web pages)
fix: fixed the problem that the api will be called directly after selecting the chat assistant picture (#1034)
### What problem does this PR solve?
fix: fixed the problem that the api will be called directly after
selecting the chat assistant picture #1033
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Add backend API support for moving files into other directory
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?
#724#162
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
feat: add rerank models to the project #724#162
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Add support for VolcEngine - the current version supports SDK2 (#885)
- The main idea is to assemble **ak**, **sk**, and **ep_id** into a
dictionary and store it in the database **api_key** field
- I don’t know much about the front-end, so I learned from Ollama, which
may be redundant.
### Configuration method
- model name
- Format requirements: {"VolcEngine model name":"endpoint_id"}
- For example: {"Skylark-pro-32K":"ep-xxxxxxxxx"}
- Volcano ACCESS_KEY
- Format requirements: VOLC_ACCESSKEY of the volcano engine
corresponding to the model
- Volcano SECRET_KEY
- Format requirements: VOLC_SECRETKEY of the volcano engine
corresponding to the model
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Delete files from a knowledge base.
#717
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
The back-end code adds Feishu oauth
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
### What problem does this PR solve?
api_app.py
/document/upload
add two non mandatory parameters
parser_id:
[naive,qaresume,manual,table,paper,book,laws,presentation,picture,one]
run: 1
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Bug Fix (non-breaking change
- [x] Other (please describe): Fix typo
let file in knowledgebases visible in file manager (#714)
### What problem does this PR solve?
Let file in knowledgebases visible in file manager.
#162
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Refactor: Use TaskStatus enum for task status handling (#646)
### What problem does this PR solve?
This commit changes the status 'not started' from being hard-coded to
being maintained by the TaskStatus enum. This enhancement ensures
consistency across the codebase and improves maintainability.
### Type of change
- [x] Refactoring
Wang Baoling
cecilia-uu
KevinHuSh
Jin Hai
Fakai Zhao
balibabu
Zhedong Cen
GYH
dashi6174
yungongzi
LiYongHui
GYH
Sawyer
Moonlit