Kevin Hu
152072f900
Add graphrag (#1793)
### What problem does this PR solve?
#1594
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
H
ac7a0d4fbf
Add ParsertType Audio (#1637)
### What problem does this PR solve?
#1514
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
6f99bbbb08
add raptor (#899)
### What problem does this PR solve?
#882
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
95f809187e
add stream chat (#811)
### What problem does this PR solve?
#709
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
4d47b2b459
fix a string format error (#781)
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
hace 1 año
KevinHuSh
aa1c915d6e
support gpt-4o (#773)
### What problem does this PR solve?
#771
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
Jin Hai
77b1520b66
Refactor message output format (#772)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
hace 1 año
KevinHuSh
ffe5737f7d
let index be batchly. (#733)
### What problem does this PR solve?
let index be batchly.
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
04a9e95161
let file in knowledgebases visible in file manager (#714)
### What problem does this PR solve?
Let file in knowledgebases visible in file manager.
#162
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
c28f7b5d38
make sure the error will be recorded. (#672)
### What problem does this PR solve?
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
a6e4b74d94
remove unused dependency (#664)
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
hace 1 año
Fakai Zhao
de839fc3f0
optimize srv broker and executor logic (#630)
### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
hace 1 año
KevinHuSh
5f03a4de11
remove redis (#629)
### What problem does this PR solve?
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
9d60a84958
refactor code (#583)
### What problem does this PR solve?
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
944776f207
fix bug about fetching file from minio (#574)
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
hace 1 año
KevinHuSh
66f8d35632
Refactor (#537)
### What problem does this PR solve?
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
0dfc8ddc0f
enlarge docker memory usage (#501)
### What problem does this PR solve?
### Type of change
- [x] Refactoring
hace 1 año
KevinHuSh
b8e58fe27a
add redis to accelerate access of minio (#482)
### What problem does this PR solve?
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
ed6081845a
Fit a lot of encodings for text file. (#458)
### What problem does this PR solve?
#384
### Type of change
- [x] Performance Improvement
hace 1 año
KevinHuSh
890561703b
Add bce-embedding and fastembed (#383)
### What problem does this PR solve?
Issue link:#326
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
91068edf16
Support Xinference (#320)
### What problem does this PR solve?
Issue link:#299
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
923cbe488e
fix #258 task_executor occupy cpu too much (#288)
### What problem does this PR solve?
Issue link:#285
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
hace 1 año
KevinHuSh
3708b97db9
Support Ollama (#261)
### What problem does this PR solve?
Issue link:#221
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
hace 1 año
KevinHuSh
38e5737067
add base url for OpenAI (#166)
hace 1 año
KevinHuSh
fd7fcb5baf
apply pep8 formalize (#155)
hace 1 año
KevinHuSh
979b3a5b4b
support snapshot download from local (#153)
* support snapshot download from local
* let snapshot download from local
hace 1 año
KevinHuSh
71fe314955
refine page ranges (#147)
hace 1 año
KevinHuSh
5875c8ba08
Add 'One' chunk method (#137)
hace 1 año
KevinHuSh
675a9f8d9a
add dockerfile for cuda envirement. Refine table search strategy, (#123)
hace 1 año
KevinHuSh
0feb085c88
refine table parser (#120)
hace 1 año
KevinHuSh
f1f09df901
add local llm implementation (#119)
hace 1 año
KevinHuSh
b89ac3c4be
chage tas execution logic (#103)
hace 1 año
KevinHuSh
602038ac49
fix task cancling bug (#98)
hace 1 año
KevinHuSh
8a57f2afd5
change callback strategy, add timezone to docker (#96)
hace 1 año
KevinHuSh
685b4d8a95
fix table desc bugs, add positions to chunks (#91)
hace 1 año
KevinHuSh
8a726fb04b
solve task execution issues (#90)
hace 1 año
KevinHuSh
0429107e80
fix user login issue (#85)
hace 1 año
KevinHuSh
320cbf545f
docker debugging modification (#82)
hace 1 año
KevinHuSh
7fd1eca582
init README of deepdoc, add picture processer. (#71)
* init README of deepdoc, add picture processer.
* add resume parsing
hace 1 año
KevinHuSh
a8294f2168
Refine resume parts and fix bugs in retrival using sql (#66)
hace 1 año
KevinHuSh
5e0a689c43
refactor retieval_test, add SQl retrieval methods (#61)
hace 1 año
KevinHuSh
c5ea37cd30
Add resume parser and fix bugs (#59)
* Update .gitignore
* Update .gitignore
* Add resume parser and fix bugs
hace 1 año
KevinHuSh
407b2523b6
remove unused codes, seperate layout detection out as a new api. Add new rag methed 'table' (#55)
hace 1 año
KevinHuSh
e6acaf6738
Add Q&A and Book, fix task running bugs (#50)
hace 1 año
KevinHuSh
6224edcd1b
Add task moduel, and pipline the task and every parser (#49)
hace 1 año
KevinHuSh
e32ef75e99
Test chat API and refine ppt chunker (#42)
hace 1 año
KevinHuSh
34b2ab3b2f
Test APIs and fix bugs (#41)
hace 1 año
KevinHuSh
484e5abc1f
llm configuation refine and trievalTest API refine (#40)
hace 1 año
KevinHuSh
9bf75d4511
add dialog api (#33)
hace 1 año
KevinHuSh
c372afe40a
change licence (#28)
* add front end code
* change licence
hace 1 año