Jin Hai
							
						 
						
							
								77b1520b66
								
									
										
											 
										
									
								
							 
						 
						
							
									Refactor message output format (#772) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com> 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								ffe5737f7d
								
									
										
											 
										
									
								
							 
						 
						
							
									let index be batchly. (#733) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
let index be batchly.
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								04a9e95161
								
									
										
											 
										
									
								
							 
						 
						
							
									let file in knowledgebases visible in file manager (#714) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Let file in knowledgebases visible in file manager.
#162  
### Type of change
- [x] New Feature (non-breaking change which adds functionality) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								c28f7b5d38
								
									
										
											 
										
									
								
							 
						 
						
							
									make sure  the error will be recorded. (#672) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								a6e4b74d94
								
									
										
											 
										
									
								
							 
						 
						
							
									remove unused dependency (#664) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   Fakai Zhao
							
						 
						
							
								de839fc3f0
								
									
										
											 
										
									
								
							 
						 
						
							
									optimize srv broker and executor logic (#630) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com> 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								5f03a4de11
								
									
										
											 
										
									
								
							 
						 
						
							
									remove redis (#629) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								9d60a84958
								
									
										
											 
										
									
								
							 
						 
						
							
									refactor code (#583) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								944776f207
								
									
										
											 
										
									
								
							 
						 
						
							
									fix bug about fetching file from minio (#574) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								66f8d35632
								
									
										
											 
										
									
								
							 
						 
						
							
									Refactor (#537) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								0dfc8ddc0f
								
									
										
											 
										
									
								
							 
						 
						
							
									enlarge docker memory usage (#501) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] Refactoring 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								b8e58fe27a
								
									
										
											 
										
									
								
							 
						 
						
							
									add redis to accelerate access of minio (#482) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
### Type of change
- [x] New Feature (non-breaking change which adds functionality) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								ed6081845a
								
									
										
											 
										
									
								
							 
						 
						
							
									Fit a lot of encodings for text file. (#458) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
#384 
### Type of change
- [x] Performance Improvement 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								890561703b
								
									
										
											 
										
									
								
							 
						 
						
							
									Add bce-embedding and fastembed (#383) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Issue link:#326
### Type of change
- [x] New Feature (non-breaking change which adds functionality) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								91068edf16
								
									
										
											 
										
									
								
							 
						 
						
							
									Support Xinference (#320) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Issue link:#299
### Type of change
- [x] New Feature (non-breaking change which adds functionality) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								923cbe488e
								
									
										
											 
										
									
								
							 
						 
						
							
									fix #258 task_executor occupy cpu too much (#288) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Issue link:#285
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								3708b97db9
								
									
										
											 
										
									
								
							 
						 
						
							
									Support Ollama (#261) 
							 
							
							 
							
							
							
							
### What problem does this PR solve?
Issue link:#221
### Type of change
- [x] New Feature (non-breaking change which adds functionality) 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								38e5737067
								
									
										
											 
										
									
								
							 
						 
						
							
									add base url for OpenAI (#166) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								fd7fcb5baf
								
									
										
											 
										
									
								
							 
						 
						
							
									apply pep8 formalize (#155) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								979b3a5b4b
								
									
										
											 
										
									
								
							 
						 
						
							
									support snapshot download from local (#153) 
							 
							
							 
							
							
							
							
* support snapshot download from local
* let snapshot download from local 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								71fe314955
								
									
										
											 
										
									
								
							 
						 
						
							
									refine page ranges (#147) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								5875c8ba08
								
									
										
											 
										
									
								
							 
						 
						
							
									Add 'One' chunk method (#137) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								675a9f8d9a
								
									
										
											 
										
									
								
							 
						 
						
							
									add dockerfile for cuda envirement. Refine table search strategy, (#123) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								0feb085c88
								
									
										
											 
										
									
								
							 
						 
						
							
									refine table parser (#120) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								f1f09df901
								
									
										
											 
										
									
								
							 
						 
						
							
									add local llm implementation (#119) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								b89ac3c4be
								
									
										
											 
										
									
								
							 
						 
						
							
									chage tas execution logic (#103) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								602038ac49
								
									
										
											 
										
									
								
							 
						 
						
							
									fix task cancling bug (#98) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								8a57f2afd5
								
									
										
											 
										
									
								
							 
						 
						
							
									change callback strategy, add timezone to docker (#96) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								685b4d8a95
								
									
										
											 
										
									
								
							 
						 
						
							
									fix table desc bugs, add positions to chunks (#91) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								8a726fb04b
								
									
										
											 
										
									
								
							 
						 
						
							
									solve task execution issues (#90) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								0429107e80
								
									
										
											 
										
									
								
							 
						 
						
							
									fix user login issue (#85) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								320cbf545f
								
									
										
											 
										
									
								
							 
						 
						
							
									docker debugging modification (#82) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								7fd1eca582
								
									
										
											 
										
									
								
							 
						 
						
							
									init README of deepdoc, add picture processer. (#71) 
							 
							
							 
							
							
							
							
* init README of deepdoc, add picture processer.
* add resume parsing 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								a8294f2168
								
							 
						 
						
							
									Refine resume parts and fix bugs in retrival using sql (#66) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								5e0a689c43
								
							 
						 
						
							
									refactor retieval_test, add SQl retrieval methods (#61) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								c5ea37cd30
								
							 
						 
						
							
									Add resume parser and fix bugs (#59) 
							 
							
							 
							
							
							
							
* Update .gitignore
* Update .gitignore
* Add resume parser and fix bugs 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								407b2523b6
								
							 
						 
						
							
									remove unused codes, seperate layout detection out as a new api. Add new rag methed 'table' (#55) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								e6acaf6738
								
							 
						 
						
							
									Add Q&A and Book, fix task running bugs (#50) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								6224edcd1b
								
							 
						 
						
							
									Add task moduel, and pipline the task and every parser (#49) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								e32ef75e99
								
							 
						 
						
							
									Test chat API and refine ppt chunker (#42) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								34b2ab3b2f
								
							 
						 
						
							
									Test APIs and fix bugs (#41) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								484e5abc1f
								
							 
						 
						
							
									llm configuation refine and trievalTest API refine (#40) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								9bf75d4511
								
							 
						 
						
							
									add dialog api (#33) 
							 
							
							
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								c372afe40a
								
							 
						 
						
							
									change licence (#28) 
							 
							
							 
							
							
							
							
* add front end code
* change licence 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								3198faf2d2
								
							 
						 
						
							
									add alot of api (#23) 
							 
							
							 
							
							
							
							
* clean rust version project
* clean rust version project
* build python version rag-flow
* add alot of api 
							
						 
						преди 1 година  
					 
				
					
						
							
								   KevinHuSh
							
						 
						
							
								30791976d5
								
							 
						 
						
							
									build python version rag-flow (#21) 
							 
							
							 
							
							
							
							
* clean rust version project
* clean rust version project
* build python version rag-flow 
							
						 
						преди 1 година