### What problem does this PR solve?

Bump infinity to 0.5.0

### Type of change

- [x] Refactoring

### What problem does this PR solve?

Elasticsearch disk-based shard allocator use absolute byte values
instead of ratio. Close #4018

### Type of change

- [ ] Documentation Update
- [x] Refactoring

### What problem does this PR solve?

Migrate infinity at startup

#3809
https://github.com/infiniflow/infinity/issues/2321

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>

### What problem does this PR solve?

Added an infinity configuration file to easily customize the settings of
Infinity

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?

Bump infinity to v0.5.0-dev5

### Type of change

- [x] Refactoring

### What problem does this PR solve?

Added doc for switching elasticsearch to infinity

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring

### What problem does this PR solve?

Replaced redis with Valkey. Close #3070

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### What problem does this PR solve?

Build multi-arch docker image `infiniflow/ragflow:poetry` on
`linux/amd64` and `linux/arm64`.

### Type of change

- [x] Refactoring

### What problem does this PR solve?

Redis post config is missing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>

### What problem does this PR solve?

Using password authentication to access ElasticSearch is essential,
especially in a production environment.

This PR will enable password access support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?

Optimize task broker and executor for reduce memory usage and deployment
complexity.

### Type of change
- [x] Performance Improvement
- [x] Refactoring

### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>

### What problem does this PR solve?

Since Kibana service is not currently being used, the associated volume
'kibanadata' has been commented out in the Docker Compose file. This
change helps to prevent the allocation of unnecessary resources and
simplifies the configuration.

### Type of change

- [x] Refactoring
unused Kibana volume storage

### What problem does this PR solve?

### Type of change

- [x] Refactoring

### What problem does this PR solve?

The docker-compose file can't config minio related port by .env file. So
I just add env `MINIO_CONSOLE_PORT=9001
MINIO_PORT=9000` to .env file.

### Type of change

- [x] Refactoring

### What problem does this PR solve?

![image](https://github.com/infiniflow/ragflow/assets/106524776/56eec972-e92a-4cd2-92cc-4c20ba478a52)

- remove version top-level element in the compose.yml file
when use command `docker compose -f docker-compose-CN.yml up -d` ,it
will waring,[maybe it appear after
v25,5](https://github.com/mailcow/mailcow-dockerized/issues/5797#issuecomment-2011349228)

![image](https://github.com/infiniflow/ragflow/assets/106524776/e42ebfb7-d51e-4423-b6dd-3a7568a6be4f)

ref to
https://docs.docker.com/compose/intro/history/#docker-compose-cli-versioning
> Version one of the Docker Compose command-line binary was first
released in 2014. It was written in Python, and is invoked with
docker-compose. Typically, Compose V1 projects include a top-level
version element in the compose.yml file, with values ranging from 2.0 to
3.8, which refer to the specific [file
formats](https://docs.docker.com/compose/intro/history/#compose-file-format-versioning).

> Version two of the Docker Compose command-line binary was announced in
2020, is written in Go, and is invoked with docker compose. Compose V2
ignores the version top-level element in the compose.yml file.
 
Also in `README.MD` show `Docker >= 24.0.0 & Docker Compose >= v2.26.1`.
so i want to remove the version.

### Type of change

- [x] Other (please describe):
remove warning

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/226)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

* clean rust version project

* clean rust version project

* build python version rag-flow

* add field progress msg into docinfo; add file processing procedure

* go through upload, create kb, add doc to kb

* smoke test for all API

* smoke test for all API

* add field progress msg into docinfo; add file processing procedure

* go through upload, create kb, add doc to kb

* add docker compose

* add docker compose