Fixed OpenAI compatibility stream [DONE]
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
DOCS: add OpenAI-compatible http and python api reference (#5374)
### What problem does this PR solve?
Add OpenAI-compatible http and python api reference
### Type of change
- [x] Documentation Update
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Fix：validate knowledge base association before document upload (#5373)
### What problem does this PR solve?
fix this bug: https://github.com/infiniflow/ragflow/issues/5368
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: wenju.li <wenju.li@deepctr.cn>
### What problem does this PR solve?
Added OpenAI-like completion api, related to #4672, #4705
This function allows users to interact with a model to get responses
based on a series of messages.
If `stream` is set to True, the response will be streamed in chunks,
mimicking the OpenAI-style API.
#### Example usage:
```bash
curl -X POST https://ragflow_address.com/api/v1/chats_openai/<chat_id>/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $RAGFLOW_API_KEY" \
-d '{
"model": "model",
"messages": [{"role": "user", "content": "Say this is a test!"}],
"stream": true
}'
```
Alternatively, you can use Python's `OpenAI` client:
```python
from openai import OpenAI
model = "model"
client = OpenAI(api_key="ragflow-api-key", base_url=f"http://ragflow_address/api/v1/chats_openai/<chat_id>")
completion = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who you are?"},
{"role": "assistant", "content": "I am an AI assistant named..."},
{"role": "user", "content": "Can you tell me how to install neovim"},
],
stream=True
)
stream = True
if stream:
for chunk in completion:
print(chunk)
else:
print(completion.choices[0].message.content)
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### Related Issues
Related to #4672, #4705
Fix: 'Chunk not found!' error in team-sharing knowledge base. (#5361)
### What problem does this PR solve?
As issue #3268 mentioned, "Chun not found!" exception will occur,
especially during the teamwork of knowledge bases.
### The reason of this bug
"tenants" are the people on current_user's team, including the team
owner itself. The old one only checks the first "tenant", tenants[0],
which will cause error when anyone editing the chunk that is not in
tenants[0]'s knowledge base.
My modification won't introduce new errors while iterate all the tenant
then retrieve knowledge bases of each.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Add configuration to choose default llm models (#5245)
### What problem does this PR solve?
This pull request includes changes to the `api/settings.py` and
`docker/service_conf.yaml.template` files to add support for default
models in the LLM configuration (specially for LIGHTEN builds). The most
important changes include adding default model configurations and
updating the initialization settings to use these defaults.
For example:
With this configuration Bedrock will be enable by default with claude
and titan embeddings.
```
user_default_llm:
factory: 'Bedrock'
api_key: '{}'
base_url: ''
default_models:
chat_model: 'anthropic.claude-3-5-sonnet-20240620-v1:0'
embedding_model: 'amazon.titan-embed-text-v2:0'
rerank_model: ''
asr_model: ''
image2text_model: ''
```
### Type of change
- [X] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Support downloading models from ModelScope Community. (#5073)
This PR supports downloading models from ModelScope. The main
modifications are as follows:
-New Feature (non-breaking change which adds functionality)
-Documentation Update
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Fix: Normalize embedding model ID comparison across datasets (#5169)
Modify embedding model ID comparison to remove vendor suffixes, ensuring
consistent model identification when working with multiple knowledge
bases. This change affects dialog creation, chat operations, and
document retrieval test functions.
### What problem does this PR solve?
resolve this bug: https://github.com/infiniflow/ragflow/issues/5166
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Feat: Add support for document meta fields update through api (#5120)
### What problem does this PR solve?
add support for update document meta data through api
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
API options like `stream` was ignored when no session_id was provided.
This PR fixes the issue.
Test command and expected result:
```
curl --request POST \
--url http://:9222/api/v1/chats/2f2e1d30ee6111efafe211749b004925/completions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ragflow-xxx' \
--data '{
"question":"Who are you",
"stream":false
}'
{"code":0,"data":"data:{\"code\": 0, \"message\": \"\", \"data\": {\"answer\": \"Hi! I'm your assistant, what can I do for you?\", \"reference\": {}, \"audio_binary\": null, \"id\": null, \"session_id\": \"82ceb0fcee7111efafe211749b004925\"}}\n\n"}
```
### Type of change
- [*] Bug Fix (non-breaking change which fixes an issue)
Fix: The max tokens defined by the tenant are not used (#4297) (#2817) (#5066)
### What problem does this PR solve?
Fix: The max tokens defined by the tenant are not used (#4297) (#2817)
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Update agent session API, to support uploading files while create a new session (#5039)
### What problem does this PR solve?
Update the agent session API "POST /api/v1/agents/{agent_id}/sessions",
to support uploading files while create a new session:
- currently, the API only supports requesting with a json body. If user
wants to upload a doc or image when create session, like what is already
supported on the web client, we need to update the API.
- if upload an image, ragflow will call image2text, and a user_id is
needed for the image2text model. So we need to send user_id in the API
request. As form-data is needed to upload files, not json body, seems we
need to put the user_id in the url as an optional parameter (currently
user_id is an optional in json body).
### Type of change
- [x] Documentation Update
- [x] Other (please describe):
Fix knowledge graph node not found (#4968) (#4970)
### What problem does this PR solve?
Fix knowledge graph node not found (#4968)
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Fixes KeyError: 'content' when using stream=False (#4944)
### 🛠 Fixes `KeyError: 'content'` when using `stream=False`
#### 🔍 Problem
When calling the chat API with `stream=False`, the code attempts to
access `msg[-1]["content"]` without verifying if the key exists. This
causes a `KeyError` when the message structure does not contain
`"content"`.
This issue was discussed in
[#4885](https://github.com/infiniflow/ragflow/issues/4885), where we
analyzed the root cause. The error does not occur with `stream=True`, as
the response is processed differently.
#### ✅ Solution
- **Logging Fix:**
- Before accessing `msg[-1]["content"]`, we check if the key exists.
- If it does not exist, a default value (`"[content not available]"`) is
used to prevent errors.
- **Structural Fix in `msg` Construction:**
- Ensured that every message in `msg` contains the `"content"` key, even
if empty.
- This fixes the issue at its root and ensures consistent behavior
between `stream=True` and `stream=False`.
#### 🔄 Impact
- Prevents the `KeyError` without affecting normal application flow.
- Ensures the integrity of the `msg` structure, avoiding future
failures.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
optimize TenantLLMService.increase_usage for "can't update token usag… (#4755)
…e error " message
### What problem does this PR solve?
optimize TenantLLMService.increase_usage Performance
### Type of change
- [x] Performance Improvement
Co-authored-by: che_shuai <che_shuai@massclouds.com>
Kevin Hu
so95
Yongteng Lei
k
liwenju0
Zhenglin Dong
Omar Leonardo Sanchez Granados
yrk111222
petertc
zhxlp
flygithub
hy89
kuschzzp
Peterson Alves
davidche
Zhichang Yu
Jin Hai