您最多选择25个主题 主题必须以字母或数字开头,可以包含连字符 (-),并且长度不得超过35个字符

accelerate_question_answering.mdx 2.7KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950
  1. ---
  2. sidebar_position: 2
  3. slug: /accelerate_doc_indexing_and_question_answering
  4. ---
  5. # Accelerate document indexing and question answering
  6. import APITable from '@site/src/components/APITable';
  7. A checklist to speed up document parsing and question answering.
  8. ---
  9. Please note that some of your settings may consume a significant amount of time. If you often find that document parsing and question answering are time-consuming, here is a checklist to consider:
  10. ## 1. Accelerate document indexing
  11. - Use GPU to reduce embedding time.
  12. - On the configuration page of your knowledge base, toggle off **Use RAPTOR to enhance retrieval**.
  13. - The **Knowledge Graph** chunk method (GraphRAG) is time-consuming.
  14. - Disable **Auto-keyword** and **Auto-question** on the configuration page of yor knowledge base, as both depend on the LLM.
  15. ## 2. Accelerate question answering
  16. - In the **Prompt Engine** tab of your **Chat Configuration** dialogue, disabling **Multi-turn optimization** will reduce the time required to get an answer from the LLM.
  17. - In the **Prompt Engine** tab of your **Chat Configuration** dialogue, leaving the **Rerank model** field empty will significantly decrease retrieval time.
  18. - In the **Assistant Setting** tab of your **Chat Configuration** dialogue, disabling **Keyword analysis** will reduce the time to receive an answer from the LLM.
  19. - When chatting with your chat assistant, click the light bulb icon above the *current* dialogue and scroll down the popup window to view the time taken for each task:
  20. ![enlighten](https://github.com/user-attachments/assets/fedfa2ee-21a7-451b-be66-20125619923c)
  21. ```mdx-code-block
  22. <APITable>
  23. ```
  24. | Item name | Description |
  25. | ----------------- | ------------------------------------------------------------ |
  26. | Total | Total time spent on this conversation round, including chunk retrieval and answer generation. |
  27. | Check LLM | Time to validate the specified LLM. |
  28. | Create retriever | Time to create a chunk retriever. |
  29. | Bind embedding | Time to initialize an embedding model instance. |
  30. | Bind LLM | Time to initialize an LLM instance. |
  31. | Tune question | Time to optimize the user query using the context of the mult-turn conversation. |
  32. | Bind reranker | Time to initialize an reranker model instance for chunk retrieval. |
  33. | Generate keywords | Time to extract keywords from the user query. |
  34. | Retrieval | Time to retrieve the chunks. |
  35. | Generate answer | Time to generate the answer. |
  36. ```mdx-code-block
  37. </APITable>
  38. ```