You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

accelerate_question_answering.mdx 2.6KB

1234567891011121314151617181920212223242526272829303132333435363738394041
  1. ---
  2. sidebar_position: 2
  3. slug: /accelerate_question_answering
  4. ---
  5. # Accelerate answering
  6. import APITable from '@site/src/components/APITable';
  7. A checklist to speed up question answering.
  8. ---
  9. Please note that some of your settings may consume a significant amount of time. If you often find that your question answering is time-consuming, here is a checklist to consider:
  10. - In the **Prompt Engine** tab of your **Chat Configuration** dialogue, disabling **Multi-turn optimization** will reduce the time required to get an answer from the LLM.
  11. - In the **Prompt Engine** tab of your **Chat Configuration** dialogue, leaving the **Rerank model** field empty will significantly decrease retrieval time.
  12. - In the **Assistant Setting** tab of your **Chat Configuration** dialogue, disabling **Keyword analysis** will reduce the time to receive an answer from the LLM.
  13. - When chatting with your chat assistant, click the light bulb icon above the *current* dialogue and scroll down the popup window to view the time taken for each task:
  14. ![enlighten](https://github.com/user-attachments/assets/fedfa2ee-21a7-451b-be66-20125619923c)
  15. ```mdx-code-block
  16. <APITable>
  17. ```
  18. | Item name | Description |
  19. | ----------------- | --------------------------------------------------------------------------------------------- |
  20. | Total | Total time spent on this conversation round, including chunk retrieval and answer generation. |
  21. | Check LLM | Time to validate the specified LLM. |
  22. | Create retriever | Time to create a chunk retriever. |
  23. | Bind embedding | Time to initialize an embedding model instance. |
  24. | Bind LLM | Time to initialize an LLM instance. |
  25. | Tune question | Time to optimize the user query using the context of the mult-turn conversation. |
  26. | Bind reranker | Time to initialize an reranker model instance for chunk retrieval. |
  27. | Generate keywords | Time to extract keywords from the user query. |
  28. | Retrieval | Time to retrieve the chunks. |
  29. | Generate answer | Time to generate the answer. |
  30. ```mdx-code-block
  31. </APITable>
  32. ```