Nevar pievienot vairāk kā 25 tēmas Tēmai ir jāsākas ar burtu vai ciparu, tā var saturēt domu zīmes ('-') un var būt līdz 35 simboliem gara.

python_api_reference.md 38KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375
  1. # DRAFT Python API Reference
  2. **THE API REFERENCES BELOW ARE STILL UNDER DEVELOPMENT.**
  3. ---
  4. :::tip NOTE
  5. Dataset Management
  6. :::
  7. ---
  8. ## Create dataset
  9. ```python
  10. RAGFlow.create_dataset(
  11. name: str,
  12. avatar: str = "",
  13. description: str = "",
  14. embedding_model: str = "BAAI/bge-zh-v1.5",
  15. language: str = "English",
  16. permission: str = "me",
  17. chunk_method: str = "naive",
  18. parser_config: DataSet.ParserConfig = None
  19. ) -> DataSet
  20. ```
  21. Creates a dataset.
  22. ### Parameters
  23. #### name: `str`, *Required*
  24. The unique name of the dataset to create. It must adhere to the following requirements:
  25. - Permitted characters include:
  26. - English letters (a-z, A-Z)
  27. - Digits (0-9)
  28. - "_" (underscore)
  29. - Must begin with an English letter or underscore.
  30. - Maximum 65,535 characters.
  31. - Case-insensitive.
  32. #### avatar: `str`
  33. Base64 encoding of the avatar. Defaults to `""`
  34. #### description: `str`
  35. A brief description of the dataset to create. Defaults to `""`.
  36. #### language: `str`
  37. The language setting of the dataset to create. Available options:
  38. - `"English"` (Default)
  39. - `"Chinese"`
  40. #### permission
  41. Specifies who can access the dataset to create. You can set it only to `"me"` for now.
  42. #### chunk_method, `str`
  43. The chunking method of the dataset to create. Available options:
  44. - `"naive"`: General (default)
  45. - `"manual`: Manual
  46. - `"qa"`: Q&A
  47. - `"table"`: Table
  48. - `"paper"`: Paper
  49. - `"book"`: Book
  50. - `"laws"`: Laws
  51. - `"presentation"`: Presentation
  52. - `"picture"`: Picture
  53. - `"one"`: One
  54. - `"knowledge_graph"`: Knowledge Graph
  55. #### parser_config
  56. The parser configuration of the dataset. A `ParserConfig` object's attributes vary based on the selected `"chunk_method"`:
  57. - `"chunk_method"`=`"naive"`:
  58. `{"chunk_token_num":128,"delimiter":"\\n!?;。;!?","html4excel":False,"layout_recognize":True,"raptor":{"user_raptor":False}}`.
  59. - `chunk_method`=`"qa"`:
  60. `{"raptor": {"user_raptor": False}}`
  61. - `chunk_method`=`"manuel"`:
  62. `{"raptor": {"user_raptor": False}}`
  63. - `chunk_method`=`"table"`:
  64. `None`
  65. - `chunk_method`=`"paper"`:
  66. `{"raptor": {"user_raptor": False}}`
  67. - `chunk_method`=`"book"`:
  68. `{"raptor": {"user_raptor": False}}`
  69. - `chunk_method`=`"laws"`:
  70. `{"raptor": {"user_raptor": False}}`
  71. - `chunk_method`=`"presentation"`:
  72. `{"raptor": {"user_raptor": False}}`
  73. - `chunk_method`=`"one"`:
  74. `None`
  75. - `chunk_method`=`"knowledge-graph"`:
  76. `{"chunk_token_num":128,"delimiter":"\\n!?;。;!?","entity_types":["organization","person","location","event","time"]}`
  77. ### Returns
  78. - Success: A `dataset` object.
  79. - Failure: `Exception`
  80. ### Examples
  81. ```python
  82. from ragflow import RAGFlow
  83. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  84. dataset = rag_object.create_dataset(name="kb_1")
  85. ```
  86. ---
  87. ## Delete datasets
  88. ```python
  89. RAGFlow.delete_datasets(ids: list[str] = None)
  90. ```
  91. Deletes datasets by ID.
  92. ### Parameters
  93. #### ids: `list[str]`, *Required*
  94. The IDs of the datasets to delete. Defaults to `None`. If it is not specified, all datasets will be deleted.
  95. ### Returns
  96. - Success: No value is returned.
  97. - Failure: `Exception`
  98. ### Examples
  99. ```python
  100. rag_object.delete_datasets(ids=["id_1","id_2"])
  101. ```
  102. ---
  103. ## List datasets
  104. ```python
  105. RAGFlow.list_datasets(
  106. page: int = 1,
  107. page_size: int = 1024,
  108. orderby: str = "create_time",
  109. desc: bool = True,
  110. id: str = None,
  111. name: str = None
  112. ) -> list[DataSet]
  113. ```
  114. Lists datasets.
  115. ### Parameters
  116. #### page: `int`
  117. Specifies the page on which the datasets will be displayed. Defaults to `1`.
  118. #### page_size: `int`
  119. The number of datasets on each page. Defaults to `1024`.
  120. #### orderby: `str`
  121. The field by which datasets should be sorted. Available options:
  122. - `"create_time"` (default)
  123. - `"update_time"`
  124. #### desc: `bool`
  125. Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.
  126. #### id: `str`
  127. The ID of the dataset to retrieve. Defaults to `None`.
  128. #### name: `str`
  129. The name of the dataset to retrieve. Defaults to `None`.
  130. ### Returns
  131. - Success: A list of `DataSet` objects.
  132. - Failure: `Exception`.
  133. ### Examples
  134. #### List all datasets
  135. ```python
  136. for dataset in rag_object.list_datasets():
  137. print(dataset)
  138. ```
  139. #### Retrieve a dataset by ID
  140. ```python
  141. dataset = rag_object.list_datasets(id = "id_1")
  142. print(dataset[0])
  143. ```
  144. ---
  145. ## Update dataset
  146. ```python
  147. DataSet.update(update_message: dict)
  148. ```
  149. Updates configurations for the current dataset.
  150. ### Parameters
  151. #### update_message: `dict[str, str|int]`, *Required*
  152. A dictionary representing the attributes to update, with the following keys:
  153. - `"name"`: `str` The revised name of the dataset.
  154. - `"embedding_model"`: `str` The updated embedding model name.
  155. - Ensure that `"chunk_count"` is `0` before updating `"embedding_model"`.
  156. - `"chunk_method"`: `str` The chunking method for the dataset. Available options:
  157. - `"naive"`: General
  158. - `"manual`: Manual
  159. - `"qa"`: Q&A
  160. - `"table"`: Table
  161. - `"paper"`: Paper
  162. - `"book"`: Book
  163. - `"laws"`: Laws
  164. - `"presentation"`: Presentation
  165. - `"picture"`: Picture
  166. - `"one"`: One
  167. - `"knowledge_graph"`: Knowledge Graph
  168. ### Returns
  169. - Success: No value is returned.
  170. - Failure: `Exception`
  171. ### Examples
  172. ```python
  173. from ragflow import RAGFlow
  174. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  175. dataset = rag_object.list_datasets(name="kb_name")
  176. dataset.update({"embedding_model":"BAAI/bge-zh-v1.5", "chunk_method":"manual"})
  177. ```
  178. ---
  179. :::tip API GROUPING
  180. File Management within Dataset
  181. :::
  182. ---
  183. ## Upload documents
  184. ```python
  185. DataSet.upload_documents(document_list: list[dict])
  186. ```
  187. Uploads documents to the current dataset.
  188. ### Parameters
  189. #### document_list: `list[dict]`, *Required*
  190. A list of dictionaries representing the documents to upload, each containing the following keys:
  191. - `"display_name"`: (Optional) The file name to display in the dataset.
  192. - `"blob"`: (Optional) The binary content of the file to upload.
  193. ### Returns
  194. - Success: No value is returned.
  195. - Failure: `Exception`
  196. ### Examples
  197. ```python
  198. dataset = rag_object.create_dataset(name="kb_name")
  199. dataset.upload_documents([{"display_name": "1.txt", "blob": "<BINARY_CONTENT_OF_THE_DOC>"}, {"display_name": "2.pdf", "blob": "<BINARY_CONTENT_OF_THE_DOC>"}])
  200. ```
  201. ---
  202. ## Update document
  203. ```python
  204. Document.update(update_message:dict)
  205. ```
  206. Updates configurations for the current document.
  207. ### Parameters
  208. #### update_message: `dict[str, str|dict[]]`, *Required*
  209. A dictionary representing the attributes to update, with the following keys:
  210. - `"display_name"`: `str` The name of the document to update.
  211. - `"chunk_method"`: `str` The parsing method to apply to the document.
  212. - `"naive"`: General
  213. - `"manual`: Manual
  214. - `"qa"`: Q&A
  215. - `"table"`: Table
  216. - `"paper"`: Paper
  217. - `"book"`: Book
  218. - `"laws"`: Laws
  219. - `"presentation"`: Presentation
  220. - `"picture"`: Picture
  221. - `"one"`: One
  222. - `"knowledge_graph"`: Knowledge Graph
  223. - `"parser_config"`: `dict[str, Any]` The parsing configuration for the document. Its attributes vary based on the selected `"chunk_method"`:
  224. - `"chunk_method"`=`"naive"`:
  225. `{"chunk_token_num":128,"delimiter":"\\n!?;。;!?","html4excel":False,"layout_recognize":True,"raptor":{"user_raptor":False}}`.
  226. - `chunk_method`=`"qa"`:
  227. `{"raptor": {"user_raptor": False}}`
  228. - `chunk_method`=`"manuel"`:
  229. `{"raptor": {"user_raptor": False}}`
  230. - `chunk_method`=`"table"`:
  231. `None`
  232. - `chunk_method`=`"paper"`:
  233. `{"raptor": {"user_raptor": False}}`
  234. - `chunk_method`=`"book"`:
  235. `{"raptor": {"user_raptor": False}}`
  236. - `chunk_method`=`"laws"`:
  237. `{"raptor": {"user_raptor": False}}`
  238. - `chunk_method`=`"presentation"`:
  239. `{"raptor": {"user_raptor": False}}`
  240. - `chunk_method`=`"one"`:
  241. `None`
  242. - `chunk_method`=`"knowledge-graph"`:
  243. `{"chunk_token_num":128,"delimiter":"\\n!?;。;!?","entity_types":["organization","person","location","event","time"]}`
  244. ### Returns
  245. - Success: No value is returned.
  246. - Failure: `Exception`
  247. ### Examples
  248. ```python
  249. from ragflow import RAGFlow
  250. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  251. dataset = rag_object.list_datasets(id='id')
  252. dataset = dataset[0]
  253. doc = dataset.list_documents(id="wdfxb5t547d")
  254. doc = doc[0]
  255. doc.update([{"parser_config": {"chunk_token_count": 256}}, {"chunk_method": "manual"}])
  256. ```
  257. ---
  258. ## Download document
  259. ```python
  260. Document.download() -> bytes
  261. ```
  262. Downloads the current document.
  263. ### Returns
  264. The downloaded document in bytes.
  265. ### Examples
  266. ```python
  267. from ragflow import RAGFlow
  268. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  269. dataset = rag_object.list_datasets(id="id")
  270. dataset = dataset[0]
  271. doc = dataset.list_documents(id="wdfxb5t547d")
  272. doc = doc[0]
  273. open("~/ragflow.txt", "wb+").write(doc.download())
  274. print(doc)
  275. ```
  276. ---
  277. ## List documents
  278. ```python
  279. Dataset.list_documents(id:str =None, keywords: str=None, offset: int=1, limit:int = 1024,order_by:str = "create_time", desc: bool = True) -> list[Document]
  280. ```
  281. Lists documents in the current dataset.
  282. ### Parameters
  283. #### id: `str`
  284. The ID of the document to retrieve. Defaults to `None`.
  285. #### keywords: `str`
  286. The keywords used to match document titles. Defaults to `None`.
  287. #### offset: `int`
  288. The starting index for the documents to retrieve. Typically used in conjunction with `limit`. Defaults to `0`.
  289. #### limit: `int`
  290. The maximum number of documents to retrieve. Defaults to `1024`.
  291. #### orderby: `str`
  292. The field by which documents should be sorted. Available options:
  293. - `"create_time"` (default)
  294. - `"update_time"`
  295. #### desc: `bool`
  296. Indicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.
  297. ### Returns
  298. - Success: A list of `Document` objects.
  299. - Failure: `Exception`.
  300. A `Document` object contains the following attributes:
  301. - `id`: The document ID. Defaults to `""`.
  302. - `name`: The document name. Defaults to `""`.
  303. - `thumbnail`: The thumbnail image of the document. Defaults to `None`.
  304. - `dataset_id`: The dataset ID associated with the document. Defaults to `None`.
  305. - `chunk_method` The chunk method name. Defaults to `"naive"`.
  306. - `source_type`: The source type of the document. Defaults to `"local"`.
  307. - `type`: Type or category of the document. Defaults to `""`. Reserved for future use.
  308. - `created_by`: `str` The creator of the document. Defaults to `""`.
  309. - `size`: `int` The document size in bytes. Defaults to `0`.
  310. - `token_count`: `int` The number of tokens in the document. Defaults to `0`.
  311. - `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.
  312. - `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.
  313. - `progress_msg`: `str` A message indicating the current progress status. Defaults to `""`.
  314. - `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.
  315. - `process_duation`: `float` Duration of the processing in seconds. Defaults to `0.0`.
  316. - `run`: `str` The document's processing status:
  317. - `"UNSTART"` (default)
  318. - `"RUNNING"`
  319. - `"CANCEL"`
  320. - `"DONE"`
  321. - `"FAIL"`
  322. - `status`: `str` Reserved for future use.
  323. - `parser_config`: `ParserConfig` Configuration object for the parser. Its attributes vary based on the selected `chunk_method`:
  324. - `chunk_method`=`"naive"`:
  325. `{"chunk_token_num":128,"delimiter":"\\n!?;。;!?","html4excel":False,"layout_recognize":True,"raptor":{"user_raptor":False}}`.
  326. - `chunk_method`=`"qa"`:
  327. `{"raptor": {"user_raptor": False}}`
  328. - `chunk_method`=`"manuel"`:
  329. `{"raptor": {"user_raptor": False}}`
  330. - `chunk_method`=`"table"`:
  331. `None`
  332. - `chunk_method`=`"paper"`:
  333. `{"raptor": {"user_raptor": False}}`
  334. - `chunk_method`=`"book"`:
  335. `{"raptor": {"user_raptor": False}}`
  336. - `chunk_method`=`"laws"`:
  337. `{"raptor": {"user_raptor": False}}`
  338. - `chunk_method`=`"presentation"`:
  339. `{"raptor": {"user_raptor": False}}`
  340. - `chunk_method`=`"one"`:
  341. `None`
  342. - `chunk_method`=`"knowledge-graph"`:
  343. `{"chunk_token_num":128,"delimiter": "\\n!?;。;!?","entity_types":["organization","person","location","event","time"]}`
  344. ### Examples
  345. ```python
  346. from ragflow import RAGFlow
  347. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  348. dataset = rag_object.create_dataset(name="kb_1")
  349. filename1 = "~/ragflow.txt"
  350. blob = open(filename1 , "rb").read()
  351. dataset.upload_documents([{"name":filename1,"blob":blob}])
  352. for doc in dataset.list_documents(keywords="rag", offset=0, limit=12):
  353. print(doc)
  354. ```
  355. ---
  356. ## Delete documents
  357. ```python
  358. DataSet.delete_documents(ids: list[str] = None)
  359. ```
  360. Deletes documents by ID.
  361. ### Parameters
  362. #### ids: `list[list]`
  363. The IDs of the documents to delete. Defaults to `None`. If it is not specified, all documents in the dataset will be deleted.
  364. ### Returns
  365. - Success: No value is returned.
  366. - Failure: `Exception`
  367. ### Examples
  368. ```python
  369. from ragflow import RAGFlow
  370. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  371. dataset = rag_object.list_datasets(name="kb_1")
  372. dataset = dataset[0]
  373. dataset.delete_documents(ids=["id_1","id_2"])
  374. ```
  375. ---
  376. ## Parse documents
  377. ```python
  378. DataSet.async_parse_documents(document_ids:list[str]) -> None
  379. ```
  380. Parses documents in the current dataset.
  381. ### Parameters
  382. #### document_ids: `list[str]`, *Required*
  383. The IDs of the documents to parse.
  384. ### Returns
  385. - Success: No value is returned.
  386. - Failure: `Exception`
  387. ### Examples
  388. ```python
  389. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  390. dataset = rag_object.create_dataset(name="dataset_name")
  391. documents = [
  392. {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',"rb").read()},
  393. {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',"rb").read()},
  394. {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',"rb").read()}
  395. ]
  396. dataset.upload_documents(documents)
  397. documents = dataset.list_documents(keywords="test")
  398. ids = []
  399. for document in documents:
  400. ids.append(document.id)
  401. dataset.async_parse_documents(ids)
  402. print("Async bulk parsing initiated.")
  403. ```
  404. ---
  405. ## Stop parsing documents
  406. ```python
  407. DataSet.async_cancel_parse_documents(document_ids:list[str])-> None
  408. ```
  409. Stops parsing specified documents.
  410. ### Parameters
  411. #### document_ids: `list[str]`, *Required*
  412. The IDs of the documents for which parsing should be stopped.
  413. ### Returns
  414. - Success: No value is returned.
  415. - Failure: `Exception`
  416. ### Examples
  417. ```python
  418. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  419. dataset = rag_object.create_dataset(name="dataset_name")
  420. documents = [
  421. {'display_name': 'test1.txt', 'blob': open('./test_data/test1.txt',"rb").read()},
  422. {'display_name': 'test2.txt', 'blob': open('./test_data/test2.txt',"rb").read()},
  423. {'display_name': 'test3.txt', 'blob': open('./test_data/test3.txt',"rb").read()}
  424. ]
  425. dataset.upload_documents(documents)
  426. documents = dataset.list_documents(keywords="test")
  427. ids = []
  428. for document in documents:
  429. ids.append(document.id)
  430. dataset.async_parse_documents(ids)
  431. print("Async bulk parsing initiated.")
  432. dataset.async_cancel_parse_documents(ids)
  433. print("Async bulk parsing cancelled.")
  434. ```
  435. ---
  436. ## Add chunk
  437. ```python
  438. Document.add_chunk(content:str, important_keywords:list[str] = []) -> Chunk
  439. ```
  440. Adds a chunk to the current document.
  441. ### Parameters
  442. #### content: `str`, *Required*
  443. The text content of the chunk.
  444. #### important_keywords: `list[str]`
  445. The key terms or phrases to tag with the chunk.
  446. ### Returns
  447. - Success: A `Chunk` object.
  448. - Failure: `Exception`.
  449. A `Chunk` object contains the following attributes:
  450. - `id`: `str`: The chunk ID.
  451. - `content`: `str` The text content of the chunk.
  452. - `important_keywords`: `list[str]` A list of key terms or phrases tagged with the chunk.
  453. - `create_time`: `str` The time when the chunk was created (added to the document).
  454. - `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.
  455. - `dataset_id`: `str` The ID of the associated dataset.
  456. - `document_name`: `str` The name of the associated document.
  457. - `document_id`: `str` The ID of the associated document.
  458. - `available`: `bool` The chunk's availability status in the dataset. Value options:
  459. - `False`: Unavailable
  460. - `True`: Available (default)
  461. ### Examples
  462. ```python
  463. from ragflow import RAGFlow
  464. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  465. dataset = rag_object.list_datasets(id="123")
  466. dtaset = dataset[0]
  467. doc = dataset.list_documents(id="wdfxb5t547d")
  468. doc = doc[0]
  469. chunk = doc.add_chunk(content="xxxxxxx")
  470. ```
  471. ---
  472. ## List chunks
  473. ```python
  474. Document.list_chunks(keywords: str = None, offset: int = 1, limit: int = 1024, id : str = None) -> list[Chunk]
  475. ```
  476. Lists chunks in the current document.
  477. ### Parameters
  478. #### keywords: `str`
  479. The keywords used to match chunk content. Defaults to `None`
  480. #### offset: `int`
  481. The starting index for the chunks to retrieve. Defaults to `1`.
  482. #### limit: `int`
  483. The maximum number of chunks to retrieve. Default: `1024`
  484. #### id: `str`
  485. The ID of the chunk to retrieve. Default: `None`
  486. ### Returns
  487. - Success: A list of `Chunk` objects.
  488. - Failure: `Exception`.
  489. ### Examples
  490. ```python
  491. from ragflow import RAGFlow
  492. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  493. dataset = rag_object.list_datasets("123")
  494. dataset = dataset[0]
  495. dataset.async_parse_documents(["wdfxb5t547d"])
  496. for chunk in doc.list_chunks(keywords="rag", offset=0, limit=12):
  497. print(chunk)
  498. ```
  499. ---
  500. ## Delete chunks
  501. ```python
  502. Document.delete_chunks(chunk_ids: list[str])
  503. ```
  504. Deletes chunks by ID.
  505. ### Parameters
  506. #### chunk_ids: `list[str]`
  507. The IDs of the chunks to delete. Defaults to `None`. If it is not specified, all chunks of the current document will be deleted.
  508. ### Returns
  509. - Success: No value is returned.
  510. - Failure: `Exception`
  511. ### Examples
  512. ```python
  513. from ragflow import RAGFlow
  514. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  515. dataset = rag_object.list_datasets(id="123")
  516. dataset = dataset[0]
  517. doc = dataset.list_documents(id="wdfxb5t547d")
  518. doc = doc[0]
  519. chunk = doc.add_chunk(content="xxxxxxx")
  520. doc.delete_chunks(["id_1","id_2"])
  521. ```
  522. ---
  523. ## Update chunk
  524. ```python
  525. Chunk.update(update_message: dict)
  526. ```
  527. Updates content or configurations for the current chunk.
  528. ### Parameters
  529. #### update_message: `dict[str, str|list[str]|int]` *Required*
  530. A dictionary representing the attributes to update, with the following keys:
  531. - `"content"`: `str` The text content of the chunk.
  532. - `"important_keywords"`: `list[str]` A list of key terms or phrases to tag with the chunk.
  533. - `"available"`: `bool` The chunk's availability status in the dataset. Value options:
  534. - `False`: Unavailable
  535. - `True`: Available (default)
  536. ### Returns
  537. - Success: No value is returned.
  538. - Failure: `Exception`
  539. ### Examples
  540. ```python
  541. from ragflow import RAGFlow
  542. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  543. dataset = rag_object.list_datasets(id="123")
  544. dataset = dataset[0]
  545. doc = dataset.list_documents(id="wdfxb5t547d")
  546. doc = doc[0]
  547. chunk = doc.add_chunk(content="xxxxxxx")
  548. chunk.update({"content":"sdfx..."})
  549. ```
  550. ---
  551. ## Retrieve chunks
  552. ```python
  553. RAGFlow.retrieve(question:str="", dataset_ids:list[str]=None, document_ids=list[str]=None, offset:int=1, limit:int=1024, similarity_threshold:float=0.2, vector_similarity_weight:float=0.3, top_k:int=1024,rerank_id:str=None,keyword:bool=False,higlight:bool=False) -> list[Chunk]
  554. ```
  555. Retrieves chunks from specified datasets.
  556. ### Parameters
  557. #### question: `str` *Required*
  558. The user query or query keywords. Defaults to `""`.
  559. #### dataset_ids: `list[str]`, *Required*
  560. The IDs of the datasets to search. Defaults to `None`. If you do not set this argument, ensure that you set `document_ids`.
  561. #### document_ids: `list[str]`
  562. The IDs of the documents to search. Defaults to `None`. You must ensure all selected documents use the same embedding model. Otherwise, an error will occur. If you do not set this argument, ensure that you set `dataset_ids`.
  563. #### offset: `int`
  564. The starting index for the documents to retrieve. Defaults to `1`.
  565. #### limit: `int`
  566. The maximum number of chunks to retrieve. Defaults to `1024`.
  567. #### Similarity_threshold: `float`
  568. The minimum similarity score. Defaults to `0.2`.
  569. #### vector_similarity_weight: `float`
  570. The weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.
  571. #### top_k: `int`
  572. The number of chunks engaged in vector cosine computaton. Defaults to `1024`.
  573. #### rerank_id: `str`
  574. The ID of the rerank model. Defaults to `None`.
  575. #### keyword: `bool`
  576. Indicates whether to enable keyword-based matching:
  577. - `True`: Enable keyword-based matching.
  578. - `False`: Disable keyword-based matching (default).
  579. #### highlight: `bool`
  580. Specifies whether to enable highlighting of matched terms in the results:
  581. - `True`: Enable highlighting of matched terms.
  582. - `False`: Disable highlighting of matched terms (default).
  583. ### Returns
  584. - Success: A list of `Chunk` objects representing the document chunks.
  585. - Failure: `Exception`
  586. ### Examples
  587. ```python
  588. from ragflow import RAGFlow
  589. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  590. dataset = rag_object.list_datasets(name="ragflow")
  591. dataset = dataset[0]
  592. name = 'ragflow_test.txt'
  593. path = './test_data/ragflow_test.txt'
  594. rag_object.create_document(dataset, name=name, blob=open(path, "rb").read())
  595. doc = dataset.list_documents(name=name)
  596. doc = doc[0]
  597. dataset.async_parse_documents([doc.id])
  598. for c in rag_object.retrieve(question="What's ragflow?",
  599. dataset_ids=[dataset.id], document_ids=[doc.id],
  600. offset=1, limit=30, similarity_threshold=0.2,
  601. vector_similarity_weight=0.3,
  602. top_k=1024
  603. ):
  604. print(c)
  605. ```
  606. ---
  607. :::tip API GROUPING
  608. Chat Assistant Management
  609. :::
  610. ---
  611. ## Create chat assistant
  612. ```python
  613. RAGFlow.create_chat(
  614. name: str,
  615. avatar: str = "",
  616. dataset_ids: list[str] = [],
  617. llm: Chat.LLM = None,
  618. prompt: Chat.Prompt = None
  619. ) -> Chat
  620. ```
  621. Creates a chat assistant.
  622. ### Parameters
  623. #### name: `str`, *Required*
  624. The name of the chat assistant.
  625. #### avatar: `str`
  626. Base64 encoding of the avatar. Defaults to `""`.
  627. #### dataset_ids: `list[str]`
  628. The IDs of the associated datasets. Defaults to `[""]`.
  629. #### llm: `Chat.LLM`
  630. The LLM settings for the chat assistant to create. Defaults to `None`. When the value is `None`, a dictionary with the following values will be generated as the default. An `LLM` object contains the following attributes:
  631. - `model_name`: `str`
  632. The chat model name. If it is `None`, the user's default chat model will be used.
  633. - `temperature`: `float`
  634. Controls the randomness of the model's predictions. A lower temperature increases the model's confidence in its responses; a higher temperature increases creativity and diversity. Defaults to `0.1`.
  635. - `top_p`: `float`
  636. Also known as “nucleus sampling”, this parameter sets a threshold to select a smaller set of words to sample from. It focuses on the most likely words, cutting off the less probable ones. Defaults to `0.3`
  637. - `presence_penalty`: `float`
  638. This discourages the model from repeating the same information by penalizing words that have already appeared in the conversation. Defaults to `0.2`.
  639. - `frequency penalty`: `float`
  640. Similar to the presence penalty, this reduces the model’s tendency to repeat the same words frequently. Defaults to `0.7`.
  641. - `max_token`: `int`
  642. The maximum length of the model’s output, measured in the number of tokens (words or pieces of words). Defaults to `512`.
  643. #### prompt: `Chat.Prompt`
  644. Instructions for the LLM to follow. A `Prompt` object contains the following attributes:
  645. - `similarity_threshold`: `float` RAGFlow uses a hybrid of weighted keyword similarity and vector cosine similarity during retrieval. This argument sets the threshold for similarities between the user query and chunks. If a similarity score falls below this threshold, the corresponding chunk will be excluded from the results. The default value is `0.2`.
  646. - `keywords_similarity_weight`: `float` This argument sets the weight of keyword similarity in the hybrid similarity score with vector cosine similarity or reranking model similarity. By adjusting this weight, you can control the influence of keyword similarity in relation to other similarity measures. The default value is `0.7`.
  647. - `top_n`: `int` This argument specifies the number of top chunks with similarity scores above the `similarity_threshold` that are fed to the LLM. The LLM will *only* access these 'top N' chunks. The default value is `8`.
  648. - `variables`: `list[dict[]]` This argument lists the variables to use in the 'System' field of **Chat Configurations**. Note that:
  649. - `knowledge` is a reserved variable, which represents the retrieved chunks.
  650. - All the variables in 'System' should be curly bracketed.
  651. - The default value is `[{"key": "knowledge", "optional": True}]`.
  652. - `rerank_model`: `str` If it is not specified, vector cosine similarity will be used; otherwise, reranking score will be used. Defaults to `""`.
  653. - `empty_response`: `str` If nothing is retrieved in the dataset for the user's question, this will be used as the response. To allow the LLM to improvise when nothing is found, leave this blank. Defaults to `None`.
  654. - `opener`: `str` The opening greeting for the user. Defaults to `"Hi! I am your assistant, can I help you?"`.
  655. - `show_quote`: `bool` Indicates whether the source of text should be displayed. Defaults to `True`.
  656. - `prompt`: `str` The prompt content. Defaults to `You are an intelligent assistant. Please summarize the content of the dataset to answer the question. Please list the data in the knowledge base and answer in detail. When all knowledge base content is irrelevant to the question, your answer must include the sentence "The answer you are looking for is not found in the knowledge base!" Answers need to consider chat history.
  657. Here is the knowledge base:
  658. {knowledge}
  659. The above is the knowledge base.`
  660. ### Returns
  661. - Success: A `Chat` object representing the chat assistant.
  662. - Failure: `Exception`
  663. ### Examples
  664. ```python
  665. from ragflow import RAGFlow
  666. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  667. datasets = rag_object.list_datasets(name="kb_1")
  668. dataset_ids = []
  669. for dataset in datasets:
  670. dataset_ids.append(dataset.id)
  671. assistant = rag_object.create_chat("Miss R", dataset_ids=dataset_ids)
  672. ```
  673. ---
  674. ## Update chat assistant
  675. ```python
  676. Chat.update(update_message: dict)
  677. ```
  678. Updates configurations for the current chat assistant.
  679. ### Parameters
  680. #### update_message: `dict[str, str|list[str]|dict[]]`, *Required*
  681. A dictionary representing the attributes to update, with the following keys:
  682. - `"name"`: `str` The revised name of the chat assistant.
  683. - `"avatar"`: `str` Base64 encoding of the avatar. Defaults to `""`
  684. - `"dataset_ids"`: `list[str]` The datasets to update.
  685. - `"llm"`: `dict` The LLM settings:
  686. - `"model_name"`, `str` The chat model name.
  687. - `"temperature"`, `float` Controls the randomness of the model's predictions.
  688. - `"top_p"`, `float` Also known as “nucleus sampling”, this parameter sets a threshold to select a smaller set of words to sample from.
  689. - `"presence_penalty"`, `float` This discourages the model from repeating the same information by penalizing words that have appeared in the conversation.
  690. - `"frequency penalty"`, `float` Similar to presence penalty, this reduces the model’s tendency to repeat the same words.
  691. - `"max_token"`, `int` The maximum length of the model’s output, measured in the number of tokens (words or pieces of words).
  692. - `"prompt"` : Instructions for the LLM to follow.
  693. - `"similarity_threshold"`: `float` RAGFlow uses a hybrid of weighted keyword similarity and vector cosine similarity during retrieval. This argument sets the threshold for similarities between the user query and chunks. If a similarity score falls below this threshold, the corresponding chunk will be excluded from the results. The default value is `0.2`.
  694. - `"keywords_similarity_weight"`: `float` This argument sets the weight of keyword similarity in the hybrid similarity score with vector cosine similarity or reranking model similarity. By adjusting this weight, you can control the influence of keyword similarity in relation to other similarity measures. The default value is `0.7`.
  695. - `"top_n"`: `int` This argument specifies the number of top chunks with similarity scores above the `similarity_threshold` that are fed to the LLM. The LLM will *only* access these 'top N' chunks. The default value is `8`.
  696. - `"variables"`: `list[dict[]]` This argument lists the variables to use in the 'System' field of **Chat Configurations**. Note that:
  697. - `knowledge` is a reserved variable, which represents the retrieved chunks.
  698. - All the variables in 'System' should be curly bracketed.
  699. - The default value is `[{"key": "knowledge", "optional": True}]`.
  700. - `"rerank_model"`: `str` If it is not specified, vector cosine similarity will be used; otherwise, reranking score will be used. Defaults to `""`.
  701. - `"empty_response"`: `str` If nothing is retrieved in the dataset for the user's question, this will be used as the response. To allow the LLM to improvise when nothing is retrieved, leave this blank. Defaults to `None`.
  702. - `"opener"`: `str` The opening greeting for the user. Defaults to `"Hi! I am your assistant, can I help you?"`.
  703. - `"show_quote`: `bool` Indicates whether the source of text should be displayed Defaults to `True`.
  704. - `"prompt"`: `str` The prompt content. Defaults to `You are an intelligent assistant. Please summarize the content of the knowledge base to answer the question. Please list the data in the knowledge base and answer in detail. When all knowledge base content is irrelevant to the question, your answer must include the sentence "The answer you are looking for is not found in the knowledge base!" Answers need to consider chat history.
  705. Here is the knowledge base:
  706. {knowledge}
  707. The above is the knowledge base.`.
  708. ### Returns
  709. - Success: No value is returned.
  710. - Failure: `Exception`
  711. ### Examples
  712. ```python
  713. from ragflow import RAGFlow
  714. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  715. datasets = rag_object.list_datasets(name="kb_1")
  716. dataset_id = datasets[0].id
  717. assistant = rag_object.create_chat("Miss R", dataset_ids=[dataset_id])
  718. assistant.update({"name": "Stefan", "llm": {"temperature": 0.8}, "prompt": {"top_n": 8}})
  719. ```
  720. ---
  721. ## Delete chat assistants
  722. ```python
  723. RAGFlow.delete_chats(ids: list[str] = None)
  724. ```
  725. Deletes chat assistants by ID.
  726. ### Parameters
  727. #### ids: `list[str]`
  728. The IDs of the chat assistants to delete. Defaults to `None`. If it is ot specified, all chat assistants in the system will be deleted.
  729. ### Returns
  730. - Success: No value is returned.
  731. - Failure: `Exception`
  732. ### Examples
  733. ```python
  734. from ragflow import RAGFlow
  735. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  736. rag_object.delete_chats(ids=["id_1","id_2"])
  737. ```
  738. ---
  739. ## List chat assistants
  740. ```python
  741. RAGFlow.list_chats(
  742. page: int = 1,
  743. page_size: int = 1024,
  744. orderby: str = "create_time",
  745. desc: bool = True,
  746. id: str = None,
  747. name: str = None
  748. ) -> list[Chat]
  749. ```
  750. Lists chat assistants.
  751. ### Parameters
  752. #### page: `int`
  753. Specifies the page on which the chat assistants will be displayed. Defaults to `1`.
  754. #### page_size: `int`
  755. The number of chat assistants on each page. Defaults to `1024`.
  756. #### orderby: `str`
  757. The attribute by which the results are sorted. Available options:
  758. - `"create_time"` (default)
  759. - `"update_time"`
  760. #### desc: `bool`
  761. Indicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.
  762. #### id: `str`
  763. The ID of the chat assistant to retrieve. Defaults to `None`.
  764. #### name: `str`
  765. The name of the chat assistant to retrieve. Defaults to `None`.
  766. ### Returns
  767. - Success: A list of `Chat` objects.
  768. - Failure: `Exception`.
  769. ### Examples
  770. ```python
  771. from ragflow import RAGFlow
  772. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  773. for assistant in rag_object.list_chats():
  774. print(assistant)
  775. ```
  776. ---
  777. :::tip API GROUPING
  778. Chat Session APIs
  779. :::
  780. ---
  781. ## Create session
  782. ```python
  783. Chat.create_session(name: str = "New session") -> Session
  784. ```
  785. Creates a chat session.
  786. ### Parameters
  787. #### name: `str`
  788. The name of the chat session to create.
  789. ### Returns
  790. - Success: A `Session` object containing the following attributes:
  791. - `id`: `str` The auto-generated unique identifier of the created session.
  792. - `name`: `str` The name of the created session.
  793. - `message`: `list[Message]` The messages of the created session assistant. Default: `[{"role": "assistant", "content": "Hi! I am your assistant,can I help you?"}]`
  794. - `chat_id`: `str` The ID of the associated chat assistant.
  795. - Failure: `Exception`
  796. ### Examples
  797. ```python
  798. from ragflow import RAGFlow
  799. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  800. assistant = rag_object.list_chats(name="Miss R")
  801. assistant = assistant[0]
  802. session = assistant.create_session()
  803. ```
  804. ---
  805. ## Update session
  806. ```python
  807. Session.update(update_message: dict)
  808. ```
  809. Updates the current session.
  810. ### Parameters
  811. #### update_message: `dict[str, Any]`, *Required*
  812. A dictionary representing the attributes to update, with only one key:
  813. - `"name"`: `str` The revised name of the session.
  814. ### Returns
  815. - Success: No value is returned.
  816. - Failure: `Exception`
  817. ### Examples
  818. ```python
  819. from ragflow import RAGFlow
  820. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  821. assistant = rag_object.list_chats(name="Miss R")
  822. assistant = assistant[0]
  823. session = assistant.create_session("session_name")
  824. session.update({"name": "updated_name"})
  825. ```
  826. ---
  827. ## List sessions
  828. ```python
  829. Chat.list_sessions(
  830. page: int = 1,
  831. page_size: int = 1024,
  832. orderby: str = "create_time",
  833. desc: bool = True,
  834. id: str = None,
  835. name: str = None
  836. ) -> list[Session]
  837. ```
  838. Lists sessions associated with the current chat assistant.
  839. ### Parameters
  840. #### page: `int`
  841. Specifies the page on which the sessions will be displayed. Defaults to `1`.
  842. #### page_size: `int`
  843. The number of sessions on each page. Defaults to `1024`.
  844. #### orderby: `str`
  845. The field by which sessions should be sorted. Available options:
  846. - `"create_time"` (default)
  847. - `"update_time"`
  848. #### desc: `bool`
  849. Indicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.
  850. #### id: `str`
  851. The ID of the chat session to retrieve. Defaults to `None`.
  852. #### name: `str`
  853. The name of the chat session to retrieve. Defaults to `None`.
  854. ### Returns
  855. - Success: A list of `Session` objects associated with the current chat assistant.
  856. - Failure: `Exception`.
  857. ### Examples
  858. ```python
  859. from ragflow import RAGFlow
  860. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  861. assistant = rag_object.list_chats(name="Miss R")
  862. assistant = assistant[0]
  863. for session in assistant.list_sessions():
  864. print(session)
  865. ```
  866. ---
  867. ## Delete sessions
  868. ```python
  869. Chat.delete_sessions(ids:list[str] = None)
  870. ```
  871. Deletes sessions by ID.
  872. ### Parameters
  873. #### ids: `list[str]`
  874. The IDs of the sessions to delete. Defaults to `None`. If it is not specified, all sessions associated with the current chat assistant will be deleted.
  875. ### Returns
  876. - Success: No value is returned.
  877. - Failure: `Exception`
  878. ### Examples
  879. ```python
  880. from ragflow import RAGFlow
  881. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  882. assistant = rag_object.list_chats(name="Miss R")
  883. assistant = assistant[0]
  884. assistant.delete_sessions(ids=["id_1","id_2"])
  885. ```
  886. ---
  887. ## Converse
  888. ```python
  889. Session.ask(question: str, stream: bool = False) -> Optional[Message, iter[Message]]
  890. ```
  891. Asks a question to start an AI-powered conversation.
  892. ### Parameters
  893. #### question: `str` *Required*
  894. The question to start an AI-powered conversation.
  895. #### stream: `bool`
  896. Indicates whether to output responses in a streaming way:
  897. - `True`: Enable streaming.
  898. - `False`: Disable streaming (default).
  899. ### Returns
  900. - A `Message` object containing the response to the question if `stream` is set to `False`
  901. - An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`
  902. The following shows the attributes of a `Message` object:
  903. #### id: `str`
  904. The auto-generated message ID.
  905. #### content: `str`
  906. The content of the message. Defaults to `"Hi! I am your assistant, can I help you?"`.
  907. #### reference: `list[Chunk]`
  908. A list of `Chunk` objects representing references to the message, each containing the following attributes:
  909. - `id` `str`
  910. The chunk ID.
  911. - `content` `str`
  912. The content of the chunk.
  913. - `image_id` `str`
  914. The ID of the snapshot of the chunk. Applicable only when the source of the chunk is an image, PPT, PPTX, or PDF file.
  915. - `document_id` `str`
  916. The ID of the referenced document.
  917. - `document_name` `str`
  918. The name of the referenced document.
  919. - `position` `list[str]`
  920. The location information of the chunk within the referenced document.
  921. - `dataset_id` `str`
  922. The ID of the dataset to which the referenced document belongs.
  923. - `similarity` `float`
  924. A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity. It is the weighted sum of `vector_similarity` and `term_similarity`.
  925. - `vector_similarity` `float`
  926. A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.
  927. - `term_similarity` `float`
  928. A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.
  929. ### Examples
  930. ```python
  931. from ragflow import RAGFlow
  932. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  933. assistant = rag_object.list_chats(name="Miss R")
  934. assistant = assistant[0]
  935. session = assistant.create_session()
  936. print("\n==================== Miss R =====================\n")
  937. print(assistant.get_prologue())
  938. while True:
  939. question = input("\n==================== User =====================\n> ")
  940. print("\n==================== Miss R =====================\n")
  941. cont = ""
  942. for ans in session.ask(question, stream=True):
  943. print(answer.content[len(cont):], end='', flush=True)
  944. cont = answer.content
  945. ```