Nevar pievienot vairāk kā 25 tēmas Tēmai ir jāsākas ar burtu vai ciparu, tā var saturēt domu zīmes ('-') un var būt līdz 35 simboliem gara.

python_api_reference.md 35KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322
  1. # DRAFT Python API Reference
  2. **THE API REFERENCES BELOW ARE STILL UNDER DEVELOPMENT.**
  3. ---
  4. :::tip NOTE
  5. Dataset Management
  6. :::
  7. ---
  8. ## Create dataset
  9. ```python
  10. RAGFlow.create_dataset(
  11. name: str,
  12. avatar: str = "",
  13. description: str = "",
  14. language: str = "English",
  15. permission: str = "me",
  16. document_count: int = 0,
  17. chunk_count: int = 0,
  18. chunk_method: str = "naive",
  19. parser_config: DataSet.ParserConfig = None
  20. ) -> DataSet
  21. ```
  22. Creates a dataset.
  23. ### Parameters
  24. #### name: `str`, *Required*
  25. The unique name of the dataset to create. It must adhere to the following requirements:
  26. - Permitted characters include:
  27. - English letters (a-z, A-Z)
  28. - Digits (0-9)
  29. - "_" (underscore)
  30. - Must begin with an English letter or underscore.
  31. - Maximum 65,535 characters.
  32. - Case-insensitive.
  33. #### avatar: `str`
  34. Base64 encoding of the avatar. Defaults to `""`
  35. #### description: `str`
  36. A brief description of the dataset to create. Defaults to `""`.
  37. #### language: `str`
  38. The language setting of the dataset to create. Available options:
  39. - `"English"` (Default)
  40. - `"Chinese"`
  41. #### permission
  42. Specifies who can access the dataset to create. You can set it only to `"me"` for now.
  43. #### chunk_method, `str`
  44. The chunking method of the dataset to create. Available options:
  45. - `"naive"`: General (default)
  46. - `"manual`: Manual
  47. - `"qa"`: Q&A
  48. - `"table"`: Table
  49. - `"paper"`: Paper
  50. - `"book"`: Book
  51. - `"laws"`: Laws
  52. - `"presentation"`: Presentation
  53. - `"picture"`: Picture
  54. - `"one"`:One
  55. - `"knowledge_graph"`: Knowledge Graph
  56. - `"email"`: Email
  57. #### parser_config
  58. The parser configuration of the dataset. A `ParserConfig` object contains the following attributes:
  59. - `chunk_token_count`: Defaults to `128`.
  60. - `layout_recognize`: Defaults to `True`.
  61. - `delimiter`: Defaults to `"\n!?。;!?"`.
  62. - `task_page_size`: Defaults to `12`.
  63. ### Returns
  64. - Success: A `dataset` object.
  65. - Failure: `Exception`
  66. ### Examples
  67. ```python
  68. from ragflow import RAGFlow
  69. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  70. dataset = rag_object.create_dataset(name="kb_1")
  71. ```
  72. ---
  73. ## Delete datasets
  74. ```python
  75. RAGFlow.delete_datasets(ids: list[str] = None)
  76. ```
  77. Deletes specified datasets or all datasets in the system.
  78. ### Parameters
  79. #### ids: `list[str]`
  80. The IDs of the datasets to delete. Defaults to `None`. If not specified, all datasets in the system will be deleted.
  81. ### Returns
  82. - Success: No value is returned.
  83. - Failure: `Exception`
  84. ### Examples
  85. ```python
  86. rag_object.delete_datasets(ids=["id_1","id_2"])
  87. ```
  88. ---
  89. ## List datasets
  90. ```python
  91. RAGFlow.list_datasets(
  92. page: int = 1,
  93. page_size: int = 1024,
  94. orderby: str = "create_time",
  95. desc: bool = True,
  96. id: str = None,
  97. name: str = None
  98. ) -> list[DataSet]
  99. ```
  100. Retrieves a list of datasets.
  101. ### Parameters
  102. #### page: `int`
  103. Specifies the page on which the datasets will be displayed. Defaults to `1`.
  104. #### page_size: `int`
  105. The number of datasets on each page. Defaults to `1024`.
  106. #### orderby: `str`
  107. The field by which datasets should be sorted. Available options:
  108. - `"create_time"` (default)
  109. - `"update_time"`
  110. #### desc: `bool`
  111. Indicates whether the retrieved datasets should be sorted in descending order. Defaults to `True`.
  112. #### id: `str`
  113. The ID of the dataset to retrieve. Defaults to `None`.
  114. #### name: `str`
  115. The name of the dataset to retrieve. Defaults to `None`.
  116. ### Returns
  117. - Success: A list of `DataSet` objects.
  118. - Failure: `Exception`.
  119. ### Examples
  120. #### List all datasets
  121. ```python
  122. for dataset in rag_object.list_datasets():
  123. print(dataset)
  124. ```
  125. #### Retrieve a dataset by ID
  126. ```python
  127. dataset = rag_object.list_datasets(id = "id_1")
  128. print(dataset[0])
  129. ```
  130. ---
  131. ## Update dataset
  132. ```python
  133. DataSet.update(update_message: dict)
  134. ```
  135. Updates configurations for the current dataset.
  136. ### Parameters
  137. #### update_message: `dict[str, str|int]`, *Required*
  138. A dictionary representing the attributes to update, with the following keys:
  139. - `"name"`: `str` The name of the dataset to update.
  140. - `"embedding_model"`: `str` The embedding model name to update.
  141. - Ensure that `"chunk_count"` is `0` before updating `"embedding_model"`.
  142. - `"chunk_method"`: `str` The chunking method for the dataset. Available options:
  143. - `"naive"`: General
  144. - `"manual`: Manual
  145. - `"qa"`: Q&A
  146. - `"table"`: Table
  147. - `"paper"`: Paper
  148. - `"book"`: Book
  149. - `"laws"`: Laws
  150. - `"presentation"`: Presentation
  151. - `"picture"`: Picture
  152. - `"one"`:One
  153. - `"knowledge_graph"`: Knowledge Graph
  154. - `"email"`: Email
  155. ### Returns
  156. - Success: No value is returned.
  157. - Failure: `Exception`
  158. ### Examples
  159. ```python
  160. from ragflow import RAGFlow
  161. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  162. dataset = rag_object.list_datasets(name="kb_name")
  163. dataset.update({"embedding_model":"BAAI/bge-zh-v1.5", "chunk_method":"manual"})
  164. ```
  165. ---
  166. :::tip API GROUPING
  167. File Management within Dataset
  168. :::
  169. ---
  170. ## Upload documents
  171. ```python
  172. DataSet.upload_documents(document_list: list[dict])
  173. ```
  174. Uploads documents to the current dataset.
  175. ### Parameters
  176. #### document_list: `list[dict]`, *Required*
  177. A list of dictionaries representing the documents to upload, each containing the following keys:
  178. - `"display_name"`: (Optional) The file name to display in the dataset.
  179. - `"blob"`: (Optional) The binary content of the file to upload.
  180. ### Returns
  181. - Success: No value is returned.
  182. - Failure: `Exception`
  183. ### Examples
  184. ```python
  185. dataset = rag_object.create_dataset(name="kb_name")
  186. dataset.upload_documents([{"display_name": "1.txt", "blob": "<BINARY_CONTENT_OF_THE_DOC>"}, {"display_name": "2.pdf", "blob": "<BINARY_CONTENT_OF_THE_DOC>"}])
  187. ```
  188. ---
  189. ## Update document
  190. ```python
  191. Document.update(update_message:dict)
  192. ```
  193. Updates configurations for the current document.
  194. ### Parameters
  195. #### update_message: `dict[str, str|dict[]]`, *Required*
  196. A dictionary representing the attributes to update, with the following keys:
  197. - `"name"`: `str` The name of the document to update.
  198. - `"parser_config"`: `dict[str, Any]` The parsing configuration for the document:
  199. - `"chunk_token_count"`: Defaults to `128`.
  200. - `"layout_recognize"`: Defaults to `True`.
  201. - `"delimiter"`: Defaults to `'\n!?。;!?'`.
  202. - `"task_page_size"`: Defaults to `12`.
  203. - `"chunk_method"`: `str` The parsing method to apply to the document.
  204. - `"naive"`: General
  205. - `"manual`: Manual
  206. - `"qa"`: Q&A
  207. - `"table"`: Table
  208. - `"paper"`: Paper
  209. - `"book"`: Book
  210. - `"laws"`: Laws
  211. - `"presentation"`: Presentation
  212. - `"picture"`: Picture
  213. - `"one"`: One
  214. - `"knowledge_graph"`: Knowledge Graph
  215. - `"email"`: Email
  216. ### Returns
  217. - Success: No value is returned.
  218. - Failure: `Exception`
  219. ### Examples
  220. ```python
  221. from ragflow import RAGFlow
  222. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  223. dataset = rag_object.list_datasets(id='id')
  224. dataset = dataset[0]
  225. doc = dataset.list_documents(id="wdfxb5t547d")
  226. doc = doc[0]
  227. doc.update([{"parser_config": {"chunk_token_count": 256}}, {"chunk_method": "manual"}])
  228. ```
  229. ---
  230. ## Download document
  231. ```python
  232. Document.download() -> bytes
  233. ```
  234. Downloads the current document.
  235. ### Returns
  236. The downloaded document in bytes.
  237. ### Examples
  238. ```python
  239. from ragflow import RAGFlow
  240. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  241. dataset = rag_object.list_datasets(id="id")
  242. dataset = dataset[0]
  243. doc = dataset.list_documents(id="wdfxb5t547d")
  244. doc = doc[0]
  245. open("~/ragflow.txt", "wb+").write(doc.download())
  246. print(doc)
  247. ```
  248. ---
  249. ## List documents
  250. ```python
  251. Dataset.list_documents(id:str =None, keywords: str=None, offset: int=0, limit:int = 1024,order_by:str = "create_time", desc: bool = True) -> list[Document]
  252. ```
  253. Retrieves a list of documents from the current dataset.
  254. ### Parameters
  255. #### id: `str`
  256. The ID of the document to retrieve. Defaults to `None`.
  257. #### keywords: `str`
  258. The keywords used to match document titles. Defaults to `None`.
  259. #### offset: `int`
  260. The starting index for the documents to retrieve. Typically used in confunction with `limit`. Defaults to `0`.
  261. #### limit: `int`
  262. The maximum number of documents to retrieve. Defaults to `1024`. A value of `-1` indicates that all documents should be returned.
  263. #### orderby: `str`
  264. The field by which documents should be sorted. Available options:
  265. - `"create_time"` (default)
  266. - `"update_time"`
  267. #### desc: `bool`
  268. Indicates whether the retrieved documents should be sorted in descending order. Defaults to `True`.
  269. ### Returns
  270. - Success: A list of `Document` objects.
  271. - Failure: `Exception`.
  272. A `Document` object contains the following attributes:
  273. - `id`: The document ID. Defaults to `""`.
  274. - `name`: The document name. Defaults to `""`.
  275. - `thumbnail`: The thumbnail image of the document. Defaults to `None`.
  276. - `knowledgebase_id`: The dataset ID associated with the document. Defaults to `None`.
  277. - `chunk_method` The chunk method name. Defaults to `""`. ?????naive??????
  278. - `parser_config`: `ParserConfig` Configuration object for the parser. Defaults to `{"pages": [[1, 1000000]]}`.
  279. - `source_type`: The source type of the document. Defaults to `"local"`.
  280. - `type`: Type or category of the document. Defaults to `""`. Reserved for future use.
  281. - `created_by`: `str` The creator of the document. Defaults to `""`.
  282. - `size`: `int` The document size in bytes. Defaults to `0`.
  283. - `token_count`: `int` The number of tokens in the document. Defaults to `0`.
  284. - `chunk_count`: `int` The number of chunks in the document. Defaults to `0`.
  285. - `progress`: `float` The current processing progress as a percentage. Defaults to `0.0`.
  286. - `progress_msg`: `str` A message indicating the current progress status. Defaults to `""`.
  287. - `process_begin_at`: `datetime` The start time of document processing. Defaults to `None`.
  288. - `process_duation`: `float` Duration of the processing in seconds. Defaults to `0.0`.
  289. - `run`: `str` The document's processing status:
  290. - `"0"`: UNSTART (default)
  291. - `"1"`: RUNNING
  292. - `"2"`: CANCEL
  293. - `"3"`: DONE
  294. - `"4"`: FAIL
  295. - `status`: `str` Reserved for future use.
  296. ### Examples
  297. ```python
  298. from ragflow import RAGFlow
  299. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  300. dataset = rag_object.create_dataset(name="kb_1")
  301. filename1 = "~/ragflow.txt"
  302. blob = open(filename1 , "rb").read()
  303. dataset.upload_documents([{"name":filename1,"blob":blob}])
  304. for doc in dataset.list_documents(keywords="rag", offset=0, limit=12):
  305. print(doc)
  306. ```
  307. ---
  308. ## Delete documents
  309. ```python
  310. DataSet.delete_documents(ids: list[str] = None)
  311. ```
  312. Deletes documents by ID.
  313. ### Parameters
  314. #### ids: `list[list]`
  315. The IDs of the documents to delete. Defaults to `None`. If not specified, all documents in the dataset will be deleted.
  316. ### Returns
  317. - Success: No value is returned.
  318. - Failure: `Exception`
  319. ### Examples
  320. ```python
  321. from ragflow import RAGFlow
  322. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  323. dataset = rag_object.list_datasets(name="kb_1")
  324. dataset = dataset[0]
  325. dataset.delete_documents(ids=["id_1","id_2"])
  326. ```
  327. ---
  328. ## Parse documents
  329. ```python
  330. DataSet.async_parse_documents(document_ids:list[str]) -> None
  331. ```
  332. Parses documents in the current dataset.
  333. ### Parameters
  334. #### document_ids: `list[str]`, *Required*
  335. The IDs of the documents to parse.
  336. ### Returns
  337. - Success: No value is returned.
  338. - Failure: `Exception`
  339. ### Examples
  340. ```python
  341. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  342. dataset = rag_object.create_dataset(name="dataset_name")
  343. documents = [
  344. {'name': 'test1.txt', 'blob': open('./test_data/test1.txt',"rb").read()},
  345. {'name': 'test2.txt', 'blob': open('./test_data/test2.txt',"rb").read()},
  346. {'name': 'test3.txt', 'blob': open('./test_data/test3.txt',"rb").read()}
  347. ]
  348. dataset.upload_documents(documents)
  349. documents = dataset.list_documents(keywords="test")
  350. ids = []
  351. for document in documents:
  352. ids.append(document.id)
  353. dataset.async_parse_documents(ids)
  354. print("Async bulk parsing initiated.")
  355. ```
  356. ---
  357. ## Stop parsing documents
  358. ```python
  359. DataSet.async_cancel_parse_documents(document_ids:list[str])-> None
  360. ```
  361. Stops parsing specified documents.
  362. ### Parameters
  363. #### document_ids: `list[str]`, *Required*
  364. The IDs of the documents for which parsing should be stopped.
  365. ### Returns
  366. - Success: No value is returned.
  367. - Failure: `Exception`
  368. ### Examples
  369. ```python
  370. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  371. dataset = rag_object.create_dataset(name="dataset_name")
  372. documents = [
  373. {'name': 'test1.txt', 'blob': open('./test_data/test1.txt',"rb").read()},
  374. {'name': 'test2.txt', 'blob': open('./test_data/test2.txt',"rb").read()},
  375. {'name': 'test3.txt', 'blob': open('./test_data/test3.txt',"rb").read()}
  376. ]
  377. dataset.upload_documents(documents)
  378. documents = dataset.list_documents(keywords="test")
  379. ids = []
  380. for document in documents:
  381. ids.append(document.id)
  382. dataset.async_parse_documents(ids)
  383. print("Async bulk parsing initiated.")
  384. dataset.async_cancel_parse_documents(ids)
  385. print("Async bulk parsing cancelled.")
  386. ```
  387. ---
  388. ## Add chunk
  389. ```python
  390. Document.add_chunk(content:str) -> Chunk ?????????????????????
  391. ```
  392. Adds a chunk to the current document.
  393. ### Parameters
  394. #### content: `str`, *Required*
  395. The text content of the chunk.
  396. #### important_keywords: `list[str]` ??????????????????????
  397. The key terms or phrases to tag with the chunk.
  398. ### Returns
  399. - Success: A `Chunk` object.
  400. - Failure: `Exception`.
  401. A `Chunk` object contains the following attributes:
  402. - `id`: `str`
  403. - `content`: `str` Content of the chunk.
  404. - `important_keywords`: `list[str]` A list of key terms or phrases to tag with the chunk.
  405. - `create_time`: `str` The time when the chunk was created (added to the document).
  406. - `create_timestamp`: `float` The timestamp representing the creation time of the chunk, expressed in seconds since January 1, 1970.
  407. - `knowledgebase_id`: `str` The ID of the associated dataset.
  408. - `document_name`: `str` The name of the associated document.
  409. - `document_id`: `str` The ID of the associated document.
  410. - `available`: `int`???? The chunk's availability status in the dataset. Value options:
  411. - `0`: Unavailable
  412. - `1`: Available
  413. ### Examples
  414. ```python
  415. from ragflow import RAGFlow
  416. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  417. dataset = rag_object.list_datasets(id="123")
  418. dtaset = dataset[0]
  419. doc = dataset.list_documents(id="wdfxb5t547d")
  420. doc = doc[0]
  421. chunk = doc.add_chunk(content="xxxxxxx")
  422. ```
  423. ---
  424. ## List chunks
  425. ```python
  426. Document.list_chunks(keywords: str = None, offset: int = 0, limit: int = -1, id : str = None) -> list[Chunk]
  427. ```
  428. Retrieves a list of chunks from the current document.
  429. ### Parameters
  430. #### keywords: `str`
  431. The keywords used to match chunk content. Defaults to `None`
  432. #### offset: `int`
  433. The starting index for the chunks to retrieve. Defaults to `1`??????
  434. #### limit
  435. The maximum number of chunks to retrieve. Default: `30`?????????
  436. #### id
  437. The ID of the chunk to retrieve. Default: `None`
  438. ### Returns
  439. - Success: A list of `Chunk` objects.
  440. - Failure: `Exception`.
  441. ### Examples
  442. ```python
  443. from ragflow import RAGFlow
  444. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  445. dataset = rag_object.list_datasets("123")
  446. dataset = dataset[0]
  447. dataset.async_parse_documents(["wdfxb5t547d"])
  448. for chunk in doc.list_chunks(keywords="rag", offset=0, limit=12):
  449. print(chunk)
  450. ```
  451. ---
  452. ## Delete chunks
  453. ```python
  454. Document.delete_chunks(chunk_ids: list[str])
  455. ```
  456. Deletes chunks by ID.
  457. ### Parameters
  458. #### chunk_ids: `list[str]`
  459. The IDs of the chunks to delete. Defaults to `None`. If not specified, all chunks of the current document will be deleted.
  460. ### Returns
  461. - Success: No value is returned.
  462. - Failure: `Exception`
  463. ### Examples
  464. ```python
  465. from ragflow import RAGFlow
  466. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  467. dataset = rag_object.list_datasets(id="123")
  468. dataset = dataset[0]
  469. doc = dataset.list_documents(id="wdfxb5t547d")
  470. doc = doc[0]
  471. chunk = doc.add_chunk(content="xxxxxxx")
  472. doc.delete_chunks(["id_1","id_2"])
  473. ```
  474. ---
  475. ## Update chunk
  476. ```python
  477. Chunk.update(update_message: dict)
  478. ```
  479. Updates content or configurations for the current chunk.
  480. ### Parameters
  481. #### update_message: `dict[str, str|list[str]|int]` *Required*
  482. A dictionary representing the attributes to update, with the following keys:
  483. - `"content"`: `str` Content of the chunk.
  484. - `"important_keywords"`: `list[str]` A list of key terms or phrases to tag with the chunk.
  485. - `"available"`: `int` The chunk's availability status in the dataset. Value options:
  486. - `0`: Unavailable
  487. - `1`: Available
  488. ### Returns
  489. - Success: No value is returned.
  490. - Failure: `Exception`
  491. ### Examples
  492. ```python
  493. from ragflow import RAGFlow
  494. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  495. dataset = rag_object.list_datasets(id="123")
  496. dataset = dataset[0]
  497. doc = dataset.list_documents(id="wdfxb5t547d")
  498. doc = doc[0]
  499. chunk = doc.add_chunk(content="xxxxxxx")
  500. chunk.update({"content":"sdfx..."})
  501. ```
  502. ---
  503. ## Retrieve chunks
  504. ```python
  505. RAGFlow.retrieve(question:str="", datasets:list[str]=None, document=list[str]=None, offset:int=1, limit:int=30, similarity_threshold:float=0.2, vector_similarity_weight:float=0.3, top_k:int=1024,rerank_id:str=None,keyword:bool=False,higlight:bool=False) -> list[Chunk]
  506. ```
  507. ???????
  508. ### Parameters
  509. #### question: `str` *Required*
  510. The user query or query keywords. Defaults to `""`.
  511. #### datasets: `list[str]`, *Required*?????
  512. The datasets to search from.
  513. #### document: `list[str]`
  514. The documents to search from. `None` means no limitation. Defaults to `None`.
  515. #### offset: `int`
  516. The starting index for the documents to retrieve. Defaults to `0`??????.
  517. #### limit: `int`
  518. The maximum number of chunks to retrieve. Defaults to `6`.???????????????
  519. #### Similarity_threshold: `float`
  520. The minimum similarity score. Defaults to `0.2`.
  521. #### similarity_threshold_weight: `float`
  522. The weight of vector cosine similarity. Defaults to `0.3`. If x represents the vector cosine similarity, then (1 - x) is the term similarity weight.
  523. #### top_k: `int`
  524. The number of chunks engaged in vector cosine computaton. Defaults to `1024`.
  525. #### rerank_id: `str`
  526. The ID of the rerank model. Defaults to `None`.
  527. #### keyword: `bool`
  528. Indicates whether keyword-based matching is enabled:
  529. - `True`: Enabled.
  530. - `False`: Disabled (default).
  531. #### highlight: `bool`
  532. Specifying whether to enable highlighting of matched terms in the results (True) or not (False).
  533. ### Returns
  534. - Success: A list of `Chunk` objects representing the document chunks.
  535. - Failure: `Exception`
  536. ### Examples
  537. ```python
  538. from ragflow import RAGFlow
  539. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  540. dataset = rag_object.list_datasets(name="ragflow")
  541. dataset = dataset[0]
  542. name = 'ragflow_test.txt'
  543. path = './test_data/ragflow_test.txt'
  544. rag_object.create_document(dataset, name=name, blob=open(path, "rb").read())
  545. doc = dataset.list_documents(name=name)
  546. doc = doc[0]
  547. dataset.async_parse_documents([doc.id])
  548. for c in rag_object.retrieve(question="What's ragflow?",
  549. datasets=[dataset.id], documents=[doc.id],
  550. offset=1, limit=30, similarity_threshold=0.2,
  551. vector_similarity_weight=0.3,
  552. top_k=1024
  553. ):
  554. print(c)
  555. ```
  556. ---
  557. :::tip API GROUPING
  558. Chat Assistant Management
  559. :::
  560. ---
  561. ## Create chat assistant
  562. ```python
  563. RAGFlow.create_chat(
  564. name: str,
  565. avatar: str = "",
  566. knowledgebases: list[str] = [],
  567. llm: Chat.LLM = None,
  568. prompt: Chat.Prompt = None
  569. ) -> Chat
  570. ```
  571. Creates a chat assistant.
  572. ### Parameters
  573. The following shows the attributes of a `Chat` object:
  574. #### name: `str`, *Required*????????
  575. The name of the chat assistant. Defaults to `"assistant"`.
  576. #### avatar: `str`
  577. Base64 encoding of the avatar. Defaults to `""`.
  578. #### knowledgebases: `list[str]`
  579. The IDs of the associated datasets. Defaults to `[""]`.
  580. #### llm: `Chat.LLM`
  581. The llm of the created chat. Defaults to `None`. When the value is `None`, a dictionary with the following values will be generated as the default.
  582. An `LLM` object contains the following attributes:
  583. - `model_name`, `str`
  584. The chat model name. If it is `None`, the user's default chat model will be returned.
  585. - `temperature`, `float`
  586. Controls the randomness of the model's predictions. A lower temperature increases the model's conficence in its responses; a higher temperature increases creativity and diversity. Defaults to `0.1`.
  587. - `top_p`, `float`
  588. Also known as “nucleus sampling”, this parameter sets a threshold to select a smaller set of words to sample from. It focuses on the most likely words, cutting off the less probable ones. Defaults to `0.3`
  589. - `presence_penalty`, `float`
  590. This discourages the model from repeating the same information by penalizing words that have already appeared in the conversation. Defaults to `0.2`.
  591. - `frequency penalty`, `float`
  592. Similar to the presence penalty, this reduces the model’s tendency to repeat the same words frequently. Defaults to `0.7`.
  593. - `max_token`, `int`
  594. This sets the maximum length of the model’s output, measured in the number of tokens (words or pieces of words). Defaults to `512`.
  595. #### prompt: `Chat.Prompt`
  596. Instructions for the LLM to follow. A `Prompt` object contains the following attributes:
  597. - `"similarity_threshold"`: `float` A similarity score to evaluate distance between two lines of text. It's weighted keywords similarity and vector cosine similarity. If the similarity between query and chunk is less than this threshold, the chunk will be filtered out. Defaults to `0.2`.
  598. - `"keywords_similarity_weight"`: `float` It's weighted keywords similarity and vector cosine similarity or rerank score (0~1). Defaults to `0.7`.
  599. - `"top_n"`: `int` Not all the chunks whose similarity score is above the 'similarity threshold' will be feed to LLMs. LLM can only see these 'Top N' chunks. Defaults to `8`.
  600. - `"variables"`: `list[dict[]]` If you use dialog APIs, the variables might help you chat with your clients with different strategies. The variables are used to fill in the 'System' part in prompt in order to give LLM a hint. The 'knowledge' is a very special variable which will be filled-in with the retrieved chunks. All the variables in 'System' should be curly bracketed. Defaults to `[{"key": "knowledge", "optional": True}]`
  601. - `"rerank_model"`: `str` If it is not specified, vector cosine similarity will be used; otherwise, reranking score will be used. Defaults to `""`.
  602. - `"empty_response"`: `str` If nothing is retrieved in the dataset for the user's question, this will be used as the response. To allow the LLM to improvise when nothing is retrieved, leave this blank. Defaults to `None`.
  603. - `"opener"`: `str` The opening greeting for the user. Defaults to `"Hi! I am your assistant, can I help you?"`.
  604. - `"show_quote`: `bool` Indicates whether the source of text should be displayed Defaults to `True`.
  605. - `"prompt"`: `str` The prompt content. Defaults to `You are an intelligent assistant. Please summarize the content of the dataset to answer the question. Please list the data in the knowledge base and answer in detail. When all knowledge base content is irrelevant to the question, your answer must include the sentence "The answer you are looking for is not found in the knowledge base!" Answers need to consider chat history.
  606. Here is the knowledge base:
  607. {knowledge}
  608. The above is the knowledge base.`.
  609. ### Returns
  610. - Success: A `Chat` object representing the chat assistant.
  611. - Failure: `Exception`
  612. ### Examples
  613. ```python
  614. from ragflow import RAGFlow
  615. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  616. datasets = rag_object.list_datasets(name="kb_1")
  617. dataset_ids = []
  618. for dataset in datasets:
  619. dataset_ids.append(dataset.id)
  620. assistant = rag_object.create_chat("Miss R", knowledgebases=dataset_ids)
  621. ```
  622. ---
  623. ## Update chat assistant
  624. ```python
  625. Chat.update(update_message: dict)
  626. ```
  627. Updates configurations for the current chat assistant.
  628. ### Parameters
  629. #### update_message: `dict[str, str|list[str]|dict[]]`, *Required*
  630. A dictionary representing the attributes to update, with the following keys:
  631. - `"name"`: `str` The name of the chat assistant to update.
  632. - `"avatar"`: `str` Base64 encoding of the avatar. Defaults to `""`
  633. - `"knowledgebases"`: `list[str]` The datasets to update.
  634. - `"llm"`: `dict` The LLM settings:
  635. - `"model_name"`, `str` The chat model name.
  636. - `"temperature"`, `float` Controls the randomness of the model's predictions.
  637. - `"top_p"`, `float` Also known as “nucleus sampling”, this parameter sets a threshold to select a smaller set of words to sample from.
  638. - `"presence_penalty"`, `float` This discourages the model from repeating the same information by penalizing words that have appeared in the conversation.
  639. - `"frequency penalty"`, `float` Similar to presence penalty, this reduces the model’s tendency to repeat the same words.
  640. - `"max_token"`, `int` This sets the maximum length of the model’s output, measured in the number of tokens (words or pieces of words).
  641. - `"prompt"` : Instructions for the LLM to follow.
  642. - `"similarity_threshold"`: `float` A score to evaluate distance between two lines of text. It's weighted keywords similarity and vector cosine similarity. If the similarity between query and chunk is less than this threshold, the chunk will be filtered out. Defaults to `0.2`.
  643. - `"keywords_similarity_weight"`: `float` It's weighted keywords similarity and vector cosine similarity or rerank score (0~1). Defaults to `0.7`.
  644. - `"top_n"`: `int` Not all the chunks whose similarity score is above the 'similarity threshold' will be feed to LLMs. LLM can only see these 'Top N' chunks. Defaults to `8`.
  645. - `"variables"`: `list[dict[]]` If you use dialog APIs, the variables might help you chat with your clients with different strategies. The variables are used to fill in the 'System' part in prompt in order to give LLM a hint. The 'knowledge' is a very special variable which will be filled-in with the retrieved chunks. All the variables in 'System' should be curly bracketed. Defaults to `[{"key": "knowledge", "optional": True}]`
  646. - `"rerank_model"`: `str` If it is not specified, vector cosine similarity will be used; otherwise, reranking score will be used. Defaults to `""`.
  647. - `"empty_response"`: `str` If nothing is retrieved in the dataset for the user's question, this will be used as the response. To allow the LLM to improvise when nothing is retrieved, leave this blank. Defaults to `None`.
  648. - `"opener"`: `str` The opening greeting for the user. Defaults to `"Hi! I am your assistant, can I help you?"`.
  649. - `"show_quote`: `bool` Indicates whether the source of text should be displayed Defaults to `True`.
  650. - `"prompt"`: `str` The prompt content. Defaults to `You are an intelligent assistant. Please summarize the content of the knowledge base to answer the question. Please list the data in the knowledge base and answer in detail. When all knowledge base content is irrelevant to the question, your answer must include the sentence "The answer you are looking for is not found in the knowledge base!" Answers need to consider chat history.
  651. Here is the knowledge base:
  652. {knowledge}
  653. The above is the knowledge base.`.
  654. ### Returns
  655. - Success: No value is returned.
  656. - Failure: `Exception`
  657. ### Examples
  658. ```python
  659. from ragflow import RAGFlow
  660. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  661. datasets = rag_object.list_datasets(name="kb_1")
  662. assistant = rag_object.create_chat("Miss R", knowledgebases=datasets)
  663. assistant.update({"name": "Stefan", "llm": {"temperature": 0.8}, "prompt": {"top_n": 8}})
  664. ```
  665. ---
  666. ## Delete chat assistants
  667. ```python
  668. RAGFlow.delete_chats(ids: list[str] = None)
  669. ```
  670. Deletes chat assistants by ID.
  671. ### Parameters
  672. #### ids: `list[str]`
  673. The IDs of the chat assistants to delete. Defaults to `None`. If not specified, all chat assistants in the system will be deleted.
  674. ### Returns
  675. - Success: No value is returned.
  676. - Failure: `Exception`
  677. ### Examples
  678. ```python
  679. from ragflow import RAGFlow
  680. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  681. rag_object.delete_chats(ids=["id_1","id_2"])
  682. ```
  683. ---
  684. ## List chat assistants
  685. ```python
  686. RAGFlow.list_chats(
  687. page: int = 1,
  688. page_size: int = 1024,
  689. orderby: str = "create_time",
  690. desc: bool = True,
  691. id: str = None,
  692. name: str = None
  693. ) -> list[Chat]
  694. ```
  695. Retrieves a list of chat assistants.
  696. ### Parameters
  697. #### page: `int`
  698. Specifies the page on which the chat assistants will be displayed. Defaults to `1`.
  699. #### page_size: `int`
  700. The number of chat assistants on each page. Defaults to `1024`.
  701. #### orderby: `str`
  702. The attribute by which the results are sorted. Available options:
  703. - `"create_time"` (default)
  704. - `"update_time"`
  705. #### desc: `bool`
  706. Indicates whether the retrieved chat assistants should be sorted in descending order. Defaults to `True`.
  707. #### id: `str`
  708. The ID of the chat assistant to retrieve. Defaults to `None`.
  709. #### name: `str`
  710. The name of the chat assistant to retrieve. Defaults to `None`.
  711. ### Returns
  712. - Success: A list of `Chat` objects.
  713. - Failure: `Exception`.
  714. ### Examples
  715. ```python
  716. from ragflow import RAGFlow
  717. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  718. for assistant in rag_object.list_chats():
  719. print(assistant)
  720. ```
  721. ---
  722. :::tip API GROUPING
  723. Chat Session APIs
  724. :::
  725. ---
  726. ## Create session
  727. ```python
  728. Chat.create_session(name: str = "New session") -> Session
  729. ```
  730. Creates a chat session.
  731. ### Parameters
  732. #### name: `str`
  733. The name of the chat session to create.
  734. ### Returns
  735. - Success: A `Session` object containing the following attributes:
  736. - `id`: `str` The auto-generated unique identifier of the created session.
  737. - `name`: `str` The name of the created session.
  738. - `message`: `list[Message]` The messages of the created session assistant. Default: `[{"role": "assistant", "content": "Hi! I am your assistant,can I help you?"}]`
  739. - `chat_id`: `str` The ID of the associated chat assistant.
  740. - Failure: `Exception`
  741. ### Examples
  742. ```python
  743. from ragflow import RAGFlow
  744. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  745. assistant = rag_object.list_chats(name="Miss R")
  746. assistant = assistant[0]
  747. session = assistant.create_session()
  748. ```
  749. ---
  750. ## Update session
  751. ```python
  752. Session.update(update_message: dict)
  753. ```
  754. Updates the current session name.
  755. ### Parameters
  756. #### update_message: `dict[str, Any]`, *Required*
  757. A dictionary representing the attributes to update, with only one key:
  758. - `"name"`: `str` The name of the session to update.
  759. ### Returns
  760. - Success: No value is returned.
  761. - Failure: `Exception`
  762. ### Examples
  763. ```python
  764. from ragflow import RAGFlow
  765. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  766. assistant = rag_object.list_chats(name="Miss R")
  767. assistant = assistant[0]
  768. session = assistant.create_session("session_name")
  769. session.update({"name": "updated_name"})
  770. ```
  771. ---
  772. ## List sessions
  773. ```python
  774. Chat.list_sessions(
  775. page: int = 1,
  776. page_size: int = 1024,
  777. orderby: str = "create_time",
  778. desc: bool = True,
  779. id: str = None,
  780. name: str = None
  781. ) -> list[Session]
  782. ```
  783. Lists sessions associated with the current chat assistant.
  784. ### Parameters
  785. #### page: `int`
  786. Specifies the page on which the sessions will be displayed. Defaults to `1`.
  787. #### page_size: `int`
  788. The number of sessions on each page. Defaults to `1024`.
  789. #### orderby: `str`
  790. The field by which sessions should be sorted. Available options:
  791. - `"create_time"` (default)
  792. - `"update_time"`
  793. #### desc: `bool`
  794. Indicates whether the retrieved sessions should be sorted in descending order. Defaults to `True`.
  795. #### id: `str`
  796. The ID of the chat session to retrieve. Defaults to `None`.
  797. #### name: `str`
  798. The name of the chat session to retrieve. Defaults to `None`.
  799. ### Returns
  800. - Success: A list of `Session` objects associated with the current chat assistant.
  801. - Failure: `Exception`.
  802. ### Examples
  803. ```python
  804. from ragflow import RAGFlow
  805. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  806. assistant = rag_object.list_chats(name="Miss R")
  807. assistant = assistant[0]
  808. for session in assistant.list_sessions():
  809. print(session)
  810. ```
  811. ---
  812. ## Delete sessions
  813. ```python
  814. Chat.delete_sessions(ids:list[str] = None)
  815. ```
  816. Deletes sessions by ID.
  817. ### Parameters
  818. #### ids: `list[str]`
  819. The IDs of the sessions to delete. Defaults to `None`. If not specified, all sessions associated with the current chat assistant will be deleted.
  820. ### Returns
  821. - Success: No value is returned.
  822. - Failure: `Exception`
  823. ### Examples
  824. ```python
  825. from ragflow import RAGFlow
  826. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  827. assistant = rag_object.list_chats(name="Miss R")
  828. assistant = assistant[0]
  829. assistant.delete_sessions(ids=["id_1","id_2"])
  830. ```
  831. ---
  832. ## Chat
  833. ```python
  834. Session.ask(question: str, stream: bool = False) -> Optional[Message, iter[Message]]
  835. ```
  836. Asks a question to start a conversation.
  837. ### Parameters
  838. #### question: `str` *Required*
  839. The question to start an AI chat.
  840. #### stream: `str`
  841. Indicates whether to output responses in a streaming way:
  842. - `True`: Enable streaming.
  843. - `False`: (Default) Disable streaming.
  844. ### Returns
  845. - A `Message` object containing the response to the question if `stream` is set to `False`
  846. - An iterator containing multiple `message` objects (`iter[Message]`) if `stream` is set to `True`
  847. The following shows the attributes of a `Message` object:
  848. #### id: `str`
  849. The auto-generated message ID.
  850. #### content: `str`
  851. The content of the message. Defaults to `"Hi! I am your assistant, can I help you?"`.
  852. #### reference: `list[Chunk]`
  853. A list of `Chunk` objects representing references to the message, each containing the following attributes:
  854. - `id` `str`
  855. The chunk ID.
  856. - `content` `str`
  857. The content of the chunk.
  858. - `image_id` `str`
  859. The ID of the snapshot of the chunk.
  860. - `document_id` `str`
  861. The ID of the referenced document.
  862. - `document_name` `str`
  863. The name of the referenced document.
  864. - `position` `list[str]`
  865. The location information of the chunk within the referenced document.
  866. - `knowledgebase_id` `str`
  867. The ID of the dataset to which the referenced document belongs.
  868. - `similarity` `float`
  869. A composite similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity.
  870. - `vector_similarity` `float`
  871. A vector similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between vector embeddings.
  872. - `term_similarity` `float`
  873. A keyword similarity score of the chunk ranging from `0` to `1`, with a higher value indicating greater similarity between keywords.
  874. ### Examples
  875. ```python
  876. from ragflow import RAGFlow
  877. rag_object = RAGFlow(api_key="<YOUR_API_KEY>", base_url="http://<YOUR_BASE_URL>:9380")
  878. assistant = rag_object.list_chats(name="Miss R")
  879. assistant = assistant[0]
  880. session = assistant.create_session()
  881. print("\n==================== Miss R =====================\n")
  882. print(assistant.get_prologue())
  883. while True:
  884. question = input("\n==================== User =====================\n> ")
  885. print("\n==================== Miss R =====================\n")
  886. cont = ""
  887. for ans in session.ask(question, stream=True):
  888. print(answer.content[len(cont):], end='', flush=True)
  889. cont = answer.content
  890. ```