Feat: launch sandbox from docker-compose (#7671)

### What problem does this PR solve?

Launch sandbox from docker-compose.
#4977
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>

README.md

@@ -139,6 +139,7 @@ releases! 🌟
 - Docker >= 24.0.0 & Docker Compose >= v2.26.1
   > If you have not installed Docker on your local machine (Windows, Mac, or Linux),
   > see [Install Docker Engine](https://docs.docker.com/engine/install/).
+  > The [gVisor](https://gvisor.dev/docs/user_guide/install/) is optional and only needed if you plan to use the code executor (sandbox) feature of RAGFlow.
 
 ### 🚀 Start up the server
 
@@ -318,7 +319,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    Add the following line to `/etc/hosts` to resolve all hosts specified in **docker/.env** to `127.0.0.1`:
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. If you cannot access HuggingFace, set the `HF_ENDPOINT` environment variable to use a mirror site:

README_id.md

@@ -284,7 +284,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    Tambahkan baris berikut ke `/etc/hosts` untuk memetakan semua host yang ditentukan di **conf/service_conf.yaml** ke `127.0.0.1`:
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. Jika Anda tidak dapat mengakses HuggingFace, atur variabel lingkungan `HF_ENDPOINT` untuk menggunakan situs mirror:

README_ja.md

@@ -280,7 +280,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    `/etc/hosts` に以下の行を追加して、**conf/service_conf.yaml** に指定されたすべてのホストを `127.0.0.1` に解決します:
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. HuggingFace にアクセスできない場合は、`HF_ENDPOINT` 環境変数を設定してミラーサイトを使用してください:

README_ko.md

@@ -279,7 +279,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    `/etc/hosts` 에 다음 줄을 추가하여 **conf/service_conf.yaml** 에 지정된 모든 호스트를 `127.0.0.1` 로 해결합니다:
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. HuggingFace에 접근할 수 없는 경우, `HF_ENDPOINT` 환경 변수를 설정하여 미러 사이트를 사용하세요:

README_pt_br.md

@@ -303,7 +303,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    Adicione a seguinte linha ao arquivo `/etc/hosts` para resolver todos os hosts especificados em **docker/.env** para `127.0.0.1`:
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. Se não conseguir acessar o HuggingFace, defina a variável de ambiente `HF_ENDPOINT` para usar um site espelho:

README_tzh.md

@@ -292,7 +292,7 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
    在 `/etc/hosts` 中加入以下程式碼，將 **conf/service_conf.yaml** 檔案中的所有 host 位址都解析為 `127.0.0.1`：
 
    ```
-   127.0.0.1 es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 
 4. 如果無法存取 HuggingFace，可以把環境變數 `HF_ENDPOINT` 設為對應的鏡像網站：

README_zh.md

@@ -113,6 +113,7 @@
 - Disk >= 50 GB
 - Docker >= 24.0.0 & Docker Compose >= v2.26.1
   > 如果你并没有在本机安装 Docker（Windows、Mac，或者 Linux）, 可以参考文档 [Install Docker Engine](https://docs.docker.com/engine/install/) 自行安装。
+  > [gVisor](https://gvisor.dev/docs/user_guide/install/) 是可选的，仅在你打算使用 RAGFlow 的代码执行器（沙箱）功能时才需要安装。
 
 ### 🚀 启动服务器
 
@@ -293,7 +294,7 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
    在 `/etc/hosts` 中添加以下代码，目的是将 **conf/service_conf.yaml** 文件中的所有 host 地址都解析为 `127.0.0.1`：
 
    ```
-   127.0.0.1       es01 infinity mysql minio redis
+   127.0.0.1       es01 infinity mysql minio redis sandbox-executor-manager
    ```
 4. 如果无法访问 HuggingFace，可以把环境变量 `HF_ENDPOINT` 设成相应的镜像站点：
 

api/settings.py

@@ -66,6 +66,11 @@ kg_retrievaler = None
 # user registration switch
 REGISTER_ENABLED = 1
 
+
+# sandbox-executor-manager
+SANDBOX_ENABLED = 0
+SANDBOX_HOST = None
+
 BUILTIN_EMBEDDING_MODELS = ["BAAI/bge-large-zh-v1.5@BAAI", "maidalun1020/bce-embedding-base_v1@Youdao"]
 
 
@@ -146,6 +151,10 @@ def init_settings():
     retrievaler = search.Dealer(docStoreConn)
     kg_retrievaler = kg_search.KGSearch(docStoreConn)
 
+    if int(os.environ.get("SANDBOX_ENABLED", "0")):
+        global SANDBOX_HOST
+        SANDBOX_HOST = os.environ.get("SANDBOX_HOST", "sandbox-executor-manager")
+
 
 class CustomEnum(Enum):
     @classmethod

docker/.env

@@ -1,11 +1,10 @@
 # The type of doc engine to use.
 # Available options:
-# - `elasticsearch` (default) 
+# - `elasticsearch` (default)
 # - `infinity` (https://github.com/infiniflow/infinity)
 # - `opensearch` (https://github.com/opensearch-project/OpenSearch)
 DOC_ENGINE=${DOC_ENGINE:-elasticsearch}
 
-
 # ------------------------------
 # docker env var for specifying vector db type at startup
 # (based on the vector db type, the corresponding docker
@@ -19,11 +18,11 @@ STACK_VERSION=8.11.3
 # The hostname where the Elasticsearch service is exposed
 ES_HOST=es01
 
-# The port used to expose the Elasticsearch service to the host machine, 
+# The port used to expose the Elasticsearch service to the host machine,
 # allowing EXTERNAL access to the service running inside the Docker container.
 ES_PORT=1200
 
-# The password for Elasticsearch. 
+# The password for Elasticsearch.
 ELASTIC_PASSWORD=infini_rag_flow
 
 # the hostname where OpenSearch service is exposed, set it not the same as elasticsearch
@@ -36,7 +35,7 @@ OS_HOST=opensearch01
 # At least one uppercase letter, one lowercase letter, one digit, and one special character
 OPENSEARCH_PASSWORD=infini_rag_flow_OS_01
 
-# The port used to expose the Kibana service to the host machine, 
+# The port used to expose the Kibana service to the host machine,
 # allowing EXTERNAL access to the service running inside the Docker container.
 KIBANA_PORT=6601
 KIBANA_USER=rag_flow
@@ -54,40 +53,40 @@ INFINITY_THRIFT_PORT=23817
 INFINITY_HTTP_PORT=23820
 INFINITY_PSQL_PORT=5432
 
-# The password for MySQL. 
+# The password for MySQL.
 MYSQL_PASSWORD=infini_rag_flow
 # The hostname where the MySQL service is exposed
 MYSQL_HOST=mysql
 # The database of the MySQL service to use
 MYSQL_DBNAME=rag_flow
-# The port used to expose the MySQL service to the host machine, 
-# allowing EXTERNAL access to the MySQL database running inside the Docker container. 
+# The port used to expose the MySQL service to the host machine,
+# allowing EXTERNAL access to the MySQL database running inside the Docker container.
 MYSQL_PORT=5455
 
 # The hostname where the MinIO service is exposed
 MINIO_HOST=minio
-# The port used to expose the MinIO console interface to the host machine, 
-# allowing EXTERNAL access to the web-based console running inside the Docker container. 
+# The port used to expose the MinIO console interface to the host machine,
+# allowing EXTERNAL access to the web-based console running inside the Docker container.
 MINIO_CONSOLE_PORT=9001
-# The port used to expose the MinIO API service to the host machine, 
-# allowing EXTERNAL access to the MinIO object storage service running inside the Docker container. 
+# The port used to expose the MinIO API service to the host machine,
+# allowing EXTERNAL access to the MinIO object storage service running inside the Docker container.
 MINIO_PORT=9000
-# The username for MinIO. 
+# The username for MinIO.
 # When updated, you must revise the `minio.user` entry in service_conf.yaml accordingly.
 MINIO_USER=rag_flow
-# The password for MinIO. 
+# The password for MinIO.
 # When updated, you must revise the `minio.password` entry in service_conf.yaml accordingly.
 MINIO_PASSWORD=infini_rag_flow
 
 # The hostname where the Redis service is exposed
 REDIS_HOST=redis
-# The port used to expose the Redis service to the host machine, 
+# The port used to expose the Redis service to the host machine,
 # allowing EXTERNAL access to the Redis service running inside the Docker container.
 REDIS_PORT=6379
 # The password for Redis.
 REDIS_PASSWORD=infini_rag_flow
 
-# The port used to expose RAGFlow's HTTP API service to the host machine, 
+# The port used to expose RAGFlow's HTTP API service to the host machine,
 # allowing EXTERNAL access to the service running inside the Docker container.
 SVR_HTTP_PORT=9380
 
@@ -97,7 +96,7 @@ RAGFLOW_IMAGE=infiniflow/ragflow:v0.18.0-slim
 #
 # To download the RAGFlow Docker image with embedding models, uncomment the following line instead:
 # RAGFLOW_IMAGE=infiniflow/ragflow:v0.18.0
-# 
+#
 # The Docker image of the v0.18.0 edition includes built-in embedding models:
 #   - BAAI/bge-large-zh-v1.5
 #   - maidalun1020/bce-embedding-base_v1
@@ -151,3 +150,32 @@ TIMEZONE='Asia/Shanghai'
 # - Enable registration: 1
 # - Disable registration: 0
 REGISTER_ENABLED=1
+
+# Sandbox settings
+# Important: To enable sandbox, you must re-declare the compose profiles. See hints at the end of file.
+# Double check if you add `sandbox-executor-manager` to your `/etc/hosts`
+# Pull the required base images before running:
+#   docker pull infiniflow/sandbox-base-nodejs:latest
+#   docker pull infiniflow/sandbox-base-python:latest
+# Our default sandbox environments include:
+#   - Node.js base image: includes axios
+#   - Python base image: includes requests, numpy, and pandas
+# Specify custom executor images below if you're using non-default environments.
+# SANDBOX_ENABLED=1
+# SANDBOX_HOST=sandbox-executor-manager
+# SANDBOX_EXECUTOR_MANAGER_IMAGE=infiniflow/sandbox-executor-manager:latest
+# SANDBOX_EXECUTOR_MANAGER_POOL_SIZE=3
+# SANDBOX_BASE_PYTHON_IMAGE=infiniflow/sandbox-base-python:latest
+# SANDBOX_BASE_NODEJS_IMAGE=infiniflow/sandbox-base-nodejs:latest
+# SANDBOX_EXECUTOR_MANAGER_PORT=9385
+# SANDBOX_ENABLE_SECCOMP=false
+
+# Important: To enable sandbox, you must re-declare the compose profiles.
+# 1. Comment out the COMPOSE_PROFILES line above.
+# 2. Uncomment one of the following based on your chosen document engine:
+#    - For Elasticsearch:
+# COMPOSE_PROFILES=elasticsearch,sandbox
+#    - For Infinity:
+# COMPOSE_PROFILES=infinity,sandbox
+#    - For OpenSearch:
+# COMPOSE_PROFILES=opensearch,sandbox

docker/docker-compose-base.yml

@@ -103,6 +103,33 @@ services:
       retries: 120
     restart: on-failure
 
+  sandbox-executor-manager:
+    container_name: ragflow-sandbox-executor-manager
+    profiles:
+      - sandbox
+    image: ${SANDBOX_EXECUTOR_MANAGER_IMAGE}
+    privileged: true
+    ports:
+      - ${SANDBOX_EXECUTOR_MANAGER_PORT}:9385
+    env_file: .env
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    networks:
+      - ragflow
+    security_opt:
+      - no-new-privileges:true
+    environment:
+      - TZ=${TIMEZONE}
+      - SANDBOX_EXECUTOR_MANAGER_POOL_SIZE=${SANDBOX_EXECUTOR_MANAGER_POOL_SIZE:-3}
+      - SANDBOX_BASE_PYTHON_IMAGE=${SANDBOX_BASE_PYTHON_IMAGE:-infiniflow/sandbox-base-python:latest}
+      - SANDBOX_BASE_NODEJS_IMAGE=${SANDBOX_BASE_NODEJS_IMAGE:-infiniflow/sandbox-base-nodejs:latest}
+      - SANDBOX_ENABLE_SECCOMP=${SANDBOX_ENABLE_SECCOMP:-false}
+    healthcheck:
+      test: ["CMD", "curl", "http://localhost:9385/healthz"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    restart: on-failure
 
   mysql:
     # mysql:5.7 linux/arm64 image is unavailable.