fix(security): P0-1 — port exposure + service-token auth on /internal/* + jupyter RPC

The fix lands in three concentric layers, all backed by a single
INTERNAL_SERVICE_TOKEN shared secret so we have one mechanism
instead of three:

1. docker-compose: drop the backend.ports: 8891:8000 and
   runtime.ports: 8892:8000 mappings. Nginx is the only host
   ingress again (architecture §2.2).
2. /internal/v1/*: the storage control plane had six endpoints, five
   of which were dead code (frontend already migrated to
   /api/v1/data-resources/* with JWT; schedule only ever called
   POST /internal/v1/objects). Delete the dead routes, mount the
   one survivor with Depends(require_internal_service) that
   compares the X-Internal-Service-Token header against
   settings.internal_service_token with secrets.compare_digest.
3. POST /api/v1/jupyter on the runtime container: previously open
   inside the Docker network. Same token mechanism — backend's
   runtime_http_client now carries the header, runtime's
   handle_jupyter_action requires the same header. /api/v1/health
   stays open for the Nginx and compose healthchecks.

The schedule worker was already configured to call
POST /internal/v1/objects; build_storage_http_client now
sets the token header so its existing call site keeps working
without changes.

Files touched:
  backend/src/backend/storage_api.py   # 5 dead routes deleted + token guard
  backend/src/backend/main.py          # runtime_http_client header
  runtime/src/runtime/main.py          # require_internal_service Depends
  common/src/common/config.py          # internal_service_token setting
  schedule/src/schedule/service.py     # httpx client header
  docker-compose.yml                   # ports dropped, INTERNAL_SERVICE_TOKEN env
  .env.example                         # INTERNAL_SERVICE_TOKEN placeholder
  API.md / README.md / DEVELOP.md      # §9 trimmed to 1 endpoint

Verified:
  compileall -> 0 errors
  pytest backend/tests -> 37 passed
  in-process ASGI smoke:
    POST /internal/v1/objects no/wrong/correct token -> 401/401/200
    POST /api/v1/jupyter   no/wrong/correct token -> 401/401/200
    5 deleted internal routes -> 404
  docker compose config (with env) -> OK

P0-1 still has one open sub-item (rclone RC --rc-no-auth) that
the user has explicitly deferred; not touched here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
tao.chen
2026-08-17 16:48:46 +08:00
co-authored by Claude Fable 5
parent 6258cf5d12
commit dfe3f0b118
10 changed files with 195 additions and 138 deletions
+5 -4
View File
@@ -36,7 +36,8 @@
▼ ▼
┌──────────────────┐ ┌──────────────────────┐
│ FastAPI Backend │ │ Runtime (Jupyter) │
│ + /internal/v1 │ control │ - rclone FUSE mount │
│ + /internal/v1 │ control │ - rclone FUSE mount │ (P0-1)
│ /objects │ token-Auth │ │
│ (storage) ├──────────────►│ - subprocess pool │
│ - DAG CRUD │ │ (per workspace) │
│ - script CRUD │ └──────────┬───────────┘
@@ -93,12 +94,12 @@ default.conf Nginx 模板(挂载,启动时渲染)
| 服务 | 镜像 | 暴露 | 用途 |
|---|---|---|---|
| `web` | `nginx:alpine` | 宿主机 `:8888``:80` | SPA、`/api/` 反向代理、`/jupyter/{ws}/` auth_request 代理、`/storage/` S3 直通(仅 s3 模式) |
| `backend` | `Dockerfile` | 仅内网 | DAG CRUD、script CRUD、schedule 触发、`/api/v1/auth/jupyter``/internal/v1/*` 存储控制面 |
| `backend` | `Dockerfile` | 仅内网 | DAG CRUD、script CRUD、schedule 触发、`/api/v1/auth/jupyter``/internal/v1/objects` 服务间 RPC(共享 `INTERNAL_SERVICE_TOKEN` 鉴权,P0-1)|
| `runtime` | `Dockerfile` | 仅内网 | 每个 workspace 一个 Jupyter 子进程池、rclone FUSE 挂载 `workspace` 桶(s3 模式) |
| `schedule` | `Dockerfile` | 仅内网 | cron tick + DAG 执行(轮询 MySQL Outbox |
架构**故意只暴露一个宿主机端口**网关);其他服务都在 Docker 内网。
这一点在 `docker-compose.yml` 里强制执行 — backend / runtime / schedule 都没有 `ports:`
架构**故意只暴露一个宿主机端口**(网关);其他服务都在 Docker 内网。
这一点在 `docker-compose.yml` 里强制执行 — backend / runtime / schedule 都没有 `ports:`在 P0-1 之前,backend 与 runtime 曾短暂地把 `8891` / `8892` 映射到宿主机;此映射已被删除,改用 `INTERNAL_SERVICE_TOKEN` 头对 `/internal/v1/*` 做服务间鉴权,见 `API.md §9`
## 快速启动