fix(security): P0-1 — port exposure + service-token auth on /internal/* + jupyter RPC

The fix lands in three concentric layers, all backed by a single
INTERNAL_SERVICE_TOKEN shared secret so we have one mechanism
instead of three:

1. docker-compose: drop the backend.ports: 8891:8000 and
   runtime.ports: 8892:8000 mappings. Nginx is the only host
   ingress again (architecture §2.2).
2. /internal/v1/*: the storage control plane had six endpoints, five
   of which were dead code (frontend already migrated to
   /api/v1/data-resources/* with JWT; schedule only ever called
   POST /internal/v1/objects). Delete the dead routes, mount the
   one survivor with Depends(require_internal_service) that
   compares the X-Internal-Service-Token header against
   settings.internal_service_token with secrets.compare_digest.
3. POST /api/v1/jupyter on the runtime container: previously open
   inside the Docker network. Same token mechanism — backend's
   runtime_http_client now carries the header, runtime's
   handle_jupyter_action requires the same header. /api/v1/health
   stays open for the Nginx and compose healthchecks.

The schedule worker was already configured to call
POST /internal/v1/objects; build_storage_http_client now
sets the token header so its existing call site keeps working
without changes.

Files touched:
  backend/src/backend/storage_api.py   # 5 dead routes deleted + token guard
  backend/src/backend/main.py          # runtime_http_client header
  runtime/src/runtime/main.py          # require_internal_service Depends
  common/src/common/config.py          # internal_service_token setting
  schedule/src/schedule/service.py     # httpx client header
  docker-compose.yml                   # ports dropped, INTERNAL_SERVICE_TOKEN env
  .env.example                         # INTERNAL_SERVICE_TOKEN placeholder
  API.md / README.md / DEVELOP.md      # §9 trimmed to 1 endpoint

Verified:
  compileall -> 0 errors
  pytest backend/tests -> 37 passed
  in-process ASGI smoke:
    POST /internal/v1/objects no/wrong/correct token -> 401/401/200
    POST /api/v1/jupyter   no/wrong/correct token -> 401/401/200
    5 deleted internal routes -> 404
  docker compose config (with env) -> OK

P0-1 still has one open sub-item (rclone RC --rc-no-auth) that
the user has explicitly deferred; not touched here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
tao.chen
2026-08-17 16:48:46 +08:00
co-authored by Claude Fable 5
parent 6258cf5d12
commit dfe3f0b118
10 changed files with 195 additions and 138 deletions
+15 -4
View File
@@ -72,6 +72,9 @@ services:
max-file: "10"
restart: unless-stopped
# No host port: architecture §2.2 — only Nginx is externally reachable.
# The previous ``8891:8000`` mapping (P0-1) was removed: the
# ``/internal/v1/*`` storage control plane is now guarded by a
# shared ``INTERNAL_SERVICE_TOKEN`` instead of network isolation.
# No local-FS volume: backend stores everything in S3 (S3_*).
environment:
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
@@ -81,6 +84,8 @@ services:
DEMO_AUTH_ENABLED: ${DEMO_AUTH_ENABLED:-false}
INITIAL_ADMIN_PASSWORD: ${INITIAL_ADMIN_PASSWORD:-admin12345}
RUNTIME_API_URL: http://runtime:8000
# P0-1 fix: shared secret required by /internal/v1/* routes.
INTERNAL_SERVICE_TOKEN: ${INTERNAL_SERVICE_TOKEN:?INTERNAL_SERVICE_TOKEN is required}
STORAGE_BACKEND: ${STORAGE_BACKEND:-s3}
LOCAL_STORAGE_BASE_DIR: ${LOCAL_STORAGE_BASE_DIR:-/data}
# S3_* only matter when STORAGE_BACKEND=s3. Defaults are kept so local
@@ -101,8 +106,6 @@ services:
condition: service_completed_successfully
runtime:
condition: service_healthy
ports:
- 8891:8000
volumes:
- ${PWD}:/app
- ./data:/data
@@ -125,8 +128,10 @@ services:
max-size: "200m"
max-file: "10"
restart: unless-stopped
ports:
- 8892:8000
# No host port: architecture §2.2 — only Nginx is externally reachable.
# The previous ``8892:8000`` mapping (P0-1) was removed: the runtime
# container is reachable only from the Docker internal network and
# Nginx-authenticated Jupyter paths.
cap_add:
- SYS_ADMIN
devices:
@@ -137,6 +142,10 @@ services:
environment:
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
SERVICE_NAME: runtime-manager
# P0-1 fix: runtime's /api/v1/jupyter is token-guarded. The same
# ``INTERNAL_SERVICE_TOKEN`` value backend uses for /internal/v1/*
# auth — see ``require_internal_service`` in runtime/main.py.
INTERNAL_SERVICE_TOKEN: ${INTERNAL_SERVICE_TOKEN:?INTERNAL_SERVICE_TOKEN is required}
STORAGE_BACKEND: ${STORAGE_BACKEND:-s3}
LOCAL_STORAGE_BASE_DIR: ${LOCAL_STORAGE_BASE_DIR:-/data}
# WORKSPACES_ROOT defaults to /data/workspace (settings.workspaces_root);
@@ -190,6 +199,8 @@ services:
SERVICE_NAME: schedule-executor
SCHEDULE_EVENT_NAMESPACE: ${SCHEDULE_EVENT_NAMESPACE:-model-platform-local}
BACKEND_API_URL: http://backend:8000
# P0-1 fix: must match the backend's INTERNAL_SERVICE_TOKEN exactly.
INTERNAL_SERVICE_TOKEN: ${INTERNAL_SERVICE_TOKEN:?INTERNAL_SERVICE_TOKEN is required}
STORAGE_BACKEND: ${STORAGE_BACKEND:-s3}
LOCAL_STORAGE_BASE_DIR: ${LOCAL_STORAGE_BASE_DIR:-/data}
S3_ENDPOINT: ${S3_ENDPOINT:-http://s3:9000}