tao.chenandClaude Fable 5
dfe3f0b118
The fix lands in three concentric layers, all backed by a single
INTERNAL_SERVICE_TOKEN shared secret so we have one mechanism
instead of three:
1. docker-compose: drop the backend.ports: 8891:8000 and
runtime.ports: 8892:8000 mappings. Nginx is the only host
ingress again (architecture §2.2).
2. /internal/v1/*: the storage control plane had six endpoints, five
of which were dead code (frontend already migrated to
/api/v1/data-resources/* with JWT; schedule only ever called
POST /internal/v1/objects). Delete the dead routes, mount the
one survivor with Depends(require_internal_service) that
compares the X-Internal-Service-Token header against
settings.internal_service_token with secrets.compare_digest.
3. POST /api/v1/jupyter on the runtime container: previously open
inside the Docker network. Same token mechanism — backend's
runtime_http_client now carries the header, runtime's
handle_jupyter_action requires the same header. /api/v1/health
stays open for the Nginx and compose healthchecks.
The schedule worker was already configured to call
POST /internal/v1/objects; build_storage_http_client now
sets the token header so its existing call site keeps working
without changes.
Files touched:
backend/src/backend/storage_api.py # 5 dead routes deleted + token guard
backend/src/backend/main.py # runtime_http_client header
runtime/src/runtime/main.py # require_internal_service Depends
common/src/common/config.py # internal_service_token setting
schedule/src/schedule/service.py # httpx client header
docker-compose.yml # ports dropped, INTERNAL_SERVICE_TOKEN env
.env.example # INTERNAL_SERVICE_TOKEN placeholder
API.md / README.md / DEVELOP.md # §9 trimmed to 1 endpoint
Verified:
compileall -> 0 errors
pytest backend/tests -> 37 passed
in-process ASGI smoke:
POST /internal/v1/objects no/wrong/correct token -> 401/401/200
POST /api/v1/jupyter no/wrong/correct token -> 401/401/200
5 deleted internal routes -> 404
docker compose config (with env) -> OK
P0-1 still has one open sub-item (rclone RC --rc-no-auth) that
the user has explicitly deferred; not touched here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
86 lines
3.9 KiB
Bash
86 lines
3.9 KiB
Bash
COMPOSE_PROJECT_NAME=model-platform-develop
|
|
SCHEDULE_EVENT_NAMESPACE=model-platform-develop
|
|
|
|
# External port of the Nginx gateway. Only Nginx is exposed to the host
|
|
# (architecture §2.2); backend/runtime/schedule stay on the Docker internal
|
|
# network. Override here to expose Nginx on a different host port.
|
|
GATEWAY_PORT=8890
|
|
|
|
# External MySQL. URL-encode reserved characters in DATABASE_URL.
|
|
MYSQL_HOST=127.0.0.1
|
|
MYSQL_PORT=3306
|
|
MYSQL_USER=root
|
|
MYSQL_PASSWORD=change-me
|
|
MYSQL_DATABASE=model_platform
|
|
DATABASE_URL=mysql+asyncmy://root:change-me@127.0.0.1:3306/model_platform?charset=utf8mb4
|
|
|
|
# Demo login is only intended for this self-hosted development UI.
|
|
DEMO_AUTH_ENABLED=true
|
|
JWT_SECRET=change-this-development-secret
|
|
|
|
# ============================================================================
|
|
# CRITICAL: must set BEFORE first run. The initial admin user is seeded by
|
|
# the deployment bootstrap. Never keep the development default in production.
|
|
# ============================================================================
|
|
INITIAL_ADMIN_PASSWORD=admin12345
|
|
|
|
# Backend loguru stderr sink level. One of DEBUG / INFO / WARNING / ERROR
|
|
# / CRITICAL. Anything else (e.g. lowercase) falls back to INFO inside
|
|
# configure_logging(). Change to DEBUG to see request bodies in
|
|
# runtime_client._jupyter_request.
|
|
LOG_LEVEL=INFO
|
|
|
|
# Object storage. Two modes are supported:
|
|
# STORAGE_BACKEND=s3 — connects to an S3-compatible service (MinIO,
|
|
# RustFS, SeaweedFS, AWS S3, …). Requires the
|
|
# S3_* block below.
|
|
# STORAGE_BACKEND=local — stores objects on the local filesystem under
|
|
# LOCAL_STORAGE_BASE_DIR. Backend and runtime
|
|
# share this directory via a Docker volume
|
|
# (docker-compose.yml mounts `local-storage`).
|
|
# Useful for dev, single-node, air-gapped.
|
|
STORAGE_BACKEND=s3
|
|
LOCAL_STORAGE_BASE_DIR=/data
|
|
|
|
# Object storage (S3-compatible). Only used when STORAGE_BACKEND=s3.
|
|
# S3_ENDPOINT is the single upstream URL consumed by all 4 services:
|
|
# - nginx (via scripts/nginx-entrypoint.sh, which parses host + port)
|
|
# - backend / runtime / schedule (passed through to boto3 / rclone)
|
|
# S3_ACCESS_KEY / S3_SECRET_KEY are read by Python code in
|
|
# backend/ and schedule/ (boto3 credentials).
|
|
#
|
|
# S3 buckets are purpose-named:
|
|
# S3_WORKSPACE_BUCKET — workspace files (notebooks, scripts, working
|
|
# copies); layout is ``s3://<bucket>/<workspace_id>/...``.
|
|
# S3_VERSION_BUCKET — immutable script-version artifacts.
|
|
# S3_RUN_LOG_BUCKET — schedule run logs and execution results.
|
|
# S3_TRASH_BUCKET — soft-deleted objects; source bucket key is preserved
|
|
# as a prefix so restore is a same-key move.
|
|
S3_HOST=127.0.0.1
|
|
S3_PORT=9000
|
|
S3_ENDPOINT=http://127.0.0.1:9000
|
|
S3_ACCESS_KEY=change-me
|
|
S3_SECRET_KEY=change-me
|
|
S3_WORKSPACE_BUCKET=workspace
|
|
S3_VERSION_BUCKET=version
|
|
S3_RUN_LOG_BUCKET=run-log
|
|
S3_TRASH_BUCKET=trash
|
|
S3_TRASH_RETENTION_DAYS=30
|
|
|
|
# rclone RC (HTTP control API). The runtime container starts rclone with
|
|
# `--rc --rc-addr 0.0.0.0:5572 --rc-no-auth` (see runtime/src/runtime/mount.py),
|
|
# so the backend can POST /vfs/refresh here to invalidate the FUSE dir-cache
|
|
# after writing new workspace files. Default points at the runtime service
|
|
# over the compose network.
|
|
RCLONE_RC_URL=http://runtime:5572
|
|
|
|
# ============================================================================
|
|
# Service-to-service auth (P0-1 fix).
|
|
# Backend's /internal/v1/* storage control plane requires this shared secret.
|
|
# The schedule worker reads the same value and sends it as the
|
|
# ``X-Internal-Service-Token`` header. Value MUST match between backend and
|
|
# schedule. Generate a random 64-char string for any non-dev deployment:
|
|
# python -c "import secrets; print(secrets.token_urlsafe(48))"
|
|
# ============================================================================
|
|
INTERNAL_SERVICE_TOKEN=change-me-internal-service-token
|