Model Platform
A self-hosted Jupyter-based model development platform that combines an interactive workspace, a DAG scheduler, an object-storage-backed artifact store, and per-workspace runtime isolation — all behind a single Nginx gateway.
Stack: React Router SPA · FastAPI · APScheduler · MySQL · S3-compatible storage (or local filesystem via
STORAGE_BACKEND=local) · shared Jupyter · FUSE mount via rclone (s3 mode only) Single ingress (Nginx :80); all other services are Docker-internal.
What it does
| Capability | Where |
|---|---|
| Workspace-scoped notebook editing with row-level lock | backend/jupyter.py + scripts.is_locked |
| Authenticated Jupyter routing (browser never sees the runtime token) | nginx/default.conf + auth_request + backend/jupyter.py |
| Object storage for notebooks / scripts / versions / run logs (s3 / local toggle) | common/storage/ + backend/scripts.py |
| DAG-style scheduling: nodes, edges, cron, manual trigger, retries, snapshots | backend/schedules.py + backend/schedule_runs.py + schedule/ (5 modules) |
| DAG execution via MySQL Outbox (no Redis, no in-process queues) | schedule/orchestrator.py + schedule/worker.py |
| Per-workspace Jupyter sub-process pool with asyncio locks | runtime/process.py |
| rclone FUSE mount of the workspace bucket into the runtime (s3 mode) | runtime/mount.py |
| MySQL-only persistence (26 tables, soft-delete, no foreign keys) | common/db/models/ |
Architecture at a glance
┌────────────────────┐
│ Browser (SPA) │
└─────────┬──────────┘
│ HTTPS / WS
┌─────────▼──────────┐
│ Nginx (only :80) │ ← templates/default.conf
│ /api/ /jupyter/ /storage/
└────┬───────┬──────┘
│ │
┌──────────────┘ └─────────────┐
▼ ▼
┌──────────────────┐ ┌──────────────────────┐
│ FastAPI Backend │ │ Runtime (Jupyter) │
│ + /internal/v1 │ control │ - rclone FUSE mount │
│ (storage) ├──────────────►│ - subprocess pool │
│ - DAG CRUD │ │ (per workspace) │
│ - script CRUD │ └──────────┬───────────┘
│ - auth_request │ │ FUSE / shared vol
│ - /api/v1/... │ ▼
└────┬──────┬──────┘ ┌──────────────────────┐
│ │ │ Object storage │
│ └──────── HTTP ───────►│ (s3: S3 service / │
▼ │ local: shared vol) │
┌────────────┐ │ 4 buckets per usage │
│ MySQL │◄───────── poll ─────│ │
│ - 26 tbls │ └──────────────────────┘
│ - outbox │
│ - jobstore │
└────┬───────┘
▲
│ outbox poll
┌────┴──────────────────────────┐
│ Schedule Executor │
│ - CronScheduler (APScheduler) │
│ - DispatchOrchestrator │
│ - NodeExecutor (worker) │
│ - SchedulerService (facade) │
└───────────────────────────────┘
Object storage is selectable via STORAGE_BACKEND (s3 | local). In s3 mode
the 4 purpose-named buckets (workspaces / versions / run-logs / trash)
are S3 buckets; in local mode they're subdirectories of LOCAL_STORAGE_BASE_DIR,
shared via the local-storage Docker volume. See DEVELOP.md §Storage.
Detailed design lives in ARCHITECTURE.md. Implementation deviations and
recent refactors are recorded in HANDOVER.md.
Repository layout
frontend/ React Router SPA
backend/ FastAPI: public API + internal storage API
runtime/ Jupyter subprocess manager + rclone FUSE
schedule/ DAG scheduler (5 modules: context/scheduler/
orchestrator/worker/service)
common/ Settings, SQLAlchemy models, storage SDK,
outbox events, jobstore
migrations/ Alembic baseline + per-feature revisions
nginx/ (concept only — see "Container" below)
scripts/ nginx-entrypoint.sh (template renderer)
docker-compose.yml 4 services — web / backend / runtime / schedule
default.conf Nginx template (mounted, rendered at start)
.env.example All env vars consumed by common.config.Settings
Containers
| Service | Image | Exposed | Purpose |
|---|---|---|---|
web |
nginx:alpine |
host :8888 → :80 |
SPA, /api/ reverse-proxy, /jupyter/{ws}/ auth_request proxy, /storage/ S3 passthrough (s3 mode only) |
backend |
Dockerfile |
internal only | DAG CRUD, script CRUD, schedule triggers, /api/v1/auth/jupyter, /internal/v1/* storage control plane |
runtime |
Dockerfile |
internal only | Per-workspace Jupyter sub-process pool, rclone FUSE mount of workspaces bucket (s3 mode) |
schedule |
Dockerfile |
internal only | Cron tick + DAG execution via MySQL Outbox polling |
The architecture deliberately has only one host port (the gateway);
all other services are on the Docker internal network. This is enforced in
docker-compose.yml — no ports: on backend / runtime / schedule.
Quick start
cp .env.example .env
# Edit .env — at minimum change MYSQL password and (in s3 mode) S3 credentials.
# Static check
uv sync --all-packages
uv run --frozen --package backend python -m compileall -q backend/src common/src
uv run --frozen --package schedule python -m compileall -q schedule/src
uv run --frozen --package runtime python -m compileall -q runtime/src
# Apply schema
uv run --frozen --package backend alembic upgrade head
# Bring up the stack
docker compose config # validate
docker compose up -d --build
docker compose ps
Visit http://localhost:8888.
Logs
docker compose logs -f backend
docker compose logs -f schedule
docker compose logs -f runtime
Tear down (keeps MySQL + S3 / local-storage volumes)
docker compose down
Wipe data
docker compose down -v
Configuration
All environment variables are declared once in common/src/common/config.py
as a pydantic-settings Settings class, with a @lru_cache singleton.
Adding a new env var:
- Add the field to
Settingsincommon/src/common/config.py(with a sensible default so dev-env "just works"). - Add the line to
.env.examplewith a comment. - Use
settings.<name>at the call site. Neveros.environ["..."].
See DEVELOP.md for the full list of variables and their meanings.
Storage layout
Four purpose-named buckets. The mapping from StorageObjects.usage_type
to bucket is decided in one place (backend/storage_api.py:resolve_bucket):
usage_type |
Bucket (env var) | Default name |
|---|---|---|
working_copy, public_script, data_resource, snapshot |
S3_WORKSPACE_BUCKET |
workspaces |
version_artifact |
S3_VERSION_BUCKET |
versions |
run_log, run_result |
S3_RUN_LOG_BUCKET |
run-logs |
| (soft-delete target) | S3_TRASH_BUCKET |
trash |
In STORAGE_BACKEND=s3 mode these are 4 separate S3 buckets. In
STORAGE_BACKEND=local mode they are 4 subdirectories under
LOCAL_STORAGE_BASE_DIR (default /data), so the layout above
becomes:
/data/
├── workspace/ # S3_WORKSPACE_BUCKET
├── version/ # S3_VERSION_BUCKET
├── run_log/ # S3_RUN_LOG_BUCKET
└── trash/ # S3_TRASH_BUCKET
A workspace's artifact_bucket column (when non-null) overrides the
default for that workspace, regardless of usage_type — useful for
isolating a paying customer to their own bucket.
The object key is a flat two-level path — workspace_id and a server-
issued ulid for the object:
<workspace_bucket>/<workspace_id>/<ulid>{.<ext>}
The file name, extension, content type, and logical path live in the
StorageObjects and Scripts rows, not in the object key, so the
storage can be re-organised without a database rewrite.
Backend code never writes to the container's local filesystem (except
in STORAGE_BACKEND=local mode, where the shared local-storage volume
is the canonical store). Schedule Executor stages node artifacts in
tempfile.TemporaryDirectory() (auto-cleaned). Only the runtime
container keeps a host volume — required by the rclone FUSE mount in
s3 mode, and a no-op pass-through in local mode.
Documentation
README.md(this file) — quick orientationARCHITECTURE.md— design diagrams + simplification historyHANDOVER.md— implementation deviations, recent refactors, pending workDEVELOP.md— developer guide (env vars, code conventions, common tasks)CLAUDE.md— agent-facing conventions for the repo
License
Internal.