tao.chen
baee7a60e1
- Drop ipykernel from the root virtual dependency list; the workspace relies on its subpackages and uv lockfile to manage per-service dependencies. - Refresh uv.lock to match the trimmed root dependency set. - Add CLAUDE.md describing the workspace layout, common development and database commands, the Nginx/backend/runtime auth request flow, and the boundaries to keep in mind when changing services.
79 lines
6.3 KiB
Markdown
79 lines
6.3 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Repository overview
|
||
|
||
This is a Python 3.12 `uv` workspace for a small Jupyter workspace platform. The root `pyproject.toml` includes four workspace packages:
|
||
|
||
- `backend`: FastAPI service used by Nginx `auth_request`. It reads the original workspace/URI headers, performs the current authentication and notebook-lock checks, asks Runtime to find or start a workspace, and returns the selected Jupyter upstream and internal token in response headers for Nginx.
|
||
- `runtime`: FastAPI process manager. At startup it mounts `REMOTE_BUCKET` with `rclone`, scans `WORKSPACES_ROOT`, and auto-starts a Jupyter Notebook process for each workspace directory. It keeps process metadata (PID, port, token, URL, start time) in the in-memory `JUPYTER_PROCESSES` dictionary and exposes health and start/stop/list/get actions.
|
||
- `common`: Shared package scaffold for SQLAlchemy/Alembic database code and object-storage helpers. `common/src/common/migrations/env.py` uses `common.db.base.Base.metadata`; the model/session/storage layers are currently mostly placeholders.
|
||
- `schedule`: Package scaffold only; `main.py` and `executor.py` are currently empty apart from module headers.
|
||
|
||
The request path in the deployed stack is **client → Nginx (`web`) → backend auth subrequest → runtime → per-workspace Jupyter**. `default.conf` also proxies `/storage/` to RustFS and configures WebSocket forwarding for Jupyter. The frontend is not implemented in this repository; `frontend/README.md` only identifies it as the frontend.
|
||
|
||
## Common commands
|
||
|
||
Run these from the repository root. Dependencies are managed by the committed `uv.lock` file.
|
||
|
||
```bash
|
||
# Install/synchronize all workspace dependencies
|
||
uv sync
|
||
|
||
# Run the services locally
|
||
make runtime # Runtime on 0.0.0.0:8001
|
||
make runtime-dev # Runtime with Uvicorn reload
|
||
make backend # Backend on 0.0.0.0:8000
|
||
make backend-dev # Backend with Uvicorn reload
|
||
|
||
# Equivalent direct commands
|
||
uv run --package runtime uvicorn runtime.main:app --host 0.0.0.0 --port 8001
|
||
uv run --package backend uvicorn backend.main:app --host 0.0.0.0 --port 8000
|
||
|
||
# Run the containerized stack (Nginx is exposed on localhost:8888)
|
||
docker compose up --build
|
||
docker compose down
|
||
```
|
||
|
||
`Makefile` variables can be overridden on the command line, for example:
|
||
|
||
```bash
|
||
make runtime-dev WORKSPACES_ROOT=./test/workspaces RUNTIME_PORT=8001
|
||
make backend-dev RUNTIME_BASE_URL=http://127.0.0.1:8001
|
||
```
|
||
|
||
The local Runtime defaults to `WORKSPACES_ROOT=/app/workspaces` in code, while the Makefile overrides it to `./test/workspaces`; make sure the directory exists and contains workspace directories before expecting auto-start behavior. Runtime also needs `rclone`, FUSE support, and a valid remote configuration when exercising the mount path. The Docker image provides `rclone` and `fuse3`; a local process does not.
|
||
|
||
### Tests, linting, and formatting
|
||
|
||
There is currently no test suite, test configuration, or lint/format command in the repository. Do not assume `make test`, `pytest`, Ruff, or Black is configured. If tests are added, run a single test with the project’s chosen runner (the conventional pytest form is `uv run pytest path/to/test_file.py::test_name`), then add the corresponding dependency and documented command rather than silently relying on a globally installed tool.
|
||
|
||
A lightweight current smoke check is to compile the Python sources:
|
||
|
||
```bash
|
||
python -m compileall backend/src common/src runtime/src schedule/src
|
||
```
|
||
|
||
### Database migrations
|
||
|
||
Alembic is configured in `common/alembic.ini`, with scripts under `common/src/common/migrations`. The checked-in `sqlalchemy.url` is the generated placeholder `driver://user:pass@localhost/dbname`, so replace/configure it for a real database before running migrations. From the root, the usual commands are:
|
||
|
||
```bash
|
||
uv run alembic -c common/alembic.ini current
|
||
uv run alembic -c common/alembic.ini upgrade head
|
||
uv run alembic -c common/alembic.ini revision --autogenerate -m "describe change"
|
||
```
|
||
|
||
## Important implementation details
|
||
|
||
- `backend/src/backend/main.py` is the ASGI entrypoint (`backend.main:app`). Nginx sends `/internal-auth` to `GET /api/v1/auth/jupyter`; the backend expects `X-Original-Workspace-Id` and `X-Original-URI`, chooses a bearer token or `access_token` cookie, and returns `x-upstream-addr` plus `x-jupyter-internal-token` headers. The JWT validation and database lock lookup are currently mocked/commented, so treat this as demo behavior rather than complete production authentication.
|
||
- `runtime/src/runtime/main.py` is the ASGI entrypoint (`runtime.main:app`). Startup/shutdown is implemented through FastAPI lifespan: mount/scan/start on startup, stop Jupyter processes and unmount on shutdown. The API is `GET /api/v1/health` and `POST /api/v1/jupyter` with `action` values `start`, `stop`, `list`, or `get` (workspace ID is required for all except `list`).
|
||
- Runtime workspace IDs are directory names below `WORKSPACES_ROOT`. A started Jupyter server receives a dynamic port, a generated token, and base URL `/jupyter/<workspace_id>/`. Because process state is only in memory, restarting Runtime loses the registry and causes startup scanning to rediscover workspaces.
|
||
- `common` is a uv workspace dependency declared by backend/runtime/schedule, but the current service code does not yet contain substantial shared model or storage integration. Keep shared DB/storage behavior in `common` rather than duplicating it in services.
|
||
- `docker-compose.yml` builds backend and runtime from the root workspace, exposes backend on `8000`, runtime on `8001`, and Nginx on `8888`. Runtime requires Linux FUSE capabilities (`SYS_ADMIN`, `/dev/fuse`, and unconfined AppArmor). Its RustFS/rclone settings are supplied as compose environment variables; use deployment secrets or environment overrides instead of committing credentials.
|
||
|
||
## Change boundaries
|
||
|
||
Keep service entrypoints and the Nginx header contract compatible when changing backend/runtime behavior. If an API action or response field changes, update both the caller in `backend` and the proxy rules in `default.conf` together. For runtime process changes, preserve cleanup in the lifespan shutdown path and consider failures from missing mounts, dead subprocesses, dynamic port allocation, and remote storage separately.
|