tao.chen
309b657d35
All operator- and developer-facing docs updated to reflect:
- The unified AsyncStorageBackend abstraction (s3 + local backends).
- The STORAGE_BACKEND toggle ("s3" default, "local" for dev /
single-node / air-gapped deployments).
- The 4-purpose-bucket layout (workspace / version / run_log / trash)
in both modes — 4 separate S3 buckets in s3 mode, 4 subdirectories
of LOCAL_STORAGE_BASE_DIR in local mode.
- The S3_* env var naming (was RUSTFS_*).
- The server-proxied upload flow (was browser-direct presign-PUT):
POST /internal/v1/uploads → PUT /internal/v1/uploads/{id} with
raw bytes → server calls backend.put().
- The factory helpers workspaces_root() (runtime's view of the
workspace bucket on disk) and rclone_remote_spec() (s3-mode mount
source).
- The "two settings describing the same thing" cleanup: the deleted
settings.workspace_root, settings.workspaces_root, and
settings.remote_bucket fields.
Files touched:
- API.md (§5 data-resource upload flow, §9 storage control plane,
§10 readiness example)
- ARCHITECTURE.md (storage layer diagram)
- CLAUDE.md (architecture description + volume-preservation note)
- DEVELOP.md (settings list, Storage section, "Wire a new bucket"
how-to, dev-export example, troubleshooting network hint)
- README.md (architecture diagram, container table, quick-start
credentials note, tear-down note, Storage layout section)
- REFACTOR_NOTES.md (final container list with s3 explanation)
- backend/README.md (storage backend description)
- migrations/data/README.md (step 11/12 record mentioning object
storage)
A handful of historical "RustFS" mentions are intentionally retained
where they name a specific S3-compatible product (e.g. as an example
in REFACTOR_NOTES.md's container list) or document the pre-2026
abstraction name (DEVELOP.md Storage section).
40 lines
1.3 KiB
Markdown
40 lines
1.3 KiB
Markdown
# 简化系统架构
|
||
|
||
```text
|
||
Browser
|
||
|
|
||
v
|
||
Nginx Gateway (静态 React Router SPA + /api + /jupyter 代理)
|
||
|-----------------------------|
|
||
| /api/v1 | /jupyter/
|
||
v v
|
||
FastAPI Backend Shared Jupyter Server
|
||
| ^
|
||
| Runtime HTTP | Runtime 管理会话/票据
|
||
v |
|
||
Runtime Manager ---------------|
|
||
|
|
||
+------ MySQL（编辑租约、运行实例）
|
||
|
||
FastAPI Backend
|
||
| 1. 写 schedule_runs + outbox_events
|
||
| 2. 尝试 HTTP 立即推送
|
||
v
|
||
Schedule Executor（APScheduler）
|
||
|-- MySQL APSchedulerJobStore
|
||
|-- MySQL Outbox 轮询兜底
|
||
|-- DAG 节点执行与重试
|
||
|-- S3 日志/结果
|
||
+-- Backend 内部 Storage API
|
||
```
|
||
|
||
## 关键简化
|
||
|
||
1. 删除 Redis 服务、Redis Streams 和 Redis 文件锁。
|
||
2. 调度定义、运行记录、Outbox、Inbox、Cron JobStore 都由 MySQL 保存。
|
||
3. 立即运行采用 Backend -> Schedule Executor 内部 HTTP 推送；推送失败由 MySQL Outbox 轮询兜底。
|
||
4. Schedule Executor 自带 APScheduler，负责 Cron 触发和 DAG 执行。
|
||
5. 文件编辑锁改为 MySQL 租约，Runtime 单副本运行。
|
||
6. Jupyter 使用一个共享容器，工作区目录通过 Volume 挂载同步。
|
||
7. 前端改为 React Router SPA，并按 feature / route / service / component 分层。
|