feat(scripts): 跨 owner 懒加载目录树 + 跨用户可见 workspace/public

修两个后端接口问题:
1) /api/v1/workspace-directories 返回为空,目录树结构消失
2) 同 workspace 内脚本/数据互相可见但默认排除 private

后端改动
--------
* list_scripts / list_resources / list_workspace_directories 新增
  owner_user_id 可选 query 参数;缺省 = 当前请求者本人(scope 到
  workspace/{me}/...),传值时 scope 到该 owner 的子树。前端根加载
  默认只见自己一级,其他成员以折叠分组呈现。
* visibility 过滤统一:非 admin 请求者只返回 owner==me 或
  visibility ∈ {workspace, public};admin 跳过。owner=me 含自己
  的 private,owner=other 只剩其 workspace/public,排除他人 private。
* create_workspace_directory 两个分支 visibility 默认 'public'
  (非 private),使跨 owner 目录树可见;响应新增 owner_user_id 字段。
* platform.list_members 鉴权从 system_admin_context 放宽为
  系统管理员或该 workspace 活跃成员(让普通用户也能渲染同
  workspace 成员名册,用于跨 owner 分组)。
* main.py 注册 platform 模块(随 list_members 改动补齐导入)。
* .env.example 同步 common/config.py 26 个字段。

前端改动
--------
* ScriptExplorer.memberScriptGroups 改由 members 列表播种分组,
  display_name 取 members.display_name;inferredDirectories 现在按
  owner_user_id 标记,统一跨 owner 目录渲染。删除脚本目录页头与
  树分组标题的工作副本数量角标。
* WorkspaceTree 新增 ownerUserId 透传到 store.toggleExpanded;
  仅"我"的分组 mount 时 auto-expand,他人分组默认折叠,展开才
  调 loadOwnerGroup / owner-scoped loadScripts / loadChildren。
* scriptWorkspaceStore 引入 namespaced cache key
  (ownerCacheKey = `${ownerUserId ?? me}:${path}`),loadedScriptPaths
  / loadedChildPaths / loadedOwnerGroups 全部按 owner 隔离;
  toggleExpanded 用 loadPath === undefined 区分 group 头与真实
  目录,修"他人子目录点击不触发接口"的 loadPath 前缀误判 bug。
* api.ts / AuthContext 透传 ownerUserId 给 listScripts /
  listResources / listWorkspaceDirectories。

文档
----
* API.md: §3.2 创建目录 visibility 默认 public + 响应加 owner_user_id;
  §3.3.1 GET directories 加 owner_user_id 参数 + 响应字段;
  §3.4 GET scripts 改写为 owner 作用域 + visibility 过滤语义;
  §五.1 GET data-resources 新增,同一套统一语义;
  §7 intro 例外 — GET members 对系统管理员或 workspace 活跃成员开放。
* DEVELOP.md: Code layout 重写以反映 backend api/services/clients/
  schemas 拆分 + schedule domain/scheduling/application/execution/
  infrastructure 拆分 + common 子包(auth/storage/backends);
  Configuration 系统补全 26 个 settings 字段;新增
  "Owner-scoping + visibility (cross-owner browsing)" 小节;
  Per-service dev 注释用 uv run 的源布局要求;Add a new DAG endpoint /
  storage bucket 路径改为 backend/src/backend/api/* 与 services/*。

测试
----
* test_list_scripts_parent_path.py /
  test_resources.py 补充 owner_user_id 参数化直接调用 + LIKE
  前缀断言(workspace/{owner}/... 前缀)。

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
tao.chen
2026-08-21 19:26:53 +08:00
co-authored by Claude
parent 8cf4b53dd4
commit b493907775
15 changed files with 929 additions and 303 deletions
+28
View File
@@ -16,6 +16,15 @@ DATABASE_URL=mysql+asyncmy://root:change-me@127.0.0.1:3306/model_platform?charse
JWT_SECRET=change-this-development-secret JWT_SECRET=change-this-development-secret
# Force the Secure flag on the session cookie even when the inbound request
# scheme is plain HTTP. Enable behind a TLS-terminating reverse proxy that
# strips/rewrites X-Forwarded-Proto — otherwise the cookie is written without
# Secure and browsers refuse to send it back over HTTPS.
COOKIE_FORCE_SECURE=false
# Service label surfaced in lifespan / health checks.
SERVICE_NAME=service
# ============================================================================ # ============================================================================
# CRITICAL: must set BEFORE first run. The initial admin user is seeded by # CRITICAL: must set BEFORE first run. The initial admin user is seeded by
# the deployment bootstrap. Never keep the development default in production. # the deployment bootstrap. Never keep the development default in production.
@@ -84,6 +93,13 @@ S3_TRASH_RETENTION_DAYS=30
# over the compose network. # over the compose network.
RCLONE_RC_URL=http://runtime:5572 RCLONE_RC_URL=http://runtime:5572
# Backend → Runtime HTTP endpoint (Jupyter contents API, file ops).
RUNTIME_API_URL=http://runtime:8000
# Public base URL for the runtime container (surfaced to clients for
# Jupyter access tickets / embedded URLs).
PUBLIC_BASE_URL=http://runtime
# ============================================================================ # ============================================================================
# Service-to-service auth (P0-1 fix). # Service-to-service auth (P0-1 fix).
# Backend's /internal/v1/* storage control plane requires this shared secret. # Backend's /internal/v1/* storage control plane requires this shared secret.
@@ -93,3 +109,15 @@ RCLONE_RC_URL=http://runtime:5572
# python -c "import secrets; print(secrets.token_urlsafe(48))" # python -c "import secrets; print(secrets.token_urlsafe(48))"
# ============================================================================ # ============================================================================
INTERNAL_SERVICE_TOKEN=change-me-internal-service-token INTERNAL_SERVICE_TOKEN=change-me-internal-service-token
# Schedule → Backend HTTP base URL (cron post-back / status callbacks).
BACKEND_API_URL=http://backend:8000
# Max concurrent notebooks running in the schedule worker. Each notebook is
# dispatched as an asyncio task bounded by a semaphore; the polling loop is
# never blocked.
SCHEDULE_EXECUTION_CONCURRENCY=4
# Readiness probe targets. Comma-separated host:port list checked by
# /health/ready; empty disables the check. e.g. mysql:3306,s3:9000
READINESS_TARGETS=
+53 -10
View File
@@ -99,7 +99,12 @@
### 3.2 `POST /api/v1/workspace-directories` ### 3.2 `POST /api/v1/workspace-directories`
创建一个**目录**。后端会落一行 `StorageObjects(object_type='directory', storage_backend='rustfs', storage_uri='inline://directory/{path}', visibility='private')`,因此空目录也能在 §3.1 树里出现并保留下来。 创建一个**目录**。后端会落一行 `StorageObjects(object_type='directory', storage_backend='rustfs', storage_uri='inline://directory/{path}', visibility='public')`,因此空目录也能在 §3.1 树里出现并保留下来。
> 目录行默认 `visibility='public'`(非 `private`)。目录是结构性导航行,
> 默认 public 使同一 workspace 内其他成员可以浏览彼此的目录结构(目录树
> 跨 owner 可见);文件级私密仍由 §3.4 / §五 的 visibility 过滤兜底
> —— 其他 owner 的 `private` 脚本 / 数据资源不会返回。
- **请求体**: - **请求体**:
```json ```json
@@ -123,7 +128,8 @@
"storage_object_id": "01HXY...", "storage_object_id": "01HXY...",
"path": "scripts/etl", "path": "scripts/etl",
"name": "etl", "name": "etl",
"parent_path": "scripts" "parent_path": "scripts",
"owner_user_id": "01HXX..."
} }
} }
``` ```
@@ -155,10 +161,12 @@
- **行为**: `parent_path` 为空字符串或缺省 → 用户根目录;非空 → 该父目录的直接子目录(workspace-relative)。仅返回 `object_status='available' AND is_deleted=0` 的 `StorageObjects` 行。 - **行为**: `parent_path` 为空字符串或缺省 → 用户根目录;非空 → 该父目录的直接子目录(workspace-relative)。仅返回 `object_status='available' AND is_deleted=0` 的 `StorageObjects` 行。
- **鉴权**: workspace 成员 - **鉴权**: workspace 成员
- **owner 作用域**: `owner_user_id` 缺省时 scope 为**当前请求者**本人根目录(`scoped_prefix = workspace/{me}`);传 `owner_user_id` 时 scope 为该 owner 的根目录(`scoped_prefix = workspace/{owner_user_id}`),用于跨 owner 浏览目录树(见 §3.4 visibility 模型)。该接口本身不施加 visibility 过滤——目录行默认 `visibility='public'`(见 §3.2),跨 owner 均可见。
- **查询参数**: - **查询参数**:
| 名 | 类型 | 必填 | 说明 | | 名 | 类型 | 必填 | 说明 |
|---|---|---|---| |---|---|---|---|
| `parent_path` | string | 否 | 父目录相对路径,空字符串或缺省表示用户根目录 | | `parent_path` | string | 否 | 父目录相对路径,空字符串或缺省表示用户根目录 |
| `owner_user_id` | string | 否 | 目标 owner 的 user_id;缺省=请求者本人。指定后 scope 到 `workspace/{owner_user_id}/{parent_path}` |
- **谓词(SQL 等价)**: `relative_path LIKE '<prefix>/%' AND relative_path NOT LIKE '<prefix>/%/%'`,其中 `prefix = scoped_prefix/{parent_path}`,索引走 `idx_storage_workspace_relative_path(workspace_id, relative_path(255))`。 - **谓词(SQL 等价)**: `relative_path LIKE '<prefix>/%' AND relative_path NOT LIKE '<prefix>/%/%'`,其中 `prefix = scoped_prefix/{parent_path}`,索引走 `idx_storage_workspace_relative_path(workspace_id, relative_path(255))`。
- **响应**: - **响应**:
```json ```json
@@ -166,9 +174,9 @@
"request_id": "...", "request_id": "...",
"data": { "data": {
"directories": [ "directories": [
{"path": "scripts", "name": "scripts", "parent_path": "", "has_children": true}, {"path": "scripts", "name": "scripts", "parent_path": "", "owner_user_id": "01HXX...", "has_children": true},
{"path": "scripts/etl", "name": "etl", "parent_path": "scripts", "has_children": false}, {"path": "scripts/etl", "name": "etl", "parent_path": "scripts", "owner_user_id": "01HXX...", "has_children": false},
{"path": "scripts/etl/daily", "name": "daily", "parent_path": "scripts/etl", "has_children": false} {"path": "scripts/etl/daily", "name": "daily", "parent_path": "scripts/etl", "owner_user_id": "01HXX...", "has_children": false}
] ]
}, },
"meta": {"directory_count": 3} "meta": {"directory_count": 3}
@@ -180,15 +188,28 @@
| `path` | string | workspace 内相对路径 | | `path` | string | workspace 内相对路径 |
| `name` | string | `path` 的最后一段 | | `name` | string | `path` 的最后一段 |
| `parent_path` | string | 父目录相对路径,根目录用空串 | | `parent_path` | string | 父目录相对路径,根目录用空串 |
| `owner_user_id` | string | 该目录行所属 owner 的 user_id(`owner_user_id` 参数缺省时=请求者本人) |
| `has_children` | bool | 该目录下是否还有直接子目录(后端额外 `EXISTS` 查询,可为空目录为 `false`) | | `has_children` | bool | 该目录下是否还有直接子目录(后端额外 `EXISTS` 查询,可为空目录为 `false`) |
- **空结果**: 不返回 404,空目录列表即 `directories: []`。 - **空结果**: 不返回 404,空目录列表即 `directories: []`。
- **错误**: 401(未登录)/ 403(非 workspace 成员)同其他接口。 - **错误**: 401(未登录)/ 403(非 workspace 成员)同其他接口。
### 3.4 `GET /api/v1/scripts` ### 3.4 `GET /api/v1/scripts`
列出当前 workspace 内**全部 active 脚本**。不受 is_locked 影响(读路径不锁)。 列出脚本,按 **owner 作用域 + visibility 过滤**返回。不受 is_locked 影响(读路径不锁)。
- **响应**: `data` 为 `ScriptPayload` 数组(见 §3.10) - **owner 作用域**: `owner_user_id` 缺省=当前请求者本人,scope 到 `workspace/{me}/...`;传 `owner_user_id` 时 scope 到 `workspace/{owner_user_id}/...`,用于跨 owner 浏览他人脚本
- **visibility 过滤(统一)**: 非 admin 请求者只返回 `owner_user_id == me OR visibility IN (workspace, public)`;admin 请求者跳过过滤返回全部。
- **owner=me(缺省)**: scope 是我的子树,行都是我的 → `owner==me` 恒成立 → **含我的 private 脚本** ✓
- **owner=other**: scope 是他人的子树,`owner==me` 不成立 → 只剩其 `workspace/public` 脚本(排除他人的 `private`) ✓
- 即"本人可见自己全部;他人只见其 workspace/public",私密仅在 owner==me 时可见。
- **非递归**: 仅返回 `parent_path` 下的**直接子级**脚本(懒加载用);子目录脚本需带 `parent_path` 再次请求。
- **查询参数**:
| 名 | 类型 | 必填 | 说明 |
|---|---|---|---|
| `parent_path` | string | 否 | 父目录相对路径,空串/缺省=该 owner 根目录下的一级脚本 |
| `owner_user_id` | string | 否 | 目标 owner;缺省=请求者本人 |
| `keyword` | string | 否 | 名称模糊匹配 |
- **响应**: `data` 为 `ScriptPayload` 数组(见 §3.10),每条带 `owner_user_id`。
### 3.5 `GET /api/v1/scripts/{script_id}` ### 3.5 `GET /api/v1/scripts/{script_id}`
@@ -485,7 +506,7 @@ queued ──→ running ──┬─→ succeeded
| `POST` | `/api/v1/data-resources/uploads` | 创建上传会话,返回 `upload_id` + `upload_path` | | `POST` | `/api/v1/data-resources/uploads` | 创建上传会话,返回 `upload_id` + `upload_path` |
| `PUT` | `/api/v1/data-resources/uploads/{upload_id}` | 上传字节(请求体即文件内容) | | `PUT` | `/api/v1/data-resources/uploads/{upload_id}` | 上传字节(请求体即文件内容) |
| `POST` | `/api/v1/data-resources/uploads/{upload_id}/bind` | 绑定已完成上传为数据资源 | | `POST` | `/api/v1/data-resources/uploads/{upload_id}/bind` | 绑定已完成上传为数据资源 |
| `GET` | `/api/v1/data-resources` | 列表(workspace 范围) | | `GET` | `/api/v1/data-resources` | 列表(owner 作用域 + visibility 过滤) |
| `GET` | `/api/v1/data-resources/{id}` | 详情 | | `GET` | `/api/v1/data-resources/{id}` | 详情 |
| `POST` | `/api/v1/data-resources/{id}/download-url` | 生成 presigned GET URL | | `POST` | `/api/v1/data-resources/{id}/download-url` | 生成 presigned GET URL |
| `DELETE` | `/api/v1/data-resources/{id}` | 软删 | | `DELETE` | `/api/v1/data-resources/{id}` | 软删 |
@@ -521,6 +542,22 @@ queued ──→ running ──┬─→ succeeded
`content_base64` 字段(JSON 体里走),内部走同一条 `AsyncStorageBackend.put` `content_base64` 字段(JSON 体里走),内部走同一条 `AsyncStorageBackend.put`
路径,前端无需分两步。 路径,前端无需分两步。
### 五.1 `GET /api/v1/data-resources`
列出数据资源,按 **owner 作用域 + visibility 过滤**返回(与 §3.4 `GET /scripts` 同一套统一语义)。
- **owner 作用域**: `owner_user_id` 缺省=当前请求者本人,scope 到 `object_key` 前缀 `{workspace_id}/{me}/...`;传 `owner_user_id` 时 scope 到 `{workspace_id}/{owner_user_id}/...`。
- **visibility 过滤(统一)**: 非 admin 请求者只返回 `owner_user_id == me OR visibility IN (workspace, public)`;admin 请求者跳过过滤返回全部。语义同 §3.4——owner=me 含自己的 private;owner=other 只见其 workspace/public。
- **非递归**: 仅返回 `parent_path` 下的**直接子级**资源(懒加载用)。
- **查询参数**:
| 名 | 类型 | 必填 | 说明 |
|---|---|---|---|
| `parent_path` | string | 否 | 父目录相对路径,空串/缺省=该 owner 根目录下的一级资源 |
| `owner_user_id` | string | 否 | 目标 owner;缺省=请求者本人 |
| `visibility` | string | 否 | `workspace` \| `public` \| `private`,二次过滤 |
| `keyword` | string | 否 | 名称模糊匹配 |
- **响应**: `data` 为 `ResourcePayload` 数组,每条带 `owner_user_id`。
--- ---
## 六、管理后台 ## 六、管理后台
@@ -581,10 +618,16 @@ Base 前缀 `/api/v1/admin`。
## 七、系统管理 (`/api/v1/platform/...`) ## 七、系统管理 (`/api/v1/platform/...`)
平台级(跨 workspace)管理接口,用于管理员工、workspace 实体与 workspace 成员。 平台级(跨 workspace)管理接口,用于管理员工、workspace 实体与 workspace 成员。
所有端点要求调用者是**系统管理员**——其 `users.platform_role_id` 指向 除特别注明外,所有端点要求调用者是**系统管理员**——其 `users.platform_role_id` 指向
`role_code='admin'` 的角色行,且 `users.status == 'active'`。系统管理员判定 `role_code='admin'` 的角色行,且 `users.status == 'active'`。系统管理员判定
通过 `GET /api/v1/auth/me` 响应中的 `data.user.is_system_admin` 字段(详见 §一)。 通过 `GET /api/v1/auth/me` 响应中的 `data.user.is_system_admin` 字段(详见 §一)。
> **例外 — `GET /workspaces/{id}/members`**:该端点对**系统管理员(任意
> workspace)**与**该 workspace 的活跃成员**(`workspace_members.is_deleted=0`
> 且 `member_status='active'`)均开放。这是为了让普通(非 admin)用户能在
> 脚本目录树里渲染同 workspace 其他成员的折叠分组(跨 owner 浏览,见 §3.4)。
> 其余 members 写端点(POST/PATCH/DELETE members)仍仅限系统管理员。
| 方法 | 路径 | 说明 | | 方法 | 路径 | 说明 |
|---|---|---| |---|---|---|
| `GET` | `/api/v1/platform/employees` | 列全平台未软删员工;包含停用、锁定及无平台角色用户 | | `GET` | `/api/v1/platform/employees` | 列全平台未软删员工;包含停用、锁定及无平台角色用户 |
@@ -596,7 +639,7 @@ Base 前缀 `/api/v1/admin`。
| `GET` | `/api/v1/platform/workspaces/{workspace_id}` | 单个 workspace(含已 disabled 的,用于恢复) | | `GET` | `/api/v1/platform/workspaces/{workspace_id}` | 单个 workspace(含已 disabled 的,用于恢复) |
| `PATCH` | `/api/v1/platform/workspaces/{workspace_id}` | 改 workspace 字段;`status` 仅允许 `active`/`archived` | | `PATCH` | `/api/v1/platform/workspaces/{workspace_id}` | 改 workspace 字段;`status` 仅允许 `active`/`archived` |
| `DELETE` | `/api/v1/platform/workspaces/{workspace_id}` | 软删 workspace;级联软删其成员 | | `DELETE` | `/api/v1/platform/workspaces/{workspace_id}` | 软删 workspace;级联软删其成员 |
| `GET` | `/api/v1/platform/workspaces/{workspace_id}/members` | 列成员 | | `GET` | `/api/v1/platform/workspaces/{workspace_id}/members` | 列成员(**系统管理员或该 workspace 活跃成员**;为跨 owner 目录树提供成员名册,见 §7 intro 例外) |
| `POST` | `/api/v1/platform/workspaces/{workspace_id}/members` | 添加成员(返回 201) | | `POST` | `/api/v1/platform/workspaces/{workspace_id}/members` | 添加成员(返回 201) |
| `PATCH` | `/api/v1/platform/workspaces/{workspace_id}/members/{user_id}` | 改成员 `member_status`;**不能改 role_code**(workspace 角色继承自平台角色) | | `PATCH` | `/api/v1/platform/workspaces/{workspace_id}/members/{user_id}` | 改成员 `member_status`;**不能改 role_code**(workspace 角色继承自平台角色) |
| `DELETE` | `/api/v1/platform/workspaces/{workspace_id}/members/{user_id}` | 软删成员 | | `DELETE` | `/api/v1/platform/workspaces/{workspace_id}/members/{user_id}` | 软删成员 |
+137 -61
View File
@@ -6,58 +6,89 @@ refactors see `HANDOVER.md`.
## Code layout ## Code layout
All Python packages use the `src/<pkg>/` layout; `uv` workspace glues them into
one `.venv`. Always invoke via `uv run [--package <pkg>] <cmd>` (see "Local
development" for the gotcha).
``` ```
common/ Pure-Python shared library common/src/common/ Pure-Python shared library
config.py Settings (pydantic-settings, lru_cache singleton) config.py Settings (pydantic-settings, lru_cache singleton)
db/ SQLAlchemy 2.0 async engine, session_scope, Base db/ SQLAlchemy 2.0 async engine, session_scope, Base
db/models/ 26 tables in 9 domain files (zero FK, zero relationship) db/models/ 26 tables in 9 domain files (zero FK, zero relationship)
scheduler/ build_sqlalchemy_jobstore (delayed import) auth/ JWT / bcrypt / workspace membership helpers
storage/ AsyncStorageBackend abstraction (s3 + local impls) + Pydantic schemas scheduler/ APScheduler trigger helpers (delayed import)
eventing.py add_outbox_event / utcnow / event_time storage/ AsyncStorageBackend abstraction + Pydantic schemas
service_app.py /health/ready TCP probe, /api/v1/health base.py Abstract interface
schemas.py StrictModel base factory.py create_storage + build_storage_config + PURPOSE_BUCKETS
utils.py get_free_port, start_process schemas.py CreateUploadRequest / ServerObjectRequest
backends/local.py Local filesystem impl
backends/s3.py S3-compatible impl (boto3)
registry.py Bucket registry
eventing.py add_outbox_event / utcnow / event_time
service_app.py /health/ready TCP probe, /api/v1/health
logging.py loguru config (LOG_LEVEL)
schemas.py StrictModel base
ids.py ULID generation helpers
utils.py get_free_port, start_process
backend/ Public FastAPI service + tiny /internal/v1/objects RPC backend/src/backend/ Public FastAPI service
main.py lifespan + route registration main.py lifespan + route registration
jupyter.py /api/v1/auth/jupyter — the ONLY auth entry audit.py HTTP access log middleware (loguru sink)
scripts.py CRUD for scripts/notebooks (object storage via AsyncStorageBackend) api/ HTTP route handlers (one module per bounded context)
schedules.py DAG CRUD: schedules, nodes, edges auth.py /api/v1/auth/* (login / me / jupyter)
schedule_runs.py Trigger / list / get runs jupyter.py /api/v1/auth/jupyter — the ONLY auth entry
schedule_schemas.py Pydantic request/response models dependencies.py request_context, database_session
admin.py Admin endpoints platform.py /api/v1/platform/* (system admin)
resources.py Misc data resources admin.py /api/v1/admin/* (workspace-internal admin)
storage_api.py /internal/v1/objects — single token-guarded endpoint (P0-1) scripts.py /api/v1/scripts/* + /api/v1/workspace-directories
storage_client.py Stub (HTTP client removed post-migration; rewrite pending) resources.py /api/v1/data-resources/*
schedule_client.py Placeholder module (was the HTTP-push executor client) schedules/schedules.py DAG CRUD
runtime_client.py Self-contained httpx wrapper for the runtime schedules/runs.py Run lifecycle
jupyter.py auth_request handler storage.py /internal/v1/objects — single token-guarded endpoint (P0-1)
dependencies.py request_context, database_session services/ Pure-Python business logic (no HTTP / no DI)
scripts.py create_workspace_directory, visibility-filtered queries
resources.py owner-scoped resource listing helpers
jupyter.py jupyter_path / lock helpers
storage.py object store helpers
schedules.py DAG validation (cycle / orphan detection)
schemas/ Pydantic request / response models
auth.py / common.py / jupyter.py / platform.py / resources.py / schedules.py / scripts.py
clients/ Outbound HTTP / RPC clients
runtime.py Self-contained httpx wrapper for the runtime
scheduler.py Backend → Schedule HTTP client (callback / dispatch)
rclone.py rclone RC API client (FUSE cache invalidation)
schedule/ Schedule Executor (DAG worker) schedule/src/schedule/ Schedule Executor (DAG worker)
context.py Constants + naive_utc main.py Lifespan + FastAPI app
scheduler.py CronScheduler (APScheduler + 5s sync loop) notebook_runner.py Subprocess entry point (nbclient) — DO NOT RENAME
orchestrator.py DispatchOrchestrator (Outbox poll + DAG advance) domain/ Pure-Python domain types
worker.py NodeExecutor (notebook / python execution) execution.py ExecutionResult (frozen dataclass) + state enums
service.py SchedulerService facade (composes the three) context.py Constants + naive_utc
main.py Lifespan + FastAPI app scheduling/ Time-based trigger
storage_client.py SchedulerStorageClient — talks to backend /internal/v1/objects scheduler.py CronScheduler (APScheduler + 5s sync loop)
execution.py execute_artifact (notebook + python paths) application/ Facades / orchestrators
notebook_runner.py Subprocess entry point (nbclient) service.py SchedulerService (composes the three)
orchestrator.py DispatchOrchestrator (Outbox poll + DAG advance)
execution/ DAG node execution
executor.py NodeExecutor (notebook / python dispatch)
worker.py asyncio entry, schedule-spawned task boundary
runners/notebook.py nbclient subprocess path (6-line `notebook_runner` shim re-exports `main`)
infrastructure/ External-system adapters
storage/client.py SchedulerStorageClient — talks to backend /internal/v1/objects
runtime/ Jupyter Runtime runtime/src/runtime/ Jupyter Runtime
main.py FastAPI entry: jupyter action endpoints main.py FastAPI entry: jupyter action endpoints
process.py Per-workspace subprocess pool + asyncio locks process.py Per-workspace subprocess pool + asyncio locks
mount.py rclone FUSE mount lifecycle mount.py rclone FUSE mount lifecycle
frontend/ React Router SPA (vite build → nginx) frontend/ React Router SPA (vite build → nginx)
app/ features/ routes/ services/ components/ app/ features/ routes/ services/ components/
migrations/ Alembic schema versions migrations/ Alembic schema versions
docker-compose.yml 4 services docker-compose.yml 4 services (gateway / backend / schedule / runtime)
default.conf Nginx template default.conf Nginx template
scripts/nginx-entrypoint.sh scripts/nginx-entrypoint.sh
.env.example .env.example All 26 config.py keys documented
``` ```
## Configuration system ## Configuration system
@@ -67,10 +98,31 @@ All env vars go through one place: `common/src/common/config.py`.
```python ```python
from common.config import settings from common.config import settings
settings.database_url # str # Auth / runtime
settings.storage_backend # str: "s3" (default) or "local" settings.database_url # str — SQLAlchemy async URL (mysql+asyncmy, charset utf8mb4)
settings.local_storage_base_dir # str: root dir for storage data (default "/data"); see "Storage" below for per-mode derivation settings.jwt_secret # HS256 secret for the auth_request handler
settings.s3_endpoint # str (full URL, e.g. "http://s3:9000"; s3 mode only) settings.cookie_force_secure # bool — write Secure flag even on plain HTTP (TLS-terminating proxy)
settings.service_name # surfaced in /health
settings.schedule_event_namespace # APScheduler JobStore namespace + Outbox scope prefix
settings.readiness_targets # CSV host:port list for /health/ready
# HTTP clients (intra-cluster URLs)
settings.runtime_api_url # backend → runtime HTTP base
settings.public_base_url # runtime public base URL (browser-facing /jupyter/)
settings.backend_api_url # schedule → backend HTTP base
settings.rclone_rc_url # backend → rclone RC control API
settings.internal_service_token # Backend ↔ Schedule shared secret (X-Internal-Service-Token)
# Logging / audit
settings.log_level # DEBUG / INFO / WARNING / ERROR / CRITICAL (lowercase → fallback INFO)
settings.audit_log_dir # dir for daily audit logs (relative to cwd; "" disables file sink)
settings.audit_log_retention_days # 0 disables cleanup
settings.audit_excluded_paths # list[str] — paths skipped from audit (health probes, etc.)
# Storage
settings.storage_backend # "s3" (default) or "local"
settings.local_storage_base_dir # root dir for storage data (default "/data")
settings.s3_endpoint # str (s3 mode only)
settings.s3_access_key # str (s3 mode only) settings.s3_access_key # str (s3 mode only)
settings.s3_secret_key # str (s3 mode only) settings.s3_secret_key # str (s3 mode only)
settings.s3_workspace_bucket # str (s3 mode only) settings.s3_workspace_bucket # str (s3 mode only)
@@ -78,17 +130,15 @@ settings.s3_version_bucket # str (s3 mode only)
settings.s3_run_log_bucket # str (s3 mode only) settings.s3_run_log_bucket # str (s3 mode only)
settings.s3_trash_bucket # str (s3 mode only) settings.s3_trash_bucket # str (s3 mode only)
settings.s3_trash_retention_days # int (s3 mode only) settings.s3_trash_retention_days # int (s3 mode only)
settings.jwt_secret # HS256 secret for the auth_request handler
settings.backend_api_url # schedule → backend HTTP base # Schedule
settings.runtime_api_url # backend → runtime HTTP base settings.schedule_execution_concurrency # int — max concurrent notebook subprocesses
settings.public_base_url # runtime public base URL
settings.service_name # surfaced in /health
settings.readiness_targets # CSV host:port list for /health/ready
``` ```
`Settings` reads from process env first, then from a `.env` file at CWD `Settings` reads from process env first, then from a `.env` file at CWD
if present. `pydantic-settings` auto-loads. `case_sensitive=False` so if present. `pydantic-settings` auto-loads. `case_sensitive=False` so
`DATABASE_URL` / `database_url` both work. `DATABASE_URL` / `database_url` both work. The full list of 26 fields
is in `common/src/common/config.py`.
### Adding a new env var ### Adding a new env var
@@ -96,7 +146,8 @@ if present. `pydantic-settings` auto-loads. `case_sensitive=False` so
```python ```python
new_var: str = Field(default="x", description="...") new_var: str = Field(default="x", description="...")
``` ```
2. Add the line to `.env.example` with a comment. 2. Add the line to `.env.example` with a comment (keep it synced — every
field in config.py must have a matching `.env.example` entry).
3. Use `settings.new_var` at the call site. 3. Use `settings.new_var` at the call site.
Do **not** call `os.environ["NEW_VAR"]` or `os.getenv("NEW_VAR")` in Do **not** call `os.environ["NEW_VAR"]` or `os.getenv("NEW_VAR")` in
@@ -190,13 +241,34 @@ grep -rnE 'os\.(environ\[?["\x27][A-Z_]+|getenv\(["\x27][A-Z_]+)' --include="*.p
- For write operations on a script/notebook, call - For write operations on a script/notebook, call
`require_script_modify_access(script, user_id=..., is_admin=...)` `require_script_modify_access(script, user_id=..., is_admin=...)`
from `backend/scripts.py`. It enforces: from `backend/src/backend/services/scripts.py`. It enforces:
- admin or owner → allow - admin or owner → allow
- non-owner, `is_locked == 0` → allow - non-owner, `is_locked == 0` → allow
- non-owner, `is_locked == 1` → 403 - non-owner, `is_locked == 1` → 403
- Read endpoints (`list_scripts`, `get_script`) intentionally do **not** - Read endpoints (`list_scripts`, `get_script`) intentionally do **not**
check `is_locked` — workspace members can see the script list. check `is_locked` — workspace members can see the script list.
### Owner-scoping + visibility (cross-owner browsing)
`GET /api/v1/scripts`, `GET /api/v1/data-resources`, and
`GET /api/v1/workspace-directories` all accept an optional
`owner_user_id` query param and follow the same model:
- `owner_user_id` 缺省 = 当前请求者本人(scope to `workspace/{me}/...`).
- 传值时 scope 到 `workspace/{owner_user_id}/...`,用于前端"点开其他成员
分组"的懒加载(见 §3.4 of `API.md`).
- `visibility` 过滤(非 admin):`owner_user_id == me OR visibility IN
(workspace, public)` — 自己可见自己全部(含 private),他人只见其
workspace/public,排除他人 private.
- 系统管理员跳过 visibility 过滤.
- 目录行默认 `visibility='public'`(由 `create_workspace_directory` 写入),
不施加 visibility 过滤,使跨 owner 目录树可见.
实现 helper 在 `backend/src/backend/services/scripts.py`
(`_build_list_scripts_owner_descendant_prefix`) 和
`backend/src/backend/api/resources.py` 中按相同模式分别构造 owner-scoped
LIKE 前缀。
### Outbox events ### Outbox events
- The platform's only async-messaging fabric is the MySQL - The platform's only async-messaging fabric is the MySQL
@@ -238,6 +310,10 @@ cd frontend && pnpm install && cd ..
### Per-service dev ### Per-service dev
Always go through `uv run` so the workspace `.venv` is used — bare
`uvicorn` / `python` resolves to system Python and `from backend.X`
imports fail with ModuleNotFoundError.
```bash ```bash
# Backend (terminal 1) # Backend (terminal 1)
export DATABASE_URL="mysql+asyncmy://model_platform:model_platform@127.0.0.1:3306/model_platform?charset=utf8mb4" export DATABASE_URL="mysql+asyncmy://model_platform:model_platform@127.0.0.1:3306/model_platform?charset=utf8mb4"
@@ -248,13 +324,13 @@ export S3_ENDPOINT=http://127.0.0.1:9000
# Or for local mode: # Or for local mode:
# export STORAGE_BACKEND=local # export STORAGE_BACKEND=local
# export LOCAL_STORAGE_BASE_DIR=/data # export LOCAL_STORAGE_BASE_DIR=/data
uv run --frozen --package backend uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload uv run --package backend uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
# Schedule Executor (terminal 2) # Schedule Executor (terminal 2)
uv run --frozen --package schedule uvicorn schedule.main:app --host 0.0.0.0 --port 8001 --reload uv run --package schedule uvicorn schedule.main:app --host 0.0.0.0 --port 8001 --reload
# Runtime (terminal 3 — needs SYS_ADMIN, FUSE, devmode) # Runtime (terminal 3 — needs SYS_ADMIN, FUSE, devmode)
uv run --frozen --package runtime python -m runtime.main uv run --package runtime python -m runtime.main
``` ```
### Frontend dev ### Frontend dev
+42 -3
View File
@@ -243,6 +243,21 @@ async def system_admin_context(
) )
async def _is_system_admin(session: AsyncSession, user: Users) -> bool:
"""True if ``user`` holds the platform-scoped admin role.
Mirrors the check inside :func:`system_admin_context` so member-listing
endpoints can admit workspace members *or* system admins without pulling
in the full :class:`SystemAdminContext` (which 403s non-admins outright).
"""
if user.platform_role_id is None:
return False
platform_role = await session.scalar(
select(Roles).where(Roles.role_id == user.platform_role_id)
)
return platform_role is not None and platform_role.role_code == "admin"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Payload helpers # Payload helpers
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -809,10 +824,33 @@ async def delete_workspace(
@router.get("/workspaces/{workspace_id}/members") @router.get("/workspaces/{workspace_id}/members")
async def list_members( async def list_members(
workspace_id: str, workspace_id: str,
context: SystemAdminContext = Depends(system_admin_context), request: Request,
session: AsyncSession = Depends(database_session), session: AsyncSession = Depends(database_session),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""List active and historical (non-soft-deleted) members of a workspace.""" """List active and historical (non-soft-deleted) members of a workspace.
Accessible to system admins (any workspace) and to active members of the
workspace itself. The script explorer calls this to seed the per-owner
directory-tree groups for non-admin users; visibility filters on the
scripts/data-resources endpoints still keep each peer's private content
hidden, so this only exposes membership (names), not private files.
"""
user = await current_user(request, session)
is_system_admin = await _is_system_admin(session, user)
if not is_system_admin:
membership = await session.scalar(
select(WorkspaceMembers).where(
WorkspaceMembers.workspace_id == workspace_id,
WorkspaceMembers.user_id == user.user_id,
WorkspaceMembers.is_deleted == 0,
WorkspaceMembers.member_status == "active",
)
)
if membership is None:
raise HTTPException(
status.HTTP_403_FORBIDDEN,
"需要系统管理员或该工作区成员权限",
)
await _load_workspace(session, workspace_id) await _load_workspace(session, workspace_id)
rows = ( rows = (
await session.execute( await session.execute(
@@ -830,8 +868,9 @@ async def list_members(
.limit(LIST_PAGE_SIZE) .limit(LIST_PAGE_SIZE)
) )
).all() ).all()
request_id = request.headers.get("X-Request-ID") or new_ulid()
return _envelope( return _envelope(
context.request_id, request_id,
[member_payload(u, r, m) for u, r, m in rows], [member_payload(u, r, m) for u, r, m in rows],
{"count": len(rows), "page_size": LIST_PAGE_SIZE}, {"count": len(rows), "page_size": LIST_PAGE_SIZE},
) )
+32 -24
View File
@@ -51,13 +51,13 @@ def _build_list_resources_descendant_prefix(parent_path: str) -> str:
"""Return the escaped materialized-path prefix for direct children """Return the escaped materialized-path prefix for direct children
of ``parent_path`` against ``StorageObjects.object_key``. of ``parent_path`` against ``StorageObjects.object_key``.
Data resources are workspace-wide (no per-user scoping at the API The full object_key is ``{ws_id}/{owner_user_id}/{jupyter_path}``.
level). The full object_key is ``{ws_id}/{user_id}/{jupyter_path}``; ``list_resources`` prepends ``{ws_id}/{owner_user_id}`` (the requester
we filter on object_key with the pattern ``{ws_id}/%/{parent_path}`` by default, or the ``owner_user_id`` query param) to this prefix and
so any owner whose jupyter_accessible_path starts with parent_path applies ``LIKE '{ws_id}/{owner}/{prefix}%' AND NOT LIKE '...%/%'`` so
matches. LIKE wildcards in parent_path are escaped; the ``%`` between only that owner's direct children under ``parent_path`` match. LIKE
``{ws_id}/`` and the escaped parent is an intentional SQL wildcard wildcards in parent_path are escaped so folder names containing ``_``
matching the ``owner_user_id`` segment across all owners. or ``%`` do not act as wildcards.
""" """
normalized = normalize_user_path(parent_path) normalized = normalize_user_path(parent_path)
escaped = _escape_like_pattern(normalized) escaped = _escape_like_pattern(normalized)
@@ -383,6 +383,7 @@ async def bind_resource(
@router.get("") @router.get("")
async def list_resources( async def list_resources(
parent_path: str = Query(default="", max_length=1024), parent_path: str = Query(default="", max_length=1024),
owner_user_id: str | None = Query(default=None, max_length=64),
context: RequestContext = Depends(request_context), context: RequestContext = Depends(request_context),
session: AsyncSession = Depends(database_session), session: AsyncSession = Depends(database_session),
visibility: str | None = Query(default=None), visibility: str | None = Query(default=None),
@@ -403,23 +404,30 @@ async def list_resources(
) )
.order_by(DataResources.updated_at.desc()) .order_by(DataResources.updated_at.desc())
) )
if parent_path: # ``parent_path`` scopes to DIRECT children of that jupyter path
# ``parent_path`` scopes to DIRECT children of that jupyter path # (matching /api/v1/scripts). Per-owner listing: default (no
# (matching /api/v1/scripts). Data resources are workspace-wide, so # owner_user_id) scopes to the requester's own object_key subtree
# the middle ``%`` is an intentional wildcard that matches the # (``{ws_id}/{me}/...``); passing owner_user_id scopes to that owner's
# ``owner_user_id`` segment across all owners. The parent's ``_`` / # subtree so the tree can lazily fetch another member's data resources
# ``%`` are escaped so sibling folders (e.g. ``fooXbar``) don't leak. # on group expand. The parent's ``_`` / ``%`` are escaped so sibling
descendant_prefix = _build_list_resources_descendant_prefix(parent_path) # folders (e.g. ``fooXbar``) don't leak. Empty parent_path still applies
statement = statement.where( # the filter: it resolves to that owner's root-level direct children
StorageObjects.object_key.like( # (``{ws_id}/{owner}/%`` and NOT ``{ws_id}/{owner}/%/%``), symmetric with
f"{context.workspace.workspace_id}/%/{descendant_prefix}%", # list_scripts. Skipping the filter for empty input would silently
escape="\\", # surface nested descendants and break the directory tree.
), target_owner = owner_user_id or context.user.user_id
~StorageObjects.object_key.like( owner_prefix = f"{context.workspace.workspace_id}/{target_owner}"
f"{context.workspace.workspace_id}/%/{descendant_prefix}%/%", descendant_prefix = _build_list_resources_descendant_prefix(parent_path)
escape="\\", statement = statement.where(
), StorageObjects.object_key.like(
) f"{owner_prefix}/{descendant_prefix}%",
escape="\\",
),
~StorageObjects.object_key.like(
f"{owner_prefix}/{descendant_prefix}%/%",
escape="\\",
),
)
# 只返回 owner 自己的资源(含 private),或 visibility 为 # 只返回 owner 自己的资源(含 private),或 visibility 为
# workspace/public 的其他成员资源;A 的 private 资源对非 owner 不可见。 # workspace/public 的其他成员资源;A 的 private 资源对非 owner 不可见。
# admin 跳过过滤,全部可见。 # admin 跳过过滤,全部可见。
+50 -35
View File
@@ -129,40 +129,36 @@ def _escape_like_pattern(value: str) -> str:
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_") return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
# @deprecated — legacy user-scoped prefix helper. list_scripts/count_scripts def _build_list_scripts_owner_descendant_prefix(
# are now workspace-wide via _build_list_scripts_workspace_descendant_prefix; owner_user_id: str, parent_path: str
# this helper is kept (with its original user-scoped semantics) so existing
# callers/tests that still reference it do not break.
def _build_list_scripts_descendant_prefix(
context: RequestContext, parent_path: str
) -> str: ) -> str:
"""Return the escaped materialized-path prefix for direct children of """Return the escaped materialized-path prefix for direct children of
``parent_path`` within the **requester's own subtree**. ``parent_path`` within ``owner_user_id``'s own subtree.
.. note:: Storage is physically laid out as ``workspace/{owner_user_id}/...``, so a
Legacy user-scoped helper. list_scripts / count_scripts are now per-owner listing matches ``workspace/{owner_user_id}/{parent}``. The
workspace-wide — use endpoint appends ``LIKE '<prefix>%' AND NOT LIKE '<prefix>%/%'`` against
:func:`_build_list_scripts_workspace_descendant_prefix` instead ``storage_objects.relative_path`` so only scripts whose parent directory
(visibility filtering handles non-admin scoping in the SQL). is exactly ``parent_path`` match (no deeper descendants, no
prefix-siblings like ``foo/bar`` vs ``foo/bar2``).
The endpoint appends ``LIKE '<prefix>/%' AND NOT LIKE '<prefix>/%/%'`` Empty ``parent_path`` produces the owner-scoped root prefix — i.e. the
against ``storage_objects.relative_path`` so only scripts whose parent endpoint returns that owner's root-level scripts only. ``list_scripts``
directory is exactly ``parent_path`` (no deeper descendants, no calls this with ``owner_user_id`` = the requester by default (so a
prefix-siblings like ``foo/bar`` vs ``foo/bar2``) match. non-admin sees their own subtree, including private) or with the
``owner_user_id`` query param so the tree can lazily fetch another
Empty ``parent_path`` produces the user-scoped root prefix — i.e. the member's content on group expand; the route's visibility filter then
endpoint returns root-level scripts only, not the full workspace. excludes the other owner's private rows.
The prefix is run through ``_escape_like_pattern`` so folder names The prefix is run through ``_escape_like_pattern`` so folder names
containing ``_`` / ``%`` do not act as wildcards. The trailing ``/`` containing ``_`` / ``%`` do not act as wildcards. The trailing ``/``
is appended AFTER escaping so it remains a literal slash. is appended AFTER escaping so it remains a literal slash.
""" """
normalized_parent = normalize_user_path(parent_path) normalized_parent = normalize_user_path(parent_path)
scoped_prefix = user_relative_path(context)
if normalized_parent: if normalized_parent:
target_prefix = f"{scoped_prefix}/{normalized_parent}" target_prefix = f"workspace/{owner_user_id}/{normalized_parent}"
else: else:
target_prefix = scoped_prefix target_prefix = f"workspace/{owner_user_id}"
return f"{_escape_like_pattern(target_prefix)}/" return f"{_escape_like_pattern(target_prefix)}/"
@@ -836,16 +832,25 @@ async def get_workspace_tree(
@router.get("/workspace-directories") @router.get("/workspace-directories")
async def list_workspace_directories( async def list_workspace_directories(
parent_path: str = Query(default=""), parent_path: str = Query(default=""),
owner_user_id: str | None = Query(default=None, max_length=64),
context: RequestContext = Depends(request_context), context: RequestContext = Depends(request_context),
session: AsyncSession = Depends(database_session), session: AsyncSession = Depends(database_session),
) -> dict[str, Any]: ) -> dict[str, Any]:
"""List direct child directories of a workspace path. """List direct child directories of a workspace path.
Empty ``parent_path`` returns the directories immediately under the Empty ``parent_path`` returns the directories immediately under the
user's scoped root. Only available, non-deleted StorageObjects are target owner's scoped root. Only available, non-deleted StorageObjects
considered. are considered.
``owner_user_id`` defaults to the requester, so a member lists their
own directories. Passing another member's id scopes to that owner's
subtree so the script explorer can lazily render their directory
structure on expand (directories are structural rows; file-level
visibility is still enforced by the scripts/data-resources endpoints,
which exclude the other owner's private files).
""" """
scoped_prefix = user_relative_path(context) target_owner = owner_user_id or context.user.user_id
scoped_prefix = f"workspace/{target_owner}"
parent = normalize_user_path(parent_path) parent = normalize_user_path(parent_path)
target_prefix = f"{scoped_prefix}/{parent}" if parent else scoped_prefix target_prefix = f"{scoped_prefix}/{parent}" if parent else scoped_prefix
descendant_prefix = f"{_escape_like_pattern(target_prefix)}/" descendant_prefix = f"{_escape_like_pattern(target_prefix)}/"
@@ -878,6 +883,7 @@ async def list_workspace_directories(
"path": child_path, "path": child_path,
"name": suffix, "name": suffix,
"parent_path": parent, "parent_path": parent,
"owner_user_id": target_owner,
"has_children": False, "has_children": False,
}, },
) )
@@ -1018,7 +1024,8 @@ async def create_workspace_directory(
path_hash=path_hash, path_hash=path_hash,
object_status="available", object_status="available",
size_bytes=0, size_bytes=0,
visibility="private", visibility="public",
owner_user_id=context.user.user_id,
created_by=context.user.user_id, created_by=context.user.user_id,
) )
session.add(directory) session.add(directory)
@@ -1032,7 +1039,8 @@ async def create_workspace_directory(
directory.storage_uri = f"inline://directory/{relative_path}" directory.storage_uri = f"inline://directory/{relative_path}"
directory.file_name = name directory.file_name = name
directory.size_bytes = 0 directory.size_bytes = 0
directory.visibility = "private" directory.visibility = "public"
directory.owner_user_id = context.user.user_id
directory.created_by = context.user.user_id directory.created_by = context.user.user_id
try: try:
@@ -1058,6 +1066,7 @@ async def create_workspace_directory(
"path": child_path, "path": child_path,
"name": name, "name": name,
"parent_path": parent, "parent_path": parent,
"owner_user_id": context.user.user_id,
}, },
"meta": {}, "meta": {},
} }
@@ -1165,17 +1174,23 @@ async def delete_workspace_directory(
@router.get("/scripts") @router.get("/scripts")
async def list_scripts( async def list_scripts(
parent_path: str = Query(default="", max_length=1024), parent_path: str = Query(default="", max_length=1024),
owner_user_id: str | None = Query(default=None, max_length=64),
context: RequestContext = Depends(request_context), context: RequestContext = Depends(request_context),
session: AsyncSession = Depends(database_session), session: AsyncSession = Depends(database_session),
) -> dict[str, Any]: ) -> dict[str, Any]:
# Workspace-wide listing: storage is physically laid out as # Per-owner listing: storage is physically laid out as
# ``workspace/{user_id}/...``. Empty parent_path wildcards the owner # ``workspace/{user_id}/...``. Default (no owner_user_id) scopes to the
# segment (``workspace/%/``) so each owner's root files are returned; # requester's own subtree — root-level files when parent_path is empty —
# non-empty parent_path embeds the same owner wildcard # so the tree's initial load fetches only "me". Passing owner_user_id
# (``workspace/%/foo``) so every owner's ``foo`` subtree matches, # scopes to that owner's subtree so the tree can lazily fetch another
# mirroring list_resources. Non-admin scoping is applied below via # member's content when their group is expanded. Non-admin scoping is
# visibility, matching list_resources (69a9a48). # applied below via visibility, so the other owner's private rows are
descendant_prefix = _build_list_scripts_workspace_descendant_prefix(parent_path) # excluded (workspace/public only); the requester's own private rows
# pass because ``owner_user_id = me``.
target_owner = owner_user_id or context.user.user_id
descendant_prefix = _build_list_scripts_owner_descendant_prefix(
target_owner, parent_path
)
statement = ( statement = (
select(Scripts, StorageObjects, Users.display_name) select(Scripts, StorageObjects, Users.display_name)
+4 -4
View File
@@ -33,16 +33,16 @@ from fastapi import Request
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from loguru import logger from loguru import logger
from backend.audit import configure_audit_logging
from backend.api.admin import router as admin_router from backend.api.admin import router as admin_router
from backend.api.auth import router as auth_router from backend.api.auth import router as auth_router
from backend.api.jupyter import router as jupyter_router from backend.api.jupyter import router as jupyter_router
from backend.api.platform import router as platform_router from backend.api.platform import router as platform_router
from backend.api.resources import router as resources_router from backend.api.resources import router as resources_router
from backend.api.scripts import router as scripts_router
from backend.api.schedules.runs import router as schedule_runs_router from backend.api.schedules.runs import router as schedule_runs_router
from backend.api.schedules.schedules import router as schedules_router from backend.api.schedules.schedules import router as schedules_router
from backend.api.scripts import router as scripts_router
from backend.api.storage import router as storage_api_router from backend.api.storage import router as storage_api_router
from backend.audit import configure_audit_logging
from backend.clients.rclone import RcloneRCClient from backend.clients.rclone import RcloneRCClient
from backend.clients.runtime import RuntimeClient from backend.clients.runtime import RuntimeClient
@@ -161,7 +161,7 @@ async def access_log(request: Request, call_next):
method=request.method, method=request.method,
path=request.url.path, path=request.url.path,
status=500, status=500,
).info("audit") ).info(f"{request.url.path} skip audit")
raise raise
elapsed_ms = (time.perf_counter() - start) * 1000 elapsed_ms = (time.perf_counter() - start) * 1000
logger.info( logger.info(
@@ -175,7 +175,7 @@ async def access_log(request: Request, call_next):
method=request.method, method=request.method,
path=request.url.path, path=request.url.path,
status=response.status_code, status=response.status_code,
).info("audit") ).info(f"{request.url.path} skip audit")
return response return response
+126 -27
View File
@@ -28,7 +28,7 @@ from sqlalchemy import Column, MetaData, String, Table, create_engine, select, t
from sqlalchemy.dialects import mysql as mysql_dialect from sqlalchemy.dialects import mysql as mysql_dialect
from backend.api.scripts import ( from backend.api.scripts import (
_build_list_scripts_descendant_prefix, _build_list_scripts_owner_descendant_prefix,
_build_list_scripts_workspace_descendant_prefix, _build_list_scripts_workspace_descendant_prefix,
_escape_like_pattern, _escape_like_pattern,
normalize_user_path, normalize_user_path,
@@ -85,14 +85,14 @@ class TestEscapeLikePattern:
def test_descendant_prefix_root() -> None: def test_descendant_prefix_root() -> None:
"""Empty parent_path → descendant prefix is the scoped root + '/'.""" """Empty parent_path → descendant prefix is the owner-scoped root + '/'."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "") prefix = _build_list_scripts_owner_descendant_prefix("alice", "")
assert prefix == "workspace/alice/" assert prefix == "workspace/alice/"
def test_descendant_prefix_subdir() -> None: def test_descendant_prefix_subdir() -> None:
"""Non-empty parent_path → appended under the scoped root.""" """Non-empty parent_path → appended under the owner-scoped root."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "foo/bar") prefix = _build_list_scripts_owner_descendant_prefix("alice", "foo/bar")
assert prefix == "workspace/alice/foo/bar/" assert prefix == "workspace/alice/foo/bar/"
@@ -100,18 +100,18 @@ def test_descendant_prefix_escapes_metachars() -> None:
"""Folder name ``foo_bar`` MUST produce ``foo\\_bar`` in the prefix """Folder name ``foo_bar`` MUST produce ``foo\\_bar`` in the prefix
so the trailing ``%`` doesn't become 'match any single char before so the trailing ``%`` doesn't become 'match any single char before
b'.""" b'."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "foo_bar") prefix = _build_list_scripts_owner_descendant_prefix("alice", "foo_bar")
assert prefix == r"workspace/alice/foo\_bar/" assert prefix == r"workspace/alice/foo\_bar/"
def test_descendant_prefix_normalizes_leading_trailing_slashes() -> None: def test_descendant_prefix_normalizes_leading_trailing_slashes() -> None:
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "/foo/bar/") prefix = _build_list_scripts_owner_descendant_prefix("alice", "/foo/bar/")
assert prefix == "workspace/alice/foo/bar/" assert prefix == "workspace/alice/foo/bar/"
def test_descendant_prefix_rejects_traversal() -> None: def test_descendant_prefix_rejects_traversal() -> None:
with pytest.raises(HTTPException) as exc: with pytest.raises(HTTPException) as exc:
_build_list_scripts_descendant_prefix(_ctx("alice"), "foo/../bar") _build_list_scripts_owner_descendant_prefix("alice", "foo/../bar")
assert exc.value.status_code == 422 assert exc.value.status_code == 422
@@ -202,14 +202,19 @@ async def test_list_scripts_where_clause_uses_like_prefix_and_excludes_deeper()
) )
) )
await list_scripts(parent_path="foo/bar", context=_ctx("alice"), session=mock_session) await list_scripts(
parent_path="foo/bar",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
assert len(captured_sql) == 1 assert len(captured_sql) == 1
sql = captured_sql[0].lower() sql = captured_sql[0].lower()
# 中间 ``%`` 是 owner 段通配符(跨所有 owner),父路径本身按字面匹配, # Default (no owner_user_id) scopes to the requester's own subtree:
# 与 list_resources 对 object_key 的过滤一致。 # LIKE workspace/alice/foo/bar/% (direct children), excluding deeper.
assert "like 'workspace/%%/foo/bar/%%'" in sql assert "like 'workspace/alice/foo/bar/%%'" in sql
assert "not like 'workspace/%%/foo/bar/%%/%%'" in sql assert "not like 'workspace/alice/foo/bar/%%/%%'" in sql
async def test_list_scripts_where_clause_escapes_pattern_literal() -> None: async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
@@ -230,7 +235,12 @@ async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
) )
) )
await list_scripts(parent_path="foo_bar", context=_ctx("alice"), session=mock_session) await list_scripts(
parent_path="foo_bar",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0] sql = captured_sql[0]
# Normalize keyword case so we don't depend on SQLAlchemy casing. # Normalize keyword case so we don't depend on SQLAlchemy casing.
sql_lower = sql.lower() sql_lower = sql.lower()
@@ -238,9 +248,9 @@ async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
# doubles the escape char inside the SQL string literal, so what # doubles the escape char inside the SQL string literal, so what
# the helper emits as `foo\_bar` renders as `foo\\_bar` here # the helper emits as `foo\_bar` renders as `foo\\_bar` here
# (2 backslash chars in the actual SQL string). # (2 backslash chars in the actual SQL string).
assert r"like 'workspace/%%/foo\\_bar/%%'" in sql_lower assert r"like 'workspace/alice/foo\\_bar/%%'" in sql_lower
# NOT LIKE clause also escaped. # NOT LIKE clause also escaped.
assert r"not like 'workspace/%%/foo\\_bar/%%/%%'" in sql_lower assert r"not like 'workspace/alice/foo\\_bar/%%/%%'" in sql_lower
# And both declare ESCAPE '\\'. # And both declare ESCAPE '\\'.
assert sql.count("ESCAPE '\\\\'") == 2, sql assert sql.count("ESCAPE '\\\\'") == 2, sql
@@ -262,18 +272,26 @@ async def test_list_scripts_where_clause_escapes_percent_pattern() -> None:
) )
) )
await list_scripts(parent_path="100%match", context=_ctx("alice"), session=mock_session) await list_scripts(
parent_path="100%match",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0] sql = captured_sql[0]
sql_lower = sql.lower() sql_lower = sql.lower()
# SQLAlchemy doubles the escape char so `%` → `\%` becomes `\\%` # SQLAlchemy doubles the escape char so `%` → `\%` becomes `\\%`
# in the SQL string literal. # in the SQL string literal.
assert r"workspace/%%/100\\%%match/%%" in sql_lower assert r"workspace/alice/100\\%%match/%%" in sql_lower
async def test_list_scripts_non_admin_adds_visibility_filter() -> None: async def test_list_scripts_non_admin_adds_visibility_filter() -> None:
"""Workspace-wide listing is narrowed by visibility for non-admin: """Owner-scoped listing is narrowed by visibility for non-admin:
owner_user_id = me OR visibility IN (workspace, public) — exactly like owner_user_id = me OR visibility IN (workspace, public) — exactly like
list_resources. The workspace prefix contains NO user_id (cross-owner).""" list_resources. Default (no owner_user_id) scopes to the requester's own
subtree, so the visibility predicate is redundant-but-present here; it
becomes load-bearing when an owner_user_id query param browses another
member's subtree (their private rows are then excluded)."""
from backend.api.scripts import list_scripts from backend.api.scripts import list_scripts
captured_sql: list[str] = [] captured_sql: list[str] = []
@@ -289,11 +307,50 @@ async def test_list_scripts_non_admin_adds_visibility_filter() -> None:
) )
) )
await list_scripts(parent_path="", context=_ctx("alice"), session=mock_session) await list_scripts(
parent_path="",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0].lower() sql = captured_sql[0].lower()
# Root listing wildcards the owner segment: LIKE workspace/%/% # Default (no owner_user_id) scopes to the requester's own root:
# (each owner's root files), excluding 3+ segment descendants. # LIKE workspace/alice/% (alice's root files), excluding nested.
assert "like 'workspace/%%/%%'" in sql assert "like 'workspace/alice/%%'" in sql
assert "scripts.owner_user_id = 'alice'" in sql
assert "scripts.visibility in ('workspace', 'public')" in sql
async def test_list_scripts_owner_param_scopes_to_other_owner() -> None:
"""owner_user_id=<other> scopes the LIKE to that owner's subtree so the
tree can lazily fetch another member's content on group expand. The
non-admin visibility predicate is still applied, so the other owner's
private rows are excluded (only workspace/public survive)."""
from backend.api.scripts import list_scripts
captured_sql: list[str] = []
class _MockResult:
def all(self):
return []
mock_session = MagicMock()
mock_session.execute = AsyncMock(
side_effect=lambda stmt: (
captured_sql.append(_compile_sql(stmt)) or _MockResult()
)
)
await list_scripts(
parent_path="",
owner_user_id="bob",
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0].lower()
assert "like 'workspace/bob/%%'" in sql
assert "not like 'workspace/bob/%%/%%'" in sql
# non-admin: visibility predicate still present (excludes bob's private)
assert "scripts.owner_user_id = 'alice'" in sql assert "scripts.owner_user_id = 'alice'" in sql
assert "scripts.visibility in ('workspace', 'public')" in sql assert "scripts.visibility in ('workspace', 'public')" in sql
@@ -316,15 +373,47 @@ async def test_list_scripts_admin_skips_visibility_filter() -> None:
) )
await list_scripts( await list_scripts(
parent_path="", context=_ctx("alice", is_admin=True), session=mock_session parent_path="",
owner_user_id=None,
context=_ctx("alice", is_admin=True),
session=mock_session,
) )
sql = captured_sql[0].lower() sql = captured_sql[0].lower()
assert "like 'workspace/%%/%%'" in sql assert "like 'workspace/alice/%%'" in sql
# owner_user_id / visibility still appear in the SELECT projection; what # owner_user_id / visibility still appear in the SELECT projection; what
# must be absent is the visibility WHERE predicate for non-admins. # must be absent is the visibility WHERE predicate for non-admins.
assert "scripts.visibility in ('workspace', 'public')" not in sql assert "scripts.visibility in ('workspace', 'public')" not in sql
async def test_list_scripts_admin_owner_param_skips_visibility() -> None:
"""Admin browsing another owner's subtree scopes to that owner and skips
the visibility predicate (admin sees the other owner's private too)."""
from backend.api.scripts import list_scripts
captured_sql: list[str] = []
class _MockResult:
def all(self):
return []
mock_session = MagicMock()
mock_session.execute = AsyncMock(
side_effect=lambda stmt: (
captured_sql.append(_compile_sql(stmt)) or _MockResult()
)
)
await list_scripts(
parent_path="sub",
owner_user_id="bob",
context=_ctx("alice", is_admin=True),
session=mock_session,
)
sql = captured_sql[0].lower()
assert "like 'workspace/bob/sub/%%'" in sql
assert "scripts.visibility in ('workspace', 'public')" not in sql
async def test_list_workspace_directories_where_clause_escapes_pattern() -> None: async def test_list_workspace_directories_where_clause_escapes_pattern() -> None:
"""list_workspace_directories must escape user input too (was """list_workspace_directories must escape user input too (was
pre-existing debt).""" pre-existing debt)."""
@@ -351,11 +440,21 @@ async def test_list_workspace_directories_where_clause_escapes_pattern() -> None
) )
await list_workspace_directories( await list_workspace_directories(
parent_path="foo_bar", context=_ctx("alice"), session=mock_session parent_path="foo_bar", owner_user_id=None,
context=_ctx("alice"), session=mock_session,
) )
sql = " ".join(captured_sql) sql = " ".join(captured_sql)
assert r"workspace/alice/foo\\_bar/" in sql, sql assert r"workspace/alice/foo\\_bar/" in sql, sql
# owner_user_id scopes the prefix to that owner's subtree.
captured_sql.clear()
await list_workspace_directories(
parent_path="foo_bar", owner_user_id="bob",
context=_ctx("alice"), session=mock_session,
)
sql = " ".join(captured_sql)
assert r"workspace/bob/foo\\_bar/" in sql, sql
# ─── layer 3: behavioral test on real LIKE execution ────────────── # ─── layer 3: behavioral test on real LIKE execution ──────────────
+70 -15
View File
@@ -573,6 +573,7 @@ async def test_list_resources_where_clause_uses_like_prefix_and_excludes_deeper(
await list_resources( await list_resources(
parent_path="foo/bar", parent_path="foo/bar",
owner_user_id=None,
context=_resource_ctx(), context=_resource_ctx(),
session=mock_session, session=mock_session,
visibility=None, visibility=None,
@@ -581,9 +582,10 @@ async def test_list_resources_where_clause_uses_like_prefix_and_excludes_deeper(
assert len(captured_sql) == 1 assert len(captured_sql) == 1
sql = captured_sql[0].lower() sql = captured_sql[0].lower()
# 中间 ``%`` 是 owner 段通配符(跨所有 owner),父路径本身按字面匹配。 # Default (no owner_user_id) scopes to the requester's own object_key
assert "like 'w001/%%/foo/bar/%%'" in sql # subtree: LIKE w001/u001/foo/bar/% (direct children), excluding deeper.
assert "not like 'w001/%%/foo/bar/%%/%%'" in sql assert "like 'w001/u001/foo/bar/%%'" in sql
assert "not like 'w001/u001/foo/bar/%%/%%'" in sql
async def test_list_resources_where_clause_escapes_underscore() -> None: async def test_list_resources_where_clause_escapes_underscore() -> None:
@@ -596,6 +598,7 @@ async def test_list_resources_where_clause_escapes_underscore() -> None:
await list_resources( await list_resources(
parent_path="foo_bar", parent_path="foo_bar",
owner_user_id=None,
context=_resource_ctx(), context=_resource_ctx(),
session=mock_session, session=mock_session,
visibility=None, visibility=None,
@@ -605,15 +608,20 @@ async def test_list_resources_where_clause_escapes_underscore() -> None:
sql_lower = sql.lower() sql_lower = sql.lower()
# SQLAlchemy doubles the escape char inside the SQL string literal, so # SQLAlchemy doubles the escape char inside the SQL string literal, so
# the helper's ``foo\_bar`` renders as ``foo\\_bar`` in the SQL text. # the helper's ``foo\_bar`` renders as ``foo\\_bar`` in the SQL text.
assert r"like 'w001/%%/foo\\_bar/%%'" in sql_lower assert r"like 'w001/u001/foo\\_bar/%%'" in sql_lower
assert r"not like 'w001/%%/foo\\_bar/%%/%%'" in sql_lower assert r"not like 'w001/u001/foo\\_bar/%%/%%'" in sql_lower
# Both LIKE clauses declare ESCAPE '\\' (two in total). # Both LIKE clauses declare ESCAPE '\\' (two in total).
assert sql.count("ESCAPE '\\\\'") == 2, sql assert sql.count("ESCAPE '\\\\'") == 2, sql
async def test_list_resources_without_parent_path_adds_no_like_clause() -> None: async def test_list_resources_empty_parent_path_adds_root_like_clause() -> None:
"""Empty parent_path keeps the legacy workspace-wide behaviour — no """Empty parent_path still applies the directory filter (symmetric with
object_key LIKE filter at all.""" list_scripts): ``{ws_id}/{owner}/%`` AND NOT ``{ws_id}/{owner}/%/%`` so
the owner-scoped root view returns only direct children of the
requester's root, never nested descendants. Skipping the filter for
empty input used to surface nested resources at the root and visually
broke the directory tree.
"""
from backend.api.resources import list_resources from backend.api.resources import list_resources
captured_sql: list[str] = [] captured_sql: list[str] = []
@@ -621,14 +629,42 @@ async def test_list_resources_without_parent_path_adds_no_like_clause() -> None:
await list_resources( await list_resources(
parent_path="", parent_path="",
owner_user_id=None,
context=_resource_ctx(), context=_resource_ctx(),
session=mock_session, session=mock_session,
visibility=None, visibility=None,
keyword=None, keyword=None,
) )
sql = captured_sql[0].lower() sql = captured_sql[0].lower()
assert " like " not in sql assert " like 'w001/u001/%%'" in sql
assert " not like " not in sql assert " not like 'w001/u001/%%/%%'" in sql
async def test_list_resources_owner_param_scopes_to_other_owner() -> None:
"""owner_user_id=<other> scopes object_key LIKE to that owner's subtree
so the tree can lazily fetch another member's data resources on group
expand. Non-admin visibility predicate is still applied, so the other
owner's private resources are excluded (workspace/public only).
"""
from backend.api.resources import list_resources
captured_sql: list[str] = []
mock_session = _list_resources_capturing_session(captured_sql)
await list_resources(
parent_path="",
owner_user_id="U002",
context=_resource_ctx(),
session=mock_session,
visibility=None,
keyword=None,
)
sql = captured_sql[0].lower()
assert " like 'w001/u002/%%'" in sql
assert " not like 'w001/u002/%%/%%'" in sql
# non-admin: visibility predicate still present (excludes U002 private)
assert "data_resources.owner_user_id = 'u001'" in sql
assert "data_resources.visibility in ('workspace', 'public')" in sql
async def test_list_resources_joins_users_for_display_name() -> None: async def test_list_resources_joins_users_for_display_name() -> None:
@@ -641,6 +677,7 @@ async def test_list_resources_joins_users_for_display_name() -> None:
await list_resources( await list_resources(
parent_path="", parent_path="",
owner_user_id=None,
context=_resource_ctx(), context=_resource_ctx(),
session=mock_session, session=mock_session,
visibility=None, visibility=None,
@@ -745,11 +782,29 @@ def test_sqlite_parent_path_with_slash_direct_children(sqlite_object_key_table)
assert matched == ["W001/U001/data/deep/nested.csv"], matched assert matched == ["W001/U001/data/deep/nested.csv"], matched
def test_sqlite_empty_parent_path_has_no_like_filter(sqlite_object_key_table) -> None: def test_sqlite_empty_parent_path_returns_root_level_across_owners(
"""Empty parent_path → no LIKE filter → the endpoint's base WHERE only sqlite_object_key_table,
(workspace-wide active resources). Stand-in: every row is returned.""" ) -> None:
"""Empty parent_path now applies the root filter (symmetric with
list_scripts): ``{ws_id}/%/%`` AND NOT ``{ws_id}/%/%/%`` returns only
direct children of every owner's root, excluding nested descendants.
Earlier 'no LIKE' behaviour used to surface every row in the
workspace at the root, which is exactly what made scripts and data
appear mutually visible and broke the tree.
"""
engine, table = sqlite_object_key_table engine, table = sqlite_object_key_table
like = "W001/%/%"
not_like = "W001/%/%/%"
with engine.connect() as conn: with engine.connect() as conn:
rows = conn.execute(select(table.c.object_key)).fetchall() rows = conn.execute(
select(table.c.object_key).where(
table.c.object_key.like(like, escape="\\"),
~table.c.object_key.like(not_like, escape="\\"),
)
).fetchall()
matched = sorted(r[0] for r in rows) matched = sorted(r[0] for r in rows)
assert len(matched) == 8, matched # ``W001/U001/root.csv`` is the only 3-segment path (= direct child
# of the owner root); all 4+ segment paths (data/*, database/*) are
# excluded by the NOT LIKE clause. The other 4+ segment files would
# be returned when the user expands the corresponding subdirectory.
assert matched == ["W001/U001/root.csv"], matched
@@ -60,6 +60,7 @@ export function ScriptExplorer({
const loadingChildrenPaths = useScriptWorkspaceStore( const loadingChildrenPaths = useScriptWorkspaceStore(
(s) => s.loadingChildrenPaths, (s) => s.loadingChildrenPaths,
); );
const members = useScriptWorkspaceStore((s) => s.members);
const onToggle = useScriptWorkspaceStore((s) => s.toggleExpanded); const onToggle = useScriptWorkspaceStore((s) => s.toggleExpanded);
const dataByOwner = useMemo(() => { const dataByOwner = useMemo(() => {
@@ -83,9 +84,15 @@ export function ScriptExplorer({
item.visibility === "public", item.visibility === "public",
); );
// 用工作区成员列表播种分组 —— 顶层"我 / user1 / user2 / …"折叠分组
// 的来源。即使某成员尚未加载任何脚本/数据(默认折叠、点击才拉取),
// 也作为空分组出现,保证目录树结构稳定可见(修"目录树结构消失")。
const byOwner = new Map<string, ScriptItem[]>(); const byOwner = new Map<string, ScriptItem[]>();
// 当前用户的目录树即使没有脚本也要渲染,所以预置空组。 for (const m of members) {
if (user?.user_id) { if (!byOwner.has(m.user_id)) byOwner.set(m.user_id, []);
}
// 当前用户兜底(members 未就绪时仍渲染"我"的分组)。
if (user?.user_id && !byOwner.has(user.user_id)) {
byOwner.set(user.user_id, []); byOwner.set(user.user_id, []);
} }
for (const item of visibleScripts) { for (const item of visibleScripts) {
@@ -93,15 +100,17 @@ export function ScriptExplorer({
list.push(item); list.push(item);
byOwner.set(item.owner_user_id, list); byOwner.set(item.owner_user_id, list);
} }
// data-only owner(只有数据资源、没有 scripts 的用户,且不在 members 列表
// data-only owner(只有数据资源、没有 scripts 的用户)也要出现在分组里, // 里,如已移除成员遗留的资源)也要出现在分组里
// 因为 data resources 与 scripts 共享同一棵目录树。
for (const ownerUserId of dataByOwner.keys()) { for (const ownerUserId of dataByOwner.keys()) {
if (!byOwner.has(ownerUserId)) { if (!byOwner.has(ownerUserId)) {
byOwner.set(ownerUserId, []); byOwner.set(ownerUserId, []);
} }
} }
// 成员 id → display_name 优先取 members 列表(最准)。
const memberName = new Map(members.map((m) => [m.user_id, m.display_name]));
const groups: { const groups: {
user: AuthUser | null; user: AuthUser | null;
scripts: ScriptItem[]; scripts: ScriptItem[];
@@ -110,9 +119,8 @@ export function ScriptExplorer({
}[] = []; }[] = [];
for (const [ownerUserId, groupScripts] of byOwner.entries()) { for (const [ownerUserId, groupScripts] of byOwner.entries()) {
const groupDataResources = dataByOwner.get(ownerUserId) ?? []; const groupDataResources = dataByOwner.get(ownerUserId) ?? [];
// data-only owner(没有 scripts 的用户)回退到 data resources 的
// owner_display_name,否则 data-only owner 前端只能显示 userId 末 6 位。
const displayName = const displayName =
memberName.get(ownerUserId) ??
groupScripts[0]?.owner_display_name ?? groupScripts[0]?.owner_display_name ??
groupDataResources[0]?.owner_display_name ?? groupDataResources[0]?.owner_display_name ??
(ownerUserId === user?.user_id ? user?.display_name : null) ?? (ownerUserId === user?.user_id ? user?.display_name : null) ??
@@ -129,14 +137,16 @@ export function ScriptExplorer({
role_code: null, role_code: null,
is_system_admin: false, is_system_admin: false,
} as AuthUser); } as AuthUser);
const inferred = inferredDirectories(groupScripts); const inferred = inferredDirectories(groupScripts, ownerUserId);
// directories flat 数组现在按 owner_user_id 标记,按 owner 切分后与
// inferred 合并(inferred 补全 fetched 目录行未覆盖的祖先路径)。
const ownerDirs = directories.filter(
(d) => d.owner_user_id === ownerUserId,
);
groups.push({ groups.push({
user: groupUser, user: groupUser,
scripts: groupScripts, scripts: groupScripts,
directories: directories: mergeDirectories(ownerDirs, inferred),
ownerUserId === user?.user_id
? mergeDirectories(directories, inferred)
: inferred,
dataResources: groupDataResources, dataResources: groupDataResources,
}); });
} }
@@ -144,18 +154,19 @@ export function ScriptExplorer({
groups.sort((a, b) => { groups.sort((a, b) => {
if (a.user?.user_id === user?.user_id) return -1; if (a.user?.user_id === user?.user_id) return -1;
if (b.user?.user_id === user?.user_id) return 1; if (b.user?.user_id === user?.user_id) return 1;
return (a.user?.user_id ?? "").localeCompare(b.user?.user_id ?? ""); return (a.user?.display_name ?? a.user?.user_id ?? "").localeCompare(
b.user?.display_name ?? b.user?.user_id ?? "",
);
}); });
return groups; return groups;
}, [filteredScripts, directories, user, dataByOwner]); }, [filteredScripts, directories, user, dataByOwner, members]);
return ( return (
<aside className="explorer"> <aside className="explorer">
<div className="explorer__header"> <div className="explorer__header">
<div> <div>
<h2></h2> <h2></h2>
<span>{scripts.length + dataResources.length} </span>
</div> </div>
<div className="explorer__actions"> <div className="explorer__actions">
<button <button
@@ -217,6 +228,7 @@ export function ScriptExplorer({
<WorkspaceTreeGroup <WorkspaceTreeGroup
key={ownerKey} key={ownerKey}
groupKey={`__group__${ownerKey}`} groupKey={`__group__${ownerKey}`}
ownerUserId={ownerKey}
title={`${group.user?.display_name}`} title={`${group.user?.display_name}`}
scripts={group.scripts} scripts={group.scripts}
directories={group.directories} directories={group.directories}
@@ -272,14 +284,22 @@ function ownedScriptPath(item: ScriptItem) {
return item.relative_path.replaceAll("\\", "/").split("/").slice(2).join("/"); return item.relative_path.replaceAll("\\", "/").split("/").slice(2).join("/");
} }
function inferredDirectories(items: ScriptItem[]): WorkspaceDirectory[] { function inferredDirectories(
items: ScriptItem[],
ownerUserId: string,
): WorkspaceDirectory[] {
const result = new Map<string, WorkspaceDirectory>(); const result = new Map<string, WorkspaceDirectory>();
for (const item of items) { for (const item of items) {
const parts = parentOf(ownedScriptPath(item)).split("/").filter(Boolean); const parts = parentOf(ownedScriptPath(item)).split("/").filter(Boolean);
let parentPath = ""; let parentPath = "";
for (const name of parts) { for (const name of parts) {
const path = parentPath ? `${parentPath}/${name}` : name; const path = parentPath ? `${parentPath}/${name}` : name;
result.set(path, { path, name, parent_path: parentPath }); result.set(path, {
path,
name,
parent_path: parentPath,
owner_user_id: ownerUserId,
});
parentPath = path; parentPath = path;
} }
} }
+4 -3
View File
@@ -221,7 +221,8 @@ export function useApi(): WorkspaceBoundApi {
const workspaceId = currentWorkspace?.workspace_id ?? ""; const workspaceId = currentWorkspace?.workspace_id ?? "";
return useMemo<WorkspaceBoundApi>(() => ({ return useMemo<WorkspaceBoundApi>(() => ({
listScripts: (parentPath) => rawApi.listScripts(workspaceId, parentPath), listScripts: (parentPath, ownerUserId) =>
rawApi.listScripts(workspaceId, parentPath, ownerUserId),
countScripts: () => rawApi.countScripts(workspaceId), countScripts: () => rawApi.countScripts(workspaceId),
listResources: (parentPath, opts) => listResources: (parentPath, opts) =>
rawApi.listResources(workspaceId, parentPath, opts), rawApi.listResources(workspaceId, parentPath, opts),
@@ -241,8 +242,8 @@ export function useApi(): WorkspaceBoundApi {
deleteScript: (scriptId) => rawApi.deleteScript(workspaceId, scriptId), deleteScript: (scriptId) => rawApi.deleteScript(workspaceId, scriptId),
setScriptLock: (scriptId, isLocked) => setScriptLock: (scriptId, isLocked) =>
rawApi.setScriptLock(workspaceId, scriptId, isLocked), rawApi.setScriptLock(workspaceId, scriptId, isLocked),
listWorkspaceDirectories: (parentPath?: string) => listWorkspaceDirectories: (parentPath, ownerUserId) =>
rawApi.listWorkspaceDirectories(workspaceId, parentPath ?? ""), rawApi.listWorkspaceDirectories(workspaceId, parentPath ?? "", ownerUserId),
createWorkspaceDirectory: (directoryName, parentPath) => createWorkspaceDirectory: (directoryName, parentPath) =>
rawApi.createWorkspaceDirectory(workspaceId, directoryName, parentPath), rawApi.createWorkspaceDirectory(workspaceId, directoryName, parentPath),
deleteWorkspaceDirectory: (path) => deleteWorkspaceDirectory: (path) =>
@@ -28,8 +28,11 @@ type WorkspaceTreeProps = {
onCopyResourcePath?: (jupyterPath: string) => void; onCopyResourcePath?: (jupyterPath: string) => void;
// 唯一标识此 group(通常 `__group__<owner_user_id>`),让多个 owner 的 group 各自独立展开。 // 唯一标识此 group(通常 `__group__<owner_user_id>`),让多个 owner 的 group 各自独立展开。
groupKey: string; groupKey: string;
// 该 group 所属 owner 的 user_id —— 透传给 store.toggleExpanded,使他人
// 分组展开时走 loadOwnerGroup / 带 owner 的 loadScripts(懒加载)。
ownerUserId: string;
expandedPaths: Set<string>; expandedPaths: Set<string>;
onToggle: (path: string, loadPath?: string) => void; onToggle: (path: string, loadPath?: string, ownerUserId?: string) => void;
loadingChildrenPaths: Set<string>; loadingChildrenPaths: Set<string>;
}; };
@@ -79,6 +82,7 @@ export function WorkspaceTreeGroup({
dataResources, dataResources,
onCopyResourcePath, onCopyResourcePath,
groupKey, groupKey,
ownerUserId,
expandedPaths, expandedPaths,
onToggle, onToggle,
loadingChildrenPaths, loadingChildrenPaths,
@@ -125,14 +129,16 @@ export function WorkspaceTreeGroup({
parent_path: path.includes("/") parent_path: path.includes("/")
? path.split("/").slice(0, -1).join("/") ? path.split("/").slice(0, -1).join("/")
: "", : "",
owner_user_id: ownerUserId,
})); }));
}, [dataResources]); }, [dataResources, ownerUserId]);
// 首次挂载自动展开(保留原本 useState(true) 的默认展开行为)。 // 默认只展开"我"的分组(!readOnly);其他成员分组默认折叠,点击才
// toggleExpanded 内识别 `__group__` 前缀,不会触发 loadChildren。 // 按需拉取其可见内容(懒加载设计)。原本对所有 group 无条件 onToggle
// 会让所有 owner 的内容在根加载时就被全量拉取,违背"默认只拉取自己的一级"。
useEffect(() => { useEffect(() => {
if (!expandedPaths.has(groupKey)) { if (!readOnly && !expandedPaths.has(groupKey)) {
void onToggle(groupKey); void onToggle(groupKey, undefined, ownerUserId);
} }
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
}, [groupKey]); }, [groupKey]);
@@ -142,7 +148,7 @@ export function WorkspaceTreeGroup({
className={`tree-group__title${open ? " is-open" : ""}`} className={`tree-group__title${open ? " is-open" : ""}`}
type="button" type="button"
aria-expanded={open} aria-expanded={open}
onClick={() => onToggle(groupKey)} onClick={() => onToggle(groupKey, undefined, ownerUserId)}
onContextMenu={onContextMenu onContextMenu={onContextMenu
? (event) => onContextMenu(event, { kind: "root", path: "" }) ? (event) => onContextMenu(event, { kind: "root", path: "" })
: undefined} : undefined}
@@ -150,12 +156,12 @@ export function WorkspaceTreeGroup({
<Icon name="chevron" size={14} /> <Icon name="chevron" size={14} />
<Icon name="folder" size={17} /> <Icon name="folder" size={17} />
<span>{title}</span> <span>{title}</span>
<em>{scripts.length + (dataResources ?? []).length}</em>
</button> </button>
{open && ( {open && (
<div className="tree-group__items"> <div className="tree-group__items">
<WorkspaceTreeItems <WorkspaceTreeItems
groupKey={groupKey} groupKey={groupKey}
ownerUserId={ownerUserId}
path="" path=""
depth={0} depth={0}
scripts={[...scripts, ...dataResourceScripts]} scripts={[...scripts, ...dataResourceScripts]}
@@ -186,6 +192,7 @@ export function WorkspaceTreeGroup({
function WorkspaceTreeItems({ function WorkspaceTreeItems({
groupKey, groupKey,
ownerUserId,
path, path,
depth, depth,
scripts, scripts,
@@ -210,6 +217,7 @@ function WorkspaceTreeItems({
<DirectoryBranch <DirectoryBranch
key={directory.path} key={directory.path}
groupKey={groupKey} groupKey={groupKey}
ownerUserId={ownerUserId}
directory={directory} directory={directory}
depth={depth} depth={depth}
scripts={scripts} scripts={scripts}
@@ -271,6 +279,7 @@ function WorkspaceTreeItems({
function DirectoryBranch({ function DirectoryBranch({
groupKey, groupKey,
ownerUserId,
directory, directory,
depth, depth,
scripts, scripts,
@@ -294,7 +303,7 @@ function DirectoryBranch({
className="directory-row" className="directory-row"
style={{ paddingLeft: 10 + depth * 16 }} style={{ paddingLeft: 10 + depth * 16 }}
type="button" type="button"
onClick={() => onToggle(expandKey, directory.path)} onClick={() => onToggle(expandKey, directory.path, ownerUserId)}
onContextMenu={onContextMenu onContextMenu={onContextMenu
? (event) => onContextMenu(event, { ? (event) => onContextMenu(event, {
kind: "directory", kind: "directory",
@@ -312,6 +321,7 @@ function DirectoryBranch({
{open && ( {open && (
<WorkspaceTreeItems <WorkspaceTreeItems
groupKey={groupKey} groupKey={groupKey}
ownerUserId={ownerUserId}
path={directory.path} path={directory.path}
depth={depth + 1} depth={depth + 1}
scripts={scripts} scripts={scripts}
@@ -9,6 +9,7 @@ import type {
Visibility, Visibility,
WorkspaceBoundApi, WorkspaceBoundApi,
WorkspaceDirectory, WorkspaceDirectory,
WorkspaceMember,
} from "~/services/api"; } from "~/services/api";
import type { NewScriptForm } from "./uiStore"; import type { NewScriptForm } from "./uiStore";
@@ -41,11 +42,35 @@ let _previewController: AbortController | null = null;
let _previewRequest = 0; let _previewRequest = 0;
let _pythonEditorOpeningIds = new Set<string>(); let _pythonEditorOpeningIds = new Set<string>();
let _scriptCountSeq = 0; let _scriptCountSeq = 0;
// 当前登录用户 id —— 与 `_api` 一样由 layout 在 render body 绑定。
// 用于:(1) `loadScripts`/`loadOwnerGroup` 区分"我"与他人;
// (2) `toggleExpanded` 判定展开真实目录时是否需要 loadChildren(他人的
// 目录全靠脚本路径推断,不调 listWorkspaceDirectories)。
let _currentUserId: string | null = null;
// 当前工作区 id —— listWorkspaceMembers 不像 listScripts 那样把 workspaceId
// 烤进 bound api,需要显式传入,故由 layout 绑定。
let _workspaceId: string | null = null;
export const bindScriptWorkspaceApi = (api: WorkspaceBoundApi | null) => { export const bindScriptWorkspaceApi = (api: WorkspaceBoundApi | null) => {
_api = api; _api = api;
}; };
export const bindScriptWorkspaceUser = (userId: string | null) => {
_currentUserId = userId;
};
export const bindScriptWorkspaceId = (workspaceId: string | null) => {
_workspaceId = workspaceId;
};
// `loadedScriptPaths` / `loadedChildPaths` 的 key 命名空间:
// `${owner_user_id}:${parent_path}`,让"我"与他人的同名子目录缓存互不串扰。
// owner 缺省时回退当前用户,再回退字面量 "me"(仅作占位 key,不会发到后端)。
function ownerCacheKey(ownerUserId: string | undefined, path: string): string {
const owner = ownerUserId ?? _currentUserId ?? "me";
return `${owner}:${path}`;
}
export const getSessionCache = () => sessionCache; export const getSessionCache = () => sessionCache;
export const clearSessionCache = () => { export const clearSessionCache = () => {
sessionCache.clear(); sessionCache.clear();
@@ -86,10 +111,9 @@ type State = {
expandedPaths: Set<string>; expandedPaths: Set<string>;
loadingChildrenPaths: Set<string>; loadingChildrenPaths: Set<string>;
loadedChildPaths: Set<string>; loadedChildPaths: Set<string>;
// Per-parent-path script cache. Keys are user-relative parent paths; // Per-owner × parent-path script cache. Keys are namespaced
// values are scripts whose storage path lives directly under that parent. // `${owner_user_id}:${parent_path}` (see ownerCacheKey) so "我"与他人
// The flat `scripts` array above is the union (deduped by script_id) of // 的同名子目录互不串扰。flat `scripts` 数组是其并集(按 script_id 去重)。
// every cache entry that has been loaded in this session.
loadedScriptPaths: Set<string>; loadedScriptPaths: Set<string>;
loadingScriptPaths: Set<string>; loadingScriptPaths: Set<string>;
// Workspace-wide active-script total — separate from the lazy-loaded // Workspace-wide active-script total — separate from the lazy-loaded
@@ -98,6 +122,14 @@ type State = {
scriptCount: number | null; scriptCount: number | null;
scriptCountLoading: boolean; scriptCountLoading: boolean;
// 工作区成员列表 —— 目录树顶层"我 / user1 / user2 / …"折叠分组的来源。
// 默认只加载"我"的一级目录;其他成员分组折叠,点击才按需拉取
// 其可见(workspace/public)内容、排除其 private。
members: WorkspaceMember[];
// 已拉取根级内容的 ownerloadOwnerGroup 标记),避免重复拉取。
loadedOwnerGroups: Set<string>;
loadingOwnerGroups: Set<string>;
// 只读内容刷新版本号(用于触发已打开标签页的内容刷新) // 只读内容刷新版本号(用于触发已打开标签页的内容刷新)
readOnlyRefreshVersion: number; readOnlyRefreshVersion: number;
@@ -106,7 +138,7 @@ type State = {
setKeyword: (keyword: string) => void; setKeyword: (keyword: string) => void;
reset: () => void; reset: () => void;
load: (silent?: boolean) => Promise<void>; load: (silent?: boolean) => Promise<void>;
loadDataResources: (parentPath?: string) => Promise<void>; loadDataResources: (parentPath?: string, ownerUserId?: string) => Promise<void>;
selectScript: (id: string | null) => void; selectScript: (id: string | null) => void;
openTab: (id: string) => void; openTab: (id: string) => void;
closeTab: (id: string, event?: { stopPropagation: () => void }) => Promise<void>; closeTab: (id: string, event?: { stopPropagation: () => void }) => Promise<void>;
@@ -132,11 +164,16 @@ type State = {
deleteScript: (script: ScriptItem) => Promise<void>; deleteScript: (script: ScriptItem) => Promise<void>;
deleteDataResource: (resourceId: string) => Promise<void>; deleteDataResource: (resourceId: string) => Promise<void>;
deleteDirectory: (path: string) => Promise<void>; deleteDirectory: (path: string) => Promise<void>;
toggleExpanded: (path: string, loadPath?: string) => Promise<void>; toggleExpanded: (path: string, loadPath?: string, ownerUserId?: string) => Promise<void>;
loadChildren: (parentPath: string) => Promise<void>; loadChildren: (parentPath: string, ownerUserId?: string) => Promise<void>;
// Lazy-load scripts directly under `parentPath`. Idempotent — repeated // Lazy-load scripts directly under `parentPath` for `ownerUserId`(缺省
// calls for an already-loaded path are no-ops; in-flight calls dedupe. // = 当前用户)。Idempotent — repeated calls for an already-loaded
loadScripts: (parentPath: string) => Promise<void>; // owner×path are no-ops; in-flight calls dedupe.
loadScripts: (parentPath: string, ownerUserId?: string) => Promise<void>;
// 按需拉取某成员的根级可见脚本 + 数据资源(点击其折叠分组时触发)。
// Always fetches (refresh-safe); toggleExpanded 的分组头分支负责守门
// 避免重复拉取,loadedOwnerGroups 标记已加载状态。
loadOwnerGroup: (ownerUserId: string) => Promise<void>;
// Fetch the workspace-wide active-script total. Cheap; the dashboard // Fetch the workspace-wide active-script total. Cheap; the dashboard
// uses this for its hero count so it doesn't depend on lazy-loaded state. // uses this for its hero count so it doesn't depend on lazy-loaded state.
loadScriptCount: () => Promise<void>; loadScriptCount: () => Promise<void>;
@@ -227,6 +264,10 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
scriptCount: null, scriptCount: null,
scriptCountLoading: false, scriptCountLoading: false,
members: [],
loadedOwnerGroups: new Set<string>(),
loadingOwnerGroups: new Set<string>(),
readOnlyRefreshVersion: 0, readOnlyRefreshVersion: 0,
setApiOnline: (online) => set({ apiOnline: online }), setApiOnline: (online) => set({ apiOnline: online }),
@@ -269,6 +310,9 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
loadingScriptPaths: new Set<string>(), loadingScriptPaths: new Set<string>(),
scriptCount: null, scriptCount: null,
scriptCountLoading: false, scriptCountLoading: false,
members: [],
loadedOwnerGroups: new Set<string>(),
loadingOwnerGroups: new Set<string>(),
}); });
}, },
@@ -277,54 +321,86 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
if (!silent) set({ loading: true }); if (!silent) set({ loading: true });
set({ refreshing: silent }); set({ refreshing: silent });
try { try {
// Re-fetch every path currently in the cache. On initial mount the const me = _currentUserId;
// cache is empty so this degrades to a single root fetch; on // 拉取工作区成员列表 —— 顶层"我 / user1 / user2 / …"折叠分组的来源。
// toolbar refresh / after createFolder / after deleteScript the const memberList =
// previously-expanded folders are also re-fetched so the UI stays _workspaceId != null
// consistent (otherwise the cache would keep stale "loaded" ? await api
// markers while the corresponding scripts had been replaced by .listWorkspaceMembers(_workspaceId)
// the root-only payload, leaving subfolders empty on re-expand). .catch(() => [] as WorkspaceMember[])
const cachedScriptPaths = Array.from(get().loadedScriptPaths); : [];
const cachedChildPaths = Array.from(get().loadedChildPaths).filter( // 只重新拉取"我"的已缓存脚本路径(含根)。首次挂载缓存为空 → 退化为
(p) => p !== "", // 单次根级拉取。他人脚本不动(保留在 flat scripts 里,见下方合并)。
const myCachedScriptKeys = Array.from(get().loadedScriptPaths).filter(
(k) => k.startsWith(`${me}:`) || k.startsWith("me:"),
); );
const rootScriptKey = ownerCacheKey(undefined, "");
const scriptFetches = const scriptFetches =
cachedScriptPaths.length > 0 myCachedScriptKeys.length > 0
? cachedScriptPaths.map((p) => ? myCachedScriptKeys.map((k) => {
api.listScripts(p).catch(() => [] as ScriptItem[]), const p = k.slice(k.indexOf(":") + 1);
) return api.listScripts(p).catch(() => [] as ScriptItem[]);
: [api.listScripts("")]; })
const dirFetches = [ : [api.listScripts("").catch(() => [] as ScriptItem[])];
api.listWorkspaceDirectories(""), // 只重新拉取"我"的已缓存目录路径(含根)。他人目录不动(保留在
...cachedChildPaths.map((p) => // flat directories 里,按 (owner,path) 去重合并)。
api.listWorkspaceDirectories(p).catch(() => [] as WorkspaceDirectory[]), const myCachedDirKeys = Array.from(get().loadedChildPaths).filter(
), (k) => (me != null && k.startsWith(`${me}:`)) || k.startsWith("me:"),
]; );
const rootDirKey = ownerCacheKey(undefined, "");
const dirFetches =
myCachedDirKeys.length > 0
? myCachedDirKeys.map((k) => {
const p = k.slice(k.indexOf(":") + 1);
return api
.listWorkspaceDirectories(p)
.catch(() => [] as WorkspaceDirectory[]);
})
: [api
.listWorkspaceDirectories("")
.catch(() => [] as WorkspaceDirectory[])];
const [scriptLists, dirLists] = await Promise.all([ const [scriptLists, dirLists] = await Promise.all([
Promise.all(scriptFetches), Promise.all(scriptFetches),
Promise.all(dirFetches), Promise.all(dirFetches),
]); ]);
const freshScripts = scriptLists.flat(); const myFreshScripts = scriptLists.flat();
const freshDirs = dirLists.flat(); // "我"的脚本用 fresh 集合替换;他人脚本原样保留(按 script_id 去重合并)。
// Dedup: later occurrences win so fresh per-path payloads override const otherScripts = get().scripts.filter(
// any duplicates coming through different fetch slots. (s) => s.owner_user_id !== me,
);
const dedupedScripts = Array.from( const dedupedScripts = Array.from(
new Map(freshScripts.map((s) => [s.script_id, s])).values(), new Map(
[...otherScripts, ...myFreshScripts].map((s) => [s.script_id, s]),
).values(),
);
// "我"的目录用 fresh 集合替换;他人目录原样保留(按 (owner,path) 去重)。
const otherDirs = get().directories.filter(
(d) => d.owner_user_id !== me,
); );
const dedupedDirs = Array.from( const dedupedDirs = Array.from(
new Map(freshDirs.map((d) => [d.path, d])).values(), new Map(
[...otherDirs, ...dirLists.flat()].map((d) => [
`${d.owner_user_id}:${d.path}`,
d,
]),
).values(),
); );
const nextLoadedScripts = new Set(cachedScriptPaths); const nextLoadedScripts = new Set(myCachedScriptKeys);
nextLoadedScripts.add(""); nextLoadedScripts.add(rootScriptKey);
const nextLoadedChildren = new Set(get().loadedChildPaths); const nextLoadedChildren = new Set(get().loadedChildPaths);
nextLoadedChildren.add(""); nextLoadedChildren.add(rootDirKey);
set({ set({
scripts: dedupedScripts, scripts: dedupedScripts,
directories: dedupedDirs, directories: dedupedDirs,
members: memberList,
apiOnline: true, apiOnline: true,
loadedScriptPaths: nextLoadedScripts, loadedScriptPaths: nextLoadedScripts,
loadedChildPaths: nextLoadedChildren, loadedChildPaths: nextLoadedChildren,
}); });
// 刷新已展开的其他成员分组(loadOwnerGroup 总是发起请求,刷新安全)。
for (const owner of get().loadedOwnerGroups) {
if (owner !== me) void get().loadOwnerGroup(owner);
}
const validIds = new Set(dedupedScripts.map((item) => item.script_id)); const validIds = new Set(dedupedScripts.map((item) => item.script_id));
const currentSelected = get().selectedId; const currentSelected = get().selectedId;
if (!currentSelected || !validIds.has(currentSelected)) { if (!currentSelected || !validIds.has(currentSelected)) {
@@ -347,33 +423,37 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
} }
}, },
loadScripts: async (parentPath) => { loadScripts: async (parentPath, ownerUserId) => {
const api = requireApi(); const api = requireApi();
if (get().loadedScriptPaths.has(parentPath)) return; const cacheKey = ownerCacheKey(ownerUserId, parentPath);
// Dedupe in-flight requests for the same path. if (get().loadedScriptPaths.has(cacheKey)) return;
if (get().loadingScriptPaths.has(parentPath)) return; // Dedupe in-flight requests for the same owner×path.
if (get().loadingScriptPaths.has(cacheKey)) return;
const next = new Set(get().loadingScriptPaths); const next = new Set(get().loadingScriptPaths);
next.add(parentPath); next.add(cacheKey);
set({ loadingScriptPaths: next }); set({ loadingScriptPaths: next });
try { try {
const items = await api.listScripts(parentPath); const items = await api.listScripts(parentPath, ownerUserId);
set((state) => { set((state) => {
const existingIds = new Set(state.scripts.map((s) => s.script_id)); // append-only 合并(按 script_id 去重,fresh 覆盖 stale 同 id 值)。
const fresh = items.filter((s) => !existingIds.has(s.script_id)); // 该 owner×path 的全量刷新由 load()(我)/ loadOwnerGroup(他人)
// 负责 drop-by-owner 后重并入;这里是子目录展开,append 即可。
const byId = new Map(state.scripts.map((s) => [s.script_id, s]));
for (const item of items) byId.set(item.script_id, item);
const nextLoaded = new Set(state.loadedScriptPaths); const nextLoaded = new Set(state.loadedScriptPaths);
nextLoaded.add(parentPath); nextLoaded.add(cacheKey);
return { return {
scripts: [...state.scripts, ...fresh], scripts: Array.from(byId.values()),
loadedScriptPaths: nextLoaded, loadedScriptPaths: nextLoaded,
loadingScriptPaths: new Set( loadingScriptPaths: new Set(
[...state.loadingScriptPaths].filter((p) => p !== parentPath), [...state.loadingScriptPaths].filter((p) => p !== cacheKey),
), ),
}; };
}); });
} catch (error) { } catch (error) {
set((state) => ({ set((state) => ({
loadingScriptPaths: new Set( loadingScriptPaths: new Set(
[...state.loadingScriptPaths].filter((p) => p !== parentPath), [...state.loadingScriptPaths].filter((p) => p !== cacheKey),
), ),
})); }));
pushToast( pushToast(
@@ -383,14 +463,96 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
} }
}, },
loadDataResources: async (parentPath = "") => { loadOwnerGroup: async (ownerUserId) => {
const api = requireApi();
if (get().loadingOwnerGroups.has(ownerUserId)) return;
const nextLoading = new Set(get().loadingOwnerGroups);
nextLoading.add(ownerUserId);
set({ loadingOwnerGroups: nextLoading });
try {
const [scripts, resources, directories] = await Promise.all([
api.listScripts("", ownerUserId).catch(() => [] as ScriptItem[]),
api.listResources("", { ownerUserId }).catch(() => [] as ResourceItem[]),
api.listWorkspaceDirectories("", ownerUserId).catch(
() => [] as WorkspaceDirectory[],
),
]);
set((state) => {
// 丢弃该 owner 的旧脚本/资源/目录(按 owner 过滤后保留他人),再并入 fresh。
const keptScripts = state.scripts.filter(
(s) => s.owner_user_id !== ownerUserId,
);
const keptResources = state.dataResources.filter(
(r) => r.owner_user_id !== ownerUserId,
);
const keptDirs = state.directories.filter(
(d) => d.owner_user_id !== ownerUserId,
);
const scriptIds = new Set(keptScripts.map((s) => s.script_id));
const freshScripts = scripts.filter((s) => !scriptIds.has(s.script_id));
const resourceIds = new Set(keptResources.map((r) => r.resource_id));
const freshResources = resources.filter(
(r) => !resourceIds.has(r.resource_id),
);
const dirIds = new Set(
keptDirs.map((d) => `${d.owner_user_id}:${d.path}`),
);
const freshDirs = directories.filter(
(d) => !dirIds.has(`${d.owner_user_id}:${d.path}`),
);
const nextLoaded = new Set(state.loadedOwnerGroups);
nextLoaded.add(ownerUserId);
const nextScriptPaths = new Set(state.loadedScriptPaths);
nextScriptPaths.add(ownerCacheKey(ownerUserId, ""));
const nextChildPaths = new Set(state.loadedChildPaths);
nextChildPaths.add(ownerCacheKey(ownerUserId, ""));
return {
scripts: [...keptScripts, ...freshScripts],
dataResources: [...keptResources, ...freshResources],
directories: [...keptDirs, ...freshDirs],
loadedOwnerGroups: nextLoaded,
loadedScriptPaths: nextScriptPaths,
loadedChildPaths: nextChildPaths,
loadingOwnerGroups: new Set(
[...state.loadingOwnerGroups].filter((o) => o !== ownerUserId),
),
};
});
} catch {
set((state) => ({
loadingOwnerGroups: new Set(
[...state.loadingOwnerGroups].filter((o) => o !== ownerUserId),
),
}));
}
},
loadDataResources: async (parentPath = "", ownerUserId) => {
const api = requireApi(); const api = requireApi();
set({ dataResourcesLoading: true }); set({ dataResourcesLoading: true });
try { try {
const list = await api.listResources(parentPath); const list = await api.listResources(parentPath, { ownerUserId });
set({ dataResources: Array.isArray(list) ? list : [] }); const fresh = Array.isArray(list) ? list : [];
set((state) => {
// 按 owner 范围合并:丢弃该 owner 的旧资源再并入 freshfresh 覆盖
// 同 id)。owner 缺省=我,故根加载/刷新我的数据时替换我的一级资源。
const targetOwner = ownerUserId ?? _currentUserId ?? null;
const kept = state.dataResources.filter(
(r) => r.owner_user_id !== targetOwner,
);
const byId = new Map(kept.map((r) => [r.resource_id, r]));
for (const item of fresh) byId.set(item.resource_id, item);
return { dataResources: Array.from(byId.values()) };
});
} catch { } catch {
set({ dataResources: [] }); set((state) => {
const targetOwner = ownerUserId ?? _currentUserId ?? null;
return {
dataResources: state.dataResources.filter(
(r) => r.owner_user_id !== targetOwner,
),
};
});
} finally { } finally {
set({ dataResourcesLoading: false }); set({ dataResourcesLoading: false });
} }
@@ -420,32 +582,40 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
} }
}, },
loadChildren: async (parentPath) => { loadChildren: async (parentPath, ownerUserId) => {
const api = requireApi(); const api = requireApi();
if (get().loadedChildPaths.has(parentPath)) return; const cacheKey = ownerCacheKey(ownerUserId, parentPath);
if (get().loadedChildPaths.has(cacheKey)) return;
const next = new Set(get().loadingChildrenPaths); const next = new Set(get().loadingChildrenPaths);
next.add(parentPath); next.add(cacheKey);
set({ loadingChildrenPaths: next }); set({ loadingChildrenPaths: next });
try { try {
const children = await api.listWorkspaceDirectories(parentPath); const children = await api.listWorkspaceDirectories(parentPath, ownerUserId);
set((state) => { set((state) => {
const nextLoaded = new Set(state.loadedChildPaths); const nextLoaded = new Set(state.loadedChildPaths);
nextLoaded.add(parentPath); nextLoaded.add(cacheKey);
// 丢弃该 owner 该 parent 下的旧目录行,再并入 fresh(按 (owner,path) 去重)。
const targetOwner = ownerUserId ?? _currentUserId ?? null;
const trimmed = state.directories.filter( const trimmed = state.directories.filter(
(d) => d.parent_path !== parentPath, (d) =>
!(d.owner_user_id === targetOwner && d.parent_path === parentPath),
); );
const byId = new Map(
trimmed.map((d) => [`${d.owner_user_id}:${d.path}`, d]),
);
for (const c of children) byId.set(`${c.owner_user_id}:${c.path}`, c);
return { return {
directories: [...trimmed, ...children], directories: Array.from(byId.values()),
loadedChildPaths: nextLoaded, loadedChildPaths: nextLoaded,
loadingChildrenPaths: new Set( loadingChildrenPaths: new Set(
[...state.loadingChildrenPaths].filter((p) => p !== parentPath), [...state.loadingChildrenPaths].filter((p) => p !== cacheKey),
), ),
}; };
}); });
} catch (error) { } catch (error) {
set((state) => ({ set((state) => ({
loadingChildrenPaths: new Set( loadingChildrenPaths: new Set(
[...state.loadingChildrenPaths].filter((p) => p !== parentPath), [...state.loadingChildrenPaths].filter((p) => p !== cacheKey),
), ),
})); }));
pushToast( pushToast(
@@ -455,7 +625,7 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
} }
}, },
toggleExpanded: async (path, loadPath) => { toggleExpanded: async (path, loadPath, ownerUserId) => {
const state = get(); const state = get();
const isOpen = state.expandedPaths.has(path); const isOpen = state.expandedPaths.has(path);
const next = new Set(state.expandedPaths); const next = new Set(state.expandedPaths);
@@ -463,16 +633,30 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
next.delete(path); next.delete(path);
} else { } else {
next.add(path); next.add(path);
// group key (`__group__<owner_id>`) 不是 workspace 路径,不调 loadChildren / loadScripts const me = _currentUserId;
const actualLoadPath = loadPath ?? path; // 用 `loadPath === undefined` 区分分组头与真实目录,而不是用
if (!actualLoadPath.startsWith("__group__")) { // `path.startsWith("__group__")`:目录的 expandKey 是
// Load both sub-directories and scripts directly under this folder // `${groupKey}/${dir.path}` 即 `__group__<owner>/dir`,同样以
// in parallel. Both are idempotent + cached; cheap when already loaded. // `__group__` 开头,前缀判断会把子目录点击误当成分组头,导致
if (!state.loadedChildPaths.has(actualLoadPath)) { // 既不调 loadChildren 也不调 loadScripts"子目录点击不触发接口")。
void get().loadChildren(actualLoadPath); // 分组头 always 传 loadPath=undefined;目录 always 传 loadPath=dir.path。
if (loadPath === undefined) {
// 分组头:他人分组首次展开 → loadOwnerGroup 按需拉取其根级可见
// 脚本+数据+目录(守门去重)。仅翻转 expand;真实子目录的懒加载
// 由目录分支(loadPath !== undefined)负责。
if (ownerUserId && ownerUserId !== me && !state.loadedOwnerGroups.has(ownerUserId)) {
void get().loadOwnerGroup(ownerUserId);
} }
if (!state.loadedScriptPaths.has(actualLoadPath)) { } else {
void get().loadScripts(actualLoadPath); // 真实目录展开:loadChildrenowner 限定的显式目录行)+ loadScripts
// 并行。两者都 idempotent + 缓存;ownerUserId 缺省=我。他人目录同样
// 调 loadChildren(owner) 拉取其目录结构,否则嵌套子目录无法被发现
// list_scripts 非递归,只能看到直接子脚本)。
if (!state.loadedChildPaths.has(ownerCacheKey(ownerUserId, loadPath))) {
void get().loadChildren(loadPath, ownerUserId);
}
if (!state.loadedScriptPaths.has(ownerCacheKey(ownerUserId, loadPath))) {
void get().loadScripts(loadPath, ownerUserId);
} }
} }
} }
@@ -1028,16 +1212,25 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
if (parentPath === "") { if (parentPath === "") {
await get().load(true); await get().load(true);
} else { } else {
const me = _currentUserId ?? "me";
const parentKey = ownerCacheKey(me, parentPath);
set((state) => { set((state) => {
const nextLoaded = new Set(state.loadedChildPaths); const nextLoaded = new Set(state.loadedChildPaths);
for (const p of state.loadedChildPaths) { for (const p of state.loadedChildPaths) {
if (p.startsWith(`${parentPath}/`)) nextLoaded.delete(p); // 仅失效"我"在该 parent 之下的缓存(namespaced key)。
if (p.startsWith(`${me}:`) && p.slice(me.length + 1).startsWith(`${parentPath}/`)) {
nextLoaded.delete(p);
}
} }
nextLoaded.delete(parentPath); nextLoaded.delete(parentKey);
return { return {
loadedChildPaths: nextLoaded, loadedChildPaths: nextLoaded,
directories: state.directories.filter( directories: state.directories.filter(
(d) => !d.parent_path.startsWith(`${parentPath}/`), (d) =>
!(
d.owner_user_id === me
&& d.parent_path.startsWith(`${parentPath}/`)
),
), ),
expandedPaths: new Set(state.expandedPaths), expandedPaths: new Set(state.expandedPaths),
}; };
@@ -1124,20 +1317,30 @@ export const useScriptWorkspaceStore = create<State>((set, get) => {
) { ) {
get().selectScript(null); get().selectScript(null);
} }
const me = _currentUserId ?? "me";
const pathKey = ownerCacheKey(me, path);
set((state) => { set((state) => {
const nextLoaded = new Set(state.loadedChildPaths); const nextLoaded = new Set(state.loadedChildPaths);
const nextExpanded = new Set(state.expandedPaths); const nextExpanded = new Set(state.expandedPaths);
for (const p of state.loadedChildPaths) { for (const p of state.loadedChildPaths) {
if (p === path || p.startsWith(`${path}/`)) nextLoaded.delete(p); // 仅失效"我"该 path 及其子目录的缓存(namespaced key)。
if (!p.startsWith(`${me}:`)) continue;
const bare = p.slice(me.length + 1);
if (bare === path || bare.startsWith(`${path}/`)) nextLoaded.delete(p);
} }
for (const p of state.expandedPaths) { for (const p of state.expandedPaths) {
if (p === path || p.startsWith(`${path}/`)) nextExpanded.delete(p); if (p === path || p.startsWith(`${path}/`)) nextExpanded.delete(p);
} }
void pathKey;
return { return {
loadedChildPaths: nextLoaded, loadedChildPaths: nextLoaded,
expandedPaths: nextExpanded, expandedPaths: nextExpanded,
directories: state.directories.filter( directories: state.directories.filter(
(d) => d.parent_path !== path && !d.parent_path.startsWith(`${path}/`), (d) =>
!(
d.owner_user_id === me
&& (d.parent_path === path || d.parent_path.startsWith(`${path}/`))
),
), ),
}; };
}); });
+9
View File
@@ -9,6 +9,8 @@ import { useApi, useAuth } from "../context/AuthContext";
import { useEditSessionLifecycle } from "../features/platform/hooks/useEditSessionLifecycle"; import { useEditSessionLifecycle } from "../features/platform/hooks/useEditSessionLifecycle";
import { import {
bindScriptWorkspaceApi, bindScriptWorkspaceApi,
bindScriptWorkspaceId,
bindScriptWorkspaceUser,
editSessionHandle, editSessionHandle,
useScriptWorkspaceStore, useScriptWorkspaceStore,
} from "../features/platform/state/scriptWorkspaceStore"; } from "../features/platform/state/scriptWorkspaceStore";
@@ -88,11 +90,18 @@ function AuthenticatedLayout() {
bindScriptWorkspaceApi(api); bindScriptWorkspaceApi(api);
bindSchedulesApi(api); bindSchedulesApi(api);
bindAdminApi(api); bindAdminApi(api);
// 当前用户 id + 工作区 id 同步绑定到 script workspace storerender body
// 与 bindScriptWorkspaceApi 同理)。store 的 lazy 跨 owner 逻辑据此区分
// "我"与他人、并调用需要显式 workspaceId 的 listWorkspaceMembers。
bindScriptWorkspaceUser(user?.user_id ?? null);
bindScriptWorkspaceId(currentWorkspace?.workspace_id ?? null);
useEffect(() => { useEffect(() => {
return () => { return () => {
bindScriptWorkspaceApi(null); bindScriptWorkspaceApi(null);
bindSchedulesApi(null); bindSchedulesApi(null);
bindAdminApi(null); bindAdminApi(null);
bindScriptWorkspaceUser(null);
bindScriptWorkspaceId(null);
}; };
}, []); }, []);
+35 -15
View File
@@ -191,6 +191,7 @@ export type WorkspaceDirectory = {
path: string; path: string;
name: string; name: string;
parent_path: string; parent_path: string;
owner_user_id: string;
has_children?: boolean; has_children?: boolean;
}; };
@@ -287,13 +288,21 @@ async function apiRequest<T>(
export async function listScripts( export async function listScripts(
workspaceId: string, workspaceId: string,
parentPath: string = "", parentPath: string = "",
ownerUserId?: string,
): Promise<ScriptItem[]> { ): Promise<ScriptItem[]> {
// Empty parentPath omits the query string entirely so the backend's // Default (no ownerUserId) scopes to the requester's own subtree; passing
// root-level filter is applied symmetrically with non-empty paths. // ownerUserId scopes to that owner's subtree (workspace/public only — the
const query = parentPath // backend excludes their private) so the tree can lazily fetch another
? `?parent_path=${encodeURIComponent(parentPath)}` // member's content when their group is expanded.
: ""; const parameters = new URLSearchParams();
return apiRequest<ScriptItem[]>(`/api/v1/scripts${query}`, {}, workspaceId); if (parentPath) parameters.set("parent_path", parentPath);
if (ownerUserId) parameters.set("owner_user_id", ownerUserId);
const query = parameters.toString();
return apiRequest<ScriptItem[]>(
`/api/v1/scripts${query ? `?${query}` : ""}`,
{},
workspaceId,
);
} }
export async function countScripts( export async function countScripts(
@@ -439,15 +448,16 @@ export type ResourceItem = {
export async function listResources( export async function listResources(
workspaceId: string, workspaceId: string,
parentPath: string = "", parentPath: string = "",
opts?: { visibility?: string; keyword?: string }, opts?: { visibility?: string; keyword?: string; ownerUserId?: string },
): Promise<ResourceItem[]> { ): Promise<ResourceItem[]> {
// Empty parentPath omits the query string entirely so the backend's // Default (no ownerUserId) scopes to the requester's own object_key
// workspace-wide (root-level) filter is applied symmetrically with // subtree; passing ownerUserId scopes to that owner's subtree so the tree
// non-empty paths, matching listScripts. // can lazily fetch another member's data resources on group expand.
const parameters = new URLSearchParams(); const parameters = new URLSearchParams();
if (parentPath) parameters.set("parent_path", parentPath); if (parentPath) parameters.set("parent_path", parentPath);
if (opts?.visibility) parameters.set("visibility", opts.visibility); if (opts?.visibility) parameters.set("visibility", opts.visibility);
if (opts?.keyword) parameters.set("keyword", opts.keyword); if (opts?.keyword) parameters.set("keyword", opts.keyword);
if (opts?.ownerUserId) parameters.set("owner_user_id", opts.ownerUserId);
const query = parameters.toString(); const query = parameters.toString();
// apiRequest<T> already unwraps the envelope's `data` field, so we // apiRequest<T> already unwraps the envelope's `data` field, so we
// request `ResourceItem[]` directly here (matching listScripts). // request `ResourceItem[]` directly here (matching listScripts).
@@ -613,12 +623,18 @@ export async function deleteScript(
export async function listWorkspaceDirectories( export async function listWorkspaceDirectories(
workspaceId: string, workspaceId: string,
parentPath: string = "", parentPath: string = "",
ownerUserId?: string,
): Promise<WorkspaceDirectory[]> { ): Promise<WorkspaceDirectory[]> {
const query = parentPath // Default (no ownerUserId) scopes to the requester's own subtree; passing
? `?parent_path=${encodeURIComponent(parentPath)}` // ownerUserId scopes to that owner so the tree can lazily render their
: ""; // directory structure on expand. Directories are structural rows; file
// visibility is still enforced by the scripts/data-resources endpoints.
const parameters = new URLSearchParams();
if (parentPath) parameters.set("parent_path", parentPath);
if (ownerUserId) parameters.set("owner_user_id", ownerUserId);
const query = parameters.toString();
const data = await apiRequest<{ directories: WorkspaceDirectory[] }>( const data = await apiRequest<{ directories: WorkspaceDirectory[] }>(
`/api/v1/workspace-directories${query}`, `/api/v1/workspace-directories${query ? `?${query}` : ""}`,
{}, {},
workspaceId, workspaceId,
); );
@@ -1487,6 +1503,7 @@ export async function getScheduleNodeRunArtifacts(
export type WorkspaceBoundApi = { export type WorkspaceBoundApi = {
listScripts: ( listScripts: (
parentPath?: Parameters<typeof listScripts>[1], parentPath?: Parameters<typeof listScripts>[1],
ownerUserId?: Parameters<typeof listScripts>[2],
) => Promise<ScriptItem[]>; ) => Promise<ScriptItem[]>;
countScripts: () => Promise<number>; countScripts: () => Promise<number>;
listResources: ( listResources: (
@@ -1527,7 +1544,10 @@ export type WorkspaceBoundApi = {
scriptId: string, scriptId: string,
isLocked: boolean, isLocked: boolean,
) => Promise<ScriptItem>; ) => Promise<ScriptItem>;
listWorkspaceDirectories: (parentPath?: string) => Promise<WorkspaceDirectory[]>; listWorkspaceDirectories: (
parentPath?: Parameters<typeof listWorkspaceDirectories>[1],
ownerUserId?: Parameters<typeof listWorkspaceDirectories>[2],
) => Promise<WorkspaceDirectory[]>;
createWorkspaceDirectory: ( createWorkspaceDirectory: (
directoryName: string, directoryName: string,
parentPath?: string, parentPath?: string,