tao.chen/model-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(scripts): 跨 owner 懒加载目录树 + 跨用户可见 workspace/public

Browse Source 
修两个后端接口问题:
1) /api/v1/workspace-directories 返回为空,目录树结构消失
2) 同 workspace 内脚本/数据互相可见但默认排除 private

后端改动
--------
* list_scripts / list_resources / list_workspace_directories 新增
  owner_user_id 可选 query 参数;缺省 = 当前请求者本人(scope 到
  workspace/{me}/...),传值时 scope 到该 owner 的子树。前端根加载
  默认只见自己一级,其他成员以折叠分组呈现。
* visibility 过滤统一:非 admin 请求者只返回 owner==me 或
  visibility ∈ {workspace, public};admin 跳过。owner=me 含自己
  的 private,owner=other 只剩其 workspace/public,排除他人 private。
* create_workspace_directory 两个分支 visibility 默认 'public'
  (非 private),使跨 owner 目录树可见;响应新增 owner_user_id 字段。
* platform.list_members 鉴权从 system_admin_context 放宽为
  系统管理员或该 workspace 活跃成员(让普通用户也能渲染同
  workspace 成员名册,用于跨 owner 分组)。
* main.py 注册 platform 模块(随 list_members 改动补齐导入)。
* .env.example 同步 common/config.py 26 个字段。

前端改动
--------
* ScriptExplorer.memberScriptGroups 改由 members 列表播种分组,
  display_name 取 members.display_name;inferredDirectories 现在按
  owner_user_id 标记,统一跨 owner 目录渲染。删除脚本目录页头与
  树分组标题的工作副本数量角标。
* WorkspaceTree 新增 ownerUserId 透传到 store.toggleExpanded;
  仅"我"的分组 mount 时 auto-expand,他人分组默认折叠,展开才
  调 loadOwnerGroup / owner-scoped loadScripts / loadChildren。
* scriptWorkspaceStore 引入 namespaced cache key
  (ownerCacheKey = `${ownerUserId ?? me}:${path}`),loadedScriptPaths
  / loadedChildPaths / loadedOwnerGroups 全部按 owner 隔离;
  toggleExpanded 用 loadPath === undefined 区分 group 头与真实
  目录,修"他人子目录点击不触发接口"的 loadPath 前缀误判 bug。
* api.ts / AuthContext 透传 ownerUserId 给 listScripts /
  listResources / listWorkspaceDirectories。

文档
----
* API.md: §3.2 创建目录 visibility 默认 public + 响应加 owner_user_id;
  §3.3.1 GET directories 加 owner_user_id 参数 + 响应字段;
  §3.4 GET scripts 改写为 owner 作用域 + visibility 过滤语义;
  §五.1 GET data-resources 新增,同一套统一语义;
  §7 intro 例外 — GET members 对系统管理员或 workspace 活跃成员开放。
* DEVELOP.md: Code layout 重写以反映 backend api/services/clients/
  schemas 拆分 + schedule domain/scheduling/application/execution/
  infrastructure 拆分 + common 子包(auth/storage/backends);
  Configuration 系统补全 26 个 settings 字段;新增
  "Owner-scoping + visibility (cross-owner browsing)" 小节;
  Per-service dev 注释用 uv run 的源布局要求;Add a new DAG endpoint /
  storage bucket 路径改为 backend/src/backend/api/* 与 services/*。

测试
----
* test_list_scripts_parent_path.py /
  test_resources.py 补充 owner_user_id 参数化直接调用 + LIKE
  前缀断言(workspace/{owner}/... 前缀)。

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
tao.chen
2026-09-02 10:10:41 +08:00
committed by tao.chen
co-authored by Claude
parent 256f369661
commit cb0205fcd2
15 changed files with 929 additions and 303 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/tests/test_list_scripts_parent_path.py
+126 -27
View File
@@ -28,7 +28,7 @@ from sqlalchemy import Column, MetaData, String, Table, create_engine, select, t
from sqlalchemy.dialects import mysql as mysql_dialect
from backend.api.scripts import (
_build_list_scripts_descendant_prefix,
_build_list_scripts_owner_descendant_prefix,
_build_list_scripts_workspace_descendant_prefix,
_escape_like_pattern,
normalize_user_path,
@@ -85,14 +85,14 @@ class TestEscapeLikePattern:
def test_descendant_prefix_root() -> None:
"""Empty parent_path → descendant prefix is the scoped root + '/'."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "")
"""Empty parent_path → descendant prefix is the owner-scoped root + '/'."""
prefix = _build_list_scripts_owner_descendant_prefix("alice", "")
assert prefix == "workspace/alice/"
def test_descendant_prefix_subdir() -> None:
"""Non-empty parent_path → appended under the scoped root."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "foo/bar")
"""Non-empty parent_path → appended under the owner-scoped root."""
prefix = _build_list_scripts_owner_descendant_prefix("alice", "foo/bar")
assert prefix == "workspace/alice/foo/bar/"
@@ -100,18 +100,18 @@ def test_descendant_prefix_escapes_metachars() -> None:
"""Folder name ``foo_bar`` MUST produce ``foo\\_bar`` in the prefix
so the trailing ``%`` doesn't become 'match any single char before
b'."""
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "foo_bar")
prefix = _build_list_scripts_owner_descendant_prefix("alice", "foo_bar")
assert prefix == r"workspace/alice/foo\_bar/"
def test_descendant_prefix_normalizes_leading_trailing_slashes() -> None:
prefix = _build_list_scripts_descendant_prefix(_ctx("alice"), "/foo/bar/")
prefix = _build_list_scripts_owner_descendant_prefix("alice", "/foo/bar/")
assert prefix == "workspace/alice/foo/bar/"
def test_descendant_prefix_rejects_traversal() -> None:
with pytest.raises(HTTPException) as exc:
_build_list_scripts_descendant_prefix(_ctx("alice"), "foo/../bar")
_build_list_scripts_owner_descendant_prefix("alice", "foo/../bar")
assert exc.value.status_code == 422
@@ -202,14 +202,19 @@ async def test_list_scripts_where_clause_uses_like_prefix_and_excludes_deeper()
)
)
await list_scripts(parent_path="foo/bar", context=_ctx("alice"), session=mock_session)
await list_scripts(
parent_path="foo/bar",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
assert len(captured_sql) == 1
sql = captured_sql[0].lower()
# 中间 ``%`` 是 owner 段通配符(跨所有 owner),父路径本身按字面匹配,
# 与 list_resources 对 object_key 的过滤一致。
assert "like 'workspace/%%/foo/bar/%%'" in sql
assert "not like 'workspace/%%/foo/bar/%%/%%'" in sql
# Default (no owner_user_id) scopes to the requester's own subtree:
# LIKE workspace/alice/foo/bar/% (direct children), excluding deeper.
assert "like 'workspace/alice/foo/bar/%%'" in sql
assert "not like 'workspace/alice/foo/bar/%%/%%'" in sql
async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
@@ -230,7 +235,12 @@ async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
)
)
await list_scripts(parent_path="foo_bar", context=_ctx("alice"), session=mock_session)
await list_scripts(
parent_path="foo_bar",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0]
# Normalize keyword case so we don't depend on SQLAlchemy casing.
sql_lower = sql.lower()
@@ -238,9 +248,9 @@ async def test_list_scripts_where_clause_escapes_pattern_literal() -> None:
# doubles the escape char inside the SQL string literal, so what
# the helper emits as `foo\_bar` renders as `foo\\_bar` here
# (2 backslash chars in the actual SQL string).
assert r"like 'workspace/%%/foo\\_bar/%%'" in sql_lower
assert r"like 'workspace/alice/foo\\_bar/%%'" in sql_lower
# NOT LIKE clause also escaped.
assert r"not like 'workspace/%%/foo\\_bar/%%/%%'" in sql_lower
assert r"not like 'workspace/alice/foo\\_bar/%%/%%'" in sql_lower
# And both declare ESCAPE '\\'.
assert sql.count("ESCAPE '\\\\'") == 2, sql
@@ -262,18 +272,26 @@ async def test_list_scripts_where_clause_escapes_percent_pattern() -> None:
)
)
await list_scripts(parent_path="100%match", context=_ctx("alice"), session=mock_session)
await list_scripts(
parent_path="100%match",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0]
sql_lower = sql.lower()
# SQLAlchemy doubles the escape char so `%` → `\%` becomes `\\%`
# in the SQL string literal.
assert r"workspace/%%/100\\%%match/%%" in sql_lower
assert r"workspace/alice/100\\%%match/%%" in sql_lower
async def test_list_scripts_non_admin_adds_visibility_filter() -> None:
"""Workspace-wide listing is narrowed by visibility for non-admin:
"""Owner-scoped listing is narrowed by visibility for non-admin:
owner_user_id = me OR visibility IN (workspace, public) — exactly like
list_resources. The workspace prefix contains NO user_id (cross-owner)."""
list_resources. Default (no owner_user_id) scopes to the requester's own
subtree, so the visibility predicate is redundant-but-present here; it
becomes load-bearing when an owner_user_id query param browses another
member's subtree (their private rows are then excluded)."""
from backend.api.scripts import list_scripts
captured_sql: list[str] = []
@@ -289,11 +307,50 @@ async def test_list_scripts_non_admin_adds_visibility_filter() -> None:
)
)
await list_scripts(parent_path="", context=_ctx("alice"), session=mock_session)
await list_scripts(
parent_path="",
owner_user_id=None,
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0].lower()
# Root listing wildcards the owner segment: LIKE workspace/%/%
# (each owner's root files), excluding 3+ segment descendants.
assert "like 'workspace/%%/%%'" in sql
# Default (no owner_user_id) scopes to the requester's own root:
# LIKE workspace/alice/% (alice's root files), excluding nested.
assert "like 'workspace/alice/%%'" in sql
assert "scripts.owner_user_id = 'alice'" in sql
assert "scripts.visibility in ('workspace', 'public')" in sql
async def test_list_scripts_owner_param_scopes_to_other_owner() -> None:
"""owner_user_id=<other> scopes the LIKE to that owner's subtree so the
tree can lazily fetch another member's content on group expand. The
non-admin visibility predicate is still applied, so the other owner's
private rows are excluded (only workspace/public survive)."""
from backend.api.scripts import list_scripts
captured_sql: list[str] = []
class _MockResult:
def all(self):
return []
mock_session = MagicMock()
mock_session.execute = AsyncMock(
side_effect=lambda stmt: (
captured_sql.append(_compile_sql(stmt)) or _MockResult()
)
)
await list_scripts(
parent_path="",
owner_user_id="bob",
context=_ctx("alice"),
session=mock_session,
)
sql = captured_sql[0].lower()
assert "like 'workspace/bob/%%'" in sql
assert "not like 'workspace/bob/%%/%%'" in sql
# non-admin: visibility predicate still present (excludes bob's private)
assert "scripts.owner_user_id = 'alice'" in sql
assert "scripts.visibility in ('workspace', 'public')" in sql
@@ -316,15 +373,47 @@ async def test_list_scripts_admin_skips_visibility_filter() -> None:
)
await list_scripts(
parent_path="", context=_ctx("alice", is_admin=True), session=mock_session
parent_path="",
owner_user_id=None,
context=_ctx("alice", is_admin=True),
session=mock_session,
)
sql = captured_sql[0].lower()
assert "like 'workspace/%%/%%'" in sql
assert "like 'workspace/alice/%%'" in sql
# owner_user_id / visibility still appear in the SELECT projection; what
# must be absent is the visibility WHERE predicate for non-admins.
assert "scripts.visibility in ('workspace', 'public')" not in sql
async def test_list_scripts_admin_owner_param_skips_visibility() -> None:
"""Admin browsing another owner's subtree scopes to that owner and skips
the visibility predicate (admin sees the other owner's private too)."""
from backend.api.scripts import list_scripts
captured_sql: list[str] = []
class _MockResult:
def all(self):
return []
mock_session = MagicMock()
mock_session.execute = AsyncMock(
side_effect=lambda stmt: (
captured_sql.append(_compile_sql(stmt)) or _MockResult()
)
)
await list_scripts(
parent_path="sub",
owner_user_id="bob",
context=_ctx("alice", is_admin=True),
session=mock_session,
)
sql = captured_sql[0].lower()
assert "like 'workspace/bob/sub/%%'" in sql
assert "scripts.visibility in ('workspace', 'public')" not in sql
async def test_list_workspace_directories_where_clause_escapes_pattern() -> None:
"""list_workspace_directories must escape user input too (was
pre-existing debt)."""
@@ -351,11 +440,21 @@ async def test_list_workspace_directories_where_clause_escapes_pattern() -> None
)
await list_workspace_directories(
parent_path="foo_bar", context=_ctx("alice"), session=mock_session
parent_path="foo_bar", owner_user_id=None,
context=_ctx("alice"), session=mock_session,
)
sql = " ".join(captured_sql)
assert r"workspace/alice/foo\\_bar/" in sql, sql
# owner_user_id scopes the prefix to that owner's subtree.
captured_sql.clear()
await list_workspace_directories(
parent_path="foo_bar", owner_user_id="bob",
context=_ctx("alice"), session=mock_session,
)
sql = " ".join(captured_sql)
assert r"workspace/bob/foo\\_bar/" in sql, sql
# ─── layer 3: behavioral test on real LIKE execution ──────────────
backend/tests/test_resources.py
+70 -15
View File
@@ -573,6 +573,7 @@ async def test_list_resources_where_clause_uses_like_prefix_and_excludes_deeper(
await list_resources(
parent_path="foo/bar",
owner_user_id=None,
context=_resource_ctx(),
session=mock_session,
visibility=None,
@@ -581,9 +582,10 @@ async def test_list_resources_where_clause_uses_like_prefix_and_excludes_deeper(
assert len(captured_sql) == 1
sql = captured_sql[0].lower()
# 中间 ``%`` 是 owner 段通配符(跨所有 owner),父路径本身按字面匹配。
assert "like 'w001/%%/foo/bar/%%'" in sql
assert "not like 'w001/%%/foo/bar/%%/%%'" in sql
# Default (no owner_user_id) scopes to the requester's own object_key
# subtree: LIKE w001/u001/foo/bar/% (direct children), excluding deeper.
assert "like 'w001/u001/foo/bar/%%'" in sql
assert "not like 'w001/u001/foo/bar/%%/%%'" in sql
async def test_list_resources_where_clause_escapes_underscore() -> None:
@@ -596,6 +598,7 @@ async def test_list_resources_where_clause_escapes_underscore() -> None:
await list_resources(
parent_path="foo_bar",
owner_user_id=None,
context=_resource_ctx(),
session=mock_session,
visibility=None,
@@ -605,15 +608,20 @@ async def test_list_resources_where_clause_escapes_underscore() -> None:
sql_lower = sql.lower()
# SQLAlchemy doubles the escape char inside the SQL string literal, so
# the helper's ``foo\_bar`` renders as ``foo\\_bar`` in the SQL text.
assert r"like 'w001/%%/foo\\_bar/%%'" in sql_lower
assert r"not like 'w001/%%/foo\\_bar/%%/%%'" in sql_lower
assert r"like 'w001/u001/foo\\_bar/%%'" in sql_lower
assert r"not like 'w001/u001/foo\\_bar/%%/%%'" in sql_lower
# Both LIKE clauses declare ESCAPE '\\' (two in total).
assert sql.count("ESCAPE '\\\\'") == 2, sql
async def test_list_resources_without_parent_path_adds_no_like_clause() -> None:
"""Empty parent_path keeps the legacy workspace-wide behaviour — no
object_key LIKE filter at all."""
async def test_list_resources_empty_parent_path_adds_root_like_clause() -> None:
"""Empty parent_path still applies the directory filter (symmetric with
list_scripts): ``{ws_id}/{owner}/%`` AND NOT ``{ws_id}/{owner}/%/%`` so
the owner-scoped root view returns only direct children of the
requester's root, never nested descendants. Skipping the filter for
empty input used to surface nested resources at the root and visually
broke the directory tree.
"""
from backend.api.resources import list_resources
captured_sql: list[str] = []
@@ -621,14 +629,42 @@ async def test_list_resources_without_parent_path_adds_no_like_clause() -> None:
await list_resources(
parent_path="",
owner_user_id=None,
context=_resource_ctx(),
session=mock_session,
visibility=None,
keyword=None,
)
sql = captured_sql[0].lower()
assert " like " not in sql
assert " not like " not in sql
assert " like 'w001/u001/%%'" in sql
assert " not like 'w001/u001/%%/%%'" in sql
async def test_list_resources_owner_param_scopes_to_other_owner() -> None:
"""owner_user_id=<other> scopes object_key LIKE to that owner's subtree
so the tree can lazily fetch another member's data resources on group
expand. Non-admin visibility predicate is still applied, so the other
owner's private resources are excluded (workspace/public only).
"""
from backend.api.resources import list_resources
captured_sql: list[str] = []
mock_session = _list_resources_capturing_session(captured_sql)
await list_resources(
parent_path="",
owner_user_id="U002",
context=_resource_ctx(),
session=mock_session,
visibility=None,
keyword=None,
)
sql = captured_sql[0].lower()
assert " like 'w001/u002/%%'" in sql
assert " not like 'w001/u002/%%/%%'" in sql
# non-admin: visibility predicate still present (excludes U002 private)
assert "data_resources.owner_user_id = 'u001'" in sql
assert "data_resources.visibility in ('workspace', 'public')" in sql
async def test_list_resources_joins_users_for_display_name() -> None:
@@ -641,6 +677,7 @@ async def test_list_resources_joins_users_for_display_name() -> None:
await list_resources(
parent_path="",
owner_user_id=None,
context=_resource_ctx(),
session=mock_session,
visibility=None,
@@ -745,11 +782,29 @@ def test_sqlite_parent_path_with_slash_direct_children(sqlite_object_key_table)
assert matched == ["W001/U001/data/deep/nested.csv"], matched
def test_sqlite_empty_parent_path_has_no_like_filter(sqlite_object_key_table) -> None:
"""Empty parent_path → no LIKE filter → the endpoint's base WHERE only
(workspace-wide active resources). Stand-in: every row is returned."""
def test_sqlite_empty_parent_path_returns_root_level_across_owners(
sqlite_object_key_table,
) -> None:
"""Empty parent_path now applies the root filter (symmetric with
list_scripts): ``{ws_id}/%/%`` AND NOT ``{ws_id}/%/%/%`` returns only
direct children of every owner's root, excluding nested descendants.
Earlier 'no LIKE' behaviour used to surface every row in the
workspace at the root, which is exactly what made scripts and data
appear mutually visible and broke the tree.
"""
engine, table = sqlite_object_key_table
like = "W001/%/%"
not_like = "W001/%/%/%"
with engine.connect() as conn:
rows = conn.execute(select(table.c.object_key)).fetchall()
rows = conn.execute(
select(table.c.object_key).where(
table.c.object_key.like(like, escape="\\"),
~table.c.object_key.like(not_like, escape="\\"),
)
).fetchall()
matched = sorted(r[0] for r in rows)
assert len(matched) == 8, matched
# ``W001/U001/root.csv`` is the only 3-segment path (= direct child
# of the owner root); all 4+ segment paths (data/*, database/*) are
# excluded by the NOT LIKE clause. The other 4+ segment files would
# be returned when the user expands the corresponding subdirectory.
assert matched == ["W001/U001/root.csv"], matched