Commit Graph
5 Commits
Author SHA1 Message Date
tao.chen 48fe49df3b feat(scripts/data-resources): merge tree + add parent_path filter
- Backend: GET /api/v1/data-resources accepts parent_path; LIKE
  '{ws_id}/%/{escaped}/%' AND NOT LIKE '{ws_id}/%/{escaped}/%/%' on
  StorageObjects.object_key (workspace-wide, escapes _ and %, mirrors
  list_scripts parent_path semantics). 13 new tests in
  test_resources.py (helper unit / SQL compile / SQLite behavioral).
- Frontend: listResources gains parentPath arg, propagated through
  WorkspaceBoundApi + AuthContext binding. WorkspaceTreeGroup title
  count and ScriptExplorer header count now include dataResources.
  memberScriptGroups backfills data-only owners so users with only
  data resources still render a group. loadDataResources accepts an
  optional parentPath, default empty preserves prior behavior.
2026-08-21 12:36:43 +08:00
tao.chen b6eb069849 fix(backend): enhance storage/resource concurrency, idempotency, and deletion safety
Summary of changes:

- Resources Bind Idempotency (High 1+2):
  - Check existing active storage object bindings before duplicate check.
  - Return existing binding (`reused: true`) on retry with same upload_id.
  - Filter by `status == "active"` to bypass dead/deleted rows during reuse check.

- Storage Concurrent Overwrite (High 3):
  - Add `acquire_named_lock` and `release_named_lock` helpers using MySQL `GET_LOCK`/`RELEASE_LOCK` hashed to <= 64 chars.
  - Wrap `upload_bytes_to_session` PUT+INSERT critical section with named lock on `object_key`.
  - Re-check key collision inside lock; append ULID suffix on collision.

- Shared Reference Deletion Protection (Medium 4):
  - Check active references before deleting storage objects in `delete_resource`.
  - Delete only `DataResources` record if storage object is still referenced elsewhere.

- Robust Usage Type Fallback (Medium 5):
  - Replace direct dict lookup for `USAGE_TYPE_TO_PURPOSE[item.usage_type]` with `.get(..., "workspace")` default.

- Idempotency Key Path Matching (Medium 6):
  - Move `file_name`/`target_path` validation forward and include path dimension in comparison.
  - Strip uniqueness suffix via `_strip_uniqueness_suffix` before key comparison to avoid false 409s on valid retries.

- Usage Type & Bind Concurrency Control (Low 7 & 8):
  - Reject bind requests with 409 if upload session purpose is not `data_resource`.
  - Wrap resource duplicate check and creation in named lock using `(owner, directory, name)`.

- Trash Key Uniqueness & Restore Compatibility (Low 9):
  - Update `trash_key` format to `{purpose}/{object_key}-{storage_object_id}` to prevent collisions.
  - Update `object_key_hash` on trash move.
  - Update restore logic in `storage_api.py` to strip suffix while maintaining backward compatibility with legacy keys.

- Dead Code Removal (Low 10):
  - Remove unreachable `upload_status = "failed"` and redundant `session.rollback()` in `IntegrityError` block.

- Tests & Mocks:
  - Add/update 5 test cases covering non-data_resource bind rejection, suffix stripping, and path recovery.
  - Add named lock statement mocks for DB testing.
2026-08-14 20:53:07 +08:00
tao.chen f72dfd10e8 fix: delete bug 2026-08-14 20:37:25 +08:00
tao.chen 25e563dcaa update: ruff check --fix 2026-08-14 19:51:58 +08:00
tao.chen 5d49ff5e34 fix: file upload error 2026-08-14 18:22:46 +08:00