fix(backend): enhance storage/resource concurrency, idempotency, and deletion safety
Summary of changes:
- Resources Bind Idempotency (High 1+2):
- Check existing active storage object bindings before duplicate check.
- Return existing binding (`reused: true`) on retry with same upload_id.
- Filter by `status == "active"` to bypass dead/deleted rows during reuse check.
- Storage Concurrent Overwrite (High 3):
- Add `acquire_named_lock` and `release_named_lock` helpers using MySQL `GET_LOCK`/`RELEASE_LOCK` hashed to <= 64 chars.
- Wrap `upload_bytes_to_session` PUT+INSERT critical section with named lock on `object_key`.
- Re-check key collision inside lock; append ULID suffix on collision.
- Shared Reference Deletion Protection (Medium 4):
- Check active references before deleting storage objects in `delete_resource`.
- Delete only `DataResources` record if storage object is still referenced elsewhere.
- Robust Usage Type Fallback (Medium 5):
- Replace direct dict lookup for `USAGE_TYPE_TO_PURPOSE[item.usage_type]` with `.get(..., "workspace")` default.
- Idempotency Key Path Matching (Medium 6):
- Move `file_name`/`target_path` validation forward and include path dimension in comparison.
- Strip uniqueness suffix via `_strip_uniqueness_suffix` before key comparison to avoid false 409s on valid retries.
- Usage Type & Bind Concurrency Control (Low 7 & 8):
- Reject bind requests with 409 if upload session purpose is not `data_resource`.
- Wrap resource duplicate check and creation in named lock using `(owner, directory, name)`.
- Trash Key Uniqueness & Restore Compatibility (Low 9):
- Update `trash_key` format to `{purpose}/{object_key}-{storage_object_id}` to prevent collisions.
- Update `object_key_hash` on trash move.
- Update restore logic in `storage_api.py` to strip suffix while maintaining backward compatibility with legacy keys.
- Dead Code Removal (Low 10):
- Remove unreachable `upload_status = "failed"` and redundant `session.rollback()` in `IntegrityError` block.
- Tests & Mocks:
- Add/update 5 test cases covering non-data_resource bind rejection, suffix stripping, and path recovery.
- Add named lock statement mocks for DB testing.
This commit is contained in:
@@ -120,9 +120,9 @@ class StorageObjects(Base):
|
||||
String(1100),
|
||||
comment=(
|
||||
"Path inside the trash bucket where the soft-deleted bytes "
|
||||
"live. Format: '{source_bucket}/{object_key}' so a restore "
|
||||
"is a same-key copy back to the source bucket. NULL while "
|
||||
"the row is still available."
|
||||
"live. Format: '{source_purpose}/{object_key}-{storage_object_id}' "
|
||||
"(id 后缀防止同名文件多次删除互相覆盖); 恢复时去掉该 id 后缀 "
|
||||
"拷回原 object_key。NULL while the row is still available."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user