Commit Graph
316 Commits
Author SHA1 Message Date
tao.chen 6258cf5d12 fix: offline deploy 2026-08-17 15:59:19 +08:00
tao.chen 0974881fbb fix: offline deploy 2026-08-17 15:30:21 +08:00
tao.chen cbfbee7fe0 chore: add CODE_REVIEW.md v0.1.0 2026-08-14 20:55:59 +08:00
tao.chen b6eb069849 fix(backend): enhance storage/resource concurrency, idempotency, and deletion safety
Summary of changes:

- Resources Bind Idempotency (High 1+2):
  - Check existing active storage object bindings before duplicate check.
  - Return existing binding (`reused: true`) on retry with same upload_id.
  - Filter by `status == "active"` to bypass dead/deleted rows during reuse check.

- Storage Concurrent Overwrite (High 3):
  - Add `acquire_named_lock` and `release_named_lock` helpers using MySQL `GET_LOCK`/`RELEASE_LOCK` hashed to <= 64 chars.
  - Wrap `upload_bytes_to_session` PUT+INSERT critical section with named lock on `object_key`.
  - Re-check key collision inside lock; append ULID suffix on collision.

- Shared Reference Deletion Protection (Medium 4):
  - Check active references before deleting storage objects in `delete_resource`.
  - Delete only `DataResources` record if storage object is still referenced elsewhere.

- Robust Usage Type Fallback (Medium 5):
  - Replace direct dict lookup for `USAGE_TYPE_TO_PURPOSE[item.usage_type]` with `.get(..., "workspace")` default.

- Idempotency Key Path Matching (Medium 6):
  - Move `file_name`/`target_path` validation forward and include path dimension in comparison.
  - Strip uniqueness suffix via `_strip_uniqueness_suffix` before key comparison to avoid false 409s on valid retries.

- Usage Type & Bind Concurrency Control (Low 7 & 8):
  - Reject bind requests with 409 if upload session purpose is not `data_resource`.
  - Wrap resource duplicate check and creation in named lock using `(owner, directory, name)`.

- Trash Key Uniqueness & Restore Compatibility (Low 9):
  - Update `trash_key` format to `{purpose}/{object_key}-{storage_object_id}` to prevent collisions.
  - Update `object_key_hash` on trash move.
  - Update restore logic in `storage_api.py` to strip suffix while maintaining backward compatibility with legacy keys.

- Dead Code Removal (Low 10):
  - Remove unreachable `upload_status = "failed"` and redundant `session.rollback()` in `IntegrityError` block.

- Tests & Mocks:
  - Add/update 5 test cases covering non-data_resource bind rejection, suffix stripping, and path recovery.
  - Add named lock statement mocks for DB testing.
2026-08-14 20:53:07 +08:00
tao.chen f72dfd10e8 fix: delete bug 2026-08-14 20:37:25 +08:00
tao.chen 1079b62f0a update 2026-08-14 20:08:51 +08:00
tao.chen 85b2916850 fix: local_storage_base_dir 2026-08-14 20:03:40 +08:00
tao.chen 25e563dcaa update: ruff check --fix 2026-08-14 19:51:58 +08:00
tao.chen d855912791 fix: soft delete helper 2026-08-14 19:39:09 +08:00
tao.chen 76af0f763c update: README.md 2026-08-14 19:30:25 +08:00
tao.chen 139f2c02c1 fix: delete error 2026-08-14 19:19:41 +08:00
tao.chen 2e070fae78 fix: pre check 2026-08-14 19:13:17 +08:00
tao.chen 5c5dec9a4a fix: allows_same_name_different_parent 2026-08-14 18:49:27 +08:00
tao.chen 5d49ff5e34 fix: file upload error 2026-08-14 18:22:46 +08:00
tao.chen c65d6dc684 fix: build error 2026-08-14 17:19:07 +08:00
tao.chen 45cb2f8409 fix: build error 2026-08-14 17:16:53 +08:00
tao.chen de1682c00d update: Dockerfile 2026-08-14 17:03:27 +08:00
tao.chen 341cc77a79 update: Dockerfile 2026-08-14 16:02:19 +08:00
tao.chen 2014d59681 update: Dockerfile 2026-08-14 15:55:00 +08:00
tao.chen 7347b409e5 update: port 2026-08-14 15:32:34 +08:00
tao.chen af9466c4e6 update: UV_HTTP_TIMEOUT 2026-08-14 15:27:16 +08:00
tao.chen e9a7e1a24e update: docker-compose.yml 2026-08-14 15:19:29 +08:00
Winnie be475dd0d9 fix: support local storage runtime and schedule logs 2026-08-14 15:06:36 +08:00
tao.chen 65d14b8610 update: dir tree 2026-08-14 15:02:55 +08:00
tao.chen 4530811441 update: spec 2026-08-14 14:40:34 +08:00
tao.chen d415154f41 update: migrate 2026-08-14 14:32:44 +08:00
tao.chen c45687ef18 fix: update unique key 2026-08-14 13:44:57 +08:00
tao.chen 7330f313ff update: add custom dep 2026-08-14 13:15:18 +08:00
tao.chen 518853cbb0 fix 2026-08-14 12:40:36 +08:00
tao.chen 604e66ffe0 fix: DataResourceUploadModal.tsx 2026-08-14 12:28:22 +08:00
tao.chen 480e1cc638 update: data source 2026-08-14 11:59:06 +08:00
tao.chen 225a585499 update: validate_target_path 2026-08-14 11:32:15 +08:00
tao.chen b9becffab2 update: 数据资源文件上传 2026-08-14 11:31:48 +08:00
tao.chen 462617b66d feat: datasource 2026-08-13 20:13:37 +08:00
tao.chen 0b5b50edb9 update: rel path 2026-08-13 19:51:20 +08:00
tao.chen 63a14b1884 fix: params error
NameError: name 'session' is not defined
2026-08-13 19:20:36 +08:00
tao.chen f5cb22e8cd feat: upload resource 2026-08-13 19:09:13 +08:00
tao.chen 912c0ed22d Merge branch 'refactor/storage' into develop 2026-08-13 18:11:50 +08:00
tao.chen 19157ad20b test 2026-08-13 18:08:12 +08:00
xiaozhu bcddaac083 fix:修复只读文件切换编辑文件时iframe重新加载问题 2026-08-13 17:24:55 +08:00
xiaozhu 9160175d51 update:只读文件隐藏操作按钮 2026-08-13 16:05:16 +08:00
tao.chen 532f0d6c91 fix: dir tree 2026-08-13 12:12:58 +08:00
tao.chen 3094a47298 fix: path error 2026-08-13 11:48:24 +08:00
tao.chen 83d0e0adca fix: bug
1. Cannot find name 'PythonEditorBuffer'
2. Cannot find name 'PythonEditor'
2026-08-13 11:26:31 +08:00
tao.chen 1ad0ec9dbf update: docker-compose.yml 2026-08-13 11:03:08 +08:00
tao.chen dc68739216 Merge remote-tracking branch 'aliyun/develop' into develop 2026-08-13 11:01:43 +08:00
tao.chen 49c8a27841 update: update storage struct 2026-08-13 11:01:37 +08:00
xiaozhu a2f515cf87 fix:锁定文件只读 2026-08-13 10:26:01 +08:00
xiaozhu 7b10f08484 Merge branch 'develop' of http://8.153.151.51:8888/team_group/model-develop into develop 2026-08-13 10:06:46 +08:00
tao.chen da9a1ec472 update: storage path 2026-08-13 09:35:27 +08:00