Commit Graph
117 Commits
Author SHA1 Message Date
tao.chenandClaude 245f4d346d fix: repair storage hot-path bugs and missing imports
- scripts.update_script: repoint script.current_object_id to the
  newly uploaded StorageObject and best-effort delete the old
  working copy. Previously the old row was mutated with the new
  content_hash while the script still pointed at it, silently
  losing user edits on the next publish_version.
- scripts.publish_version: read object bytes via the new
  RustFSObjectStore.get_bytes() instead of the non-existent
  .get_object(); publish 500'd on every call.
- scripts.upload_script / storage_api.complete_upload_record:
  Path(file_name) raised NameError (only PurePosixPath imported),
  crashing every upload and every upload finalization. Use
  PurePosixPath.
- RustFSObjectStore: add get_bytes() helper (sync, body.close in
  finally) for in-process callers that need raw bytes.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-31 15:07:35 +08:00
tao.chen d377ba3cfe refactor: object_key flat layout + usage_type→bucket routing + Settings singleton
- common.config.Settings: pydantic-settings with @lru_cache singleton;
  all env vars now declared in one place (database / JWT / RUSTFS_*
  credentials + 3 purpose-named buckets / workspace FS roots / etc.).
  Replaces os.environ / os.getenv in backend / schedule / runtime /
  common modules.

- storage_api: object_key layout flattens from
  "{ws}/{usage_type}/{ulid}/{name}" to "{ws}/{ulid}". File name, type,
  and logical path live in the StorageObjects / Scripts row, not in
  the S3 key, so the bucket can be re-organised without a DB rewrite.

- storage_api: new BUCKET_FOR_USAGE map and resolve_bucket() helper
  route uploads by usage_type to the right purpose-named bucket:
    working_copy / public_script / data_resource / snapshot
      → RUSTFS_WORKSPACE_BUCKET (workspaces)
    version_artifact
      → RUSTFS_VERSION_BUCKET (versions)
    run_log / run_result
      → RUSTFS_RUN_LOG_BUCKET (run-logs)
  workspace.artifact_bucket override wins over the default for that
  workspace. Unknown usage_type falls through to the workspace bucket
  so uploads are never silently dropped.

- backend.main lifespan: ensure_bucket loops over all three buckets at
  startup.

- common.storage.schemas: extend usage_type Literal to include
  working_copy / public_script (consumed by scripts.py after the local
  FS removal).

- common.storage.client: raise StorageClientError / StorageUnavailable /
  StorageRequestFailed instead of FastAPI HTTPException, so the client
  is usable from non-FastAPI contexts (e.g. schedule worker). The
  register_workspace_object method is removed (the local-FS path it
  routed to no longer exists).

- common.pyproject.toml: add greenlet>=3.0.0 (SQLAlchemy 2.0 async
  engine.dispose() requires it) and pydantic-settings>=2.14.2.

Verified: backend.main 57 routes; docker compose config; 20 SQLAlchemy
tables, 0 ForeignKey; grep os.environ / os.getenv in
backend|schedule|runtime|common = 0.
2026-07-31 13:37:04 +08:00
tao.chen 5cbbe8213a review 2026-07-31 13:34:33 +08:00
tao.chen 28069f516b refactor: delete table 2026-07-31 13:21:15 +08:00
tao.chen f288c90d37 refactor: remove local fs, add config class to common pacakge 2026-07-31 12:46:31 +08:00
tao.chen 62c7d7465e Merge branch 'optimize-by-taochen' into develop
# Conflicts:
#	.env.example
#	backend/src/backend/main.py
#	common/src/common/db/models.py
#	common/src/common/eventing.py
#	docker-compose.yml
#	runtime/Dockerfile
#	runtime/pyproject.toml
#	runtime/src/runtime/main.py
#	schedule/src/schedule/main.py
#	schedule/src/schedule/service.py
2026-07-31 09:29:03 +08:00
tao.chen 91767461d6 refactor 2026-07-30 20:52:46 +08:00
tao.chen c32e6e4cae Merge branch 'refactor/remove-redis' into develop
# Conflicts:
#	.env.example
#	CLAUDE.md
#	backend/Dockerfile
#	backend/pyproject.toml
#	backend/src/backend/main.py
#	backend/src/backend/schedule_runs.py
#	common/src/common/db/models.py
#	common/src/common/eventing.py
#	contracts/README.md
#	contracts/demo-core-v1.md
#	contracts/events/README.md
#	contracts/events/event-envelope-v1.json
#	contracts/locks/README.md
#	contracts/locks/file-edit-lock-v1.md
#	contracts/schedules/schedule-definition-api-v1.md
#	docker-compose.yml
#	frontend/README.md
#	migrations/README.md
#	migrations/versions/20260724_0001_v1_schema_baseline.py
#	runtime/Dockerfile
#	runtime/README.md
#	runtime/pyproject.toml
#	runtime/src/runtime/main.py
#	schedule/Dockerfile
#	schedule/README.md
#	schedule/pyproject.toml
#	schedule/src/schedule/main.py
#	schedule/src/schedule/service.py
2026-07-30 20:14:53 +08:00
tao.chen ec53edbce5 refactor 2026-07-30 20:02:19 +08:00
Winnie c74b2abb48 重构模型平台前后端并移除Redis依赖 2026-07-30 19:03:00 +08:00
tao.chen c1e15758a3 refactor 2026-07-30 18:57:18 +08:00
Winnie 6d6c70cea8 refactor: integrate model platform backend 2026-07-30 13:43:29 +08:00
tao.chen 128aa7563f feat: deploy demo test 2026-07-28 13:57:54 +08:00
tao.chen af74f2f242 update: optimize environment 2026-07-28 11:32:06 +08:00
tao.chen ab731bdc3b feat: add nginx, proxy to jupyter after auth 2026-07-27 19:58:17 +08:00
tao.chen 1c03a32298 update: add common 2026-07-27 15:37:58 +08:00
tao.chen e56bf0e56b init 2026-07-27 15:04:31 +08:00