tao.chen
4b2a67ae5d
Replace the old RustFS-specific storage layer (common.storage.client /
RustFSObjectStore) with a minimal sync/async abstraction:
AsyncStorageBackend: put / get / get_stream / delete / exists / stat /
list / get_url / copy
StorageBackend: same surface, sync implementations
create_storage({"type": "s3" | "local", "mode": "async", ...})
backends/s3.py: S3-compatible (boto3 / aioboto3)
backends/local.py: on-disk filesystem (aiofiles)
Concretely:
- Drop RustFSObjectStore + common.storage.client (deleted).
- Drop the RustFS-specific ensure_bucket / presign_put / move_to_trash /
rewrite_to_public_path / sha256 / put_bytes methods.
- Migrate backend/storage_api.py + backend/main.py + backend/scripts.py
+ schedule/service.py + schedule/worker.py to the new abstraction.
- Migrate backend/storage_client.py + schedule/storage_client.py to
stub status (HTTP wrapper is dead code post-migration; rewrite pending).
- Rename all RUSTFS_* env vars to S3_* across .env.example,
docker-compose.yml, default.conf, scripts/nginx-entrypoint.sh,
common/config.py.
- Replace hardcoded rclone remote name "rustfs" with "s3" in
docker-compose.yml + config.py default.
- Rename "rustfs" SQLAlchemy column comments + table comments to
provider-neutral wording; StorageObjects.storage_backend enum
value moves from "rustfs" to "s3" (DB rows with the old value will
fail the != "s3" check until a one-shot migration is applied).
- Drop unused common/src/common/migrations/{README,env.py,script.py.mako}
(the alembic setup lives in /migrations/, not here).
Migration of the old abstractions has been done in one pass; per-route
method calls (delete / stat / put / get_url) are now direct one-liners
against AsyncStorageBackend.
After this commit:
- All Python imports resolve; routes compile (compileall green).
- s3 mode is fully wired.
- Routes that depended on removed methods (presign_put, move_to_trash,
rewrite_to_public_path, head() metadata) raise NotImplementedError
with a one-line TODO; rewriting these route handlers is the next step.
22 lines
641 B
Bash
Executable File
22 lines
641 B
Bash
Executable File
#!/bin/sh
|
|
# nginx entrypoint for the model-platform gateway.
|
|
#
|
|
# Overrides the default nginx:alpine /docker-entrypoint.sh to render the
|
|
# templated default.conf (mounted at /etc/nginx/conf.d/default.conf.template)
|
|
# by substituting the single ${S3_ENDPOINT} placeholder, then writes
|
|
# the result to /etc/nginx/conf.d/default.conf and execs the CMD
|
|
# (typically `nginx -g 'daemon off;'`).
|
|
#
|
|
# Uses sed (busybox-safe) so no extra packages are needed.
|
|
|
|
set -eu
|
|
|
|
: "${S3_ENDPOINT:=http://s3:9000}"
|
|
|
|
sed \
|
|
-e "s|\${S3_ENDPOINT}|${S3_ENDPOINT}|g" \
|
|
/etc/nginx/conf.d/default.conf.template \
|
|
> /etc/nginx/conf.d/default.conf
|
|
|
|
exec "$@"
|