tmp-upload: 大文件分片上传 (前端 1GB/片, 后端聚合) #43

New Issue

2026-09-11T19:09:33+08:00

tao.chen commented

2026-09-11 19:09:33 +08:00

文件 >5GB 时前端切 1GB 分片 POST /upload-chunk, 后端聚合落盘. ≤5GB 仍走原 /upload.

_{Task: a5cpgatkbzr9otz0v04px00m}

文件 >5GB 时前端切 1GB 分片 POST /upload-chunk, 后端聚合落盘. ≤5GB 仍走原 /upload. --- <sub>Task: a5cpgatkbzr9otz0v04px00m</sub>

tao.chen commented

2026-09-11 19:09:34 +08:00

MODEL-DEVELOP-PLATFORM-68

[MODEL-DEVELOP-PLATFORM-68](https://kaneo.maimaicuizhiji.top/dashboard/workspace/UVjqrB68D9jAIx4sh6ucRfsPbbUCj97c/project/hna942mjy8cwyv1o77mw0cnv/task/fjmtczebexq7wfjmnde0vgg0)

tao.chen added the priority:medium status:in-progress labels 2026-09-11 19:09:45 +08:00

tao.chen added status:done and removed status:in-progress labels 2026-09-11 19:49:22 +08:00

tao.chen closed this issue

2026-09-11 19:49:24 +08:00

tao.chen commented

2026-09-11 19:49:29 +08:00

tao.chen commented:

完成。go build/vet/test 通过, JS node --check 通过, 拒绝路径 + happy path + 审计已实测。

改动:

新增 chunk.go (366 行) + chunk_test.go (84 行)
handlers.go: 抽取 checkFilenameSafe (NUL+200), uploadHandler 调用它 (主逻辑不变)
main.go: r.POST("/upload-chunk", uploadChunkHandler)
cleaner.go: 复用 scanEvery ticker, 每小时 cleanupChunks(fileTTL) 清孤儿分片并释放预留
static/helpers.js: CHUNK_THRESHOLD / CHUNK_SIZE / uuid()
static/app.js: uploadChunked / uploadOneChunk (3 并发, 单片 3 次重试 500/1000/2000ms), 取消时 abort 全部分片 xhr, 累计进度

关键决策:

并发: RWMutex 只护 map+meta; io.Copy 在锁外(避免 3 并发串行化)
配额: Index=0 预留 TotalSize; 聚合后 quotaSub+quotaReserve(realSize); 所有失败路径释放
校验: regex+filepath.Base 防穿越; TotalSize>5GB; totalChunks==ceil(±1); 聚合前逐片 stat 校验

已知风险:

rescanQuota 不扫 chunkDir, in-flight 预留可能被抹零 → 聚合后 quotaReserve 补回 (与现有漂移一致)
末片重传在 200 丢失后会 404 (前端串行重试不会并发双聚合)
crypto.randomUUID 需要 secure context (HTTP 非 localhost 会失败)

**tao.chen** commented: > 完成。go build/vet/test 通过, JS node --check 通过, 拒绝路径 + happy path + 审计已实测。改动: - 新增 chunk.go (366 行) + chunk_test.go (84 行) - handlers.go: 抽取 checkFilenameSafe (NUL+200), uploadHandler 调用它 (主逻辑不变) - main.go: r.POST("/upload-chunk", uploadChunkHandler) - cleaner.go: 复用 scanEvery ticker, 每小时 cleanupChunks(fileTTL) 清孤儿分片并释放预留 - static/helpers.js: CHUNK_THRESHOLD / CHUNK_SIZE / uuid() - static/app.js: uploadChunked / uploadOneChunk (3 并发, 单片 3 次重试 500/1000/2000ms), 取消时 abort 全部分片 xhr, 累计进度关键决策: - 并发: RWMutex 只护 map+meta; io.Copy 在锁外(避免 3 并发串行化) - 配额: Index=0 预留 TotalSize; 聚合后 quotaSub+quotaReserve(realSize); 所有失败路径释放 - 校验: regex+filepath.Base 防穿越; TotalSize>5GB; totalChunks==ceil(±1); 聚合前逐片 stat 校验已知风险: - rescanQuota 不扫 chunkDir, in-flight 预留可能被抹零 → 聚合后 quotaReserve 补回 (与现有漂移一致) - 末片重传在 200 丢失后会 404 (前端串行重试不会并发双聚合) - crypto.randomUUID 需要 secure context (HTTP 非 localhost 会失败)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: tao.chen/model-platform#43