修复 spark_submit 参数重复、上传校验绕过等 8 处缺陷

对应代码审查发现的问题(#1, #2, #4-#8, #10):

#1 CRITICAL:spark_submit 在 argv 中重复拼接 binary。此前
    cmd := []string{binary} 后又把 cmd 作为 Args 传给 executor.Run,
    而 executor 会再拼一次 Binary,导致 OS argv 为 [binary, binary, ...],
    spark-submit 会把自身当作应用 jar。现在 cmd 从 --master 开始,
    executor.Run 使用 Binary + Args,argv 正确。
#2:Validate 曾接受 .meta.json 路径本身。现在显式拒绝 sidecar 路径,
    要求传入数据文件路径。
#4:Sweep 对 sidecar 损坏的数据文件跳过清理。现在损坏 sidecar 会回退
    到数据文件 mtime,超期即删除。
#5:upload_file 描述仍引用已移除的 args 字段,已改为引用 script_path
    及结构化字段。
#6:Deps.UploadStore 改为值类型 uploads.Store,避免 nil 绕过上传校验;
    移除 spark_submit/upload_file 中的 nil 检查。
#7:master/queue/executor_memory 增加空字符串校验。
#8:提取 buildSparkSubmitCommand 构建 argv,消除双写参数的结构性根因。
#10:Validate 失败时记录 slog.Warn("spark_submit.unminted_path_rejected")。

新增测试:
- TestSparkSubmit_StructuredCommand:断言 argv 首行为 --master,末行
  仍为 script_path。
- TestSparkSubmit_EmptyMaster:空 master 返回错误。
- TestStore_Validate_RejectsSidecarPath:拒绝 .meta.json 路径。
- TestStore_Sweep_DeletesDataWithCorruptSidecar:损坏 sidecar 的数据文件
  被清理。

未在本提交处理:
- #9 cluster.DefaultSubmitArgs 弃用留作后续批次。

Co-Authored-By: tao.chen <93983997+taochen-ct@users.noreply.github.com>
This commit is contained in:
tao.chen
2026-07-14 10:09:48 +08:00
parent 4a28886502
commit dab4cd062e
6 changed files with 181 additions and 43 deletions
+1 -5
View File
@@ -18,7 +18,7 @@ var filenameRegex = regexp.MustCompile(`^[a-zA-Z0-9._-]+$`)
// NewUploadFileTool returns the schema for the upload_file MCP Tool.
func NewUploadFileTool() mcp.Tool {
return mcp.NewTool(UploadFileName,
mcp.WithDescription("Upload a script or config file to the server's data/uploads/ directory. Returns {file_id, path, name, size, sha256}. To submit it, pass the returned `path` as one of the args in spark_submit (e.g. args=[<path>, '--class', 'Main']). The path is server-side; the agent does not need to construct one."),
mcp.WithDescription("Upload a script or config file to the server's data/uploads/ directory. Returns {file_id, path, name, size, sha256}. To submit it, pass the returned `path` as the `script_path` field in spark_submit, along with the structured fields (master, deploy_mode, queue, executor_memory, executor_cores, num_executors; optional spark_conf, extra_args)."),
mcp.WithString("filename",
mcp.Required(),
mcp.Description("Plain file name without path separators (1-128 chars, [a-zA-Z0-9._-])"),
@@ -59,10 +59,6 @@ func (d *Deps) UploadFileHandler(ctx context.Context, req mcp.CallToolRequest) (
return errResult("upload_file: " + err.Error()), nil
}
if d.UploadStore == nil {
return errResult("upload_file: upload store not configured"), nil
}
var data []byte
switch encoding {
case "base64":