cluster: deprecate DefaultSubmitArgs (no longer prepended)

cluster.DefaultSubmitArgs is now deprecated. It used to be prepended to
spark_submit Tool calls, but since 7920dc9 the spark-submit builder
constructs argv from the structured fields in the tool call. The field is
still accepted by the admin API and still persisted to the DB, so this is
a non-breaking change.

This commit deliberately does NOT remove:
- the Cluster.DefaultSubmitArgs field in internal/cluster/types.go
- the default_submit_args DB column
- the admin API read/write support in internal/admin/router.go
- existing storage/repo tests

Runtime now logs a one-time slog.Info when the PUT handler receives a
non-empty default_submit_args value, warning operators that the field is
ignored by spark_submit and that they should use spark_conf / extra_args
instead.

Follow-up removal should touch:
- the Cluster struct field and its JSON tag
- storage/cluster_repo.go scan/insert/update SQL and params
- admin/router.go JSON round-tripping
- storage/cluster_repo_test.go assertions
- the DB migration dropping the default_submit_args column

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
tao.chen
2026-07-14 10:11:39 +08:00
co-authored by Claude
parent dab4cd062e
commit 97f6184f65
2 changed files with 27 additions and 17 deletions
+5
View File
@@ -5,6 +5,7 @@ package admin
import (
"encoding/json"
"errors"
"log/slog"
"net/http"
"strconv"
@@ -170,6 +171,10 @@ func updateCluster(repo *storage.ClusterRepo, auditRepo *audit.Repo) gin.Handler
merged.DefaultSubmitArgs = decodeStringSlice(v)
}
if len(merged.DefaultSubmitArgs) > 0 {
slog.Default().Info("admin: cluster sets default_submit_args which is deprecated; the spark_submit Tool no longer prepends these args. Configure flags via spark_conf / extra_args in the structured tool call.", "cluster_id", merged.ID)
}
// Re-validate after merging so a missing auth_type on a fresh
// cluster (defaulted by DB to "none") doesn't get clobbered.
if err := validateClusterUpdate(&merged); err != nil {
+22 -17
View File
@@ -34,23 +34,28 @@ const (
// password" — keeps the existing encrypted blob intact.
// - URLAllowlist: extra glob patterns beyond the RM/SHS hosts; fetch_url
// uses this to permit long-tail SHS endpoints the agent may construct.
// - DefaultSubmitArgs: prepended to every spark_submit Tool call's args.
// - DefaultSubmitArgs: deprecated; no longer prepended by spark_submit
// (post-7920dc9 the builder constructs argv from structured fields).
// Kept for backward compatibility with the admin API and DB schema;
// actual removal is a follow-up. New cluster configurations should
// leave this empty.
// - RateLimitPerMin: 0 disables the per-cluster limiter.
type Cluster struct {
ID string `json:"id"`
Name string `json:"name"`
RMURL string `json:"rm_url"`
SHSURL string `json:"shs_url"`
SparkSubmitExecuteBin string `json:"spark_submit_execute_bin"`
IsActive bool `json:"is_active"`
AuthType AuthType `json:"auth_type"`
AuthUsername string `json:"auth_username,omitempty"`
AuthPassword string `json:"-"` // never serialized; encrypted at rest in auth_password_enc
SSLVerify bool `json:"ssl_verify"`
SSLCABundle string `json:"ssl_ca_bundle,omitempty"`
URLAllowlist []string `json:"url_allowlist,omitempty"`
DefaultSubmitArgs []string `json:"default_submit_args,omitempty"`
RateLimitPerMin int `json:"rate_limit_per_min"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
ID string `json:"id"`
Name string `json:"name"`
RMURL string `json:"rm_url"`
SHSURL string `json:"shs_url"`
SparkSubmitExecuteBin string `json:"spark_submit_execute_bin"`
IsActive bool `json:"is_active"`
AuthType AuthType `json:"auth_type"`
AuthUsername string `json:"auth_username,omitempty"`
AuthPassword string `json:"-"` // never serialized; encrypted at rest in auth_password_enc
SSLVerify bool `json:"ssl_verify"`
SSLCABundle string `json:"ssl_ca_bundle,omitempty"`
URLAllowlist []string `json:"url_allowlist,omitempty"`
// Deprecated: see godoc.
DefaultSubmitArgs []string `json:"default_submit_args,omitempty"`
RateLimitPerMin int `json:"rate_limit_per_min"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}