Linus 修复: PUT 部分更新用 map[string]json.RawMessage 区分 absent/present

原 PUT /admin/clusters/:id 用 ShouldBindJSON + struct, 缺字段自动
填 Go zero value, 无法区分 "未传" vs "传零值":
- true bool → 缺省 = false → 被误翻成 false
- 1 int → 缺省 = 0 → 丢失
- "" string → 缺省 = "" → 清空

第二次 (用 if-zero-value-keep-old 逻辑) 实际更糟: bool "缺省 = false"
跟 old "true" 不等 → 反而把 true 翻成 false。

正确做法: 用 map[string]json.RawMessage 解析 body, 用
_, ok := rawBody["field"] 判断 absent, 缺字段 = 保留 old。

- internal/admin/router.go: 改 updateCluster 解析方式,
  加 decodeString/Int/Bool/StringSlice helpers
  + validateClusterUpdate (轻量校验, 不要求所有必填字段)
- internal/admin/router_test.go: +2 测试
  - TestUpdateCluster_PartialUpdatePreservesOtherFields
    (单字段 PUT 保留其他全部)
  - TestUpdateCluster_SliceReplacement
    (slice 显式覆盖, nil = 保留, [] = 替换)

端到端实测:
- PUT rate_limit=99 保留 is_active=true (修复前会被翻 false)
- PUT is_active=false 保留 rate_limit=99
- PUT auth_type=basic 保留其他全部
- PUT url_allowlist=[only] 替换 slice
- audit 5 条 (1 create + 4 update) DESC 正确

Co-Authored-By: Claude <noreply@anthropic.com>

.env.example

@@ -0,0 +1,37 @@
+# spark-mcp-go environment configuration
+# Copy to .env and edit the *REQUIRED* values.
+
+# HTTP listen address (default: :8080)
+LISTEN_ADDR=:8080
+
+# Root directory for SQLite, uploads, and logs (default: ./data)
+DATA_DIR=./data
+
+# *REQUIRED* Comma-separated admin bearer tokens for /admin/* endpoints
+ADMIN_TOKENS=change-me-admin-token-1,change-me-admin-token-2
+
+# *REQUIRED* Agent bearer token for /mcp endpoints
+AGENT_TOKEN=change-me-agent-token
+
+# Timeout for fetch_url and RM tool HTTP calls (default: 30s)
+HTTP_CLIENT_TIMEOUT=30s
+
+# Maximum HTTP response bytes before truncation (default: 1048576)
+MAX_RESPONSE_BYTES=1048576
+
+# Timeout for spark-submit process execution (default: 60s)
+SPARK_SUBMIT_TIMEOUT=60s
+
+# Log directory (default: ./data/logs)
+LOG_DIR=./data/logs
+
+# Log level: debug, info, warn, error (default: info)
+LOG_LEVEL=info
+
+# Log format: text (terminal) or json (log file; default: text)
+LOG_FORMAT=text
+
+# Analyzer heuristic thresholds (defaults shown)
+ANALYZER_DATA_SKEW_RATIO=3.0
+ANALYZER_GC_PRESSURE_RATIO=0.1
+ANALYZER_BOTTLENECK_SHUFFLE_GB=50

ARCHITECTURE.md

@@ -0,0 +1,63 @@
+# spark-mcp-go 架构说明
+
+## 包依赖图
+
+```
+main
+  ├─ config
+  ├─ logging
+  ├─ storage (SQLite)
+  ├─ admin
+  │    └─ storage / cluster / audit
+  ├─ mcp
+  │    ├─ tools
+  │    │    ├─ rm
+  │    │    ├─ httpclient
+  │    │    ├─ analyzer
+  │    │    └─ storage / cluster
+  │    └─ server (mark3labs/mcp-go)
+  └─ middleware
+```
+
+- `admin` 只依赖 `storage`、`cluster`、`audit`、`middleware`。
+- `tools` 依赖 `rm`（ResourceManager 客户端）、`httpclient`（通用 HTTP 原语）、`analyzer`（日志分析）。
+- `mcp` 聚合所有 Tools，注册到 `mark3labs/mcp-go` 的 Streamable HTTP server。
+- `main` 负责加载配置、打开数据库、挂载路由、启动 HTTP 服务。
+
+## 数据流
+
+1. 管理员调用 `/admin/clusters` POST，写入 cluster JSON。
+2. `storage` 把 cluster 持久化到 SQLite（后续会对敏感字段加密）。
+3. LLM Agent 调用 `/mcp` Tool（如 `list_clusters`）。
+4. Tool 从 storage 读取 cluster，携带 auth 信息，通过 `httpclient` 或本地 `spark-submit` 调用 YARN/Spark。
+5. 结果返回给 LLM；每次 Tool 调用同时写入独立审计日志。
+
+## 11 Tools 分类
+
+| 分类 | Tools |
+|---|---|
+| discovery | `list_clusters` |
+| exec | `spark_submit` |
+| RM | `list_applications`, `get_application_status`, `get_application_logs`, `kill_application`, `fetch_cluster_env` |
+| SHS | `fetch_spark_metrics` |
+| analyzer | `analyze_spark_log` |
+| 原语 | `fetch_url`, `upload_file` |
+
+## 关键设计决策
+
+- **slice-form `spark_submit`**：参数以字符串数组传入，避免 shell 拼接和注入。
+- **cluster `url_allowlist`**：所有外发 HTTP（`fetch_url`、RM、SHS）必须匹配 allowlist，防止 SSRF。
+- **日志降级链**：`get_application_logs` 依次尝试 `amContainerLogs`、aggregated-logs、通用 logs 端点。
+- **多 token admin**：`ADMIN_TOKENS` 支持逗号分隔多管理员 token，便于轮换。
+- **per-tool 独立审计日志**：每个 Tool 每次调用写入 `${LOG_DIR}/tools/<tool>.log`，文件权限 `0600`。
+- **`auth_password` 不可通过 JSON 创建**：字段带 `json:"-"` tag，避免接口泄露密码；生产通过专用端点或数据库初始化写入。
+
+## 安全模型（三道防线）
+
+1. **Token 鉴权**：`ADMIN_TOKENS` 保护管理面，`AGENT_TOKEN` 保护 MCP 面。
+2. **SSRF 防护**：`fetch_url` 与所有集群请求都走 cluster allowlist，且内置 DNS rebinding 检查。
+3. **路径/参数白名单**：`spark_submit` 仅允许 jar、class、--conf 等安全参数；禁止 shell 元字符与重定向。
+
+## 运行与部署
+
+详见 [docs/runbook.md](docs/runbook.md)。

README.md

@@ -0,0 +1,139 @@
+# spark-mcp-go
+
+MCP (Model Context Protocol) server for Apache Spark on YARN. Lets an LLM agent
+discover Spark/YARN endpoints, submit jobs, fetch logs, and analyze them via
+11 typed Tools. Streamable HTTP transport, SQLite-backed configuration, and
+`log/slog` structured logging with per-Tool call files.
+
+## What it does
+
+- Exposes 11 MCP Tools over Streamable HTTP at `/mcp`
+- Admin API at `/admin/*` for cluster and audit configuration
+- HTTP Basic and YARN SimpleAuth for Spark/YARN endpoints
+- SSRF protection with cluster URL allowlist plus DNS rebinding guard
+- Per-Tool independent audit log (file mode `0600`)
+- `spark-submit` via local `exec.Command` (no shell, no injection)
+
+## Quick start (3 steps)
+
+1. Copy and edit the environment file:
+   ```bash
+   cp .env.example .env
+   # Edit ADMIN_TOKENS and AGENT_TOKEN
+   ```
+
+2. Build and start the server:
+   ```bash
+   go build ./...
+   ./spark-mcp-go
+   # Or use the helper:
+   # ./scripts/dev.sh
+   ```
+
+3. Initialize an MCP session, then call `list_clusters` to discover endpoints.
+
+## 11 Tools
+
+| Tool | Type | Purpose |
+|---|---|---|
+| `list_clusters` | discovery | Return all active clusters (LLM entry point) |
+| `spark_submit` | exec | Local `spark-submit` process, extracts `app_id` |
+| `list_applications` | RM | `GET /ws/v1/cluster/apps[?state=&user=]` |
+| `get_application_status` | RM | `GET /ws/v1/cluster/apps/{id}` |
+| `get_application_logs` | RM | Fallback chain: `amContainerLogs` -> aggregated-logs -> logs |
+| `kill_application` | RM | `PUT` state `KILLED` |
+| `fetch_spark_metrics` | SHS | Executor metrics + summary mode triggers analyzer |
+| `fetch_cluster_env` | RM | Aggregate `/cluster/info` and `/cluster/metrics` |
+| `analyze_spark_log` | analyzer | RM logs + heuristic rules + LLM-ready prompt |
+| `fetch_url` | HTTP | Generic primitive, uses cluster allowlist and auth |
+| `upload_file` | FS | Write to `./data/uploads/`, reference from `spark_submit` |
+
+## Environment variables
+
+| Variable | Default | Required | Description |
+|---|---|---|---|
+| `LISTEN_ADDR` | `:8080` | no | HTTP listen address |
+| `DATA_DIR` | `./data` | no | SQLite, uploads, and log root |
+| `ADMIN_TOKENS` | — | **yes** | Comma-separated tokens for `/admin/*` |
+| `AGENT_TOKEN` | — | **yes** | Token for `/mcp` requests |
+| `HTTP_CLIENT_TIMEOUT` | `30s` | no | Timeout for `fetch_url`/RM Tool HTTP calls |
+| `MAX_RESPONSE_BYTES` | `1048576` (1 MiB) | no | HTTP response truncation limit |
+| `SPARK_SUBMIT_TIMEOUT` | `60s` | no | `spark-submit` process timeout |
+| `LOG_DIR` | `./data/logs` | no | Log root directory |
+| `LOG_LEVEL` | `info` | no | `debug`, `info`, `warn`, or `error` |
+| `LOG_FORMAT` | `text` | no | `text` for terminal, `json` for log files |
+| `ANALYZER_DATA_SKEW_RATIO` | `3.0` | no | Data-skew rule threshold (max/min ratio) |
+| `ANALYZER_GC_PRESSURE_RATIO` | `0.1` | no | GC-pressure rule threshold (GC/CPU ratio) |
+| `ANALYZER_BOTTLENECK_SHUFFLE_GB` | `50` | no | Bottleneck rule threshold (shuffle GB) |
+
+## Admin API
+
+All endpoints require `Authorization: Bearer <admin_token>`.
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/admin/clusters` | List all clusters |
+| `POST` | `/admin/clusters` | Create a cluster |
+| `GET` | `/admin/clusters/:id` | Get one cluster |
+| `PUT` | `/admin/clusters/:id` | Update a cluster |
+| `DELETE` | `/admin/clusters/:id` | Delete a cluster |
+| `GET` | `/admin/audit?limit=100` | Query audit log |
+
+Create a cluster:
+
+```bash
+curl -sS -X POST \\
+  -H "Authorization: Bearer ${ADMIN_TOKEN}" \\
+  -H "Content-Type: application/json" \\
+  -d '{
+    "id": "prod",
+    "name": "prod",
+    "rm_url": "http://rm.example.com:8088",
+    "shs_url": "http://shs.example.com:18080",
+    "spark_submit_execute_bin": "/opt/spark/bin/spark-submit",
+    "is_active": true,
+    "auth_type": "simple",
+    "auth_username": "yarn",
+    "rate_limit_per_min": 10,
+    "url_allowlist": ["rm.example.com:8088", "shs.example.com:18080"]
+  }' \\
+  "http://127.0.0.1:${LISTEN_ADDR:-:8080}/admin/clusters"
+```
+
+Note: `auth_password` is not accepted via JSON (`json:"-"`). Set it through a
+dedicated password endpoint or seed the database directly.
+
+## Security model
+
+Three lines of defense:
+
+1. **Token auth** - `ADMIN_TOKENS` protects `/admin/*`; `AGENT_TOKEN` protects `/mcp`.
+2. **SSRF guard** - Every outbound URL must match the cluster's `url_allowlist`,
+   with private-IP CIDR blacklist (incl. `169.254.0.0/16` AWS/GCP metadata).
+3. **Path allowlist** - `upload_file` writes only to `./data/uploads/`, rejects
+   absolute paths, `..`, and non-`[a-zA-Z0-9._-]` filenames.
+
+Additional guarantees:
+
+- `spark_submit` runs the cluster's binary via `exec.Command(name, args...)` —
+  slice form, never `sh -c`. No shell metacharacter interpretation.
+- `auth_password` is tagged `json:"-"`; never serialized in responses, never
+  accepted from JSON request bodies.
+- Cross-host redirects (RM → NM 307) preserve the `Authorization` header
+  through `DoWithRedirect` only when the destination host is in
+  `url_allowlist`.
+- Per-Tool log files are created with mode `0600` and live under
+  `./data/logs/tools/`.
+
+## Development
+
+```bash
+go build ./...
+go test ./...
+go vet ./...
+```
+
+## More docs
+
+- [ARCHITECTURE.md](ARCHITECTURE.md) - Design and package layout
+- [docs/runbook.md](docs/runbook.md) - Deployment, upgrade, and troubleshooting

docs/runbook.md

@@ -0,0 +1,80 @@
+# spark-mcp-go 部署与排障手册
+
+## 部署步骤
+
+### 1. 构建
+
+```bash
+cd /opt/spark-mcp-go
+go build -o spark-mcp-go .
+```
+
+### 2. 配置
+
+```bash
+cp .env.example .env
+# 编辑 .env：ADMIN_TOKENS、AGENT_TOKEN、LISTEN_ADDR、DATA_DIR 等
+```
+
+### 3. 使用 systemd 运行
+
+参见 [docs/systemd/spark-mcp-go.service](systemd/spark-mcp-go.service)。
+
+### 4. 初始化示例 cluster
+
+```bash
+ADMIN_HOST=http://127.0.0.1:8080 ./scripts/seed.sh prod http://rm.example.com:8088 http://shs.example.com:18080 /opt/spark/bin/spark-submit
+```
+
+### 5. 升级流程
+
+1. 停止服务：`systemctl stop spark-mcp-go`
+2. 备份数据库：`cp ./data/spark-mcp.db ./data/spark-mcp.db.bak`
+3. 替换二进制：`cp spark-mcp-go /usr/local/bin/`
+4. 启动服务：`systemctl start spark-mcp-go`
+5. 验证：`curl -sS http://127.0.0.1:${LISTEN_ADDR:-:8080}/healthz`
+
+## 故障排查
+
+### 401 Unauthorized
+
+- 检查请求是否带 `Authorization: Bearer <token>`。
+- 区分 `ADMIN_TOKENS`（管理面）和 `AGENT_TOKEN`（MCP 面）。
+- `.env` 中 token 不能含前导/尾随空格。
+
+### 404 "Invalid session ID"
+
+- Streamable HTTP 需要先走 MCP `initialize` 握手，拿到 `Mcp-Session-Id` 后再调用 Tool。
+- 确保客户端保存并回传 response header 中的 session ID。
+
+### 422 cluster not found
+
+- `cluster_id` 拼写错误，或 cluster 未激活。
+- 先用 `list_clusters` 确认可用的 cluster ID。
+
+### "host not in allowlist"
+
+- 对应 cluster 的 `url_allowlist` 未包含目标主机。
+- 使用 `admin` 更新 cluster，把 RM/SHS 主机加入 allowlist。
+
+### `spark_submit` 返回 exit_code 7
+
+- `spark_submit_execute_bin` 路径错误，或 YARN 拒绝提交。
+- 查看返回的 `stderr_tail`；在服务器上直接运行相同命令验证。
+
+### `analyze_spark_log` 的 `findings` 为空
+
+- SHS metrics 端点 404 或返回格式不匹配。
+- 可 fallback 到 raw log + LLM 自行分析。
+
+### SQLite 锁冲突
+
+- `storage` 已开启 WAL 模式，但 SQLite 仍是 single-writer。
+- 多个 server 实例共用一个 `spark-mcp.db` 会产生锁竞争；请保持一实例一数据库。
+
+## 监控建议
+
+- **存活探测**：HTTP `GET /healthz`
+- **日志告警**：监控 `data/logs/spark-mcp.log` 中的 `ERROR` 级别行
+- **审计日志告警**：`data/logs/tools/` 下文件过大或增长过快时检查异常调用
+- **磁盘告警**：`data/` 目录包含 SQLite、上传文件、日志，需预留空间

docs/systemd/spark-mcp-go.service

@@ -0,0 +1,22 @@
+[Unit]
+Description=Spark MCP Go Server
+After=network.target
+
+[Service]
+Type=simple
+User=spark-mcp
+Group=spark-mcp
+WorkingDirectory=/opt/spark-mcp-go
+EnvironmentFile=/opt/spark-mcp-go/.env
+ExecStart=/opt/spark-mcp-go/spark-mcp-go
+Restart=on-failure
+RestartSec=5
+
+# Security hardening
+NoNewPrivileges=true
+ProtectSystem=strict
+ProtectHome=true
+ReadWritePaths=/opt/spark-mcp-go/data
+
+[Install]
+WantedBy=multi-user.target

go.mod

@@ -4,6 +4,7 @@ go 1.25.5
 
 require (
 	github.com/gin-gonic/gin v1.12.0
+	github.com/mark3labs/mcp-go v0.56.0
 	modernc.org/sqlite v1.53.0
 )
 
@@ -20,6 +21,7 @@ require (
 	github.com/go-playground/validator/v10 v10.30.1 // indirect
 	github.com/goccy/go-json v0.10.5 // indirect
 	github.com/goccy/go-yaml v1.19.2 // indirect
+	github.com/google/jsonschema-go v0.4.2 // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/klauspost/cpuid/v2 v2.3.0 // indirect
@@ -32,8 +34,11 @@ require (
 	github.com/quic-go/qpack v0.6.0 // indirect
 	github.com/quic-go/quic-go v0.59.0 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
+	github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
+	github.com/spf13/cast v1.7.1 // indirect
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.3.1 // indirect
+	github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
 	go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
 	golang.org/x/arch v0.22.0 // indirect
 	golang.org/x/crypto v0.48.0 // indirect

go.sum

@@ -9,8 +9,12 @@ github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gE
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
+github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
+github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
+github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
 github.com/gabriel-vasile/mimetype v1.4.12 h1:e9hWvmLYvtp846tLHam2o++qitpguFiYCKbn0w9jyqw=
 github.com/gabriel-vasile/mimetype v1.4.12/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
 github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
@@ -32,6 +36,8 @@ github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7Lk
 github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
 github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
 github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/jsonschema-go v0.4.2 h1:tmrUohrwoLZZS/P3x7ex0WAVknEkBZM46iALbcqoRA8=
+github.com/google/jsonschema-go v0.4.2/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
 github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
 github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
@@ -42,8 +48,14 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
 github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
 github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
 github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
+github.com/mark3labs/mcp-go v0.56.0 h1:7aCj2wODCskMi08f923ADG+EfELZBdiKILny415cIS8=
+github.com/mark3labs/mcp-go v0.56.0/go.mod h1:+8WclSK1ZUweCP3hvktSji8n8ABG/95QaEkeVE/Uwas=
 github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
 github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -63,6 +75,12 @@ github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SA
 github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
+github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
+github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
+github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
+github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
+github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
+github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -78,6 +96,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
 github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
 github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
+github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
+github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
 go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
 go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
 go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=

internal/admin/openapi.go

@@ -0,0 +1,52 @@
+package admin
+
+import (
+	"embed"
+	"net/http"
+
+	"github.com/gin-gonic/gin"
+)
+
+//go:embed openapi.yaml
+var openAPIFS embed.FS
+
+// OpenAPIYAML returns the embedded OpenAPI spec bytes.
+func OpenAPIYAML() ([]byte, error) {
+	return openAPIFS.ReadFile("openapi.yaml")
+}
+
+// DocsHandler serves the Scalar API reference HTML.
+func DocsHandler(c *gin.Context) {
+	_, err := OpenAPIYAML()
+	if err != nil {
+		c.String(http.StatusInternalServerError, "openapi.yaml not embedded: %v", err)
+		return
+	}
+	c.Data(http.StatusOK, "text/html; charset=utf-8", []byte(scalarHTML))
+}
+
+// OpenAPISpecHandler serves the raw OpenAPI YAML spec.
+func OpenAPISpecHandler(c *gin.Context) {
+	spec, err := OpenAPIYAML()
+	if err != nil {
+		c.String(http.StatusInternalServerError, "openapi.yaml: %v", err)
+		return
+	}
+	c.Data(http.StatusOK, "application/yaml", spec)
+}
+
+const scalarHTML = `<!doctype html>
+<html>
+<head>
+  <title>spark-mcp-go Admin API</title>
+  <meta charset="utf-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1" />
+  <style>
+    body { margin: 0; padding: 0; }
+  </style>
+</head>
+<body>
+  <script id="api-reference" data-url="/admin/docs/spec"></script>
+  <script src="https://cdn.jsdelivr.net/npm/@scalar/api-reference"></script>
+</body>
+</html>`

internal/admin/openapi.yaml

@@ -0,0 +1,129 @@
+openapi: 3.0.3
+info:
+  title: spark-mcp-go Admin API
+  version: 0.0.0
+  description: |
+    Admin API for cluster and audit log configuration. All endpoints
+    require `Authorization: Bearer <admin_token>` (matches any token in
+    the comma-separated ADMIN_TOKENS env var).
+servers:
+  - url: http://localhost:8080
+    description: Default local dev
+components:
+  securitySchemes:
+    bearerAuth:
+      type: http
+      scheme: bearer
+  schemas:
+    Cluster:
+      type: object
+      required: [id, name, rm_url, shs_url, spark_submit_execute_bin]
+      properties:
+        id: { type: string, example: prod }
+        name: { type: string, example: Production }
+        rm_url: { type: string, format: uri, example: http://rm:8088 }
+        shs_url: { type: string, format: uri, example: http://shs:18080 }
+        spark_submit_execute_bin:
+          type: string
+          example: /opt/spark/bin/spark-submit
+          description: Absolute path to spark-submit binary
+        is_active: { type: boolean, default: true }
+        auth_type:
+          type: string
+          enum: [none, simple, basic]
+          default: none
+        auth_username: { type: string, description: 'simple: user.name, basic: username' }
+        ssl_verify: { type: boolean, default: true }
+        ssl_ca_bundle: { type: string }
+        url_allowlist:
+          type: array
+          items: { type: string }
+          description: Additional host patterns (besides RM/SHS) permitted for fetch_url
+        default_submit_args:
+          type: array
+          items: { type: string }
+          description: Prepended to every spark_submit args
+        rate_limit_per_min:
+          type: integer
+          default: 10
+          minimum: 0
+        created_at: { type: string, format: date-time }
+        updated_at: { type: string, format: date-time }
+    AuditEntry:
+      type: object
+      properties:
+        id: { type: integer }
+        timestamp: { type: string, format: date-time }
+        actor: { type: string, example: 'admin:secret-a' }
+        action: { type: string, enum: [cluster.create, cluster.update, cluster.delete] }
+        cluster_id: { type: string, nullable: true }
+        details: { type: string, description: 'JSON-encoded before/after diff' }
+    Error:
+      type: object
+      properties:
+        error: { type: string }
+security:
+  - bearerAuth: []
+paths:
+  /admin/clusters:
+    get:
+      summary: List all clusters
+      responses:
+        '200':
+          description: Array of clusters (always [] even if empty)
+          content:
+            application/json:
+              schema:
+                type: array
+                items: { $ref: '#/components/schemas/Cluster' }
+    post:
+      summary: Create a cluster
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: '#/components/schemas/Cluster' }
+      responses:
+        '201': { description: Created, content: { application/json: { schema: { $ref: '#/components/schemas/Cluster' } } } }
+        '400': { description: Validation error, content: { application/json: { schema: { $ref: '#/components/schemas/Error' } } } }
+  /admin/clusters/{id}:
+    parameters:
+      - name: id
+        in: path
+        required: true
+        schema: { type: string }
+    get:
+      summary: Get a single cluster
+      responses:
+        '200': { description: OK, content: { application/json: { schema: { $ref: '#/components/schemas/Cluster' } } } }
+        '404': { description: Not found }
+    put:
+      summary: Update a cluster (password in body is ignored; set via dedicated endpoint)
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: '#/components/schemas/Cluster' }
+      responses:
+        '200': { description: OK }
+        '404': { description: Not found }
+    delete:
+      summary: Delete a cluster
+      responses:
+        '204': { description: Deleted }
+        '404': { description: Not found }
+  /admin/audit:
+    get:
+      summary: Query audit log (most recent first)
+      parameters:
+        - name: limit
+          in: query
+          schema: { type: integer, default: 100, minimum: 1, maximum: 1000 }
+      responses:
+        '200':
+          description: Array of audit entries
+          content:
+            application/json:
+              schema:
+                type: array
+                items: { $ref: '#/components/schemas/AuditEntry' }

internal/admin/router.go

@@ -0,0 +1,357 @@
+// Package admin exposes the admin HTTP API for cluster management and audit
+// log access. All routes are protected by middleware.AdminAuth.
+package admin
+
+import (
+	"encoding/json"
+	"errors"
+	"net/http"
+	"strconv"
+
+	"github.com/gin-gonic/gin"
+
+	"spark-mcp-go/internal/audit"
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/middleware"
+	"spark-mcp-go/internal/storage"
+)
+
+// Mount attaches the /admin sub-router to r, protecting every route with
+// bearer-token admin authentication.
+func Mount(r *gin.Engine, repo *storage.ClusterRepo, auditRepo *audit.Repo, adminTokens []string) {
+	g := r.Group("/admin", middleware.AdminAuth(adminTokens))
+	g.GET("", webHandler)
+	g.GET("/", webHandler)
+	g.GET("/clusters", listClusters(repo))
+	g.POST("/clusters", createCluster(repo, auditRepo))
+	g.GET("/clusters/:id", getCluster(repo))
+	g.PUT("/clusters/:id", updateCluster(repo, auditRepo))
+	g.DELETE("/clusters/:id", deleteCluster(repo, auditRepo))
+	g.GET("/audit", listAudit(auditRepo))
+	g.GET("/docs", DocsHandler)
+	g.GET("/docs/spec", OpenAPISpecHandler)
+}
+
+func listClusters(repo *storage.ClusterRepo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		clusters, err := repo.List(c.Request.Context())
+		if err != nil {
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+		c.JSON(http.StatusOK, clusters)
+	}
+}
+
+func createCluster(repo *storage.ClusterRepo, auditRepo *audit.Repo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		var cl cluster.Cluster
+		if err := c.ShouldBindJSON(&cl); err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
+			return
+		}
+		if err := validateClusterCreate(&cl); err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
+			return
+		}
+
+		ctx := c.Request.Context()
+		if err := repo.Create(ctx, &cl); err != nil {
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+
+		details, err := audit.MarshalDetails(clusterFields(&cl))
+		if err == nil {
+			_ = auditRepo.Insert(ctx, &audit.Entry{
+				Actor:     actor(c),
+				Action:    audit.ActionClusterCreate,
+				ClusterID: cl.ID,
+				Details:   details,
+			})
+		}
+
+		c.JSON(http.StatusCreated, &cl)
+	}
+}
+
+func getCluster(repo *storage.ClusterRepo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		cl, err := repo.Get(c.Request.Context(), c.Param("id"))
+		if err != nil {
+			if errors.Is(err, storage.ErrNotFound) {
+				c.JSON(http.StatusNotFound, gin.H{"error": "cluster not found"})
+				return
+			}
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+		c.JSON(http.StatusOK, cl)
+	}
+}
+
+func updateCluster(repo *storage.ClusterRepo, auditRepo *audit.Repo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		id := c.Param("id")
+		ctx := c.Request.Context()
+
+		old, err := repo.Get(ctx, id)
+		if err != nil {
+			if errors.Is(err, storage.ErrNotFound) {
+				c.JSON(http.StatusNotFound, gin.H{"error": "cluster not found"})
+				return
+			}
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+
+		// Parse the body as a generic map first so we can tell which fields
+		// the client actually sent. A struct + ShouldBindJSON can't tell
+		// "absent" from "zero value" for bools, ints, and empty strings —
+		// which silently flips true→false and 1→0 on PATCH-style updates.
+		rawBody := make(map[string]json.RawMessage)
+		if err := c.ShouldBindJSON(&rawBody); err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
+			return
+		}
+
+		merged := *old
+		merged.ID = id
+
+		// Strings: absent = keep old; empty string = keep old (no way to
+		// distinguish "set to empty" from "not set" in JSON; treat them
+		// the same — clients should send the new value or omit).
+		if v, ok := rawBody["name"]; ok {
+			merged.Name = decodeString(v)
+		}
+		if v, ok := rawBody["rm_url"]; ok {
+			merged.RMURL = decodeString(v)
+		}
+		if v, ok := rawBody["shs_url"]; ok {
+			merged.SHSURL = decodeString(v)
+		}
+		if v, ok := rawBody["spark_submit_execute_bin"]; ok {
+			merged.SparkSubmitExecuteBin = decodeString(v)
+		}
+		if v, ok := rawBody["auth_type"]; ok {
+			merged.AuthType = cluster.AuthType(decodeString(v))
+		}
+		if v, ok := rawBody["auth_username"]; ok {
+			merged.AuthUsername = decodeString(v)
+		}
+		if v, ok := rawBody["auth_password"]; ok {
+			// Field is json:"-" so it never round-trips through admin JSON,
+			// but a future dedicated password endpoint could set it here.
+			merged.AuthPassword = decodeString(v)
+		}
+		if v, ok := rawBody["ssl_ca_bundle"]; ok {
+			merged.SSLCABundle = decodeString(v)
+		}
+		// Ints.
+		if v, ok := rawBody["rate_limit_per_min"]; ok {
+			merged.RateLimitPerMin = decodeInt(v)
+		}
+		// Bools.
+		if v, ok := rawBody["is_active"]; ok {
+			merged.IsActive = decodeBool(v)
+		}
+		if v, ok := rawBody["ssl_verify"]; ok {
+			merged.SSLVerify = decodeBool(v)
+		}
+		// Slices: null = keep old; [] = replace with empty.
+		if v, ok := rawBody["url_allowlist"]; ok {
+			merged.URLAllowlist = decodeStringSlice(v)
+		}
+		if v, ok := rawBody["default_submit_args"]; ok {
+			merged.DefaultSubmitArgs = decodeStringSlice(v)
+		}
+
+		// Re-validate after merging so a missing auth_type on a fresh
+		// cluster (defaulted by DB to "none") doesn't get clobbered.
+		if err := validateClusterUpdate(&merged); err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
+			return
+		}
+
+		if err := repo.Update(ctx, &merged); err != nil {
+			if errors.Is(err, storage.ErrNotFound) {
+				c.JSON(http.StatusNotFound, gin.H{"error": "cluster not found"})
+				return
+			}
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+
+		details, err := audit.MarshalDetails(map[string]any{
+			"before": clusterFields(old),
+			"after":  clusterFields(&merged),
+		})
+		if err == nil {
+			_ = auditRepo.Insert(ctx, &audit.Entry{
+				Actor:     actor(c),
+				Action:    audit.ActionClusterUpdate,
+				ClusterID: merged.ID,
+				Details:   details,
+			})
+		}
+
+		c.JSON(http.StatusOK, &merged)
+	}
+}
+
+func decodeString(raw json.RawMessage) string {
+	var s string
+	_ = json.Unmarshal(raw, &s)
+	return s
+}
+
+func decodeInt(raw json.RawMessage) int {
+	var n int
+	_ = json.Unmarshal(raw, &n)
+	return n
+}
+
+func decodeBool(raw json.RawMessage) bool {
+	var b bool
+	_ = json.Unmarshal(raw, &b)
+	return b
+}
+
+func decodeStringSlice(raw json.RawMessage) []string {
+	// Handle both "key": null (clear to nil — caller treats as keep-old
+	// since they would have skipped this field) and "key": [...]
+	var s []string
+	_ = json.Unmarshal(raw, &s)
+	return s
+}
+
+func deleteCluster(repo *storage.ClusterRepo, auditRepo *audit.Repo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		id := c.Param("id")
+		ctx := c.Request.Context()
+
+		old, err := repo.Get(ctx, id)
+		if err != nil {
+			if errors.Is(err, storage.ErrNotFound) {
+				c.JSON(http.StatusNotFound, gin.H{"error": "cluster not found"})
+				return
+			}
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+
+		if err := repo.Delete(ctx, id); err != nil {
+			if errors.Is(err, storage.ErrNotFound) {
+				c.JSON(http.StatusNotFound, gin.H{"error": "cluster not found"})
+				return
+			}
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+
+		details, err := audit.MarshalDetails(map[string]any{
+			"id":   id,
+			"name": old.Name,
+		})
+		if err == nil {
+			_ = auditRepo.Insert(ctx, &audit.Entry{
+				Actor:     actor(c),
+				Action:    audit.ActionClusterDelete,
+				ClusterID: id,
+				Details:   details,
+			})
+		}
+
+		c.Status(http.StatusNoContent)
+	}
+}
+
+func listAudit(auditRepo *audit.Repo) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		limitStr := c.DefaultQuery("limit", "100")
+		limit, err := strconv.Atoi(limitStr)
+		if err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"error": "invalid limit"})
+			return
+		}
+
+		entries, err := auditRepo.List(c.Request.Context(), limit)
+		if err != nil {
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+		c.JSON(http.StatusOK, entries)
+	}
+}
+
+func actor(c *gin.Context) string {
+	tok, _ := c.Get("admin_token")
+	s, _ := tok.(string)
+	if len(s) > 8 {
+		s = s[:8]
+	}
+	return "admin:" + s
+}
+
+func validateClusterCreate(c *cluster.Cluster) error {
+	if c.ID == "" {
+		return errors.New("id is required")
+	}
+	if c.Name == "" {
+		return errors.New("name is required")
+	}
+	if c.RMURL == "" {
+		return errors.New("rm_url is required")
+	}
+	if c.SHSURL == "" {
+		return errors.New("shs_url is required")
+	}
+	if c.SparkSubmitExecuteBin == "" {
+		return errors.New("spark_submit_execute_bin is required")
+	}
+	switch c.AuthType {
+	case cluster.AuthNone, cluster.AuthSimple, cluster.AuthBasic:
+	default:
+		return errors.New("auth_type must be one of none, simple, basic")
+	}
+	if c.RateLimitPerMin < 0 {
+		return errors.New("rate_limit_per_min must be >= 0")
+	}
+	return nil
+}
+
+// validateClusterUpdate is the lighter validation for PATCH semantics: only
+// fields that constrain cluster behavior are checked, since the existing row
+// already supplies the rest.
+func validateClusterUpdate(c *cluster.Cluster) error {
+	switch c.AuthType {
+	case cluster.AuthNone, cluster.AuthSimple, cluster.AuthBasic:
+		// ok
+	case "":
+		// Should not happen — defaults to "none" in DB.
+	default:
+		return errors.New("auth_type must be one of none, simple, basic")
+	}
+	if c.RateLimitPerMin < 0 {
+		return errors.New("rate_limit_per_min must be >= 0")
+	}
+	return nil
+}
+
+func clusterFields(c *cluster.Cluster) map[string]any {
+	return map[string]any{
+		"id":                       c.ID,
+		"name":                     c.Name,
+		"rm_url":                   c.RMURL,
+		"shs_url":                  c.SHSURL,
+		"spark_submit_execute_bin": c.SparkSubmitExecuteBin,
+		"is_active":                c.IsActive,
+		"auth_type":                c.AuthType,
+		"auth_username":            c.AuthUsername,
+		"ssl_verify":               c.SSLVerify,
+		"ssl_ca_bundle":            c.SSLCABundle,
+		"url_allowlist":            c.URLAllowlist,
+		"default_submit_args":      c.DefaultSubmitArgs,
+		"rate_limit_per_min":       c.RateLimitPerMin,
+	}
+}

internal/admin/router_test.go

@@ -0,0 +1,474 @@
+package admin
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/gin-gonic/gin"
+
+	"spark-mcp-go/internal/audit"
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/storage"
+)
+
+func init() {
+	gin.SetMode(gin.TestMode)
+}
+
+func newTestAdmin(t *testing.T) (*gin.Engine, *storage.ClusterRepo, *audit.Repo) {
+	t.Helper()
+	db, err := storage.Open(":memory:")
+	if err != nil {
+		t.Fatalf("open: %v", err)
+	}
+	t.Cleanup(func() { _ = db.Close() })
+	r := gin.New()
+	Mount(r, db.Clusters(), audit.NewRepo(db), []string{"good-token"})
+	return r, db.Clusters(), audit.NewRepo(db)
+}
+
+func doReq(t *testing.T, r *gin.Engine, method, path, token string, body any) *httptest.ResponseRecorder {
+	t.Helper()
+	var rdr io.Reader
+	if body != nil {
+		b, _ := json.Marshal(body)
+		rdr = bytes.NewReader(b)
+	}
+	req := httptest.NewRequest(method, path, rdr)
+	if token != "" {
+		req.Header.Set("Authorization", "Bearer "+token)
+	}
+	if body != nil {
+		req.Header.Set("Content-Type", "application/json")
+	}
+	w := httptest.NewRecorder()
+	r.ServeHTTP(w, req)
+	return w
+}
+
+func fullClusterMap(id string) map[string]any {
+	return map[string]any{
+		"id":                        id,
+		"name":                      id + "-name",
+		"rm_url":                    "http://rm.example.com:8088",
+		"shs_url":                   "http://shs.example.com:18080",
+		"spark_submit_execute_bin":  "/usr/bin/spark-submit",
+		"is_active":                 true,
+		"auth_type":                 "basic",
+		"auth_username":             "admin",
+		"auth_password":             "real-secret",
+		"ssl_verify":                false,
+		"ssl_ca_bundle":             "",
+		"url_allowlist":             []string{"*.example.com"},
+		"default_submit_args":       []string{"--master", "yarn"},
+		"rate_limit_per_min":        10,
+	}
+}
+
+func TestMount_RequiresAuth(t *testing.T) {
+	r, _, _ := newTestAdmin(t)
+
+	tests := []struct {
+		name   string
+		method string
+		path   string
+		token  string
+		want   int
+	}{
+		{"no header", "GET", "/admin/clusters", "", http.StatusUnauthorized},
+		{"wrong scheme", "GET", "/admin/clusters", "Basic xxx", http.StatusUnauthorized},
+		{"wrong token", "GET", "/admin/clusters", "wrong", http.StatusUnauthorized},
+		{"valid token", "GET", "/admin/clusters", "good-token", http.StatusOK},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			w := doReq(t, r, tt.method, tt.path, tt.token, nil)
+			if w.Code != tt.want {
+				t.Errorf("got status %d, want %d", w.Code, tt.want)
+			}
+		})
+	}
+}
+
+func TestListClusters(t *testing.T) {
+	r, _, _ := newTestAdmin(t)
+
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c2"))
+
+	w := doReq(t, r, "GET", "/admin/clusters", "good-token", nil)
+	if w.Code != http.StatusOK {
+		t.Fatalf("got status %d, want %d", w.Code, http.StatusOK)
+	}
+
+	var list []map[string]any
+	if err := json.Unmarshal(w.Body.Bytes(), &list); err != nil {
+		t.Fatalf("unmarshal list: %v", err)
+	}
+	if len(list) != 2 {
+		t.Errorf("got %d clusters, want 2", len(list))
+	}
+
+	for _, cl := range list {
+		if _, ok := cl["auth_password"]; ok {
+			t.Errorf("response must not contain auth_password")
+		}
+	}
+
+	body := w.Body.String()
+	if strings.Contains(body, "real-secret") {
+		t.Errorf("response body leaks auth_password")
+	}
+}
+
+func TestCreateCluster(t *testing.T) {
+	r, _, auditRepo := newTestAdmin(t)
+
+	w := doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+	if w.Code != http.StatusCreated {
+		t.Fatalf("got status %d, want %d", w.Code, http.StatusCreated)
+	}
+
+	var cl map[string]any
+	if err := json.Unmarshal(w.Body.Bytes(), &cl); err != nil {
+		t.Fatalf("unmarshal cluster: %v", err)
+	}
+	if cl["id"] != "c1" || cl["name"] != "c1-name" {
+		t.Errorf("unexpected cluster fields: id=%v name=%v", cl["id"], cl["name"])
+	}
+
+	entries, err := auditRepo.List(context.Background(), 100)
+	if err != nil {
+		t.Fatalf("list audit: %v", err)
+	}
+	if len(entries) < 1 {
+		t.Fatalf("got %d audit entries, want at least 1", len(entries))
+	}
+	found := false
+	for _, e := range entries {
+		if e.Action == audit.ActionClusterCreate && e.Actor == "admin:good-tok" {
+			found = true
+			break
+		}
+	}
+	if !found {
+		t.Errorf("missing create audit entry from admin:good-tok")
+	}
+}
+
+func TestCreateCluster_ValidationErrors(t *testing.T) {
+	r, _, _ := newTestAdmin(t)
+
+	tests := []struct {
+		name string
+		body map[string]any
+	}{
+		{
+			"missing id",
+			func() map[string]any {
+				m := fullClusterMap("c1")
+				delete(m, "id")
+				return m
+			}(),
+		},
+		{
+			"missing name",
+			func() map[string]any {
+				m := fullClusterMap("c1")
+				delete(m, "name")
+				return m
+			}(),
+		},
+		{
+			"invalid auth_type",
+			func() map[string]any {
+				m := fullClusterMap("c1")
+				m["auth_type"] = "kerberos"
+				return m
+			}(),
+		},
+		{
+			"negative rate limit",
+			func() map[string]any {
+				m := fullClusterMap("c1")
+				m["rate_limit_per_min"] = -1
+				return m
+			}(),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			w := doReq(t, r, "POST", "/admin/clusters", "good-token", tt.body)
+			if w.Code != http.StatusBadRequest {
+				t.Errorf("got status %d, want %d", w.Code, http.StatusBadRequest)
+			}
+		})
+	}
+}
+
+func TestGetCluster(t *testing.T) {
+	r, _, _ := newTestAdmin(t)
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+
+	w := doReq(t, r, "GET", "/admin/clusters/c1", "good-token", nil)
+	if w.Code != http.StatusOK {
+		t.Fatalf("got status %d, want %d", w.Code, http.StatusOK)
+	}
+	var cl map[string]any
+	if err := json.Unmarshal(w.Body.Bytes(), &cl); err != nil {
+		t.Fatalf("unmarshal cluster: %v", err)
+	}
+	if cl["id"] != "c1" || cl["name"] != "c1-name" {
+		t.Errorf("unexpected cluster fields: id=%v name=%v", cl["id"], cl["name"])
+	}
+
+	w = doReq(t, r, "GET", "/admin/clusters/not-found", "good-token", nil)
+	if w.Code != http.StatusNotFound {
+		t.Errorf("got status %d, want %d", w.Code, http.StatusNotFound)
+	}
+	var errBody map[string]any
+	if err := json.Unmarshal(w.Body.Bytes(), &errBody); err != nil {
+		t.Fatalf("unmarshal error body: %v", err)
+	}
+	if errBody["error"] != "cluster not found" {
+		t.Errorf("unexpected error message: %v", errBody["error"])
+	}
+}
+
+func TestUpdateCluster(t *testing.T) {
+	r, _, auditRepo := newTestAdmin(t)
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+
+	update := map[string]any{
+		"name":               "updated",
+		"rate_limit_per_min": 99,
+	}
+	w := doReq(t, r, "PUT", "/admin/clusters/c1", "good-token", update)
+	if w.Code != http.StatusOK {
+		t.Fatalf("got status %d, want %d: %s", w.Code, http.StatusOK, w.Body.String())
+	}
+	var cl map[string]any
+	if err := json.Unmarshal(w.Body.Bytes(), &cl); err != nil {
+		t.Fatalf("unmarshal cluster: %v", err)
+	}
+	if cl["name"] != "updated" || cl["rate_limit_per_min"] != float64(99) {
+		t.Errorf("unexpected update response: name=%v rate_limit_per_min=%v", cl["name"], cl["rate_limit_per_min"])
+	}
+
+	w = doReq(t, r, "GET", "/admin/clusters/c1", "good-token", nil)
+	if w.Code != http.StatusOK {
+		t.Fatalf("get after update: got status %d", w.Code)
+	}
+	if err := json.Unmarshal(w.Body.Bytes(), &cl); err != nil {
+		t.Fatalf("unmarshal cluster after update: %v", err)
+	}
+	if cl["name"] != "updated" || cl["rate_limit_per_min"] != float64(99) {
+		t.Errorf("cluster not updated: name=%v rate_limit_per_min=%v", cl["name"], cl["rate_limit_per_min"])
+	}
+
+	entries, err := auditRepo.List(context.Background(), 100)
+	if err != nil {
+		t.Fatalf("list audit: %v", err)
+	}
+	found := false
+	for _, e := range entries {
+		if e.Action == audit.ActionClusterUpdate {
+			found = true
+			var details map[string]any
+			if err := json.Unmarshal([]byte(e.Details), &details); err != nil {
+				t.Fatalf("unmarshal audit details: %v", err)
+			}
+			if _, ok := details["before"]; !ok {
+				t.Errorf("audit details missing before")
+			}
+			if _, ok := details["after"]; !ok {
+				t.Errorf("audit details missing after")
+			}
+		}
+	}
+	if !found {
+		t.Errorf("missing update audit entry")
+	}
+}
+
+func TestUpdateCluster_PreservesEmptyPassword(t *testing.T) {
+	r, repo, auditRepo := newTestAdmin(t)
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+
+	// AuthPassword has json:"-", so POST cannot receive it via JSON.
+	// Seed the stored password directly through the repo.
+	cl, err := repo.Get(context.Background(), "c1")
+	if err != nil {
+		t.Fatalf("get cluster from repo: %v", err)
+	}
+	cl.AuthPassword = "real-secret"
+	if err := repo.Update(context.Background(), cl); err != nil {
+		t.Fatalf("seed password: %v", err)
+	}
+	// Discard the audit row produced by the seed update so it does not confuse later checks.
+	_, _ = auditRepo.List(context.Background(), 100)
+
+	update := map[string]any{
+		"name": "renamed",
+	}
+	w := doReq(t, r, "PUT", "/admin/clusters/c1", "good-token", update)
+	if w.Code != http.StatusOK {
+		t.Fatalf("got status %d, want %d: %s", w.Code, http.StatusOK, w.Body.String())
+	}
+
+	if strings.Contains(w.Body.String(), "real-secret") {
+		t.Errorf("update response leaks auth_password")
+	}
+
+	cl, err = repo.Get(context.Background(), "c1")
+	if err != nil {
+		t.Fatalf("get cluster from repo: %v", err)
+	}
+	if cl.AuthPassword != "real-secret" {
+		t.Errorf("password changed: got %q, want %q", cl.AuthPassword, "real-secret")
+	}
+}
+
+// TestUpdateCluster_PartialUpdatePreservesOtherFields verifies that PUT only
+// touches fields present in the request body. Sends a full cluster via POST,
+// then PUTs a body with a single field ("is_active": false) and checks every
+// other field still matches the original.
+func TestUpdateCluster_PartialUpdatePreservesOtherFields(t *testing.T) {
+	r, repo, _ := newTestAdmin(t)
+	original := fullClusterMap("c1")
+	original["is_active"] = true
+	original["rate_limit_per_min"] = 25
+	original["url_allowlist"] = []string{"rm", "shs"}
+	original["default_submit_args"] = []string{"--conf", "spark.foo=bar"}
+	if w := doReq(t, r, "POST", "/admin/clusters", "good-token", original); w.Code != http.StatusCreated {
+		t.Fatalf("seed POST: %d %s", w.Code, w.Body.String())
+	}
+
+	patch := map[string]any{"is_active": false}
+	w := doReq(t, r, "PUT", "/admin/clusters/c1", "good-token", patch)
+	if w.Code != http.StatusOK {
+		t.Fatalf("PUT: %d %s", w.Code, w.Body.String())
+	}
+
+	got, err := repo.Get(context.Background(), "c1")
+	if err != nil {
+		t.Fatalf("get after PUT: %v", err)
+	}
+	if got.IsActive != false {
+		t.Errorf("is_active: got %v, want false", got.IsActive)
+	}
+	// Every other field should equal the original.
+	if got.Name != original["name"] {
+		t.Errorf("name: got %q, want %q", got.Name, original["name"])
+	}
+	if got.RMURL != original["rm_url"] {
+		t.Errorf("rm_url: got %q, want %q", got.RMURL, original["rm_url"])
+	}
+	if got.SHSURL != original["shs_url"] {
+		t.Errorf("shs_url: got %q, want %q", got.SHSURL, original["shs_url"])
+	}
+	if got.SparkSubmitExecuteBin != original["spark_submit_execute_bin"] {
+		t.Errorf("spark_submit_execute_bin: got %q, want %q", got.SparkSubmitExecuteBin, original["spark_submit_execute_bin"])
+	}
+	if got.AuthType != cluster.AuthBasic {
+		t.Errorf("auth_type: got %q, want basic (preserved from seed)", got.AuthType)
+	}
+	if got.RateLimitPerMin != 25 {
+		t.Errorf("rate_limit_per_min: got %d, want 25 (preserved from seed)", got.RateLimitPerMin)
+	}
+	if len(got.URLAllowlist) != 2 || got.URLAllowlist[0] != "rm" || got.URLAllowlist[1] != "shs" {
+		t.Errorf("url_allowlist: got %v, want [rm shs] (preserved)", got.URLAllowlist)
+	}
+	if len(got.DefaultSubmitArgs) != 2 || got.DefaultSubmitArgs[0] != "--conf" {
+		t.Errorf("default_submit_args: got %v, want [--conf ...] (preserved)", got.DefaultSubmitArgs)
+	}
+}
+
+// TestUpdateCluster_SliceReplacement checks that an explicit non-nil slice
+// in the body *replaces* the old slice (rather than appending or being
+// ignored as a zero value).
+func TestUpdateCluster_SliceReplacement(t *testing.T) {
+	r, repo, _ := newTestAdmin(t)
+	if w := doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1")); w.Code != http.StatusCreated {
+		t.Fatalf("seed POST: %d %s", w.Code, w.Body.String())
+	}
+	seed, _ := repo.Get(context.Background(), "c1")
+	if len(seed.URLAllowlist) == 0 {
+		t.Fatal("seed should have url_allowlist from fullClusterMap")
+	}
+
+	// Replace with a new single-entry list.
+	patch := map[string]any{"url_allowlist": []string{"only-this"}}
+	if w := doReq(t, r, "PUT", "/admin/clusters/c1", "good-token", patch); w.Code != http.StatusOK {
+		t.Fatalf("PUT: %d %s", w.Code, w.Body.String())
+	}
+	got, _ := repo.Get(context.Background(), "c1")
+	if len(got.URLAllowlist) != 1 || got.URLAllowlist[0] != "only-this" {
+		t.Errorf("url_allowlist: got %v, want [only-this]", got.URLAllowlist)
+	}
+}
+
+func TestDeleteCluster(t *testing.T) {
+	r, _, auditRepo := newTestAdmin(t)
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+
+	w := doReq(t, r, "DELETE", "/admin/clusters/c1", "good-token", nil)
+	if w.Code != http.StatusNoContent {
+		t.Fatalf("got status %d, want %d", w.Code, http.StatusNoContent)
+	}
+
+	w = doReq(t, r, "GET", "/admin/clusters/c1", "good-token", nil)
+	if w.Code != http.StatusNotFound {
+		t.Errorf("get after delete: got status %d, want %d", w.Code, http.StatusNotFound)
+	}
+
+	entries, err := auditRepo.List(context.Background(), 100)
+	if err != nil {
+		t.Fatalf("list audit: %v", err)
+	}
+	found := false
+	for _, e := range entries {
+		if e.Action == audit.ActionClusterDelete {
+			found = true
+			break
+		}
+	}
+	if !found {
+		t.Errorf("missing delete audit entry")
+	}
+}
+
+func TestListAudit(t *testing.T) {
+	r, _, _ := newTestAdmin(t)
+	doReq(t, r, "POST", "/admin/clusters", "good-token", fullClusterMap("c1"))
+	doReq(t, r, "PUT", "/admin/clusters/c1", "good-token", map[string]any{"name": "updated"})
+	doReq(t, r, "DELETE", "/admin/clusters/c1", "good-token", nil)
+
+	w := doReq(t, r, "GET", "/admin/audit", "good-token", nil)
+	if w.Code != http.StatusOK {
+		t.Fatalf("got status %d, want %d", w.Code, http.StatusOK)
+	}
+
+	var entries []*audit.Entry
+	if err := json.Unmarshal(w.Body.Bytes(), &entries); err != nil {
+		t.Fatalf("unmarshal audit list: %v", err)
+	}
+	if len(entries) != 3 {
+		t.Fatalf("got %d audit entries, want 3", len(entries))
+	}
+
+	wantActions := []string{string(audit.ActionClusterDelete), string(audit.ActionClusterUpdate), string(audit.ActionClusterCreate)}
+	for i, want := range wantActions {
+		got := string(entries[i].Action)
+		if got != want {
+			t.Errorf("entry[%d].action=%s, want %s", i, got, want)
+		}
+	}
+}

internal/admin/web.go

@@ -0,0 +1,20 @@
+package admin
+
+import (
+	"embed"
+	"net/http"
+
+	"github.com/gin-gonic/gin"
+)
+
+//go:embed web/cluster.html
+var webFS embed.FS
+
+func webHandler(c *gin.Context) {
+	data, err := webFS.ReadFile("web/cluster.html")
+	if err != nil {
+		c.String(http.StatusInternalServerError, "cluster.html: %v", err)
+		return
+	}
+	c.Data(http.StatusOK, "text/html; charset=utf-8", data)
+}

internal/admin/web/cluster.html

@@ -0,0 +1,401 @@
+<!DOCTYPE html>
+<html lang='en'>
+<head>
+  <meta charset='utf-8'>
+  <meta name='viewport' content='width=device-width, initial-scale=1'>
+  <title>spark-mcp-go Admin</title>
+  <style>
+    :root {
+      --bg: #0d1117;
+      --panel: #161b22;
+      --border: #30363d;
+      --text: #c9d1d9;
+      --muted: #8b949e;
+      --accent: #58a6ff;
+      --danger: #f85149;
+      --ok: #3fb950;
+    }
+    * { box-sizing: border-box; }
+    body {
+      margin: 0;
+      font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', monospace;
+      background: var(--bg);
+      color: var(--text);
+      line-height: 1.5;
+    }
+    header {
+      display: flex;
+      align-items: center;
+      justify-content: space-between;
+      gap: 1rem;
+      padding: 1rem;
+      border-bottom: 1px solid var(--border);
+      background: var(--panel);
+      flex-wrap: wrap;
+    }
+    header h1 { margin: 0; font-size: 1.1rem; color: var(--accent); }
+    .auth { display: flex; align-items: center; gap: .5rem; flex-wrap: wrap; }
+    input[type='text'], input[type='password'], input[type='number'], select, textarea {
+      background: var(--bg);
+      color: var(--text);
+      border: 1px solid var(--border);
+      border-radius: 4px;
+      padding: .4rem .5rem;
+      font: inherit;
+    }
+    input[type='text'], input[type='password'], select { min-width: 220px; }
+    button {
+      background: var(--accent);
+      color: #fff;
+      border: none;
+      border-radius: 4px;
+      padding: .4rem .8rem;
+      font: inherit;
+      cursor: pointer;
+    }
+    button:hover { opacity: .9; }
+    button.danger { background: var(--danger); }
+    button.secondary { background: var(--border); color: var(--text); }
+    main { padding: 1rem; max-width: 1200px; margin: 0 auto; }
+    .toolbar { display: flex; gap: .5rem; margin-bottom: 1rem; align-items: center; }
+    .error {
+      background: rgba(248, 81, 73, .15);
+      border: 1px solid var(--danger);
+      color: var(--danger);
+      padding: .75rem;
+      border-radius: 4px;
+      margin-bottom: 1rem;
+      white-space: pre-wrap;
+    }
+    .hidden { display: none; }
+    table { width: 100%; border-collapse: collapse; }
+    th, td { text-align: left; padding: .5rem; border-bottom: 1px solid var(--border); }
+    th { color: var(--muted); font-weight: 600; }
+    td { vertical-align: middle; }
+    .actions { display: flex; gap: .4rem; }
+    .form-panel {
+      background: var(--panel);
+      border: 1px solid var(--border);
+      border-radius: 6px;
+      padding: 1rem;
+      margin-bottom: 1rem;
+    }
+    .form-grid {
+      display: grid;
+      grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
+      gap: 1rem;
+    }
+    .field { display: flex; flex-direction: column; gap: .25rem; }
+    .field label { color: var(--muted); font-size: .85rem; }
+    .field input, .field select { width: 100%; }
+    .form-actions { display: flex; gap: .5rem; margin-top: 1rem; }
+    .bool { flex-direction: row; align-items: center; gap: .5rem; }
+    .bool input { width: auto; }
+  </style>
+</head>
+<body>
+  <header>
+    <h1>spark-mcp-go Admin</h1>
+    <div class='auth'>
+      <label for='token'>Token</label>
+      <input id='token' type='password' placeholder='Bearer token' autocomplete='off'>
+      <button id='save-token'>Save</button>
+      <button id='logout' class='danger'>Logout</button>
+    </div>
+  </header>
+
+  <main>
+    <div id='error' class='error hidden'></div>
+
+    <section class='toolbar'>
+      <button id='new-cluster'>+ New Cluster</button>
+      <button id='refresh' class='secondary'>Refresh</button>
+    </section>
+
+    <section id='form-panel' class='form-panel hidden'>
+      <h2 id='form-title'>New Cluster</h2>
+      <form id='cluster-form'>
+        <div class='form-grid'>
+          <div class='field'>
+            <label for='f_id'>ID</label>
+            <input id='f_id' name='id' type='text' required>
+          </div>
+          <div class='field'>
+            <label for='f_name'>Name</label>
+            <input id='f_name' name='name' type='text' required>
+          </div>
+          <div class='field'>
+            <label for='f_rm_url'>RM URL</label>
+            <input id='f_rm_url' name='rm_url' type='text' required>
+          </div>
+          <div class='field'>
+            <label for='f_shs_url'>SHS URL</label>
+            <input id='f_shs_url' name='shs_url' type='text' required>
+          </div>
+          <div class='field' style='grid-column: 1 / -1;'>
+            <label for='f_spark_submit_execute_bin'>Spark Submit Binary</label>
+            <input id='f_spark_submit_execute_bin' name='spark_submit_execute_bin' type='text' required>
+          </div>
+          <div class='field bool'>
+            <input id='f_is_active' name='is_active' type='checkbox' checked>
+            <label for='f_is_active'>Active</label>
+          </div>
+          <div class='field bool'>
+            <input id='f_ssl_verify' name='ssl_verify' type='checkbox' checked>
+            <label for='f_ssl_verify'>SSL verify</label>
+          </div>
+          <div class='field'>
+            <label for='f_auth_type'>Auth type</label>
+            <select id='f_auth_type' name='auth_type'>
+              <option value='none' selected>none</option>
+              <option value='simple'>simple</option>
+              <option value='basic'>basic</option>
+            </select>
+          </div>
+          <div class='field'>
+            <label for='f_auth_username'>Auth username</label>
+            <input id='f_auth_username' name='auth_username' type='text'>
+          </div>
+          <div class='field'>
+            <label for='f_ssl_ca_bundle'>SSL CA bundle</label>
+            <input id='f_ssl_ca_bundle' name='ssl_ca_bundle' type='text'>
+          </div>
+          <div class='field'>
+            <label for='f_url_allowlist'>URL allowlist (comma separated)</label>
+            <input id='f_url_allowlist' name='url_allowlist' type='text'>
+          </div>
+          <div class='field'>
+            <label for='f_default_submit_args'>Default submit args (comma separated)</label>
+            <input id='f_default_submit_args' name='default_submit_args' type='text'>
+          </div>
+          <div class='field'>
+            <label for='f_rate_limit_per_min'>Rate limit / min</label>
+            <input id='f_rate_limit_per_min' name='rate_limit_per_min' type='number' min='0' value='10'>
+          </div>
+        </div>
+        <div class='form-actions'>
+          <button type='submit' id='btn-save'>Save</button>
+          <button type='button' id='btn-cancel' class='secondary'>Cancel</button>
+        </div>
+      </form>
+    </section>
+
+    <section>
+      <table>
+        <thead>
+          <tr>
+            <th>ID</th>
+            <th>Name</th>
+            <th>RM URL</th>
+            <th>Active</th>
+            <th>Auth</th>
+            <th>Actions</th>
+          </tr>
+        </thead>
+        <tbody id='clusters-body'>
+          <tr><td colspan='6'>Loading...</td></tr>
+        </tbody>
+      </table>
+    </section>
+  </main>
+
+  <script>
+    const $ = (sel) => document.querySelector(sel);
+    const API = '/admin';
+
+    const tokenInput = $('#token');
+    tokenInput.value = localStorage.getItem('adminToken') || '';
+
+    function headers() {
+      return {
+        'Authorization': 'Bearer ' + (localStorage.getItem('adminToken') || ''),
+        'Content-Type': 'application/json'
+      };
+    }
+
+    function showError(msg) {
+      const el = $('#error');
+      el.textContent = msg || '';
+      el.classList.toggle('hidden', !msg);
+    }
+
+    async function api(method, path, body) {
+      const opts = { method, headers: headers() };
+      if (body !== undefined) opts.body = JSON.stringify(body);
+      const res = await fetch(API + path, opts);
+      if (!res.ok) {
+        let detail = res.statusText;
+        try {
+          const j = await res.json();
+          if (j.error) detail = j.error;
+        } catch (_) {}
+        throw new Error(`${res.status} ${detail}`);
+      }
+      if (res.status === 204) return null;
+      return res.json();
+    }
+
+    function toArray(value) {
+      if (!value) return [];
+      return value.split(',').map(s => s.trim()).filter(Boolean);
+    }
+
+    function escapeHtml(s) {
+      return String(s)
+        .replace(/&/g, '&amp;')
+        .replace(/</g, '&lt;')
+        .replace(/>/g, '&gt;')
+        .replace(/"/g, '&quot;')
+        .replace(/'/g, '&#39;');
+    }
+
+    function renderList(clusters) {
+      const tbody = $('#clusters-body');
+      tbody.innerHTML = '';
+      if (!clusters.length) {
+        tbody.innerHTML = '<tr><td colspan="6">No clusters.</td></tr>';
+        return;
+      }
+      for (const c of clusters) {
+        const tr = document.createElement('tr');
+        tr.innerHTML = `
+          <td>${escapeHtml(c.id)}</td>
+          <td>${escapeHtml(c.name)}</td>
+          <td>${escapeHtml(c.rm_url)}</td>
+          <td>${c.is_active ? 'yes' : 'no'}</td>
+          <td>${escapeHtml(c.auth_type)}</td>
+          <td class='actions'>
+            <button data-id='${escapeHtml(c.id)}' class='edit'>Edit</button>
+            <button data-id='${escapeHtml(c.id)}' class='danger delete'>Delete</button>
+          </td>
+        `;
+        tbody.appendChild(tr);
+      }
+      tbody.querySelectorAll('.edit').forEach(b => b.addEventListener('click', handleEdit));
+      tbody.querySelectorAll('.delete').forEach(b => b.addEventListener('click', handleDelete));
+    }
+
+    async function loadClusters() {
+      showError('');
+      try {
+        const clusters = await api('GET', '/clusters');
+        renderList(clusters);
+      } catch (e) {
+        showError('Failed to load clusters: ' + e.message);
+      }
+    }
+
+    let editingId = null;
+
+    function showForm(id) {
+      editingId = id || null;
+      $('#form-title').textContent = editingId ? `Edit Cluster: ${editingId}` : 'New Cluster';
+      $('#f_id').disabled = !!editingId;
+      $('#form-panel').classList.remove('hidden');
+      if (!editingId) resetForm();
+    }
+
+    function hideForm() {
+      editingId = null;
+      $('#form-panel').classList.add('hidden');
+      resetForm();
+    }
+
+    function resetForm() {
+      $('#cluster-form').reset();
+      $('#f_id').disabled = false;
+      $('#f_is_active').checked = true;
+      $('#f_ssl_verify').checked = true;
+      $('#f_auth_type').value = 'none';
+      $('#f_rate_limit_per_min').value = '10';
+    }
+
+    function formToCluster() {
+      return {
+        id: $('#f_id').value.trim(),
+        name: $('#f_name').value.trim(),
+        rm_url: $('#f_rm_url').value.trim(),
+        shs_url: $('#f_shs_url').value.trim(),
+        spark_submit_execute_bin: $('#f_spark_submit_execute_bin').value.trim(),
+        is_active: $('#f_is_active').checked,
+        auth_type: $('#f_auth_type').value,
+        auth_username: $('#f_auth_username').value.trim(),
+        ssl_verify: $('#f_ssl_verify').checked,
+        ssl_ca_bundle: $('#f_ssl_ca_bundle').value.trim(),
+        url_allowlist: toArray($('#f_url_allowlist').value),
+        default_submit_args: toArray($('#f_default_submit_args').value),
+        rate_limit_per_min: parseInt($('#f_rate_limit_per_min').value, 10) || 0
+      };
+    }
+
+    async function handleEdit(e) {
+      const id = e.target.dataset.id;
+      showError('');
+      try {
+        const c = await api('GET', `/clusters/${encodeURIComponent(id)}`);
+        showForm(id);
+        $('#f_id').value = c.id || '';
+        $('#f_name').value = c.name || '';
+        $('#f_rm_url').value = c.rm_url || '';
+        $('#f_shs_url').value = c.shs_url || '';
+        $('#f_spark_submit_execute_bin').value = c.spark_submit_execute_bin || '';
+        $('#f_is_active').checked = !!c.is_active;
+        $('#f_ssl_verify').checked = c.ssl_verify !== false;
+        $('#f_auth_type').value = c.auth_type || 'none';
+        $('#f_auth_username').value = c.auth_username || '';
+        $('#f_ssl_ca_bundle').value = c.ssl_ca_bundle || '';
+        $('#f_url_allowlist').value = (c.url_allowlist || []).join(', ');
+        $('#f_default_submit_args').value = (c.default_submit_args || []).join(', ');
+        $('#f_rate_limit_per_min').value = String(c.rate_limit_per_min ?? 10);
+      } catch (err) {
+        showError('Failed to load cluster: ' + err.message);
+      }
+    }
+
+    async function handleDelete(e) {
+      const id = e.target.dataset.id;
+      if (!confirm(`Delete cluster ${id}?`)) return;
+      showError('');
+      try {
+        await api('DELETE', `/clusters/${encodeURIComponent(id)}`);
+        await loadClusters();
+      } catch (err) {
+        showError('Failed to delete cluster: ' + err.message);
+      }
+    }
+
+    $('#new-cluster').addEventListener('click', () => showForm(null));
+    $('#refresh').addEventListener('click', loadClusters);
+    $('#btn-cancel').addEventListener('click', hideForm);
+
+    $('#save-token').addEventListener('click', () => {
+      localStorage.setItem('adminToken', tokenInput.value.trim());
+      loadClusters();
+    });
+    $('#logout').addEventListener('click', () => {
+      localStorage.removeItem('adminToken');
+      tokenInput.value = '';
+      showError('Token cleared; reload the page after supplying a new token.');
+    });
+
+    $('#cluster-form').addEventListener('submit', async (e) => {
+      e.preventDefault();
+      showError('');
+      const payload = formToCluster();
+      try {
+        if (editingId) {
+          await api('PUT', `/clusters/${encodeURIComponent(editingId)}`, payload);
+        } else {
+          await api('POST', '/clusters', payload);
+        }
+        hideForm();
+        await loadClusters();
+      } catch (err) {
+        showError('Save failed: ' + err.message);
+      }
+    });
+
+    loadClusters();
+  </script>
+</body>
+</html>

internal/analyzer/rules.go

@@ -0,0 +1,116 @@
+package analyzer
+
+import "fmt"
+
+// Analyze runs all heuristic rules and returns the triggered findings.
+func Analyze(in Input, t Thresholds) []Finding {
+	var out []Finding
+	out = append(out, checkDataSkew(in, t)...)
+	out = append(out, checkGCPressure(in, t)...)
+	out = append(out, checkBottleneck(in, t)...)
+	return out
+}
+
+func checkDataSkew(in Input, t Thresholds) []Finding {
+	var out []Finding
+	for stage, m := range in.StageMetrics {
+		if m.MinPartitionBytes <= 0 || m.MaxPartitionBytes <= 0 {
+			continue
+		}
+		ratio := float64(m.MaxPartitionBytes) / float64(m.MinPartitionBytes)
+		if ratio > t.DataSkewRatio {
+			sev := SeverityWarning
+			if ratio > t.DataSkewRatio*2 {
+				sev = SeverityCritical
+			}
+			out = append(out, Finding{
+				Rule:     "data_skew",
+				Severity: sev,
+				Stage:    stage,
+				Evidence: map[string]any{
+					"max_partition_bytes": m.MaxPartitionBytes,
+					"min_partition_bytes": m.MinPartitionBytes,
+					"ratio":               ratio,
+					"threshold":           t.DataSkewRatio,
+				},
+				Message: fmt.Sprintf("stage %q 数据倾斜: max/min = %.1fx (阈值 %.1fx)", stage, ratio, t.DataSkewRatio),
+			})
+		}
+	}
+	return out
+}
+
+func checkGCPressure(in Input, t Thresholds) []Finding {
+	var out []Finding
+	var totalGC, totalCPU int64
+	var worstExec string
+	var worstRatio float64
+	for id, m := range in.ExecutorMetrics {
+		if m.CPUTimeMS <= 0 {
+			continue
+		}
+		r := float64(m.GCTimeMS) / float64(m.CPUTimeMS)
+		totalGC += m.GCTimeMS
+		totalCPU += m.CPUTimeMS
+		if r > worstRatio {
+			worstRatio = r
+			worstExec = id
+		}
+	}
+	if totalCPU > 0 {
+		aggRatio := float64(totalGC) / float64(totalCPU)
+		if aggRatio > t.GCPressureRatio {
+			sev := SeverityWarning
+			if aggRatio > t.GCPressureRatio*2 {
+				sev = SeverityCritical
+			}
+			out = append(out, Finding{
+				Rule:     "gc_pressure",
+				Severity: sev,
+				Evidence: map[string]any{
+					"aggregate_gc_ratio": aggRatio,
+					"worst_executor":     worstExec,
+					"worst_ratio":        worstRatio,
+					"threshold":          t.GCPressureRatio,
+				},
+				Message: fmt.Sprintf("GC 压力过高: 聚合 GC/CPU 比率 = %.1f%% (阈值 %.1f%%, 最差 executor %s = %.1f%%)",
+					aggRatio*100, t.GCPressureRatio*100, worstExec, worstRatio*100),
+			})
+		}
+	}
+	return out
+}
+
+func checkBottleneck(in Input, t Thresholds) []Finding {
+	var out []Finding
+	thresholdBytes := int64(t.BottleneckShuffleGB * (1 << 30))
+	for stage, m := range in.StageMetrics {
+		total := m.ShuffleReadBytes + m.ShuffleWriteBytes
+		if total < 0 {
+			continue // overflow guard
+		}
+		if total > thresholdBytes {
+			sev := SeverityInfo
+			if total > thresholdBytes*2 {
+				sev = SeverityWarning
+			}
+			if total > thresholdBytes*5 {
+				sev = SeverityCritical
+			}
+			out = append(out, Finding{
+				Rule:     "bottleneck",
+				Severity: sev,
+				Stage:    stage,
+				Evidence: map[string]any{
+					"shuffle_read_bytes":  m.ShuffleReadBytes,
+					"shuffle_write_bytes": m.ShuffleWriteBytes,
+					"shuffle_total_gb":    float64(total) / (1 << 30),
+					"threshold_gb":        t.BottleneckShuffleGB,
+				},
+				Message: fmt.Sprintf("stage %q 是潜在瓶颈: shuffle 总 %.1f GB (阈值 %.1f GB)",
+					stage, float64(total)/(1<<30), t.BottleneckShuffleGB),
+			})
+		}
+	}
+	return out
+}

internal/analyzer/rules_test.go

@@ -0,0 +1,149 @@
+package analyzer
+
+import (
+	"testing"
+)
+
+func TestAnalyze(t *testing.T) {
+	defaultThresholds := Thresholds{
+		DataSkewRatio:       3.0,
+		GCPressureRatio:     0.1,
+		BottleneckShuffleGB: 50.0,
+	}
+
+	t.Run("data_skew", func(t *testing.T) {
+		t.Run("normal", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 0": {MaxPartitionBytes: 100, MinPartitionBytes: 50},
+				},
+			}
+			if got := Analyze(in, defaultThresholds); len(got) != 0 {
+				t.Fatalf("expected no findings, got %+v", got)
+			}
+		})
+
+		t.Run("critical", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 1": {MaxPartitionBytes: 1000, MinPartitionBytes: 10},
+				},
+			}
+			got := Analyze(in, defaultThresholds)
+			if len(got) != 1 {
+				t.Fatalf("expected 1 finding, got %+v", got)
+			}
+			if got[0].Rule != "data_skew" || got[0].Severity != SeverityCritical || got[0].Stage != "stage 1" {
+				t.Fatalf("unexpected finding: %+v", got[0])
+			}
+		})
+
+		t.Run("zero_min_partition_bytes", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 2": {MaxPartitionBytes: 1000, MinPartitionBytes: 0},
+				},
+			}
+			if got := Analyze(in, defaultThresholds); len(got) != 0 {
+				t.Fatalf("expected no findings for zero min, got %+v", got)
+			}
+		})
+	})
+
+	t.Run("gc_pressure", func(t *testing.T) {
+		t.Run("normal", func(t *testing.T) {
+			in := Input{
+				ExecutorMetrics: map[string]ExecutorMetric{
+					"1": {GCTimeMS: 100, CPUTimeMS: 10000},
+				},
+			}
+			if got := Analyze(in, defaultThresholds); len(got) != 0 {
+				t.Fatalf("expected no findings, got %+v", got)
+			}
+		})
+
+		t.Run("warning", func(t *testing.T) {
+			in := Input{
+				ExecutorMetrics: map[string]ExecutorMetric{
+					"1": {GCTimeMS: 2000, CPUTimeMS: 10000},
+				},
+			}
+			got := Analyze(in, defaultThresholds)
+			if len(got) != 1 {
+				t.Fatalf("expected 1 finding, got %+v", got)
+			}
+			if got[0].Rule != "gc_pressure" || got[0].Severity != SeverityWarning {
+				t.Fatalf("unexpected finding: %+v", got[0])
+			}
+		})
+
+		t.Run("critical", func(t *testing.T) {
+			in := Input{
+				ExecutorMetrics: map[string]ExecutorMetric{
+					"1": {GCTimeMS: 5000, CPUTimeMS: 10000},
+				},
+			}
+			got := Analyze(in, defaultThresholds)
+			if len(got) != 1 {
+				t.Fatalf("expected 1 finding, got %+v", got)
+			}
+			if got[0].Rule != "gc_pressure" || got[0].Severity != SeverityCritical {
+				t.Fatalf("unexpected finding: %+v", got[0])
+			}
+		})
+
+		t.Run("zero_cpu_time", func(t *testing.T) {
+			in := Input{
+				ExecutorMetrics: map[string]ExecutorMetric{
+					"1": {GCTimeMS: 5000, CPUTimeMS: 0},
+				},
+			}
+			if got := Analyze(in, defaultThresholds); len(got) != 0 {
+				t.Fatalf("expected no findings for zero cpu, got %+v", got)
+			}
+		})
+	})
+
+	t.Run("bottleneck", func(t *testing.T) {
+		t.Run("normal", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 0": {ShuffleReadBytes: 1 << 30, ShuffleWriteBytes: 0},
+				},
+			}
+			if got := Analyze(in, defaultThresholds); len(got) != 0 {
+				t.Fatalf("expected no findings, got %+v", got)
+			}
+		})
+
+		t.Run("warning", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 1": {ShuffleReadBytes: 120 << 30, ShuffleWriteBytes: 0},
+				},
+			}
+			got := Analyze(in, defaultThresholds)
+			if len(got) != 1 {
+				t.Fatalf("expected 1 finding, got %+v", got)
+			}
+			if got[0].Rule != "bottleneck" || got[0].Severity != SeverityWarning {
+				t.Fatalf("unexpected finding: %+v", got[0])
+			}
+		})
+
+		t.Run("critical", func(t *testing.T) {
+			in := Input{
+				StageMetrics: map[string]StageMetric{
+					"stage 2": {ShuffleReadBytes: 300 << 30, ShuffleWriteBytes: 0},
+				},
+			}
+			got := Analyze(in, defaultThresholds)
+			if len(got) != 1 {
+				t.Fatalf("expected 1 finding, got %+v", got)
+			}
+			if got[0].Rule != "bottleneck" || got[0].Severity != SeverityCritical {
+				t.Fatalf("unexpected finding: %+v", got[0])
+			}
+		})
+	})
+}

internal/analyzer/types.go

@@ -0,0 +1,47 @@
+package analyzer
+
+// Severity is the heuristic finding severity.
+type Severity string
+
+const (
+	SeverityInfo     Severity = "info"
+	SeverityWarning  Severity = "warning"
+	SeverityCritical Severity = "critical"
+)
+
+// Finding is a single triggered heuristic rule.
+type Finding struct {
+	Rule     string         `json:"rule"`
+	Severity Severity       `json:"severity"`
+	Stage    string         `json:"stage,omitempty"`
+	Evidence map[string]any `json:"evidence"`
+	Message  string         `json:"message"`
+}
+
+// Thresholds holds analyzer thresholds injected at runtime.
+type Thresholds struct {
+	DataSkewRatio       float64
+	GCPressureRatio     float64
+	BottleneckShuffleGB float64
+}
+
+// Input is the data passed to the heuristic rules.
+type Input struct {
+	StageMetrics    map[string]StageMetric    `json:"stage_metrics"`
+	ExecutorMetrics map[string]ExecutorMetric `json:"executor_metrics"`
+}
+
+// StageMetric contains per-stage measurements.
+type StageMetric struct {
+	DurationMS        int64 `json:"duration_ms"`
+	ShuffleReadBytes  int64 `json:"shuffle_read_bytes"`
+	ShuffleWriteBytes int64 `json:"shuffle_write_bytes"`
+	MaxPartitionBytes int64 `json:"max_partition_bytes"`
+	MinPartitionBytes int64 `json:"min_partition_bytes"`
+}
+
+// ExecutorMetric contains per-executor measurements.
+type ExecutorMetric struct {
+	GCTimeMS  int64 `json:"gc_time_ms"`
+	CPUTimeMS int64 `json:"cpu_time_ms"`
+}

internal/audit/repo.go

@@ -0,0 +1,126 @@
+package audit
+
+import (
+	"context"
+	"database/sql"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"time"
+
+	"spark-mcp-go/internal/storage"
+)
+
+// Action enumerates admin write operations that are persisted to the audit log.
+type Action string
+
+const (
+	ActionClusterCreate Action = "cluster.create"
+	ActionClusterUpdate Action = "cluster.update"
+	ActionClusterDelete Action = "cluster.delete"
+)
+
+// Entry is one admin write operation recorded for accountability.
+//
+// Actor is stored as "admin:<token_name>" (the first 8 characters of the
+// admin token when multiple tokens are in use).
+// Details is a JSON-encoded string produced by MarshalDetails; callers that
+// do not need structured details can leave it empty.
+type Entry struct {
+	ID        int64     `json:"id"`
+	Timestamp time.Time `json:"timestamp"`
+	Actor     string    `json:"actor"`
+	Action    Action    `json:"action"`
+	ClusterID string    `json:"cluster_id,omitempty"`
+	Details   string    `json:"details,omitempty"`
+}
+
+// Repo writes and reads audit_log rows.
+type Repo struct {
+	db *storage.DB
+}
+
+// NewRepo returns a repository bound to the supplied DB handle.
+func NewRepo(db *storage.DB) *Repo {
+	return &Repo{db: db}
+}
+
+// Insert persists a single audit entry. A zero Timestamp is replaced with
+// the current time before storage.
+func (r *Repo) Insert(ctx context.Context, e *Entry) error {
+	if e.Timestamp.IsZero() {
+		e.Timestamp = time.Now()
+	}
+	var details any
+	if e.Details != "" {
+		details = e.Details
+	}
+	_, err := r.db.SQLDB().ExecContext(ctx, `
+		INSERT INTO audit_log (timestamp, actor, action, cluster_id, details)
+		VALUES (?, ?, ?, ?, ?)`,
+		e.Timestamp.UnixNano(), e.Actor, string(e.Action), e.ClusterID, details,
+	)
+	if err != nil {
+		return fmt.Errorf("audit: insert: %w", err)
+	}
+	return nil
+}
+
+// List returns the most recent audit entries ordered by timestamp descending.
+// A limit of zero or less falls back to 100.
+func (r *Repo) List(ctx context.Context, limit int) ([]*Entry, error) {
+	if limit <= 0 {
+		limit = 100
+	}
+	rows, err := r.db.SQLDB().QueryContext(ctx, `
+		SELECT id, timestamp, actor, action, cluster_id, details
+		FROM audit_log ORDER BY timestamp DESC LIMIT ?`, limit)
+	if err != nil {
+		return nil, fmt.Errorf("audit: list: %w", err)
+	}
+	defer rows.Close()
+
+	var out []*Entry
+	for rows.Next() {
+		var e Entry
+		var ts int64
+		var clusterID, details sql.NullString
+		var action string
+		if err := rows.Scan(&e.ID, &ts, &e.Actor, &action, &clusterID, &details); err != nil {
+			return nil, fmt.Errorf("audit: scan: %w", err)
+		}
+		e.Timestamp = time.Unix(0, ts)
+		e.Action = Action(action)
+		if clusterID.Valid {
+			e.ClusterID = clusterID.String
+		}
+		if details.Valid {
+			e.Details = details.String
+		}
+		out = append(out, &e)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, fmt.Errorf("audit: rows: %w", err)
+	}
+	if out == nil {
+		// Force empty array (not null) in JSON.
+		out = []*Entry{}
+	}
+	return out, nil
+}
+
+// MarshalDetails encodes any value as a JSON string for Entry.Details.
+func MarshalDetails(v any) (string, error) {
+	if v == nil {
+		return "", nil
+	}
+	b, err := json.Marshal(v)
+	if err != nil {
+		return "", err
+	}
+	return string(b), nil
+}
+
+// ErrNotFound is unused in this package (single-entry lookups are not
+// supported), but keeps the errors import available for future use.
+var _ = errors.New

internal/executor/spark_submit.go

@@ -0,0 +1,159 @@
+// Package executor runs the local spark-submit binary for the spark_submit Tool.
+//
+// It is the single place in the codebase that shells out to user-configured
+// binaries. To prevent command injection the binary is taken verbatim from
+// cluster.SparkSubmitExecuteBin and arguments are passed via exec.Command's
+// slice form — never as a single string, never via "sh -c".
+package executor
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"os/exec"
+	"regexp"
+	"strings"
+	"time"
+)
+
+// stdoutCap caps spark-submit stdout/stderr to keep the per-Tool log file
+// from blowing up on verbose jobs. 10 MiB is generous for the "Submitted
+// application application_xxx" line and a few progress lines.
+const stdoutCap = 10 << 20
+
+// appIDPattern matches the YARN line printed on successful submission:
+//
+//	"Submitted application application_12345_0001"
+var appIDPattern = regexp.MustCompile(`application_\d+_\d+`)
+
+// SparkSubmitOpts is the resolved command line for a single spark-submit run.
+//
+// Binary must be an absolute path (validated at cluster create/update time).
+// Args are the post-binary CLI args, already including any cluster-default
+// args prepended by the caller.
+type SparkSubmitOpts struct {
+	Binary string
+	Args   []string
+	// Timeout is the wall-clock budget for the entire run. Zero means no
+	// timeout. We use context.WithTimeout to enforce it.
+	Timeout time.Duration
+}
+
+// Result is what spark_submit returns to the LLM.
+type Result struct {
+	AppID      string `json:"app_id,omitempty"`
+	ExitCode   int    `json:"exit_code"`
+	StdoutTail string `json:"stdout_tail"`
+	StderrTail string `json:"stderr_tail"`
+	DurationMS int64  `json:"duration_ms"`
+}
+
+// ErrBinaryEmpty is returned when the cluster has no spark-submit binary
+// configured. The agent should call list_clusters to surface this.
+var ErrBinaryEmpty = errors.New("executor: spark_submit_execute_bin is empty")
+
+// Run executes the configured spark-submit and returns the parsed result.
+//
+// Command construction: exec.Command(name, args...) — the slice form means
+// no shell, so a malicious arg like "foo; rm -rf /" is passed as a single
+// argument to spark-submit, never executed.
+func Run(ctx context.Context, opts SparkSubmitOpts) (*Result, error) {
+	if strings.TrimSpace(opts.Binary) == "" {
+		return nil, ErrBinaryEmpty
+	}
+
+	if opts.Timeout > 0 {
+		var cancel context.CancelFunc
+		ctx, cancel = context.WithTimeout(ctx, opts.Timeout)
+		defer cancel()
+	}
+
+	cmd := exec.CommandContext(ctx, opts.Binary, opts.Args...)
+	// Critical: do NOT call cmd.Run() with /bin/sh. The slice form above is
+	// the entire injection defense.
+
+	stdout := &capBuf{cap: stdoutCap}
+	stderr := &capBuf{cap: stdoutCap}
+	cmd.Stdout = stdout
+	cmd.Stderr = stderr
+
+	start := time.Now()
+	err := cmd.Run()
+	dur := time.Since(start)
+
+	res := &Result{
+		ExitCode:   exitCodeFromErr(err),
+		StdoutTail: stdout.String(),
+		StderrTail: stderr.String(),
+		DurationMS: dur.Milliseconds(),
+	}
+
+	// app_id: prefer stdout, fall back to stderr. Both are searched only on
+	// the tail (cap-bounded) so we don't pay for the full output.
+	if id := matchAppID(res.StdoutTail); id != "" {
+		res.AppID = id
+	} else if id := matchAppID(res.StderrTail); id != "" {
+		res.AppID = id
+	}
+
+	if err != nil {
+		// Wrap the underlying error but keep the parsed result so the LLM
+		// can see exit_code and stderr even on failure.
+		return res, fmt.Errorf("executor: spark-submit failed: %w", err)
+	}
+	return res, nil
+}
+
+func matchAppID(s string) string {
+	if s == "" {
+		return ""
+	}
+	return appIDPattern.FindString(s)
+}
+
+// exitCodeFromErr returns 0 on nil, the process's exit code on *exec.ExitError,
+// and 1 on other errors (e.g. context deadline). exec.ExitError is the typed
+// way to ask the OS-level exit code without inspecting strings.
+func exitCodeFromErr(err error) int {
+	if err == nil {
+		return 0
+	}
+	var exitErr *exec.ExitError
+	if errors.As(err, &exitErr) {
+		return exitErr.ExitCode()
+	}
+	// context.DeadlineExceeded or failed to start — surface as 1.
+	return 1
+}
+
+// capBuf is an io.Writer that keeps at most `cap` bytes from the end.
+//
+// The point isn't to deliver partial stdout to the LLM (LLM rarely needs the
+// full stream) — it's to bound memory and the per-Tool log file. We keep
+// the tail because that's where "Submitted application application_xxx" and
+// final error messages live.
+type capBuf struct {
+	cap int
+	buf bytes.Buffer
+}
+
+func (b *capBuf) Write(p []byte) (int, error) {
+	n, err := b.buf.Write(p)
+	if err != nil {
+		return n, err
+	}
+	if b.buf.Len() > b.cap {
+		// Drop the front. Cheaper than a ring buffer; works fine at 10 MiB.
+		overflow := b.buf.Len() - b.cap
+		_ = b.buf.Next(overflow)
+	}
+	return len(p), nil
+}
+
+func (b *capBuf) String() string { return b.buf.String() }
+
+// Ensure io.Discard stays referenced; we keep this in case future variants
+// want to write to a logger instead of capturing.
+var _ = io.Discard

internal/executor/spark_submit_test.go

@@ -0,0 +1,104 @@
+package executor
+
+import (
+	"context"
+	"runtime"
+	"strings"
+	"testing"
+	"time"
+)
+
+// TestRun_BinaryEmpty covers the ErrBinaryEmpty path. The agent-facing
+// behavior is "list the cluster, fix the binary field" — we just guard the
+// error here.
+func TestRun_BinaryEmpty(t *testing.T) {
+	_, err := Run(context.Background(), SparkSubmitOpts{Binary: "", Args: []string{"--help"}})
+	if err == nil || !strings.Contains(err.Error(), "empty") {
+		t.Fatalf("want empty-binary error, got %v", err)
+	}
+}
+
+// TestRun_FakeBinary uses /bin/echo as a stand-in for spark-submit. We
+// verify slice-form arg passing (no shell) by including an arg that would
+// be catastrophic if interpreted by a shell.
+func TestRun_FakeBinary(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("uses unix binary")
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+	defer cancel()
+
+	res, err := Run(ctx, SparkSubmitOpts{
+		Binary: "/bin/echo",
+		Args:   []string{"--master", "yarn; rm -rf /tmp/this-should-not-exist"},
+		Timeout: 3 * time.Second,
+	})
+	if err != nil {
+		t.Fatalf("Run: %v", err)
+	}
+	if res.ExitCode != 0 {
+		t.Fatalf("ExitCode: got %d, want 0", res.ExitCode)
+	}
+	// The arg must be passed through verbatim, not interpreted by a shell.
+	if !strings.Contains(res.StdoutTail, "yarn; rm -rf /tmp/this-should-not-exist") {
+		t.Errorf("stdout missing literal arg, got: %q", res.StdoutTail)
+	}
+	if res.AppID != "" {
+		t.Errorf("AppID should be empty for echo, got %q", res.AppID)
+	}
+}
+
+// TestRun_NonZeroExit captures the exit code when the binary returns !=0.
+func TestRun_NonZeroExit(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		t.Skip("uses unix binary")
+	}
+	res, err := Run(context.Background(), SparkSubmitOpts{
+		Binary: "/bin/sh",
+		Args:   []string{"-c", "exit 7"},
+		Timeout: 3 * time.Second,
+	})
+	if err == nil {
+		t.Fatal("want non-nil error from non-zero exit")
+	}
+	if res.ExitCode != 7 {
+		t.Errorf("ExitCode: got %d, want 7", res.ExitCode)
+	}
+}
+
+// TestMatchAppID locks down the regex so a YARN output tweak doesn't
+// silently break app_id extraction.
+func TestMatchAppID(t *testing.T) {
+	cases := []struct {
+		in, want string
+	}{
+		{"Submitted application application_12345_0001", "application_12345_0001"},
+		{"... some prefix application_999_42 ... tail", "application_999_42"},
+		{"no app id here", ""},
+		{"", ""},
+		{"application_1_2 application_3_4", "application_1_2"}, // first match
+	}
+	for _, tc := range cases {
+		got := matchAppID(tc.in)
+		if got != tc.want {
+			t.Errorf("matchAppID(%q) = %q, want %q", tc.in, got, tc.want)
+		}
+	}
+}
+
+// TestCapBuf ensures the tail-capping behavior: writing more than `cap`
+// bytes keeps the most recent `cap` bytes.
+func TestCapBuf(t *testing.T) {
+	b := &capBuf{cap: 10}
+	// Write 20 bytes total in two chunks.
+	if _, err := b.Write([]byte("0123456789")); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := b.Write([]byte("ABCDEFGHIJ")); err != nil {
+		t.Fatal(err)
+	}
+	// Tail should be the last 10 bytes: "ABCDEFGHIJ".
+	if got := b.String(); got != "ABCDEFGHIJ" {
+		t.Errorf("capBuf tail: got %q, want %q", got, "ABCDEFGHIJ")
+	}
+}

internal/httpclient/auth.go

@@ -0,0 +1,53 @@
+package httpclient
+
+import (
+	"errors"
+	"fmt"
+	"net/http"
+	"net/url"
+
+	"spark-mcp-go/internal/cluster"
+)
+
+// ApplyAuth rewrites req according to the cluster's configured AuthType.
+//   - none / empty: no-op
+//   - simple:       appends ?user.name=<username> (YARN SimpleAuth)
+//   - basic:        attaches HTTP Basic credentials
+//
+// baseURL is kept for future use (e.g. reconstructing absolute URLs in Phase 5)
+// and is intentionally unused in this version.
+func ApplyAuth(req *http.Request, baseURL string, c *cluster.Cluster) error {
+	if baseURL == "" {
+		// baseURL is reserved for Phase 5 host rewriting.
+	}
+
+	switch c.AuthType {
+	case cluster.AuthNone, "":
+		return nil
+
+	case cluster.AuthSimple:
+		u, err := url.Parse(req.URL.String())
+		if err != nil {
+			return fmt.Errorf("httpclient: simple auth parse url: %w", err)
+		}
+		q := u.Query()
+		user := c.AuthUsername
+		if user == "" {
+			user = "yarn"
+		}
+		q.Set("user.name", user)
+		u.RawQuery = q.Encode()
+		req.URL = u
+		return nil
+
+	case cluster.AuthBasic:
+		if c.AuthUsername == "" || c.AuthPassword == "" {
+			return errors.New("httpclient: basic auth requires username and password")
+		}
+		req.SetBasicAuth(c.AuthUsername, c.AuthPassword)
+		return nil
+
+	default:
+		return fmt.Errorf("httpclient: unknown auth type %q", c.AuthType)
+	}
+}

internal/httpclient/auth_test.go

@@ -0,0 +1,99 @@
+package httpclient
+
+import (
+	"encoding/base64"
+	"net/http"
+	"strings"
+	"testing"
+
+	"spark-mcp-go/internal/cluster"
+)
+
+func TestApplyAuth_None(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://example.com/foo", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthNone}
+	if err := ApplyAuth(req, "", c); err != nil {
+		t.Fatalf("apply auth: %v", err)
+	}
+	if req.Header.Get("Authorization") != "" {
+		t.Errorf("expected no Authorization header")
+	}
+	if req.URL.Query().Get("user.name") != "" {
+		t.Errorf("expected no user.name query parameter")
+	}
+}
+
+func TestApplyAuth_Simple_DefaultUser(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/ws/v1/cluster/apps", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthSimple}
+	if err := ApplyAuth(req, "", c); err != nil {
+		t.Fatalf("apply auth: %v", err)
+	}
+	if got := req.URL.Query().Get("user.name"); got != "yarn" {
+		t.Errorf("user.name=%q, want yarn", got)
+	}
+}
+
+func TestApplyAuth_Simple_CustomUser(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/ws/v1/cluster/apps", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthSimple, AuthUsername: "alice"}
+	if err := ApplyAuth(req, "", c); err != nil {
+		t.Fatalf("apply auth: %v", err)
+	}
+	if got := req.URL.Query().Get("user.name"); got != "alice" {
+		t.Errorf("user.name=%q, want alice", got)
+	}
+}
+
+func TestApplyAuth_Simple_PreservesExistingQuery(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/ws/v1/cluster/apps?foo=bar", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthSimple}
+	if err := ApplyAuth(req, "", c); err != nil {
+		t.Fatalf("apply auth: %v", err)
+	}
+	q := req.URL.Query()
+	if q.Get("foo") != "bar" {
+		t.Errorf("foo=%q, want bar", q.Get("foo"))
+	}
+	if q.Get("user.name") != "yarn" {
+		t.Errorf("user.name=%q, want yarn", q.Get("user.name"))
+	}
+	raw := req.URL.RawQuery
+	if !strings.Contains(raw, "foo=bar") || !strings.Contains(raw, "user.name=yarn") {
+		t.Errorf("raw query %q missing expected parameters", raw)
+	}
+}
+
+func TestApplyAuth_Basic(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/foo", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthBasic, AuthUsername: "u", AuthPassword: "p"}
+	if err := ApplyAuth(req, "", c); err != nil {
+		t.Fatalf("apply auth: %v", err)
+	}
+	want := "Basic " + base64.StdEncoding.EncodeToString([]byte("u:p"))
+	if got := req.Header.Get("Authorization"); got != want {
+		t.Errorf("Authorization=%q, want %q", got, want)
+	}
+}
+
+func TestApplyAuth_Basic_MissingCreds(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/foo", nil)
+	c := &cluster.Cluster{AuthType: cluster.AuthBasic, AuthUsername: "", AuthPassword: "p"}
+	if err := ApplyAuth(req, "", c); err == nil {
+		t.Error("expected error for missing username")
+	}
+
+	req2, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/foo", nil)
+	c2 := &cluster.Cluster{AuthType: cluster.AuthBasic, AuthUsername: "u", AuthPassword: ""}
+	if err := ApplyAuth(req2, "", c2); err == nil {
+		t.Error("expected error for missing password")
+	}
+}
+
+func TestApplyAuth_UnknownAuthType(t *testing.T) {
+	req, _ := http.NewRequest(http.MethodGet, "http://rm.example.com/foo", nil)
+	c := &cluster.Cluster{AuthType: "kerberos"}
+	if err := ApplyAuth(req, "", c); err == nil {
+		t.Error("expected error for unknown auth type")
+	}
+}

internal/httpclient/client.go

@@ -0,0 +1,60 @@
+package httpclient
+
+import (
+	"context"
+	"io"
+	"net/http"
+	"time"
+)
+
+// Config tunes the shared HTTP client.
+type Config struct {
+	Timeout          time.Duration // e.g. 30s
+	MaxResponseBytes int64         // e.g. 1 << 20
+}
+
+// Client is a thin, SSRF-aware wrapper around net/http.Client.
+type Client struct {
+	cfg Config
+	hc  *http.Client
+}
+
+// New creates a Client with redirect following disabled.
+// Redirect handling is intentionally left to DoWithRedirect.
+func New(cfg Config) *Client {
+	return &Client{
+		cfg: cfg,
+		hc: &http.Client{
+			Timeout: cfg.Timeout,
+			CheckRedirect: func(req *http.Request, via []*http.Request) error {
+				return http.ErrUseLastResponse
+			},
+		},
+	}
+}
+
+// Do executes a single HTTP request. The response body is truncated to
+// MaxResponseBytes. The request URL is validated by CheckURL before sending;
+// hosts listed in allowedHosts bypass SSRF checks.
+//
+// TODO(prod): implement IP-pinned dialer to fully close TOCTOU between the
+// SSRF check here and the actual TCP dial performed by net/http.
+func (c *Client) Do(ctx context.Context, req *http.Request, allowedHosts ...string) (*http.Response, error) {
+	if err := CheckURL(req.URL.String(), allowedHosts...); err != nil {
+		return nil, err
+	}
+
+	resp, err := c.hc.Do(req.WithContext(ctx))
+	if err != nil {
+		return nil, err
+	}
+
+	resp.Body = struct {
+		io.Reader
+		io.Closer
+	}{
+		Reader: io.LimitReader(resp.Body, c.cfg.MaxResponseBytes),
+		Closer: resp.Body,
+	}
+	return resp, nil
+}

internal/httpclient/client_test.go

@@ -0,0 +1,56 @@
+package httpclient
+
+import (
+	"context"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestClient_Timeout(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		time.Sleep(2 * time.Second)
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 100 * time.Millisecond, MaxResponseBytes: 1024})
+	req, err := http.NewRequest(http.MethodGet, srv.URL, nil)
+	if err != nil {
+		t.Fatalf("new request: %v", err)
+	}
+	_, err = c.Do(context.Background(), req)
+	if err == nil {
+		t.Fatal("expected timeout error, got nil")
+	}
+}
+
+func TestClient_MaxResponseBytes(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+		w.Write([]byte(strings.Repeat("a", 2048)))
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 512})
+	req, err := http.NewRequest(http.MethodGet, srv.URL, nil)
+	if err != nil {
+		t.Fatalf("new request: %v", err)
+	}
+	resp, err := c.Do(context.Background(), req, "127.0.0.1")
+	if err != nil {
+		t.Fatalf("do: %v", err)
+	}
+	defer resp.Body.Close()
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		t.Fatalf("read body: %v", err)
+	}
+	if len(body) > 512 {
+		t.Fatalf("expected at most 512 bytes, got %d", len(body))
+	}
+}

internal/httpclient/redirect.go

@@ -0,0 +1,62 @@
+package httpclient
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"net/http"
+)
+
+const defaultMaxRedirects = 5
+
+// DoWithRedirect 手动跟随 3xx,跨主机跳转保留 Authorization header。
+// maxRedirects == 0 禁止 redirect(等同 Do)
+// maxRedirects < 0 用 defaultMaxRedirects
+func (c *Client) DoWithRedirect(ctx context.Context, req *http.Request, maxRedirects int, allowedHosts ...string) (*http.Response, error) {
+	if maxRedirects == 0 {
+		return c.Do(ctx, req, allowedHosts...)
+	}
+	if maxRedirects < 0 {
+		maxRedirects = defaultMaxRedirects
+	}
+
+	current := req
+	remaining := maxRedirects
+	for {
+		resp, err := c.Do(ctx, current, allowedHosts...)
+		if err != nil {
+			return nil, err
+		}
+		if resp.StatusCode < 300 || resp.StatusCode >= 400 {
+			return resp, nil
+		}
+
+		// 3xx
+		if remaining == 0 {
+			_ = resp.Body.Close()
+			return nil, errors.New("httpclient: too many redirects")
+		}
+		remaining--
+
+		loc := resp.Header.Get("Location")
+		if loc == "" {
+			_ = resp.Body.Close()
+			return nil, fmt.Errorf("httpclient: %d %s without Location header", resp.StatusCode, resp.Status)
+		}
+		next, err := current.URL.Parse(loc)
+		if err != nil {
+			_ = resp.Body.Close()
+			return nil, fmt.Errorf("httpclient: parse redirect Location %q: %w", loc, err)
+		}
+		_ = resp.Body.Close()
+
+		newReq, err := http.NewRequestWithContext(ctx, current.Method, next.String(), nil)
+		if err != nil {
+			return nil, fmt.Errorf("httpclient: build redirect request: %w", err)
+		}
+		// **保留** header (Authorization 等)
+		newReq.Header = current.Header.Clone()
+		newReq.Body = nil
+		current = newReq
+	}
+}

internal/httpclient/redirect_test.go

@@ -0,0 +1,142 @@
+package httpclient
+
+import (
+	"context"
+	"encoding/base64"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestDoWithRedirect_SameHost(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/start":
+			w.Header().Set("Location", "/final")
+			w.WriteHeader(http.StatusFound)
+		case "/final":
+			w.WriteHeader(http.StatusOK)
+			w.Write([]byte("ok"))
+		default:
+			w.WriteHeader(http.StatusNotFound)
+		}
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 1024})
+	req, err := http.NewRequest(http.MethodGet, srv.URL+"/start", nil)
+	if err != nil {
+		t.Fatalf("new request: %v", err)
+	}
+	resp, err := c.DoWithRedirect(context.Background(), req, 5, "127.0.0.1")
+	if err != nil {
+		t.Fatalf("do with redirect: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		t.Fatalf("status=%d, want 200", resp.StatusCode)
+	}
+	body, _ := io.ReadAll(resp.Body)
+	if string(body) != "ok" {
+		t.Fatalf("body=%q, want ok", string(body))
+	}
+}
+
+func TestDoWithRedirect_CrossHost_PreservesAuth(t *testing.T) {
+	srv2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+		w.Write([]byte(r.Header.Get("Authorization")))
+	}))
+	defer srv2.Close()
+
+	srv1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/start" {
+			w.Header().Set("Location", srv2.URL+"/final")
+			w.WriteHeader(http.StatusTemporaryRedirect)
+			return
+		}
+		w.WriteHeader(http.StatusNotFound)
+	}))
+	defer srv1.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 1024})
+	req, err := http.NewRequest(http.MethodGet, srv1.URL+"/start", nil)
+	if err != nil {
+		t.Fatalf("new request: %v", err)
+	}
+	req.SetBasicAuth("u", "p")
+
+	resp, err := c.DoWithRedirect(context.Background(), req, 5, "127.0.0.1")
+	if err != nil {
+		t.Fatalf("do with redirect: %v", err)
+	}
+	defer resp.Body.Close()
+
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		t.Fatalf("read body: %v", err)
+	}
+	want := "Basic " + base64.StdEncoding.EncodeToString([]byte("u:p"))
+	if string(body) != want {
+		t.Fatalf("body=%q, want %q", string(body), want)
+	}
+}
+
+func TestDoWithRedirect_MaxLimit(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Location", "/loop")
+		w.WriteHeader(http.StatusFound)
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 1024})
+	req, _ := http.NewRequest(http.MethodGet, srv.URL+"/loop", nil)
+	_, err := c.DoWithRedirect(context.Background(), req, 2, "127.0.0.1")
+	if err == nil {
+		t.Fatal("expected too many redirects error")
+	}
+	if !strings.Contains(err.Error(), "too many redirects") {
+		t.Fatalf("expected too many redirects, got %v", err)
+	}
+}
+
+func TestDoWithRedirect_NoLocation(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusFound)
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 1024})
+	req, _ := http.NewRequest(http.MethodGet, srv.URL, nil)
+	_, err := c.DoWithRedirect(context.Background(), req, 5, "127.0.0.1")
+	if err == nil {
+		t.Fatal("expected error for missing Location")
+	}
+	if !strings.Contains(err.Error(), "without Location header") {
+		t.Fatalf("expected missing Location error, got %v", err)
+	}
+}
+
+func TestDoWithRedirect_ZeroDisallows(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Location", "/elsewhere")
+		w.WriteHeader(http.StatusFound)
+	}))
+	defer srv.Close()
+
+	c := New(Config{Timeout: 5 * time.Second, MaxResponseBytes: 1024})
+	req, _ := http.NewRequest(http.MethodGet, srv.URL, nil)
+	resp, err := c.DoWithRedirect(context.Background(), req, 0, "127.0.0.1")
+	if err != nil {
+		t.Fatalf("do: %v", err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusFound {
+		t.Fatalf("status=%d, want 302", resp.StatusCode)
+	}
+}

internal/httpclient/ssrf.go

@@ -0,0 +1,127 @@
+package httpclient
+
+import (
+	"fmt"
+	"net"
+	"net/url"
+	"strings"
+)
+
+// privateCIDRs lists IPv4 and IPv6 ranges that must not be reached by the
+// outbound HTTP client (SSRF prevention). Includes RFC 1918, loopback, link-
+// local, multicast, documentation, and IPv4-mapped IPv6 ranges.
+var privateCIDRs = []string{
+	"0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10",
+	"127.0.0.0/8", "169.254.0.0/16",
+	"172.16.0.0/12", "192.0.0.0/24", "192.168.0.0/16",
+	"198.18.0.0/15", "224.0.0.0/4", "240.0.0.0/4",
+	"::1/128", "fc00::/7", "fe80::/10", "::ffff:0:0/96",
+}
+
+// privateNets holds the parsed CIDR blocks from privateCIDRs.
+var privateNets []*net.IPNet
+
+func init() {
+	for _, cidr := range privateCIDRs {
+		_, ipNet, err := net.ParseCIDR(cidr)
+		if err != nil {
+			// net.ParseCIDR only fails on malformed input; the list is static.
+			panic(fmt.Sprintf("httpclient: unable to parse private CIDR %q: %v", cidr, err))
+		}
+		privateNets = append(privateNets, ipNet)
+	}
+}
+
+// CheckURL validates that rawURL is an http(s) URL and that its hostname does
+// not resolve to a private/reserved IP address. Hosts matching any entry in
+// allowedHosts bypass the IP check.
+//
+// allowedHosts entries support exact matches, suffix matches (e.g.
+// "example.com" matches "rm.example.com"), and wildcard suffixes (e.g.
+// "*.example.com").
+func CheckURL(rawURL string, allowedHosts ...string) error {
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		return fmt.Errorf("httpclient: parse url: %w", err)
+	}
+
+	if !strings.EqualFold(u.Scheme, "http") && !strings.EqualFold(u.Scheme, "https") {
+		return fmt.Errorf("httpclient: unsupported url scheme %q", u.Scheme)
+	}
+
+	host := strings.ToLower(u.Hostname())
+	if host == "" {
+		return fmt.Errorf("httpclient: url has no hostname")
+	}
+
+	for _, allowed := range allowedHosts {
+		if matchAllowedHost(host, allowed) {
+			return nil
+		}
+	}
+
+	ips, err := net.LookupIP(host)
+	if err != nil {
+		return fmt.Errorf("httpclient: lookup host %q: %w", host, err)
+	}
+	if len(ips) == 0 {
+		return fmt.Errorf("httpclient: host %q resolved to no IPs", host)
+	}
+
+	for _, ip := range ips {
+		if isPrivateIP(ip) {
+			return fmt.Errorf("httpclient: host %q resolves to private IP %s", host, ip)
+		}
+	}
+
+	return nil
+}
+
+// isPrivateIP reports whether ip falls inside any of the reserved CIDR blocks.
+func isPrivateIP(ip net.IP) bool {
+	if v4 := ip.To4(); v4 != nil {
+		for _, ipNet := range privateNets {
+			if isIPv4MappedNet(ipNet) {
+				continue
+			}
+			if ipNet.Contains(v4) {
+				return true
+			}
+		}
+		return false
+	}
+	for _, ipNet := range privateNets {
+		if ipNet.Contains(ip) {
+			return true
+		}
+	}
+	return false
+}
+
+// isIPv4MappedNet reports whether n is an IPv4-mapped IPv6 CIDR such as
+// ::ffff:0:0/96. These must not be used to classify plain IPv4 addresses.
+func isIPv4MappedNet(n *net.IPNet) bool {
+	return len(n.IP) == net.IPv6len && n.IP.To4() != nil && n.IP[10] == 0xff && n.IP[11] == 0xff
+}
+
+// matchAllowedHost reports whether host matches pattern. Patterns may be exact
+// hostnames, domain suffixes ("example.com" matches "rm.example.com"), or
+// wildcard suffixes ("*.example.com").
+func matchAllowedHost(host, pattern string) bool {
+	pattern = strings.ToLower(strings.TrimSpace(pattern))
+	if pattern == "" {
+		return false
+	}
+	if pattern == host {
+		return true
+	}
+
+	if strings.HasPrefix(pattern, "*.") {
+		pattern = pattern[2:]
+	}
+	if strings.HasPrefix(pattern, ".") {
+		pattern = pattern[1:]
+	}
+
+	return host == pattern || strings.HasSuffix(host, "."+pattern)
+}

internal/httpclient/ssrf_test.go

@@ -0,0 +1,79 @@
+package httpclient
+
+import (
+	"net"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+func TestCheckURL_RejectsLoopback(t *testing.T) {
+	err := CheckURL("http://127.0.0.1:8080/path")
+	if err == nil {
+		t.Fatal("expected error for loopback URL")
+	}
+}
+
+func TestCheckURL_RejectsLinkLocal(t *testing.T) {
+	err := CheckURL("http://169.254.169.254/latest/meta-data")
+	if err == nil {
+		t.Fatal("expected error for link-local URL")
+	}
+}
+
+func TestCheckURL_AllowsAllowlisted(t *testing.T) {
+	err := CheckURL("http://127.0.0.1:8080", "127.0.0.1")
+	if err != nil {
+		t.Fatalf("expected allowlisted host to pass: %v", err)
+	}
+	if !matchAllowedHost("rm.example.com", "*.example.com") {
+		t.Fatal("expected *.example.com to match rm.example.com")
+	}
+}
+
+func TestCheckURL_RejectsBadScheme(t *testing.T) {
+	cases := []string{"file:///etc/passwd", "gopher://x", "ftp://x"}
+	for _, raw := range cases {
+		err := CheckURL(raw)
+		if err == nil {
+			t.Errorf("expected error for %q", raw)
+		}
+	}
+}
+
+func TestCheckURL_RejectsEmptyHost(t *testing.T) {
+	err := CheckURL("http:///path")
+	if err == nil {
+		t.Fatal("expected error for URL with empty host")
+	}
+}
+
+func TestCheckURL_AcceptsPublic(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+	}))
+	defer srv.Close()
+
+	err := CheckURL(srv.URL, "127.0.0.1")
+	if err != nil {
+		t.Fatalf("expected allowlisted server URL to pass: %v", err)
+	}
+}
+
+func TestIsPrivateIP(t *testing.T) {
+	cases := []struct {
+		ip   string
+		want bool
+	}{
+		{"10.1.2.3", true},
+		{"8.8.8.8", false},
+		{"::1", true},
+		{"2001:db8::1", false},
+	}
+	for _, tc := range cases {
+		got := isPrivateIP(net.ParseIP(tc.ip))
+		if got != tc.want {
+			t.Errorf("isPrivateIP(%q) = %v, want %v", tc.ip, got, tc.want)
+		}
+	}
+}

internal/mcp/server.go

@@ -0,0 +1,36 @@
+package mcp
+
+import (
+	"net/http"
+
+	"github.com/mark3labs/mcp-go/server"
+
+	"spark-mcp-go/internal/mcp/tools"
+)
+
+// NewServer builds the MCP server with all configured Tools.
+func NewServer(deps *tools.Deps) *server.MCPServer {
+	s := server.NewMCPServer("spark-mcp-go", "0.0.0",
+		server.WithToolCapabilities(false),
+		server.WithRecovery(),
+		server.WithLogging(),
+	)
+	s.AddTool(tools.NewListClustersTool(), deps.ListClustersHandler)
+	s.AddTool(tools.NewSparkSubmitTool(), deps.SparkSubmitHandler)
+	s.AddTool(tools.NewListApplicationsTool(), deps.ListApplicationsHandler)
+	s.AddTool(tools.NewGetApplicationStatusTool(), deps.GetApplicationStatusHandler)
+	s.AddTool(tools.NewGetApplicationLogsTool(), deps.GetApplicationLogsHandler)
+	s.AddTool(tools.NewKillApplicationTool(), deps.KillApplicationHandler)
+	s.AddTool(tools.NewFetchURLTool(), deps.FetchURLHandler)
+	s.AddTool(tools.NewUploadFileTool(), deps.UploadFileHandler)
+	s.AddTool(tools.NewFetchSparkMetricsTool(), deps.FetchSparkMetricsHandler)
+	s.AddTool(tools.NewFetchClusterEnvTool(), deps.FetchClusterEnvHandler)
+	s.AddTool(tools.NewAnalyzeSparkLogTool(), deps.AnalyzeSparkLogHandler)
+	return s
+}
+
+// Handler exposes the MCP server as an http.Handler for main.go to mount.
+func Handler(deps *tools.Deps) (http.Handler, error) {
+	s := NewServer(deps)
+	return server.NewStreamableHTTPServer(s), nil
+}

internal/mcp/tools/analyze_spark_log.go

@@ -0,0 +1,116 @@
+package tools
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/analyzer"
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/rm"
+)
+
+const AnalyzeSparkLogName = "analyze_spark_log"
+
+const defaultAnalyzeMaxLogBytes = 10240
+
+// NewAnalyzeSparkLogTool returns the schema for the analyze_spark_log MCP Tool.
+func NewAnalyzeSparkLogTool() mcp.Tool {
+	return mcp.NewTool(AnalyzeSparkLogName,
+		mcp.WithDescription("Fetch YARN application logs and produce an LLM-ready analysis prompt with heuristic findings."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		mcp.WithString("app_id",
+			mcp.Required(),
+			mcp.Description("YARN application ID, e.g. application_1234567890_0001"),
+		),
+		mcp.WithString("container",
+			mcp.Description("Container ID used for the aggregated-logs fallback"),
+		),
+		mcp.WithNumber("max_log_bytes",
+			mcp.Description("Maximum bytes of the log tail to include in the prompt"),
+			mcp.DefaultNumber(10240),
+		),
+	)
+}
+
+// AnalyzeSparkLogHandler retrieves logs and builds an LLM-ready prompt.
+func (d *Deps) AnalyzeSparkLogHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("analyze_spark_log: " + err.Error()), nil
+	}
+	appID, err := req.RequireString("app_id")
+	if err != nil {
+		return errResult("analyze_spark_log: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	container := ""
+	if v, ok := args["container"].(string); ok {
+		container = v
+	}
+	if container == "" {
+		container = fmt.Sprintf("container_%s_01", appID)
+	}
+
+	maxLogBytes := defaultAnalyzeMaxLogBytes
+	if v, ok := args["max_log_bytes"].(float64); ok {
+		maxLogBytes = int(v)
+	}
+	if maxLogBytes <= 0 {
+		maxLogBytes = defaultAnalyzeMaxLogBytes
+	}
+
+	callLog := startToolCall(ctx, d.Logger, AnalyzeSparkLogName, map[string]any{
+		"cluster_id":    clusterID,
+		"app_id":        appID,
+		"container":     container,
+		"max_log_bytes": maxLogBytes,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("analyze_spark_log: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	rmc := rm.New(d.HTTPClient, cl)
+	body, source, err := rmc.GetLogs(ctx, appID, container)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("analyze_spark_log: " + err.Error()), nil
+	}
+
+	logTail := truncateTail(string(body), maxLogBytes)
+
+	var findings []analyzer.Finding
+	input, metricsErr := d.sparkMetricsInput(ctx, cl, appID)
+	if metricsErr == nil {
+		findings = analyzer.Analyze(input, d.AnalyzerThresholds)
+	}
+
+	prompt := buildSparkLogPrompt(appID, cl, source, len(body), maxLogBytes, logTail, findings)
+
+	result := map[string]any{
+		"findings":   findings,
+		"log_source": source,
+		"log_tail":   logTail,
+		"prompt":     prompt,
+	}
+	callLog.WithResult(map[string]any{"source": source, "bytes": len(body), "findings": len(findings)})
+	return textResult(encodeJSON(result)), nil
+}
+
+func buildSparkLogPrompt(appID string, cl *cluster.Cluster, source string, totalBytes, maxBytes int, logTail string, findings []analyzer.Finding) string {
+	findingsBlock := "no heuristic findings; rely on raw log + LLM analysis"
+	if len(findings) > 0 {
+		findingsBlock = encodeJSON(findings)
+	}
+
+	return fmt.Sprintf("# Spark Log Analysis for %s\n\n## Cluster\n%s (%s) — RM=%s, SHS=%s\n\n## Log Source\n%s (%d bytes, last %d shown below)\n\n## Log Tail\n"+"```"+"\n%s\n"+"```"+"\n\n## Heuristic Findings\n%s\n\n## Suggested LLM Analysis\n1. Check for ERROR/Exception stack traces\n2. Look for OOM/Timeout/Shuffle fetch failures\n3. Identify slow stages (compare to median)\n4. Suggest mitigations\n", appID, cl.Name, cl.ID, cl.RMURL, cl.SHSURL, source, totalBytes, maxBytes, logTail, findingsBlock)
+}

internal/mcp/tools/deps.go

@@ -0,0 +1,22 @@
+package tools
+
+import (
+	"log/slog"
+	"time"
+
+	"spark-mcp-go/internal/analyzer"
+	"spark-mcp-go/internal/httpclient"
+	"spark-mcp-go/internal/storage"
+)
+
+// Deps bundles the dependencies shared by all MCP Tool handlers.
+type Deps struct {
+	Logger             *slog.Logger
+	ClusterRepo        *storage.ClusterRepo
+	SparkSubmitTimeout time.Duration
+	HTTPClient         *httpclient.Client
+	MaxResponseBytes   int64
+	DataDir            string // upload_file writes to DataDir/uploads
+
+	AnalyzerThresholds analyzer.Thresholds
+}

internal/mcp/tools/fetch_cluster_env.go

@@ -0,0 +1,66 @@
+package tools
+
+import (
+	"context"
+	"encoding/json"
+	"strings"
+	"time"
+
+	"github.com/mark3labs/mcp-go/mcp"
+)
+
+const FetchClusterEnvName = "fetch_cluster_env"
+
+// NewFetchClusterEnvTool returns the schema for the fetch_cluster_env MCP Tool.
+func NewFetchClusterEnvTool() mcp.Tool {
+	return mcp.NewTool(FetchClusterEnvName,
+		mcp.WithDescription("Fetch YARN cluster environment information and metrics from the ResourceManager."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+	)
+}
+
+// FetchClusterEnvHandler fetches RM cluster info and metrics.
+func (d *Deps) FetchClusterEnvHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("fetch_cluster_env: " + err.Error()), nil
+	}
+
+	callLog := startToolCall(ctx, d.Logger, FetchClusterEnvName, map[string]any{
+		"cluster_id": clusterID,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_cluster_env: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	base := strings.TrimRight(cl.RMURL, "/")
+	infoBody, err := d.fetchInternal(ctx, cl, base+"/ws/v1/cluster/info")
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_cluster_env: info: " + err.Error()), nil
+	}
+	metricsBody, err := d.fetchInternal(ctx, cl, base+"/ws/v1/cluster/metrics")
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_cluster_env: metrics: " + err.Error()), nil
+	}
+
+	var info, metrics any
+	_ = json.Unmarshal(infoBody, &info)
+	_ = json.Unmarshal(metricsBody, &metrics)
+
+	result := map[string]any{
+		"cluster_info": info,
+		"metrics":      metrics,
+		"fetched_at":   time.Now().UTC().Format(time.RFC3339),
+	}
+	callLog.WithResult(map[string]any{"info_bytes": len(infoBody), "metrics_bytes": len(metricsBody)})
+	return textResult(encodeJSON(result)), nil
+}

internal/mcp/tools/fetch_spark_metrics.go

@@ -0,0 +1,207 @@
+package tools
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/analyzer"
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+)
+
+const FetchSparkMetricsName = "fetch_spark_metrics"
+
+type shsExecutor struct {
+	ID        string `json:"id"`
+	GCTimeMS  int64  `json:"gc_time_ms"`
+	CPUTimeMS int64  `json:"cpu_time_ms"`
+}
+
+type shsStage struct {
+	StageID           int   `json:"stageId"`
+	DurationMS        int64 `json:"duration_ms"`
+	ShuffleReadBytes  int64 `json:"shuffle_read_bytes"`
+	ShuffleWriteBytes int64 `json:"shuffle_write_bytes"`
+	MaxPartitionBytes int64 `json:"max_partition_bytes"`
+	MinPartitionBytes int64 `json:"min_partition_bytes"`
+}
+
+// NewFetchSparkMetricsTool returns the schema for the fetch_spark_metrics MCP Tool.
+func NewFetchSparkMetricsTool() mcp.Tool {
+	return mcp.NewTool(FetchSparkMetricsName,
+		mcp.WithDescription("Fetch Spark History Server metrics for an application. Returns SHS executor/stage data plus optional heuristic findings."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster whose SHS is used"),
+		),
+		mcp.WithString("app_id",
+			mcp.Required(),
+			mcp.Description("Spark application ID, e.g. application_1234567890_0001"),
+		),
+		mcp.WithString("format",
+			mcp.Description("Output format: raw SHS JSON or summary with heuristic findings"),
+			mcp.Enum("summary", "raw"),
+			mcp.DefaultString("summary"),
+		),
+	)
+}
+
+// FetchSparkMetricsHandler fetches SHS executor and stage data and optionally runs analysis.
+func (d *Deps) FetchSparkMetricsHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("fetch_spark_metrics: " + err.Error()), nil
+	}
+	appID, err := req.RequireString("app_id")
+	if err != nil {
+		return errResult("fetch_spark_metrics: " + err.Error()), nil
+	}
+
+	format := "summary"
+	if v, ok := req.GetArguments()["format"].(string); ok && v != "" {
+		format = v
+	}
+
+	callLog := startToolCall(ctx, d.Logger, FetchSparkMetricsName, map[string]any{
+		"cluster_id": clusterID,
+		"app_id":     appID,
+		"format":     format,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_spark_metrics: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	base := strings.TrimRight(cl.SHSURL, "/")
+	execURL := base + "/api/v1/applications/" + appID + "/executors"
+	stageURL := base + "/api/v1/applications/" + appID + "/stages"
+
+	execBody, err := d.fetchInternal(ctx, cl, execURL)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_spark_metrics: executors: " + err.Error()), nil
+	}
+	stageBody, err := d.fetchInternal(ctx, cl, stageURL)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_spark_metrics: stages: " + err.Error()), nil
+	}
+
+	var rawExec, rawStage any
+	_ = json.Unmarshal(execBody, &rawExec)
+	_ = json.Unmarshal(stageBody, &rawStage)
+
+	if format != "summary" {
+		return textResult(encodeJSON(map[string]any{
+			"executors": rawExec,
+			"stages":    rawStage,
+		})), nil
+	}
+
+	input, err := d.sparkMetricsInput(ctx, cl, appID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_spark_metrics: analyze: " + err.Error()), nil
+	}
+	findings := analyzer.Analyze(input, d.AnalyzerThresholds)
+
+	return textResult(encodeJSON(map[string]any{
+		"executors": rawExec,
+		"stages":    rawStage,
+		"findings":  findings,
+	})), nil
+}
+
+// fetchInternal performs an SSRF-aware, authenticated GET to a cluster URL.
+func (d *Deps) fetchInternal(ctx context.Context, cl *cluster.Cluster, rawURL string) ([]byte, error) {
+	allowedHosts := buildAllowedHosts(cl)
+
+	targetURL, err := url.Parse(rawURL)
+	if err != nil {
+		return nil, fmt.Errorf("parse url: %w", err)
+	}
+	targetHost := strings.ToLower(targetURL.Hostname())
+	if !hostAllowed(targetHost, allowedHosts) {
+		return nil, fmt.Errorf("host %q is not in the cluster allowlist", targetHost)
+	}
+
+	if err := httpclient.CheckURL(rawURL, allowedHosts...); err != nil {
+		return nil, err
+	}
+
+	httpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
+	if err != nil {
+		return nil, fmt.Errorf("build request: %w", err)
+	}
+	if err := httpclient.ApplyAuth(httpReq, "", cl); err != nil {
+		return nil, fmt.Errorf("auth: %w", err)
+	}
+
+	resp, err := d.HTTPClient.DoWithRedirect(ctx, httpReq, 5, allowedHosts...)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode >= 400 {
+		return nil, fmt.Errorf("%s returned %d", rawURL, resp.StatusCode)
+	}
+	return io.ReadAll(resp.Body)
+}
+
+// sparkMetricsInput converts SHS executor/stage JSON into analyzer input.
+func (d *Deps) sparkMetricsInput(ctx context.Context, cl *cluster.Cluster, appID string) (analyzer.Input, error) {
+	base := strings.TrimRight(cl.SHSURL, "/")
+	execURL := base + "/api/v1/applications/" + appID + "/executors"
+	stageURL := base + "/api/v1/applications/" + appID + "/stages"
+
+	execBody, err := d.fetchInternal(ctx, cl, execURL)
+	if err != nil {
+		return analyzer.Input{}, err
+	}
+	stageBody, err := d.fetchInternal(ctx, cl, stageURL)
+	if err != nil {
+		return analyzer.Input{}, err
+	}
+
+	var execs []shsExecutor
+	if err := json.Unmarshal(execBody, &execs); err != nil {
+		return analyzer.Input{}, fmt.Errorf("parse executors: %w", err)
+	}
+	var stages []shsStage
+	if err := json.Unmarshal(stageBody, &stages); err != nil {
+		return analyzer.Input{}, fmt.Errorf("parse stages: %w", err)
+	}
+
+	input := analyzer.Input{
+		StageMetrics:    make(map[string]analyzer.StageMetric),
+		ExecutorMetrics: make(map[string]analyzer.ExecutorMetric),
+	}
+	for _, e := range execs {
+		input.ExecutorMetrics[e.ID] = analyzer.ExecutorMetric{
+			GCTimeMS:  e.GCTimeMS,
+			CPUTimeMS: e.CPUTimeMS,
+		}
+	}
+	for _, s := range stages {
+		key := fmt.Sprintf("stage %d", s.StageID)
+		input.StageMetrics[key] = analyzer.StageMetric{
+			DurationMS:        s.DurationMS,
+			ShuffleReadBytes:  s.ShuffleReadBytes,
+			ShuffleWriteBytes: s.ShuffleWriteBytes,
+			MaxPartitionBytes: s.MaxPartitionBytes,
+			MinPartitionBytes: s.MinPartitionBytes,
+		}
+	}
+	return input, nil
+}

internal/mcp/tools/fetch_upload_test.go

@@ -0,0 +1,405 @@
+package tools
+
+import (
+	"context"
+	"encoding/base64"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+	"spark-mcp-go/internal/storage"
+)
+
+// testDepsWithDataDir returns dependencies backed by an in-memory DB and a
+// temporary data directory.
+func testDepsWithDataDir(t *testing.T) (*Deps, *storage.ClusterRepo) {
+	t.Helper()
+	db, err := storage.Open(":memory:")
+	if err != nil {
+		t.Fatalf("open db: %v", err)
+	}
+	t.Cleanup(func() { _ = db.Close() })
+
+	return &Deps{
+		HTTPClient: httpclient.New(httpclient.Config{
+			Timeout:          5 * time.Second,
+			MaxResponseBytes: 1 << 20,
+		}),
+		ClusterRepo:      db.Clusters(),
+		MaxResponseBytes: 1 << 20,
+		DataDir:          t.TempDir(),
+	}, db.Clusters()
+}
+
+// createCluster creates a cluster in the repository with the given fields.
+func createCluster(t *testing.T, repo *storage.ClusterRepo, c *cluster.Cluster) {
+	t.Helper()
+	if c.RMURL == "" {
+		c.RMURL = "http://rm.example.com:8088"
+	}
+	if c.SHSURL == "" {
+		c.SHSURL = "http://shs.example.com:18080"
+	}
+	if c.SparkSubmitExecuteBin == "" {
+		c.SparkSubmitExecuteBin = "/usr/bin/spark-submit"
+	}
+	if err := repo.Create(context.Background(), c); err != nil {
+		t.Fatalf("create cluster: %v", err)
+	}
+}
+
+func resultText(t *testing.T, res *mcp.CallToolResult) string {
+	t.Helper()
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	if len(res.Content) == 0 {
+		t.Fatal("empty result content")
+	}
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	return text.Text
+}
+
+func resultJSON(t *testing.T, res *mcp.CallToolResult) map[string]any {
+	t.Helper()
+	var out map[string]any
+	if err := json.Unmarshal([]byte(resultText(t, res)), &out); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	return out
+}
+
+func TestFetchURL(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"hello":"world"}`))
+	}))
+	defer srv.Close()
+
+	deps, repo := testDepsWithDataDir(t)
+	createCluster(t, repo, &cluster.Cluster{
+		ID:           "cluster-a",
+		Name:         "Cluster A",
+		IsActive:     true,
+		AuthType:     cluster.AuthNone,
+		URLAllowlist: []string{srv.URL},
+	})
+
+	t.Run("success", func(t *testing.T) {
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-a",
+			"url":        srv.URL + "/foo",
+			"method":     "GET",
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		payload := resultJSON(t, res)
+		if payload["status_code"] != float64(200) {
+			t.Errorf("status_code=%v, want 200", payload["status_code"])
+		}
+		if !strings.Contains(payload["body"].(string), "hello") {
+			t.Errorf("body missing hello: %v", payload["body"])
+		}
+	})
+
+	t.Run("cluster not found", func(t *testing.T) {
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "missing",
+			"url":        srv.URL,
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		if !res.IsError {
+			t.Errorf("expected error result, got: %v", res.Content)
+		}
+	})
+
+	t.Run("private ip without allowlist", func(t *testing.T) {
+		createCluster(t, repo, &cluster.Cluster{
+			ID:       "cluster-private",
+			Name:     "Private",
+			IsActive: true,
+			AuthType: cluster.AuthNone,
+		})
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-private",
+			"url":        "http://127.0.0.1:12345/",
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		if !res.IsError {
+			t.Errorf("expected error result, got: %v", res.Content)
+		}
+	})
+
+	t.Run("public url without allowlist", func(t *testing.T) {
+		createCluster(t, repo, &cluster.Cluster{
+			ID:       "cluster-public",
+			Name:     "Public",
+			IsActive: true,
+			AuthType: cluster.AuthNone,
+		})
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-public",
+			"url":        "http://example.com/",
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		if !res.IsError {
+			t.Errorf("expected error result, got: %v", res.Content)
+		}
+	})
+
+	t.Run("invalid method", func(t *testing.T) {
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-a",
+			"url":        srv.URL,
+			"method":     "INVALID",
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		if !res.IsError {
+			t.Errorf("expected error result, got: %v", res.Content)
+		}
+	})
+
+	t.Run("basic auth header sent", func(t *testing.T) {
+		authSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			auth := r.Header.Get("Authorization")
+			if auth == "" {
+				http.Error(w, "missing auth", http.StatusUnauthorized)
+				return
+			}
+			parts := strings.SplitN(auth, " ", 2)
+			if len(parts) != 2 || parts[0] != "Basic" {
+				http.Error(w, "bad auth", http.StatusUnauthorized)
+				return
+			}
+			w.Write([]byte(parts[1]))
+		}))
+		defer authSrv.Close()
+
+		createCluster(t, repo, &cluster.Cluster{
+			ID:           "cluster-auth",
+			Name:         "Auth",
+			IsActive:     true,
+			AuthType:     cluster.AuthBasic,
+			AuthUsername: "u",
+			AuthPassword: "p",
+			URLAllowlist: []string{authSrv.URL},
+		})
+
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-auth",
+			"url":        authSrv.URL,
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		payload := resultJSON(t, res)
+		got := payload["body"].(string)
+		want := base64.StdEncoding.EncodeToString([]byte("u:p"))
+		if got != want {
+			t.Errorf("auth body=%q, want %q", got, want)
+		}
+	})
+
+	t.Run("user authorization header preserved", func(t *testing.T) {
+		authSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.Write([]byte(r.Header.Get("Authorization")))
+		}))
+		defer authSrv.Close()
+
+		createCluster(t, repo, &cluster.Cluster{
+			ID:           "cluster-user-auth",
+			Name:         "UserAuth",
+			IsActive:     true,
+			AuthType:     cluster.AuthBasic,
+			AuthUsername: "u",
+			AuthPassword: "p",
+			URLAllowlist: []string{authSrv.URL},
+		})
+
+		req := newToolRequest(FetchURLName, map[string]any{
+			"cluster_id": "cluster-user-auth",
+			"url":        authSrv.URL,
+			"headers": map[string]any{
+				"Authorization": "Bearer user-token",
+			},
+		})
+		res, err := deps.FetchURLHandler(context.Background(), req)
+		if err != nil {
+			t.Fatalf("handler error: %v", err)
+		}
+		payload := resultJSON(t, res)
+		got := payload["body"].(string)
+		if got != "Bearer user-token" {
+			t.Errorf("authorization header=%q, want user token preserved", got)
+		}
+	})
+}
+
+func TestFetchURL_RedirectPreservesAuth(t *testing.T) {
+	var server2URL string
+	server2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		auth := r.Header.Get("Authorization")
+		parts := strings.SplitN(auth, " ", 2)
+		if len(parts) == 2 && parts[0] == "Basic" {
+			w.Write([]byte(parts[1]))
+			return
+		}
+		w.Write([]byte(auth))
+	}))
+	defer server2.Close()
+	server2URL = server2.URL
+
+	server1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Location", server2URL+"/final")
+		w.WriteHeader(http.StatusTemporaryRedirect)
+	}))
+	defer server1.Close()
+
+	deps, repo := testDepsWithDataDir(t)
+	createCluster(t, repo, &cluster.Cluster{
+		ID:           "cluster-redirect",
+		Name:         "Redirect",
+		IsActive:     true,
+		AuthType:     cluster.AuthBasic,
+		AuthUsername: "u",
+		AuthPassword: "p",
+		URLAllowlist: []string{server1.URL, server2.URL},
+	})
+
+	req := newToolRequest(FetchURLName, map[string]any{
+		"cluster_id": "cluster-redirect",
+		"url":        server1.URL + "/start",
+	})
+	res, err := deps.FetchURLHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+
+	payload := resultJSON(t, res)
+	got := payload["body"].(string)
+	want := base64.StdEncoding.EncodeToString([]byte("u:p"))
+	if got != want {
+		t.Errorf("redirect auth body=%q, want %q", got, want)
+	}
+}
+
+func TestUploadFile(t *testing.T) {
+	deps, _ := testDepsWithDataDir(t)
+
+	req := newToolRequest(UploadFileName, map[string]any{
+		"filename": "hello.txt",
+		"content":  "hello world",
+	})
+	res, err := deps.UploadFileHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+
+	payload := resultJSON(t, res)
+	if payload["path"] != "uploads/hello.txt" {
+		t.Errorf("path=%v, want uploads/hello.txt", payload["path"])
+	}
+	if payload["size"] != float64(len("hello world")) {
+		t.Errorf("size=%v, want %d", payload["size"], len("hello world"))
+	}
+
+	final := filepath.Join(deps.DataDir, "uploads", "hello.txt")
+	got, err := os.ReadFile(final)
+	if err != nil {
+		t.Fatalf("read uploaded file: %v", err)
+	}
+	if string(got) != "hello world" {
+		t.Errorf("content=%q, want %q", got, "hello world")
+	}
+
+	info, err := os.Stat(final)
+	if err != nil {
+		t.Fatalf("stat uploaded file: %v", err)
+	}
+	if info.Mode().Perm() != 0o640 {
+		t.Errorf("mode=%o, want %o", info.Mode().Perm(), 0o640)
+	}
+}
+
+func TestUploadFile_PathTraversal(t *testing.T) {
+	deps, _ := testDepsWithDataDir(t)
+
+	cases := []string{
+		"../../../etc/passwd",
+		"/etc/passwd",
+		"foo/bar",
+		"foo\\bar",
+		"",
+		"hello world",
+		"中文.txt",
+		"..",
+	}
+
+	for _, name := range cases {
+		t.Run(strconv.Quote(name), func(t *testing.T) {
+			req := newToolRequest(UploadFileName, map[string]any{
+				"filename": name,
+				"content":  "x",
+			})
+			res, err := deps.UploadFileHandler(context.Background(), req)
+			if err != nil {
+				t.Fatalf("handler error: %v", err)
+			}
+			if !res.IsError {
+				t.Errorf("expected error for filename %q, got: %v", name, res.Content)
+			}
+		})
+	}
+}
+
+func TestUploadFile_Base64(t *testing.T) {
+	deps, _ := testDepsWithDataDir(t)
+
+	req := newToolRequest(UploadFileName, map[string]any{
+		"filename": "hello.bin",
+		"content":  "aGVsbG8=",
+		"encoding": "base64",
+	})
+	res, err := deps.UploadFileHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+
+	_ = resultJSON(t, res)
+	final := filepath.Join(deps.DataDir, "uploads", "hello.bin")
+	got, err := os.ReadFile(final)
+	if err != nil {
+		t.Fatalf("read uploaded file: %v", err)
+	}
+	if string(got) != "hello" {
+		t.Errorf("content=%q, want %q", got, "hello")
+	}
+}

internal/mcp/tools/fetch_url.go

@@ -0,0 +1,223 @@
+package tools
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strconv"
+	"strings"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+)
+
+const FetchURLName = "fetch_url"
+
+var fetchURLMethods = map[string]bool{
+	http.MethodGet:    true,
+	http.MethodPost:   true,
+	http.MethodPut:    true,
+	http.MethodDelete: true,
+	http.MethodHead:   true,
+}
+
+// NewFetchURLTool returns the schema for the fetch_url MCP Tool.
+func NewFetchURLTool() mcp.Tool {
+	return mcp.NewTool(FetchURLName,
+		mcp.WithDescription("Perform an HTTP request to an allowed URL using a cluster's auth and allowlist. Returns {status, status_code, headers, body, body_truncated}."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster whose allowlist and auth are used"),
+		),
+		mcp.WithString("url",
+			mcp.Required(),
+			mcp.Description("Absolute http(s) URL to fetch"),
+		),
+		mcp.WithString("method",
+			mcp.Description("HTTP method"),
+			mcp.Enum("GET", "POST", "PUT", "DELETE", "HEAD"),
+			mcp.DefaultString("GET"),
+		),
+		mcp.WithObject("headers",
+			mcp.Description("Extra HTTP headers as a JSON object"),
+			mcp.AdditionalProperties(map[string]any{"type": "string"}),
+		),
+		mcp.WithString("body",
+			mcp.Description("Request body for POST/PUT"),
+		),
+	)
+}
+
+// FetchURLHandler performs an HTTP request with cluster auth and SSRF protection.
+func (d *Deps) FetchURLHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("fetch_url: " + err.Error()), nil
+	}
+	fetchURL, err := req.RequireString("url")
+	if err != nil {
+		return errResult("fetch_url: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	method := http.MethodGet
+	if v, ok := args["method"].(string); ok && v != "" {
+		method = strings.ToUpper(v)
+	}
+	if !fetchURLMethods[method] {
+		return errResult(fmt.Sprintf("fetch_url: invalid method %q", method)), nil
+	}
+
+	var body string
+	if v, ok := args["body"].(string); ok {
+		body = v
+	}
+
+	var userHeaders map[string]any
+	if v, ok := args["headers"].(map[string]any); ok {
+		userHeaders = v
+	}
+
+	callLog := startToolCall(ctx, d.Logger, FetchURLName, map[string]any{
+		"cluster_id": clusterID,
+		"url":        fetchURL,
+		"method":     method,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	allowedHosts := buildAllowedHosts(cl)
+
+	targetURL, err := url.Parse(fetchURL)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: parse url: " + err.Error()), nil
+	}
+	targetHost := strings.ToLower(targetURL.Hostname())
+	if !hostAllowed(targetHost, allowedHosts) {
+		return errResult(fmt.Sprintf("fetch_url: host %q is not in the cluster allowlist", targetHost)), nil
+	}
+
+	if err := httpclient.CheckURL(fetchURL, allowedHosts...); err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: " + err.Error()), nil
+	}
+
+	var bodyReader io.Reader
+	if body != "" {
+		bodyReader = strings.NewReader(body)
+	}
+	httpReq, err := http.NewRequestWithContext(ctx, method, fetchURL, bodyReader)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: build request: " + err.Error()), nil
+	}
+
+	for k, v := range userHeaders {
+		httpReq.Header.Set(k, fmt.Sprint(v))
+	}
+	if body != "" {
+		httpReq.Header.Set("Content-Length", strconv.Itoa(len(body)))
+	}
+
+	// Preserve an explicitly provided Authorization header; otherwise apply cluster auth.
+	if httpReq.Header.Get("Authorization") == "" {
+		if err := httpclient.ApplyAuth(httpReq, "", cl); err != nil {
+			callLog.WithError(err)
+			return errResult("fetch_url: auth: " + err.Error()), nil
+		}
+	}
+
+	resp, err := d.HTTPClient.DoWithRedirect(ctx, httpReq, 5, allowedHosts...)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: " + err.Error()), nil
+	}
+	defer resp.Body.Close()
+
+	respBody, err := io.ReadAll(resp.Body)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("fetch_url: read body: " + err.Error()), nil
+	}
+
+	bodyTruncated := int64(len(respBody)) >= d.MaxResponseBytes &&
+		(resp.ContentLength < 0 || resp.ContentLength > d.MaxResponseBytes)
+
+	result := map[string]any{
+		"status":         resp.Status,
+		"status_code":    resp.StatusCode,
+		"headers":        resp.Header,
+		"body":           string(respBody),
+		"body_truncated": bodyTruncated,
+	}
+	callLog.WithResult(map[string]any{"status_code": resp.StatusCode, "bytes": len(respBody)})
+	return textResult(encodeJSON(result)), nil
+}
+
+// buildAllowedHosts extracts hostnames from the cluster's allowlist
+// and RM/SHS URLs. Full URLs are parsed and only their host is used; entries
+// that do not parse as URLs are passed through as-is so glob patterns such as
+// "*.example.com" continue to work.
+func buildAllowedHosts(cl *cluster.Cluster) []string {
+	var hosts []string
+	seen := make(map[string]bool)
+	add := func(s string) {
+		s = strings.ToLower(strings.TrimSpace(s))
+		if s == "" || seen[s] {
+			return
+		}
+		seen[s] = true
+		hosts = append(hosts, s)
+	}
+
+	for _, raw := range cl.URLAllowlist {
+		if u, err := url.Parse(raw); err == nil && u.Hostname() != "" {
+			add(u.Hostname())
+		} else {
+			add(raw)
+		}
+	}
+
+	for _, raw := range []string{cl.RMURL, cl.SHSURL} {
+		if u, err := url.Parse(raw); err == nil && u.Hostname() != "" {
+			add(u.Hostname())
+		}
+	}
+
+	return hosts
+}
+
+// hostAllowed reports whether host matches any pattern in allowed. Patterns may
+// be exact hostnames, domain suffixes ("example.com" matches "rm.example.com"),
+// or wildcard suffixes ("*.example.com"). It mirrors httpclient.matchAllowedHost.
+func hostAllowed(host string, allowed []string) bool {
+	for _, pattern := range allowed {
+		pattern = strings.ToLower(strings.TrimSpace(pattern))
+		if pattern == "" {
+			continue
+		}
+		if pattern == host {
+			return true
+		}
+		if strings.HasPrefix(pattern, "*.") {
+			pattern = pattern[2:]
+		}
+		if strings.HasPrefix(pattern, ".") {
+			pattern = pattern[1:]
+		}
+		if host == pattern || strings.HasSuffix(host, "."+pattern) {
+			return true
+		}
+	}
+	return false
+}

internal/mcp/tools/get_application_logs.go

@@ -0,0 +1,93 @@
+package tools
+
+import (
+	"context"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/rm"
+)
+
+const GetApplicationLogsName = "get_application_logs"
+
+const defaultTailBytes = 1 << 20
+
+// NewGetApplicationLogsTool returns the schema for the get_application_logs MCP Tool.
+func NewGetApplicationLogsTool() mcp.Tool {
+	return mcp.NewTool(GetApplicationLogsName,
+		mcp.WithDescription("Fetch YARN application logs. Tries amContainerLogs first, then aggregated-logs, then the legacy /logs endpoint. Returns {source, text}."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		mcp.WithString("app_id",
+			mcp.Required(),
+			mcp.Description("YARN application ID, e.g. application_1234567890_0001"),
+		),
+		mcp.WithString("container",
+			mcp.Description("Container ID used for the aggregated-logs fallback, e.g. container_1234567890_0001_01_000001."),
+		),
+		mcp.WithNumber("tail_bytes",
+			mcp.Description("Maximum bytes to return. 0 means use the server-wide response byte limit."),
+		),
+	)
+}
+
+// GetApplicationLogsHandler retrieves logs using the RM fallback chain.
+func (d *Deps) GetApplicationLogsHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("get_application_logs: " + err.Error()), nil
+	}
+	appID, err := req.RequireString("app_id")
+	if err != nil {
+		return errResult("get_application_logs: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	container := ""
+	if v, ok := args["container"].(string); ok {
+		container = v
+	}
+
+	tailBytes := defaultTailBytes
+	if v, ok := args["tail_bytes"].(float64); ok {
+		tailBytes = int(v)
+	}
+	if tailBytes == 0 {
+		tailBytes = int(d.MaxResponseBytes)
+	}
+
+	callLog := startToolCall(ctx, d.Logger, GetApplicationLogsName, map[string]any{
+		"cluster_id": clusterID,
+		"app_id":     appID,
+		"container":  container,
+		"tail_bytes": tailBytes,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("get_application_logs: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	rmc := rm.New(d.HTTPClient, cl)
+	body, source, err := rmc.GetLogs(ctx, appID, container)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("get_application_logs: " + err.Error()), nil
+	}
+
+	text := string(body)
+	if len(text) > tailBytes {
+		text = truncateMiddle(text, tailBytes)
+	}
+
+	result := map[string]any{
+		"source": source,
+		"text":   text,
+	}
+	callLog.WithResult(map[string]any{"source": source, "bytes": len(body), "returned_bytes": len(text)})
+	return textResult(encodeJSON(result)), nil
+}

internal/mcp/tools/get_application_status.go

@@ -0,0 +1,60 @@
+package tools
+
+import (
+	"context"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/rm"
+)
+
+const GetApplicationStatusName = "get_application_status"
+
+// NewGetApplicationStatusTool returns the schema for the get_application_status MCP Tool.
+func NewGetApplicationStatusTool() mcp.Tool {
+	return mcp.NewTool(GetApplicationStatusName,
+		mcp.WithDescription("Get the detailed status of a single YARN application from the ResourceManager. Returns the raw RM JSON for the app."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		mcp.WithString("app_id",
+			mcp.Required(),
+			mcp.Description("YARN application ID, e.g. application_1234567890_0001"),
+		),
+	)
+}
+
+// GetApplicationStatusHandler fetches a single application's details.
+func (d *Deps) GetApplicationStatusHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("get_application_status: " + err.Error()), nil
+	}
+	appID, err := req.RequireString("app_id")
+	if err != nil {
+		return errResult("get_application_status: " + err.Error()), nil
+	}
+
+	callLog := startToolCall(ctx, d.Logger, GetApplicationStatusName, map[string]any{
+		"cluster_id": clusterID,
+		"app_id":     appID,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("get_application_status: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	rmc := rm.New(d.HTTPClient, cl)
+	raw, err := rmc.GetApp(ctx, appID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("get_application_status: " + err.Error()), nil
+	}
+
+	callLog.WithResult(map[string]any{"bytes": len(raw)})
+	return textResult(string(raw)), nil
+}

internal/mcp/tools/helpers.go

@@ -0,0 +1,80 @@
+package tools
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"log/slog"
+
+	"github.com/mark3labs/mcp-go/mcp"
+	"spark-mcp-go/internal/logging"
+)
+
+// textResult wraps a string into a text CallToolResult.
+func textResult(s string) *mcp.CallToolResult {
+	return mcp.NewToolResultText(s)
+}
+
+// errResult wraps a string into an error CallToolResult.
+// MCP protocol errors are reserved for exceptional conditions; business
+// errors are reported inside the tool result with IsError set by the library.
+func errResult(s string) *mcp.CallToolResult {
+	return mcp.NewToolResultError(s)
+}
+
+// encodeJSON marshals v to JSON. On failure it returns a string literal that
+// embeds the error so the caller never receives a nil or empty result.
+func encodeJSON(v any) string {
+	b, err := json.Marshal(v)
+	if err != nil {
+		return fmt.Sprintf("<marshal error: %s>", err)
+	}
+	return string(b)
+}
+
+// startToolCall starts per-tool-call logging. If logging is not initialized
+// (e.g. during tests) it falls back to a no-op logger so handlers never panic.
+func startToolCall(ctx context.Context, logger *slog.Logger, toolName string, params any) logging.ToolCallLogger {
+	callLog, err := logging.StartToolCall(ctx, logger, toolName, params)
+	if err != nil {
+		return &noopToolCallLogger{}
+	}
+	return callLog
+}
+
+type noopToolCallLogger struct{}
+
+func (n *noopToolCallLogger) WithResult(any)  {}
+func (n *noopToolCallLogger) WithError(error) {}
+func (n *noopToolCallLogger) End()            {}
+
+// truncateMiddle limits s to roughly max bytes by keeping the first and
+// last halves, separated by a marker.  It preserves UTF-8 rune boundaries
+// so the result never starts or ends with a broken multi-byte rune.
+func truncateMiddle(s string, max int) string {
+	if len(s) <= max {
+		return s
+	}
+	half := max / 2
+	start := s[:half]
+	end := s[len(s)-half:]
+	for len(start) > 0 && start[len(start)-1] >= 0x80 && start[len(start)-1] < 0xC0 {
+		start = start[:len(start)-1]
+	}
+	for len(end) > 0 && end[0]&0xC0 == 0x80 {
+		end = end[1:]
+	}
+	return start + "\n... [truncated middle] ...\n" + end
+}
+
+// truncateTail keeps the last max bytes of s, preserving UTF-8 rune boundaries.
+func truncateTail(s string, max int) string {
+	if len(s) <= max {
+		return s
+	}
+	start := len(s) - max
+	for start < len(s) && s[start]&0xC0 == 0x80 {
+		start++
+	}
+	return s[start:]
+}

internal/mcp/tools/high_level_test.go

@@ -0,0 +1,240 @@
+package tools
+
+import (
+	"context"
+	"encoding/json"
+	"github.com/mark3labs/mcp-go/mcp"
+	"net/http"
+	"net/http/httptest"
+	"spark-mcp-go/internal/analyzer"
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+	"spark-mcp-go/internal/storage"
+	"strings"
+	"testing"
+	"time"
+)
+
+func testDepsWithCluster(t *testing.T, rmURL, shsURL string) *Deps {
+	t.Helper()
+	db, err := storage.Open(":memory:")
+	if err != nil {
+		t.Fatalf("open db: %v", err)
+	}
+	t.Cleanup(func() { _ = db.Close() })
+	cl := &cluster.Cluster{ID: "cluster-a", Name: "Cluster A", RMURL: rmURL, SHSURL: shsURL, SparkSubmitExecuteBin: "/usr/bin/spark-submit", IsActive: true, AuthType: cluster.AuthNone}
+	if err := db.Clusters().Create(context.Background(), cl); err != nil {
+		t.Fatalf("create cluster: %v", err)
+	}
+	return &Deps{HTTPClient: httpclient.New(httpclient.Config{Timeout: 5 * time.Second, MaxResponseBytes: 1 << 20}), ClusterRepo: db.Clusters(), MaxResponseBytes: 1 << 20, DataDir: t.TempDir(), AnalyzerThresholds: analyzer.Thresholds{DataSkewRatio: 3.0, GCPressureRatio: 0.1, BottleneckShuffleGB: 50.0}}
+}
+
+func TestFetchSparkMetrics_Summary(t *testing.T) {
+	shsSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/api/v1/applications/app_123/executors":
+			w.Header().Set("Content-Type", "application/json")
+			w.Write([]byte(`[{"id":"1","gc_time_ms":5000,"cpu_time_ms":10000}]`))
+		case "/api/v1/applications/app_123/stages":
+			w.Header().Set("Content-Type", "application/json")
+			w.Write([]byte(`[{"stageId":1,"duration_ms":30000,"shuffle_read_bytes":0,"shuffle_write_bytes":0,"max_partition_bytes":100,"min_partition_bytes":10}]`))
+		default:
+			t.Errorf("unexpected SHS path %q", r.URL.Path)
+		}
+	}))
+	defer shsSrv.Close()
+	deps := testDepsWithCluster(t, "http://unused", shsSrv.URL)
+	req := newToolRequest("fetch_spark_metrics", map[string]any{"cluster_id": "cluster-a", "app_id": "app_123", "format": "summary"})
+	res, err := deps.FetchSparkMetricsHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	findings, ok := payload["findings"].([]any)
+	if !ok || len(findings) == 0 {
+		t.Fatalf("expected findings, got %+v", payload["findings"])
+	}
+}
+
+func TestFetchClusterEnv(t *testing.T) {
+	rmSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/ws/v1/cluster/info":
+			w.Header().Set("Content-Type", "application/json")
+			w.Write([]byte(`{"clusterInfo":{"id":"rm1","name":"test"}}`))
+		case "/ws/v1/cluster/metrics":
+			w.Header().Set("Content-Type", "application/json")
+			w.Write([]byte(`{"clusterMetrics":{"appsSubmitted":5}}`))
+		default:
+			t.Errorf("unexpected RM path %q", r.URL.Path)
+		}
+	}))
+	defer rmSrv.Close()
+	deps := testDepsWithCluster(t, rmSrv.URL, "http://unused")
+	req := newToolRequest("fetch_cluster_env", map[string]any{"cluster_id": "cluster-a"})
+	res, err := deps.FetchClusterEnvHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	if payload["cluster_info"] == nil {
+		t.Errorf("cluster_info missing")
+	}
+	if payload["metrics"] == nil {
+		t.Errorf("metrics missing")
+	}
+	if _, ok := payload["fetched_at"].(string); !ok {
+		t.Errorf("fetched_at missing or not string")
+	}
+}
+
+func TestAnalyzeSparkLog(t *testing.T) {
+	rmSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/ws/v1/cluster/apps/app_123/amContainerLogs":
+			w.WriteHeader(http.StatusNotFound)
+		case "/ws/v1/cluster/apps/app_123/aggregated-logs":
+			w.Write([]byte("first line\nsecond line\nERROR: something"))
+		default:
+			t.Errorf("unexpected RM path %q", r.URL.Path)
+		}
+	}))
+	defer rmSrv.Close()
+	shsSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/api/v1/applications/app_123/executors":
+			w.Write([]byte(`[]`))
+		case "/api/v1/applications/app_123/stages":
+			w.Write([]byte(`[]`))
+		default:
+			t.Errorf("unexpected SHS path %q", r.URL.Path)
+		}
+	}))
+	defer shsSrv.Close()
+	deps := testDepsWithCluster(t, rmSrv.URL, shsSrv.URL)
+	req := newToolRequest("analyze_spark_log", map[string]any{"cluster_id": "cluster-a", "app_id": "app_123"})
+	res, err := deps.AnalyzeSparkLogHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	prompt := payload["prompt"].(string)
+	if !strings.Contains(prompt, "Spark Log Analysis") {
+		t.Errorf("prompt missing header")
+	}
+	if !strings.Contains(payload["log_tail"].(string), "ERROR: something") {
+		t.Errorf("log tail missing content")
+	}
+}
+
+func TestAnalyzeSparkLog_LogSourceReported(t *testing.T) {
+	rmSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/ws/v1/cluster/apps/app_123/amContainerLogs":
+			w.WriteHeader(http.StatusNotFound)
+		case "/ws/v1/cluster/apps/app_123/aggregated-logs":
+			w.Write([]byte("log body"))
+		default:
+			t.Errorf("unexpected RM path %q", r.URL.Path)
+		}
+	}))
+	defer rmSrv.Close()
+	shsSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Write([]byte(`[]`))
+	}))
+	defer shsSrv.Close()
+	deps := testDepsWithCluster(t, rmSrv.URL, shsSrv.URL)
+	req := newToolRequest("analyze_spark_log", map[string]any{"cluster_id": "cluster-a", "app_id": "app_123"})
+	res, err := deps.AnalyzeSparkLogHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, _ := mcp.AsTextContent(res.Content[0])
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	if payload["log_source"] == "" {
+		t.Errorf("log_source empty")
+	}
+}
+
+func TestAnalyzeSparkLog_PromptIncludesFindings(t *testing.T) {
+	rmSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/ws/v1/cluster/apps/app_123/amContainerLogs":
+			w.WriteHeader(http.StatusNotFound)
+		case "/ws/v1/cluster/apps/app_123/aggregated-logs":
+			w.Write([]byte("log body"))
+		default:
+			t.Errorf("unexpected RM path %q", r.URL.Path)
+		}
+	}))
+	defer rmSrv.Close()
+	shsSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/api/v1/applications/app_123/executors":
+			w.Write([]byte(`[{"id":"1","gc_time_ms":5000,"cpu_time_ms":10000}]`))
+		case "/api/v1/applications/app_123/stages":
+			w.Write([]byte(`[]`))
+		default:
+			t.Errorf("unexpected SHS path %q", r.URL.Path)
+		}
+	}))
+	defer shsSrv.Close()
+	deps := testDepsWithCluster(t, rmSrv.URL, shsSrv.URL)
+	req := newToolRequest("analyze_spark_log", map[string]any{"cluster_id": "cluster-a", "app_id": "app_123"})
+	res, err := deps.AnalyzeSparkLogHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, _ := mcp.AsTextContent(res.Content[0])
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	prompt := payload["prompt"].(string)
+	if !strings.Contains(prompt, "GC 压力") {
+		t.Errorf("prompt missing GC finding; prompt:\n%s", prompt)
+	}
+	findings, ok := payload["findings"].([]any)
+	if !ok || len(findings) == 0 {
+		t.Fatalf("expected findings in result")
+	}
+}

internal/mcp/tools/kill_application.go

@@ -0,0 +1,60 @@
+package tools
+
+import (
+	"context"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/rm"
+)
+
+const KillApplicationName = "kill_application"
+
+// NewKillApplicationTool returns the schema for the kill_application MCP Tool.
+func NewKillApplicationTool() mcp.Tool {
+	return mcp.NewTool(KillApplicationName,
+		mcp.WithDescription("Kill (move to KILLED state) a YARN application through the ResourceManager. Returns the RM JSON response."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		mcp.WithString("app_id",
+			mcp.Required(),
+			mcp.Description("YARN application ID, e.g. application_1234567890_0001"),
+		),
+	)
+}
+
+// KillApplicationHandler sends a KILLED state update to the ResourceManager.
+func (d *Deps) KillApplicationHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("kill_application: " + err.Error()), nil
+	}
+	appID, err := req.RequireString("app_id")
+	if err != nil {
+		return errResult("kill_application: " + err.Error()), nil
+	}
+
+	callLog := startToolCall(ctx, d.Logger, KillApplicationName, map[string]any{
+		"cluster_id": clusterID,
+		"app_id":     appID,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("kill_application: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	rmc := rm.New(d.HTTPClient, cl)
+	raw, err := rmc.KillApp(ctx, appID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("kill_application: " + err.Error()), nil
+	}
+
+	callLog.WithResult(map[string]any{"bytes": len(raw)})
+	return textResult(string(raw)), nil
+}

internal/mcp/tools/list_applications.go

@@ -0,0 +1,69 @@
+package tools
+
+import (
+	"context"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/rm"
+)
+
+const ListApplicationsName = "list_applications"
+
+// NewListApplicationsTool returns the schema for the list_applications MCP Tool.
+func NewListApplicationsTool() mcp.Tool {
+	return mcp.NewTool(ListApplicationsName,
+		mcp.WithDescription("List YARN applications from a cluster's ResourceManager. Returns the raw RM JSON response so the agent can inspect app IDs, states, and owners."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		mcp.WithString("state",
+			mcp.Description("Filter by application state. YARN accepts comma-separated states; common values: NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED."),
+		),
+		mcp.WithString("user",
+			mcp.Description("Filter by submitting user."),
+		),
+	)
+}
+
+// ListApplicationsHandler queries the ResourceManager for applications.
+func (d *Deps) ListApplicationsHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("list_applications: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	state := ""
+	if v, ok := args["state"].(string); ok {
+		state = v
+	}
+	user := ""
+	if v, ok := args["user"].(string); ok {
+		user = v
+	}
+
+	callLog := startToolCall(ctx, d.Logger, ListApplicationsName, map[string]any{
+		"cluster_id": clusterID,
+		"state":      state,
+		"user":       user,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("list_applications: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	rmc := rm.New(d.HTTPClient, cl)
+	raw, err := rmc.ListApps(ctx, state, user)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("list_applications: " + err.Error()), nil
+	}
+
+	callLog.WithResult(map[string]any{"bytes": len(raw)})
+	return textResult(string(raw)), nil
+}

internal/mcp/tools/list_clusters.go

@@ -0,0 +1,39 @@
+package tools
+
+import (
+	"context"
+
+	"github.com/mark3labs/mcp-go/mcp"
+)
+
+const ListClustersName = "list_clusters"
+
+// NewListClustersTool returns the schema for the list_clusters MCP Tool.
+func NewListClustersTool() mcp.Tool {
+	return mcp.NewTool(ListClustersName,
+		mcp.WithDescription("List active Spark/YARN clusters configured for the agent. Returns the full Cluster struct (minus auth password) for every active cluster — this is the discovery root: from here the agent knows RM/SHS endpoints, auth flavor, and URL allowlists."),
+	)
+}
+
+// ListClustersHandler lists configured clusters and returns only active ones.
+func (d *Deps) ListClustersHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	callLog := startToolCall(ctx, d.Logger, ListClustersName, nil)
+	defer callLog.End()
+
+	all, err := d.ClusterRepo.List(ctx)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("list_clusters: " + err.Error()), nil
+	}
+
+	// task_plan.md specifies active clusters only.
+	active := all[:0]
+	for _, c := range all {
+		if c.IsActive {
+			active = append(active, c)
+		}
+	}
+
+	callLog.WithResult(map[string]any{"count": len(active)})
+	return textResult(encodeJSON(active)), nil
+}

internal/mcp/tools/rm_test.go

@@ -0,0 +1,176 @@
+package tools
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+	"spark-mcp-go/internal/storage"
+)
+
+func testDeps(t *testing.T, srv *httptest.Server) (*Deps, *storage.ClusterRepo) {
+	t.Helper()
+	db, err := storage.Open(":memory:")
+	if err != nil {
+		t.Fatalf("open db: %v", err)
+	}
+	t.Cleanup(func() { _ = db.Close() })
+
+	cl := &cluster.Cluster{
+		ID:                    "cluster-a",
+		Name:                  "Cluster A",
+		RMURL:                 srv.URL,
+		SHSURL:                "http://shs.example.com:18080",
+		SparkSubmitExecuteBin: "/usr/bin/spark-submit",
+		IsActive:              true,
+		AuthType:              cluster.AuthNone,
+	}
+	if err := db.Clusters().Create(context.Background(), cl); err != nil {
+		t.Fatalf("create cluster: %v", err)
+	}
+
+	return &Deps{
+		HTTPClient: httpclient.New(httpclient.Config{
+			Timeout:          5 * time.Second,
+			MaxResponseBytes: 1 << 20,
+		}),
+		ClusterRepo:      db.Clusters(),
+		MaxResponseBytes: 1 << 20,
+	}, db.Clusters()
+}
+
+func newToolRequest(name string, args map[string]any) mcp.CallToolRequest {
+	return mcp.CallToolRequest{
+		Request: mcp.Request{Method: "tools/call"},
+		Params: mcp.CallToolParams{
+			Name:      name,
+			Arguments: args,
+		},
+	}
+}
+
+func TestListApplications_EndToEnd(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/ws/v1/cluster/apps" {
+			t.Errorf("path=%q", r.URL.Path)
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"apps":{"app":[{"id":"application_1"}]}}`))
+	}))
+	defer srv.Close()
+
+	deps, _ := testDeps(t, srv)
+	req := newToolRequest("list_applications", map[string]any{
+		"cluster_id": "cluster-a",
+		"state":      "RUNNING",
+	})
+
+	res, err := deps.ListApplicationsHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	if !strings.Contains(text.Text, "application_1") {
+		t.Errorf("result missing app: %s", text.Text)
+	}
+}
+
+func TestKillApplication_AppIDNotProvided(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("unexpected RM call")
+	}))
+	defer srv.Close()
+
+	deps, _ := testDeps(t, srv)
+	req := newToolRequest("kill_application", map[string]any{
+		"cluster_id": "cluster-a",
+	})
+
+	res, err := deps.KillApplicationHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if !res.IsError {
+		t.Errorf("expected error result, got: %v", res.Content)
+	}
+}
+
+func TestGetApplicationLogs_SourceReported(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/ws/v1/cluster/apps/app_123/amContainerLogs" {
+			w.Write([]byte("driver stdout"))
+			return
+		}
+		t.Errorf("unexpected path %q", r.URL.Path)
+	}))
+	defer srv.Close()
+
+	deps, _ := testDeps(t, srv)
+	req := newToolRequest("get_application_logs", map[string]any{
+		"cluster_id": "cluster-a",
+		"app_id":     "app_123",
+	})
+
+	res, err := deps.GetApplicationLogsHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if res.IsError {
+		t.Fatalf("unexpected error result: %v", res.Content)
+	}
+
+	text, ok := mcp.AsTextContent(res.Content[0])
+	if !ok {
+		t.Fatalf("content is not text: %T", res.Content[0])
+	}
+	var payload map[string]any
+	if err := json.Unmarshal([]byte(text.Text), &payload); err != nil {
+		t.Fatalf("unmarshal result: %v", err)
+	}
+	if payload["source"] != "amContainerLogs" {
+		t.Errorf("source=%v, want amContainerLogs", payload["source"])
+	}
+	if !strings.Contains(payload["text"].(string), "driver stdout") {
+		t.Errorf("text missing logs: %v", payload["text"])
+	}
+}
+
+func TestKillApplication_NotFound(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/ws/v1/cluster/apps/app_123/state" {
+			w.WriteHeader(http.StatusNotFound)
+			w.Write([]byte(`{"error":"Not Found"}`))
+			return
+		}
+		t.Errorf("unexpected path %q", r.URL.Path)
+	}))
+	defer srv.Close()
+
+	deps, _ := testDeps(t, srv)
+	req := newToolRequest("kill_application", map[string]any{
+		"cluster_id": "cluster-a",
+		"app_id":     "app_123",
+	})
+
+	res, err := deps.KillApplicationHandler(context.Background(), req)
+	if err != nil {
+		t.Fatalf("handler error: %v", err)
+	}
+	if !res.IsError {
+		t.Errorf("expected error result, got: %v", res.Content)
+	}
+}

internal/mcp/tools/spark_submit.go

@@ -0,0 +1,100 @@
+package tools
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/mark3labs/mcp-go/mcp"
+
+	"spark-mcp-go/internal/executor"
+)
+
+const SparkSubmitName = "spark_submit"
+
+// NewSparkSubmitTool returns the schema for the spark_submit MCP Tool.
+func NewSparkSubmitTool() mcp.Tool {
+	return mcp.NewTool(SparkSubmitName,
+		mcp.WithDescription("Submit a Spark application to a configured cluster using the cluster's local spark-submit binary. Returns {app_id, exit_code, stdout_tail, stderr_tail, duration_ms}. The cluster's default_submit_args are prepended to your args. Binary is invoked as a child process — no shell, no command injection."),
+		mcp.WithString("cluster_id",
+			mcp.Required(),
+			mcp.Description("ID of the configured cluster (from list_clusters)"),
+		),
+		// mcp-go v0.56.0 uses PropertyOption for array item schema.
+		mcp.WithArray("args",
+			mcp.Description("Spark-submit CLI args (after default_submit_args). Each element becomes one argv entry — no shell interpretation."),
+			mcp.Items(map[string]any{"type": "string"}),
+		),
+	)
+}
+
+// SparkSubmitHandler runs spark-submit against the requested cluster.
+func (d *Deps) SparkSubmitHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	clusterID, err := req.RequireString("cluster_id")
+	if err != nil {
+		return errResult("spark_submit: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	var userArgs []string
+	if rawArgs, ok := args["args"]; ok && rawArgs != nil {
+		arr, ok := rawArgs.([]any)
+		if !ok {
+			return errResult("spark_submit: args is not an array"), nil
+		}
+		for i, item := range arr {
+			s, ok := item.(string)
+			if !ok {
+				return errResult(fmt.Sprintf("spark_submit: args[%d] is not a string", i)), nil
+			}
+			userArgs = append(userArgs, s)
+		}
+	}
+
+	callLog := startToolCall(ctx, d.Logger, SparkSubmitName, map[string]any{
+		"cluster_id": clusterID,
+		"args":       userArgs,
+	})
+	defer callLog.End()
+
+	cl, err := d.ClusterRepo.Get(ctx, clusterID)
+	if err != nil {
+		callLog.WithError(err)
+		return errResult("spark_submit: cluster " + clusterID + ": " + err.Error()), nil
+	}
+
+	fullArgs := append([]string{}, cl.DefaultSubmitArgs...)
+	fullArgs = append(fullArgs, userArgs...)
+
+	result, err := executor.Run(ctx, executor.SparkSubmitOpts{
+		Binary:  cl.SparkSubmitExecuteBin,
+		Args:    fullArgs,
+		Timeout: d.SparkSubmitTimeout,
+	})
+	if err != nil {
+		callLog.WithError(err)
+		// Even on error, result carries ExitCode/Stderr for the LLM.
+		if result != nil {
+			callLog.WithResult(map[string]any{
+				"app_id":      result.AppID,
+				"exit_code":   result.ExitCode,
+				"duration_ms": result.DurationMS,
+			})
+			return textResult(encodeJSON(map[string]any{
+				"app_id":      result.AppID,
+				"exit_code":   result.ExitCode,
+				"stdout_tail": result.StdoutTail,
+				"stderr_tail": result.StderrTail,
+				"duration_ms": result.DurationMS,
+				"error":       err.Error(),
+			})), nil
+		}
+		return errResult("spark_submit: " + err.Error()), nil
+	}
+
+	callLog.WithResult(map[string]any{
+		"app_id":      result.AppID,
+		"exit_code":   result.ExitCode,
+		"duration_ms": result.DurationMS,
+	})
+	return textResult(encodeJSON(result)), nil
+}

internal/mcp/tools/upload_file.go

@@ -0,0 +1,104 @@
+package tools
+
+import (
+	"context"
+	"encoding/base64"
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+
+	"github.com/mark3labs/mcp-go/mcp"
+)
+
+const UploadFileName = "upload_file"
+
+// filenameRegex restricts upload names to a safe, portable character set.
+var filenameRegex = regexp.MustCompile(`^[a-zA-Z0-9._-]+$`)
+
+// NewUploadFileTool returns the schema for the upload_file MCP Tool.
+func NewUploadFileTool() mcp.Tool {
+	return mcp.NewTool(UploadFileName,
+		mcp.WithDescription("Upload a script or config file to ./data/uploads/ so it can be referenced by spark_submit later."),
+		mcp.WithString("filename",
+			mcp.Required(),
+			mcp.Description("Plain file name without path separators (1-128 chars, [a-zA-Z0-9._-])"),
+		),
+		mcp.WithString("content",
+			mcp.Required(),
+			mcp.Description("File contents; text or base64-encoded binary"),
+		),
+		mcp.WithString("encoding",
+			mcp.Description("Encoding of content"),
+			mcp.Enum("text", "base64"),
+			mcp.DefaultString("text"),
+		),
+	)
+}
+
+// UploadFileHandler writes user-provided content to DataDir/uploads/filename.
+func (d *Deps) UploadFileHandler(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+	filename, err := req.RequireString("filename")
+	if err != nil {
+		return errResult("upload_file: " + err.Error()), nil
+	}
+	content, err := req.RequireString("content")
+	if err != nil {
+		return errResult("upload_file: " + err.Error()), nil
+	}
+
+	args := req.GetArguments()
+	encoding := "text"
+	if v, ok := args["encoding"].(string); ok && v != "" {
+		encoding = v
+	}
+	if encoding != "text" && encoding != "base64" {
+		return errResult(fmt.Sprintf("upload_file: invalid encoding %q", encoding)), nil
+	}
+
+	if err := validateUploadFilename(filename); err != nil {
+		return errResult("upload_file: " + err.Error()), nil
+	}
+
+	var data []byte
+	switch encoding {
+	case "base64":
+		decoded, err := base64.StdEncoding.DecodeString(content)
+		if err != nil {
+			return errResult("upload_file: decode base64: " + err.Error()), nil
+		}
+		data = decoded
+	default:
+		data = []byte(content)
+	}
+
+	final := filepath.Join(d.DataDir, "uploads", filename)
+	if err := os.MkdirAll(filepath.Dir(final), 0o750); err != nil {
+		return errResult("upload_file: create uploads dir: " + err.Error()), nil
+	}
+	if err := os.WriteFile(final, data, 0o640); err != nil {
+		return errResult("upload_file: write file: " + err.Error()), nil
+	}
+
+	result := map[string]any{
+		"path": fmt.Sprintf("uploads/%s", filename),
+		"size": len(data),
+	}
+	return textResult(encodeJSON(result)), nil
+}
+
+func validateUploadFilename(filename string) error {
+	if len(filename) == 0 || len(filename) > 128 {
+		return fmt.Errorf("filename length must be 1-128")
+	}
+	if filepath.Base(filename) != filename {
+		return fmt.Errorf("filename must not contain path separators or '..'")
+	}
+	if filename == "." || filename == ".." {
+		return fmt.Errorf("filename must not be '.' or '..'")
+	}
+	if !filenameRegex.MatchString(filename) {
+		return fmt.Errorf("filename contains invalid characters")
+	}
+	return nil
+}

internal/middleware/auth.go

@@ -0,0 +1,66 @@
+package middleware
+
+import (
+	"crypto/subtle"
+	"net/http"
+	"strings"
+
+	"github.com/gin-gonic/gin"
+)
+
+// AdminAuth validates Authorization: Bearer <token> against a list of
+// allowed admin tokens. On failure it aborts with 401 and a JSON error body.
+// Successful requests have the matched token stored under "admin_token".
+func AdminAuth(adminTokens []string) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		token := bearerToken(c)
+		if token == "" {
+			c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "missing bearer token"})
+			return
+		}
+		if !matchAny(token, adminTokens) {
+			c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "invalid bearer token"})
+			return
+		}
+		c.Set("admin_token", token)
+		c.Next()
+	}
+}
+
+// AgentAuth validates a single Authorization: Bearer <token> against the
+// configured agent token. It is intended for the LLM-facing /mcp endpoint.
+func AgentAuth(agentToken string) gin.HandlerFunc {
+	return func(c *gin.Context) {
+		token := bearerToken(c)
+		if token == "" || subtle.ConstantTimeCompare([]byte(token), []byte(agentToken)) != 1 {
+			c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
+			return
+		}
+		c.Set("agent_token", token)
+		c.Next()
+	}
+}
+
+func bearerToken(c *gin.Context) string {
+	h := c.GetHeader("Authorization")
+	if h == "" {
+		return ""
+	}
+	const prefix = "Bearer "
+	if !strings.HasPrefix(h, prefix) {
+		return ""
+	}
+	return strings.TrimSpace(h[len(prefix):])
+}
+
+// matchAny compares the provided token against every candidate using a
+// constant-time comparison to reduce timing side-channels.
+func matchAny(token string, candidates []string) bool {
+	t := []byte(token)
+	for _, c := range candidates {
+		if subtle.ConstantTimeCompare(t, []byte(c)) == 1 {
+			return true
+		}
+	}
+	return false
+}

internal/rm/client.go

@@ -0,0 +1,131 @@
+package rm
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+)
+
+// Client is a YARN ResourceManager REST API client bound to a single cluster.
+type Client struct {
+	hc      *httpclient.Client
+	cluster *cluster.Cluster
+}
+
+// New creates a ResourceManager client bound to c.
+func New(hc *httpclient.Client, c *cluster.Cluster) *Client {
+	return &Client{hc: hc, cluster: c}
+}
+
+// ListApps returns raw JSON from GET /ws/v1/cluster/apps[?state=&user=].
+func (r *Client) ListApps(ctx context.Context, state, user string) (json.RawMessage, error) {
+	base := r.cluster.RMURL + "/ws/v1/cluster/apps"
+	u, err := url.Parse(base)
+	if err != nil {
+		return nil, err
+	}
+	q := u.Query()
+	if state != "" {
+		q.Set("state", state)
+	}
+	if user != "" {
+		q.Set("user", user)
+	}
+	u.RawQuery = q.Encode()
+	return r.do(ctx, http.MethodGet, u.String(), nil)
+}
+
+// GetApp returns raw JSON from GET /ws/v1/cluster/apps/{id}.
+func (r *Client) GetApp(ctx context.Context, appID string) (json.RawMessage, error) {
+	return r.do(ctx, http.MethodGet, r.cluster.RMURL+"/ws/v1/cluster/apps/"+appID, nil)
+}
+
+// KillApp PUT {"state":"KILLED"} to /ws/v1/cluster/apps/{id}/state.
+func (r *Client) KillApp(ctx context.Context, appID string) (json.RawMessage, error) {
+	body := `{"state":"KILLED"}`
+	return r.do(ctx, http.MethodPut, r.cluster.RMURL+"/ws/v1/cluster/apps/"+appID+"/state",
+		strings.NewReader(body))
+}
+
+// GetLogs walks a fallback chain to retrieve YARN application logs:
+//  1. GET /ws/v1/cluster/apps/{id}/amContainerLogs (RM often 307 to a NodeManager)
+//  2. GET /ws/v1/cluster/apps/{id}/aggregated-logs?container=...
+//  3. GET /ws/v1/cluster/apps/{id}/logs
+//
+// The first successful endpoint wins.  The returned source names the endpoint
+// that produced the body.
+func (r *Client) GetLogs(ctx context.Context, appID, container string) (body []byte, source string, err error) {
+	endpoints := []struct{ path, name string }{
+		{fmt.Sprintf("/ws/v1/cluster/apps/%s/amContainerLogs", appID), "amContainerLogs"},
+		{fmt.Sprintf("/ws/v1/cluster/apps/%s/aggregated-logs?container=%s", appID, url.QueryEscape(container)), "aggregated-logs"},
+		{fmt.Sprintf("/ws/v1/cluster/apps/%s/logs", appID), "logs"},
+	}
+	var lastErr error
+	for _, ep := range endpoints {
+		b, e := r.doBytes(ctx, http.MethodGet, r.cluster.RMURL+ep.path, nil)
+		if e == nil {
+			return b, ep.name, nil
+		}
+		lastErr = e
+	}
+	return nil, "", fmt.Errorf("rm: all log endpoints failed: %w", lastErr)
+}
+
+// do executes a single HTTP request with auth and allowed-host exemptions,
+// returning the validated JSON body.
+func (r *Client) do(ctx context.Context, method, rawURL string, body io.Reader) (json.RawMessage, error) {
+	b, err := r.doBytes(ctx, method, rawURL, body)
+	if err != nil {
+		return nil, err
+	}
+	raw := json.RawMessage(b)
+	if !json.Valid(raw) {
+		return nil, fmt.Errorf("rm: invalid JSON response from %s", rawURL)
+	}
+	return raw, nil
+}
+
+func (r *Client) doBytes(ctx context.Context, method, rawURL string, body io.Reader) ([]byte, error) {
+	req, err := http.NewRequestWithContext(ctx, method, rawURL, body)
+	if err != nil {
+		return nil, err
+	}
+	if err := httpclient.ApplyAuth(req, "", r.cluster); err != nil {
+		return nil, err
+	}
+	resp, err := r.hc.DoWithRedirect(ctx, req, 5, r.allowedHosts()...)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode >= 400 {
+		return nil, fmt.Errorf("rm: %s %s returned %d", method, rawURL, resp.StatusCode)
+	}
+	return io.ReadAll(resp.Body)
+}
+
+func (r *Client) allowedHosts() []string {
+	hosts := []string{extractHost(r.cluster.RMURL)}
+	for _, pat := range r.cluster.URLAllowlist {
+		if pat == "" {
+			continue
+		}
+		hosts = append(hosts, pat)
+	}
+	return hosts
+}
+
+func extractHost(u string) string {
+	parsed, err := url.Parse(u)
+	if err != nil {
+		return ""
+	}
+	return parsed.Hostname()
+}

internal/rm/client_test.go

@@ -0,0 +1,238 @@
+package rm
+
+import (
+	"context"
+	"encoding/base64"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+
+	"spark-mcp-go/internal/cluster"
+	"spark-mcp-go/internal/httpclient"
+)
+
+func testClient(srv *httptest.Server, c *cluster.Cluster) *Client {
+	hc := httpclient.New(httpclient.Config{Timeout: 5 * time.Second, MaxResponseBytes: 1 << 20})
+	return New(hc, c)
+}
+
+func TestListApps(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/ws/v1/cluster/apps" {
+			t.Errorf("path=%q, want /ws/v1/cluster/apps", r.URL.Path)
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"apps":{"app":[]}}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	raw, err := testClient(srv, cl).ListApps(context.Background(), "", "")
+	if err != nil {
+		t.Fatalf("ListApps: %v", err)
+	}
+	if string(raw) != `{"apps":{"app":[]}}` {
+		t.Errorf("body=%q", string(raw))
+	}
+}
+
+func TestListApps_WithQuery(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		q := r.URL.Query()
+		if q.Get("state") != "RUNNING" {
+			t.Errorf("state=%q, want RUNNING", q.Get("state"))
+		}
+		if q.Get("user") != "yarn" {
+			t.Errorf("user=%q, want yarn", q.Get("user"))
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"apps":{"app":[{"id":"app_1"}]}}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	raw, err := testClient(srv, cl).ListApps(context.Background(), "RUNNING", "yarn")
+	if err != nil {
+		t.Fatalf("ListApps: %v", err)
+	}
+	if !strings.Contains(string(raw), "app_1") {
+		t.Errorf("body=%q", string(raw))
+	}
+}
+
+func TestGetApp(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/ws/v1/cluster/apps/app_123" {
+			t.Errorf("path=%q", r.URL.Path)
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"app":{"id":"app_123","state":"RUNNING"}}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	raw, err := testClient(srv, cl).GetApp(context.Background(), "app_123")
+	if err != nil {
+		t.Fatalf("GetApp: %v", err)
+	}
+	if !strings.Contains(string(raw), "RUNNING") {
+		t.Errorf("body=%q", string(raw))
+	}
+}
+
+func TestKillApp(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/ws/v1/cluster/apps/app_123/state" {
+			t.Errorf("path=%q", r.URL.Path)
+		}
+		if r.Method != http.MethodPut {
+			t.Errorf("method=%q, want PUT", r.Method)
+		}
+		body, _ := io.ReadAll(r.Body)
+		if string(body) != `{"state":"KILLED"}` {
+			t.Errorf("body=%q", string(body))
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.Write([]byte(`{"state":"KILLED"}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	raw, err := testClient(srv, cl).KillApp(context.Background(), "app_123")
+	if err != nil {
+		t.Fatalf("KillApp: %v", err)
+	}
+	if string(raw) != `{"state":"KILLED"}` {
+		t.Errorf("body=%q", string(raw))
+	}
+}
+
+func TestGetLogs_PrimarySuccess(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/ws/v1/cluster/apps/app_123/amContainerLogs" {
+			w.Write([]byte("am logs"))
+			return
+		}
+		t.Errorf("unexpected path %q", r.URL.Path)
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	body, source, err := testClient(srv, cl).GetLogs(context.Background(), "app_123", "container_1")
+	if err != nil {
+		t.Fatalf("GetLogs: %v", err)
+	}
+	if source != "amContainerLogs" {
+		t.Errorf("source=%q, want amContainerLogs", source)
+	}
+	if string(body) != "am logs" {
+		t.Errorf("body=%q", string(body))
+	}
+}
+
+func TestGetLogs_PrimaryRedirects(t *testing.T) {
+	srvNM := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		auth := r.Header.Get("Authorization")
+		if auth == "" {
+			t.Error("Authorization header missing after redirect")
+		}
+		w.Write([]byte("nm logs"))
+	}))
+	defer srvNM.Close()
+
+	srvRM := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/ws/v1/cluster/apps/app_123/amContainerLogs" {
+			w.Header().Set("Location", srvNM.URL+"/node/containerlogs/container_1/root")
+			w.WriteHeader(http.StatusTemporaryRedirect)
+			return
+		}
+		t.Errorf("unexpected RM path %q", r.URL.Path)
+	}))
+	defer srvRM.Close()
+
+	nmHost := srvNM.Listener.Addr().String()
+	cl := &cluster.Cluster{
+		RMURL:        srvRM.URL,
+		AuthType:     cluster.AuthBasic,
+		AuthUsername: "u",
+		AuthPassword: "p",
+		URLAllowlist: []string{nmHost},
+	}
+	body, source, err := testClient(srvRM, cl).GetLogs(context.Background(), "app_123", "container_1")
+	if err != nil {
+		t.Fatalf("GetLogs: %v", err)
+	}
+	if source != "amContainerLogs" {
+		t.Errorf("source=%q, want amContainerLogs", source)
+	}
+	if string(body) != "nm logs" {
+		t.Errorf("body=%q", string(body))
+	}
+}
+
+func TestGetLogs_FallbackToAggregated(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/ws/v1/cluster/apps/app_123/amContainerLogs":
+			w.WriteHeader(http.StatusInternalServerError)
+		case "/ws/v1/cluster/apps/app_123/aggregated-logs":
+			if r.URL.Query().Get("container") != "container_1" {
+				t.Errorf("container=%q, want container_1", r.URL.Query().Get("container"))
+			}
+			w.Write([]byte("aggregated logs"))
+		default:
+			t.Errorf("unexpected path %q", r.URL.Path)
+		}
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL}
+	body, source, err := testClient(srv, cl).GetLogs(context.Background(), "app_123", "container_1")
+	if err != nil {
+		t.Fatalf("GetLogs: %v", err)
+	}
+	if source != "aggregated-logs" {
+		t.Errorf("source=%q, want aggregated-logs", source)
+	}
+	if string(body) != "aggregated logs" {
+		t.Errorf("body=%q", string(body))
+	}
+}
+
+func TestApplyAuth_SimpleUserName(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if got := r.URL.Query().Get("user.name"); got != "alice" {
+			t.Errorf("user.name=%q, want alice", got)
+		}
+		w.Write([]byte(`{"apps":{"app":[]}}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL, AuthType: cluster.AuthSimple, AuthUsername: "alice"}
+	_, err := testClient(srv, cl).ListApps(context.Background(), "", "")
+	if err != nil {
+		t.Fatalf("ListApps: %v", err)
+	}
+}
+
+func TestApplyAuth_BasicHeader(t *testing.T) {
+	captured := ""
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		captured = r.Header.Get("Authorization")
+		w.Write([]byte(`{"apps":{"app":[]}}`))
+	}))
+	defer srv.Close()
+
+	cl := &cluster.Cluster{RMURL: srv.URL, AuthType: cluster.AuthBasic, AuthUsername: "u", AuthPassword: "p"}
+	_, err := testClient(srv, cl).ListApps(context.Background(), "", "")
+	if err != nil {
+		t.Fatalf("ListApps: %v", err)
+	}
+	want := "Basic " + base64.StdEncoding.EncodeToString([]byte("u:p"))
+	if captured != want {
+		t.Errorf("Authorization=%q, want %q", captured, want)
+	}
+}

internal/storage/cluster_repo.go

@@ -118,6 +118,10 @@ func (r *ClusterRepo) List(ctx context.Context) ([]*cluster.Cluster, error) {
 	if err := rows.Err(); err != nil {
 		return nil, fmt.Errorf("storage: list clusters: %w", err)
 	}
+	if out == nil {
+		// Force empty array (not null) in JSON for LLM/front-end ergonomics.
+		out = []*cluster.Cluster{}
+	}
 	return out, nil
 }
 

internal/storage/sqlite.go

@@ -83,3 +83,6 @@ func (d *DB) Close() error {
 	}
 	return d.sqlDB.Close()
 }
+
+// SQLDB returns the underlying *sql.DB handle.
+func (d *DB) SQLDB() *sql.DB { return d.sqlDB }

main.go

@@ -1,7 +1,7 @@
 // Command spark-mcp-go launches the Spark MCP Server.
 //
-// Phase 0 scope: wire config + slog + Gin + /healthz.
-// Full MCP transport, admin API, and Tool surface land in later phases.
+// Phase 5 scope: config + slog + Gin + /healthz + /admin/* + /mcp
+// Streamable HTTP. 2 MCP Tools wired (list_clusters, spark_submit).
 package main
 
 import (
@@ -16,8 +16,16 @@ import (
 
 	"github.com/gin-gonic/gin"
 
+	"spark-mcp-go/internal/admin"
+	"spark-mcp-go/internal/analyzer"
+	"spark-mcp-go/internal/audit"
 	"spark-mcp-go/internal/config"
+	"spark-mcp-go/internal/httpclient"
 	"spark-mcp-go/internal/logging"
+	mcpsrv "spark-mcp-go/internal/mcp"
+	"spark-mcp-go/internal/mcp/tools"
+	"spark-mcp-go/internal/middleware"
+	"spark-mcp-go/internal/storage"
 )
 
 func main() {
@@ -50,6 +58,13 @@ func run() error {
 		return err
 	}
 
+	db, err := storage.Open(cfg.SQLitePath)
+	if err != nil {
+		return err
+	}
+	defer db.Close()
+	logger.Info("storage.open", "path", cfg.SQLitePath)
+
 	gin.SetMode(gin.ReleaseMode)
 	r := gin.New()
 	r.Use(gin.Recovery())
@@ -58,6 +73,38 @@ func run() error {
 		c.JSON(http.StatusOK, gin.H{"ok": true, "version": "0.0.0"})
 	})
 
+	admin.Mount(r, db.Clusters(), audit.NewRepo(db), cfg.AdminTokens)
+
+	mcpHandler, err := mcpsrv.Handler(&tools.Deps{
+		Logger:             logger.With("component", "mcp"),
+		ClusterRepo:        db.Clusters(),
+		SparkSubmitTimeout: cfg.SparkSubmitTimeout,
+		HTTPClient: httpclient.New(httpclient.Config{
+			Timeout:          cfg.HTTPClientTimeout,
+			MaxResponseBytes: cfg.MaxResponseBytes,
+		}),
+		MaxResponseBytes: cfg.MaxResponseBytes,
+		DataDir:          cfg.DataDir,
+		AnalyzerThresholds: analyzer.Thresholds{
+			DataSkewRatio:       cfg.AnalyzerDataSkewRatio,
+			GCPressureRatio:     cfg.AnalyzerGCPressureRatio,
+			BottleneckShuffleGB: cfg.AnalyzerBottleneckShuffleGB,
+		},
+	})
+	if err != nil {
+		return err
+	}
+	// Wire AgentAuth for /mcp. We can't use gin.WrapH alone because
+	// AgentAuth is a gin middleware; compose it manually:
+	agentAuth := middleware.AgentAuth(cfg.AgentToken)
+	r.Any("/mcp", func(c *gin.Context) {
+		agentAuth(c)
+		if c.IsAborted() {
+			return
+		}
+		mcpHandler.ServeHTTP(c.Writer, c.Request)
+	})
+
 	srv := &http.Server{
 		Addr:              cfg.ListenAddr,
 		Handler:           r,

scripts/dev.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# One-command build + start + health check for spark-mcp-go.
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+# Load .env if present
+if [ -f .env ]; then
+    set -a
+    # shellcheck source=/dev/null
+    . ./.env
+    set +a
+fi
+
+# Required tokens
+: "${ADMIN_TOKENS:?ADMIN_TOKENS is required}"
+: "${AGENT_TOKEN:?AGENT_TOKEN is required}"
+
+LISTEN_ADDR="${LISTEN_ADDR:-:8080}"
+PORT="${LISTEN_ADDR##*:}"
+PID_FILE="./data/spark-mcp.pid"
+
+echo ">> go build"
+go build -o ./spark-mcp-go .
+
+mkdir -p ./data/logs/tools
+
+echo ">> starting server on port ${PORT}"
+nohup ./spark-mcp-go > ./data/logs/spark-mcp.log 2>&1 &
+PID=$!
+echo "${PID}" > "${PID_FILE}"
+
+sleep 2
+
+echo ">> health check"
+if curl -sS "http://127.0.0.1:${PORT}/healthz"; then
+    echo
+    echo ">> running, PID=${PID}, log=./data/logs/spark-mcp.log"
+else
+    echo
+    echo ">> health check failed; see ./data/logs/spark-mcp.log"
+    kill "${PID}" 2>/dev/null || true
+    rm -f "${PID_FILE}"
+    exit 1
+fi

scripts/seed.sh

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+# Admin POST to create a sample cluster.
+# Usage: ./scripts/seed.sh [cluster_id] [rm_url] [shs_url] [spark_submit_bin]
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+# Load .env if present
+if [ -f .env ]; then
+    set -a
+    # shellcheck source=/dev/null
+    . ./.env
+    set +a
+fi
+
+: "${ADMIN_TOKENS:?ADMIN_TOKENS is required}"
+
+HOST="${ADMIN_HOST:-http://127.0.0.1:8080}"
+TOKEN="${ADMIN_TOKENS%%,*}"
+AUTH="Authorization: Bearer ${TOKEN}"
+
+CLUSTER_ID="${1:-prod}"
+RM_URL="${2:-http://rm.example.com:8088}"
+SHS_URL="${3:-http://shs.example.com:18080}"
+SPARK_BIN="${4:-/opt/spark/bin/spark-submit}"
+
+echo ">> creating cluster '${CLUSTER_ID}' via ${HOST}/admin/clusters"
+
+curl -sS -X POST -H "${AUTH}" -H "Content-Type: application/json" \
+    -d "{\"id\":\"${CLUSTER_ID}\",\"name\":\"${CLUSTER_ID}\",\"rm_url\":\"${RM_URL}\",\"shs_url\":\"${SHS_URL}\",\"spark_submit_execute_bin\":\"${SPARK_BIN}\",\"is_active\":true,\"auth_type\":\"simple\",\"auth_username\":\"yarn\",\"rate_limit_per_min\":10,\"url_allowlist\":[\"${RM_URL#*://}\",\"${SHS_URL#*://}\"]}" \
+    "${HOST}/admin/clusters"
+
+echo