tao.chen/spark-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity
36 Commits 1 Branch 2 Tags
d6e3952a4b4874a427f3254538f81dd45b053e85
Commit Graph
8 Commits
Author SHA1 Message Date
tao.chenandClaude df9b01272f admin: uploads list view with search, sort, and delete 
Add /admin/uploads HTML page and JSON API backed by the upload_files

DB index. The page lists file_id, name, size (KB), uploaded_at, and

sha256 with client-side search by name, column sort, and per-row delete.

DELETE /admin/uploads/:id removes the DB index row and the on-disk

data file plus .meta.json sidecar. Admin deletes are written to the

audit log.

Wire the new dependencies through admin.Mount and main.go, and run

the backfill step on startup so pre-existing sidecars are indexed.

Tests cover auth, HTML rendering, JSON search, delete removing all

artifacts, and audit entry creation.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-14 11:39:56 +08:00
tao.chenandClaude 97f6184f65 cluster: deprecate DefaultSubmitArgs (no longer prepended) 
cluster.DefaultSubmitArgs is now deprecated. It used to be prepended to
spark_submit Tool calls, but since 7920dc9 the spark-submit builder
constructs argv from the structured fields in the tool call. The field is
still accepted by the admin API and still persisted to the DB, so this is
a non-breaking change.

This commit deliberately does NOT remove:
- the Cluster.DefaultSubmitArgs field in internal/cluster/types.go
- the default_submit_args DB column
- the admin API read/write support in internal/admin/router.go
- existing storage/repo tests

Runtime now logs a one-time slog.Info when the PUT handler receives a
non-empty default_submit_args value, warning operators that the field is
ignored by spark_submit and that they should use spark_conf / extra_args
instead.

Follow-up removal should touch:
- the Cluster struct field and its JSON tag
- storage/cluster_repo.go scan/insert/update SQL and params
- admin/router.go JSON round-tripping
- storage/cluster_repo_test.go assertions
- the DB migration dropping the default_submit_args column

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-14 10:11:39 +08:00
tao.chenandClaude 67d1e9a6c5 gofmt: 清理 admin/executor 测试文件的对齐空格 
这两个文件在 HEAD 上就未通过 gofmt, 但 go test 不检查 gofmt,
所以一直没暴露. 修一下让 gofmt -l . 干净, 否则新加的 commit
之前都得先 stash.

纯格式变更, 无逻辑变化.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-13 19:55:26 +08:00
tao.chenandClaude bf2640d8fd admin: /admin HTML 公开可访问，让 token 弹窗真正能触发 
上一轮 (90eba22) 把 admin 页改成 token 弹窗模式，但 webHandler 还
挂在带 AdminAuth 的 group 里。结果无 token 访问 /admin 拿到的不是
HTML，而是 {"error":"missing bearer token"} 的 401 JSON，弹窗
变成死代码。

修复: 拆出 gPublic group，只放 webHandler (GET /admin 和 /admin/)。
API 路由 (/clusters*, /audit) 和 OpenAPI docs (/docs, /docs/spec)
继续受 AdminAuth 保护 (后者暴露内部 API 结构，不该公网可见)。

HTML 自身的 token 输入流不变 — 弹窗仍然在客户端拦截 401、清旧 token、
记录 pendingRetry、保存新 token 后自动重试。

补一个 TestAdminWeb_NoTokenReturnsHTML 锁定新契约:
  - GET /admin 无 token -> 200 + text/html (含 token-modal 元素)
  - GET /admin/ 同上
  - GET /admin/clusters 无 token -> 401 (回归保护)
  - GET /admin/clusters good-token -> 200 (回归保护)

旧 TestMount_RequiresAuth 测的全是 API 路径, 不受影响, 全绿。

验证:
  - go build / go vet / go test 全部通过
  - 端到端 curl 复现用户报告场景: /admin 无 token -> 200 + 弹窗 HTML
  - API + docs 鉴权行为不变

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-13 14:16:49 +08:00
tao.chenandClaude 90eba221d4 admin: token 输入改为弹窗模式 (modal) 
把 header 内嵌的 token 输入框 + Save 按钮拆掉，改用原生 <dialog>
弹窗。三种触发场景: 首访无 token 自动弹、API 401 自动弹 (清旧 token
后弹带错误信息)、用户主动点 Set Token 按钮弹。无有效 token 时阻止
Escape 关闭 (cancel 事件 preventDefault), 防止误操作丢失输入。
localStorage 键名 adminToken 不变, 零迁移成本。401 路径在 api()
内部清 token + 弹窗 + 记录 pendingRetry 闭包, 用户保存新 token 后
自动重试失败请求。

设计取舍:
  - 选原生 <dialog> 而非自造 div + JS, 自带 modal/背板/焦点陷阱/a11y
  - 选 <form> + preventDefault 而非 method='dialog', 避免自动关闭
  - CSS 全部复用现有 dark theme 变量, 0 视觉割裂
  - 0 新依赖 (纯 HTML5 + 原生 JS)

验证:
  - go build / go vet / go test 全部通过
  - 端到端 curl 确认新元素嵌入、旧元素删除、401/200 路径正常
  - router_test.go 无回归 (HTML 改动不涉及 Go)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-13 14:08:16 +08:00
tao.chenandClaude 756a88a800 Linus 修复: PUT 部分更新用 map[string]json.RawMessage 区分 absent/present 
原 PUT /admin/clusters/:id 用 ShouldBindJSON + struct, 缺字段自动
填 Go zero value, 无法区分 "未传" vs "传零值":
- true bool → 缺省 = false → 被误翻成 false
- 1 int → 缺省 = 0 → 丢失
- "" string → 缺省 = "" → 清空

第二次 (用 if-zero-value-keep-old 逻辑) 实际更糟: bool "缺省 = false"
跟 old "true" 不等 → 反而把 true 翻成 false。

正确做法: 用 map[string]json.RawMessage 解析 body, 用
_, ok := rawBody["field"] 判断 absent, 缺字段 = 保留 old。

- internal/admin/router.go: 改 updateCluster 解析方式,
  加 decodeString/Int/Bool/StringSlice helpers
  + validateClusterUpdate (轻量校验, 不要求所有必填字段)
- internal/admin/router_test.go: +2 测试
  - TestUpdateCluster_PartialUpdatePreservesOtherFields
    (单字段 PUT 保留其他全部)
  - TestUpdateCluster_SliceReplacement
    (slice 显式覆盖, nil = 保留, [] = 替换)

端到端实测:
- PUT rate_limit=99 保留 is_active=true (修复前会被翻 false)
- PUT is_active=false 保留 rate_limit=99
- PUT auth_type=basic 保留其他全部
- PUT url_allowlist=[only] 替换 slice
- audit 5 条 (1 create + 4 update) DESC 正确

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-10 18:23:44 +08:00
tao.chenandClaude 9a48f9e68e Phase 7 起步: Admin OpenAPI + Scalar UI + Cluster CRUD HTML 
用户问题: 有 list_clusters MCP tool 但没简易 admin HTML。
A + B 方案:
- A: OpenAPI 文档 + Scalar UI(让人看 + 试 admin API)
- B: Cluster CRUD HTML 页面(让人填表创建/改 cluster)

实现:
- internal/admin/openapi.yaml: 手写 6 admin 端点 + Cluster/AuditEntry/Error
  schema。auth_password 不在 spec(因 json:"-" tag)
- internal/admin/openapi.go: //go:embed openapi.yaml + Scalar HTML
  GET /admin/docs 返 Scalar UI (CDN), GET /admin/docs/spec 返 YAML
- internal/admin/web/cluster.html: 单文件 401 行暗色 monospace 风格
  cluster CRUD 页面, 14 字段表单, vanilla JS + localStorage token
  + 内嵌 CSS, 无前端框架, 无 bundler
- internal/admin/web.go: //go:embed web/cluster.html
- internal/admin/router.go: Mount 里追加 4 路由 (GET /admin, /admin/,
  /admin/docs, /admin/docs/spec), 全部在 AdminAuth 组内

Linus 视角决策: 整个 /admin 子树 (含 HTML) 统一用 AdminAuth —
用户进页面前输 token, JS 存 localStorage, 后续 fetch 自动加 header。
简单且一致, 不搞"白名单 + 不带 token 也能看页"这种特殊 case。

Constraint:
- 不引第三方 Go 库 (不用 gofiber/swagger/scalar-go)
- 不引前端框架 (no React/Vue)
- 不开 JS bundler (no webpack/vite)
- HTML/CSS/JS 全内嵌在 cluster.html 一个文件
- Scalar 走 CDN (@scalar/api-reference latest)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-10 18:17:37 +08:00
tao.chenandClaude 2c39091706 Phase 3+4: 共享 HTTP 客户端 + admin API + audit log 
Phase 3 (httpclient):
- shared Client (Timeout + MaxResponseBytes 截断)
- SSRF: 16 段私网 CIDR (含 169.254/::ffff:0:0/96 IPv4-mapped)
  + hostname allowlist (精确/后缀/*. 通配)
  + scheme 校验 (仅 http/https)
- auth 适配器: none / simple (YARN user.name) / basic (SetBasicAuth)
- DoWithRedirect: 跨主机跳转保留 Authorization, max 5 默认
- 19 个测试全绿 (httptest 模拟)

Phase 4 (admin + audit):
- internal/audit: Entry + Repo.Insert/List + MarshalDetails, snake_case JSON
- internal/middleware: AdminAuth/AgentAuth (constant-time 比对)
- internal/admin: 6 端点 (GET/POST/PUT/DELETE /admin/clusters, GET /admin/audit)
  + 写操作触发 audit_log (含 before/after diff)
  + AuthPassword 空=保留旧密码
  + 校验: 必填字段 + auth_type 枚举 + rate_limit>=0
- main.go: 挂载 storage.Open + admin.Mount
- L fix: audit.Entry 加 json tag (smoke test 发现大写 key 不规范)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-10 13:54:24 +08:00