Self-review caught a contradiction: the spec's prose said
'MCP_SERVICES="" (empty) -> zero services mount' but the code example
in the spec used 'if not raw: return services' which conflates unset
and empty (both fall through to 'all mount').
Fix: _filter_by_env now uses 'MCP_SERVICES' not in os.environ to
distinguish unset (backward compat: all mount) from empty (explicit
choice: zero mount). Test cases already specified the correct behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds docs/superpowers/specs/2026-06-30-pluggable-mcp-design.md describing
a refactor of main.py: McpService dataclass moves to common/mcp_service.py
alongside BUILTIN_SERVICES list and discover_and_filter() helper. Each
service package gets a service.py exporting a SERVICE McpService instance.
Deployments opt in/out via MCP_SERVICES env whitelist (unset = all).
Branch: feat/files-mcp (continues from file-MCP service work; same branch,
not pushed; can be split into separate PRs later)
Backward compat: 0 existing tools / tests / dependencies changed;
unsetting MCP_SERVICES reproduces today's behavior exactly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds docs/superpowers/plans/2026-06-30-files-mcp.md — 5-task
implementation plan covering scaffold+path_guard, models+fs_ops,
business wrappers+request models, server+routes+integration tests,
and main.py+conftest wiring. 77 new tests across 3 unit files and
1 integration file; verified acceptance includes the spark_executor
test suite (242 existing) still passing unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds docs/superpowers/specs/2026-06-30-files-mcp-design.md describing a
sibling FastAPI sub-app (mount: /files-mcp) that exposes 9 file operations
(create/read/update/delete/list_dir/stat/search/move/copy) over fastapi-mcp,
sandboxed to FILES_MCP_ROOT with utf-8 text only and atomic writes.
Branch: feat/files-mcp
Mount path: /files-mcp
New package: files_mcp/
Backward compat: 0 existing tools / tests / dependencies changed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three changes the user wanted in the operation skill, all aimed at
making the LLM agent pause and ask the human instead of silently
choosing defaults or auto-confirming:
1. At task submission, ASK for connection (which one to use) and
app_name. The service has no default for app_name and rejects
implicit queue/memory/cores/num_executors defaults, so the
agent has to surface these to the user anyway — better to do
it deliberately than to call a tool, get 400, and re-ask.
2. Before prepare_submit_job, ASK for the submit parameters
(queue, executor_memory, executor_cores, num_executors,
extra_args). The agent tells the user the server-side defaults
it WOULD use so they can approve or override, instead of
picking on their behalf.
3. Before confirm_submit_job, present a one-screen summary of
exactly what will run (connection, app_name, script, queue,
resources, tracking URL) and wait for the user's explicit
affirmative ('yes', 'confirm', 'go', 'y', even an emoji).
A non-answer or 'wait' / 'let me think' is NOT consent. The
agent NEVER calls confirm_submit_job without that go-ahead.
Where the changes live in the skill:
* §0 Mindset — added a 4th 'must' about not auto-confirming,
cross-referencing the new §0.1.
* §0.1 Mandatory user confirmations — new sub-section, spells out
the three checkpoints in detail (what to ask, what counts as
consent, what doesn't).
* §2 The canonical happy path — the numbered list now interleaves
CHECKPOINT 1/2/3 lines with the tool calls, so the pauses are
visually unmistakable.
* §8 Common pitfalls — two new rows: 'auto-confirming without
explicit yes' and 'picking defaults on the user's behalf'.
* §9 End-to-end example — the word-count walkthrough now shows
the full ask flow with sample agent / user dialogue at each
checkpoint, ending in 'only NOW may you call confirm_submit_job'.
The example also adds a 'kill_job: ASK THE USER FIRST' note, since
killing a running YARN app is the same class of irreversible action
as confirming a new one.
File: 449 → 557 lines (+108).
No code changes; tests not affected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The MCP tool was named `generate_job_file` from Stage 2 but it does
NOT generate PySpark code — the calling LLM writes the code in its own
context, and this tool only persists it to a file under
SPARK_EXECUTOR_JOBS_DIR so `spark-submit` can see it. The misleading
`generate_` prefix sent agents (and humans) looking for a code
generator that doesn't exist.
This commit folds three related polish changes into one (split later
with rebase -i if you want them as separate history):
1. The rename itself:
- `tools/generate.py` → `tools/write_job.py`
- `generate_job_file` → `write_job_file`
- `GenerateJobFileRequest` → `WriteJobFileRequest`
- `/generate_job_file` route → `/write_job_file`
- `operation_id="generate_job_file"` → `operation_id="write_job_file"`
The internal helper `core.job_writer.write_job_file` (which just
writes bytes to disk with no SQL guard) is imported with an
`_write_to_disk` alias to avoid the name collision with the
MCP-exposed function in the same module.
The description for the tool now explicitly states 'this tool
does NOT generate PySpark code. The calling LLM is expected to
have already written the code; this tool only persists it.'
2. Skill for LLM agents operating the service
(`docs/superpowers/skills/spark-executor-mcp-operate/SKILL.md`,
449 lines). Covers the 16 tools, the two-step prepare/confirm
flow, the dual-ID contract (job_id vs application_id), the
PendingSubmission state machine, the Connection profile, the
job-file workflow, the error reference, common pitfalls, and a
full end-to-end word-count example.
3. Default `executor_memory` lowered 4G → 2G
(`_DEFAULTS_TO_CONFIRM` in `server.py`). Mirrors the matching
change in `test_mcp_routes.py` and the 5 unit tests that
reference the default. Aligns with the lighter workloads the
service is sized for in its current container profile.
Also tracked in git for the first time:
- `docs/superpowers/plans/2026-06-24-spark-executor-mcp.md`
(the original Stage 1/2/3 design plan, updated to use the new
tool name throughout).
Test rename:
- `tests/unit/test_generate_tool.py` → `test_write_job_tool.py`
- the new test file picks up an extra assertion that the SQL guard
rejects a `DROP TABLE` statement at write time.
243 tests pass (was 242; +1 new SQL-guard assertion). Zero regressions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
