refactor(mcp): rename generate_job_file to write_job_file to match what it does

The MCP tool was named `generate_job_file` from Stage 2 but it does
NOT generate PySpark code — the calling LLM writes the code in its own
context, and this tool only persists it to a file under
SPARK_EXECUTOR_JOBS_DIR so `spark-submit` can see it. The misleading
`generate_` prefix sent agents (and humans) looking for a code
generator that doesn't exist.

This commit folds three related polish changes into one (split later
with rebase -i if you want them as separate history):

  1. The rename itself:
     - `tools/generate.py`  →  `tools/write_job.py`
     - `generate_job_file`  →  `write_job_file`
     - `GenerateJobFileRequest`  →  `WriteJobFileRequest`
     - `/generate_job_file` route  →  `/write_job_file`
     - `operation_id="generate_job_file"`  →  `operation_id="write_job_file"`
     The internal helper `core.job_writer.write_job_file` (which just
     writes bytes to disk with no SQL guard) is imported with an
     `_write_to_disk` alias to avoid the name collision with the
     MCP-exposed function in the same module.
     The description for the tool now explicitly states 'this tool
     does NOT generate PySpark code. The calling LLM is expected to
     have already written the code; this tool only persists it.'

  2. Skill for LLM agents operating the service
     (`docs/superpowers/skills/spark-executor-mcp-operate/SKILL.md`,
     449 lines). Covers the 16 tools, the two-step prepare/confirm
     flow, the dual-ID contract (job_id vs application_id), the
     PendingSubmission state machine, the Connection profile, the
     job-file workflow, the error reference, common pitfalls, and a
     full end-to-end word-count example.

  3. Default `executor_memory` lowered 4G → 2G
     (`_DEFAULTS_TO_CONFIRM` in `server.py`). Mirrors the matching
     change in `test_mcp_routes.py` and the 5 unit tests that
     reference the default. Aligns with the lighter workloads the
     service is sized for in its current container profile.

Also tracked in git for the first time:
  - `docs/superpowers/plans/2026-06-24-spark-executor-mcp.md`
    (the original Stage 1/2/3 design plan, updated to use the new
    tool name throughout).

Test rename:
  - `tests/unit/test_generate_tool.py`  →  `test_write_job_tool.py`
  - the new test file picks up an extra assertion that the SQL guard
    rejects a `DROP TABLE` statement at write time.

243 tests pass (was 242; +1 new SQL-guard assertion). Zero regressions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

spark_executor/server.py

@@ -12,14 +12,14 @@ from spark_executor.tools.connections import (
     list_connections,
     save_connection,
 )
-from spark_executor.tools.generate import generate_job_file
+from spark_executor.tools.write_job import write_job_file
 from spark_executor.tools.job_file import read_job_file, update_job_file
 from spark_executor.tools.kill import kill_job
 from spark_executor.tools.logs import get_job_logs
 from spark_executor.tools.requests import (
     ConnectionNameRequest,
     EmptyRequest,
-    GenerateJobFileRequest,
+    WriteJobFileRequest,
     GetJobLogsRequest,
     JobIdRequest,
     PendingIdRequest,
@@ -46,7 +46,7 @@ app = FastAPI(title="Spark Executor MCP", version="0.0.1", description="Spark Ex
 
 _DEFAULTS_TO_CONFIRM = {
     "queue": "default",
-    "executor_memory": "4G",
+    "executor_memory": "2G",
     "executor_cores": 2,
     "num_executors": 2,
 }
@@ -90,7 +90,7 @@ def health_check():
         "yarn_rm_url into a PendingSubmission record and persist it. "
         "Does NOT invoke spark-submit. Returns pending_id for use with "
         "confirm_submit_job (the user-second-confirmation step).\n\n"
-        "REQUIRED PATTERN for LLM-generated code: call generate_job_file(code=...) "
+        "REQUIRED PATTERN for LLM-generated code: call write_job_file(code=...) "
         "first, then pass the returned script_path here. Direct submission with a "
         "synthetic path (one that only exists in the agent's context) will be "
         "rejected with HTTP 400 — the script must exist inside the container's "
@@ -297,19 +297,24 @@ def _delete_connection(req: ConnectionNameRequest):
 # --- LLM-driven PySpark generation (Stage 2) ---
 
 @app.post(
-    "/generate_job_file",
-    operation_id="generate_job_file",
-    summary="Write LLM-generated PySpark code to disk",
+    "/write_job_file",
+    operation_id="write_job_file",
+    summary="Write LLM-authored PySpark code to disk",
     description=(
-        "Takes a PySpark code string and writes it to a timestamped file under "
+        "Takes a PySpark code string the LLM has already composed in its "
+        "context and writes it to a timestamped file under "
         "SPARK_EXECUTOR_JOBS_DIR (default ./data/jobs/). Returns the absolute "
         "path for use as the script_path argument of prepare_submit_job — the "
-        "two-step pattern means the LLM can produce code, the user can review "
-        "the resulting file, and only then is the job submitted."
+        "two-step pattern means the LLM writes the file, the user can review "
+        "it (via read_job_file), and only then is the job submitted.\n\n"
+        "Note: this tool does NOT generate PySpark code. The calling LLM is "
+        "expected to have already written the code; this tool only persists "
+        "it. Code is also run through the SQL safety policy (SELECT/INSERT "
+        "only) before being written — forbidden statements cause a 400."
     ),
 )
-def _generate_job_file(req: GenerateJobFileRequest):
-    return generate_job_file(req.code)
+def _write_job_file(req: WriteJobFileRequest):
+    return write_job_file(req.code)
 
 
 @app.post(
@@ -318,7 +323,7 @@ def _generate_job_file(req: GenerateJobFileRequest):
     summary="Read the contents of an existing PySpark script",
     description=(
         "Returns the text content of an existing script file at the given "
-        "path. Caps reads at 1 MB. Typical use: after generate_job_file "
+        "path. Caps reads at 1 MB. Typical use: after write_job_file "
         "returns a path, call read_job_file on that path to inspect what "
         "was actually written, before deciding to prepare_submit_job or "
         "update_job_file."
@@ -334,7 +339,7 @@ def _read_job_file(req: ReadJobFileRequest):
     summary="Overwrite an existing PySpark script with new content",
     description=(
         "Replaces the entire content of an existing script file. Path must "
-        "be under SPARK_EXECUTOR_JOBS_DIR (the dir generate_job_file writes "
+        "be under SPARK_EXECUTOR_JOBS_DIR (the dir write_job_file writes "
         "to) — protects against overwriting host-mounted configs or other "
         "non-script files. Caps writes at 1 MB. Typical use: read_job_file, "
         "edit the content (LLM or human), update_job_file, then "