feat(result): get_job_result tool for terminal view of Spark job
`get_job_status` returns YARN state + the raw response blob, so the terminal fields (finalStatus, diagnostics, trackingUrl, startedTime, finishedTime) are buried inside `raw` and not surfaced in a structured form. Add a new tool that parses them. `get_job_result(job_id)` reuses `yarn_client.get_application_status` and extracts: - finalStatus (SUCCEEDED / FAILED / KILLED / UNDEFINED) - diagnostics (YARN final message) - tracking_url (Spark Web UI) - started_time / finished_time (epoch ms) All five fields are optional: running jobs have no `finishedTime`, and older YARN versions (CDH 5 / H2) may omit some fields. Missing fields stay None — never raise. Coexistence with `get_job_status` is intentional: the latter is for polling the running YARN state, the former is the terminal view. Tests: 4 cases — happy path, running job (no finishedTime), bare-minimum raw (all optionals None), unknown job_id raises KeyError. uv run pytest -> 171 passed.
This commit is contained in:
@@ -37,6 +37,8 @@ from spark_executor.tools.submit import (
|
||||
prepare_submit_job,
|
||||
)
|
||||
|
||||
from spark_executor.tools.result import get_job_result
|
||||
|
||||
app = FastAPI(title="Spark Executor MCP", version="0.0.1", description="Spark Executor MCP Server")
|
||||
|
||||
|
||||
@@ -149,6 +151,20 @@ def _get_job_status(req: JobIdRequest):
|
||||
return get_job_status(req.job_id)
|
||||
|
||||
|
||||
@app.post(
|
||||
"/get_job_result",
|
||||
summary="Query YARN for a job's terminal result view",
|
||||
description=(
|
||||
"Return a terminal-oriented view of a Spark job: final_status, "
|
||||
"diagnostics, tracking_url, started_time, and finished_time. "
|
||||
"This is distinct from get_job_status, which is for polling the "
|
||||
"running YARN state and returns the raw YARN response."
|
||||
),
|
||||
)
|
||||
def _get_job_result(req: JobIdRequest):
|
||||
return get_job_result(req.job_id)
|
||||
|
||||
|
||||
@app.post(
|
||||
"/get_job_logs",
|
||||
summary="Fetch aggregated container logs for a job",
|
||||
|
||||
Reference in New Issue
Block a user