feat(result): get_job_result tool for terminal view of Spark job

`get_job_status` returns YARN state + the raw response blob, so the
terminal fields (finalStatus, diagnostics, trackingUrl, startedTime,
finishedTime) are buried inside `raw` and not surfaced in a structured
form. Add a new tool that parses them.

`get_job_result(job_id)` reuses `yarn_client.get_application_status` and
extracts:
  - finalStatus  (SUCCEEDED / FAILED / KILLED / UNDEFINED)
  - diagnostics  (YARN final message)
  - tracking_url (Spark Web UI)
  - started_time / finished_time (epoch ms)

All five fields are optional: running jobs have no `finishedTime`, and
older YARN versions (CDH 5 / H2) may omit some fields. Missing fields
stay None — never raise.

Coexistence with `get_job_status` is intentional: the latter is for
polling the running YARN state, the former is the terminal view.

Tests: 4 cases — happy path, running job (no finishedTime), bare-minimum
raw (all optionals None), unknown job_id raises KeyError. uv run pytest
-> 171 passed.
This commit is contained in:
Claude
2026-06-26 11:21:53 +08:00
parent a5c587dc5f
commit 70400d3ed1
4 changed files with 185 additions and 0 deletions
+36
View File
@@ -0,0 +1,36 @@
# coding=utf-8
"""
@Time :2026/6/26
@Author :tao.chen
"""
import json
from common.logging import logger
from spark_executor.core.job_store import JobStore
from spark_executor.core.yarn_client import get_application_status
from spark_executor.models import JobResult
store = JobStore()
def get_job_result(job_id: str) -> JobResult:
logger.debug(f"get_job_result enter job_id={job_id}")
job = store.get(job_id)
if job is None:
raise KeyError(f"Unknown job_id: {job_id}")
state, raw = get_application_status(job.application_id, job.yarn_rm_url)
app = json.loads(raw).get("app", {})
result = JobResult(
application_id=job.application_id,
state=state,
final_status=app.get("finalStatus"),
diagnostics=app.get("diagnostics"),
tracking_url=app.get("trackingUrl"),
started_time=app.get("startedTime"),
finished_time=app.get("finishedTime"),
)
logger.info(
f"get_job_result ok job_id={job_id} application_id={job.application_id} "
f"state={state} final_status={result.final_status}"
)
return result