Two follow-ups to the previous error-handling fix (16fe011):
1) Drop the 1 MB cap on read_job_file and update_job_file.
The cap was originally there to keep MCP responses bounded, but it
blocks legitimate use of large PySpark scripts and large update
payloads. With the new model (LLM composes scripts via
write_job_file and edits them via read+update), a fixed 1 MB
cap is more hindrance than protection. MCP response size is
already bounded by the JSON transport and httpx; the tool itself
doesn't need a second limit.
Changes:
- tools/job_file.py: delete MAX_FILE_BYTES constant, drop the two
size checks in read_job_file and update_job_file, update
module docstring.
- tests/unit/test_job_file.py: delete test_update_rejects_
oversized_content and test_update_rejects_1mb_plus_1_byte
(the two tests that asserted the cap), replace with
test_update_accepts_content_larger_than_former_1mb_cap.
- server.py: drop "Caps reads at 1 MB" and "Caps writes at 1 MB"
from the two route descriptions.
2) Add YarnError -> 502 handler.
yarn_client wraps every httpx call: on connect / TLS / timeout /
4xx / 5xx / parse failure it raises YarnError. Previously this
was unhandled, so all six external job tools (get_external_job_*,
list_applications, plus anything else that hits YARN) returned
500 "Internal Server Error" with no detail — the LLM couldn't
tell whether the cluster was down or the request was bad.
Same fix as 16fe011 (which did this for fetch_url directly).
The new handler returns HTTP 502 Bad Gateway with the YarnError
message in the response detail. 502 because the MCP service is
acting as a gateway to YARN — 502 is the standard status for
"upstream didn't respond correctly".
Changes:
- server.py: import YarnError, add @app.exception_handler
returning 502 + the YarnError message.
- tests/integration/test_mcp_routes.py: new test asserts that
when get_application_status raises YarnError, /get_job_status
returns 502 with the YarnError message in the detail.
Note: ValueError (request was bad) is still 400, KeyError (job
not in JobStore) is still 404. The three handlers form a clean
3-way classification of tool-layer errors.
Tests: 401 passed (was 401, +1 YarnError test, -2 cap tests = net -1).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
