tao.chen/mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: validate script_path exists + tell agent to call generate_job_file

Browse Source 
Two problems with the prior prepare_submit_job flow:

  1. The agent could pass a script_path that only existed in its own
     context (LLM-generated code not yet on disk) or a path on the host
     filesystem that's invisible inside the container. The new check
     surfaces this as a 400 with a clear remediation hint instead of
     letting spark-submit fail later with an opaque FileNotFoundError -> 500.

  2. The container-isolation issue: any path the agent gives is interpreted
     inside the container. The two ways a file can legitimately exist there
     are (a) generate_job_file(code=...) just wrote it to
     SPARK_EXECUTOR_JOBS_DIR (the default ./data/jobs/ is the only
     gitignored dir that survives restarts), or (b) a host dir was
     mounted via -v. The error message spells both out so an agent can
     self-correct.

Implementation:
  - submit.py: new _check_script_path() that raises ValueError (-> 400)
    when the path is missing, empty, or a directory. Called in both
    prepare_submit_job and confirm_submit_job (defense in depth).
  - prepare_submit_job checks the connection FIRST (KeyError -> 404)
    before the script (ValueError -> 400), so an agent with both problems
    sees the more fundamental 'unknown connection' error first.
  - server.py / requests.py: route description and Pydantic field
    description spell out the generate_job_file pattern so an LLM
    reading the tool schema learns the right next step.

8 new tests; 8 existing tests adjusted to create real files (they used
synthetic /tmp/*.py paths that don't exist).
This commit is contained in:
Claude
2026-06-25 10:41:19 +08:00
parent 994c2d67ab
commit dd197019f9
6 changed files with 173 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/unit/test_logging.py
+5 -3
View File
@@ -42,14 +42,16 @@ def test_save_connection_emits_info_log(log_capture):
assert "connection saved" in text
def test_prepare_submit_job_emits_debug_and_info(log_capture):
def test_prepare_submit_job_emits_debug_and_info(log_capture, tmp_path):
connections.save_connection(name="prod", master="yarn", deploy_mode="cluster")
script = tmp_path / "demo.py"
script.write_text("print('hi')\n")
log_capture.truncate(0); log_capture.seek(0)
submit.prepare_submit_job(connection="prod", script_path="/tmp/j.py", queue="research")
submit.prepare_submit_job(connection="prod", script_path=str(script), queue="research")
text = log_capture.getvalue()
assert "DEBUG|prepare_submit_job enter" in text
assert "INFO|prepare_submit_job ok" in text
assert "script_path=/tmp/j.py" in text
assert f"script_path={script}" in text
assert "queue=research" in text