refactor: replace yarn CLI shell-out with YARN REST API

yarn_client.py no longer invokes the 'yarn' binary via subprocess; it uses
httpx against /ws/v1/cluster/apps/* endpoints. This means the runtime image
no longer needs the Hadoop client installation — the only YARN-side
dependency left in the container is the config dir consumed by
spark-submit itself.

New model field:
  - Job.yarn_rm_url: str | None
  - PendingSubmission.yarn_rm_url: str | None (snapshotted at prepare)

prepare_submit_job snapshots Connection.yarn_rm_url into the pending
record (consistent with the existing master/deploy_mode/spark_conf
snapshot pattern); confirm_submit_job copies it onto the Job so
status/logs/kill can use it without re-looking-up the connection.

Resolution order for the RM URL at runtime:
  1. Job.yarn_rm_url (preferred — survives connection edits/deletes)
  2. Connection.yarn_rm_url fallback (if a future tool is added that
     doesn't go through a Job)
  3. YARN_RESOURCE_MANAGER_URL env var

Errors:
  - YarnConfigError (HTTP 4xx semantics) when URL is missing/malformed
  - YarnError for HTTP 4xx/5xx from the RM, network failures, missing
    state field, or unparseable log responses

10 new tests in test_yarn_client.py cover the REST surface:
success, 404, 5xx, missing state field, env-var fallback, malformed
URL, log 404 with log-aggregation hint, kill PUT body shape, and
httpx connection-error wrapping.

spark_executor/tools/kill.py

@@ -15,7 +15,7 @@ def kill_job(job_id: str) -> dict[str, str]:
     job = store.get(job_id)
     if job is None:
         raise KeyError(f"Unknown job_id: {job_id}")
-    kill_application(job.application_id)
+    kill_application(job.application_id, job.yarn_rm_url)
     logger.info(f"kill_job ok job_id={job_id} application_id={job.application_id}")
     return {
         "job_id": job_id,

spark_executor/tools/logs.py

@@ -15,7 +15,7 @@ def get_job_logs(job_id: str, tail_chars: int = 5000) -> str:
     job = store.get(job_id)
     if job is None:
         raise KeyError(f"Unknown job_id: {job_id}")
-    full = get_application_logs(job.application_id)
+    full = get_application_logs(job.application_id, job.yarn_rm_url)
     tailed = full[-tail_chars:] if len(full) > tail_chars else full
     logger.info(
         f"get_job_logs ok job_id={job_id} application_id={job.application_id} "

spark_executor/tools/status.py

@@ -16,6 +16,6 @@ def get_job_status(job_id: str) -> JobStatus:
     job = store.get(job_id)
     if job is None:
         raise KeyError(f"Unknown job_id: {job_id}")
-    state, raw = get_application_status(job.application_id)
+    state, raw = get_application_status(job.application_id, job.yarn_rm_url)
     logger.info(f"get_job_status ok job_id={job_id} application_id={job.application_id} state={state}")
     return JobStatus(application_id=job.application_id, state=state, raw=raw)

spark_executor/tools/submit.py

@@ -49,6 +49,7 @@ def prepare_submit_job(
         connection=connection,
         master=conn.master,
         deploy_mode=conn.deploy_mode,
+        yarn_rm_url=conn.yarn_rm_url,
         script_path=script_path,
         queue=queue,
         executor_memory=executor_memory,
@@ -119,6 +120,7 @@ def confirm_submit_job(*, pending_id: str) -> SubmitResult:
             queue=pending.queue,
             submit_time=datetime.utcnow(),
             connection=pending.connection,
+            yarn_rm_url=pending.yarn_rm_url,
         )
     )