fix: address review findings on fetch-url-tool
- fetch_url: revalidate allowlist on every redirect hop (fixes SSRF where 302 to disallowed host / 169.254.169.254 / file:// bypassed the url_allowlist). Stream response body with iter_bytes and cap at 1MB so a multi-GB response from an allowlisted host cannot OOM the service. Reuses the manual-redirect-loop pattern from yarn_client. - list_applications: stop swallowing 404 (YARN returns 200+empty for "no match"; 404 means the RM doesn't support the endpoint — surface the YarnError instead of hiding it as an empty result). Add Field(ge=1, le=10000) to ListApplicationsRequest.limit so a runaway limit is rejected at the Pydantic layer with 422. - save_connection: PATCH semantics for existing records. Re-route to update_connection when the name already exists so partial updates (e.g. only master) no longer wipe url_allowlist back to []. Uses an _UNSET sentinel in the tool function to distinguish "omitted" from "None" without breaking the existing parameter list. - README: drop leading space on 5 new connection-tool table rows that was breaking GitHub Flavored Markdown table continuity. - Indentation: normalize connections.py and requests.py to 4-space indent (auth_password/auth_principal/auth_keytab were 3-space). Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,7 @@ from unittest.mock import patch
|
||||
import pytest
|
||||
|
||||
from spark_executor.core import connection_store
|
||||
from spark_executor.core.yarn_client import YarnError
|
||||
from spark_executor.models import ApplicationSummary, Connection
|
||||
from spark_executor.tools import connections, external_jobs
|
||||
from spark_executor.tools.requests import ListApplicationsRequest
|
||||
@@ -290,3 +291,15 @@ def test_list_applications_maps_yarn_json_to_summary(fresh_stores):
|
||||
assert summary.finished_time == 0
|
||||
assert summary.tracking_url == "http://rm:8088/proxy/application_42"
|
||||
assert summary.progress == 12.5
|
||||
|
||||
|
||||
def test_list_applications_raises_on_404(fresh_stores):
|
||||
external_jobs.conn_store.save(
|
||||
Connection(name="prod", master="yarn", yarn_rm_url="http://rm:8088")
|
||||
)
|
||||
with patch(
|
||||
"spark_executor.tools.external_jobs.list_applications_yarn",
|
||||
side_effect=YarnError("YARN list applications failed: 404"),
|
||||
):
|
||||
with pytest.raises(YarnError, match="YARN list applications failed"):
|
||||
external_jobs.list_applications("prod")
|
||||
|
||||
Reference in New Issue
Block a user