feat(fetch_url): per-Connection allowed_url_hosts glob allowlist
The default 'URL host shares >= 2 labels of suffix with yarn_rm_url host' rule is too strict for clusters whose hostnames are single-label (e.g. 'ccam1' through 'ccam99'). The user's cluster is reachable at http://ccam1:8088, and they want fetch_url to work for any ccamN host — but ccam1 and ccam50 share 0 suffix labels, so the existing rule rejects everything. Add an opt-in allowlist field on Connection: allowed_url_hosts: list[str] | None Each entry is an fnmatch glob pattern. The URL host is allowed if it matches ANY pattern, regardless of the suffix rule. Save a Connection with ['ccam*'] to allow ccam1, ccam2, ..., ccam99. SSRF safety: the implementation does NOT use vanilla fnmatch on the flat host string (because '*' in fnmatch crosses '.', so 'ccam*' would match 'ccam50.evil.com' — a security hole). Instead, both the pattern and the host are split on '.' and matched LABEL-BY-LABEL with the label counts required to match exactly. So 'ccam*' matches 'ccam50' but NOT 'ccam50.evil.com' (different label counts). Test coverage: - 8 new tests in tests/unit/test_fetch_url.py (glob allow, glob deny, dot-boundary SSRF test, multiple globs, empty list, None fallback, error message hint) - All 21 fetch_url tests pass; 377 total. - spark_executor/models.py: Connection.allowed_url_hosts (with description) - spark_executor/tools/requests.py: SaveConnectionRequest.allowed_url_hosts - spark_executor/tools/fetch_url.py: new _host_matches_any_glob helper; _validate_url_host now takes allowed_hosts and checks globs BEFORE the suffix rule - tests/unit/test_fetch_url.py: 8 new tests - README.md: fetch_url row mentions the allowlist Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -117,12 +117,22 @@ class SaveConnectionRequest(BaseModel):
|
||||
description=(
|
||||
"Absolute path to a Kerberos keytab file. Optional convenience "
|
||||
"for 'kinit -kt' workflows. The service does NOT auto-initialize "
|
||||
"from the keytab — you must `kinit -kt <auth_keytab> <auth_principal>` "
|
||||
"yourself before calling the tools."
|
||||
"from the keytab — you must `kinit -kt <auth_keytab> <auth_principal>` "
|
||||
"yourself before calling the tools."
|
||||
),
|
||||
)
|
||||
|
||||
allowed_url_hosts: list[str] | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Optional list of fnmatch glob patterns for hosts the fetch_url tool "
|
||||
"may access. See Connection.allowed_url_hosts for full semantics. "
|
||||
"Example for single-label host clusters: ['ccam*'] allows any host "
|
||||
"starting with 'ccam' (ccam1, ccam2, ..., ccam99). Default None means "
|
||||
"only the default 'shares >= 2 labels of suffix with yarn_rm_url' rule."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class PrepareSubmitJobRequest(BaseModel):
|
||||
connection: str = Field(
|
||||
...,
|
||||
|
||||
Reference in New Issue
Block a user