feat(fetch_url): per-Connection allowed_url_hosts glob allowlist
The default 'URL host shares >= 2 labels of suffix with yarn_rm_url host' rule is too strict for clusters whose hostnames are single-label (e.g. 'ccam1' through 'ccam99'). The user's cluster is reachable at http://ccam1:8088, and they want fetch_url to work for any ccamN host — but ccam1 and ccam50 share 0 suffix labels, so the existing rule rejects everything. Add an opt-in allowlist field on Connection: allowed_url_hosts: list[str] | None Each entry is an fnmatch glob pattern. The URL host is allowed if it matches ANY pattern, regardless of the suffix rule. Save a Connection with ['ccam*'] to allow ccam1, ccam2, ..., ccam99. SSRF safety: the implementation does NOT use vanilla fnmatch on the flat host string (because '*' in fnmatch crosses '.', so 'ccam*' would match 'ccam50.evil.com' — a security hole). Instead, both the pattern and the host are split on '.' and matched LABEL-BY-LABEL with the label counts required to match exactly. So 'ccam*' matches 'ccam50' but NOT 'ccam50.evil.com' (different label counts). Test coverage: - 8 new tests in tests/unit/test_fetch_url.py (glob allow, glob deny, dot-boundary SSRF test, multiple globs, empty list, None fallback, error message hint) - All 21 fetch_url tests pass; 377 total. - spark_executor/models.py: Connection.allowed_url_hosts (with description) - spark_executor/tools/requests.py: SaveConnectionRequest.allowed_url_hosts - spark_executor/tools/fetch_url.py: new _host_matches_any_glob helper; _validate_url_host now takes allowed_hosts and checks globs BEFORE the suffix rule - tests/unit/test_fetch_url.py: 8 new tests - README.md: fetch_url row mentions the allowlist Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -69,6 +69,21 @@ class Connection(BaseModel):
|
||||
auth_principal: str | None = None
|
||||
auth_keytab: str | None = None
|
||||
|
||||
allowed_url_hosts: list[str] | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Optional list of fnmatch glob patterns for hosts the fetch_url tool "
|
||||
"may access, in addition to the default 'shares >= 2 labels of suffix "
|
||||
"with yarn_rm_url host' rule. Useful for clusters whose hostnames do "
|
||||
"NOT share a 2+ label suffix — e.g. single-label hosts like 'ccam1'-"
|
||||
"'ccam99' (configure ['ccam*']) or HDFS namenode on a different "
|
||||
"subdomain ('*.hadoop.internal'). Patterns are matched against the "
|
||||
"URL host only (no port, no path). fnmatch rules apply: '*' does NOT "
|
||||
"match '.', so 'ccam*' matches 'ccam50' but not 'ccam50.evil.com'. "
|
||||
"Default None means the suffix rule alone applies."
|
||||
),
|
||||
)
|
||||
|
||||
@field_validator("master")
|
||||
@classmethod
|
||||
def _check_master(cls, v: str) -> str:
|
||||
|
||||
Reference in New Issue
Block a user