tao.chen/mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(fetch_url): trust the allowlist, rename to url_allowlist

Browse Source 
Two changes:

1) Drop every check except the allowlist lookup.

  Old _validate_url_host did: scheme check, host-presence check,
  IP-literal check, empty-allowlist check, then glob match.

  New _validate_url_host does: parse host, return on glob match,
  raise on miss. That's it. The only remaining structural check is
  'the URL must have a host' (otherwise the glob has nothing to
  test against).

  Security implication: scheme (file://, gopher://, ftp://) and
  IP literals (10.0.0.1, ::1) are NO LONGER rejected by the
  validator. The allowlist is the single source of truth. If the
  user writes ['*.*.*.*'], they have opted in to 4-label hosts
  including IP literals; if they write ['ccam*'], they get ccam1-
  ccam99 and nothing else. The default ['ccam*'] / [] pattern is
  tight by construction.

  Removed: import ipaddress, the scheme/IP rejection branches, the
  'allowlist empty' explicit branch (the empty list naturally
  matches nothing).

2) Rename allowed_url_hosts -> url_allowlist.

  The previous name was a verbose double-negative ('allowed ... hosts').
  The new name is short, modern (allowlist > whitelist), and matches
  the pattern of the field (URL hosts allowed). Renamed in:
    - Connection (models.py)
    - SaveConnectionRequest, UpdateConnectionRequest, FetchUrlRequest
      (requests.py)
    - _validate_url_host, _host_matches_any_glob parameters
      (fetch_url.py)
    - save_connection / update_connection call sites and error
      messages (connections.py, fetch_url.py)
    - Route descriptions (server.py)
    - All test files
    - README

  No backward-compat alias: the field was added in 0291f36 and
  hasn't shipped, so no production migration. Local dev data
  (data/connections.json, gitignored) with allowed_url_hosts set
  will be silently dropped by Pydantic v2 (default for extra
  fields is ignore) — those connections lose their allowlist and
  fetch_url will reject everything until re-saved.

Test cleanup:
  - Removed 9 obsolete tests (scheme/IP/suffix rejection)
  - Renamed allowed_url_hosts -> url_allowlist in 11 surviving tests
  - Added 4 new tests documenting the 'allowlist is the only gate'
    model: IP literal accepted, HTTPS accepted, no-host rejected,
    empty allowlist rejected, error message mentions url_allowlist

  -1 obsolete test, net -4 from 386 -> 382 tests passing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Claude
2026-07-09 12:09:41 +08:00
co-authored by Claude Fable 5
parent 6d7387b022
commit a5b9539663
8 changed files with 138 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files
spark_executor/tools/requests.py
+13 -14
View File
@@ -119,17 +119,17 @@ class SaveConnectionRequest(BaseModel):
"for 'kinit -kt' workflows. The service does NOT auto-initialize "
"from the keytab — you must `kinit -kt <auth_keytab> <auth_principal>` "
"yourself before calling the tools."
),
)
),
)
allowed_url_hosts: list[str] | None = Field(
url_allowlist: list[str] | None = Field(
default=None,
description=(
"Optional list of fnmatch glob patterns for hosts the fetch_url tool "
"may access. See Connection.allowed_url_hosts for full semantics. "
"may access. See Connection.url_allowlist for full semantics. "
"Example for single-label host clusters: ['ccam*'] allows any host "
"starting with 'ccam' (ccam1, ccam2, ..., ccam99). If omitted, None, "
"or empty, the saved connection will have allowed_url_hosts=[] (the "
"or empty, the saved connection will have url_allowlist=[] (the "
"default), meaning fetch_url will reject every URL until the list is "
"populated via update_connection."
),
@@ -395,13 +395,12 @@ class FetchUrlRequest(BaseModel):
url: str = Field(
...,
description=(
"Absolute http:// or https:// URL to fetch. The host must share "
"at least 2 labels of suffix with the named Connection's "
"yarn_rm_url host (e.g. if yarn_rm_url is 'rm.prod.internal:8088', "
"you may fetch 'http://nm01.prod.internal:8042/...' but NOT "
"'http://evil.com/...' or 'http://10.0.0.1/...'). IP literals "
"and non-http(s) schemes are rejected. The Connection's saved "
"auth is reused for the outbound request — the agent does not "
"Absolute URL to fetch. The only access control is the named "
"Connection's url_allowlist: the URL host must match one of the "
"fnmatch glob patterns in that list. An empty or omitted allowlist "
"denies every host. IP literals and non-HTTP schemes are allowed "
"if and only if they are matched by the allowlist. The Connection's "
"saved auth is reused for the outbound request — the agent does not "
"need cluster credentials."
),
)
@@ -468,10 +467,10 @@ class UpdateConnectionRequest(BaseModel):
default=None,
description="New Kerberos keytab path. Omit to keep current.",
)
allowed_url_hosts: list[str] | None = Field(
url_allowlist: list[str] | None = Field(
default=None,
description=(
"Replacement allowed_url_hosts list (not merged). Omit to keep current. "
"Replacement url_allowlist list (not merged). Omit to keep current. "
"Pass an empty list to deny all hosts (the default for new connections). "
"Example: ['ccam*'] allows ccam1-ccam99."
),