refactor(fetch_url): trust the allowlist, rename to url_allowlist
Two changes:
1) Drop every check except the allowlist lookup.
Old _validate_url_host did: scheme check, host-presence check,
IP-literal check, empty-allowlist check, then glob match.
New _validate_url_host does: parse host, return on glob match,
raise on miss. That's it. The only remaining structural check is
'the URL must have a host' (otherwise the glob has nothing to
test against).
Security implication: scheme (file://, gopher://, ftp://) and
IP literals (10.0.0.1, ::1) are NO LONGER rejected by the
validator. The allowlist is the single source of truth. If the
user writes ['*.*.*.*'], they have opted in to 4-label hosts
including IP literals; if they write ['ccam*'], they get ccam1-
ccam99 and nothing else. The default ['ccam*'] / [] pattern is
tight by construction.
Removed: import ipaddress, the scheme/IP rejection branches, the
'allowlist empty' explicit branch (the empty list naturally
matches nothing).
2) Rename allowed_url_hosts -> url_allowlist.
The previous name was a verbose double-negative ('allowed ... hosts').
The new name is short, modern (allowlist > whitelist), and matches
the pattern of the field (URL hosts allowed). Renamed in:
- Connection (models.py)
- SaveConnectionRequest, UpdateConnectionRequest, FetchUrlRequest
(requests.py)
- _validate_url_host, _host_matches_any_glob parameters
(fetch_url.py)
- save_connection / update_connection call sites and error
messages (connections.py, fetch_url.py)
- Route descriptions (server.py)
- All test files
- README
No backward-compat alias: the field was added in 0291f36 and
hasn't shipped, so no production migration. Local dev data
(data/connections.json, gitignored) with allowed_url_hosts set
will be silently dropped by Pydantic v2 (default for extra
fields is ignore) — those connections lose their allowlist and
fetch_url will reject everything until re-saved.
Test cleanup:
- Removed 9 obsolete tests (scheme/IP/suffix rejection)
- Renamed allowed_url_hosts -> url_allowlist in 11 surviving tests
- Added 4 new tests documenting the 'allowlist is the only gate'
model: IP literal accepted, HTTPS accepted, no-host rejected,
empty allowlist rejected, error message mentions url_allowlist
-1 obsolete test, net -4 from 386 -> 382 tests passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -148,10 +148,10 @@ def test_update_connection_with_no_fields_is_noop(_fresh_store):
|
||||
assert out["deploy_mode"] == "client"
|
||||
|
||||
|
||||
def test_update_connection_allows_url_hosts_change(_fresh_store):
|
||||
def test_update_connection_changes_url_allowlist(_fresh_store):
|
||||
connections.save_connection(name="prod", master="yarn")
|
||||
out = connections.update_connection(name="prod", allowed_url_hosts=["ccam*"])
|
||||
assert out["allowed_url_hosts"] == ["ccam*"]
|
||||
out = connections.update_connection(name="prod", url_allowlist=["ccam*"])
|
||||
assert out["url_allowlist"] == ["ccam*"]
|
||||
|
||||
|
||||
def test_update_connection_replaces_not_merges_dict(_fresh_store):
|
||||
@@ -160,14 +160,14 @@ def test_update_connection_replaces_not_merges_dict(_fresh_store):
|
||||
assert out["spark_conf"] == {"b": "2"}
|
||||
|
||||
|
||||
def test_update_connection_replaces_not_merges_list(_fresh_store):
|
||||
def test_update_connection_replaces_not_merges_url_allowlist(_fresh_store):
|
||||
connections.save_connection(
|
||||
name="prod",
|
||||
master="yarn",
|
||||
allowed_url_hosts=["ccam*"],
|
||||
url_allowlist=["ccam*"],
|
||||
)
|
||||
out = connections.update_connection(name="prod", allowed_url_hosts=["nm*"])
|
||||
assert out["allowed_url_hosts"] == ["nm*"]
|
||||
out = connections.update_connection(name="prod", url_allowlist=["nm*"])
|
||||
assert out["url_allowlist"] == ["nm*"]
|
||||
|
||||
|
||||
def test_update_connection_raises_for_unknown_name(_fresh_store):
|
||||
|
||||
Reference in New Issue
Block a user