Follow-up to the layered refactor (review-driven):
- Move scheduling/orchestrator.py -> application/orchestrator.py
(orchestrator is application-level coordination, not a cron-trigger
primitive; matches the intended target tree)
- Migrate orchestrator re-exports from scheduling/__init__.py to
application/__init__.py; scheduling/ now exposes only CronScheduler
- Rewrite imports + 5 mock.patch string targets in test_janitor.py and
the orchestrator import in test_layering.py
- Update docstring refs in application/service.py + execution/worker.py
- Add 4 runner smoke tests (test_layering.py): _limited_log under-limit /
empty-sentinel / above-MAX_LOG_BYTES truncation; execute_artifact
rejects unsupported script_type with ValueError
- infrastructure/__init__.py re-exports SchedulerStorageClient so
``from schedule.infrastructure import SchedulerStorageClient`` is a
stable top-level surface
- CLAUDE.md engineering note: extend the commit trail to 7476c27 and
note the stage-8 orchestrator placement
Zero behavior change; schedule/pyproject.toml untouched. 29 tests green.
Co-Authored-By: Claude <noreply@anthropic.com>
Pin the new five-layer contract so a later refactor can't silently break
an import surface or lifecycle:
- domain: ExecutionResult defaults, terminal/failed state-set invariants,
naive_utc normalization (import-side-effect-free)
- infrastructure.storage: SchedulerStorageClient base64 upload via
httpx.MockTransport (no live server; base_url required for relative URL)
- scheduling: CronScheduler start/close lifecycle, global trigger cleared
- application: SchedulerService wires cron + orchestrator + worker, handler
dispatch table points at the wired NodeExecutor
- execution: schedule.notebook_runner shim re-exports the real main
25 schedule tests green; schedule/pyproject.toml untouched.
Co-Authored-By: Claude <noreply@anthropic.com>
- Move service.py -> application/service.py (SchedulerService class name
unchanged; build_object_store / build_storage_http_client move along)
- main.py imports schedule.application.service
- Rewrite worker.py's lazy `from schedule.service import build_object_store`
and test_worker.py's mock patch string targets — same class of bug as the
test_janitor patch strings (silent no-op until the old file is deleted)
- Delete flat service.py (orphaned; only docstring refs remain in
orchestrator, cleaned up in stage 6)
- Zero behavior change; schedule/pyproject.toml untouched
Co-Authored-By: Claude <noreply@anthropic.com>
- Move worker.py -> execution/worker.py, executor.py -> execution/executor.py
(byte-identical copies; import sites updated)
- Merge old execution.py + notebook_runner.py into
execution/runners/notebook.py: subprocess CLI (main/emit_outputs) plus the
in-process helpers (_execute_notebook/_execute_python/execute_artifact)
- schedule/notebook_runner.py becomes a compatibility shim so
`python -m schedule.notebook_runner` (the worker's stable -m string) still works
- Delete flat execution.py (shadowed by the new execution/ package)
- Zero behavior change; schedule/pyproject.toml untouched
Co-Authored-By: Claude <noreply@anthropic.com>
Stage 3 of the layered refactor. Relocate the two scheduling components
into their own package so that domain / application / scheduling /
execution / infrastructure boundaries actually exist on disk.
- Add schedule/src/schedule/scheduling/__init__.py
- Move scheduler.py (232 lines) -> scheduling/scheduler.py
(byte-identical via diff; CronScheduler class name unchanged)
- Move orchestrator.py (946 lines) -> scheduling/orchestrator.py
(byte-identical via diff; DispatchOrchestrator + event constants
unchanged; NOT further split this round, per plan)
- service.py lines 39-40: import paths rewritten to the new module
- tests/test_janitor.py: rewrite the import + 5 patch() string targets
The 5 patch() targets ("schedule.orchestrator.session_scope" x3,
"schedule.orchestrator.asyncio.sleep" x2) were NOT caught by the
import-line grep — they patch module attributes at runtime and would
have become dead no-ops after the move (and would hard-raise once
the old module is deleted in stage 6). Rewriting them to
"schedule.scheduling.orchestrator.*" keeps the janitor tests meaningfully
exercising the new module.
- old flat scheduler.py / orchestrator.py left on disk; stage 6 deletes
them once all layers are extracted.
Validation:
- uv run --package schedule pytest schedule/tests -q: 18 passed
- uv run python -m compileall schedule/src: zero errors
- grep 'from schedule.(scheduler|orchestrator)\\b' (old paths): 0 matches
- grep '"schedule.orchestrator.' (old patch targets): 0 matches
- main.py / worker.py / domain/ / infrastructure/ / pyproject.toml
byte-identical to HEAD
Co-Authored-By: Claude <noreply@anthropic.com>
B1: `_mark_upload_failed_and_raise` now commits on a separate session
- Helper takes `request + upload_id`, opens a fresh session from
`request.app.state.session_factory` and commits there before raising.
- Closes the named-lock connection-pool leak Codex flagged: the old
"commit-on-the-same-session" implementation could return the
GET_LOCK connection to the pool before the enclosing
`finally: release_named_lock` ran, leaking `mp:<hash>` for up to
`pool_recycle` and re-opening the same-key upload race.
- Same helper now used by `create_server_object_payload`'s put-failure
branch — two failure paths have identical semantics.
B2: streaming copy for soft-delete + restore (`get_stream() + put()`)
- LOCAL backend: zero-copy (aiofiles stream write). OOM fixed.
- S3 backend: still OOMs on multi-GB objects — `put()` materializes
the async iter via `b"".join(chunks)`. Multipart `put` is a
follow-up; do NOT claim "OOM fixed on production" since production
defaults to S3.
C1: worker re-verifies `Users.status='active' AND is_deleted=0`
- `_assert_user_active` called from `_execution_context` after
resolving `triggered_by`; skips `SYSTEM_CRON_USER_ID`.
- `USER_DISABLED` error_code goes into the `NODE_FINISHED_EVENT`
outbox payload — `schedule_node_runs` has no `error_code` column,
the row only carries the `message` text. Docstrings corrected to
say so explicitly (previous docstring falsely promised row-level
observability).
F1: honest browser-local file lock
- `api.ts` `acquireFileLock/heartbeatFileLock/releaseFileLock/
releaseFileLockOnUnload` are now no-ops with comments stating they
never call the network.
- `scriptWorkspaceStore` dropped `tickHeartbeats`; `tickCleanup`
simplified to just clear cache.
- `useEditSessionLifecycle` dropped its 15s heartbeat `setInterval`.
- `ScriptWorkspace.tsx` renders `.local-lock-banner` info bar when
`isEditing`. Two tabs may still silently last-write — banner is the
only guard (acceptable disclosure-only tradeoff).
Dead code: deleted the duplicate `upload_bytes_to_session` in
`backend/src/backend/storage_api.py`. The `services.storage` import
is now the only source of the function; `create_upload_record`'s
docstring updated to point at `backend.resources`.
S2: schedule worker add janitor task that force-terminals node_runs
whose deadline (timeout_seconds + retry_count*retry_interval + 120s
slack from started_at) has passed. Closes the gap where outbox retry
exhaustion (5 tries, capped 30s backoff) marked the *event* failed
but left the *node_run* stuck in queued/running forever. Re-reads the
row under FOR UPDATE before writing so a worker that races us to a
real terminal state is not overwritten; idempotency key uses
:timed_out variant so the :finished path cannot collide.
R1: extract _reap_once() from _reap_loop for testability; in the
dead-process branch, re-verify (process.pid, started_at) against the
live JUPYTER_PROCESSES entry before del. A start_workspace that
replaced the dead record mid-cycle used to have its new entry
silently erased by the reaper's stale snapshot — leaked the port.
R2: delete _drop_workspace_lock and its two call sites
(stop_workspace tail, get_workspace 404 path). Popping the lock
object after release breaks mutual exclusion for any coroutine still
holding the old reference while a fresh caller gets a new lock
object — same ws_id can race two starts. The dict is bounded by the
number of workspaces so the leak is negligible; invariant lives on
WORKSPACE_LOCKS in a comment.
Tests:
- schedule/tests/test_janitor.py — 8 tests covering normal kill /
healthy-skip / worker-race / never-started / multi-row batch /
cancellation propagation / per-iteration self-heal
- runtime/tests/test_process.py — 7 tests covering reaper identity
match / replacement-skip / alive-preserved + lock
same-object / concurrent-serialize / survives-stop /
helper-removed guard
uv run --package schedule pytest schedule/tests → 14 passed
uv run --package runtime pytest runtime/tests → 7 passed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
