The drop-sync refactor is now contractually enforced:
* create_storage({"mode": "sync"|"async"|<any>}) raises StorageConfigError
-- the regression we most want to catch is someone "restoring" the
sync shape by re-introducing mode= handling.
* create_storage() preserves caller's dict, returns an AsyncStorageBackend
subclass, and wraps constructor TypeError into a useful StorageConfigError.
* build_storage_config emits no mode key on either local or s3 branch.
* register_backend() refuses to silently overwrite a different class on
the same name (collision guard), and accepts same-class re-registration.
* registry surfaces the built-in local + s3 classes after import; the
conflict-test cleanup pattern avoids leaking global state across tests.
Adds [tool.pytest.ini_options] (asyncio_mode=auto, testpaths=tests) so
uv run --package common pytest common/tests works from the workspace root.
17 new tests, all green. Backend suite still 136 passed.
The async-only direction was already the only one used in production:
* create_storage never accepted mode=sync; build_storage_config always
emitted mode=async; zero callers referenced StorageBackend / SyncData
/ S3StorageBackend.sync / LocalStorageBackend.sync anywhere.
* Drop the parallel sync base class, the sync concrete classes in
backends/local.py and backends/s3.py, and the boto3 dependency.
* Drop example_usage.py (zero importers; demonstration code, not part
of the public surface).
* Rename LocalAsyncStorageBackend -> LocalStorageBackend,
S3AsyncStorageBackend -> S3StorageBackend to reflect the single
remaining class per type.
* Tighten create_storage: any mode=... key now raises StorageConfigError
with the new pointer (settings.storage_backend controls behavior).
* Cleanup call sites: schedule.application.service.build_object_store
no longer passes mode=async to create_storage.
* Cosmetic touch-ups in backend/services/storage.py and
common/config.py docstrings where they still said "boto3" instead of
"S3 client".
Public API surface preserved: AsyncStorageBackend / ObjectMeta /
create_storage / build_storage_config / register_backend all keep
their names and call signatures. backend tests: 136 passed.
Review (2026-08-21) found that stage 1 promoted ExecutionResult from a
plain class to @dataclass(frozen=True) along the way. No caller mutates
or compares these objects by identity, so the only externally visible
change is structured log output. User opted to keep the upgrade.
- domain/execution.py module docstring: explicit note that the frozen +
value-equality form is a deliberate enhancement, not a behavioral
accident
- CLAUDE.md "Schedule service layering" lesson: add a "don't silently
upgrade dataclass-ness during a structural-only refactor" note so
future refactors copy class definitions verbatim unless they intend
to tighten semantics explicitly
No code change; tests still 29 green.
Co-Authored-By: Claude <noreply@anthropic.com>
Follow-up to the layered refactor (review-driven):
- Move scheduling/orchestrator.py -> application/orchestrator.py
(orchestrator is application-level coordination, not a cron-trigger
primitive; matches the intended target tree)
- Migrate orchestrator re-exports from scheduling/__init__.py to
application/__init__.py; scheduling/ now exposes only CronScheduler
- Rewrite imports + 5 mock.patch string targets in test_janitor.py and
the orchestrator import in test_layering.py
- Update docstring refs in application/service.py + execution/worker.py
- Add 4 runner smoke tests (test_layering.py): _limited_log under-limit /
empty-sentinel / above-MAX_LOG_BYTES truncation; execute_artifact
rejects unsupported script_type with ValueError
- infrastructure/__init__.py re-exports SchedulerStorageClient so
``from schedule.infrastructure import SchedulerStorageClient`` is a
stable top-level surface
- CLAUDE.md engineering note: extend the commit trail to 7476c27 and
note the stage-8 orchestrator placement
Zero behavior change; schedule/pyproject.toml untouched. 29 tests green.
Co-Authored-By: Claude <noreply@anthropic.com>
Pin the new five-layer contract so a later refactor can't silently break
an import surface or lifecycle:
- domain: ExecutionResult defaults, terminal/failed state-set invariants,
naive_utc normalization (import-side-effect-free)
- infrastructure.storage: SchedulerStorageClient base64 upload via
httpx.MockTransport (no live server; base_url required for relative URL)
- scheduling: CronScheduler start/close lifecycle, global trigger cleared
- application: SchedulerService wires cron + orchestrator + worker, handler
dispatch table points at the wired NodeExecutor
- execution: schedule.notebook_runner shim re-exports the real main
25 schedule tests green; schedule/pyproject.toml untouched.
Co-Authored-By: Claude <noreply@anthropic.com>
- Move service.py -> application/service.py (SchedulerService class name
unchanged; build_object_store / build_storage_http_client move along)
- main.py imports schedule.application.service
- Rewrite worker.py's lazy `from schedule.service import build_object_store`
and test_worker.py's mock patch string targets — same class of bug as the
test_janitor patch strings (silent no-op until the old file is deleted)
- Delete flat service.py (orphaned; only docstring refs remain in
orchestrator, cleaned up in stage 6)
- Zero behavior change; schedule/pyproject.toml untouched
Co-Authored-By: Claude <noreply@anthropic.com>
- Move worker.py -> execution/worker.py, executor.py -> execution/executor.py
(byte-identical copies; import sites updated)
- Merge old execution.py + notebook_runner.py into
execution/runners/notebook.py: subprocess CLI (main/emit_outputs) plus the
in-process helpers (_execute_notebook/_execute_python/execute_artifact)
- schedule/notebook_runner.py becomes a compatibility shim so
`python -m schedule.notebook_runner` (the worker's stable -m string) still works
- Delete flat execution.py (shadowed by the new execution/ package)
- Zero behavior change; schedule/pyproject.toml untouched
Co-Authored-By: Claude <noreply@anthropic.com>
Stage 3 of the layered refactor. Relocate the two scheduling components
into their own package so that domain / application / scheduling /
execution / infrastructure boundaries actually exist on disk.
- Add schedule/src/schedule/scheduling/__init__.py
- Move scheduler.py (232 lines) -> scheduling/scheduler.py
(byte-identical via diff; CronScheduler class name unchanged)
- Move orchestrator.py (946 lines) -> scheduling/orchestrator.py
(byte-identical via diff; DispatchOrchestrator + event constants
unchanged; NOT further split this round, per plan)
- service.py lines 39-40: import paths rewritten to the new module
- tests/test_janitor.py: rewrite the import + 5 patch() string targets
The 5 patch() targets ("schedule.orchestrator.session_scope" x3,
"schedule.orchestrator.asyncio.sleep" x2) were NOT caught by the
import-line grep — they patch module attributes at runtime and would
have become dead no-ops after the move (and would hard-raise once
the old module is deleted in stage 6). Rewriting them to
"schedule.scheduling.orchestrator.*" keeps the janitor tests meaningfully
exercising the new module.
- old flat scheduler.py / orchestrator.py left on disk; stage 6 deletes
them once all layers are extracted.
Validation:
- uv run --package schedule pytest schedule/tests -q: 18 passed
- uv run python -m compileall schedule/src: zero errors
- grep 'from schedule.(scheduler|orchestrator)\\b' (old paths): 0 matches
- grep '"schedule.orchestrator.' (old patch targets): 0 matches
- main.py / worker.py / domain/ / infrastructure/ / pyproject.toml
byte-identical to HEAD
Co-Authored-By: Claude <noreply@anthropic.com>
Stage 2 of the layered refactor. Move the storage HTTP client one
package deeper so that infrastructure code lives under a dedicated
namespace.
- Add schedule/src/schedule/infrastructure/__init__.py
- Add schedule/src/schedule/infrastructure/storage/__init__.py
- Add schedule/src/schedule/infrastructure/storage/client.py
(verbatim copy of old schedule/src/schedule/storage_client.py,
byte-identical via diff — 2682 bytes)
- main.py line 17: import path rewrite to the new module
(only consumer — service.py and worker.py take storage_client as
an `Any` constructor param and never imported the class)
- old schedule/src/schedule/storage_client.py left on disk; stage 6
deletes it once all layers are extracted.
Validation:
- uv run --package schedule pytest schedule/tests -q: 18 passed
- uv run python -m compileall schedule/src: zero errors
- grep 'from schedule.storage_client' (old path): 0 matches
- service.py and worker.py byte-identical to HEAD
- main.py / pyproject.toml / tests/ unchanged apart from the 1 import line
Co-Authored-By: Claude <noreply@anthropic.com>
Add the three files that complete stage 1 of the layered refactor:
- schedule/src/schedule/domain/__init__.py (empty package marker)
- schedule/src/schedule/domain/context.py
(TERMINAL_NODE_STATES / FAILED_NODE_STATES / TERMINAL_RUN_STATES / naive_utc —
pure types, no I/O)
- schedule/src/schedule/domain/execution.py
(ExecutionResult dataclass, frozen=True)
The corresponding import-path rewrites in worker.py / orchestrator.py /
scheduler.py / execution.py were already landed in cdcfcb2 (the prior
commit on this branch). This commit only adds the missing domain/
package files those imports point at.
Validation:
- uv run --package schedule pytest schedule/tests -q: 18 passed
- uv run python -m compileall schedule/src: zero errors
- from schedule.domain.context / schedule.domain.execution importable
- main.py / pyproject.toml / tests/ unchanged
Co-Authored-By: Claude <noreply@anthropic.com>
tao.chen