perf: decouple notebook execution and tune pools

schedule:
- New _execution_loop runs alongside _database_event_loop. It claims
  job.node.execute rows, sets a 30-min lease on available_at, then
  dispatches each as asyncio.create_task under a Semaphore(N).
  Polling loop is back to sub-millisecond turnaround for
  schedule.run.requested and job.node.finished. Long notebook
  execution no longer blocks DAG advance events.
- _process_pending_events filters by event_type IN
  ('schedule.run.requested', 'job.node.finished'); the executor
  loop owns job.node.execute exclusively.
- _process_outbox_event builds a plain dict envelope before
  handler dispatch; the previous ORM-row handoff risked
  DetachedInstanceError once the outer session closed.
- _sync_once uses get_job + reschedule_job for existing job ids
  instead of add_job(replace_existing=True). Each cron schedule
  no longer removed-and-readded every 5s.
- service.py threads settings.schedule_execution_concurrency into
  the orchestrator (default 4).

common:
- create_async_engine gets explicit pool_size=10, max_overflow=20,
  pool_recycle=1800. No more relying on SQLAlchemy defaults.
- New schedule_execution_concurrency setting.

runtime:
- scan_workspaces: add missing 'import os' (NameError on startup)
  and switch to asyncio.gather bounded by Semaphore(4) so N
  workspaces start in parallel instead of sequentially.

Co-Authored-By: Claude <noreply@anthropic.com>

common/src/common/config.py

@@ -112,6 +112,16 @@ class Settings(BaseSettings):
         description="Public base URL for the runtime container.",
     )
 
+    # ── schedule execution tuning ────────────────────────────────
+    schedule_execution_concurrency: int = Field(
+        default=4,
+        description=(
+            "Max concurrent notebooks running in the schedule worker. "
+            "Each notebook is dispatched as an asyncio task bounded by "
+            "a semaphore; the polling loop is never blocked."
+        ),
+    )
+
     # ── readiness probes ──────────────────────────────────────────
     readiness_targets: str = Field(
         default="",

common/src/common/db/session.py

@@ -18,12 +18,18 @@ def create_database_engine(
     *,
     echo: bool = False,
     pool_pre_ping: bool = True,
+    pool_size: int = 10,
+    max_overflow: int = 20,
+    pool_recycle: int = 1800,
 ) -> AsyncEngine:
     """Create an async SQLAlchemy engine without storing global connection state."""
     return create_async_engine(
         database_url,
         echo=echo,
         pool_pre_ping=pool_pre_ping,
+        pool_size=pool_size,
+        max_overflow=max_overflow,
+        pool_recycle=pool_recycle,
     )