refactor: unify env-var config in common/config.py

Before: SPARK_EXECUTOR_DATA_DIR, SPARK_EXECUTOR_JOBS_DIR, and
YARN_RESOURCE_MANAGER_URL were each read directly via os.environ.get()
inside the module that used them. Log level was hardcoded DEBUG in
common/logging.py. There was no single file showing what the full set
of env vars the app reads is.

After: common/config.py defines a single Settings dataclass that
reads all env vars at import time and exposes them as fields on a
module-level singleton. App code uses "from common.config import
settings; settings.data_dir" etc. New SPARK_EXECUTOR_LOG_LEVEL env
var controls stderr + info file verbosity (debug file always gets full
DEBUG).

Improvements:
  - One file lists every env var the app reads (was: grep the codebase)
  - Tests can monkeypatch fields on the settings singleton directly
    instead of monkeypatching the env + reloading
  - Adding a new env var means adding one field in config.py, not
    editing 3+ call sites
  - settings.reload() method for tests that prefer env-var style

Out of scope (kept where they are):
  - GUNICORN_* env vars live in gunicorn.conf.py (gunicorn concept)
  - PYTHONUNBUFFERED in Dockerfile (Python runtime flag)
  - SPARK_SUBMIT_OPTS not in config (JVM flag, not Python)

Test changes:
  - test_job_writer.py: settings.jobs_dir instead of monkeypatching
    SPARK_EXECUTOR_JOBS_DIR
  - test_yarn_client.py: settings.yarn_resource_manager_url instead of
    monkeypatching YARN_RESOURCE_MANAGER_URL
  - test_generate_tool.py: same as job_writer
  - Each test file gets an autouse fixture that snapshots+restores
    settings so one test mutation does not leak into the next

116/116 still pass. Live verified: SPARK_EXECUTOR_LOG_LEVEL=INFO
suppresses DEBUG loguru output as expected.

common/config.py

@@ -0,0 +1,90 @@
+# coding=utf-8
+"""
+@Time   :2026/6/24
+@Author :tao.chen
+
+Single source of truth for environment-driven application configuration.
+
+Why this module exists: every prior `os.environ.get("SPARK_EXECUTOR_*")`
+was scattered across `connection_store.py`, `pending_store.py`,
+`yarn_client.py`, `job_writer.py`. The same env var name appeared in
+multiple files, and there was no single place to see "what's
+configurable?". This file is that place.
+
+All application code should:
+    from common.config import settings
+    ... settings.data_dir / settings.yarn_resource_manager_url / ...
+
+...rather than `os.environ.get(...)` directly. This way:
+  - The env var name appears in exactly one place.
+  - Ops can audit the full config surface from one file.
+  - Tests can monkeypatch fields on the `settings` singleton directly
+    (more reliable than monkeypatching env + reloading).
+
+NOT in this module:
+  - Gunicorn-specific env vars (workers, threads, bind, timeout) live in
+    gunicorn.conf.py because they configure gunicorn, not the app.
+  - `PYTHONUNBUFFERED` — Python runtime flag, set in Dockerfile.
+  - `SPARK_SUBMIT_OPTS` — JVM flags, not Python config; users pass them
+    to spark-submit directly.
+"""
+import os
+from dataclasses import dataclass
+
+
+@dataclass
+class Settings:
+    """Application settings loaded from environment variables at import time.
+
+    Mutable so tests can reassign fields directly. For tests that use
+    `monkeypatch.setenv(...)`, call `settings.reload()` to re-read from
+    the (now-patched) environment.
+    """
+
+    # --- Data persistence (./data/ in dev, /var/lib/... in prod) ---
+    data_dir: str = "./data"
+
+    # --- Job files (LLM-generated PySpark) ---
+    # Defaults to <data_dir>/jobs; can be pointed at a larger disk via
+    # SPARK_EXECUTOR_JOBS_DIR (e.g. /var/spark-jobs on a big-disk host).
+    jobs_dir: str = ""
+
+    # --- YARN REST client (fallback for Job.yarn_rm_url snapshot) ---
+    # Set in the env OR per-Connection via save_connection.
+    yarn_resource_manager_url: str | None = None
+
+    # --- Loguru ---
+    # Controls stderr verbosity and the info-level file sink. The debug
+    # file sink always captures DEBUG (full audit trail).
+    log_level: str = "DEBUG"
+
+    @classmethod
+    def from_env(cls) -> "Settings":
+        data_dir = os.environ.get("SPARK_EXECUTOR_DATA_DIR", "./data")
+        jobs_dir = os.environ.get(
+            "SPARK_EXECUTOR_JOBS_DIR", os.path.join(data_dir, "jobs")
+        )
+        return cls(
+            data_dir=data_dir,
+            jobs_dir=jobs_dir,
+            # `or None` collapses empty string to None for the URL fallback
+            yarn_resource_manager_url=os.environ.get("YARN_RESOURCE_MANAGER_URL") or None,
+            log_level=os.environ.get("SPARK_EXECUTOR_LOG_LEVEL", "DEBUG"),
+        )
+
+    def reload(self) -> "Settings":
+        """Re-read from environment, mutate in place, return self (chainable).
+
+        Useful in tests that `monkeypatch.setenv(...)` and want the
+        singleton to pick up the new value without a re-import.
+        """
+        fresh = self.from_env()
+        self.data_dir = fresh.data_dir
+        self.jobs_dir = fresh.jobs_dir
+        self.yarn_resource_manager_url = fresh.yarn_resource_manager_url
+        self.log_level = fresh.log_level
+        return self
+
+
+# Module-level singleton. Loaded once at import time.
+settings = Settings.from_env()