refactor: unify env-var config in common/config.py

Before: SPARK_EXECUTOR_DATA_DIR, SPARK_EXECUTOR_JOBS_DIR, and
YARN_RESOURCE_MANAGER_URL were each read directly via os.environ.get()
inside the module that used them. Log level was hardcoded DEBUG in
common/logging.py. There was no single file showing what the full set
of env vars the app reads is.

After: common/config.py defines a single Settings dataclass that
reads all env vars at import time and exposes them as fields on a
module-level singleton. App code uses "from common.config import
settings; settings.data_dir" etc. New SPARK_EXECUTOR_LOG_LEVEL env
var controls stderr + info file verbosity (debug file always gets full
DEBUG).

Improvements:
  - One file lists every env var the app reads (was: grep the codebase)
  - Tests can monkeypatch fields on the settings singleton directly
    instead of monkeypatching the env + reloading
  - Adding a new env var means adding one field in config.py, not
    editing 3+ call sites
  - settings.reload() method for tests that prefer env-var style

Out of scope (kept where they are):
  - GUNICORN_* env vars live in gunicorn.conf.py (gunicorn concept)
  - PYTHONUNBUFFERED in Dockerfile (Python runtime flag)
  - SPARK_SUBMIT_OPTS not in config (JVM flag, not Python)

Test changes:
  - test_job_writer.py: settings.jobs_dir instead of monkeypatching
    SPARK_EXECUTOR_JOBS_DIR
  - test_yarn_client.py: settings.yarn_resource_manager_url instead of
    monkeypatching YARN_RESOURCE_MANAGER_URL
  - test_generate_tool.py: same as job_writer
  - Each test file gets an autouse fixture that snapshots+restores
    settings so one test mutation does not leak into the next

116/116 still pass. Live verified: SPARK_EXECUTOR_LOG_LEVEL=INFO
suppresses DEBUG loguru output as expected.

tests/unit/test_yarn_client.py

@@ -4,6 +4,7 @@ from unittest.mock import patch
 import httpx
 import pytest
 
+from common import config
 from spark_executor.core.yarn_client import (
     YarnConfigError,
     YarnError,
@@ -16,6 +17,23 @@ from spark_executor.core.yarn_client import (
 RM = "http://rm:8088"
 
 
+@pytest.fixture(autouse=True)
+def _restore_settings():
+    """Each test gets a clean copy of `settings` so env-var-style overrides
+    in one test don't leak into the next."""
+    snapshot = config.Settings(
+        data_dir=config.settings.data_dir,
+        jobs_dir=config.settings.jobs_dir,
+        yarn_resource_manager_url=config.settings.yarn_resource_manager_url,
+        log_level=config.settings.log_level,
+    )
+    yield
+    config.settings.data_dir = snapshot.data_dir
+    config.settings.jobs_dir = snapshot.jobs_dir
+    config.settings.yarn_resource_manager_url = snapshot.yarn_resource_manager_url
+    config.settings.log_level = snapshot.log_level
+
+
 def _resp(status: int, *, json_data=None, text: str | None = None) -> httpx.Response:
     if json_data is not None:
         return httpx.Response(status, json=json_data)
@@ -57,16 +75,16 @@ def test_status_raises_when_state_field_missing():
 
 
 def test_status_requires_rm_url():
-    with patch.dict("os.environ", {}, clear=True):
-        with pytest.raises(YarnConfigError):
-            get_application_status("application_1", None)
+    config.settings.yarn_resource_manager_url = None
+    with pytest.raises(YarnConfigError):
+        get_application_status("application_1", None)
 
 
-def test_status_falls_back_to_env_var():
+def test_status_falls_back_to_settings_yarn_rm_url():
+    config.settings.yarn_resource_manager_url = "http://env-rm:8088"
     fake = _resp(200, json_data={"app": {"state": "FINISHED"}})
-    with patch.dict("os.environ", {"YARN_RESOURCE_MANAGER_URL": "http://env-rm:8088"}, clear=False):
-        with patch("spark_executor.core.yarn_client.httpx.request", return_value=fake) as m:
-            state, _ = get_application_status("application_1", None)
+    with patch("spark_executor.core.yarn_client.httpx.request", return_value=fake) as m:
+        state, _ = get_application_status("application_1", None)
     assert state == "FINISHED"
     assert "env-rm:8088" in m.call_args.args[1]