tao.chen/mcp-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: env-configurable loguru file paths

Browse Source 
common/logging.py used hardcoded 'data/logs/{debug,info}/' paths,
which forced logs into the same dir as connections.json. Operators
couldn't put logs on a dedicated volume or a different disk for
retention policy reasons.

Now the base log dir comes from settings.log_dir (env var
SPARK_EXECUTOR_LOG_DIR, default <data_dir>/logs). Subdirs debug/ and
info/ are auto-created. Same behavior in dev (./data/logs/), but in
prod you can do:

  SPARK_EXECUTOR_LOG_DIR=/var/log/spark-executor   # dedicated volume
  SPARK_EXECUTOR_LOG_DIR=/mnt/slow-storage/logs   # cold storage for old logs

Other changes:
  - .env.example: new SPARK_EXECUTOR_LOG_DIR entry, split into its own
    'Loguru file sinks' section
  - docker-compose.yml: forwards SPARK_EXECUTOR_LOG_DIR with the
    container-side default /app/data/logs (the existing ./data:/app/data
    volume mount already covers this)
  - common/config.py: Settings gets a new log_dir field, defaults
    derived from data_dir; reload() also resets it

116/116 still pass. Live smoke verified: with
SPARK_EXECUTOR_LOG_DIR=/tmp/spark-logs, both
  /tmp/spark-logs/info/2026-06-25.log
  /tmp/spark-logs/debug/2026-06-25.log
are created on the first request.
This commit is contained in:
Claude
2026-06-25 10:57:50 +08:00
parent 52624aacff
commit 13e39b39f3
4 changed files with 36 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/logging.py
+8 -4
View File
@@ -22,8 +22,12 @@ from loguru import logger
from common.config import settings
Path("data/logs/debug").mkdir(parents=True, exist_ok=True)
Path("data/logs/info").mkdir(parents=True, exist_ok=True)
# Loguru file sinks — paths derived from settings.log_dir (env-var driven
# via SPARK_EXECUTOR_LOG_DIR, default <data_dir>/logs).
DEBUG_LOG_DIR = Path(settings.log_dir) / "debug"
INFO_LOG_DIR = Path(settings.log_dir) / "info"
DEBUG_LOG_DIR.mkdir(parents=True, exist_ok=True)
INFO_LOG_DIR.mkdir(parents=True, exist_ok=True)
logger.remove()
logger.add(
@@ -40,7 +44,7 @@ logger.add(
)
logger.add(
"data/logs/debug/{time:YYYY-MM-DD}.log",
str(DEBUG_LOG_DIR / "{time:YYYY-MM-DD}.log"),
level="DEBUG", # always full at the file level for audit
enqueue=True,
retention="30 days",
@@ -49,7 +53,7 @@ logger.add(
)
logger.add(
"data/logs/info/{time:YYYY-MM-DD}.log",
str(INFO_LOG_DIR / "{time:YYYY-MM-DD}.log"),
level=settings.log_level,
enqueue=True,
retention="30 days",