spark_executor/core/job_writer.py provides write_job_file(code, jobs_dir=None)
that resolves the output directory in this order:
1. explicit jobs_dir argument (for tests + programmatic override)
2. SPARK_EXECUTOR_JOBS_DIR environment variable (operator override)
3. DEFAULT_JOBS_DIR = './data/jobs' (gitignored, persists via volume mount)
Filenames are job_<UTC-stamp>_<random-hex>.py — same-second writes still
get distinct names via the random suffix, so concurrent agents won't
clobber each other.
8 tests cover: arg>env>default precedence, dir auto-creation, env-var
override, default fallback in a tmp cwd, and uniqueness under concurrent
calls.
Claude
9340061b30