From 33294af47703ea9a3348985174bd3dcd6db2fedf Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 24 Jun 2026 17:37:44 +0800 Subject: [PATCH] fix(Dockerfile): install openjdk-17-jre-headless for spark-submit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spark 4.1.2 requires Java 17 at runtime (it boots a JVM via the bin/spark-submit shell script). The previous Dockerfile only installed curl, ca-certificates, and tar — leaving spark-submit unable to find the 'java' binary. A JRE is sufficient (not the full JDK) because spark-submit does not compile Java at runtime — it loads pre-built JARs from $SPARK_HOME/jars/. JRE-only keeps the image ~160 MB smaller than JDK-headless. Sets JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 and prepends it to PATH so spark-submit (which checks JAVA_HOME) finds the right JVM. Combined with the prior yarn-REST rewrite, the runtime image now needs exactly three things in addition to python:3.12-slim: - uv (from ghcr.io/astral-sh/uv) - openjdk-17-jre-headless - Spark 4.1.2 from Tsinghua mirror --- Dockerfile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Dockerfile b/Dockerfile index efd3aba..95effa4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -18,7 +18,8 @@ RUN apt-get update && \ apt-get install -y --no-install-recommends \ curl \ ca-certificates \ - tar && \ + tar \ + openjdk-17-jre-headless && \ curl -L \ https://mirrors.tuna.tsinghua.edu.cn/apache/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-hadoop3.tgz \ -o /tmp/spark.tgz && \ @@ -29,8 +30,9 @@ RUN apt-get update && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* +ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 ENV SPARK_HOME=/opt/spark -ENV PATH=${SPARK_HOME}/bin:${PATH} +ENV PATH=${JAVA_HOME}/bin:${SPARK_HOME}/bin:${PATH} # Hadoop/Yarn 配置目录(运行时挂载) RUN mkdir -p /etc/hadoop/conf