单机Docker运行spark

Docker 安装spark

        方便进行简单的验证测试。

Dockerfile

FROM centos:7

MAINTAINER  macintosh

ADD jdk-8u261-linux-x64.tar.gz /usr/local/

ADD scala-2.12.12.tgz /usr/local/

ADD spark-3.0.0-bin-hadoop3.2.tgz /usr/local/

ENV JAVA_HOME /usr/local/jdk1.8.0_261

ENV CLASSPATH $JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

ENV SPARK_HOME /usr/local/spark-3.0.0-bin-hadoop3.2

ENV SCALA_HOME /usr/local/scala-scala-2.12.12

ENV PATH=$JAVA_HOME/bin:$SPARK_HOME/bin:$SCALA_HOME/bin:$PATH

jdk scala spark 可以切换相应的版本

docker build -t spark:v1 .

docker run spark:v1 /bin/bash