Install Spark 2.2.1 in Windows
This page summarizes the steps to install Spark 2.2.1 in your Windows environment.
Tools and Environment
- GIT Bash
- Command Prompt
- Windows 10
Download Binary Package
Download the latest binary from the following site:
In my case, I am saving the file to folder: F:\DataAnalytics.
UnZip binary package
Open Git Bash, and change directory (cd) to the folder where you save the binary package and then unzip:
$ cd F:\DataAnalytics
fahao@Raymond-Alienware MINGW64 /f/DataAnalytics
$ tar -xvzf spark-2.2.1-bin-hadoop2.7.tgz
In my case, spark is extracted to: F:\DataAnalytics\spark-2.2.1-bin-hadoop2.7
Setup environment variables
JAVA_HOME
Follow section ‘JAVA_HOME environment variable’ in the following page to setup JAVA_HOME
https://kontext.tech/docs/DataAndBusinessIntelligence/p/install-zeppelin-073-in-windows
SPARK_HOME
Setup SPARK_HOME environment variable with value of your spark installation directory.
PATH
Added ‘%SPARK_HOME%\bin’ to your path environment variable.
Verify the installation
Verify command
Run the following command in Command Prompt to verify the installation.
%SPARK_HOME%\bin\spark-shell
The screen should be similar to the following screenshot:
Run examples
Execute the following command in Command Prompt to run one example provided as part of Spark installation (class SparkPi with param 10).
https://spark.apache.org/docs/latest/
%SPARK_HOME%\bin\run-example.cmd SparkPi 10
The output looks like the following:
Spark context UI
As printed out, Spark context Web UI available at http://172.24.144.1:4040.
The following is a screenshot of the UI:
Spark developer tools
Refer to the following page if you are interested in any Spark developer tools.
https://spark.apache.org/developer-tools.html