access_time 4 years ago languageEnglish
more_vert

Install Spark 2.2.1 in Windows

visibility 2,680 comment 0

This page summarizes the steps to install Spark 2.2.1 in your Windows environment.

Tools and Environment

  • GIT Bash
  • Command Prompt
  • Windows 10

Download Binary Package

Download the latest binary from the following site:

https://spark.apache.org/downloads.html

In my case, I am saving the file to folder: F:\DataAnalytics.

UnZip binary package

Open Git Bash, and change directory (cd) to the folder where you save the binary package and then unzip:

$ cd F:\DataAnalytics

fahao@Raymond-Alienware MINGW64 /f/DataAnalytics
$ tar -xvzf   spark-2.2.1-bin-hadoop2.7.tgz

In my case, spark is extracted to: F:\DataAnalytics\spark-2.2.1-bin-hadoop2.7

Setup environment variables

JAVA_HOME

Follow section ‘JAVA_HOME environment variable’ in the following page to setup JAVA_HOME

https://kontext.tech/docs/DataAndBusinessIntelligence/p/install-zeppelin-073-in-windows

SPARK_HOME

Setup SPARK_HOME environment variable with value of your spark installation directory.

image

PATH

Added ‘%SPARK_HOME%\bin’ to your path environment variable.

Verify the installation

Verify command

Run the following command in Command Prompt to verify the installation.

%SPARK_HOME%\bin\spark-shell

The screen should be similar to the following screenshot:

image

Run examples

Execute the following command in Command Prompt to run one example provided as part of Spark installation (class SparkPi with param 10).

https://spark.apache.org/docs/latest/

%SPARK_HOME%\bin\run-example.cmd SparkPi 10

The output looks like the following:
image

Spark context UI

As printed out, Spark context Web UI available at http://172.24.144.1:4040.

The following is a screenshot of the UI:

image

Spark developer tools

Refer to the following page if you are interested in any Spark developer tools.

https://spark.apache.org/developer-tools.html

info Last modified by Raymond 4 years ago copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Follow Kontext

Get our latest updates on LinkedIn.

Want to contribute on Kontext to help others?

Learn more

More from Kontext

visibility 201
thumb_up 0
access_time 7 months ago
Apache Spark 3.0.0 Installation on Linux Guide
visibility 3711
thumb_up 0
access_time 11 months ago