PySpark - Fix PermissionError: [WinError 5] Access is denied

access_time 2 years ago visibility4546 comment 0

When running pyspark or spark-submit command in Windows to execute python scripts, you may encounter the following error:

PermissionError: [WinError 5] Access is denied

As it’s self-explained, permissions are not setup correctly.

To resolve this issue you can try different approaches:

  • Run the program as an Administrator
  • Or configure the security for your PySpark executable properly.

pyspark command exists in your ‘bin’ subfolder under Spark home folder. Make sure the permissions are configured correctly for the user that you are using to execute the script.

image

In my system, the error occurred because my current user has no permission to the Spark folder and its subfolders.

info Last modified by Raymond at 2 years ago copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Want to publish your article on Kontext?

Learn more

Kontext Column

Created for everyone to publish data, programming and cloud related articles.
Follow three steps to create your columns.


Learn more arrow_forward

More from Kontext

local_offer python local_offer spark

visibility 993
thumb_up 0
access_time 12 months ago

This code snippet shows how to convert string to date.

local_offer python local_offer spark local_offer pyspark local_offer spark-advanced

visibility 9274
thumb_up 0
access_time 2 years ago

For SQL developers that are familiar with SCD and merge statements, you may wonder how to implement the same in big data platforms, considering database or storages in Hadoop are not designed/optimised for record level updates and inserts. In this post, I’m going to demonstrate how to implement ...

local_offer python local_offer python-file-operations

visibility 1112
thumb_up 0
access_time 7 months ago

CSV is a common data format used in many applications. It's also a common task for data workers to read and parse CSV and then save it into another storage such as RDBMS (Teradata, SQL Server, MySQL). In my previous article  PySpark Read Multiple Lines Records from CSV I demonstrated how to ...

About column

Apache Spark installation guides, performance tuning tips, general tutorials, etc.

*Spark logo is a registered trademark of Apache Spark.

rss_feed Subscribe RSS