languageEnglish

Convert string to date in Python / Spark

visibility 2,596 comment 0 access_time 3 years ago

This code snippet shows how to convert string to date.

In PySpark/Python, we can use unix_timestamp and from_unixtime functions.

Code snippet

from pyspark.sql import SparkSession
from pyspark.sql.functions import unix_timestamp, from_unixtime

appName = "PySpark Date Parse Example"
master = "local"

# Create Spark session with Hive supported.
spark = SparkSession.builder \
    .appName(appName) \
    .master(master) \
    .getOrCreate()

df = spark.createDataFrame([('2019-06-01',)], ['DATE_STR_COL'])
df.select(from_unixtime(unix_timestamp(df.DATE_STR_COL, 'yyyy-MM-dd')).alias('DATE_COL'))
df.show()
info Last modified by Raymond 3 years ago copyright This page is subject to Site terms.

Subscribe newsletter

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

More from Kontext

visibility 2784
thumb_up 0
access_time 3 years ago
visibility 6671
thumb_up 0
access_time 9 months ago
Apache Spark 3.0.1 Installation on Linux or WSL Guide
visibility 1627
thumb_up 0
access_time 9 months ago