Convert string to date in Python / Spark

visibility 2,820 access_time 3 years ago languageEnglish

This code snippet shows how to convert string to date.

In PySpark/Python, we can use unix_timestamp and from_unixtime functions.

Code snippet

from pyspark.sql import SparkSession
from pyspark.sql.functions import unix_timestamp, from_unixtime

appName = "PySpark Date Parse Example"
master = "local"

# Create Spark session with Hive supported.
spark = SparkSession.builder \
    .appName(appName) \
    .master(master) \
    .getOrCreate()

df = spark.createDataFrame([('2019-06-01',)], ['DATE_STR_COL'])
df.select(from_unixtime(unix_timestamp(df.DATE_STR_COL, 'yyyy-MM-dd')).alias('DATE_COL'))
df.show()
info Last modified by Raymond 3 years ago copyright This page is subject to Site terms.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

timeline Stats
Page index 2.60
More from Kontext
Debug PySpark Code in Visual Studio Code
visibility 15,318
thumb_up 1
access_time 3 years ago
pre-commit
visibility 68
thumb_up 1
access_time 4 months ago