Spark SQL - Convert String to Timestamp

access_time 7 days ago visibility11 comment 0

Similar as Convert String to Date using Spark SQL, you can convert string of timestamp to Spark SQL timestamp data type.

Function to_timestamp

Function to_timestamp(timestamp_str[, fmt]) parses the `timestamp_str` expression with the `fmt` expression to a timestamp data type in Spark. 

Example:

spark-sql> select to_timestamp('2021-01-09 17:34:59');
to_timestamp('2021-01-09 17:34:59')
2021-01-09 17:34:59

In the above example, the string is in default format of timestamp, thus there is no need to pass in the format parameter.  If it is not in the default format, you need to pass in the format string.

Example:

spark-sql> select to_timestamp('2021010917:34:59','yyyyMMddHH:mm:ss');
to_timestamp('2021010917:34:59', 'yyyyMMddHH:mm:ss')
2021-01-09 17:34:59

All the datetime patterns are list in Datetime patterns.

copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Follow Kontext

Get our latest updates on LinkedIn or Twitter.

Want to publish your article on Kontext?

Learn more

More from Kontext

visibility 11
thumb_up 0
access_time 7 days ago

Similar as  Convert String to Date using Spark SQL , you can convert string of timestamp to Spark SQL timestamp data type. Function  to_timestamp(timestamp_str[, fmt]) p arses the `timestamp_str` expression with the `fmt` expression to a timestamp data type in Spark.  Example ...

visibility 10
thumb_up 0
access_time 9 days ago

RANK in Spark calculates the rank of a value in a group of values. It returns one plus the number of rows proceeding or equals to the current row in the ordering of a partition. The returned values are not sequential.   The following sample SQL uses RANK function without PARTITION BY ...

visibility 7
thumb_up 0
access_time 6 days ago

In Spark SQL, MapType is designed for key values, which is like dictionary object type in many other programming languages. This article summarize the commonly used map functions in Spark SQL. Function map is used to create a map.  Example: spark-sql> select ...

About column

Apache Spark installation guides, performance tuning tips, general tutorials, etc.

*Spark logo is a registered trademark of Apache Spark.