Spark SQL - UNIX timestamp functions

access_time 7 days ago visibility8 comment 0

Current UNIX timestamp

Function unix_timestamp() returns the UNIX timestamp of current time. You can also specify a input timestamp value. 

Example:

spark-sql> select unix_timestamp();
unix_timestamp(current_timestamp(), yyyy-MM-dd HH:mm:ss)
1610174099

spark-sql> select unix_timestamp(current_timestamp, 'yyyy-MM-dd');
unix_timestamp(current_timestamp(), yyyy-MM-dd)
1610174365

UNIX timestamp string to date

Function from_unixtime(unix_time, format) can be used to convert UNIX time to Spark SQL date data type. 

Example:

spark-sql> select from_unixtime(1610174365, 'yyyy-MM-dd');
from_unixtime(CAST(1610174365 AS BIGINT), yyyy-MM-dd)
2021-01-09

UNIX timestamp string to timestamp

Function from_unixtime(unix_time, format) can also be used to convert UNIX time to Spark SQL timestamp data type. 

Example:

spark-sql> select from_unixtime(1610174099, 'yyyy-MM-dd HH:mm:ss');
from_unixtime(CAST(1610174099 AS BIGINT), yyyy-MM-dd HH:mm:ss)
2021-01-09 17:34:59
copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Follow Kontext

Get our latest updates on LinkedIn or Twitter.

Want to publish your article on Kontext?

Learn more

More from Kontext

visibility 18
thumb_up 0
access_time 19 days ago

In Spark-SQL CLI tool, the result print will omit headings (column names) by default. To display columns, we need to update Spark setting spark.hadoop.hive.cli.print.header. To make the changes for all spark-sql sessions, edit file $SPARK_HOME/conf/spark-defaults.conf . Add the following ...

visibility 9
thumb_up 0
access_time 9 days ago

DENSE_RANK is similar as  Spark SQL - RANK Window Function . It  calculates the rank of a value in a group of values. It returns one plus the number of rows proceeding or equals to the current row in the ordering of a partition. The returned values are sequential in each window thus no ...

visibility 6
thumb_up 0
access_time 7 days ago

Function current_date() or current_date can be used to return the current date at the start of query evaluation.  Example: spark-sql> select current_date(); current_date() 2021-01-09 spark-sql> select current_date; current_date() 2021-01-09 *Brackets are optional for this ...