By using this site, you acknowledge that you have read and understand our Cookie policy, Privacy policy and Terms .

Context

This pages demonstrates how to convert string to java.util.Date in Spark via Scala.

Prerequisites

If you have not installed Spark, follow the page below to install it:

Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice

Sample code

The following code snippet uses pattern yyyy-MM-dd to parse string to Date.

import java.text.SimpleDateFormat

import java.util.Date

val format = new SimpleDateFormat("yyyy-MM-dd")

val date = format.parse("2018-03-03")

Output

image

Retrieve week day name

scala> val format2 = new SimpleDateFormat("EEEEE")
format2: java.text.SimpleDateFormat = java.text.SimpleDateFormat@3ecbf05

scala> format2.format(date)
res0: String = Saturday

Summary

For the complete list of Java date and time pattern, please refer to the following link.

https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

info Last modified by Raymond at 3 years ago * This page is subject to Site terms.

More from Kontext

local_offer scala local_offer spark-2-x

visibility 648
thumb_up 0
access_time 5 months ago

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

open_in_new View open_in_new Code snippets

local_offer scala local_offer spark-2-x

visibility 258
thumb_up 0
access_time 5 months ago

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

open_in_new View open_in_new Code snippets

local_offer lite-log local_offer spark local_offer pyspark

visibility 2305
thumb_up 0
access_time 10 months ago

When creating Spark date frame using schemas, you may encounter errors about “field **: **Type can not accept object ** in type <class '*'>”. The actual error can vary, for instances, the following are some examples: field xxx: BooleanType can not accept object 100 in type ...

open_in_new View open_in_new Spark + PySpark

local_offer spark local_offer scala

visibility 67
thumb_up 0
access_time 5 months ago

Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files. 

open_in_new View open_in_new Code snippets

info About author

Dark theme mode

Dark theme mode is available on Kontext.

Learn more arrow_forward
Kontext Column

Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.

Learn more arrow_forward
info Follow us on Twitter to get the latest article updates. Follow us