access_time 3 years ago visibility8785 comment 0 languageEnglish
more_vert

Context

This pages demonstrates how to convert string to java.util.Date in Spark via Scala.

Prerequisites

If you have not installed Spark, follow the page below to install it:

Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice

Sample code

The following code snippet uses pattern yyyy-MM-dd to parse string to Date.

import java.text.SimpleDateFormat

import java.util.Date

val format = new SimpleDateFormat("yyyy-MM-dd")

val date = format.parse("2018-03-03")

Output

image

Retrieve week day name

scala> val format2 = new SimpleDateFormat("EEEEE")
format2: java.text.SimpleDateFormat = java.text.SimpleDateFormat@3ecbf05

scala> format2.format(date)
res0: String = Saturday

Summary

For the complete list of Java date and time pattern, please refer to the following link.

https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

info Last modified by Raymond at 3 years ago * This page is subject to Site terms.

More from Kontext

local_offer scala local_offer spark-2-x

visibility 1710
thumb_up 1
access_time 7 months ago

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

open_in_new View open_in_new Code snippets

local_offer scala local_offer spark-2-x

visibility 475
thumb_up 0
access_time 7 months ago

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

open_in_new View open_in_new Code snippets

local_offer spark local_offer pyspark

visibility 3378
thumb_up 0
access_time 11 months ago

When creating Spark date frame using schemas, you may encounter errors about “field **: **Type can not accept object ** in type <class '*'>”. The actual error can vary, for instances, the following are some examples: field xxx: BooleanType can not accept object 100 in type ...

open_in_new View open_in_new Spark + PySpark

local_offer spark local_offer scala

visibility 130
thumb_up 0
access_time 7 months ago

Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files. 

open_in_new View open_in_new Code snippets

info About author

Dark theme mode

Dark theme mode is available on Kontext.

Learn more arrow_forward

Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.


Learn more arrow_forward