This pages demonstrates how to convert string to java.util.Date in Spark via Scala.
If you have not installed Spark, follow the page below to install it:
The following code snippet uses pattern yyyy-MM-dd to parse string to Date.
val format = new SimpleDateFormat("yyyy-MM-dd")
val date = format.parse("2018-03-03")
scala> val format2 = new SimpleDateFormat("EEEEE")
format2: java.text.SimpleDateFormat = java.text.SimpleDateFormat@3ecbf05
res0: String = Saturday
For the complete list of Java date and time pattern, please refer to the following link.
This post shows how to derive new column in a Spark data frame from a JSON array string column. I am running the code in Spark 2.2.1 though it is compatible with Spark 1.6.0 (with less JSON SQL functions). Prerequisites Refer to the following post to install Spark in Windows. ...View detail
In this page, I’m going to demonstrate how to write and read parquet files in Spark/Scala by using Spark SQLContext class. Reference What is parquet format? Go the following project site to understand more about parquet. ...View detail
Are you a Windows/.NET developer and willing to learn big data concepts and tools in your Windows? If yes, you can follow the links below to install them in your PC. The installations are usually easier to do in Linux/UNIX but they are not difficult to implement in Windows either since the...View detail
This page shows how to import data from SQL Server into Hadoop via Apache Sqoop. Prerequisites Please follow the link below to install Sqoop in your machine if you don’t have one environment ready. ...View detail
In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page...View detail
Background This page provides an example to load text file from HDFS through SparkContext in Zeppelin (sc). Reference The details about this method can be found at: SparkContext.textFile ...View detail