local_offer scala
local_offer scala local_offer spark-2-x
In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.
local_offer scala local_offer spark-2-x
Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object.
Convert string to date in Scala / Spark
local_offer spark local_offer scala
This code snippet shows how to convert string to date.
local_offer spark local_offer scala
Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files.
local_offer spark local_offer hdfs local_offer scala local_offer parquet
In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page...
local_offer spark local_offer scala local_offer parquet
In this page, I’m going to demonstrate how to write and read parquet files in Spark/Scala by using Spark SQLContext class. Reference What is parquet format? Go the following project site to understand more about parquet. ...
Convert String to Date in Spark (Scala)
Context This pages demonstrates how to convert string to java.util.Date in Spark via Scala. Prerequisites If you have not installed Spark, follow the page below to install it: ...
