Spark (v2.x) Code snippets

Spark (v2.x) Python

Read JSON file as Spark DataFrame in Python / Spark (v2.x)

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

View detail
Spark (v2.x) Scala

Read JSON file as Spark DataFrame in Scala / Spark (v2.x)

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

View detail
Spark (v2.x) Python

Convert List to Spark Data Frame in Python / Spark (v2.x)

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

View detail
Spark (v2.x) Scala

Convert List to Spark Data Frame in Scala / Spark (v2.x)

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

View detail