Scala: Read JSON file as Spark DataFrame

visibility 162 comment 0 access_time 11m languageEnglish

In article Scala: Parse JSON String as Spark DataFrame, it shows how to convert an in-memory JSON string object to a Spark DataFrame. This article shows how to read directly from a JSON file. In fact, this is even simpler. 

Read from local JSON file

The following code snippet reads from a local JSON file named test.json.

The content of the JSON file is:


Code snippet

|  ABC|  1|
|  DEF|  2|
|  GHI|  3|

Read from HDFS JSON file

The following code snippet reads from a path in HDFS (/big-data/test.json):

info Last modified by Raymond 11m copyright This page is subject to Site terms.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

More from Kontext
visibility 15
thumb_up 0
access_time 8d
visibility 1,111
thumb_up 1
access_time 8m