scala

Articles tagged with scala.

local_offer spark local_offer hdfs local_offer scala local_offer parquet local_offer spark-file-operations

visibility 15253
thumb_up 0
access_time 3 years ago

In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page, I am going to demonstrate how to write and read parquet files in HDFS. import ...

local_offer spark local_offer scala local_offer parquet local_offer spark-file-operations

visibility 21529
thumb_up 0
access_time 3 years ago

In this page, I’m going to demonstrate how to write and read parquet files in Spark/Scala by using Spark SQLContext class. Go the following project site to understand more about parquet. https://parquet.apache.org/ If you have not installed Spark, follow this page to setup: Install Big Data ...

local_offer scala local_offer spark-2-x

visibility 5657
thumb_up 1
access_time 12 months ago

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

local_offer scala local_offer spark-2-x

visibility 1087
thumb_up 0
access_time 12 months ago

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

local_offer spark local_offer scala

visibility 322
thumb_up 0
access_time 12 months ago

Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files. 

local_offer spark local_offer scala

visibility 2442
thumb_up 0
access_time 12 months ago

This code snippet shows how to convert string to date.

local_offer scala

visibility 9511
thumb_up 0
access_time 3 years ago

This pages demonstrates how to convert string to java.util.Date in Spark via Scala. If you have not installed Spark, follow the page below to install it: Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice The following code snippet uses pattern yyyy-MM-dd ...

Read more

Find more tags on tag cloud.

launch Tag cloud