scala

Articles tagged with scala.

local_offer scala local_offer spark-2-x

visibility 4928
thumb_up 1
access_time 11 months ago

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

local_offer scala local_offer spark-2-x

visibility 923
thumb_up 0
access_time 11 months ago

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

local_offer spark local_offer scala

visibility 2088
thumb_up 0
access_time 11 months ago

This code snippet shows how to convert string to date.

local_offer spark local_offer scala

visibility 276
thumb_up 0
access_time 11 months ago

Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files. 

local_offer spark local_offer hdfs local_offer scala local_offer parquet local_offer spark-file-operations

visibility 14639
thumb_up 0
access_time 3 years ago

In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page, I am going to demonstrate how to write and read parquet files in HDFS. import ...

local_offer spark local_offer scala local_offer parquet local_offer spark-file-operations

visibility 20943
thumb_up 0
access_time 3 years ago

In this page, I’m going to demonstrate how to write and read parquet files in Spark/Scala by using Spark SQLContext class. Go the following project site to understand more about parquet. https://parquet.apache.org/ If you have not installed Spark, follow this page to setup: Install Big Data ...

local_offer scala

visibility 9438
thumb_up 0
access_time 3 years ago

This pages demonstrates how to convert string to java.util.Date in Spark via Scala. If you have not installed Spark, follow the page below to install it: Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice The following code snippet uses pattern yyyy-MM-dd ...

Read more

Find more tags on tag cloud.

launch Tag cloud