Python Code snippets

Spark Python

Convert string to date in Python / Spark

This code snippet shows how to convert string to date.

View detail
Spark Python

Write and read parquet files in Python / Spark

Parquet is columnar store format published by Apache. It's commonly used in Hadoop ecosystem. There are many programming language APIs that have been implemented to support writing and reading parquet files. 

View detail
Python

Read and parse JSON in Python

JSON is commonly used in modern applications for data storage and transfers. Pretty much all programming languages provide APIs to parse JSON. 

View detail
Spark (v2.x) Python

Read JSON file as Spark DataFrame in Python / Spark (v2.x)

Spark has easy fluent APIs that can be used to read data from JSON file as DataFrame object. 

View detail
Spark (v2.x) Python

Convert List to Spark Data Frame in Python / Spark (v2.x)

In Spark, SparkContext.parallelize function can be used to convert list of objects to RDD and then RDD can be converted to DataFrame object through SparkSession.

View detail
Python

Check installed packages in Python

Different programming languages have different package management tools.

View detail