local_offer hadoop local_offer yarn
This detailed step-by-step guide shows you how to install the latest Hadoop (v3.2.1) on Windows 10. It also provides a temporary fix for bug HDFS-14084 (java.lang.UnsupportedOperationException INFO).
Kontext release v0.6.7
Kontext v0.6.7 is now released with a few changes/enhancements. Changes The following sections list the new features/changes i...
Kontext release v0.6.6
Kontext v0.6.6 is now released with a few changes/enhancements. Changes SEO enhancements Added a number of Facebook and twitter m...
local_offer machine-learning local_offer jupyter-notebook local_offer C# local_offer dotnet core
In this article, I'm going to show you how to install Jupyter in Windows and then install .NET kernel for Jupyter notebooks. It also shows a machine learning example using ML.NET. The target audience are .NET developers who want to expand their skills in data engineering and science domain...
Kontext release v0.6.5
Kontext v0.6.5 is now released with a few changes/enhancements. Changes RSS changes RSS subscriptions are now increased to 200 items and the d...
local_offer pyspark local_offer spark-2-x local_offer python
This articles show you how to convert a Python dictionary list to a Spark DataFrame. The code snippets runs on Spark 2.x environments. Input The input data (dictionary list looks like the following): data = [{"Category": 'Category A', 'ItemID': 1, 'Amount': 12.40}, ...
local_offer pyspark local_offer spark local_offer spark-2-x local_offer pandas
Apache Arrow is an in-memory columnar data format that can be used in Spark to efficiently transfer data between JVM and Python processes. This currently is most beneficial to Python users that work with Pandas/NumPy data. In this article, ...
Read and Write XML files in PySpark
local_offer pyspark local_offer spark-2-x local_offer spark
This article shows you how to read and write XML files in Spark. Sample XML file Create a sample XML file named test.xml with the following content: <?xml version="1.0"?> <data> <record id="1"> <rid>1</rid> <nam...
Create and Read Pickle Files in Python
local_offer python local_offer pandas
Pickle files are commonly used Python data related projects. This article shows how to create and load pickle files using Pandas. Create pickle file import pandas as pd import numpy as np file_name="data/test.pkl" data = np.random.randn(1000, 2) # pd.set_option('displ...
local_offer kontext local_offer Azure
From release v0.6.0, Kontext website now runs from a package (zip file) in Azure App Services. This ensures storage (incl. database storage, blob storage for uploaded files) is separated from the application and the application itself is read-only. Azure DevOps support I...
