By using this site, you acknowledge that you have read and understand our Cookie policy, Privacy policy and Terms .

Share. Ask. Meet.

Join Our Global Community for Cloud, Data and AI Professionals.

person_add Register account_circle Log in

Featured columns/forums

account_circle Raymond

Articles about Apache Spark

open_in_new View

account_circle Raymond

Articles about Apache Hadoop

open_in_new View

account_circle Raymond

Everything about .NET framework.

open_in_new View

account_circle Raymond

Blog posts about ASP.NET Core

open_in_new View

account_circle Raymond

Code snippets for various programming languages/frameworks.

open_in_new View

account_circle Raymond

Posts and tutorials about Power BI.

open_in_new View

account_circle Raymond

Tutorials and informations about Teradata.

open_in_new View

account_circle Administrator

About the releases of Kontext Project.

open_in_new View

local_offer .net core local_offer entity-framework

visibility 13470
comment 4
thumb_up 0
access_time 2 years ago

SQLite is a self-contained and embedded SQL database engine. In .NET Core, Entity Framework Core provides APIs to work with SQLite. This page provides sample code to create a SQLite database using package Microsoft.EntityFrameworkCore.Sqlite . Create sample project ...

open_in_new View

local_offer hadoop local_offer yarn local_offer hdfs

visibility 25890
comment 30
thumb_up 1
access_time 2 years ago

This page summarizes the steps to install Hadoop 3.0.0 in your Windows environment. Reference page: https://wiki.apache.org/hadoop/Hadoop2OnWindows ...

open_in_new View

local_offer hadoop local_offer linux local_offer WSL

visibility 8578
comment 18
thumb_up 5
access_time 8 months ago

In my previous post , I showed how to configure a single node Hadoop instance on Windows 10. The steps are not too difficult to follow if you have Java programming backgr...

open_in_new View

local_offer hadoop local_offer hive

visibility 10465
comment 11
thumb_up 1
access_time 9 months ago

If you have been following my website, you would know I’ve published a number of articles about installing big data tools/framewo...

open_in_new View

local_offer python local_offer spark local_offer pyspark

visibility 4976
comment 0
thumb_up 0
access_time 6 months ago

In Spark, SparkContext.parallelize function can be used to convert Python list to RDD and then RDD can be converted to DataFrame object. The following sample code is based on Spark 2.x. In this page, I am going to show you how to convert the following list to a data frame: data = [(...

open_in_new View

local_offer python local_offer spark local_offer pyspark

visibility 3468
comment 0
thumb_up 2
access_time 9 months ago

Data partitioning is critical to data processing performance especially for large volume of data processing in Spark. Partitions in Spark won’t span across nodes though one node can contains more than one partitions. When processing, Spark assigns one task for each partition and each worker threa...

open_in_new View

local_offer asp.net core local_offer identity core 2

visibility 12728
comment 0
thumb_up 0
access_time 3 years ago

The identity system in ASP.NET has evolved over time. If you are using ASP.NET Core, you probably found User property is an instance of ClaimsPrincipal in Controller or Razor views. Thus to retrieve the information, you need to utilize the claims.

open_in_new View

local_offer python local_offer spark

visibility 11980
comment 0
thumb_up 0
access_time 12 months ago

This post shows how to derive new column in a Spark data frame from a JSON array string column. I am running the code in Spark 2.2.1 though it is compatible with Spark 1.6.0 (with less JSON SQL functions). Prerequisites Refer to the following post to install Spark in Windows. ...

open_in_new View

local_offer angular local_offer lite-log

visibility 9197
comment 3
thumb_up 0
access_time 2 years ago

Problem When you follow Angular CLI installation guide in Windows, you may encounter the following error: ng is not recognized as an internal or external command The resolutions are available in the following link: ...

open_in_new View

local_offer power-bi local_offer google-analytics

visibility 2324
comment 0
thumb_up 0
access_time 2 years ago

Power BI is my favourite BI and visualization tool as it is very simple yet powerful. It doesn’t only support traditional data sources like databases, CSV, JSON, XML and etc., but also supports emerging sources that are available in HDFS, Spark, R, Salesforce, Google Analytics and cloud platforms...

open_in_new View

...

local_offer kontext

visibility 4
comment 0
thumb_up 0
access_time 11 hours ago

Kontext v0.6.4 is now released with a few changes. Changes Standalone pages Administrators can now add standalone pages. These pages are not linked...

open_in_new View

...

local_offer kontext

visibility 7
comment 0
thumb_up 0
access_time 12 days ago

Kontext v0.6.3 is now released with a few changes. Changes User profile enhancements User created latest columns are now also added to user profile page. ...

open_in_new View

local_offer pyspark local_offer spark-2-x local_offer spark

visibility 7
comment 0
thumb_up 0
access_time 12 days ago

Sometime it is necessary to pass environment variables to Spark executors. To pass environment variable to executors, use setExecutorEnv function of SparkConf class. Code snippet In the following code snippet, an environment variable name ENV_NAME is set up with value ...

open_in_new View

local_offer pyspark local_offer spark local_offer spark-2-x

visibility 11
comment 0
thumb_up 0
access_time 12 days ago

Spark provides rich APIs to save data frames to many different formats of files such as CSV, Parquet, Orc, Avro, etc. CSV is commonly used in data application though nowadays binary formats are getting momentum. In this article, I am going to show you how to save Spark data frame as CSV file in b...

open_in_new View

local_offer kontext

visibility 9
comment 0
thumb_up 0
access_time 14 days ago

There are a number of small improvements and enhancements to Kontext in release v0.6.2. Changes The following are the main changes in this release: In site navbar for column site and articles pages, the author of column are added to feature the owner and column. ...

open_in_new View

...

local_offer devops local_offer Azure

visibility 8
comment 0
thumb_up 0
access_time 14 days ago

It might be useful to push artifact or build variables to your Azure websites environment variables in Release pipelines.

open_in_new View

local_offer kontext

account_circle Raymond arrow_drop_down
visibility 20
comment 0
thumb_up 0
access_time 21 days ago

At the moment, tags are managed by the site administrators and there are already a number of tags created. Refer to the following link for more details: Existing tags If you think there is other tags we need to add, please let us ...

open_in_new View

local_offer kontext

visibility 18
comment 0
thumb_up 3
access_time 22 days ago

Kontext v0.6.1 is now released. New features Azure blob storage as file storage All the user uploaded images are now stored in Azure blob storage. Eac...

open_in_new View

local_offer kontext local_offer Azure

visibility 123
comment 0
thumb_up 4
access_time 22 days ago

From release v0.6.1, Kontext platform is server-less and cloud native with horizontal scalabilities.

open_in_new View

...

local_offer kontext local_offer Azure

visibility 125
comment 0
thumb_up 1
access_time 26 days ago

Kontext website is now upgraded to ASP.NET Core 3.0 with many new features.

open_in_new View