Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.


Learn more arrow_forward
Install Hadoop 3.2.1 on Windows 10 Step by Step Guide

local_offer windows10 local_offer hadoop local_offer yarn

visibility 5657
thumb_up 11
access_time 6 months ago

This detailed step-by-step guide shows you how to install the latest Hadoop (v3.2.1) on Windows 10. It also provides a temporary fix for bug HDFS-14084 (java.lang.UnsupportedOperationException INFO).

open_in_new Hadoop

local_offer hadoop local_offer hive

visibility 19441
thumb_up 5
access_time 2 years ago

In this article, I’m going to demo how to install Hive 3.0.0 on Windows 10. Prerequisites Before installation of Apache Hive, please ensure you have Hadoop available on your Windows environment. We cannot run Hive without Hadoop.  ...

open_in_new Hadoop

local_offer python local_offer spark local_offer pyspark

visibility 19903
thumb_up 6
access_time 2 years ago

Data partitioning is critical to data processing performance especially for large volume of data processing in Spark. Partitions in Spark won’t span across nodes though one node can contains more than one partitions. When processing, Spark assigns one task for each partition and each worker threa...

open_in_new Spark + PySpark

local_offer sqlite local_offer .net core local_offer entity-framework

visibility 22628
thumb_up 2
access_time 3 years ago

SQLite is a self-contained and embedded SQL database engine. In .NET Core, Entity Framework Core provides APIs to work with SQLite. This page provides sample code to create a SQLite database using package Microsoft.EntityFrameworkCore.Sqlite . Create sample project ...

open_in_new .NET Framework

local_offer linux local_offer WSL local_offer ubuntu

visibility 5669
thumb_up 3
access_time 2 years ago

This page shows how to install Windows Subsystem for Linux (WSL) system on a non-system drive manually. Enable Windows Subsystem for Linux system feature Open PowerShell as Administrator and run the following command to enable WSL feature: Enable-WindowsOptionalFea...

open_in_new Tools

local_offer pyspark local_offer spark local_offer spark-2-x

visibility 4365
thumb_up 0
access_time 8 months ago

Spark provides rich APIs to save data frames to many different formats of files such as CSV, Parquet, Orc, Avro, etc. CSV is commonly used in data application though nowadays binary formats are getting momentum. In this article, I am going to show you how to save Spark data frame as CSV file in b...

open_in_new Spark + PySpark

Install Hadoop 3.0.0 on Windows (Single Node)

local_offer hadoop local_offer yarn local_offer hdfs

visibility 32924
thumb_up 3
access_time 3 years ago

This page summarizes the steps to install Hadoop 3.0.0 on your Windows environment. Reference page: https://wiki.apache.org/hadoop/Hadoop2OnWindows ...

open_in_new Hadoop

local_offer python local_offer spark local_offer pyspark

visibility 16571
thumb_up 0
access_time 13 months ago

In Spark, SparkContext.parallelize function can be used to convert Python list to RDD and then RDD can be converted to DataFrame object. The following sample code is based on Spark 2.x. In this page, I am going to show you how to convert the following list to a data frame: data = [(...

open_in_new Spark + PySpark

local_offer asp.net core local_offer identity core 2

visibility 20239
thumb_up 0
access_time 3 years ago

The identity system in ASP.NET has evolved over time. If you are using ASP.NET Core, you probably found User property is an instance of ClaimsPrincipal in Controller or Razor views. Thus to retrieve the information, you need to utilize the claims.

open_in_new ASP.NET Core

local_offer python local_offer spark local_offer pyspark local_offer hive

visibility 16246
thumb_up 3
access_time 2 years ago

From Spark 2.0, you can easily read data from Hive data warehouse and also write/append new data to Hive tables. This page shows how to operate with Hive in Spark including: Create DataFrame from existing Hive table Save DataFrame to a new Hive table Append data ...

open_in_new Spark + PySpark

local_offer SQL Server local_offer python local_offer spark local_offer pyspark

visibility 15705
thumb_up 2
access_time 2 years ago

Spark is an analytics engine for big data processing. There are various ways to connect to a database in Spark. This page summarizes some of common approaches to connect to SQL Server using Python as programming language. ...

open_in_new Spark + PySpark

local_offer python local_offer spark

visibility 21272
thumb_up 0
access_time 2 years ago

This post shows how to derive new column in a Spark data frame from a JSON array string column. I am running the code in Spark 2.2.1 though it is compatible with Spark 1.6.0 (with less JSON SQL functions). Prerequisites Refer to the following post to install Spark in Windows. ...

open_in_new Spark + PySpark

local_offer angular

visibility 13263
thumb_up 1
access_time 2 years ago

Problem When you follow Angular CLI installation guide in Windows, you may encounter the following error: ng is not recognized as an internal or external command The resolutions are available in the following link: ...

open_in_new Angular

local_offer hadoop local_offer linux local_offer WSL

visibility 14200
thumb_up 8
access_time 2 years ago

In my previous post , I showed how to configure a single node Hadoop instance on Windows 10. The steps are not too difficult to follow if you have Java programming backgr...

open_in_new Hadoop

local_offer pyspark local_offer spark-2-x local_offer teradata local_offer SQL Server

visibility 1210
thumb_up 0
access_time 4 months ago

In my previous article about  Connect to SQL Server in Spark (PySpark) , I mentioned the ways t...

open_in_new Spark + PySpark

Entity Framework Core Code-First - Generate Covering Index with Columns Included

local_offer entity-framework local_offer asp.net core local_offer Azure local_offer C#

visibility 17
thumb_up 0
access_time 7 days ago

In SQL Server or some other relational databases, it is a very common requirement to create covering index with columns included in index pages beside the index key columns. With Entity Framework Core, you can also easily generate covering indexes using purely C# code. Scenario For ...

open_in_new ASP.NET Core

"Welcome to Kontext" Email Notifications

local_offer kontext

visibility 8
thumb_up 0
access_time 8 days ago

From 2020-06-27, Kontext will send out one "Welcome to Kontext!" email notifications to users for the following scenarios: when users register as a local user successfully and email is activated. when the first time users log in with Google and Microsoft accounts to register as...

open_in_new Kontext Information

Google One-tap Sign-in, Sign-in with Google and Microsoft Account are now Enabled

local_offer oauth local_offer kontext

visibility 50
thumb_up 0
access_time 23 days ago

From release 0.7.4, Kontext users can now easily log in/sign up with Google or Microsoft accounts. Log in with Google or Microsoft accounts To log in using your Google or Microsoft accounts, simply click Log in button on the top navigation bar.  ...

open_in_new Kontext Information

local_offer asp.net core local_offer asp.net core 3

visibility 11
thumb_up 0
access_time 25 days ago

Sign-in with social accounts like Google, Microsoft, Twitter and Facebook accounts are very commonly used in websites to allow website users to logon easily without registering an separate account. Issue summary During the implementation of Kontext Google sign-in function, I encoun...

open_in_new ASP.NET Core

Bootstrap 4 Mega Full Width Dropdown Menu

local_offer HTML local_offer Javascript local_offer bootstrap

visibility 67
thumb_up 0
access_time 2 months ago

On Kontext, a full width mega dropdown menu is implemented using Bootstrap 4. If you'd like to implement similar menus, please follow these steps.  Environment The following code snippets work with Bootstrap 4.4.1. It should also work with other 4.x versions but I didn't test i...

open_in_new Frontend & Javascript

local_offer teradata local_offer python

visibility 572
thumb_up 1
access_time 3 months ago

Pandas is commonly used by Python users to perform data operations. In many scenarios, the results need to be saved to a storage like Teradata. This article shows you how to do that easily using JayDeBeApi or  ...

open_in_new Spark + PySpark

local_offer python

visibility 153
thumb_up 0
access_time 3 months ago

CSV is a common data format used in many applications. It's also a common task for data workers to read and parse CSV and then save it into another storage such as RDBMS (Teradata, SQL Server, MySQL). In my previous article  ...

open_in_new Python Programming

local_offer teradata local_offer SQL

visibility 55
thumb_up 0
access_time 3 months ago

In SQL Server, we can use TRUNCATE statement to clear all the records in a table and it usually performs better compared with DELETE statements as no transaction log for each individual row deletion. The syntax looks like the following: TRUNCATE TABLE { database_name.schema_name.tab...

open_in_new Code snippets

local_offer teradata local_offer python local_offer Java

visibility 319
thumb_up 0
access_time 3 months ago

Python JayDeBeApi module allows you to connect from Python to Teradata databases using Java JDBC drivers. In article Connect to Teradata database through Python , I showed ho...

open_in_new Python Programming

local_offer hadoop local_offer hive local_offer Java

visibility 417
thumb_up 1
access_time 3 months ago

When I was configuring Hive 3.0.0 in Hadoop 3.2.1 environment, I encountered the following error: Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V Ro...

open_in_new Hadoop

local_offer pandas local_offer sqlite

visibility 107
thumb_up 0
access_time 3 months ago

In my previous posts, I showed how to use  jaydebeapi or sqlite3 pack...

open_in_new Python Programming

local_offer Azure local_offer kontext

visibility 9
thumb_up 0
access_time 3 months ago

In the past few months, Kontext's DNS server was not very stable as it went offline several times accidentally. When the DNS server is down, our domain (kontext.tech) cannot be parsed successfully to the IP address of Azure App Service (the service that hosts this website); thus it became unreach...

open_in_new Kontext Information

Modern Web Application - Azure Blob Storage for Uploaded Files

local_offer Azure local_offer asp.net core local_offer dotnetcore

visibility 274
thumb_up 0
access_time 3 months ago

With cloud platforms like Azure, we can totally separate user content storage from web application storage to decouple components from each other and to make the application easy to scale and deploy. This article provides detailed information with code snippets about how to use Azure server-less product Blob Storage and App Service to enable horizontally scalable web application for users to upload files (BLOBs).

open_in_new Azure

Azure SQL Database Automated Backup Strategy

local_offer Azure local_offer SQL Server

visibility 78
thumb_up 0
access_time 3 months ago

When designing the architecture of Kontext platform, Azure SQL Database is chosen as the storage for relational data. TDE and other advanced security features are always enabled to protect the database. Backup plans are also employed to ensure I can always restore the database for as point of tim...

open_in_new Azure

local_offer sqlite local_offer python local_offer Java

visibility 81
thumb_up 0
access_time 3 months ago

To read data from SQLite database in Python, you can use the built-in sqlite3 package . Another approach is to use SQLite JDBC driver via  ...

open_in_new Python Programming

All columns

ML.NET is an open source and cross-platform machine learning framework. With ML.NET, you can create custom ML models using C# or F# without having to leave the .NET ecosystem. This column publish articles about ML.NET.

Code snippets for various programming languages/frameworks.

AspNetCore.XmlRpc - a XML Remote Procedure Call library for ASP.NET Core.

A
Administrator

For more details about Docu, please visit:

https://github.com/kontext-tech/docu 

Data analytics with Google Cloud Platform.

Data analytics, application development with Microsoft Azure cloud platform.

Posts and tutorials about Power BI.