This site uses cookies to deliver our services. By using this site, you acknowledge that you have read and understand our Cookie and Privacy policy. Your use of Kontext website is subject to this policy. Allow Cookies and Dismiss
lite-log power-bi

Data Analysis Expressions to Create Static Tables in PowerBI

26 views   0 comments last modified about 18 days ago

DATATABLE StaticTable1 = DATATABLE("IntCol",INTEGER,"StringCol",STRING,{{1,"User1"},{2,"User2"}}) The above expression generates a table with two columns IntCol and StringCol : ...

View detail
power-bi google-analytics

Power Analytics with Power BI and Google Analytics

80 views   0 comments last modified about 18 days ago

Power BI is my favourite BI and visualization tool as it is very simple yet powerful. It doesn’t only support traditional data sources like databases, CSV, JSON, XML and etc., but also supports emerging sources that are available in HDFS, Spark, R, Salesforce, Google Analytics and cloud platforms...

View detail
power-bi

Power BI Analytics - Connect to DBMS and Card

16 views   0 comments last modified about 18 days ago

Power BI supports connecting to most of the DBMS databases such as SQL Server, Oracle, Teradata, MySQL, DB2, Sybase, Snowflake, Google BigQuery, Impala and etc. This page summarizes the steps to connect to SQL Azure and to create the following part of the sample dashboard of this series: ...

View detail
lite-log powershell

Script to Start Multiple Virtual Machines

39 views   0 comments last modified about 2 months ago

The following is a PowerShell code snippet to start multiple VMs through vmplayer.exe: # Navigate to the folder where vmplayer.exe file exists cd "D:\Program Files (x86)\VM Player" # Start the VMs by using vmplayer...

View detail
teradata

Useful DBC (Data Base Computer) System Views in Teradata

39 views   0 comments last modified about 2 months ago

This page summarize some of the commonly used views in Teradata. Conventions In all the views in the following sections, X views are also available though they only return rows that contain information on objects that the requesting database user owns, created, granted privilige on,...

View detail
lite-log hadoop hdfs

Resolve Hadoop RemoteException - Name node is in safe mode

29 views   0 comments last modified about 2 months ago

In Safe Mode, the HDFS cluster is read-only. After completion of block replication maintenance activity, the name node leaves safe mode automatically. If you try to delete files in safe mode, the following exception may raise: org.apache.hadoop.ipc.RemoteException(org.apac...

View detail
hadoop hdfs parquet sqoop

Configure Sqoop in a Edge Node of Hadoop Cluster

140 views   0 comments last modified about 2 months ago

This page continues with the following documentation about configuring a Hadoop multi-nodes cluster via adding a new edge node to configure administration or client tools. ...

View detail
hadoop yarn

Configure YARN and MapReduce Resources in Hadoop Cluster

31 views   0 comments last modified about 2 months ago

When configuring YARN and MapReduce in Hadoop cluster, it is very important to configure the memory and virtual processors correctly. If the configurations are incorrect, the nodes may not be able to start properly and the applications may not be able to run successfully. For example...

View detail
sql server lite-log

A New SQL Server Client Tool - SQL Operations Studio

32 views   0 comments last modified about 2 months ago

If you have used Visual Studio Code, you probably will also enjoy a new client tool SQL Operations Studio. Currently it is available as preview version and not formally released yet. What is SQL Operations Studio SQL Operations Studio is a data management tool that enables working w...

View detail
hadoop yarn hdfs

Configure Hadoop 3.1.0 in a Multi Node Cluster

639 views   0 comments last modified about 2 months ago

Previously, I summarized the steps to install Hadoop in a single node Windows machine. Install Hadoop 3.0.0 in Windows (Single Node) In this page, I...

View detail
lite-log

Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice

210 views   2 comments last modified about 2 months ago

Are you a Windows/.NET developer and willing to learn big data concepts and tools in your Windows? If yes, you can follow the links below to install them in your PC. The installations are usually easier to do in Linux/UNIX but they are not difficult to implement in Windows either since the...

View detail
hadoop yarn hdfs

Default Ports Used by Hadoop Services (HDFS, MapReduce, YARN)

146 views   0 comments last modified about 2 months ago

This page summarizes the default ports used by Hadoop services. It is useful when configuring network interfaces in a cluster. Hadoop 3.1.0 HDFS The secondary namenode http/https server address and port. ...

View detail
sql server spark hdfs parquet sqoop

Load Data into HDFS from SQL Server via Sqoop

113 views   0 comments last modified about 2 months ago

This page shows how to import data from SQL Server into Hadoop via Apache Sqoop. Prerequisites Please follow the link below to install Sqoop in your machine if you don’t have one environment ready. ...

View detail
sqoop

Install Apache Sqoop in Windows

101 views   0 comments last modified about 2 months ago

This page summarizes the steps required to install Apache Sqoop (v1.4.7) in Windows 10 environment. What is Sqoop Sqoop is an ETL tool for Hadoop,which is designed to efficiently transfer data between structured (RDBMS), semi-structured (Cassandra, Hbase and etc.) and unstructured ...

View detail
sql server zeppelin

Connecting Apache Zeppelin to your SQL Server

99 views   0 comments last modified about 2 months ago

This page demonstrates the steps you need to connect to SQL Server in Zeppelin. There are many ways to implement this, for example SQL Server interpreters in GitHub. In this page, I am going to use the JDBC driver to connect to SQL Server instead of using third party interpreters. For authe...

View detail
lite-log linux

Mount Windows Shared Folder in SUSE or other Linux VMware Guest

20 views   0 comments last modified about 3 months ago

This page summarizes the steps to mount a windows shared folder in a Linux VMware guest machine, i.e. SUSE. Setup shared folder In Virtual Machine Settings windows, add shared folder. ...

View detail

Querying Teradata and SQL Server - Tutorial 1: The SELECT Statement

28335 views   7 comments last modified about 4 years ago

SELECT is one of the most commonly used statements. In this tutorial, I will cover the following items: Two of the principal query clauses—FROM and SELECT Data Types Built-in functions CASE expressions and variations like ISNULL and COALESCE. * The functio...

View detail

Install Teradata Express 15.0.0.8 by Using VMware Player 6.0 in Windows

11775 views   23 comments last modified about 4 years ago

In this article, I am going to introduce how to install Teradata Express in virtual machines in Windows. Download software 1) Download VMware Player for Windows 32-bit and 64-bit from the following link (version 6.0): ...

View detail

Working with SQL Server Compact 4.0 using Entity Framework 6 and ADO.NET

10430 views   0 comments last modified about 4 years ago

SQL Server Compact 4.0 (CE 4.0) is a free SQL Server embedded database ideal for building standalone and occasionally connected applications for mobile devices, desktops, Web clients and others. In one of my projects, I used it as the database for logging errors, which assumes the errors will onl...

View detail

Generate Formatted Excel Destination (Output) in SSIS Data Flow Task

8252 views   0 comments last modified about 4 years ago

SSIS (SQL Server Integration Service) provides a number of convenient tasks to enable data integration. Exporting data from database to Excel file is a common task in ETL (Extract, Transform, Load) projects. Constantly the users/customers may raise format request regarding the Excel extract. To g...

View detail

Create ETL Project with Teradata through SSIS

7926 views   2 comments last modified about 4 years ago

Infosphere DataStage is adopted as ETL (Extract, Transform, Load) tool in many Teradata based data warehousing projects. With the Teradata ODBC and .NET data providers, you can also use the BI tools from Microsoft, i.e. SSIS. In my previous post, I demonstrated how to install Teradata Tool...

View detail
dotnet core angular asp.net core 2

Issue - Unable to get property 'apply' of undefined or null reference occurred in Angular 4.*, VS2017 15.3, ASP.NET Core 2.0

6128 views   10 comments last modified about 2 years ago

Issue Context After installed Visual Studio 2017 15.3 preview and .net core 2.0 preview SDK, I upgraded one of my existing asp.net core project to 2.0. The project was created using ‘dotnet new angular’ SPA template.  I also upgraded all the client app packages to the latest. For exa...

View detail

Connect to Teradata Virtual Machine Guest from Windows Host

5375 views   16 comments last modified about 4 years ago

In my previous posts about Querying Teradata and SQL Server, I logged into the virtual machine graphic interface to manage the database. However, I constantly found it is resource intensive as there is only 4GB memory in my laptop. Instead, I will use text mode to start the virtual machine and co...

View detail
java kerberos

Java Kerberos Authentication Configuration Sample & SQL Server Connection Practice

4303 views   2 comments last modified about 3 years ago

Overview Recently, I have been working on an ETL framework to load various source data (i.e. files, SQL Server, Oracle and Teradata) into Teradata. Due to some limitations, Java was chosen as the implementation language though IBM Infosphere DataStage is available to use. DataStage has p...

View detail

[C#] Connect to Teradata Database via .NET Data Provider

3987 views   2 comments last modified about 3 years ago

In this post, I will demonstrate how to connect to Teradata database via .NET Data Provider for Teradata using C#. Prerequisites Install the .NET Data Provider for Teradata from the following link: ...

View detail

Resolve the Issues in Upgrading Entity Framework to Version 6.1

3747 views   0 comments last modified about 4 years ago

When upgrading your Entity Framework to Entity Framework 6.1 (EF6) from version 5.0, you may meet a number of issues. I have summarized all the issues I’ve encountered and their resolutions for your reference. Upgrade to EF6 Microsoft has provided one summary about upgrading to E...

View detail
asp.net core 2

Server.MapPath Equivalent in ASP.NET Core 2

3450 views   0 comments last modified about 9 months ago

In traditional asp.net applications, Server.MapPath is commonly used to generate absolute path in the web server. However, this has been removed from ASP.NET Core. So what is the equivalent way of doing it?

View detail

Create and Debug C/C++ Programs with Eclipse and Cygwin in Windows

3176 views   0 comments last modified about 3 years ago

In this post, I am going to demonstrate how to use Eclipse to create and debug C/C++ programs for Unix/Linux in Windows. I am going to use Cygwin GCC as toolchains. Cygwin GDB will also be installed for debugging purpose. I am using Windows 10 and JRE 1.8 in the following steps. Install E...

View detail

Querying Teradata and SQL Server - Tutorial 2 Filtering and Sorting

2840 views   2 comments last modified about 4 years ago

Tutorial –1 For Teradata developers, if you have no SQL Server installed, please go to the following link to download the SQL Server 2014 Expression Edition. ...

View detail

Querying Teradata and SQL Server - Tutorial 0: Preparation

2631 views   0 comments last modified about 4 years ago

In this serial, I will compare Teradata SQL with T-SQL with samples provided. This is mainly prepared for SQL Server DBAs, Developers and other users to help them quickly master the common used SQLs in Teradata platform. Similar to T-SQL, Teradata SQL implements most part of the ANSI SQL with ext...

View detail

Surface RT/Pro 版本Windows 8 QQ下载与安装

2508 views   0 comments last modified about 5 years ago

Surface的用户可能会遇到一个难题,在应用商城里边无法搜索到QQ应用程序。但是让我疑惑的是,我的Windows 8.1 系统之前是成功安装了QQ。

后来在腾讯QQ下载官网发现,其实是有QQ应用安装的链接:

按照如下方法安装Surface QQ.

Have fun!

image

View detail

Migrate from Universal Membership Provider to ASP.NET Identity 2.1.0 - PasswordHarsher

2246 views   2 comments last modified about 4 years ago

In my previous post, I demonstrated how to migrate from ASP.NET Universal Membership Provider to ASP.NET Identity 2.1.0. ...

View detail

about 5 days ago

I can get it work by using the following approach:

1) Create an IIS website (http://localhost/Test/) with one page index.html:

<html>
    <head>
        <title>Test</title>
    </head>
    <body>
        <h1>Test iframe</h1>
        <iframe src="http://localhost:8080/#/notebook/2D7J63CN7" width="600px" height="400px" style="border:1px solid #000000"></iframe>
    </body>
</html>

2) And then open the website in the browser: http://localhost/Test/

If you open with a file URL, then the content cannot be displayed due to security reasons:

file:///F:/Projects/HtmlProjTest/Index.html

When you publish your website, your Zeppelin site should also be deployed into a server that your user can access.


about 5 days ago

Hi Raymond Tang. I come back because I tried but I didn't succeded to embed a zeppelin notebook as an iframe in my website. I have something like that  

<div id="interactivForm">

                                <iframe id="MyInterpreter" src="http://localhost:8085/#/notebook/2DHDGTVNU"></iframe>
                           </div> in my website but it doesn't work. But I can access to http://localhost:8085/#/notebook/2DHDGTVNU without problem. Do you know how to do that?
Thank you.

about 12 days ago

You can use <iframe> html element to embed Zeppelin into your website.

This also means that your Zeppelin website (*:8080 by default) needs to be exposed to all your users (i.e. their networks).

about 12 days ago

Hi Raymond. Now I don't really want to do any authentification. I want only to give an opportunity to anonymous user to execute spark in my website using zeppelin. So do you know how to do that or do you have a tutorial where I can see how to do that step by step? Thank you very much.

about 12 days ago

You can embed Zeppelin into a website.

However you need to decide how to pass through user credentials from your website to Zeppelin website depends on the authentication type. 

The authentication part you can reference to the following website:

https://zeppelin.apache.org/docs/0.6.2/security/shiroauthentication.html#4-login 

Based on my current understanding, I don't think currently you can directly implement the automatic logon without extending Zeppelin (but I might be wrong). 

about 13 days ago

Hi everyone,

do you know if it is possible to embed Zeppelin Notebook in a webpage? Like as an iframe or another method? So that users can come and execute their own code? Do you think that is possible? Has someone an idea to do that? Thank you!

about 14 days ago

Nw, I'm glad it worked. :)

about 14 days ago

I am sorry. It works!

about 14 days ago

I come again and sorry. When I run this commande %HADOOP_HOME%\bin\hdfs dfs -put file:///G:/DataAnalytics/test.txt / I get an error: put: `G:/DataAnalytics/test.txt': No such file or directory. But I followed the configuration step by step and my DataAnalytics folder is in G:.

about 14 days ago

Hi. thank you for your answer. I found a solution for my problem. If it can help someone, the problem was related to the syntax of my system username. It contains a space. So, to fixe it, you can edit /etc/hadoop/hadoop-env.cmd, at the end of this file, you will find set HADOOP_IDENT_STRING=%USERNAME% , change this with a string that you want but without space. For example: set HADOOP_IDENT_STRING=myuser, the problem will be fixed.

about 14 days ago

Did you follow all the steps in this post? For example, you need to ensure winutils tool is installed: 

Overwrite your bin folder (%HADOOP_HOME%\bin) with the files from this link:

https://github.com/steveloughran/winutils/tree/master/hadoop-3.0.0/bin

The current available version is 3.0.0 and I am not very sure whether it can fix the issue for 3.0.1 but worth giving it a try.

about 14 days ago

Good morning, I am trying to install hadoop 3.0.1 in my windows but when I want to test my configuration it gives me that error: Error: Can not find or load the main class. Can someone help me please?

Thank you

about 2 months ago

@Ansh

Yes, you can.

For example, the following code is used to read parquet files from a Hadoop cluster.

def readParquet(sqlContext: SQLContext) = {
// read back parquet to DF
val newDataDF = sqlContext.read.parquet("hdfs://hdp-master:19000/user/hadoop/sqoop_test/blogs")
// show contents
newDataDF.show()
}

The cluster was setup by following this post:

Configure Hadoop 3.1.0 in a Multi Node Cluster

Of source the hdp-master:19000 needs to be accessible from the server that running the Spark/Scala code.

At the moment, my HDFS is set as readable for all servers/users in the LAN. In a production environment, you may need to manage the permissions too.

Furthermore, you can also run Spark apps in a Spark Cluster instead of in stand-alone or local machine.  I will cover more about this in my future post.

about 2 months ago

Can we connect and read remotely located HDFS Parquet file? by using above code