This site uses cookies to deliver our services. By using this site, you acknowledge that you have read and understand our Cookie and Privacy policy. Your use of Kontext website is subject to this policy. Allow Cookies and Dismiss

Tag - hadoop

hadoop

Install Hadoop 3.0.0 in Windows (Single Node)

5937 views   14 comments last modified about 8 months ago

This page summarizes the steps to install Hadoop 3.0.0 in your Windows environment. Reference page: https://wiki.apache.org/hadoop/Hadoop2OnWindows ...

View detail
hadoop

Resolve Hadoop RemoteException - Name node is in safe mode

106 views   0 comments last modified about 6 months ago

In Safe Mode, the HDFS cluster is read-only. After completion of block replication maintenance activity, the name node leaves safe mode automatically. If you try to delete files in safe mode, the following exception may raise: org.apache.hadoop.ipc.RemoteException(org.apac...

View detail
hadoop

Configure Sqoop in a Edge Node of Hadoop Cluster

504 views   0 comments last modified about 6 months ago

This page continues with the following documentation about configuring a Hadoop multi-nodes cluster via adding a new edge node to configure administration or client tools. ...

View detail
hadoop

Configure YARN and MapReduce Resources in Hadoop Cluster

234 views   0 comments last modified about 6 months ago

When configuring YARN and MapReduce in Hadoop cluster, it is very important to configure the memory and virtual processors correctly. If the configurations are incorrect, the nodes may not be able to start properly and the applications may not be able to run successfully. For example...

View detail
hadoop

Configure Hadoop 3.1.0 in a Multi Node Cluster

2280 views   0 comments last modified about 6 months ago

Previously, I summarized the steps to install Hadoop in a single node Windows machine. Install Hadoop 3.0.0 in Windows (Single Node) In this page, I...

View detail
hadoop

Default Ports Used by Hadoop Services (HDFS, MapReduce, YARN)

1082 views   0 comments last modified about 6 months ago

This page summarizes the default ports used by Hadoop services. It is useful when configuring network interfaces in a cluster. Hadoop 3.1.0 HDFS The secondary namenode http/https server address and port. ...

View detail
hadoop

Read Text File from Hadoop in Zeppelin through Spark Context

1314 views   0 comments last modified about 8 months ago

Background This page provides an example to load text file from HDFS through SparkContext in Zeppelin (sc). Reference The details about this method can be found at: SparkContext.textFile ...

View detail
hadoop

Use Hadoop File System Task in SSIS to Write File into HDFS

677 views   0 comments last modified about 8 months ago

Context SQL Server Integration Service ( SSIS ) has tasks to perform operations against Hadoop, for example: Hadoop File System Task Hadoop Hive Task Hadoop Pig Task In Data Flow Task, you can also use: Hadoop HDFS Source ...

View detail
hadoop

Invoke Hadoop WebHDFS APIs in .NET Core

371 views   0 comments last modified about 8 months ago

Background Apache doesn't provide native official .NET APIs for Hadoop HDFS. The HTTP REST API supports the complete FileSystem / ...

View detail
hadoop

Hadoop datanode issue and resolution - ‘Incompatible clusterIDs’

97 views   0 comments last modified about 9 months ago

Issue After finishing installation Hadoop 3.0.0 in my Windows: Install Hadoop 3.0.0 in Windows (Single Node) , I got the following error after I formated the name node several tim...

View detail