This site uses cookies to deliver our services. By using this site, you acknowledge that you have read and understand our Cookie and Privacy policy. Your use of Kontext website is subject to this policy. Allow Cookies and Dismiss

Tag - hdfs

hdfs

Install Hadoop 3.0.0 in Windows (Single Node)

1659 views   14 comments last modified about 4 months ago

This page summarizes the steps to install Hadoop 3.0.0 in your Windows environment. Reference page: https://wiki.apache.org/hadoop/Hadoop2OnWindows ...

View detail
hdfs

Resolve Hadoop RemoteException - Name node is in safe mode

29 views   0 comments last modified about 2 months ago

In Safe Mode, the HDFS cluster is read-only. After completion of block replication maintenance activity, the name node leaves safe mode automatically. If you try to delete files in safe mode, the following exception may raise: org.apache.hadoop.ipc.RemoteException(org.apac...

View detail
hdfs

Configure Sqoop in a Edge Node of Hadoop Cluster

140 views   0 comments last modified about 2 months ago

This page continues with the following documentation about configuring a Hadoop multi-nodes cluster via adding a new edge node to configure administration or client tools. ...

View detail
hdfs

Configure Hadoop 3.1.0 in a Multi Node Cluster

642 views   0 comments last modified about 2 months ago

Previously, I summarized the steps to install Hadoop in a single node Windows machine. Install Hadoop 3.0.0 in Windows (Single Node) In this page, I...

View detail
hdfs

Default Ports Used by Hadoop Services (HDFS, MapReduce, YARN)

148 views   0 comments last modified about 2 months ago

This page summarizes the default ports used by Hadoop services. It is useful when configuring network interfaces in a cluster. Hadoop 3.1.0 HDFS The secondary namenode http/https server address and port. ...

View detail
hdfs

Load Data into HDFS from SQL Server via Sqoop

113 views   0 comments last modified about 2 months ago

This page shows how to import data from SQL Server into Hadoop via Apache Sqoop. Prerequisites Please follow the link below to install Sqoop in your machine if you don’t have one environment ready. ...

View detail
hdfs

Write and Read Parquet Files in HDFS through Spark/Scala

418 views   0 comments last modified about 4 months ago

In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page...

View detail
hdfs

Use Hadoop File System Task in SSIS to Write File into HDFS

213 views   0 comments last modified about 4 months ago

Context SQL Server Integration Service ( SSIS ) has tasks to perform operations against Hadoop, for example: Hadoop File System Task Hadoop Hive Task Hadoop Pig Task In Data Flow Task, you can also use: Hadoop HDFS Source ...

View detail
hdfs

Invoke Hadoop WebHDFS APIs in .NET Core

138 views   0 comments last modified about 4 months ago

Background Apache doesn't provide native official .NET APIs for Hadoop HDFS. The HTTP REST API supports the complete FileSystem / ...

View detail
hdfs

Hadoop datanode issue and resolution - ‘Incompatible clusterIDs’

32 views   0 comments last modified about 5 months ago

Issue After finishing installation Hadoop 3.0.0 in my Windows: Install Hadoop 3.0.0 in Windows (Single Node) , I got the following error after I formated the name node several tim...

View detail