Browse Courses

Hadoop Administration+Development

Course Description

Curriculum

Chapter 1 : Hadoop Administration 

1Introduction to Big Data

  •     What is Big Data ?
  •     Big Data Facts
  •     The Three V’s of Big Data

Chapter 2 : Understanding Hadoop 

1 Understanding Hadoop

  •     Understanding Hadoop
  •     What is Hadoop ?
  •     Why learn Hadoop ?
  •     Relational Databases Vs. Hadoop
  •     Motivation for Hadoop
  •     6 Key Hadoop Data Types

Chapter 3 : The Hadoop Distributed File system (HDFS) 

1The Hadoop Distributed File system (HDFS)

  •     What is HDFS ?
  •     HDFS components
  •     Understanding Block storage
  •     The Name Node
  •     The Data Nodes
  •     Data Node Failures
  •     HDFS Commands
  •     HDFS File Permissions

Chapter 4 : The MapReduce Framework 

1The MapReduce Framework

  •     Overview of MapReduce
  •     Understanding MapReduce
  •     The Map Phase
  •     The Reduce Phase
  •     WordCount in MapReduce
  •     Running MapReduce Job

Chapter 5 : Planning Your Hadoop Cluster 

1Planning Your Hadoop Cluster

  •     Single Node Cluster Configuration
  •     Multi-Node Cluster Configuration

Chapter 6 : Cluster Maintenance 

1Cluster Maintenance

  •     Checking HDFS Status
  •     Breaking the cluster
  •     Copying Data Between Clusters
  •     Adding and Removing Cluster Nodes
  •     Rebalancing the cluster
  •     Name Node Metadata Backup
  •     Cluster Upgrading

Chapter 7 : Installing and Managing Hadoop Ecosystem Projects 

1Installing and Managing Hadoop Ecosystem Projects

  •     Sqoop
  •     Flume
  •     Hive
  •     Pig
  •     HBase
  •     Oozie

Chapter 8 : Managing and Scheduling Jobs 

1Managing and Scheduling Jobs

  •     Managing Jobs
  •     The FIFO Scheduler
  •     The Fair Schedule
  •     How to stop and start jobs running on the cluster

Chapter 9 : Cluster Monitoring, Troubleshooting, and Optimizing 

1Cluster Monitoring, Troubleshooting, and Optimizing

  •     General System conditions to Monitor
  •     Name Node and Job Tracker Web Uis
  •     View and Manage Hadoop’s Log files
  •     Ganglia Monitoring Tool
  •     Common cluster issues and their resolutions
  •     Benchmark your cluster’s performance

Chapter 10 : Populating HDFS from External Sources 

1 Populating HDFS from External Sources

  •     How to use Sqoop to import data from RDBMSs to HDFS
  •     How to gather logs from multiple systems using Flume
  •     Features of Hive, Hbase and Pig
  •     How to populate HDFS from external Sources

Chapter 11 : Hadoop Development 

1Introduction to BigData, Hadoop

  • Big Data Introduction

         -  Hadoop Introduction
         - What is Hadoop? Why Hadoop?
         -  Hadoop History?
         -  Different types of Components in Hadoop?
         -  HDFS, MapReduce, PIG, Hive, SQOOP, HBASE, OOZIE, Flume, Zookeeper and so on…
         -  What is the scope of Hadoop?

Chapter 12 : Deep Drive in HDFS (for Storing the Data) 

1Introduction of HDFS

 

  • HDFS Design
  • HDFS role in Hadoop
  • Features of HDFS
  • Daemons of Hadoop and its functionality

                        - Name Node
                        - Secondary Name Node
                        - Job Tracker
                        - Data Node
                        - Task Tracker

  • Anatomy of File Wright
  • Anatomy of File Read
  • Network Topology

                        - Nodes
                        - Racks
                        - Data Center

  •  Parallel Copying using DistCp
  •  Basic Configuration for HDFS
  •  Data Organization

                        - Blocks and
                        - Replication

  • Rack Awareness
  • Heartbeat Signal
  • How to Store the Data into HDFS
  • How to Read the Data from HDFS
  • Accessing HDFS (Introduction of Basic UNIX commands)
  • CLI commands

Chapter 13 : MapReduce using Java (Processing the Data) 

1The introduction of MapReduce.

 

  • MapReduce Architecture
  • Data flow in MapReduce

                        - Splits
                        - Mapper
                        - Portioning
                        - Sort and shuffle
                        - Combiner
                        - Reducer

  • Understand Difference Between Block and InputSplit
  • Role of RecordReader
  • Basic Configuration of MapReduce
  • MapReduce life cycle

                        - Driver Code
                        - Mapper
                        - and Reducer

  • How MapReduce Works
  • Writing and Executing the Basic MapReduce Program using Java
  • Submission & Initialization of MapReduce Job.
  • File Input/output Formats in MapReduce Jobs

                        - Text Input Format
                        - Key Value Input Format
                        - Sequence File Input Format
                        - NLine Input Format

  •  Joins

                        - Map-side Joins
                        - Reducer-side Joins
                        

  • Word Count Example
  • Partition MapReduce Program
  • Side Data Distribution

                        - Distributed Cache (with Program)

  • Counters (with Program)

                        - Types of Counters
                        - Task Counters
                        - Job Counters
                        - User Defined Counters
                        - Propagation of Counters

  • Job Scheduling

Chapter 14 : PIG 

1Introduction to Apache PIG

 

  • Introduction to PIG Data Flow Engine
  • MapReduce vs PIG in detail
  • When should PIG use?
  • Data Types in PIG
  • Basic PIG programming
  • Modes of Execution in PIG

                        - Local Mode and
                        - MapReduce Mode
                        

  • Execution Mechanisms

                        - Grunt Shell
                        - Script
                        - Embedded
                        

  • Operators/Transformations in PIG
  • PIG UDF’s with Program
  • Word Count Example in PIG
  • The difference between the MapReduce and PIG

Chapter 15 : SQOOP 

1Introduction to SQOOP

 

  • Use of SQOOP
  • Connect to mySql database
  • SQOOP commands

                        - Import
                        - Export
                        - Eval
                        - Codegen and etc…

  • Joins in SQOOP
  • Export to MySQL
  • Export to HBase

Chapter 16 : HIVE 

1Introduction to HIVE

  • HIVE Meta Store
  • HIVE Architecture
  • Tables in HIVE

                        - Managed Tables
                        - External Tables

  • Hive Data Types

                        - Primitive Types
                        - Complex Types

 

  • Partition
  • Joins in HIVE
  • HIVE UDF’s and UADF’s with Programs
  • Word Count Example

Chapter 17 : HBASE 

1Introduction to HBASE

  • Basic Configurations of HBASE
  • Fundamentals of HBase
  • What is NoSQL?
  • HBase Data Model

                        - Table and Row
                        - Column Family and Column Qualifier
                        - Cell and its Versioning
                        

  • Categories of NoSQL Data Bases

                        - Key-Value Database
                        - Document Database
                        - Column Family Database

  • HBASE Architecture

                        - HMaster
                        - Region Servers
                        - Regions
                        - MemStore
                        - Store

  • SQL vs NOSQL
  • How HBASE is differed from RDBMS
  • HDFS vs HBase
  • Client side buffering or bulk uploads
  • HBase Designing Tables
  • HBase Operations

                        - Get
                        - Scan
                        - Put
                        - Delete

Chapter 18 : MongoDB 

1What is MongoDB?

  •  Where to Use?
  •  Configuration On Windows
  •  Inserting the data into MongoDB?
  •  Reading the MongoDB data.

Chapter 19 : Cluster Setup 

1Downloading and installing the Ubuntu12.x

  • Installing Java
  • Installing Hadoop
  • Creating Cluster
  • Increasing Decreasing the Cluster size
  • Monitoring the Cluster Health
  • Starting and Stopping the Nodes

Chapter 20 : Zookeeper 

1Introduction Zookeeper

  • Data Modal
  • Operations

Chapter 21 : OOZIE 

1Introduction to OOZIE

  •  Use of OOZIE
  •  Where to use?

Chapter 22 : Flume 

1Introduction to Flume

  • Uses of Flume
  • Flume Architecture


                        - Flume Master
                        - Flume Collectors
                        - Flume Agents

$600

  • Lectures0
  • Video
  • Skill level Full Course
  • Languages English

Take the world's best courses

and take your career to the next level!


New to Truway? Create a new account

By signing up, you agree to our Terms of Use and Privacy Policy.
Already have an account? Login