Hadoop: The Definitive Guide

Front Cover
"O'Reilly Media, Inc.", Sep 24, 2010 - Computers - 628 pages

Discover how Apache Hadoop can unleash the power of your data. This comprehensive resource shows you how to build and maintain reliable, scalable, distributed systems with the Hadoop framework -- an open source implementation of MapReduce, the algorithm on which Google built its empire. Programmers will find details for analyzing datasets of any size, and administrators will learn how to set up and run Hadoop clusters.

This revised edition covers recent changes to Hadoop, including new features such as Hive, Sqoop, and Avro. It also provides illuminating case studies that illustrate how Hadoop is used to solve specific problems. Looking to get the most out of your data? This is your book.

  • Use the Hadoop Distributed File System (HDFS) for storing large datasets, then run distributed computations over those datasets with MapReduce
  • Become familiar with Hadoop’s data and I/O building blocks for compression, data integrity, serialization, and persistence
  • Discover common pitfalls and advanced features for writing real-world MapReduce programs
  • Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud
  • Use Pig, a high-level query language for large-scale data processing
  • Analyze datasets with Hive, Hadoop’s data warehousing system
  • Take advantage of HBase, Hadoop’s database for structured and semi-structured data
  • Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems

"Now you have the opportunity to learn about Hadoop from a master -- not only of the technology, but also of common sense and plain talk."

--Doug Cutting, Cloudera

 

Contents

Chapter 1 Meet Hadoop
1
Chapter 2 MapReduce
15
Chapter 3 The Hadoop Distributed Filesystem
41
Chapter 4 Hadoop IO
75
Chapter 5 Developing a MapReduce Application
129
Chapter 6 How MapReduce Works
167
Chapter 7 MapReduce Types and Formats
189
Chapter 8 MapReduce Features
225
Chapter 11 Pig
321
Chapter 12 Hive
365
Chapter 13 HBase
411
Chapter 14 ZooKeeper
441
Chapter 15 Sqoop
477
Chapter 16 Case Studies
497
Appendix A Installing Apache Hadoop
565
Appendix B Clouderas Distribution for Hadoop
571

Chapter 9 Setting Up a Hadoop Cluster
259
Chapter 10 Administering Hadoop
293
Appendix C Preparing the NCDC Weather Data
573

Common terms and phrases

Bibliographic information