Apache Yarn to Manage Resources a Solution

Apache Hadoop is one of the most popular tools for big data processing. It has been successfully deployed in production by many companies for several years. 

Though Hadoop is considered a reliable, scalable, and cost-effective solution, it is constantly being improved by a large community of developers. As a result, the 2.0 version offers several revolutionary features, including Yet Another Resource Negotiator (YARN), HDFS Federation, and a highly available NameNode, which make the Hadoop cluster much more efficient, powerful, and reliable. 

Apache Yarn

Apache Hadoop 2.0 includes YARN, which separates the resource management and processing components. The YARN-based architecture is not constrained to MapReduce.
  • New developmens in Hadoop 2.0 Architecture with YARN: 
  • ResourceManager instead of a cluster manager 
  • ApplicationMaster instead of a dedicated and short-lived JobTracker 
  • NodeManager instead of TaskTracker 
  • A distributed application instead of a MapReduce job 

Basic changes in Hadoop 2.0 architecture

  • The ResourceManager, the NodeManager, and a container are not concerned about the type of application or task.
  • All application framework-specific code is simply moved to its ApplicationMaster so that any distributed framework can be supported by YARN — as long as someone implements an appropriate ApplicationMaster for it.
  • Thanks to this generic approach, the dream of a Hadoop YARN cluster running many various workloads comes true. Imagine: a single Hadoop cluster in your data center that can run MapReduce, Giraph, Storm, Spark, Tez/Impala, MPI, and more.

Read More

Comments

Post a Comment

Thanks for your message. We will get back you.

Popular posts from this blog

How to Fix Python Syntax Errors Quickly

Image
In Python, you can call parsing errors Syntax errors. How to fix the syntax errors quickly, I can show in this post. Also, check Python Syntax Errors Cheat Sheet. Python Syntax Error. If you run the Python code, the interpreter first parses the logic. If any errors are found, it is the developer's responsibility to fix them. Courtesy: Python documentation. >>> while True print ( 'Hello world' ) File "<stdin>" , line 1 while True print ( 'Hello world' ) ^ SyntaxError : invalid syntax How to Fix the Syntax Error First, check for the line number. In the above case, it is '1'. Then look for the symbol '^'. The issue is at the PRINT function. The reason is before the PRINT function you need a colon ':'. That is the reason, you are getting errors. This is a syntax error. Find below the corrected logic: >>> while True : print ( 'Hello world' ) File "<
Read more

7 AWS Interview Questions asked in Infosys, TCS

Image
I have shared 7 amazing AWS interview questions for your quick reference. 1). What is AWS? Amazon Web Services (AWS) provides on-demand computing resources and services in the cloud, with pay-as-you-go pricing. For example, you can run a server on AWS that you can log on to, configure, secure, and run just as you would a server that's sitting in front of you 2). What you can do with AWS? Store public or private data. Host a static website. These websites use client-side technologies (such as HTML, CSS, and JavaScript) to display content that doesn't change frequently. A static website doesn't require server-side technologies (such as PHP and ASP.NET). Host a dynamic website or web app. These websites include classic three-tier applications, with web, application, and database tiers. Support students or online training programs. Process business and scientific data. Handle peak loads. AWS Questions 3). What is the Security group? A security group acts as a virtual fi
Read more

Hyperledger Fabric: 20 Real Interview Questions

Image
I am practicing Hyperledger. This is one of the top listed blockchains. This architecture follows R3 Corda specifications. Sharing the interview questions with you that I have prepared for my interview. Though Ethereum leads in the real-time applications. The latest Hyperledger version is now ready for production applications. It has now become stable for production applications. The Hyperledger now backed by IBM. But, it is still an open source. These interview questions help you to read quickly. The below set of interview questions help you like a tutorial on Hyperledger fabric. Hyperledger Fabric Interview Questions 1). What are Nodes? In Hyperledger the communication entities are called Nodes. 2). What are the three different types of Nodes? - Client Node - Peer Node - Order Node The Client node initiates transactions. The peer node commits the transaction. The order node guarantees the delivery. 3). What is Channel? A channel in Hyperledger is the subnet of
Read more