Skip to main content

Featured post

12 DevOps principles read today

DevOps means a concept to bring Development team and operation team together. So that you can speed up the deployment process. Before you start learning the tools you need to understand core principles.
The useful DevOps Principles to know for your project and interviewsTo deliver rapidly without affecting qualityCommunication and Collaboration are the key ideas in DevOps conceptMultiple deploys are possible- if code in Development team automatedOnce you commit the repository, it tests the code automatically against the automated test scriptsIf Build is successfully passed, it installs automatically in Testing environment When infrastructure is automated, it installs automatically to other serversMinor changes takes place in isolation, that means , it creates separate server to deploy minor changesSpeed in Devops, organizations to better serve their customers and compete more effectively in the market.Quality and Security teams are part of DevOpsAutomating the process, much better pro…

The best 5 differences of AWS EMR and Hadoop

With Amazon Elastic MapReduce (Amazon EMR) you can analyze and process vast amounts of data. It does this by distributing the computational work across a cluster of virtual servers running in the Amazon cloud. The cluster is managed using an open-source framework called Hadoop.
With Amazon Elastic MapReduce (Amazon EMR) you can analyze and process vast amounts of data.

Amazon EMR - Elastic MapReduce, The Unique Features


Amazon EMR has made enhancements to Hadoop and other open-source applications to work seamlessly with AWS.

For example, Hadoop clusters running on Amazon EMR use EC2 instances as virtual Linux servers for the master and slave nodes, Amazon S3 for bulk storage of input and output data, and CloudWatch to monitor cluster performance and raise alarms.

You can also move data into and out of DynamoDB using Amazon EMR and Hive.

All of this is orchestrated by Amazon EMR control software that launches and manages the Hadoop cluster. This process is called an Amazon EMR cluster.

What does Hadoop do...

Hadoop uses a distributed processing architecture called MapReduce in which a task is mapped to a set of servers for processing.

The results of the computation performed by those servers is then reduced down to a single output set.

One node, designated as the master node, controls the distribution of tasks. The following diagram shows a Hadoop cluster with the master node directing a group of slave nodes which process the data.

One Master node handles multiple slave nodes.
AWS Cloud Formation
All open-source projects that run on top of the Hadoop architecture can also be run on Amazon EMR. The most popular applications, such as Hive, Pig, HBase, DistCp, and Ganglia, are already integrated with Amazon EMR.
By running Hadoop on Amazon EMR you will get the following benefits of the cloud:
  1. The ability to provision clusters of virtual servers within minutes.
  2. You can scale the number of virtual servers in your cluster to manage your computation needs, and only pay for what you use. 
  3. Integration with other AWS services.

Comments

Most Viewed

Hyperledger Fabric Real Interview Questions Read Today

I am practicing Hyperledger. This is one of the top listed blockchains. This architecture follows R3 Corda specifications. Sharing the interview questions with you that I have prepared for my interview.

Though Ethereum leads in the real-time applications. The latest Hyperledger version is now ready for production applications. It has now become stable for production applications.
The Hyperledger now backed by IBM. But, it is still an open source. These interview questions help you to read quickly. The below set of interview questions help you like a tutorial on Hyperledger fabric. Hyperledger Fabric Interview Questions1). What are Nodes?
In Hyperledger the communication entities are called Nodes.

2). What are the three different types of Nodes?
- Client Node
- Peer Node
- Order Node
The Client node initiates transactions. The peer node commits the transaction. The order node guarantees the delivery.

3). What is Channel?
A channel in Hyperledger is the subnet of the main blockchain. You c…