Posts

Showing posts with the label hadoop cluster setup

Featured Post

Step-by-Step Guide to Reading Different Files in Python

Image
 In the world of data science, automation, and general programming, working with files is unavoidable. Whether you’re dealing with CSV reports, JSON APIs, Excel sheets, or text logs, Python provides rich and easy-to-use libraries for reading different file formats. In this guide, we’ll explore how to read different files in Python , with code examples and best practices. 1. Reading Text Files ( .txt ) Text files are the simplest form of files. Python’s built-in open() function handles them effortlessly. Example: # Open and read a text file with open ( "sample.txt" , "r" ) as file: content = file.read() print (content) Explanation: "r" mode means read . with open() automatically closes the file when done. Best Practice: Always use with to handle files to avoid memory leaks. 2. Reading CSV Files ( .csv ) CSV files are widely used for storing tabular data. Python has a built-in csv module and a powerful pandas library. Using cs...

How to Setup Hadoop Cluster Top Ideas

Image
Hadoop cluster setup in Centos Operating System explained in this post. So you can install CentOs either in your Laptop or in Virtual Machine. Hadoop Cluster Setup Process 9 Steps Process to Setup Hadoop Cluster Step 1:  Installing Sun Java on Linux. Commands to execute for the same: sudo apt-add-repository ppa:flexiondotorg/java sudo apt-get update sudo apt-get install sun-java6-jre sun-java6-plugin sudo update-java-alternatives -s java-6-sun Step 2:  Create Hadoop User. Commands to execute for the same: $sudo addgroup hadoop $sudo adduser —ingroup hadoop hduser Step 3:  Install SSH Server if not already present. Commands are: $ sudo apt-get install openssh-server $ su - hduser $ ssh-keygen -t rsa -P "" $ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys Step 4:  Installing Hadoop. Commands for the same are: $wget http://www.eng.lsu.edu/mirrors/apache/hadoop/core/hadoop-0.22.0/hadoop-0.22.0.tar.gz $ cd /home/hduser $ tar xzf ...