Featured Post

How to Check Column Nulls and Replace: Pandas

Here is a post that shows how to count Nulls and replace them with the value you want in the Pandas Dataframe. We have explained the process in two steps - Counting and Replacing the Null values. Count null values (column-wise) in Pandas ## count null values column-wise null_counts = df.isnull(). sum() print(null_counts) ``` Output: ``` Column1    1 Column2    1 Column3    5 dtype: int64 ``` In the above code, we first create a sample Pandas DataFrame `df` with some null values. Then, we use the `isnull()` function to create a DataFrame of the same shape as `df`, where each element is a boolean value indicating whether that element is null or not. Finally, we use the `sum()` function to count the number of null values in each column of the resulting DataFrame. The output shows the count of null values column-wise. to count null values column-wise: ``` df.isnull().sum() ``` ##Code snippet to count null values row-wise: ``` df.isnull().sum(axis=1) ``` In the above code, `df` is the Panda

Sqoop Real Use in Hadoop Framework

Why Sqoop you need while working on Hadoop-The Sqoop and its primary reason is to import data from structural data sources such as Oracle/DB2 into HDFS(also called Hadoop file system).

To our readers, I have collected a good video from Edureka which helps you to understand the functionality of Sqoop.

The comparison between Sqoop and Flume


How name come for Sqoop

Sqoop word came from SQL+HADOOP=SQOOP. And Sqoop is a data transfer tool. The main use of Sqoop is to import and export a large amount of data from RDBMS to HDFS and vice versa.

List of basic Sqoop commands

  1. Codegen- It helps to generate code to interact with database records.
  2. Create-hive-table- It helps to Import a table definition into a hive
  3. Eval- It helps to evaluate SQL statement and display the results
  4. Export-It helps to export an HDFS directory into a database table
  5. Help- It helps to list the available commands
  6. Import- It helps to import a table from a database to HDFS
  7. Import-all-tables- It helps to import tables from a database to HDFS
  8. List-databases- It helps to list available databases on a server
  9. List-tables-It helps to list tables in a database
  10. Version-It helps to display the version information


Popular posts from this blog

Explained Ideal Structure of Python Class

How to Check Kafka Available Brokers

6 Python file Methods Real Usage