Posts

Featured post

Analytics Today: Python Tuple Vs Lists

Image
Tuple Vs. List in Python is the most searchable word. Here, I have shared the differences between these two. 1. Python List It is simple to create a list in Python.  Comma-separated elements inside a square bracket [] make a list. The list can contain any number of elements of different data types (integer, float, character, etc.)  A list can contain another list. And it is called a nested list List Example #Illustration of creating a list  new_list=[1, 2, 3, 4]  print(new_list)  # Homogeneous data elements  new_list1=[1, "John", 55.5]  print(new_list1)  # Heterogeneous data elements  new_list2=[111, [1, "Clara", 75.5]]  # Nested list  print(new_list2) Output [1, 2, 3, 4] [1, ‘John’, 55.5] [111, [1, ‘Clara’, 75.5]] 2. Python Tuple It is simple to create a tuple in Python.  The elements of a list are enclosed in square brackets []. But these are put in parenthesis (), separated with commas.  A tuple can have any number of elements of heterogeneous type (integer, floa

How to Check Log File in Kafka

Image
In Kafka, the log file's function is to store entries. Here, you can find entries for the producer's incoming messages. These log files you can call as topics. Again topics are divided into partitions. IN THIS PAGE Kafka Logs How Producer Messages Store Benefits of Kafka Logs How to check Logs in Kafka 1. Kafka Logs The mechanism underlying Kafka is the log. Most software engineers are familiar with this. It tracks what an application is doing.  If you have performance issues or errors in your application, the first place to check is the application logs. But it is a different sort of log.  In the context of Kafka (or any other distributed system), a log is "an append-only, totally ordered sequence of records - ordered by time. 2. How Producer Messages Store The producer writes the messages to Kafka, the records store in a log file. The records sored as 0,1,2,3 and so on. 4. Benefits of Kafka Logs Logs are a simple data abstraction with powerful implications. If you have r

Kafka: How to Check Number of Brokers Present

Image
Here is the command on how to check the number of brokers present in a Kafka Cluster. The hear of Kafka cluster is a broker. In simple terms, you can say it as a process. That receives messages from publishers and gives permission to access messages by consumers. Broker Id The default broker id is -1 . When you create a new broker, it adds to -1. Then, it gives a new broker id. The broker ids will be generated from  reserved.broker.max.id + 1. According to Wiki: Kafka runs on a cluster of one or more servers (called brokers), and the partitions of all topics are distributed across the cluster nodes. Additionally, partitions are replicated to multiple brokers. This architecture allows Kafka to deliver massive streams of messages in a fault-tolerant fashion and has allowed it to replace some of the conventional messaging systems like Java Message Service (JMS), Advanced Message Queuing Protocol (AMQP), etc. Since the 0.11.0.0 release, Kafka offers transactional writes, which provide an

6 Top Takeaways from MapReduce Flowchart

Image
Take any large data there arise two problems. Firstly, to read it and secondly process it. Read it means traditionally whole file need to read once then divided manually but it is not convenient, with that respect Hadoop provide facility to read file automatically irrespective to its size. So whole file read data line by line that is by using offset and line value. Hadoop MapReduce Process Flowchart MapReduce Process MapReduce Process Six Top Functions Step 1 : Take the file as input for processing purpose. Any file will consist of group of lines. These lines containing key-value pair of data. Whole file can be read out with this method. Step 2: In next step file will be in "splitting" mode. This mode will divide file into key, value pair of data. This time key will be offset and data will be value part of program. Each line will be read individually so there is no need to split data manually. Step 3: Further step is to process the value of each line with associate from co

5 HBase Vs. RDBMS Top Functional Differences

Image
HBASE in the Big data context has a lot of benefits over RDBMS. Clear difference makes you feel great why you need HBASE in Hadoop or Bigdata platform. Let us check one by one quickly. HBASE Vs. RDBMS HBase handles a large amount of data that is store in a distributed manner in the column-oriented format while RDBMS is systematic storage of a database that cannot support a random manner for accessing the database. RDBMS strictly follow Codd's 12 rules with fixed schemas and row-oriented manner of database and also follow ACID properties, while HBase follows BASE properties and implement complex queries. Secondary indexes, complex inner and outer joins, count, sum, sort, group, and data of page and table can easily be accessible by RDBMS. From small to medium storage application there is the use of RDBMS that provide the solution with MySQL and PostgreSQL whose size increase with concurrency and performance. Codd's rules always need to keep in mind while extending the size of th

The Growth of Machine Learning till TensorFlow

Image
The Internet and the vast amount of data are inspiration CEOs of big corporations started to use Machine learning. It is to provide a better experience to users. Let us take Amazon, online retail that uses   Machine learning.  The algorithm's purpose is to generate revenue. Based on user search data, the ML application provides information or insights. The other example is the advertising platform where Google is a leader in this line. Where it shows ads based on the user movements while surfing the web. These are just a few, but there are many in reality. Machine Learning 1. Evolution of TensorFlow 2. Machine Learning Frameworks 1. Torch The torch is the first framework developed in 2002 by Ronan Collobert. Initially, IBM and Facebook have shown much interest. The interface language is Lua. The primary focus is matrix calculations. It is suitable for developing neural networks. 2. Theano It is developed in 2010 by the University of Montreal. It is highly reliable to process graphs

How to Use the ps Command in Linux

Image
The real use of the ps command is to know the process details. In fact, along with process details, you would know many other parameters about the process. How many processes are running in a current Session you will get from the below examples. ps command Linux ps Command in Linux PID (Process Id) - It is the process identification number. TTY(Terminal name) - associated with this process TIME(Time) - the format is hh: mm: ss (Hours, Minutes, and Seconds). Cumulated time of that particular process. CMD (Executable Command name) I will show you an example, which is very basic, how it appears when you issue the  ps command. ps command You May Also Like: PS Command Options in Linux What is Slave/Master Terminal in Linux The pts/0 is the slave terminal. The  pts  means pseudo terminal slave. Raise your knowledge by bringing in these quick definitions.  LINUX - DIFFERENCE BETWEEN TTY AND PTS. In reality, the terminals are two types. One is ptmx (Master) and pts (Slave). The master termi

Linux Relative Vs. Absolute Path Top Differences

Image
Linux directories are helpful to store files. Many a time, the programmer would trade in directory paths. The paths are two types - absolute and relative. I have shared simple ideas to remember the differences between these two. Linux Absolute Vs. Relative Path in Linux Below are the differences between these two. 1. Absolute pathname An example is /usr/lib, which is an exact directory in the directory tree.  Think of the absolute pathname as being the complete mailing address for a package that the postal service will deliver to your next-door neighbor. 2. Relative pathname An example is cups, which represents the cups subdirectory of the current directory, whatever that may be.  Think of the relative directory name as giving the postal carrier directions from your house to the one next door so that the carrier can deliver the package. Useful differences, how to differentiate the differences between absolute and real paths. How to go back to the home directory If I type  cd cups  in