Hyderabad Based Startup Built Largest Ever Big data Electoral Repository

-
I was gone through an email from my friend saying that they are creating a Hadoop project to analyze voters data. This project in my view is both academic and research oriented.
hadoop project
The real challenge was extraction of voter info from 2.5 crore PDF pages and translation of the same into English to fuse with other sources. The technology was a big hurdle. 

Hadoop Project

The infrastructure, built especially for the project, included 64 node Hadoop, PostgreSQL and servers that process a master file containing over 8 Terabytes of Data.

Besides, Testing and Validation was another big task. ‘First of a Kind’ Heuristic (machine learning) algorithms were developed for people classification based on name, geography etc., which help in the identification of religion, caste, and even ethnicity.

Data from Sources

“Data from multiple sources like census, economic and social surveys were mapped to polling booths. Simultaneously, external and propriety data sources had to be fused with individual voters’ data,” informed Joshi. 



Also Read

Comments

Post a Comment

Thanks for your message. We will get back you.

Popular Posts

Hyperledger Fabric: 20 Real Interview Questions

-
Image
I am practicing Hyperledger. This is one of the top listed blockchains. This architecture follows R3 Corda specifications. Sharing the interview questions with you that I have prepared for my interview. Though Ethereum leads in the real-time applications. The latest Hyperledger version is now ready for production applications. It has now become stable for production applications. The Hyperledger now backed by IBM. But, it is still an open source. These interview questions help you to read quickly. The below set of interview questions help you like a tutorial on Hyperledger fabric. Hyperledger Fabric Interview Questions 1). What are Nodes? In Hyperledger the communication entities are called Nodes. 2). What are the three different types of Nodes? - Client Node - Peer Node - Order Node The Client node initiates transactions. The peer node commits the transaction. The order node guarantees the delivery. 3). What is Channel? A channel in Hyperledger is the subnet of
Read more

7 AWS Interview Questions asked in Infosys, TCS

-
Image
I have shared 7 amazing AWS interview questions for your quick reference. 1). What is AWS? Amazon Web Services (AWS) provides on-demand computing resources and services in the cloud, with pay-as-you-go pricing. For example, you can run a server on AWS that you can log on to, configure, secure, and run just as you would a server that's sitting in front of you 2). What you can do with AWS? Store public or private data. Host a static website. These websites use client-side technologies (such as HTML, CSS, and JavaScript) to display content that doesn't change frequently. A static website doesn't require server-side technologies (such as PHP and ASP.NET). Host a dynamic website or web app. These websites include classic three-tier applications, with web, application, and database tiers. Support students or online training programs. Process business and scientific data. Handle peak loads. AWS Questions 3). What is the Security group? A security group acts as a virtual fi
Read more

How to Fix Python Syntax Errors Quickly

-
Image
In Python, you can call parsing errors Syntax errors. How to fix the syntax errors quickly, I can show in this post. Also, check Python Syntax Errors Cheat Sheet. Python Syntax Error. If you run the Python code, the interpreter first parses the logic. If any errors found, it is the developer's responsibility to fix them. Courtesy: Python documentation. >>> while True print('Hello world')   File "<stdin>", line 1     while True print('Hello world')                    ^ SyntaxError: invalid syntax How to Fix the Syntax Error First, check for the line number. In the above case, it is '1'. Then look for the symbol '^'. The issue is at the PRINT function. The reason is before the PRINT function you need a colon ':'. That is the reason, you are getting error. This is a syntax error. Find below the corrected logic: >>> while True : print('Hello world')   File "<stdin>", line 1     while Tr
Read more

Python 'getsizeof' Command the Real Purpose

-
Image
In Python, there are many objects. Out of those List (arrays) is one type.  To get the number of elements present in a List is not easy. So how to get this? To do you need the 'getsizeof' command. Import 'sys' Library. You need to import the 'sys' library to use the SIZE commands.  >>> import sys Congrats to you since you have now imported the SYS library. The command name is " getsizeof" . Check the below example. print("An Integer:") i = int(100) print(sys.getsizeof(i)) The result will be: 100 What is the size command?  - is to get the size of elements. It gives the result based on the object you are being used. Suppose, if you use Strings, you will get String length. How to use the  'getsizeof' command? Example:1 print("Empty string:") empty_string = "" print(sys.getsizeof(empty_string)) Example:2 print("A normal string") s = "Hello world" print(sys.getsizeof(s)) Example:3 print(&quo
Read more

Blue Prism complete tutorials download now

-
Image
Blue Prism is an automation tool useful to execute repetitive tasks without human effort. To learn this tool you need the right material. Provided below quick reference materials to understand detailed elements, architecture, and creating new bots . Useful if you are a new learner and trying to enter into an automation career. The number one and most popular tool in automation is a Blue prism. In this post, I have given references for popular materials and resources so that you can use for your interviews. RPA Blue Prism RPA blue prism tutorial popular resources I have given in this post. You can download it quickly. Learning Blue Prism is a really good option if you are a learner of Robotic process automation . RPA Advantages The RPA is also called "Robotic Process Automation"- Real advantages are you can automate any business process and you can complete the customer requests in less time. The Books Available on Blue Prism  Blue Prism resources David chap
Read more

Vi Editor to Quit use Esc and Colon

-
Image
Here are vi editor commands you can use to quit from Vi editor with or without saving your work. These are useful to use in day-to-day work. w => To Save Your Work q => To Quit wq => To Save and Quit q! => To quit without saving vi editor also called Visual editor in terms of Unix or Linux or Ubuntu Operating systems. Since all are the same UNIX flavor, and UNIX is the mother of all these operating systems. Recently many of my friends asked to share some daily use vi editor commands . I am sharing those for your quick reference. 6 Top vi Editor Commands 1. How to Edit a file? vi filename This is the first command to use for editing a file. 2. A command to Repeat Previous Command? Use the  UP arrow to repeat the previous command. 3. How to Insert data? Use insert command to over-type data. 4. How to Save File? Press Esc then :w 5. How to Save and Quit (Exit)? Press Esc then :wq 6 How to quit without Save? Press Esc then :q!
Read more

Python Dictionary Vs List With Examples

-
Image
Dictionary and List we use interchangeably in Python to store values. For beginners, both look the same. In reality, they both differ. I want to share some key differences between Python Dictionary and List . Python Dictionary Vs List Top Differences. Values in lists are accessed by means of integers called indices, which indicate where in the list a given value is found. Dictionaries access values by means of integers, strings, or other Python objects called keys , which indicate where in the dictionary a given value is found.  In other words, both lists and dictionaries provide indexed access to arbitrary values, but the set of items that can be used as dictionary indices is much larger than, and contains, the set of items that can be used as list indices.  Also, the mechanism that dictionaries use to provide indexed access is quite different from that used by lists. Both lists and dictionaries can store objects of any type. Values stored in a list are implicitly ordered by their p
Read more

AWS Vs Azure Load Balancers Top Insights

-
Image
In AWS, you have three types of load-balancers. In the case of Azure, you have only two load balancers. This post tells you the comparison between these two. 1. AWS Load Balancers: Application Load balancer Network Load balancer Classic Load balancers. a. Application Load Balancer Balancing of HTTP and HTTPS traffic and provides advanced request routing targeted at the delivery of modern application architectures, including micro-services and containers.  Operating at the individual request level (Layer 7), Application Load Balancer routes traffic to targets within Amazon Virtual Private Cloud (Amazon VPC) based on the content of the request. b. Network Load Balancer Network Load Balancer is best suited for load balancing of Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Transport Layer Security (TLS) traffic where extreme performance is required. c. Classic Load Balancer The Classic Load Balancer provides the basic load-balancing across multiple Amazon EC2 inst
Read more

How to Decode TLV Quickly

-
Image
In TLV, the format is Tag, Length, and Value. The TLV protocol needs this type of data. Here you will know how to decode TLV data.  According to IBM , TLV data is three parts. The tag tells what type of data it is. The length field denotes the length of the value. The Value-field denotes the actual value. Structure of TLV. TLV comprises three field values.  Tag Length Value EMV formulated different tags. They have their meanings. Usually, the Tag and Length together takes 1 to 4 bytes. The Best example for TLV. In the below example, you can find the sample TAG, LENGTH, and VALUE fields. [Tag][Value Length][Value] (ex. " 9F4005F000F0A001 ") where Tag Name =  9F40 Value Length (in bytes) =  05  Value (Hex representation of bytes. Example, "F0" – 1-byte) =  F000F0A001 In the above message, tag 9F40 has some meaning designed by EMV company. Here  you can find a list of EMV Tags. How to read the TLV Tag: 1 or 2 bytes Length: Length of the Value. F0-00-F0-A0-01 ==> 5 B
Read more

Tokenization: Vault based Vs Vault-less

-
Image
Tokenization means generating a random number as an alternative for a Credit card (PAN) number. The random number can be a Numeric/Non-numeric. It is in such a format that others cannot decode it.    *** The popular tokenization server is the Vault server. It generates a random Token for each transaction customer made at Merchant outlets. You can find how card numbers masked with junk characters in the pdf provided. Here is the  Visa Security Tokenization Pdf. Two Top Tokenization Servers. Vault based server Vault-less server What is Tokenization? 1. Vault based server It stores a  Card number (PAN) and a Token  in a Table of the Teradata database as Key and value pair. Due to the growing volume of transactions, the handling of the database became a big challenge for acquirers.  For each transaction, the vault server stores a record. That row contains both the Credit card number and its Token. When you use the same card multiple times, it generates different Tokens.  It is th
Read more