Featured Post

Best Practices for Handling Duplicate Elements in Python Lists

Image
Here are three awesome ways that you can use to remove duplicates in a list. These are helpful in resolving your data analytics solutions.  01. Using a Set Convert the list into a set , which automatically removes duplicates due to its unique element nature, and then convert the set back to a list. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = list(set(original_list)) 02. Using a Loop Iterate through the original list and append elements to a new list only if they haven't been added before. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = [] for item in original_list:     if item not in unique_list:         unique_list.append(item) 03. Using List Comprehension Create a new list using a list comprehension that includes only the elements not already present in the new list. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = [] [unique_list.append(item) for item in original_list if item not in unique_list] All three methods will result in uni

How to Identify Data Relevant for Data Science Analytics

Your government, your web server, your business partners, even your body. While we aren’t drowning in a sea of data, we’re finding that almost everything can (or has) been instrumented. We frequently combine publishing industry data from Nielsen Book Scan with our own sales data, publicly available Amazon data, and even job data to see what’s happening in the publishing industry.

Data is everywhere
Sites like Infochimps and Factual provide access to many large datasets, including climate data, MySpace activity streams, and game logs from sporting events. Factual enlists users to update and improve its datasets, which cover topics as diverse as endocrinologists to hiking trails.

How the data is growing

Much of the data we currently work with is the direct consequence of Web 2.0, and of Moore’s Law applied to data. The Web has people spending more time online and leaving a trail of data wherever they go. Mobile applications leave an even richer data trail since many of them are annotated with geolocation, or involve video or audio, all of which can be mined.

Point-of-sale devices and frequent shoppers cards make it possible to capture all of your retail transactions, not just the ones you make online. All of this data would be useless if we couldn’t store it, and that’s where Moore’s Law comes in. Since the early ’80s, processor speed has increased from 10 MHz to 3.6 GHz—an increase of 360 (not counting increases in word length and number of cores).

The need for Storage capacity

But we’ve seen much bigger increases in storage capacity, on every level. RAM has moved from $1,000/MB to roughly $25/GB—a price reduction of about 40000, to say nothing of the reduction in size and increase in speed. Hitachi made the first-gigabyte disk drives in 1982, weighing in at roughly 250 pounds; now terabyte drives are consumer equipment, and a 32 GB microSD card weighs about half a gram. Whether you look at bits per gram, bits per dollar, or raw capacity, storage has more than kept pace with the increase of CPU speed.

Comments

Popular posts from this blog

Explained Ideal Structure of Python Class

6 Python file Methods Real Usage

How to Decode TLV Quickly