Python: How to Remove Duplicates From the List

In this post, I have shared top ideas to remove duplicates from the list. Those are with Append and Dictionary.


Python: How to Remove Duplicates From List

1. How to Remove Duplicates Using Append?

# Here is a list with duplicates

list_with_duplicates = [1,2,3,12,1,2,3,4,5,6,1,2,3,7,8,9]

It is simple if you follow the first-approach - brute force approach:

list_without_duplicates = []

for pd in list_with_duplicates:
  if pd not in list_without_duplicates:
      list_without_duplicates.append(pd)
print(list_without_duplicates)

Result:

[1, 2, 3, 12, 4, 5, 6, 7, 8, 9]

This method has performance issues when the list is bigger in size. 

Real-time.

Idea 1:  Remove Duplicates Using Append.



2. How to Remove Duplicates Using Dictionary?


# Here is you can convert a list to a dictionary

dict_without_duplicates = dict(zip(list_with_duplicates, list_with_duplicates))
print(dictionary_without_duplicates)

Result:

{1: 1, 2: 2, 3: 3, 12: 12, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9}


Real-time.

Idea 2: Remove Duplicates Using Dictionary.


Once again, this works and has the advantage of taking less space than duplicating the entire list. 


*Notes: Of course, we still need to convert it back to a list when we did, which will be somewhat painful since we must extract the keys and add them to a list.

Comments

Popular Posts

Hyperledger Fabric: 20 Real Interview Questions

7 AWS Interview Questions asked in Infosys, TCS

How to Fix Python Syntax Errors Quickly

Python 'getsizeof' Command the Real Purpose

Blue Prism complete tutorials download now

Vi Editor to Quit use Esc and Colon

Python Dictionary Vs List With Examples

How to Use the ps Command in Linux

AWS Vs Azure Load Balancers Top Insights

How to Decode TLV Quickly