Remove duplicate words from a string in Python

In this tutorial, we are going to learn about how to Remove duplicate words from a string in Python along with some examples.

What are duplicate words/strings in a string?

• Duplicate words/strings are similar words in a sentence or in a string.

Example: “powerful people come from powerful places”

So here powerful is the duplicate word in our sentence and we have to remove this one “powerful” from our string/sentence using Python.

output: powerful people come from places

Let’s start coding

There are many ways to find duplicate words in a string but here only we discuss two of them

1. Using for loop to delete duplicate words from a string

• 1^st we will define a string

string=“powerful people come from powerful places”

Remember Python is case sensitive, if you use “P” capital in 1^st powerful and “p” small in 2^nd powerful then Python takes them as a different word, not a similar word.

• Then we will use an inbuilt function split() which split our string into a list where each word is a list item and we will store this item into a variable “i”.
• Then we will create an empty list “e” in which we will store those words that are not duplicated.
• Then we create or will run a for loop through the list items and then an if statement in which we use a
  1. “count()” function in our string which will return the number of elements that were there in our sentence.
  2. “(i not in e)” – This will check whether “i”(Have a list item from a list “l”) in “e”(new list “e”) or not and return a respective value True or False.
  3. The conditional statement “and” will return a True if both conditions are True or False if one of the conditions is False.

If both conditions are true then our if statement will execute and we will append that particular list item “i”(Have a list item from list “l”) to our new list “e” which is not a duplicate word for “e”.

This for loop and if statement will filter those duplicate word

• At last, we use “ ‘ ‘.join(d)” The .join() method- Join all items in our list into a string followed by a or separated by a space “ ”.

And the last line code prints our new string or words which will not contain any duplicate words.

 

string="powerful people come from powerful places”
l=string.split()
e=[]
for i in l:
    if (string.count(i)>=1 and (i not in e)):
        e.append(i)
print(' '.join(e))

Output:

powerful people come from places

2. Using a set to remove duplicate words

Set is an inbuilt data type in python which used to store multiple items in a single variable.

Set not allow any repetition value or items in it. This means sets cannot store two items with the same value.

If we have a duplicate item in our list then when we convert that list into a set then at the time set removes all those duplicate items that are present in our list.

So basically we will use this concept to remove duplicate words in a string using Python.

• 1^st two lines of code are the same
• In the 3^rd line, we will create a set “s” of our list “l” using the set() in-built function and this line of code removes all the duplicate items from our list.
• At last, we will print that set “s” using print(“ ”.join(s)) to get a desirable output.

string="powerful people come from powerful places”
l=string.split()
s=set(l)
print(' '.join(s))

Output:

people powerful from places come

Thus, we have learned how to Remove duplicate words from a string in Python.