Styjun

I am trying to get all the similar sounding words from a list.

I tried to get them using cosine similarity but that does not fulfil my purpose.

from sklearn.metrics.pairwise import cosine_similarity
dataList = ['two','fourth','forth','dessert','to','desert']
cosine_similarity(dataList)

I know this is not the right approach, I cannot seem to get a result like:

result = ['xx', 'xx', 'yy', 'yy', 'zz', 'zz'] 

where they mean that the words which sound similar

First, you need to use a right way to get the similar sounding words i.e. string similarity, I would suggest:

Using jellyfish:

from jellyfish import soundex

print(soundex("two"))
print(soundex("to"))

OUTPUT:

T000
T000

Now perhaps, create a function that would handle the list and then sort it to get them:

def getSoundexList(dList):
 res = [soundex(x) for x in dList] # iterate over each elem in the dataList
 # print(res) # ['T000', 'F630', 'F630', 'D263', 'T000', 'D263']
 return res

dataList = ['two','fourth','forth','dessert','to','desert'] 
print([x for x in sorted(getSoundexList(dataList))])

OUTPUT:

['D263', 'D263', 'F630', 'F630', 'T000', 'T000']

EDIT:

Another way could be:

Using fuzzy:

import fuzzy
soundex = fuzzy.Soundex(4)

print(soundex("to"))
print(soundex("two"))

OUTPUT:

T000
T000

EDIT 2:

If you want them grouped, you could use groupby:

from itertools import groupby

def getSoundexList(dList):
 return sorted([soundex(x) for x in dList])

dataList = ['two','fourth','forth','dessert','to','desert'] 
print([list(g) for _, g in groupby(getSoundexList(dataList), lambda x: x)])

OUTPUT:

[['D263', 'D263'], ['F630', 'F630'], ['T000', 'T000']]

EDIT 3:

This ones for @Eric Duminil, let's say you want both the names and their respective val:

Using a dict along with itemgetter:

from operator import itemgetter

def getSoundexDict(dList):
 return sorted(dict_.items(), key=itemgetter(1)) # sorting the dict_ on val

dataList = ['two','fourth','forth','dessert','to','desert']
res = [soundex(x) for x in dataList] # to get the val for each elem
dict_ = dict(list(zip(dataList, res))) # dict_ with k,v as name/val

print([list(g) for _, g in groupby(getSoundexDict(dataList), lambda x: x[1])])

OUTPUT:

[[('dessert', 'D263'), ('desert', 'D263')], [('fourth', 'F630'), ('forth', 'F630')], [('two', 'T000'), ('to', 'T000')]]

EDIT 4 (for OP):

Soundex:

Soundex is a system whereby values are assigned to names in such a
manner that similar-sounding names get the same value. These values
are known as soundex encodings. A search application based on soundex
will not search for a name directly but rather will search for the
soundex encoding. By doing so, it will obtain all names that sound
like the name being sought.

read more..