[python] TypeError: unhashable type: 'list' when using built-in set function

I have a list containing multiple lists as its elements

eg: [[1,2,3,4],[4,5,6,7]]

If I use the built in set function to remove duplicates from this list, I get the error

TypeError: unhashable type: 'list'

The code I'm using is

TopP = sorted(set(TopP),reverse=True)

Where TopP is a list just like in the e.g. Above

Is this usage of set() wrong? Is there any other way in which I can sort the above list?

This question is related to python list duplicates

The answer is


Sets require their items to be hashable. Out of types predefined by Python only the immutable ones, such as strings, numbers, and tuples, are hashable. Mutable types, such as lists and dicts, are not hashable because a change of their contents would change the hash and break the lookup code.

Since you're sorting the list anyway, just place the duplicate removal after the list is already sorted. This is easy to implement, doesn't increase algorithmic complexity of the operation, and doesn't require changing sublists to tuples:

def uniq(lst):
    last = object()
    for item in lst:
        if item == last:
            continue
        yield item
        last = item

def sort_and_deduplicate(l):
    return list(uniq(sorted(l, reverse=True)))

Sets remove duplicate items. In order to do that, the item can't change while in the set. Lists can change after being created, and are termed 'mutable'. You cannot put mutable things in a set.

Lists have an unmutable equivalent, called a 'tuple'. This is how you would write a piece of code that took a list of lists, removed duplicate lists, then sorted it in reverse.

result = sorted(set(map(tuple, my_list)), reverse=True)

Additional note: If a tuple contains a list, the tuple is still considered mutable.

Some examples:

>>> hash( tuple() )
3527539
>>> hash( dict() )

Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    hash( dict() )
TypeError: unhashable type: 'dict'
>>> hash( list() )

Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    hash( list() )
TypeError: unhashable type: 'list'

Definitely not the ideal solution, but it's easier for me to understand if I convert the list into tuples and then sort it.

mylist = [[1,2,3,4],[4,5,6,7]]
mylist2 = []
for thing in mylist:
    thing = tuple(thing)
    mylist2.append(thing)
set(mylist2)

    python 3.2


    >>>> from itertools import chain
    >>>> eg=sorted(list(set(list(chain(*eg)))), reverse=True)
        [7, 6, 5, 4, 3, 2, 1]


   ##### eg contain 2 list within a list. so if you want to use set() function
   you should flatten the list like [1, 2, 3, 4, 4, 5, 6, 7]

   >>> res= list(chain(*eg))       # [1, 2, 3, 4, 4, 5, 6, 7]                   
   >>> res1= set(res)                    #   [1, 2, 3, 4, 5, 6, 7]
   >>> res1= sorted(res1,reverse=True)

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to list

Convert List to Pandas Dataframe Column Python find elements in one list that are not in the other Sorting a list with stream.sorted() in Java Python Loop: List Index Out of Range How to combine two lists in R How do I multiply each element in a list by a number? Save a list to a .txt file The most efficient way to remove first N elements in a list? TypeError: list indices must be integers or slices, not str Parse JSON String into List<string>

Examples related to duplicates

Remove duplicates from dataframe, based on two columns A,B, keeping row with max value in another column C Remove duplicates from a dataframe in PySpark How to "select distinct" across multiple data frame columns in pandas? How to find duplicate records in PostgreSQL Drop all duplicate rows across multiple columns in Python Pandas Left Join without duplicate rows from left table Finding duplicate integers in an array and display how many times they occurred How do I use SELECT GROUP BY in DataTable.Select(Expression)? How to delete duplicate rows in SQL Server? Python copy files to a new directory and rename if file name already exists