How to remove duplicates from Python list and keep order

Question

Given a list of strings  I want to sort it alphabetically and remove duplicates  I know I can do this   from sets import Set       myHash   Set myList    but I don t know how to retrieve the list members from the hash in alphabetical order   I m not married to the hash  so any way to accomplish this will work  Also  performance is not an issue  so I d prefer a solution that is expressed in code clearly to a fast but more opaque one

User · Answer

If it s clarity you re after  rather than speed  I think this is very clear   def sortAndUniq input     output        for x in input      if x not in output        output append x    output sort     return output   It s O n 2  though  with the repeated use of not in for each element of the input list

User · Answer

For the string data   output            def uniq input            if input not in output              output append input   print output

User · Answer

but I don t know how to retrieve the list members from the hash in alphabetical order   Not really your main question  but for future reference Rod s answer using sorted can be used for traversing a dict s keys in sorted order   for key in sorted my dict keys        print key  my dict key           and also because tuple s are ordered by the first member of the tuple  you can do the same with items   for key  val in sorted my dict items         print key  val

User · Answer

If your input is already sorted  then there may be a simpler way to do it   from operator import itemgetter from itertools import groupby unique list   list map itemgetter 0   groupby yourList

User · Answer

but I don t know how to retrieve the list members from the hash in alphabetical order   Not really your main question  but for future reference Rod s answer using sorted can be used for traversing a dict s keys in sorted order   for key in sorted my dict keys        print key  my dict key           and also because tuple s are ordered by the first member of the tuple  you can do the same with items   for key  val in sorted my dict items         print key  val

User · Answer

but I don t know how to retrieve the list members from the hash in alphabetical order   Not really your main question  but for future reference Rod s answer using sorted can be used for traversing a dict s keys in sorted order   for key in sorted my dict keys        print key  my dict key           and also because tuple s are ordered by the first member of the tuple  you can do the same with items   for key  val in sorted my dict items         print key  val

User · Answer

If it s clarity you re after  rather than speed  I think this is very clear   def sortAndUniq input     output        for x in input      if x not in output        output append x    output sort     return output   It s O n 2  though  with the repeated use of not in for each element of the input list

User · Answer

If your input is already sorted  then there may be a simpler way to do it   from operator import itemgetter from itertools import groupby unique list   list map itemgetter 0   groupby yourList

User · Answer

If your input is already sorted  then there may be a simpler way to do it   from operator import itemgetter from itertools import groupby unique list   list map itemgetter 0   groupby yourList

User · Answer

but I don t know how to retrieve the list members from the hash in alphabetical order   Not really your main question  but for future reference Rod s answer using sorted can be used for traversing a dict s keys in sorted order   for key in sorted my dict keys        print key  my dict key           and also because tuple s are ordered by the first member of the tuple  you can do the same with items   for key  val in sorted my dict items         print key  val

User · Answer

If you want to keep order of the original list  just use OrderedDict with None as values   In Python2       from collections import OrderedDict     from itertools import izip  repeat      unique list   list OrderedDict izip my list  repeat None       In Python3 it s even simpler       from collections import OrderedDict     from itertools import repeat      unique list   list OrderedDict zip my list  repeat None       If you don t like iterators  zip and repeat  you can use a generator  works both in 2  amp  3        from collections import OrderedDict     unique list   list OrderedDict  element  None  for element in my list

User · Answer

A list can be sorted and deduplicated using built-in functions   myList   sorted set myList      set is a built-in function for Python    2 3 sorted is a built-in function for Python    2 4

User · Answer

If you want to keep order of the original list  just use OrderedDict with None as values   In Python2       from collections import OrderedDict     from itertools import izip  repeat      unique list   list OrderedDict izip my list  repeat None       In Python3 it s even simpler       from collections import OrderedDict     from itertools import repeat      unique list   list OrderedDict zip my list  repeat None       If you don t like iterators  zip and repeat  you can use a generator  works both in 2  amp  3        from collections import OrderedDict     unique list   list OrderedDict  element  None  for element in my list

User · Answer

A list can be sorted and deduplicated using built-in functions   myList   sorted set myList      set is a built-in function for Python    2 3 sorted is a built-in function for Python    2 4

User · Answer

If it s clarity you re after  rather than speed  I think this is very clear   def sortAndUniq input     output        for x in input      if x not in output        output append x    output sort     return output   It s O n 2  though  with the repeated use of not in for each element of the input list

User · Answer

A list can be sorted and deduplicated using built-in functions   myList   sorted set myList      set is a built-in function for Python    2 3 sorted is a built-in function for Python    2 4

User · Answer

If your input is already sorted  then there may be a simpler way to do it   from operator import itemgetter from itertools import groupby unique list   list map itemgetter 0   groupby yourList

User · Answer

For the string data   output            def uniq input            if input not in output              output append input   print output

User · Answer

A list can be sorted and deduplicated using built-in functions   myList   sorted set myList      set is a built-in function for Python    2 3 sorted is a built-in function for Python    2 4

User · Answer

If it s clarity you re after  rather than speed  I think this is very clear   def sortAndUniq input     output        for x in input      if x not in output        output append x    output sort     return output   It s O n 2  though  with the repeated use of not in for each element of the input list

[python] How to remove duplicates from Python list and keep order?

Examples related to python

Examples related to list

Examples related to sorting

Examples related to unique