Get difference between two lists

Question

I have two lists in Python  like these   temp1     One    Two    Three    Four   temp2     One    Two     I need to create a third list with items from the first list which aren t present in the second one  From the example I have to get   temp3     Three    Four     Are there any fast ways without cycles and checking

User · Answer

most simple way   use set   difference set     list a    1 2 3  list b    2 3  print set list a  difference set list b     answer is set  1    can print as a list   print list set list a  difference set list b

User · Answer

i ll toss in since none of the present solutions yield a tuple   temp3   tuple set temp1  - set temp2     alternatively    edited using  Mark Byers idea  If you accept this one as answer  just accept his instead  temp3   tuple x for x in temp1 if x not in set temp2     Like the other non-tuple yielding answers in this direction  it preserves order

User · Answer

Try this   temp3   set temp1  - set temp2

User · Answer

The existing solutions all offer either one or the other of    Faster than O n m  performance  Preserve order of input list    But so far no solution has both  If you want both  try this   s   set temp2  temp3    x for x in temp1 if x not in s    Performance test  import timeit init    temp1   list range 100    temp2    i   2 for i in range 50    print timeit timeit  list set temp1  - set temp2     init  number   100000  print timeit timeit  s   set temp2   x for x in temp1 if x not in s    init  number   100000  print timeit timeit   item for item in temp1 if item not in temp2    init  number   100000    Results   4 34620224079   ars  answer 4 2770634955    This answer 30 7715615392   matt b s answer   The method I presented as well as preserving order is also  slightly  faster than the set subtraction because it doesn t require construction of an unnecessary set  The performance difference would be more noticable if the first list is considerably longer than the second and if hashing is expensive  Here s a second test demonstrating this   init       temp1    str i  for i in range 100000   temp2    str i   2  for i in range 50         Results   11 3836875916   ars  answer 3 63890368748   this answer  3 times faster   37 7445402279   matt b s answer

User · Answer

If you run into TypeError  unhashable type   list  you need to turn lists or sets into tuples  e g   set map tuple  list of lists1   symmetric difference set map tuple  list of lists2      See also How to compare a list of lists sets in python

User · Answer

The difference between two lists  say list1 and list2  can be found using the following simple function   def diff list1  list2       c   set list1  union set list2      or c   set list1    set list2      d   set list1  intersection set list2      or d   set list1   amp  set list2      return list c - d    or  def diff list1  list2       return list set list1  symmetric difference set list2       or return list set list1    set list2     By Using the above function  the difference can be found using diff temp2  temp1  or diff temp1  temp2   Both will give the result   Four    Three    You don t have to worry about the order of the list or which list is to be given first   Python doc reference

User · Answer

We can calculate intersection minus union of lists   temp1     One    Two    Three    Four   temp2     One    Two    Five    set temp1 temp2 - set temp1  amp set temp2    Out  set   Four    Five    Three

User · Answer

This can be solved with one line  The question is given two lists  temp1 and temp2  return their difference in a third list  temp3    temp3   list set temp1  difference set temp2

User · Answer

Can be done using python XOR operator    This will remove the duplicates in each list This will show difference of temp1 from temp2 and temp2 from temp1      set temp1    set temp2

User · Answer

You can cycle through the first list and  for every item that isn t in the second list but is in the first list  add it to the third list  E g  temp3      for i in temp1      if i not in temp2          temp3 append i  print temp3

User · Answer

def diffList list1  list2         returns the difference between two lists      if len list1   gt  len list2           return  list set list1  - set list2        else          return  list set list2  - set list1      e g  if list1    10  15  20  25  30  35  40  and list2    25  40  35  then the returned list will be output    10  20  30  15

User · Answer

Here is an simple way to distinguish two lists  whatever the contents are   you can get the result as shown below     gt  gt  gt  from sets import Set  gt  gt  gt   gt  gt  gt  l1     xvda   False   xvdbb   12   xvdbc    gt  gt  gt  l2     xvda    xvdbb    xvdbc    xvdbd   None   gt  gt  gt   gt  gt  gt  Set l1  symmetric difference Set l2   Set  False   xvdbd   None  12     Hope this will helpful

User · Answer

single line version of arulmr solution  def diff listA  listB       return set listA  - set listB    set listA  -set listB

User · Answer

I prefer to use converting to sets and then using the  difference    function  The full code is    temp1     One    Two    Three    Four                        temp2     One    Two   set1   set temp1  set2   set temp2  set3   set1 difference set2  temp3   list set3  print temp3     Output    gt  gt  gt print temp3    Three    Four     It s the easiest to undersand  and morover in future if you work with large data  converting it to sets will remove duplicates if duplicates are not required  Hope it helps  -

User · Answer

Here s a Counter answer for the simplest case   This is shorter than the one above that does two-way diffs because it only does exactly what the question asks  generate a list of what s in the first list but not the second   from collections import Counter  lst1     One    Two    Three    Four   lst2     One    Two    c1   Counter lst1  c2   Counter lst2  diff   list  c1 - c2  elements      Alternatively  depending on your readability preferences  it makes for a decent one-liner   diff   list  Counter lst1  - Counter lst2   elements      Output     Three    Four     Note that you can remove the list      call if you are just iterating over it   Because this solution uses counters  it handles quantities properly vs the many set-based answers   For example on this input   lst1     One    Two    Two    Two    Three    Three    Four   lst2     One    Two     The output is     Two    Two    Three    Three    Four

User · Answer

I wanted something that would take two lists and could do what diff in bash does  Since this question pops up first when you search for  python diff two lists  and is not very specific  I will post what I came up with   Using SequenceMather from difflib you can compare two lists like diff does  None of the other answers will tell you the position where the difference occurs  but this one does  Some answers give the difference in only one direction  Some reorder the elements  Some don t handle duplicates  But this solution gives you a true difference between two lists   a    A quick fox jumps the lazy dog  split   b    A quick brown mouse jumps over the dog  split    from difflib import SequenceMatcher  for tag  i  j  k  l in SequenceMatcher None  a  b  get opcodes      if tag     equal   print  both have   a i j     if tag in   delete    replace    print    1st has   a i j     if tag in   insert    replace    print    2nd has   b k l     This outputs   both have   A    quick     1st has   fox     2nd has   brown    mouse   both have   jumps     2nd has   over   both have   the     1st has   lazy   both have   dog     Of course  if your application makes the same assumptions the other answers make  you will benefit from them the most  But if you are looking for a true diff functionality  then this is the only way to go   For example  none of the other answers could handle   a    1 2 3 4 5  b    5 4 3 2 1    But this one does     2nd has  5  4  3  2  both have  1    1st has  2  3  4  5

User · Answer

Let s say we have two lists  list1    1  3  5  7  9  list2    1  2  3  4  5    we can see from the above two lists that items 1  3  5 exist in list2 and items 7  9 do not  On the other hand  items 1  3  5 exist in list1 and items 2  4 do not   What is the best solution to return a new list containing items 7  9 and 2  4   All answers above find the solution  now whats the most optimal   def difference list1  list2       new list          for i in list1          if i not in list2              new list append i       for j in list2          if j not in list1              new list append j      return new list   versus  def sym diff list1  list2       return list set list1  symmetric difference set list2      Using timeit we can see the results  t1   timeit Timer  difference list1  list2     from   main   import difference   list1  list2   t2   timeit Timer  sym diff list1  list2     from   main   import sym diff   list1  list2    print  Using two for loops   t1 timeit number 100000    Milliseconds   print  Using two for loops   t2 timeit number 100000    Milliseconds     returns   7  9  2  4  Using two for loops 0 11572412995155901 Milliseconds Using symmetric difference 0 11285737506113946 Milliseconds  Process finished with exit code 0

User · Answer

You could use a naive method if the elements of the difflist are sorted and sets   list1  1 2 3 4 5  list2  1 2 3   print list1 len list2      or with native set methods   subset set list1  difference list2   print subset  import timeit init    temp1   list range 100    temp2    i   2 for i in range 50    print  Naive solution     timeit timeit  temp1 len temp2      init  number   100000  print  Native set solution     timeit timeit  set temp1  difference temp2    init  number   100000    Naive solution   0 0787101593292  Native set solution   0 998837615564

User · Answer

temp3    item for item in temp1 if item not in temp2

User · Answer

Here are a few simple  order-preserving ways of diffing two lists of strings   Code  An unusual approach using pathlib   import pathlib   temp1     One    Two    Three    Four   temp2     One    Two    p   pathlib Path  temp1  r   p relative to  temp2  list r parts      Three    Four     This assumes both lists contain strings with equivalent beginnings   See the docs for more details   Note  it is not particularly fast compared to set operations     A straight-forward implementation using itertools zip longest   import itertools as it    x for x  y in it zip longest temp1  temp2  if x    y      Three    Four

User · Answer

In  5   list set temp1  - set temp2   Out 5     Four    Three     Beware that   In  5   set  1  2   - set  2  3   Out 5   set  1      where you might expect want it to equal set  1  3    If you do want set  1  3   as your answer  you ll need to use set  1  2   symmetric difference set  2  3

User · Answer

I am little too late in the game for this but you can do a comparison of performance of some of the above mentioned code with this  two of the fastest contenders are   list set x  symmetric difference set y    list set x    set y     I apologize for the elementary level of coding   import time import random from itertools import filterfalse    1 - performance  time taken    2 - correctness  answer - 1 4 5 6    set performance performance   1 numberoftests   7  def answer x y z       if z    0          start   time clock           lists    str list set x -set y   list set y -set y             times     1       str time clock   - start           return  lists times       elif z    1          start   time clock           lists    str list set x  symmetric difference set y              times     2       str time clock   - start           return  lists times       elif z    2          start   time clock           lists    str list set x    set y             times     3       str time clock   - start           return  lists times       elif z    3          start   time clock           lists    filterfalse set y    contains    x           times     4       str time clock   - start           return  lists times       elif z    4          start   time clock           lists    tuple set x  - set y            times     5       str time clock   - start           return  lists times       elif z    5          start   time clock           lists     tt for tt in x if tt not in y           times     6       str time clock   - start           return  lists times       else              start   time clock           Xarray    iDa for iDa in x if iDa not in y          Yarray    iDb for iDb in y if iDb not in x          lists    str Xarray   Yarray           times     7       str time clock   - start           return  lists times   n   numberoftests  if performance    2      a    1 2 3 4 5      b    3 2 6      for c in range 0 n           d   answer a b c          print d 0    elif performance    1      for tests in range 0 10           print  Test Number    str tests   1           a   random sample range 1  900000   9999          b   random sample range 1  900000   9999          for c in range 0 n                if c not in  1 4 5 6               d   answer a b c              print d 1

User · Answer

If you should remove all values from list a  which are present in list b  def list diff a  b       r           for i in a          if i not in b              r append i      return r  list diff  1 2 2    1   Result   2 2  or def list diff a  b       return  x for x in a if x not in b

User · Answer

This is another solution   def diff a  b       xa    i for i in set a  if i not in b      xb    i for i in set b  if i not in a      return xa   xb

User · Answer

I know this question got great answers already but I wish to add the following method using numpy  temp1     One    Two    Three    Four   temp2     One    Two    list np setdiff1d temp1 temp2      Four    Three    Output

User · Answer

If you are really looking into performance  then use numpy   Here is the full notebook as a gist on github with comparison between list  numpy  and pandas   https   gist github com denfromufa 2821ff59b02e9482be15d27f2bbd4451

User · Answer

In case you want the difference recursively  I have written a package for python  https   github com seperman deepdiff  Installation  Install from PyPi   pip install deepdiff   Example usage  Importing   gt  gt  gt  from deepdiff import DeepDiff  gt  gt  gt  from pprint import pprint  gt  gt  gt  from   future   import print function   In case running on Python 2   Same object returns empty   gt  gt  gt  t1    1 1  2 2  3 3   gt  gt  gt  t2   t1  gt  gt  gt  print DeepDiff t1  t2        Type of an item has changed   gt  gt  gt  t1    1 1  2 2  3 3   gt  gt  gt  t2    1 1  2  2   3 3   gt  gt  gt  pprint DeepDiff t1  t2   indent 2     type changes      root 2       newtype    lt class  str  gt                                     newvalue    2                                     oldtype    lt class  int  gt                                     oldvalue   2      Value of an item has changed   gt  gt  gt  t1    1 1  2 2  3 3   gt  gt  gt  t2    1 1  2 4  3 3   gt  gt  gt  pprint DeepDiff t1  t2   indent 2    values changed     root 2      newvalue   4   oldvalue   2      Item added and or removed   gt  gt  gt  t1    1 1  2 2  3 3  4 4   gt  gt  gt  t2    1 1  2 4  3 3  5 5  6 6   gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff    dic item added     root 5     root 6       dic item removed     root 4       values changed     root 2      newvalue   4   oldvalue   2      String difference   gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   world     gt  gt  gt  t2    1 1  2 4  3 3  4   a   hello    b   world      gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2     values changed      root 2      newvalue   4   oldvalue   2                          root 4   b        newvalue    world                                             oldvalue    world       String difference 2   gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   world  nGoodbye  n1 n2 nEnd     gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   world n1 n2 nEnd     gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2     values changed      root 4   b        diff    ---  n                                                        n                                                      -1 5  1 4    n                                                   -world  n                                                   -Goodbye  n                                                    world n                                                    1 n                                                    2 n                                                    End                                            newvalue    world n1 n2 nEnd                                            oldvalue    world  n                                                       Goodbye  n                                                       1 n                                                       2 n                                                       End       gt  gt  gt    gt  gt  gt  print  ddiff  values changed    root 4   b      diff    ---          -1 5  1 4    -world  -Goodbye   world  1  2  End   Type change   gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   1  2  3     gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   world n n nEnd     gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2     type changes      root 4   b        newtype    lt class  str  gt                                          newvalue    world n n nEnd                                          oldtype    lt class  list  gt                                          oldvalue    1  2  3       List difference   gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   1  2  3  4     gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   1  2     gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2    iterable item removed     root 4   b   2    3   root 4   b   3    4     List difference 2    gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   1  2  3     gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   1  3  2  3     gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2     iterable item added     root 4   b   3    3      values changed      root 4   b   1      newvalue   3   oldvalue   2                          root 4   b   2      newvalue   2   oldvalue   3      List difference ignoring order or duplicates   with the same dictionaries as above    gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   1  2  3     gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   1  3  2  3     gt  gt  gt  ddiff   DeepDiff t1  t2  ignore order True   gt  gt  gt  print  ddiff       List that contains dictionary    gt  gt  gt  t1    1 1  2 2  3 3  4   a   hello    b   1  2   1 1  2 2      gt  gt  gt  t2    1 1  2 2  3 3  4   a   hello    b   1  2   1 3      gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  ddiff  indent   2     dic item removed     root 4   b   2  2        values changed     root 4   b   2  1      newvalue   3   oldvalue   1      Sets    gt  gt  gt  t1    1  2  8   gt  gt  gt  t2    1  2  3  5   gt  gt  gt  ddiff   DeepDiff t1  t2   gt  gt  gt  pprint  DeepDiff t1  t2     set item added     root 3     root 5      set item removed     root 8       Named Tuples    gt  gt  gt  from collections import namedtuple  gt  gt  gt  Point   namedtuple  Point     x    y     gt  gt  gt  t1   Point x 11  y 22   gt  gt  gt  t2   Point x 11  y 23   gt  gt  gt  pprint  DeepDiff t1  t2     values changed     root y     newvalue   23   oldvalue   22      Custom objects    gt  gt  gt  class ClassA object           a   1         def   init   self  b               self b   b       gt  gt  gt  t1   ClassA 1   gt  gt  gt  t2   ClassA 2   gt  gt  gt    gt  gt  gt  pprint DeepDiff t1  t2     values changed     root b     newvalue   2   oldvalue   1      Object attribute added    gt  gt  gt  t2 c    new attribute   gt  gt  gt  pprint DeepDiff t1  t2     attribute added     root c      values changed     root b     newvalue   2   oldvalue   1

User · Answer

this could be even faster than Mark s list comprehension   list itertools filterfalse set temp2    contains    temp1

User · Answer

if you want something more like a changeset    could use Counter  from collections import Counter  def diff a  b         more verbose than needs to be  for clarity       ca  cb   Counter a   Counter b    to add   cb - ca   to remove   ca - cb   changes   Counter to add    changes subtract to remove    return changes  lista     one    three    four    four    one   listb     one    two    three    In  127   diff lista  listb  Out 127   Counter   two   1   one   -1   four   -2     in order to go from lista to list b  you need to add a  two   remove a  one   and remove two  four s  In  128   diff listb  lista  Out 128   Counter   four   2   one   1   two   -1     in order to go from listb to lista  you must add two  four s  add a  one   and remove a  two

[python] Get difference between two lists

Examples related to python

Examples related to performance

Examples related to list

Examples related to set

Examples related to set-difference