Extract subset of key-value pairs from Python dictionary object

Question

I have a big dictionary object that has several key value pairs  about 16   but I am only interested in 3 of them  What is the best way  shortest efficient most elegant  to achieve that   The best I know is   bigdict     a  1  b  2       z  26   subdict     l  bigdict  l     m  bigdict  m     n  bigdict  n      I am sure there is a more elegant way than this  Ideas

User · Answer

A bit of speed comparison for all mentioned methods:

UPDATED on 2020.07.13 (thx to @user3780389): ONLY for keys from bigdict.

 IPython 5.5.0 -- An enhanced Interactive Python.
Python 2.7.18 (default, Aug  8 2019, 00:00:00) 
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux2
import numpy.random as nprnd
  ...: keys = nprnd.randint(100000, size=10000)
  ...: bigdict = dict([(_, nprnd.rand()) for _ in range(100000)])
  ...: 
  ...: %timeit {key:bigdict[key] for key in keys}
  ...: %timeit dict((key, bigdict[key]) for key in keys)
  ...: %timeit dict(map(lambda k: (k, bigdict[k]), keys))
  ...: %timeit {key:bigdict[key] for key in set(keys) & set(bigdict.keys())}
  ...: %timeit dict(filter(lambda i:i[0] in keys, bigdict.items()))
  ...: %timeit {key:value for key, value in bigdict.items() if key in keys}
100 loops, best of 3: 2.36 ms per loop
100 loops, best of 3: 2.87 ms per loop
100 loops, best of 3: 3.65 ms per loop
100 loops, best of 3: 7.14 ms per loop
1 loop, best of 3: 577 ms per loop
1 loop, best of 3: 563 ms per loop

As it was expected: dictionary comprehensions are the best option.

User · Answer

Using map  halfdanrump s answer  is best for me  though haven t timed it    But if you go for a dictionary  and if you have a big dict   Make absolutely certain you loop through the the req  This is crucial  and affects the running time of the algorithm  big O  theta  you name it  Write it generic enough to avoid errors if keys are not there   so e g   big dict     a  1  b  2  c  3                                                   req     a   c   w     k big dict get k None  for k in req     or   k big dict k  for k in req if k in big dict   Note that in the converse case  that the req is big  but my dict is small  you should loop through my dict instead  In general  we are doing an intersection and the complexity of the problem is O min len dict   min len req     Python s own implementation of intersection considers the size of the two sets  so it seems optimal  Also  being in c and part of the core library  is probably faster than most not optimized python statements  Therefore  a solution that I would consider is  dict     a  1  b  2  c  3                                                   req     a   c   w                         k dic k  for k in set req  intersection dict keys      It moves the critical operation inside python s c code and will work for all cases

User · Answer

Yet another one  I prefer Mark Longair s answer   di     a  1  b  2  c  3  req     a   c   w   dict  i for i in di iteritems   if i 0  in di and i 0  in req

User · Answer

This answer uses a dictionary comprehension similar to the selected answer  but will not except on a missing item   python 2 version    k v for k  v in bigDict iteritems   if k in   l    m    n      python 3 version    k v for k  v in bigDict items   if k in   l    m    n

User · Answer

Maybe   subdict dict   x bigdict x   for x in   l    m    n       Python 3 even supports the following   subdict  a bigdict a  for a in   l   m   n      Note that you can check for existence in dictionary as follows   subdict dict   x bigdict x   for x in   l    m    n   if x in bigdict     resp  for python 3  subdict  a bigdict a  for a in   l   m   n   if a in bigdict

User · Answer

A bit shorter  at least   wanted keys     l    m    n     The keys you want dict  k  bigdict k   for k in wanted keys if k in bigdict

User · Answer

solution  from operator import itemgetter from typing import List  Dict  Union   def subdict d  Union Dict  List   columns  List str   - gt  Union Dict  List Dict           Return a dict or list of dicts with subset of      columns from the d argument              getter   itemgetter  columns       if isinstance d  list           result              for subset in map getter  d               record   dict zip columns  subset               result append record          return result     elif isinstance d  dict           return dict zip columns  getter d         raise ValueError  Unsupported type for  d      examples of use    pure dict  d   dict a 1  b 2  c 3  print subdict d    a    c       gt  gt  gt  In  5     a   1   c   3      list of dicts  d         dict a 1  b 2  c 3       dict a 2  b 4  c 6       dict a 4  b 8  c 12      print subdict d    a    c       gt  gt  gt  In  5      a   1   c   3     a   2   c   6     a   4   c   12

User · Answer

interesting keys     l    m    n   subdict    x  bigdict x  for x in interesting keys if x in bigdict

User · Answer

Okay  this is something that has bothered me a few times  so thank you Jayesh for asking it   The answers above seem like as good a solution as any  but if you are using this all over your code  it makes sense to wrap the functionality IMHO  Also  there are two possible use cases here  one where you care about whether all keywords are in the original dictionary  and one where you don t  It would be nice to treat both equally   So  for my two-penneth worth  I suggest writing a sub-class of dictionary  e g   class my dict dict       def subdict self  keywords  fragile False           d              for k in keywords              try                  d k    self k              except KeyError                  if fragile                      raise         return d   Now you can pull out a sub-dictionary with orig dict subdict keywords    Usage examples        our keywords are letters of the alphabet keywords    abcdefghijklmnopqrstuvwxyz       our dictionary maps letters to their index d   my dict   k i  for i k in enumerate keywords    print  Original dictionary  n r n n     d         constructing a sub-dictionary with good keywords oddkeywords   keywords   2  subd   d subdict oddkeywords  print  Dictionary from odd numbered keys  n r n n     subd         constructing a sub-dictionary with mixture of good and bad keywords somebadkeywords   keywords 1  2     A  try      subd2   d subdict somebadkeywords      print  We shouldn t see this message   except KeyError      print  subd2 construction fails        print   toriginal dictionary doesn t contain some keys n n        Trying again with fragile set to false try      subd3   d subdict somebadkeywords  fragile False      print  Dictionary constructed using some bad keys  n r n n     subd3    except KeyError      print  We shouldn t see this message     If you run all the above code  you should see  something like  the following output  sorry for the formatting       Original dictionary             a   0   c   2   b   1   e   4   d   3   g   6   f   5             i   8   h   7   k   10   j   9   m   12   l   11   o   14             n   13   q   16   p   15   s   18   r   17   u   20             t   19   w   22   v   21   y   24   x   23   z   25         Dictionary from odd numbered keys             a   0   c   2   e   4   g   6   i   8   k   10   m   12   o   14   q   16   s   18   u   20   w   22   y   24         subd2 construction fails           original dictionary doesn t contain some keys        Dictionary constructed using some bad keys             b   1   d   3   f   5   h   7   j   9   l   11   n   13   p   15   r   17   t   19   v   21   x   23   z   25

User · Answer

You could try   dict  k  bigdict k   for k in   l    m    n          or in Python 3 Python versions 2 7 or later  thanks to F  bio Diniz for pointing that out that it works in 2 7 too     k  bigdict k  for k in   l    m    n      Update  As H  vard S points out  I m assuming that you know the keys are going to be in the dictionary - see his answer if you aren t able to make that assumption  Alternatively  as timbo points out in the comments  if you want a key that s missing in bigdict to map to None   you can do    k  bigdict get k  None  for k in   l    m    n      If you re using Python 3  and you only want keys in the new dict that actually exist in the original one  you can use the fact to view objects implement some set operations    k  bigdict k  for k in bigdict keys    amp    l    m    n

User · Answer

You can also use map  which is a very useful function to get to know anyway    sd   dict map lambda k   k  l get k  None    l    Example   large dictionary     a1  123   a2  45   a3  344  list of keys     a1    a3   small dictionary   dict map lambda key   key  large dictionary get key  None    list of keys     PS  I borrowed the  get key  None  from a previous answer

[python] Extract subset of key-value pairs from Python dictionary object?

Examples related to python

Examples related to dictionary

Examples related to associative-array