[python] Proper way to use **kwargs in Python

What is the proper way to use **kwargs in Python when it comes to default values?

kwargs returns a dictionary, but what is the best way to set default values, or is there one? Should I just access it as a dictionary? Use get function?

class ExampleClass:
    def __init__(self, **kwargs):
        self.val = kwargs['val']
        self.val2 = kwargs.get('val2')

A simple question, but one that I can't find good resources on. People do it different ways in code that I've seen and it's hard to know what to use.

While most answers are saying that, e.g.,

def f(**kwargs):
    foo = kwargs.pop('foo')
    bar = kwargs.pop('bar')
    ...etc...

is "the same as"

def f(foo=None, bar=None, **kwargs):
    ...etc...

this is not true. In the latter case, f can be called as f(23, 42), while the former case accepts named arguments only -- no positional calls. Often you want to allow the caller maximum flexibility and therefore the second form, as most answers assert, is preferable: but that is not always the case. When you accept many optional parameters of which typically only a few are passed, it may be an excellent idea (avoiding accidents and unreadable code at your call sites!) to force the use of named arguments -- threading.Thread is an example. The first form is how you implement that in Python 2.

The idiom is so important that in Python 3 it now has special supporting syntax: every argument after a single * in the def signature is keyword-only, that is, cannot be passed as a positional argument, but only as a named one. So in Python 3 you could code the above as:

def f(*, foo=None, bar=None, **kwargs):
    ...etc...

Indeed, in Python 3 you can even have keyword-only arguments that aren't optional (ones without a default value).

However, Python 2 still has long years of productive life ahead, so it's better to not forget the techniques and idioms that let you implement in Python 2 important design ideas that are directly supported in the language in Python 3!

Following up on @srhegde suggestion of using setattr:

class ExampleClass(object):
    __acceptable_keys_list = ['foo', 'bar']

    def __init__(self, **kwargs):
        [self.__setattr__(key, kwargs.get(key)) for key in self.__acceptable_keys_list]

This variant is useful when the class is expected to have all of the items in our acceptable list.

You'd do

self.attribute = kwargs.pop('name', default_value)

or

self.attribute = kwargs.get('name', default_value)

If you use pop, then you can check if there are any spurious values sent, and take the appropriate action (if any).

I suggest something like this

def testFunc( **kwargs ):
    options = {
            'option1' : 'default_value1',
            'option2' : 'default_value2',
            'option3' : 'default_value3', }

    options.update(kwargs)
    print options

testFunc( option1='new_value1', option3='new_value3' )
# {'option2': 'default_value2', 'option3': 'new_value3', 'option1': 'new_value1'}

testFunc( option2='new_value2' )
# {'option1': 'default_value1', 'option3': 'default_value3', 'option2': 'new_value2'}

And then use the values any way you want

dictionaryA.update(dictionaryB) adds the contents of dictionaryB to dictionaryA overwriting any duplicate keys.

Here's another approach:

def my_func(arg1, arg2, arg3):
    ... so something ...

kwargs = {'arg1': 'Value One', 'arg2': 'Value Two', 'arg3': 'Value Three'}
# Now you can call the function with kwargs like this:

my_func(**kwargs)

Since **kwargs is used when the number of arguments is unknown, why not doing this?

class Exampleclass(object):
  def __init__(self, **kwargs):
    for k in kwargs.keys():
       if k in [acceptable_keys_list]:
          self.__setattr__(k, kwargs[k])

I think the proper way to use **kwargs in Python when it comes to default values is to use the dictionary method setdefault, as given below:

class ExampleClass:
    def __init__(self, **kwargs):
        kwargs.setdefault('val', value1)
        kwargs.setdefault('val2', value2)

In this way, if a user passes 'val' or 'val2' in the keyword args, they will be used; otherwise, the default values that have been set will be used.

Another simple solution for processing unknown or multiple arguments can be:

class ExampleClass(object):

    def __init__(self, x, y, **kwargs):
      self.x = x
      self.y = y
      self.attributes = kwargs

    def SomeFunction(self):
      if 'something' in self.attributes:
        dosomething()

If you want to combine this with *args you have to keep *args and **kwargs at the end of the definition.

So:

def method(foo, bar=None, *args, **kwargs):
    do_something_with(foo, bar)
    some_other_function(*args, **kwargs)

Using **kwargs and default values is easy. Sometimes, however, you shouldn't be using **kwargs in the first place.

In this case, we're not really making best use of **kwargs.

class ExampleClass( object ):
    def __init__(self, **kwargs):
        self.val = kwargs.get('val',"default1")
        self.val2 = kwargs.get('val2',"default2")

The above is a "why bother?" declaration. It is the same as

class ExampleClass( object ):
    def __init__(self, val="default1", val2="default2"):
        self.val = val
        self.val2 = val2

When you're using **kwargs, you mean that a keyword is not just optional, but conditional. There are more complex rules than simple default values.

When you're using **kwargs, you usually mean something more like the following, where simple defaults don't apply.

class ExampleClass( object ):
    def __init__(self, **kwargs):
        self.val = "default1"
        self.val2 = "default2"
        if "val" in kwargs:
            self.val = kwargs["val"]
            self.val2 = 2*self.val
        elif "val2" in kwargs:
            self.val2 = kwargs["val2"]
            self.val = self.val2 / 2
        else:
            raise TypeError( "must provide val= or val2= parameter values" )

**kwargs gives the freedom to add any number of keyword arguments. One may have a list of keys for which he can set default values. But setting default values for an indefinite number of keys seems unnecessary. Finally, it may be important to have the keys as instance attributes. So, I would do this as follows:

class Person(object):
listed_keys = ['name', 'age']

def __init__(self, **kwargs):
    _dict = {}
    # Set default values for listed keys
    for item in self.listed_keys: 
        _dict[item] = 'default'
    # Update the dictionary with all kwargs
    _dict.update(kwargs)

    # Have the keys of kwargs as instance attributes
    self.__dict__.update(_dict)

You could do something like this

class ExampleClass:
    def __init__(self, **kwargs):
        arguments = {'val':1, 'val2':2}
        arguments.update(kwargs)
        self.val = arguments['val']
        self.val2 = arguments['val2']

@AbhinavGupta and @Steef suggested using update(), which I found very helpful for processing large argument lists:

args.update(kwargs)

What if we want to check that the user hasn't passed any spurious/unsupported arguments? @VinaySajip pointed out that pop() can be used to iteratively process the list of arguments. Then, any leftover arguments are spurious. Nice.

Here's another possible way to do this, which keeps the simple syntax of using update():

# kwargs = dictionary of user-supplied arguments
# args = dictionary containing default arguments

# Check that user hasn't given spurious arguments
unknown_args = user_args.keys() - default_args.keys()
if unknown_args:
    raise TypeError('Unknown arguments: {}'.format(unknown_args))

# Update args to contain user-supplied arguments
args.update(kwargs)

unknown_args is a set containing the names of arguments that don't occur in the defaults.