I'm pretty much new in Python object oriented programming and I have trouble
understanding the super()
function (new style classes) especially when it comes to multiple inheritance.
For example if you have something like:
class First(object):
def __init__(self):
print "first"
class Second(object):
def __init__(self):
print "second"
class Third(First, Second):
def __init__(self):
super(Third, self).__init__()
print "that's it"
What I don't get is: will the Third()
class inherit both constructor methods? If yes, then which one will be run with super()
and why?
And what if you want to run the other one? I know it has something to do with Python method resolution order (MRO).
This question is related to
python
multiple-inheritance
Posting this answer for my future referance.
Python Multiple Inheritance should use a diamond model and the function signature shouldn't change in the model.
A
/ \
B C
\ /
D
The sample code snippet would be ;-
class A:
def __init__(self, name=None):
# this is the head of the diamond, no need to call super() here
self.name = name
class B(A):
def __init__(self, param1='hello', **kwargs):
super().__init__(**kwargs)
self.param1 = param1
class C(A):
def __init__(self, param2='bye', **kwargs):
super().__init__(**kwargs)
self.param2 = param2
class D(B, C):
def __init__(self, works='fine', **kwargs):
super().__init__(**kwargs)
print(f"{works=}, {self.param1=}, {self.param2=}, {self.name=}")
d = D(name='Testing')
Here class A is object
class First(object):
def __init__(self, a):
print "first", a
super(First, self).__init__(20)
class Second(object):
def __init__(self, a):
print "second", a
super(Second, self).__init__()
class Third(First, Second):
def __init__(self):
super(Third, self).__init__(10)
print "that's it"
t = Third()
Output is
first 10
second 20
that's it
Call to Third() locates the init defined in Third. And call to super in that routine invokes init defined in First. MRO=[First, Second]. Now call to super in init defined in First will continue searching MRO and find init defined in Second, and any call to super will hit the default object init. I hope this example clarifies the concept.
If you don't call super from First. The chain stops and you will get the following output.
first 10
that's it
This is known as the Diamond Problem, the page has an entry on Python, but in short, Python will call the superclass's methods from left to right.
Consider calling super().Foo()
called from a sub-class. The Method Resolution Order (MRO) method is the order in which method calls are resolved.
In this, super().Foo() will be searched up in the hierarchy and will consider the closest implementation, if found, else raise an Exception. The "is a" relationship will always be True in between any visited sub-class and its super class up in the hierarchy. But this story isn't the same always in Multiple Inheritance.
Here, while searching for super().Foo() implementation, every visited class in the hierarchy may or may not have is a relation. Consider following examples:
class A(object): pass
class B(object): pass
class C(A): pass
class D(A): pass
class E(C, D): pass
class F(B): pass
class G(B): pass
class H(F, G): pass
class I(E, H): pass
Here, I
is the lowest class in the hierarchy. Hierarchy diagram and MRO for I
will be
(Red numbers showing the MRO)
MRO is I E C D A H F G B object
Note that a class X
will be visited only if all its sub-classes, which inherit from it, have been visited(i.e., you should never visit a class that has an arrow coming into it from a class below that you have not yet visited).
Here, note that after visiting class C
, D
is visited although C
and D
DO NOT have is a relationship between them(but both have with A
). This is where super()
differs from single inheritance.
Consider a slightly more complicated example:
(Red numbers showing the MRO)
MRO is I E C H D A F G B object
In this case we proceed from I
to E
to C
. The next step up would be A
, but we have yet to visit D
, a subclass of A
. We cannot visit D
, however, because we have yet to visit H
, a subclass of D
. The leaves H
as the next class to visit. Remember, we attempt to go up in hierarchy, if possible, so we visit its leftmost superclass, D
. After D
we visit A
, but we cannot go up to object because we have yet to visit F
, G
, and B
. These classes, in order, round out the MRO for I
.
Note that no class can appear more than once in MRO.
This is how super() looks up in the hierarchy of inheritance.
Credits for resources: Richard L Halterman Fundamentals of Python Programming
Assuming everything descends from object
(you are on your own if it doesn't), Python computes a method resolution order (MRO) based on your class inheritance tree. The MRO satisfies 3 properties:
If no such ordering exists, Python errors. The inner workings of this is a C3 Linerization of the classes ancestry. Read all about it here: https://www.python.org/download/releases/2.3/mro/
Thus, in both of the examples below, it is:
When a method is called, the first occurrence of that method in the MRO is the one that is called. Any class that doesn't implement that method is skipped. Any call to super
within that method will call the next occurrence of that method in the MRO. Consequently, it matters both what order you place classes in inheritance, and where you put the calls to super
in the methods.
super
first in each methodclass Parent(object):
def __init__(self):
super(Parent, self).__init__()
print "parent"
class Left(Parent):
def __init__(self):
super(Left, self).__init__()
print "left"
class Right(Parent):
def __init__(self):
super(Right, self).__init__()
print "right"
class Child(Left, Right):
def __init__(self):
super(Child, self).__init__()
print "child"
Child()
Outputs:
parent
right
left
child
super
last in each methodclass Parent(object):
def __init__(self):
print "parent"
super(Parent, self).__init__()
class Left(Parent):
def __init__(self):
print "left"
super(Left, self).__init__()
class Right(Parent):
def __init__(self):
print "right"
super(Right, self).__init__()
class Child(Left, Right):
def __init__(self):
print "child"
super(Child, self).__init__()
Child()
Outputs:
child
left
right
parent
Another not yet covered point is passing parameters for initialization of classes. Since the destination of super
depends on the subclass the only good way to pass parameters is packing them all together. Then be careful to not have the same parameter name with different meanings.
Example:
class A(object):
def __init__(self, **kwargs):
print('A.__init__')
super().__init__()
class B(A):
def __init__(self, **kwargs):
print('B.__init__ {}'.format(kwargs['x']))
super().__init__(**kwargs)
class C(A):
def __init__(self, **kwargs):
print('C.__init__ with {}, {}'.format(kwargs['a'], kwargs['b']))
super().__init__(**kwargs)
class D(B, C): # MRO=D, B, C, A
def __init__(self):
print('D.__init__')
super().__init__(a=1, b=2, x=3)
print(D.mro())
D()
gives:
[<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>]
D.__init__
B.__init__ 3
C.__init__ with 1, 2
A.__init__
Calling the super class __init__
directly to more direct assignment of parameters is tempting but fails if there is any super
call in a super class and/or the MRO is changed and class A may be called multiple times, depending on the implementation.
To conclude: cooperative inheritance and super and specific parameters for initialization aren't working together very well.
I would like to add to what @Visionscaper says at the top:
Third --> First --> object --> Second --> object
In this case the interpreter doesnt filter out the object class because its duplicated, rather its because Second appears in a head position and doesnt appear in the tail position in a hierarchy subset. While object only appears in tail positions and is not considered a strong position in C3 algorithm to determine priority.
The linearisation(mro) of a class C, L(C), is the
Linearised Merge is done by selecting the common classes that appears as the head of lists and not the tail since order matters(will become clear below)
The linearisation of Third can be computed as follows:
L(O) := [O] // the linearization(mro) of O(object), because O has no parents
L(First) := [First] + merge(L(O), [O])
= [First] + merge([O], [O])
= [First, O]
// Similarly,
L(Second) := [Second, O]
L(Third) := [Third] + merge(L(First), L(Second), [First, Second])
= [Third] + merge([First, O], [Second, O], [First, Second])
// class First is a good candidate for the first merge step, because it only appears as the head of the first and last lists
// class O is not a good candidate for the next merge step, because it also appears in the tails of list 1 and 2,
= [Third, First] + merge([O], [Second, O], [Second])
// class Second is a good candidate for the second merge step, because it appears as the head of the list 2 and 3
= [Third, First, Second] + merge([O], [O])
= [Third, First, Second, O]
Thus for a super() implementation in the following code:
class First(object):
def __init__(self):
super(First, self).__init__()
print "first"
class Second(object):
def __init__(self):
super(Second, self).__init__()
print "second"
class Third(First, Second):
def __init__(self):
super(Third, self).__init__()
print "that's it"
it becomes obvious how this method will be resolved
Third.__init__() ---> First.__init__() ---> Second.__init__() --->
Object.__init__() ---> returns ---> Second.__init__() -
prints "second" - returns ---> First.__init__() -
prints "first" - returns ---> Third.__init__() - prints "that's it"
I understand this doesn't directly answer the super()
question, but I feel it's relevant enough to share.
There is also a way to directly call each inherited class:
class First(object):
def __init__(self):
print '1'
class Second(object):
def __init__(self):
print '2'
class Third(First, Second):
def __init__(self):
Second.__init__(self)
Just note that if you do it this way, you'll have to call each manually as I'm pretty sure First
's __init__()
won't be called.
About @calfzhou's comment, you can use, as usually, **kwargs
:
class A(object):
def __init__(self, a, *args, **kwargs):
print("A", a)
class B(A):
def __init__(self, b, *args, **kwargs):
super(B, self).__init__(*args, **kwargs)
print("B", b)
class A1(A):
def __init__(self, a1, *args, **kwargs):
super(A1, self).__init__(*args, **kwargs)
print("A1", a1)
class B1(A1, B):
def __init__(self, b1, *args, **kwargs):
super(B1, self).__init__(*args, **kwargs)
print("B1", b1)
B1(a1=6, b1=5, b="hello", a=None)
Result:
A None
B hello
A1 6
B1 5
You can also use them positionally:
B1(5, 6, b="hello", a=None)
but you have to remember the MRO, it's really confusing. You can avoid this by using keyword-only parameters:
class A(object):
def __init__(self, *args, a, **kwargs):
print("A", a)
etcetera.
I can be a little annoying, but I noticed that people forgot every time to use *args
and **kwargs
when they override a method, while it's one of few really useful and sane use of these 'magic variables'.
I wanted to elaborate the answer by lifeless a bit because when I started reading about how to use super() in a multiple inheritance hierarchy in Python, I did't get it immediately.
What you need to understand is that super(MyClass, self).__init__()
provides the next __init__
method according to the used Method Resolution Ordering (MRO) algorithm in the context of the complete inheritance hierarchy.
This last part is crucial to understand. Let's consider the example again:
#!/usr/bin/env python2
class First(object):
def __init__(self):
print "First(): entering"
super(First, self).__init__()
print "First(): exiting"
class Second(object):
def __init__(self):
print "Second(): entering"
super(Second, self).__init__()
print "Second(): exiting"
class Third(First, Second):
def __init__(self):
print "Third(): entering"
super(Third, self).__init__()
print "Third(): exiting"
According to this article about Method Resolution Order by Guido van Rossum, the order to resolve __init__
is calculated (before Python 2.3) using a "depth-first left-to-right traversal" :
Third --> First --> object --> Second --> object
After removing all duplicates, except for the last one, we get :
Third --> First --> Second --> object
So, lets follow what happens when we instantiate an instance of the Third
class, e.g. x = Third()
.
Third.__init__
executes.
Third(): entering
super(Third, self).__init__()
executes and MRO returns First.__init__
which is called.First.__init__
executes.
First(): entering
super(First, self).__init__()
executes and MRO returns Second.__init__
which is called.Second.__init__
executes.
Second(): entering
super(Second, self).__init__()
executes and MRO returns object.__init__
which is called.object.__init__
executes (no print statements in the code there)Second.__init__
which then prints Second(): exiting
First.__init__
which then prints First(): exiting
Third.__init__
which then prints Third(): exiting
This details out why instantiating Third() results in to :
Third(): entering
First(): entering
Second(): entering
Second(): exiting
First(): exiting
Third(): exiting
The MRO algorithm has been improved from Python 2.3 onwards to work well in complex cases, but I guess that using the "depth-first left-to-right traversal" + "removing duplicates expect for the last" still works in most cases (please comment if this is not the case). Be sure to read the blog post by Guido!
In python 3.5+ inheritance looks predictable and very nice for me. Please looks at this code:
class Base(object):
def foo(self):
print(" Base(): entering")
print(" Base(): exiting")
class First(Base):
def foo(self):
print(" First(): entering Will call Second now")
super().foo()
print(" First(): exiting")
class Second(Base):
def foo(self):
print(" Second(): entering")
super().foo()
print(" Second(): exiting")
class Third(First, Second):
def foo(self):
print(" Third(): entering")
super().foo()
print(" Third(): exiting")
class Fourth(Third):
def foo(self):
print("Fourth(): entering")
super().foo()
print("Fourth(): exiting")
Fourth().foo()
print(Fourth.__mro__)
Outputs:
Fourth(): entering
Third(): entering
First(): entering Will call Second now
Second(): entering
Base(): entering
Base(): exiting
Second(): exiting
First(): exiting
Third(): exiting
Fourth(): exiting
(<class '__main__.Fourth'>, <class '__main__.Third'>, <class '__main__.First'>, <class '__main__.Second'>, <class '__main__.Base'>, <class 'object'>)
As you can see, it calls foo exactly ONE time for each inherited chain in the same order as it was inherited. You can get that order by calling .mro :
Fourth -> Third -> First -> Second -> Base -> object
In learningpythonthehardway I learn something called super() an in-built function if not mistaken. Calling super() function can help the inheritance to pass through the parent and 'siblings' and help you to see clearer. I am still a beginner but I love to share my experience on using this super() in python2.7.
If you have read through the comments in this page, you will hear of Method Resolution Order (MRO), the method being the function you wrote, MRO will be using Depth-First-Left-to-Right scheme to search and run. You can do more research on that.
By adding super() function
super(First, self).__init__() #example for class First.
You can connect multiple instances and 'families' with super(), by adding in each and everyone in them. And it will execute the methods, go through them and make sure you didn't miss out! However, adding them before or after does make a difference you will know if you have done the learningpythonthehardway exercise 44. Let the fun begins!!
Taking example below, you can copy & paste and try run it:
class First(object):
def __init__(self):
print("first")
class Second(First):
def __init__(self):
print("second (before)")
super(Second, self).__init__()
print("second (after)")
class Third(First):
def __init__(self):
print("third (before)")
super(Third, self).__init__()
print("third (after)")
class Fourth(First):
def __init__(self):
print("fourth (before)")
super(Fourth, self).__init__()
print("fourth (after)")
class Fifth(Second, Third, Fourth):
def __init__(self):
print("fifth (before)")
super(Fifth, self).__init__()
print("fifth (after)")
Fifth()
How does it run? The instance of fifth() will goes like this. Each step goes from class to class where the super function added.
1.) print("fifth (before)")
2.) super()>[Second, Third, Fourth] (Left to right)
3.) print("second (before)")
4.) super()> First (First is the Parent which inherit from object)
The parent was found and it will go continue to Third and Fourth!!
5.) print("third (before)")
6.) super()> First (Parent class)
7.) print ("Fourth (before)")
8.) super()> First (Parent class)
Now all the classes with super() have been accessed! The parent class has been found and executed and now it continues to unbox the function in the inheritances to finished the codes.
9.) print("first") (Parent)
10.) print ("Fourth (after)") (Class Fourth un-box)
11.) print("third (after)") (Class Third un-box)
12.) print("second (after)") (Class Second un-box)
13.) print("fifth (after)") (Class Fifth un-box)
14.) Fifth() executed
The outcome of the program above:
fifth (before)
second (before
third (before)
fourth (before)
first
fourth (after)
third (after)
second (after)
fifth (after)
For me by adding super() allows me to see clearer on how python would execute my coding and make sure the inheritance can access the method I intended.
This is to how I solved to issue of having multiple inheritance with different variables for initialization and having multiple MixIns with the same function call. I had to explicitly add variables to passed **kwargs and add a MixIn interface to be an endpoint for super calls.
Here A
is an extendable base class and B
and C
are MixIn classes both who provide function f
. A
and B
both expect parameter v
in their __init__
and C
expects w
.
The function f
takes one parameter y
. Q
inherits from all three classes. MixInF
is the mixin interface for B
and C
.
class A(object):
def __init__(self, v, *args, **kwargs):
print "A:init:v[{0}]".format(v)
kwargs['v']=v
super(A, self).__init__(*args, **kwargs)
self.v = v
class MixInF(object):
def __init__(self, *args, **kwargs):
print "IObject:init"
def f(self, y):
print "IObject:y[{0}]".format(y)
class B(MixInF):
def __init__(self, v, *args, **kwargs):
print "B:init:v[{0}]".format(v)
kwargs['v']=v
super(B, self).__init__(*args, **kwargs)
self.v = v
def f(self, y):
print "B:f:v[{0}]:y[{1}]".format(self.v, y)
super(B, self).f(y)
class C(MixInF):
def __init__(self, w, *args, **kwargs):
print "C:init:w[{0}]".format(w)
kwargs['w']=w
super(C, self).__init__(*args, **kwargs)
self.w = w
def f(self, y):
print "C:f:w[{0}]:y[{1}]".format(self.w, y)
super(C, self).f(y)
class Q(C,B,A):
def __init__(self, v, w):
super(Q, self).__init__(v=v, w=w)
def f(self, y):
print "Q:f:y[{0}]".format(y)
super(Q, self).f(y)
Your code, and the other answers, are all buggy. They are missing the super()
calls in the first two classes that are required for co-operative subclassing to work.
Here is a fixed version of the code:
class First(object):
def __init__(self):
super(First, self).__init__()
print("first")
class Second(object):
def __init__(self):
super(Second, self).__init__()
print("second")
class Third(First, Second):
def __init__(self):
super(Third, self).__init__()
print("third")
The super()
call finds the next method in the MRO at each step, which is why First and Second have to have it too, otherwise execution stops at the end of Second.__init__()
.
This is what I get:
>>> Third()
second
first
third
Maybe there's still something that can be added, a small example with Django rest_framework, and decorators. This provides an answer to the implicit question: "why would I want this anyway?"
As said: we're with Django rest_framework, and we're using generic views, and for each type of objects in our database we find ourselves with one view class providing GET and POST for lists of objects, and an other view class providing GET, PUT, and DELETE for individual objects.
Now the POST, PUT, and DELETE we want to decorate with Django's login_required. Notice how this touches both classes, but not all methods in either class.
A solution could go through multiple inheritance.
from django.utils.decorators import method_decorator
from django.contrib.auth.decorators import login_required
class LoginToPost:
@method_decorator(login_required)
def post(self, arg, *args, **kwargs):
super().post(arg, *args, **kwargs)
Likewise for the other methods.
In the inheritance list of my concrete classes, I would add my LoginToPost
before ListCreateAPIView
and LoginToPutOrDelete
before RetrieveUpdateDestroyAPIView
. My concrete classes' get
would stay undecorated.
Source: Stackoverflow.com