What are metaclasses in Python

Question

In Python  what are metaclasses and what do we use them for

User · Answer

Python classes are themselves objects - as in instance - of their meta-class.

The default metaclass, which is applied when when you determine classes as:

class foo:
    ...

meta class are used to apply some rule to an entire set of classes. For example, suppose you're building an ORM to access a database, and you want records from each table to be of a class mapped to that table (based on fields, business rules, etc..,), a possible use of metaclass is for instance, connection pool logic, which is share by all classes of record from all tables. Another use is logic to to support foreign keys, which involves multiple classes of records.

when you define metaclass, you subclass type, and can overrided the following magic methods to insert your logic.

class somemeta(type):
    __new__(mcs, name, bases, clsdict):
      """
  mcs: is the base metaclass, in this case type.
  name: name of the new class, as provided by the user.
  bases: tuple of base classes 
  clsdict: a dictionary containing all methods and attributes defined on class

  you must return a class object by invoking the __new__ constructor on the base metaclass. 
 ie: 
    return type.__call__(mcs, name, bases, clsdict).

  in the following case:

  class foo(baseclass):
        __metaclass__ = somemeta

  an_attr = 12

  def bar(self):
      ...

  @classmethod
  def foo(cls):
      ...

      arguments would be : ( somemeta, "foo", (baseclass, baseofbase,..., object), {"an_attr":12, "bar": <function>, "foo": <bound class method>}

      you can modify any of these values before passing on to type
      """
      return type.__call__(mcs, name, bases, clsdict)


    def __init__(self, name, bases, clsdict):
      """ 
      called after type has been created. unlike in standard classes, __init__ method cannot modify the instance (cls) - and should be used for class validaton.
      """
      pass


    def __prepare__():
        """
        returns a dict or something that can be used as a namespace.
        the type will then attach methods and attributes from class definition to it.

        call order :

        somemeta.__new__ ->  type.__new__ -> type.__init__ -> somemeta.__init__ 
        """
        return dict()

    def mymethod(cls):
        """ works like a classmethod, but for class objects. Also, my method will not be visible to instances of cls.
        """
        pass

anyhow, those two are the most commonly used hooks. metaclassing is powerful, and above is nowhere near and exhaustive list of uses for metaclassing.

User · Answer

Metaclass is a kind of class which defines how the class will behave like or we can say that A class is itself an instance of a metaclass

User · Answer

Note that in python 3 6 a new dunder method   init subclass   cls    kwargs  was introduced to replace a lot of common use cases for metaclasses  Is is called when a subclass of the defining class is created  See python docs

User · Answer

In object-oriented programming  a metaclass is a class whose instances are classes  Just as an ordinary class defines the behavior of certain objects  a metaclass defines the behavior of certain class and their instances The term metaclass simply means something used to create classes  In other words  it is the class of a class  The metaclass is used to create the class so like the object being an instance of a class  a class is an instance of a metaclass  In python classes are also considered objects

User · Answer

A metaclass is a class that tells how  some  other class should be created   This is a case where I saw metaclass as a solution to my problem  I had a really complicated problem  that probably could have been solved differently  but I chose to solve it using a metaclass   Because of the complexity  it is one of the few modules I have written where the comments in the module surpass the amount of code that has been written   Here it is        usr bin env python    Copyright  C  2013-2014 Craig Phillips   All rights reserved     This requires some explaining   The point of this metaclass excercise is to   create a static abstract class that is in one way or another  dormant until   queried   I experimented with creating a singlton on import  but that did   not quite behave how I wanted it to   See now here  we are creating a class   called GsyncOptions  that on import  will do nothing except state that its   class creator is GsyncOptionsType   This means  docopt doesn t parse any   of the help document  nor does it start processing command line options    So importing this module becomes really efficient   The complicated bit   comes from requiring the GsyncOptions class to be static   By that  I mean   any property on it  may or may not exist  since they are not statically   defined  so I can t simply just define the class with a whole bunch of   properties that are  property  staticmethods      So here s how it works      Executing  from libgsync options import GsyncOptions  does nothing more   than load up this module  define the Type and the Class and import them   into the callers namespace   Simple      Invoking  GsyncOptions debug  for the first time  or any other property   causes the   metaclass     getattr   method to be called  since the class   is not instantiated as a class instance yet   The   getattr   method on   the type then initialises the class  GsyncOptions  via the   initialiseClass   method   This is the first and only time the class will actually have its   dictionary statically populated   The docopt module is invoked to parse the   usage document and generate command line options from it   These are then   paired with their defaults and what s in sys argv   After all that  we   setup some dynamic properties that could not be defined by their name in   the usage  before everything is then transplanted onto the actual class   object  or static class GsyncOptions       Another piece of magic  is to allow command line options to be set in   in their native form and be translated into argparse style properties      Finally  the GsyncListOptions class is actually where the options are   stored   This only acts as a mechanism for storing options as lists  to   allow aggregation of duplicate options or options that can be specified   multiple times   The   getattr   call hides this by default  returning the   last item in a property s list   However  if the entire list is required    calling the  list    method on the GsyncOptions class  returns a reference   to the GsyncListOptions class  which contains all of the same properties   but as lists and without the duplication of having them as both lists and   static singlton values      So this actually means that GsyncOptions is actually a static proxy class           And all this is neatly hidden within a closure for safe keeping  def GetGsyncOptionsType        class GsyncListOptions object             initialised   False      class GsyncOptionsType type           def   initialiseClass cls               if GsyncListOptions  GsyncListOptions  initialised  return              from docopt import docopt             from libgsync options import doc             from libgsync import   version                options   docopt                  doc   doc       version                    version     version                    options first   True                            paths   options pop   lt path gt    None              setattr cls   destination path   paths pop   if paths else None              setattr cls   source paths   paths              setattr cls   options   options               for k  v in options iteritems                    setattr cls  k  v               GsyncListOptions  GsyncListOptions  initialised   True          def list cls               return GsyncListOptions          def   getattr   cls  name               cls   initialiseClass               return getattr GsyncListOptions  name  -1           def   setattr   cls  name  value                 Substitut option names  --an-option-name for an option name             import re             name   re sub r           re sub r -        name               listvalue                     Ensure value is converted to a list type for GsyncListOptions             if isinstance value  list                   if value                      listvalue        value                 else                      listvalue     None               else                  listvalue     value                type   setattr   GsyncListOptions  name  listvalue         Cleanup this module to prevent tinkering      import sys     module   sys modules   name        del module   dict    GetGsyncOptionsType        return GsyncOptionsType    Our singlton abstract proxy class  class GsyncOptions object         metaclass     GetGsyncOptionsType

User · Answer

Python 3 update  There are  at this point  two key methods in a metaclass      prepare    and   new       prepare   lets you supply a custom mapping  such as an OrderedDict  to be used as the namespace while the class is being created   You must return an instance of whatever namespace you choose   If you don t implement   prepare   a normal dict is used     new   is responsible for the actual creation modification of the final class   A bare-bones  do-nothing-extra metaclass would like   class Meta type        def   prepare   metaclass  cls  bases           return dict        def   new   metacls  cls  bases  clsdict           return super     new   metacls  cls  bases  clsdict    A simple example   Say you want some simple validation code to run on your attributes -- like it must always be an int or a str   Without a metaclass  your class would look something like   class Person      weight   ValidateType  weight   int      age   ValidateType  age   int      name   ValidateType  name   str    As you can see  you have to repeat the name of the attribute twice   This makes typos possible along with irritating bugs   A simple metaclass can address that problem   class Person metaclass Validator       weight   ValidateType int      age   ValidateType int      name   ValidateType str    This is what the metaclass would look like  not using   prepare   since it is not needed    class Validator type       def   new   metacls  cls  bases  clsdict             search clsdict looking for ValidateType descriptors         for name  attr in clsdict items                if isinstance attr  ValidateType                   attr name   name                 attr attr         name           create final class and return it         return super     new   metacls  cls  bases  clsdict    A sample run of   p   Person   p weight   9 print p weight  p weight    9    produces   9 Traceback  most recent call last     File  simple meta py   line 36  in  lt module gt      p weight    9    File  simple meta py   line 24  in   set        self name  self type  value   TypeError  weight must be of type s   lt class  int  gt   got  9       Note   This example is simple enough it could have also been accomplished with a class decorator  but presumably an actual metaclass would be doing much more   The  ValidateType  class for reference   class ValidateType      def   init   self  type           self name   None    will be set by metaclass         self attr   None    will be set by metaclass         self type   type     def   get   self  inst  cls           if inst is None              return self         else              return inst   dict   self attr      def   set   self  inst  value           if not isinstance value  self type               raise TypeError   s must be of type s   s  got  r                          self name  self type  value           else              inst   dict   self attr    value

User · Answer

In addition to the published answers I can say that a metaclass defines the behaviour for a class  So  you can explicitly set your metaclass  Whenever Python gets a keyword class then it starts searching for the metaclass  If it s not found     the default metaclass type is used to create the class s object  Using the   metaclass   attribute  you can set metaclass of your class   class MyClass       metaclass     type      write here other method      write here one more method  print MyClass   metaclass      It ll produce the output like this   class  type    And  of course  you can create your own metaclass to define the behaviour of any class that are created using your class   For doing that  your default metaclass type class must be inherited as this is the main metaclass   class MyMetaClass type        metaclass     type      you can write here any behaviour you want  class MyTestClass       metaclass     MyMetaClass  Obj   MyTestClass   print Obj   metaclass    print MyMetaClass   metaclass      The output will be   class    main   MyMetaClass  class  type

User · Answer

The type   function can return the type of an object or create a new type    for example  we can create a Hi class with the type   function and do not  need to use this way with class Hi object    def func self  name  mike        print  Hi   s     name   Hi   type  Hi    object    dict hi func   h   Hi   h hi   Hi  mike   type Hi  type  type h    main   Hi   In addition to using type   to create classes dynamically  you can control creation behavior of class and use metaclass   According to the Python object model  the class is the object  so the class must be an instance of another certain class  By default  a Python class is instance of the type class  That is  type is metaclass of most of the built-in classes and metaclass of user-defined classes   class ListMetaclass type       def   new   cls  name  bases  attrs           attrs  add     lambda self  value  self append value          return type   new   cls  name  bases  attrs   class CustomList list  metaclass ListMetaclass       pass  lst   CustomList   lst add  custom list 1   lst add  custom list 2    lst   custom list 1    custom list 2     Magic will take effect when we passed keyword arguments in metaclass  it indicates the Python interpreter to create the CustomList through ListMetaclass  new     at this point  we can modify the class definition  for example  and add a new method and then return the revised definition

User · Answer

A class  in Python  is an object  and just like any other object  it is an instance of  something   This  something  is what is termed as a Metaclass  This metaclass is a special type of class that creates other class s objects  Hence  metaclass is responsible for making new classes  This allows the programmer to customize the way classes are generated   To create a metaclass  overriding of new   and init   methods is usually done  new   can be overridden to change the way objects are created  while init   can be overridden to change the way of initializing the object  Metaclass can be created by a number of ways  One of the ways is to use type   function  type   function  when called with 3 parameters  creates a metaclass  The parameters are  -   Class Name Tuple having base classes inherited by class A dictionary having all class methods and class variables   Another way of creating a metaclass comprises of  metaclass  keyword  Define the metaclass as a simple class  In the parameters of inherited class  pass metaclass metaclass name  Metaclass can be specifically used in the following situations  -   when a particular effect has to be applied to all the subclasses Automatic change of class  on creation  is required By API developers

User · Answer

A metaclass is the class of a class  A class defines how an instance of the class  i e  an object  behaves while a metaclass defines how a class behaves  A class is an instance of a metaclass   While in Python you can use arbitrary callables for metaclasses  like Jerub shows   the better approach is to make it an actual class itself  type is the usual metaclass in Python  type is itself a class  and it is its own type  You won t be able to recreate something like type purely in Python  but Python cheats a little  To create your own metaclass in Python you really just want to subclass type   A metaclass is most commonly used as a class-factory  When you create an object by calling the class  Python creates a new class  when it executes the  class  statement  by calling the metaclass  Combined with the normal   init   and   new   methods  metaclasses therefore allow you to do  extra things  when creating a class  like registering the new class with some registry or replace the class with something else entirely   When the class statement is executed  Python first executes the body of the class statement as a normal block of code  The resulting namespace  a dict  holds the attributes of the class-to-be  The metaclass is determined by looking at the baseclasses of the class-to-be  metaclasses are inherited   at the   metaclass   attribute of the class-to-be  if any  or the   metaclass   global variable  The metaclass is then called with the name  bases and attributes of the class to instantiate it   However  metaclasses actually define the type of a class  not just a factory for it  so you can do much more with them  You can  for instance  define normal methods on the metaclass  These metaclass-methods are like classmethods in that they can be called on the class without an instance  but they are also not like classmethods in that they cannot be called on an instance of the class  type   subclasses     is an example of a method on the type metaclass  You can also define the normal  magic  methods  like   add      iter   and   getattr    to implement or change how the class behaves   Here s an aggregated example of the bits and pieces   def make hook f          Decorator to turn  foo  method into    foo           f is hook   1     return f  class MyType type       def   new   mcls  name  bases  attrs            if name startswith  None                return None            Go over attributes and see if they should be renamed          newattrs              for attrname  attrvalue in attrs iteritems                if getattr attrvalue   is hook   0                   newattrs     s      attrname    attrvalue             else                  newattrs attrname    attrvalue          return super MyType  mcls    new   mcls  name  bases  newattrs       def   init   self  name  bases  attrs           super MyType  self    init   name  bases  attrs             classregistry register self  self interfaces          print  Would register class  s now     self      def   add   self  other           class AutoClass self  other               pass         return AutoClass           Alternatively  to autogenerate the classname as well as the class            return type self   name     other   name     self  other            def unregister self             classregistry unregister self          print  Would unregister class  s now     self  class MyObject        metaclass     MyType   class NoneSample MyObject       pass    Will print  NoneType None  print type NoneSample   repr NoneSample   class Example MyObject       def   init   self  value           self value   value      make hook     def add self  other           return self   class   self value   other value     Will unregister the class Example unregister    inst   Example 10    Will fail with an AttributeError  inst unregister    print inst   inst class Sibling MyObject       pass  ExampleSibling   Example   Sibling   ExampleSibling is now a subclass of both Example and Sibling  with no   content of its own  although it will believe it s called  AutoClass  print ExampleSibling print ExampleSibling   mro

User · Answer

Others have explained how metaclasses work and how they fit into the Python type system  Here s an example of what they can be used for  In a testing framework I wrote  I wanted to keep track of the order in which classes were defined  so that I could later instantiate them in this order  I found it easiest to do this using a metaclass   class MyMeta type        counter   0      def   init   cls  name  bases  dic           type   init   cls  name  bases  dic          cls  order   MyMeta counter         MyMeta counter    1  class MyType object                  Python 2       metaclass     MyMeta  class MyType metaclass MyMeta        Python 3     pass   Anything that s a subclass of MyType then gets a class attribute  order that records the order in which the classes were defined

User · Answer

Here s another example of what it can be used for    You can use the metaclass to change the function of its instance  the class     class MetaMemberControl type         slots              classmethod     def   prepare   mcs  f cls name  f cls parents     f cls means  future class                     meta args None  meta options None      meta args and meta options is not necessarily needed  just so you know          f cls attr   dict           if not  do something or if you want to define your cool stuff of dict                  return dict make your special dict None          else              return f cls attr      def   new   mcs  f cls name  f cls parents  f cls attr                  meta args None  meta options None            original getattr   f cls attr get    getattribute             original setattr   f cls attr get    setattr              def init getattr self  item               if not item startswith          you can set break points at here                 alias name         item                 if alias name in f cls attr    slots                          item   alias name             if original getattr is not None                  return original getattr self  item              else                  return super eval f cls name   self    getattribute   item           def init setattr self  key  value               if not key startswith      and        key  in f cls attr    slots                      raise AttributeError f you can t modify private members   key                if original setattr is not None                  original setattr self  key  value              else                  super eval f cls name   self    setattr   key  value           f cls attr    getattribute       init getattr         f cls attr    setattr       init setattr          cls   super     new   mcs  f cls name  f cls parents  f cls attr          return cls   class Human metaclass MetaMemberControl         slots        age     name        def   init   self  name  age           self  name   name         self  age   age      def   getattribute   self  item                       is just for IDE recognize                      return super     getattribute   item           with MetaMemberControl then you don t have to write as following      property     def name self           return self  name       property     def age self           return self  age           def test demo        human   Human  Carson   27        human age   18    you can t modify private members  age   lt -- this is defined by yourself        human k   18     Human  object has no attribute  k    lt -- system error      age1   human  age    It s OK  although the IDE will show some warnings   Access to a protected member  age of a class       age2   human age    It s OK  see below              if you do not define    getattribute    at the class of Human      the IDE will show you  Unresolved attribute reference  age  for class  Human      but it s ok on running since the MetaMemberControl will help you            if   name         main         test demo      The metaclass is powerful  there are many things  such as monkey magic  you can do with it  but be careful this may only be known to you

User · Answer

One use for metaclasses is adding new properties and methods to an instance automatically   For example  if you look at Django models  their definition looks a bit confusing  It looks as if you are only defining class properties   class Person models Model       first name   models CharField max length 30      last name   models CharField max length 30    However  at runtime the Person objects are filled with all sorts of useful methods  See the source for some amazing metaclassery

User · Answer

type is actually a metaclass -- a class that creates another classes  Most metaclass are the subclasses of type  The metaclass receives the new class as its first argument and provide access to class object with details as mentioned below    gt  gt  gt  class MetaClass type           def   init   cls  name  bases  attrs               print   class name   s   name               print   Defining class  s   cls              print  Bases  s     bases              print  Attributes               for  name  value  in attrs items                    print    s   r    name  value          gt  gt  gt  class NewClass object  metaclass MetaClass          get choch  dairy       class name  NewClass Bases  lt class  object  gt    Defining class  lt class  NewClass  gt  get choch   dairy    module     builtins    qualname     NewClass    Note   Notice that the class was not instantiated at any time  the simple act of creating the class triggered execution of the metaclass

User · Answer

The tl dr version  The type obj  function gets you the type of an object    The type   of a class is its metaclass   To use a metaclass   class Foo object         metaclass     MyMetaClass   type is its own metaclass  The class of a class is a metaclass-- the body of a class is the arguments passed to the metaclass that is used to construct the class   Here you can read about how to use metaclasses to customize class construction

User · Answer

Role of a metaclass    call     method when creating a class instance  If you ve done Python programming for more than a few months you ll eventually stumble upon code that looks like this     define a class class SomeClass object                   some definition here                  create an instance of it instance   SomeClass      then call the object as if it s a function result   instance  foo    bar     The latter is possible when you implement the   call     magic method on the class   class SomeClass object                   some definition here                    def   call   self  foo  bar           return bar   foo   The   call     method is invoked when an instance of a class is used as a callable  But as we ve seen from previous answers a class itself is an instance of a metaclass  so when we use the class as a callable  i e  when we create an instance of it  we re actually calling its metaclass    call     method  At this point most Python programmers are a bit confused because they ve been told that when creating an instance like this instance   SomeClass   you re calling its   init     method  Some who ve dug a bit deeper know that before   init     there s   new      Well  today another layer of truth is being revealed  before   new     there s the metaclass    call       Let s study the method call chain from specifically the perspective of creating an instance of a class   This is a metaclass that logs exactly the moment before an instance is created and the moment it s about to return it   class Meta 1 type       def   call   cls           print  Meta 1   call     before creating an instance of    cls         instance   super Meta 1  cls    call             print  Meta 1   call     about to return instance           return instance   This is a class that uses that metaclass  class Class 1 object          metaclass     Meta 1      def   new   cls           print  Class 1   new     before creating an instance           instance   super Class 1  cls    new   cls          print  Class 1   new     about to return instance           return instance      def   init   self           print  entering Class 1   init     for instance initialization           super Class 1 self    init             print  exiting Class 1   init         And now let s create an instance of Class 1  instance   Class 1     Meta 1   call     before creating an instance of  lt class    main   Class 1  gt     Class 1   new     before creating an instance    Class 1   new     about to return instance    entering Class 1   init     for instance initialization    exiting Class 1   init        Meta 1   call     about to return instance    Observe that the code above doesn t actually do anything more than logging the tasks  Each method delegates the actual work to its parent s implementation  thus keeping the default behavior  Since type is Meta 1 s parent class  type being the default parent metaclass  and considering the ordering sequence of the output above  we now have a clue as to what would be the pseudo implementation of type   call       class type      def   call   cls   args    kwarg                  maybe a few things done to cls here            then we call   new     on the class to create an instance         instance   cls   new   cls   args    kwargs                 maybe a few things done to the instance here            then we initialize the instance with its   init     method         instance   init    args    kwargs                 maybe a few more things done to instance here            then we return it         return instance   We can see that the metaclass    call     method is the one that s called first  It then delegates creation of the instance to the class s   new     method and initialization to the instance s   init      It s also the one that ultimately returns the instance   From the above it stems that the metaclass    call     is also given the opportunity to decide whether or not a call to Class 1   new     or Class 1   init     will eventually be made  Over the course of its execution it could actually return an object that hasn t been touched by either of these methods  Take for example this approach to the singleton pattern   class Meta 2 type       singletons           def   call   cls   args    kwargs           if cls in Meta 2 singletons                we return the only instance and skip a call to   new                   and   init                 print      singleton returning from Meta 2   call                            skipping creation of new instance   format cls               return Meta 2 singletons cls             else if the singleton isn t present we proceed as usual         print  Meta 2   call     before creating an instance           instance   super Meta 2  cls    call    args    kwargs          Meta 2 singletons cls    instance         print  Meta 2   call     returning new instance           return instance  class Class 2 object          metaclass     Meta 2      def   new   cls   args    kwargs           print  Class 2   new     before creating instance           instance   super Class 2  cls    new   cls          print  Class 2   new     returning instance           return instance      def   init   self   args    kwargs           print  entering Class 2   init     for initialization           super Class 2  self    init             print  exiting Class 2   init         Let s observe what happens when repeatedly trying to create an object of type Class 2  a   Class 2     Meta 2   call     before creating an instance    Class 2   new     before creating instance    Class 2   new     returning instance    entering Class 2   init     for initialization    exiting Class 2   init        Meta 2   call     returning new instance   b   Class 2      lt class    main   Class 2  gt  singleton returning from Meta 2   call      skipping creation of new instance   c   Class 2      lt class    main   Class 2  gt  singleton returning from Meta 2   call      skipping creation of new instance   a is b is c   True

User · Answer

I think the ONLamp introduction to metaclass programming is well written and gives a really good introduction to the topic despite being several years old already   http   www onlamp com pub a python 2003 04 17 metaclasses html  archived at https   web archive org web 20080206005253 http   www onlamp com pub a python 2003 04 17 metaclasses html   In short  A class is a blueprint for the creation of an instance  a metaclass is a blueprint for the creation of a class  It can be easily seen that in Python classes need to be first-class objects too to enable this behavior   I ve never written one myself  but I think one of the nicest uses of metaclasses can be seen in the Django framework  The model classes use a metaclass approach to enable a declarative style of writing new models or form classes  While the metaclass is creating the class  all members get the possibility to customize the class itself    Creating a new model The metaclass enabling this   The thing that s left to say is  If you don t know what metaclasses are  the probability that you will not need them is 99

User · Answer

Classes as objects Before understanding metaclasses  you need to master classes in Python  And Python has a very peculiar idea of what classes are  borrowed from the Smalltalk language  In most languages  classes are just pieces of code that describe how to produce an object  That s kinda true in Python too   gt  gt  gt  class ObjectCreator object             pass       gt  gt  gt  my object   ObjectCreator    gt  gt  gt  print my object   lt   main   ObjectCreator object at 0x8974f2c gt   But classes are more than that in Python  Classes are objects too  Yes  objects  As soon as you use the keyword class  Python executes it and creates an OBJECT  The instruction  gt  gt  gt  class ObjectCreator object             pass      creates in memory an object with the name  quot ObjectCreator quot   This object  the class  is itself capable of creating objects  the instances   and this is why it s a class  But still  it s an object  and therefore   you can assign it to a variable you can copy it you can add attributes to it you can pass it as a function parameter  e g    gt  gt  gt  print ObjectCreator    you can print a class because it s an object  lt class    main   ObjectCreator  gt   gt  gt  gt  def echo o             print o       gt  gt  gt  echo ObjectCreator    you can pass a class as a parameter  lt class    main   ObjectCreator  gt   gt  gt  gt  print hasattr ObjectCreator   new attribute    False  gt  gt  gt  ObjectCreator new attribute    foo    you can add attributes to a class  gt  gt  gt  print hasattr ObjectCreator   new attribute    True  gt  gt  gt  print ObjectCreator new attribute  foo  gt  gt  gt  ObjectCreatorMirror   ObjectCreator   you can assign a class to a variable  gt  gt  gt  print ObjectCreatorMirror new attribute  foo  gt  gt  gt  print ObjectCreatorMirror     lt   main   ObjectCreator object at 0x8997b4c gt   Creating classes dynamically Since classes are objects  you can create them on the fly  like any object  First  you can create a class in a function using class   gt  gt  gt  def choose class name           if name     foo               class Foo object                   pass             return Foo   return the class  not an instance         else              class Bar object                   pass             return Bar      gt  gt  gt  MyClass   choose class  foo    gt  gt  gt  print MyClass    the function returns a class  not an instance  lt class    main   Foo  gt   gt  gt  gt  print MyClass      you can create an object from this class  lt   main   Foo object at 0x89c6d4c gt   But it s not so dynamic  since you still have to write the whole class yourself  Since classes are objects  they must be generated by something  When you use the class keyword  Python creates this object automatically  But as with most things in Python  it gives you a way to do it manually  Remember the function type  The good old function that lets you know what type an object is   gt  gt  gt  print type 1    lt type  int  gt   gt  gt  gt  print type  quot 1 quot     lt type  str  gt   gt  gt  gt  print type ObjectCreator    lt type  type  gt   gt  gt  gt  print type ObjectCreator      lt class    main   ObjectCreator  gt   Well  type has a completely different ability  it can also create classes on the fly  type can take the description of a class as parameters  and return a class   I  know  it s silly that the same function can have two completely different uses according to the parameters you pass to it  It s an issue due to backward compatibility in Python  type works this way  type name  bases  attrs   Where   name  name of the class bases  tuple of the parent class  for inheritance  can be empty  attrs  dictionary containing attributes names and values  e g    gt  gt  gt  class MyShinyClass object             pass  can be created manually this way   gt  gt  gt  MyShinyClass   type  MyShinyClass             returns a class object  gt  gt  gt  print MyShinyClass   lt class    main   MyShinyClass  gt   gt  gt  gt  print MyShinyClass      create an instance with the class  lt   main   MyShinyClass object at 0x8997cec gt   You ll notice that we use  quot MyShinyClass quot  as the name of the class and as the variable to hold the class reference  They can be different  but there is no reason to complicate things  type accepts a dictionary to define the attributes of the class  So   gt  gt  gt  class Foo object             bar   True  Can be translated to   gt  gt  gt  Foo   type  Foo         bar  True    And used as a normal class   gt  gt  gt  print Foo   lt class    main   Foo  gt   gt  gt  gt  print Foo bar  True  gt  gt  gt  f   Foo    gt  gt  gt  print f   lt   main   Foo object at 0x8a9b84c gt   gt  gt  gt  print f bar  True  And of course  you can inherit from it  so   gt  gt  gt    class FooChild Foo               pass  would be   gt  gt  gt  FooChild   type  FooChild    Foo         gt  gt  gt  print FooChild   lt class    main   FooChild  gt   gt  gt  gt  print FooChild bar    bar is inherited from Foo True  Eventually  you ll want to add methods to your class  Just define a function with the proper signature and assign it as an attribute   gt  gt  gt  def echo bar self             print self bar       gt  gt  gt  FooChild   type  FooChild    Foo      echo bar   echo bar    gt  gt  gt  hasattr Foo   echo bar   False  gt  gt  gt  hasattr FooChild   echo bar   True  gt  gt  gt  my foo   FooChild    gt  gt  gt  my foo echo bar   True  And you can add even more methods after you dynamically create the class  just like adding methods to a normally created class object   gt  gt  gt  def echo bar more self             print  yet another method        gt  gt  gt  FooChild echo bar more   echo bar more  gt  gt  gt  hasattr FooChild   echo bar more   True  You see where we are going  in Python  classes are objects  and you can create a class on the fly  dynamically  This is what Python does when you use the keyword class  and it does so by using a metaclass  What are metaclasses  finally  Metaclasses are the  stuff  that creates classes  You define classes in order to create objects  right  But we learned that Python classes are objects  Well  metaclasses are what create these objects  They are the classes  classes  you can picture them this way  MyClass   MetaClass   my object   MyClass    You ve seen that type lets you do something like this  MyClass   type  MyClass            It s because the function type is in fact a metaclass  type is the metaclass Python uses to create all classes behind the scenes  Now you wonder why the heck is it written in lowercase  and not Type  Well  I guess it s a matter of consistency with str  the class that creates strings objects  and int the class that creates integer objects  type is just the class that creates class objects  You see that by checking the   class   attribute  Everything  and I mean everything  is an object in Python  That includes ints  strings  functions and classes  All of them are objects  And all of them have been created from a class   gt  gt  gt  age   35  gt  gt  gt  age   class    lt type  int  gt   gt  gt  gt  name    bob   gt  gt  gt  name   class    lt type  str  gt   gt  gt  gt  def foo    pass  gt  gt  gt  foo   class    lt type  function  gt   gt  gt  gt  class Bar object   pass  gt  gt  gt  b   Bar    gt  gt  gt  b   class    lt class    main   Bar  gt   Now  what is the   class   of any   class      gt  gt  gt  age   class     class    lt type  type  gt   gt  gt  gt  name   class     class    lt type  type  gt   gt  gt  gt  foo   class     class    lt type  type  gt   gt  gt  gt  b   class     class    lt type  type  gt   So  a metaclass is just the stuff that creates class objects  You can call it a  class factory  if you wish  type is the built-in metaclass Python uses  but of course  you can create your own metaclass  The   metaclass   attribute In Python 2  you can add a   metaclass   attribute when you write a class  see next section for the Python 3 syntax   class Foo object         metaclass     something               If you do so  Python will use the metaclass to create the class Foo  Careful  it s tricky  You write class Foo object  first  but the class object Foo is not created in memory yet  Python will look for   metaclass   in the class definition  If it finds it  it will use it to create the object class Foo  If it doesn t  it will use type to create the class  Read that several times  When you do  class Foo Bar       pass  Python does the following  Is there a   metaclass   attribute in Foo  If yes  create in-memory a class object  I said a class object  stay with me here   with the name Foo by using what is in   metaclass    If Python can t find   metaclass    it will look for a   metaclass   at the MODULE level  and try to do the same  but only for classes that don t inherit anything  basically old-style classes   Then if it can t find any   metaclass   at all  it will use the Bar s  the first parent  own metaclass  which might be the default type  to create the class object  Be careful here that the   metaclass   attribute will not be inherited  the metaclass of the parent  Bar   class    will be  If Bar used a   metaclass   attribute that created Bar with type    and not type   new       the subclasses will not inherit that behavior  Now the big question is  what can you put in   metaclass    The answer is something that can create a class  And what can create a class  type  or anything that subclasses or uses it  Metaclasses in Python 3 The syntax to set the metaclass has been changed in Python 3  class Foo object  metaclass something            i e  the   metaclass   attribute is no longer used  in favor of a keyword argument in the list of base classes  The behavior of metaclasses however stays largely the same  One thing added to metaclasses in Python 3 is that you can also pass attributes as keyword-arguments into a metaclass  like so  class Foo object  metaclass something  kwarg1 value1  kwarg2 value2            Read the section below for how python handles this  Custom metaclasses The main purpose of a metaclass is to change the class automatically  when it s created  You usually do this for APIs  where you want to create classes matching the current context  Imagine a stupid example  where you decide that all classes in your module should have their attributes written in uppercase  There are several ways to do this  but one way is to set   metaclass   at the module level  This way  all classes of this module will be created using this metaclass  and we just have to tell the metaclass to turn all attributes to uppercase  Luckily    metaclass   can actually be any callable  it doesn t need to be a formal class  I know  something with  class  in its name doesn t need to be a class  go figure    but it s helpful   So we will start with a simple example  by using a function    the metaclass will automatically get passed the same argument   that you usually pass to  type  def upper attr future class name  future class parents  future class attrs        quot  quot  quot        Return a class object  with the list of its attribute turned       into uppercase       quot  quot  quot        pick up any attribute that doesn t start with      and uppercase it     uppercase attrs             attr if attr startswith  quot    quot   else attr upper    v         for attr  v in future class attrs items                let  type  do the class creation     return type future class name  future class parents  uppercase attrs     metaclass     upper attr   this will affect all classes in the module  class Foo      global   metaclass   won t work with  quot object quot  though       but we can define   metaclass   here instead to affect only this class       and this will work with  quot object quot  children     bar    bip   Let s check   gt  gt  gt  hasattr Foo   bar   False  gt  gt  gt  hasattr Foo   BAR   True  gt  gt  gt  Foo BAR  bip   Now  let s do exactly the same  but using a real class for a metaclass    remember that  type  is actually a class like  str  and  int    so you can inherit from it class UpperAttrMetaclass type           new   is the method called before   init         it s the method that creates the object and returns it       while   init   just initializes the object passed as parameter       you rarely use   new    except when you want to control how the object       is created        here the created object is the class  and we want to customize it       so we override   new         you can do some stuff in   init   too if you wish       some advanced use involves overriding   call   as well  but we won t       see this     def   new   upperattr metaclass  future class name                  future class parents  future class attrs           uppercase attrs                 attr if attr startswith  quot    quot   else attr upper    v             for attr  v in future class attrs items                     return type future class name  future class parents  uppercase attrs   Let s rewrite the above  but with shorter and more realistic variable names now that we know what they mean  class UpperAttrMetaclass type       def   new   cls  clsname  bases  attrs           uppercase attrs                 attr if attr startswith  quot    quot   else attr upper    v             for attr  v in attrs items                     return type clsname  bases  uppercase attrs   You may have noticed the extra argument cls  There is nothing special about it    new   always receives the class it s defined in  as the first parameter  Just like you have self for ordinary methods which receive the instance as the first parameter  or the defining class for class methods  But this is not proper OOP  We are calling type directly and we aren t overriding or calling the parent s   new    Let s do that instead  class UpperAttrMetaclass type       def   new   cls  clsname  bases  attrs           uppercase attrs                 attr if attr startswith  quot    quot   else attr upper    v             for attr  v in attrs items                     return type   new   cls  clsname  bases  uppercase attrs   We can make it even cleaner by using super  which will ease inheritance  because yes  you can have metaclasses  inheriting from metaclasses  inheriting from type   class UpperAttrMetaclass type       def   new   cls  clsname  bases  attrs           uppercase attrs                 attr if attr startswith  quot    quot   else attr upper    v             for attr  v in attrs items                     return super UpperAttrMetaclass  cls    new                cls  clsname  bases  uppercase attrs   Oh  and in python 3 if you do this call with keyword arguments  like this  class Foo object  metaclass MyMetaclass  kwarg1 value1            It translates to this in the metaclass to use it  class MyMetaclass type       def   new   cls  clsname  bases  dct  kwargs1 default                That s it  There is really nothing more about metaclasses  The reason behind the complexity of the code using metaclasses is not because of metaclasses  it s because you usually use metaclasses to do twisted stuff relying on introspection  manipulating inheritance  vars such as   dict    etc  Indeed  metaclasses are especially useful to do black magic  and therefore complicated stuff  But by themselves  they are simple   intercept a class creation modify the class return the modified class  Why would you use metaclasses classes instead of functions  Since   metaclass   can accept any callable  why would you use a class since it s obviously more complicated  There are several reasons to do so   The intention is clear  When you read UpperAttrMetaclass type   you know what s going to follow You can use OOP  Metaclass can inherit from metaclass  override parent methods  Metaclasses can even use metaclasses  Subclasses of a class will be instances of its metaclass if you specified a metaclass-class  but not with a metaclass-function  You can structure your code better  You never use metaclasses for something as trivial as the above example  It s usually for something complicated  Having the ability to make several methods and group them in one class is very useful to make the code easier to read  You can hook on   new      init   and   call    Which will allow you to do different stuff  Even if usually you can do it all in   new    some people are just more comfortable using   init    These are called metaclasses  damn it  It must mean something   Why would you use metaclasses  Now the big question  Why would you use some obscure error-prone feature  Well  usually you don t   Metaclasses are deeper magic that 99  of users should never worry about it  If you wonder whether you need them  you don t  the people who actually need them to know with certainty that they need them and don t need an explanation about why    Python Guru Tim Peters The main use case for a metaclass is creating an API  A typical example of this is the Django ORM  It allows you to define something like this  class Person models Model       name   models CharField max length 30      age   models IntegerField    But if you do this  person   Person name  bob   age  35   print person age   It won t return an IntegerField object  It will return an int  and can even take it directly from the database  This is possible because models Model defines   metaclass   and it uses some magic that will turn the Person you just defined with simple statements into a complex hook to a database field  Django makes something complex look simple by exposing a simple API and using metaclasses  recreating code from this API to do the real job behind the scenes  The last word First  you know that classes are objects that can create instances  Well  in fact  classes are themselves instances  Of metaclasses   gt  gt  gt  class Foo object   pass  gt  gt  gt  id Foo  142630324  Everything is an object in Python  and they are all either instance of classes or instances of metaclasses  Except for type  type is actually its own metaclass  This is not something you could reproduce in pure Python  and is done by cheating a little bit at the implementation level  Secondly  metaclasses are complicated  You may not want to use them for very simple class alterations  You can change classes by using two different techniques   monkey patching class decorators  99  of the time you need class alteration  you are better off using these  But 98  of the time  you don t need class alteration at all

User · Answer

Note  this answer is for Python 2 x as it was written in 2008  metaclasses are slightly different in 3 x   Metaclasses are the secret sauce that make  class  work  The default metaclass for a new style object is called  type    class type object       type object  - gt  the object s type      type name  bases  dict  - gt  a new type   Metaclasses take 3 args   name    bases  and  dict   Here is where the secret starts  Look for where name  bases and the dict come from in this example class definition   class ThisIsTheName Bases  Are  Here       All the code here     def doesIs create  a           dict   Lets define a metaclass that will demonstrate how  class   calls it   def test metaclass name  bases  dict       print  The Class Name is   name     print  The Class Bases are   bases     print  The dict has   len dict    elems  the keys are   dict keys        return  yellow   class TestName object  None  int  1         metaclass     test metaclass     foo   1     def baz self  arr           pass  print  TestName      repr TestName     output   gt   The Class Name is TestName The Class Bases are   lt type  object  gt   None   lt type  int  gt   1  The dict has 4 elems  the keys are   baz      module      foo      metaclass     TestName     yellow    And now  an example that actually means something  this will automatically make the variables in the list  attributes  set on the class  and set to None   def init attributes name  bases  dict       if  attributes  in dict          for attr in dict  attributes                dict attr    None      return type name  bases  dict   class Initialised object         metaclass     init attributes     attributes     foo    bar    baz    print  foo   gt    Initialised foo   output  gt  foo   gt  None   Note that the magic behaviour that Initialised gains by having the metaclass init attributes is not passed onto a subclass of Initialised   Here is an even more concrete example  showing how you can subclass  type  to make a metaclass that performs an action when the class is created  This is quite tricky   class MetaSingleton type       instance   None     def   call   cls   args    kw           if cls instance is None              cls instance   super MetaSingleton  cls    call    args    kw          return cls instance  class Foo object         metaclass     MetaSingleton  a   Foo   b   Foo   assert a is b

User · Answer

What are metaclasses  What do you use them for    TLDR  A metaclass instantiates and defines behavior for a class just like a class instantiates and defines behavior for an instance    Pseudocode    gt  gt  gt  Class      instance   The above should look familiar  Well  where does Class come from  It s an instance of a metaclass  also pseudocode     gt  gt  gt  Metaclass      Class   In real code  we can pass the default metaclass  type  everything we need to instantiate a class and we get a class    gt  gt  gt  type  Foo    object          requires a name  bases  and a namespace  lt class    main   Foo  gt    Putting it differently   A class is to an instance as a metaclass is to a class    When we instantiate an object  we get an instance    gt  gt  gt  object                              instantiation of class  lt object object at 0x7f9069b4e0b0 gt        instance   Likewise  when we define a class explicitly with the default metaclass  type  we instantiate it    gt  gt  gt  type  Object    object              instantiation of metaclass  lt class    main   Object  gt                instance  Put another way  a class is an instance of a metaclass    gt  gt  gt  isinstance object  type  True  Put a third way  a metaclass is a class s class    gt  gt  gt  type object     type True  gt  gt  gt  object   class    lt class  type  gt     When you write a class definition and Python executes it  it uses a metaclass to instantiate the class object  which will  in turn  be used to instantiate instances of that class    Just as we can use class definitions to change how custom object instances behave  we can use a metaclass class definition to change the way a class object behaves   What can they be used for  From the docs      The potential uses for metaclasses are boundless  Some ideas that have been explored include logging  interface checking  automatic delegation  automatic property creation  proxies  frameworks  and automatic resource locking synchronization    Nevertheless  it is usually encouraged for users to avoid using metaclasses unless absolutely necessary   You use a metaclass every time you create a class   When you write a class definition  for example  like this   class Foo object         demo    You instantiate a class object    gt  gt  gt  Foo  lt class    main   Foo  gt   gt  gt  gt  isinstance Foo  type   isinstance Foo  object   True  True    It is the same as functionally calling type with the appropriate arguments and assigning the result to a variable of that name   name    Foo  bases    object   namespace       doc      demo   Foo   type name  bases  namespace    Note  some things automatically get added to the   dict    i e   the namespace    gt  gt  gt  Foo   dict   dict proxy     dict      lt attribute    dict    of  Foo  objects gt       module        main        weakref      lt attribute    weakref     of  Foo  objects gt      doc      demo      The metaclass of the object we created  in both cases  is type     A side-note on the contents of the class   dict      module   is there because classes must know where they are defined  and    dict   and   weakref   are there because we don t define   slots   - if we define   slots   we ll save a bit of space in the instances  as we can disallow   dict   and   weakref   by excluding them  For example    gt  gt  gt  Baz   type  Bar    object        doc      demo      slots           gt  gt  gt  Baz   dict   mappingproxy     doc      demo      slots            module        main            but I digress    We can extend type just like any other class definition   Here s the default   repr   of classes    gt  gt  gt  Foo  lt class    main   Foo  gt    One of the most valuable things we can do by default in writing a Python object is to provide it with a good   repr    When we call help repr  we learn that there s a good test for a   repr   that also requires a test for equality - obj    eval repr obj    The following simple implementation of   repr   and   eq   for class instances of our type class provides us with a demonstration that may improve on the default   repr   of classes   class Type type       def   repr   cls                        gt  gt  gt  Baz         Type  Baz    Foo  Bar        module        main        doc     None            gt  gt  gt  eval repr Baz           Type  Baz    Foo  Bar        module        main        doc     None                       metaname   type cls    name           name   cls   name           parents        join b   name   for b in cls   bases            if parents              parents                namespace        join      join             repr k   repr v  if not isinstance v  type  else v   name                    for k  v in cls   dict   items            return   0     1       2       3      format metaname  name  parents  namespace      def   eq   cls  other                        gt  gt  gt  Baz    eval repr Baz           True                                 return  cls   name    cls   bases    cls   dict                         other   name    other   bases    other   dict      So now when we create an object with this metaclass  the   repr   echoed on the command line provides a much less ugly sight than the default    gt  gt  gt  class Bar object   pass  gt  gt  gt  Baz   Type  Baz    Foo  Bar        module        main        doc     None    gt  gt  gt  Baz Type  Baz    Foo  Bar        module        main        doc     None     With a nice   repr   defined for the class instance  we have a stronger ability to debug our code  However  much further checking with eval repr Class   is unlikely  as functions would be rather impossible to eval from their default   repr   s    An expected usage    prepare   a namespace  If  for example  we want to know in what order a class s methods are created in  we could provide an ordered dict as the namespace of the class  We would do this with   prepare   which returns the namespace dict for the class if it is implemented in Python 3    from collections import OrderedDict  class OrderedType Type        classmethod     def   prepare   metacls  name  bases    kwargs           return OrderedDict       def   new   cls  name  bases  namespace    kwargs           result   Type   new   cls  name  bases  dict namespace           result members   tuple namespace          return result   And usage   class OrderedMethodsObject object  metaclass OrderedType       def method1 self   pass     def method2 self   pass     def method3 self   pass     def method4 self   pass   And now we have a record of the order in which these methods  and other class attributes  were created    gt  gt  gt  OrderedMethodsObject members     module        qualname      method1    method2    method3    method4     Note  this example was adapted from the documentation - the new enum in the standard library does this   So what we did was instantiate a metaclass by creating a class  We can also treat the metaclass as we would any other class  It has a method resolution order    gt  gt  gt  inspect getmro OrderedType    lt class    main   OrderedType  gt    lt class    main   Type  gt    lt class  type  gt    lt class  object  gt     And it has approximately the correct repr  which we can no longer eval unless we can find a way to represent our functions      gt  gt  gt  OrderedMethodsObject OrderedType  OrderedMethodsObject    object      method1    lt function OrderedMethodsObject method1 at 0x0000000002DB01E0 gt    members       module        qualname      method1    method2    method3    method4     method3    lt function OrderedMet hodsObject method3 at 0x0000000002DB02F0 gt    method2    lt function OrderedMethodsObject method2 at 0x0000000002DB0268 gt      module        main        weakref      lt attribute    weakref    of  OrderedMethodsObject  objects gt      doc     None     d ict      lt attribute    dict    of  OrderedMethodsObject  objects gt    method4    lt function OrderedMethodsObject method4 at 0x0000000002DB0378 gt

[python] What are metaclasses in Python?

Examples related to python

Examples related to oop

Examples related to metaclass

Examples related to python-class

Examples related to python-datamodel