Can someone explain all in Python

Question

I have been using Python more and more  and I keep seeing the variable   all   set in different   init   py files   Can someone explain what this does

User · Answer

all   affects how from foo import   works   Code that is inside a module body  but not in the body of a function or class  may use an asterisk     in a from statement   from foo import     The   requests that all attributes of module foo  except those beginning with underscores  be bound as global variables in the importing module  When foo has an attribute   all    the attribute s value is the list of the names that are bound by this type of from statement    If foo is a package and its   init   py defines a list named   all    it is taken to be the list of submodule names that should be imported when from foo import   is encountered  If   all   is not defined  the statement from foo import   imports whatever names are defined in the package  This includes any names defined  and submodules explicitly loaded  by   init   py   Note that   all   doesn t have to be a list  As per the documentation on the import statement  if defined    all   must be a sequence of strings which are names defined or imported by the module  So you may as well use a tuple to save some memory and CPU cycles  Just don t forget a comma in case the module defines a single public name     all       some name      See also Why is    import      bad

User · Answer

Short answer    all   affects from  lt module gt  import   statements   Long answer  Consider this example   foo  -- bar py  --   init   py   In foo   init   py     Implicit  If we don t define   all    then from foo import   will only import names defined in foo   init   py   Explicit  If we define   all         then from foo import   will import nothing   Explicit  If we define   all        lt name1 gt          then from foo import   will only import those names    Note that in the implicit case  python won t import names starting with    However  you can force importing such names using   all     You can view the Python document here

User · Answer

This is defined in PEP8 here   Global Variable Names  Let s hope that these variables are meant for use inside one module only   The conventions are about the same as those for functions  Modules that are designed for use via from M import   should use the   all   mechanism to prevent exporting globals  or use the older convention of prefixing such globals with an underscore  which you might want to do to indicate these globals are  quot module non-public quot     PEP8 provides coding conventions for the Python code comprising the standard library in the main Python distribution  The more you follow this  closer you are to the original intent

User · Answer

Explain   all   in Python       I keep seeing the variable   all   set in different   init   py files       What does this do    What does   all   do   It declares the semantically  public  names from a module  If there is a name in   all    users are expected to use it  and they can have the expectation that it will not change    It also will have programmatic affects   import      all   in a module  e g  module py     all       foo    Bar     means that when you import   from the module  only those names in the   all   are imported   from module import                   imports foo and Bar   Documentation tools  Documentation and code autocompletion tools may  in fact  should  also inspect the   all   to determine what names to show as available from a module     init   py makes a directory a Python package  From the docs      The   init   py files are required to make Python treat the directories as containing packages  this is done to prevent directories with a common name  such as string  from unintentionally hiding valid modules that occur later on the module search path       In the simplest case    init   py can just be an empty file  but it can also execute initialization code for the package or set the   all   variable    So the   init   py can declare the   all   for a package   Managing an API   A package is typically made up of modules that may import one another  but that are necessarily tied together with an   init   py file  That file is what makes the directory an actual Python package  For example  say you have the following files in a package   package  --   init   py  -- module 1 py  -- module 2 py   Let s create these files with Python so you can follow along - you could paste the following into a Python 3 shell   from pathlib import Path  package   Path  package   package mkdir     package      init   py   write text     from  module 1 import   from  module 2 import         package module 1   package    module 1 py  package module 1 write text       all       foo   imp detail1   imp detail2   imp detail3   None def foo    pass       package module 2   package    module 2 py  package module 2 write text       all       Bar   imp detail1   imp detail2   imp detail3   None class Bar  pass        And now you have presented a complete api that someone else can use when they import your package  like so    import package package foo   package Bar     And the package won t have all the other implementation details you used when creating your modules cluttering up the package namespace     all   in   init   py  After more work  maybe you ve decided that the modules are too big  like many thousands of lines   and need to be split up  So you do the following   package  --   init   py  -- module 1         -- foo implementation py         --   init   py  -- module 2      -- Bar implementation py      --   init   py   First make the subpackage directories with the same names as the modules   subpackage 1   package    module 1  subpackage 1 mkdir   subpackage 2   package    module 2  subpackage 2 mkdir     Move the implementations   package module 1 rename subpackage 1    foo implementation py   package module 2 rename subpackage 2    Bar implementation py     create   init   pys for the subpackages that declare the   all   for each    subpackage 1      init   py   write text     from  foo implementation import     all       foo         subpackage 2      init   py   write text     from  Bar implementation import     all       Bar          And now you still have the api provisioned at the package level    gt  gt  gt  import package  gt  gt  gt  package foo    gt  gt  gt  package Bar    lt package module 2 Bar implementation Bar object at 0x7f0c2349d210 gt    And you can easily add things to your API that you can manage at the subpackage level instead of the subpackage s module level  If you want to add a new name to the API  you simply update the   init   py  e g  in module 2   from  Bar implementation import   from  Baz implementation import     all       Bar    Baz     And if you re not ready to publish Baz in the top level API  in your top level   init   py you could have   from  module 1 import           also constrained by   all   s from  module 2 import           in the   init   py s   all       foo    Bar         further constraining the names advertised   and if your users are aware of the availability of Baz  they can use it   import package package Baz     but if they don t know about it  other tools  like pydoc  won t inform them   You can later change that when Baz is ready for prime time   from  module 1 import   from  module 2 import     all       foo    Bar    Baz     Prefixing   versus   all     By default  Python will export all names that do not start with an    You certainly could rely on this mechanism  Some packages in the Python standard library  in fact  do rely on this  but to do so  they alias their imports  for example  in ctypes   init   py   import os as  os  sys as  sys   Using the   convention can be more elegant because it removes the redundancy of naming the names again  But it adds the redundancy for imports  if you have a lot of them  and it is easy to forget to do this consistently - and the last thing you want is to have to indefinitely support something you intended to only be an implementation detail  just because you forgot to prefix an   when naming a function   I personally write an   all   early in my development lifecycle for modules so that others who might use my code know what they should use and not use   Most packages in the standard library also use   all     When avoiding   all   makes sense  It makes sense to stick to the   prefix convention in lieu of   all   when    You re still in early development mode and have no users  and are constantly tweaking your API  Maybe you do have users  but you have unittests that cover the API  and you re still actively adding to the API and tweaking in development    An export decorator  The downside of using   all   is that you have to write the names of functions and classes being exported twice - and the information is kept separate from the definitions  We could use a decorator to solve this problem   I got the idea for such an export decorator from David Beazley s talk on packaging  This implementation seems to work well in CPython s traditional importer  If you have a special import hook or system  I do not guarantee it  but if you adopt it  it is fairly trivial to back out - you ll just need to manually add the names back into the   all    So in  for example  a utility library  you would define the decorator   import sys  def export fn       mod   sys modules fn   module        if hasattr mod     all              mod   all   append fn   name        else          mod   all      fn   name        return fn   and then  where you would define an   all    you do this     cat  gt  main py from lib import export   all          optional - we create a list if   all   is not there    export def foo    pass   export def bar         bar   def main        print  main    if   name         main         main     And this works fine whether run as main or imported by another function     cat  gt  run py import main main main      python run py main   And API provisioning with import   will work too     cat  gt  run py from main import   foo   bar   main     expected to error here  not exported    python run py Traceback  most recent call last     File  run py   line 4  in  lt module gt      main     expected to error here  not exported NameError  name  main  is not defined

User · Answer

It also changes what pydoc will show   module1 py  a    A  b    B  c    C    module2 py    all       a    b    a    A  b    B  c    C      pydoc module1   Help on module module1   NAME     module1  FILE     module1 py  DATA     a    A      b    B      c    C      pydoc module2   Help on module module2   NAME     module2  FILE     module2 py  DATA       all       a    b       a    A      b    B    I declare   all   in all my modules  as well as underscore internal details  these really help when using things you ve never used before in live interpreter sessions

User · Answer

I m just adding this to be precise   All other answers refer to modules  The original question explicitely mentioned   all   in   init   py files  so this is about python packages   Generally    all   only comes into play when the from xxx import   variant of the import statement is used  This applies to packages as well as to modules   The behaviour for modules is explained in the other answers  The exact behaviour for packages is described here in detail   In short    all   on package level does approximately the same thing as for modules  except it deals with modules within the package   in contrast to specifying names within the module   So   all   specifies all modules that shall be loaded and imported into the current namespace when us use from package import     The big difference is  that when you omit the declaration of   all   in a package s   init   py  the statement from package import   will not import anything at all  with exceptions explained in the documentation  see link above     On the other hand  if you omit   all   in a module  the  starred import  will import all names  not starting with an underscore  defined in the module

User · Answer

all   is used to document the public API of a Python module  Although it is optional    all   should be used  Here is the relevant excerpt from the Python language reference   The public names defined by a module are determined by checking the module   s namespace for a variable named   all    if defined  it must be a sequence of strings which are names defined or imported by that module  The names given in   all   are all considered public and are required to exist  If   all   is not defined  the set of public names includes all names found in the module   s namespace which do not begin with an underscore character          all   should contain the entire public API  It is intended to avoid accidentally exporting items that are not part of the API  such as library modules which were imported and used within the module    PEP 8 uses similar wording  although it also makes it clear that imported names are not part of the public API when   all   is absent   To better support introspection  modules should explicitly declare the names in their public API using the   all   attribute  Setting   all   to an empty list indicates that the module has no public API        Imported names should always be considered an implementation detail  Other modules must not rely on indirect access to such imported names unless they are an explicitly documented part of the containing module s API  such as os path or a package s   init   module that exposes functionality from submodules   Furthermore  as pointed out in other answers    all   is used to enable wildcard importing for packages   The import statement uses the following convention  if a package   s   init   py code defines a list named   all    it is taken to be the list of module names that should be imported when from package import   is encountered

User · Answer

Linked to  but not explicitly mentioned here  is exactly when   all   is used  It is a list of strings defining what symbols in a module will be exported when from  lt module gt  import   is used on the module   For example  the following code in a foo py explicitly exports the symbols bar and baz     all       bar    baz    waz   5 bar   10 def baz    return  baz    These symbols can then be imported like so   from foo import    print bar  print baz     The following will trigger an exception  as  waz  is not exported by the module print waz    If the   all   above is commented out  this code will then execute to completion  as the default behaviour of import   is to import all symbols that do not begin with an underscore  from the given namespace   Reference  https   docs python org tutorial modules html importing-from-a-package  NOTE    all   affects the from  lt module gt  import   behavior only  Members that are not mentioned in   all   are still accessible from outside the module and can be imported with from  lt module gt  import  lt member gt

User · Answer

all   customizes   in from  lt module gt  import     all   customizes   in from  lt package gt  import    A module is a  py file meant to be imported  A package is a directory with a   init   py file  A package usually contains modules   MODULES  quot  quot  quot  cheese py - an example module  quot  quot  quot     all       swiss    cheddar    swiss   4 99 cheddar   3 99 gouda   10 99    all   lets humans know the  quot public quot  features of a module   AaronHall   Also  pydoc recognizes them   Longpoke  from module import   See how swiss and cheddar are brought into the local namespace  but not gouda   gt  gt  gt  from cheese import    gt  gt  gt  swiss  cheddar  4 99  3 99   gt  gt  gt  gouda Traceback  most recent call last     File  quot  lt stdin gt  quot   line 1  in  lt module gt  NameError  name  gouda  is not defined  Without   all    any symbol  that doesn t start with an underscore  would have been available   Imports without   are not affected by   all    import module  gt  gt  gt  import cheese  gt  gt  gt  cheese swiss  cheese cheddar  cheese gouda  4 99  3 99  10 99   from module import names  gt  gt  gt  from cheese import swiss  cheddar  gouda  gt  gt  gt  swiss  cheddar  gouda  4 99  3 99  10 99   import module as localname  gt  gt  gt  import cheese as ch  gt  gt  gt  ch swiss  ch cheddar  ch gouda  4 99  3 99  10 99    PACKAGES In the   init   py file of a package   all   is a list of strings with the names of public modules or other objects  Those features are available to wildcard imports  As with modules    all   customizes the   when wildcard-importing from the package   MartinStettner  Here s an excerpt from the Python MySQL Connector   init   py    all            MySQLConnection    Connect    custom error exception          Some useful constants      FieldType    FieldFlag    ClientFlag    CharacterSet    RefreshOption        HAVE CEXT          Error handling      Error    Warning           etc            The default case  asterisk with no   all   for a package  is complicated  because the obvious behavior would be expensive  to use the file system to search for all modules in the package  Instead  in my reading of the docs  only the objects defined in   init   py are imported   If   all   is not defined  the statement from sound effects import   does not import all submodules from the package sound effects into the current namespace  it only ensures that the package sound effects has been imported  possibly running any initialization code in   init   py  and then imports whatever names are defined in the package  This includes any names defined  and submodules explicitly loaded  by   init   py  It also includes any submodules of the package that were explicitly loaded by previous import statements    And lastly  a venerated tradition for stack overflow answers  professors  and mansplainers everywhere  is the bon mot of reproach for asking a question in the first place   Wildcard imports     should be avoided  as they  confuse  readers and many automated tools    PEP 8   ToolmakerSteve

User · Answer

From  An Unofficial  Python Reference Wiki      The public names defined by a module are determined by checking the module s namespace for a variable named   all    if defined  it must be a sequence of strings which are names defined or imported by that module  The names given in   all   are all considered public and are required to exist  If   all   is not defined  the set of public names includes all names found in the module s namespace which do not begin with an underscore character          all   should contain the entire public API  It is intended to avoid accidentally exporting items that are not part of the API  such as library modules which were imported and used within the module

User · Answer

It s a list of public objects of that module  as interpreted by import    It overrides the default of hiding everything that begins with an underscore

[python] Can someone explain all in Python?

Examples related to python

Examples related to syntax

Examples related to namespaces

[python] Can someone explain __all__ in Python?

Examples related to python

Examples related to syntax

Examples related to namespaces

[python] Can someone explain all in Python?