[python] How to make a cross-module variable?

The __debug__ variable is handy in part because it affects every module. If I want to create another variable that works the same way, how would I do it?

The variable (let's be original and call it 'foo') doesn't have to be truly global, in the sense that if I change foo in one module, it is updated in others. I'd be fine if I could set foo before importing other modules and then they would see the same value for it.

I could achieve cross-module modifiable (or mutable) variables by using a dictionary:

# in myapp.__init__
Timeouts = {} # cross-modules global mutable variables for testing purpose
Timeouts['WAIT_APP_UP_IN_SECONDS'] = 60

# in myapp.mod1
from myapp import Timeouts

def wait_app_up(project_name, port):
    # wait for app until Timeouts['WAIT_APP_UP_IN_SECONDS']
    # ...

# in myapp.test.test_mod1
from myapp import Timeouts

def test_wait_app_up_fail(self):
    timeout_bak = Timeouts['WAIT_APP_UP_IN_SECONDS']
    Timeouts['WAIT_APP_UP_IN_SECONDS'] = 3
    with self.assertRaises(hlp.TimeoutException) as cm:
        wait_app_up(PROJECT_NAME, PROJECT_PORT)
    self.assertEqual("Timeout while waiting for App to start", str(cm.exception))
    Timeouts['WAIT_JENKINS_UP_TIMEOUT_IN_SECONDS'] = timeout_bak

When launching test_wait_app_up_fail, the actual timeout duration is 3 seconds.

I believe that there are plenty of circumstances in which it does make sense and it simplifies programming to have some globals that are known across several (tightly coupled) modules. In this spirit, I would like to elaborate a bit on the idea of having a module of globals which is imported by those modules which need to reference them.

When there is only one such module, I name it "g". In it, I assign default values for every variable I intend to treat as global. In each module that uses any of them, I do not use "from g import var", as this only results in a local variable which is initialized from g only at the time of the import. I make most references in the form g.var, and the "g." serves as a constant reminder that I am dealing with a variable that is potentially accessible to other modules.

If the value of such a global variable is to be used frequently in some function in a module, then that function can make a local copy: var = g.var. However, it is important to realize that assignments to var are local, and global g.var cannot be updated without referencing g.var explicitly in an assignment.

Note that you can also have multiple such globals modules shared by different subsets of your modules to keep things a little more tightly controlled. The reason I use short names for my globals modules is to avoid cluttering up the code too much with occurrences of them. With only a little experience, they become mnemonic enough with only 1 or 2 characters.

It is still possible to make an assignment to, say, g.x when x was not already defined in g, and a different module can then access g.x. However, even though the interpreter permits it, this approach is not so transparent, and I do avoid it. There is still the possibility of accidentally creating a new variable in g as a result of a typo in the variable name for an assignment. Sometimes an examination of dir(g) is useful to discover any surprise names that may have arisen by such accident.

This sounds like modifying the __builtin__ name space. To do it:

import __builtin__
__builtin__.foo = 'some-value'

Do not use the __builtins__ directly (notice the extra "s") - apparently this can be a dictionary or a module. Thanks to ??O????? for pointing this out, more can be found here.

Now foo is available for use everywhere.

I don't recommend doing this generally, but the use of this is up to the programmer.

Assigning to it must be done as above, just setting foo = 'some-other-value' will only set it in the current namespace.

I use this for a couple built-in primitive functions that I felt were really missing. One example is a find function that has the same usage semantics as filter, map, reduce.

def builtin_find(f, x, d=None):
    for i in x:
        if f(i):
            return i
    return d

import __builtin__
__builtin__.find = builtin_find

Once this is run (for instance, by importing near your entry point) all your modules can use find() as though, obviously, it was built in.

find(lambda i: i < 0, [1, 3, 0, -5, -10])  # Yields -5, the first negative.

Note: You can do this, of course, with filter and another line to test for zero length, or with reduce in one sort of weird line, but I always felt it was weird.

If you need a global cross-module variable maybe just simple global module-level variable will suffice.

a.py:

var = 1

b.py:

import a
print a.var
import c
print a.var

c.py:

import a
a.var = 2

Test:

$ python b.py
# -> 1 2

Real-world example: Django's global_settings.py (though in Django apps settings are used by importing the object django.conf.settings).

Define a module ( call it "globalbaz" ) and have the variables defined inside it. All the modules using this "pseudoglobal" should import the "globalbaz" module, and refer to it using "globalbaz.var_name"

This works regardless of the place of the change, you can change the variable before or after the import. The imported module will use the latest value. (I tested this in a toy example)

For clarification, globalbaz.py looks just like this:

var_name = "my_useful_string"

I wanted to post an answer that there is a case where the variable won't be found.

Cyclical imports may break the module behavior.

For example:

first.py

import second
var = 1

second.py

import first
print(first.var)  # will throw an error because the order of execution happens before var gets declared.

main.py

import first

On this is example it should be obvious, but in a large code-base, this can be really confusing.

You can pass the globals of one module to onother:

In Module A:

import module_b
my_var=2
module_b.do_something_with_my_globals(globals())
print my_var

In Module B:

def do_something_with_my_globals(glob): # glob is simply a dict.
    glob["my_var"]=3

Global variables are usually a bad idea, but you can do this by assigning to __builtins__:

__builtins__.foo = 'something'
print foo

Also, modules themselves are variables that you can access from any module. So if you define a module called my_globals.py:

# my_globals.py
foo = 'something'

Then you can use that from anywhere as well:

import my_globals
print my_globals.foo

Using modules rather than modifying __builtins__ is generally a cleaner way to do globals of this sort.

You can already do this with module-level variables. Modules are the same no matter what module they're being imported from. So you can make the variable a module-level variable in whatever module it makes sense to put it in, and access it or assign to it from other modules. It would be better to call a function to set the variable's value, or to make it a property of some singleton object. That way if you end up needing to run some code when the variable's changed, you can do so without breaking your module's external interface.

It's not usually a great way to do things — using globals seldom is — but I think this is the cleanest way to do it.

I wondered if it would be possible to avoid some of the disadvantages of using global variables (see e.g. http://wiki.c2.com/?GlobalVariablesAreBad) by using a class namespace rather than a global/module namespace to pass values of variables. The following code indicates that the two methods are essentially identical. There is a slight advantage in using class namespaces as explained below.

The following code fragments also show that attributes or variables may be dynamically created and deleted in both global/module namespaces and class namespaces.

wall.py

# Note no definition of global variables

class router:
    """ Empty class """

I call this module 'wall' since it is used to bounce variables off of. It will act as a space to temporarily define global variables and class-wide attributes of the empty class 'router'.

source.py

import wall
def sourcefn():
    msg = 'Hello world!'
    wall.msg = msg
    wall.router.msg = msg

This module imports wall and defines a single function sourcefn which defines a message and emits it by two different mechanisms, one via globals and one via the router function. Note that the variables wall.msg and wall.router.message are defined here for the first time in their respective namespaces.

dest.py

import wall
def destfn():

    if hasattr(wall, 'msg'):
        print 'global: ' + wall.msg
        del wall.msg
    else:
        print 'global: ' + 'no message'

    if hasattr(wall.router, 'msg'):
        print 'router: ' + wall.router.msg
        del wall.router.msg
    else:
        print 'router: ' + 'no message'

This module defines a function destfn which uses the two different mechanisms to receive the messages emitted by source. It allows for the possibility that the variable 'msg' may not exist. destfn also deletes the variables once they have been displayed.

main.py

import source, dest

source.sourcefn()

dest.destfn() # variables deleted after this call
dest.destfn()

This module calls the previously defined functions in sequence. After the first call to dest.destfn the variables wall.msg and wall.router.msg no longer exist.

The output from the program is:

global: Hello world!
router: Hello world!
global: no message
router: no message

The above code fragments show that the module/global and the class/class variable mechanisms are essentially identical.

If a lot of variables are to be shared, namespace pollution can be managed either by using several wall-type modules, e.g. wall1, wall2 etc. or by defining several router-type classes in a single file. The latter is slightly tidier, so perhaps represents a marginal advantage for use of the class-variable mechanism.