[python] Python circular importing?

So I'm getting this error

Traceback (most recent call last):
  File "/Users/alex/dev/runswift/utils/sim2014/simulator.py", line 3, in <module>
    from world import World
  File "/Users/alex/dev/runswift/utils/sim2014/world.py", line 2, in <module>
    from entities.field import Field
  File "/Users/alex/dev/runswift/utils/sim2014/entities/field.py", line 2, in <module>
    from entities.goal import Goal
  File "/Users/alex/dev/runswift/utils/sim2014/entities/goal.py", line 2, in <module>
    from entities.post import Post
  File "/Users/alex/dev/runswift/utils/sim2014/entities/post.py", line 4, in <module>
    from physics import PostBody
  File "/Users/alex/dev/runswift/utils/sim2014/physics.py", line 21, in <module>
    from entities.post import Post
ImportError: cannot import name Post

and you can see that I use the same import statement further up and it works? Is there some unwritten rule about circular importing? How do I use the same class further down the call stack?

I was using the following:

from module import Foo

foo_instance = Foo()

but to get rid of circular reference I did the following and it worked:

import module.foo

foo_instance = foo.Foo()

For those of you who, like me, come to this issue from Django, you should know that the docs provide a solution: https://docs.djangoproject.com/en/1.10/ref/models/fields/#foreignkey

"...To refer to models defined in another application, you can explicitly specify a model with the full application label. For example, if the Manufacturer model above is defined in another application called production, you’d need to use:

class Car(models.Model):
    manufacturer = models.ForeignKey(
        'production.Manufacturer',
        on_delete=models.CASCADE,
)

This sort of reference can be useful when resolving circular import dependencies between two applications...."

If you run into this issue in a fairly complex app it can be cumbersome to refactor all your imports. PyCharm offers a quickfix for this that will automatically change all usage of the imported symbols as well.

To understand circular dependencies, you need to remember that Python is essentially a scripting language. Execution of statements outside methods occurs at compile time. Import statements are executed just like method calls, and to understand them you should think about them like method calls.

When you do an import, what happens depends on whether the file you are importing already exists in the module table. If it does, Python uses whatever is currently in the symbol table. If not, Python begins reading the module file, compiling/executing/importing whatever it finds there. Symbols referenced at compile time are found or not, depending on whether they have been seen, or are yet to be seen by the compiler.

Imagine you have two source files:

File X.py

def X1:
    return "x1"

from Y import Y2

def X2:
    return "x2"

File Y.py

def Y1:
    return "y1"

from X import X1

def Y2:
    return "y2"

Now suppose you compile file X.py. The compiler begins by defining the method X1, and then hits the import statement in X.py. This causes the compiler to pause compilation of X.py and begin compiling Y.py. Shortly thereafter the compiler hits the import statement in Y.py. Since X.py is already in the module table, Python uses the existing incomplete X.py symbol table to satisfy any references requested. Any symbols appearing before the import statement in X.py are now in the symbol table, but any symbols after are not. Since X1 now appears before the import statement, it is successfully imported. Python then resumes compiling Y.py. In doing so it defines Y2 and finishes compiling Y.py. It then resumes compilation of X.py, and finds Y2 in the Y.py symbol table. Compilation eventually completes w/o error.

Something very different happens if you attempt to compile Y.py from the command line. While compiling Y.py, the compiler hits the import statement before it defines Y2. Then it starts compiling X.py. Soon it hits the import statement in X.py that requires Y2. But Y2 is undefined, so the compile fails.

Please note that if you modify X.py to import Y1, the compile will always succeed, no matter which file you compile. However if you modify file Y.py to import symbol X2, neither file will compile.

Any time when module X, or any module imported by X might import the current module, do NOT use:

from X import Y

Any time you think there may be a circular import you should also avoid compile time references to variables in other modules. Consider the innocent looking code:

import X
z = X.Y

Suppose module X imports this module before this module imports X. Further suppose Y is defined in X after the import statement. Then Y will not be defined when this module is imported, and you will get a compile error. If this module imports Y first, you can get away with it. But when one of your co-workers innocently changes the order of definitions in a third module, the code will break.

In some cases you can resolve circular dependencies by moving an import statement down below symbol definitions needed by other modules. In the examples above, definitions before the import statement never fail. Definitions after the import statement sometimes fail, depending on the order of compilation. You can even put import statements at the end of a file, so long as none of the imported symbols are needed at compile time.

Note that moving import statements down in a module obscures what you are doing. Compensate for this with a comment at the top of your module something like the following:

#import X   (actual import moved down to avoid circular dependency)

In general this is a bad practice, but sometimes it is difficult to avoid.

I was able to import the module within the function (only) that would require the objects from this module:

def my_func():
    import Foo
    foo_instance = Foo()

When you import a module (or a member of it) for the first time, the code inside the module is executed sequentially like any other code; e.g., it is not treated any differently that the body of a function. An import is just a command like any other (assignment, a function call, def, class). Assuming your imports occur at the top of the script, then here's what's happening:

• When you try to import World from world, the world script gets executed.
• The world script imports Field, which causes the entities.field script to get executed.
• This process continues until you reach the entities.post script because you tried to import Post
• The entities.post script causes physics module to be executed because it tries to import PostBody
• Finally, physics tries to import Post from entities.post
• I'm not sure whether the entities.post module exists in memory yet, but it really doesn't matter. Either the module is not in memory, or the module doesn't yet have a Post member because it hasn't finished executing to define Post
• Either way, an error occurs because Post is not there to be imported

So no, it's not "working further up in the call stack". This is a stack trace of where the error occurred, which means it errored out trying to import Post in that class. You shouldn't use circular imports. At best, it has negligible benefit (typically, no benefit), and it causes problems like this. It burdens any developer maintaining it, forcing them to walk on egg shells to avoid breaking it. Refactor your module organization.