Home › Session 09
Program structure · Session 09

Modules & Imports

Nearly every Python file you'll ever write starts with import, and most people never see what that line does. It finds a file, runs it once, keeps the result in a cache and binds a name to it. Once you can see those four moves, the odd cases (a setting that won't update, a circular import, a file called random.py that breaks everything) stop being odd.

~40 minutes
5 interactive demos
Builds on Sessions 01, 02 and 08
Learning goal

A module is an object, built once and kept in sys.modules.

import checks a cache, runs the file into a new module object if it has to, and binds a name in your file. Everything else in this session (from-imports, the main guard, circular imports, a file that hides a library) is one of those moves happening at a moment you didn't expect.

module sys.modules __name__ sys.path __globals__
Recommended path
1 Watch one import. Demo 1 runs a file once and then finds it in the cache. Keep an eye on sys.modules.
2 Then the two surprises. Demo 2 is a from-import that goes stale, and demo 3 is a file that behaves differently depending on who loads it.
3 Then the two classic bugs. Demos 4 and 5 are a circular import and a file that hides a standard module. In both, the cache is only doing its job.
Interactive lab

Each frame is a file running its top-level code

When a module is imported, its code runs in a frame of its own, named here after the file. A module object appears on the heap with a __class__ link to module and its __dict__ underneath. While the file runs, that dict is the frame's global namespace, so a new name shows up in both places at once. The dict labelled sys.modules is the cache every import checks first.

Tip: at every import line, ask one question before anything else. Is this name already in sys.modules?
1 Choose a demo
›
2 Step with → or Play
›
3 Watch sys.modules
›
4 Read the model below
Call stack: one frame per file whose top-level code is running
Module objects, sys.modules and what they hold
→ next ← back Space play R reset
python
1 / 1
Memory
stack heap
1

Nothing has run yet

Press Play or use Next to watch a module being imported.

What import actually does

In plain words Think of a workshop with a tool wall. The first time anyone asks for a drill, somebody fetches it from the store room and hangs it on the wall. After that, everyone who asks for a drill gets the one on the wall. Nobody fetches a second drill. sys.modules is the wall, and import is the asking.

import config does four things. It looks up "config" in sys.modules, and if it's there, it skips straight to the last step. Otherwise it searches sys.path for config.py (or a config/ package), creates an empty module object and stores it in sys.modules, then runs the file's code with the module's __dict__ as its global namespace. Last, it binds the name config in the namespace that ran the import. The module is cached before its body runs, not after, and that's what makes circular imports possible at all.

Already in the cache?

sys.modules is a plain dict from names to module objects. A hit costs one lookup, so the second import of a module, anywhere in the program, is nearly free.

Run the file once

A new module object is registered, then the file runs top to bottom in a frame whose globals are the module's __dict__. Every top-level def, class and assignment becomes an attribute.

A name in your file

import config binds config. import config as cfg binds cfg. from config import retries binds retries, and doesn't bind config at all.

Read
A module's attributes are its globals
config.TIMEOUT from outside and the name TIMEOUT inside config.py are the same dict entry, seen from two sides. That's also why global in Session 02 means module level, not program-wide. A function looks up its globals in the module where it was defined (its __globals__), whichever file happens to call it.
Runs once

Top-level code, on the first import

Prints, opened connections, files read, settings parsed. All of it runs the first time the module is imported and never again in this process, however many files import it afterwards.

Doesn't rerun

Edits to the file while it's running

Changing config.py under a running program changes nothing. importlib.reload(config) runs the body again into the same module object, but names that other files copied with from config import ... still point at the old objects.

Care
Keep import-time work small
A module's body runs the moment someone imports it, so slow or risky work at the top level (a network call, loading a large file, connecting to a database) happens before the importer's next line runs. That includes a test that only wanted one small function. Put that work in a function and call it when you actually need it.
Try this in a REPL

import sys, json, then sys.modules["json"] is json gives True. len(sys.modules) shows how many modules were loaded before you typed anything, and json.__file__ shows which file this one came from. Finally type(json): <class 'module'>, an ordinary class with ordinary instances.

Import forms

from x import y copies a name

In plain words import config gives you the office's phone number. from config import retries is writing down what the office told you this morning. If the office changes its answer at lunchtime, the person with the phone number hears the new one. The person with the note doesn't.

from config import retries imports config as usual, then does the equivalent of retries = config.retries in your namespace and stops. It doesn't create an alias or a link. If code later rebinds config.retries to a new object, your retries still points at the old one. If code mutates the object instead, say by appending to a list both names share, both names see it. That's Session 01's rule again, with the two names living in different files.

You write Names bound in your file Sees the module rebind it later?
import config config Yes, config.retries is looked up every time
import config as cfg cfg only Yes, the same module under another name
from config import retries retries only, not config No, it's a copy of one binding
from config import * every public name, or just the ones listed in __all__ No, and a reader can't tell where each name came from
import os.path os Yes. The package is bound, and os.path is reached through it
Care
mock.patch aims at a name, not at a module
This is why tests sometimes patch the wrong thing. If user.py does from config import retries, then patch("config.retries", 99) changes the module's slot, and user.py never looks there again. Patch "user.retries", the name your code actually reads. The unittest.mock docs put it as: patch where an object is looked up, not where it's defined.
Try this in a REPL

Save retries = 3 as config.py. Then from config import retries, then import config, then config.retries = 5. retries is still 3, and retries is config.retries is False. Before the import config line, typing config would have given NameError, even though "config" was already in sys.modules.

The main guard

__name__ and if __name__ == "__main__":

In plain words A file can't tell from its own name whether it's the whole program or just a part of one. Python tells it, by setting __name__ before the first line runs: "__main__" if you started it, and its module name if something imported it.

The script you run is executed as a module registered in sys.modules under the key "__main__", with __name__ set to "__main__". The same goes for python -m pkg.tool. When it's imported instead, the same file gets __name__ == "pkg.tool". The guard is an ordinary if comparing two strings, and it separates what a file defines from what it does when you start it.

python
import sys def main(argv): print("hello", *argv) if __name__ == "__main__": main(sys.argv[1:]) # python greet.py Sam prints: hello Sam # import greet defines main and runs nothing
Key
One file can turn into two modules
If app.py is the script you run and some other module does import app, the file runs twice: once as __main__ and once as app. Those are two separate module objects with two separate sets of globals, so a counter or a cache at the top of app.py now exists twice, and changes to one don't show in the other. Keep the file you run thin, and import the real code from somewhere else.
Try this in a REPL

Type __name__ in a fresh REPL: '__main__'. The REPL is a module too. Then import json; json.__name__ gives 'json', and import sys; sys.modules["__main__"] shows the module you're typing into.

Circular imports

Two modules that need each other

In plain words Two people each wait for the other to finish a sentence before they finish their own. Python breaks the deadlock by letting the second person carry on with whatever the first had said so far. That works fine, as long as the second person doesn't need the part that hasn't been said yet.

Because a module goes into sys.modules before its body runs, an import that comes back round to it finds the partly run module and returns it without running anything. Names defined above the import line are there. Names below it don't exist yet, and reading one of them during the import raises AttributeError (for a.NAME) or ImportError: cannot import name (for from a import NAME). When an import fails, every module whose body didn't finish is removed from sys.modules again, so nothing is left half built in the cache.

1

Read names later. Keep import a at the top, but use a.VALUE inside functions. By the time a function is called, both modules have finished loading.

2

Move the shared part out. If both modules need VALUE, put it in a third module that imports neither of them. This is usually the real fix, because the cycle was a sign the code wanted splitting.

3

Import inside the function. def f(): from a import VALUE waits until the call. It's a fair last resort, and after the first call it costs one dict lookup.

Read
Reading the error on Python 3.13
Python 3.13 ends the message with a hint in brackets. For modules inside a package it usually says partially initialized module ... (most likely due to a circular import), which is right. For files sitting next to the script you ran, as in demo 4, it says consider renaming ... if it has the same name as a library you intended to import, which sends you the wrong way. If the traceback shows two of your own modules importing each other, it's circular, whatever the hint says.
The search

Where Python looks: sys.path

In plain words Finding random.py is like calling a name down a corridor of rooms, nearest room first. The first room that answers wins, even if the person you wanted was further down.

On a cache miss, Python asks the modules built into the interpreter first (sys, for one), then walks sys.path, a plain list of folders, in order, and takes the first match: a random.py file, a random/ package folder, or a compiled extension. Entry 0 is the folder of the script you ran (the current folder for python -m, and an empty string meaning the current folder in the REPL). After it come the standard library and then site-packages, where pip installs things.

sys.modules Already loaded? Then you get that object, and the search never starts. cache
built in Modules compiled into the interpreter, such as sys, and a few frozen copies of standard modules it needs at startup. interpreter
sys.path[0] The script's own folder. Your files live here, which is why one of them can hide anything further down. your code
stdlib The standard library folders, such as lib/python3.13 and lib-dynload. batteries
site-packages Everything pip installed, inside your virtual environment if one is active. third party
Care
Don't name your files after modules you use
random.py, json.py, email.py, test.py, requests.py: each one hides the real module from every import in the program, including imports made inside the standard library itself. Python 3.13 notices the standard-library case and says so in the error. For a pip package like requests you only get the plain AttributeError.

Packages are modules with a __path__

A folder with an __init__.py is a package. import shop.orders runs shop/__init__.py first, then shop/orders.py, registers both "shop" and "shop.orders" in sys.modules, and binds only shop in your file. The package's __path__ is the list of folders its submodules are searched in, a small sys.path of its own.

What __pycache__ is

The first import of config.py compiles it to bytecode and saves that as __pycache__/config.cpython-313.pyc. Next time, if the source hasn't changed, Python loads the saved bytecode and skips compiling. It speeds up start-up, not running time, and it's safe to delete. The script you run directly isn't saved this way.

Try this in a REPL

import sys; print(*sys.path, sep="\n") prints the search order on your own machine. Then import random; random.__file__ shows where the real random lives. If that path is inside your project, you've found a shadow.

What you should be able to draw

A sys.modules dict with one entry per loaded module, each pointing at a module object whose __dict__ holds that file's top-level names. A frame that exists only while a file's body runs. Two names in two files pointing at one object, which is all a from-import leaves behind. And a module that sits in the cache half built while its own imports are still running. Draw those four and you can predict every case in this session.

Common mistakes

Six ways this goes wrong

Naming a file after a module you import

A random.py or requests.py in your project folder wins the search, and the real module never loads. The error shows up somewhere else entirely, as a missing attribute. print(module.__file__) settles it in one line.

Expecting a from-imported name to update

from config import retries takes a copy of one binding at import time. If the module rebinds retries later, your copy doesn't move. Use config.retries, or a function that returns the current value.

Doing real work at import time

Top-level code runs for everyone who imports the module, including tests and tools that only wanted one function. Connections, big file loads and prints belong in a function, called from the main guard or by whoever needs them.

Running a file inside a package directly

python shop/orders.py with a relative import such as from . import users fails with ImportError: attempted relative import with no known parent package, because a file run directly is __main__, not part of shop. Run it from the project folder as python -m shop.orders.

Re-importing to pick up an edit

In a REPL or a notebook, a second import config after you edit the file does nothing, because the old module is still in the cache. importlib.reload(config) runs it again, but from-imported names and instances of old classes keep pointing at the old objects. Restarting is the honest fix.

Two modules that need each other at import time

A circular import is fine until one side reads the other's names during loading. Read them inside functions instead, or move the shared names into a third module. The traceback's tell is two of your modules importing each other.

Check
Two questions catch nearly all of it
Is it already in sys.modules? If it is, nothing runs and you get that object. Which namespace does this name live in? A from-import put a copy in yours, and module.name reads theirs.
Check your understanding

Can you predict the memory?

Every question comes back to the cache and to which namespace a name lives in.

1. config.py starts with print("loading"). Three different files in one program each run import config. How many times is loading printed?
The first import runs the whole file into a new module object and caches it. Every import after that, in any file, is a lookup in sys.modules plus a name binding. Nothing runs again unless you call importlib.reload.
2. When is a new module added to sys.modules?
Registering first is what lets a circular import find the module instead of loading it forever. It's also why the other module can see it half built. If the body then fails, the entry is removed again.
3. main.py runs from config import retries, and later config.set_retries(5) rebinds the module's retries from 3 to 5. What does retries in main.py hold now?
from config import retries works like retries = config.retries. Rebinding the module's name moves that one name only. If the object had been mutated instead (a list appended to, say), both names would see the change, because they share it.
4. After from config import retries and nothing else, you type config. What happens?
The whole module was loaded and cached, but a from-import binds only the names you list. import config is what binds config. Being in the cache and being a name in your namespace are two different things.
5. You start a program with python tools.py. Inside tools.py, what is __name__?
Python sets __name__ from how the file was loaded. The script you start, or the one you run with python -m, is "__main__". The same file imported by another module gets "tools", and then its guarded block doesn't run.
6. a.py is import b then VALUE = 1. b.py is import a then print(a.VALUE). You run import a. What happens?
a is registered, starts running, and pauses on import b. b's import a finds a in sys.modules and gets it as it is, before line 2 of a.py has run. The cache stops the loop, and it's also what hands over an unfinished module.
7. Your project folder has game.py and a helper you saved as random.py. game.py runs import random. Which file is loaded?
The search walks sys.path in order and stops at the first match, and your script's folder comes before the standard library. From then on sys.modules["random"] is your file, for every import in the process. Rename the helper.