Python3 Modules

In the previous chapters, we basically programmed using the Python interpreter. If you exit and re-enter the Python interpreter, all the methods and variables you defined will be lost.

To solve this, Python provides a way to store these definitions in a file for use by scripts or interactive interpreter instances. This file is called a module.

A module in Python is a file containing Python definitions and statements. The filename is the module name plus the .py suffix.

Modules can contain functions, classes, variables, and executable code. Through modules, we can organize code into reusable units, making it easier to manage and maintain.

The Role of Modules

• Code Reuse: Encapsulate common functionality into modules that can be reused across multiple programs.

• Namespace Management: Modules avoid naming conflicts; functions or variables with the same name in different modules do not interfere with each other.

• Code Organization: Divide code by functionality into different modules, making the program structure clearer.

Below is an example of using modules from the Python standard library.

Example (Python 3.0+)

#!/usr/bin/python3
# Filename: using_sys.py
 
import sys
 
print('Command line arguments are as follows:')
for i in sys.argv:
   print(i)
 
print('\n\nPython path is: ', sys.path, '\n')

The execution result is as follows:

$ python using_sys.py arg1 arg2
Command line arguments are as follows:
using_sys.py
arg1
arg2

Python path is: ['/root', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages'] 

• 1. import sys introduces the sys.py module from the Python standard library; this is the method for importing a module.
• 2. sys.argv is a list containing command-line arguments.
• 3. sys.path contains a list of paths that the Python interpreter automatically searches for the required modules.

---

The import Statement

To use a Python source file, simply execute an import statement in another source file. The syntax is as follows:

import module1[, module2[,... moduleN]

When the interpreter encounters an import statement, the module is imported if it is in the current search path.

The search path is a list of all directories that the interpreter will first search. If you want to import the support module, you need to put the command at the top of the script:

support.py File Code

#!/usr/bin/python3
# Filename: support.py
 
def print_func( par ):
    print ("Hello : ", par)
    return

test.py imports the support module:

test.py File Code

#!/usr/bin/python3
# Filename: test.py
 
# Import module
import support
 
# Now you can call the functions contained in the module
support.print_func("Runoob")

Output of the above example:

$ python3 test.py 
Hello :  Runoob

Download Code

A module is imported only once, regardless of how many times you execute import. This prevents the imported module from being executed over and over again.

When we use the import statement, how does the Python interpreter find the corresponding file?

This involves Python’s search path, which consists of a series of directory names. The Python interpreter searches these directories in order to find the imported module.

Module Search Path

When importing a module, Python searches for the module in the following order:

1. The current directory.

2. Directories specified by the PYTHONPATH environment variable.

3. The Python standard library directory.

4. Directories specified in .pth files.

The search path is determined when Python is compiled or installed, and installing new libraries should also modify it. The search path is stored in the path variable of the sys module. Let’s do a simple experiment; in the interactive interpreter, enter the following code:

>>> import sys
>>> sys.path
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>> 

The sys.path output is a list, where the first item is an empty string ‘’, representing the current directory (if printed from a script, it can show more clearly which directory it is), i.e., the directory where we executed the Python interpreter (for scripts, it is the directory where the running script is located).

Therefore, if there is a file with the same name as the module to be imported in the current directory, it will shadow the module to be imported.

Once you understand the concept of the search path, you can modify sys.path in a script to import modules that are not in the search path.

Now, create a fibo.py file in the current directory of the interpreter or in one of the directories in sys.path, with the following code:

Example

# Fibonacci sequence module
 
def fib(n):    # Define Fibonacci sequence up to n
    a, b = 0, 1
    while b < n:
        print(b, end=' ')
        a, b = b, a+b
    print()
 
def fib2(n): # Return Fibonacci sequence up to n
    result = []
    a, b = 0, 1
    while b < n:
        result.append(b)
        a, b = b, a+b
    return result

Then enter the Python interpreter and use the following command to import this module:

>>> import fibo

This does not write the function names defined directly in fibo into the current symbol table; it only writes the module name fibo there.

You can use the module name to access functions:

Example

>>>fibo.fib(1000)
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>> fibo.__name__
'fibo'

If you plan to use a function frequently, you can assign it to a local name:

>>> fib = fibo.fib
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

---

The from … import Statement

Python’s from statement lets you import a specific part from a module into the current namespace. The syntax is as follows:

from modname import name1[, name2[, ... nameN]]

For example, to import the fib function from the fibo module, use the following statement:

>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This declaration does not import the entire fibo module into the current namespace; it only introduces the fib function from fibo.

Aliasing Modules

Use the as keyword to alias modules or functions:

import numpy as np  # Set the numpy module alias to np
from math import sqrt as square_root  # Set the sqrt function alias to square_root

---

The from … import * Statement

It is also possible to import all contents of a module into the current namespace, using the following declaration:

from modname import *

This provides a simple way to import all items from a module.

Not recommended, as it can easily cause naming conflicts.

---

Module Deep Dive

In addition to method definitions, modules can also include executable code. This code is generally used to initialize the module. This code is only executed the first time it is imported.

Each module has its own independent symbol table, which is used as a global symbol table for all functions inside the module.

Thus, module authors can confidently use these global variables inside the module without worrying about mixing them up with other users’ global variables.

On the other hand, when you know exactly what you are doing, you can also access functions within a module using notation like modname.itemname.

Modules can import other modules. Use import at the top of a module (or script, or elsewhere) to import a module, though this is just a convention, not mandatory. The name of the imported module will be placed in the symbol table of the currently operating module.

There is another import method where you can use import to directly import the names of functions and variables from a module into the currently operating module. For example:

>>> from fibo import fib, fib2
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This import method does not place the name of the imported module in the current character table (so in this example, the name fibo is undefined).

There is yet another method to import all names (functions, variables) from a module into the current module’s character table at once:

>>> from fibo import *
>>> fib(500)
1 1 2 3 5 8 13 21 34 55 89 144 233 377

This will import all names, but names starting with a single underscore (_) are excluded. In most cases, Python programmers do not use this method because names introduced from other sources may overwrite existing definitions.

---

The __name__ Attribute

When a module is first imported by another program, its main program will run.

If we want a certain program block in the module not to execute when the module is imported, we can use the __name__ attribute to make that program block execute only when the module itself is run.

#!/usr/bin/python3
# Filename: using_name.py

if __name__ == '__main__':
   print('The program is running on its own')
else:
   print('I come from another module')

The output is as follows:

$ python using_name.py
The program is running on its own

$ python
>>> import using_name
I come from another module
>>>

Explanation: Each module has a __name__ attribute.

• If the module is run directly, the value of __name__ is __main__.

• If the module is imported, the value of __name__ is the module name.

Note: __name__ and __main__ have double underscores underneath.

---

The dir() Function

The built-in function dir() can find all names defined within a module. It returns them as a list of strings:

>>> import fibo, sys
>>> dir(fibo)
['__name__', 'fib', 'fib2']
>>> dir(sys)  
['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__',
 '__package__', '__stderr__', '__stdin__', '__stdout__',
 '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe',
 '_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv',
 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder',
 'call_tracing', 'callstats', 'copyright', 'displayhook',
 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix',
 'executable', 'exit', 'flags', 'float_info', 'float_repr_style',
 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags',
 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit',
 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount',
 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info',
 'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path',
 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1',
 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit',
 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout',
 'thread_info', 'version', 'version_info', 'warnoptions']

If no argument is given, the dir() function will list all names currently defined:

>>> a = [1, 2, 3, 4, 5]
>>> import fibo
>>> fib = fibo.fib
>>> dir() # Get a list of attributes defined in the current module
['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']
>>> a = 5 # Create a new variable 'a'
>>> dir()
['__builtins__', '__doc__', '__name__', 'a', 'sys']
>>>
>>> del a # Delete the variable name a
>>>
>>> dir()
['__builtins__', '__doc__', '__name__', 'sys']
>>>

---

Standard Modules

Python comes with some standard module libraries, which will be introduced in the Python Library Reference documentation (that is, the “Library Reference” later).

Module Name	Function Description
math	Mathematical operations (e.g., square roots, trigonometry)
os	OS-related functionality (e.g., file and directory operations)
sys	System-related parameters and functions
random	Generate random numbers
datetime	Handle dates and times
json	Handle JSON data
re	Regular expression operations
collections	Provide additional data structures (e.g., defaultdict, deque)
itertools	Provide iterator tools
functools	Higher-order function tools (e.g., reduce, lru_cache)

Some modules are built directly into the parser. Although these are not built-in language features, they can be used very efficiently, even for system-level calls.

These components are configured differently depending on the operating system. For example, the winreg module is only available on Windows systems.

It should be noted that there is a special module called sys, which is built into every Python parser. The variables sys.ps1 and sys.ps2 define the strings corresponding to the primary and secondary prompts:

>>> import sys
>>> sys.ps1
'>>> '
>>> sys.ps2
'... '
>>> sys.ps1 = 'C> '
C> print('Runoob!')
Runoob!
C> 

---

Packages

A package is a form of managing Python module namespaces, using “dotted module names.”

For example, if a module name is A.B, then it represents a submodule B within package A.

Just like when using modules, you don’t have to worry about global variables from different modules interfering with each other. Using dotted module names also avoids the problem of modules with the same name across different libraries.

This way, different authors can provide NumPy modules or Python graphics libraries.

Let’s suppose you want to design a set of modules for uniformly handling sound files and data (or what you might call a “package”).

There are many different audio file formats (basically distinguished by suffix, for example: .wav, .aiff, .au), so you need a growing set of modules for converting between different formats.

And for handling these audio data, there are many different operations (such as mixing, adding echo, applying equalizer functions, creating artificial stereo effects), so you also need a set of modules that you can never finish writing to handle these operations.

Here is a possible package structure (in a hierarchical file system):

sound/                          Top-level package
      __init__.py               Initialize the sound package
      formats/                  File format conversion subpackage
              __init__.py
              wavread.py
              wavwrite.py
              aiffread.py
              aiffwrite.py
              auread.py
              auwrite.py
              ...
      effects/                  Sound effects subpackage
              __init__.py
              echo.py
              surround.py
              reverse.py
              ...
      filters/                  Filters subpackage
              __init__.py
              equalizer.py
              vocoder.py
              karaoke.py
              ...

When importing a package, Python searches for subdirectories contained in the package based on the directories in sys.path.

A directory is only considered a package if it contains a file called __init__.py. This is mainly to prevent some common names (like string) from accidentally affecting valid modules in the search path.

In the simplest case, just put an empty __init__.py file. Of course, this file can also contain some initialization code or assign values to the __all__ variable (which will be introduced later).

Users can import only specific modules from a package, for example:

import sound.effects.echo

This will import the submodule sound.effects.echo. It must be accessed using its full name:

sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)

Another way to import a submodule is:

from sound.effects import echo

This also imports the submodule echo, and it does not require those verbose prefixes, so it can be used like this:

echo.echofilter(input, output, delay=0.7, atten=4)

Another variation is to directly import a function or variable:

from sound.effects.echo import echofilter

Similarly, this method imports the submodule echo and allows you to directly use its echofilter() function:

echofilter(input, output, delay=0.7, atten=4)

Note that when using the form from package import item, the corresponding item can be a submodule (or subpackage) within the package, or other names defined within the package, such as functions, classes, or variables.

The import syntax first tries to interpret item as a name defined by the package. If not found, it tries to import it as a module. If still not found, it throws an :exc:ImportError exception.

Conversely, if you use the import form like import item.subitem.subsubitem, all items except the last one must be packages, and the last item can be a module or a package, but cannot be the name of a class, function, or variable.

---

Importing * from a Package

What happens if we use from sound.effects import *?

Python will enter the file system, find all submodules within this package, and import them one by one.

However, this method does not work very well on Windows platforms, because Windows is a case-insensitive system.

On Windows platforms, we cannot determine whether a file called ECHO.py should be imported as the module echo, Echo, or ECHO.

To solve this problem, we just need to provide a precise package index.

The import statement follows this rule: if the package definition file __init__.py has a list variable called __all__, then when using from package import *, all names in this list are imported as package contents.

As the package author, don’t forget to keep __all__ updated after updating the package.

The following example in file sounds/effects/__init__.py contains this code:

__all__ = ["echo", "surround", "reverse"]

This means that when you use from sound.effects import *, you will only import these three submodules from the package.

If __all__ is truly not defined, then using the syntax from sound.effects import * will not import any submodules from the package sound.effects. It only imports the package sound.effects and all the contents defined within it (possibly running the initialization code defined in __init__.py).

This will import all names defined in __init__.py. And it will not break any explicitly specified modules imported before this statement. Look at this code:

import sound.effects.echo
import sound.effects.surround
from sound.effects import *

In this example, before executing from…import, the echo and surround modules in the package sound.effects have already been imported into the current namespace. (Of course, if __all__ is defined, it’s even more problem-free.)

We generally do not recommend using the * method to import modules, because it often leads to reduced code readability. However, it does save a lot of typing, and some modules are designed to be imported only through specific methods.

Remember, using from Package import specific_submodule is never wrong. In fact, this is the recommended method. Unless the submodule you want to import might have the same name as submodules from other packages.

If a package is a subpackage within a structure (as in this example for the package sound), and you want to import a sibling package (a package at the same level), you must use absolute paths for importing. For example, if the module sound.filters.vocoder needs to use the echo module from the package sound.effects, you would write from sound.effects import echo.

from . import echo
from .. import formats
from ..filters import equalizer

Whether implicit or explicit, relative imports start from the current module. The name of the main module is always “__main__”. The main module of a Python application should always use absolute path references.

Packages also provide an additional attribute __path__. This is a list of directories, each containing a directory that has an __init__.py serving this package. You must define it before other __init__.py files are executed. You can modify this variable to affect the modules and subpackages contained within the package.

This feature is not commonly used and is generally used to extend the modules within a package.