Modules, Packages, and Libraries¶
Python allows you to organize code into modules and packages. Using this system, we can share and use each other’s code.
In this video, I’m going to cover virtual environments and the import statement. I’ll also give some very basic directions on how to create your own packages.
Virtualenv¶
If you plan on importing anything, let alone installing anything, you should create a virtualenv.
A virtualenv (venv for short) will isolate your packages and modules from the system packages and modules. What someone else installs in their virtualenv will not affect yours. And you can have different packages than the system packages.
Creating a virtualenv is ridiculously easy.
$ python -m venv <path>
<path>
is the path you want to create your virtualenv inside. Typically, I
name it venv
and I put it at the top-level of whatever project I am
working on. For instance:
$ cd Projects/calculator
$ python -m venv venv
Note that you should be invoking venv with your Python 3.7 installation.
Once you have created your virtualenv, it’s time to activate it.
$ . venv/bin/activate # Linux/MacOS
$ venv/Scripts/activate # Windows
Once you’ve activated your virtualenv, all the packages you install with
pip
will be installed in your virtualenv and be immediately accessible.
See https://PyPI.org/ for a list of publicly available packages. You may also
find packages you want to install at your favorite project’s website.
import statement¶
The simplest form of the import statement is:
import <identifier>
This will import the module named <identifier> and assign it to a variable of the same name.
As an example:
import sys
The exact steps Python follows are:
Check
sys.modules
to see if the module has already been loaded. If so, just assign the module to a variable of the same name in your namespace.If the module has not been loaded, search for a file named
<identifier>.py
or<identifier>/__init__.py
in the virtualenv. (Note: This is not the whole story. It’s quite a bit more complicated than this.)Once the file is found, run it in its own namespace. (The module’s global namespace is not the same as your global namespace. You can access its global namespace through the module attributes.)
Registers the module in
sys.modules
and assign the module to a variable of the same name in your namespace.
Note
You can import at any time. People typically like to import before running code, but this is not necessary.
If you execute the import statement in a function call, then a local variable will be used for the module name, rather than a global.
Accessing the Contents of a Module¶
Once you’ve imported a module, you can access objects within its namespace via
the attribute access .
syntax:
import sys
sys.modules
You should read the documentation on the module if you want to know what is in it.
If you don’t have the documentation available, or you just want a quick
reminder, then use help()
or dir()
>>> import sys
>>> help(sys) # Documentation of all the objects
>>> dir(sys) # List of all the objects
ImportError¶
If anything goes wrong during the import process, an ImportError
is
raised. You can catch this error and handle it:
try:
import foo
except ImportError:
print("Error!")
Typically, ImportError
isn’t caught and handled. The only case where I
have done this is when I have code that needs to run in Python 2 and 3, and
the module is named differently.
ImportErrors could arise for any one of these problems:
There is no module with that name.
The module has a SyntaxError that prevents Python from compiling and running it.
While running the module, an exception was raised.
There isn’t much you can do except resolve the underlying problems. Maybe you
misspelled the module name, or you forgot to install it with pip
, or you
wrote the module yourself and created a SyntaxError or some other bug in it.
Importing multiple modules¶
You can import multiple modules at the same time:
import os, sys
This is the same as importing each of the modules one at a time.
import os
import sys
Importing Modules in Packages¶
Some modules are organized in a package.
Note
Modules are files in Python.
Packages are paths to the files in Python.
For instance, foo/bar/baz.py
would be the module foo.bar.baz
and the
package foo.bar
.
This is an important distinction in Python, as we’ll see when we write our own packages and modules.
import <package>.<module>
This will assign the top-level package to a variable of the same name. You can
access the module with the .
syntax.
You may have packages nested in packages nested in packages. Use as many .
as needed.
There aren’t many nested packages in the Python library. Here’s an example, though:
import os.path
os.path.join(...)
Importing with a different name¶
Sometimes you want to import a module, but store it under a different name.
import sys as s
s.modules
This will import sys but assign it to s
.
This works for package imports as well:
import os.path as p
p.join
The benefit of this is the module, not the package, is assigned to the identifier.
Importing specific objects¶
You can import individual objects from a module:
from <module> import <id>, <id>, <id>
If you have a lot of things, using parentheses is OK:
from <module> import (
<id>,
<id>,
<id>,
)
Note that you can also import from modules in a package this way.
This will import the module, and then assign the objects to variables of the same name.
If you want a different name, use as
:
from <module> import (
x as a,
y as b,
z as c,
)
Module Special Attributes¶
Modules have a few special attributes.
__name__
: The name of the module.__package__
: The name of the package of the module.__file__
: The location of the file the module came from.__doc__
: The doc string.
Running a module as a script¶
You can invoke any module as a script from the commandline:
$ python -m <module name> <script parameters>
We did this with the venv
module when creating a virtual environment:
$ python -m venv venv
if __name__ == '__main__':
¶
Sometimes you want to write a module that also functions as a script.
When a python file is loaded as a script, Python treats it as a module named
__main__
. You can check the module’s __name__
from within the module
itself.
This means you can hide the code that should only be run as a script behind this if statement:
... module code ...
if __name__ == '__main__':
... script code ...
There is a better way to do this using setuptools
that I’ll cover in a
later video.
Creating your own Packages and Modules¶
Creating your own Packages and Modules is a fairly involved process that
involves writing a setup.py
file. There isn’t a lot of consensus in the
Python community how to do this properly, so there is a lot of confusion
around it.
I recommend something simple and robust.
mycode/ # Project directory
setup.py
venv/ # For your virtual environment
mycode/ # If a package
__init__.py # Empty
mycode.py
...
mycode.py # If a single module
To structure your code, first create a directory. You’ll likely want to manage
this directory with git
or a similar tool. (I’ll create a video on what
you really need to know about git later.)
You need a package or module name. I’m using mycode
as a stand-in for
that.
The package would live under a directory called mycode
. In that directory,
you need a __init__.py
file, which I recommend leaving blank. If you have
any subdirectories, they each need their own __init__.py
, which I
recommend leaving blank. Any modules you want you just drop in the appropriate
directory with the appropriate name. Let’s say you want a mycode.foo.bar
module. In that case, create a directory called mycode/foo
and drop an
__init__.py
there, then add in mycode/foo/bar.py
for the module file.
If you just want a single module with no packages, then just create
mycode.py
in your project directory.
Next, you need a setup.py
file. People try to get really fancy with their
setup.py
files, but I strongly discourage this.
Here’s bare minimum of what I would put in my setup.py
file:
from setuptools import setup, find_packages
setup(
name='mycode',
version='0.0.1',
python_requires='>=3.7',
# If a package, then...
packages=find_packages(include=['mycode', 'mycode.*']),
# If a module, then...
py_modules=['mycode'],
)
If you are serious about creating packages and modules, then I highly recommend reading and understanding the content at https://packaging.python.org/. I really can’t do a better job than what they have done.
That said, every project is different, and you’re likely going to run into cases where you’re at a loss for how to properly package it. I can help. Find me on discord.
Installing your Own Code for Testing¶
Once you get your setup.py written, then install your code with pip install -e
<project directory>
. That’s all you need. With the venv activated, you can
import your package or module however you see fit.
The -e
option tells pip to not copy the files over, but to link them. This
means that when you change one of the files, those changes will appear when
someone imports your modules and packages again.
Note that if you add directories, you’ll have to pip install
again,
otherwise they won’t be linked in to the venv.
Distributing your Code¶
Before you go launching your code to PyPI, talk to me first, please. I can help you get your package production ready so that you won’t make some rookie mistakes when deploying packages.
Future Videos¶
Going into setuptools.setup in depth
Conflict resolution
git