Python Packages ############### .. wordcount:: .. vim:filetype=rst spell: .. include:: /references.inc Last time we looked at Python's ``Modules``. Basically, a ``module`` is a simple way to break up a program that is getting too big. We cut the big file up into chunks, hopefully containing related parts of the program, and place them in a separate file. That really helps keep yu focused on pne part o f the program at a time. As the program grows in size, you will appreciate this system a lot. Too Many Modules **************** A program can get even bigger, and the ``module`` idea starts to fall apart. Sure, you can create hundreds of files in a single folder, but do you really want to wade through a giant list of files looking for the one you need to look at? We can take the ``module`` idea one step further. We can group modules into a separate directory. With a little tuning, we will call this new directory a ``Python Package``. Package Structure ***************** You will find many Python programmers using a simple scheme to set up a ``package``. You simply move your modules into the new folder, then add a file named ``__init__.py``. Often that file has no text in it, but in older versions of Python, that file needed to be in a folder for Python to accept it as a ``pacKage``! Modern Python versions (3.3+) do not need that ``__init__.py`` file, but it is often put in a ``package`` in case it is needed later (More on that soon in this discussion!) Let's create a simple example. Package Setup ************* In this example, I will show ``command line`` steps to set up a simple Python Project. We will see a new way to build a program as we do this. .. note:: Of course, you can create this setup using a tool like the ``Windows File Explorer`` on a PC, or the ``Finder`` tool on a Mac. Pick a Project Name =================== Picking a name is an important first step. Developers looking for programs (maybe on GitHub_) to help them with something will search for information based on some name, so your name should be helpful to those folks. For our simple example, we will use ``UnitConverter`` as our project name: .. code-block:: bash $ mkdir UnitConverter $ cd UnitConverter Create Basic Folders ==================== We willl organize our project in a few basic folders: * source - where the Python code will live * tests - where testing code will live (see below) * docs - where documentation will live Here is how we can set this part up: .. code-block:: bash > mkdir source > mkdir tests > mkdir docs Ready to Code? ************** Let's start coding. In the ``source`` folder, create this file: .. literalinclude:: code/units1/temps.py :linenos: :caption: source/temps.py Notice that last ``if`` block. This is a python ``module``, but we are going to test it directly. Now do this: .. command-output:: python -m temps :cwd: code/units1 Let's run this as a module. To do that, we need a simple application: .. literalinclude:: code/units1/units.py :linenos: :caption: source/units.py Now, run this application: .. command-output:: python units.py :cwd: code/units1 It looks like this works as expected. Notice that I used that standard ``if`` block at the end. This is a common practice in the Python world. Package Time ************ Now, create a new folder under ``source``: .. code-block:: bash > mkdir units Create an empty file named ``__init__.py`` in this new folder. (You can just open up gVim and save the initially empty file with the name needed.) Now, move your ``temps.py`` into this folder and modify the application so it looks like this: .. literalinclude:: code/units2/units.py :linenos: :caption: source/units.py Now, run this modified application: .. command-output:: python units.py :cwd: code/units2 This works, but it has an ugly ``import`` line. We created a ``package`` we named ``units``, but we needed to know the name of the python ``module`` inside of that package. Users do not need to see that, or even worry about that. So, how do we fix this? Remember how we used the ``sqrt`` function: All we did was something like this: .. code-block:: python from math import sqrt ... x = sqrt(2) Or, we could do this: .. code-block:: python import math x = math.sqrt(3) Namespaces ========== Python maintains a list of all names defined in your code at the moment. This list has a bunch of names Python uses internally, and that list expands as your define things in your code. There is a built-in function named ``dir`` that you can use to see what names are defined. Let's modify our application so we can see this list: In the first case, we are making the name ``sqrt`` directly part of our local ``namespace``, a fancy term meaning all the names of things defined in our code at the moment. In the second case, ``sqrt`` is not in our ``namespace, but ``math`` is. We know that ``sqrt`` is in that ``math`` module )or is os a package?) So we tell Python to access the needed function with a slightly longer call. .. note:: Best practices in programming tells us to NEVER use ``from >> dir() ['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__'] Nothins looks familiar there. These are all default names created by Python. Next, import our package: .. code-block:: text >>> import units >>> dir() ['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'units'] We still do not have access to the functions. Try again: .. code-block:: text >>> from units import * >>> dir() ['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'units'] That did not work. One more try: .. code-block:: text >>> from units.temps import * >>> dir() ['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'ctof', 'ftoc', 'units'] Now, we see our two functions. Initializing the Namespace -------------------------- What we want to do is to get Python to add these names to the namespace using the first simple import option. That is where the ``__init__.py`` file comes in! This file is processed by Python when you import the package. We can add any code here we like, but it is common to keep "real" code out of here for now. Modify your ``_init__.py`` file (in the ``units`` folder) so it looks like this: .. literalinclude:: code/units3/init.py :linenos: :caption: source/units/__init__.py Check that we can now access our functions: .. code-block:: text >>> import units >>> dir(units) ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'ctof', 'ftoc', 'temps'] Success! It seems odd to need to reference the package name inside of the package. That makes it hard to "reuse" this package in another project where the package name might need to be changed. Python lets us simplify the import line so this problem does not exist. Simply drop the package name from the import line, leaving the dot and the module name: .. literalinclude:: code/units3/units/__init__.py :caption: source/units/__init__.py Now, our application can look like this: .. literalinclude:: code/units3/units.py :linenos: :caption: source/units.py Try it out: .. command-output:: python units.py :cwd: code/units3 Now we have a working package. We can add more unit conversions! Add Another Module ****************** We can add anothe rmodule to our 11units`` package as easliy as dropping another file in the folder, and adding a line to the ``__init__.py`` file: .. literalinclude:: code/units4/units/lengths.py :linenos: :caption: source/units/length.py .. literalinclude:: code/units4/units/__init__.py :linenos: :caption: source/units/__init__.py We need to add more calls to out test application. Hmmm, let's rename it to ``test_units.py`` .. literalinclude:: code/units4/test_units.py :linenos: :caption: source/test_units.py Try it out: .. command-output:: python test_units.py :cwd: code/units4 Wrapping Up *********** We started off by building an application suitable for public use. We need a real front end interface to our new package to make is "user friendly". Try this in the ``source`` directory: .. literalinclude:: UnitConverter/source/run.py :linenos: :caption: source/run.py Try it out, in the ``source`` folder: .. command-output:: python run.py :cwd: UnitConverter/source .. command-output:: python run.py -ftom 5280 :cwd: UnitConverter/source I will leave it to you to complete the unit conversion package. Testing ******* Why did we change the name of our old test program to ``test_units``? It is vitally important that you test your code. That is why we set up a ``tests`` folder for this project. Move the ``test_units.py`` file into ``tests``. This is where we will put all test code for this project. But how do we actually test? The code we wrote originally was just a test application that does test the package code, but this is a mess to set up for a large project. Python developers fixed that problem long ago by creating a cool package (!) called ``PyTest``. Python has literally hundreds (thousands) of cool packages around. We will focus on testing here. Assuming you have ``pip`` installed, which should be true if you installed Python from the official website, do this from a command line: .. command-output:: pip --version That proves ``pip`` is installed. ``Pip`` is the standard tool for loading publicly available Python packages from the ``Python Package Index`` (``PyPi``), where open source Python developers publish their code. We will try out ``PyTest`` for testing our package. .. code-block:: bash $ pip install pytest .. warning:: You need to be connected to the Internet when you run this command. The output will show you messages as all the needed files are downloaded and installed on your system. Ditch that original test code, and replace it with this version: .. literalinclude:: UnitConverter/tests/test_units.py :linenos: :caption: tests/test_units.py Once that file is on place, run the test command from the command line at the top of your project folder: .. command-output:: pytest tests :cwd: UnitConverter Hey, you are well on your way to becoming a "real" Python developer!