Distutils and You

Writing a python module is great, but what’s really important is getting it out to others so they can make use of it too. What’s the best way to do that?

Of course, the best way is to use existing infrastructure in the Python community. PyPI is the place to get Python modules. You can browse them on the web, and they are all just a “easy_install” or “pip install” away.

Want people to be able to “pip install” or “easy_install” your module?

First, you must have your module in an standard module format. In this case, we’re just going to assume we’re starting a new project.

PasteScript is great for creating skeletons for projects. Let’s install it.

$ pip install PasteScript

In this case, we just need to creat a the basic Python package layout.

$ cd projects
$ paster create --template=basic_package fibtools
Selected and implied templates:
  PasteScript#basic_package  A basic setuptools-enabled package

Variables:
  egg:      fibtools
  package:  fibtools
  project:  fibtools
Enter version (Version (like 0.1)) ['']: 0.1.0
Enter description (One-line description of the package) ['']: A set of tools for working with fibonacci numbers
Enter long_description (Multi-line description (in reST)) ['']: This set of tools is great for generating fibonacci numbers and doing things with them!
Enter keywords (Space-separated keywords/tags) ['']: fibonacci, numbers, math
Enter author (Author name) ['']: Your Name
Enter author_email (Author email) ['']: email@site.com
Enter url (URL of homepage) ['']: homepage.my.site
Enter license_name (License name) ['']: MIT
Enter zip_safe (True/False: if the package can be distributed as a .zip file) [False]:
Creating template basic_package
Creating directory ./fibtools
  Recursing into +package+
    Creating ./fibtools/fibtools/
    Copying __init__.py to ./fibtools/fibtools/__init__.py
  Copying setup.cfg to ./fibtools/setup.cfg
  Copying setup.py_tmpl to ./fibtools/setup.py

It creates a directory for your module and creates files required by distutils.

$ ls -R fibtools
fibtools:
fibtools/  fibtools.egg-info/  setup.cfg  setup.py

fibtools/fibtools:
__init__.py

fibtools/fibtools.egg-info:
dependency_links.txt  entry_points.txt  not-zip-safe  PKG-INFO  SOURCES.txt  top_level.txt

Now let’s edit the init.py for our module, and put in some basic functionality.

We’ll just make a simple generator that spits out Fibonacci numbers.

#!/usr/bin/env python
def fibs():
	a = 1
	b = 1
	while True:
		yield a
		(a, b) = (b, a+b)

if __name__ == "__main__":
	generator = fibs()
	for i in range(10):
		print generator.next()

To test that your module works on your system you can go into your “projects/fibtools” directory and run “python setup.py develop”.

What this does is make links in your “/usr/lib/python/site-packages” directory to wherever the module is in your file system. This way, if you make changes to your module then the site-packages version always stays up to date. This is great, obviously, if you’re doing development and will need to test your changes. Once you’ve done this, you can import your module in a python shell just as you’d expect.

>>> import fibtools
>>> g = fibtools.fibs()
>>> print g.next()
1
>>> print g.next()
1
>>> print g.next()
2
>>> print g.next()
3

Once your module works to your satisfaction, you can now register your module with PyPI.

$ pwd
~/projects/fibtools
$ python setup.py register
running register
running egg_info
writing fibtools.egg-info/PKG-INFO
writing top-level names to fibtools.egg-info/top_level.txt
writing dependency_links to fibtools.egg-info/dependency_links.txt
writing entry points to fibtools.egg-info/entry_points.txt
reading manifest file 'fibtools.egg-info/SOURCES.txt'
writing manifest file 'fibtools.egg-info/SOURCES.txt'
running check
We need to know who you are, so please choose either:
 1. use your existing login,
 2. register as a new user,
 3. have the server generate a new password for you (and email it to you), or
 4. quit
Your selection [default 1]:

At this point, if you already have a PyPI account, you can just use your existing login to register your module. Otherwise, you’ll have to make an account. It’s quick and painless, of course, and just requires a username, password, and email address. It’s easiest if you cache your credentials locally, it’s simpler to upload updated versions of your module later.

Distutils will then automatically upload your code to PyPI and you can tell your friends to “pip install” or “easy_install” it.

By default, there is one flaw with the setup.py that is generated by Paster. It’s not big, but can be a bit of a hindrance. So let’s fix it, and re-upload your script to PyPI with a fixed version of setup.py.

from setuptools import setup, find_packages
import sys, os

version = '0.1.0'

setup(name='fibtools',
    version=version,
    description="A set of tools for working with fibonacci numbers",
    long_description="""\
    his set of tools is great for generating fibonacci numbers and doing things with them!""",
    classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers
    keywords='fibonacci, numbers, math',
    author='Your Name',
    author_email='email@site.com',
    url='homepage.my.site',
    license='MIT',
    packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),
    include_package_data=True,
    zip_safe=False,
    install_requires=[
    # -*- Extra requirements: -*-
    ],
    entry_points="""
    # -*- Entry points: -*-
    """,
)

We need to add a keywords list so that people can find our module. Add this anywhere inside of the parentheses:

keywords = ['fibtools', 'fibonacci', 'numbers']

And while we’re at it, we have to update the version number. PyPI does not let you re-upload a package with the same version twice. This is so that people can always know if they have the latest available version of a package; they can’t know if version numbers don’t increment. So change the “version” variable to “0.1.1” and now let’s re-upload the corrected version.

pwd ~/projects/fibtools
python setup.py sdist upload

running sdist
running egg_info
writing requirements to fibtools.egg-info/requires.txt
writing fibtools.egg-info/PKG-INFO
writing top-level names to fibtools.egg-info/top_level.txt
writing dependency_links to fibtools.egg-info/dependency_links.txt
reading manifest file 'fibtools.egg-info/SOURCES.txt'
writing manifest file 'fibtools.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.txt

creating fibtools-0.1.1
creating fibtools-0.1.1
creating fibtools-0.1.1/fibtools
creating fibtools-0.1.1/fibtools.egg-info
making hard links in fibtools-0.1.1...
Writing fibtools-0.1.1/setup.cfg
Creating tar archive
removing 'fibtools-0.1.1' (and everything under it)
running upload
Submitting dist/fibtools-0.1.1.tar.gz to http://pypi.python.org/pypi
Server response (200): OK

And that’s it! Now people can “pip install” or “easy_install” your module.

Ryan Scott Brown

Distutils and You