[python] setup.py examples?

After studying this page:

http://docs.python.org/distutils/builtdist.html

I am hoping to find some setup.py files to study so as to make my own (with the goal of making a fedora rpm file).

Could the s.o. community point me towards some good examples?

You may find the HitchHiker's Guide to Packaging helpful, even though it is incomplete. I'd start with the Quick Start tutorial. Try also just browsing through Python packages on the Python Package Index. Just download the tarball, unpack it, and have a look at the setup.py file. Or even better, only bother looking through packages that list a public source code repository such as one hosted on GitHub or BitBucket. You're bound to run into one on the front page.

My final suggestion is to just go for it and try making one; don't be afraid to fail. I really didn't understand it until I started making them myself. It's trivial to create a new package on PyPI and just as easy to remove it. So, create a dummy package and play around.

I recommend the setup.py of the Python Packaging User Guide's example project.

The Python Packaging User Guide "aims to be the authoritative resource on how to package, publish and install Python distributions using current tools".

Minimal example

from setuptools import setup, find_packages


setup(
    name="foo",
    version="1.0",
    packages=find_packages(),
)

More info in docs

Here is the utility I wrote to generate a simple setup.py file (template) with useful comments and links. I hope, it will be useful.

Installation

sudo pip install setup-py-cli

Usage

To generate setup.py file just type in the terminal.

setup-py

Now setup.py file should occur in the current directory.

Generated setup.py

from distutils.core import setup
from setuptools import find_packages
import os


# User-friendly description from README.md
current_directory = os.path.dirname(os.path.abspath(__file__))
try:
    with open(os.path.join(current_directory, 'README.md'), encoding='utf-8') as f:
        long_description = f.read()
except Exception:
    long_description = ''

setup(
    # Name of the package
    name=<name of current directory>,

    # Packages to include into the distribution
    packages=find_packages('.'), 

    # Start with a small number and increase it with every change you make
    # https://semver.org
    version='1.0.0',

    # Chose a license from here: https://help.github.com/articles/licensing-a-repository
    # For example: MIT
    license='',

    # Short description of your library
    description='',

    # Long description of your library
    long_description = long_description,
    long_description_context_type = 'text/markdown',

    # Your name
    author='', 

    # Your email
    author_email='',     

    # Either the link to your github or to your website
    url='',

    # Link from which the project can be downloaded
    download_url='',

    # List of keyword arguments
    keywords=[],

    # List of packages to install with this one
    install_requires=[],

    # https://pypi.org/classifiers/
    classifiers=[]  
)

Content of the generated setup.py:

• automatically fulfilled package name based on the name of the current directory.
• some basic fields to fulfill.
• clarifying comments and links to useful resources.
• automatically inserted description from README.md or an empty string if there is no README.md.

Here is the link to the repository. Fill free to enhance the solution.

Look at this complete example https://github.com/marcindulak/python-mycli of a small python package. It is based on packaging recommendations from https://packaging.python.org/en/latest/distributing.html, uses setup.py with distutils and in addition shows how to create RPM and deb packages.

The project's setup.py is included below (see the repo for the full source):

#!/usr/bin/env python

import os
import sys

from distutils.core import setup

name = "mycli"

rootdir = os.path.abspath(os.path.dirname(__file__))

# Restructured text project description read from file
long_description = open(os.path.join(rootdir, 'README.md')).read()

# Python 2.4 or later needed
if sys.version_info < (2, 4, 0, 'final', 0):
    raise SystemExit, 'Python 2.4 or later is required!'

# Build a list of all project modules
packages = []
for dirname, dirnames, filenames in os.walk(name):
        if '__init__.py' in filenames:
            packages.append(dirname.replace('/', '.'))

package_dir = {name: name}

# Data files used e.g. in tests
package_data = {name: [os.path.join(name, 'tests', 'prt.txt')]}

# The current version number - MSI accepts only version X.X.X
exec(open(os.path.join(name, 'version.py')).read())

# Scripts
scripts = []
for dirname, dirnames, filenames in os.walk('scripts'):
    for filename in filenames:
        if not filename.endswith('.bat'):
            scripts.append(os.path.join(dirname, filename))

# Provide bat executables in the tarball (always for Win)
if 'sdist' in sys.argv or os.name in ['ce', 'nt']:
    for s in scripts[:]:
        scripts.append(s + '.bat')

# Data_files (e.g. doc) needs (directory, files-in-this-directory) tuples
data_files = []
for dirname, dirnames, filenames in os.walk('doc'):
        fileslist = []
        for filename in filenames:
            fullname = os.path.join(dirname, filename)
            fileslist.append(fullname)
        data_files.append(('share/' + name + '/' + dirname, fileslist))

setup(name='python-' + name,
      version=version,  # PEP440
      description='mycli - shows some argparse features',
      long_description=long_description,
      url='https://github.com/marcindulak/python-mycli',
      author='Marcin Dulak',
      author_email='[email protected]',
      license='ASL',
      # https://pypi.python.org/pypi?%3Aaction=list_classifiers
      classifiers=[
          'Development Status :: 1 - Planning',
          'Environment :: Console',
          'License :: OSI Approved :: Apache Software License',
          'Natural Language :: English',
          'Operating System :: OS Independent',
          'Programming Language :: Python :: 2',
          'Programming Language :: Python :: 2.4',
          'Programming Language :: Python :: 2.5',
          'Programming Language :: Python :: 2.6',
          'Programming Language :: Python :: 2.7',
          'Programming Language :: Python :: 3',
          'Programming Language :: Python :: 3.2',
          'Programming Language :: Python :: 3.3',
          'Programming Language :: Python :: 3.4',
      ],
      keywords='argparse distutils cli unittest RPM spec deb',
      packages=packages,
      package_dir=package_dir,
      package_data=package_data,
      scripts=scripts,
      data_files=data_files,
      )

and and RPM spec file which more or less follows Fedora/EPEL packaging guidelines may look like:

# Failsafe backport of Python2-macros for RHEL <= 6
%{!?python_sitelib: %global python_sitelib      %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
%{!?python_sitearch:    %global python_sitearch     %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
%{!?python_version: %global python_version      %(%{__python} -c "import sys; sys.stdout.write(sys.version[:3])")}
%{!?__python2:      %global __python2       %{__python}}
%{!?python2_sitelib:    %global python2_sitelib     %{python_sitelib}}
%{!?python2_sitearch:   %global python2_sitearch    %{python_sitearch}}
%{!?python2_version:    %global python2_version     %{python_version}}

%{!?python2_minor_version: %define python2_minor_version %(%{__python} -c "import sys ; print sys.version[2:3]")}

%global upstream_name mycli


Name:           python-%{upstream_name}
Version:        0.0.1
Release:        1%{?dist}
Summary:        A Python program that demonstrates usage of argparse
%{?el5:Group:       Applications/Scientific}
License:        ASL 2.0

URL:            https://github.com/marcindulak/%{name}
Source0:        https://github.com/marcindulak/%{name}/%{name}-%{version}.tar.gz

%{?el5:BuildRoot:   %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)}
BuildArch:      noarch

%if 0%{?suse_version}
BuildRequires:      python-devel
%else
BuildRequires:      python2-devel
%endif


%description
A Python program that demonstrates usage of argparse.


%prep
%setup -qn %{name}-%{version}


%build
%{__python2} setup.py build


%install
%{?el5:rm -rf $RPM_BUILD_ROOT}
%{__python2} setup.py install --skip-build --prefix=%{_prefix} \
   --optimize=1 --root $RPM_BUILD_ROOT


%check
export PYTHONPATH=`pwd`/build/lib
export PATH=`pwd`/build/scripts-%{python2_version}:${PATH}
%if 0%{python2_minor_version} >= 7
%{__python2} -m unittest discover -s %{upstream_name}/tests -p '*.py'
%endif


%clean
%{?el5:rm -rf $RPM_BUILD_ROOT}


%files
%doc LICENSE README.md
%{_bindir}/*
%{python2_sitelib}/%{upstream_name}
%{?!el5:%{python2_sitelib}/*.egg-info}


%changelog
* Wed Jan 14 2015 Marcin Dulak <[email protected]> - 0.0.1-1
- initial version

Here you will find the simplest possible example of using distutils and setup.py:

https://docs.python.org/2/distutils/introduction.html#distutils-simple-example

This assumes that all your code is in a single file and tells how to package a project containing a single module.

READ THIS FIRST https://packaging.python.org/en/latest/current.html

Installation Tool Recommendations

1. Use pip to install Python packages from PyPI.
2. Use virtualenv, or pyvenv to isolate application specific dependencies from a shared Python installation.
3. Use pip wheel to create a cache of wheel distributions, for the purpose of > speeding up subsequent installations.
4. If you’re looking for management of fully integrated cross-platform software stacks, consider buildout (primarily focused on the web development community) or Hashdist, or conda (both primarily focused on the scientific community).

Packaging Tool Recommendations

1. Use setuptools to define projects and create Source Distributions.
2. Use the bdist_wheel setuptools extension available from the wheel project to create wheels. This is especially beneficial, if your project contains binary extensions.
3. Use twine for uploading distributions to PyPI.

This anwser has aged, and indeed there is a rescue plan for python packaging world called

wheels way

I qoute pythonwheels.com here:

What are wheels?

Wheels are the new standard of python distribution and are intended to replace eggs. Support is offered in pip >= 1.4 and setuptools >= 0.8.

Advantages of wheels

1. Faster installation for pure python and native C extension packages.
2. Avoids arbitrary code execution for installation. (Avoids setup.py)
3. Installation of a C extension does not require a compiler on Windows or OS X.
4. Allows better caching for testing and continuous integration.
5. Creates .pyc files as part of installation to ensure they match the python interpreter used.
6. More consistent installs across platforms and machines.

The full story of correct python packaging (and about wheels) is covered at packaging.python.org

conda way

For scientific computing (this is also recommended on packaging.python.org, see above) I would consider using CONDA packaging which can be seen as a 3rd party service build on top of PyPI and pip tools. It also works great on setting up your own version of binstar so I would imagine it can do the trick for sophisticated custom enterprise package management.

Conda can be installed into a user folder (no super user permisssions) and works like magic with

conda install

and powerful virtual env expansion.

eggs way

This option was related to python-distribute.org and is largerly outdated (as well as the site) so let me point you to one of the ready to use yet compact setup.py examples I like:

• A very practical example/implementation of mixing scripts and single python files into setup.py is giving here
• Even better one from hyperopt

This quote was taken from the guide on the state of setup.py and still applies:

• setup.py gone!
• distutils gone!
• distribute gone!
• pip and virtualenv here to stay!
• eggs ... gone!

I add one more point (from me)

• wheels!

I would recommend to get some understanding of packaging-ecosystem (from the guide pointed by gotgenes) before attempting mindless copy-pasting.

from distutils.core import setup

from setuptools import setup

pip install Distutils2

python -m distutils2.run install

PS

Packaging never was trivial (one learns this by trying to develop a new one), so I assume a lot of things have gone for reason. I just hope this time it will be is done correctly.