[python] Including non-Python files with setup.py

How do I make setup.py include a file that isn't part of the code? (Specifically, it's a license file, but it could be any other thing.)

I want to be able to control the location of the file. In the original source folder, the file is in the root of the package. (i.e. on the same level as the topmost __init__.py.) I want it to stay exactly there when the package is installed, regardless of operating system. How do I do that?

In setup.py under setup( :

setup(
   name = 'foo library'
   ...
  package_data={
   'foolibrary.folderA': ['*'],     # All files from folder A
   'foolibrary.folderB': ['*.txt']  #All text files from folder B
   },

It is 2019, and here is what is working - despite advice here and there, what I found on the internet halfway documented is using setuptools_scm, passed as options to setuptools.setup. This will include any data files that are versioned on your VCS, be it git or any other, to the wheel package, and will make "pip install" from the git repository to bring those files along.

So, I just added these two lines to the setup call on "setup.py". No extra installs or import required:

    setup_requires=['setuptools_scm'],
    include_package_data=True,

No need to manually list package_data, or in a MANIFEST.in file - if it is versioned, it is included in the package. The docs on "setuptools_scm" put emphasis on creating a version number from the commit position, and disregard the really important part of adding the data files. (I can't care less if my intermediate wheel file is named "*0.2.2.dev45+g3495a1f" or will use the hardcoded version number "0.3.0dev0" I've typed in - but leaving crucial files for the program to work behind is somewhat important)

This works in 2020!

As others said create "MANIFEST.in" where your setup.py is located.

Next in manifest include/exclude all the necessary things. Be careful here regarding the syntax. Ex: lets say we have template folder to be included in the source package.

in manifest file do this :

recursive-include template *

Make sure you leave space between dir-name and pattern for files/dirs like above. Dont do like this like we do in .gitignore

recursive-include template/* [this won't work]

Other option is to use include. There are bunch of options. Look up here at their docs for Manifest.in

And the final important step, include this param in your setup.py and you are good to go!

   setup(
    ...
    include_package_data=True,
    ......
)

Hope that helps! Happy Coding!

Step 1: create a MANIFEST.in file in the same folder with setup.py

Step 2: include the relative path to the files you want to add in MANIFEST.in

include README.rst
include docs/*.txt
include funniest/data.json

Step 3: set include_package_data=True in the setup() function to copy these files to site-package

Reference is here.

Here is a simpler answer that worked for me.

First, per a Python Dev's comment above, setuptools is not required:

package_data is also available to pure distutils setup scripts 
since 2.3. – Éric Araujo

That's great because putting a setuptools requirement on your package means you will have to install it also. In short:

from distutils.core import setup

setup(
    # ...snip...
    packages          = ['pkgname'],
    package_data      = {'pkgname': ['license.txt']},
)

None of the answers worked for me because my files were at the top level, outside the package. I used a custom build command instead.

import os
import setuptools
from setuptools.command.build_py import build_py
from shutil import copyfile

HERE = os.path.abspath(os.path.dirname(__file__))
NAME = "thepackage"

class BuildCommand(build_py):
    def run(self):
        build_py.run(self)

        if not self.dry_run:
            target_dir = os.path.join(self.build_lib, NAME)
            for fn in ["VERSION", "LICENSE.txt"]:
                copyfile(os.path.join(HERE, fn), os.path.join(target_dir,fn))

 
 
setuptools.setup(
    name=NAME,
    cmdclass={"build_py": BuildCommand},
    description=DESCRIPTION,
    ...
)

I wanted to post a comment to one of the questions but I don't enough reputation to do that >.>

Here's what worked for me (came up with it after referring the docs):

package_data={
    'mypkg': ['../*.txt']
},

include_package_data: False

The last line was, strangely enough, also crucial for me (you can also omit this keyword argument - it works the same).

What this does is it copies all text files in your top-level or root directory (one level up from the package mypkg you want to distribute).

Hope this helps!

Figured out a workaround: I renamed my lgpl2.1_license.txt to lgpl2.1_license.txt.py, and put some triple quotes around the text. Now I don't need to use the data_files option nor to specify any absolute paths. Making it a Python module is ugly, I know, but I consider it less ugly than specifying absolute paths.

I just wanted to follow up on something I found working with Python 2.7 on Centos 6. Adding the package_data or data_files as mentioned above did not work for me. I added a MANIFEST.IN with the files I wanted which put the non-python files into the tarball, but did not install them on the target machine via RPM.

In the end, I was able to get the files into my solution using the "options" in the setup/setuptools. The option files let you modify various sections of the spec file from setup.py. As follows.

from setuptools import setup


setup(
    name='theProjectName',
    version='1',
    packages=['thePackage'],
    url='',
    license='',
    author='me',
    author_email='[email protected]',
    description='',
    options={'bdist_rpm': {'install_script': 'filewithinstallcommands'}},
)

file - MANIFEST.in:

include license.txt

file - filewithinstallcommands:

mkdir -p $RPM_BUILD_ROOT/pathtoinstall/
#this line installs your python files
python setup.py install -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
#install license.txt into /pathtoinstall folder
install -m 700 license.txt $RPM_BUILD_ROOT/pathtoinstall/
echo /pathtoinstall/license.txt >> INSTALLED_FILES

create MANIFEST.in in the project root with recursive-include to the required directory or include with the file name.

include LICENSE
include README.rst
recursive-include package/static *
recursive-include package/templates *

documentation can be found here

To accomplish what you're describing will take two steps...

• The file needs to be added to the source tarball
• setup.py needs to be modified to install the data file to the source path

Step 1: To add the file to the source tarball, include it in the MANIFEST

Create a MANIFEST template in the folder that contains setup.py

The MANIFEST is basically a text file with a list of all the files that will be included in the source tarball.

Here's what the MANIFEST for my project look like:

• CHANGELOG.txt
• INSTALL.txt
• LICENSE.txt
• pypreprocessor.py
• README.txt
• setup.py
• test.py
• TODO.txt

Note: While sdist does add some files automatically, I prefer to explicitly specify them to be sure instead of predicting what it does and doesn't.

Step 2: To install the data file to the source folder, modify setup.py

Since you're looking to add a data file (LICENSE.txt) to the source install folder you need to modify the data install path to match the source install path. This is necessary because, by default, data files are installed to a different location than source files.

To modify the data install dir to match the source install dir...

Pull the install dir info from distutils with:

from distutils.command.install import INSTALL_SCHEMES

Modify the data install dir to match the source install dir:

for scheme in INSTALL_SCHEMES.values():
    scheme['data'] = scheme['purelib']

And, add the data file and location to setup():

data_files=[('', ['LICENSE.txt'])]

Note: The steps above should accomplish exactly what you described in a standard manner without requiring any extension libraries.