How do I run all Python unit tests in a directory

Question

I have a directory that contains my Python unit tests  Each unit test module is of the form test   py  I am attempting to make a file called all test py that will  you guessed it  run all files in the aforementioned test form and return the result  I have tried two methods so far  both have failed  I will show the two methods  and I hope someone out there knows how to actually do this correctly   For my first valiant attempt  I thought  If I just import all my testing modules in the file  and then call this unittest main   doodad  it will work  right   Well  turns out I was wrong   import glob import unittest  testSuite   unittest TestSuite   test file strings   glob glob  test   py   module strings    str 0 len str -3  for str in test file strings   if   name         main          unittest main     This did not work  the result I got was     python all test py   ---------------------------------------------------------------------- Ran 0 tests in 0 000s  OK   For my second try  I though  ok  maybe I will try to do this whole testing thing in a more  manual  fashion  So I attempted to do that below   import glob import unittest  testSuite   unittest TestSuite   test file strings   glob glob  test   py   module strings    str 0 len str -3  for str in test file strings     import   str  for str in module strings  suites    unittest TestLoader   loadTestsFromName str  for str in module strings   testSuite addTest suite  for suite in suites  print testSuite   result   unittest TestResult   testSuite run result  print result   Ok  at this point I have a result  How do I display it as the normal unit test command line output  if   name         main         unittest main     This also did not work  but it seems so close     python all test py   lt unittest TestSuite tests   lt unittest TestSuite tests   lt unittest TestSuite tests   lt test main TestMain testMethod test respondes to get gt   gt   gt   gt   lt unittest TestResult run 1 errors 0 failures 0 gt   ---------------------------------------------------------------------- Ran 0 tests in 0 000s  OK   I seem to have a suite of some sort  and I can execute the result  I am a little concerned about the fact that it says I have only run 1  seems like that should be run 2  but it is progress  But how do I pass and display the result to main  Or how do I basically get it working so I can just run this file  and in doing so  run all the unit tests in this directory

User · Answer

I have used the discover method and an overloading of load tests to achieve this result in a  minimal  I think  number lines of code   def load tests loader  tests  pattern       Discover and load all unit tests in all files named     test py   in     src            suite   TestSuite       for all test suite in unittest defaultTestLoader discover  src   pattern    tests py            for test suite in all test suite              suite addTests test suite      return suite  if   name         main         unittest main     Execution on fives something like  Ran 27 tests in 0 187s OK

User · Answer

This BASH script will execute the python unittest test directory from ANYWHERE in the file system  no matter what working directory you are in  its working directory always be where that test directory is located   ALL TESTS  independent  PWD  unittest Python module is sensitive to your current directory  unless you tell it where  using discover -s option    This is useful when staying in the   src or   example working directory and you need a quick overall unit test        bin bash this program   0  dirname   dirname  this program   readlink   readlink -e  dirname    python -m unittest discover -s   readlink  test -v   SELECTED TESTS  independent  PWD  I name this utility file  runone py and use it like this   runone py  lt test-python-filename-minus-dot-py-fileextension gt         bin bash this program   0  dirname   dirname  this program   readlink   readlink -e  dirname     cd   dirname  test  python -m unittest  1    No need for a test   init   py file to burden your package memory-overhead during production

User · Answer

With Python 2 7 and higher you don t have to write new code or use third-party tools to do this  recursive test execution via the command line is built-in   Put an   init   py in your test directory and  python -m unittest discover  lt test directory gt    or python -m unittest discover -s  lt directory gt  -p    test py   You can read more in the python 2 7 or python 3 x unittest documentation   Update for 2021  Lots of modern python projects use more advanced tools like nosetests and pytest  For example pull down matplotlib or scikit-learn and you will see they both use pytest  It is important to know about these newer tools because when you have more than 7000 tests you need   more advanced ways to summarize what passes  skipped  warnings  errors easy ways to see how they failed percent complete as it is running total run time ways to generate a test report etc etc

User · Answer

In python 3  if you re using unittest TestCase    You must have an empty  or otherwise    init   py file in your test directory  must be named test   Your test files inside test  match the pattern test   py  They can be inside a subdirectory under test   and those subdirs can be named as anything    Then  you can run all the tests with   python -m unittest   Done  A solution less than 100 lines  Hopefully another python beginner saves time by finding this

User · Answer

Well by studying the code above a bit  specifically using TextTestRunner and defaultTestLoader   I was able to get pretty close  Eventually I fixed my code by also just passing all test suites to a single suites constructor  rather than adding them  manually   which fixed my other problems  So here is my solution   import glob import unittest  test files   glob glob  test   py   module strings    test file 0 len test file -3  for test file in test files  suites    unittest defaultTestLoader loadTestsFromName test file  for test file in module strings  test suite   unittest TestSuite suites  test runner   unittest TextTestRunner   run test suite    Yeah  it is probably easier to just use nose than to do this  but that is besides the point

User · Answer

You could use a test runner that would do this for you   nose is very good for example   When run  it will find tests in the current tree and run them   Updated   Here s some code from my pre-nose days   You probably don t want the explicit list of module names  but maybe the rest will be useful to you   testmodules          cogapp test makefiles        cogapp test whiteutils        cogapp test cogapp          suite   unittest TestSuite    for t in testmodules      try            If the module defines a suite   function  call it to get the suite          mod     import   t  globals    locals      suite            suitefn   getattr mod   suite           suite addTest suitefn        except  ImportError  AttributeError             else  just load all the test cases from the module          suite addTest unittest defaultTestLoader loadTestsFromName t    unittest TextTestRunner   run suite

User · Answer

This is an old question  but what worked for me now  in 2019  is   python -m unittest   test py   All my test files are in the same folder as the source files and they end with  test

User · Answer

In case of a packaged library or application  you don t want to do it  setuptools will do it for you       To use this command  your project   s tests must be wrapped in a unittest test suite by either a function  a TestCase class or method  or a module or package containing TestCase classes  If the named suite is a module  and the module has an additional tests   function  it is called and the result  which must be a unittest TestSuite  is added to the tests to be run  If the named suite is a package  any submodules and subpackages are recursively added to the overall test suite    Just tell it where your root test package is  like   setup                test suite    somepkg test      And run python setup py test   File-based discovery may be problematic in Python 3  unless you avoid relative imports in your test suite  because discover uses file import  Even though it supports optional top level dir  but I had some infinite recursion errors  So a simple solution for a non-packaged code is to put the following in   init   py of your test package  see load tests Protocol    import unittest  from   import foo  bar   def load tests loader  tests  pattern       suite   unittest TestSuite       suite addTests loader loadTestsFromModule foo       suite addTests loader loadTestsFromModule bar        return suite

User · Answer

Because Test discovery seems to be a complete subject  there is some dedicated framework to test discovery      nose Py Test   More reading here   https   wiki python org moin PythonTestingToolsTaxonomy

User · Answer

Here is my approach by creating a wrapper to run tests from the command line      usr bin env python3 import os  sys  unittest  argparse  inspect  logging  if   name         main           Parse arguments      parser   argparse ArgumentParser add help False      parser add argument  -     --help       action  help                          help  show this help message and exit        parser add argument  -v    --verbose    action  store true   dest  verbose    help  increase output verbosity        parser add argument  -d    --debug      action  store true   dest  debug      help  show debug messages        parser add argument  -h    --host       action  store        dest  host       help  Destination host        parser add argument  -b    --browser    action  store        dest  browser    help  Browser driver    choices   Firefox    Chrome    IE    Opera    PhantomJS         parser add argument  -r    --reports-dir   action  store     dest  dir        help  Directory to save screenshots    default  reports       parser add argument  files   nargs          args   parser parse args          Load files from the arguments      for filename in args files          exec open filename  read           See  http   codereview stackexchange com q 88655 15346     def make suite tc class           testloader   unittest TestLoader           testnames   testloader getTestCaseNames tc class          suite   unittest TestSuite           for name in testnames              suite addTest tc class name  cargs args           return suite        Add all tests      alltests   unittest TestSuite       for name  obj in inspect getmembers sys modules   name              if inspect isclass obj  and name startswith  FooTest                alltests addTest make suite obj          Set-up logger     verbose   bool os environ get  VERBOSE   args verbose       debug     bool os environ get  DEBUG   args debug       if verbose or debug          logging basicConfig  stream sys stdout           root   logging getLogger           root setLevel logging INFO if verbose else logging DEBUG          ch   logging StreamHandler sys stdout          ch setLevel logging INFO if verbose else logging DEBUG          ch setFormatter logging Formatter    asctime s   levelname s    name s    message s            root addHandler ch      else          logging basicConfig stream sys stderr         Run tests      result   unittest TextTestRunner verbosity 2  run alltests      sys exit not result wasSuccessful      For sake of simplicity  please excuse my non-PEP8 coding standards   Then you can create BaseTest class for common components for all your tests  so each of your test would simply look like   from BaseTest import BaseTest class FooTestPagesBasic BaseTest       def test foo self           driver   self driver         driver get self base url          To run  you simply specifying tests as part of the command line arguments  e g      run tests py -h http   example com  tests      py

User · Answer

I use PyDev LiClipse and haven t really figured out how to run all tests at once from the GUI   edit  you right click the root test folder and choose Run as - gt  Python unit-test  This is my current workaround   import unittest  def load tests loader  tests  pattern       return loader discover       if   name         main         unittest main     I put this code in a module called all in my test directory  If I run this module as a unittest from LiClipse then all tests are run  If I ask to only repeat specific or failed tests then only those tests are run  It doesn t interfere with my commandline test runner either  nosetests  -- it s ignored   You may need to change the arguments to discover based on your project setup

User · Answer

Based on the answer of Stephen Cagle I added support for nested test modules   import fnmatch import os import unittest  def all test modules root dir  pattern       test file names   all files in root dir  pattern      return  path to module str  for str in test file names   def all files in root dir  pattern       matches           for root  dirnames  filenames in os walk root dir           for filename in fnmatch filter filenames  pattern               matches append os path join root  filename        return matches  def path to module py file       return strip leading dots            replace slash by dot                 strip extension py file     def strip extension py file       return py file 0 len py file  - len   py     def replace slash by dot str       return str replace            replace            def strip leading dots str       while str startswith              str   str 1 len str       return str  module names   all test modules        Tests py   suites    unittest defaultTestLoader loadTestsFromName mname  for mname      in module names   testSuite   unittest TestSuite suites  runner   unittest TextTestRunner verbosity 1  runner run testSuite    The code searches all subdirectories of   for  Tests py files which are then loaded  It expects each  Tests py to contain a single class  Tests unittest TestCase  which is loaded in turn and executed one after another   This works with arbitrary deep nesting of directories modules  but each directory in between needs to contain an empty   init   py file at least  This allows the test to load the nested modules by replacing slashes  or backslashes  by dots  see replace slash by dot

User · Answer

This is now possible directly from unittest  unittest TestLoader discover   import unittest loader   unittest TestLoader   start dir    path to your test files  suite   loader discover start dir   runner   unittest TextTestRunner   runner run suite

User · Answer

If you want to run all the tests from various test case classes and you re happy to specify them explicitly then you can do it like this    from unittest import TestLoader  TextTestRunner  TestSuite from uclid test test symbols import TestSymbols from uclid test test patterns import TestPatterns  if   name         main          loader   TestLoader       tests             loader loadTestsFromTestCase test          for test in  TestSymbols  TestPatterns            suite   TestSuite tests       runner   TextTestRunner verbosity 2      runner run suite    where uclid is my project and TestSymbols and TestPatterns are subclasses of TestCase

User · Answer

I just created a discover py file in my base test directory and added import statements for anything in my sub directories  Then discover is able to find all my tests in those directories by running it on discover py python -m unittest discover   test -p    py      test discover py import unittest  from test package1 mod1 import XYZTest from test package1 package2 mod2 import ABCTest      if   name       quot   main   quot      unittest main

User · Answer

I tried various approaches but all seem flawed or I have to makeup some code  that s annoying    But there s a convinient way under linux  that is simply to find every test through certain pattern and then invoke them one by one   find   -name  Test py  -exec python           and most importantly  it definitely works

[python] How do I run all Python unit tests in a directory?

Examples related to python

Examples related to unit-testing

Examples related to testing

Examples related to python-unittest