Testing with files and directories

Working with files and directories in tests can often require excessive amounts of boilerplate code to make sure that the tests happen in their own sandbox, files and directories contain what they should or code processes test files correctly, and the sandbox is cleared up at the end of the tests.

Methods of use

To help with this, TestFixtures provides the TempDirectory class that hides most of the boilerplate code you would need to write.

Suppose you wanted to test the following function:

import os

def foo2bar(dirpath, filename):
  path = os.path.join(dirpath, filename)
  with open(path, 'rb') as input:
      data = input.read()
  data = data.replace(b'foo', b'bar')
  with open(path, 'wb') as output:
      output.write(data)

There are several different ways depending on the type of test you are writing:

The context manager

If you’re using a version of Python where the with keyword is available, a TempDirectory can be used as a context manager:

>>> from testfixtures import TempDirectory
>>> with TempDirectory() as d:
...   d.write('test.txt', b'some foo thing')
...   foo2bar(d.path, 'test.txt')
...   d.read('test.txt')
'...'
b'some bar thing'

The decorator

If you are working in a traditional unittest environment and only work with files or directories in a particular test function, you may find the decorator suits your needs better:

from testfixtures import tempdir, compare

@tempdir()
def test_function(d):
    d.write('test.txt', b'some foo thing')
    foo2bar(d.path, 'test.txt')
    compare(d.read('test.txt'), b'some bar thing')

Manual usage

If you want to work with files or directories for the duration of a doctest or in every test in a TestCase, then you can use the TempDirectory manually.

The instantiation and replacement are done in the setUp function of the TestCase or passed to the DocTestSuite constructor:

>>> from testfixtures import TempDirectory
>>> d = TempDirectory()

You can then use the temporary directory for your testing:

>>> d.write('test.txt', b'some foo thing')
'...'
>>> foo2bar(d.path, 'test.txt')
>>> d.read('test.txt') == b'some bar thing'
True

Then, in the tearDown function of the TestCase or passed to the DocTestSuite constructor, you should make sure the temporary directory is cleaned up:

>>> d.cleanup()

If you have multiple TempDirectory objects in use, you can easily clean them all up:

>>> TempDirectory.cleanup_all()

Features of a temporary directory

No matter which usage pattern you pick, you will always end up with a TempDirectory object. These have an array of methods that let you perform common file and directory related tasks without all the manual boiler plate. The following sections show you how to perform the various tasks you’re likely to bump into in the course of testing.

Computing paths

If you need to know the real path of the temporary directory, the TempDirectory object has a path attribute:

>>> tempdir.path
'...tmp...'

A common use case is to want to compute a path within the temporary directory to pass to code under test. This can be done with the getpath() method:

>>> tempdir.getpath('foo').rsplit(os.sep,1)[-1]
'foo'

If you want to compute a deeper path, you can either pass either a tuple or a forward slash-separated path:

>>> tempdir.getpath(('foo', 'baz')).rsplit(os.sep, 2)[-2:]
['foo', 'baz']
>>> tempdir.getpath('foo/baz') .rsplit(os.sep, 2)[-2:]
['foo', 'baz']

Note

If passing a string containing path separators, a forward slash should be used as the separator regardless of the underlying platform separator.

Writing files

To write to a file in the root of the temporary directory, you pass the name of the file and the content you want to write:

>>> tempdir.write('myfile.txt', b'some text')
'...'
>>> with open(os.path.join(tempdir.path, 'myfile.txt')) as f:
...     print(f.read())
some text

The full path of the newly written file is returned:

>>> path = tempdir.write('anotherfile.txt', b'some more text')
>>> with open(path) as f:
...     print(f.read())
some more text

You can also write files into a sub-directory of the temporary directory, whether or not that directory exists, as follows:

>>> path = tempdir.write(('some', 'folder', 'afile.txt'), b'the text')
>>> with open(path) as f:
...     print(f.read())
the text

You can also specify the path to write to as a forward-slash separated string:

>>> path = tempdir.write('some/folder/bfile.txt', b'the text')
>>> with open(path) as f:
...     print(f.read())
the text

Note

Forward slashes should be used regardless of the file system or operating system in use.

Creating directories

If you just want to create a sub-directory in the temporary directory you can do so as follows:

>>> tempdir.makedir('output')
'...'
>>> os.path.isdir(os.path.join(tempdir.path, 'output'))
True

As with file creation, the full path of the sub-directory that has just been created is returned:

>>> path = tempdir.makedir('more_output')
>>> os.path.isdir(path)
True

Finally, you can create a nested sub-directory even if the intervening parent directories do not exist:

>>> os.path.exists(os.path.join(tempdir.path, 'some'))
False
>>> path = tempdir.makedir(('some', 'sub', 'dir'))
>>> os.path.exists(path)
True

You can also specify the path to write to as a forward-slash separated string:

>>> os.path.exists(os.path.join(tempdir.path, 'another'))
False
>>> path = tempdir.makedir('another/sub/dir')
>>> os.path.exists(path)
True

Note

Forward slashes should be used regardless of the file system or operating system in use.

Checking the contents of files

Once a file has been written into the temporary directory, you will often want to check its contents. This is done with the TempDirectory.read() method.

Suppose the code you are testing creates some files:

def spew(path):
   with open(os.path.join(path, 'root.txt'), 'wb') as f:
       f.write(b'root output')
   os.mkdir(os.path.join(path, 'subdir'))
   with open(os.path.join(path, 'subdir', 'file.txt'), 'wb') as f:
       f.write(b'subdir output')
   os.mkdir(os.path.join(path, 'subdir', 'logs'))

We can test this function by passing it the temporary directory’s path and then using the TempDirectory.read() method to check the files were created with the correct content:

>>> spew(tempdir.path)
>>> tempdir.read('root.txt')
b'root output'
>>> tempdir.read(('subdir', 'file.txt'))
b'subdir output'

The second part of the above test shows how to use the TempDirectory.read() method to check the contents of files that are in sub-directories of the temporary directory. This can also be done by specifying the path relative to the root of the temporary directory as a forward-slash separated string:

>>> tempdir.read('subdir/file.txt')
b'subdir output'

Note

Forward slashes should be used regardless of the file system or operating system in use.

Checking the contents of directories

It’s good practice to test that your code is only writing files you expect it to and to check they are being written to the path you expect. TempDirectory.compare() is the method to use to do this.

As an example, we could check that the spew() function above created no extraneous files as follows:

>>> tempdir.compare([
...     'root.txt',
...     'subdir/',
...     'subdir/file.txt',
...     'subdir/logs/',
... ])

If we only wanted to check the sub-directory, we would specify the path to start from, relative to the root of the temporary directory:

>>> tempdir.compare([
...     'file.txt',
...     'logs/',
... ], path='subdir')

If, like git, we only cared about files, we could do the comparison as follows:

>>> tempdir.compare([
...     'root.txt',
...     'subdir/file.txt',
... ], files_only=True)

And finally, if we only cared about files at a particular level, we could turn off the recursive comparison as follows:

>>> tempdir.compare([
...     'root.txt',
...     'subdir',
... ], recursive=False)

The compare() method can also be used to check whether a directory contains nothing, for example:

>>> tempdir.compare(path=('subdir', 'logs'), expected=())

The above can also be done by specifying the sub-directory to be checked as a forward-slash separated path:

>>> tempdir.compare(path='subdir/logs', expected=())

If the actual directory contents do not match the expected contents passed in, an AssertionError is raised, which will show up as a unit test failure:

>>> tempdir.compare(['subdir'], recursive=False)
Traceback (most recent call last):
...
AssertionError: sequence not as expected:

same:
()

expected:
('subdir',)

actual:
('root.txt', 'subdir')

In some circumstances, you may want to ignore certain files or sub-directories when checking contents. To make this easy, the TempDirectory constructor takes an optional ignore parameter which, if provided, should contain a sequence of regular expressions. If any of the regular expressions return a match when used to search through the results of any of the the methods covered in this section, that result will be ignored.

For example, suppose we are testing some revision control code, but don’t really care about the revision control system’s metadata directories, which may or may not be present:

from random import choice

def svn_ish(dirpath, filename):
  if choice((True, False)):
    os.mkdir(os.path.join(dirpath, '.svn'))
  with open(os.path.join(dirpath, filename), 'wb') as f:
    f.write(b'something')

To test this, we can use any of the previously described methods.

When used manually or as a context manager, this would be as follows:

>>> with TempDirectory(ignore=['.svn']) as d:
...   svn_ish(d.path, 'test.txt')
...   d.compare(['test.txt'])

The decorator would be as follows:

from testfixtures import tempdir, compare

@tempdir(ignore=['.svn'])
def test_function(d):
    svn_ish(d.path, 'test.txt')
    d.compare(['test.txt'])

If you are working with doctests, the listdir() method can be used instead:

>>> tempdir.listdir()
root.txt
subdir
>>> tempdir.listdir('subdir')
file.txt
logs
>>> tempdir.listdir(('subdir', 'logs'))
No files or directories found.

The above example also shows how to check the contents of sub-directories of the temporary directory and also shows what is printed when a directory contains nothing. The listdir() method can also take a path separated by forward slashes, which can make doctests a little more readable. The above test could be written as follows:

>>> tempdir.listdir('subdir/logs')
No files or directories found.

However, if you have a nested folder structure, such as that created by our spew() function, it can be easier to just inspect the whole tree of files and folders created. You can do this by using the recursive parameter to listdir():

>>> tempdir.listdir(recursive=True)
root.txt
subdir/
subdir/file.txt
subdir/logs/

Bytes versus Strings

You’ll notice that all of the examples so far have used raw bytes as their data and written to and read from files only in binary mode. This keeps all the examples nice and simple and working consistently between Python 2 and Python 3. One of the big changes between Python 2 and Python 3 was that the default string type became unicode instead of binary, and a new type for bytes was introduced. This little snippet shows the difference by defining two constants for the British Pound symbol:

import sys
PY3 = sys.version_info[:2] >= (3, 0)

if PY3:
    some_bytes = '\xa3'.encode('utf-8')
    some_text = '\xa3'
else:
    some_bytes = '\xc2\xa3'
    some_text = '\xc2\xa3'.decode('utf-8')

Python 3 is much stricter than Python 2 about the byte versus string boundary and TempDirectory has been changed to help work with this by only reading and writing files in binary mode and providing parameters to control decoding and encoding when you want to read and write text.

For example, when writing, you can either write bytes directly, as we have been in the examples so far:

>>> path = tempdir.write('currencies.txt', some_bytes)
>>> with open(path, 'rb') as currencies:
...     currencies.read()
b'\xc2\xa3'

Or, you can write text, but must specify an encoding to use when writing the data to the file:

>>> path = tempdir.write('currencies.txt', some_text, 'utf-8')
>>> with open(path, 'rb') as currencies:
...     currencies.read()
b'\xc2\xa3'

The same is true when reading files. You can either read bytes:

>>> tempdir.read('currencies.txt') == some_bytes
True

Or, you can read text, but must specify an encoding that will be used to decode the data in the file:

>>> tempdir.read('currencies.txt', 'utf-8') == some_text
True

Working with an existing sandbox

Some testing infrastructure already provides a sandbox temporary directory, however that infrastructure might not provide the same level of functionality that TempDirectory provides.

For this reason, it is possible to wrap an existing directory such as the following with a TempDirectory:

>>> from tempfile import mkdtemp
>>> thedir = mkdtemp()

When working with the context manager, this is done as follows:

>>> with TempDirectory(path=thedir) as d:
...   d.write('file', b'data')
...   d.makedir('directory')
...   sorted(os.listdir(thedir))
'...'
'...'
['directory', 'file']

For the decorator, usage would be as follows:

from testfixtures import tempdir, compare

@tempdir(path=thedir)
def test_function(d):
    d.write('file', b'data')
    d.makedir('directory')
    assert sorted(os.listdir(thedir))==['directory', 'file']

It is important to note that if an existing directory is used, it will not be deleted by either the decorator or the context manager. You will need to make sure that the directory is cleaned up as required.

Using with Manuel

Manuel is an excellent take on testing the examples found in documentation. It works by applying a set of specialised parsers to the documentation and testing or otherwise using the the blocks returned by those parsers.

The key differences between testing with Manuel and the traditional doctest are that it is possible to plug in different types of parser, not just the “python console session” one, and so it is possible to test different types of examples. TestFixtures provides one these plugins to aid working with TempDirectory objects. This plugin makes use of topic directives with specific classes set to perform different actions.

The following sections describe how to use this plugin to help with writing temporary files and checking their contents.

Setting up

To use the Manuel plugin, you need to make sure a TempDirectory instance is available under a particular name in the test globals. This name is then passed to the plugin’s constructor and the plugin is passed to Manuel’s TestSuite constructor.

The following example shows how to return a test suite that will execute all of the examples below. These require not only the TestFixtures plugin but also the Manuel plugins that give more traditional doctest behaviour, hidden code blocks that are useful for setting things up and checking examples without breaking up the flow of the documentation, and capturing of examples from the documentation to use for use in other forms of testing:

from glob import glob
from manuel import doctest, capture
from manuel.testing import TestSuite
from os.path import join
from testfixtures import TempDirectory
from testfixtures.manuel import Files

from . import compat


def setUp(test):
    test.globs['tempdir'] = TempDirectory()


def tearDown(test):
    test.globs['tempdir'].cleanup()


def test_suite():
    m = doctest.Manuel()
    m += compat.Manuel()
    m += capture.Manuel()
    m += Files('tempdir')
    return TestSuite(
        m,
        setUp=setUp,
        tearDown=tearDown,
        *glob(join(path_to_your_docs, '*.txt'))
        )

Writing files

To write a file with the plugin, a topic with a class of write-file is included in the documentation. The following example is a complete reStructuredText file that shows how to write a file that is then used by a later example:

Here's an example configuration file:

.. topic:: example.cfg
 :class: write-file

 ::

   [A Section]
   dir=frob
   long: this value continues
     on the next line

.. invisible-code-block: python
   
  from testfixtures.compat import PY3
  # change to the temp directory
  import os
  original_dir = os.getcwd()
  os.chdir(tempdir.path)

To parse this file using the :mod:`ConfigParser` module, you would
do the following:

.. code-block:: python

   if PY3:
       from configparser import ConfigParser
   else:
       from ConfigParser import ConfigParser
   config = ConfigParser()
   config.read('example.cfg')

The items in the section are now available as follows:

>>> for name, value in sorted(config.items('A Section')):
...     print('{0!r}:{1!r}'.format(name, value))
'dir':'frob'
'long':'this value continues\non the next line'

.. invisible-code-block: python
   
  # change out again
  import os
  os.chdir(original_dir)

Checking the contents of files

To read a file with the plugin, a topic with a class of read-file is included in the documentation. The following example is a complete reStructuredText file that shows how to check the values written by the code being documented while also using this check as part of the documentation:

.. invisible-code-block: python
   
  from testfixtures.compat import PY3
  # change to the temp directory
  import os
  original_dir = os.getcwd()
  os.chdir(tempdir.path)

To construct a configuration file using the :mod:`ConfigParser`
module, you would do the following: 

.. code-block:: python

   if PY3:
       from configparser import ConfigParser
   else:
       from ConfigParser import ConfigParser
   config = ConfigParser()
   config.add_section('A Section')
   config.set('A Section', 'dir', 'frob')
   f = open('example.cfg','w')
   config.write(f)
   f.close()

The generated configuration file will be as follows:

.. topic:: example.cfg
 :class: read-file

 ::

   [A Section]
   dir = frob

   
.. config parser writes whitespace at the end, be careful when testing!

.. invisible-code-block: python
   
  # change out again
  import os
  os.chdir(original_dir)

Checking the contents of directories

While the TestFixtures plugin itself does not offer any facility for checking the contents of directories, Manuel’s capture plugin can be used in conjunction with the existing features of a TempDirectory to illustrate the contents expected in a directory seamlessly within the documentation.

Here’s a complete reStructuredText document that illustrates this technique:

Here's an example piece of code that creates some files and
directories: 

.. code-block:: python

  import os

  def spew(path):
    with open(os.path.join(path, 'root.txt'), 'wb') as f:
        f.write(b'root output')
    os.mkdir(os.path.join(path, 'subdir'))
    with open(os.path.join(path, 'subdir', 'file.txt'), 'wb') as f:
        f.write(b'subdir output')
    os.mkdir(os.path.join(path, 'subdir', 'logs'))

This function is used as follows:

>>> spew(tempdir.path)

This will create the following files and directories::

  root.txt
  subdir/
  subdir/file.txt
  subdir/logs/
 
.. -> expected_listing

.. invisible-code-block: python

  # check the listing was as expected
  tempdir.compare(expected_listing.strip().split('\n'))

A note on encoding and line endings

As currently implemented, the plugin provided by TestFixtures only works with textual file content that can be encoded using the ASCII character set. This content will always be written with '\n' line seperators and, when read, will always have its line endings normalised to '\n'. If you hit any limitations caused by this, please raise an issue in the tracker on GitHub.