Testing exceptions
==================

.. currentmodule:: testfixtures

.. invisible-code-block: python

    try:
        import pydantic
    except ImportError:
        pydantic = None

Testfixtures has tools to help when making assertions about exceptions that should be raised by
a piece of code.

The :class:`ShouldRaise` context manager
----------------------------------------

The :class:`ShouldRaise` context manager is the recommended way to make assertions about
a piece of code that should raise exceptions.

Suppose we wanted to test the following function to make sure that the
right exception was raised:

.. code-block:: python

  def the_thrower(throw=True):
      if throw:
          raise ValueError('Not good!')

The following example shows how to test that the correct exception is
raised:

>>> from testfixtures import ShouldRaise
>>> with ShouldRaise(ValueError('Not good!')):
...     the_thrower()

If the exception raised doesn't match the one expected,
:class:`ShouldRaise` will raise an :class:`AssertionError`
causing the tests in which it occurs to fail:

>>> with ShouldRaise(ValueError('Is good!')):
...     the_thrower()
Traceback (most recent call last):
...
AssertionError: not equal:
ValueError('Is good!') (expected)
ValueError('Not good!') (raised)

If you're not concerned about anything more than the type of the
exception that's raised, you can check as follows:

>>> from testfixtures import ShouldRaise
>>> with ShouldRaise(ValueError):
...     the_thrower()

If you want to match :func:`pytest.raises` behaviour and just check that ``str(exception)`` matches
a pattern you supply, see :ref:`shouldraise-match` below.

If you're feeling slack and just want to check that an exception is
raised, but don't care about the type of that exception, the following
will suffice:

>>> from testfixtures import ShouldRaise
>>> with ShouldRaise():
...     the_thrower()

If no exception is raised by the code under test, :class:`ShouldRaise`
will raise an :class:`AssertionError` to indicate this:

>>> from testfixtures import ShouldRaise
>>> with ShouldRaise():
...     the_thrower(throw=False)
Traceback (most recent call last):
...
testfixtures.shouldraise.NoException: No exception raised!

:class:`ShouldRaise` has been implemented such that it can be
used to test code that raises all exceptions, including :class:`SystemExit` and
:class:`KeyboardInterrupt` exceptions.

To help with :class:`SystemExit` and other exceptions that are
tricky to construct yourself, :class:`ShouldRaise` instances have a
:attr:`~ShouldRaise.raised` attribute. This will contain the actual
exception raised and can be used to inspect parts of it:

>>> import sys
>>> from testfixtures import ShouldRaise
>>> with ShouldRaise() as s:
...     sys.exit(42)
>>> s.raised.code
42

The :func:`should_raise` decorator
-----------------------------------------

If you are working in a traditional :mod:`unittest` environment and
want to check that a particular test function raises an exception, you
may find the decorator suits your needs better:
 
.. code-block:: python

  from testfixtures import should_raise
  
  @should_raise(ValueError('Not good!'))
  def test_function():
      the_thrower()

This decorator behaves exactly as the :class:`ShouldRaise` context
manager described in the documentation above.

.. note:: 

  It is slightly recommended that you use the context manager rather
  than the decorator in most cases. With the decorator, all exceptions
  raised within the decorated function will be checked, which can
  hinder test development. With the context manager, you can make
  assertions about only the exact lines of code that you expect to
  raise the exception.

Parameterised testing with optional exceptions
----------------------------------------------

When writing parameterised tests that may or may not expect an exception,
you can pass ``None`` to :class:`ShouldRaise` to indicate that no exception
should be raised. This avoids the need for conditional context manager selection:

.. code-block:: python

  import pytest
  from testfixtures import ShouldRaise

  @pytest.mark.parametrize('value,exception', [
      (1, None),
      (-1, ValueError('negative value')),
  ])
  def test_something(value, exception):
      with ShouldRaise(exception):
          if value < 0:
              raise ValueError('negative value')

.. _shouldraise-match:

Matching exception messages
---------------------------

If you want just match a pattern in the ``str(exception)`` of a raised exception you can do this
as follows:

>>> with ShouldRaise(ValueError, match='Not good'):
...     the_thrower()

The ``match`` parameter can also be a :class:`re.Pattern`:

>>> import re
>>> with ShouldRaise(ValueError, match=re.compile('not good', re.IGNORECASE)):
...     the_thrower()

If the pattern doesn't match, an :class:`AssertionError` is raised:

>>> with ShouldRaise(ValueError, match='^All good$'):
...     the_thrower()
Traceback (most recent call last):
...
AssertionError: Pattern did not match:
'^All good$' (expected)
'Not good!' (actual)

``match`` can also be used without specifying an exception type, when you only care that
something was raised and its message matches:

>>> with ShouldRaise(match='Not good'):
...     the_thrower()

``match`` is not permitted when passing an exception instance or ``None``, indicating exception is
expected; both will raise a :class:`TypeError` at construction time.

Matching the type and ``repr()`` or ``str()``
---------------------------------------------

Sometimes you want to assert both the type of the exception and its
:func:`repr` or :class:`str` without constructing the exception instance
yourself. The :func:`repr_like` and :func:`str_like` matchers do this and are
typed to stand in for the exception.

This is useful for exceptions that carry structured parameters rather than a
plain message, where it isn't obvious what ends up in the rendered text:

.. code-block:: python

  class InvalidRecord(Exception):
      def __init__(self, id, reason):
          self.id = id
          self.reason = reason
          super().__init__(f'record {id} invalid: {reason}')

  class SpecialInvalidRecord(InvalidRecord):
      pass

>>> from testfixtures import repr_like
>>> with ShouldRaise(
...     repr_like(InvalidRecord, "InvalidRecord('record 42 invalid: missing name')")
... ):
...     raise InvalidRecord(42, 'missing name')

If a subclass is raised instead, :class:`ShouldRaise` lets it propagate,
since the type must match exactly:

>>> with ShouldRaise(
...     repr_like(InvalidRecord, "InvalidRecord('record 42 invalid: missing name')")
... ):
...     raise SpecialInvalidRecord(42, 'missing name')
Traceback (most recent call last):
...
SpecialInvalidRecord: record 42 invalid: missing name

If the type matches but the rendering does not, an :class:`AssertionError`
explains the difference:

>>> with ShouldRaise(
...     repr_like(InvalidRecord, "InvalidRecord('record 42 invalid: missing surname')")
... ):
...     raise InvalidRecord(42, 'missing name')
Traceback (most recent call last):
...
AssertionError: not equal:
<ReprComparison: builtins.InvalidRecord: InvalidRecord('record 42 invalid: missing surname')> (expected)
InvalidRecord('record 42 invalid: missing name') (raised)

:func:`str_like` works the same way but checks :class:`str` instead of
:func:`repr`. This is particularly useful for exceptions such as
:class:`pydantic.ValidationError <pydantic:pydantic_core.ValidationError>`, which has no public
constructor that accepts a plain message:

.. skip: start if(pydantic is None, reason="No pydantic installed")

.. code-block:: python

  from pydantic import BaseModel, ValidationError

  class Point(BaseModel):
      x: int
      y: int

``match`` is a regular expression, so literal square brackets in pydantic's
rendering need escaping:

>>> from testfixtures import str_like
>>> with ShouldRaise(
...     str_like(
...         ValidationError,
...         match=(
...             "Input should be a valid integer, "
...             "unable to parse string as an integer "
...             r"\[type=int_parsing, input_value='not-an-int', input_type=str\]"
...         )
...     )
... ):
...     Point(x='not-an-int', y=2)

.. skip: end

The type must match exactly, so a subclass of the expected exception will not
match. See :ref:`comparison-objects` for more about :func:`repr_like` and
:func:`str_like`.

Exceptions that are conditionally raised
----------------------------------------

Some exceptions are only raised in certain versions of Python. For
example, in Python 2, ``bytes()`` will turn both bytes and strings into
bytes, while in Python 3, it will raise an exception when presented
with a string. If you wish to make assertions that this behaviour is
expected, you can use the ``unless`` option to :class:`ShouldRaise`
as follows:

.. code-block:: python

  import sys
  from testfixtures import ShouldRaise

  PY2 = sys.version_info[:2] <  (3, 0)

  with ShouldRaise(TypeError, unless=PY2):
      bytes('something')

.. note:: 

  Do **not** abuse this functionality to make sloppy assertions. It is
  always better have two different tests that cover a case when an
  exception should be raised and a case where an exception should not
  be raised rather than using it above functionality. It is *only*
  provided to help in cases where something in the environment that
  cannot be mocked out or controlled influences whether or not an
  exception is raised.