Comparing objects and sequences#
The helpers here provide ways of making assertions about object equality even when those objects don’t natively support comparison. Where differences are found, feedback is provided in a way that makes it quick and easy to see what the difference was, even in the case of deeply nested data structures.
The compare function#
The compare()
function can be used as a replacement for
assertEqual()
or pytest-style assert statements.
It raises an AssertionError
when its parameters are not equal, which will be
reported as a test failure:
>>> from testfixtures import compare
>>> compare(1, 2)
Traceback (most recent call last):
...
AssertionError: 1 != 2
It allows you to specify a prefix for the message to be used in the event of failure:
>>> compare(1, 2, prefix='wrong number of orders')
Traceback (most recent call last):
...
AssertionError: wrong number of orders: 1 != 2
You can also optionally specify a suffix, which will be appended to the message on a new line:
>>> compare(1, 2, suffix='(Except for very large values of 1)')
Traceback (most recent call last):
...
AssertionError: 1 != 2
(Except for very large values of 1)
The expected and actual value can also be explicitly supplied, making it clearer as to what has gone wrong:
>>> compare(expected=1, actual=2)
Traceback (most recent call last):
...
AssertionError: 1 (expected) != 2 (actual)
The real strengths of this function come when comparing more complex data types. A number of common python data types will give more detailed output when a comparison fails as described below:
sets#
Comparing sets that aren’t the same will attempt to highlight where the differences lie:
>>> compare(set([1, 2]), set([2, 3]))
Traceback (most recent call last):
...
AssertionError: set not as expected:
in first but not second:
[1]
in second but not first:
[3]
dicts#
Comparing dictionaries that aren’t the same will attempt to highlight where the differences lie:
>>> compare(dict(x=1, y=2, a=4), dict(x=1, z=3, a=5))
Traceback (most recent call last):
...
AssertionError: dict not as expected:
same:
['x']
in first but not second:
'y': 2
in second but not first:
'z': 3
values differ:
'a': 4 != 5
lists and tuples#
Comparing lists or tuples that aren’t the same will attempt to highlight where the differences lie:
>>> compare([1, 2, 3], [1, 2, 4])
Traceback (most recent call last):
...
AssertionError: sequence not as expected:
same:
[1, 2]
first:
[3]
second:
[4]
namedtuples#
When two namedtuple()
instances are compared, if
they are of the same type, the description given will highlight which
elements were the same and which were different:
>>> from collections import namedtuple
>>> TestTuple = namedtuple('TestTuple', 'x y z')
>>> compare(TestTuple(1, 2, 3), TestTuple(1, 4, 3))
Traceback (most recent call last):
...
AssertionError: TestTuple not as expected:
same:
['x', 'z']
values differ:
'y': 2 != 4
generators#
When two generators are compared, they are both first unwound into tuples and those tuples are then compared.
The generator helper is useful for creating a generator to represent the expected results:
>>> from testfixtures import generator
>>> def my_gen(t):
... i = 0
... while i<t:
... i += 1
... yield i
>>> compare(generator(1, 2, 3), my_gen(2))
Traceback (most recent call last):
...
AssertionError: sequence not as expected:
same:
(1, 2)
first:
(3,)
second:
()
Warning
If you wish to assert that a function returns a generator, say, for performance reasons, then you should use strict comparison.
strings#
Comparison of strings can be tricky, particularly when those strings contain multiple lines; spotting the differences between the expected and actual values can be hard.
To help with this, long strings give a more helpful representation when comparison fails:
>>> compare("1234567891011", "1234567789")
Traceback (most recent call last):
...
AssertionError:
'1234567891011'
!=
'1234567789'
Likewise, multi-line strings give unified diffs when their comparison fails:
>>> compare("""
... This is line 1
... This is line 2
... This is line 3
... """,
... """
... This is line 1
... This is another line
... This is line 3
... """)
Traceback (most recent call last):
...
AssertionError:
--- first
+++ second
@@ -1,5 +1,5 @@
This is line 1
- This is line 2
+ This is another line
This is line 3
Such comparisons can still be confusing as white space is taken into account. If you need to care about whitespace characters, you can make spotting the differences easier as follows:
>>> compare("\tline 1\r\nline 2"," line1 \nline 2", show_whitespace=True)
Traceback (most recent call last):
...
AssertionError:
--- first
+++ second
@@ -1,2 +1,2 @@
-'\tline 1\r\n'
+' line1 \n'
'line 2'
However, you may not care about some of the whitespace involved. To
help with this, compare()
has two options that can be set to
ignore certain types of whitespace.
If you wish to compare two strings that contain blank lines or lines containing only whitespace characters, but where you only care about the content, you can use the following:
compare('line1\nline2', 'line1\n \nline2\n\n',
blanklines=False)
If you wish to compare two strings made up of lines that may have trailing whitespace that you don’t care about, you can do so with the following:
compare('line1\nline2', 'line1 \t\nline2 \n',
trailing_whitespace=False)
objects#
Even if your objects do not natively support comparison, when they are compared they will be considered identical if they are of the same type and have identical attributes. Take instances of this class as an example:
class MyObject(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return '<MyObject>'
If the attributes and type of instances are the same, they will be considered equal:
>>> compare(MyObject('foo'), MyObject('foo'))
However, if their attributes differ, you will get an informative error:
>>> compare(MyObject('foo'), MyObject('bar'))
Traceback (most recent call last):
...
AssertionError: MyObject not as expected:
attributes differ:
'name': 'foo' != 'bar'
While comparing .name: 'foo' != 'bar'
This type of comparison is also used on objects that make use of __slots__
.
Recursive comparison#
Where compare()
is able to provide a descriptive comparison for
a particular type, it will then recurse to do the same for the
elements contained within objects of that type.
For example, when comparing a list of dictionaries, the description
will not only tell you where in the list the difference occurred, but
also what the differences were within the dictionaries that weren’t
equal:
>>> compare([{'one': 1}, {'two': 2, 'text':'foo\nbar\nbaz'}],
... [{'one': 1}, {'two': 2, 'text':'foo\nbob\nbaz'}])
Traceback (most recent call last):
...
AssertionError: sequence not as expected:
same:
[{'one': 1}]
first:
[{'text': 'foo\nbar\nbaz', 'two': 2}]
second:
[{'text': 'foo\nbob\nbaz', 'two': 2}]
While comparing [1]: dict not as expected:
same:
['two']
values differ:
'text': 'foo\nbar\nbaz' != 'foo\nbob\nbaz'
While comparing [1]['text']:
--- first
+++ second
@@ -1,3 +1,3 @@
foo
-bar
+bob
baz
This also applies to any comparers you have provided, as can be seen in the next section.
Providing your own comparers#
When using compare()
frequently for your own complex objects,
it can be beneficial to give more descriptive output when two objects
don’t compare as equal.
Note
If you are reading this section as a result of needing to test objects that don’t natively support comparison, or as a result of needing to infrequently compare your own subclasses of python basic types, take a look at Comparison objects as this may well be an easier solution.
As an example, suppose you have a class whose instances have a timestamp and a name as attributes, but you’d like to ignore the timestamp when comparing:
from datetime import datetime
class MyObject(object):
def __init__(self, name):
self.timestamp = datetime.now()
self.name = name
To compare lots of these, you would first write a comparer:
def compare_my_object(x, y, context):
if x.name == y.name:
return
return 'MyObject named %s != MyObject named %s' % (
context.label('x', repr(x.name)),
context.label('y', repr(y.name)),
)
Then you’d register that comparer for your type:
from testfixtures.comparison import register
register(MyObject, compare_my_object)
Now, it’ll get used when comparing objects of that type, even if they’re contained within other objects:
>>> compare([1, MyObject('foo')], [1, MyObject('bar')])
Traceback (most recent call last):
...
AssertionError: sequence not as expected:
same:
[1]
first:
[<MyObject ...>]
second:
[<MyObject ...>]
While comparing [1]: MyObject named 'foo' != MyObject named 'bar'
From this example, you can also see that a comparer can indicate that
two objects are equal, for compare()
’s purposes, by returning
None
:
>>> MyObject('foo') == MyObject('foo')
False
>>> compare(MyObject('foo'), MyObject('foo'))
You can also see that you can, and should, use the context object passed in to add labels to the representations of the objects being compared if the comparison fails:
>>> compare(expected=MyObject('foo'), actual=MyObject('bar'))
Traceback (most recent call last):
...
AssertionError: MyObject named 'foo' (expected) != MyObject named 'bar' (actual)
It may be that you only want to use a comparer or set of
comparers for a particular test. If that’s the case, you can pass
compare()
a comparers
parameter consisting of a
dictionary that maps types to comparers. These will be added to the
global registry for the duration of the call:
>>> compare(MyObject('foo'), MyObject('bar'),
... comparers={MyObject: compare_my_object})
Traceback (most recent call last):
...
AssertionError: MyObject named 'foo' != MyObject named 'bar'
A full list of the available comparers included can be found below the
API documentation for compare()
. These make good candidates for
registering for your own classes, if they provide the necessary
behaviour, and their source is also good to read when wondering how to
implement your own comparers.
You may be wondering what the context
object passed to the
comparer is for; it allows you to hand off comparison of parts of the
two objects currently being compared back to the compare()
machinery, it also allows you to pass options to your comparison
function.
For example, you may have an object that has a couple of dictionaries as attributes:
class Request(object):
def __init__(self, uri, headers, body):
self.uri = uri
self.headers = headers
self.body = body
When your tests encounter instances of these that are not as expected, you want feedback about which bits of the request or response weren’t as expected. This can be achieved by implementing a comparer as follows:
def compare_request(x, y, context):
uri_different = x.uri != y.uri
headers_different = context.different(x.headers, y.headers, '.headers')
body_different = context.different(x.body, y.body, '.body')
if uri_different or headers_different or body_different:
return 'Request for %r != Request for %r' % (
x.uri, y.uri
)
Note
A comparer should always return some text when it considers the two objects it is comparing to be different.
This comparer can either be registered globally or passed to each
compare()
call and will give detailed feedback about how the
requests were different:
>>> compare(Request('/foo', {'method': 'POST'}, {'my_field': 'value_1'}),
... Request('/foo', {'method': 'GET'}, {'my_field': 'value_2'}),
... comparers={Request: compare_request})
Traceback (most recent call last):
...
AssertionError: Request for '/foo' != Request for '/foo'
While comparing .headers: dict not as expected:
values differ:
'method': 'POST' != 'GET'
While comparing .headers['method']: 'POST' != 'GET'
While comparing .body: dict not as expected:
values differ:
'my_field': 'value_1' != 'value_2'
While comparing .body['my_field']: 'value_1' != 'value_2'
As an example of passing options through to a comparer, suppose you wanted to compare all decimals in a nested data structure by rounding them to a number of decimal places that varies from test to test. The comparer could be implemented and registered as follows:
from decimal import Decimal
from testfixtures.comparison import register
def compare_decimal(x, y, context):
precision = context.get_option('precision', 2)
if round(x, precision) != round(y, precision):
return '%r != %r when rounded to %i places' % (
x, y, precision
)
register(Decimal, compare_decimal)
Now, this comparer will be used for comparing all decimals and the
precision used will be that passed to compare()
:
>>> expected_order = {'price': Decimal('1.234'), 'quantity': 5}
>>> actual_order = {'price': Decimal('1.236'), 'quantity': 5}
>>> compare(expected_order, actual_order, precision=1)
>>> compare(expected_order, actual_order, precision=3)
Traceback (most recent call last):
...
AssertionError: dict not as expected:
same:
['quantity']
values differ:
'price': Decimal('1.234') != Decimal('1.236')
While comparing ['price']: Decimal('1.234') != Decimal('1.236') when rounded to 3 places
If no precision is passed, the default of 2
will be used:
>>> compare(Decimal('2.006'), Decimal('2.009'))
>>> compare(Decimal('2.001'), Decimal('2.009'))
Traceback (most recent call last):
...
AssertionError: Decimal('2.001') != Decimal('2.009') when rounded to 2 places
Ignoring __eq__
#
Some objects, such as those from the Django ORM, have pretty broken
implementations of __eq__
. Since compare()
normally relies on this,
it can result in objects appearing to be equal when they are not.
Take this class, for example:
class OrmObj(object):
def __init__(self, a):
self.a = a
def __eq__(self, other):
return True
def __repr__(self):
return 'OrmObj: '+str(self.a)
If we compare normally, we erroneously understand the objects to be equal:
>>> compare(actual=OrmObj(1), expected=OrmObj(2))
In order to get a sane comparison, we need to both supply a custom comparer
as described above, and use the ignore_eq
parameter:
def compare_orm_obj(x, y, context):
if x.a != y.a:
return 'OrmObj: %s != %s' % (x.a, y.a)
>>> compare(actual=OrmObj(1), expected=OrmObj(2),
... comparers={OrmObj: compare_orm_obj}, ignore_eq=True)
Traceback (most recent call last):
...
AssertionError: OrmObj: 2 != 1
Strict comparison#
If is it important that the two values being compared are of exactly
the same type, rather than just being equal as far as Python is
concerned, then the strict mode of compare()
should be used.
For example, these two instances will normally appear to be equal provided the elements within them are the same:
>>> TypeA = namedtuple('A', 'x')
>>> TypeB = namedtuple('B', 'x')
>>> compare(TypeA(1), TypeB(1))
If this type difference is important, then the strict parameter should be used:
>>> compare(TypeA(1), TypeB(1), strict=True)
Traceback (most recent call last):
...
AssertionError: A(x=1) (<class '__main__.A'>) != B(x=1) (<class '__main__.B'>)
Comparison objects#
Another common problem with the checking in tests is that you may only want to make
assertions about the type of an object that is nested in a data structure, or even just compare
a subset of an object’s attributes. Testfixtures provides the Comparison
class to help in situations like these.
Comparisons will appear to be equal to any object they are compared with that matches their specification. For example, take the following class:
class SomeClass:
def __init__(self, x, y):
self.x, self.y = x, y
When a comparison fails, the Comparison
will not equal the object it
was compared with and its representation changes to give information about what went wrong:
>>> from testfixtures import Comparison as C
>>> c = C(SomeClass, x=2)
>>> print(repr(c))
<C:...SomeClass>x: 2</>
>>> c == SomeClass(1, 2)
False
>>> print(repr(c))
<C:...SomeClass(failed)>
attributes in actual but not Comparison:
'y': 2
attributes differ:
'x': 2 (Comparison) != 1 (actual)
</C:...SomeClass>
Note
Some test frameworks and helpers, including assertEqual()
,
truncate the text shown in assertions. Use compare()
instead, which will
give you other desirable behaviour as well as showing you the full
output of failed comparisons.
Types of comparison#
There are several ways a comparison can be set up depending on what you want to check.
If you only care about the type of an object, you can set up the comparison with only the class:
>>> C(SomeClass) == SomeClass(1, 2)
True
This can also be achieved by specifying the type of the object as a dotted name:
>>> import sys
>>> C('types.ModuleType') == sys
True
Alternatively, if you happen to have an object already around, comparison can be done with it:
>>> C(SomeClass(1, 2)) == SomeClass(1, 2)
True
If you only care about certain attributes, this can also easily be achieved by doing a partial comparison:
>>> C(SomeClass, x=1, partial=True) == SomeClass(1, 2)
True
The above can be problematic if you want to compare an object with
attributes that share names with parameters to the Comparison
constructor. For this reason, you can pass the attributes in a
dictionary:
>>> compare(C(SomeClass, {'partial': 3}, partial=True), SomeClass(1, 2))
Traceback (most recent call last):
...
AssertionError:
<C:...SomeClass(failed)>
attributes in Comparison but not actual:
'partial': 3
</C:...SomeClass> != <...SomeClass...>
Gotchas#
If the object being compared has an
__eq__
method, such as Django model instances, then theComparison
must be the first object in the equality check.The following class is an example of this:
class SomeModel: def __eq__(self,other): if isinstance(other, SomeModel): return True return False
It will not work correctly if used as the second object in the expression:
>>> SomeModel() == C(SomeModel) False
However, if the comparison is correctly placed first, then everything will behave as expected:
>>> C(SomeModel)==SomeModel() True
It probably goes without saying, but comparisons should not be used on both sides of an equality check:
>>> C(SomeClass) == C(SomeClass) False
Mapping Comparison objects#
When comparing mappings such as dict
and OrderedDict
,
you may need to check the order of the keys is as you expect.
MappingComparison
objects can be used for this:
>>> from collections import OrderedDict
>>> from testfixtures import compare, MappingComparison as M
>>> compare(expected=M((('a', 1), ('c', 3), ('d', 2)), ordered=True),
... actual=OrderedDict((('a', 1), ('d', 2), ('c', 3))))
Traceback (most recent call last):
...
AssertionError:...
<MappingComparison(ordered=True, partial=False)(failed)>
wrong key order:
same:
['a']
expected:
['c', 'd']
actual:
['d', 'c']
</MappingComparison(ordered=True, partial=False)> (expected) != OrderedDict([('a', 1), ('d', 2), ('c', 3)]) (actual)
You may also only care about certain keys being present in a mapping. This
can also be achieved with MappingComparison
objects:
>>> compare(expected=M(a=1, d=2, partial=True), actual={'a': 1, 'c': 3})
Traceback (most recent call last):
...
AssertionError:...
<MappingComparison(ordered=False, partial=True)(failed)>
ignored:
['c']
same:
['a']
in expected but not actual:
'd': 2
</MappingComparison(ordered=False, partial=True)> (expected) != {'a': 1, 'c': 3} (actual)
Where there are differences, they may be hard to spot. In this case, you can ask for a more detailed explanation of what wasn’t as expected:
>>> compare(expected=M((('a', [1, 2]), ('d', [1, 3])), ordered=True, recursive=True),
... actual=OrderedDict((('a', [1, 2]), ('d', [1, 4]))))
Traceback (most recent call last):
...
AssertionError:...
<MappingComparison(ordered=True, partial=False)(failed)>
same:
['a']
values differ:
'd': [1, 3] (expected) != [1, 4] (actual)
While comparing ['d']: sequence not as expected:
same:
[1]
expected:
[3]
actual:
[4]
</MappingComparison(ordered=True, partial=False)> (expected) != OrderedDict([('a', [1, 2]), ('d', [1, 4])]) (actual)
Round Comparison objects#
When comparing numerics you often want to be able to compare to a given precision to allow for rounding issues which make precise equality impossible.
For these situations, you can use RoundComparison
objects
wherever you would use floats or Decimals, and they will compare equal to
any float or Decimal that matches when both sides are rounded to the
specified precision.
Here’s an example:
from testfixtures import compare, RoundComparison as R
compare(expected=R(1234.5678, 2), actual=1234.5681)
Note
You should always pass the same type of object to the
RoundComparison
object as you intend to compare it with. If
the type of the rounded expected value is not the same as the type of
the rounded value it is being compared to, a TypeError
will be raised.
Range Comparison objects#
When comparing numbers, dates, times and any other type that can be ordered, you may only
want to assert what range a value will fall into. RangeComparison
objects
let you confirm a value is within a certain tolerance or range.
Here’s an example with numbers:
from testfixtures import compare, RangeComparison as R
compare(expected=R(123.456, 789), actual=Decimal(555.01))
Here’s an example with dates:
from datetime import date
from testfixtures import compare, RangeComparison as R
compare(expected=R(date(1978, 6, 13), date(1978, 10, 31)), actual=date(1978, 7, 1))
Note
RangeComparison
is inclusive of both the lower and upper bound.
Sequence Comparison objects#
When comparing sequences, you may not care about the order of items in the
sequence. While this type of comparison can often be achieved by pouring
the sequence into a set
, this may not be possible if the items in the
sequence are unhashable, or part of a nested data structure.
SequenceComparison
objects can be used in this case:
>>> from testfixtures import compare, SequenceComparison as S
>>> compare(expected={'k': S({1}, {2}, ordered=False)}, actual={'k': [{2}, {1}]})
You may also only care about certain items being present in a sequence, but where
it is important that those items are in the order you expected. This
can also be achieved with SequenceComparison
objects:
>>> compare(expected=S(1, 3, 5, partial=True), actual=[1, 2, 3, 4, 6])
Traceback (most recent call last):
...
AssertionError:...
<SequenceComparison(ordered=True, partial=True)(failed)>
ignored:
[2, 4, 6]
same:
[1, 3]
expected:
[5]
actual:
[]
</SequenceComparison(ordered=True, partial=True)> (expected) != [1, 2, 3, 4, 6] (actual)
Where there are differences, they may be hard to spot. In this case, you can ask for a more detailed explanation of what wasn’t as expected:
>>> compare(expected=S({1: 'a'}, {2: 'c'}, recursive=True), actual=[{1: 'a'}, {2: 'd'}])
Traceback (most recent call last):
...
AssertionError:...
<SequenceComparison(ordered=True, partial=False)(failed)>
same:
[{1: 'a'}]
expected:
[{2: 'c'}]
actual:
[{2: 'd'}]
While comparing [1]: dict not as expected:
values differ:
2: 'c' (expected) != 'd' (actual)
While comparing [1][2]: 'c' (expected) != 'd' (actual)
</SequenceComparison(ordered=True, partial=False)> (expected) != [{1: 'a'}, {2: 'd'}] (actual)
There are also the Subset
and Permutation
shortcuts:
>>> from testfixtures import Subset, Permutation
>>> assert Subset({1}, {2}) == [{1}, {2}, {3}]
>>> assert Permutation({1}, {2}) == [{2}, {1}]
String Comparison objects#
When comparing sequences of strings, particularly those coming from things like the python logging package, you often end up wanting to express a requirement that one string should be almost like another, or maybe fit a particular pattern expressed as a regular expression.
For these situations, you can use StringComparison
objects
wherever you would use normal strings, and they will compare equal to
any string that matches the regular expression they are created with.
Here’s an example:
from testfixtures import compare, StringComparison as S
compare(expected=S(r'Starting thread \d+'), actual='Starting thread 132356')
If you need to specify flags, this can be done in one of three ways:
As parameters:
compare(expected=S(".*BaR", dotall=True, ignorecase=True), actual="foo\nbar")
As you would to
re.compile()
:import re compare(expected=S(".*BaR", re.DOTALL|re.IGNORECASE), actual="foo\nbar")
Inline:
compare(expected=S("(?s:.*bar)"), actual="foo\nbar")
Differentiating chunks of text#
Testfixtures provides a function that will compare two strings and
give a unified diff as a result. This can be handy as a third
parameter to assertEqual()
or just as a
general utility function for comparing two lumps of text.
As an example:
>>> from testfixtures import diff
>>> print(diff('line1\nline2\nline3',
... 'line1\nlineA\nline3'))
--- first
+++ second
@@ -1,3 +1,3 @@
line1
-line2
+lineA
line3