Trial basics
Trial is Twisted's testing framework. It provides a library
for building test cases (similar to the Python standard library's
unittest
module) and utility functions for working with the Twisted
environment in your tests, and a command-line utility for running your tests.
For instance, to run all the Twisted tests, do:
$ trial -R twisted
Refer to the Trial man page for other command-line options.
Twisted-specific quirks: reactor, Deferreds, callLater
The standard Python unittest
framework, from which Trial is
derived, is ideal for testing code with a fairly linear flow of control.
Twisted is an asynchronous networking framework which provides a clean,
sensible way to establish functions that are run in response to events (like
timers and incoming data), which creates a highly non-linear flow of control.
Trial has a few extensions which help to test this kind of code. This section
provides some hints on how to use these extensions and how to best structure
your tests.
Leave the Reactor as you found it
Trial runs the entire test suite (over one thousand tests) in a single process, with a single reactor. Therefore it is important that your test leave the reactor in the same state as it found it. Leftover timers may expire during somebody else's unsuspecting test. Leftover connection attempts may complete (and fail) during a later test. These lead to intermittent failures that wander from test to test and are very time-consuming to track down.
Your test is responsible for cleaning up after itself. The
tearDown
method is an ideal place for this cleanup code: it is
always run regardless of whether your test passes or fails (like a bare
except
clause in a try-except construct). Exceptions in
tearDown
are flagged as errors and flunk the test.
reactor.stop
is considered very harmful, and should only be used by
reactor-specific test cases which know how to restore the state that it
kills. If you must use reactor.run, use reactor.crash
to stop it instead of
reactor.stop
.
Calls to reactor.callLater
create IDelayedCall
s. These need
to be run or cancelled during a test, otherwise they will outlive the test.
This would be bad, because they could interfere with a later test, causing
confusing failures in unrelated tests! For this reason, Trial checks the
reactor to make sure there are no leftover IDelayedCall
s in the reactor after a
test, and will fail the test if there are.
Similarly, sockets created during a test should be closed by the end of the
test. This applies to both listening ports and client connections. So, calls
to reactor.listenTCP
(and listenUNIX
, and so on)
return IListeningPort
s, and these should be
cleaned up before a test ends by calling their stopListening
method
(note that this can return a Deferred, so you should wait
on it to
make sure it has really stopped before the test ends. Calls to
reactor.connectTCP
return IConnector
s, which should be cleaned
up by calling their disconnect
method. Trial
will warn about unclosed sockets.
deferredResult
If your test creates a Deferred
and simply wants to verify
something about its result, use deferredResult
. It will wait for the
Deferred to fire and give you the result. If the Deferred runs the errback
handler instead, it will raise an exception so your test can fail. Note that
the only thing that will terminate a deferredResult
call is if the Deferred fires; in particular, timers which raise exceptions
will not cause it to return.
Waiting for Things
The preferred way to run a test that waits for something to happen (always
triggered by other things that you have done) is to use a while not self.done
loop that does reactor.iterate(0.1)
at the beginning of each pass. The
0.1
argument sets a limit on how long the reactor will wait to return
if there is nothing to do. 100 milliseconds is long enough to avoid spamming
the CPU while your timers wait to expire.
Using Timers to Detect Failing Tests
It is common for tests to establish some kind of fail-safe timeout that will terminate the test in case something unexpected has happened and none of the normal test-failure paths are followed. This timeout puts an upper bound on the time that a test can consume, and prevents the entire test suite from stalling because of a single test. This is especially important for the Twisted test suite, because it is run automatically by the buildbot whenever changes are committed to the Subversion repository.
Trial tests indicate they have failed by raising a FailTest exception
(self.fail and friends are just wrappers around this raise
statement). Exceptions that are raised inside a
callRemote timer are caught and logged but otherwise ignored. Trial uses a
logging hook to notice when errors have been logged by the test that just
completed (so such errors will flunk the test), but this happens after the
fact: they will not be noticed by the main body of your test code. Therefore
callRemote timers can not be used directly to establish timeouts which
terminate and flunk the test.
The right way to implement this sort of timeout is to have a
self.done
flag, and a while loop which iterates the reactor
until it becomes true. Anything that causes the test to be finished (success
or failure) can set self.done to cause the loop to exit.
Most of the code in Twisted is run by the reactor as a result of socket activity. This is almost always started by Protocol.connectionMade or Protocol.dataReceived (because the output side goes through a buffer which queues data for transmission). Exceptions that are raised by code called in this way (by the reactor, through doRead or doWrite) are caught, logged, handed to connectionLost, and otherwise ignored.
This means that your Protocol's connectionLost method, if invoked because of an exception, must also set this self.done flag. Otherwise the test will not terminate.
Exceptions that are raised in a Deferred callback are turned into a Failure and stashed inside the Deferred. When an errback handler is attached, the Failure is given to it. If the Deferred goes out of scope while an error is still pending, the error is logged just like exceptions that happen in timers or protocol handlers. This will cause the current test to flunk (eventually), but it is not checked until after the test fails. So again, it is a good idea to add errbacks to your Deferreds that will terminate your test's main loop.
Here is a brief example that demonstrates a few of these techniques.
class MyTest(unittest.TestCase): def setUp(self): self.done = False self.failure = None def tearDown(self): self.server.stopListening() # TODO: also shut down client try: self.timeout.cancel() except (error.AlreadyCancelled, error.AlreadyCalled): pass def succeeded(self): self.done = True def failed(self, why): self.done = True self.failure = why def testServer(self): self.server = reactor.listenTCP(port, factory) self.client = reactor.connectTCP(port, factory) # you should give the factories a way to call our 'succeeded' or # 'failed' methods self.timeout = reactor.callLater(5, self.failed, "timeout") while not self.done: reactor.iterate(0.1) # we get here if the test is finished, for good or for bad if self.failure: self.fail(self.failure) # otherwise it probably passed. Cleanup will be done in tearDown()