When coding I/O based programs - networking code, databases, file access - there are many APIs that are blocking, and many methods where the common idiom is to block until a result is gotten.
class Getter: def getData(self, x): self.blockUntilResult(x) return result g = Getter() print g.getData(3)
Unfortunately, Twisted can not support blocking calls in most of its code, since it is single threaded, and event based. The solution for this issue is to refactor the code, so that instead of blocking until data is available, we return immediately, and use a callback to notify the requester once the data eventuall arrives. However, this is harder to use, doesn't deal with error raised while waiting for the data, and makes connecting such classes together rather difficult. Nonetheless, looking at how this is implemented will help us understand Deferreds.
class Getter: def getData(self, x, callback): self.callback = callback # this call does not block, it ensure self.gotResult is called # when we have the result self.onResult(x, self.gotResult) def gotResult(self, result): self.callback(result) def gotData(d): print d g = Getter() g.getData(3, gotData) # run the program's event loop here
A twisted.python.defer.Deferred
is a promise that a function will at some point have a result.
We can attach callback functions to a Deferred, and once it gets a result
these callbacks will be called. In addition Deferreds allow the developer
to register a callback for an error, with the default behavior of logging
the error. This is an asynchronous equivalent of the common idiom of blocking
until a result is returned or an exception it raised.
As we said, multiple callbacks can be added to a Deferred. The first callback in the Deferred's callback chain will be called with the result, the second with the result of the first callback, and so on. Why do we need this? Well, consider a Deferred returned by twisted.enterprise.adbapi - the result of a SQL query. A web widget might add a callback that converts this result into HTML, and pass the Deferred onwards, where the callback will be used by twisted to return the result to the HTTP client.
In order for a Deferred to pass its results to a register callback function,
it needs to be armed. Frameworks that support Deferreds -
twisted.web.widgets
,
twisted.web.xmlrpc
,
twisted.spread.pb
server-side objects - will arm the
Deferred automatically for you.
import sys from twisted.python import defer class Getter: def getResult(self, x): self.d = defer.Deferred() self.doNonblockingStuff(x) return self.d def gotResult(self, result): """Called when we get some info from somewhere via the event loop. E.g. this may be called because we got a chunk of data off a socket. """ if self.goodResult(result): # tell the Deferred that we have a result for it self.d.callback(result) else: # tell the Deferred that we have an error self.d.errback("An error has occured.") def printData(d): sys.stdout.write(d) def printError(e): sys.stderr.write(e) g = Getter() d = g.getResult(3) # notice how this is similar to the blocking version d.addCallback(printData) # printData will be called when a result is available d.addErrback(printError) # printError will be called on an error d.arm() # run main event loop here