Twisted is an asynchronous networking framework, but most database API
implementations unfortunately have blocking interfaces -- for this
reason, twisted.enterprise.adbapi
was
created. It is a non-blocking interface to the standardized 'dbapi'
module, which allows you to access a number of different RDBMSes.
Twisted is an asynchronous framework. This means standard database modules cannot be used directly, as they typically work something like:
# Create connection... db = dbmodule.connect('mydb', 'andrew', 'password') # ...which blocks for an unknown amount of time # Create a cursor cursor = db.cursor() # Do a query... resultset = cursor.query('SELECT * FROM table WHERE ...') # ...which could take a long time, perhaps even minutes.
Those delays are unacceptable when using an asynchronous framework
such as Twisted. For this reason, twisted provides twisted.enterprise.adbapi
, an asynchronous wrapper
for any
DB-API 2.0-compliant
module.
enterprise.adbapi
will do blocking database operations
in seperate threads, which trigger callbacks in the originating thread
when they complete. In the meantime, the original thread can continue
doing normal work, like servicing other requests.
Rather than creating a database connection directly, use the adbapi.ConnectionPool
class to manage a connections
for you. This allows enterprise.adbapi
to use multiple
connections, one per thread. This is easy:
# Using the "dbmodule" from the previous example, create a ConnectionPool from twisted.enterprise import adbapi dbpool = adbapi.ConnectionPool("dbmodule", 'mydb', 'andrew', 'password')
Things to note about doing this:
adbapi.ConnectionPool
's
constructor.adbapi.ConnectionPool
's
constructor. Keyword parameters work as well.
So, now you need to be able to dispatch queries to your
ConnectionPool. We do this by subclassing adbapi.Augmentation
. Here's an example:
class AgeDatabase(adbapi.Augmentation): """A simple example that can retrieve an age from the database""" def getAge(self, name): # Define the query sql = """SELECT Age FROM People WHERE name = ?""" # Run the query, and return a Deferred to the caller to add # callbacks to. return self.runOperation(sql, name) def gotAge(resultlist, name): """Callback for handling the result of the query""" age = resultlist[0][0] # First field of first record print "%s is %d years old" % (name, age) db = MyDatabase(dbpool) # These will *not* block. Hooray! db.getAge("Andrew").addCallbacks(gotAge, db.operationError, callbackArgs=name).arm() db.getAge("Glyph").addCallbacks(gotAge, db.operationError, callbackArgs=name).arm()
This is straightforward, except perhaps for the return value of
getAge
. It returns a twisted.python.defer.Deferred
,
which allows arbitrary callbacks to be called upon completion (or upon
failure).
Also worth noting is that I'm assuming that dbmodule uses the "qmarks" paramstyle in my SQL query.
That's all you need to know to use a database from within Twisted. You probably should read the adbapi module's documentation to get an idea of the other functions it has, but hopefully this document presents the core ideas.
This tutorial was written by Andrew Bennetts (andrew at puzzling dot org). I would like to thank the Twisted crew for their fun (and useful!) framework, and also dash in #python for explaining the database stuff to me in the first place. The final example was improved with the aid of suggestion from Glyph.
Suggestions to improve this document are welcomed.