MiddleKit User's Guide

Version 0.8.1
Webware for Python 0.8.1

Table of Contents

Synopsis
Data Types
      Basic Types
      Date and time types
      Enumerations
      Object references
Model Files
      Classes.csv
      Samples.csv
      Configuration
Generated Python
      Lists: addToBars()
Miscellaneous Topics
      Database Name
      Iterating over attributes
      Deleting objects
      Derived attributes
Model Inheritance
Related Links
Credit
Implementor's Guide
Test Cases

Synopsis

MiddleKit provides an object-relational mapping layer that enables developers to write object-oriented, Python-centric code while enjoying the benefits of a relational database.

The benefits of Python include: generality, OOP (encapsulation, inheritance, polymorphism), a mature language, and many libraries, both standard and 3rd party.

The benefits of a relational database include: data storage, concurrent access and 3rd party tools (such as report generators).

Data Types

All attributes in an object model must be typed. MiddleKit divides types into 4 major categories:

  1. Basic types
  2. Date and time types
  3. Enumerations
  4. Object references

Basic Types

The basic types are easy to work with and do what you expect. They are the same as the Python types with the addition of one special case: bool.

While MiddleKit is designed to be generic and therefore, database agnostic, concrete examples pay good dividends. So the following table includes the type equivalents for MySQL.

Basic Types
MiddleKit Python MySQL Notes
bool int bool Py int = 0 or 1
int int int  
long long bigint 64-bit int
float float double 64-bit float
string string varchar (*)  

(*) The MySQL type is char if the minimum and maximum length are equal.

Date and time types

MiddleKit supports types of date, time and datetime. With a default Python installation and MiddleKit, these are expressed as Python strings such as '2001-02-27'. If you have installed the mxDateTime package, then these values will be expressed as instances of DateTime and DateTimeDelta.

With a default Python installation and MiddleKit, you can pass strings

Date/Time Types
MiddleKit Python MySQL Notes
datetime string/DateTime datetime  
date string/DateTime datetime  
time string/DateTimeDelta time  

Enumerations

Enumerations are provided through the enum type which is directly supported in MySQL. In Python, these enumerations are kept as case sensitive strings. The object model must specify the valid values of the enumeration. While that be done via a column named "Enums", this is more frequently specified in the "Extras" column, where less common attribute specs are usually placed:

Enums='red, green, blue'

Object references

There are two types of object references: a single reference and a list of references. In relational terminology, that would be "1 to 1" and "1 to many" respectively.

The type for a single reference in Python is indicated by simply naming the class of the object that can be referred to. That class must be defined in the object model and will therefore have it's own table in the database. User-defined classes are required to be capitalized (while other types are lower case). For example:

Attribute Type
address Address
billingInfo BillingInfo

The type for a list of references is specified by list of ClassName and represents the 1-to-many relationship. This will be an ordinary Python list, except that some invisible MiddleKit machinery in the background will perform various services (fetch objects on demand, insert new objects, etc.). For example:

Attribute Type
contacts list of Contact
customers list of Customer

There are two additional properties that applies to attributes whose types are object references.

For the purpose of discussion, the object containing the attribute is self while the objects being referred to are the others. Now then, the onDeleteSelf property specifies what happens to the other object(s) when the self object is deleted:

  1. deny - do not allow self to be deleted
  2. cascade - also delete other when self is deleted
  3. detach - allow self to be deleted with no effect on other (this is the default)

There is a similar property onDeleteOther which specifies what happens to the self object when the other object(s) is deleted:

  1. deny - do not allow other to be deleted (this is the default)
  2. cascade - also delete self when other is deleted
  3. detach - allow other to be deleted, and set the reference attribute in self that referred to other to None

The default value of onDeleteSelf is detach, and the default value of onDeleteOther is deny. In other words, by default, you can delete an object which references other objects, but you can't delete an object which is referenced by other objects. An example specification would be onDeleteOther=cascade.

Note: onDeleteSelf can also be specified for "list of reference" attributes, where it has the same effect as it does when applied to reference attributes.

Model Files

Classes.csv

This is the object model where classes and attributes are defined. See the Quick Start for an example.

Samples.csv

This is an optional file containing sample data for the model. See the Quick Start for an example.

Note that a blank field in the samples will be substituted with the default value of the attribute (as specified in the object model, e.g., Classes.csv). To force a None value (NULL in SQL), use 'none' (without the quotes).

Configuration

An MK model can have configuration files inside it that affect things like code generation.

Settings.config is the primary configuration file.

The Package setting can be used to declare the package that your set of middle objects are contained by. This is useful for keeping your middle objects packaged away from other parts of your programs, thereby reducing the chances of a name conflict. This is the recommended way of using MK.

An example Settings.config:

{
    'Package': 'Middle',
}

Your code would then import classes like so:

from Middle.Foo import Foo

Don't forget to put an __init__.py in the directory so that Python recognizes it as a package.

The SQLLog setting can be used to get MiddleKit to echo all SQL statements to 'stdout', 'stderr' or a filename. For filenames, an optional 'Mode' setting inside SQLLog can be used to write over or append to an existing file. The default is write. Here are some examples:

{
    'SQLLog': { 'File': 'stdout' },
}
{
    'SQLLog': { 'File': 'middlekit.sql' },
}
{
    'SQLLog': { 'File': 'middlekit.sql', 'Mode': 'append' },
}

The Database setting overrides the database name, which is otherwise assumed to be same name as the model. This is particularly useful if you are running two instances of the same application on one host.

{
    'Database': 'foobar',
}

The DeleteBehavior setting can be used to change what MiddleKit does when you delete objects. The default behavior is "delete" which means that objects are deleted from the SQL database when they are deleted from the MiddleKit object store. But setting DeleteBehavior to "mark" causes an extra SQL datetime column called "deleted" to be added to each SQL table, and records that are deleted from the object store in MiddleKit are kept in SQL tables with the deleted field set to the date/time when the object was deleted. This setting has no effect on the visible behavior of MiddleKit; it only changes what happens behind the scenes in the SQL store.

{
    'DeleteBehavior': 'mark',
}

The SQLConnectionPoolSize setting is used to create a MiscUtils.DBPool instance for use by the store. For DB API modules with a threadsafety of only 1 (such as MySQLdb), this is particularly useful (in one benchmark, the speed up was 15 - 20%). Simply set the size of the pool in order to have one created and used:

{
	'SQLConnectionPoolSize': 20,
}

The UsePickledClassesCache setting defaults to 1 and causes MiddleKit to cache the Classes.csv text file as a binary pickle file named Classes.pickle.cache. This reduces subsequent load times by about 40%. The cache will be ignored if it can't be read, is older than the CSV file, has a different Python version, etc. You don't normally even need to think about this, but if for some reason you would like to turn off the use of the cache, you can do so through this setting.

SQLGenerator.config has one setting: DropStatements whose potential values are:

An example SQLGenerator.config:

{
    'DropStatements': 'database',  # database, tables
}

Generated Python

Attributes: foo() and setFoo()

For each attribute, foo, MiddleKit stores its value in the attribute _foo, returns it in the accessor method foo() and allows you to set it with setFoo(). You should always use foo() to get the value of an attribute, as there could be some logic there behind the scenes.

Lists: addToBars()

Given an attribute of type list, with the name "bars", MK will generate a Python method named addToBars() that will make it easy for you to add a new object to the list:

newBar = Bar()
newBar.setXY(1, 2)
foo.addToBars(newBar)

This method actually does a lot more for you, ensuring that you're not adding an object of the wrong type, adding the same object twice, etc. Here is a complete list:

You don't have to remember the details since this behavior is both supplied and what you would expect. Just remember to use the various addToBars() methods.

Not setBars() method is provided for list typed attributes.

Miscellaneous Topics

Database Name

MiddleKit uses the name of the store as the name of the database. This works well most of the time. However, if you need to use a different database name, there are two techniques available:

1. You can specify the 'Database' setting in Settings.config. See Configuration for an example.

2. You can pass the database name via the object store's constructor arguments, which are then passed on to the DB API module. This technique overrides both the default model name and the model settings. For example:

store = MySQLObjectStore(db='foobar', user='prog', passwd='asdklfj')

Iterating over attributes

Every once in a while, you might get a hankering for iterating over the attributes of an MK object. You can do so like this:

for attr in obj.klass().allAttrs():
	print attr.name()

The klass() method seen above, returns the object's MiddleKit Klass, which is the class specification that came from the object model you created. The allAttrs() method returns a klass' list of attributes.

The attributes are instances of MiddleKit.Core.Attr (or one of its subclasses such as ObjRefAttr) which inherits from UserDict and acquires additional methods from mix-ins located in MiddleKit.Design and MiddleKit.Run. Since attributes are essentially dictionaries, you can treat them like so, although if you modify them you are asking for serious trouble.

for attr in obj.klass().allAttrs():
	keys = attr.keys()
	keys.sort()
	print '%s: %s' % (attr.name(), keys)

If you had asked the klass for its attrs() instead of allAttrs(), you would have missed out on attributes that were inherited.

If you want to get a dictionary of all the attribute values for a particular object, don't roll your own code. You can already ask your middle objects for allAttrs(), in which case you get values instead of definitions (which is what Klass returns for allAttrs()).

Deleting objects

If you need to delete an object from the object store, you can do so like this:

store.deleteObject(object)

As with other changes, the deletion is is not committed until you perform store.saveChanges().

This may raise one of these two exceptions defined in MiddleKit.Run.ObjectStore:

See Object references for the specifications of onDeleteSelf and onDeleteOther.

Derived attributes

Sometimes it can be convenient to define an attribute in MiddleKit that does not exist in the SQL database back end. Perhaps you want to compute the value from other attributes, or store the value somewhere else outside of the SQL database. Yet you still want to be able to iterate over the attribute using the allAttrs() method provided in MiddleKit.

To do this, simply set the property isDerived on the attribute in the model file. You will have to write your own setter and getter methods for the attribute.

Model Inheritance

Model inheritance is an advanced feature for developers who wish to reuse models in other projects that are also model based. In Settings.config, you can specify other models to inhert class definitions from, which are termed parent models:

{
    'Inherits': ['/usr/lib/mkmodels/News', 'Users'],
}

Note that the .mkmodel extension is assumed. Also, relative filenames are relative to the path of the model inheriting them.

The essential effect is that the classes found in parent models are available to instantiate, subclass and create sample data from, and are termed inherited classes. You can also redefine an inherited class before using it in other class declarations. Classes are identified strictly by name.

The resolution order for finding a class in a model that has parent classes is the same as the basic method resolution order in Python 2.2, although don't take that mean that MiddleKit requires Python 2.2 (it requires 2.0 or greater).

Model inheritance does not affect the files found in the parent model directories. Also, settings and sample data are not inherited from parents; only class definitions.

In MiddleKit.Core.Model, the methods of interest that relate to model inheritance are klass(), which will traverse the parent model hierarchy if necessary, and allKlassesInOrder() and allKlassesByName(). See the doc strings for more info.

Caveats:

Related Links

The topic of object-relational mapping (ORM) is an old one. Here are some related links if you wish to explore the topic further:

Scott Ambler has written some papers on the topic of ORM and also maintains a set of ORM related links:

Apple has a very mature (and perhaps complex) ORM framework named Enterprise Objects Framework, or EOF, available in both Java and Objective-C. All the docs are online at the WebObjects page:

The only other Python ORM that we're aware of is dbObj by Boudewijn Rempt:

Here's a Perl ORM that someone recommended as interesting:

Credit

Authors: Chuck Esterbrook

Implementor's Guide

@@ 2000-10-28 ce: This should be a separate guide.

Test Cases

In the Tests directory of MiddleKit you will find several test case object models.

@@ 2001-02-13 ce: complete this