BRL-CAD Database Format

Version 5 (DRAFT)


Table of Contents

Background and Terminology
Format of Data Elements/Database External Format
Definition of a Single, Generic Database Object
Object Structure
Flags
Object Type
Object Length
Object Name
Object Attributes
Object Body
Padding and Length Rounding
How Objects Are Grouped into a Database
Details of BRL-CAD-Specific Nongeometric Database Object Types
Details of BRL-CAD-Specific Geometric Database Object Types
Extensions for Deferred Implementation
Community Feedback on the Proposal
Database Library Application Programming Interface (API)

Abstract

BRL-CAD software uses its own binary file format to store the geometric information and other properties required to define CAD models. BRL-CAD used its version 4 (v4) binary ".g" database format for over 10 years, but a variety of long standing issues with that format prompted the development of version 5. Issues addressed by the new format include a machine-independent format, expanding the upper limit of numbers to double precision floating point, and lifting a 16 character size limit on object names.

BRL-CAD is a constructive solid geometry (CSG) modeling system. Primitive solid shapes are combined using boolean operations to form regions of homogeneous material.

The database is organized as a directed acyclic graph (DAG), which comprises

In a slight abuse of terminology, the DAG is often spoken of as a tree or collection of trees. In this context, the solids are also called leaves.

The database access library stores objects as a collection of data with a globally unique name and places no interpretation on the content of those data. The object is the smallest granularity of an item in the database; objects must be read from and written to the database in a single atomic operation.

In the case of librt, each database object will contain exactly one combination node or leaf (solid) node.

All objects share certain common properties, which are stored in a standardized object wrapper consisting of an Object Header and an Object Footer.

The Object Header consists of:

The Object Footer consists of:

Objects may store application-specific information in an Object Interior.

The on-disk version of each object consists of three distinct parts: Object Header, Object Interior, and Object Footer. This is called the external format of the object.


????Need a description that says that an object can now have (1) EITHER an attribute OR a body, (2) BOTH an attribute and a body, or (3) NEITHER an attribute nor a body.

The routines db_get_external() and db_put_external() are used to move objects in external format between memory and the database disk file. The routines db_wrap_external() and db_unwrap_external() are used to wrap and unwrap the (??? Object_Body or Object_Interior ???) (already in external form) in a standardized database object's wrapper.

The Flags element consists of three 8-bit fields: HFlags, AFlags, and BFlags. The HFlags field is 1 byte containing flag bits that pertain to the noncompressible basic header and the database object as a whole. The AFlags and BFlags fields are each single bytes containing flag bits that pertain to the (potentially compressed) attributes and body, respectively, in the object interior.


The length of an object or subelement in the database is recorded using an unsigned integer. These are variable-width fields based on the magnitude of the maximum number needed. The Wid bits specify the size of the unsigned integer employed in each instance. There are four 2-bit width (Wid) flags: Object_Wid and Name_Wid (stored in HFlags), Attribute_Wid (stored in AFlags), and Body_Wid (stored in BFlags). The Wid fields are interpreted in this manner:


The Object_Wid flag, at the high end of HFlags, encodes the width of the Object_Length field. The Name_Wid flag, in bits 3 and 4 of HFlags, encodes the width of the Name_Length field (when the name element is present; see the N bit, shown later.). Attribute_Wid (or Body_Wid, as the case may be) encodes the width of the Attribute_Length field (when the Object_Attributes (or Object_Attributes) element is present. http://ftp.arl.mil/~mike/papers/brlcad5.0/newdb.html - bbitSee the P bit, below.)??????????.

The rationale for allowing the width of the Object_Length field to be specified independently of the other widths is to save space on objects in which the values in many of the length fields nearly overflow the specified field width, so that their sum requires a wider field. For example, for four 255-byte interior fields, the corresponding length fields need be no more than 8 bits wide, so the choice Interior_Wid=00 suffices, but their combined length of 1020 bytes would require Object_Wid=01. Because all of the length fields besides Object_Length must have the same width, the largest of the values stored in these length fields determines the value of Interior_Wid required. Both Object_Wid and Interior_Wid may vary from object to object. It is expected that the routines that write an object to the disk will use the narrowest width possible for each object.

The DLI flag is a 2-bit flag that indicates whether the object is an Application Data Object or a Database Layer Internal Object. The bits are interpreted as follows:


The DLI flag is not available to the higher database access layers.

Note

Implementation note: Before writing a new object into the database in a free area, the library should read the object header from the database and confirm that the space is indeed free. Similarly, additions to the end should be checked by ensuring that the file hasn't been extended. In case the check fails, the database write should fail, the user should be notified, and the internal library mode (not the operating system file access permissions) should be changed over to read-only access so that no further attempts to write will be issued. These checks will provide protection against two or more users trying to modify the same database simultaneously and accidentally stepping on each other. In the NFS world, file locking isn't a strong enough assurance.

The Object_Type element is always 16 bits wide, organized into two 8-bit-wide fields: the Major_Type and the Minor_Type.


Each different Major_Type value is assigned to a different class of database objects. The following values are defined in this specification:


The remainder are available for extending the types of objects that may be stored in the database, allowing BRL-CAD users to extend the database for their own particular purposes far beyond what the "attribute" method permits.

This type of object stores only attributes in the object interior section; it has no object body elements.

For example, if several objects need to have the same shader parameters, it would be possible to create one attribute-only object to hold these common attributes and serve as a simple form of "macro". Objects that needed to share these attributes could all reference the same attribute object. If the attribute object is altered, then all of the objects that reference it would be updated together. Without this capability, the user would have to update each element individually to alter the attributes.

Conventions will have to be established regarding which attributes of an attribute-only object will be used when a macro reference is performed. For example, rt shaders will only be interested in the value of the "oshader=" attribute, while librt's tree-walker might also be interested in the "rgb=", "giftmater=", "nsn=", "material=", and "los=" attributes (assuming that a convention was developed so that a combination could macro-reference an attribute-only object too).

An attribute-only object may not have an object body; thus, flag bit B must always be zero for this type of object.

As used by the rt family of applications codes, these attribute-only objects will contain "macros" for shaders. The shader name and its parameters shall be encoded as a single ASCII string, which is the value of the "oshader=" attribute. An rt shader named "macro" (or equivalent) would take a single parameter "obj=", which would specify the name of the attribute-only object in the database from which the actual shader and shader parameter information would be extracted.

There will be one attribute-only object with a reserved object name of "_GLOBAL" that will be used to contain various kinds of states that are global to the entire ".g" database and that had previously been found in the database header itself. There will be the following BRL-CAD-specific attributes whose meaning is predefined for the _GLOBAL object:

In addition, the "comment=" attribute of the "_GLOBAL" object may be used to store human-readable remarks about the database that are not more properly associated with a specific database item. These might include remarks about data sources, model evolution, security classification, and release restrictions. In the absense of some outboard revision-control system, this might also be a place to record modification history, although such use is discouraged.

This class of objects contains various "bulk" binary data that might otherwise have been placed in auxiliary files.

MGED and standalone commands must be built to store/extract these opaque?? binary objects between a ".g" file and stdin/stdout/auxiliary files. A user might want to use those same MGED commands to store/extract the binary object body of any object for external processing. An easy example to imagine is the importing and exporting of texture maps for external processing, but the same commands could be used for importing and exporting solid parameters in their external binary form.

These objects may be referenced in combination nodes, for organizational purposes, but they cannot be drawn in MGED or raytraced, and doing so would result in a warning message being printed by the tree walker as that arc is traversed. This class may be used by all applications and layers.

The data's purpose may be placed in the "purpose=" attribute. (????????Need a table/registry of presently known values for this attribute.)

Routines that retrieve bulk binary objects should check the minor type and the "purpose=" attribute and send a warning message in the event of a mismatch, but best-effort processing of the object should continue. This will permit some degree of error checking, which should benefit novice users without standing in the way of "creatively" reusing one set of data, (e.g., using one array of values as both a height field and a bwtexture). This allows common data perversion practices, such as interpreting an array of floats as an array of bytes, to continue.

Each application will need to have its own syntax for the user to specify whether the data source is an outboard file or a raw-binary object. For example, the current RT sh_texture module uses the keyword file="name" to indicate an outboard file; that might be supplemented with an additional obj="name" possibility for retrieving from an inboard raw-binary object.

This class of objects contain various "bulk" binary data that might otherwise have been placed in an auxiliary file.

Point of Discussion?????Has ramifications... we have to implement type advising, so that applications that use these data can compare the type provided in the minor type code with the type that they're expecting and advise the user (with a warning message) that there is a potential type mismatch.


The 3-bit ``Atom'' flag indicates the fundamental data type of the atomic elements in the array according to the following scheme:


The ``S'' bit indicates whether an integer type is signed (1) or unsigned (0). Floats and doubles (i.e., atomic types with the highest atom bit equal to 0) are explicitly signed, so they will have the ``S'' bit equal to 1. (The bit patterns corresponding to unsigned floats and doubles are reserved for possible other use.)

The 2-bit ``Wid'' flag specifies the length (in atomic elements) of the array elements:


The remaining Minor_Type bits ``r'' are reserved for the design committee to use for other purposes, possibly including extensions of the ``Atom'' and/or ``Wid'' flags.

As examples, data in PIX(5) format, which might be used for a texture map, would have Minor_Type ``0010 0100'', indicating a triple of unsigned char, and CMYK data might be stored with Minor_Type ``0011 1011'', indicating a quadruple of doubles.

The data's purpose (e.g., height field, texture, bump, displacement, etc.) may be placed in the "purpose=" attribute. ?????Point of Discussion???(Need a table/registry of presently known values for this attribute.)

An object may optionally have an Object_Attributes element, which stores an association list

binding attributes to values.

These are ASCII strings of unlimited length. These attributes are intended for direct use by programs. There will be a WWW registry of attribute names presently in use to prevent two application developers from using the same attribute_name for different purposes.

For attribute names and attribute values, The decision was taken to support 8-bit ASCII only. The on-disk encoding of this will simply be:

where NULL represents a byte with all bits zero. The NULL in place of anameN+1 signals the end of the attribute data, simplifying the job of the reader.

Every object in the database may have zero or more attributes attached to it; the meaning of these attributes will vary depending on which application or library processes them.

There are several aname conventions that all BRL-CAD applications are expected to respect. There will be a WWW extendable registry of "in-use" anames, so that independent applications developers may select aname strings for their own use without fear of name conflicts later. The initial registry would include:

All other attributes, from whatever source, would be stored similarly, including application-specific and end-user-created attributes.

The minimal object is a Free object (with no name) 8 bytes long:

This is why we have chosen the 8-bit size for our chunks. Pad bytes are inserted as necessary in the Object Footer immediately before the second magic number so that the final byte of the object is the Magic2 byte. The pad bytes are not counted as part of the Body_Length, but are counted as part of the Object_Length.

The minimal valid object is thus the following Free object:

The header of the database will always look like this:

The minimal valid allocated database storage object (with an Object_Name, no Object_Attributes or Object_Body) would thus be:

Without the padding, this (rather useless) object would be 10 bytes long. Given the rounding requirements, it is clear that all allocated storage objects in the database must be at least 16 bytes long. A database object with a minimal Object_Body would need 12 bytes, which would need to be padded out to 16 bytes as well: