C++ Boost

Boost.Python

Frequently Asked Questions (FAQs)


Is return_internal reference efficient?
How can I wrap containers which take C++ containers as arguments?

Is return_internal reference efficient?

Q: I have an object composed of 12 doubles. A const& to this object is returned by a member function of another class. From the viewpoint of using the returned object in Python I do not care if I get a copy or a reference to the returned object. In Boost.Python Version 2 I have the choice of using copy_const_reference or return_internal_reference. Are there considerations that would lead me to prefer one over the other, such as size of generated code or memory overhead?

A: copy_const_reference will make an instance with storage for one of your objects, size = base_size + 12 * sizeof(double). return_internal_reference will make an instance with storage for a pointer to one of your objects, size = base_size + sizeof(void*). However, it will also create a weak reference object which goes in the source object's weakreflist and a special callback object to manage the lifetime of the internally-referenced object. My guess? copy_const_reference is your friend here, resulting in less overall memory use and less fragmentation, also probably fewer total cycles.

How can I wrap functions which take C++ containers as arguments?

Ralf W. Grosse-Kunstleve provides these notes:

  1. Using the regular class_<> wrapper:
    class_<std::vector<double> >("std_vector_double")
      .def(...)
      ...
      ;
    
    This can be moved to a template so that several types (double, int, long, etc.) can be wrapped with the same code. This technique is used in the file
    scitbx/include/scitbx/array_family/boost_python/flex_wrapper.h
    in the "scitbx" package. The file could easily be modified for wrapping std::vector<> instantiations.

    This type of C++/Python binding is most suitable for containers that may contain a large number of elements (>10000).

  2. Using custom rvalue converters. Boost.Python "rvalue converters" match function signatures such as:
    void foo(std::vector<double> const& array); // pass by const-reference
    void foo(std::vector<double> array); // pass by value
    
    Some custom rvalue converters are implemented in the file
    scitbx/include/scitbx/boost_python/container_conversions.h
    This code can be used to convert from C++ container types such as std::vector<> or std::list<> to Python tuples and vice versa. A few simple examples can be found in the file
    scitbx/array_family/boost_python/regression_test_module.cpp
    Automatic C++ container <-> Python tuple conversions are most suitable for containers of moderate size. These converters generate significantly less object code compared to alternative 1 above.
A disadvantage of using alternative 2 is that operators such as arithmetic +,-,*,/,% are not available. It would be useful to have custom rvalue converters that convert to a "math_array" type instead of tuples. This is currently not implemented but is possible within the framework of Boost.Python V2 as it will be released in the next couple of weeks. [ed.: this was posted on 2002/03/10]

It would also be useful to also have "custom lvalue converters" such as std::vector<> <-> Python list. These converters would support the modification of the Python list from C++. For example:

C++:

void foo(std::vector<double>& array)
{
  for(std::size_t i=0;i<array.size();i++) {
    array[i] *= 2;
  }
}
Python:
>>> l = [1, 2, 3]
>>> foo(l)
>>> print l
[2, 4, 6]
Custom lvalue converters require changes to the Boost.Python core library and are currently not available.

P.S.:

The "scitbx" files referenced above are available via anonymous CVS:

cvs -d:pserver:anonymous@cvs.cctbx.sourceforge.net:/cvsroot/cctbx login
cvs -d:pserver:anonymous@cvs.cctbx.sourceforge.net:/cvsroot/cctbx co scitbx

Revised 05 November, 2002

© Copyright Dave Abrahams 2002. All Rights Reserved.