Hello World using Python embedded in C++, with RAII library initialization

Question

I have written a small Hello World application that uses Python's C-API. The C library requires some functions to be called for global library (de-)initialization and the objects created by the library eventually need to be released by passing the pointer to a specific function. I think this is a fairly common pattern in C libraries.

I attempted to use RAII, to get automatic cleanup, even when exceptions interfere with the planned control flow.

#include <tr1/memory>
#include <Python.h>

// helper to get function pointer for the macro
void Python_DecreaseRefCount(PyObject* p) {
    Py_DECREF(p);
}

int main() {
    // declare struct and create instance that handles global initialization and
    // deinitialization of Python API
    static struct PythonApi {
        PythonApi() { Py_Initialize(); }
        ~PythonApi() { Py_Finalize(); }
    } python_api;
    // create object
    // use smart pointer for automatic refcount decrement at scope end
    std::tr1::shared_ptr<PyObject> string(
        PyString_FromString("Hello World!\n"),
        &Python_DecreaseRefCount);
    // do something with object
    PyObject_Print(string.get(), stdout, Py_PRINT_RAW);
}

Compiles and runs with:

$ g++ test.cc `pkg-config --cflags --libs python2` && ./a.out
Hello World!

I'm interested in all kinds of feedback, as to whether my attempt makes sense and if there is room for improvement. Are there best practices, or certain idioms that facilitate this, that I might have missed in my approach?

Answer 1

1. Are you sure also scoping de-initialization of Python to main (by removing static) is not an option?
   As-is, you cannot use Python in a globals initializer or destructor anyway.
   Having taken a look at Py_Finalize, I actually would not deinitialize it at all, at least not in a release-build, as none of the reasons that option is provided apply to you:

   This function is provided for a number of reasons. An embedding application might want to restart Python without having to restart the application itself. An application that has loaded the Python interpreter from a dynamically loadable library (or DLL) might want to free all memory allocated by Python before unloading the DLL. During a hunt for memory leaks in an application a developer might want to free all memory allocated by Python before exiting from the application

   Bugs and caveats: [... lots ...]

2. Next, you shouldn't be using a function-pointer, use an empty function-object instead.
   Using a function-object facilitates inlining and as it's empty, reduces memory-overhead.

3. Actually, are you sure you actually need / want a std::shared_ptr?
   Either because you are already irretrievably locked into using that type, or because you actually might need weak pointers somewhere?

   If the answer is no, look into boost::intrusive_ptr, as that will save you an allocation every time you hand a PyObject to a smart-pointer's oversight, and reduces smartpointer-size by half, to raw-pointer-size.

   // Hooks in same scope as PyObject (global scope), so ADL in intrusive_ptr finds them:
   void intrusive_ptr_add_ref(PyObject* p) noexcept { Py_INCREF(p); }
   void intrusive_ptr_release(PyObject* p) noexcept { Py_DECREF(p); }
   // typedef for ease of use:
   typedef boost::intrusive_ptr<PyObject> PyPtr;
   // Use as:
   PyPtr string(PyString_FromString("Hello World!\n"), false);
   

Answer 2

C++11

If your compiler has proper C++11 support or better, you don't need, and shouldn't, use the stuff from std::tr1. shared_ptr is a member of namespace std and lives under the include file <memory>.

Global init/shutdown

Your approach is valid, it ensures proper termination if an exception is thrown and never caught. Still you have to be careful to avoid ever declaring a global object that might need to call the Py_ API in its constructor, because global constructors will be called before main is entered. There is no silver bullet there. The best course of action is to just avoid global objects with constructors & destructors that rely on other globals or APIs that require explicit init/shutdown.

Type alias?

Since all of your objects are stored as pointers to PyObject, you can make life easier and code shorter by defining an alias:

using CppPyObjectPtr = std::shared_ptr<PyObject>;

And to avoid resupplying the deleter every time, you can define a helper akin to std::make_unique. I also suggest using a functor structure or a lambda rather than a function pointer, the compiler should have better chances of optimizing it:

CppPyObjectPtr makePyObjectPtr(PyObject* obj)
{
    return CppPyObjectPtr(obj, [](PyObject* toFree) { Py_DECREF(toFree); });
}

// Usage:
auto pyString = makePyObjectPtr(PyString_FromString("Hello World!\n"));
PyObject_Print(pyString.get(), stdout, Py_PRINT_RAW);