234 lines
8.5 KiB
ReStructuredText
234 lines
8.5 KiB
ReStructuredText
Smart pointers & ``py::class_``
|
|
###############################
|
|
|
|
The binding generator for classes, ``py::class_``, can be passed a template
|
|
type that denotes a special *holder* type that is used to manage references to
|
|
the object. If no such holder type template argument is given, the default for
|
|
a type ``T`` is ``std::unique_ptr<T>``.
|
|
|
|
.. note::
|
|
|
|
A ``py::class_`` for a given C++ type ``T`` — and all its derived types —
|
|
can only use a single holder type.
|
|
|
|
|
|
.. _smart_holder:
|
|
|
|
``py::smart_holder``
|
|
====================
|
|
|
|
Starting with pybind11v3, ``py::smart_holder`` is built into pybind11. It is
|
|
the recommended ``py::class_`` holder for most situations. However, for
|
|
backward compatibility it is **not** the default holder, and there are no
|
|
plans to make it the default holder in the future.
|
|
|
|
It is extremely easy to use the safer and more versatile ``py::smart_holder``:
|
|
simply add ``py::smart_holder`` to ``py::class_``:
|
|
|
|
* ``py::class_<T>`` to
|
|
|
|
* ``py::class_<T, py::smart_holder>``.
|
|
|
|
.. note::
|
|
|
|
A shorthand, ``py::classh<T>``, is provided for
|
|
``py::class_<T, py::smart_holder>``. The ``h`` in ``py::classh`` stands
|
|
for **smart_holder** but is shortened for brevity, ensuring it has the
|
|
same number of characters as ``py::class_``. This design choice facilitates
|
|
easy experimentation with ``py::smart_holder`` without introducing
|
|
distracting whitespace noise in diffs.
|
|
|
|
The ``py::smart_holder`` functionality includes the following:
|
|
|
|
* Support for **two-way** Python/C++ conversions for both
|
|
``std::unique_ptr<T>`` and ``std::shared_ptr<T>`` **simultaneously**.
|
|
|
|
* Passing a Python object back to C++ via ``std::unique_ptr<T>``, safely
|
|
**disowning** the Python object.
|
|
|
|
* Safely passing "trampoline" objects (objects with C++ virtual function
|
|
overrides implemented in Python, see :ref:`overriding_virtuals`) via
|
|
``std::unique_ptr<T>`` or ``std::shared_ptr<T>`` back to C++:
|
|
associated Python objects are automatically kept alive for the lifetime
|
|
of the smart-pointer.
|
|
|
|
* Full support for ``std::enable_shared_from_this`` (`cppreference
|
|
<http://en.cppreference.com/w/cpp/memory/enable_shared_from_this>`_).
|
|
|
|
|
|
``std::unique_ptr``
|
|
===================
|
|
|
|
This is the default ``py::class_`` holder and works as expected in
|
|
most situations. However, handling base-and-derived classes involves a
|
|
``reinterpret_cast``, which is, strictly speaking, undefined behavior.
|
|
Also note that the ``std::unique_ptr`` holder only supports passing a
|
|
``std::unique_ptr`` from C++ to Python, but not the other way around.
|
|
For example, the following code works as expected with ``py::class_<Example>``:
|
|
|
|
.. code-block:: cpp
|
|
|
|
std::unique_ptr<Example> create_example() { return std::unique_ptr<Example>(new Example()); }
|
|
|
|
.. code-block:: cpp
|
|
|
|
m.def("create_example", &create_example);
|
|
|
|
However, this will fail with ``py::class_<Example>`` (but works with
|
|
``py::class_<Example, py::smart_holder>``):
|
|
|
|
.. code-block:: cpp
|
|
|
|
void do_something_with_example(std::unique_ptr<Example> ex) { ... }
|
|
|
|
.. note::
|
|
|
|
The ``reinterpret_cast`` mentioned above is `here
|
|
<https://github.com/pybind/pybind11/blob/30eb39ed79d1e2eeff15219ac00773034300a5e6/include/pybind11/cast.h#L235>`_.
|
|
For completeness: The same cast is also applied to ``py::smart_holder``,
|
|
but that is safe, because ``py::smart_holder`` is not templated.
|
|
|
|
|
|
``std::shared_ptr``
|
|
===================
|
|
|
|
It is possible to use ``std::shared_ptr`` as the holder, for example:
|
|
|
|
.. code-block:: cpp
|
|
|
|
py::class_<Example, std::shared_ptr<Example> /* <- holder type */>(m, "Example");
|
|
|
|
Compared to using ``py::class_<Example, py::smart_holder>``, there are two noteworthy disadvantages:
|
|
|
|
* Because a ``py::class_`` for a given C++ type ``T`` can only use a
|
|
single holder type, ``std::unique_ptr<T>`` cannot even be passed from C++
|
|
to Python. This will become apparent only at runtime, often through a
|
|
segmentation fault.
|
|
|
|
* Similar to the ``std::unique_ptr`` holder, the handling of base-and-derived
|
|
classes involves a ``reinterpret_cast`` that has strictly speaking undefined
|
|
behavior, although it works as expected in most situations.
|
|
|
|
|
|
.. _smart_pointers:
|
|
|
|
Custom smart pointers
|
|
=====================
|
|
|
|
For custom smart pointers (e.g. ``c10::intrusive_ptr`` in pytorch), transparent
|
|
conversions can be enabled using a macro invocation similar to the following.
|
|
It must be declared at the top namespace level before any binding code:
|
|
|
|
.. code-block:: cpp
|
|
|
|
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>)
|
|
|
|
The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
|
|
placeholder name that is used as a template parameter of the second argument.
|
|
Thus, feel free to use any identifier, but use it consistently on both sides;
|
|
also, don't use the name of a type that already exists in your codebase.
|
|
|
|
The macro also accepts a third optional boolean parameter that is set to false
|
|
by default. Specify
|
|
|
|
.. code-block:: cpp
|
|
|
|
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>, true)
|
|
|
|
if ``SmartPtr<T>`` can always be initialized from a ``T*`` pointer without the
|
|
risk of inconsistencies (such as multiple independent ``SmartPtr`` instances
|
|
believing that they are the sole owner of the ``T*`` pointer). A common
|
|
situation where ``true`` should be passed is when the ``T`` instances use
|
|
*intrusive* reference counting.
|
|
|
|
Please take a look at the :ref:`macro_notes` before using this feature.
|
|
|
|
By default, pybind11 assumes that your custom smart pointer has a standard
|
|
interface, i.e. provides a ``.get()`` member function to access the underlying
|
|
raw pointer. If this is not the case, pybind11's ``holder_helper`` must be
|
|
specialized:
|
|
|
|
.. code-block:: cpp
|
|
|
|
// Always needed for custom holder types
|
|
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>)
|
|
|
|
// Only needed if the type's `.get()` goes by another name
|
|
namespace PYBIND11_NAMESPACE { namespace detail {
|
|
template <typename T>
|
|
struct holder_helper<SmartPtr<T>> { // <-- specialization
|
|
static const T *get(const SmartPtr<T> &p) { return p.getPointer(); }
|
|
};
|
|
}}
|
|
|
|
The above specialization informs pybind11 that the custom ``SmartPtr`` class
|
|
provides ``.get()`` functionality via ``.getPointer()``.
|
|
|
|
.. note::
|
|
|
|
The two noteworthy disadvantages mentioned under the ``std::shared_ptr``
|
|
section apply similarly to custom smart pointer holders, but there is no
|
|
established safe alternative in this case.
|
|
|
|
.. seealso::
|
|
|
|
The file :file:`tests/test_smart_ptr.cpp` contains a complete example
|
|
that demonstrates how to work with custom reference-counting holder types
|
|
in more detail.
|
|
|
|
|
|
Be careful not to accidentally undermine automatic lifetime management
|
|
======================================================================
|
|
|
|
``py::class_``-wrapped objects automatically manage the lifetime of the
|
|
wrapped C++ object, in collaboration with the chosen holder type.
|
|
When wrapping C++ functions involving raw pointers, care needs to be taken
|
|
to not inadvertently transfer ownership, resulting in multiple Python
|
|
objects acting as owners, causing heap-use-after-free or double-free errors.
|
|
For example:
|
|
|
|
.. code-block:: cpp
|
|
|
|
class Child { };
|
|
|
|
class Parent {
|
|
public:
|
|
Parent() : child(std::make_shared<Child>()) { }
|
|
Child *get_child() { return child.get(); } /* DANGER */
|
|
private:
|
|
std::shared_ptr<Child> child;
|
|
};
|
|
|
|
PYBIND11_MODULE(example, m) {
|
|
py::class_<Child, std::shared_ptr<Child>>(m, "Child");
|
|
|
|
py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
|
|
.def(py::init<>())
|
|
.def("get_child", &Parent::get_child); /* PROBLEM */
|
|
}
|
|
|
|
The following Python code leads to undefined behavior, likely resulting in
|
|
a segmentation fault.
|
|
|
|
.. code-block:: python
|
|
|
|
from example import Parent
|
|
|
|
print(Parent().get_child())
|
|
|
|
Part of the ``/* PROBLEM */`` here is that pybind11 falls back to using
|
|
``return_value_policy::take_ownership`` as the default (see
|
|
:ref:`return_value_policies`). The fact that the ``Child`` instance is
|
|
already managed by ``std::shared_ptr<Child>`` is lost. Therefore pybind11
|
|
will create a second independent ``std::shared_ptr<Child>`` that also
|
|
claims ownership of the pointer, eventually leading to heap-use-after-free
|
|
or double-free errors.
|
|
|
|
There are various ways to resolve this issue, either by changing
|
|
the ``Child`` or ``Parent`` C++ implementations (e.g. using
|
|
``std::enable_shared_from_this<Child>`` as a base class for
|
|
``Child``, or adding a member function to ``Parent`` that returns
|
|
``std::shared_ptr<Child>``), or if that is not feasible, by using
|
|
``return_value_policy::reference_internal``. What is the best approach
|
|
depends on the exact situation.
|