Skip to content

Commit 1605ff3

Browse files
committed
Update upgrade guide for new method signature
1 parent b01ddde commit 1605ff3

File tree

1 file changed

+25
-35
lines changed

1 file changed

+25
-35
lines changed

docs/source/user-guide/upgrade-guides.rst

Lines changed: 25 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -25,69 +25,59 @@ This version includes a major update to the :ref:`ffi` due to upgrades
2525
to the `Foreign Function Interface <https://doc.rust-lang.org/nomicon/ffi.html>`_.
2626
Users who contribute their own ``CatalogProvider``, ``SchemaProvider``,
2727
``TableProvider`` or ``TableFunction`` via FFI must now provide access to a
28-
``LogicalExtensionCodec`` and a ``TaskContextProvider``. The most convenient
29-
way to provide these is from the ``datafusion-python`` ``SessionContext`` Python
30-
object. The ``SessionContext`` now has a method to export a
31-
``FFI_LogicalExtensionCodec``, which can satisfy this new requirement.
28+
``LogicalExtensionCodec`` and a ``TaskContextProvider``. The function signatures
29+
for the methods to get these ``PyCapsule`` objects now requires an additional
30+
parameter, which is a Python object that can be used to extract the
31+
``FFI_LogicalExtensionCodec`` that is necessary.
3232

3333
A complete example can be found in the `FFI example <https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example>`_.
34-
The constructor for your provider needs to take as an input the ``SessionContext``
35-
python object. Instead of calling ``FFI_CatalogProvider::new`` you can use the
36-
added method ``FFI_CatalogProvider::new_with_ffi_codec`` as follows:
34+
Your methods need to be updated to take an additional parameter like in this
35+
example.
3736

3837
.. code-block:: rust
3938
4039
#[pymethods]
4140
impl MyCatalogProvider {
42-
#[new]
43-
pub fn new(session: &Bound<PyAny>) -> PyResult<Self> {
44-
let logical_codec = ffi_logical_codec_from_pycapsule(session)?;
45-
let inner = Arc::new(MemoryCatalogProvider::new());
46-
47-
Ok(Self {
48-
inner,
49-
logical_codec,
50-
})
51-
}
52-
5341
pub fn __datafusion_catalog_provider__<'py>(
5442
&self,
5543
py: Python<'py>,
44+
session: Bound<PyAny>,
5645
) -> PyResult<Bound<'py, PyCapsule>> {
5746
let name = cr"datafusion_catalog_provider".into();
58-
let codec = self.logical_codec.clone();
59-
let catalog_provider =
60-
FFI_CatalogProvider::new_with_ffi_codec(Arc::new(self.clone()), None, codec);
6147
62-
PyCapsule::new(py, catalog_provider, Some(name))
48+
let provider = Arc::clone(&self.inner) as Arc<dyn CatalogProvider + Send>;
49+
50+
let codec = ffi_logical_codec_from_pycapsule(session)?;
51+
let provider = FFI_CatalogProvider::new_with_ffi_codec(provider, None, codec);
52+
53+
PyCapsule::new(py, provider, Some(name))
6354
}
6455
}
6556
66-
To extract the logical extension codec FFI object from the ``SessionContext`` you
57+
To extract the logical extension codec FFI object from the provided object you
6758
can implement a helper method such as:
6859

6960
.. code-block:: rust
7061
7162
pub(crate) fn ffi_logical_codec_from_pycapsule(
72-
obj: &Bound<PyAny>,
63+
obj: Bound<PyAny>,
7364
) -> PyResult<FFI_LogicalExtensionCodec> {
7465
let attr_name = "__datafusion_logical_extension_codec__";
66+
let capsule = if obj.hasattr(attr_name)? {
67+
obj.getattr(attr_name)?.call0()?
68+
} else {
69+
obj
70+
};
7571
76-
if obj.hasattr(attr_name)? {
77-
let capsule = obj.getattr(attr_name)?.call0()?;
78-
let capsule = capsule.downcast::<PyCapsule>()?;
79-
validate_pycapsule(capsule, "datafusion_logical_extension_codec")?;
72+
let capsule = capsule.downcast::<PyCapsule>()?;
73+
validate_pycapsule(capsule, "datafusion_logical_extension_codec")?;
8074
81-
let provider = unsafe { capsule.reference::<FFI_LogicalExtensionCodec>() };
75+
let codec = unsafe { capsule.reference::<FFI_LogicalExtensionCodec>() };
8276
83-
Ok(provider.clone())
84-
} else {
85-
Err(PyValueError::new_err(
86-
"Expected PyCapsule object for FFI_LogicalExtensionCodec, but attribute does not exist",
87-
))
88-
}
77+
Ok(codec.clone())
8978
}
9079
80+
9181
The DataFusion FFI interface updates no longer depend directly on the
9282
``datafusion`` core crate. You can improve your build times and potentially
9383
reduce your library binary size by removing this dependency and instead

0 commit comments

Comments
 (0)