-
Notifications
You must be signed in to change notification settings - Fork 5
Advanced Topics
On this page, we are going to cover several advanced topics. These topics may involve performance issues, and the caveats of LibraryLink and wll-interface.
When using wll-interface, we can consider Wolfram Language as a C++ caller of our function so that the function signature determines how the arguments are passed to the callee. Commonly, an argument can be passed to the callee in the following ways:
| Pass by | Callee gets (semantically) | Can be changed? | Written as |
|---|---|---|---|
| value | copy of the value | Yes | Type |
| constant value | copy of the value | No | const Type |
| reference | argument itself | Yes | Type& |
| constant reference | argument itself | No | const Type& |
In the framework of LibraryLink communication, tensors that are passed from Wolfram Language follow three memory management possibilities: Automatic, "Constant", and "Shared". The following table shows how they are matched with C++ type specifications.
| Dynamic library (C++) | Wolfram Language | Owned by |
|---|---|---|
wll::tensor<T,R> |
{T,R,Automatic} |
Wolfram Kernel |
const wll::tensor<T,R> |
{T,R,"Constant"} |
Wolfram Kernel |
wll::tensor<T,R>& |
{T,R,"Shared"} |
both sides |
const wll::tensor<T,R>& |
{T,R,"Constant"} |
Wolfram Kernel |
For example, if we are implementing a function doing summation of a list, we should choose pass-by-const-reference. Because we are not going to change the content of the input list so that it does not need to be copied. The function signature should look like:
double sum(const wll::list<double>&);And the function is loaded as:
sum = LibraryFunctionLoad[(*libpath*), "wll_sum", {{Real, 1, "Constant"}}, Real];Caveat: Due to a behavior in the implementation of LibraryLink by Wolfram Language, pass a tensor with incorrect type by reference may lead to unexpected result. Why is the tensor not changing, when I pass it by reference?
wll-interface only allows the function to return a tensor by value. The reason why a tensor does not need to be returned by reference can be divided into two cases. First, if the tensor is a local variable, since local variables are destructed by the end of the function, returning references to them causes undefined behavior unless they are never used. Second, if the tensor is initially passed from Wolfram Language as a reference, there is no point to return it because the Kernel already has access to that tensor. For example, an in-place sort function may have a signature of void sort(wll::list<double>&).
Copying tensors can have various degrees of impact on the performance of LibraryLink functions. As an extreme example, function part takes a tensor and a index as position then returns the value on that position, where an extra copy of the tensor for each function call is devastating to the performance.
There are in general two techniques to avoid extra tensor copies, related to resource management in wll::tensor class and automatic type conversions.
wll::tensor template class is implemented to have four resource management types, specifically, how the memory is managed.
memory_type |
Owned by | Managed by (APIs) |
|---|---|---|
owned |
wll::tensor |
malloc/free
|
proxy |
Wolfram Kernel | --/-- |
manual |
wll::tensor |
MTensor_new/MTensor_free
|
shared |
both | --/MTensor_disown
|
When we construct a tensor from an MTensor, we can pass the intended memory type to the constructor as the second argument, and it will behave accordingly. For example, if we construct a wll::matrix<char> as shared, an assertion failure will be triggered because Wolfram language do not have char as a value type. But if we construct it as
proxy, even though it is also impossible, it will be constructed as owned, because they cannot be distinguished.
This feature is especially useful when we want to return tensors from our LibraryLink functions. A wll::tensor is by default constructed as owned. When we return such a tensor from a LibraryLink function, wll-interface has to create an MTensor copy, then pass the MTensor through LibraryLink. By constructing tensor as manual, the memory of the tensor is internally managed as an MTensor. For example,
auto identity_matrix(size_t s)
{
wll::matrix<double> m({s, s}, memory_type::manual);
for (size_t i = 0; i < 10; ++i)
m(i, i) = 1.0;
return identity;
}Since a manual tensor can only be create on the C++ side, and a manual tensor returned from a LibraryLink function must be movable, the internal MTensor can be returned directly without copy.
The only restriction is that the value type of a manual tensor must be, or be equivalent to, one of three MTensor data types, i.e. mint, mreal, and mcomplex. See scalar type specifications.
Automatic type conversions can sometimes happen, unnoticed. The most common case is when the value type of a tensor passed through LibraryLink is not the same as what is declared in the argument. For example,
auto some_function(const wll::list<float>&);Because real numbers in Wolfram Language are implemented as double, a tensor of float would require a copy of the original tensor. wll-interface thinks that you intend to do so, and will actually copy the tensors.
Try to avoid such confusing argument types, and use the following instead:
auto some_function(wll::list<float>); // do make a copy, use single precision
auto some_function(const wll::list<double>&); // A const-ref to whatever is passedThe elements of wll::tensor and wll::sparse_array are accessed by the same methods, i.e. .at(...) and .operator()(...). But they have different return types when the object is not constant, thus require special attentions.
In the following examples, the usages of accessing tensor elements are compared with those of accessing sparse array elements.
Non-constant wll::tensor<int, 2>
int u0 = t(2, 3); // okay
auto u2 = t(2, 3); // okay, u2 is int
int& u1 = t(2, 3); // okayConstant wll::tensor<int, 2>
int u4 = ct(2, 3); // okay
int& u5 = ct(2, 3); // error, r-value bind to non-const l-value reference
const int& u6 = ct(2, 3); // okay, const l-value referenceNon-constant wll::sparse_array<int, 2>
int v0 = s(2, 3); // okay
auto v1 = s(2, 3); // okay, v1 is a reference to s(2, 3)
int& v1 = s(2, 3); // error, r-value bind to non-const l-value referenceConstant wll::sparse_array<int, 2>
int v4 = cs(2, 3); // okay, u4 is initialized with the value of ct(2, 3)
int& v5 = cs(2, 3); // error, r-value bind to non-const l-value reference
const int& v6 = ct(2, 3); // okay, const l-value referenceThe behavior of non-constant wll::sparse_array is not as predicted because of the special data storage format. (A similar case is std::vector<bool> in C++ STL.) The reference to an element in sparse arrays is not of type value_type& but wll::sparse_array<T,R>::reference, a type that can either represent an explicit value in the sparse array, or a position with no explicit value. operator=(const value_type&) and operator value_type() is overloaded for this ::reference type in order to make it looks like a reference type, but it is a non-reference type after all, and it does not have address-of operator.