Using custom tensor type

[ramcdona]

I've recently started working with libpny. Thanks for making it available.

In my use case, I am only reading some *.npz files. I do not need to write any files.

I would like to use my own tensor type so I can read the data directly into the data structures that will be used in my program.

Unfortunately, I don't see how this is possible. The documentation states that all you need to implement in your tensor class are the methods:

data
shape
size
dtype
fortran_order

I can imagine that this would suffice if you were writing your data to file. However, without any constructor or ability to specify the size of the data structure, I can't see how you will be able to read into a custom tensor type. Likewise, it seems that you must provide a tensor type that can support both row and column major ordering. This seems like a lot of requirements -- largely making it impractical to use your own tensor type for reading.

Perhaps it is impossible to use a custom tensor type for reading -- and I should just stick to the supplied tensor for read, copying the data out as soon as the read is complete.

I initially implemented a test program using the default tensor class. The test program read in a large matrix from file and then performed a simple matrix / vector multiplication. Profiling this test application revealed that (i,j) indexing the tensor was by far the most expensive part of the operation -- copying the data to vector<vector> improved the matrix/vector multiply time from .05 seconds to .0005 seconds. Clearly using a custom tensor class is worth doing.

Another challenge is

const std::vector<size_t> &shape() const

Since this returns a reference to a std::vector<size_t>, I think this effectively forces you to store the shape of your tensor in a std::vector<size_t> in your class.

I am developing a library (that uses libpny) and some of my expected users prefer to avoid using the STL. So, my preference is to implement a tensor class using only raw C++ data types -- i.e. raw C-Style arrays.

If libpny is going to force STL, one solution would be for me to create the required size vector on the fly -- if it was returned by a copy, or passed as an output reference as an argument. However, since it is returned as a reference, then I believe I would be forced to keep it around as a member variable of the class (so it remains valid after the call to size()).

Any suggestions are appreciated.

[matajoh]

Thanks for opening this issue! I'll do my best to address your concerns here:

• I think you are highlighting the need for some sample code for custom tensors. I'll take this on board use this issue to track that work.
• It is certainly not the intention that the tensors produced by libnpy be used directly for mathematical operations. The data is stored in a contiguous array in either row-major or column-major order. The intention is that the data gets moved to be used by another library, e.g., Eigen, which will then provide performant matrix maths.
• So, as re: your comment on the use of std::vector in the interface and a desire to avoid using the STL, it would be helpful to understand where the constraint is coming from. I currently provide the shape() method. Does this not suffice? I suppose I am missing a "shape_size" member which would return the number of dimensions in the tensor. Would these do the job?

I think some sample code for custom tensors would help here, and potentially also clarify for me where the issues are in the current interface, so we'll start there.

[ramcdona]

I agree on both points. A sample that uses a custom tensor type would document the process and it would help you go through the exercise of using the interface to confirm it can do everything you'd like.

The low-level way to replace shape() would be two api calls -- size_t shape_size() that returns the number of dimensions in the tensor, and then a size_t shape( size_t idim ); that returns the size of a particular dimension.

Sticking with std::vector, the current std::vector<size_t>& shape(); method would be improved if it returned a copy of the shape vector instead of a reference to it. Returning the reference forces the implementation to keep a std::vector of the shape in-scope. That is cheap, simple, and easy if you're using a std::vector to store that information under the hood -- but if you aren't it gets messy. If it returned a copy (instead of a reference), then you could create one on the fly, pass it out, and all would be done. Instead, since it returns a reference (that you must keep in scope), you now need to maintain a member variable of that type -- which means you either use a std::vector as the implementation, or you have to maintain two sources of truth.

While you don't need the built-in type to be super performant, and users will always want to be able to use a custom type (whether Eigen or something else), you still might want to do some benchmarking and profiling of the built-in type. A matrix-vector multiply taking two orders of magnitude longer using the default type (vs naive vectors of vectors) is a huge amount of overhead that boils down to the time required for a random lookup of a value. An array access A(i,j) should not take 100x the cost of a multiply.

I really appreciate you making this library available -- there are a ton of *.npz libraries out there. I tried a few and most sucked one way or the other.

I particularly appreciate that you managed to use miniz instead of depending on gzip or another external library. I believe that libraries should be as self-contained as possible -- I hate when a lib for something small like this wants to include Boost. Just say no....

[matajoh]

So, the size_t shape(size_t index) method already exists, and is the shape interface that a custom tensor type needs to implement. The vector version that takes no parameters is purely for convenience.

As for the array access, the issue fundamentally boils down to the complexity of ravel. Keep in mind that tensors can be of any dimensionality. At present, the tensor class trades access performance for simplicity of storage. In your case, the matrix/vector multiply could be performed by simply iterating through the data directly using the pointer, instead of using the random access functionality (this is how a BLAS library would do this operation, with vectorisation to allow full use of parallelism in the hardware). While I can write these routines, this library is not intended to be a replacement for numpy, but simply a way to read numpy files for downstream use in C++ programs.

[ramcdona]

Ok, that is helpful.

The documentation says that a custom tensor must implement shape but then the interface defines two versions of shape, with no indication that a custom tensor only needs to implement one of them. I could have dug deeper into the code to try and figure out how much of the interface was actually used, but I assumed both were required.

The custom tensor interface requirements made sense from a perspective of writing a *.npz file to disk, but it didn't seem like there were enough methods for reading from disk directly into a custom tensor. I.e. reading would need the ability to set number of dimensions, size per dimension, and allocate the required memory.

I understand that *.npz supports both row and column ordering, but it also was not clear whether a custom tensor implementation was required to support both. I.e. is it possible to read a tensor stored in column order on disk into a custom tensor type that only supports row ordering? I would expect a custom tensor to need some combination of get_order / set_order -- and clarity on what happens if you try to set_order() to an ordering that is not supported by the custom type.