ARROW-4226: [C++] Add sparse CSF tensor support

[rok]

This is to resolve ARROW-4226.

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - How to contribute patches

[pitrou]

Just a pre-review.

[rok]

This is missing the MakeSparseTensorFromTensor part for CSF.

[mrkn]

@rok Thank you for working to implement CSF format.

We should research the existing implementations of CSF format so that our implementation can be used to exchange the data without copying buffers.

Did you find the existing libraries that have CSF format? If you know some such libraries, could you tell me them for the investigation of the data layout?

[rok]

We should research the existing implementations of CSF format so that our implementation can be used to exchange the data without copying buffers.

Agreed! :)

Did you find the existing libraries that have CSF format? If you know some such libraries, could you tell me them for the investigation of the data layout?

To my knowledge CSF is implemented in taco and in pydata/sparse as GXCS, see this PR. I did not research the taco implementation.
As for sparse.GXCS: I'm skeptical of the way it indexes with a global index, see discussion here. My proposal uses offsets to enable per dimension indexing and it should be easily possible to interface it with sparse.GXCS.

My main reference for implementation is Figure 2 in this paper. I'm not sure this implementation is optimal, please give me your thoughts.

[rok]

Another point: CSF is a generalization of CSR and CSC. Because of this transformations CSF <-> CSC and CSF <-> CSR don't require changes to the index. I'm proposing a potential COO -> CSF in this PR, so we just need COO <- CSF and we have all sparse index transformations covered.
This would enable us to compare SparseTensors with different indices.

[mrkn]

GCXS in pydata/sparse seems an implementation of GCRS and GCCS proposed in this article.

I confirmed that taco supports CSF format.

Now I'm reading this paper and the implementation of taco to study taco's abstraction sparse tensor format.

I guess we need more researches on the real-world implementations of CSF format right now.

[rok]

There is also a CSF implementation by @ShadenSmith in SPLATT.
Another nice resource for this discussion is FROSTT, a collection of sparse tensors by the same author.

[ShadenSmith]

Hello! It's great to see your interest in the CSF format. I'm happy to answer any questions that you have.

[rok]

Hi @ShadenSmith. We're currently discussing layout of arrow's CSF implementation. As @mrkn mentioned we want to design it in a way that lets us avoid copying data when interfacing with existing libraries and rather make it possible to just pass pointers.

This PR currently proposes a structure like:

table SparseTensorIndexCSF {
  indptrType: Int;
  indptrBuffer: Buffer;
  indptrOffsets: [int];
  indicesType: Int;
  indicesBuffer: Buffer;
  indicesOffsets: [int];
  axisOrder: [long];
}

Here indices and inptrs are stored in indptrBuffer and indicesBuffer respectively. Offsets are used to split these buffers into per dimension indices and indptrs. See here for comments.

If I understand correctly SPLATT uses a different structure where indices and indptrs are stored in a '2D' data structure where dimension is used to accesses data per dimension, eg: fptr[dimension][fiber] and fids[dimension][fiber].

@ShadenSmith could you say:

• do you think other CSF implementations variables are laid out like SPLATT is?
• what number (range) of dimensions would you expect to encounter / need in real world applications?

[ShadenSmith]

Hi @rok, the other CSF implementations that I have seen either include the code from SPLATT or also just use the 2D layout for the per-mode structures. Here is an example. I'm not sure if taco uses the 1D or 2D implementation and did not find the details from a quick search of the repo.

I think five or six modes (dimensions) is the largest I have encountered in an application, with three being the most common. I have heard of some ongoing research with O(100-1000) mode tensors, but haven't seen results yet and am not sure something so high dimensional will pan out.

[rok]

Hi @rok, the other CSF implementations that I have seen either include the code from SPLATT or also just use the 2D layout for the per-mode structures. Here is an example. I'm not sure if taco uses the 1D or 2D implementation and did not find the details from a quick search of the repo.

Got it, thanks a lot! This is good to know. :)
I'll make another proposal with a 2D structure.

I think five or six modes (dimensions) is the largest I have encountered in an application, with three being the most common. I have heard of some ongoing research with O(100-1000) mode tensors, but haven't seen results yet and am not sure something so high dimensional will pan out.

Interesting, do you think higher dimensional tensors will be used if available?

[ShadenSmith]

Interesting, do you think higher dimensional tensors will be used if available?

Let me first say that I can really only speak for the tensor factorization community, and even there I'm more of a scalability researcher than an application researcher.

Today's software can deal with very high dimensional tensors (SPLATT for example just needs a compile-time flag to increase the max dimensionality). I think the issue is more on the side of the data and quality of a tensor factorization with many dimensions; you may just be learning noise unless the data truly has a nice tensor form with many dimensions. It's quite possible that future application areas do use tensors of that form...I just haven't seen it. Off the top of my head, some of the electronic health record tensors have ~5-10 dimensions, but still not 100-1000 AFAIK.

Also, I would usually not recommend using CSF for tensors with more than 5 or 6 dimensions, unless you have a lot of data that has a non-uniform sparsity pattern. A simple coordinate format would be better unless the non-zeros arrange nicely into fibers. As you increase the number of dimensions in a tensor, the idea of a fiber gets more and more specific, meaning that the savings you get from storing the sparse tensor in CSF disappear because most fibers will only have a few non-zeros in them.

[rok]

Thanks for the review @pitrou! I've pushed changes you suggested.
It's good to see this is slowly coming together.

[rok]

It seems flight tests are failing in appveyor.

[pitrou]

A couple of details yet. We're getting there :-)

[pitrou]

You don't seem to mutate those vector inputs, so should make them a const reference really :-)

[rok]

Thanks for the review @pitrou. I pushed the suggested changes :).

[pitrou]

Thank you @rok :-)

[rok]

Thanks @pitrou @mrkn and @ShadenSmith!