[ntuple] add support for leaf count arrays in classes #20633
Conversation
jblomer
force-pushed
the
ntuple-leafcount-class
branch
from
57e7292 to
614cc6c
Compare
New field for internal use: maps an on-disk collection to a leaf count array in a class. Writes RVec collection. Must only be used inside a class field.
Maps leaf count arrays in classes to RVecs using Internal::RLeafCountArrayField
Test Results 20 files 20 suites 3d 15h 42m 21s ⏱️ For more details on these failures, see this check. Results for commit 614cc6c. |
| \brief A field for an RVec field as a class member on disk that is represented as leaf count array` in memory. | ||
| \ingroup NTuple | ||
| */ | ||
| class RLeafCountArrayField final : public RFieldBase { |
There was a problem hiding this comment.
The commit message says it's Internal, but the declaration doesn't seem to be?
|
|
||
| struct LeafCountInClass : public LeafCountInClassBase { | ||
| unsigned char *fPayload1 = nullptr; //[fSize]; | ||
| unsigned char *fPayload2 = nullptr; //[fSize]; |
There was a problem hiding this comment.
I'm probably missing the context behind this PR, but looking at this class makes me wonder if we're subtly introducing support for raw pointers in RNTuple on-disk format.
There was a problem hiding this comment.
One could "emulate" a raw pointer by using fSize = 1, but since this only works for owned payloads it's not conceptually different from supporting unique_ptr (I believe the problematic part of raw pointers is the fact that they may not be owning and reference data outside the parent class)
There was a problem hiding this comment.
Also note that on-disk this is stored as an RVec<char>. This PR adds support for a peculiar memory layout of a vector, if you will.
| friend class RClassField; // only class fields are able to construct this | ||
|
|
||
| private: | ||
| std::size_t fItemSize{0}; ///< The size of a child field's item |
There was a problem hiding this comment.
For better or worse, the leaf count is currently limited (hard coded) to 32bits.
| if (idxRight == std::string::npos) | ||
| return nullptr; | ||
|
|
||
| auto countLeaf = title.substr(1, idxRight - 1); |
There was a problem hiding this comment.
| auto countLeaf = title.substr(1, idxRight - 1); | |
| auto nameCountLeaf = title.substr(1, idxRight - 1); |
(and subsequent change)
pcanal
left a comment
pcanal
left a comment
There was a problem hiding this comment.
LGTM code wise. Inclusion shall be discussed at next ROOT I/O workshop.
5 participants
Transparently translates a leaf count array in a custom class in an RVec (read & write). Classes with leaf count arrays are used, e.g., in the ALICE RAW EDM but most likely there are other relevant cases, too.
Since this approach diverges the in-memory representation from the on-disk representation, we should discuss if we actually want this.
In any case, we probably want the first commit that detects leaf count arrays in classes and fails gracefully in such a case.