# This is a combination of 119 commits.

# This is the 1st commit message:

Issue #303: framework for reference_dense added

This commit comes from a split of the original commit whose details are as follows:
SHA1 ID: a7c1db0
Author: Albert-Jan Yzelman <yzelman@linux.fritz.box>  2022-01-14 15:52:26
Committer: Albert-Jan Yzelman <yzelman@linux.fritz.box>  2022-01-30 13:48:35

The reason for splitting is to separate the changes irrelevant to the branch 303 into a separate branch.
The split produced 3 separate commits.
This commit is number 1.
This commit in particular is relevant to 303 and will remain in it.

# This is the commit message #2:

Issue #303: unify script argument similar to --no-reference

# This is the commit message #3:

Issue #303: complete implementation of Vector< reference_dense >, modulo output iteration.

# This is the commit message #4:

Issue #303: modified vector implementation according to option 3 in handling dense container construction.

# This is the commit message #5:

Issue #303: vectors now have an iterator for the user to extract data from.

# This is the commit message #6:

Issue #303, fixed silly oops: the difference between two iterators should be an integer

# This is the commit message #7:

Issue #303: containers are now uninitialized on construction. Iteration over uninitialized containers returns nothing.

# This is the commit message #8:

Issue #303: configure and cmake now properly do make install with reference_dense

# This is the commit message #9:

Issue #303: cmake infra has updated-- make dense_reference additions compatible to it

# This is the commit message #10:

Drafting structured matrix interface using reference_dense backend

# This is the commit message #11:

Issue #303: Copy test mxm.cpp to dense_mxm.cpp as a starting point

# This is the commit message #12:

Issue #303: Add partial first version of Matrix interface for denseref

# This is the commit message #13:

Issue #303: WIP: dense mxm test

# This is the commit message #14:

Issue #303: Add dense mxm test to CMakeLists

# This is the commit message #15:

Issue #303: Separate reference and reference_dense includes

# This is the commit message #16:

Issue #303: Fix grb include name mismatch

# This is the commit message #17:

Issue #303: Add src files for reference dense backend

# This is the commit message #18:

Issue #303: Add project related option for dense backend

# This is the commit message #19:

Issue #303: Remove dense_reference-related code from reference cmake

# This is the commit message #20:

Keeping a clean separate denseref backend when building with cmake.

# This is the commit message #21:

Fix cmake option name

# This is the commit message #22:

Use a proper data container for a denseref matrix

# This is the commit message #23:

Change PRIVATE to INTERFACE for denseref backend

# This is the commit message #24:

fix cmake config for building backend

# This is the commit message #25:

Add friend functions for accessing matrix dimensions and raw data

# This is the commit message #26:

Add first version of blas3 mxm for reference dense backend

# This is the commit message #27:

Add temporary code for matrix data initialization

# This is the commit message #28:

Complete first working dense mxm unit test

# This is the commit message #29:

Added structured matrix test including draft static views and references

# This is the commit message #30:

Convert spaces to tabs

# This is the commit message #31:

Make use of operators and monoid provided to mxm

# This is the commit message #32:

Add friend getter/setter for field 'initialized' of denseref matrix

# This is the commit message #33:

Revert "Add temporary code for matrix data initialization"

This reverts commit 676cb4d.

# This is the commit message #34:

Add temporary way for allocating data of a denseref matrix

# This is the commit message #35:

Add functions to initialize matrix data from provided forward iterator

# This is the commit message #36:

Make getters const

# This is the commit message #37:

Exploit new features in mxm unit test

New features are building a matrix from an iterator and handling of
uninitialized matrices.

# This is the commit message #38:

added get/remove_ref type traits for StructuredMatrix types

# This is the commit message #39:

Temporary switch to pointers to target matrices to enable empty init.

# This is the commit message #40:

First draft of index mapping functions

# This is the commit message #41:

Cleaned up StructuredMatrix-related doxygen drafts

# This is the commit message #42:

Added to the spec draft

# This is the commit message #43:

Add alloc primitives for denseref

Currently they are a copy of the same primitives for reference backend.
Think whether it makes sense to group the alloc functions for these two
backends.

# This is the commit message #44:

Handle memory allocation failure. Implement destructor.

# This is the commit message #45:

WIP: Add data size calculator depending on Storage scheme and structure.

Also add a simple test to showcase the usage of this feature.

# This is the commit message #46:

Add buildMatrix for general full StructuredMatrix

# This is the commit message #47:

Add a check for size mismatch in buildMatrix

# This is the commit message #48:

Add mxm for dense Structured Matrices

# This is the commit message #49:

Introducing grb::get_view<> draft

# This is the commit message #50:

Add UpperTriangular StructuredMatrix (container and view)

# This is the commit message #51:

Implement polymorphic comparison of IMFs

# This is the commit message #52:

Add to structures a compatibility check from source structure and IMF functions

This is to be used when creating a view over a source matrix using
provided IMF functions. The compatibility of TargetStructure depends on
the SourceStructure and provided IMF functions. This is a runtime check.

# This is the commit message #53:

Add constructor taking two IMFs to reference SM with general structure

# This is the commit message #54:

Implement gather through view with provided TargetStructure and IMFs

# This is the commit message #55:

Add test for gathers on StructuredMatrix using views with IMFs

# This is the commit message #56:

Basic VectorView definition based on grb::vector container

# This is the commit message #57:

Draft of diagonal (vector) view of a matrix

# This is the commit message #58:

Initial get_view for VectorViews

# This is the commit message #59:

Add a stub for Storage Mapping Functions

# This is the commit message #60:

Define interface for dense StructuredMatrices and scalars

# This is the commit message #61:

Add test showcasing eWiseApply operations on Structured Matrices

# This is the commit message #62:

Minor changes for dense reference vector

# This is the commit message #63:

Add outer product of two vectors for reference_dense backend

# This is the commit message #64:

Expand the gather via IMF to general view of general Storage Matrix

# This is the commit message #65:

Add more Structures for StructuredMatrix

# This is the commit message #66:

Add dot product and norm2 in blas1 for dense reference backend

Also add a stub for a test showcasing their use

# This is the commit message #67:

WIP Constant matrix factory methods

# This is the commit message #68:

WIP Base classes for StructuredMatrix

# This is the commit message #69:

WIP Refactor StructuredMatrix using inheritance from Base classes

# This is the commit message #70:

Implement friend functions through base classes

This avoids the need to declare friend functions inside every
StructuredMatrix specialization.

# This is the commit message #71:

Add getter and setter for initialized field in StructuredMatrix

# This is the commit message #72:

Revert "WIP Constant matrix factory methods"

This reverts commit 742fe9f.

# This is the commit message #73:

Add interface for creating constant matrices

This includes Identity matrix, Zero matrix and Givens rotations

# This is the commit message #74:

Add test showcasing the use of Identity and Zero matrices

# This is the commit message #75:

Revert "WIP: Add data size calculator depending on Storage scheme and structure."

This reverts commit 1f7b6c4.

# This is the commit message #76:

draft of selection and range views.

# This is the commit message #77:

Using default coords template param for VectorView.

# This is the commit message #78:

Added matrix-related doxygen draft. In need of compilation check (had to push to from machine without compiler support).

# This is the commit message #79:

Fixed interface compilation (need to double check buildMatrix friend functions though)

# This is the commit message #80:

Add doxygen for reference_dense vector

Also minor code cleanup.

# This is the commit message #81:

WIP blas1 interface for reference dense

# This is the commit message #82:

WIP: Add blas2 interface for reference_dense backend

# This is the commit message #83:

Fix compilation errors in denseref blas1

# This is the commit message #84:

Complete blas2 interface for denseref backend

# This is the commit message #85:

Fix compilation of vector

# This is the commit message #86:

Add possibility for VectorView to be a temporary

# This is the commit message #87:

Add VectorView tmp parameter to blas1 and blas2

# This is the commit message #88:

Add doxygen for VectorView

Provides detailed description of relevant concepts.

# This is the commit message #89:

Fix specification of tmp template parameters in blas1 and blas2

# This is the commit message #90:

Fix remaining tmp template errors in blas1

# This is the commit message #91:

Correct tmp template specializaiton in denseref VectorView

# This is the commit message #92:

Add missing tmp parameter to view source type

# This is the commit message #93:

Expand doxygen for denseref blas1

# This is the commit message #94:

Comment out performance semantics doxygen from blas1

It requires revision if still applies to the denseref backend

# This is the commit message #95:

Started working on doxygen description for level3 functions

# This is the commit message #96:

Added doxygen draft to blas3 operations

# This is the commit message #97:

compiler messages cleanup

# This is the commit message #98:

Add capacity to dense vector

# This is the commit message #99:

Expand doxygen for dense vector

# This is the commit message #100:

Inline functions to avoid "multiple definitions" linking error

# This is the commit message #101:

Remove tmp template parameter from VectorView interface.

This commit is the result of squashing the commits listed above and
cleanup of other locations where tmp parameter was introduced (doxygen,
blas3).

Revert "Add missing tmp parameter to view source type"
This reverts commit cd70999.

Revert "Correct tmp template specializaiton in denseref VectorView"
This reverts commit bdc728e.

Revert "Fix remaining tmp template errors in blas1"
This reverts commit 8821830.

Revert "Fix specification of tmp template parameters in blas1 and blas2"
This reverts commit d00b3ed.

Revert "Add VectorView tmp parameter to blas1 and blas2"
This reverts commit d009aac.

Revert "Add possibility for VectorView to be a temporary"
This reverts commit fd5bf70.

# This is the commit message #102:

Rename Identity view to Original

including related variables, typenames and doxygen fixes

# This is the commit message #103:

Rename StructuredMatrix base classes

This includes StructuredMatrixBase, StructuredMatrixContainer and
StructuredMatrixReference

# This is the commit message #104:

Remove tmp template parameter from StructuredMatrix interface

# This is the commit message #105:

Remove 'initialized' field from StructureMatrix classes

as it is already present in the underlying container grb::Matrix.
Expose that field through friend functions operaiting on
StructuredMatrix classes.

# This is the commit message #106:

Store reference instead of a pointer to a source matrix

# This is the commit message #107:

Adjust comments to reflect renaming Identity to Original view

# This is the commit message #108:

Remove 'initialized' field from VectorView

as it is already present in the underlying container grb::Vector.
Expose that field through friend functions operaiting on
VectorView.

# This is the commit message #109:

!19 Resolve internal issue #360: Make VectorView "compatible" with StructuredMatrix in terms of template parameters
This commit mainly adds Structure to the VectorView and reorders VectorView's template parameters to follow the same order as in StructuredMatrix. Doxygen is updated to reflect these changes.

In addition, there are the following changes:
* Remove VectorView's Tmp parameter from blas1 doxygen
* Use consistent ordering of template name parameters across blas 1/2/3. The order is Output, Mask, Inputs.
* Fix minor unrelated issues
* Generalize over storage scheme type for VectorView in level 1/2/3 routines

# This is the commit message #110:

!18 Resolve internal issue #357: Include capacity argument to VectorView and StructuredMatrix

# This is the commit message #111:

!20 Resolve internal issue #361: Introduce Scalar container

# This is the commit message #112:

!25 Resolve internal issue #366: Scalar version of operations at level 1

This commit introduces the following changes:
* Update ALP tests to use ALP Scalar
* Update level 1 functions to operate on ALP scalars

# This is the commit message #113:

!30 Resolve internal issue #377: Do not expose coordinates template argument in alp vectors

# This is the commit message #114:

!24 Resolves internal issue #371: Introducing Union of Bands Structure
* Using past-the-end notation for static grb::Interval
* Indent comments from MR !24
* simple example of usage in test
* Initial draft of Band structure

# This is the commit message #115:

!26 Resolve internal issue #364: Resize operations for reference dense containers

* Add resize functions for dense ALP containers
* Resize for dense containers succeeds if new_cap <= current_cap

# This is the commit message #116:

!29 Resolve internal issue #362: Implement bracket operator for accessing single element

* Add bracket operator for ALP scalars.
* Add bracket operator to VectorView

# This is the commit message #117:

!31 Resolves internal issue #376: All views applied via get_view
* Dropped leftover coord arguments in VectorView
* Integrating comments from MR !31
* Doxygen cleanup
* Started using get_view for all views in matrix.hpp

# This is the commit message #118:

!23 Resolve internal issue #370: Introduce sort operation on vectors
* Move sort to level-1 functions
* clean buffer and move to C++ comparators
* define sort interface

# This is the commit message #119:

!32 Resolve internal issue #378: Convert grb storage from namespace to enum

This enum now has two values - Dense and Sparse.
The actual implementation of the storage scheme
will be handled by the Storage Mapping Functions.

Author: hidanielesgit

include/graphblas/base/matrix.hpp

@@ -37,7 +37,7 @@
 #include <graphblas/descriptors.hpp>
 #include <graphblas/ops.hpp>
 #include <graphblas/rc.hpp>
-#include <graphblas/storage.hpp>
+#include <graphblas/density.hpp>
 #include <graphblas/structures.hpp>
 #include <graphblas/utils.hpp>
 #include <graphblas/views.hpp>
@@ -402,7 +402,7 @@ RC clear( Matrix< InputType, backend > & A ) noexcept {
 	return UNSUPPORTED;
 }
 
-template< typename T, typename Structure, typename StorageSchemeType, typename View, enum Backend backend >
+template< typename T, typename Structure, enum Density density, typename View, enum Backend backend >
 class StructuredMatrix {
 
 	/**
@@ -435,10 +435,12 @@ class StructuredMatrix {
 	 * If \a backend is set to \a mlir then the scheme could be fixed by the JIT compiler to effectively
 	 * support its optimization strategy.
 	 * At construction time and until the moment the scheme decision is made it may be set to
-	 * an appropriate default choice, e.g. if \a StorageSchemeType is \a storage::Dense then
-	 * \a storage::Dense::full could be used.
+	 * an appropriate default choice, e.g. if \a density is \a Density::Dense then
+	 * \a Density::Dense::full could be used.
+	 * \internal \todo Revisit this. The change of storage scheme type to enum (dense/sparse) and
+	 * implementing storage mapping functions requires a change of this spec.
 	 */
-	StorageSchemeType storage_scheme;
+	// Storage storage_scheme;
 
 	/**
 	 * When a structured matrix defines a View over another matrix, it contains a pointer
@@ -451,9 +453,9 @@ class StructuredMatrix {
 
 	StructuredMatrix( const size_t m, const size_t n );
 
-	StructuredMatrix( const StructuredMatrix< T, Structure, StorageSchemeType, View, backend > & other );
+	StructuredMatrix( const StructuredMatrix< T, Structure, density, View, backend > & other );
 
-	StructuredMatrix( StructuredMatrix< T, Structure, StorageSchemeType, View, backend > && other );
+	StructuredMatrix( StructuredMatrix< T, Structure, density, View, backend > && other );
 
 	~StructuredMatrix();
 
@@ -464,8 +466,8 @@ class StructuredMatrix {
  */
 template< typename T >
 struct is_structured_matrix : std::false_type {};
-template< typename T, typename Structure, typename StorageSchemeType, typename View, enum Backend backend >
-struct is_structured_matrix< StructuredMatrix< T, Structure, StorageSchemeType, View, backend > > : std::true_type {};
+template< typename T, typename Structure, enum Density density, typename View, enum Backend backend >
+struct is_structured_matrix< StructuredMatrix< T, Structure, density, View, backend > > : std::true_type {};
 
 } // end namespace ``grb''
 

include/graphblas/base/scalar.hpp

@@ -54,6 +54,9 @@ namespace grb {
 			/** @see Vector::value_type. */
 			typedef T value_type;
 
+			/** @see Vector::lambda_reference */
+			typedef T& lambda_reference;
+
 			/**
 			 * The default ALP scalar constructor.
 			 *
@@ -164,6 +167,60 @@ namespace grb {
 				(void)other;
 			}
 
+			/**
+			 * Returns a lambda reference to the value of this Scalar. The user
+			 * ensures that the requested reference only corresponds to a pre-existing
+			 * nonzero in this scalar, <em>or undefined behaviour will occur</em>.
+			 * This addresses the sparse specialization of scalars. In the dense
+			 * context, scalar is considered to have a nonzero value \em iff initialized.
+			 *
+			 * A lambda reference to the value of this scalar is only valid when used
+			 * inside a lambda function evaluated via grb::eWiseLambda. Outside this
+			 * scope the returned reference incurs undefined behaviour.
+			 *
+			 *
+			 * \warning In parallel contexts the use of a returned lambda reference
+			 *          outside the context of an eWiseLambda will incur at least one of
+			 *          the following ill effects: it may
+			 *            -# fail outright,
+			 *            -# work on stale data,
+			 *            -# work on incorrect data, or
+			 *            -# incur high communication costs to guarantee correctness.
+			 *          In short, such usage causes undefined behaviour. Implementers are
+			 *          \em not advised to provide GAS-like functionality through this
+			 *          interface, as it invites bad programming practices and bad
+			 *          algorithm design decisions. This operator is instead intended to
+			 *          provide for generic BLAS0-type operations only.
+			 *
+			 * \note    For I/O, use the iterator retrieved via cbegin() instead of
+			 *          relying on a lambda_reference.
+			 *
+			 * @return      A lambda reference to the value of this scalar
+			 *
+			 * \par Example.
+			 * See grb::eWiseLambda() for a practical and useful example.
+			 *
+			 * \warning There is no similar concept in the official GraphBLAS specs.
+			 *
+			 * @see lambda_reference For more details on the returned reference type.
+			 * @see grb::eWiseLambda For one legal way in which to use the returned
+			 *      #lambda_reference.
+			 */
+			lambda_reference operator*() noexcept {
+#ifndef _GRB_NO_EXCEPTIONS
+				assert( false ); // Requesting lambda reference of unimplemented Scalar backend.
+#endif
+			}
+
+			/** Returns a constant reference to the scalar value.
+			 * See the non-constant variant for further details.
+			 */
+			const lambda_reference operator*() const noexcept {
+#ifndef _GRB_NO_EXCEPTIONS
+				assert( false ); // Requesting lambda reference of unimplemented Scalar backend.
+#endif
+			}
+
 	}; // class Scalar
 
 } // namespace grb

include/graphblas/base/vector.hpp

@@ -36,6 +36,7 @@
 #include <graphblas/descriptors.hpp>
 #include <graphblas/ops.hpp>
 #include <graphblas/rc.hpp>
+#include <graphblas/density.hpp>
 
 
 namespace grb {
@@ -948,7 +949,7 @@ namespace grb {
 			}
 	};
 
-	template< typename T, typename Structure, typename StorageSchemeType, typename View, enum Backend backend >
+	template< typename T, typename Structure, enum Density density, typename View, enum Backend backend >
 	class VectorView { };
 
 } // end namespace ``grb''