SVDDC

[rconan]

Hi,
I am trying to compute the SVD of a rather large matrix [13824,273] , the svd routine takes about 260s to perform the decomposition , compare to 7s with the svd routine from numpy linalg module.
And I cannot figure out how to use the svddc routine instead of the svd routine, it is not clear what values uvt_flag should be given.

Also I have noticed that matrix multiplication is very slow with ndarray dot routine compared again to numpy, for example , after the SVD if I do the matrix multiplications USVT with ndarray it takes 160s and with numpy it takes 25ms.

Is these timings expected for ndarray-linalg at this stage of development or did I do something wrong and I should expect the same speed than numpy?

Thanks

[jturner314]

The performance should not be that much worse; I would expect comparable performance between ndarray-linalg and NumPy. For arrays not in Fortran layout, ndarray-linalg may currently perform unnecessary copies of the array data which NumPy may avoid, but that should not cause such a large performance difference. Are you running in release mode (cargo run --release)? Also, what is the memory layout of your arrays (println!("{:?}", arr); or println!("shape={:?}, strides={:?}", arr.shape(), arr.strides());)?

Regarding svddc -- we need to add docs for the uvt_flag. [Edit: I just created #183.] It corresponds to the JOBZ parameter of the *gesdd LAPACK command. The LAPACK docs say:

          Specifies options for computing all or part of the matrix U:
          = 'A':  all M columns of U and all N rows of V**T are
                  returned in the arrays U and VT;
          = 'S':  the first min(M,N) columns of U and the first
                  min(M,N) rows of V**T are returned in the arrays U
                  and VT;
          = 'O':  If M >= N, the first N columns of U are overwritten
                  on the array A and all rows of V**T are returned in
                  the array VT;
                  otherwise, all columns of U are returned in the
                  array U and the first M rows of V**T are overwritten
                  in the array A;
          = 'N':  no columns of U or rows of V**T are computed.

where A is UVTFlag::Full, S is UVTFlag::Some, and N is UVTFlag::None.

[rconan]

using svddc with uvt_flag set to UVTFlag::Some and with cargo run --release, now the SVD is done in 2.5s, the same setup with numpy gives 110ms.
Here is the memory layout of the array I perform the SVD on:
shape=[13824, 273], strides=[1, 13824], layout=F (0x2), const ndim=2
Matrix multiplication has also improved down to 1.3s!