Divide broadcast performance

[npolina4]

import dpctl.tensor as dpt
c = dpt.ones(8192, device='cpu', dtype='f4')
%timeit dpt.divide(c, c[:, None])
#435 ms ± 777 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

c_gpu = dpt.ones(8192, device='gpu', dtype='f4')
%timeit dpt.divide(c_gpu, c_gpu[:, None])
#1.17 s ± 4.41 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

import numpy
c_np = numpy.ones(8192, dtype='f4')
%timeit numpy.divide(c_np, c_np[:, None])
#67.2 ms ± 3.02 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

[ndgrigorian]

In [1]: import dpctl.tensor as dpt, numpy as np

In [2]: x1 = dpt.arange(8192, dtype="f4")

In [3]: x2 = x1[:, None]

In [4]: %time dpt.divide(x1, x2)
CPU times: user 1.45 s, sys: 1.01 s, total: 2.46 s
Wall time: 2.47 s

In [5]: %time dpt.divide(x2, x1)
CPU times: user 86 ms, sys: 67.4 ms, total: 153 ms
Wall time: 154 ms

In [6]: %time dpt.divide(x1, x2, order="F")
CPU times: user 144 ms, sys: 92 ms, total: 235 ms
Wall time: 236 ms

In [7]: res = dpt.empty((x1.size, x1.size), dtype=x1.dtype, order="F")

In [8]: %time dpt.divide(x1, x2, out=res)
CPU times: user 51.6 ms, sys: 10.6 ms, total: 62.2 ms
Wall time: 60.1 ms

The problem seems to be the stride simplification code. Upon investigating, it's returning the strides of all three arrays reversed when the first array has decreasing strides. When the first array has increasing strides, it leaves them as they are.

An example using _contract_iter3:

In [1]: import dpctl.tensor as dpt, numpy as np

In [2]: from dpctl.tensor._tensor_impl import _contract_iter3

In [1]: import dpctl.tensor as dpt, numpy as np

In [2]: from dpctl.tensor._tensor_impl import _contract_iter3

In [3]: sh, st1, st2, st3 = (8192, 8192), (0, 1), (1, 0), (8192, 1) # shape, strides in issue

In [4]: _contract_iter3(sh, st1, st2, st3)
Out[4]: ([8192, 8192], [1, 0], 0, [0, 1], 0, [1, 8192], 0)

In [5]: _contract_iter3(sh, st2, st1, st3)
Out[5]: ([8192, 8192], [1, 0], 0, [0, 1], 0, [8192, 1], 0)

When the order is switched in source/elementwise_functions.cpp it reverses the problem, as well.