broadcast makes sparse matrix full

[mauro3]

julia> sp = sprand(3,3,0.1)
3x3 sparse matrix with 1 Float64 entries:
        [2, 2]  =  0.885289

julia> broadcast(*, [1,1,1], sp)
3x3 Array{Float64,2}:
 0.0  0.0       0.0
 0.0  0.885289  0.0
 0.0  0.0       0.0

[pao]

@mauro3 Which version of Julia?

[mauro3]

Sorry, both 0.3 and 0.4. (edit: oddly broadcasting .* works in 0.4 but not in 0.3: #11475)

[tkelman]

This is because of the Array on this line:

julia/base/broadcast.jl

Line 236 in 6b35206

broadcast(f::Function, As...) = broadcast!(f, Array(promote_eltype(As...), broadcast_shape(As...)), As...)

You should be able to do either broadcast!(*, copy(sp), [1,1,1], sp), or the more-specialized but non-exported SparseMatrix.broadcast_zpreserving(*, [1,1,1], sp), ref #10106

[mauro3]

So, should it be a similar there instead?

And the broadcast! trick works on 0.3 as well, thanks for the work-around. (Which is good enough for me, so no need to fix this on 0.3 from my point of view).

[tkelman]

Well, similar to which of the inputs? We'd need something to the effect of promote_containertype which we don't really have right now, and would seem to be quite difficult to do fully generically. We should try to move in that direction of being smarter and more generic about container types instead of falling back to dense Array all the time, but it's going to take some time and effort to get there.

[ViralBShah]

I am pretty sure the dense output is due to a generic fallback.

[ViralBShah]

Perhaps if one of the inputs is sparse, we should dispatch to the sparse version.

julia> @which broadcast(*, [1,1,1], sp)
broadcast(f::Function, As...) at broadcast.jl:236

julia> @which broadcast(*, sparse([1,1,1]), sp)
broadcast{Tv1,Ti1,Tv2,Ti2}(f::Function, A_1::Base.SparseMatrix.SparseMatrixCSC{Tv1,Ti1}, A_2::Base.SparseMatrix.SparseMatrixCSC{Tv2,Ti2}) at sparse/sparsematrix.jl:854

[tkelman]

Not unless we know a-priori that the function is zero-preserving?

On second thought, maybe that would make sense, but the behavior of the function on zeros would have to be checked for that case.

[toivoh]

With only one sparse input, I think it doesn't make sense to produce a
sparse output unless the function products a zero if either input is zero,
like * does (but not so many other functions).

[ViralBShah]

This seems like it should be an easy thing to test. The unfortunate thing is that we will end up with a type unstable solution. I doubt that will be a problem, but still worth keeping in mind.

[tkelman]

I don't think we can add a whole lot of cleverness to broadcast on sparse matrices in time for 0.5. There are sparse-specialized internal methods but you have to look for them and your operation needs to satisfy the invariants.