masked_select: Filtering tensors with a boolean mask tensor

[brentp]

given a 2d array/tensor, how would i do the arraymancer equivalent of this numpy expression:

a[:,a.mean(axis=0) > 0.5] = -1

I think it's probably some combination of the map/apply/fold_inline, but it's not obvious how to do that.
Once I understand, I can open a PR with some common examples like this if it's helpful.

[brentp]

I have come up with this:

import arraymancer

var T = randomTensor[float32](8, 3, 1'f32)
let m = T.mean(axis=0).broadcast(T.shape)

apply2_inline(T, m, if x > y: x else: -1)
echo T

which seems reasonable. not sure if there's a simpler way.

[mratsim]

So what's happening is that Numpy accepts a Tensor of booleans as a mask for selecting things.

Your solution works for your case because you don't need to discard/filter the values, however implementing filtering with a Tensor of bool would also be very useful for dataframes/analysis related stuff.

For filtering, I see the following difficulties:

• Implementing the algorithm: it may be easier to use PyTorch's masked_select as a reference but while not as complex as Numpy, the codebase is very complex at the moment due to their 3 backens mixed together: C Torch, Aten and C10.
• Updating the indexing macro to accept a Tensor of bool and dispatch to the masked_select proc(s). This may require multiple dispatch/allocations if 2 dimensions are sliced at the same time.

[brentp]

Thanks for considering it. For my additional 0.02...
I guess that, for example indexing with a tensor of booleans or indexes as in numpy is a fundamental feature, but for other stuff, I think I'd be fine just writing the 2-3 lines myself if I had a better handle on the map* and apply* functions.

To that end, it'd be nice if the broadcasting was done auto-magically (as in numpy) as well.

[mratsim]

Broadcast is done auto-magically with the .+ and other series of dot function. I intentionally have 2 different operators because it's a pain in Numpy to not broadcast and have a silent error to debug.

Arraymancer/src/tensor/operators_broadcasted.nim

Lines 25 to 28 in 94efff3

	proc `.+`*[T: SomeNumber|Complex[float32]|Complex[float64]](a, b: Tensor[T]): Tensor[T] {.noInit,inline.} =
	## Broadcasted addition for tensors of incompatible but broadcastable shape.
	let (tmp_a, tmp_b) = broadcast2(a, b)
	result = tmp_a + tmp_b

[brentp]

I am also having trouble just using the map/apply stuff. for example, I can't understand how to change this (which either segfaults or runs endlessly):

import arraymancer

var T = randomTensor[float32](250, 17384, 1'f32)
let m = T.map_inline():
  if x < 0: 1'f32 else: 0'f32

echo m.shape
echo m
echo m.mean(axis=0)

[Vindaar]

@brentp It's been a while since your last post, so maybe you've noticed this after posting at some point (or you encountered a real bug):

I believe the reason you're seeing the code run endlessly, is simply that the tensor you create is huge and arraymancer's printing is pretty slow (and doesn't just cut off after a fixed N elements).

[mratsim]

Note that masked_select implementation is planned soon.

[mratsim]

Tentative implementation and names at #429

If you have suggestion on proc name and description to limit confusion especially for

proc masked_axis_fill(t: var Tensor[T], mask: Tensor[bool], axis: int, value: T) 
  ## Take a 1D-mask 
  ## iterate on t along the axis and fill the slice of t with `value`
  ## if the mask[current_iteration_index] is true

proc masked_fill_along_axis(t: var Tensor[T], mask: Tensor[bool], axis: int, value: T) 
  ## Take a N-D mask. Dimension along the axis must be 1
  ## iterate on t along the axis
  ##   On the slice of t, apply masked_fill

I'm taking them (let's have the name discussion in the PR)