Support guards/filters in comprehensions

[pao]

Quoting @rtzui in #547:
On the other hand more powerful:

x=[1,2,3,4,510,1,2,3,1,9]
y=[sqrt(a) | a ∈ x, x%2==0]

(Discussion from @pao)
There is an open syntactic question. The Haskell syntax (shown above) has guards as boolean expressions separated by commas and interspersed with membership assertions, which works as long as you can tell the difference between an expression evaluating to a boolean and a loop assignment statement. Python uses the keyword "if" to precede each guard. I'm sure there are other approaches as well.

(From @JeffBezanson)
It's maybe not ideal, but you can accomplish this as:

x=[1,2,3,4,510,1,2,3,1,9]
y=[sqrt(a) | a in x[x%2==0] ]

[pao]

Jeff's workaround for this particular case is fine, but in general guards could include multiple comprehension variables, as in the Cartesian product except the diagonal:

Prelude> [(a,b) | a <- [1..4], b <- [1..4], a /= b]
[(1,2),(1,3),(1,4),(2,1),(2,3),(2,4),(3,1),(3,2),(3,4),(4,1),(4,2),(4,3)]

>>> [(a,b) for a in range(1,5) for b in range(1,5) if a != b]
[(1, 2), (1, 3), (1, 4), (2, 1), (2, 3), (2, 4), (3, 1), (3, 2), (3, 4), (4, 1), (4, 2), (4, 3)]

I don't see how to get there with the current list comprehension capabilities.

[ViralBShah]

I call what we have array comprehensions. The size of the output array is determined by the ranges before the computation starts, and that would be difficult with guards. One could always extract a subarray at the end though, if guards are used, and they should not be too difficult to implement either.

We do need to debate if we want this, and the actual syntax. @StefanKarpinski Given that you came up with the original comprehensions idea, what are your thoughts?

[StefanKarpinski]

Filters and guards don't mix with multidimensional comprehensions. For 1-d comprehensions, I'm not convinced that it's really worth the additional syntax. What's the case for making it part of the syntax instead of just using a filter function?

[rtzui]

/edit. here was something else, but it was BS.

I don't see the advantage of guards to array slices a la x[x>3], but it don't see the problem with multi dimensions since the slices already operate on multi dimension arrays.

multiple comprehension variables would be cool thou, but I think it would be cooler this way:

((1..4) insert appropriate operator here (1..4))[ (x,y) -> x /= y]

[StefanKarpinski]

Because it doesn't work in higher dimensions: you can't just excise arbitrary items out of a matrix and still get a matrix.

[ViralBShah]

One could set values to be excised to zero or some other default provided by the user. One could implement diag, triu, tril, spdiags, etc. using comprehensions. This kind of stuff would be great for experimenting and exploration. However, these would be bad implementations, since even though the running time complexity is the same, one ends up consuming many more flops than necessary.

-viral

On 09-Mar-2012, at 3:24 PM, Stefan Karpinski wrote:

Because it doesn't work in higher dimensions: you can't just excise arbitrary items out of a matrix and still get a matrix.

---

Reply to this email directly or view it on GitHub:
#550 (comment)

[rtzui]

julia> x=[1 2 3 4; 5 6 7 9]
2x4 Int64 Array:
 1  2  3  4
 5  6  7  9

julia> [ a | a = x ]
{1, 5, 2, 6, 3, 7, 4, 9}

Today the output is always one dimensional. Or am i missing something?

[pao]

Above comments re: full array comprehensions seem reasonable. There are contexts (like the Cartesian product minus the diagonal) where you'd be okay with that array flattened, and some where you would really prefer to never evaluate "invalid" pairs:

[ does_something_useful_but_throws_exception_if_equal(a, b) | a in 1:4, b in 1:4 ]

It might be the case that we'd rather this idiom be written with a double-for-loop-with-inner-if, but adding a guard to this expression is definitely more concise Cartesian for loop and if statement, which was mentioned in #330 but doesn't seem to be documented in the Control Flow section of the manual.

[pao]

@rtzui Try [(a,b) | a in 1:4, b in 1:4]. The final array has one dimension per expression on the RHS of the bar.

[pao]

I've written a small patch for the manual to mention the Cartesian for feature, JuliaLang/www_old.julialang.org#8.

[JeffBezanson]

I guess this could be implemented by transforming 1d comprehensions with guards to loops using push.

[StefanKarpinski]

I guess this could be implemented by transforming 1d comprehensions with guards to loops using push.

That's a solid idea. Another option would be to pre-allocate the whole thing and then shrink at the end. Might want to choose between the two approaches based on a threshold. (Guards would still only work in the 1d case.)