Question: Is there planned support for iterator in version 1.23?

[amikai]

Hi @antonmedv

Golang 1.23 will support the iter package and define the type Seq[V any] as an iterator. The expr package supports numerous array functions. I believe iterators will be widely used in many libraries in the future. Is there any plan for these array functions to support not only arrays but also iterators in version 1.23?

Use slice.Values as example

// Values is slices.Values inject by expr.Func
let v = Values([1, 2, 3]);
any(Values, { # > 1}) //  [2, 3]

I understand this example is quite simple and not necessary (because we can directly use the array in any). However, please imagine that the iterator might not be iterating over an array.

Reference

https://tip.golang.org/doc/go1.23
https://pkg.go.dev/iter@master

[antonmedv]

Hey @amikai

Yes, definitely. I'm going to work on bring support for Go iterators in Expr.

[amikai]

Hi @antonmedv,

Although I am not familiar with the source code for expr, I would like to try working on this feature. If it's convenient for you, could you please explain how to proceed, mention any high-level steps, or reference similar PRs? I will evaluate my capabilities to decide whether to take on this task. If this change affects a large portion of the source code and you prefer to handle it yourself, just let me know.

[antonmedv]

Sure you can try next things.

First of all find an example of using an iterator and thing what is not working in expr.

[amikai]

Take all as example. I think the implementation like this:

• all only accept reflect.Array and reflect.Slice as a first element in checker.go. The 1.23 expose reflect.CanSeq, we can leverage on this function to check it is iterable or not.
• The function with predicate will treat # as PointerNode, I guess it want to use the array (or slice) element in the first parameter, and calculate is true or false. It should point to the value which pulled from iterator.

The built in function with predicate look like directly compile to byte code , that i quite not understand that part (compiler fundamental).

I think I can't help, so I'll leave what I imagined before here. I hope it helps you when you're working on it.

function
all(array | iter.Seq, predicate) → bool
any(array | iter.Seq, predicate) → bool
one(array | iter.Seq, predicate) → bool
none(array | iter.Seq, predicate) → bool
map(array, predicate) → array map(iter.Seq, predicate) → iter.Seq
filter(array, predicate) → array filter(iter.Seq, predicate) → iter.Seq
find(array | iter.Seq, predicate) → (element type)
findLast(array | iter.Seq, predicate) → (element type)
findIndex(array | iter.Seq, predicate) → int
count(array | iter.Seq, predicate) → int
concat(array1, array2[, ...]) → array concat(seq1, seq2[, ...]) → iter.Seq
concat(array, [, delimiter]) → string concat(iter.Seq, [, delimiter]) → string
(TODO) reduce
sum(array | iter.Seq) → int
mean(array | iter.Seq) → int
meidan(array | iter.Seq) → int
fist(array | iter.Seq) → (element type)
last(array | iter.Seq) → (element type)
take(array, n) → array take(iter.Seq, n) → iter.Seq
reverse(array) → array reverse(iter.Seq) → iter.Seq
sort(array) → array sort(iter.Seq) → iter.Seq

[antonmedv]

Nice 👍🏻

I like how you evaluated all the builtin functions. We can use it to better plan the feature.

One point about map:

map(iter.Seq, predicate) → iter.Seq

I think map will return array. Not seq. As lazy evaluation is more complicated.

[amikai]

Hi @antonmedv,

I hope you're doing well. I wanted to share my thoughts on the map function for iter.Seq. I believe it is important, and when you're implementing it, it might be worth considering the potential for future support.

For example, a scenario that I think could be beneficial is to prevent exhausting the entire array with a predicate:

any(map(seq, expensive_predicate), predicate)

[delaneyj]

Has there been any movement on this?

[antonmedv]

I've been working on a prototype for iterators. @delaneyj it would be very useful if you can share your setup: how do you use expr, what env?

[delaneyj]

I'm looking at integrating expr into NATS tooling and iterators would be amazing as the whole context is not available. I wanted to start filtering as messages came in. The iterator interface is ideal in this case

[antonmedv]

Could you please provide an example of environment what is on your mind?