Fold variation that captures remainder elements

[atifaziz]

The current implementation of Fold throws when the source sequence doesn't contain exactly the same number of elements as the arity of the folder function. Consider adding a variation that captures remainder or tail elements.

Prototype

Below is an example shown for folder function with an arity of 3:

public static TResult FoldVar<T, TResult>(this IEnumerable<T> source,
                                          Func<T, T, T, IEnumerable<T>, TResult> folder)
{
    InvalidOperationException Error() =>
        throw new InvalidOperationException("Insufficient elements.");
        
    using (var e = source.GetEnumerator())
    {
        if (!e.MoveNext()) throw Error(); var first  = e.Current;
        if (!e.MoveNext()) throw Error(); var second = e.Current;
        if (!e.MoveNext()) throw Error(); var third  = e.Current;
        var rest = new List<T>();
        while (e.MoveNext())
            rest.Add(e.Current);
        return folder(first, second, third, rest);
    }
}

Note that this version still throws an error if there are insufficient elements in the source sequence but not otherwise.

Usage

var result =
    Enumerable.Range(1, 10)
              .FoldVar((x, y, z, rest) => new
              {
                  First  = x,
                  Second = y,
                  Third  = z,
                  Tail   = rest,
              });

// result = { First = 1, Second = 2, Third = 3, Tail = { 4, 5, 6, 7, 8, 9, 10 } }

[fsateler]

I don't have personally much use for this, but I suppose you have a real use-case that needed this?

BTW, maybe it'd be better if the whole sequence was not eagerly iterated.

[atifaziz]

but I suppose you have a real use-case that needed this?

Variable records would be one real use-case…

var input =
    @"Bob;Alice;2;Jack;Jill
      Tom;Mary;0
      Jane;Adam;3;Mark;Susan;Charlie";

var q = 
    from line in input.Split('\n')
    select line.Trim() into line
    select line.Split(';') into row
    select row.FoldVar((p1, p2, _, children) => new
    {
        Partner1 = p1,
        Partner2 = p2,
        Children = children
    });

BTW, maybe it'd be better if the whole sequence was not eagerly iterated.

Not sure how that would be possible without a redesign of the API.

[fsateler]

Thanks for the example. Looks useful.

BTW, maybe it'd be better if the whole sequence was not eagerly iterated.

Not sure how that would be possible without a redesign of the API.

Well, it could be done easily if we had Memoize:

public static TResult FoldVar<T, TResult>(this IEnumerable<T> source,
                                          Func<T, T, T, IEnumerable<T>, TResult> folder)
{
    InvalidOperationException Error() =>
        throw new InvalidOperationException("Insufficient elements.");
        
    using (var e = source.GetEnumerator())
    {
        if (!e.MoveNext()) throw Error(); var first  = e.Current;
        if (!e.MoveNext()) throw Error(); var second = e.Current;
        if (!e.MoveNext()) throw Error(); var third  = e.Current;

        IEnumerable<T> Rest() {
            while (e.MoveNext())
                yield return e.Current;
        }
        return folder(first, second, third, Rest().Memoize());
    }
}

[atifaziz]

Well, it could be done easily if we had Memoize

Except then we enter into a whole discussion about disposal (who should do it and when should it happen), which is what kept #211 from being merged.

[fsateler]

> Well, it could be done easily if we had Memoize Except then we enter into whole discussion about disposal (who should do it and when it should it happen), which is what kept #211 from being merged.

Ah, right. Nevermind then. Lets not open that can of worms again ;)

[atifaziz]

BTW, maybe it'd be better if the whole sequence was not eagerly iterated.

Not sure how that would be possible without a redesign of the API.

Well, it could be done easily if we had Memoize:

I think that the right way to solve the issue where we seem to be concerned about full iteration of the source when it may not be needed is to create an API over a reader monad. For something like Fold, it would look like this:

var result =
    MoreEnumerable
        .Generate(1, x => x + 1)
        .Fold(_ =>
            from a in _.Skip(1).Pop()
            from b in _.Pop()
            from c in _.Pop()
            from q in _.Pop(4)
            select new
            {
                First  = a,
                Second = b,
                Third  = c,
                Quad   = $"[{string.Join(", ", q)}]",
            });

// { First = 2, Second = 3, Third = 4, Quad = [5, 6, 7, 8] }

In other words, a computational expression would tell the operator what bits you want. This has the following benefits:

• you don't need overloads with arity going from 1 to 16
• you don't need to be limited by Func with 16 type parameters
• you can use infinite sequences
• you can extend the operators (Pop, Skip, List, etc.)
• same infrastructure is reusable where the same requirements come up

The one big downside is performance but I often think that those who are sensitive to performance of certain parts of their code shouldn't be using LINQ there in the first place.

Another downside may be that some LINQ operators may need to be re-implemented in terms of this.

This is what I meant by a redesign of the API.