Aff, Pipes and IAsyncEnumerable #1176
timmi-on-rails
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Motivation
I have a couple of
Aff
s that I want to run in parallel with awindowSize
to limit the degree of parallelism.Furthermore, I'd like to have a
Producer
thatyield
s the results in the order in which they finish.Problem
My solution relies on
Proxy.enumerate
which expectsIAsyncEnumerable
.I assume that this is the way to go, because of the samples and implementations I have sighted.
While I can live with my own solution, I'd appreciate support from the library.
Custom implementations can be error prone and not performant.
Idea
There is already utility such as
TraverseParallel
andSequenceParallel
for sequences ofAff
s, which covers parallel execution andwindowSize
s.However, these functions return an
Aff
that does not complete until every inputAff
has completed.So they are not useful when it comes to streaming the results.
I propose to add similar utility functions which return e.g.
Eff<IAsyncEnumerable<Fin<T>>>
, which is useful for streaming.From a library perspective we should consider wrapping
IAsyncEnumerable
.What do you think?
Beta Was this translation helpful? Give feedback.
All reactions