Provide a ProducerConsumer helper for processing a sequence of items in parallel. #73323
Conversation
ghost
added
Area-IDE
untriaged
| using var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); | ||
|
|
||
| // Defer to the ProducerConsumer. We're search the unreferenced projects in parallel. As we get results, we'll | ||
| // add them to the 'allSymbolReferences' queue. If we get enough results, we'll cancel all the other work. |
There was a problem hiding this comment.
view with whitespace off.
| }), | ||
| await ProducerConsumer<ImmutableArray<SymbolReference>>.RunParallelAsync( | ||
| viableUnreferencedProjects, | ||
| produceItems: static async (project, onItemsFound, args) => |
There was a problem hiding this comment.
this form is much easier to understand (for me at least). instead of hte nested lambdas, you just pass hte single lambda. It will be called in parallel, passing a single item from "viableUnreferencedProjects", and teh callback to call when items are produced based on htat item.
CyrusNajmabadi
changed the title
| { | ||
| return RunAsync( | ||
| // We're running in parallel, so we def have multiple writers | ||
| ProducerConsumerOptions.SingleReaderOptions, |
There was a problem hiding this comment.
never mind, I can see clearly now
| ProducerConsumerOptions options, | ||
| Func<Action<TItem>, TArgs, Task> produceItems, | ||
| Func<ImmutableArray<TItem>, TArgs, Task> consumeItems, | ||
| Func<Action<TItem>, TArgs, CancellationToken, Task> produceItems, |
There was a problem hiding this comment.
I like produceItems. It's the callback responsible for producing all the items. It will just do that by producing them and passing them to callback it itself is provided.
| return RunAsync( | ||
| // We're running in parallel, so we def have multiple writers | ||
| ProducerConsumerOptions.SingleReaderOptions, | ||
| produceItems: static (callback, args, cancellationToken) => |
There was a problem hiding this comment.
Have coffee! I can also extract into local functions on a follow-up if that's ok.
| /// <code>Parallel.ForEachAsync</code>, allowing for parallel processing of the items, with a preference towards | ||
| /// earlier items. | ||
| /// </summary> | ||
| private static Task PerformParallelSearchAsync<T>( |
There was a problem hiding this comment.
I can consider removing in follow up
|
@ToddGrun can I make those changes in follow-up? |
|
@jasonmalinowski For review when you get back. |
No description provided.