Repository navigation
Adding method to consume an iterator efficiently #64117
Description
Currently, mapping methods for iterator, e.g. map and filter are implemented in a lazy manner. Which means they would not be calculated until further consumption.
However, sometimes it can be the case that given an Iterator (whatever it actually is) . we just wanted its side-effect and to simply consume it. for instance :
#[derive(Debug)]
struct Item(i32);
fn take_iter<T : Iterator>(iter: T) {
// we just want to consume the `iter` here
// do something else
}
fn main() {
let ve = vec![Item(1), Item(2), Item(3), Item(1)];
take_iter(ve.iter().filter(|value| value.0 < 3).map(|value| {
println!("{:?}", value);
value
}));
}Currently, there're several ways of achieving this goal, for example :
fn take_iter<T : Iterator>(iter: T) {
iter.for_each(|_| {}); // consumed
// do something else
}fn take_iter<T : Iterator>(iter: T) {
iter.count(); // consumed
// do something else
}While they're both usable, it's obvious that these methods are at the cost of efficiency, for_each yields a call to closure which is totally unnecessary, while count finally counting the number of elements in vain.
Activity
Phosphorus15 commentedon Sep 3, 2019
I suggest adding a method
consumefor theIteratortrait, which takes no params and effectively consumes each element in an iterator to preserve their side-effects only. Then the example above could be written as :jakubadamw commentedon Sep 3, 2019
That call to a no-op closure will most certainly be optimized out in release builds.
In the case of the code that you've provided using
map()for side effects is not really idiomatic. The lazyinspect()combinator or the strictfor_each()consumer would be more suited for invoking aprintln!()on each element.inspect()for when you're trying to observe/debug the evaluation of the lazy iterator andfor_each()for when you want to consume the iterator and produce predictable output. I'm not sure which it is in your case. I guess the latter?In either case, I think the proposed scenario does not deserve a separate method. If you find yourself in need of force-consuming an iterator that produces side effects, that is a strong hint that you should move the effectful operations (like
println!()) from closures within the lazy composition of your iterator to the part of the program that consumes the iterator, be that afor_each, aforloop or whatever else.Phosphorus15 commentedon Sep 3, 2019
Yea, most of the time a
for_each()is quiet enough, and it would be a rare case that one would like to preserve the side effect of previous mappings only.But in case that someone wants to perform an I/O action, or just to swiftly printing some debug output by
inspectetc. . It would do no harm that we add this method for good (while indeed, not that necessary).clarfonthey commentedon Sep 3, 2019
I recommend the idiom
.for_each(drop)because it's clearer; what you're suggesting is to basically make an alias for this. And, like was said, this is usually optimised out.scottmcm commentedon Sep 4, 2019
As @clarfon noted, this has been discussed before (#48945), leading to the current suggestion to use
.for_each(drop)for this, which is recommended in warnings as well:rust/src/libcore/iter/traits/iterator.rs
Lines 1476 to 1477 in b9de4ef
As such I'm going to close this.