Skip to content

gh-113479: Link to workaround for subtle issue with takewhile() #115890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 25, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 41 additions & 33 deletions Doc/library/itertools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -688,6 +688,14 @@ loops that truncate the stream.
else:
break

Note, the element that first fails the predicate condition is
consumed from the input iterator and there is no way to access it.
This could be an issue if an application wants to further consume the
input iterator after takewhile has been run to exhaustion. To work
around this problem, consider using `more-iterools before_and_after()
<https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.before_and_after>`_
instead.


.. function:: tee(iterable, n=2)

Expand Down Expand Up @@ -1004,32 +1012,6 @@ which incur interpreter overhead.
except exception:
pass

def before_and_after(predicate, it):
""" Variant of takewhile() that allows complete
access to the remainder of the iterator.

>>> it = iter('ABCdEfGhI')
>>> all_upper, remainder = before_and_after(str.isupper, it)
>>> ''.join(all_upper)
'ABC'
>>> ''.join(remainder) # takewhile() would lose the 'd'
'dEfGhI'

Note that the true iterator must be fully consumed
before the remainder iterator can generate valid results.
"""
it = iter(it)
transition = []

def true_iterator():
for elem in it:
if predicate(elem):
yield elem
else:
transition.append(elem)
return

return true_iterator(), chain(transition, it)


The following recipes have a more mathematical flavor:
Expand Down Expand Up @@ -1543,13 +1525,6 @@ The following recipes have a more mathematical flavor:
>>> list(odds)
[1, 3, 5, 7, 9]

>>> it = iter('ABCdEfGhI')
>>> all_upper, remainder = before_and_after(str.isupper, it)
>>> ''.join(all_upper)
'ABC'
>>> ''.join(remainder)
'dEfGhI'

>>> list(subslices('ABCD'))
['A', 'AB', 'ABC', 'ABCD', 'B', 'BC', 'BCD', 'C', 'CD', 'D']

Expand Down Expand Up @@ -1640,6 +1615,32 @@ The following recipes have a more mathematical flavor:
result.append(pool[-1-n])
return tuple(result)

def before_and_after(predicate, it):
""" Variant of takewhile() that allows complete
access to the remainder of the iterator.

>>> it = iter('ABCdEfGhI')
>>> all_upper, remainder = before_and_after(str.isupper, it)
>>> ''.join(all_upper)
'ABC'
>>> ''.join(remainder) # takewhile() would lose the 'd'
'dEfGhI'

Note that the true iterator must be fully consumed
before the remainder iterator can generate valid results.
"""
it = iter(it)
transition = []

def true_iterator():
for elem in it:
if predicate(elem):
yield elem
else:
transition.append(elem)
return

return true_iterator(), chain(transition, it)

.. doctest::
:hide:
Expand Down Expand Up @@ -1669,3 +1670,10 @@ The following recipes have a more mathematical flavor:
>>> combos = list(combinations(iterable, r))
>>> all(nth_combination(iterable, r, i) == comb for i, comb in enumerate(combos))
True

>>> it = iter('ABCdEfGhI')
>>> all_upper, remainder = before_and_after(str.isupper, it)
>>> ''.join(all_upper)
'ABC'
>>> ''.join(remainder)
'dEfGhI'