Skip to content

gh-69893: Add the close() method for xml.etree.ElementTree.iterparse() iterator #114534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Doc/library/xml.etree.elementtree.rst
Original file line number Diff line number Diff line change
Expand Up @@ -625,6 +625,8 @@ Functions
target. Returns an :term:`iterator` providing ``(event, elem)`` pairs;
it has a ``root`` attribute that references the root element of the
resulting XML tree once *source* is fully read.
The iterator has the :meth:`!close` method that closes the internal
file object if *source* is a filename.

Note that while :func:`iterparse` builds the tree incrementally, it issues
blocking reads on *source* (or the file it names). As such, it's unsuitable
Expand All @@ -647,6 +649,9 @@ Functions
.. versionchanged:: 3.8
The ``comment`` and ``pi`` events were added.

.. versionchanged:: 3.13
Added the :meth:`!close` method.


.. function:: parse(source, parser=None)

Expand Down
8 changes: 8 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,14 @@ warnings
warning may also be emitted when a decorated function or class is used at runtime.
See :pep:`702`. (Contributed by Jelle Zijlstra in :gh:`104003`.)

xml.etree.ElementTree
---------------------

* Add the :meth:`!close` method for the iterator returned by
:func:`~xml.etree.ElementTree.iterparse` for explicit cleaning up.
(Contributed by Serhiy Storchaka in :gh:`69893`.)


Optimizations
=============

Expand Down
85 changes: 82 additions & 3 deletions Lib/test/test_xml_etree.py
Original file line number Diff line number Diff line change
Expand Up @@ -553,6 +553,17 @@ def test_iterparse(self):
('end', '{namespace}root'),
])

with open(SIMPLE_XMLFILE, 'rb') as source:
context = iterparse(source)
action, elem = next(context)
self.assertEqual((action, elem.tag), ('end', 'element'))
self.assertEqual([(action, elem.tag) for action, elem in context], [
('end', 'element'),
('end', 'empty-element'),
('end', 'root'),
])
self.assertEqual(context.root.tag, 'root')

events = ()
context = iterparse(SIMPLE_XMLFILE, events)
self.assertEqual([(action, elem.tag) for action, elem in context], [])
Expand Down Expand Up @@ -644,12 +655,81 @@ def test_iterparse(self):

# Not exhausting the iterator still closes the resource (bpo-43292)
with warnings_helper.check_no_resource_warning(self):
it = iterparse(TESTFN)
it = iterparse(SIMPLE_XMLFILE)
del it

with warnings_helper.check_no_resource_warning(self):
it = iterparse(SIMPLE_XMLFILE)
it.close()
del it

with warnings_helper.check_no_resource_warning(self):
it = iterparse(SIMPLE_XMLFILE)
action, elem = next(it)
self.assertEqual((action, elem.tag), ('end', 'element'))
del it, elem

with warnings_helper.check_no_resource_warning(self):
it = iterparse(SIMPLE_XMLFILE)
action, elem = next(it)
it.close()
self.assertEqual((action, elem.tag), ('end', 'element'))
del it, elem

with self.assertRaises(FileNotFoundError):
iterparse("nonexistent")

def test_iterparse_close(self):
iterparse = ET.iterparse

it = iterparse(SIMPLE_XMLFILE)
it.close()
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

with open(SIMPLE_XMLFILE, 'rb') as source:
it = iterparse(source)
it.close()
self.assertFalse(source.closed)
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

it = iterparse(SIMPLE_XMLFILE)
action, elem = next(it)
self.assertEqual((action, elem.tag), ('end', 'element'))
it.close()
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

with open(SIMPLE_XMLFILE, 'rb') as source:
it = iterparse(source)
action, elem = next(it)
self.assertEqual((action, elem.tag), ('end', 'element'))
it.close()
self.assertFalse(source.closed)
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

it = iterparse(SIMPLE_XMLFILE)
list(it)
it.close()
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

with open(SIMPLE_XMLFILE, 'rb') as source:
it = iterparse(source)
list(it)
it.close()
self.assertFalse(source.closed)
with self.assertRaises(StopIteration):
next(it)
it.close() # idempotent

def test_writefile(self):
elem = ET.Element("tag")
elem.text = "text"
Expand Down Expand Up @@ -3042,8 +3122,7 @@ def test_basic(self):
# With an explicit parser too (issue #9708)
sourcefile = serialize(doc, to_string=False)
parser = ET.XMLParser(target=ET.TreeBuilder())
self.assertEqual(next(ET.iterparse(sourcefile, parser=parser))[0],
'end')
self.assertEqual(next(ET.iterparse(sourcefile, parser=parser))[0], 'end')

tree = ET.ElementTree(None)
self.assertRaises(AttributeError, tree.iter)
Expand Down
9 changes: 8 additions & 1 deletion Lib/xml/etree/ElementTree.py
Original file line number Diff line number Diff line change
Expand Up @@ -1248,10 +1248,17 @@ def iterator(source):
if close_source:
source.close()

gen = iterator(source)
class IterParseIterator(collections.abc.Iterator):
__next__ = iterator(source).__next__
__next__ = gen.__next__
def close(self):
if close_source:
source.close()
gen.close()

def __del__(self):
# TODO: Emit a ResourceWarning if it was not explicitly closed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any plans for when this TODO will be done? They can end up languishing for decades :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When all maintained Python versions have the close() method, so you can call it without checking the version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe update the TODO to mention this, so when someone sees it they know whether they can do the TODO or wait a bit longer?

Anyway, I'll approve this.

# (When the close() method will be supported in all maintained Python versions.)
if close_source:
source.close()

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add the :meth:`!close` method for the iterator returned by
:func:`xml.etree.ElementTree.iterparse`.