Fix doctest runable examples in python manual

[abalkin]

BPO	10225
Nosy	@birkenfeld, @rhettinger, @terryjreedy, @abalkin, @ezio-melotti, @merwok, @ambv, @andy-maier
Dependencies	bpo-10224: Build 3.x documentation using python3.x
Files	issue10225-r27.diff: Patch agains 2.7 branch issue10225a-r27.diff: Patch against 2.7 branch sorting-howto.diff issue10225-py3k.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-10-29.02:18:53.672>
labels = ['docs']
title = 'Fix doctest runable examples in python manual'
updated_at = <Date 2016-07-16.01:27:06.527>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2016-07-16.01:27:06.527>
actor = 'belopolsky'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2010-10-29.02:18:53.672>
creator = 'belopolsky'
dependencies = ['10224']
files = ['19409', '19413', '20359', '20377']
hgrepos = []
issue_num = 10225
keywords = ['patch']
message_count = 21.0
messages = ['119840', '119841', '119842', '119869', '119908', '119917', '125860', '125906', '126050', '126051', '126063', '126067', '126073', '126074', '126107', '126111', '131794', '132019', '132020', '222164', '226121']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'rhettinger', 'terry.reedy', 'belopolsky', 'ezio.melotti', 'eric.araujo', 'docs@python', 'lukasz.langa', 'andymaier']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue10225'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

Linked PRs

• gh-54434: Make difflib.rst doctests pass. #111677
• [3.12] gh-54434: Make difflib.rst doctests pass. (GH-111677) #111678
• [3.11] gh-54434: Make difflib.rst doctests pass. (GH-111677) #111679
• gh-54434: Execute difflib.rst doctests in our test suite #111680

[abalkin]

As noted in bpo-10224, python 3.x documentation is not being built with python 3.x yet, so a simple cd Doc; make doctest does not work. However, hg trunk version os sphinx works with python 3.x and can be used to validate examples in ReST documentation.

[merwok]

hg trunk version [of] sphinx works with python 3.x and can be used to
validate examples in ReST documentation.

How do you replace “make doctest”?

[abalkin]

On Thu, Oct 28, 2010 at 10:20 PM, Éric Araujo <report@bugs.python.org> wrote:

How do you replace “make doctest”?

$ sphinx-build -b html -d Doc/build3/doctrees -D latex_paper_size=
Doc  Doc/build3/html

but I had to run 2to3 on Doc/conf.py and Doc/tools first.

[abalkin]

I started with 2.7 branch because some of the issues are the same there, but the tools work better at the moment. I a posting a work-in-progress patch to solicit early feedback.

[abalkin]

I am attaching a new patch which fixes all but two doctest examples.

I suspect that the remaining failures are due to a bug in sphinx. (The examples are executed even though marked up with ::).

[abalkin]

I started porting the patch to 3.x and it looks like I've uncovered another bug in sphinx:

$ sphinx-build -b doctest -d build/doctrees  . build/doctest library/traceback.rst
..
**********************************************************************
File "library/traceback.rst", line 327, in default
..

Differences (ndiff with -expected +actual):
*** print_tb:
- File "<doctest...>", line 10, in <module>
? ^^^
+ File "<doctest default[0]>", line 10, in <module>
? ^^^^^^^^^^^
lumberjack()

(I added +REPORT_NDIFF to testoutput options for clarity.)

I even tried adding +ELLIPSIS to options, but this did not help. The same example works in 2.7.

[terryjreedy]

As I understand this, your patch would fix numerous errors in the docs, but has not been applied. It would be nice to get the fixes into 3.2. What sort of feedback do you want? Proofreading of text? Recheck of doctest?

Does sphinx suppress doctest comments like
#doctest: +IGNORE_EXCEPTION_DETAIL
?

doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False) fails 23 of 37 attempts, so improvement is needed ;-).

[abalkin]

On Sun, Jan 9, 2011 at 3:52 PM, Terry J. Reedy <report@bugs.python.org> wrote:

.. It would be nice to get the fixes into 3.2.

Yes, it would be nice, but I don't have a 3.2 patch (yet). As you can
see from my last comment, I ran into a bug in the alpha version of
sphinx that I used to validate py3k docs.

What sort of feedback do you want? Proofreading of text? Recheck of doctest?

All of the above. Some of the fixes may be at the expense of
readability. I would appreciate feedback in the instances when human
and computer reader's interests are in conflict. Some examples are
"fixed" by excluding them from being checked. Better ideas are
welcome.

Does sphinx suppress doctest comments like
    #doctest: +IGNORE_EXCEPTION_DETAIL
?

Yes.

doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False)
fails 23 of 37 attempts, so improvement is needed ;-).

You cannot run documentation examples with a plain doctest. You have
to use sphinx-build.

[terryjreedy]

Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed. See bpo-10875 where is was very helpful. So your last comment puzzles me.

In any case, your patch is too big to digest at once. I extracted the part for howto/sorting.rst and tried to apply to 3.2, but the file has changed too much (at least for TortoiseSVN). So I pasted the '... 's (and adjusted spacing) where needed, and did the two other changes you had. This left just two failures -- for the 2.7 code using 'cmp=xxx' args. With '# doctest: +SKIP' added twice, doctest passes. Patch commited for 3.2 as r87946.

instances when human and computer reader's interests are in conflict.
What is one (or more) that you were thinking of?

[abalkin]

On Tue, Jan 11, 2011 at 6:10 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..

Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed.
See bpo-10875 where is was very helpful. So your last comment puzzles me.

Sphinx doctest runner supports doctest fixtures that plain doctest
does not. Some of the examples in the docs rely on these.

See http://sphinx.pocoo.org/ext/doctest.html .

[rhettinger]

Terry, I'm unhappy with the changes to the sorting how-to. IMO, it was a not a net win (transforming code that already ran fine in something doctest would swallow). The code snippets now have the visual clutter of the ">>>" and "..." PS1 and PS2 prompts and they can no longer be readily cut and pasted into the interpreter so that people can experiment with them.

[terryjreedy]

There are two reasons I forward ported the changes.

1. Without running doctest on doc examples, they sometimes have errors either originally, after patches to the doc, or after patches to Python. On other issues, I found 4 errors in the json doc (probably original; my fixes were augmented and committed by GB, I believe), and several in the re howto (due to non-update after re changes). There have been other error reports on the tracker and I presume more (other than missing '...') shown in AB's patch. (I just started with the first file changed in the patch.) So I thought it pretty well settled that getting doc examples correct (and test-ready) is be a good idea.

2. Examples with output always start with '>>> ' to differentiate input from output. I thought it pretty standard that continuation lines start with '... '. This is true of all applicable examples in re howto, module chapters for difflib, json, bisect, and 4 of 5 applicable examples in the Built-in Types chapter. Other chapters for binhex, itertools, and tkinter have no multiline interactive examples, so provide no precedent either way.

What is the alternative? The current sorting howto is not consistent:

For some examples, '... ' is just deleted:
>>> student_tuples = [
    ('john', 'A', 15),
    ('jane', 'B', 12),
    ('dave', 'B', 10),
]

For others, '... ' is replaced with '    ':
>>> class Student:
        def __init__(self, name, grade, age):
            self.name = name
            self.grade = grade
            self.age = age

The cmp_to_key function def is a special case: there is no output in the same box with the def and so (normally) should not have prompts. The only reason I followed AB's patch here and added them is because the definition is used in the next box. If the next example were changed for 3.2 to use the new-in-3.2 functools.cmp_to_key(), then the def would not be needed. (I already planned to suggest thisOr the followup example could just be marked SKIP. With either change, I would remove the prompts added to this function, which are the ones I presume bother you the most.

Summary: according to my current (and subject-to-update ;-) understanding of precedent and policy, the changes other than for cmp_to_key seem correct. The rationale for cmp_to_key changes could easily be negated, in which case they should be reverted.

I will put my thoughts on cut-and-paste in a separate message.

[rhettinger]

You're welcome to discuss this with me on IRC at some point. For now, the important thing is that I put a good deal of time and effort working on the sorting howto and I like it as it stands (your notions of precedent or consistency not withstanding).