IDLE: Shell calltips bugged by printed unclosed "(''' "

[SectorCorruptor]

When executing

help()

and giving an input of, say, ghi to the prompt, help terminates with an error.

However, the IDLE calltip for help changes from

(*args, **kwds)
Define the builtin 'help'

to

Import can't find module, or can't find name in module.

(This is after ending the help session and typing help()

What is weird is that if I do the same thing again, except that I provide an input of abc, as an example of a valid name, the calltip restores back to what it originally was.

Besides, the calltip also changes for other names. If I reproduce this issue, and then I type print(, the same calltip

Import can't find module, or can't find name in module.

comes.

Is this intended behaviour?

NOTE: This behaviour is not specific to help. See #105689 (comment).

My environment

• CPython versions tested on: 3.11.3
• Operating system and architecture: Windows 11

(Note: I wanted to attach a picture, but calltips go away as soon as I take a screenshot)

[terryjreedy]

The first bug, which is not an IDLE issue, is the regression of help() crashing upon invalid names. It should catch the exception and print stuff, including a new help prompt, as it used to do. #105702. The issue here is whether IDLE could be made immune to this or whether something like inspect, which IDLE uses, is messed up. So I will try to find out where the wrong tip is coming from.

[SectorCorruptor]

It seems to me that the first issue only appeared in 3.11 or so. Before that, it would say:

No Python documentation found for 'thing'.
Use help() to get the interactive help utility.
Use help(str) for help on the str class.

and continue.
Now, it's an error, and the prompt ends there.

[terryjreedy]

The discussion of the help bug is on the issue I opened and the issue it duplicated, #102541. There was a related issue just before 3.11.0 was released.

The wrong message line Import can't find module, or can't find name in module. comes from exceptions.c, 1674-9.

ComplexExtendsException(PyExc_Exception, ImportError,
                        ImportError, 0 /* new */,
                        ImportError_methods, ImportError_members,
                        0 /* getset */, ImportError_str,
                        "Import can't find module, or can't find name in "
                        "module.");

Among whatever else this call does, it makes the line the doc string of ImportError. Since ImportError has no accessible signature, the line is the calltip for ImportError. For some reason I will investigate another day, calltip.getargspec(ob) is being called with ob == ImportError, not with the function the user types.

[SectorCorruptor]

But why does that also affect every other object's calltip (like print's?)

[SectorCorruptor]

Also, I've found another bug.

What is weird is that if I do the same thing again, except that I provide an input of abc, as an example of a valid name, the calltip restores back to what it originally was.

Apparently, only some inputs will restore it back to its original value. Trying print does not change it back.

[terryjreedy]

But why does that also affect every other object's calltip (like print's?)

Because IDLE is displaying the calltip for ImportError regardless of what is typed. I am looking into this now. Directly causing ImportError does not do this. help is doing something strange.

[terryjreedy]

Progress: let m = help error line number with raise ImportError('''\ and n = user input line number with, for instance, print(. (Initially, n = m+5 because of the additional help error lines.) The latter invokes calltip.Calltip.open_calltip, which starts with

        hp = HyperParser(self.editwin, "insert")
        sur_paren = hp.get_surrounding_brackets('(')

sur_paren should be and normally is ('n.0', 'n.end') (with n replaced with the actual line number). But after the help error, it is instead ('m.21', 'n.end') (21 is the column with '('.) Hyperparser subsequently looks backward from the error '(' to find 'ImportError'. Even after more input and output, sur_paren continues to find 'm.21'. Print the error message from help() has somehow left Shell in an incorrect state.

The HyperParser initialization correctly sees 'insert' as referring to line n. The problem is with get_surrounding_brackets. This is exposed to users as Edit => Show Surrounding Parens (default hotkey ^0 (zero) as least on Windows). Normally, if one types help( and ^0, only the last '(' is highlighted. After the help error message that prints an unmatched '(', ^0 highlights back to that unmatched paren in the error output. There are really two bugs here: 1) looking for a '(' before the first one found, and 2) crossing the boundary between output and following input.

Debugging this appears to involve the separate pyparse module. A deep solution would be separate text widgets for code input and resulting output but I have only started a draft issue for this.

What may be a factor is that Shell treat responses to executed input('prompt') statements, as with help>, as code input -- colorizing and displaying calltips in response to '('s. This is a known bug but has so far merely been annoying.