bpo-36492: Deprecate passing some arguments as keyword arguments.

[serhiy-storchaka]

Deprecated passing the following arguments as keyword arguments:

• "func" in functools.partialmethod(), weakref.finalize(),
  profile.Profile.runcall(), cProfile.Profile.runcall(),
  bdb.Bdb.runcall(), trace.Trace.runfunc() and
  curses.wrapper().
• "function" in unittest.addModuleCleanup() and
  unittest.TestCase.addCleanup().
• "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor
  and concurrent.futures.ProcessPoolExecutor.
• "callback" in contextlib.ExitStack.callback(),
  contextlib.AsyncExitStack.callback() and
  contextlib.AsyncExitStack.push_async_callback().
• "c" and "typeid" in the create() method of multiprocessing.managers.Server
  and multiprocessing.managers.SharedMemoryServer.
• "obj" in weakref.finalize().

Also allowed to pass arbitrary keyword arguments (even "self" and "func")
if the above arguments are passed as positional argument.

https://bugs.python.org/issue36492

[gvanrossum]

This is cool! It shows how arbitrary and inconsistent argument names often are across similar functions -- 'fn', 'func', 'function', 'callback', and more.

[gvanrossum]

Did anyone review this? I just commented "This is cool ..." but had not read the whole diff nor Terry's comments. Terry is right that the help() output is inferior with the named argument removed, so for any public functions we should not do this. And for non-public functions there's no point in changing the code. @serhiy-storchaka Why did you commit this?

[serhiy-storchaka]

Oh, sorry. I thought you approved this.

Deterioration of the help() output is inevitable when fix this bug. But it is temporary until we implement PEP 570 or something like. #12620 will return named arguments back.

The deprecation is necessary to avoid the breakage from making parameters positional-only.

[gvanrossum]

Is there a way to add the “right” signature to the docstrings? That might help.

On Mon, Apr 1, 2019 at 8:10 AM Serhiy Storchaka ***@***.***> wrote: Oh, sorry. I thought you approved this. Deterioration of the help() output is inevitable when fix this bug. But it is temporary until we implement PEP 570 or something like. #12620 <#12620> will return named arguments back. The deprecation is necessary to avoid the breakage from making parameters positional-only. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12637 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMnk2Dpcesrl7EMK9orTHWfK6ksX-ks5vciFQgaJpZM4cUKvn> .

-- --Guido (mobile)

[serhiy-storchaka]

Is there a way to add the “right” signature to the docstrings?

Currently there is no a way to override the signature. Otherwise we would use it in other methods that use the *args hack. We can add a signature to the docstring, but it will result in having both signatures in the help() output. It is one of purposes of PEP 570 -- having clearer signatures in such cases.

We can patch inspect to use __text_signature__ for non-builtins, but PEP 570 (or alternate solutions) will make this hack unneeded.

[gvanrossum]

On Tue, Apr 2, 2019 at 4:21 AM Serhiy Storchaka ***@***.***> wrote: Is there a way to add the “right” signature to the docstrings? Currently there is no a way to *override* the signature. Otherwise we would use it in other methods that use the *args hack. We can add a signature to the docstring, but it will result in having both signatures in the help() output. It is one of purposes of PEP 570 -- having clearer signatures in such cases.

Hm, I was thinking of an old help() feature where if the first line of the docstring looks like a signature, help() would use that. But somehow that was thrown out. (Maybe it only worked for C functions?)

We can patch inspect to use __text_signature__ for non-builtins, but PEP 570 (or alternate solutions) will make this hack unneeded.

I could still imagine other uses of such a feature that PEP 570 wouldn't address. But then I agree with Terry that this is a regression. And the desire to deprecate these makes it impossible to use PEP 570 at first. Maybe you could come up with a decorator that allows keyword args with a deprecation warning? E.g. @deprecated_non_positional_arguments def foo(name, *args): return name, args

>> help(foo)

foo(name, /, *args)

>> foo("x")

('x', ())

>> foo(name="x")

DeprecationWarning: please don't pass 'name' as a keyword ('x', ())

…

>>

[terryjreedy]

Guido: "help() feature where if the first line of the docstring looks like a signature, help() would use that."

help(object) prints (still) str(inspect.signature(object)) if it exists, and then object.doc. (It used .argspec or .fullargspec previously.) The signature always exists for python-coded functions. It never did for C-coded functions until ArgClinic. So the convention was to put the signature in the first line(s) of the docstring of C-coded functions. Some C-coded functions have been converted, some not. For builtins, the issue is whether there is a comprehensible 1-line signature producible with .signature, as with "list(iterable=(), /)") or whether to stick with multiple lines in the docstring, as with int, bytes, and range.

IDLE tool tips include either .signature and the first line of the docstring or up to 5 lines of the docstring, up to a blank line, to accommodate bytes, with its 5 different 'signatures'.

[terryjreedy]

I wrote a decorator for

def foo(name, *args):
      return name, args`

based on Serhiy's patch for Lib/curses/__init__.py, def wrapper. Serhiy, did you intend to preserve the existing bug that foo(1, name='x') raises "TypeError: test() got multiple values for argument 'name'" instead of printing "'a', (1,)"? In any case, testing 'func' in kwds:` first fixes the bug in the process of deprecating the usage.

def positional_arg(name):
    def deco(func):
        def wrapper(*args, **kwds):
            if name in kwds:
                arg = kwds.pop(name)
                import warnings
                warnings.warn(
                        "Passing %s as keyword argument is deprecated" % name,
                        DeprecationWarning, stacklevel=2)
            elif args:
                arg, *args = args
            else:
                raise TypeError(f"{func.__name__} expected at least 1 positional"
                                f" argument, got 0")
            func(arg, *args, **kwds)
            
        wrapper.__wrapped__ = func
        wrapper.__name__ = func.__name__
        return wrapper
    return deco

@positional_arg('name')
def test(name, *args, **kwds):
    print(name, args, kwds)

print(help(test))
print('---')
test('abc', 1, 2, k='key')
test(1, 2, name='abc', k='key')
print()
test()

prints the following, with help giving the original function signature.

Help on function test in module __main__:

test(name, *args, **kwds)

None
---
abc (1, 2) {'k': 'key'}

Warning (from warnings module):
  File "F:\Python\a\tem3.py", line 29
    test(1, 2, name='abc', k='key')
DeprecationWarning: Passing name as keyword argument is deprecated
abc (1, 2) {'k': 'key'}

Traceback (most recent call last):
  File "F:\Python\a\tem3.py", line 30, in <module>
    test()
  File "F:\Python\a\tem3.py", line 13, in wrapper
    raise TypeError(f"{func.__name__} expected at least 1 positional"
TypeError: test expected at least 1 positional argument, got 0

[serhiy-storchaka]

I wrote a decorator for

It does not work this way. We want test('abc', 1, 2, name='def', k='key') to work and do not emit a warning.

Currently it is not possible to solve this issue with a decorator.

[gvanrossum]

This is an interesting corner case. I think it should be added to PEP 570.