Inner method discards type narrowing done in outer method

[euresti]

So everybody knows not to write
def foo(arg={'bar': 'baz'}):

So instead one usually writes something like:

def foo(arg=None):
    if arg is None:
        arg = {'bar', 'baz'}

Now add some types:

def func(arg=None):
    # type: (Dict[str, str]) -> Dict[str,str]
    reveal_type(arg)  # Revealed type is 'Union[builtins.dict[builtins.str, builtins.str], builtins.None]'
    if arg is None:
        arg = {'foo': 'bar'}
    reveal_type(arg)  #  Revealed type is 'builtins.dict[builtins.str, builtins.str]'
    return arg

So all is good, unless you try to create an inner method:

def func(arg=None):
    # type: (Dict[str, str]) -> Dict[str,str]
    reveal_type(arg)  # Revealed type is 'Union[builtins.dict[builtins.str, builtins.str], builtins.None]'
    if arg is None:
        arg = {'foo': 'bar'}
    reveal_type(arg)  #  Revealed type is 'builtins.dict[builtins.str, builtins.str]'

    def inner():
        # type: () -> Dict[str, str]
        reveal_type(arg)  # Revealed type is 'Union[builtins.dict[builtins.str, builtins.str], builtins.None]'
        return arg  # Incompatible return value type (got "Optional[Dict[str, str]]", expected Dict[str, str])

    return inner()

I guess there are lots of workarounds (assign to temporary variable, pull out the inner method, pass arguments into the inner) but they all require editing existing code which is always scary. Hmm. I guess the easiest thing is to just add an assert arg is not None in the inner. That's not too intrusive, anyway thought you might want to know.

[gvanrossum]

This looks like the same as #2145 which was closed because the binder doesn't keep track of this -- maybe we should reopen that? Or at least keep this one open.

[JukkaL]

Let's keep this open.

[ilevkivskyi]

(Rising priority to normal, since this appeared third time.)

[gvanrossum]

How should we do this? I think it's safe to trust the narrower type *if it prevails over the entire rest of the (outer) function* -- maybe that's enough? After all the "None" is an artifact of the idiom for mutable default. (I almost wish there was a way to say "this None isn't really here" like we have at the class level.)

[carljm]

FWIW you can work around this by early-binding arg to the inner function as an argument default (def inner(a=arg):).

And yes, I think it’s safe to trust the narrowed type iff it applies to the entire rest of the outer function.

[ilevkivskyi]

And yes, I think it’s safe to trust the narrowed type iff it applies to the entire rest of the outer function.

+1 also from me.