Fast parser doesn't know the location of starargs in function calls

[ddfisher]

In Python 2, the AST for function calls looks like this:

Call(expr func, expr* args, keyword* keywords, expr? starargs, expr? kwargs)

Keyword arguments are allowed to come either before or after starargs, but unfortunately this distinction is not made clear in the Call node. Therefore, when displaying errors about function calls with starargs and keyword arguments, we sometimes give an incorrect argument number.

[gvanrossum]

Aren't you confusing Python 2 and 3? In Python 2 you cannot have a keyword arg after *args. This is a syntax error in Python 2:

def f(a=1, *v, b=2):
    pass

It is only allowed in Python 3.

[ddfisher]

I should've given an example. This is for function calls, not definitions. E.g.

f(x=0, *[0])

vs

f(*[0], x=0)

[ddfisher]

That said, I thought this was a Python 2-specific problem, but I was wrong. The relevant Python 3 AST is:

Call(expr func, expr* args, keyword* keywords)

starargs is now part of args, but this is still an issue, because it doesn't give you any indication of the ordering with respect to keywords.

[gvanrossum]

Well, that's because they mean the same thing at runtime. Try disassembling the generated code -- it's identical.

>>> def f():
...  g(x=0, *[0])
...  g(*[0], x=0)
... 
>>> dis.dis(f)
  2           0 LOAD_GLOBAL              0 (g)
              3 LOAD_CONST               1 ('x')
              6 LOAD_CONST               2 (0)
              9 LOAD_CONST               2 (0)
             12 BUILD_LIST               1
             15 CALL_FUNCTION_VAR      256
             18 POP_TOP             

  3          19 LOAD_GLOBAL              0 (g)
             22 LOAD_CONST               1 ('x')
             25 LOAD_CONST               2 (0)
             28 LOAD_CONST               2 (0)
             31 BUILD_LIST               1
             34 CALL_FUNCTION_VAR      256
             37 POP_TOP             
             38 LOAD_CONST               0 (None)
             41 RETURN_VALUE        
>>> 

[ddfisher]

Sure. The only issue is we can't refer to these arguments by number. Take this testcase, for example:

def f(x: int, y: str) -> None: pass
  f(x=1, *[2])

Part of the expected output is:

main:2: error: Argument 2 to "f" has incompatible type *List[int]; expected "str"

... but the fast parser can't know that's "Argument 2" as opposed to, say, "Argument 1".

The solution is probably to refer to these arguments a different way. Maybe non-starargs positional arguments by position (which we know), keyword arguments by name, and starargs and kwargs by "starargs" and "kwargs" or similar.

[gvanrossum]

Yeah, keyword args should really be referenced by name anyway, since it's ambiguous whether the numbering is about the call or about the definition. Regarding how to refer to *args and **kwds, I don't particularly like the terms "starargs" and "kwargs", maybe we can use the name from the definition with 1 or two "*"s as a prefix?

[ddfisher]

The definition doesn't necessarily have a *args or **kwds, though...

[gvanrossum]

Argh. Then "*args" and "**kwds" sound like the best alternative to me.

[ddfisher]

I think you mean "*Argh". :P

Agreed. That sounds reasonable to me too.

[hauntsaninja]

mypy gives a reasonable error here:

main.py:2: error: "f" gets multiple values for keyword argument "x"
main.py:2: error: Argument 1 to "f" has incompatible type "*List[int]"; expected "str"