open() returns IO[str] instead of io.TextIOWrapper

[jcrmatos]

Hello,

My env is Win7P+SP1 x64, Py 3.5.2 32b, Mypy 0.501.

I opened this issue (#3286) in mypy, but GVR told me to open it here because it was a "(complex) typeshed issue".

The var used in the with context is of type <class '_io.TextIOWrapper'>.

with open(pathname, encoding='utf-8') as f_in:
    print(type(f_in))

<class '_io.TextIOWrapper'>

But when I try to set it's type, mypy returns the following error

from io import TextIOWrapper
with open(pathname, encoding='utf-8') as f_in:  # type: TextIOWrapper
    print(type(f_in))

data_processing.py:917: error: Incompatible types in assignment (expression has type IO[Any], variable has type "TextIOWrapper")

If I try to use module typing's own TextIO type it returns

from typing import TextIO
with open(pathname, encoding='utf-8') as f_in:  # type: TextIO
    print(type(f_in))

data_processing.py:917: error: Incompatible types in assignment (expression has type IO[Any], variable has type "TextIO")

It only works if I do

from typing import IO
with open(pathname, encoding='utf-8') as f_in:  # type: IO[Any]
    print(type(f_in))

Best regards,

JM

[lincolnq]

The solution to your problem is probably to use IO[str], which is compatible with the IO[Any] returned by open() and has (most of) the right methods. It doesn't have the TextIO specific ones (like .encoding, .line_buffering, etc.), so if you want those you should use cast(TextIO, open(...)).

However, I agree that this is an interesting problem and worth thinking about. It means TextIO and BinaryIO are annoyingly impractical to use in real code (a cast is necessary whenever open() is called). The documentation doesn't give many examples or suggest that IO[str] is what most people should use.

[gvanrossum]

Yeah, this is not one of the smoothest APIs. Note that in master, open(fn, 'rb') will now return IO[bytes] while open(fn, 'r') will return IO[str]. (However, if the mode argument is a variable, it returns IO[Any]; and in PY2 they all return BinaryIO regardless of the mode.)

I think this is still wrong though. open() with a recognized mode argument should return BinaryIO or TextIO. That way you can write

f: TextIO = open(fn, 'rb')

Some classes used by the implementation in the io module are subclasses of TextIO or BinaryIO, but you're better off not referencing them.

(Another note: it's not great to use runtime inspection to come up with type names to put into annotations -- things at runtime are a bit different from how things are dine on the static typing side of things.)

[srittau]

I think any further improvements in typeshed for this are dependent on implementing dependent types, see python/mypy#3062.

[srittau]

This is part of the IO classes restructuring.

[hauntsaninja]

This was fixed in #4146 and others!