Skip to content

Fix #2070 - bytes formatting incorrect in python 3 #2168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Sep 26, 2016
Merged

Fix #2070 - bytes formatting incorrect in python 3 #2168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
34ffd12
Add a code example for the typing.IO module
Sep 14, 2016
190a120
Merge branch 'master' of https://github.com/python/mypy
Sep 15, 2016
ca645f2
Merge branch 'master' of https://github.com/python/mypy
Sep 22, 2016
818adf7
Add a parallel path for bytes and str for checking % interpolation
Sep 22, 2016
76d6d71
Update the expected type of first arg to check_byte_interpolation
Sep 22, 2016
9920e9b
Refactor code to reduce duplication in changes
Sep 26, 2016
af5dc00
Reinstate type check for StrExpr, etc
Sep 26, 2016
3ce554c
Add byte interpolation unit test cases
Sep 26, 2016
b769a8f
Add TODO comments re bytes formatting functionality
Sep 26, 2016
1c1d18b
Fix two lint errors in checkstrformat
Sep 26, 2016
31c7412
Add a single Unicode python2 test
Sep 26, 2016
9a9b4ae
Clean up trailing comma.
gvanrossum Sep 26, 2016
41bce18
Add periods to sentences.
gvanrossum Sep 26, 2016
592e456
Don't assume.
gvanrossum Sep 26, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions mypy/checkexpr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -965,8 +965,9 @@ def visit_op_expr(self, e: OpExpr) -> Type:
if e.op == '*' and isinstance(e.left, ListExpr):
# Expressions of form [...] * e get special type inference.
return self.check_list_multiply(e)
if e.op == '%' and isinstance(e.left, (StrExpr, BytesExpr)):
return self.strfrm_checker.check_str_interpolation(cast(StrExpr, e.left), e.right)
if e.op == '%':
if isinstance(e.left, (StrExpr, BytesExpr, UnicodeExpr)):
return self.strfrm_checker.check_str_interpolation(e.left, e.right)
left_type = self.accept(e.left)

if e.op in nodes.op_methods:
Expand Down
21 changes: 17 additions & 4 deletions mypy/checkstrformat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

import re

from typing import cast, List, Tuple, Dict, Callable
from typing import cast, List, Tuple, Dict, Callable, Union

from mypy.types import (
Type, AnyType, TupleType, Instance, UnionType
)
from mypy.nodes import (
Node, StrExpr, BytesExpr, TupleExpr, DictExpr, Context
Node, StrExpr, BytesExpr, UnicodeExpr, TupleExpr, DictExpr, Context
)
if False:
# break import cycle only needed for mypy
Expand Down Expand Up @@ -55,7 +55,12 @@ def __init__(self,
self.exprchk = exprchk
self.msg = msg

def check_str_interpolation(self, str: StrExpr, replacements: Node) -> Type:
# TODO: In Python 3, the bytes formatting has a more restricted set of options
# compared to string formatting.
# TODO: Bytes formatting in Python 3 is only supported in 3.5 and up.
def check_str_interpolation(self,
str: Union[StrExpr, BytesExpr, UnicodeExpr],
replacements: Node) -> Type:
"""Check the types of the 'replacements' in a string interpolation
expression: str % replacements
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a TODO around here that in Python 3 the bytes format characters are actually more restricted than str/unicode format characters? (a) There are fewer supported characters; (b) %s only supports bytes. (In Python 2 these limitations do not apply.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also bytes formatting in Python 3 is only supported in 3.5 and up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the bytes formatting in 3.5 a TODO? Or is it something that I'll need to check for in code?

Expand All @@ -67,7 +72,15 @@ def check_str_interpolation(self, str: StrExpr, replacements: Node) -> Type:
self.check_mapping_str_interpolation(specifiers, replacements)
else:
self.check_simple_str_interpolation(specifiers, replacements)
return self.named_type('builtins.str')

if isinstance(str, BytesExpr):
return self.named_type('builtins.bytes')
elif isinstance(str, UnicodeExpr):
return self.named_type('builtins.unicode')
elif isinstance(str, StrExpr):
return self.named_type('builtins.str')
else:
assert False

def parse_conversion_specifiers(self, format: str) -> List[ConversionSpecifier]:
key_regex = r'(\(([^()]*)\))?' # (optional) parenthesised sequence of characters
Expand Down
13 changes: 13 additions & 0 deletions test-data/unit/check-expressions.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1111,6 +1111,19 @@ main:4: error: Incompatible types in string interpolation (expression has type "
[case testStringInterpolationSpaceKey]
'%( )s' % {' ': 'foo'}

[case testByteByteInterpolation]
def foo(a: bytes, b: bytes):
b'%s:%s' % (a, b)
foo(b'a', b'b') == b'a:b'

[case testBytePercentInterpolationSupported]
b'%s' % (b'xyz',)
b'%(name)s' % {'name': 'jane'}
b'%c' % (123)

[case testUnicodeInterpolation_python2]
u'%s' % (u'abc',)

-- Lambdas
-- -------

Expand Down