hashlib segmentation fault

[httpsgithubcomxcainiao]

BPO	34922
Nosy	@terryjreedy, @vstinner, @tiran, @ned-deily, @serhiy-storchaka, @tirkarthi, @https://github.com/xcainiao
PRs	bpo-34922: Fixed integer overflow in the digest() and hexdigest() methods #9751 [3.6] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) #9797 [3.7] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) #9798 [3.6] bpo-34922: Fix integer overflow in the digest() and hexdigest() methods (GH-9751) (GH-9798) #9801

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-14.22:56:24.735>
created_at = <Date 2018-10-07.12:40:29.942>
labels = ['extension-modules', '3.7', '3.8', 'type-crash']
title = 'hashlib segmentation fault'
updated_at = <Date 2020-01-14.22:56:24.734>
user = 'https://github.com/httpsgithubcomxcainiao'

bugs.python.org fields:

activity = <Date 2020-01-14.22:56:24.734>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-14.22:56:24.735>
closer = 'ned.deily'
components = ['Extension Modules']
creation = <Date 2018-10-07.12:40:29.942>
creator = 'shuoz'
dependencies = []
files = []
hgrepos = []
issue_num = 34922
keywords = ['patch']
message_count = 16.0
messages = ['327277', '327283', '327285', '327288', '327294', '327306', '327312', '327495', '327515', '327518', '327519', '327604', '327606', '327619', '327677', '360009']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'vstinner', 'christian.heimes', 'ned.deily', 'serhiy.storchaka', 'xtreak', 'shuoz']
pr_nums = ['9751', '9797', '9798', '9801']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue34922'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[httpsgithubcomxcainiao]

python hashlib a signd overflow maybe cause a memory over read.

python version:
Python 3.6.7rc1+ (heads/3.6:cb0bec3, Oct 1 2018, 02:19:39)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

[----------------------------------registers-----------------------------------]
RAX: 0x0
RBX: 0x7fffffffd5f0 --> 0x41b58ab3
RCX: 0x0
RDX: 0x1ffffffffffffff6
RSI: 0x7ffff35ae880 --> 0x0
RDI: 0x7fffffffd650 --> 0x7d828fe8a42b9c7f
RBP: 0xffffffffabe --> 0x0
RSP: 0x7fffffffd5c8 --> 0x7ffff2a5f793 (<_sha3_shake_128_hexdigest+627>:	test   eax,eax)
RIP: 0x7ffff2a5ec60 (<_PySHA3_KeccakWidth1600_SpongeSqueeze>:	push   r15)
R8 : 0x65fc7ba985946aff
R9 : 0xefbdaa140b587a16
R10: 0x50573373c9b2b8dc
R11: 0xfba4d93abbdabffc
R12: 0x7fffffffd770 --> 0x7fffffffd7d0 --> 0xffffffffb00 --> 0x0
R13: 0x7fffffffd650 --> 0x7d828fe8a42b9c7f
R14: 0x7ffff35ae880 --> 0x0
R15: 0xfffffffffffffff6
EFLAGS: 0xa06 (carry PARITY adjust zero sign trap INTERRUPT direction OVERFLOW)
[-------------------------------------code-------------------------------------]
   0x7ffff2a5ec50 <_PySHA3_KeccakP1600_ExtractBytes+160>:	jmp    0x7ffff2a54d10 <_PySHA3_KeccakP1600_ExtractBytesInLane@plt>
   0x7ffff2a5ec55:	nop
   0x7ffff2a5ec56:	nop    WORD PTR cs:[rax+rax*1+0x0]
=> 0x7ffff2a5ec60 <_PySHA3_KeccakWidth1600_SpongeSqueeze>:	push   r15
   0x7ffff2a5ec62 <_PySHA3_KeccakWidth1600_SpongeSqueeze+2>:	push   r14
   0x7ffff2a5ec64 <_PySHA3_KeccakWidth1600_SpongeSqueeze+4>:	push   r13
   0x7ffff2a5ec66 <_PySHA3_KeccakWidth1600_SpongeSqueeze+6>:	push   r12
   0x7ffff2a5ec68 <_PySHA3_KeccakWidth1600_SpongeSqueeze+8>:	mov    r13,rdx
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffd5c8 --> 0x7ffff2a5f793 (<_sha3_shake_128_hexdigest+627>:	test   eax,eax)
0008| 0x7fffffffd5d0 --> 0x7fffffffd5f0 --> 0x41b58ab3
0016| 0x7fffffffd5d8 --> 0xffffefdb33b --> 0x0
0024| 0x7fffffffd5e0 --> 0x7ffff7ed99d8 --> 0x0
0032| 0x7fffffffd5e8 --> 0x7ffff3606910 --> 0x6190000096e5 --> 0x9000009828000000
0040| 0x7fffffffd5f0 --> 0x41b58ab3
0048| 0x7fffffffd5f8 --> 0x7ffff2a68c08 ("2 32 8 6 length 96 224 4 temp ")
0056| 0x7fffffffd600 --> 0x7ffff2a5f520 (<_sha3_shake_128_hexdigest>:	push   r15)
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value

Breakpoint 2, _PySHA3_KeccakWidth1600_SpongeSqueeze (instance=0x7fffffffd650, data=0x7ffff35ae880 "", dataByteLen=0x1ffffffffffffff6) at /home/test/cpython/Modules/_sha3/kcp/KeccakSponge.inc:272
``````py
dataByteLen=0x1ffffffffffffff6

RAX: 0x7ffff3615f90 --> 0xfffffffffffffffa
RBX: 0xa8
RCX: 0x7ffff3616028 --> 0xf938000001a4
RDX: 0x18
RSI: 0x7fffffffd6e0 --> 0x6ab2a5fe4fe8efd
RDI: 0x7ffff3615fe0 --> 0x44b6a41dfdc1a3df
RBP: 0x7fffffffd510 --> 0xa8
RSP: 0x7fffffffcc78 --> 0x7ffff6e936cf (mov    rcx,QWORD PTR [rbp-0x38])
RIP: 0x7ffff6120786 (<__memmove_sse2_unaligned_erms+614>:	movntdq XMMWORD PTR [rdi+0x20],xmm2)
R8 : 0xfffffffffffffff0
R9 : 0x10007e6bac07 --> 0x0
R10: 0x7ffff3616038 --> 0x0
R11: 0x7ffff3615f90 --> 0xfffffffffffffffa
R12: 0x7ffff3615f90 --> 0xfffffffffffffffa
R13: 0x7fffffffd650 --> 0xa35bf3e9cd13e78e
R14: 0x7ffff3615f90 --> 0xfffffffffffffffa
R15: 0x0
EFLAGS: 0x10206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x7ffff6120779 <__memmove_sse2_unaligned_erms+601>:	sub    rdx,0x40
   0x7ffff612077d <__memmove_sse2_unaligned_erms+605>:	movntdq XMMWORD PTR [rdi],xmm0
   0x7ffff6120781 <__memmove_sse2_unaligned_erms+609>:	movntdq XMMWORD PTR [rdi+0x10],xmm1
=> 0x7ffff6120786 <__memmove_sse2_unaligned_erms+614>:	movntdq XMMWORD PTR [rdi+0x20],xmm2
   0x7ffff612078b <__memmove_sse2_unaligned_erms+619>:	movntdq XMMWORD PTR [rdi+0x30],xmm3
   0x7ffff6120790 <__memmove_sse2_unaligned_erms+624>:	add    rdi,0x40
   0x7ffff6120794 <__memmove_sse2_unaligned_erms+628>:	cmp    rdx,0x40
   0x7ffff6120798 <__memmove_sse2_unaligned_erms+632>:	ja     0x7ffff6120758 <__memmove_sse2_unaligned_erms+568>
[------------------------------------stack-------------------------------------]
0000| 0x7fffffffcc78 --> 0x7ffff6e936cf (mov    rcx,QWORD PTR [rbp-0x38])
0008| 0x7fffffffcc80 --> 0x7fffffffccf0 --> 0x41b58ab3
0016| 0x7fffffffcc88 --> 0x7fffffffcd90 --> 0x6
0024| 0x7fffffffcc90 --> 0xffffffff99e --> 0x0
0032| 0x7fffffffcc98 --> 0x7fffffffcd50 --> 0x0
0040| 0x7fffffffcca0 --> 0x0
0048| 0x7fffffffcca8 --> 0x7ffff3616038 --> 0x0
0056| 0x7fffffffccb0 --> 0x7ffff358a068 --> 0x1
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
__memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:492
492	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
gdb-peda$ bt
#0  __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:492
#1  0x00007ffff6e936cf in ?? () from /usr/lib/x86_64-linux-gnu/libasan.so.4
#2  0x00007ffff2a5eab4 in memcpy (__len=0xa8, __src=<optimized out>, __dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34
#3  _PySHA3_KeccakP1600_ExtractLanes (state=<optimized out>, data=<optimized out>, laneCount=0x15) at /home/test/cpython/Modules/_sha3/kcp/KeccakP-1600-opt64.c:342
#4  0x00007ffff2a5ec2c in _PySHA3_KeccakP1600_ExtractBytes (state=0x7fffffffd650, data=0x7ffff3615f90 "\372\377\377\377\377\377\377\377\002", offset=<optimized out>, length=0xa8)
    at /home/test/cpython/Modules/_sha3/kcp/KeccakP-1600-opt64.c:375
#5  0x00007ffff2a5ee1d in _PySHA3_KeccakWidth1600_SpongeSqueeze (instance=0x7fffffffd650, data=<optimized out>, dataByteLen=0x1ffffffffffffff6)
    at /home/test/cpython/Modules/_sha3/kcp/KeccakSponge.inc:287
#6  0x00007ffff2a5f793 in _SHAKE_digest (hex=0x1, digestlen=0xfffffffffffffff6, self=0x7ffff7ed98e8) at /home/test/cpython/Modules/_sha3/sha3module.c:620
#7  _sha3_shake_128_hexdigest_impl (length=0xfffffffffffffff6, self=0x7ffff7ed98e8) at /home/test/cpython/Modules/_sha3/sha3module.c:669
#8  _sha3_shake_128_hexdigest (self=0x7ffff7ed98e8, args=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>) at /home/test/cpython/Modules/_sha3/clinic/sha3module.c.h:149
#9  0x000055555583eab6 in _PyCFunction_FastCallDict (kwargs=0x0, nargs=0x1, args=0x616000021518, func_obj=0x7ffff2e86f30) at Objects/methodobject.c:250
#10 _PyCFunction_FastCallKeywords (func=func@entry=0x7ffff2e86f30, stack=0x616000021518, nargs=nargs@entry=0x1, kwnames=kwnames@entry=0x0) at Objects/methodobject.c:294
#11 0x0000555555995945 in call_function (pp_stack=pp_stack@entry=0x7fffffffdc30, oparg=oparg@entry=0x1, kwnames=kwnames@entry=0x0) at Python/ceval.c:4837
#12 0x000055555599feaa in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3335
#13 0x0000555555994939 in PyEval_EvalFrameEx (throwflag=0x0, f=0x616000021398) at Python/ceval.c:754
#14 _PyEval_EvalCodeWithName (_co=_co@entry=0x7ffff36088a0, globals=globals@entry=0x0, locals=locals@entry=0x7ffff355a9d8, args=args@entry=0x0, argcount=argcount@entry=0x0, kwnames=kwnames@entry=0x0,
    kwargs=0x0, kwcount=0x0, kwstep=0x2, defs=0x0, defcount=0x0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4166
#15 0x0000555555997b73 in PyEval_EvalCodeEx (closure=0x0, kwdefs=0x0, defcount=0x0, defs=0x0, kwcount=0x0, kws=0x0, argcount=0x0, args=0x0, locals=locals@entry=0x7ffff355a9d8, globals=globals@entry=0x0,
    _co=_co@entry=0x7ffff36088a0) at Python/ceval.c:4187
#16 PyEval_EvalCode (co=co@entry=0x7ffff36088a0, globals=globals@entry=0x7ffff7e5a318, locals=locals@entry=0x7ffff7e5a318) at Python/ceval.c:731
#17 0x00005555556b5b3b in run_mod (arena=0x7ffff7e75150, flags=<optimized out>, locals=0x7ffff7e5a318, globals=0x7ffff7e5a318, filename=0x7ffff358d270, mod=0x62500001e300) at Python/pythonrun.c:1025
#18 PyRun_FileExFlags (fp=<optimized out>, filename_str=<optimized out>, start=<optimized out>, globals=<optimized out>, locals=<optimized out>, closeit=<optimized out>, flags=<optimized out>)
    at Python/pythonrun.c:978
#19 0x00005555556b5fdc in PyRun_SimpleFileExFlags (fp=<optimized out>,
    filename=0x7ffff35c2680 "\314\070\064\302\227\a\254\bJf\331u\230N\273\022\355@\200\352\024`z[\267&\257+\022Q\324\017\310\nSyF2+\001{\327\354\355\245\275\002\064d-\235x\\\327O\230٧\036ތF\222\326\336\060\027q\220\037\217\b\364#=\366\224,\362\355\224i4h\030.c\377\225\360.׀M\033\066\251\ve'M=\261\t\365\307\016\267\203Q\316\313n\251]+\351H\222\244\266{\224FG\257\022\340\071\233r\300\220\065\031\236][\266\v\027\071#\354Ɣ\310\\\243M\243\251\250\372_\362^Φ\306ڝ\222\365\062O1nY\224pĥ\243IV\364\070\356\232\\\222z\242\321\v\027|\342\027\325\325O֬\300\252a0\250"..., closeit=0x1, flags=<optimized out>)
    at Python/pythonrun.c:419
#20 0x00005555556f2704 in run_file (p_cf=0x7fffffffe2b0, filename=0x604000000010 L"crash.py", fp=0x616000034880) at Modules/main.c:340
#21 Py_Main (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:810
#22 0x000055555569a293 in main (argc=argc@entry=0x2, argv=argv@entry=0x7fffffffe528) at ./Programs/python.c:69
#23 0x00007ffff6086b97 in __libc_start_main (main=0x55555569a050 <main>, argc=0x2, argv=0x7fffffffe528, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe518)
    at ../csu/libc-start.c:310
#24 0x000055555569bb2a in _start ()

x.py

import hashlib
hashlib.shake_128().hexdigest(-10)

[ned-deily]

See also bpo-33729. We need this addressed for 3.6.7.

[tirkarthi]

Thanks for the report. Interesting, this is not reproducible on master and latest 3.7 branches though both have different errors but reproducible in latest 3.6 and v3.7.0 . As Ned noted this seems to have been fixed with bpo-33729 but still there is no decision on reverting/keeping the commits made with the linked issue.

# master

./python.exe
Python 3.8.0a0 (heads/master:7dfbd49671, Oct  7 2018, 16:00:31)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.shake_128().hexdigest(-10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: value must be positive

# upstream/3.7

./python.exe
Python 3.7.1rc1+ (remotes/upstream/3.7:3b699932e5, Oct  7 2018, 21:44:03)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.shake_128().hexdigest(-10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: can't convert negative value to unsigned int

# 3.7.0 segfaults

./python.exe
Python 3.7.0 (tags/v3.7.0:1bf9cc5093, Oct  7 2018, 21:51:43)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.shake_128().hexdigest(-10)
[1]    67585 bus error  ./python.exe

# upstream/3.6 segfaults

./python.exe
Python 3.6.7rc1+ (remotes/upstream/3.6:177254c96f, Oct  7 2018, 21:42:19)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.shake_128().hexdigest(-10)
[1]    49096 bus error  ./python.exe

Thanks

[tirkarthi]

Sorry Ned, my comment seems to have changed the priority while submitting the comment. I would also propose adding the attached report as a unit test.

[ned-deily]

No problem; that's something to watch out for when you get an update conflict message from the bug tracker! Regarding this issue, I believe Serhiy is going to do a PR but perhaps you can work with him on providing the test case.