Access Violation when mixing scalar values and shared clones

[p5pRT]

Migrated from rt.perl.org#67902 (status was 'open')

Searchable as RT67902$

[p5pRT]

From esquel@sommarskog.se

Created by esquel@sommarskog.se

The script below may or may not be correct. That is, maybe it is not
intended that you should do things like this. However, the outcome
when running it on ActivePerl 1005 is that I get an Access Violation
on the last statement in the script. Either I should get an error
message telling me "you can't do that", or the assignment should
succeed.

...............................
use strict;
use threads;
use threads​::shared;

my $td : shared = shared_clone({});

warn "Write a string\n";
$td->{Input} = "somestring";

warn "Write an array reference\n";
$td->{Input} = shared_clone([]);

warn "Write a new string\n";
$td->{Input} = "shoestring";
.................................

This is a summary of the crash details I get from Dr Watson

Problem Event Name​: APPCRASH
  Application Name​: perl.exe
  Application Version​: 5.10.0.1005
  Application Timestamp​: 4a199ba1
  Fault Module Name​: msvcrt.dll
  Fault Module Version​: 7.0.6002.18005
  Fault Module Timestamp​: 49e04189
  Exception Code​: c0000005
  Exception Offset​: 000000000000114a
  OS Version​: 6.0.6002.2.2.0.256.1
  Locale ID​: 1053
  Additional Information 1​: c9f2
  Additional Information 2​: 37d41fa3f85b1e40f599f5dffe4ff997
  Additional Information 3​: def6
  Additional Information 4​: e54449e06bf3570e70d3b1010a27855a

Perl Info

Flags:
    category=library
    severity=medium

Site configuration information for perl 5.10.0:

Configured by SYSTEM at Sun May 24 12:17:14 2009.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='14.0.50727', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"F:\Perl\AS1005-x86\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"F:\Perl\AS1005-x86\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY
    f7bbab select() generates 'Invalid parameter' messages on Windows Vista.
    8dc00b fix buffer overflow in win32_select()
    36f064 do/require don't treat '.\foo' or '..\foo' as absolute paths on Windows
    287a96 fix -p function and Fcntl::S_IFIFO constant under Microsoft VC compiler
    406878 avoids segfaults invoking S_raise_signal() (on Linux)
    40c7cc Win32 process ids can have more than 16 bits
    37589e Load 'loadable object' with non-default file extension
    d374f9 64-bit fix for Time::Local


@INC for perl 5.10.0:
    F:\Utveckling\AbaPerls\Perl
    F:/Perl/AS1005-x86/site/lib
    F:/Perl/AS1005-x86/lib
    .


Environment for perl 5.10.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=F:\Perl\AS1005-x86\bin;C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\IDE;C:\Program Files (x86)\Microsoft Visual Studio 8\VC\BIN;C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\Tools;C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\Tools\bin;C:\Program Files (x86)\Microsoft Visual Studio 8\VC\PlatformSDK\bin;C:\Program Files (x86)\Microsoft Visual Studio 8\SDK\v2.0\bin;C:\Windows\Microsoft.NET\Framework\v2.0.50727;C:\Program Files (x86)\Microsoft Visual Studio 8\VC\VCPackages;F:\Perl\AS1002-AMD64\site\bin;F:\Perl\AS1002-AMD64\bin;F:\Utveckling\AbaPerls\Perl;F:\Utveckling\Diverse\;F:\Perl\AS1002-x86\site\bin;F:\Perl\AS1002-x86\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Program Files\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files\Microsoft SQL Server\100\Tools\binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\binn\;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\TextPad 4;C:\Program Files (x86)\Beyond Compare 3;C:\Program Files\Diskeeper Corporation\Diskeeper\;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\90\Tools\binn\;C:\Program Files (x86)\Microsoft SQL Server\90\Tools\binn\;C:\Program Files\Microsoft SQL Server\90\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\90\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft SQL Server\80\Tools\Binn\;C:\strawberry\c\bin;C:\strawberry\perl\bin
    PERLLIB=F:\Utveckling\AbaPerls\Perl
    PERL_BADLANG (unset)
    SHELL (unset)

[p5pRT]

From @jandubois

I can confirm that this bug still exists in maint-5.10.x. I'm attaching
a stack trace; don't have time right now to look into it further.

STACK_TEXT​:
0140f9cc 28072e28 00000000 0035c50c 0000000a MSVCRT!memmove+0x154
0140f9ec 000d1849 0034cff4 00e0400a 00000000
perl510!Perl_sv_setsv_flags+0x66f [..\sv.c @​ 3871]
0140fa2c 000d1e78 01c335bc 01c381cc 0036afdc
shared!sharedsv_scalar_store+0x120 [shared.xs @​ 767]
0140fa68 28014c07 00000005 01c381cc 00000005
shared!sharedsv_elem_mg_STORE+0x166 [shared.xs @​ 948]
0140fa8c 280428a5 00000014 01c381cc 00000001 perl510!Perl_mg_set+0x73
[..\mg.c @​ 293]
0140fabc 2806ecdd 01c335bc 01c335bc 28029b38
perl510!Perl_pp_sassign+0x1d4 [..\pp_hot.c @​ 212]
0140fac8 28029b38 01c335bc 01c335bc 01c315f8
perl510!Perl_runops_standard+0xc [..\run.c @​ 38]
0140fad8 280299f1 01c335bc 00000001 00000000 perl510!S_run_body+0xdb
[..\perl.c @​ 2431]
0140fb44 2809ef00 01c335bc 00000000 00000000 perl510!perl_run+0x179
[..\perl.c @​ 2349]
0140ff04 00401012 00000003 01c31118 01c31960 perl510!RunPerl+0xed
[perllib.c @​ 270]
0140ff14 004010f9 00000003 01c31118 01c31960 perl!main+0x12 [perlmain.c
@​ 22]
0140ff88 75cceccb 7efde000 0140ffd4 7773d24d perl!mainCRTStartup+0xe3
0140ff94 7773d24d 7efde000 458a770b 00000000
kernel32!BaseThreadInitThunk+0xe
0140ffd4 7773d45f 00401016 7efde000 00000000 ntdll!__RtlUserThreadStart+0x23
0140ffec 00000000 00401016 7efde000 00000000 ntdll!_RtlUserThreadStart+0x1b

[p5pRT]

@jandubois - Status changed from 'new' to 'open'

[p5pRT]

From @jandubois

Bug is not Windows specific; I can reproduce it on OS X too.

[p5pRT]

From @nwc10

On Sun, Jul 26, 2009 at 01​:14​:24PM -0700, Erland Sommarskog wrote​:

The script below may or may not be correct. That is, maybe it is not
intended that you should do things like this. However, the outcome
when running it on ActivePerl 1005 is that I get an Access Violation
on the last statement in the script. Either I should get an error
message telling me "you can't do that", or the assignment should
succeed.

Thanks for the report, with the clear test case. Yes, you're right, it should
either work or croak sweetly.

...............................
use strict;
use threads;
use threads​::shared;

my $td : shared = shared_clone({});

warn "Write a string\n";
$td->{Input} = "somestring";

warn "Write an array reference\n";
$td->{Input} = shared_clone([]);

warn "Write a new string\n";
$td->{Input} = "shoestring";
.................................

The bug is still present in blead, and I can replicate it on Linux.
valgrind shows it to be a NULL pointer dereference​:

==22776== Invalid write of size 1
==22776== at 0x4C23C84​: memmove (mc_replace_strmem.c​:517)
==22776== by 0x593E81​: Perl_sv_setsv_flags (sv.c​:4063)
==22776== by 0x66A2829​: sharedsv_scalar_store (shared.xs​:766)
==22776== by 0x66A4911​: sharedsv_elem_mg_STORE (shared.xs​:947)
==22776== by 0x520F0D​: Perl_mg_set (mg.c​:293)
==22776== by 0x550F65​: Perl_pp_sassign (pp_hot.c​:198)
==22776== by 0x50D14D​: Perl_runops_debug (dump.c​:2017)
==22776== by 0x450885​: S_run_body (perl.c​:2282)
==22776== by 0x44FBCB​: perl_run (perl.c​:2208)
==22776== by 0x421435​: main (perlmain.c​:117)
==22776== Address 0x0 is not stack'd, malloc'd or (recently) free'd

The call to memmove is the Move() macro here​:

  /* Failed the swipe test, and it's not a shared hash key either.
  Have to copy the string. */
  STRLEN len = SvCUR(sstr);
  SvGROW(dstr, len + 1); /* inlined from sv_setpvn */
  Move(SvPVX_const(sstr),SvPVX(dstr),len,char);
  SvCUR_set(dstr, len);
  *SvEND(dstr) = '\0';

SvPVX(dstr) is NULL - it's the "PV = 0" in this dump output​:

(gdb) call Perl_sv_dump(my_perl, dstr)
SV = PVNV(0xacae98) at 0xacbfd8
  REFCNT = 2
  FLAGS = (POK,pPOK)
  IV = 0
  NV = 0
  PV = 0

That shouldn't happen. That SV is malformed - it should have the flag pPOK set
if PV is 0. I don't know how it got to be that way, and I'm not in a position
to investigate further, but I suspect that solving that will resolve the bug.

I hope that someone else will pick it up from here.

Nicholas Clark

[p5pRT]

From vega.james@gmail.com

On Tue, Jul 28, 2009 at 12​:46​:44PM +0100, Nicholas Clark wrote​:

On Sun, Jul 26, 2009 at 01​:14​:24PM -0700, Erland Sommarskog wrote​:

...............................
use strict;
use threads;
use threads​::shared;

my $td : shared = shared_clone({});

warn "Write a string\n";
$td->{Input} = "somestring";

warn "Write an array reference\n";
$td->{Input} = shared_clone([]);

warn "Write a new string\n";
$td->{Input} = "shoestring";
.................................

The bug is still present in blead, and I can replicate it on Linux.
valgrind shows it to be a NULL pointer dereference​:

==22776== Invalid write of size 1
==22776== at 0x4C23C84​: memmove (mc_replace_strmem.c​:517)
==22776== by 0x593E81​: Perl_sv_setsv_flags (sv.c​:4063)
==22776== by 0x66A2829​: sharedsv_scalar_store (shared.xs​:766)
==22776== by 0x66A4911​: sharedsv_elem_mg_STORE (shared.xs​:947)
==22776== by 0x520F0D​: Perl_mg_set (mg.c​:293)
==22776== by 0x550F65​: Perl_pp_sassign (pp_hot.c​:198)
==22776== by 0x50D14D​: Perl_runops_debug (dump.c​:2017)
==22776== by 0x450885​: S_run_body (perl.c​:2282)
==22776== by 0x44FBCB​: perl_run (perl.c​:2208)
==22776== by 0x421435​: main (perlmain.c​:117)
==22776== Address 0x0 is not stack'd, malloc'd or (recently) free'd

The call to memmove is the Move() macro here​:

        /\* Failed the swipe test\, and it's not a shared hash key either\.
           Have to copy the string\.  \*/
    STRLEN len = SvCUR\(sstr\);
        SvGROW\(dstr\, len \+ 1\);    /\* inlined from sv\_setpvn \*/
        Move\(SvPVX\_const\(sstr\)\,SvPVX\(dstr\)\,len\,char\);
        SvCUR\_set\(dstr\, len\);
        \*SvEND\(dstr\) = '\\0';

SvPVX(dstr) is NULL - it's the "PV = 0" in this dump output​:

(gdb) call Perl_sv_dump(my_perl, dstr)
SV = PVNV(0xacae98) at 0xacbfd8
REFCNT = 2
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0

At $work, we have been seeing crashes that look like this when using
5.16. Doing some more testing, and a bisect, the specific crash in
this bug appears to have been fixed by e8c6a47, released in 5.20.0.

Hopefully I'll be able to say the same about the crash at work if we
switch to 5.20.

Cheers,
--
James
GPG Key​: 4096R/331BA3DB 2011-12-05 James McCoy <vega.james@​gmail.com>

[p5pRT]

From @dcollinsn

On Fri Jul 10 20​:01​:12 2015, vega.james@​gmail.com wrote​:

On Tue, Jul 28, 2009 at 12​:46​:44PM +0100, Nicholas Clark wrote​:

On Sun, Jul 26, 2009 at 01​:14​:24PM -0700, Erland Sommarskog wrote​:

...............................
use strict;
use threads;
use threads​::shared;

my $td : shared = shared_clone({});

warn "Write a string\n";
$td->{Input} = "somestring";

warn "Write an array reference\n";
$td->{Input} = shared_clone([]);

warn "Write a new string\n";
$td->{Input} = "shoestring";
.................................

The bug is still present in blead, and I can replicate it on Linux.
valgrind shows it to be a NULL pointer dereference​:

==22776== Invalid write of size 1
==22776== at 0x4C23C84​: memmove (mc_replace_strmem.c​:517)
==22776== by 0x593E81​: Perl_sv_setsv_flags (sv.c​:4063)
==22776== by 0x66A2829​: sharedsv_scalar_store (shared.xs​:766)
==22776== by 0x66A4911​: sharedsv_elem_mg_STORE (shared.xs​:947)
==22776== by 0x520F0D​: Perl_mg_set (mg.c​:293)
==22776== by 0x550F65​: Perl_pp_sassign (pp_hot.c​:198)
==22776== by 0x50D14D​: Perl_runops_debug (dump.c​:2017)
==22776== by 0x450885​: S_run_body (perl.c​:2282)
==22776== by 0x44FBCB​: perl_run (perl.c​:2208)
==22776== by 0x421435​: main (perlmain.c​:117)
==22776== Address 0x0 is not stack'd, malloc'd or (recently) free'd

The call to memmove is the Move() macro here​:

/* Failed the swipe test, and it's not a shared hash key either.
Have to copy the string. */
STRLEN len = SvCUR(sstr);
SvGROW(dstr, len + 1); /* inlined from sv_setpvn */
Move(SvPVX_const(sstr),SvPVX(dstr),len,char);
SvCUR_set(dstr, len);
*SvEND(dstr) = '\0';

SvPVX(dstr) is NULL - it's the "PV = 0" in this dump output​:

(gdb) call Perl_sv_dump(my_perl, dstr)
SV = PVNV(0xacae98) at 0xacbfd8
REFCNT = 2
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0

At $work, we have been seeing crashes that look like this when using
5.16. Doing some more testing, and a bisect, the specific crash in
this bug appears to have been fixed by e8c6a47, released in 5.20.0.

Hopefully I'll be able to say the same about the crash at work if we
switch to 5.20.

Cheers,

This was definitely not fixed, at least, not across all platforms. in 5.25.2, valgrind gives the following​:

Write a string
Write an array reference
Write a new string
==43648== Invalid write of size 8
==43648== at 0x4C2D04B​: memcpy@​GLIBC_2.2.5 (vg_replace_strmem.c​:1017)
==43648== by 0x4F2820​: Perl_sv_setsv_flags (sv.c​:4726)
==43648== by 0x6760A3A​: sharedsv_scalar_store (shared.xs​:792)
==43648== by 0x6761E6F​: sharedsv_elem_mg_STORE (shared.xs​:983)
==43648== by 0x4C02A5​: Perl_mg_set (mg.c​:277)
==43648== by 0x4DCB22​: Perl_pp_sassign (pp_hot.c​:226)
==43648== by 0x4DC2F5​: Perl_runops_standard (run.c​:41)
==43648== by 0x44B3FC​: S_run_body (perl.c​:2521)
==43648== by 0x44B3FC​: perl_run (perl.c​:2444)
==43648== by 0x41D0BA​: main (perlmain.c​:123)
==43648== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==43648==
==43648==
==43648== Process terminating with default action of signal 11 (SIGSEGV)
==43648== Access not within mapped region at address 0x0
==43648== at 0x4C2D04B​: memcpy@​GLIBC_2.2.5 (vg_replace_strmem.c​:1017)
==43648== by 0x4F2820​: Perl_sv_setsv_flags (sv.c​:4726)
==43648== by 0x6760A3A​: sharedsv_scalar_store (shared.xs​:792)
==43648== by 0x6761E6F​: sharedsv_elem_mg_STORE (shared.xs​:983)
==43648== by 0x4C02A5​: Perl_mg_set (mg.c​:277)
==43648== by 0x4DCB22​: Perl_pp_sassign (pp_hot.c​:226)
==43648== by 0x4DC2F5​: Perl_runops_standard (run.c​:41)
==43648== by 0x44B3FC​: S_run_body (perl.c​:2521)
==43648== by 0x44B3FC​: perl_run (perl.c​:2444)
==43648== by 0x41D0BA​: main (perlmain.c​:123)
==43648== If you believe this happened as a result of a stack
==43648== overflow in your program's main thread (unlikely but
==43648== possible), you can try to increase the size of the
==43648== main thread stack using the --main-stacksize= flag.
==43648== The main thread stack size used in this run was 8388608.
==43648==
==43648== HEAP SUMMARY​:
==43648== in use at exit​: 1,216,776 bytes in 4,331 blocks
==43648== total heap usage​: 11,907 allocs, 7,576 frees, 2,466,846 bytes allocated
==43648==
==43648== LEAK SUMMARY​:
==43648== definitely lost​: 12 bytes in 1 blocks
==43648== indirectly lost​: 0 bytes in 0 blocks
==43648== possibly lost​: 577,798 bytes in 900 blocks
==43648== still reachable​: 638,966 bytes in 3,430 blocks
==43648== of which reachable via heuristic​:
==43648== newarray : 2,344 bytes in 74 blocks
==43648== suppressed​: 0 bytes in 0 blocks
==43648== Rerun with --leak-check=full to see details of leaked memory
==43648==
==43648== For counts of detected and suppressed errors, rerun with​: -v
==43648== ERROR SUMMARY​: 1 errors from 1 contexts (suppressed​: 0 from 0)
Segmentation fault

--
Respectfully,
Dan Collins