Perl crash when run under AppVerifier #12986
Comments
From karthik.rajagopalan@schrodinger.comHi Folks, We use perl widely for one of our job monitoring application. And recently use IO::Socket::INET; # Pass an open handle of tempfile to child process. print "here\n"; Exactly after notepad.exe process is spawned we see the crash with -Karthik 0:000> !analyze -v *** WARNING: Unable to verify checksum for FAULTING_IP: EXCEPTION_RECORD: 000000000108eaa0 -- (.exr 0x108eaa0) FAULTING_THREAD: 0000000000000ae8 DEFAULT_BUCKET_ID: STATUS_BREAKPOINT PROCESS_NAME: perl.exe CONTEXT: 000000000108e470 -- (.cxr 0x108e470) BAD_HANDLE: 0000000000000003 (!htrace 0000000000000003) ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments EXCEPTION_PARAMETER1: 0000000000000000 MOD_LIST: <ANALYSIS/> NTGLOBALFLAG: 100 APPLICATION_VERIFIER_FLAGS: 80000004 PRIMARY_PROBLEM_CLASS: STATUS_BREAKPOINT BUGCHECK_STR: APPLICATION_FAULT_STATUS_BREAKPOINT LAST_CONTROL_TRANSFER: from 000007fef56578dd to 000007fef76637ed STACK_TEXT: FOLLOWUP_IP: SYMBOL_STACK_INDEX: 7 SYMBOL_NAME: mswsock!SockImportHandle+108 FOLLOWUP_NAME: MachineOwner MODULE_NAME: mswsock IMAGE_NAME: mswsock.dll DEBUG_FLR_IMAGE_TIMESTAMP: 4a5bdfc4 STACK_COMMAND: ~0s ; kb FAILURE_BUCKET_ID: STATUS_BREAKPOINT_80000003_mswsock.dll!SockImportHandle BUCKET_ID: Followup: MachineOwner |
From @bulk88On Wed May 22 13:57:37 2013, kartlee05 wrote:
NT Kernel does not throw exceptions AKA crashes AKA SEH for What my first guess is that this is a double free-ing of a socket 5.14 I think is now out of support since 5.18 was released a few days -- |
|
The RT System itself - Status changed from 'new' to 'open' |
From karthik.rajagopalan@schrodinger.comHi, Thanks for your reply. I don't think debugging flag is turned on at -Karthik 0:000> !analyze -v *** ERROR: Symbol file could not be found. Defaulted to export symbols for FAULTING_IP: EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff) DEFAULT_BUCKET_ID: INVALID_POINTER_READ PROCESS_NAME: perl.exe ERROR_CODE: (NTSTATUS) 0xc0000008 - An invalid HANDLE was specified. EXCEPTION_CODE: (NTSTATUS) 0xc0000008 - An invalid HANDLE was specified. MOD_LIST: <ANALYSIS/> NTGLOBALFLAG: 0 APPLICATION_VERIFIER_FLAGS: 0 FAULTING_THREAD: 0000000000001554 PRIMARY_PROBLEM_CLASS: INVALID_POINTER_READ BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ LAST_CONTROL_TRANSFER: from 0000000076da8e59 to 0000000076e0d7d8 STACK_TEXT: STACK_COMMAND: ~0s; .ecxr ; kb FOLLOWUP_IP: SYMBOL_STACK_INDEX: 6 SYMBOL_NAME: uxtheme!_delayLoadHelper2+96 FOLLOWUP_NAME: MachineOwner MODULE_NAME: uxtheme IMAGE_NAME: uxtheme.dll DEBUG_FLR_IMAGE_TIMESTAMP: 4a5be093 FAILURE_BUCKET_ID: BUCKET_ID: WATSON_STAGEONE_URL: Followup: MachineOwner On Wed, May 22, 2013 at 11:04 PM, bulk88 via RT
|
From @bulk88On Thu Jun 06 11:27:32 2013, kartlee05 wrote:
Use Dependency Walker since something went wrong with loading a DLL. I'm Googling STATUS_INVALID_HANDLE and loadlibraryex gave me -- |
From @bulk88I think I found another report On Server 2003, (not the NT 6 the OP is using), I tried a
|
From @steve-m-hayFor the record, I can reproduce this, debugging blead perl in VC++ 2010 with AppVerifier running. My call stack is: ntdll.dll!77b1f921()
|
This comment has been minimized.
This comment has been minimized.
From @steve-m-hayOn Wed Oct 30 01:49:04 2013, shay wrote:
I can also still reproduce this, with the same call stack, with the patch from #13328 (which is related but doesn't claim to fix this) applied. |
From @bulk88I'll add this here for archival reasons/google pickup. People using Procmon (MS/Sysinternals Process Monitor) on a Perl process, may sometimes see a couple calls in sequence to NtDeviceIOControlFile with IOCTL 0x12043. The exact line in procmon is "INVALID PARAMETER Control: 0x12043 (Device:0x1 Function:2064 Method: 3)" with a valid on disk file name the Path column (in most cases a .pm module). Do not be alarmed. 0x12043 is IOCTL_AFD_GET_CONTEXT. The C callstack that generates the failed 0x12043 is
Below is a sample from procmon on my machine of what the error looks like. I got alarmed seeing it, since IOCTLs on disk files are very very rare in procmon logs, so curiosity and a wonder if I have anti-security or pro-security software running on the machine made me investigate it. 9:25:30.3284509 PM perl.exe 21088 CreateFile C:\p519\src\lib\Errno.pm SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: Opened The "(Device:0x1 Function:2064 Method: 3)" is total garbage BTW. Device 1 (FILE_DEVICE_BEEP) is the system beeper. I dont hear beeps coming out of the machine. Something else is up. #define FILE_DEVICE_NETWORK 18 ((18<<12)=0x12000, so ioctl call that starts with this is winsock related C:\Documents and Settings\Administrator>perl -e"printf('%x',((0x12<<12)|(0x10<<2 But the definition of IOCTL bitfield that MS and Promon use is different from winsock's _AFD_CONTROL_CODE #define CTL_CODE(DeviceType, Function, Method, Access) \ Once the ioctl code 0x12043 is decoded as a CTL_CODE it is "(Device:0x1 Function:2064 Method: 3)". -- |
From @bulk88On Wed Oct 30 02:08:09 2013, shay wrote:
Posting a patch to this problem for comments/review. The idea is to tag sockets handles with a special low 2 bits pattern. The kernel ignores the last 2 bits of all handles, so the only meaning the low 2 bits have are on a user-mode (that includes kernel32.dll) level. 2 closesocket calls are necessary to avoid a leak in ws2_32.dll's socket descriptor to winsock provider vtable hash table. I'm not 100% sure that this won't create another race like #118059 fixed. This patch breaks non-IFS/non-kernel handle socket provider protocols even harder than before. Before=Perl already calls dup/dup2 on socket handles, which calls DuplicateHandle in the CRT, if the socket handle isn't a kernel handle (but a user mode pointer), that causes breakage. Also I assume Perl's sysread/syswrite/buffered IO calls also won't work on non-kernel socket handles, I think I've seen a ticket about that somewhere on RT before. Perl may have had support for doing recv instead of read() on sockets in the past, but I might be imagining it, will need to research. -- |
From @bulk880001-WIP-don-t-closesocket-on-non-socket-handles.patchFrom b431f923ee773478da393bb26dd45b07d1dbf7e8 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Sat, 23 Nov 2013 17:03:32 -0500
Subject: [PATCH] WIP don't closesocket on non-socket handles
---
win32/win32.c | 19 ++++++++++-
win32/win32.h | 29 ++++++++++++++++++
win32/win32io.c | 5 ++-
win32/win32sck.c | 86 ++++++++++++++++++++++++++++++++++-------------------
4 files changed, 105 insertions(+), 34 deletions(-)
diff --git a/win32/win32.c b/win32/win32.c
index fb24bbb..8a565d8 100644
--- a/win32/win32.c
+++ b/win32/win32.c
@@ -3332,13 +3332,28 @@ win32_isatty(int fd)
DllExport int
win32_dup(int fd)
{
- return dup(fd);
+ int ret;
+ int old = _osfhnd(fd);
+ //DebugBreak();
+ ret = dup(fd);
+ if(ret != -1) {
+ int h = _osfhnd(ret);
+ assert(!SOCKET_FLAG_FROM_HANDLE(h));
+ _osfhnd(ret) |= SOCKET_FLAG_FROM_HANDLE(_osfhnd(fd));
+ }
+ return ret;
}
DllExport int
win32_dup2(int fd1,int fd2)
{
- return dup2(fd1,fd2);
+ int ret = dup2(fd1,fd2);
+ if(ret != -1) {
+ int h = _osfhnd(fd2);
+ assert(!SOCKET_FLAG_FROM_HANDLE(h));
+ _osfhnd(fd2) |= SOCKET_FLAG_FROM_HANDLE(_osfhnd(fd1));
+ }
+ return ret;
}
DllExport int
diff --git a/win32/win32.h b/win32/win32.h
index a521a41..df51ffa 100644
--- a/win32/win32.h
+++ b/win32/win32.h
@@ -599,6 +599,35 @@ EXTERN_C _CRTIMP ioinfo* __pioinfo[];
/* since we are not doing a dup2(), this works fine */
# define _set_osfhnd(fh, osfh) (void)(_osfhnd(fh) = (intptr_t)osfh)
+
+/* derived from IsConsoleHandle macro, (((ULONG_PTR)(h) & 0x10000003) == 0x3)
+ if low 2 bits are 11, then its a console psuedo handle that only Win32
+ subsystem understands, not a kernel handle, a kernel handle ends in 00, we
+ reserve 10 to mean socket handle, starting in Windows 8 according to reports
+ online, console handles are kernel handles, low 2 bits behaviour on Win8
+ consone handles is unresearched, 0x80000000 catches negative values are
+ other psuedo handles or invalid handle, meaning behind flag 0x10000000 is
+ unknown, 01 can not be used since 01 handles are listed as FILE_TYPE_UNKNOWN,
+ and the handle is not passed to a kernel call to determine its type, this
+ causes _fstati64 to fail, making open() from PP fail when opening a numeric
+ fd that contains 01 tagged handle
+*/
+
+//static void
+//PerlIOUnix_setfd(pTHX_ PerlIO *f, int fd, int imode)
+//{
+// PerlIOUnix * const s = PerlIOSelf(f, PerlIOUnix);
+//#if defined(WIN32)
+// Stat_t st;
+// if (PerlLIO_fstat(fd, &st) == 0) { <<<<< calls GetFileType in CRT which makes fstat fail
+//
+# define HANDLE_IS_SCK 0x2
+# define PSUEDO_HANDLES_MASK 0x90000003
+# define IS_PSUEDO_HANDLE(h) ((ULONG_PTR)(h) & PSUEDO_HANDLES_MASK)
+# define SOCKET_FLAG_FROM_HANDLE(h) ((IS_PSUEDO_HANDLE(h) == HANDLE_IS_SCK) ? HANDLE_IS_SCK : 0x0)
+/* slow, and does failing kernel calls, use only in DEBUGGING */
+# define PUBLIC_IS_SOCKET(h) (GetFileType(h) == FILE_TYPE_PIPE && GetNamedPipeInfo(h, 0, 0, 0, 0) == 0)
+
#endif /* PERL_CORE */
/* IO.xs and POSIX.xs define PERLIO_NOT_STDIO to 1 */
diff --git a/win32/win32io.c b/win32/win32io.c
index d183e3b..817628e 100644
--- a/win32/win32io.c
+++ b/win32/win32io.c
@@ -321,7 +321,10 @@ PerlIOWin32_dup(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *params, int flags)
if (DuplicateHandle(proc, os->h, proc, &new_h, 0, FALSE, DUPLICATE_SAME_ACCESS))
{
char mode[8];
- int fd = win32_open_osfhandle((intptr_t) new_h, PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
+ int fd;
+ assert(!(SOCKET_FLAG_FROM_HANDLE(new_h) || IS_PSUEDO_HANDLE(new_h)));
+ *((ULONG_PTR *)&new_h) |= SOCKET_FLAG_FROM_HANDLE(os->h);
+ fd = win32_open_osfhandle((intptr_t) new_h, PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
if (fd >= 0)
{
f = PerlIOBase_dup(aTHX_ f, o, params, flags);
diff --git a/win32/win32sck.c b/win32/win32sck.c
index 674add2..df92946 100644
--- a/win32/win32sck.c
+++ b/win32/win32sck.c
@@ -396,7 +396,11 @@ win32_accept(SOCKET s, struct sockaddr *addr, int *addrlen)
SOCKET r;
SOCKET_TEST((r = accept(TO_SOCKET(s), addr, addrlen)), INVALID_SOCKET);
- return OPEN_SOCKET(r);
+ assert(!(SOCKET_FLAG_FROM_HANDLE(r) || IS_PSUEDO_HANDLE(r)));
+ r = OPEN_SOCKET(r);
+ if( r != -1)
+ _osfhnd(r) |= HANDLE_IS_SCK;
+ return r;
}
int
@@ -670,8 +674,13 @@ win32_socket(int af, int type, int protocol)
if((s = open_ifs_socket(af, type, protocol)) == INVALID_SOCKET)
errno = get_last_socket_error();
- else
+ else {
+ assert(!(SOCKET_FLAG_FROM_HANDLE(s) || IS_PSUEDO_HANDLE(s)));
+ /* _open_osfhandle does a GetFileType which bans handles ending in 01 */
s = OPEN_SOCKET(s);
+ if( s != -1)
+ _osfhnd(s) |= HANDLE_IS_SCK;
+ }
return s;
}
@@ -690,21 +699,28 @@ int my_close(int fd)
return(close(fd)); /* Then not a socket. */
osf = TO_SOCKET(fd);/* Get it now before it's gone! */
if (osf != -1) {
- int err;
- err = closesocket(osf);
- if (err == 0) {
- assert(_osfhnd(fd) == osf); /* catch a bad ioinfo struct def */
- /* don't close freed handle */
- _set_osfhnd(fd, INVALID_HANDLE_VALUE);
- return close(fd);
+ if(SOCKET_FLAG_FROM_HANDLE(osf)) {
+ int err = closesocket(osf);
+ int err2 = closesocket(osf&~HANDLE_IS_SCK);
+ assert(err == err2); /* ???? */
+ if (err == 0) {
+ assert(_osfhnd(fd) == osf); /* catch a bad ioinfo struct def */
+ /* don't close freed handle */
+ _set_osfhnd(fd, INVALID_HANDLE_VALUE);
+ return close(fd);
+ }
+ else if (err == SOCKET_ERROR) {
+ err = get_last_socket_error();
+ assert(err != ENOTSOCK);
+ /* note this does a not allowed by MS CloseHandle
+ on the socket handle */
+ (void)close(fd);
+ errno = err;
+ return EOF;
+ }
}
- else if (err == SOCKET_ERROR) {
- err = get_last_socket_error();
- if (err != ENOTSOCK) {
- (void)close(fd);
- errno = err;
- return EOF;
- }
+ else {
+ assert(!PUBLIC_IS_SOCKET(osf));
}
}
return close(fd);
@@ -721,21 +737,29 @@ my_fclose (FILE *pf)
if (osf != -1) {
int err;
win32_fflush(pf);
- err = closesocket(osf);
- if (err == 0) {
- assert(_osfhnd(win32_fileno(pf)) == osf); /* catch a bad ioinfo struct def */
- /* don't close freed handle */
- _set_osfhnd(win32_fileno(pf), INVALID_HANDLE_VALUE);
- return fclose(pf);
- }
- else if (err == SOCKET_ERROR) {
- err = get_last_socket_error();
- if (err != ENOTSOCK) {
- (void)fclose(pf);
- errno = err;
- return EOF;
- }
- }
+ if(SOCKET_FLAG_FROM_HANDLE(osf)) {
+ int err = closesocket(osf);
+ int err2 = closesocket(osf&~HANDLE_IS_SCK);
+ assert(err == err2); /* ???? */
+ if (err == 0) {
+ assert(_osfhnd(win32_fileno(pf)) == osf); /* catch a bad ioinfo struct def */
+ /* don't close freed handle */
+ _set_osfhnd(win32_fileno(pf), INVALID_HANDLE_VALUE);
+ return fclose(pf);
+ }
+ else if (err == SOCKET_ERROR) {
+ err = get_last_socket_error();
+ assert(err != ENOTSOCK);
+ /* note this does a not allowed by MS CloseHandle
+ on the socket handle */
+ (void)fclose(pf);
+ errno = err;
+ return EOF;
+ }
+ }
+ else {
+ assert(!PUBLIC_IS_SOCKET(osf));
+ }
}
return fclose(pf);
}
--
1.7.9.msysgit.0
|
From @bulk88Bump. -- |
From @bulk88This ticket needs "Operating System" to be changed to "mswin32". I dont have the perms to do it. -- |
From @khwilliamsonOn 12/22/2013 08:42 PM, bulk88 via RT wrote:
Done |
From @bulk88On Sun Dec 22 19:40:08 2013, bulk88 wrote:
Bump. -- |
From @steve-m-hayOn Sat Nov 23 14:16:48 2013, bulk88 wrote:
What is the breakage caused to non-IFS providers that you refer to? I'm wondering what effect it will have on previous changes in this area: 036c1c1 - Fix [perl #24269] socket() call uses non-IFS providers causing subsequent print/read to hang or misbehave 1c97260 - Implement new environment variable to allow the use of non-IFS compatible LSP's on Windows to allow Perl to work in conjunction with a firewall such as McAfee Guardian |
From @bulk88On Mon Jan 27 10:12:36 2014, shay wrote:
That seems to have prohibited non-IFS protocols if I read it correctly, since if the protocol doesn't have "XP1_IFS_HANDLES" a socket using that protocol can't be created. The OP in 24269 is unclear, but he refers to sockets and kernel IO operations like WriteFile, and kernel I/O handle scheduling. He says "SO_SYNCHRONOUS_NONALERT" but i think he meant WSA_FLAG_OVERLAPPED.
That creates an option to removal the prohibition of non-IFS protocols. -- |
From @bulk88On Wed Mar 05 05:05:34 2014, bulk88 wrote:
Bump. Steve Hay? Tony C? JDB? -- |
From @tonycozOn Sat Nov 23 14:16:48 2013, bulk88 wrote:
Do Microsoft document that anywhere? Tony |
From @bulk88On Wed Jul 08 22:45:47 2015, tonyc wrote:
http://blogs.msdn.com/b/oldnewthing/archive/2008/08/27/8898863.aspx From MS headers #define STDAPI EXTERN_C HRESULT STDAPICALLTYPE #define STDMETHODIMP HRESULT STDMETHODCALLTYPE // The 'V' versions allow Variable Argument lists. #define STDAPIV EXTERN_C HRESULT STDAPIVCALLTYPE #define STDMETHODIMPV HRESULT STDMETHODVCALLTYPE // end_winnt // #define OBJ_HANDLE_TAGBITS 0x00000003L // typedef char CCHAR; // winnt typedef CCHAR *PCCHAR; // end_ntminiport end_ntndis end_ntminitape Reactos code http://doxygen.reactos.org/d0/d54/ntdef_8h_source.html#l01577 use of low 2 bits to store CWD directory handle inside Kernel32/ntdll http://doxygen.reactos.org/d9/d6e/lib_2rtl_2path_8c_source.html#l00026 reactos defintion of a kernel handle 00054 typedef union _EXHANDLE http://doxygen.reactos.org/de/d51/ntoskrnl_2ex_2handle_8c_source.html#l00045 at line 45 the low 2 bits are cleared when converting a handle to a kernel pointer MS's version Handle.TagBits = 0; MaxHandle = *(volatile ULONG *) &HandleTable->NextHandleNeedingPool; It is obvious I will have to rebase and clean up the commit a bit more before it can be applied. -- |
From @tonycozOn Thu Jul 09 04:10:20 2015, bulk88 wrote:
As we discussed in IRC, my main concern was whether the sockets returned From our discussion, I think we can reasonably assume that for, whether some As also discussed, duplicating a socket with your patch produces a handle If some external C code decides to dup() the fd without the mapping in XSUB.h Tony |
From @bulk88On Sun Jul 12 19:14:04 2015, tonyc wrote:
Perl would act very bizarre and flawed if diamond operator failed on sockets that got fdreopened/duped. Also, I dont think that win32_accept or win32_socket would even return a "fd" for a user mode socket. The SOCKET */maybe handle is turned into a fd with OPEN_SOCKET/win32_open_osfhandle/_open_osfhandle. _open_osfhandle calls GetFileType, if GetFileType fails/returns FILE_TYPE_UNKNOWN (same val), MS CRT wont alloc a fd. GetFileType is mostly a wrapper around NtQueryVolumeInformationFile, which obviously requires a real kernel handle. int __cdecl _open_osfhandle( /* copy relevant flags from second parameter */ ...........cut..................... /* find out what type of file (file/device/pipe) */ isdev = GetFileType((HANDLE)osfhandle); Perl internally has IoTYPE_SOCKET flag but I am not sure if reopening a fd preserves IoTYPE_SOCKET flag or not, and if there is adequate testing of the flag (and how to see the flag from PP without B::/Devel::Peek). http://perl5.git.perl.org/perl.git/commitdiff/036c1c1eb70a0dfc5a7187959eb5e39d499c9396 and http://perl5.git.perl.org/perl.git/commitdiff/1c97260979b979af03b946d71d50e8e4c075665c would basically have to go away, possibly replaced with panics in win32_accept and win32_socket (the only 2 ways to make a socket, correct me if I am wrong) if the low 2 bits are not 0. There is also the problem, that assuming a SOCKET * is a memory addr instead of kernel handle, the low 2 bits will be zero because that private C struct behind the SOCKET * is going to be aligned to atleast 4 bytes if not 8 or 16, so low 2 bits being zero can't separate a mem addr socket from a handle socket. So maybe either the open_ifs_socket/WSCEnumProtocols/XP1_IFS_HANDLES code stays, or GetHandleInformation has to be used to figure out if its a kernel handle socket. There is a slightly remote remote (should I round it to impossible?) chance that a mem addr might be a valid handle if over ~350K handles have been allocated. Windows can not allocate VM below addr 0x00010000 and on my system most handles are between 0x100 and 0x1000, and 0x2000 at tops. I'll skip the details (nobody wants a tour of Win32 VM layout), so the first possible malloc address is 0x150000. 0x150000/4=0x54000=344064 kernel handles. http://blogs.technet.com/b/markrussinovich/archive/2009/09/29/3283844.aspx says the limit is 16 million handles. So a [perl] process with mem addr SOCKETs, plus a large handle leak, could wind up mem addr socket pointers that are valid kernel handles, although the failure in this case would be that the panic croak of GetHandleInformation stops being thrown and GetHandleInformation returns true, is a kernel handle, even though the number is really a user mode SOCKET * and not a handle, and by coincidence, user mode SOCKET * happens to be a handle (maybe a file, maybe a mutex, maybe a process). But talking about deciding if its a user mode SOCKET or a kernel handle SOCKET and supporting user mode sockets is all in vain if _open_osfhandle will never allocate a fd for a user mode socket. User mode sockets aren't inheritable http://stackoverflow.com/questions/11847793/are-tcp-socket-handles-inheritable
Would anyone still use such a firewall in the 2010s? How many non-perl apps (php/python/ruby/non interpretated lang progs) would be broken if sockets aren't kernel handles? Now lets see what other engines do about closesocket vs close/CloseHandle on Win32. http://caml.inria.fr/mantis/view.php?id=5258 ocaml remembers what is a file and what is a socket CPYTHON cpython doesn't use _open_osfhandle on sockets, sockets are not fds https://docs.python.org/2/library/socket.html " socket.fileno() Return the socket’s file descriptor (a small integer). This is useful with select.select(). Under Windows the small integer returned by this method cannot be used where a file descriptor can be used (such as os.fdopen()). Unix does not have this limitation. https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L8484 only use of _open_osfhandle that isn't on stderr/stdout/stdin in cpython https://github.com/python/cpython/blob/master/Lib/multiprocessing/connection.py#L354 socket object knows its a socket and not a fd on win32 PHP https://github.com/php/php-src/blob/master/ext/sockets/windows_common.h#L34 sockets are socket objects/class, they know they aren't fds on Win32, no fileno method, no _open_osfhandle "fastcgi" knows if its a socket or not RUBY https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L7374 Ruby in a separate table remembers what handles are sockets. That is an analog of using Perl's ptr_table_fetch Also Ruby does not use _open_osfhandle the way Perl does
The attached patch tries fix that by comparing the error codes from closesocket on the original and tagged handles. if (errtagged == erruntagged) /* this catches WSAENOTSOCK */ If both closesockets return errors that are not WSAENOTSOCK, which error code is passed up to the caller from perl's my_close? IDK what it should be. Always the untagged version? which is the older/original handle. There is also a little bit of reservations I have, since this patch increases the memory that each socket takes up by one extra DSOCKET object in ws2_32.dll (0x18 bytes long), and 0xd8 bytes inside mswsock.dll at this callstack ntdll.dll!_RtlAllocateHeap@12() + 0xf There is also the CPU overhead of the "importing", but currently when perl is closesocket'ing disk files and pipes once winsock has been loaded/started to be used in the process, the "importing" is happening but failing. The ideal solution would be to get IoTYPE_SOCKET down to unix layer, and have unix layer pass it onto my_close_maybe_a_socket. perl523.dll!PerlIOUnix_open(interpreter * my_perl=0x00363efc, _PerlIO_funcs * self=0x280f86c0, PerlIO_list_s * layers=0x008f547c, long n=0, const char * mode=0x2814fbd4, int fd=3, int imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line 2668 C mode arg shows + mode 0x2814fbd4 "rb" const char * for me, not "s" or "rs" "b" is #define PIPESOCK_MODE "b" /* pipes, sockets default to binmode */ -- |
From @bulk880001-WIP-118127-don-t-closesocket-on-non-socket-handles-o.patchFrom 04461bd1387d842e9c6d431e1ed9e9927631c198 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Tue, 21 Jul 2015 02:48:44 -0400
Subject: [PATCH] WIP #118127 don't closesocket on non-socket handles on Win32
-win32io.c changes due to
win32io.c(336) : error C2296: '|=' : illegal, left operand has type 'HANDLE'
---
inline.h | 10 ++++
win32/win32.c | 32 +++++++++++++-
win32/win32.h | 44 +++++++++++++++++++
win32/win32io.c | 7 +++-
win32/win32sck.c | 122 ++++++++++++++++++++++++++++++++++++++++--------------
5 files changed, 181 insertions(+), 34 deletions(-)
diff --git a/inline.h b/inline.h
index 46a8cb6..3e99e96 100644
--- a/inline.h
+++ b/inline.h
@@ -350,6 +350,16 @@ S_should_warn_nl(const char *pv) {
#endif
+/* ------------------ win32sck.c ------------ */
+
+#if defined(WIN32) && !defined(WIN32_NO_SOCKETS)
+PERL_STATIC_INLINE BOOL
+S_is_handle_valid(HANDLE hnd) {
+ DWORD dwFlags;
+ return GetHandleInformation(hnd, &dwFlags);
+}
+#endif
+
/* ------------------ pp.c, regcomp.c, toke.c, universal.c ------------ */
#define MAX_CHARSET_NAME_LENGTH 2
diff --git a/win32/win32.c b/win32/win32.c
index 1ec045e..211db9f 100644
--- a/win32/win32.c
+++ b/win32/win32.c
@@ -3328,13 +3328,41 @@ win32_isatty(int fd)
DllExport int
win32_dup(int fd)
{
- return dup(fd);
+ int ret;
+ ret = dup(fd);
+#ifndef WIN32_NO_SOCKETS
+ if(ret != -1) {
+ int old_h = _osfhnd(fd);
+ int flag = SOCKET_FLAG_FROM_HANDLE(old_h);
+ int new_h = _osfhnd(ret);
+ assert(!SOCKET_FLAG_FROM_HANDLE(new_h)
+ && PUBLIC_IS_SOCKET(new_h) == PUBLIC_IS_SOCKET(old_h));
+ new_h |= flag;
+ _osfhnd(ret) = new_h;
+ }
+#endif
+ return ret;
}
DllExport int
win32_dup2(int fd1,int fd2)
{
- return dup2(fd1,fd2);
+ int ret;
+ ret = dup2(fd1,fd2);
+#ifndef WIN32_NO_SOCKETS
+ if(ret != -1) {
+ /* assert(!SOCKET_FLAG_FROM_HANDLE(_osfhnd(fd2)));
+ _osfhnd(fd2) |= SOCKET_FLAG_FROM_HANDLE(_osfhnd(fd1)); */
+ int old_h = _osfhnd(fd1);
+ int flag = SOCKET_FLAG_FROM_HANDLE(old_h);
+ int new_h = _osfhnd(fd2);
+ assert(!SOCKET_FLAG_FROM_HANDLE(new_h)
+ && PUBLIC_IS_SOCKET(new_h) == PUBLIC_IS_SOCKET(old_h));
+ new_h |= flag;
+ _osfhnd(fd2) = new_h;
+ }
+#endif
+ return ret;
}
DllExport int
diff --git a/win32/win32.h b/win32/win32.h
index 3b35b6c..5d90839 100644
--- a/win32/win32.h
+++ b/win32/win32.h
@@ -625,6 +625,50 @@ EXTERN_C _CRTIMP ioinfo* __pioinfo[];
/* since we are not doing a dup2(), this works fine */
# define _set_osfhnd(fh, osfh) (void)(_osfhnd(fh) = (intptr_t)osfh)
+
+/* derived from IsConsoleHandle macro, (((ULONG_PTR)(h) & 0x10000003) == 0x3)
+ if low 2 bits are 11, then its a console psuedo handle that only Win32
+ subsystem understands, not a kernel handle, a kernel handle ends in 00, we
+ reserve 10 to mean socket handle, starting in Windows 8 according to reports
+ online, console handles are kernel handles, low 2 bits behaviour on Win8
+ consone handles is unresearched, 0x80000000 catches negative values are
+ other psuedo handles (STD_INPUT_HANDLE/-10/STD_OUTPUT_HANDLE/-11) or invalid
+ handle (-1), meaning behind flag 0x10000000 is unknown, 01 can not be used
+ since 01 handles are special cased to fail by GetFileType,
+ and the handle is not passed to a kernel call to determine its type, this
+ causes _fstati64 to fail, making open() from PP fail when opening a numeric
+ fd that contains 01 tagged handle, thus leaving 10 as the only permutation
+ that can be used.
+*/
+
+/*
+static void
+PerlIOUnix_setfd(pTHX_ PerlIO *f, int fd, int imode)
+{
+ PerlIOUnix * const s = PerlIOSelf(f, PerlIOUnix);
+#if defined(WIN32)
+ Stat_t st;
+ if (PerlLIO_fstat(fd, &st) == 0) { <<<<< calls GetFileType in CRT which makes fstat fail
+*/
+
+# ifndef WIN32_NO_SOCKETS
+# define HANDLE_IS_SCK 0x2
+# define PSUEDO_HANDLES_MASK 0x90000003
+# define IS_PSUEDO_HANDLE(h) ((ULONG_PTR)(h) & PSUEDO_HANDLES_MASK)
+# define SOCKET_FLAG_FROM_HANDLE(h) ((IS_PSUEDO_HANDLE(h) == HANDLE_IS_SCK) ? HANDLE_IS_SCK : 0x0)
+/* [use] public [api to find out if the handle] is [a] socket
+
+ Slow, and does failing kernel calls, use only in DEBUGGING, not for regular builds.
+ There is no simple way to find out what is a socket, since the NT drivers for
+ named pipes (NPFS) and TCPIP (AFD or ws2ifsl) both call themselves
+ FILE_TYPE_PIPE/FILE_DEVICE_NAMED_PIPE. GetNamedPipeInfo internally is an
+ named pipe driver only ioctl. If the handle/driver sucessfully responds to a
+ NPFS ioctl, it proves that the handle is a pipe. Else it is assumed (but it
+ can't be proven without doing a winsock call) to be a socket.
+*/
+# define PUBLIC_IS_SOCKET(h) (GetFileType((HANDLE)h) == FILE_TYPE_PIPE \
+ && GetNamedPipeInfo((HANDLE)h, 0, 0, 0, 0) == 0)
+# endif
#endif /* PERL_CORE */
/* IO.xs and POSIX.xs define PERLIO_NOT_STDIO to 1 */
diff --git a/win32/win32io.c b/win32/win32io.c
index 00f5bb8..7f06f28 100644
--- a/win32/win32io.c
+++ b/win32/win32io.c
@@ -329,7 +329,12 @@ PerlIOWin32_dup(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *params, int flags)
if (DuplicateHandle(proc, os->h, proc, &new_h, 0, FALSE, DUPLICATE_SAME_ACCESS))
{
char mode[8];
- int fd = win32_open_osfhandle((intptr_t) new_h, PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
+ int fd;
+#ifndef WIN32_NO_SOCKETS
+ assert(!(SOCKET_FLAG_FROM_HANDLE(new_h) || IS_PSUEDO_HANDLE(new_h)));
+ *((ULONG_PTR *)&new_h) |= SOCKET_FLAG_FROM_HANDLE(os->h);
+#endif
+ fd = win32_open_osfhandle((intptr_t) new_h, PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
if (fd >= 0)
{
f = PerlIOBase_dup(aTHX_ f, o, params, flags);
diff --git a/win32/win32sck.c b/win32/win32sck.c
index 3f97241..07bda9f 100644
--- a/win32/win32sck.c
+++ b/win32/win32sck.c
@@ -396,7 +396,14 @@ win32_accept(SOCKET s, struct sockaddr *addr, int *addrlen)
SOCKET r;
SOCKET_TEST((r = accept(TO_SOCKET(s), addr, addrlen)), INVALID_SOCKET);
- return OPEN_SOCKET(r);
+#ifndef WIN32_NO_SOCKETS
+ assert(!(SOCKET_FLAG_FROM_HANDLE(r) || IS_PSUEDO_HANDLE(r)));
+ if (r != INVALID_SOCKET)
+ r |= HANDLE_IS_SCK;
+#endif
+ /* why is this call creating a fd for an invalud socket ??? */
+ r = OPEN_SOCKET(r);
+ return r;
}
int
@@ -619,6 +626,15 @@ open_ifs_socket(int af, int type, int protocol)
int error_code;
SOCKET out = INVALID_SOCKET;
+/* looking all this up on every socket call is stupid,
+ everything (including the convert_proto_info_w2a calls, should use WSASocketW instead)
+ but the for loop needs to be factored out into StartSockets, assuming this
+ code from Winsock 1 era isn't completly removed
+
+ https://rt.perl.org/Ticket/Display.html?id=24269
+
+ http://perl5.git.perl.org/perl.git/commitdiff/036c1c1eb70a0dfc5a7187959eb5e39d499c9396
+ */
if ((s = PerlEnv_getenv("PERL_ALLOW_NON_IFS_LSP")) && atoi(s))
return WSASocket(af, type, protocol, NULL, 0, 0);
@@ -661,6 +677,8 @@ open_ifs_socket(int af, int type, int protocol)
return out;
}
+#ifndef WIN32_NO_SOCKETS
+
SOCKET
win32_socket(int af, int type, int protocol)
{
@@ -670,8 +688,12 @@ win32_socket(int af, int type, int protocol)
if((s = open_ifs_socket(af, type, protocol)) == INVALID_SOCKET)
errno = get_last_socket_error();
- else
+ else {
+ assert(!(SOCKET_FLAG_FROM_HANDLE(s) || IS_PSUEDO_HANDLE(s)));
+ if (s != INVALID_SOCKET)
+ s |= HANDLE_IS_SCK;
s = OPEN_SOCKET(s);
+ }
return s;
}
@@ -683,6 +705,7 @@ win32_socket(int af, int type, int protocol)
* -- BKS, 11-11-2000
*/
+
int my_close(int fd)
{
int osf;
@@ -690,22 +713,49 @@ int my_close(int fd)
return(close(fd)); /* Then not a socket. */
osf = TO_SOCKET(fd);/* Get it now before it's gone! */
if (osf != -1) {
- int err;
- err = closesocket(osf);
- if (err == 0) {
- assert(_osfhnd(fd) == osf); /* catch a bad ioinfo struct def */
- /* don't close freed handle */
- _set_osfhnd(fd, INVALID_HANDLE_VALUE);
- return close(fd);
- }
- else if (err == SOCKET_ERROR) {
- err = get_last_socket_error();
- if (err != ENOTSOCK) {
+ if(SOCKET_FLAG_FROM_HANDLE(osf)) {
+ int err;
+ int osftagged = osf;
+ int errtagged = closesocket(osftagged) == SOCKET_ERROR ? WSAGetLastError() : ERROR_SUCCESS;
+ int osfuntagged = osftagged &~ HANDLE_IS_SCK;
+ int erruntagged = closesocket(osfuntagged) == SOCKET_ERROR ? WSAGetLastError() : ERROR_SUCCESS;
+/* use Socket; socket(SOCK, PF_INET, SOCK_STREAM, getprotobyname(\"tcp\"));
+open(SOCKTWO, '<&' . fileno(SOCK)); close(SOCKTWO);
+this code sample uses dup/DuplicateHandle and creates a tagged handle,
+that is never once passed to a socket function, until the 1st closesocket, and then
+the 2nd closesocket fails with WSAENOTSOCK since winsock front end never got a func call on the untagged handle before, and attempting to vivify/"Import" the untagged handle fails, since the socket was closed in the WSP layer/mswsock.dll/afd.sys in the 1st closesocket, and on the 2nd closesocket, WSP layer/mswsock.dll/afd.sys does not tell the front end "yes I created this handle/socket"
+
+Therefore, only if tagged and untagged both return WSAENOTSOCK, will the result be WSAENOTSOCK, else the result of the operation will be something other than WSAENOTSOCK
+*/
+ /* both kernel handles should be dead by now */
+ assert(!S_is_handle_valid((HANDLE)osfuntagged)
+ && !S_is_handle_valid((HANDLE)osftagged));
+ if (errtagged == erruntagged) /* this catches WSAENOTSOCK */
+ err = erruntagged;
+ else if (errtagged != WSAENOTSOCK && erruntagged == WSAENOTSOCK)
+ err = errtagged;
+ else if (errtagged == WSAENOTSOCK && erruntagged != WSAENOTSOCK)
+ err = erruntagged;
+ else
+ DebugBreak(); /* 2 different states, which to pass to caller ??? */
+ if (err == ERROR_SUCCESS) {
+ assert(_osfhnd(fd) == osftagged); /* catch a bad ioinfo struct def */
+ /* don't close freed handle */
+ _set_osfhnd(fd, INVALID_HANDLE_VALUE);
+ return close(fd);
+ }
+ else {
+ assert(err != ENOTSOCK);
+ /* note close() does a not allowed by MS CloseHandle
+ on the socket handle */
(void)close(fd);
- errno = err;
+ errno = convert_wsa_error_to_errno(err);
return EOF;
}
}
+ else {
+ assert(!PUBLIC_IS_SOCKET(osf));
+ }
}
return close(fd);
}
@@ -715,30 +765,40 @@ int
my_fclose (FILE *pf)
{
int osf;
+ int fd;
if (!wsock_started) /* No WinSock? */
return(fclose(pf)); /* Then not a socket. */
- osf = TO_SOCKET(win32_fileno(pf));/* Get it now before it's gone! */
+ osf = TO_SOCKET(fd = win32_fileno(pf));/* Get it now before it's gone! */
if (osf != -1) {
- int err;
win32_fflush(pf);
- err = closesocket(osf);
- if (err == 0) {
- assert(_osfhnd(win32_fileno(pf)) == osf); /* catch a bad ioinfo struct def */
- /* don't close freed handle */
- _set_osfhnd(win32_fileno(pf), INVALID_HANDLE_VALUE);
- return fclose(pf);
- }
- else if (err == SOCKET_ERROR) {
- err = get_last_socket_error();
- if (err != ENOTSOCK) {
- (void)fclose(pf);
- errno = err;
- return EOF;
- }
- }
+ if(SOCKET_FLAG_FROM_HANDLE(osf)) {
+ /* this code does not match my_close ATM, my_close is authoritative */
+ int err = closesocket(osf);
+ int err2 = closesocket(osf&~HANDLE_IS_SCK);
+ assert(err == err2); /* ???? */
+ if (err == 0) {
+ assert(_osfhnd(fd) == osf); /* catch a bad ioinfo struct def */
+ /* don't close freed handle */
+ _set_osfhnd(fd, INVALID_HANDLE_VALUE);
+ return fclose(pf);
+ }
+ else if (err == SOCKET_ERROR) {
+ err = get_last_socket_error();
+ assert(err != ENOTSOCK);
+ /* note this does a not allowed by MS CloseHandle
+ on the socket handle */
+ (void)fclose(pf);
+ errno = err;
+ return EOF;
+ }
+ }
+ else {
+ assert(!PUBLIC_IS_SOCKET(osf));
+ }
}
return fclose(pf);
}
+#endif /* #ifndef WIN32_NO_SOCKETS */
struct hostent *
win32_gethostbyaddr(const char *addr, int len, int type)
--
1.7.9.msysgit.0
|
Migrated from rt.perl.org#118127 (status was 'open')
Searchable as RT118127$
The text was updated successfully, but these errors were encountered: