Process get_threads() and get_num_threads() occasionally fails with NoSuchProcess exception on Windows

[giampaolo]

From g.rodola on July 07, 2011 12:49:27

What steps will reproduce the problem?  
p = psutil.Process(os.getpid())
x = 0
while 1:
    x += 1
    p.get_threads()
    print x 

What is the expected output?  


What do you see instead?  
After a while  the script above will fail with a NoSuchProcess "process no 
longer exists" exception.
Problem is OpenThread() call in _psutil_mswindows.c which suddenly fails with 
ERROR_INVALID_PARAMETER which we treat as a synonym for "NoSuchProcess".
Still not sure what is the right thing to do in that case.

Original issue: http://code.google.com/p/psutil/issues/detail?id=180

[giampaolo]

From wj32...@gmail.com on July 07, 2011 03:54:37

I don't understand the problem here. Can you please elaborate?

[giampaolo]

From g.rodola on July 07, 2011 03:59:47

Problem is the script above should never fail with "process no longer exists" 
because the process obviously still exists considering we are inside an 
infinite while loop. =)

This behavior comes from: 
https://code.google.com/p/psutil/source/browse/tags/release-0.2.1/psutil/_psutil_mswindows.c#966
 What happens is OpenThread() fails with ERROR_INVALID_PARAMETER which we 
erroneously treat as a synonym for "process has gone away" and hence we raise 
NoSuchProcess exception.

I think OpenThread() returning ERROR_INVALID_PARAMETER at that point actually 
means that the thread has disappeared on us in which case we should just 
"continue" the for loop.

[giampaolo]

From wj32...@gmail.com on July 07, 2011 04:06:40

I see. STATUS_INVALID_CID (which is what NtOpenThread returns and is translated 
to ERROR_INVALID_PARAMETER) indicates that the thread does not exist, but that 
doesn't say anything about the process. In this case you probably want to 
continue the loop.

[giampaolo]

From wj32...@gmail.com on July 07, 2011 04:09:29

By the way, if you used the get_process_info function you could tell if the 
process itself is dead, as well as getting a thread count, which is in 
SYSTEM_PROCESS_INFORMATION->NumberOfThreads. It's much faster and easier.

[giampaolo]

From g.rodola on July 07, 2011 04:13:02

We are not only getting the number of threads but also the CPU times the thread 
has spend in system and user mode.

[giampaolo]

From wj32...@gmail.com on July 07, 2011 04:15:33

It's all in there. SYSTEM_THREAD_INFORMATION has KernelTime, UserTime, 
CreateTime, and a bunch of other stuff.

[giampaolo]

From g.rodola on July 07, 2011 06:11:41

It turns out both get_threads() and get_num_threads() suffer this problem which 
is now fixed in r1088 and r1089 .
Thanks wj32.64, I'll see to replace the current implementation by using 
SYSTEM_PROCESS_INFORMATION in a future commit.
Closing this out as fixed.

Summary: Process get_threads() and get_num_threads() occasionally fails with NoSuchProcess exception on Windows
Status: FixedInSVN

[giampaolo]

From g.rodola on July 08, 2011 12:07:35

Status: Fixed

[giampaolo]

From g.rodola on March 02, 2013 04:01:38

Updated csets after the SVN -> Mercurial migration: r1088 == revision 
86f100584cd6