Add pid and msg attributes to NoSuchProcess and AccessDenied exceptions

[giampaolo]

From g.rodola on June 11, 2010 22:03:34

This would be useful, for example, when iterating over different 
Process instances to know the ID of the process which raised the exception and the exact reason.

Example:

import psutil

for pid in psutil.get_pid_list():
    try:
        p = psutil.Process(pid)
        p.create_time
    except (psutil.NoSuchProcess, psutil.AccessDenied), err:
        print "exception for pid:%s, reason:%s", % (err.pid, err.msg)
        raise err

Ideally, pid argument should always be mandatory for the function which raises the exception while msg can be left optional.

Both psutil.NoSuchProcess and psutil.AccessDenied should inherit from a generic psutil.Error exception that can be used to catch both.

Original issue: http://code.google.com/p/psutil/issues/detail?id=95

[giampaolo]

From g.rodola on June 12, 2010 09:36:51

Fixed as r588 .

Status: Fixed
Labels: -Progress-0in4

[giampaolo]

From g.rodola on June 12, 2010 09:48:03

Reopening as I realized I totally forgot the exceptions raised from C code.

Status: ReOpened

[giampaolo]

From g.rodola on June 18, 2010 14:21:18

FreeBSD changes have been committed as r561 .

The original NoSuchProcess exception written in C that was overriding 
the one written in Python has been removed and replaced by bare "return PyErr_SetFromErrno(PyExc_OSError);" statements.
This causes an OSError exception to be raised in python with ESRCH or EPERM errorcode automatically set.
From Python this gets translated in psutil.NoSuchProcess or psutil.AccessDenied with pid and msg attributes properly set.

Having a NoSuchProcess exception defined in C which overrides another 
one with the same name defined in Python was a poor design choice in the first place, imho.

This same thing should be done for Windows and OSX as well.

Labels: Progress-1in4

[giampaolo]

From g.rodola on June 19, 2010 06:45:18

Windows changes committed as r569 .
Summary:

- removed _psmswindows.NoSuchProcess
- defined a generic NoSuchProcess function in C (no longer visible 
from python) which, when called, causes an OSError(ESCHR) exception to be raised in python
- from python OSError(ESCHR) is translated in psutil.NoSuchProcess(pid, msg)

Note that from now on when we intend no raise NSP from C just we just have to call: return NoSuchProcess();
The same approach should is going to be adopted in FreeBSD and OSX as well.

The ugly NSP namespace overriding has been previously removed as r563 .

Labels: -Progress-1in4 Progress-2in4

[giampaolo]

From g.rodola on June 19, 2010 07:10:09

Here's a patch for OSX. I haven't tried it though.
Jay could you try it and see if it works?
Please, also take a look at the XXX comments I inserted.

Attachment: osx.patch

[giampaolo]

From jlo...@gmail.com on June 21, 2010 15:06:36

committed the changes, with a few modifications/fixes, and put in an 
explanation where the XXX comments were. I think we can also replace 
the existing NoSuchProcess() void function in C code with this simpler code: 

static PyObject* NoSuchProcess(void) {
    errno = ESRCH;
    return PyErr_SetFromErrno(PyExc_OSError);
}

Makes it a lot easier to read and less room for problems I think than 
the existing code. If we're OK with this change then it can be checked in on the Windows and BSD sides as well.

For the bad news, it looks like we are now running into this, at 
least on my 10.6 system: 
http://lists.apple.com/archives/darwin-dev/2009/Jan/msg00048.html 
task_for_pid() is preventing access to processes when we're trying to 
get cpu_times and memory info. I'm not sure what we can do to get 
around this at the moment but it looks like we will either need to 
avoid the OS X mach info interfaces altogether, or find a way to add 
psutil as a whole to the trusted processes security group. If I 
recall correctly, libtop for example is setuid in order to make this 
function properly. This would require either some kind of specialized 
installer scripting, or a post-installation step for the users, 
neither of which sounds particularly desirable, but I'm not sure if we'll have any better options.

[giampaolo]

From jlo...@gmail.com on June 21, 2010 15:39:06

After discussion with Giampaolo, added AccessDenied to the C code as 
well and we went back to returning AccessDenied errors when 
task_for_pid fails. This solved a lot of the test suite errors but we 
need additional work to fix remaining errors and determine whether 
we're getting useful data at all or if everything is now returning access failures now on current OS X versions.

[giampaolo]

From g.rodola on June 21, 2010 18:50:29

Can you provide the traceback of the failures and the output of such 
commands, just to figure out how many processes we're currently missing?

>>> len(psutil.get_pid_list())
>>> len(list(psutil.process_iter())

[giampaolo]

From g.rodola on June 21, 2010 18:50:43

Labels: -Progress-2in4 Progress-3in4

[giampaolo]

From jlo...@gmail.com on June 21, 2010 19:22:45

We're not missing processes: 

  >>> len(psutil.get_pid_list())
  84
  >>> len(list(psutil.process_iter()))
  84

Here's the current error output... I think it's varying on each run 
of the test suite though, most likely due to race condition scenarios 
where the process isn't initialized totally as we've seen in the past. 


======================================================================
ERROR: test_gid (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 83, in tearDown
    reap_children()
  File "test/test_psutil.py", line 73, in reap_children
    waitpid(child)
  File "test/test_psutil.py", line 58, in waitpid
    if pid == 0 and not process.is_running():
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/__init__.py", line 370, in is_running
    newproc = Process(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/__init__.py", line 147, in __init__
    self._last_user_time, self._last_kern_time = self.get_cpu_times()
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/__init__.py", line 337, in get_cpu_times
    return _platform_impl.get_cpu_times(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/_psosx.py", line 60, in wrapper
    return callable(self, pid, *args, **kwargs)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/_psosx.py", line 87, in get_cpu_times
    return _psutil_osx.get_process_cpu_times(pid)
RuntimeError: task_info(TASK_THREAD_TIMES_INFO) failed for pid 45030

======================================================================
ERROR: test_pid_0 (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 535, in test_pid_0
    p.get_memory_info()
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/__init__.py", line 348, in get_memory_info
    return _platform_impl.get_memory_info(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/_psosx.py", line 65, in wrapper
    raise AccessDenied(pid)
AccessDenied

======================================================================
ERROR: test_types (__main__.LimitedUserTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 428, in test_types
    self.assert_(isinstance(p.get_cpu_times(), tuple))
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/__init__.py", line 337, in get_cpu_times
    return _platform_impl.get_cpu_times(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/psutil/_psosx.py", line 65, in wrapper
    raise AccessDenied(pid)
AccessDenied

----------------------------------------------------------------------
Ran 90 tests in 2.590s

FAILED (errors=5)

[giampaolo]

From jlo...@gmail.com on June 21, 2010 20:17:30

It looks like task_for_pid is consistently failing on my system for 
PID 0, even in a test app that does nothing but attempt to get a 
kernel task port. We may need to add an exception for PID 0 to avoid this issue. 

Wrapping exceptions in tearDown in the test suite for reap_children 
should help solve some of the other errors and get us closer to 100% tests passed on OS X.

[giampaolo]

From g.rodola on June 22, 2010 12:37:55

Labels: -Milestone-0.1.4 Milestone-0.2.0

[giampaolo]

From g.rodola on July 19, 2010 05:02:50

r603 catches NSP in reap_children() which should solve some failures.
Aside from that, what's the current status on this one?

[giampaolo]

From jlo...@gmail.com on August 05, 2010 17:10:49

Some errors I'm still getting AccessDenied on: 

======================================================================
ERROR: test_types (__main__.LimitedUserTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 598, in test_types
    self.assert_(isinstance(p.get_cpu_times(), tuple))
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psutil/__init__.py", line 297, in get_cpu_times
    return _platform_impl.get_cpu_times(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psutil/_psosx.py", line 67, in wrapper
    raise AccessDenied(pid)
AccessDenied

======================================================================
ERROR: test_pid_0 (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 706, in test_pid_0
    p.get_memory_info()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psutil/__init__.py", line 308, in get_memory_info
    return _platform_impl.get_memory_info(self.pid)
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/psutil/_psosx.py", line 67, in wrapper
    raise AccessDenied(pid)
AccessDenied