Stack information capture fails with IndexError when Jinja2 templates in call stack

[quantus]

Our application tries to send a warning message to the Sentry, but calling the captureMessage function raises IndexError exception. The application is build on Gunicorn+Flask+Jinja2 stack and contains a lot of moving parts. My assumption is that the stack information generation fails when the call stack contains compiled Jinja2 templates. I'm assuming that Jinja2 is storing somehow incorrect source code location information to somewhere, and that causes the problems.

One traceback from the application logs:

Traceback (most recent call last): 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app 
    response = self.full_dispatch_request() 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request 
    rv = self.handle_user_exception(e) 
  File "/app/.heroku/python/lib/python2.7/site-packages/newrelic/hooks/framework_flask.py", line 108, in _nr_wrapper_Flask_handle_exception_ 
    return wrapped(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception 
    reraise(exc_type, exc_value, tb) 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request 
    rv = self.dispatch_request() 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request 
    return self.view_functions[rule.endpoint](**req.view_args) 
  File "/app/.heroku/python/lib/python2.7/site-packages/newrelic/hooks/framework_flask.py", line 45, in _nr_wrapper_handler_ 
    return wrapped(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_lazyviews/utils.py", line 38, in __call__ 
    return view(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view 
    return func(*args, **kwargs) 
[ ...redacted... ]
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/views.py", line 84, in view 
    return self.dispatch_request(*args, **kwargs) 
[ ...redacted... ]
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template 
    context, ctx.app) 
  File "/app/.heroku/python/lib/python2.7/site-packages/flask/templating.py", line 116, in _render 
    rv = template.render(context) 
  File "/app/.heroku/python/lib/python2.7/site-packages/newrelic/api/function_trace.py", line 95, in dynamic_wrapper 
    return wrapped(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render 
    return self.environment.handle_exception(exc_info, True) 
  File "/app/.heroku/python/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception 
    reraise(exc_type, exc_value, tb) 
  File "/app/[redacted].html", line [redacted], in top-level template code 
[ ...redacted... ]
  File "/app/[redacted].html", line 188, in template 
    {% with field = foo|attr(field_name) %} 
  File "/app/.heroku/python/lib/python2.7/site-packages/jinja2/filters.py", line 799, in do_attr 
    value = getattr(obj, name) 
[ ...redacted... ]
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/contrib/flask.py", line 318, in captureMessage 
    result = self.client.captureMessage(*args, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/base.py", line 796, in captureMessage 
    return self.capture('raven.events.Message', message=message, **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/base.py", line 638, in capture 
    **kwargs) 
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/base.py", line 418, in build_msg 
    capture_locals=self.capture_locals, 
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/utils/stacks.py", line 241, in get_stack_info 
    for frame_info in frames: 
  File "/app/.heroku/python/lib/python2.7/site-packages/raven/utils/stacks.py", line 135, in iter_stack_frames 
    frames = inspect.stack()[1:] 
  File "/app/.heroku/python/lib/python2.7/inspect.py", line 1062, in stack 
    return getouterframes(sys._getframe(1), context) 
  File "/app/.heroku/python/lib/python2.7/inspect.py", line 1040, in getouterframes 
    framelist.append((frame,) + getframeinfo(frame, context)) 
  File "/app/.heroku/python/lib/python2.7/inspect.py", line 1015, in getframeinfo 
    lines, lnum = findsource(frame) 
  File "/app/.heroku/python/lib/python2.7/inspect.py", line 579, in findsource 
    if pat.match(lines[lnum]): break 
IndexError: list index out of range

Environment info:

• Python 2.7.14 running on Heroku (heroku-16 stack)

flask==0.12.2
raven==6.3.0
gunicorn==19.7.1
jinja2==2.8.1
(Lots of other assumedly unrelated dependencies)

The code that sends the message is simply something like this:

sentry.captureMessage(
    message='Message',
    level='warning',
    extra={'foo': 123},
    stack=True
)

For hotfix I've set stack=False, and I don't actually even need the stack information with warning messages. But I think this same would happen if there was some exception instead. Would it be possible for raven to catch raised IndexError:s and ignore the call stack in such cases? This would allow raven to catch exception even when the call stack contains compiled Jinja2 templates.

[quantus]

I've confirmed that the Jinja2 template in call stack causes the issue. Same python version and packages as in the previous message.

test_template.html:

Hello {{ foo.bar }}!

inspect_crasher.py:

import inspect

class Foo(object):
    @property
    def bar(self):
        return str(inspect.stack())

t.render(foo=Foo())

from jinja2 import Environment, FileSystemLoader
j2_env = Environment(loader=FileSystemLoader('.'))
j2_env.get_template('test_template.html').render(foo=Foo())

Execution:

> python inspect_crasher.py 
Traceback (most recent call last):
  File "inspect_crasher.py", line 11, in <module>
    j2_env.get_template('test_template.html').render(foo=Foo())
  File "/Users/pekka/.virtualenvs/inspect_crash/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/Users/pekka/.virtualenvs/inspect_crash/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "./test_template.html", line 1, in top-level template code
    Hello {{ foo.bar }}!
  File "/Users/pekka/.virtualenvs/inspect_crash/lib/python2.7/site-packages/jinja2/environment.py", line 408, in getattr
    return getattr(obj, attribute)
  File "inspect_crasher.py", line 8, in bar
    return str(inspect.stack())
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 1059, in stack
    return getouterframes(sys._getframe(1), context)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 1037, in getouterframes
    framelist.append((frame,) + getframeinfo(frame, context))
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 1012, in getframeinfo
    lines, lnum = findsource(frame)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/inspect.py", line 579, in findsource
    if pat.match(lines[lnum]): break
IndexError: list index out of range