Search causes an internal error (Django exception: `InvalidQuery: Raw query must include the primary key`)

[msabramo]

I have a very basic setup with Sentry running on port 9000 using sqlite, basically the configuration from the Quickstart, no gevent installed.

I search and get:

There was an internal server error while processing your request.

You may reference this error as c49b2d7df977464fa7bf8d75a1b427f0$0b9143ace0a36291d19255c95b4db3f4.

View this error in Sentry

The console running Sentry in the foreground has:

Internal Server Error: /default/search/
Traceback (most recent call last):
  File "/www/sentry/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/decorators.py", line 110, in wrapped
    return func(request, *args, **kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/decorators.py", line 47, in _wrapped
    return func(request, project, *args, **kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/frontend/groups.py", line 168, in search
    documents = list(SearchDocument.objects.search(project, query, sort_by=sort))
  File "/www/sentry/lib/python2.7/site-packages/django/db/models/query.py", line 1341, in __iter__
    raise InvalidQuery('Raw query must include the primary key')
InvalidQuery: Raw query must include the primary key
Internal Server Error: /default/search/
Traceback (most recent call last):
  File "/www/sentry/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/decorators.py", line 110, in wrapped
    return func(request, *args, **kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/decorators.py", line 47, in _wrapped
    return func(request, project, *args, **kwargs)
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/frontend/groups.py", line 168, in search
    documents = list(SearchDocument.objects.search(project, query, sort_by=sort))
  File "/www/sentry/lib/python2.7/site-packages/django/db/models/query.py", line 1341, in __iter__
    raise InvalidQuery('Raw query must include the primary key')
InvalidQuery: Raw query must include the primary key

[msabramo]

Added some printing of variables to the Django exception (hmmm, enhancement for Django?) and discovered that skip contains "id".

...
  File "/www/sentry/lib/python2.7/site-packages/sentry/web/frontend/groups.py", line 168,
in search
    documents = list(SearchDocument.objects.search(project, query, sort_by=sort))
  File "/www/sentry/lib/python2.7/site-packages/django/db/models/query.py", line 1341,
in __iter__
    raise InvalidQuery('Raw query must include the primary key
("%s" appeared in "%s" for model %s; query: %r)' % 
(self.model._meta.pk.attname, skip, self.model, self.query))
InvalidQuery: Raw query must include the primary key ("id" appeared in
"set(['status', 'total_events', 'project_id', 'date_changed', 'date_added', 'group_id', 'id'])"
for model <class 'sentry.models.SearchDocument'>; 
query: <RawQuery: "\n            SELECT sd.id, sd.group_id, 
SUM(st.times_seen) / sd.total_events as score,\n
sd.date_changed, sd.date_added\n
FROM sentry_searchdocument as sd\n            INNER JOIN sentry_searchtoken as st\n
ON st.document_id = sd.id\n            WHERE st.token IN ([u'LoggerFactory'])\n
AND sd.project_id = 1\n
GROUP BY sd.id, sd.group_id, sd.total_events, sd.date_changed, sd.date_added\n
ORDER BY sd.date_changed DESC\n            LIMIT 100 OFFSET 0\n        ">)

[msabramo]

Strangely, this issue happens on machine (CentOS) that I set up yesterday but not one that I set up locally on my MacBook Pro today.

I would diff the versions of packages in the virtualenv but I can't log into the former because of IT issues.

[msabramo]

OK, well now both of my servers have the same set of packages and the CentOS one still throws an error when I search and the Mac one doesn't.

BeautifulSoup==3.2.1
Django==1.3.1
South==0.7.4
amqplib==1.0.2
anyjson==0.3.1
celery==2.5.3
cssutils==0.9.9
django-celery==2.5.5
django-indexer==0.3.0
django-paging==0.2.4
django-picklefield==0.2.0
django-templatetag-sugar==0.1
gunicorn==0.13.4
kombu==2.1.5
logan==0.2.1
pynliner==0.4.0
pysqlite==2.6.3
python-dateutil==1.5
pytz==2012c
raven==1.7.3
sentry==4.0.14
simplejson==2.5.0
wsgiref==0.1.2

[msabramo]

Hmmm if I print out some of the values of variables in django.db.models.RawQuerySet.__iter__ then there are some interesting differences.

On the machine where search is throwing an exception:

*** self.columns = ['sd.id', 'sd.group_id', 'score', 'sd.date_changed', 'sd.date_added']
*** model_init_field_names = {}

On the machine where search is working correctly:

*** self.columns = ['id', 'group_id', 'score', 'date_changed', 'date_added']
*** model_init_field_names = {'date_changed': 3, 'group_id': 1, 'id': 0, 'date_added': 4}

Me thinks that when the self.columns has column names with sd. in them, model_init_field_names doesn't get initialized properly.

It's not clear to me yet whether this is a problem in Sentry or a problem with Django.