Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix for resolve many MemoryErrors #304

Closed
wants to merge 4 commits into from
Closed

Hotfix for resolve many MemoryErrors #304

wants to merge 4 commits into from

Conversation

vadimka123
Copy link

@vadimka123 vadimka123 commented Sep 6, 2018

[ERROR] Exception when performing meta profiling, dumping trace below
Traceback (most recent call last):
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/middleware.py", line 110, in _process_response
    collector.finalise()
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/collector.py", line 158, in finalise
    query_model = models.SQLQuery.objects.create(**query)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 393, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 185, in inner
    return func(*args, **kwargs)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/models.py", line 283, in save
    self.request.save(update_fields=['num_sql_queries'])
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/models.py", line 167, in save
    Request.garbage_collect(force=False)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/models.py", line 149, in garbage_collect
    if not requests or len(requests)-1 < target_count:
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 258, in __nonzero__
    return type(self).__bool__(self)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 254, in __bool__
    self._fetch_all()
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/cacheops/query.py", line 273, in _fetch_all
    return self._no_monkey._fetch_all(self)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1102, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/silk/sql.py", line 45, in execute_sql
    return self._execute_sql(*args, **kwargs)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 876, in execute_sql
    cursor.execute(sql, params)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 101, in execute
    return self.cursor.execute(query, args)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/home/vadimka/workspace/Data_Optimization_Prototype/venv/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
MemoryError

@avelis
Copy link
Collaborator

avelis commented Sep 6, 2018

@vadimka123 Thanks for making a PR. If you could address the PR's I could take a look and potentially merge in this change.

@vadimka123
Copy link
Author

@avelis, how to do this?

@avelis
Copy link
Collaborator

avelis commented Dec 19, 2018

You can see the build failure here:
https://travis-ci.org/jazzband/django-silk/builds/425169004?utm_source=github_status&utm_medium=notification

======================================================================
FAIL: test_greedy_garbage_collect (tests.test_models.RequestTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jazzband/django-silk/project/tests/test_models.py", line 158, in test_greedy_garbage_collect
    models.Request.garbage_collect(force=True)
  File "/home/travis/build/jazzband/django-silk/project/silk/models.py", line 152, in garbage_collect
    cls.objects.all().order_by('id')[:records_count_to_delete].delete()
  File "/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/django/db/models/query.py", line 598, in delete
    "Cannot use 'limit' or 'offset' with delete."
AssertionError: Cannot use 'limit' or 'offset' with delete.

@vadimka123 It looks like it doesn't like a delete method from a slice.

@avelis
Copy link
Collaborator

avelis commented Dec 19, 2018

@vadimka123 Getting closer.
https://travis-ci.org/jazzband/django-silk/builds/469903442?utm_source=github_status&utm_medium=notification
Looks like MySQL doesn't support certain queries the tests are running.

@xrmx
Copy link
Contributor

xrmx commented Feb 15, 2019

@vadimka123 your issue should be fixed in master, i think you can close this.

@auvipy auvipy closed this Feb 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants