Closed
Description
Reported by @msarm initially as part of #207 :
I see 'ft.search' round trip got reduced now after defaulting the page size to 1000 and this helps other queries to run even faster but my results are not coming this time may be a different issue now.
When I perform the Person query:
Person.find().sort_by('-emp_no').first()Here is the ft.search performed.
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 0, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 1000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 2000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 3000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 4000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 5000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 6000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 7000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 8000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 9000, 1, 'SORTBY', 'emp_no', 'desc']
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 10000, 1, 'SORTBY', 'emp_no', 'desc'] - Fired query hangs
When the last fired ft.search is performed it does not return any results it retries for every.
Since are looking just for the first record, can we just fire the first ft.search and return the results?
args: ['ft.search', ':__main__.Person:index', '*', 'LIMIT', 0, 1, 'SORTBY', 'emp_no', 'desc']
When I set explicitly the exhaust_results=False in the first() function, I see quick results with no extra query. Is that right the fix?
def first(self):
query = self.copy(offset=0, limit=1, sort_fields=self.sort_fields)
results = query.execute(exhaust_results=False)
if not results:
raise NotFoundError()
return results[0]Original code:
import datetime
from typing import Optional
from redis_om import Field, HashModel, Migrator, get_redis_connection
# This Redis instance is tuned for durability.
REDIS_DATA_URL = "redis://localhost:6380"
class Person(HashModel):
first_name: str = Field(index=True)
last_name: str = Field(index=True)
emp_no: int = Field(index=True)
# set redis connection
Person.Meta.database = get_redis_connection(url=REDIS_DATA_URL,
decode_responses=True)
# apply migrations
Migrator().run()
for row_number in range(0,10000):
person = Person(first_name="John" + str(row_number), last_name="Doe", emp_no=row_number)
result = Person.find(Person.emp_no ==row_number).all()
if (len(result) == 0):
person.save()
print(person.pk)
# very slow to query a single record (~12 seconds)
Person.find().sort_by('-emp_no').first()Metadata
Metadata
Assignees
Labels
No labels