Open
Description
openedon Aug 29, 2021
#418 has shown that the typical steps for a search are:
- Read the zim file (to be able to locate the xapian index in it) : Cold : 7.44s | Warm : 0.12s
- Open the xapian database (internal xapian code) : Cold : 0.09s | Warm : 0.003s
- Set the enquire on the database : Cold : 0.02s | Warm: 0.0004s
- Run the enquire and get a set of (ranged) results from the enquire (internal xapian code) : Cold : 3.74s | Warm : 1.5s
Here is when it happens:
- Once at file opening
- At first search requested and then cached
- At each search
- At each search
In a attempt to speed-up searches (in particular the first one) the idea would be to have the following workflow:
- Once at file opening
- Once at file opening (optional?) and then cached
- At file opening and then keep one (more?) ready to go all the time
- At each search
He would be the related questions on my side:
- Can we secure that 2. does not slows down the opening of the file (so it should run in an other thread)?
- Can we secure that 3. does not slows down the searches (so it should run in an other thread)?
- I guess the whole search system is protected to avoid two search requests to happen at the same time. If this is secure in a multithreaded context. This will be responsible of massive slow downs in many search requests happen at the same time. Would that be possible/reasonable to have a pull of "searcher"?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment