Search: initial file search seems to be slow

[bpasero]

My target workspace is our source with all node modules. All search.exclude is disabled and the .gitignore as well to make sure really every file is searched.

I run out of sources and have a word "index" in my clipboard to paste into quick open.

I measure:

• the time it takes from spawning the RipGrep command until the done handler is called
• the time it takes from starting the search until the results are all collected

Upon cold start I see:

• RipGrep takes: ~500ms
• Overall operation takes: ~3500ms

Subsequent operations on the same search seem to trigger our cache and I see:

• RipGrep takes: ~1000ms
• Overall operation takes: ~500ms

What I am wondering is: how come the very first initial search (that does not seem to trigger our cache) is taking so long from when all results are collected until they are on the model side of things.

This may be me measuring something wrong, but I think it is worth looking into this again if we have regressed for quick open file search performance.

[chrmarti]

From what I remember: RipGrep collects the full list of files disregarding the filter input and that full list is then cached. It then uses the filter text to filter the full list and again caches that result. So the cache has multiple entries and when you filter for something having a prefix for which there already is a cache entry, the new filtering is done on the prefiltered cache entry.

The check for whether there is an existing cache entry given a filter text is here:

vscode/src/vs/workbench/services/search/node/rawSearchService.ts

Line 279 in 98574d6

if (strings.startsWith(searchValue, previousSearch)) {

[roblourens]

@chrmarti's explanation is right, I see similar numbers but I don't really see a difference in ripgrep between the first/other runs. We seem to spend a lot of time sending results through IPC. I'll look at it some more during debt week.

[roblourens]

Found something interesting, for that initial cmd+p, we are actually sending unfiltered results back from the search process when we are not supposed to. Dates back to strict null check fixes. Getting rid of this gives a slight improvement.

I don't really see anything else problematic.