Finalise filesystem_watchers

[alex-courtis]

Remove any codepaths that are conditional on either of these options.

Silently remove these options.

Update: there are genuine use cases for disabled watchers. Resolve remaining issues such as large directories.

Tasks:

• add filesystem_watchers.ignore
• tear down watcher on failure, notifying the user once

[yehuohan]

filesystem_watchers is a great feature for nvim-tree, and I think there is a case will have a better performance to keep the option filesystem_watchers with false value .

The case is that executing :!make command to build a large c/c++ project with nvim-tree opened, where a lot of intermedia files will be created that will result large memory consumption to nvim.exe.
I caught the case on windows when, and get the memory consumption from task manager.

[alex-courtis]

That is a good observation. No watchers may be a valid use case. This change should be reconsidered.

Before we reject I would be grateful if you could run a performance test. I don't have a windows environment, however it would be very valuable to see what is happening; there may be problems with the watchers that can be fixed.

Please enable profiling, run your large make and post the logs.

[gegoune]

Couldn't certain files be ignored by watchers?

[alex-courtis]

Couldn't certain files be ignored by watchers?

That would be a great solution.

Watchers only know about the directory and don't see individual file changes, as they don't get the full inotify event. However... ignoring directories would work very nicely.

We are (hardcoded) ignoring .git subdirectories now and could add regex directories. Use case of meson build directory comes to mind.

I'll be interested to see the results of @yehuohan profiling: perhaps there are some known bad or problematic directories on windows that we could document or defualt.

See also #1670

[yehuohan]

@alex-courtis

I tested with building neovim 0.8 branch from msys2 with following commands saved into a sh file:

mkdir -p build
cd build
cmake -DCMAKE_INSTALL_PREFIX=E:/nvim/neovim/install -G Ninja ..
ninja
ninja install
cd ..

And run nvim-qt (nvim 0.8) with mini nvim config:

let $DotVimDir=resolve(expand('<sfile>:p:h'))
set rtp^=$DotVimDir
call plug#begin($DotVimDir.'/bundle')
    Plug 'kyazdani42/nvim-tree.lua'
call plug#end()

lua << EOF
    local tcb = require('nvim-tree.config').nvim_tree_callback
    require('nvim-tree').setup{
        auto_reload_on_write = false,
        log = {
            enable = true,
            truncate = true,
            types = {
                git = false,
                profile = true,
            },
        },
        filesystem_watchers = { enable = true },
        diagnostics = { enable = false },
        git = { enable = false },
    }
EOF

• Build neovim from :terminal without opening nvim-tree

The memory consumption of nvim.exe is ok.

The detailed testing screen recording:

without-nt.mp4

• Build neovim from :termianl with opening nvim-tree

The memory consumption of nvim.exe is very large. (Finally I had to kill nvim-qt forcely from task manager for nvim-qt had no response)

with-nt.mp4

The attached nvim-tree.log:

[2022-10-22 14:35:32] [profile] START draw
[2022-10-22 14:35:32] [profile] END   draw  2ms
[2022-10-22 14:35:41] [profile] START draw
[2022-10-22 14:35:41] [profile] END   draw  1ms
[2022-10-22 14:35:41] [profile] START draw
[2022-10-22 14:35:41] [profile] END   draw  2ms
[2022-10-22 14:35:41] [profile] START draw
[2022-10-22 14:35:41] [profile] END   draw  2ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  2ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  2ms
[2022-10-22 14:35:42] [profile] START draw
[2022-10-22 14:35:42] [profile] END   draw  1ms
[2022-10-22 14:35:43] [profile] START draw
[2022-10-22 14:35:43] [profile] END   draw  1ms
[2022-10-22 14:35:43] [profile] START draw
[2022-10-22 14:35:43] [profile] END   draw  1ms
[2022-10-22 14:35:43] [profile] START draw
[2022-10-22 14:35:43] [profile] END   draw  1ms
[2022-10-22 14:35:54] [profile] START draw
[2022-10-22 14:35:54] [profile] END   draw  1ms
[2022-10-22 14:35:55] [profile] START draw
[2022-10-22 14:35:55] [profile] END   draw  1ms
[2022-10-22 14:38:53] [profile] START draw
[2022-10-22 14:38:53] [profile] END   draw  8ms
[2022-10-22 14:38:54] [profile] START draw
[2022-10-22 14:38:54] [profile] END   draw  1ms
[2022-10-22 14:39:36] [profile] START draw
[2022-10-22 14:39:36] [profile] END   draw  38ms

By the way, it seemed running build commands with ! will block nvim-tree and it's hard to reproduce the case with !. So I tested with :terminal where can reproduce every time. (Actually the first time I caught the memory consumption case is running build command with a job runner plugin)

[alex-courtis]

Thanks for the detailed demonstration. The problem is demonstrated and it's very useful to see that it is a meson style build.

Unforntunately there's nothing useful in the log; I apologise for not specifying the full settings, which I will update in the readme now.

Please can you re-run with:

    log = {
      enable = true,
      truncate = true,
      types = {
        all = false,
        config = false,
        copy_paste = false,
        dev = false,
        diagnostics = false,
        git = true,
        profile = true,
        watcher = true,
      },

I am hopeful that directory ignoring will resolve this.

[yehuohan]

@alex-courtis I tested with the log setting you provide. The nvim-tree.log is attached below and is about 140Mb after unzip.
The nvim memory consumption is large still. (Please let me know if you need the screen record)

nvim-tree.zip