Opening a large workspace (like a home directory) makes Theia unresponsive and can crash it

[lmcbout]

--> seems to loop forever and THEIA crash after a while
Note: use my /home folder as the workspace
trace many errors like:

Watching error: [ { Error: EPERM: operation not permitted, stat '/home/lmcbout/.local/share/teamviewer12/dosdevices/z:/proc/2646/map_files/7f327dd50000-7f327df50000'
errno: -1,
code: 'EPERM',
syscall: 'stat',
path: '/home/lmcbout/.local/share/teamviewer12/dosdevices/z:/proc/2646/
and map_files/7f327dd50000-7f327df50000' } ]
appears on the console --> END result. loosing THEIA on the web browser

..... Also on the console, I also see:

2017-10-18T14:58:19.996Z] ERROR: Theia/13521 on elxa8z6zp12:
Watching error: [ { Error: watch /home/lmcbout/.local/share/teamviewer12/dosdevices/z:/var/lock/lvm EACCES
at _errnoException (util.js:1019:11)
at FSWatcher.start (fs.js:1383:19)
at Object.fs.watch (fs.js:1409:11)
at createFsWatchInstance (/home/lmcbout/theia/theia/node_modules/chokidar/lib/nodefs-handler.js:37:15)

... then

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
...
11: 0x2e1f8046fd

The express server worker 1 has been disconnected.
The express server worker 1 has been stopped.

Done in 1793.94s.

[akosyakov]

Looks like a dup of #239

[hexa00]

I don't think it's a dup of #239 but it's close, it's just that in this case it triggers a lot of EACESS errors instead of ENOSPC.

So it can't be fixed by doing away with the errors, they have to be managed properly.

It does look like file watching errors handling is taking way to many resources.

[simark]

I can reproduce the crash (although it takes time) by using / as my root. I'd like to figure out what is taking up the memory, so I'll research memory profiling tools. If you already have some knowledge, I appreciate any tips!

[hexa00]

See these instructions bout using the chrome dev tools with node:
https://medium.com/@paul_irish/debugging-node-js-nightlies-with-chrome-devtools-7c4a1b95ae27

Then you can use the memory profiles in the memory tab.

[hexa00]

We could use https://www.npmjs.com/package/chokidar-children

To avoid Theia slowing down/crashing because of these issues.

[akosyakov]

I think we should try watching in the separate process, but this package does not seem to do much and does not support all events which we are using.

[hexa00]

Yes I think we could do your own layer of course.

There's also: https://github.com/Axosoft/nsfw to consider which seems to be more native and have less impact on JS... but I haven't dug into it much yet

[akosyakov]

I am working on the PR for it to run watching in the separate process.

[hexa00]

@akosyakov ho very nice ! :) thx

[akosyakov]

Should be fixed with #899