test: allow running locally in isolated environment

[MunifTanjim]

Running ./scripts/test.sh would:

• Install necessary plugins (e.g. nui.nvim, plenary.nvim) if not already installed
• Run the tests in an environment isolated from regular vim setup (i.e. plugins should be used from ./.testcache directory)

Running ./scripts/test.sh --clean would:

• Clean ./.testcache directory before running the test and reinstall plugins

[cseickel]

This is pretty nice. My only concern is that it does not support Windows and we do get contributions from Windows users. I'm not sure if they run the tests or not, or maybe just rely on the Action anyhow.

Are there any Windows users watching this that want to comment?

[danilshvalov]

Maybe use something like Python?

[cseickel]

or maybe use a docker container to run the tests? Is docker as ubiquitous as python yet?

[miversen33]

or maybe use a docker container to run the tests? Is docker as ubiquitous as python yet?

Docker on windows runs in WSL, so you wouldn't actually be testing "on Windows", you'd be testing "on Linux"

[cseickel]

or maybe use a docker container to run the tests? Is docker as ubiquitous as python yet?

Docker on windows runs in WSL, so you wouldn't actually be testing "on Windows", you'd be testing "on Linux"

Duh, that would not be all that helpful, now would it?

The script is simple enough that we could probably just add a ps1 or bat script that copies the functionality. I'm leaning towards that. @MunifTanjim could you add that or convert it to python? If you don't have an environment setup on Windows I can write it from memory and let the next windows contributor actually test it out. We can also add an "is windows" switch in the test setup that reverts to the old code on Windows.

[MunifTanjim]

My only concern is that it does not support Windows and we do get contributions from Windows users. I'm not sure if they run the tests or not, or maybe just rely on the Action anyhow.

If we want to support running it on powershell (or whatever the default is on windows), I think that is a separate piece of work? I'm not motivated/equipped to work on that. Maybe somebody using windows can add another small script for windows later?

Maybe use something like Python?

or maybe use a docker container to run the tests? Is docker as ubiquitous as python yet?

Those would just be increasing the complexity. The simpler solution would be to:

• Provide another script that runs on windows (test.bat or something)
• Or if the windows user has Git Bash installed, the test.sh should run normally
• Or if WSL is used, the test.sh should run normally

[MunifTanjim]

could you add that or convert it to python?

I would like to avoid it very much, if I can 😂

We can also add an "is windows" switch in the test setup that reverts to the old code on Windows.

I'm not sure what you mean by "old code". The script just clones the plugins in a specific directory. If another script does the same thing on windows, shouldn't everything just work? 🤔

[MunifTanjim]

btw, I just wanted to run the tests locally without any hassle. That's the motivation behind this PR.

Adding python or docker in the mix would increase the hassle instead of decreasing it. Python or Docker has nothing to do with this project and I don't think we should be forcing the contributors to have those installed just to run the tests locally. If we want to support Windows, there are simpler ways (mentioned here).

Also it was not supported to run the tests locally at all. So not supporting windows probably shouldn't be a blocker for now? 🤔

[cseickel]

btw, I just wanted to run the tests locally without any hassle. That's the motivation behind this PR.

Adding python or docker in the mix would increase the hassle instead of decreasing it. Python or Docker has nothing to do with this project and I don't think we should be forcing the contributors to have those installed just to run the tests locally. If we want to support Windows, there are simpler ways (mentioned #413 (comment)).

I agree about python or docker. A bat or ps1 would probably be easiest if git bash doesn't work.

Also it was not supported to run the tests locally at all. So not supporting windows probably shouldn't be a blocker for now?

The test did run locally before. It used to be make test in the repo root. Maybe it didn't work for everyone, particularly if you didn't use packer. I just thought you added this because wanted to isolate the downloaded requirements from your installed plugins.

[MunifTanjim]

Maybe it didn't work for everyone, particularly if you didn't use packer. I just thought you added this because wanted to isolate the downloaded requirements from your installed plugins.

Ah, yes... I use vim-plug. So that would be the reason why it didn't work for me.

Did it work on windows before? 🤔 Well, in that case it is a blocker 😂

Sorry for the confusion. Converting the PR to draft for now.

[MunifTanjim]

@cseickel
with the latest change, make test should behave the same as before for windows users!

[cseickel]

Looks good, thanks!