Alpha Windows support

[mitchellh]

@wez edited this issue to place this status at the top

Current status: Alpha for 64-bit Windows.
It builds and passes the test suite on Windows Server 2012 R2 (x64) and should work on Windows 7 and later. PCRE expression terms are not currently supported.

At this time we're looking for alpha testers. Watchman on Windows may hang, crash or otherwise be unreliable. If you encounter problems we'll need you to provide detailed information about how to trigger the problem, and we may ask you to use visual studio to attach and perform some rudimentary debugging.

The latest build is available here:
https://ci.appveyor.com/api/projects/wez/watchman/artifacts/watchman.zip?branch=master&job=Environment:+WATCHMAN_WIN7_COMPAT%3D

Extract the zip file and make sure that watchman.exe is located in your PATH.
The watchman.pdb file is provided to facilitate debugging.

If you are consuming watchman from python, you will need to build pywatchman for yourself.
The released package on https://pypi.python.org/pypi/pywatchman should work for this purpose.

If you are consuming watchman from nodejs, the existing fb-watchman package available via npm should work, but the version in master has slightly better diagnostics in case you experience hangs.

[wez]

Windows is not currently supported. We have no immediate plans to add support for it. We would welcome a contribution to add this support if someone has the time and inclination to produce it.

[wez]

To support Windows, we first need to tidy up our handling of case-insensitive filesystems; that work is in-progress, but we still have no immediate plans for Windows at this time.

[wez]

We're tracking the list of things that would need to be tackled for Windows support in this wiki page:
https://github.com/facebook/watchman/wiki/Changes-required-for-Windows-support

[pclouds]

This is something you guys may want to watch out when using ReadDirectoryChangesW [1]

I dropped ReadDirectoryChangesW because maintaining a 'live' file system cache became more and more
complicated. For example, according to MSDN docs, ReadDirectoryChangesW may report short DOS 8.3
names (i.e. "PROGRA~1" instead of "Program Files"), so a correct and fast cache implementation would
have to be indexed by long and short names...

[1] http://article.gmane.org/gmane.comp.version-control.git/216195

[ziriax]

You might want to consider using the NTFS Change Journal. It is rock solid. I used it for Evolver, a closed source version control system that I developed for a game company in 2002.

It seems a Python wrapper is already made

[wez]

cc: @dsp

I just pushed https://github.com/wez/watchman/tree/win with my hacking in progress; it's been sitting on my laptop for a while and I just rebased on master as of today and make sure that it still builds on both osx and windows.

You will need the microsoft visual C++ compiler to build it; you can use the community edition for this.
The compiler tools (cl, nmake and so on) must be in your path.
To run the integration tests you will need arcanist and php installed and in your path.

To build and test:

cd watchman
nmake -f winbuild\Makefile check
arc test

The integration tests are a bit rough and ready largely because of the filesystem semantics on windows are not POSIX; there are a lot of errors around attempting to delete dirs while handles are still open on them. There are also some errors with / instead of \ in path names. These are for the most part cosmetic.

The current status is 26 of 74 tests failed.

If you're interested in helping with hacking and testing, you are most welcome! We're at the cleaning things up stage.

If you're interested in making a client connect to it, you can connect to a named pipe for the current user; for example, my watchman instance listens at \\.\pipe\watchman-wez. The code to select the default name of the pipe can be found here: https://github.com/wez/watchman/blob/win/main.c#L456

Once connected, you use the standard JSON or BSER watchman protocol to communicate with the server.

[ziriax]

Great! I just tried to compile and run watchman.exe, but the code seems to use Windows 8 specific APIs, while we're still at Windows 7. For example, the API function GetOverlappedResultEx is used. Is this by design?

[wez]

Windows 8 has a number of API calls that make it easier to port posix style applications and we're currently using those. In addition, I didn't have easy access to Windows 7 while I was prototyping this. I'm not opposed to making changes to have this run on Windows 7.

[ziriax]

It seems the only Win8 specific API function call is the GetOverlappedResultEx. At first sight it seems you can just replace this with the old GetOverlappedResult function, since your timeout is either 0 or INFINITE, so the timeout maps to the BOOL bWait argument of the old function.

I was able to compile and run the watchman fine on my Win7 machine, and the log shows that watching indeed works.

I was not yet able to get the hgwatchman Mercurial extension running on Windows, but I suspect I have a conflict between 32-bit and 64-bit version. I'll dig deeper.

[wez]

@ziriax great! I just (force) pushed some fixes to my branch to pick up the python build and test bits. You'll also see how to establish the named pipe connection you'll need for hgwatchman--you have to use os.open because the higher level file abstraction in python doesn't work properly with named pipes

[wez]

I remembered that I added make.bat in my branch as a short cut, so you can just run:

make

to have it run nmake with the right makefile for you. This script will also source the visual studio build environment. We're moving towards having the tests use python, so it is best if you have python in your path when you build; I'm using python 2.7.