windows: finish adding needed interfaces to act as central

[deadprogram]

To use this module as a BLE central from a machine running Windows there are a few more interfaces that need to be implemented.

• Connect to a Device
• DiscoverServices on Device
• DiscoverCharacteristics on a discovered Services
• Write to a characteristic
• Receive notifications from a characteristic

See https://github.com/tinygo-org/bluetooth/blob/dev/gattc_darwin.go for example implementation in macOS.

The core of the low-level code needed to communicate with Windows Bluetooth interfaces has been written, see https://github.com/tinygo-org/bluetooth/tree/dev/winbt for more info.

[bketelsen]

@deadprogram let me poke around and see if there's anyone

[deadprogram]

Thanks @bketelsen

[deadprogram]

Just saw this new project: https://github.com/tdakkota/win32metadata

Seems like could be very helpful.

[jagobagascon]

Hi @deadprogram

We are really interested in having a functioning Windows implementation. And I did some tests with the library you mention, but before that let me add some background for the people that is not familiar with the windows implementation:

A tiny bit of background

The Windows implementation uses the WinRT (which stands for Windows Runtime) API. And calling this API from Go is quite a hard task: It requires structs to match the WinRT class definition. With pointers to each method of the class in the exact same order, because the pointers are initialised using syscalls. And the worst thing is that there's no way to know if you did it right. Calling a function is basically a leap of faith, a trial and error process:

type IBuffer struct {
	ole.IInspectable
}

type IBufferVtbl struct {
	ole.IInspectableVtbl

	// These methods have been obtained from windows.storage.streams.h in the WinRT API.

	// read methods
	GetCapacity uintptr // ([out] [retval] UINT32* value)
	GetLength   uintptr // ([out] [retval] UINT32* value)

	// write methods
	SetLength uintptr // ([in] UINT32 value);
}

func (v *IBuffer) Length() int {
	var n int
	hr, _, _ := syscall.SyscallN(
		v.VTable().GetLength,
		uintptr(unsafe.Pointer(v)),  // "this"
		uintptr(unsafe.Pointer(&n)), // we know that "out" param is an int
	)
	mustSucceed(hr)
	return n
}

This makes the Windows integration a real pain to implement. And even if we did so, any new feature, bug, or change made to the code in the future will require a HUGE amount of effort.

Luckily Windows provides API definitions files using its own IDL (WinMD). The win32metadata library parses these files and we could (ideally) use it to automatically generate Go files to call the WinRT API.

Testing win32metadata

That said, I did some tests with this library. I tried to read the information from C:/Windows/System32/WinMetadata/Windows.Storage.winmd. My goal was to obtain all the information required to automatically generate the buffer.go file, but I had no luck.

I was not able to find all the information I need to generate the Go file. I can list all methods from DataReader but I cannot find any mentions to IDataReaderStatics and more importantly I can't find the GUID of that interface (See the idl). And we need this to call the DataReader.FromBuffer static method:

bluetooth/winbt/buffer.go

Line 48 in e767111

inspectable, err := ole.RoGetActivationFactory("Windows.Storage.Streams.DataReader", ole.NewGUID("11FCBFC8-F93A-471B-B121-F379E349313C"))

There's probably something that I'm missing. I will try again when I have time.

I also did a little bit of research looking for alternatives:

Other options to generate the code

There's an open issue in golang/go#43838 requesting the use of windows metadata to auto-generate Go bindings more easily/accurately (for both win32 and winrt). I'm not sure if that's ever going to be implemented, at least not in the near future so we will need some other solution.

In that same issue, there's a mention to zzl/go-win32api, which is a Go library that contains automatically generated Win32 API bindings. Sadly it does not include WinRT.

The library has been generated using the zzl/go-win32api-gen generator which reads JSON files from marlersoft/win32json that are generated by marlersoft/win32jsongen using the winmd files as a source.

I did a very quick test using marlersoft/win32jsongen with the WinRT file we need C:/Windows/System32/WinMetadata/Windows.Storage.winmd file, but it just failed to run, I hope to find some time to try again later this week.

I thought it would be easier to unify efforts, so I opened an issue in zzl/go-win32api: zzl/go-win32api#1

[deadprogram]

Thanks for all this @jagobagascon that is incredibly helpful, and good idea to try to enlist others in this quest.

[jagobagascon]

A small update.

We've been working on a tool to automatically generate the required WinRT classes. Our goal was to have a simple code generator to assist us adding the missing features for windows, so we do not expect to generate all WinRT APIs, only the ones we need.

On the other hand, @zzl has also been working on it's own WinRT generated code. And created a repository with all the WinRT APIs last month: https://github.com/zzl/go-winrtapi. So we went from no WinRT generated code, to two different solutions (not complaining, I'm just happy for the steps we are making).

I tried zzl's solution but could not make it work (I sure missed something, but it's also true that the library is untested). And since I've successfully tested Windows device scan and connection with our generated WinRT classes, I will continue using our own generated code.

We expect to make the repository publicly available in the upcoming weeks. It will contain the code generator itself and also all the required WinRT APIs required by this project.

In the meantime, if @deadprogram is ok with it, I'll work on the device connection PR using a copy the generated code.

Once our tool is publicly available we could either remove the code and add a dependency to it or use the generator tool to generate keep the code in this repo (having more control over what's generated and what not).

[jagobagascon]

I'm happy to announce that we just made WinRT Go publicly available for anyone to use. And we are ready to push all the changes required to act as a BLE central on Windows.

[jagobagascon]

@deadprogram now that WinRT Go went public, are you okay with adding a dependency to it?

I think it's the best option. The library already contains all the necessary code to implement the missing Windows features. And also to replace the existing, manually written, WinRT interfaces.

But, if you wanted to, we are open to tweak the code generator a little bit to allow generating the code in this library.

Once we know how to proceed, I would:

• Add a new PR to migrate the existing WinRT classes (used for scanning and getting manufacturer data) to use the autogenerated code (windows: add winrt-go dependency and remove manually generated code #112)
• Connect to a device (windows: add device connection and disconnection #111)
• Discover services (windows: add service discovery #114)
• Discover characteristics (windows: add characteristic discovery #115)
• Write, read and enable notifications (windows: add characteristic read, write and notify operations #116)