Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce windows API bindings in the standard library to the minimum required for the standard library's own abstractions to work #4426

Closed
andrewrk opened this issue Feb 10, 2020 · 21 comments · Fixed by #17770
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. os-windows proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. standard library This issue involves writing Zig code for the standard library.
Milestone

Comments

@andrewrk
Copy link
Member

andrewrk commented Feb 10, 2020

Accepted Decision


Windows has a huge API surface. This opens the question of, what is the purpose of std lib windows API bindings?

Is it...

  • 🗳️ minimal surface area for the std lib's own abstractions to work, with the idea of eventually being supplemented by a third party package for users who want more comprehensive API coverage?
  • 🗳️ a complete, up-to-date, maintained set of all Windows API functions, for some approved set of DLLs?
  • 🗳️ some subset across DLLs that "feels right" for a standard library to have, with opinions on which functions should or should not be called (e.g. 'A' functions, see switch to using 'W' versions of windows functions instead of 'A' #534)?

In this proposal I am advocating for (1). I also think that (2) would be reasonable. I agree with @LemonBoy here that (3) is not reasonable.

@andrewrk andrewrk added standard library This issue involves writing Zig code for the standard library. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. os-windows labels Feb 10, 2020
@andrewrk andrewrk added this to the 0.7.0 milestone Feb 10, 2020
andrewrk added a commit that referenced this issue Feb 10, 2020
@Sobeston
Copy link
Contributor

By bindings are we talking just the function signatures, or wrappings with zig errors and such?
The latter obviously being a lot more work.

@mlarouche
Copy link
Contributor

My vote for 2

@LemonBoy
Copy link
Contributor

My vote is for (2) too, if the conversion process can be completely automated it's even better.

@Sobeston
Copy link
Contributor

By bindings are we talking just the function signatures, or wrappings with zig errors and such?
The latter obviously being a lot more work.

If we are talking the former, most of the work could be automated, with the caveat being that deciding which pointers are nullable will likely have to be done by a human.

@Rocknest
Copy link
Contributor

(3) it should contain preferred API's which excludes non thread safe functions and deprecated apis and possibly apis are not present in lowest supported windows version (win 8.1?)

@Sobeston
Copy link
Contributor

and possibly apis are not present in lowest supported windows version (win 8.1?)

I'm not aware of microsoft ever removing api functions (at least in the last 20 years) - they're very against this.

Personally I'm against 3 primarily because it will spawn endless avoidable discussion on what does and doesn't constitute as an excludable function.

There is also that if you're using winapi instead of the rest of the standard library you're likely doing things very wrong in the first place (...let's assume we're at 1.0 and the standard library is all good and covers all common use cases).

Another thing to add to this question: how to we deal with multiple versions of windows, which have differing sets of functions? (related: #1907)

@JesseRMeyer
Copy link

2

@Rocknest
Copy link
Contributor

@Sobeston they do not remove apis, but they add them.

@ikskuh
Copy link
Contributor

ikskuh commented Feb 10, 2020

I vote for (1) as it would fit the Zig spirit of "not doing too much".
std should only use a minimum set of APIs and if a user needs the full WinAPI including windows, fibers, ... he/she should import a WinAPI package.

This reduces the maintenance surface of std and the core of the language is kept small

@daurnimator
Copy link
Contributor

I think 1 + 2... which is 3 I guess.

  • We should cover the full functionality of NTDLL: that's the equivalent of the syscall ABI on other platforms
  • We will additionally need a few pieces of functionality from win32 and other dlls to accomplish certain operations in the standard library: these would not get full coverage.

@jayschwa
Copy link
Contributor

I think the options could be rewritten as the following:

  1. Release a new version of the standard library when its implementation changes.
  2. Like option 1, but also release new versions when a supported OS changes its API.
  3. Like option 2, but with added debates about what "feels right".

Option 1 seems the most sustainable in the long run.

@Sahnvour
Copy link
Contributor

@Sobeston IIUC this is about bindings only, not wrapping them in idiomatic Zig.

I fail to see how option (1) is different from (3) in practice.

The standard library will always need to provide OS abstraction, and as such will need the necessary bindings to windows api. Unless they're made inaccessible to the users and we force them to use the third-party windows package, this is effectively an opinionated subset of the API.

So I'd vote for 2.

@Sobeston
Copy link
Contributor

@Sahnvour if we're just talking bindings then that's pretty much as easy as getting all the function signatures, so I vote for 2. I write stuff with the windows API, and this would be extremely convenient. It also means that there isn't much room for disagreement on what functions should be there. Would be happy to help fill that out.

As for what DLLs, I think Kernel32, NtDll, User32 are the most important.
And versioning, targeting 8.1 is a good idea.

@adontz
Copy link

adontz commented Feb 21, 2020

I think std should depend on bare minimum (so option 1), but since there is an obvious need to interact with OS, there should be another project, some library with complete win32 APIs, probably with thin Zig-way wrappers. Rust and Delphi follow this path more or less successfully, and there is http://pinvoke.net for .Net. I actually hate .Net CRL approach, because it's not one-to-one mapping, but some leaking abstraction, so pinvoke.net code often replaces CLR, not just complements it.

I'd actually, highly prefer to have thin wrappers, over no wrappers. At least, because memory management strategies (who's the owner of a buffer, which function frees memory, can a buffer be moved in memory between related calls) vary wildly in win32.

Regarding version. It's not even about functions and their prototypes. TCP_FASTOPEN is available since some version (1607, but not the point) of Windows 10. Such new flags if used on older Windows versions often generate ERROR_INVALID_PARAMETER errors. Should every application interested in TCP_FASTOPEN implement own fallback logic, or should thin wrapper become a little thicker?

@BarabasGitHub
Copy link
Contributor

I'd say 1, but I'm not sure what 2 means. I guess it's a little bit more than a @cImport() would generate, but then if you wrap around it you'll have to document all the things you did as well. So... somehow that doesn't feel like the right thing to do for a language.

@qb-0
Copy link

qb-0 commented Jul 10, 2020

Def. 2

@Aransentin
Copy link
Contributor

How about the renaming of basic win32 types? I can see a lot of benefit in doing that, with the drawback that reading the win32 C docs might get a little more awkward:

I see from the current bindings that sometimes types like LPCWSTR are renamed to [*:0]const u16 and such. This is probably necessary in some cases, as the type does not otherwise specify if the string is null-terminated. MultiByteToWideChar takes a LPWSTR plus length separately for example.

Should types like INT32 use i32 instead?

Should types like UINT use u32 or perhaps c_uint instead? Note that ULONG is always 32 bits on Windows but long 64 bits on most 64-bit Linuxes, so IMHO this would make the functions a lot less confusing to use.

Should DWORD and such use u32 instead? Can there be an instance where knowing that a function takes a DWORD as opposed to an UINT is useful information?

Sometimes win32 functions do awful hacks like in UnregisterClassW, where if the u16 string argument is small enough it is treated as an ATOM id instead. Since the ATOM id isn't necessarily divisible by 2 and as such not casteable to an [*:0]u16, there is no way for safety-checked Zig to use this feature. Could we replace these arguments with a union wrapper? (would that even work with stdcall?)

@malusmundus
Copy link

2

@expikr
Copy link
Contributor

expikr commented Sep 22, 2023

I'm making the call here to stick with (1).

For those who want (2), look at marlersoft/zigwin32. There's just too much bloat in the Windows API that we don't need to ship with the Zig standard library. This is exactly what third party packages are for. Given that Zig will have a package manager built in, there is no need to bloat the standard library.

This means removing the Windows extern functions that end in "A" is fair game now.

I'm in favour of this as well, but zigwin32 kinda sucks with how it's organized by its own arbitrary namespaces, rather than the straightforward way in the current std. Being forced to use zigwin32 adds the burden of having to manually look up their arbitrary categorizations instead of just checking MSDN. Is there any chance that the current library is split into its own project with the aim of completing it the way it's organized on MSDN?

@ellacrity
Copy link

I'm in favour of this as well, but zigwin32 kinda sucks with how it's organized by its own arbitrary namespaces, rather than the straightforward way in the current std. Being forced to use zigwin32 adds the burden of having to manually look up their arbitrary categorizations instead of just checking MSDN. Is there any chance that the current library is split into its own project with the aim of completing it the way it's organized on MSDN?

I agree as well (quiet strongly) with solution 1. Solution 2 may sound great on paper, but in reality, it would be a massive undertaking and very difficult to maintain. This type of thing is exactly what third-party libraries are for, so I think it's a great call to make that explicit (as mentioned above).

I would argue that anything beyond 1 are not even within the scope of Zig's standard library. A standard library is meant to be general, targeted at the most common use-cases rather than all use-cases, or niche use cases (like this one).

I do agree that it can be extremely frustrating to try to learn systems programming on Windows, so I would like to help document the existing Win32 / NT API functions used within std. Please bear in mind that documenting a significant number of items from these Windows APIs may prove to be difficult, or at the very least a lot of work. I recently asked about this feature in Discord and some people seem to be on board with the general idea. I think the first part is worrying about getting the raw data in the first place.

andrewrk added a commit that referenced this issue Oct 28, 2023
This is progress towards #4426
Closes #17417
@andrewrk andrewrk changed the title clarify the intent of windows API bindings in the standard library reduce windows API bindings in the standard library to the minimum required for the standard library's own abstractions to work Oct 28, 2023
andrewrk added a commit that referenced this issue Oct 28, 2023
This is progress towards #4426
Closes #17417
andrewrk added a commit that referenced this issue Oct 28, 2023
The only remaining Windows APIs now are the ones that the standard
library itself depends on for its cross-platform abstractions.

closes #4426
@andrewrk andrewrk modified the milestones: 0.15.0, 0.12.0 Oct 28, 2023
andrewrk added a commit that referenced this issue Oct 29, 2023
The only remaining Windows APIs now are the ones that the standard
library itself depends on for its cross-platform abstractions.

closes #4426
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. os-windows proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. standard library This issue involves writing Zig code for the standard library.
Projects
None yet
Development

Successfully merging a pull request may close this issue.