Add Windows Dockerfile

[friism]

This is a canary PR to explore what getting Windows image into the official library would look like.

We'd want to have both windowsservercore and nanoserver derived images, maybe they should be in subfolders like alpine.

/cc @wol4max @tianon @taylorb-microsoft @StefanScherer @jstarks @jhowardmsft

[lowenna]

You can combine these into a single command to avoid an extra layer. See dockerfile.windows in docker/docker for an example.

[friism]

The download-unzip and setx instructions? Again, this is structured like the linux image: https://github.com/docker-library/golang/blob/master/1.6/Dockerfile#L20 I don't think there's any harm in having this be separate layers since this sequencing doesn't cause bloat.

[lowenna]

Sure, but less layers are quicker....

[friism]

How? It's the same number of bits over the wire (roughly), and it makes better use of caching because (for example) the RUN setx command (and subsequent ones) can be changed without having to re-run the the download and un-archive

[lowenna]

It's miniscule but every msec counts. But realistically if you're worried about caching, surely the setx should go first? It's more likely the download will change, not the path...

[friism]

@jhowardmsft but that would be semantically incorrect. Someone building the image and wanting to debug the layer after setx but before the download would run docker run -ti <layer-hash> cmd and be in a container with GOPATH and PATH pointing to a non-existent location.

[lowenna]

You honestly believe that's a realistic scenario though? I really don't.

[friism]

Here are some other reasons:

1. it's how the Linux Dockerfiles are structured
2. It reads better. If I'm scanning the Dockerfile top to bottom I'd see instruction setting PATH to random place not encountered before. Only scanning the next instruction do I realize that it's the eventual destination of the install of what's to come

[lowenna]

My $.02 - I truly don't buy that in something so simple. But your call as to what goes into any official images. Just giving my opinion 😄

[tianon]

Neat! Happy to have this discussion! 😄 👍

(@jhowardmsft probably want to wait on the specifics of the PR review until we figure out a few logistics first 😄)

A few questions right off the bat come to mind for me:

• Where does windowsservercore come from? Is that embedded into Windows itself, or is that going to be an official image too?
• Now that Windows can run Linux binaries, will it be potentially possible to run some Linux containers as-is on a Windows platform?
  (I think this is probably the most important question for how we structure this, given that it would determine whether windows is a variant or just an arch)
• What's the easiest way for us to test this and ultimately build/push it officially? (don't personally have access to any machines running the TPs) 😄

[StefanScherer]

As on Linux I would see some sort of "buildpack-deps:jessie-scm" base image to have git (and svn, bzr, ...) installed to make go get work.

[lowenna]

@tianon - Ignore the Linux binary bit. That's a client-only thing, not in containers or in server.

[tianon]

@jhowardmsft that makes me kind of sad, and I hope there are plans to change that because being able to docker run -it --rm ubuntu on Windows sounds really magical and I want it to be a thing 😇

[lowenna]

+1, but out of my hands.

[StefanScherer]

Maybe with a Hyper-V "Linux container": docker run -it --rm --isolation=hyperv ubuntu 😄

[tianon]

(reiterating that I think the Dockerfile here is a great straw-man and I'd prefer we hold off on actual review of it until we figure out the logistics first)

[StefanScherer]

The downloaded zip file should be checked as on Linux. I've added something in a node Docker image https://github.com/StefanScherer/dockerfiles-windows/blob/89f47847d082584c72741850119757414dda3625/node/5.10/Dockerfile#L10

   if ((Get-FileHash node.msi -Algorithm sha256).Hash -ne $env:NODE_SHA256) {exit 1} ; \

[lowenna]

@tianon Sure. I'll shut-up now 😺

[friism]

Where does windowsservercore come from? Is that embedded into Windows itself, or is that going to be an official image too?

This is explored here: moby/moby#21975

Currently the base images are assumed to be on the host. The hoped-for design is that the images will exist as tagged entities on Docker Hub, but that they'll be redirects to files hosted by Microsoft that the engine will seamlessly consume. @taylorb-microsoft may have more details.

Now that Windows can run Linux binaries, will it be potentially possible to run some Linux containers as-is on a Windows platform?

That's not really the case. I think it's better to think of Windows as a different architecture (but it's really a matrix, because one can conceive of ARM64 Windows Containers too).

Same as with CPU architectures, I think the goal is to have the option of Windows and Linux x86_64 images side-by-side in the same tag using manifest lists. @estesp has a prototype tool (I haven't tested it yet): https://github.com/estesp/manifest-tool

Until that's baked I think that placing the Windows images in separate tags or in a separate org (like for the non-x86_64 CPU architectures) is great.

What's the easiest way for us to test this and ultimately build/push it officially? (don't personally have access to any machines running the TPs)

The WS2016 tech previews are free to download and Microsoft has setup instructions.

You can also boot WS2016 VMs on Azure (@chanezon can perhaps help you get access to an Azure account).

@jhowardmsft and Docker currently maintains a fleet of modified WS2016 instances on Azure for the purpose of running CI on docker/docker. @mikedougherty has, I think, explored using that fleet for other things, but given how busy John is keeping the current setup going I don't think it can be re-purposed for image-building use.

Another wiggle is that the latest WS2016 (TP5) is not yet publicly available and we should build the images for that. A version is available to Docker, though.

@tianon if you think it's useful, I'd be happy to get on a hangout with you to talk more about how we might do this.

[friism]

@StefanScherer thanks for the feedback, added sha-check

[tianon]

Ok, have had some time to ponder this more (and got access to hardware/software for testing; thanks @friism and @mikedougherty!), and have more thoughts:

1. I think we need to figure out the final home of windowsservercore and nanoserver such that we're not turning around and changing all the references everywhere later, especially since as we add these variants to the official images they'll likely start to crop up elsewhere.

   If they're going to live under a namespace such as microsoft/, I don't mind much what their names are (and think that'd be one of the only cases of an exception to the official-images rule of "No official images can be derived from, or depend on, non-official images.").

   If they're going to be unprefixed top-level images, then I think having them as tags off a central windows repo makes sense (ie, windows:server-core, windows:nano-server, windows:nano-server-10.0.10586.0, etc). Perhaps it'd make sense to instead do windows-server:core + windows-server:nano, etc. or even just windows:core and windows:nano, since there's probably not much reason to put server in the name given that a desktop/client image doesn't really make a lot of sense, right? (or maybe it does?)

2. More specific to this repo (and thus establishing the pattern for replicating this out to other official images), I agree that sub-directories make sense, especially since we'll be treating these as variants for some time.

   For example, something like 1.6/windows-server or 1.6/windows/server-core, and leading to tags like golang:1.6-windows-server-core and golang:windows-server-core and perhaps even golang:windows, although I'm not sure whether that final alias would point to server-core or nano. Certainly open to suggestions on the naming for these bits.

[StefanScherer]

I would prefer shorter tags. For some tags the manifest tool can be used.

• golang:1.6.2
  • Linux: debian:jessie based
  • Windows: windowsservercore based (or how it will be named)
• golang:1.6.2-onbuild
  • Linux: debian:jessie based
  • Windows: windowsservercore based

But different tags for eg.

• golang:1.6.2-alpine
  • only for Linux
• golang:1.6.2-nanoserver
  • only for Windows

[tianon]

Introducing the manifest tool is not something we (the official images team) or the engine are prepared to support fully at this time, so this "variant" option is the patch to get us something usable in the short term while proper support is worked out, and even beyond that for supporting older daemons after we do have proper multi-manifest support worked out.

cc @estesp

[friism]

@tianon thanks for taking a look!

As far as I know, Nanoserver and Windowsservercore do not represent different userlands running on the same kernel. They're different kernels with different ABIs. For that reason, I think it's best for them to be differently named base images vs. different tags of windows.

@jstarks has been working on adding the relevant image metadata. Details on getting that wired up here: moby/moby#22816 Combined with manifest lists, this would let the engine/registry auto-negotiate whether to choose between nanoserver and windowsservercore.

I can try to add a Nanoserver Dockerfile too, maybe that'll root out some issues.

I'm fine with starting with a purely tag-based approach. I'd like to understand what it would take to adopt manifest lists though. Note that the registry does auto-fallback in case the client(daemon) doesn't know about manifest lists (this is determined from the accept header version). In that case, it just serves the Linux image manifest (no manifest list) to the client. Do you have other concerns? Better Docker Hub UI support?

cc @glennc @aaronlehmann @MichaelSimons @taylorb-microsoft @jhowardmsft

[jstarks]

Keep in mind that nanoserver images will work anywhere (post-TP5) while windowsservercore images will not work on Nano Server hosts (unless you pass --isolation=hyperv...). And if a user can use nanoserver, then he probably should since it's so much smaller and faster -- there is just a smaller API surface available.

So I'm not sure relying on manifest-list-based autodetection is the right thing here. I think the users are going to have to make a decision about which image is right for their use cases.

[MichaelSimons]

@jstarks I was thinking the manifest would be useful in the default experience. For example, if golang:latest is pulled on Ubuntu it would get the Linux variant. If pulled on a Windows Server Core host, it would get the windowsservercore variant. If pulled on a Nano Server host, it would get the nanoserver variant. Now if you want to get a nanoserver based image on Windows Server Core, then you could pull a tagged variant (golang:nanoserver) whose manifest is just tied to Windows. Additionally the manifest would be helpful in blocking the ability to pull windowsservercore based images on Nano Server.

[jstarks]

@MichaelSimons That may be true. I guess the question is whether a Windows Server Core user really wants to default to Windows Server Core or whether we can make Nano popular enough that everyone just wants that.

It seems that if we can plumb the information into the images and manifests, then it's just a policy decision that the engine can make at pull time.

[friism]

@tianon it looks like you've made progress on this - awesome!

I wanted to update you guys on the latest developments:

The Windows base layers have been regularized as normal Docker layer tarballs and will not require out-of-band installation. Currently you can download and docker load them, see here. Soon, Docker Hub will have foreign layer support. With that, microsoft/windowsservercore and microsoft/nanoserver will be almost-normal images on Docker Hub, but the base layer references will point to Microsoft servers. They can be pulled as normal images.

I think you should be able to start building prototype Windows library images. I'd recommend running WS2016, installing the base images using the current out-of-band approach and tagging them like microsoft/windowsservercore:10.0.14300.1030 (although I think the current version tag is slightly different).

For the curious, Microsoft has published a set of Windows images (typically named "-sample") here. Some of the Dockerfiles are here: https://github.com/Microsoft/Virtualization-Documentation/tree/master/windows-container-samples/windowsservercore

@tianon can you let us know if you're think you're blocked on missing pieces in the Windows - Docker engine on Windows - Windows base image toolchain?

[tianon]

Ah, nice! So microsoft/windowsservercore and microsoft/nanoserver are the for-sure final resting places? (not going to be in the store instead, or something like that?) My goal is to avoid adding a bunch of Windows Dockerfiles only to have to change them in a couple months because the images moved. 😄

Regarding nanoserver -- is it possible to run/build those on WS2016 yet (even via --isolation=hyperv or something like that), or do those still require Windows 10 / an actual Nano Server host?

[friism]

@tianon yes, microsoft/windowsservercore and microsoft/nanoserver are where they'll go - @taylorb-microsoft can confirm.

And yes, nanoserver can run on Windows Server 2016 with --isolation=hyperv!

[friism]

@tianon I think the windows base layers on Hub are just about working: moby/moby#25176 (comment)

[tianon]

That's epic! 😄 (and also points directly at microsoft/nanoserver being correct 👍 🤘)

I'll take a look at this again now (especially in the context of the image variant naming discussions that @taylorb-microsoft and I have had) and see if we can finally get this rolling again! 🎉

[thecloudtaylor]

@tianon As we're building out the image naming/versions etc... we are seeing some challenges, @enderb-ms is doing some thinking about this right now - great if you guys can discuss.

[tianon]

@taylorb-microsoft nice! I'd be happy to do whatever I can to help! Feel free to prod me via email too if you think it'd be easier to discuss/help that way 👍 (or even schedule a hangout or something)

[friism]

(following up from call)

@tianon for creating windowsservercore container images, I think a WS 2016 TP5 instance on Azure is great. You can really do this any old way that you want, including running the (free) WS2016 TP5 iso in virtualbox on your Mac or Linux machines.

nanoserver is going to be a bit more involved, so maybe it should be done later. Either you'd have run WSCore2016 on bare metal and then build nanoserver containers there (bare metal is required because nanoserver requires --isolation=hyper-v). Or you can boot nanoserver VMs directly on Azure and then remote-powershell to them to install Docker and do the builds. We'd have to get you access to create VMs in our Azure account for that to work, or you'd have to run your own. I haven't experimented with this approach and suspect it's going to be non-trivial.

WSCore2016 will eventually be able to build and run nanoserver containers without hyperv, but that's not useful for us know-if nothing else because we need to create TP5-compatible images.

@tianon I'd suggest that you start with windowsservercore images unless you (or someone else) can think of a simple way to build the nanoserver ones too.

@enderb-ms @taylorb-microsoft @PatrickLang