Kubernetes Container Runtime Interface (CRI) doesn't support WindowsContainerConfig and WindowsContainerResources

[JiangtianLi]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature

What happened:
Currently Container Runtime Interface (CRI) only supports LinuxContainerConfig and therefore LinuxContainerResources in ContainerConfig. Windows resource config is different from Linux's, although it shares some common properties. For example, docker's resource config for both platforms: https://github.com/moby/moby/blob/master/api/types/container/host_config.go#L308-L344. The resource controls for Windows containers is documented at: https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/resource-controls.

What you expected to happen:
Since LinuxContainerConfig is configuration specific to Linux containers, ideally there is WindowsContainerConfig for configuration specific to Windows containers, so that container runtime can take that config and pass to Windows containers. For example,

// WindowsContainerResources specifies Windows specific configuration for
// resources.
// https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/resource-controls
message WindowsContainerResources {
    // CPU count. Default: 0 (not specified).
    int64 cpus = 1;
    // CPU percent. Default: 0 (not specified).
    int64 cpu_percent = 2;
    // CPU shares (relative weight vs. other containers). Default: 0 (not specified).
    int64 cpu_shares = 3;
    // Memory limit in bytes. Default: 0 (not specified).
    int64 memory_limit_in_bytes = 4;
    // Maximum IOps for the container system drive. Default: 0 (not specified).
    int64 io_maximum_iops = 5;
    // Maximum IO in bytes per second for the container system drive. Default: 0 (not specified).
    int64 io_maximum_bandwidth = 6;
}

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Windows
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[JiangtianLi]

/sig windows
/sig node

[JiangtianLi]

@PatrickLang @taylorb-microsoft @feiskyer

[feiskyer]

+1 for the feature. Some questions

• Does iops is required? CRI doesn't support setting iops yet.
• How windows would handle pod resources? Kubelet set pod-level cgroups for Linux containers
• Does all current LinuxContainerResources make sense for Windows containers? e.g. all fields below

kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime/api.proto

Lines 455 to 474 in 8c1ee76

	// LinuxContainerResources specifies Linux specific configuration for
	// resources.
	// TODO: Consider using Resources from opencontainers/runtime-spec/specs-go
	// directly.
	message LinuxContainerResources {
	// CPU CFS (Completely Fair Scheduler) period. Default: 0 (not specified).
	int64 cpu_period = 1;
	// CPU CFS (Completely Fair Scheduler) quota. Default: 0 (not specified).
	int64 cpu_quota = 2;
	// CPU shares (relative weight vs. other containers). Default: 0 (not specified).
	int64 cpu_shares = 3;
	// Memory limit in bytes. Default: 0 (not specified).
	int64 memory_limit_in_bytes = 4;
	// OOMScoreAdj adjusts the oom-killer score. Default: 0 (not specified).
	int64 oom_score_adj = 5;
	// CpusetCpus constrains the allowed set of logical CPUs. Default: "" (not specified).
	string cpuset_cpus = 6;
	// CpusetMems constrains the allowed set of memory nodes. Default: "" (not specified).
	string cpuset_mems = 7;
	}

cc/ @brendanburns @dchen1107 @yujuhong @kubernetes/sig-node-feature-requests

[thecloudtaylor]

cc Windows container core (@jhowardmsft; @darrenstahlmsft; @jstarks; @PatrickLang)

• Iops is optional (without specifying a value the container is unbounded in regards to IO)

• Ideally I think a per pod spec and a per container spec would make since (possibly the pod is just a sum of the containers but pod + container settings would enable nice sharing scenarios). With the goals of being as similar to existing (Linux) behavior and also getting to a solid MVP, what do we think makes since?

• With the ability to run Linux containers on Windows we would want it in the kublet :) but even for windows we have some of these values but not all. See https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/resource-controls for some more details.

[JiangtianLi]

Going forward, I think we need Windows config in CRI to be container runtime agnostic. I am trying to find spec from OCI eventually, not from docker's config in https://github.com/moby/moby/blob/master/api/types/container/host_config.go. Is https://github.com/opencontainers/runtime-spec/blob/master/specs-go/config.go something we could refer to?

[jberkus]

a) Is this a bug or a feature?

b) if it's a feature, please add it to feature tracking.

c) also, please add a priority/ label

[feiskyer]

Feature issue is here: kubernetes/enhancements#547

[JiangtianLi]

Changed this issue from bug to feature.

[jberkus]

Thanks!

[jberkus]

Please add status/approved-for-milestone if this feature is staying in 1.10 Thanks!

[JiangtianLi]

/status approved-for-milestone

[k8s-ci-robot]

You must be a member of the kubernetes/kubernetes-milestone-maintainers github team to add status labels.