Skip to content

Commit

Permalink
config-linux: add Intel RDT/MBA Linux support
Browse files Browse the repository at this point in the history
Add support for Intel Resource Director Technology (RDT) /
Memory Bandwidth Allocation (MBA). Add memory bandwidth resource
constraints in Linux-specific configuration.

In this PR, the spec for memory bandwidth (memBwSchema) keeps
the same format as existed spec for L3 cache (l3CacheSchema)
for consistency and compatibility in runtime-spec 1.x.

Example:

"linux": {
    "intelRdt": {
        "l3CacheSchema": "L3:0=7f0;1=1f",
        "memBwSchema": "MB:0=20;1=70"
    }
}

This is the prerequisite of this runc proposal:
opencontainers/runc#1596

For more information about Intel RDT/MBA, please refer to:
opencontainers/runc#1596

Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
  • Loading branch information
xiaochenshen committed Oct 28, 2017
1 parent 4ebb31e commit dd78a43
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 10 deletions.
23 changes: 17 additions & 6 deletions config-linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -461,24 +461,35 @@ The following parameters can be specified to set up the controller:
If `intelRdt` is set, the runtime MUST write the container process ID to the `<container-id>/tasks` file in a mounted `resctrl` pseudo-filesystem, using the container ID from [`start`](runtime.md#start) and creating the `<container-id>` directory if necessary.
If no mounted `resctrl` pseudo-filesystem is available in the [runtime mount namespace](glossary.md#runtime-namespace), the runtime MUST [generate an error](runtime.md#errors).

If `intelRdt` is not set, the runtime MUST NOT manipulate any `resctrl` pseudo-filesystems.
If `intelRdt` is not set, the runtime MUST NOT manipulate any `resctrl` pseudo-filesystems.

The following parameters can be specified for the container:

* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM).
If `l3CacheSchema` is set, runtimes MUST write the value to the `schemata` file in the `<container-id>` directory discussed in `intelRdt`.
The value SHOULD start with `L3:` and SHOULD NOT contain newlines.
* **`memBwSchema`** *(string, OPTIONAL)* - specifies the schema of memory bandwidth percentage per L3 cache id.
The value MUST start with `MB:` and MUST NOT contain newlines.

If `l3CacheSchema` is not set, runtimes MUST NOT write to `schemata` files in any `resctrl` pseudo-filesystems.
If both `l3CacheSchema` and `memBwSchema` are set, runtimes MUST write the combined value to the `schemata` file in the `<container-id>` directory discussed in `intelRdt`.
If `l3CacheSchema` contains a line beginning with `MB:`, the value written to `schemata` file MUST be the non-`MB:` line(s) from `l3CacheSchema` and the line from `memBWSchema`.

If either `l3CacheSchema` or `memBwSchema` is set, runtimes MUST write the value to the `schemata` file in the `<container-id>` directory discussed in `intelRdt`.

If neither `l3CacheSchema` nor `memBwSchema` is set, runtimes MUST NOT write to `schemata` files in any `resctrl` pseudo-filesystems.

### Example

Consider a two-socket machine with two L3 caches where the default CBM is 0xfffff and the max CBM length is 20 bits.
Tasks inside the container only have access to the "upper" 80% of L3 cache id 0 and the "lower" 50% L3 cache id 1:
Consider a two-socket machine with two L3 caches where the default CBM is 0x7ff and the max CBM length is 11 bits,
and minimum memory bandwidth of 10% with a memory bandwidth granularity of 10%.

Tasks inside the container only have access to the "upper" 7/11 of L3 cache on socket 0 and the "lower" 5/11 L3 cache on socket 1,
and may use a maximum memory bandwidth of 20% on socket 0 and 70% on socket 1.

```json
"linux": {
"intelRdt": {
"l3CacheSchema": "L3:0=ffff0;1=3ff"
"l3CacheSchema": "L3:0=7f0;1=1f",
"memBwSchema": "MB:0=20;1=70"
}
}
```
Expand Down
5 changes: 5 additions & 0 deletions schema/config-linux.json
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,11 @@
"l3CacheSchema": {
"id": "https://opencontainers.org/schema/bundle/linux/intelRdt/l3CacheSchema",
"type": "string"
},
"memBwSchema": {
"id": "https://opencontainers.org/schema/bundle/linux/intelRdt/memBwSchema",
"type": "string",
"pattern": "^MB:[^\\n]*$"
}
}
}
Expand Down
12 changes: 8 additions & 4 deletions specs-go/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -158,8 +158,8 @@ type Linux struct {
ReadonlyPaths []string `json:"readonlyPaths,omitempty"`
// MountLabel specifies the selinux context for the mounts in the container.
MountLabel string `json:"mountLabel,omitempty"`
// IntelRdt contains Intel Resource Director Technology (RDT) information
// for handling resource constraints (e.g., L3 cache) for the container
// IntelRdt contains Intel Resource Director Technology (RDT) information for
// handling resource constraints (e.g., L3 cache, memory bandwidth) for the container
IntelRdt *LinuxIntelRdt `json:"intelRdt,omitempty"`
}

Expand Down Expand Up @@ -561,10 +561,14 @@ type LinuxSyscall struct {
Args []LinuxSeccompArg `json:"args,omitempty"`
}

// LinuxIntelRdt has container runtime resource constraints
// for Intel RDT/CAT which introduced in Linux 4.10 kernel
// LinuxIntelRdt has container runtime resource constraints for Intel RDT
// CAT and MBA features which introduced in Linux 4.10 and 4.12 kernel
type LinuxIntelRdt struct {
// The schema for L3 cache id and capacity bitmask (CBM)
// Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
L3CacheSchema string `json:"l3CacheSchema,omitempty"`

// The schema of memory bandwidth percentage per L3 cache id
// Format: "MB:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;..."
MemBwSchema string `json:"memBwSchema,omitempty"`
}

0 comments on commit dd78a43

Please sign in to comment.