Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config-linux: add Intel RDT CLOS name sharing support #988

Merged
merged 1 commit into from
Sep 11, 2018

Conversation

linericyang
Copy link
Contributor

@linericyang linericyang commented Aug 28, 2018

Previously, it will create a dedicated RDT Class of Service (CLOS) for each running container, even they have exactly same Scheam. This will lead to short of CLOS, since there is a hardware limit for the number of CLOS, around 16 CLOS per platform.

This PR add one parameter 'closID' into existed spec to allow user to specify which RDT Class of Service (CLOS) the container will be located. So it can place these containers with same Schema into one single CLOS.

Example:

"linux": {
    "intelRdt": {
        "closID": "guaranteed_group",
        "l3CacheSchema": "L3:0=ffff0;1=3ff"
    }
}

Fixes: runc issue 1878

Co-authored with Dakshina Ilangovan @dakshinai

Creating a dedicated RDT Class of Service (CLOS) for each running
container, even they have exactly same Scheam, will lead to short of
CLOS, since there is a hardware limit for the number of CLOS, around
16 CLOS per platform.

This PR add one parameter 'closID' into existed spec to allow user
to specify which RDT Class of Service (CLOS) the container will be
located. So it can place these containers with same Schema into one
single CLOS.

Example:

"linux": {
    "intelRdt": {
        "closID": "guaranteed_group",
        "l3CacheSchema": "L3:0=ffff0;1=3ff"
    }
}

Signed-off-by: Lin Yang <lin.a.yang@intel.com>
@xiaochenshen
Copy link
Contributor

We could also call this "Intel RDT control group sharing between containers" feature which we have discussed in:
opencontainers/runc#433 (comment)

For more information about Intel RDT CAT and MBA support in OCI:
opencontainers/runc#433
opencontainers/runc#1596

@linericyang
Copy link
Contributor Author

@crosbymichael @vishh Could you please help to review this design proposal of "Intel RDT control group sharing between containers" feature if you get chance? Thanks in advance. Really appreciated.

@crosbymichael
Copy link
Member

@xiaochenshen does this look good to you?

@xiaochenshen
Copy link
Contributor

xiaochenshen commented Sep 5, 2018

@crosbymichael

Yes, Intel RDT CLOS/group sharing between containers is a real user scenario for OCI and Intel RDT users.

Intel RDT has hardware limit (num_closids) for RDT control groups. The number of the limit is exported by kernel through /sys/fs/resctrl/info/L3/num_closids.

In current implementation of Intel RDT/CAT support for OCI (opencontainers/runc#433, opencontainers/runc#1279) and Intel RDT/MBA support for OCI (opencontainers/runc#1596, #932, opencontainers/runc#1632), we create new dedicated RDT group (CLOS) for each container which specified Intel RDT config in /sys/fs/resctrl/<container_id>, similar to how we deal with cgroups. But we will run out of CLOS hardware resource quickly.

With this PR, we could mitigate the issue of running out of RDT CLOS.

@crosbymichael
Copy link
Member

crosbymichael commented Sep 6, 2018

LGTM

Approved with PullApprove

@linericyang
Copy link
Contributor Author

@vbatts @tianon @hqhq Could one of you please review this PR? Please let me know if you have any questions.

cc @xiaochenshen

@hqhq
Copy link
Contributor

hqhq commented Sep 11, 2018

LGTM

Approved with PullApprove

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants