-
Notifications
You must be signed in to change notification settings - Fork 14.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document extended resources and OIR deprecation. #5399
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -305,6 +305,15 @@ where `OOM` stands for Out Of Memory. | |
|
||
## Opaque integer resources (Alpha feature) | ||
|
||
{% include feature-state-deprecated.md %} | ||
|
||
**Note:** Opaque Integer Resources will be removed in version 1.9. | ||
[Extended Resources](#extended-resources) are a replacement for Opaque Integer | ||
Resources. Users can use any domain name prefix outside of the `kubernetes.io/` | ||
domain instead of the previous `pod.alpha.kubernetes.io/opaque-int-resource-` | ||
prefix. | ||
{: .note} | ||
|
||
Kubernetes version 1.5 introduces Opaque integer resources. Opaque | ||
integer resources allow cluster operators to advertise new node-level | ||
resources that would be otherwise unknown to the system. | ||
|
@@ -316,6 +325,7 @@ available amount is simultaneously allocated to Pods. | |
**Note:** Opaque integer resources are Alpha in Kubernetes version 1.5. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we can drop this note entirely now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agreed |
||
Only resource accounting is implemented; node-level isolation is still | ||
under active development. | ||
{: .note} | ||
|
||
Opaque integer resources are resources that begin with the prefix | ||
`pod.alpha.kubernetes.io/opaque-int-resource-`. The API server | ||
|
@@ -395,6 +405,106 @@ spec: | |
pod.alpha.kubernetes.io/opaque-int-resource-foo: 1 | ||
``` | ||
|
||
## Extended Resources | ||
|
||
Kubernetes version 1.8 introduces Extended Resources. Extended Resources are | ||
fully-qualified resource names outside the `kubernetes.io` domain. Extended | ||
Resources allow cluster operators to advertise new node-level resources that | ||
would be otherwise unknown to the system. Extended Resource quantities must be | ||
integers and cannot be overcommitted. | ||
|
||
Users can consume Extended Resources in Pod specs just like CPU and memory. | ||
The scheduler takes care of the resource accounting so that no more than the | ||
available amount is simultaneously allocated to Pods. | ||
|
||
The API server restricts quantities of Extended Resources to whole numbers. | ||
Examples of _valid_ quantities are `3`, `3000m` and `3Ki`. Examples of | ||
_invalid_ quantities are `0.5` and `1500m`. | ||
|
||
**Note:** Extended Resources replace [Opaque Integer | ||
Resources](#opaque-integer-resources-alpha-feature). Users can use any domain | ||
name prefix outside of the `kubernetes.io/` domain instead of the previous | ||
`pod.alpha.kubernetes.io/opaque-int-resource-` prefix. | ||
{: .note} | ||
|
||
There are two steps required to use Extended Resources. First, the | ||
cluster operator must advertise a per-node opaque resource on one or more | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/opaque/extended or s/opaque// |
||
nodes. Second, users must request the Extended Resource in Pods. | ||
|
||
To advertise a new Extended Resource, the cluster operator should | ||
submit a `PATCH` HTTP request to the API server to specify the available | ||
quantity in the `status.capacity` for a node in the cluster. After this | ||
operation, the node's `status.capacity` will include a new resource. The | ||
`status.allocatable` field is updated automatically with the new resource | ||
asynchronously by the kubelet. Note that because the scheduler uses the | ||
node `status.allocatable` value when evaluating Pod fitness, there may | ||
be a short delay between patching the node capacity with a new resource and the | ||
first pod that requests the resource to be scheduled on that node. | ||
|
||
**Example:** | ||
|
||
Here is an HTTP request that advertises five "example.com/foo" resources on | ||
node `k8s-node-1` whose master is `k8s-master`. | ||
|
||
```http | ||
PATCH /api/v1/nodes/k8s-node-1/status HTTP/1.1 | ||
Accept: application/json | ||
Content-Type: application/json-patch+json | ||
Host: k8s-master:8080 | ||
|
||
[ | ||
{ | ||
"op": "add", | ||
"path": "/status/capacity/example.com~1foo", | ||
"value": "5" | ||
} | ||
] | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This http request detail is not strictly necessary because the following Shell command already covers all necessary information. So ... I'd suggest we remove it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, should we remove the similar one from the OIR section above too? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If there is no meat in it, better throw it away IMO. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest to remain it. It's very important for usual users. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @zhangxiaoyu-zidif The snippet above is not a JSON or an YAML for you to create resource from Shell. It is an explanation of the HTTP request internals, and the information is duplicated with the command below. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry to misunderstand your response. The command below is necessary. And the above one can be remove. It has no meaning to common users. But the below shell command is useful. =) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My historical note is that I first added the HTTP request, then reviewers asked for a |
||
|
||
```shell | ||
curl --header "Content-Type: application/json-patch+json" \ | ||
--request PATCH \ | ||
--data '[{"op": "add", "path": "/status/capacity/example.com~1foo", "value": "5"}]' \ | ||
http://k8s-master:8080/api/v1/nodes/k8s-node-1/status | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the '/status' part should be removed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @ConnorDoyle There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @tengqm the URL shown is the correct way to patch the node status. |
||
``` | ||
|
||
**Note**: In the preceding request, `~1` is the encoding for the character `/` | ||
in the patch path. The operation path value in JSON-Patch is interpreted as a | ||
JSON-Pointer. For more details, see | ||
[IETF RFC 6901, section 3](https://tools.ietf.org/html/rfc6901#section-3). | ||
{: .note} | ||
|
||
To consume an Extended Resource in a Pod, include the resource name as a key | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. better be explicit here: .. include the resource name There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I meant this sentence to be more general than the specific resource from the example. Did that not come across? This is the second of two steps required to use ERs. The previous section began with "To advertise a new Extended Resource..." There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alright, if it is referring to general cases, i.e. "an Extended Resource", it is fine. |
||
in the `spec.containers[].resources.requests` map. | ||
|
||
**Note:** Extended resources cannot be overcommitted, so request and limit | ||
must be equal if both are present in a container spec. | ||
{: .note} | ||
|
||
The Pod is scheduled only if all of the resource requests are | ||
satisfied, including cpu, memory and any Extended Resources. The Pod will | ||
remain in the `PENDING` state as long as the resource request cannot be met by | ||
any node. | ||
|
||
**Example:** | ||
|
||
The Pod below requests 2 cpus and 1 "example.com/foo" (an extended resource.) | ||
|
||
```yaml | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: my-pod | ||
spec: | ||
containers: | ||
- name: my-container | ||
image: myimage | ||
resources: | ||
requests: | ||
cpu: 2 | ||
example.com/foo: 1 | ||
``` | ||
|
||
## Planned Improvements | ||
|
||
Kubernetes version 1.5 only allows resource quantities to be specified on a | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move this note below https://github.com/kubernetes/kubernetes.github.io/pull/5399/files?diff=unified#diff-4c9401ed6b037e1adb958cbce20630c7L321.