Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat/nfd-master: configure CR restrictions #1592

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions deployment/base/rbac/master-clusterrole.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@ kind: ClusterRole
metadata:
name: nfd-master
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- watch
- list
- apiGroups:
- ""
resources:
Expand Down
15 changes: 15 additions & 0 deletions deployment/components/master-config/nfd-master.conf.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,21 @@
# enableTaints: false
# labelWhiteList: "foo"
# resyncPeriod: "2h"
# restrictions:
# disableLabels: true
# disableTaints: true
# disableExtendedResources: true
# disableAnnotations: true
# allowOverwrite: false
# denyNodeFeatureLabels: true
# nodeFeatureNamespaceSelector:
# matchLabels:
# kubernetes.io/metadata.name: "node-feature-discovery"
# matchExpressions:
# - key: "kubernetes.io/metadata.name"
# operator: "In"
# values:
# - "node-feature-discovery"
# klog:
# addDirHeader: false
# alsologtostderr: false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ metadata:
labels:
{{- include "node-feature-discovery.labels" . | nindent 4 }}
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- watch
- list
- apiGroups:
- ""
resources:
Expand Down
15 changes: 15 additions & 0 deletions deployment/helm/node-feature-discovery/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,21 @@ master:
# enableTaints: false
# labelWhiteList: "foo"
# resyncPeriod: "2h"
# restrictions:
# disableLabels: true
# disableTaints: true
# disableExtendedResources: true
# disableAnnotations: true
# allowOverwrite: false
# denyNodeFeatureLabels: true
# nodeFeatureNamespaceSelector:
# matchLabels:
# kubernetes.io/metadata.name: "node-feature-discovery"
# matchExpressions:
# - key: "kubernetes.io/metadata.name"
# operator: "In"
# values:
# - "node-feature-discovery"
# klog:
# addDirHeader: false
# alsologtostderr: false
Expand Down
101 changes: 101 additions & 0 deletions docs/reference/master-configuration-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -338,3 +338,104 @@ Comma-separated list of `pattern=N` settings for file-filtered logging.
Default: *empty*

Run-time configurable: yes

## restrictions (EXPERIMENTAL)

The following options specify the restrictions that can be applied by the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment. Should we call this EXPERIMENTAL in this release(?) There have been so many subtle details during the review process that we've certainly missed something 😊 WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please. I will create another issue to cover those unseen details. If you have some bullet points in mind you can leave them here.

nfd-master on the deployed Custom Resources in the cluster.

### restrictions.nodeFeatureNamespaceSelector

The `nodeFeatureNamespaceSelector` option specifies the NodeFeatures namespaces
to watch, which can be selected by using `metav1.LabelSelector` as a type for
this option. An empty value selects all namespaces to be watched.

Default: *empty*

Example:

```yaml
restrictions:
nodeFeatureNamespaceSelector:
matchLabels:
kubernetes.io/metadata.name: "node-feature-discovery"
matchExpressions:
- key: "kubernetes.io/metadata.name"
operator: "In"
values:
- "node-feature-discovery"
```

### restrictions.disableLabels

The `disableLabels` option controls whether to allow creation of node labels
from NodeFeature and NodeFeatureRule CRs or not.

Default: false

Example:

```yaml
restrictions:
disableLabels: true
```

### restrictions.disableExtendedResources

The `disableExtendedResources` option controls whether to allow creation of
node extended resources from NodeFeatureRule CR or not.

Default: false

Example:

```yaml
restrictions:
disableExtendedResources: true
```

### restrictions.disableAnnotations

he `disableAnnotations` option controls whether to allow creation of node annotations
from NodeFeatureRule CR or not.

Default: false

Example:

```yaml
restrictions:
disableAnnotations: true
```

### restrictions.allowOverwrite

The `allowOverwrite` option controls whether NFD is allowed to overwrite and
take over management of existing node labels, annotations, and extended resources.
Labels, annotations and extended resources created by NFD itself are not affected
(overwrite cannot be disabled). NFD tracks the labels, annotations and extended
resources that it manages with specific
[node annotations](../get-started/introduction.md#node-annotations).

Default: true

Example:

```yaml
restrictions:
allowOverwrite: false
```

### restrictions.denyNodeFeatureLabels

The `denyNodeFeatureLabels` option specifies whether to deny labels from 3rd party
NodeFeature objects or not. NodeFeature objects created by nfd-worker are not affected.

Default: false

Example:

```yaml
restrictions:
denyNodeFeatureLabels: true
```
58 changes: 58 additions & 0 deletions pkg/nfd-master/namespace-lister.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
Copyright 2024 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package nfdmaster

import (
"time"

corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/client-go/informers"
k8sclient "k8s.io/client-go/kubernetes"
v1lister "k8s.io/client-go/listers/core/v1"
)

// NamespaceLister lists kubernetes namespaces.
type NamespaceLister struct {
namespaceLister v1lister.NamespaceLister
labelsSelector labels.Selector
stopChan chan struct{}
}

func newNamespaceLister(k8sClient k8sclient.Interface, labelsSelector labels.Selector) *NamespaceLister {
factory := informers.NewSharedInformerFactory(k8sClient, time.Hour)
namespaceLister := factory.Core().V1().Namespaces().Lister()

stopChan := make(chan struct{})
factory.Start(stopChan) // runs in background
factory.WaitForCacheSync(stopChan)

return &NamespaceLister{
namespaceLister: namespaceLister,
labelsSelector: labelsSelector,
stopChan: stopChan,
}
}

// list returns all kubernetes namespaces.
func (lister *NamespaceLister) list() ([]*corev1.Namespace, error) {
return lister.namespaceLister.List(lister.labelsSelector)
}

// stop closes the channel used by the lister
func (lister *NamespaceLister) stop() {
close(lister.stopChan)
}
50 changes: 45 additions & 5 deletions pkg/nfd-master/nfd-api-controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
k8sclient "k8s.io/client-go/kubernetes"
restclient "k8s.io/client-go/rest"
"k8s.io/client-go/tools/cache"
"k8s.io/klog/v2"
Expand All @@ -46,12 +47,16 @@ type nfdController struct {
updateOneNodeChan chan string
updateAllNodeFeatureGroupsChan chan struct{}
updateNodeFeatureGroupChan chan string

namespaceLister *NamespaceLister
}

type nfdApiControllerOptions struct {
DisableNodeFeature bool
DisableNodeFeatureGroup bool
ResyncPeriod time.Duration
DisableNodeFeature bool
DisableNodeFeatureGroup bool
ResyncPeriod time.Duration
K8sClient k8sclient.Interface
NodeFeatureNamespaceSelector *metav1.LabelSelector
}

func init() {
Expand All @@ -67,8 +72,16 @@ func newNfdController(config *restclient.Config, nfdApiControllerOptions nfdApiC
updateNodeFeatureGroupChan: make(chan string),
}

nfdClient := nfdclientset.NewForConfigOrDie(config)
if nfdApiControllerOptions.NodeFeatureNamespaceSelector != nil {
labelMap, err := metav1.LabelSelectorAsSelector(nfdApiControllerOptions.NodeFeatureNamespaceSelector)
if err != nil {
klog.ErrorS(err, "failed to convert label selector to map", "selector", nfdApiControllerOptions.NodeFeatureNamespaceSelector)
return nil, err
}
c.namespaceLister = newNamespaceLister(nfdApiControllerOptions.K8sClient, labelMap)
}

nfdClient := nfdclientset.NewForConfigOrDie(config)
klog.V(2).InfoS("initializing new NFD API controller", "options", utils.DelayedDumper(nfdApiControllerOptions))

informerFactory := nfdinformers.NewSharedInformerFactory(nfdClient, nfdApiControllerOptions.ResyncPeriod)
Expand All @@ -89,7 +102,11 @@ func newNfdController(config *restclient.Config, nfdApiControllerOptions nfdApiC
AddFunc: func(obj interface{}) {
nfr := obj.(*nfdv1alpha1.NodeFeature)
klog.V(2).InfoS("NodeFeature added", "nodefeature", klog.KObj(nfr))
c.updateOneNode("NodeFeature", nfr)
if c.isNamespaceSelected(nfr.Namespace) {
c.updateOneNode("NodeFeature", nfr)
} else {
klog.V(2).InfoS("NodeFeature namespace is not selected, skipping", "nodefeature", klog.KObj(nfr))
}
if !nfdApiControllerOptions.DisableNodeFeatureGroup {
c.updateAllNodeFeatureGroups()
}
Expand Down Expand Up @@ -187,6 +204,7 @@ func newNfdController(config *restclient.Config, nfdApiControllerOptions nfdApiC

func (c *nfdController) stop() {
close(c.stopChan)
c.namespaceLister.stop()
}

func getNodeNameForObj(obj metav1.Object) (string, error) {
Expand All @@ -209,6 +227,28 @@ func (c *nfdController) updateOneNode(typ string, obj metav1.Object) {
c.updateOneNodeChan <- nodeName
}

func (c *nfdController) isNamespaceSelected(namespace string) bool {
// this means that the user didn't specify any namespace selector
// which means that we allow all namespaces
if c.namespaceLister == nil {
return true
}

namespaces, err := c.namespaceLister.list()
if err != nil {
klog.ErrorS(err, "failed to query namespaces by the namespace lister")
return false
}

for _, ns := range namespaces {
if ns.Name == namespace {
return true
}
}

return false
}

func (c *nfdController) updateAllNodes() {
select {
case c.updateAllNodesChan <- struct{}{}:
Expand Down
Loading