List/Watch/Get of objects associated with node

Author: wojtek-t

contributors/design-proposals/secure_watch.md

@@ -0,0 +1,99 @@
+# Background
+
+ As part of increasing security of a cluster, we are planning to limit the ability
+of a given Kubelet (in general: node), to be able to read only resources associated
+with it. Those resources, in particular means: secrets, configmaps &
+persistentvolumeclaims. This is needed to avoid situation when compromising node
+de facto means compromising a cluster. For more details & discussions see
+https://github.com/kubernetes/kubernetes/issues/40476.
+ 
+ However, by some extension to this effort, we would like to improve scalability
+of the system, by significantly reducing amount of api calls coming from kubelets.
+As of now, to avoid situation that kubelet is watching all secrets/configmaps/...
+in the system, it is not using watch. Instead of that, it is retrieving indidual pods,
+by sending individual GET requests. However, it is sending those requests periodically
+to enable automatic updates of mounted secrets/configmaps/... In large clusters,
+this is generating huge unnecessary load, as this load in principle should be
+watch-based. We would like to address this together with solving the authorization issue.
+
+# Proposal
+
+ In this proposal, I'm not focusing on how exactly security should be done - I'm just
+sketching very high level approach and exact authorization mechanism should be discussed
+separately.
+
+ At the high level, what we want to achieve is to enable LIST and WATCH requests to
+support filtering of "objects only attached to pods bound to a given node". We obviously
+want to be able to authorize other types of requests (in particular GETs), so that the
+design has to be consistent with that.
+
+ To solve this, I propose to introduce a new filtering mechanism (next to label selector
+and field selector): ```node selector ``` (we probably need better name though). Its
+semantic will be to filter only objects that are attached to pods bound to a given node
+and it will be supported for some predefined set of object types.
+
+# Detailed design
+
+ We will introduce the following ```node selector ``` filtering mechanism:
+
+```
+// TODO: Consider making an interface for it.
+type NodeSelector struct {
+  // TODO: Should this be repeated field to allow for some fancy controllers
+  // that will have access to multiple nodes?
+  nodeName string
+}
+```
+
+ The NodeSelector field will be added to ```ListOptions ``` (next to label & field
+selectors) and will be supported only by LIST and WATCH requests.
+
+ With that mechanism in place, all List/Watch requests coming from kubelets will have
+to have this field correctly set. We will create a dedicated admission plugin that will
+be responsible for checking if a given ```node selector ``` is allowed from a given
+client (the exact mechanism for doing this is out of scope for this doc) and either
+rejecting this request or letting it go. Note that doing this may require modifying
+admission attributes from implementation point of view.
+
+ TODO: Consider adding NodeSelector to ```GetOptions ``` - if we would do that,
+we could have unified patern from authorizing all requests from nodes and have the
+admission plugin to be relatively simple.
+
+ Once the request is authorized, we need to modify apiserver to be able to support
+the ```node selector ```. We would like to make the changes as local as possible,
+thus we will solve it at the apiserver storage layer. Going into details:
+
+1. we will create a new class ```NodeSelectorFilterer ``` (TODO: come up with
+better name) that will be implementing ```storage.Interface ```.
+2. ```NodeSelectorFilterer ``` will be a wrapper around what we are currently
+using as storage (which is implemenetation of this interface for etcd + (in case
+of most resource kinds) cacher).
+3. for List and Watch calls, we will send them to the wrapped implementation,
+catch the result, filter it based on ```node selector ``` and send the filtered
+result back to the user; all other requests will be simply forwarded to wrapped
+implementation
+4. ```NodeSelectorFilterer ``` will maintain (in-memory) mapping from an object
+(namespace/name) into list of nodes to which we have at least one pod referencing
+this object bound. This mapping will be build using standard reflector/informer
+framework by selflooping into kuberneter API.
+5. ```NodeSelectorFilterer ``` will be per resource type object (similarly as
+cacher is), thus we need to share e.g. pod informer between those.
+6. As an optimization, we should consider setting appropriate trigger function
+in cacher (based on the mapping from above that we will already have).
+
+
+ Once we have the ```NodeSelectorFilterer ``` implemented, the changes that will
+need to be done in apiserver will just be:
+
+1. Change ```SelectionPredicate ``` to contain also ```NodeSelector ``` and
+propage those from generic registry
+2. Correctly initialze the storage for every registry by wrapping the already
+existing one with the ```NodeSelectorFilterer ```
+
+
+TODO: If we bound a first pod referencing a given object to a node (or delete
+the last one), ADD watch event for the object (or DELETE) should be send to the
+watcher. The ADD shouldn't be problematic, ensure that DELETE will not cause
+problems (I think it shouldn't as delete pod means that either it was already
+removed by kubelet, or it is non-graceful deletion and it doesn't matter that
+much).