feat: Add ModelRegistry component (opendatahub-io#775) (opendatahub-i…

…o#776) Squashed commit due to buildability since ComponentInterface has changed. Other patches squashed as well to avoid double squashing due to merge policy. modelregistry: regenerate autogenerated files Run `make generate manifests` after all the changes Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> feat: Add ModelRegistry component (opendatahub-io#775) (opendatahub-io#776) * feat: Add ModelRegistry component (opendatahub-io#775) * fix: Fix modelregistry odh overlays path * fix: fix dsc_create_test tests err nil check * fix: refactor ModelRegistry.ReconcileComponent for new parameters * chore: added modelregistry to README.md * fix: add missing rbac rules for deploymentconfigs and daemonsets * chore: code lint cleanup * fix: added check for nil DevFlags in model-registry component * fix: add nil check for dscispec.DevFlags in model-registry ReconcileComponent * fix: remove RBAC rules for daemonsets and deploymentconfigs * fix(chore): fix lint errors in dsc_deletion_test.go (cherry picked from commit 112d3f1) Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> modelregistry: partial: chore: removes SetImageParamsMap from ComponentInterface (opendatahub-io#897) Partial application of already applied commit d10a764 Author: Bartosz Majsak <bartosz.majsak@gmail.com> Date: Thu Mar 7 15:43:37 2024 +0100 chore: removes SetImageParamsMap from ComponentInterface (opendatahub-io#897) As it's not used by any component, acting as a simple pass-return loop. This makes the API contract a bit cleaner. Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> modelregistry: partial: chore: remove the need of passing rest config (opendatahub-io#895) Partial application of already applied commit ca7fa98 Author: Bartosz Majsak <bartosz.majsak@gmail.com> Date: Fri Mar 8 17:40:54 2024 +0100 chore: remove the need of passing rest config (opendatahub-io#895) * chore: fixes ComponentInterface docs By removing reference to non-existing func. This function has been in use outside of this component. * fix: removes rest config As we are already using client.Client interface we do not have to instantiate other typed clients to e.g. list resources using their own funcs. Generic client.Client is sufficient for these needs. Additionally this change adds ctx propogation for these calls. Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> modelregistry: partial: feat(logger): for both controller level and component level (opendatahub-io#837) Partial application of already applied commit d8a83a2 Author: Wen Zhou <wenzhou@redhat.com> Date: Mon Apr 1 22:06:16 2024 +0200 feat(logger): for both controller level and component level (opendatahub-io#837) * feat(logger): for both controller level and component level Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update(logger): use logr instead of uber's zap Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: do not log error only print Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: use zap.Options for both and tune levels Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: move setting into common function Signed-off-by: Wen Zhou <wenzhou@redhat.com> Signed-off-by: Zhou, Wen <wenzhou@redhat.com> --------- Signed-off-by: Wen Zhou <wenzhou@redhat.com> Signed-off-by: Zhou, Wen <wenzhou@redhat.com> Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> update(modelregistry): rename image name (opendatahub-io#877) Signed-off-by: Wen Zhou <wenzhou@redhat.com> (cherry picked from commit b4e4d6f) modelregistry: partial: chore: cleanup duplicated functions packages and add more for godoc (opendatahub-io#981) Partial application of already applied commit 96c85f2 Author: Wen Zhou <wenzhou@redhat.com> Date: Tue Apr 23 14:05:24 2024 +0200 chore: cleanup duplicated functions packages and add more for godoc (opendatahub-io#981) * chore: cleanup duplicated functions/package and add godoc - move GetPlatform() from deploy package to cluster package - move const ManagedRhods SelfManagedRhods OpenDataHub from deploy to cluster package - move WaitForDeploymentAvailable() monitoring package to cluster package - remove monitoring package - move UpdatePodSecurityRolebinding() from common package to cluster package - deprecate GetDomain from common package, to only use GetDomain from cluster package. - remove gvk package, move its GVK to cluster package - move DeleteExistingSubscription() from deploy package to upgrade package - do not export getSubscription() Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: remove gvk into one file but under cluster package Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: rename variable, removing GVK from it Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: move gvk into a sub package under cluster Signed-off-by: Wen Zhou <wenzhou@redhat.com> --------- Signed-off-by: Wen Zhou <wenzhou@redhat.com> Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> feat(mr): create namespace for Model Registry (opendatahub-io#930) * feat(mr): create namespace for smm Signed-off-by: Wen Zhou <wenzhou@redhat.com> * fix: rebase Signed-off-by: Zhou, Wen <wenzhou@redhat.com> * update: code review comments Signed-off-by: Wen Zhou <wenzhou@redhat.com> * fix(doc): wrong comments Signed-off-by: Wen Zhou <wenzhou@redhat.com> * update: remove label to keep namespace even opreator is uninstalled Signed-off-by: Wen Zhou <wenzhou@redhat.com> --------- Signed-off-by: Wen Zhou <wenzhou@redhat.com> Signed-off-by: Zhou, Wen <wenzhou@redhat.com> (cherry picked from commit 1188ce1) feat(mr): add model registry odh extras manifests, fixes RHOAIENG-5112 (opendatahub-io#953) (cherry picked from commit 7c3e81b) modelregistry: partial: chore: Open up util functions for context propagation (opendatahub-io#1033) Partial application of already applied commit 105adae Author: Aslak Knutsen <aslak@4fs.no> Date: Tue Jun 4 15:16:21 2024 +0200 chore: Open up util functions for context propagation (opendatahub-io#1033) context should be determined by the caller and propagated down the call chain. Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> modelregistry: partial: chore: remove duplicated platform call in each component (opendatahub-io#1055) Partial application of already applied commit 1b04761 Author: Wen Zhou <wenzhou@redhat.com> Date: Fri Jun 14 14:47:33 2024 +0200 chore: remove duplicated platform call in each component (opendatahub-io#1055) - get in DSC and pass into compoment Signed-off-by: Wen Zhou <wenzhou@redhat.com> Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com> modelregistry: update api docs run `make api-docs` add +groupName=datasciencecluster.opendatahub.io On backporting of 1b86e42 ("Update readme.md (opendatahub-io#890)") Signed-off-by: Yauheni Kaliuta <ykaliuta@redhat.com>
ykaliuta · Aug 27, 2024 · 32f59c6 · 32f59c6
1 parent c4446d6
commit 32f59c6
Show file tree

Hide file tree

Showing 15 changed files with 338 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -317,6 +317,8 @@ spec:
       managementState: Managed
     workbenches:
       managementState: Managed
+    modelregistry:
+      managementState: Managed
 ```
 
 2. Enable only Dashboard and Workbenches

diff --git a/apis/datasciencecluster/v1/datasciencecluster_types.go b/apis/datasciencecluster/v1/datasciencecluster_types.go
@@ -31,6 +31,7 @@ import (
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/kserve"
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/kueue"
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/modelmeshserving"
+	"github.com/opendatahub-io/opendatahub-operator/v2/components/modelregistry"
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/ray"
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/trainingoperator"
 	"github.com/opendatahub-io/opendatahub-operator/v2/components/trustyai"
@@ -80,6 +81,9 @@ type Components struct {
 
 	//Training Operator component configuration.
 	TrainingOperator trainingoperator.TrainingOperator `json:"trainingoperator,omitempty"`
+
+	// ModelRegistry component configuration.
+	ModelRegistry modelregistry.ModelRegistry `json:"modelregistry,omitempty"`
 }
 
 // DataScienceClusterStatus defines the observed state of DataScienceCluster.

diff --git a/apis/datasciencecluster/v1/zz_generated.deepcopy.go b/apis/datasciencecluster/v1/zz_generated.deepcopy.go
diff --git a/bundle/manifests/datasciencecluster.opendatahub.io_datascienceclusters.yaml b/bundle/manifests/datasciencecluster.opendatahub.io_datascienceclusters.yaml
@@ -385,6 +385,49 @@ spec:
                         pattern: ^(Managed|Unmanaged|Force|Removed)$
                         type: string
                     type: object
+                  modelregistry:
+                    description: ModelRegistry component configuration.
+                    properties:
+                      devFlags:
+                        description: Add developer fields
+                        properties:
+                          manifests:
+                            description: List of custom manifests for the given component
+                            items:
+                              properties:
+                                contextDir:
+                                  default: ""
+                                  description: contextDir is the relative path to
+                                    the folder containing manifests in a repository
+                                  type: string
+                                sourcePath:
+                                  default: ""
+                                  description: 'sourcePath is the subpath within contextDir
+                                    where kustomize builds start. Examples include
+                                    any sub-folder or path: `base`, `overlays/dev`,
+                                    `default`, `odh` etc'
+                                  type: string
+                                uri:
+                                  default: ""
+                                  description: uri is the URI point to a git repo
+                                    with tag/branch. e.g  https://github.com/org/repo/tarball/<tag/branch>
+                                  type: string
+                              type: object
+                            type: array
+                        type: object
+                      managementState:
+                        description: "Set to one of the following values: \n - \"Managed\"
+                          : the operator is actively managing the component and trying
+                          to keep it active. It will only upgrade the component if
+                          it is safe to do so \n - \"Removed\" : the operator is actively
+                          managing the component and will not install it, or if it
+                          is installed, the operator will try to remove it"
+                        enum:
+                        - Managed
+                        - Removed
+                        pattern: ^(Managed|Unmanaged|Force|Removed)$
+                        type: string
+                    type: object
                   ray:
                     description: Ray component configuration.
                     properties:

diff --git a/bundle/manifests/rhods-operator.clusterserviceversion.yaml b/bundle/manifests/rhods-operator.clusterserviceversion.yaml
@@ -46,6 +46,9 @@ metadata:
               "modelmeshserving": {
                 "managementState": "Managed"
               },
+              "modelregistry": {
+                "managementState": "Removed"
+              },
               "ray": {
                 "managementState": "Managed"
               },
@@ -1032,6 +1035,32 @@ spec:
           - update
           - use
           - watch
+        - apiGroups:
+          - modelregistry.opendatahub.io
+          resources:
+          - modelregistries
+          verbs:
+          - create
+          - delete
+          - get
+          - list
+          - patch
+          - update
+          - watch
+        - apiGroups:
+          - modelregistry.opendatahub.io
+          resources:
+          - modelregistries/finalizers
+          verbs:
+          - update
+        - apiGroups:
+          - modelregistry.opendatahub.io
+          resources:
+          - modelregistries/status
+          verbs:
+          - get
+          - patch
+          - update
         - apiGroups:
           - monitoring.coreos.com
           resources:

diff --git a/components/modelregistry/modelregistry.go b/components/modelregistry/modelregistry.go
@@ -0,0 +1,106 @@
+// Package modelregistry provides utility functions to config ModelRegistry, an ML Model metadata repository service
+// +groupName=datasciencecluster.opendatahub.io
+package modelregistry
+
+import (
+	"context"
+	"fmt"
+	"path/filepath"
+
+	"github.com/go-logr/logr"
+	operatorv1 "github.com/openshift/api/operator/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"sigs.k8s.io/controller-runtime/pkg/client"
+
+	dsciv1 "github.com/opendatahub-io/opendatahub-operator/v2/apis/dscinitialization/v1"
+	"github.com/opendatahub-io/opendatahub-operator/v2/components"
+	"github.com/opendatahub-io/opendatahub-operator/v2/pkg/cluster"
+	"github.com/opendatahub-io/opendatahub-operator/v2/pkg/deploy"
+)
+
+var (
+	ComponentName = "model-registry-operator"
+	Path          = deploy.DefaultManifestPath + "/" + ComponentName + "/overlays/odh"
+	// we should not apply this label to the namespace, as it triggered namspace deletion during operator uninstall
+	// modelRegistryLabels = cluster.WithLabels(
+	// 	labels.ODH.OwnedNamespace, "true",
+	// ).
+)
+
+// Verifies that ModelRegistry implements ComponentInterface.
+var _ components.ComponentInterface = (*ModelRegistry)(nil)
+
+// ModelRegistry struct holds the configuration for the ModelRegistry component.
+// +kubebuilder:object:generate=true
+type ModelRegistry struct {
+	components.Component `json:""`
+}
+
+func (m *ModelRegistry) OverrideManifests(_ string) error {
+	// If devflags are set, update default manifests path
+	if len(m.DevFlags.Manifests) != 0 {
+		manifestConfig := m.DevFlags.Manifests[0]
+		if err := deploy.DownloadManifests(ComponentName, manifestConfig); err != nil {
+			return err
+		}
+		// If overlay is defined, update paths
+		defaultKustomizePath := "overlays/odh"
+		if manifestConfig.SourcePath != "" {
+			defaultKustomizePath = manifestConfig.SourcePath
+		}
+		Path = filepath.Join(deploy.DefaultManifestPath, ComponentName, defaultKustomizePath)
+	}
+
+	return nil
+}
+
+func (m *ModelRegistry) GetComponentName() string {
+	return ComponentName
+}
+
+func (m *ModelRegistry) ReconcileComponent(ctx context.Context, cli client.Client, logger logr.Logger,
+	owner metav1.Object, dscispec *dsciv1.DSCInitializationSpec, platform cluster.Platform, _ bool) error {
+	l := m.ConfigComponentLogger(logger, ComponentName, dscispec)
+	var imageParamMap = map[string]string{
+		"IMAGES_MODELREGISTRY_OPERATOR": "RELATED_IMAGE_ODH_MODEL_REGISTRY_OPERATOR_IMAGE",
+		"IMAGES_GRPC_SERVICE":           "RELATED_IMAGE_ODH_MLMD_GRPC_SERVER_IMAGE",
+		"IMAGES_REST_SERVICE":           "RELATED_IMAGE_ODH_MODEL_REGISTRY_IMAGE",
+	}
+	enabled := m.GetManagementState() == operatorv1.Managed
+
+	if enabled {
+		if m.DevFlags != nil {
+			// Download manifests and update paths
+			if err := m.OverrideManifests(string(platform)); err != nil {
+				return err
+			}
+		}
+
+		// Update image parameters only when we do not have customized manifests set
+		if (dscispec.DevFlags == nil || dscispec.DevFlags.ManifestsUri == "") && (m.DevFlags == nil || len(m.DevFlags.Manifests) == 0) {
+			if err := deploy.ApplyParams(Path, imageParamMap, false); err != nil {
+				return fmt.Errorf("failed to update image from %s : %w", Path, err)
+			}
+		}
+
+		// Create odh-model-registries namespace
+		// We do not delete this namespace even when ModelRegistry is Removed or when operator is uninstalled.
+		_, err := cluster.CreateNamespace(ctx, cli, "odh-model-registries")
+		if err != nil {
+			return err
+		}
+	}
+	// Deploy ModelRegistry Operator
+	if err := deploy.DeployManifestsFromPath(cli, owner, Path, dscispec.ApplicationsNamespace, m.GetComponentName(), enabled); err != nil {
+		return err
+	}
+	l.Info("apply manifests done")
+
+	// Create additional model registry resources, componentEnabled=true because these extras are never deleted!
+	if err := deploy.DeployManifestsFromPath(cli, owner, Path+"/extras", dscispec.ApplicationsNamespace, m.GetComponentName(), true); err != nil {
+		return err
+	}
+	l.Info("apply extra manifests done")
+
+	return nil
+}
diff --git a/components/modelregistry/zz_generated.deepcopy.go b/components/modelregistry/zz_generated.deepcopy.go
diff --git a/config/crd/bases/datasciencecluster.opendatahub.io_datascienceclusters.yaml b/config/crd/bases/datasciencecluster.opendatahub.io_datascienceclusters.yaml
@@ -386,6 +386,50 @@ spec:
                         pattern: ^(Managed|Unmanaged|Force|Removed)$
                         type: string
                     type: object
+                  modelregistry:
+                    description: ModelRegistry component configuration.
+                    properties:
+                      devFlags:
+                        description: Add developer fields
+                        properties:
+                          manifests:
+                            description: List of custom manifests for the given component
+                            items:
+                              properties:
+                                contextDir:
+                                  default: manifests
+                                  description: contextDir is the relative path to
+                                    the folder containing manifests in a repository,
+                                    default value "manifests"
+                                  type: string
+                                sourcePath:
+                                  default: ""
+                                  description: 'sourcePath is the subpath within contextDir
+                                    where kustomize builds start. Examples include
+                                    any sub-folder or path: `base`, `overlays/dev`,
+                                    `default`, `odh` etc.'
+                                  type: string
+                                uri:
+                                  default: ""
+                                  description: uri is the URI point to a git repo
+                                    with tag/branch. e.g.  https://github.com/org/repo/tarball/<tag/branch>
+                                  type: string
+                              type: object
+                            type: array
+                        type: object
+                      managementState:
+                        description: "Set to one of the following values: \n - \"Managed\"
+                          : the operator is actively managing the component and trying
+                          to keep it active. It will only upgrade the component if
+                          it is safe to do so \n - \"Removed\" : the operator is actively
+                          managing the component and will not install it, or if it
+                          is installed, the operator will try to remove it"
+                        enum:
+                        - Managed
+                        - Removed
+                        pattern: ^(Managed|Unmanaged|Force|Removed)$
+                        type: string
+                    type: object
                   ray:
                     description: Ray component configuration.
                     properties:

diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml
@@ -785,6 +785,32 @@ rules:
   - update
   - use
   - watch
+- apiGroups:
+  - modelregistry.opendatahub.io
+  resources:
+  - modelregistries
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - modelregistry.opendatahub.io
+  resources:
+  - modelregistries/finalizers
+  verbs:
+  - update
+- apiGroups:
+  - modelregistry.opendatahub.io
+  resources:
+  - modelregistries/status
+  verbs:
+  - get
+  - patch
+  - update
 - apiGroups:
   - monitoring.coreos.com
   resources:

diff --git a/config/samples/datasciencecluster_v1_datasciencecluster.yaml b/config/samples/datasciencecluster_v1_datasciencecluster.yaml
@@ -39,4 +39,6 @@ spec:
     workbenches:
       managementState: "Managed"
     trustyai:
+      managementState: "Removed"      
+    modelregistry:
       managementState: "Removed"
diff --git a/controllers/datasciencecluster/kubebuilder_rbac.go b/controllers/datasciencecluster/kubebuilder_rbac.go
@@ -122,6 +122,10 @@ package datasciencecluster
 // +kubebuilder:rbac:groups="monitoring.coreos.com",resources=probes,verbs=get;create;patch;delete;deletecollection
 // +kubebuilder:rbac:groups="monitoring.coreos.com",resources=prometheusrules,verbs=get;create;patch;delete;deletecollection
 
+// +kubebuilder:rbac:groups=modelregistry.opendatahub.io,resources=modelregistries,verbs=get;list;watch;create;update;patch;delete
+// +kubebuilder:rbac:groups=modelregistry.opendatahub.io,resources=modelregistries/status,verbs=get;update;patch
+// +kubebuilder:rbac:groups=modelregistry.opendatahub.io,resources=modelregistries/finalizers,verbs=update
+
 // +kubebuilder:rbac:groups="monitoring.coreos.com",resources=prometheuses/finalizers,verbs=get;create;patch;delete;deletecollection
 // +kubebuilder:rbac:groups="monitoring.coreos.com",resources=prometheuses/status,verbs=get;create;patch;delete;deletecollection