Add MongoDB scaler #1467

NUCsimple · 2020-12-31T08:44:27Z

Signed-off-by: carson carson_192@163.com

This PR adds a new scaler for the mongo.

Checklist

Commits are signed with Developer Certificate of Origin (DCO)
Tests have been added
A PR is opened to update the documentation on https://github.com/kedacore/keda-docs (Add Mongo scaler documentation keda-docs#343)
Changelog has been updated

Related to #1465.

tomkerkhove · 2021-01-01T13:38:17Z

Can you fix DCO & our static checks please?

tomkerkhove

I'm wondering if we should call it mongo or mongodb?

tomkerkhove · 2021-01-01T13:38:29Z

CHANGELOG.md

@@ -19,6 +19,7 @@
 ### New
 - Can use Pod Identity with Azure Event Hub scaler ([#994](https://github.com/kedacore/keda/issues/994))
 - Introducing InfluxDB scaler ([#1239](https://github.com/kedacore/keda/issues/1239))
+- Introducing Mongo scaler ([#1465](https://github.com/kedacore/keda/issues/1465))


Suggested change

- Introducing Mongo scaler ([#1465](https://github.com/kedacore/keda/issues/1465))

- Introducing MongoDB scaler ([#1465](https://github.com/kedacore/keda/issues/1465))

Yeah, I think it's better to call it mongodb scaler.

We should keep that naming consistent, in the go code all the references to the scaler are as mongo not mongodb. Shouldn't we change that as well?

You are right, i will change them to mongoDB,at the same time,add e2e tests for this scaler.

NUCsimple · 2021-01-01T16:02:43Z

Can you fix DCO & our static checks please?

DCO is fixed,but static checks failed again, bacause of the cache.

ringtail · 2021-01-04T08:43:48Z

@tomkerkhove Could we trigger the pre-commit re-test.

zroubalik · 2021-01-04T09:30:12Z

@NUCsimple thanks for the contribution! I'll get to the proper review later. Could you please add e2e tests for this scaler?

pkg/scalers/mongo.go

NUCsimple · 2021-01-14T06:46:22Z

Hi, @zroubalik .Coud you please check it again? I have changed all the non-standard contents mentioned above.

zroubalik

Minor formating related nits and mongoDB -> mongodb for consistency

zroubalik · 2021-01-14T12:44:37Z

pkg/scalers/mongo.go

@@ -0,0 +1,286 @@
+package scalers


Could you please name this file mongodb_scaler.go so it is consistent with other scalers in the package.

zroubalik · 2021-01-14T12:44:53Z

pkg/scalers/mongo.go

+	defaultQueryValue     = 1
+)
+
+var mongoDBLog = logf.Log.WithName("mongoDB_scaler")


Suggested change

var mongoDBLog = logf.Log.WithName("mongoDB_scaler")

var mongoDBLog = logf.Log.WithName("mongodb_scaler")

zroubalik · 2021-01-14T12:45:57Z

pkg/scalers/mongo_test.go

@@ -0,0 +1,87 @@
+package scalers


Could you please name this file mongodb_scaler_test.go so it is consistent with other tests in the package.

zroubalik · 2021-01-14T12:46:49Z

pkg/scaling/scale_handler.go

@@ -474,6 +474,8 @@ func buildScaler(triggerType string, config *scalers.ScalerConfig) (scalers.Scal
 		return scalers.NewCPUMemoryScaler(corev1.ResourceMemory, config)
 	case "metrics-api":
 		return scalers.NewMetricsAPIScaler(config)
+	case "mongoDB":


Suggested change

case "mongoDB":

case "mongodb":

I'd make this lowercase, to be consistent with other scalers

zroubalik · 2021-01-14T12:57:06Z

pkg/scalers/mongo.go

+func (s *mongoDBScaler) GetMetrics(ctx context.Context, metricName string, metricSelector labels.Selector) ([]external_metrics.ExternalMetricValue, error) {
+	num, err := s.getQueryResult()
+	if err != nil {
+		return []external_metrics.ExternalMetricValue{}, fmt.Errorf("failed to inspect momgo,because of %v", err)


Suggested change

return []external_metrics.ExternalMetricValue{}, fmt.Errorf("failed to inspect momgo,because of %v", err)

return []external_metrics.ExternalMetricValue{}, fmt.Errorf("failed to inspect mongoDB, because of %v", err)

zroubalik · 2021-01-14T12:58:26Z

pkg/scalers/mongo.go

+	}
+
+	if err = client.Ping(ctx, readpref.Primary()); err != nil {
+		return nil, fmt.Errorf("failed to ping mongoDB,because of %v", err)


Suggested change

return nil, fmt.Errorf("failed to ping mongoDB,because of %v", err)

return nil, fmt.Errorf("failed to ping mongoDB, because of %v", err)

zroubalik · 2021-01-14T12:58:37Z

pkg/scalers/mongo.go

+	if val, ok := config.TriggerMetadata["queryValue"]; ok {
+		queryValue, err := strconv.Atoi(val)
+		if err != nil {
+			return nil, "", fmt.Errorf("failed to convert %v to int,because of %v", queryValue, err.Error())


Suggested change

return nil, "", fmt.Errorf("failed to convert %v to int,because of %v", queryValue, err.Error())

return nil, "", fmt.Errorf("failed to convert %v to int, because of %v", queryValue, err.Error())

zroubalik · 2021-01-14T12:58:55Z

pkg/scalers/mongo.go

+	if s.client != nil {
+		err := s.client.Disconnect(context.TODO())
+		if err != nil {
+			mongoDBLog.Error(err, fmt.Sprintf("failed to close mongoDB connection,because of %v", err))


Suggested change

mongoDBLog.Error(err, fmt.Sprintf("failed to close mongoDB connection,because of %v", err))

mongoDBLog.Error(err, fmt.Sprintf("failed to close mongoDB connection, because of %v", err))

zroubalik · 2021-01-14T12:59:05Z

pkg/scalers/mongo.go

+
+	filter, err := json2BsonDoc(s.metadata.query)
+	if err != nil {
+		mongoDBLog.Error(err, fmt.Sprintf("failed to convert query param to bson.Doc,because of %v", err))


Suggested change

mongoDBLog.Error(err, fmt.Sprintf("failed to convert query param to bson.Doc,because of %v", err))

mongoDBLog.Error(err, fmt.Sprintf("failed to convert query param to bson.Doc, because of %v", err))

zroubalik · 2021-01-14T12:59:17Z

pkg/scalers/mongo.go

+
+	docsNum, err := s.client.Database(s.metadata.dbName).Collection(s.metadata.collection).CountDocuments(ctx, filter)
+	if err != nil {
+		mongoDBLog.Error(err, fmt.Sprintf("failed to query %v in %v,because of %v", s.metadata.dbName, s.metadata.collection, err))


Suggested change

mongoDBLog.Error(err, fmt.Sprintf("failed to query %v in %v,because of %v", s.metadata.dbName, s.metadata.collection, err))

mongoDBLog.Error(err, fmt.Sprintf("failed to query %v in %v, because of %v", s.metadata.dbName, s.metadata.collection, err))

Signed-off-by: 高强 <wb-gq555900@alibaba-inc.com>

zroubalik

LGTM, thanks for the contribution!

NUCsimple requested review from ahmelsayed and zroubalik as code owners December 31, 2020 08:44

tomkerkhove reviewed Jan 1, 2021

View reviewed changes

NUCsimple force-pushed the add_mongo_scaler branch 2 times, most recently from 143197b to f17a965 Compare January 1, 2021 15:27

zroubalik requested changes Jan 4, 2021

View reviewed changes

pkg/scalers/mongo.go Outdated Show resolved Hide resolved

zroubalik changed the title ~~add mongo scaler~~ Add MongoDB scaler Jan 4, 2021

NUCsimple force-pushed the add_mongo_scaler branch 3 times, most recently from dc989e6 to 974ed74 Compare January 10, 2021 14:32

zroubalik reviewed Jan 14, 2021

View reviewed changes

add mongoDB scaler

3fb066e

Signed-off-by: 高强 <wb-gq555900@alibaba-inc.com>

NUCsimple force-pushed the add_mongo_scaler branch from d709486 to 3fb066e Compare January 15, 2021 03:21

add go.mod && go.sum

30ff930

Signed-off-by: 高强 <wb-gq555900@alibaba-inc.com>

zroubalik approved these changes Jan 15, 2021

View reviewed changes

tomkerkhove merged commit efbcbe5 into kedacore:main Jan 15, 2021

ycabrer pushed a commit to ycabrer/keda that referenced this pull request Mar 1, 2021

Add MongoDB scaler (kedacore#1467)

958e225

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MongoDB scaler #1467

Add MongoDB scaler #1467

NUCsimple commented Dec 31, 2020

tomkerkhove commented Jan 1, 2021

tomkerkhove left a comment

tomkerkhove Jan 1, 2021

NUCsimple Jan 1, 2021

zroubalik Jan 4, 2021

NUCsimple Jan 4, 2021

zroubalik Jan 4, 2021

NUCsimple commented Jan 1, 2021

ringtail commented Jan 4, 2021

zroubalik commented Jan 4, 2021

NUCsimple commented Jan 14, 2021

zroubalik left a comment

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik Jan 14, 2021

zroubalik left a comment

	- Introducing Mongo scaler ([#1465](https://github.com/kedacore/keda/issues/1465))
	- Introducing MongoDB scaler ([#1465](https://github.com/kedacore/keda/issues/1465))

	var mongoDBLog = logf.Log.WithName("mongoDB_scaler")
	var mongoDBLog = logf.Log.WithName("mongodb_scaler")

	return []external_metrics.ExternalMetricValue{}, fmt.Errorf("failed to inspect momgo,because of %v", err)
	return []external_metrics.ExternalMetricValue{}, fmt.Errorf("failed to inspect mongoDB, because of %v", err)

	return nil, fmt.Errorf("failed to ping mongoDB,because of %v", err)
	return nil, fmt.Errorf("failed to ping mongoDB, because of %v", err)

	return nil, "", fmt.Errorf("failed to convert %v to int,because of %v", queryValue, err.Error())
	return nil, "", fmt.Errorf("failed to convert %v to int, because of %v", queryValue, err.Error())

	mongoDBLog.Error(err, fmt.Sprintf("failed to close mongoDB connection,because of %v", err))
	mongoDBLog.Error(err, fmt.Sprintf("failed to close mongoDB connection, because of %v", err))

Add MongoDB scaler #1467

Add MongoDB scaler #1467

Conversation

NUCsimple commented Dec 31, 2020

Checklist

tomkerkhove commented Jan 1, 2021

tomkerkhove left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NUCsimple commented Jan 1, 2021

ringtail commented Jan 4, 2021

zroubalik commented Jan 4, 2021

NUCsimple commented Jan 14, 2021

zroubalik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zroubalik left a comment

Choose a reason for hiding this comment