Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions lib/jobs/generate-tag.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
// Copyright 2016, EMC, Inc.

'use strict';

var di = require('di');

module.exports = generateTagJobFactory;
di.annotate(generateTagJobFactory, new di.Provide('Job.Catalog.GenerateTag'));
di.annotate(generateTagJobFactory, new di.Inject(
'Job.Base',
'Services.Waterline',
'Protocol.Events',
'JobUtils.CatalogSearchHelpers',
'anchor',
'Logger',
'Util',
'Promise',
'Assert',
'_'
));
function generateTagJobFactory(
BaseJob,
waterline,
eventsProtocol,
catalogSearch,
anchor,
Logger,
util,
Promise,
assert,
_
) {

var logger = Logger.initialize(generateTagJobFactory);

/**
*
* @param {Object} [options]
* @constructor
*/
function GenerateTagJob(options, context, taskId) {
GenerateTagJob.super_.call(this, logger, options, context, taskId);

this.nodeId = context.target || options.nodeId;
assert.isMongoId(this.nodeId, 'context.target || options.nodeId');
}

util.inherits(GenerateTagJob, BaseJob);

var RULE_DELIMITER = /\./g;

/**
* @memberOf GenerateTagJob
* @returns {Promise}
*/
GenerateTagJob.prototype._run = function run() {
var self = this;

waterline.tags.find({}).then(function (tags) {
var catalogTypes = _(tags)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just discovered this usage pattern of lodash a couple weeks ago, glad you are using it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, well, hat tip to @halfspiral then :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this chain style, seems very cool :)

.pluck('rules')
.flattenDeep()
.pluck('path')
.map(function (path) {
return _.compact(path.split(RULE_DELIMITER))[0];
})
.uniq()
.value();
return [ Promise.map(catalogTypes, function(type) {
return waterline.catalogs.findMostRecent({
node: self.nodeId,
source: type
});
}), tags ];
}).spread(function(catalogs, tags) {
catalogs = _(catalogs)
.flattenDeep()
.compact()
.transform(function (catalogs, catalog) {
catalogs[catalog.source] = catalog.data;
}, {})
.value();

var matches = _.filter( matchTags(catalogs, tags), function(match) {
return match.tag.rules.length && !match.errors.length;
});

var tagNames = _(matches).pluck('tag.name').value();
if( !_.isEmpty(tagNames)) {
return waterline.nodes.addTags(self.nodeId, tagNames);
}
}).then(function() {
self._done();
}).catch(function(err) {
self._done(err);
});
};

function matchTags(catalogs, tags) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor problem I'm anticipating, not with your code but with the existing system, is that we have to sanitize catalog key values with '.' characters in them to be '_' in order to be mongo compatible JSON (the ipmi sdr catalog as an example of this). I'm not sure we're very judicious about deserializing this data to restore the '.' characters, so I could see some tag rules producing false negatives in these cases. I'll make a follow-up ticket or something to make sure we are deserializing catalog data where necessary (even though it's likely a performance pain).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image a float value or string like 1.2 in catalog, changing the . to _ will lost its meaning. Of course, it should a rare case to use float string as key.

An alternative way is copying the design of % output in C language, we can use double . in rule path to explicitly tells it belongs to the key itself, don't consider it as delimiter.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meanwhile, I prefer to use some standard way to retrieve catalog in future, such as JSONPath or XPath or JPath
Waterline or Mongodb has its own path syntax?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yyscamper yeah I agree I think formalizing on something like JSONPath is a good idea to start work on. Mongodb doesn't support . characters within keys because the mongo query language allows for nested key searches, for example:

find({
    'key.subkey.subsubkey': <value>
})

would find a document like:

{
    "key": {
        "subkey": {
            "subsubkey": <value>
        }
    }
}

return _.map(tags, function (tag) {
var result = _.reduce(tag.rules, function (result, rule) {
var path = _.compact(rule.path.split(RULE_DELIMITER));
var depth = path.length;
var value = catalogSearch.getPath(catalogs, path.join('.'));
return {
maxDepth: Math.max(depth, result.maxDepth),
errors: result.errors.concat(anchor(value).to(_.omit(rule, 'path')) || []),
};
}, {
maxDepth: 0,
errors: []
});
result.tag = tag;
return result;
});
}

return GenerateTagJob;
}
15 changes: 15 additions & 0 deletions lib/task-data/base-tasks/generate-tag.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Copyright 2016, EMC, Inc.

'use strict';

module.exports = {
friendlyName: 'Generate Tag',
injectableName: 'Task.Base.Catalog.GenerateTag',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GenerateTag should not be a standard catalog job (For me, Standard Catalog Job means executing catalog command, parse the output and store it to database), so I am wondering whether it is suitable to set the injectableName to be nested under Catalog.

@RackHD/corecommitters may have other thought on this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yyscamper: The SKU command, which is exactly the same, is called Task.Base.Catalog.GenerateSKU. I believe they are named this way because they are initiated during discovery cataloging.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zyoung51 @yyscamper many of our injector names (especially older ones) have no strong reasoning behind them, and may not actually make sense anymore :). I think we initially nested it under Catalog because the job relied upon and searched through the catalogs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @benbp , OK for me.

runJob: 'Job.Catalog.GenerateTag',
requiredOptions: [
'nodeId'
],
requiredProperties: {},
properties: {}
};

13 changes: 13 additions & 0 deletions lib/task-data/tasks/generate-tag.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// Copyright 2016, EMC, Inc.

'use strict';

module.exports = {
friendlyName: 'Generate Tag',
injectableName: 'Task.Catalog.GenerateTag',
implementsTask: 'Task.Base.Catalog.GenerateTag',
options: {
nodeId: null
},
properties: {}
};
Loading