Uplift tree/forest: add feature importance and parallelize forest

[yungmsh]

Add feature importance

• Compute feature importance using similar methodology as scikit-learn (see below):

    cpdef compute_feature_importances(self, normalize=True):
        """Computes the importance of each feature (aka variable)."""
        cdef Node* left
        cdef Node* right
        cdef Node* nodes = self.nodes
        cdef Node* node = nodes
        cdef Node* end_node = node + self.node_count

        cdef double normalizer = 0.

        cdef np.ndarray[np.float64_t, ndim=1] importances
        importances = np.zeros((self.n_features,))
        cdef DOUBLE_t* importance_data = <DOUBLE_t*>importances.data

        with nogil:
            while node != end_node:
                if node.left_child != _TREE_LEAF:
                    # ... and node.right_child != _TREE_LEAF:
                    left = &nodes[node.left_child]
                    right = &nodes[node.right_child]

                    importance_data[node.feature] += (
                        node.weighted_n_node_samples * node.impurity -
                        left.weighted_n_node_samples * left.impurity -
                        right.weighted_n_node_samples * right.impurity)
                node += 1

        importances /= nodes[0].weighted_n_node_samples

        if normalize:
            normalizer = np.sum(importances)

            if normalizer > 0.0:
                # Avoid dividing by zero (e.g., when root is pure)
                importances /= normalizer

        return importances

• code above can be found here: https://github.com/scikit-learn/scikit-learn/blob/fd237278e895b42abe8d8d09105cbb82dc2cbba7/sklearn/tree/_tree.pyx#L1052:L1088
• Example can be found in updated notebook under examples

Parallelize Forest

• using joblib's Parallel to parallelize building trees for UpliftRandomForestClassifier.
• Speeds up ~60% for dataset with shape=(40000, 20) using 50 estimators, setting n_jobs=8 on a 16GB, 3.1GHz machine)

[paullo0106]

Do we need a self.feature_importances_.sum() > 0 check here as shown in your compute_feature_importances() reference to avoid dividing by zero (e.g., when root is pure)? This might be an extreme edge case though

[yungmsh]

Good callout- I left it out because it should be the rare extreme case (also, if the root is pure, the user is not using uplift trees correctly)

[yungmsh]

But yeah if we want to prevent an error from raising, we can add the condition

[paullo0106]

agree your point that it's minor and more of an user error, thanks for confirming!

[paullo0106]

LGTM!

[yungmsh]

Looks like the build is failing here (https://travis-ci.com/github/uber/causalml/jobs/366341859) - anyone seen this kind of error before? @jeongyoonlee @ppstacy @paullo0106

[yungmsh]

Never mind, I just re-ran the build and it's passing.