metric.values - speed up

[leppott]

Adding 11 dominant metrics slowed up metric.values() considerably. Try to speed up.

[leppott]

Seemed mostly ok until had to define pipe, %>%. That is when it slowed down but otherwise got an error if dplyr wasn't loaded.

line 270 in metric.values.R

define pipe

%>% <- dplyr::%>%

Should clean up the extra data frames that were created for the dominant metrics. This may free up some memory and keep things from getting sluggish.

[leppott]

Only minor improvement with removing extra data frames.

Defining pipe seems to be the issue.

[leppott]

Nothing else to do at this point other than to disable the metrics if don't need.

I could add a trigger in the calling routine to set the max number of Dominant metrics to generate.

Then I could modify the code to loop through until have all of those metrics added.

This might be a good idea even if not putting into master function call.

[leppott]

Time metric.values:

[leppott]

Better but still a bit slow.

This is example number 2 in metric.values().
678 samples.

[leppott]

Ran a 19 MB, 125k records (2000+ samples) file and it took somewhere between 45 min and 1 hour.

Have added lots of metrics so may have to break apart and then merge back together.

[leppott]

Things to try:

1. group_by first or inside of summarise (current configuration).

2. Switch packages (requiring a rewrite) from dplyr::summarise to data.table.

3. Check if can add conditions to the data then evaluate them more easily in the summarise.

https://community.rstudio.com/t/dplyr-summarise-with-condition/100885/3