colRowSums

matrixStats: Benchmark report

---

colSums() and rowSums() benchmarks

This report benchmark the performance of colSums() and rowSums() against alternative methods.

Alternative methods

• .colSums() and .rowSums()
• apply() + sum()

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), naProb = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         X <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         X <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         X <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(X) <- mode
+     if (naProb > 0) 
+         X[sample(n, size = naProb * n)] <- NA
+     dim(X) <- c(nrow, ncol)
+     X
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   806945 43.1    1442291  77.1  1442291  77.1
Vcells 12283605 93.8   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805392 43.1    1442291  77.1  1442291  77.1
Vcells 12279209 93.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0027	0.0038	0.0049	0.0046	0.0052	0.0219
1	colSums	0.0054	0.0067	0.0101	0.0092	0.0106	0.0604
3	apply+sum	0.0562	0.0589	0.0732	0.0601	0.0860	0.2637

	expr	min	lq	mean	median	uq	max
2	.colSums	1.00	1.00	1.000	1	1.000	1.000
1	colSums	2.00	1.75	2.083	2	2.037	2.754
3	apply+sum	20.86	15.30	15.078	13	16.552	12.017
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0046	0.0058	0.0071	0.0067	0.0077	0.0216
1	rowSums	0.0100	0.0115	0.0205	0.0231	0.0266	0.0397
3	apply+sum	0.0897	0.1174	0.1292	0.1292	0.1399	0.3237

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.00	1.00	1.000	1.00	1.000
1	rowSums	2.167	2.00	2.90	3.428	3.45	1.839
3	apply+sum	19.415	20.33	18.28	19.168	18.17	15.017
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+10x10 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	5.39	6.738	10.11	9.24	10.59	60.44
rowSums	10.01	11.550	20.50	23.10	26.56	39.65

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.0	1.000	1.0000
rowSums	1.857	1.714	2.027	2.5	2.509	0.6561
Figure: Benchmarking of colSums() and rowSums() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805459 43.1    1442291  77.1  1442291  77.1
Vcells 12280395 93.7   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805453 43.1    1442291  77.1  1442291  77.1
Vcells 12285438 93.8   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0131	0.0202	0.0255	0.0268	0.0296	0.0385
1	colSums	0.0166	0.0233	0.0361	0.0360	0.0466	0.0631
3	apply+sum	0.3907	0.4685	0.6268	0.6843	0.7264	0.8735

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.00
1	colSums	1.265	1.152	1.417	1.345	1.571	1.64
3	apply+sum	29.852	23.180	24.608	25.575	24.506	22.69
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0346	0.0358	0.0444	0.0387	0.0535	0.0932
1	rowSums	0.0377	0.0393	0.0508	0.0441	0.0583	0.0843
3	apply+sum	0.3865	0.4032	0.5759	0.4806	0.7395	1.6653

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.00	1.000
1	rowSums	1.089	1.097	1.144	1.139	1.09	0.905
3	apply+sum	11.155	11.263	12.972	12.423	13.82	17.876
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+100x100 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	16.55	23.29	36.10	35.99	46.58	63.13
rowSums	37.73	39.27	50.77	44.08	58.32	84.31

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.000
rowSums	2.279	1.686	1.407	1.225	1.252	1.335
Figure: Benchmarking of colSums() and rowSums() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805499 43.1    1442291  77.1  1442291  77.1
Vcells 12280634 93.7   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805493 43.1    1442291  77.1  1442291  77.1
Vcells 12285677 93.8   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0123	0.0142	0.0196	0.0154	0.0264	0.0520
1	colSums	0.0154	0.0169	0.0272	0.0208	0.0341	0.0893
3	apply+sum	0.2329	0.2398	0.3221	0.3087	0.3917	0.4639

	expr	min	lq	mean	median	uq	max
2	.colSums	1.00	1.000	1.000	1.00	1.000	1.000
1	colSums	1.25	1.189	1.389	1.35	1.292	1.718
3	apply+sum	18.90	16.837	16.470	20.05	14.853	8.926
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0462	0.0527	0.0548	0.0543	0.0560	0.0912
1	rowSums	0.0535	0.0602	0.0667	0.0643	0.0729	0.1109
3	apply+sum	0.3703	0.4092	0.4386	0.4296	0.4746	0.6433

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.000
1	rowSums	1.158	1.142	1.218	1.184	1.302	1.215
3	apply+sum	8.017	7.759	8.007	7.915	8.474	7.051
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+1000x10 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	15.40	16.94	27.16	20.79	34.07	89.31
rowSums	53.51	60.25	66.75	64.29	72.95	110.87

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.000
rowSums	3.475	3.557	2.458	3.093	2.141	1.241
Figure: Benchmarking of colSums() and rowSums() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805529 43.1    1442291  77.1  1442291  77.1
Vcells 12281209 93.7   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805523 43.1    1442291  77.1  1442291  77.1
Vcells 12286252 93.8   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0146	0.0281	0.0339	0.0308	0.0448	0.0508
1	colSums	0.0173	0.0360	0.0490	0.0474	0.0653	0.1293
3	apply+sum	1.9190	3.4055	3.3434	3.5893	3.7106	6.7436

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.000
1	colSums	1.184	1.281	1.446	1.538	1.455	2.545
3	apply+sum	131.169	121.179	98.749	116.544	82.737	132.709
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0366	0.0527	0.0589	0.0566	0.0685	0.0847
1	rowSums	0.0400	0.0612	0.0744	0.0704	0.0926	0.1132
3	apply+sum	1.9479	3.4530	3.4980	3.6186	3.7290	14.4627

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.000
1	rowSums	1.095	1.161	1.264	1.245	1.351	1.336
3	apply+sum	53.261	65.473	59.432	63.945	54.421	170.770
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+10x1000 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	17.32	35.99	48.97	47.35	65.25	129.3
rowSums	40.04	61.21	74.38	70.45	92.58	113.2

expr	min	lq	mean	median	uq	max
colSums	1.000	1.0	1.000	1.000	1.000	1.000
rowSums	2.311	1.7	1.519	1.488	1.419	0.875
Figure: Benchmarking of colSums() and rowSums() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805573 43.1    1442291  77.1  1442291  77.1
Vcells 12281605 93.8   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805567 43.1    1442291  77.1  1442291  77.1
Vcells 12331648 94.1   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.1136	0.1509	0.2103	0.2237	0.2543	0.2872
1	colSums	0.1197	0.1420	0.2144	0.2319	0.2699	0.3137
3	apply+sum	3.6255	4.5355	6.1318	6.3614	6.7223	22.7127

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.0000	1.00	1.000	1.000	1.000
1	colSums	1.054	0.9413	1.02	1.037	1.061	1.093
3	apply+sum	31.925	30.0561	29.16	28.442	26.438	79.089
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.3191	0.3274	0.4181	0.4440	0.4756	0.6598
1	rowSums	0.3268	0.3642	0.4337	0.4602	0.4943	0.5316
3	apply+sum	3.6113	4.6069	6.6670	7.0670	7.7509	20.4214

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.0000
1	rowSums	1.024	1.112	1.037	1.036	1.039	0.8057
3	apply+sum	11.316	14.071	15.947	15.915	16.297	30.9504
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+100x1000 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	119.7	142.0	214.4	231.9	269.9	313.7
rowSums	326.8	364.2	433.7	460.2	494.3	531.6

expr	min	lq	mean	median	uq	max
colSums	1.00	1.000	1.000	1.000	1.000	1.000
rowSums	2.73	2.564	2.023	1.984	1.832	1.694
Figure: Benchmarking of colSums() and rowSums() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805604 43.1    1442291  77.1  1442291  77.1
Vcells 12282062 93.8   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805598 43.1    1442291  77.1  1442291  77.1
Vcells 12332105 94.1   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.1051	0.1116	0.1815	0.1958	0.2373	0.2841
1	colSums	0.1109	0.1538	0.2058	0.2240	0.2635	0.3495
3	apply+sum	2.0056	2.6448	3.0788	3.0633	3.7006	4.4162

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.00	1.00
1	colSums	1.055	1.378	1.134	1.145	1.11	1.23
3	apply+sum	19.084	23.691	16.966	15.649	15.59	15.54
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.3241	0.4194	0.4401	0.4671	0.4829	0.5097
1	rowSums	0.3291	0.4700	0.4715	0.4954	0.5083	0.5801
3	apply+sum	2.0302	3.2563	3.6153	3.7498	3.9554	5.1026

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.000
1	rowSums	1.015	1.121	1.071	1.061	1.053	1.138
3	apply+sum	6.264	7.764	8.214	8.027	8.191	10.011
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on integer+1000x100 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	110.9	153.8	205.8	224.0	263.5	349.5
rowSums	329.1	470.0	471.5	495.4	508.3	580.1

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.00
rowSums	2.969	3.056	2.291	2.211	1.929	1.66
Figure: Benchmarking of colSums() and rowSums() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), naProb = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         X <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         X <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         X <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(X) <- mode
+     if (naProb > 0) 
+         X[sample(n, size = naProb * n)] <- NA
+     dim(X) <- c(nrow, ncol)
+     X
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805650 43.1    1442291  77.1  1442291  77.1
Vcells 12397671 94.6   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805635 43.1    1442291  77.1  1442291  77.1
Vcells 12397799 94.6   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0042	0.0054	0.0064	0.0062	0.0073	0.0169
1	colSums	0.0092	0.0112	0.0168	0.0146	0.0229	0.0416
3	apply+sum	0.0855	0.0905	0.1164	0.1161	0.1297	0.2983

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.000
1	colSums	2.181	2.071	2.626	2.375	3.131	2.454
3	apply+sum	20.175	16.784	18.194	18.840	17.735	17.613
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0046	0.0062	0.0072	0.0073	0.0081	0.0158
1	rowSums	0.0104	0.0127	0.0161	0.0141	0.0162	0.1736
3	apply+sum	0.0870	0.0985	0.1071	0.1011	0.1032	0.5301

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.00	1.000	1.000	1.000	1.00	1.00
1	rowSums	2.25	2.062	2.236	1.921	2.00	11.00
3	apply+sum	18.83	15.998	14.859	13.814	12.76	33.58
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+10x10 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	rowSums	10.39	12.70	16.12	14.05	16.17	173.62
1	colSums	9.24	11.16	16.80	14.63	22.91	41.58

	expr	min	lq	mean	median	uq	max
2	rowSums	1.000	1.0000	1.000	1.000	1.000	1.0000
1	colSums	0.889	0.8788	1.042	1.041	1.417	0.2395
Figure: Benchmarking of colSums() and rowSums() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805675 43.1    1442291  77.1  1442291  77.1
Vcells 12398411 94.6   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805669 43.1    1442291  77.1  1442291  77.1
Vcells 12408454 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0108	0.0129	0.0179	0.0160	0.0189	0.1251
1	colSums	0.0146	0.0206	0.0271	0.0246	0.0318	0.0597
3	apply+sum	0.3934	0.4706	0.6029	0.6608	0.7074	1.2523

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.0000
1	colSums	1.357	1.597	1.511	1.542	1.684	0.4769
3	apply+sum	36.499	36.488	33.598	41.359	37.499	10.0092
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0262	0.0273	0.0314	0.0293	0.0341	0.0758
1	rowSums	0.0293	0.0316	0.0398	0.0354	0.0448	0.0758
3	apply+sum	0.3896	0.4154	0.5039	0.4825	0.5243	0.7976

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.00	1.000	1.00
1	rowSums	1.118	1.155	1.265	1.21	1.316	1.00
3	apply+sum	14.882	15.197	16.030	16.49	15.390	10.52
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+100x100 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	14.63	20.60	27.11	24.64	31.76	59.67
rowSums	29.26	31.57	39.77	35.42	44.85	75.84

expr	min	lq	mean	median	uq	max
colSums	1	1.000	1.000	1.000	1.000	1.000
rowSums	2	1.533	1.467	1.438	1.412	1.271
Figure: Benchmarking of colSums() and rowSums() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805715 43.1    1442291  77.1  1442291  77.1
Vcells 12399215 94.6   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805709 43.1    1442291  77.1  1442291  77.1
Vcells 12409258 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0123	0.0169	0.0186	0.0181	0.0200	0.0262
1	colSums	0.0154	0.0239	0.0274	0.0250	0.0296	0.1032
3	apply+sum	0.2314	0.3178	0.3969	0.4188	0.4675	0.5832

	expr	min	lq	mean	median	uq	max
2	.colSums	1.00	1.000	1.00	1.000	1.000	1.000
1	colSums	1.25	1.409	1.47	1.383	1.481	3.941
3	apply+sum	18.78	18.760	21.28	23.148	23.354	22.279
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0373	0.0393	0.0451	0.0412	0.0427	0.2776
1	rowSums	0.0443	0.0483	0.0571	0.0572	0.0616	0.0947
3	apply+sum	0.3838	0.4054	0.4440	0.4329	0.4602	1.2080

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.00	1.000	1.000	1.000	1.0000
1	rowSums	1.186	1.23	1.265	1.388	1.441	0.3412
3	apply+sum	10.278	10.32	9.840	10.509	10.770	4.3523
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+1000x10 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	15.40	23.87	27.41	25.02	29.64	103.2
rowSums	44.27	48.31	57.09	57.17	61.59	94.7

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.0000
rowSums	2.875	2.024	2.083	2.285	2.078	0.9179
Figure: Benchmarking of colSums() and rowSums() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805745 43.1    1442291  77.1  1442291  77.1
Vcells 12399233 94.6   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805739 43.1    1442291  77.1  1442291  77.1
Vcells 12409276 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0119	0.0139	0.0262	0.0210	0.037	0.0493
1	colSums	0.0158	0.0185	0.0380	0.0366	0.052	0.0951
3	apply+sum	1.9937	2.1394	3.0565	3.4386	3.681	7.2175

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.00
1	colSums	1.323	1.333	1.453	1.743	1.406	1.93
3	apply+sum	167.059	154.368	116.861	163.891	99.595	146.47
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.0277	0.0352	0.0428	0.0406	0.0500	0.0627
1	rowSums	0.0320	0.0481	0.0601	0.0595	0.0699	0.0959
3	apply+sum	2.0114	2.1149	3.0317	3.3112	3.7622	7.5085

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.000
1	rowSums	1.153	1.366	1.403	1.464	1.396	1.528
3	apply+sum	72.566	60.042	70.808	81.529	75.175	119.660
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+10x1000 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	15.78	18.48	38.00	36.57	51.97	95.08
rowSums	31.95	48.12	60.07	59.48	69.87	95.85

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.000
rowSums	2.024	2.604	1.581	1.626	1.344	1.008
Figure: Benchmarking of colSums() and rowSums() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805789 43.1    1442291  77.1  1442291  77.1
Vcells 12400190 94.7   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805783 43.1    1442291  77.1  1442291  77.1
Vcells 12500233 95.4   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.0893	0.1178	0.1510	0.1272	0.1996	0.2521
1	colSums	0.0962	0.1263	0.1671	0.1453	0.2239	0.3626
3	apply+sum	3.6232	6.0942	6.2943	6.3165	6.7892	25.6360

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.000
1	colSums	1.078	1.072	1.107	1.142	1.121	1.438
3	apply+sum	40.569	51.735	41.697	49.647	34.014	101.671
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.2387	0.2812	0.3175	0.3282	0.3515	0.5663
1	rowSums	0.2460	0.3118	0.3485	0.3574	0.3848	0.5001
3	apply+sum	3.7791	6.3063	6.9822	6.9763	7.4048	28.5705

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.0000
1	rowSums	1.031	1.109	1.098	1.089	1.095	0.8831
3	apply+sum	15.834	22.426	21.994	21.258	21.068	50.4540
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+100x1000 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	96.24	126.3	167.1	145.3	223.9	362.6
rowSums	245.99	311.8	348.5	357.4	384.8	500.1

expr	min	lq	mean	median	uq	max
colSums	1.000	1.00	1.000	1.00	1.000	1.000
rowSums	2.556	2.47	2.086	2.46	1.719	1.379
Figure: Benchmarking of colSums() and rowSums() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805820 43.1    1442291  77.1  1442291  77.1
Vcells 12400209 94.7   35610798 271.7 68120027 519.8
> colStats <- microbenchmark(colSums = colSums(X, na.rm = FALSE), .colSums = .colSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 2L, FUN = sum, na.rm = FALSE), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   805814 43.1    1442291  77.1  1442291  77.1
Vcells 12500252 95.4   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowSums = rowSums(X, na.rm = FALSE), .rowSums = .rowSums(X, m = nrow(X), 
+     n = ncol(X), na.rm = FALSE), `apply+sum` = apply(X, MARGIN = 1L, FUN = sum, na.rm = FALSE), unit = "ms")

Table: Benchmarking of colSums(), .colSums() and apply+sum() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.colSums	0.1012	0.1080	0.1463	0.1193	0.1944	0.3199
1	colSums	0.1078	0.1272	0.1731	0.1517	0.2331	0.3218
3	apply+sum	2.0183	2.8562	3.4913	3.4213	3.7648	13.8918

	expr	min	lq	mean	median	uq	max
2	.colSums	1.000	1.000	1.000	1.000	1.000	1.000
1	colSums	1.065	1.178	1.183	1.271	1.199	1.006
3	apply+sum	19.935	26.451	23.861	28.669	19.366	43.426
Table: Benchmarking of rowSums(), .rowSums() and apply+sum() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

	expr	min	lq	mean	median	uq	max
2	.rowSums	0.2437	0.2510	0.3148	0.3199	0.3549	0.4030
1	rowSums	0.2498	0.3049	0.3506	0.3493	0.3923	0.8454
3	apply+sum	2.1534	3.4057	4.2447	4.0776	4.9155	14.3457

	expr	min	lq	mean	median	uq	max
2	.rowSums	1.000	1.000	1.000	1.000	1.000	1.000
1	rowSums	1.025	1.215	1.114	1.092	1.105	2.097
3	apply+sum	8.837	13.569	13.484	12.747	13.849	35.593
Figure: Benchmarking of colSums(), .colSums() and apply+sum() on double+1000x100 data as well as rowSums(), .rowSums() and apply+sum() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colSums() and rowSums() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr	min	lq	mean	median	uq	max
colSums	107.8	127.2	173.1	151.7	233.1	321.8
rowSums	249.8	304.9	350.6	349.3	392.3	845.4

expr	min	lq	mean	median	uq	max
colSums	1.000	1.000	1.000	1.000	1.000	1.000
rowSums	2.318	2.396	2.025	2.303	1.683	2.627
Figure: Benchmarking of colSums() and rowSums() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R Under development (unstable) (2015-02-27 r67909)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] markdown_0.7.7          microbenchmark_1.4-2    matrixStats_0.14.0-9000
[4] ggplot2_1.0.0           knitr_1.9.3             R.devices_2.13.0       
[7] R.utils_2.0.0           R.oo_1.19.0             R.methodsS3_1.7.0      

loaded via a namespace (and not attached):
 [1] Rcpp_0.11.4         BiocGenerics_0.13.6 splines_3.2.0      
 [4] MASS_7.3-39         munsell_0.4.2       lattice_0.20-30    
 [7] colorspace_1.2-4    R.cache_0.11.1-9000 multcomp_1.3-9     
[10] stringr_0.6.2       plyr_1.8.1          tools_3.2.0        
[13] parallel_3.2.0      grid_3.2.0          Biobase_2.27.2     
[16] gtable_0.1.2        TH.data_1.0-6       survival_2.38-1    
[19] digest_0.6.8        R.rsp_0.20.0        reshape2_1.4.1     
[22] formatR_1.0.3       base64enc_0.1-3     mime_0.2.1         
[25] evaluate_0.5.7      labeling_0.3        sandwich_2.3-2     
[28] scales_0.2.4        mvtnorm_1.0-2       zoo_1.7-12         
[31] Cairo_1.5-6         proto_0.3-10       

Total processing time was 40.78 secs.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colSums')

---

Copyright Henrik Bengtsson. Last updated on 2015-03-02 17:22:15 (-0800 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAADFBMVEX9/v0AAP/9/v3//wBEQjoBAAAABHRSTlP//wD//gy7CwAAAGJJREFUOI3N0rESwCAIA9Ag///PXdoiBk0HhmbNO49DMETQCexNCSyFgdlGoO5DYOr9ThLgPosA7osIQP0sHuDOog8UI/ALa988wzdwXJRctf4s+d36YPTJ6aMd8ux3+QO4ABTtB85yDAh9AAAAAElFTkSuQmCC" document.getElementsByTagName('head')[0].appendChild(link); </script>