Skip to content

colRowOrderStats

Jump to bottom
hb edited this page Mar 3, 2015 · 2 revisions

matrixStats: Benchmark report

colOrderStats() and rowOrderStats() benchmarks

This report benchmark the performance of colOrderStats() and rowOrderStats() against alternative methods.

Alternative methods

  • apply() + quantile(..., type=3L)
  • Biobase::rowQ()

Data type "integer"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), naProb = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         X <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         X <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         X <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(X) <- mode
+     if (naProb > 0) 
+         X[sample(n, size = naProb * n)] <- NA
+     dim(X) <- c(nrow, ncol)
+     X
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 integer matrix

> X <- data[["10x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   802765 42.9    1442291  77.1  1168576  62.5
Vcells 12278847 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801324 42.8    1442291  77.1  1442291  77.1
Vcells 12274701 93.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.0035 0.0058 0.0146 0.0125 0.0214 0.1228
3 rowQ(t(X)) 0.0524 0.0689 0.1069 0.1143 0.1344 0.2429
2 apply+quantile 1.7716 1.8141 2.2346 1.9450 2.5963 5.3366
expr min lq mean median uq max
1 colOrderStats 1.00 1.00 1.00 1.000 1.000 1.000
3 rowQ(t(X)) 15.11 11.93 7.32 9.138 6.288 1.978
2 apply+quantile 511.27 314.10 153.07 155.449 121.519 43.457
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.0035 0.0073 0.0164 0.0150 0.0221 0.1290
3 rowQ 0.0362 0.0581 0.0881 0.0708 0.1184 0.2156
2 apply+quantile 1.7739 2.1457 2.4533 2.5434 2.6096 5.8921
expr min lq mean median uq max
1 rowOrderStats 1.00 1.000 1.00 1.000 1.000 1.000
3 rowQ 10.44 7.947 5.38 4.718 5.348 1.672
2 apply+quantile 511.94 293.333 149.80 169.396 117.889 45.689
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+10x10 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 3.465 5.776 14.60 12.51 21.37 122.8
rowOrderStats 3.465 7.315 16.38 15.01 22.14 129.0
expr min lq mean median uq max
colOrderStats 1 1.000 1.000 1.0 1.000 1.00
rowOrderStats 1 1.267 1.122 1.2 1.036 1.05
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 integer matrix

> X <- data[["100x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801511 42.9    1442291  77.1  1442291  77.1
Vcells 12275910 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801504 42.9    1442291  77.1  1442291  77.1
Vcells 12281016 93.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.1386 0.1621 0.1788 0.1788 0.1956 0.3326
3 rowQ(t(X)) 0.5051 0.5913 0.7490 0.7393 0.8071 4.3123
2 apply+quantile 18.0370 21.5343 25.0443 25.6753 26.7304 42.4920
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.00
3 rowQ(t(X)) 3.644 3.648 4.188 4.135 4.127 12.97
2 apply+quantile 130.152 132.873 140.047 143.588 136.688 127.76
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.1428 0.1555 0.1751 0.1728 0.1867 0.2933
3 rowQ 0.4735 0.4904 0.5913 0.5447 0.6725 1.0887
2 apply+quantile 18.2611 18.8400 23.2920 20.9669 23.9143 149.7863
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 3.315 3.154 3.378 3.151 3.602 3.711
2 apply+quantile 127.862 121.140 133.053 121.304 128.087 510.632
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+100x100 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
2 rowOrderStats 142.8 155.5 175.1 172.8 186.7 293.3
1 colOrderStats 138.6 162.1 178.8 178.8 195.6 332.6
expr min lq mean median uq max
2 rowOrderStats 1.0000 1.000 1.000 1.000 1.000 1.000
1 colOrderStats 0.9703 1.042 1.022 1.034 1.047 1.134
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 integer matrix

> X <- data[["1000x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801551 42.9    1442291  77.1  1442291  77.1
Vcells 12276151 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801544 42.9    1442291  77.1  1442291  77.1
Vcells 12281257 93.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.1220 0.1365 0.1519 0.1507 0.1648 0.1983
3 rowQ(t(X)) 0.4858 0.6575 0.7136 0.7418 0.7645 1.4120
2 apply+quantile 2.1915 2.7234 3.1879 3.3141 3.4688 9.0179
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.00 1.000
3 rowQ(t(X)) 3.981 4.818 4.697 4.922 4.64 7.122
2 apply+quantile 17.959 19.956 20.983 21.990 21.05 45.487
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.1317 0.1559 0.1793 0.1723 0.1934 0.9682
3 rowQ 0.4473 0.5846 0.6536 0.6513 0.7100 2.4529
2 apply+quantile 2.2986 3.2236 3.4544 3.4057 3.6111 9.4714
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 3.398 3.749 3.646 3.781 3.671 2.534
2 apply+quantile 17.459 20.676 19.269 19.770 18.668 9.783
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+1000x10 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 122.0 136.5 151.9 150.7 164.8 198.3
rowOrderStats 131.7 155.9 179.3 172.3 193.4 968.2
expr min lq mean median uq max
colOrderStats 1.000 1.000 1.00 1.000 1.000 1.000
rowOrderStats 1.079 1.143 1.18 1.143 1.174 4.883
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 integer matrix

> X <- data[["10x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801581 42.9    1442291  77.1  1442291  77.1
Vcells 12276875 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801574 42.9    1442291  77.1  1442291  77.1
Vcells 12281981 93.8   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.1436 0.1632 0.1843 0.1715 0.1882 0.9616
3 rowQ(t(X)) 0.4908 0.6036 0.6808 0.6331 0.7591 1.4701
2 apply+quantile 198.0534 220.5612 238.4357 237.3883 253.2588 379.7250
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 3.418 3.698 3.694 3.691 4.033 1.529
2 apply+quantile 1379.307 1351.304 1293.645 1384.201 1345.379 394.883
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.1440 0.1584 0.1774 0.1740 0.1854 0.2421
3 rowQ 0.4558 0.4868 0.5726 0.5565 0.6309 0.9197
2 apply+quantile 196.4350 212.4889 228.4554 224.2445 240.5667 357.8924
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 3.166 3.073 3.227 3.198 3.404 3.798
2 apply+quantile 1364.378 1341.390 1287.441 1288.761 1297.860 1478.057
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+10x1000 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 143.6 163.2 184.3 171.5 188.2 961.6
rowOrderStats 144.0 158.4 177.4 174.0 185.4 242.1
expr min lq mean median uq max
colOrderStats 1.000 1.0000 1.0000 1.000 1.0000 1.0000
rowOrderStats 1.003 0.9705 0.9628 1.015 0.9847 0.2518
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 integer matrix

> X <- data[["100x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801625 42.9    1442291  77.1  1442291  77.1
Vcells 12277302 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801618 42.9    1442291  77.1  1442291  77.1
Vcells 12327408 94.1   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 1.365 1.396 1.562 1.468 1.718 2.163
3 rowQ(t(X)) 5.093 5.207 6.128 5.491 6.977 19.327
2 apply+quantile 200.416 226.750 243.862 241.777 260.357 349.187
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 3.731 3.729 3.922 3.739 4.061 8.935
2 apply+quantile 146.820 162.380 156.070 164.653 151.542 161.432
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 1.466 1.523 1.633 1.541 1.756 2.365
3 rowQ 4.829 4.946 5.753 5.382 6.456 9.347
2 apply+quantile 204.134 233.113 248.974 245.885 262.553 359.096
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 3.293 3.248 3.524 3.493 3.676 3.952
2 apply+quantile 139.218 153.093 152.514 159.605 149.487 151.828
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+100x1000 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 1.365 1.396 1.562 1.468 1.718 2.163
rowOrderStats 1.466 1.523 1.633 1.541 1.756 2.365
expr min lq mean median uq max
colOrderStats 1.000 1.00 1.000 1.000 1.000 1.000
rowOrderStats 1.074 1.09 1.045 1.049 1.022 1.093
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 integer matrix

> X <- data[["1000x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801647 42.9    1442291  77.1  1442291  77.1
Vcells 12277838 93.7   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801649 42.9    1442291  77.1  1442291  77.1
Vcells 12327959 94.1   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 1.263 1.323 1.466 1.396 1.538 2.567
3 rowQ(t(X)) 5.149 5.342 6.802 7.205 7.732 10.335
2 apply+quantile 21.900 27.126 31.508 30.887 34.894 62.135
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 4.078 4.037 4.641 5.161 5.027 4.026
2 apply+quantile 17.345 20.496 21.498 22.125 22.687 24.206
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on integer+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 1.412 1.508 1.657 1.580 1.790 2.322
3 rowQ 4.913 5.002 5.773 5.132 6.682 9.031
2 apply+quantile 22.642 24.546 28.343 27.020 32.053 45.068
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 3.478 3.318 3.483 3.249 3.734 3.889
2 apply+quantile 16.031 16.283 17.100 17.105 17.912 19.405
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on integer+1000x100 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on integer+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 1.263 1.323 1.466 1.396 1.538 2.567
rowOrderStats 1.412 1.508 1.657 1.580 1.790 2.322
expr min lq mean median uq max
colOrderStats 1.000 1.000 1.000 1.000 1.000 1.0000
rowOrderStats 1.119 1.139 1.131 1.131 1.163 0.9048
Figure: Benchmarking of colOrderStats() and rowOrderStats() on integer+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Data type "double"

Data

> rmatrix <- function(nrow, ncol, mode = c("logical", "double", "integer", "index"), range = c(-100, 
+     +100), naProb = 0) {
+     mode <- match.arg(mode)
+     n <- nrow * ncol
+     if (mode == "logical") {
+         X <- sample(c(FALSE, TRUE), size = n, replace = TRUE)
+     }     else if (mode == "index") {
+         X <- seq_len(n)
+         mode <- "integer"
+     }     else {
+         X <- runif(n, min = range[1], max = range[2])
+     }
+     storage.mode(X) <- mode
+     if (naProb > 0) 
+         X[sample(n, size = naProb * n)] <- NA
+     dim(X) <- c(nrow, ncol)
+     X
+ }
> rmatrices <- function(scale = 10, seed = 1, ...) {
+     set.seed(seed)
+     data <- list()
+     data[[1]] <- rmatrix(nrow = scale * 1, ncol = scale * 1, ...)
+     data[[2]] <- rmatrix(nrow = scale * 10, ncol = scale * 10, ...)
+     data[[3]] <- rmatrix(nrow = scale * 100, ncol = scale * 1, ...)
+     data[[4]] <- t(data[[3]])
+     data[[5]] <- rmatrix(nrow = scale * 10, ncol = scale * 100, ...)
+     data[[6]] <- t(data[[5]])
+     names(data) <- sapply(data, FUN = function(x) paste(dim(x), collapse = "x"))
+     data
+ }
> data <- rmatrices(mode = mode)

Results

10x10 double matrix

> X <- data[["10x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801702 42.9    1442291  77.1  1442291  77.1
Vcells 12393570 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801686 42.9    1442291  77.1  1442291  77.1
Vcells 12393761 94.6   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+10x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.005 0.0083 0.0176 0.0129 0.0296 0.0366
3 rowQ(t(X)) 0.025 0.0418 0.0792 0.0697 0.1201 0.1959
2 apply+quantile 1.769 2.1020 2.6685 2.7164 2.9549 7.0812
expr min lq mean median uq max
1 colOrderStats 1.0 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 5.0 5.046 4.497 5.403 4.052 5.358
2 apply+quantile 353.5 253.946 151.611 210.633 99.686 193.624
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+10x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.0058 0.0077 0.0169 0.0158 0.0246 0.0343
3 rowQ 0.0146 0.0227 0.0474 0.0331 0.0743 0.1309
2 apply+quantile 1.7885 1.8997 2.5092 2.5726 2.7692 5.7085
expr min lq mean median uq max
1 rowOrderStats 1.000 1.00 1.000 1.000 1.000 1.00
3 rowQ 2.533 2.95 2.808 2.098 3.015 3.82
2 apply+quantile 309.697 246.72 148.774 162.991 112.394 166.61
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+10x10 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+10x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 5.005 8.277 17.60 12.90 29.64 36.57
rowOrderStats 5.775 7.700 16.87 15.78 24.64 34.26
expr min lq mean median uq max
colOrderStats 1.000 1.0000 1.0000 1.000 1.0000 1.0000
rowOrderStats 1.154 0.9302 0.9582 1.224 0.8312 0.9368
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+10x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x100 double matrix

> X <- data[["100x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801727 42.9    1442291  77.1  1442291  77.1
Vcells 12394371 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801720 42.9    1442291  77.1  1442291  77.1
Vcells 12404477 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+100x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.3345 0.3467 0.3990 0.3640 0.4475 0.5355
3 rowQ(t(X)) 0.4600 0.4756 0.5617 0.5463 0.6254 0.7822
2 apply+quantile 18.9378 19.7749 23.8666 23.3303 26.6034 35.0639
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 1.375 1.372 1.408 1.501 1.397 1.461
2 apply+quantile 56.611 57.045 59.811 64.099 59.448 65.482
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+100x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.3361 0.3451 0.4156 0.3782 0.4791 0.6182
3 rowQ 0.4150 0.4417 0.5336 0.5495 0.5961 1.2734
2 apply+quantile 18.8989 20.3279 24.7056 24.8366 27.4876 37.1569
expr min lq mean median uq max
1 rowOrderStats 1.000 1.00 1.000 1.000 1.000 1.00
3 rowQ 1.235 1.28 1.284 1.453 1.244 2.06
2 apply+quantile 56.236 58.90 59.452 65.668 57.376 60.10
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+100x100 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+100x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 334.5 346.7 399.0 364.0 447.5 535.5
rowOrderStats 336.1 345.1 415.6 378.2 479.1 618.2
expr min lq mean median uq max
colOrderStats 1.000 1.0000 1.000 1.000 1.000 1.000
rowOrderStats 1.005 0.9956 1.041 1.039 1.071 1.155
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+100x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x10 double matrix

> X <- data[["1000x10"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801767 42.9    1442291  77.1  1442291  77.1
Vcells 12395258 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801760 42.9    1442291  77.1  1442291  77.1
Vcells 12405364 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+1000x10 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.2914 0.3047 0.3785 0.3819 0.4496 0.5135
3 rowQ(t(X)) 0.4154 0.4875 0.5790 0.5926 0.6675 0.7680
2 apply+quantile 2.5873 2.8306 3.6961 3.8580 4.1379 9.4040
expr min lq mean median uq max
1 colOrderStats 1.000 1.00 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 1.425 1.60 1.530 1.552 1.485 1.496
2 apply+quantile 8.878 9.29 9.766 10.103 9.203 18.313
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+1000x10 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.3049 0.3361 0.3957 0.4075 0.4267 0.5632
3 rowQ 0.3746 0.4654 0.5305 0.5359 0.5678 1.1618
2 apply+quantile 2.5480 3.3803 3.7994 3.9144 4.1454 9.4614
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 1.228 1.385 1.341 1.315 1.331 2.063
2 apply+quantile 8.357 10.058 9.601 9.607 9.714 16.800
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+1000x10 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+1000x10 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 291.4 304.7 378.5 381.9 449.6 513.5
rowOrderStats 304.9 336.1 395.7 407.5 426.7 563.2
expr min lq mean median uq max
colOrderStats 1.000 1.000 1.000 1.000 1.0000 1.000
rowOrderStats 1.046 1.103 1.046 1.067 0.9491 1.097
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+1000x10 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

10x1000 double matrix

> X <- data[["10x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801797 42.9    1442291  77.1  1442291  77.1
Vcells 12395276 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801790 42.9    1442291  77.1  1442291  77.1
Vcells 12405382 94.7   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+10x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 0.3068 0.3197 0.3633 0.3390 0.3957 0.5436
3 rowQ(t(X)) 0.4338 0.4746 0.5524 0.5411 0.6007 0.7430
2 apply+quantile 194.3855 217.2599 235.5137 229.5844 246.9935 366.9353
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ(t(X)) 1.414 1.485 1.521 1.596 1.518 1.367
2 apply+quantile 633.572 679.564 648.267 677.335 624.141 675.064
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+10x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 0.3072 0.3245 0.3855 0.3952 0.4358 0.5932
3 rowQ 0.3911 0.4365 0.4940 0.4716 0.5707 0.6575
2 apply+quantile 199.4311 221.7211 240.0376 235.4357 249.3340 404.2755
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.00 1.000
3 rowQ 1.273 1.345 1.281 1.193 1.31 1.108
2 apply+quantile 649.202 683.234 622.682 595.804 572.17 681.499
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+10x1000 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+10x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 306.8 319.7 363.3 339.0 395.7 543.6
rowOrderStats 307.2 324.5 385.5 395.2 435.8 593.2
expr min lq mean median uq max
colOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
rowOrderStats 1.001 1.015 1.061 1.166 1.101 1.091
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+10x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

100x1000 double matrix

> X <- data[["100x1000"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801841 42.9    1442291  77.1  1442291  77.1
Vcells 12396346 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801834 42.9    1442291  77.1  1442291  77.1
Vcells 12496452 95.4   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+100x1000 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 3.320 3.340 3.855 3.394 4.530 5.630
3 rowQ(t(X)) 4.712 4.826 5.762 5.784 6.477 8.141
2 apply+quantile 216.073 237.575 254.872 249.764 264.340 394.454
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.00 1.000
3 rowQ(t(X)) 1.419 1.445 1.495 1.704 1.43 1.446
2 apply+quantile 65.078 71.129 66.109 73.579 58.36 70.059
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+100x1000 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 3.485 3.513 3.910 3.607 4.394 5.414
3 rowQ 4.279 4.315 5.046 4.895 5.711 6.686
2 apply+quantile 208.476 238.384 251.722 251.117 262.862 379.678
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.00 1.000
3 rowQ 1.228 1.228 1.291 1.357 1.30 1.235
2 apply+quantile 59.828 67.863 64.380 69.612 59.82 70.129
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+100x1000 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+100x1000 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 3.320 3.340 3.855 3.394 4.530 5.630
rowOrderStats 3.485 3.513 3.910 3.607 4.394 5.414
expr min lq mean median uq max
colOrderStats 1.00 1.000 1.000 1.000 1.0000 1.0000
rowOrderStats 1.05 1.052 1.014 1.063 0.9701 0.9616
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+100x1000 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

1000x100 double matrix

> X <- data[["1000x100"]]
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801872 42.9    1442291  77.1  1442291  77.1
Vcells 12396365 94.6   35610798 271.7 68120027 519.8
> probs <- 0.3
> which <- round(probs * nrow(X))
> colStats <- microbenchmark(colOrderStats = colOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 2L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), `rowQ(t(X))` = rowQ(t(X), 
+     which = which), unit = "ms")
> X <- t(X)
> gc()
           used (Mb) gc trigger  (Mb) max used  (Mb)
Ncells   801865 42.9    1442291  77.1  1442291  77.1
Vcells 12496471 95.4   35610798 271.7 68120027 519.8
> rowStats <- microbenchmark(rowOrderStats = rowOrderStats(X, which = which, na.rm = FALSE), `apply+quantile` = apply(X, 
+     MARGIN = 1L, FUN = quantile, probs = probs, na.rm = FALSE, type = 3L), rowQ = rowQ(X, which = which), 
+     unit = "ms")

Table: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+1000x100 data. The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
1 colOrderStats 3.413 3.497 4.507 4.393 4.605 29.25
3 rowQ(t(X)) 4.753 5.410 6.513 6.693 7.153 12.54
2 apply+quantile 26.287 30.320 34.855 35.672 38.476 68.14
expr min lq mean median uq max
1 colOrderStats 1.000 1.000 1.000 1.000 1.000 1.0000
3 rowQ(t(X)) 1.392 1.547 1.445 1.524 1.553 0.4286
2 apply+quantile 7.701 8.670 7.733 8.120 8.356 2.3291
Table: Benchmarking of rowOrderStats(), apply+quantile() and rowQ() on double+1000x100 data (transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.
expr min lq mean median uq max
1 rowOrderStats 3.637 3.679 4.367 4.471 4.883 6.375
3 rowQ 4.408 4.481 5.327 5.052 6.099 7.183
2 apply+quantile 26.883 31.897 34.560 34.341 37.180 45.913
expr min lq mean median uq max
1 rowOrderStats 1.000 1.000 1.000 1.000 1.000 1.000
3 rowQ 1.212 1.218 1.220 1.130 1.249 1.127
2 apply+quantile 7.391 8.671 7.913 7.681 7.614 7.202
Figure: Benchmarking of colOrderStats(), apply+quantile() and rowQ(t(X))() on double+1000x100 data as well as rowOrderStats(), apply+quantile() and rowQ() on the same data transposed. Outliers are displayed as crosses. Times are in milliseconds.

Table: Benchmarking of colOrderStats() and rowOrderStats() on double+1000x100 data (original and transposed). The top panel shows times in milliseconds and the bottom panel shows relative times.

expr min lq mean median uq max
colOrderStats 3.413 3.497 4.507 4.393 4.605 29.254
rowOrderStats 3.637 3.679 4.367 4.471 4.883 6.375
expr min lq mean median uq max
colOrderStats 1.000 1.000 1.0000 1.000 1.00 1.0000
rowOrderStats 1.065 1.052 0.9689 1.018 1.06 0.2179
Figure: Benchmarking of colOrderStats() and rowOrderStats() on double+1000x100 data (original and transposed). Outliers are displayed as crosses. Times are in milliseconds.

Appendix

Session information

R Under development (unstable) (2015-02-27 r67909)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] markdown_0.7.7          microbenchmark_1.4-2    matrixStats_0.14.0-9000
[4] ggplot2_1.0.0           knitr_1.9.3             R.devices_2.13.0       
[7] R.utils_2.0.0           R.oo_1.19.0             R.methodsS3_1.7.0      

loaded via a namespace (and not attached):
 [1] Rcpp_0.11.4         BiocGenerics_0.13.6 splines_3.2.0      
 [4] MASS_7.3-39         munsell_0.4.2       lattice_0.20-30    
 [7] colorspace_1.2-4    R.cache_0.11.1-9000 multcomp_1.3-9     
[10] stringr_0.6.2       plyr_1.8.1          tools_3.2.0        
[13] parallel_3.2.0      grid_3.2.0          Biobase_2.27.2     
[16] gtable_0.1.2        TH.data_1.0-6       survival_2.38-1    
[19] digest_0.6.8        R.rsp_0.20.0        reshape2_1.4.1     
[22] formatR_1.0.3       base64enc_0.1-3     mime_0.2.1         
[25] evaluate_0.5.7      labeling_0.3        sandwich_2.3-2     
[28] scales_0.2.4        mvtnorm_1.0-2       zoo_1.7-12         
[31] Cairo_1.5-6         proto_0.3-10

Total processing time was 4.38 mins.

Reproducibility

To reproduce this report, do:

html <- matrixStats:::benchmark('colOrderStats')

Copyright Henrik Bengtsson. Last updated on 2015-03-02 17:14:58 (-0800 UTC). Powered by RSP.

<script> var link = document.createElement('link'); link.rel = 'icon'; link.href = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAADFBMVEX9/v0AAP/9/v3//wBEQjoBAAAABHRSTlP//wD//gy7CwAAAGJJREFUOI3N0rESwCAIA9Ag///PXdoiBk0HhmbNO49DMETQCexNCSyFgdlGoO5DYOr9ThLgPosA7osIQP0sHuDOog8UI/ALa988wzdwXJRctf4s+d36YPTJ6aMd8ux3+QO4ABTtB85yDAh9AAAAAElFTkSuQmCC" document.getElementsByTagName('head')[0].appendChild(link); </script>

[Benchmark reports](Benchmark reports)

Clone this wiki locally