Optimize permutations by implementing it via multiset_permutations

[FedericoStra]

Code changes

As noted in #151, multiset_permutations is much faster than permutations, so we can exploit it to optimize the latter (this was suggested in a comment here).

The code does the equivalent of

permutations(a, t::Integer=length(a)) =
    Iterators.map(
        indices -> [a[i] for i in indices],
        multiset_permutations(eachindex(a), t))

but we construct the iterator manually so that we can define eltype or it (otherwise, with Iterators.map, type inference would deduce Any).

This closes #151; it possibly closes #185 too.

Simple benchmark

Benchmark code

using BenchmarkTools
using Combinatorics

count_permutations(a) = count(Returns(true), permutations(a))
count_permutations(a, t) = count(Returns(true), permutations(a, t))

# compile
count_permutations(1:3)
for t in 0:3
    count_permutations(1:3, t)
end

for n in [3, 5, 7, 8, 9]
    println("\nn = $(n)\n")
    a = collect(1:n)
    display(@benchmark count_permutations($a))
end

println("\nn = 10, t = 6\n")
display(@benchmark count_permutations(1:10, 6))

Before

n = 3

BenchmarkTools.Trial: 10000 samples with 20 evaluations per sample.
 Range (min … max):  977.800 ns … 108.978 μs  ┊ GC (min … max): 0.00% … 97.93%
 Time  (median):       1.003 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):     1.160 μs ±   1.842 μs  ┊ GC (mean ± σ):  2.62% ±  1.68%

  █▇▃▅▃                                   ▃▁                    ▁
  █████▆▃▆▆▆▆▅▇▆▆▆▅▃▅▅▃▃▅▄▅▄▄▃▃▅▅▆▅▆▆▇▆▇▆▆██▇▇▇▇▇▆▇▇▆▆▇▄▅▆▅▅▄▅▆ █
  978 ns        Histogram: log(frequency) by time       2.47 μs <

 Memory estimate: 624 bytes, allocs estimate: 16.

n = 5

BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  109.018 μs …  2.817 ms  ┊ GC (min … max): 0.00% … 94.72%
 Time  (median):     118.300 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   121.273 μs ± 36.502 μs  ┊ GC (mean ± σ):  0.40% ±  1.33%

       █  ▂
  ▁▁▇▅▂█▅▃█▅▃▇▆▃▃▄▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
  109 μs          Histogram: frequency by time          173 μs <

 Memory estimate: 11.41 KiB, allocs estimate: 244.

n = 7

BenchmarkTools.Trial: 155 samples with 1 evaluation per sample.
 Range (min … max):  27.639 ms … 41.011 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     31.259 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   32.352 ms ±  3.509 ms  ┊ GC (mean ± σ):  0.06% ± 0.47%

     ▃▆▆▁ ██▆▆▁▃ ▁ ▁ ▁    ▃ ▁  ▁            ▁       ▁
  ▄▄▇████▇██████▇█▇█▇█▆▄▁▇█▄█▆▆█▇▆▁▆▁▁▇▇▄▄▄▁█▄▄▄▄▆▁▁█▆▄▁▁▄▁▇▄ ▄
  27.6 ms         Histogram: frequency by time        40.7 ms <

 Memory estimate: 551.42 KiB, allocs estimate: 10084.

n = 8

BenchmarkTools.Trial: 9 samples with 1 evaluation per sample.
 Range (min … max):  591.173 ms … 673.411 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     604.497 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   609.657 ms ±  24.487 ms  ┊ GC (mean ± σ):  0.03% ± 0.08%

  █   ██ █ ████                                               █
  █▁▁▁██▁█▁████▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  591 ms           Histogram: frequency by time          673 ms <

 Memory estimate: 4.92 MiB, allocs estimate: 80644.

n = 9

BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
 Single result which took 14.473 s (0.01% GC) to evaluate,
 with a memory estimate of 44.30 MiB, over 725764 allocations.

n = 10, t = 6

BenchmarkTools.Trial: 121 samples with 1 evaluation per sample.
 Range (min … max):  39.323 ms … 45.142 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     41.299 ms              ┊ GC (median):    3.36%
 Time  (mean ± σ):   41.409 ms ±  1.035 ms  ┊ GC (mean ± σ):  2.86% ± 1.18%

                    █       ▄
  ▄▃▃▁▁▁▁▆▆▆▆▅▄▇▄▇▄▆█▅▆▇▆▄▄▆█▄▆▅▄▄▇▃▃▇▁▃▁▃▃▁▁▁▁▁▁▃▃▁▁▁▁▁▁▁▁▁▃ ▃
  39.3 ms         Histogram: frequency by time          45 ms <

 Memory estimate: 16.15 MiB, allocs estimate: 302403.

After

n = 3

BenchmarkTools.Trial: 10000 samples with 10 evaluations per sample.
 Range (min … max):  1.071 μs … 217.489 μs  ┊ GC (min … max): 0.00% … 97.94%
 Time  (median):     1.126 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.271 μs ±   4.475 μs  ┊ GC (mean ± σ):  9.03% ±  2.59%

    █▆
  ▂▇███▄▃▂▂▂▂▂▂▂▂▂▁▁▁▂▂▂▂▂▂▂▂▂▂▂▁▂▁▂▂▂▂▂▁▂▂▂▂▂▂▂▁▁▁▂▂▂▂▂▂▂▁▂▂ ▂
  1.07 μs         Histogram: frequency by time        2.13 μs <

 Memory estimate: 2.56 KiB, allocs estimate: 63.

n = 5

BenchmarkTools.Trial: 10000 samples with 1 evaluation per sample.
 Range (min … max):  11.350 μs …  2.772 ms  ┊ GC (min … max): 0.00% … 98.82%
 Time  (median):     12.043 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   13.577 μs ± 56.440 μs  ┊ GC (mean ± σ):  9.17% ±  2.21%

    █▇
  ▃███▆▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▂▂▂▂▂▂▂▂▂▂▂▂ ▂
  11.4 μs         Histogram: frequency by time          25 μs <

 Memory estimate: 35.22 KiB, allocs estimate: 755.

n = 7

BenchmarkTools.Trial: 8577 samples with 1 evaluation per sample.
 Range (min … max):  461.444 μs …   3.088 ms  ┊ GC (min … max):  0.00% … 73.05%
 Time  (median):     480.591 μs               ┊ GC (median):     0.00%
 Time  (mean ± σ):   580.974 μs ± 262.912 μs  ┊ GC (mean ± σ):  15.87% ± 19.68%

  ██▅▄▃▂                                           ▂▃▃▃▂▁▁▁     ▂
  ███████▇▇▇▆▆▆▆▄▄▃▅▅▁▄▁▁▁▃▄▄▄▄▁▁▃▄▄▁▄▃▁▁▄▁▁▃▁▁▃▁▁█████████▇▇▆▆ █
  461 μs        Histogram: log(frequency) by time       1.38 ms <

 Memory estimate: 1.62 MiB, allocs estimate: 30283.

n = 8

BenchmarkTools.Trial: 1046 samples with 1 evaluation per sample.
 Range (min … max):  3.837 ms …  10.550 ms  ┊ GC (min … max):  0.00% … 17.33%
 Time  (median):     4.818 ms               ┊ GC (median):    19.09%
 Time  (mean ± σ):   4.778 ms ± 488.635 μs  ┊ GC (mean ± σ):  16.78% ±  7.33%

  ▃▄▁▂▂                      ▄█▇▅▅▆▅▄▂▁▁
  █████▇█▇▄▅▄▅▄▅▄▄▅▁▄▁▄▁▄▁▁▁▁████████████▇█▇▇▆▇▇▅▆▆▇▆▄▆▆▆▆▅▆▅ █
  3.84 ms      Histogram: log(frequency) by time      5.79 ms <

 Memory estimate: 14.77 MiB, allocs estimate: 241967.

n = 9

BenchmarkTools.Trial: 114 samples with 1 evaluation per sample.
 Range (min … max):  41.082 ms …  47.521 ms  ┊ GC (min … max):  8.59% … 16.64%
 Time  (median):     43.978 ms               ┊ GC (median):    16.26%
 Time  (mean ± σ):   44.119 ms ± 926.061 μs  ┊ GC (mean ± σ):  16.19% ±  1.33%

                      ▃▃  ▂ ▅▃▃█▅▃  ▅  ▆
  ▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇▇██▇▇█▅██████▅▄█▄▅█▇▅▅▁▄▄▅█▁▁▁▁▄▁▁▁▄▁▁▁▄▄ ▄
  41.1 ms         Histogram: frequency by time           47 ms <

 Memory estimate: 132.89 MiB, allocs estimate: 2177332.

n = 10, t = 6

BenchmarkTools.Trial: 308 samples with 1 evaluation per sample.
 Range (min … max):  13.125 ms …  19.379 ms  ┊ GC (min … max):  0.00% … 17.51%
 Time  (median):     16.235 ms               ┊ GC (median):    18.53%
 Time  (mean ± σ):   16.227 ms ± 654.551 μs  ┊ GC (mean ± σ):  18.60% ±  2.95%

                                ▂██ ▂▂▂▂ ▅█▄▁▃▂▁  ▂  ▁
  ▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▃▃▆▅▅▇███▇████████████▅▆█▃▇█▃▃▁▁▃▁▃▃ ▄
  13.1 ms         Histogram: frequency by time         17.9 ms <

 Memory estimate: 53.07 MiB, allocs estimate: 907255.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.17%. Comparing base (e005ee0) to head (3f6ee61).
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #186      +/-   ##
==========================================
- Coverage   97.22%   97.17%   -0.06%     
==========================================
  Files           8        8              
  Lines         829      813      -16     
==========================================
- Hits          806      790      -16     
  Misses         23       23              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[oameye]

This looks great. No comments

[FedericoStra]

Of course I don't wish to take any credit at all for this silly PR. All the merit goes to whoever coded the fast nextpermutation(m, t, state) function which is used by Base.iterate(p::MultiSetPermutations, s=p.ref) (I believe they are mostly @randy3k and @natemcintosh).

[FedericoStra]

I merged master because there was a conflict (caused my myself in #184 😅).

I then had to apply a small fix (collect the indices on line 28) because otherwise it failed a test with OffsetArray (check out the merge commit to see what I'm referring to).

[natemcintosh]

I'm glad to see this! Speedups are always great.

[randy3k]

The algorithm that I used in MultisetPremutation is adopted form http://alistairisrael.wordpress.com/2009/09/22/simple-efficient-pnk-algorithm/

10 years ago, it was the only source/algorithm that I could find over the internet that supported multisets and in lexicographic order. I guess it must be one of the algorithms on Wikipedia.

[oameye]

Shall we merge this? :)