Use shared pre-computation for by and perm orderings.

[LilithHafner]

• Add internal order preparation API to let sorting algorithms prepare elements for repeated comparisons
• Use the API
• Add performance tests

How many times by is called

cnt = Ref(0)
incr_identity = x -> (cnt[] += 1; x)
x = 1:50

cnt[] = 0
@test issorted(x; by=incr_identity)
println(cnt[]) # 98 => 50

cnt[] = 0
@test searchsortedfirst(x, 1; by=incr_identity) == 1
printlnt(cnt[]) # 17 => 7

cnt[] = 0
@test searchsorted(repeat(1:10, inner=10), 3; by=incr_identity) == 21:30
println(cnt[]) # 26 => 16

cnt[] = 0
x = hash.(1:1000) # stable rng lol
@test sort(x; by=incr_identity) == sort(x)
println(cnt[]) # 24638 => 12999

Runtime performance tests

by_f = x -> exp(x) % 3
x = 1:50
sx = sort(x; by=by_f);
y = hash.(1:1000); # stable rng lol

@btime issorted(x; by=$by_f) # 30.066 ns => 30.570 ns
@btime issorted(sx; by=$by_f) # 977.417 ns => 471.649 ns
@btime searchsortedfirst($sx, 1; by=$by_f) # 161.942 ns => 129.231 ns
@btime searchsorted($sx, 3; by=$by_f) # 176.821 ns => 132.665 ns
@btime sort($y; by=$by_f); # 8.569 μs (3 allocations: 7.88 KiB) => 7.385 μs (3 allocations: 7.88 KiB)

Something to maybe consider doing later is marking this lt_prepared API public.

A downside of this PR is that it makes comparing with an Ordering subtype that defines neither lt nor lt_prepared a stack overflow error instead of a method not found error. This could happen of someone defines MyOrder <: Ordering and lt(::MyOrder, ::MyType, ::MyType) and then calls lt(o::MyOrder, 1, 2). It's possible to remove that downside by replacing

lt(::Ordering, x, y) = lt_prepared(o, prepare(o, x), prepare(o, y))

with

lt(::Union{ReverseOrdering, By, Perm}, x, y) = lt_prepared(o, prepare(o, x), prepare(o, y))

But that makes the (internal) lt_prepared API more cludgy to use.

[nsajko]

Fabulous idea!

A downside of this PR is that it makes comparing [...]

Couldn't you fix that by using "traits" to declare that a type supports the new API?

struct Preparable end
struct NotPreparable end

preparable(::Ordering) = NotPreparable()

preparable(::ReverseOrdering) = Preparable()
preparable(::By) = Preparable()
preparable(::Perm) = Preparable()

If you need preparable to be recursive (didn't look closely), you could, e.g., replace preparable(::Perm) = Preparable() with preparable(p::Perm) = preparable(p.order).

[LilithHafner]

We could do that, or we could say the "trait" is to define

lt(::MyNewPreparableOrdering, x, y) = lt_prepared(o, prepare(o, x), prepare(o, y))

In order of verbosity:

1. PR (stack overflow)
2. This comment's "trait"s
3. Full traits

Full traits also allow algorithmic dispatch based on the presence of preparability, but I think that is actually something we do not want to support because I want supporting preparation to remain totally free (i.e. never a regression vs not supporting preparation). If you have a comparison that is 90% lt and 10% preparation, it still makes sense to share the preparation and I don't want to dispatch based on the existence of preparation for that reason.

Unfortunately, preparability is only recursive if each Ordering supports it explicitly.

MyUnpreparableBy(-, By(abs)) is not preparable because there is no way (that I know of) for us to figure out what the appropriate input to abs is without the cooperation of MyUnpreparableBy's author.

[LilithHafner]

The API this adds is internal, the downside in the OP is no longer present, and I'd like to merge this before working on sortperm performance so that that stuff cleanly generalizes to by orders as well. So I'm inclined to merge this unless anyone objects.