List(1.0, -1.0).sorted gives a deprecation warning

[eed3si9n]

Originally posted by @odersky in #10511 (comment)

Ref scala/scala#6323
Ref scala/scala#6410

steps

scala> List(1.0, -1.0).sorted
warning: there was one deprecation warning (since 2.13.0); for details, enable `:setting -deprecation' or `:replay -deprecation'
res0: List[Double] = List(-1.0, 1.0)

problem

It gives a deprecation warning. I think this is very bad, not to say catastrophic.

expectation

These things should just work. I don't care what ordering is used and whether it is inconsistent or not (we all know IEEE is broken and unfixable). But we should do the right thing for everyday users. As an everyday user I do not care what the ordering does for NaN and -0.0, maybe I do not even know that these weird numbers exist! (and they should not exist, if I go by common mathematical logic). But I do care that I can sort a list of doubles without going into the deeper mysteries of Scala's implicit system. This requirement is now violated.

notes

I think we should revert ASAP. We should keep the old total ordering and offer IEEE as an optional alternative that needs to be imported explicitly.

[eed3si9n]

@NthPortal wrote:

I think we should revert ASAP. We should keep the old total ordering and offer IEEE as an optional alternative that needs to be imported explicitly.

The issue is, the old behaviour was the IEEE ordering, so if we switch to a total ordering by default now, there's no transition period. Are we okay with that? (see also scala/scala#6410 (comment))

[eed3si9n]

@odersky wrote:

I believe we should simply keep the old behavior. My opinion about NaN and -0.0 is:

• we are under no obligation to ensure that primitive types and boxed types behave the same
• we are under no obligation to make the ordering total or not.
• we should do the same thing that Java does
• we should not make operations that do not involve NaN and -0.0 harder than before.

My rationale is that these constructs are illogical anyway. No matter what you do, there will be weirdness. So, the best thing to do is not upset the apple cart.

[eed3si9n]

@NthPortal wrote:

thoughts @Ichoran @szeiger @SethTisue?

[eed3si9n]

@NthPortal wrote:

FWIW, Java uses a total ordering on primitives:

scala> Array[Double](5.0, 3.0, NaN, 4.0)
res0: Array[Double] = Array(5.0, 3.0, NaN, 4.0)

scala> java.util.Arrays.sort(res0)

scala> res0
res3: Array[Double] = Array(3.0, 4.0, 5.0, NaN)

[eed3si9n]

@NthPortal wrote:

Personally, I think we have an obligation to have Orderings be internally consistent (compare(_, _) < 0 iff lt(_, _) == true, etc.), which happens to mean having a total ordering.

Another possibility of course is to have the default ordering just throw an exception if you give it NaN ;P

If we must un-deprecate it, I feel the default should be the total ordering. I don't care particularly about how 0.0/-0.0 compare, as long as it's consistent, but I care that operations involving NaN (which is concerningly easy to summon) behave sanely and consistently. It happens to be that the current (deprecated) default is a total ordering, so we can just remove the annotation and fix up a few tests.

[eed3si9n]

@odersky wrote:

Java uses total ordering right? So why and when did Scala deviate from this to use IEEE ordering?

[eed3si9n]

@NthPortal wrote:

@odersky it changed in #5104 and scala/scala#76

In essence the issue is that there is an expectation (a valid one) that min and max should comply with the IEEE specification (i.e. with java.lang.Math.{min,max}). Unfortunately, that is not consistent with a total ordering.

I highly recommend reading all of the discussions on scala/scala#6323 and scala/scala#6410 (even though they're long) for the full context of how we decided on the solution we chose.

[eed3si9n]

@dwijnand wrote:

It's a shame that Java is inconsistent between java.util.Arrays.sort and java.lang.Math.{min,max}.

Can we capture this explanation and put it in on a page of docs.scala-lang.org and refer to it in the Scaladoc? Or put the whole explanation in the Scaladoc. Note @som-snytt had some thoughts around those docs in scala/scala#8191 (comment) too.

[eed3si9n]

@odersky wrote:

I have already expressed my opinion. List(1.0, -1.0).sorted just has to work, no excuses allowed.

The fact that it stopped working, despite the fact that a lot of eyes looked at it, is worrying to me. Did anyone think it was acceptable that this would cause an implicit ambiguity message? Or did it just slip through with nobody realizing the consequences?

[eed3si9n]

@odersky wrote:

Can we capture this explanation and put it in on a page of docs.scala-lang.org and refer to it in the Scaladoc? Or put the whole explanation in the Scaladoc. Note @som-snytt had some thoughts around those docs in scala/scala#8191 (comment) too.

I would be skeptical about this. The fact is: NaN is irredeemably broken. It was introduced as a hack since computer manufacturers in the 70s did not think they could provide precise exceptions and Fortran did not provide exception handling. But it makes no sense, mathematically. So yes, of course things will end up being inconsistent somewhere, and the only sensible strategy is to accept that and ignore the problem (which is what Java did, and I think they did the right thing).

[eed3si9n]

@dwijnand wrote:

Did anyone think it was acceptable that this would cause an implicit ambiguity message? Or did it just slip through with nobody realizing the consequences?

Speaking only for myself, I thought it was acceptable, given the circumstances.

Can we capture this explanation and put it in on a page of docs.scala-lang.org and refer to it in the Scaladoc? Or put the whole explanation in the Scaladoc. Note @som-snytt had some thoughts around those docs in scala/scala#8191 (comment) too.

I would be skeptical about this. The fact is: NaN is irredeemably broken.

I'll give others a chance to weigh in (particularly Nth, as I expect she's spent a long time looking at all the angles), but I still think that whatever happens (change or no change) it would be good to have some kind of documentation capturing the trade-off chosen.

[eed3si9n]

@odersky wrote:

I agree it's good to document, but any such documentation cannot be a reason to keep the current broken behavior.

For background: I came across this because a professor of mathematics was stumped by the behavior and did not know how to proceed.