Thinking Longterm, Dotty support means removal of all macros

[LukaJCB]

We should start discussing the best way to handle migration to Dotty.
Fortunately we don't have a lot of macros in cats, the few that I can think of are Simulacrum/Machinist and the FunctionK.lift macro. We could manually expand the former (as suggested by @smarter), but I'm not sure how to handle the latter. Maybe someone else has other good ideas :)

[smarter]

FunctionK.lift macro

Seems like this only exists to work around the lack of polymorphic function types, so if it's really a blocker that could force me to prioritize getting them in sooner :). In fact, I'd like to add polymorphic SAM auto-adaptation too, so you could just write val lifted: FunctionK[List, Option] = headOption.

More generally, I'd welcome removing all macros from cats given its central place in the Scala ecosystem. A macro-less cats could probably easily become part of the dotty community build, which would help us make sure we don't break it, and would make it much easier to experiment with new features related to typeclasses or type-level programming in general in a real codebase.

[LukaJCB]

Seems like this only exists to work around the lack of polymorphic function types, so if it's really a blocker that could force me to prioritize getting them in sooner :). In fact, I'd like to add polymorphic SAM auto-adaptation too, so you could just write val lifted: FunctionK[List, Option] = headOption.

That is precisely why it exists and I think a lot of us would love to see them being prioritized!
SAM auto-adaption sounds even better 😍

More generally, I'd welcome removing all macros from cats given its central place in the Scala ecosystem. A macro-less cats could probably easily become part of the dotty community build, which would help us make sure we don't break it, and would make it much easier to experiment with new features related to typeclasses or type-level programming in general in a real codebase.

I think we can probably do this in source compatible way even.
Regarding FunctionK.lift, we could probably go the unsafe route and just use asInstanceOf, which works in general, but allows one to do some strange things. I'm not sure if this breaks anything or is just weird, but we could probably experiment with it.

[LukaJCB]

I played around with it a bit, and the fact that you can insert a type doesn't seem to cause any breakage, all of this compiles fine:

import cats._
import cats.arrow.FunctionK

sealed trait Example[A]

def lift[F[_], G[_]]
  (f: (F[α] ⇒ G[α]) forSome { type α }): FunctionK[F, G] =
    new FunctionK[F, G] {
      def apply[A](fa: F[A]): G[A] = 
        f.asInstanceOf[F[A] => G[A]](fa)
    }
  
def sample[A](option: Option[A]): List[A] = option.toList
  
val lifted1: FunctionK[Option, List] = lift(sample[Int])
val lifted2: FunctionK[Option, List] = lift(sample[Any])
val lifted3: FunctionK[Option, List] = lift(sample[Example[String]])
val lifted4: FunctionK[Option, List] = lift(sample)
  
def foo[F[_], A](fa: F[A])(f: F ~> List): List[A] =
  f(fa)
  
val res1: List[String] = foo(Option("2"))(lifted1)
val res2: List[String] = foo(Option("2"))(lifted2)
val res3: List[String] = foo(Option("2"))(lifted3)
val res4: List[String] = foo(Option("2"))(lifted4)

val res5: List[Int] = foo(Option(2))(lifted1)
val res6: List[Int] = foo(Option(2))(lifted2)
val res7: List[Int] = foo(Option(2))(lifted3)
val res8: List[Int] = foo(Option(2))(lifted4)

Scalafiddle: https://scalafiddle.io/sf/223M0ER/2

[smarter]

Note that there's no forSome in Dotty so you'd have to write def lift[F[_], G[_], A](f: F[A] => G[A])

[LukaJCB]

Ideally with polymorphic functions that'd be (depending on syntax) def lift[F[_], G[_]](f: (A -> F[A] => G[A])): FunctionK[F, G], correct? :)

[smarter]

modulo syntax yes

[larsrh]

I played around with it a bit, and the fact that you can insert a type doesn't seem to cause any breakage, all of this compiles fine:

When you want to provoke the CCE, you have to think like the CCE 😜

import cats._
import cats.arrow.FunctionK

def lift[F[_], G[_]]
  (f: (F[α] ⇒ G[α]) forSome { type α }): FunctionK[F, G] =
    new FunctionK[F, G] {
      def apply[A](fa: F[A]): G[A] = 
        f.asInstanceOf[F[A] => G[A]](fa)
    }
  
def sample(option: Option[String]): List[String] = List(option.getOrElse("").length.toString)
  
val lifted: FunctionK[Option, List] = lift(sample)

lifted(Option(3))

https://scastie.scala-lang.org/larsrh/gV8haQYCRtWDFU6JDi8NHA

[LukaJCB]

Oh wow I didn't know it would accept a Option[String] => List[String] yeah that's pretty broken 😅

[catostrophe]

This impl is fine:

object FunctionK {
  def lift[F[_]]: LiftPartiallyApplied[F] = new LiftPartiallyApplied

  private[arrow] final class LiftPartiallyApplied[F[_]](val dummy: Boolean = true) extends AnyVal {
    type T

    def apply[G[_]](f: F[T] => G[T]): FunctionK[F, G] = new FunctionK[F, G] {
      def apply[A](fa: F[A]): G[A] = f(fa.asInstanceOf[F[T]]).asInstanceOf[G[A]]
    }
  }
}

Works better than the original lift, doesn't need macros, may be defined with the first type param partially applied.

def sample1(option: Option[String]): List[String] =
  List(option.getOrElse("").length.toString)
def sample2[A](option: Option[A]): List[A] = option.toList
def sample3[A]: Option[A] => List[A] = _.toList

FunctionK.lift[Option](sample1) // doesn't compile - correct
FunctionK.lift[Option](sample2) // compiles
FunctionK.lift[Option](sample2[Any]) // doesn't compile - correct
FunctionK.lift[Option](sample3) // compiles, while the original lift doesn't

Props to @alexknvl and @Odomontois for showing this trick.

[LukaJCB]

Very nice, thanks @catostrophe! Care to make a quick PR out of this? :)
Otherwise, I'll create an issue for others to take

[catostrophe]

Doing this with the partial type param application would be source incompatible. Providing both params is redundant and inconvenient.

We can make a source compatible PR to 1.6 and improve it later when we're working on 2.0. What do you think?

[LukaJCB]

That sounds good to me, thanks a lot!

[catostrophe]

The thing I didn't notice is that type inference is a bit worse with the new approach.

def sample[A](option: Option[A]): List[A] = option.toList
val f1 = FunctionK.lift(sample) // doesn't infere
val f2: Option ~> List = FunctionK.lift(sample) // compiles
val f3 = FunctionK.lift[Option, List](sample) // compiles, can be simplified to lift[Option](sample)

So, unfortunately, it cannot be a replacement for the macro generated lift in 1.x due to source incompatibility.

[sir-wabbit]

https://github.com/alexknvl/polymorphic/tree/master/src/test/scala

    class Foo[A]

    test("constructors") {
        val f1: ∀[Foo] = ∀(new Foo)
        val f2: ∀[Foo] = Forall(new Foo)
        val f3: ∀[Foo] = Forall.of[Foo](new Foo)
        val f4: ∀[Foo] = Forall.of[Foo].from(new Foo)
        val f5: ∀[Foo] = Forall.from(new Prototype[Foo] {
            override def apply[X]: Foo[X] = new Foo
        })
        val f7 = ∀[Foo](new Foo)
        val f8 = Forall[Foo](new Foo)
        val f9 = Forall.of[Foo](new Foo)
        val f10 = Forall.of[Foo].from(new Foo)
        val f11 = Forall.from(new Prototype[Foo] {
            override def apply[X]: Foo[X] = new Foo
        })
    }

        val f1: Option ~> List = fn(_.toList)
        val f2 = fn[Option, List](_.toList)

        val f3: Option ~> List = FunctionK(_.toList)
        val f4 = FunctionK[Option, List](_.toList)

But indeed, type-inference is sub-optimal.

I'll update it in a bit to the latest cats if anyone wants to check it out.