Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error during derivation for recursive data types #144

Closed
tschuchortdev opened this issue May 14, 2023 · 6 comments · Fixed by #195
Closed

Error during derivation for recursive data types #144

tschuchortdev opened this issue May 14, 2023 · 6 comments · Fixed by #195
Labels
bug Something isn't working question Further information is requested

Comments

@tschuchortdev
Copy link

tschuchortdev commented May 14, 2023

I'm trying to generically derive instances of FunctorK for recursive data structures:

trait FunctorK[D[_[_]]] {
  extension[F[_]] (df: D[F])
    def mapK[G[_]](fg: [A] => F[A] => G[A]): D[G]
}

object FunctorK {
  inline def apply[D[_[_]]]: FunctorK[D] = summonInline

  given monoFunctorK[A]: FunctorK[[F[_]] =>> F[A]] with {
    extension[F[_]] (df: F[A])
      override def mapK[G[_]](fg: [B] => F[B] => G[B]): G[A] = fg(df)
  }

  given phantomFunctorK[A]: FunctorK[[_[_]] =>> A] with {
    extension[F[_]] (df: A)
      override def mapK[G[_]](fg: [B] => F[B] => G[B]): A = df
  }

  given productFunctorK[D[_[_]]] (using pInst: K11.ProductInstances[FunctorK, D]): FunctorK[D] with {
    extension[F[_]] (df: D[F])
      override def mapK[G[_]](fg: [A] => F[A] => G[A]): D[G] =
        pInst.map(df) {
          [t[_[_]]] => (fieldFunctorK: FunctorK[t], field: t[F]) =>
            fieldFunctorK.mapK(field)(fg)
        }
  }

  given coproductFunctorK[D[_[_]]] (using cInst: K11.CoproductInstances[FunctorK, D]): FunctorK[D] with {
    extension[F[_]] (df: D[F])
      override def mapK[G[_]](fg: [A] => F[A] => G[A]): D[G] =
        cInst.fold(df) {
          [t[f[_]] <: D[f]] => (caseFunctorK: FunctorK[t], _case: t[F]) =>
            caseFunctorK.mapK(_case)(fg)
        }
  }

  given wrappedFunctorK[D[_[_]], H[_]] (using fkd: FunctorK[D], functorH: Functor[H]
                                       ): FunctorK[[F[_]] =>> H[D[F]]] with {
    extension[F[_]] (hdf: H[D[F]])
      override def mapK[G[_]](fg: [A] => F[A] => G[A]): H[D[G]] =
        functorH.map(hdf) { (df: D[F]) => fkd.mapK(df)(fg) }
  }
}

Given following example

sealed trait Foo2[F[_]]

case class Foo2Rec[F[_]](x: F[Int], y: Foo2[F]) extends Foo2[F]

case class Foo2Empty[F[_]](x: F[Int]) extends Foo2[F]

val m = summon[Mirror.ProductOf[Foo2Rec[Option]]]

summon[K11.ProductInstances[FunctorK, Foo2Rec]]

an error will be thrown by the compiler:

val m: 
  scala.deriving.Mirror.Product{
    MirroredMonoType = Foo2Rec[Option]; MirroredType = Foo2Rec[Option]; 
      MirroredLabel = "Foo2Rec"
    ; MirroredElemTypes = (Option[Int], Foo2[Option]); 
      MirroredElemLabels = ("x", "y")
  } = Foo2Rec

-- Error: ----------------------------------------------------------------------
  1 |summon[K11.ProductInstances[FunctorK, Foo2Rec]]
    |                                               ^
    |No given instance of type com.tschuchort.FunctorK[Foo2] was found.
    |I found:
    |
    |    com.tschuchort.FunctorK.coproductFunctorK[Foo2](
    |      {
    |        val gen$proxy247: 
    |          deriving.Mirror.Sum{
    |            Kind = shapeless3.deriving.K11.type; MirroredType[X[_$121]] = Foo2[X]; 
    |              MirroredMonoType = Foo2[[_] =>> Any]
    |            ; MirroredElemTypes[_$123[_$124]] <: Tuple
    |          } & 
    |            scala.deriving.Mirror.Sum{
    |              MirroredMonoType = Foo2[?[_$121]]; MirroredType[X[_$121]] = Foo2[X]; 
    |                MirroredLabel = ("Foo2" : String)
    |              ; MirroredElemTypes[X[_$121]] = (Foo2Rec[X], Foo2Empty[X]); 
    |                MirroredElemLabels = (("Foo2Rec" : String), ("Foo2Empty" : String))
    |            }
    |         = 
    |          {
    |            final class $anon() extends Object(), Serializable {
    |              type MirroredMonoType = Foo2[?[_$121]]
    |            }
    |            (new $anon():Object & Serializable)
    |          }.$asInstanceOf[
    |            deriving.Mirror.Sum{
    |              Kind = shapeless3.deriving.K11.type; MirroredType[X[_$121]] = Foo2[X]
    |                ; 
    |              MirroredMonoType = Foo2[[_] =>> Any]; 
    |                MirroredElemTypes[_$123[_$124]] <: Tuple
    |            } & 
    |              scala.deriving.Mirror.Sum{
    |                MirroredMonoType = Foo2[?[_$121]]; MirroredType[X[_$121]] = Foo2[X]
    |                  ; 
    |                MirroredLabel = ("Foo2" : String); 
    |                  MirroredElemTypes[X[_$121]] = (Foo2Rec[X], Foo2Empty[X])
    |                ; 
    |                  MirroredElemLabels = (("Foo2Rec" : String), ("Foo2Empty" : String)
    |                    )
    |              }
    |          ]
    |        new 
    |          shapeless3.deriving.internals.ErasedCoproductInstances[
    |            shapeless3.deriving.K11.type
    |          , com.tschuchort.FunctorK[Foo2]]
    |        (gen$proxy247, 
    |          {
    |            val arr$proxy247: Array[Any] = new Array[Any](2)
    |            {
    |              arr$proxy247.update(0, 
    |                compiletime.summonInline[com.tschuchort.FunctorK[Foo2Rec]]
    |              )
    |              shapeless3.deriving.summonAsArray0[
    |                com.tschuchort.FunctorK[Foo2Empty] *: EmptyTuple
    |              ](1, arr$proxy247)
    |            }:Array[Any]
    |          }:Array[Any]
    |        ):
    |          shapeless3.deriving.internals.ErasedCoproductInstances[
    |            shapeless3.deriving.K11.type
    |          , com.tschuchort.FunctorK[Foo2]]
    |        :shapeless3.deriving.K11.CoproductInstances[com.tschuchort.FunctorK, Foo2]
    |      }
    |    )
    |
    |But given instance mkCoproductInstances in object K11 does not match type shapeless3.deriving.K11.CoproductInstances[com.tschuchort.FunctorK, Foo2].
    |---------------------------------------------------------------------------
    |Inline stack trace
    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |This location contains code that was inlined from deriving.scala:46
    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |This location contains code that was inlined from deriving.scala:46
    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |This location contains code that was inlined from deriving.scala:46
    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |This location contains code that was inlined from deriving.scala:46
    |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    |This location contains code that was inlined from deriving.scala:46
     ---------------------------------------------------------------------------

A similar problem also appears for classes like this:

case class Foo3[F[_]](x: F[Int], y: Option[Foo3[F]])
summon[FunctorK[Foo3]]

Note that an almost identical error with mkProductInstances can be produced depending on the order of summoning (Foo2 or Foo2Rec first).

I believe the error might occur because of an infinite recursion in the derivation logic that reaches the inlining depth limit. When the inlining limit is reached, further calls to the inline function (ErasedCoproductInstances.apply?) are no longer inlined, perhaps causing the types to no longer match. A different error message, which I am unfortunately unable to reproduce, suggested this to me.

I understand that recursive data structures were already addressed in #40, so I'm not sure if it's a bug in the library or in my own code.

Library version: 3.3.0
Scala version: 3.2.1
IntelliJ plugin version: 2023.1.548

@joroKr21
Copy link
Member

It looks like it might be a bug. I will try it out later, but in the meantime - we have a test for FunctorK in the test suite and it has a much simpler definition, since mapK is one of the methods exposed by Shapeless. I wonder if that definition works?

@tschuchortdev
Copy link
Author

we have a test for FunctorK in the test suite and it has a much simpler definition, since mapK is one of the methods exposed by Shapeless. I wonder if that definition works?

I just tried it and it doesn't work either (the implementation is functionally identical except for some additional cases in my code).

  | => derivingJVM / Test / compileIncremental 13s
[error] -- Error: C:\Users\Thilo\Documents\IntelliJProjects\shapeless-3\modules\deriving\src\test\scala\shapeless3\deriving\deriving.scala:263:26
[error] 263 |    summon[FunctorK[Foo2]]
[error]     |                          ^mental 13s
[error]     |No given instance of type shapeless3.deriving.FunctorK[Foo2Rec] was found.
[error]     |I found:
[error]     |
[error]     |    shapeless3.deriving.FunctorK.functorKGen[Foo2Rec](
[error]     |      {
[error]     |        val gen$proxy250:
[error]     |
[error]     |            deriving.Mirror{
[error]     |              Kind = shapeless3.deriving.K11.type;
[error]     |                MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |              ; MirroredMonoType = Foo2Rec[[_] =>> Any];
[error]     |                MirroredElemTypes[_$123[_$124]] <: Tuple
[error]     |            }
[error]     |           &
[error]     |            scala.deriving.Mirror.Product{
[error]     |              MirroredMonoType = Foo2Rec[?[_$121]];
[error]     |                MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |              ; MirroredLabel = ("Foo2Rec" : String);
[error]     |                MirroredElemTypes[X[_$121]] = (X[Int], Foo2[X])
[error]     |              ; MirroredElemLabels = (("x" : String), ("y" : String))
[error]     |            }
[error]     |
[error]     |         =
[error]     |          Foo2Rec.$asInstanceOf[
[error]     |
[error]     |              deriving.Mirror{
[error]     |                Kind = shapeless3.deriving.K11.type;
[error]     |                  MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                ; MirroredMonoType = Foo2Rec[[_] =>> Any];
[error]     |                  MirroredElemTypes[_$123[_$124]] <: Tuple
[error]     |              }
[error]     |             &
[error]     |              scala.deriving.Mirror.Product{
[error]     |                MirroredMonoType = Foo2Rec[?[_$121]];
[error]     |                  MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                ; MirroredLabel = ("Foo2Rec" : String);
[error]     |                  MirroredElemTypes[X[_$121]] = (X[Int], Foo2[X])
[error]     |                ; MirroredElemLabels = (("x" : String), ("y" : String))
[error]     |              }
[error]     |
[error]     |          ]
[error]     |        {
[error]     |          val $scrutinee523:
[error]     |
[error]     |              (gen$proxy250 :
[error]     |                deriving.Mirror{
[error]     |                  Kind = shapeless3.deriving.K11.type;
[error]     |                    MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                  ; MirroredMonoType = Foo2Rec[[_] =>> Any];
[error]     |                    MirroredElemTypes[_$123[_$124]] <: Tuple
[error]     |                }
[error]     |               &
[error]     |                scala.deriving.Mirror.Product{
[error]     |                  MirroredMonoType = Foo2Rec[?[_$121]];
[error]     |                    MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                  ; MirroredLabel = ("Foo2Rec" : String);
[error]     |                    MirroredElemTypes[X[_$121]] = (X[Int], Foo2[X])
[error]     |                  ; MirroredElemLabels = (("x" : String), ("y" : String))
[error]     |                }
[error]     |              )
[error]     |
[error]     |           = gen$proxy250
[error]     |          val p:
[error]     |
[error]     |              deriving.Mirror{
[error]     |                Kind = shapeless3.deriving.K11.type;
[error]     |                  MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                ; MirroredMonoType = Foo2Rec[[_] =>> Any];
[error]     |                  MirroredElemTypes[_$123[_$124]] <: Tuple
[error]     |              }
[error]     |             &
[error]     |              scala.deriving.Mirror.Product{
[error]     |                MirroredMonoType = Foo2Rec[?[_$121]];
[error]     |                  MirroredType[X[_$121]] = Foo2Rec[X]
[error]     |                ; MirroredLabel = ("Foo2Rec" : String);
[error]     |                  MirroredElemTypes[X[_$121]] = (X[Int], Foo2[X])
[error]     |                ; MirroredElemLabels = (("x" : String), ("y" : String))
[error]     |              }
[error]     |
[error]     |           = $scrutinee523
[error]     |          shapeless3.deriving.internals.ErasedProductInstancesN.apply[
[error]     |            shapeless3.deriving.K11.type
[error]     |          , shapeless3.deriving.FunctorK[Foo2Rec]](p,
[error]     |            {
[error]     |              val arr$proxy235: Array[Any] = new Array[Any](2)
[error]     |              {
[error]     |                arr$proxy235.update(0,
[error]     |                  shapeless3.deriving.FunctorK.given_FunctorK_Id[Int]
[error]     |                )
[error]     |                {
[error]     |                  arr$proxy235.update(1,
[error]     |                    compiletime.summonInline[shapeless3.deriving.FunctorK[Foo2]]
[error]     |                  )
[error]     |                  shapeless3.deriving.summonAsArray0[EmptyTuple.type](2,
[error]     |                    arr$proxy235
[error]     |                  )
[error]     |                }:Array[Any]
[error]     |              }:Array[Any]
[error]     |            }:Array[Any]
[error]     |          ):
[error]     |            shapeless3.deriving.internals.ErasedProductInstances[
[error]     |              shapeless3.deriving.K11.type
[error]     |            , shapeless3.deriving.FunctorK[Foo2Rec]]
[error]     |          :
[error]     |            shapeless3.deriving.K11.ProductInstances[shapeless3.deriving.FunctorK,
[error]     |              Foo2Rec
[error]     |            ]
[error]     |        }:shapeless3.deriving.K11.Instances[shapeless3.deriving.FunctorK, Foo2Rec]
[error]     |      }
[error]     |    )
[error]     |
[error]     |But given instance mkInstances in object K11 does not match type shapeless3.deriving.K11.Instances[shapeless3.deriving.FunctorK, Foo2Rec].
[error]     |---------------------------------------------------------------------------
[error]     |Inline stack trace
[error]     |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]     |This location contains code that was inlined from deriving.scala:46
[error]  46 |    arr(i) = summonInline[a]
[error]     |             ^^^^^^^^^^^^^^^
[error]     |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]     |This location contains code that was inlined from deriving.scala:46
[error]  41 |  summonAsArray0[T](0, new Array[Any](constValue[Tuple.Size[T]]))
[error]     |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]     |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]     |This location contains code that was inlined from deriving.scala:46
[error] 353 |    new ErasedCoproductInstances[K, FT](mirror, summonAsArray[E])
[error]     |                                                ^^^^^^^^^^^^^^^^
[error]     |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error]     |This location contains code that was inlined from deriving.scala:46
[error] 471 |    ErasedCoproductInstances[K11.type, F[T], LiftP[F, gen.MirroredElemTypes]](gen)
[error]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]      ---------------------------------------------------------------------------

@joroKr21
Copy link
Member

Ok, thanks for trying 👍

@agaro1121
Copy link

agaro1121 commented Jan 10, 2024

I'm not sure if I ran into into the same issue but I have issues with recursion as well:

// THIS WORKS
  @Test
  def testRec: Unit =
    final case class Rec(as: Vector[Rec])
    Typeable[Rec]

// THIS DOES NOT WORK
  final case class Rec2(as: Vector[Rec2])
  @Test
  def testRec2: Unit =
    Typeable[Rec2]

is this expected behavior?

The second example is generally how I'd summon the Typeable instance in a different piece of code than the case classes

@joroKr21
Copy link
Member

@tschuchortdev sorry that I didn't see this sooner, but you need to make the coproduct instances by-name if you want it to work with recursive types. The same applies to our example in the tests:

given coproductFunctorK[D[_[_]]] (using cInst: => K11.CoproductInstances[FunctorK, D]): FunctorK[D] with {
    extension[F[_]] (df: D[F])
      override def mapK[G[_]](fg: [A] => F[A] => G[A]): D[G] =
        cInst.fold(df) {
          [t[f[_]] <: D[f]] => (caseFunctorK: FunctorK[t], _case: t[F]) =>
            caseFunctorK.mapK(_case)(fg)
        }
  }
@@ -333,7 +333,7 @@ object FunctorK:
   given [T]: FunctorK[K11.Id[T]] with
     def mapK[A[_], B[_]](at: A[T])(f: A ~> B): B[T] = f(at)
 
-  given functorKGen[H[_[_]]](using inst: K11.Instances[FunctorK, H]): FunctorK[H] with
+  given functorKGen[H[_[_]]](using inst: => K11.Instances[FunctorK, H]): FunctorK[H] with
     def mapK[A[_], B[_]](ha: H[A])(f: A ~> B): H[B] =
       inst.map(ha)([t[_[_]]] => (ft: FunctorK[t], ta: t[A]) => ft.mapK(ta)(f))

@joroKr21 joroKr21 added bug Something isn't working question Further information is requested labels Jan 10, 2024
@joroKr21
Copy link
Member

@agaro1121 that's a different issue. Typeable uses a macro to materialize so it has to be fixed to work with recursive types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants