Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't transform coproduct - Java Enum to Sealed trait #482

Closed
8 of 10 tasks
frevib opened this issue Mar 20, 2024 · 24 comments
Closed
8 of 10 tasks

Can't transform coproduct - Java Enum to Sealed trait #482

frevib opened this issue Mar 20, 2024 · 24 comments
Labels
bug in compiler Possible compiler bug, either library can work around it or it's a wontfix bug Erroneous behavior in existing features wontfix

Comments

@frevib
Copy link

frevib commented Mar 20, 2024

Checklist

Describe the bug

If we try to convert a Java Enum to a Sealed trait, we get

Chimney can't derive transformation from com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala

SealedTraitScala
  can't transform coproduct instance com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala


SealedTraitScala
  derivation from enumjava: com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala is not supported in Chimney!

However, when we put one instance of the Java Enum in scope, e.g. by adding this to the top-level val enumJava = EnumJava.ONE, compilation succeeds.

Reproduction

We have the Scala file in src/main/scala:

//> using jvm 11
//> using scala 3.3.3
//> using options -
//> using dep io.scalaland::chimney::0.8.5

import com.eventlooopsoftware.javaenum.EnumJava
import io.scalaland.chimney.dsl.*

// uncomment this line and the code will compile...
//val enumJava = EnumJava.ONE

object Main {
  def main(args: Array[String]): Unit = {
    val enumJ: EnumJava = args.headOption.map(it => EnumJava.valueOf(it)).getOrElse(throw new IllegalArgumentException)
    val result = chimneyTransform(enumJ)
    println(result)
  }

  def chimneyTransform(enumJava: EnumJava) = enumJava.transformInto[SealedTraitScala]
}


sealed trait SealedTraitScala
object SealedTraitScala {
  case object ONE extends SealedTraitScala
  case object TWO extends SealedTraitScala
  case object THREE extends SealedTraitScala
}

And the Java Enum in the src/main/java directory:

package com.eventlooopsoftware.javaenum;

public enum EnumJava {
    ONE, TWO, THREE
}

Compilation now fails with can't transform coproduct instance com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala.

However if we add val enumJava = EnumJava.ONE somewhere in the file, compilation succeeds.

Full example is here: https://github.com/frevib/chimney-java-enum-bug-example/blob/main/src/main/scala/Main.scala

Expected behavior

Chimney to successfully transform the Java Enum to Scala Sealed trait.

Actual behavior

can't transform coproduct instance com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala

Which Chimney version do you use

0.8.5 and 1.0.0-M1

Which platform do you use

  • JVM
  • Scala.js
  • Scala Native

If you checked JVM

openjdk version "17.0.9" 2023-10-17
OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode)

Additional context

I have an idea that Chimney might not be able to find the Java Enum on the classpath during compile time. I've tried several CompileOrders, but to no use.

@frevib frevib added the bug Erroneous behavior in existing features label Mar 20, 2024
@MateuszKubuszok
Copy link
Member

Hello! Thanks for the bug report and the reproduction.

I am not sure when I'll have the time to take a look at this closer. If code compiles when you merely use the definition in scope, I suspect it might be a bug in compiler, that we'd have to workaround (and report).

There was such case in Scala 2 macros - see scala/bug#7755 that we have to work around by some hack - perhaps similar issue exists in Scala 3, we'd have to create a test case for it and work around it (if it's even possible somehow).

@jchyb have you heard of such issues?

@MateuszKubuszok
Copy link
Member

@frevib I tried to reproduce but when I compiled https://github.com/frevib/chimney-java-enum-bug-example/ there was no error. Is there something else I need to do to reproduce?

@resilica
Copy link

I tried and got the same compiler error, i'm on mac M3:

 chimney-java-enum-bug-example-main % sbt compile
[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 17.0.9)
[info] loading project definition from /Users/x/Downloads/chimney-java-enum-bug-example-main/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/Users/x/Downloads/chimney-java-enum-bug-example-main/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 1 Scala source to /Users/x/Downloads/chimney-java-enum-bug-example-main/target/scala-3.3.3/classes ...
[error] -- Error: /Users/x/Downloads/chimney-java-enum-bug-example-main/src/main/scala/Main.scala:14:85 
[error] 14 |  def chimneyTransform(enumJava: EnumJava) = enumJava.transformInto[SealedTraitScala]
[error]    |                                                                                     ^
[error]    |Chimney can't derive transformation from com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala
[error]    |
[error]    |SealedTraitScala
[error]    |  can't transform coproduct instance com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala
[error]    |
[error]    |
[error]    |SealedTraitScala
[error]    |  derivation from enumjava: com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala is not supported in Chimney!
[error]    |
[error]    |
[error]    |Consult https://chimney.readthedocs.io for usage examples.
[error]    |
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed Mar 20, 2024, 2:33:57 PM

@frevib
Copy link
Author

frevib commented Mar 20, 2024

@MateuszKubuszok no, just sbt compile should give the same error that @resilica has. I’m on sbt 1.9.9.

@MateuszKubuszok
Copy link
Member

@frevib, @resilica What else can you tell me about your environment? From what I see both of you are using Java 17.0.9 (Temurin and Adoption), should I assume both are MacOS with M architecture?

I ask because my result is:

sbt compile
[info] welcome to sbt 1.9.9 (Amazon.com Inc. Java 17.0.7)
[info] loading settings for project global-plugins from plugins.sbt,sonatype.sbt ...
[info] loading global plugins from /Users/dev/.sbt/1.0/plugins
[info] loading project definition from /private/tmp/chimney-java-enum-bug-example/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/private/tmp/chimney-java-enum-bug-example/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 1 Scala source and 1 Java source to /private/tmp/chimney-java-enum-bug-example/target/scala-3.3.3/classes ...
[success] Total time: 9 s, completed Mar 20, 2024, 2:43:38 PM

So on my M2 it compiles without issues. If the same code might or might not compile on different laptops there must be something different about the environment.

@resilica
Copy link

resilica commented Mar 20, 2024

I tried it on an intel with the same sbt/scala/java versions and it seems to compile fine as well... Can you try it with 17.0.9?

@frevib
Copy link
Author

frevib commented Mar 20, 2024

@MateuszKubuszok @resilica this is really confusing. I'm on Mac m3 (max). I've tried a couple of times and it succeeds, then fails. Here is my output, I've added -----OK and -----FAILED:

[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 17.0.9)
[info] loading project definition from /Users/jeff/tmp/scala_chimney_enum_bug/untitled/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/Users/jeff/tmp/scala_chimney_enum_bug/untitled/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 1 Scala source and 1 Java source to /Users/jeff/tmp/scala_chimney_enum_bug/untitled/target/scala-3.3.3/classes ...
[success] Total time: 2 s, completed Mar 20, 2024, 3:37:15 PM

------OK
jeff@jeffs-MacBook-Pro untitled % sbt
[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 17.0.9)
[info] loading project definition from /Users/jeff/tmp/scala_chimney_enum_bug/untitled/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/Users/jeff/tmp/scala_chimney_enum_bug/untitled/)
[info] sbt server started at local:///Users/jeff/.sbt/1.0/server/a04fc1dbd14c6eb321e8/sock
[info] started sbt server
sbt:chimney-java-enum-bug-example> compile

------OK 
[success] Total time: 0 s, completed Mar 20, 2024, 3:37:28 PM
sbt:chimney-java-enum-bug-example> clean
[success] Total time: 0 s, completed Mar 20, 2024, 3:37:30 PM
sbt:chimney-java-enum-bug-example> compile

------OK
[info] compiling 1 Scala source and 1 Java source to /Users/jeff/tmp/scala_chimney_enum_bug/untitled/target/scala-3.3.3/classes ...
[success] Total time: 2 s, completed Mar 20, 2024, 3:37:59 PM
sbt:chimney-java-enum-bug-example> compile
[success] Total time: 0 s, completed Mar 20, 2024, 3:38:17 PM

------OK
sbt:chimney-java-enum-bug-example> exit
[info] shutting down sbt server
jeff@jeffs-MacBook-Pro untitled % java -version
openjdk version "17.0.9" 2023-10-17
OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9)
OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode)
jeff@jeffs-MacBook-Pro untitled % sbt clean
[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 17.0.9)
[info] loading project definition from /Users/jeff/tmp/scala_chimney_enum_bug/untitled/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/Users/jeff/tmp/scala_chimney_enum_bug/untitled/)
[success] Total time: 0 s, completed Mar 20, 2024, 3:38:31 PM
jeff@jeffs-MacBook-Pro untitled % sbt
[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 17.0.9)
[info] loading project definition from /Users/jeff/tmp/scala_chimney_enum_bug/untitled/project
[info] loading settings for project root from build.sbt ...
[info] set current project to chimney-java-enum-bug-example (in build file:/Users/jeff/tmp/scala_chimney_enum_bug/untitled/)
[info] sbt server started at local:///Users/jeff/.sbt/1.0/server/a04fc1dbd14c6eb321e8/sock
[info] started sbt server
sbt:chimney-java-enum-bug-example> compile
[info] compiling 1 Scala source and 1 Java source to /Users/jeff/tmp/scala_chimney_enum_bug/untitled/target/scala-3.3.3/classes ...
[success] Total time: 2 s, completed Mar 20, 2024, 3:38:38 PM

------OK
sbt:chimney-java-enum-bug-example> clean
[success] Total time: 0 s, completed Mar 20, 2024, 3:38:41 PM
sbt:chimney-java-enum-bug-example> compile
[info] compiling 1 Scala source to /Users/jeff/tmp/scala_chimney_enum_bug/untitled/target/scala-3.3.3/classes ...
[error] -- Error: /Users/jeff/tmp/scala_chimney_enum_bug/untitled/src/main/scala/Main.scala:14:85 
[error] 14 |  def chimneyTransform(enumJava: EnumJava) = enumJava.transformInto[SealedTraitScala]
[error]    |                                                                                     ^
[error]    |Chimney can't derive transformation from com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala
[error]    |
[error]    |SealedTraitScala
[error]    |  can't transform coproduct instance com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala
[error]    |
[error]    |
[error]    |SealedTraitScala
[error]    |  derivation from enumjava: com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala is not supported in Chimney!
[error]    |
[error]    |
[error]    |Consult https://chimney.readthedocs.io for usage examples.
[error]    |
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 0 s, completed Mar 20, 2024, 3:38:44 PM
sbt:chimney-java-enum-bug-example> 

-----FAILED

Intellij also thinks it's an error:

Screenshot 2024-03-20 at 15 47 25

@MateuszKubuszok
Copy link
Member

I am pretty sure at this point that this is a compiler bug which for this code:

    def isSealed[A: Type]: Boolean = {
      val flags = TypeRepr.of[A].typeSymbol.flags
      flags.is(Flags.Enum) || flags.is(Flags.Sealed)
    }

sometimes returns false - even though Java Enums are supposed to return true on this flag check. I think I could work around this code by changing this

    def parse[A: Type]: Option[Enum[A]] =
      // no need for separate java.lang.Enum handling contrary to Scala 2
      if isSealed[A] then Some(symbolsToEnum(extractSealedSubtypes[A]))
      else None

into

    def parse[A: Type]: Option[Enum[A]] =
      // we should have no need for separate java.lang.Enum handling contrary to Scala 2
      // but there is a bug that makes it necessary
      if isSealed[A] || isJavaEnum[A] then Some(symbolsToEnum(extractSealedSubtypes[A]))
      else None

but with no reproduction I can neither test if that would actually help nor report a bug to the Scala 3 compiler team. :/

@frevib
Copy link
Author

frevib commented Mar 20, 2024

@MateuszKubuszok for now the only way to reproduce this, is doing a lot of clean and then compile, and some “luck”. Maybe a script of doing this many times will reproduce the error in x % of the cases.

For now I’ll add an Enum instance in scope.

@MateuszKubuszok
Copy link
Member

I'm on a conference now, so I have very limited capability for debugging this. However I've run:

for i in $(seq 1 100); do sbt clean compile; done

and let the code clean and compile for 19 minutes melting my 2019 Mac's bettery... and the issue haven't occurred even once.

Looking at the code I figured out that without a reproduction I am missing data for any workaround (just checking if class extends java.lang.Enum is not enough, I have to rely on flags there, and flags is what is broken), so ATM there is nothing I can do. Someone who can actually reproduce would have to print and analyze the flags and symbols for the broken case to be able to provide a workaround in the macro.

@frevib
Copy link
Author

frevib commented Mar 21, 2024

Thanks for your effort looking into it. For now I'll use the mitigation by placing val enumJava = EnumJava.ONE somewhere in scope. When I have an idea how to consistently reproduce, I will post here.

@MateuszKubuszok
Copy link
Member

I've made some blind best-effort attempt at https://github.com/scalalandio/chimney/compare/scala-3-java-enum-bugfix?expand=1 - if that would solve the issue, I could merge but, but if it doesn't then I am unfortunately out of options, and I can only hope that it will gets reproduced by someone in Scala 3 compiler team.

@frevib
Copy link
Author

frevib commented Mar 21, 2024

My Intellij (Zinc compiler) is consistent in saying this is an error, sbt is not. I could check out and build above branch and see what Intellij has to say. This will not give 100% confidence, but it's something.

@MateuszKubuszok
Copy link
Member

@frevib were you able to check if the workaround helps with anything?

@frevib
Copy link
Author

frevib commented Mar 29, 2024

Yes, adding an instance of the enum in scope solves the problem:

val enumJava = EnumJava.ONE

This can be added top-level or anywhere else in scope, e.g. inside the method.

@MateuszKubuszok
Copy link
Member

And how about the fix from the branch https://github.com/scalalandio/chimney/compare/scala-3-java-enum-bugfix?expand=1 ? Does it help with anything or is it a dead end?

@frevib
Copy link
Author

frevib commented Mar 30, 2024

I did not try that yet. I got a bit stuck in building the Chimney package from source. Are there instructions on how to build Chimney? I will do it asap then.

@MateuszKubuszok
Copy link
Member

I think RELEASE=true sbt chimneyMacroCommons3/publishLocal chimney3/publishLocal should do the trick.

@frevib
Copy link
Author

frevib commented Mar 31, 2024

Unfortunately I cannot give a definitive answer. I've tried the new 1.0.0-M2 build and it gave the same error. Then I reverted back to 0.8.5 and it compiled successfully, at least 10 times in a row. I've tried many other things for a hour, and could not find any consistent result. I found some odd behaviour in Intellij though. After a sbt clean the compiled classes are deleted from the target directory, and then automatically return (see image).

At this point I don't know any more. Intellij's UI keeps saying that Chimney can't derive transformation from com.eventlooopsoftware.javaenum.EnumJava to SealedTraitScala, but sbt compile does work most of the time now. It seems like it's something very weird with my Intellij or a compiler bug that does does not surface on Easter Sunday 😕 .

That's all I can do for now.
Screenshot 2024-03-31 at 19 30 01

@resilica can you try one more time? If no one else has any problems we can close this issue.

@MateuszKubuszok
Copy link
Member

1.0.0-M2 did not have the workaround merged (and neither did 1.0.0-M3 nor 1.0.0-M4 - I don't want to introduce a complexity that I am not sure would be justified), so at best it shows that nothing got fixed accidentally in the meantime. The workaround would have to tested by:

git clone https://github.com/scalalandio/chimney.git
cd chimney
git checkout scala-3-java-enum-bugfix
export RELEASE=true # turn off -Xfatal-warnings which fails scaladoc (sic!) generation
sbt chimneyMacroCommons3/publishLocal chimney3/publishLocal
# then use chimney in version 1.0.0-M2-2-g775e2ce6-SNAPSHOT

However, I am almost certain that it would be some weird bug in the compiler. I might have even more certainty (or rule that out) if failed logs were enriched with .enableMacrosLogging and the info output of the failed line (it will not be shown in IJ if there is an error at the same position, so the bug would have to be reproduced in console).

@frevib
Copy link
Author

frevib commented Apr 5, 2024

Yes, scala-3-java-enum-bugfix is used.

I also had to include the java-collections:

RELEASE=true sbt chimneyMacroCommons3/publishLocal chimney3/publishLocal chimneyJavaCollections3/publishLocal

Then use the 1.0.0-M2-5-g5dc1807-SNAPSHOT version.

The error still occurs at random times. 95% of the times it works though. I was able to reproduce it for a while by deleting the target directory, and then running compile in the sbt cli, but later it just constantly worked. I have absolutely no idea what's going on here.

@MateuszKubuszok
Copy link
Member

If the workaround didn't help then I am not sure anymore where the bug occurs. If it's not the flags, then maybe the .children property, but at this point I am completely unable to fix that on the library's side. Either someone from the compiler team would be able to reproduce it or it's unfixable.

@MateuszKubuszok MateuszKubuszok added the bug in compiler Possible compiler bug, either library can work around it or it's a wontfix label Apr 5, 2024
@frevib
Copy link
Author

frevib commented Apr 5, 2024

Thx for looking into it.

@resilica where you able to test?

@MateuszKubuszok
Copy link
Member

Since there is nothing we can do do fix this, and the only solution I can see is fixing it in the compiler, I'll close it now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug in compiler Possible compiler bug, either library can work around it or it's a wontfix bug Erroneous behavior in existing features wontfix
Projects
None yet
Development

No branches or pull requests

3 participants