Crash on null from Method.getGenericParameterTypes #389
Comments
|
Thanks for reporting @stuhood, I'll be looking into this today. |
|
I have bad news; this bug is caused by a fundamental deficiency in the reflection API. It's not possible to know what the type parameters of a lambda class are going to be if we're not at the call-site (since lambdas are generated at runtime based on the call-site information). References:
So, this means we need to use low-level, internal runtime tricks to get the actual type parameters of lambdas (which could not work in all the VMs). The only plausible workaround that I can implement in Zinc is to ignore There's a github repo supposed to fix this problem (https://github.com/jhalterman/typetools) but it doesn't solve it in its entirety (it cannot get type parameters that are not used in the implemented functional interfaces; see example in README) and it has certain requirements that I'm not sure we have in Zinc (like access to the class representing the lhs of a local variable). All in all, I think it may be best to synchronize with the Intellij team and see what they're doing to work around this limitation (their java incremental compiler must have encountered these issues at some point in the past). I think that a deeper investigation of this issue is required before proposing a stable fix -- and FTR it's not even clear to me if this solution exists or not. |
|
@jvican : A filter for the null (or something to widen the type to I'll be out for the next month, so no hurry to land anything. |
|
Perfect, I'll wait and look into a better fix then -- if there's such a fix. Enjoy the vacation @stuhood. |
|
@stuhood Let me know when you're back from vacation and we'll sync up on how to move forward on this one. |
|
@jvican : Hey Jorge: I'm back. So, presumably the risk here is that if we don't detect some type as used, we won't invalidate correctly when that type changes? Are there any sorts of guarantees that types used in lambdas are already in scope elsewhere? In a nearby method signature, for example? Alternatively, if the type is not known at all at compile time, is it by-definition safe not to have the type trigger invalidation? Given that this is one of the last issues blocking upgrading pants to this version of zinc, I think I'd be willing to live with hackier solutions that might cause overcompilation, or undercompilation even (as long as we continued to track it as a bug?). |
|
We can confirm this is happening in Akka as well; Example Akka build where it happened: https://jenkins.akka.io:8498/job/akka-nightly-jdk9/13/consoleFull We planned to release a JDK9 compatible and build today, hope we can work around this issue. |
|
Am I correct in assuming that a lambda implementation method is being looked at when the null's arise? These methods are private. Is there a reason that Zinc actually needs to reflect over private methods in Java compiled classes? |
|
To answer my own question: No. In @stuhood's test: import java.util.function.Supplier;
public class Example {
static <I, O> void call() {
Supplier<BaseBlah<I, O>> blah = () ->
new BaseBlah<I, O>() {
@Override
protected O getResponseInternal(I i) {
return null;
}
};
}
public static void main(String[] args) {
Example.<String, String>call();
}
}
abstract class BaseBlah<I, O> {
protected O getResponseInternal(I i) {
return null;
}
}The context for the crash is: Is the one that (sometimes?) gives: If we can assume that these null-s are only going to show up in place of references to generic type parameters, it would probably be safe to ignore the reference, as it is intra-compilation unit and should not be material to Zinc. This method has a generic signature of Is there a JDK bug that specifically covers this problem? JDK-8178523 appears to be similar but not the same. |
|
Here's a standalone reproduction of the JDK bug. import java.util.function.Supplier;
public class Example {
static <I, O> void call() {
Supplier<BaseBlah<I, O>> blah = () ->
new BaseBlah<I, O>() {
@Override
protected O getResponseInternal(I i) {
return null;
}
};
}
public static void main(String[] args) {
Example.<String, String>call();
}
}
abstract class BaseBlah<I, O> {
protected O getResponseInternal(I i) {
return null;
}
} |
|
Today I'm sending a PR to fix this. |
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
Because of the way lambdas are synthesized by the JVM, at the call-site, it is not possible to know which generic types do lambdas have via the current reflection API. Related but not strictly reasons why this happens can be found in JDK's issue tracker: https://bugs.openjdk.java.net/browse/JDK-8178523?jql=text%20%7E%20%22lambda%20generic%20type%22 As a result, we ignore nulls that are returned by `getGenericParameterTypes`. Fixes sbt#389.
|
Thanks for continuing to investigate gang! Looking forward to consuming |
`Method.getGenericParameterType` may sometimes return `null` for the return types when the method's return type is indeed generic. It's not clear yet where this "sometimes" happens, but it looks like the JDK is not able to tell the return type of a lambda returning the generic class. This can be seen in the `java-lambda-typeparams` scripted test. In this context, lambda metafactory synthesizes a lambda class that returns a class that is parameterized in its return type. In that context, when `ClassToAPI` inspects the synthesized lambda and tries to figure out the full return type of it, the `null` is returned. It looks like the Java reflection API sparingly returns `null`s. We can see that in `ClassToAPI` where we guard against `null`s in lots of other places. So the guard added by this commit is not a novelty, but rather the norm. Fixes sbt#389.
|
The bugfix is expected to ship in 1.0.3. |
`Method.getGenericParameterType` may sometimes return `null` for the return types when the method's return type is indeed generic. It's not clear yet where this "sometimes" happens, but it looks like the JDK is not able to tell the return type of a lambda returning the generic class. This can be seen in the `java-lambda-typeparams` scripted test. In this context, lambda metafactory synthesizes a lambda class that returns a class that is parameterized in its return type. In that context, when `ClassToAPI` inspects the synthesized lambda and tries to figure out the full return type of it, the `null` is returned. It looks like the Java reflection API sparingly returns `null`s. We can see that in `ClassToAPI` where we guard against `null`s in lots of other places. So the guard added by this commit is not a novelty, but rather the norm. Fixes sbt#389.
|
Confirmed fixed in |
### Problem #4729 is currently blocked on the fix for sbt/zinc#389, which is now released in zinc `1.0.3`. ### Solution Bump to zinc 1.0.3 to pick up that fix.
In some cases (seemingly related to lambdas),
Method.getGenericParameterTypeswill return an array containingnullTypes, which are then passed intodef defLiketo cause crashes.This commit has a self-contained repro: twitter/pants@aaa07c3 ... that crashes when compiled with zinc
1.0.0-rc3:The text was updated successfully, but these errors were encountered: