Enable github-api to support GraalVM native images #1908
Comments
|
I saw a lot of other issues which are caused by the heavy usage of reflections. When I add all classes which are instantiated with reflections, like explained in the issue, I received the following error Edit: I created a demo project for showcase: https://github.com/klopfdreh/github-api-native-test Here you can run the GitHubApiNativeTestApplication in Java and everything is working fine. When you perform a native image build (like explained in the All images are arm based - if you want to try it on x86_64 - you have to change the docker image and the liberica NIK which is downloaded - see Dockerfile. |
|
@klopfdreh I've created a PR to create a v2.x That removes Java 8 and most of the reflection from this project. But it is already out of date and needs more attention to complete. PRs welcome, but sounds like the changes you're asking if I would be breaking changes. So we might want to include the 2.x effort. |
|
Hey @bitwiseman - yes it seems to be better in 2.x then. I think it all narrows down to the way objects are read and the root value is passed into the newly created objects with InjectableValues. Sadly I just came across this issue as we are moving all of our projects to Spring Boot Native and one of it is using this GitHub-API. I don’t know if I have the time to fix this as we have a lot of other projects being ported. I just created a demo project which can be used to test native image. When there will be a near final version of 2.x then let me know - I could test it for you. |
|
FWIW, we are heavily using this API in Quarkus projects and you might find some inspiration here: You could probably use this work to generate a native-image reflection config file. In any case, given Jackson is heavily reflection-based (as all the general purpose JSON library out there), you will end up with some reflection. |
Hey - thanks a lot for the heads up! Is there anyway to translate this into something which can be used only by GraalVM and does not rely ony Quarkus / JBoss API? If yes we could provide it here to enable the project for native images in a more generic way. |
|
Basically, you need to provide a reflection config file in JSON that lists all the reflection that is needed. That's what Quarkus does from the classes I pointed out. You could probably script something that generates most of the JSON file from what I pointed out. Again, with an annotation and an annotation processor, things would be a lot easier. I adjust this code every time there is a release to make sure everything is listed (I'm a couple of micros late). |
|
Thank you for the explanation. I got it working with a See last commit: klopfdreh/github-api-native-test@8c50f6c @bitwiseman what we could do to make this work for Spring Boot Native:
org.springframework.aot.hint.RuntimeHintsRegistrar=org.kohsuke.github.aot.GitHubRuntimeHints
package org.kohsuke.github.aot;
import org.springframework.aot.hint.MemberCategory;
import org.springframework.aot.hint.RuntimeHints;
import org.springframework.aot.hint.RuntimeHintsRegistrar;
import org.springframework.aot.hint.TypeReference;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Arrays;
import java.util.jar.JarEntry;
import java.util.jar.JarInputStream;
import java.util.logging.Level;
import java.util.logging.Logger;
public class GitHubRuntimeHints implements RuntimeHintsRegistrar {
private static final Logger LOGGER = Logger.getLogger(GitHubRuntimeHints.class.getName());
@Override
public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
Arrays.stream(System.getProperty("java.class.path").split(File.pathSeparator)).forEach(classpathEntry -> {
// If the classpathEntry is no jar skip it
if (!classpathEntry.endsWith(".jar")) {
return;
}
try (JarInputStream is = new JarInputStream(new FileInputStream(classpathEntry))) {
JarEntry entry;
while ((entry = is.getNextJarEntry()) != null) {
if (entry.getName().endsWith(".class") && entry.getName().startsWith("org/kohsuke/github")) {
String githubApiClassName = entry.getName().replace("/", ".");
String githubApiClassNameWithoutClass = githubApiClassName.substring(0, githubApiClassName.length() - 6);
LOGGER.log(Level.INFO, "Registered class " + githubApiClassNameWithoutClass + " for reflections and serialization.");
hints.reflection().registerType(TypeReference.of(githubApiClassNameWithoutClass), MemberCategory.values());
hints.serialization().registerType(TypeReference.of(githubApiClassNameWithoutClass));
}
}
} catch (IOException e) {
LOGGER.log(Level.INFO, "Error while reading jars", e);
}
});
}
}When ever GitHub-API is used as jar in a Spring Boot Native project it should be caught up automatically all serialization and reflection hints during the aot-processing. I think the spring dependency could be introduced as optional as this See like it works in a Spring Project (In this case in Spring Batch): What I did to make this future proof is to iterate through all classes and get all in the |
|
@bitwiseman - if I should I create a PR let me know. 👍 |
|
I think it would need to be a separate artifact. Having a dependency on Spring in such a low level library looks odd. |
So you suggest to add a new repository to hub4j called „github-api-spring-boot“ and if you want to build native applications you need to add both of them? (I let the native information out of the name as this dependency may also provide other Spring related integrations) Would be fine for me, too. 👍 |
|
It’s really for @bitwiseman to decide :) I know that for Quarkus, we are dealing with this in extensions and we avoid cluttering the upstream projects. I would have a similar approach here. |
|
Note that on the opposite side, if we go the low level GraalVM metadata route, it could make sense to include them upstream. |
Yes that’s true, but it is very nice that you share your experiences here to find the right way to go. 😃 |
I think this depends on how often new classes which require reflections are introduced. As there will be a 2.x soon it might be more future proof to introduce projects for Spring / Quarkus - but let’s see how @bitwiseman decides. 👍 |
|
I currently have very little time to devote to this project. I would rather not fork to a separate repository - but creating a separate release branch that generates a separate artifact is relatively low cost. See https://github.com/hub4j/github-api/tree/release/v1.x-unbridged If there's a way to do this without adding a SpringBoot dependency that would be better. |
|
@bitwiseman in this case there is no need for a separate release branch. I created a PR which covers all reflection and serialization hints generated by the I also double checked if there is no new class requires reflection hints in the main branch compared to the latest release. |
|
For reference/thought: As noted by @gsmet, Quarkus goes the route of manually registering classes for reflection. The GraalVM docs discuss how to generate these files based on tracing: Since we enforce code coverage on our data classes, it might be possible to generate these files based on execution of all the tests. But I think I've made reasonable suggestion for how to make #1914 a workable solution without either of the above. |
|
In the PR a suggestion has been implemented. It is without generate reachable meta data, but provides a way to add GraalVM without Spring / Quarkus classes during runtime. |
Describe the bug
Due to some reflections you encounter errors during the runtime when
github-apiis used in a native image.Example
To Reproduce
Steps to reproduce the behavior:
github-apinative-imagebuildExpected behavior
github-apishould be used in a native image without any issuesDesktop (please complete the following information):
N/A
Additional context
You could add
META-INF/native-image/<groupid>/<artifactid>/reflect-config.jsonand describe the reflection usage:Example:
[ { "name": "org.kohsuke.github.GHRepository", <settings for reflections> } ]The text was updated successfully, but these errors were encountered: