Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable gRPC code generation to utilize argument file #39291

Merged
merged 1 commit into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/src/main/asciidoc/grpc-generation-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -288,3 +288,12 @@ plugins {

IMPORTANT: It is recommended to package the `proto` files in a dependency instead of the generated classes, so Quarkus can generate optimized classes.
Refer to the <<scan-for-proto, dedicated section>> for more information.

== Argument files

When the `protoc` command line exceeds the maximum command length, you can ask Quarkus to use an argument file to pass the arguments to the `protoc` command.

To enable this feature, set the `quarkus.generate-code.grpc.use-arg-file` property in your `application.properties` file to `true`.

If you are on Windows, and the command line exceeds 8190 characters, Quarkus automatically uses an argument file to pass the arguments to the `protoc` command.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
import static java.util.Arrays.asList;

import java.io.BufferedWriter;
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
Expand Down Expand Up @@ -61,6 +64,8 @@ public class GrpcCodeGen implements CodeGenProvider {
private static final String DESCRIPTOR_SET_OUTPUT_DIR = "quarkus.generate-code.grpc.descriptor-set.output-dir";
private static final String DESCRIPTOR_SET_FILENAME = "quarkus.generate-code.grpc.descriptor-set.name";

private static final String USE_ARG_FILE = "quarkus.generate-code.grpc.use-arg-file";

private Executables executables;
private String input;

Expand Down Expand Up @@ -103,6 +108,9 @@ public boolean trigger(CodeGenContext context) throws CodeGenException {
Path workDir = context.workDir();
Path inputDir = CodeGenProvider.resolve(context.inputDir());
Set<String> protoDirs = new LinkedHashSet<>();

boolean useArgFile = context.config().getOptionalValue(USE_ARG_FILE, Boolean.class).orElse(false);

try {
List<String> protoFiles = new ArrayList<>();
if (Files.isDirectory(inputDir)) {
Expand Down Expand Up @@ -160,6 +168,21 @@ public boolean trigger(CodeGenContext context) throws CodeGenException {

command.addAll(protoFiles);

// Estimate command length to avoid command line too long error
int commandLength = command.stream().mapToInt(String::length).sum();
// 8191 is the maximum command line length for Windows
if (useArgFile || (commandLength > 8190 && OS.determineOS() == OS.WINDOWS)) {
File argFile = File.createTempFile("grpc-protoc-params", ".txt");
argFile.deleteOnExit();

try (PrintWriter writer = new PrintWriter(argFile, StandardCharsets.UTF_8)) {
for (int i = 1; i < command.size(); i++) {
writer.println(command.get(i));
}
}

command = new ArrayList<>(Arrays.asList(command.get(0), "@" + argFile.getAbsolutePath()));
}
ProcessBuilder processBuilder = new ProcessBuilder(command);

final Process process = ProcessUtil.launchProcess(processBuilder, context.shouldRedirectIO());
Expand All @@ -170,6 +193,7 @@ public boolean trigger(CodeGenContext context) throws CodeGenException {
}
postprocessing(context, outDir);
log.info("Successfully finished generating and post-processing sources from proto files");

return true;
}
} catch (IOException | InterruptedException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,7 @@ quarkus.http.enable-compression=true

%vertx.quarkus.grpc.clients.hello.port=8081
%vertx.quarkus.grpc.clients.hello.use-quarkus-grpc-client=true
%vertx.quarkus.grpc.server.use-separate-server=false
%vertx.quarkus.grpc.server.use-separate-server=false

# Force the usage of an arg file
quarkus.generate-code.grpc.use-arg-file=true
Loading