Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

protoparse: match the way protoc populates aggregate value in uninterpreted options #526

Merged
merged 1 commit into from
Aug 18, 2022

Conversation

jhump
Copy link
Owner

@jhump jhump commented Aug 18, 2022

This dumps the original source input for the aggregate as its string value, but we strip formatting/whitespace/comments and just put a single space between each token (also omit enclosing braces). This is what protoc does.

This patch was ported over from https://github.com/jhump/protocompile/pull/6.

Fixes #274.


To confirm the output of protoc, beyond inspecting the implementation code linked above, I wrote a simple C++ program that calls Parser::Parse but does not do any linking or options interpretation, so I could view how the uninterpreted options fields in the descriptor protos were populated.

Here's that test program, which reads proto source from stdin and spits out the resulting descriptor proto (text format) to stdout:

// file: just-parse.cc
#include <iostream>
#include <string>
#include <unistd.h>
#include <google/protobuf/io/zero_copy_stream_impl.h>
#include <google/protobuf/io/tokenizer.h>
#include <google/protobuf/compiler/parser.h>

class PrintingErrorCollector : public google::protobuf::io::ErrorCollector {
 public:
  PrintingErrorCollector() {}

  void AddError(int line, google::protobuf::io::ColumnNumber column, const std::string& message) {
    std::cerr << "ERROR: <input>" << ":" << line << ":" << column << ": " << message << "\n";
  }
  void AddWarning(int line, google::protobuf::io::ColumnNumber column, const std::string& message) {
    std::cout << "WARN: <input>" << ":" << line << ":" << column << ": " << message << "\n";
  }
};

int main(int argc, char *argv[]) {
  google::protobuf::io::FileInputStream in(STDIN_FILENO);
  google::protobuf::FileDescriptorProto fdp;
  PrintingErrorCollector coll;
  google::protobuf::io::Tokenizer tok(&in, &coll);
  google::protobuf::compiler::Parser parser;
  parser.RecordErrorsTo(&coll);
  parser.SetRequireSyntaxIdentifier(false);
  if (! parser.Parse(&tok, &fdp)) {
    std::cerr << "parse failed\n";
    return 1;
  }
  fdp.clear_source_code_info();
  fdp.PrintDebugString();
  std::cout << "\n";
  return 0;
}

In order to build this on my Macbook Pro, I had to build/install the protobuf C++ libraries/runtime and then run the following:

clang++ -std=c++11 just-parse.cc -o just-parse `pkg-config --cflags --libs protobuf`

@jhump jhump merged commit 8dab444 into master Aug 18, 2022
@jhump jhump deleted the jh/aggregate-vals-in-uninterpreted-options branch August 18, 2022 02:36
@jhump jhump changed the title match the way protoc populates aggregate value in uninterpreted options protoparse: match the way protoc populates aggregate value in uninterpreted options Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

protoparse: aggregate values in uninterpreted options differ from those generated by protoc
1 participant