feat(otlp-grpc-exporter-base): use statically generated protobuf code

[pichlermarc]

Which problem is this PR solving?

• gRPC exporter is currently not working with bundling tools as it uses the proto files from the build directory to generate a client on runtime.
• grpc_tools_node_protoc can generate service and protobuf definitions, however, the protobuf types generated by that tool are not compatible with types from our existing @opentelemetry/otlp-transformer package.

This PR adds support for using the gRPC exporter with bundling tools by mimicking what the code generated by gRPC's static code generator is doing, while still keeping the @opentelemetry/otlp-transformer package as-is. This is accomplished by hand-rolling a service definition and using code that is statically generated using protobufjs-cli instead.

As we already use protobuf-js-cli, in the http/protobuf exporter package, I re-used the existing script and moved it to the scripts directory in the repository root.

Fixes #2786, #3632, #3656

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

• Existing tests
• Manual testing with webpack

Checklist:

• Followed the style guidelines of this project
• Unit tests have been added
• Documentation has been updated

[codecov]

Codecov Report

Merging #3705 (3a62d02) into main (abfb1bb) will decrease coverage by 0.31%.
The diff coverage is 43.33%.

❗ Current head 3a62d02 differs from pull request most recent head eb3bb5b. Consider uploading reports for the commit eb3bb5b to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3705      +/-   ##
==========================================
- Coverage   93.50%   93.19%   -0.31%     
==========================================
  Files         293      296       +3     
  Lines        8961     8999      +38     
  Branches     1853     1853              
==========================================
+ Hits         8379     8387       +8     
- Misses        582      612      +30     

Impacted Files	Coverage Δ
...ntal/packages/otlp-grpc-exporter-base/src/types.ts	100.00% <ø> (ø)
...-grpc-exporter-base/src/LogsExportServiceClient.ts	42.85% <42.85%> (ø)
...pc-exporter-base/src/MetricsExportServiceClient.ts	42.85% <42.85%> (ø)
...grpc-exporter-base/src/TraceExportServiceClient.ts	42.85% <42.85%> (ø)
...ental/packages/otlp-grpc-exporter-base/src/util.ts	76.00% <44.44%> (+0.03%)	⬆️

... and 4 files with indirect coverage changes

[dyladan]

Looks good. I tried this myself once before and got quite frustrated

[dyladan]

Looks good to me. It looks like the precompile is running npm protos which itself is running npm submodule, then after the compile is finished lerna run protos is run again. i think this isn't required anymore? Seems to only affect grpc exporter base and proto exporter base which both handle it in the precompile step. I think also the submodules are updated once per package instead of globally. Instead of --concurrency 1 it might make sense to just run once? I think the script is not doing anything package-specific.

[pichlermarc]

Looks good to me. It looks like the precompile is running npm protos which itself is running npm submodule, then after the compile is finished lerna run protos is run again. i think this isn't required anymore? Seems to only affect grpc exporter base and proto exporter base which both handle it in the precompile step. I think also the submodules are updated once per package instead of globally. Instead of --concurrency 1 it might make sense to just run once? I think the script is not doing anything package-specific.

I'm not sure I understood correctly: did you mean that I should split the protos script into submodule update and file generation script and then only trigger the submodule update once in the top-level package.json instead? 🤔

I pushed a change that does that but keeps the protos step in each package's package.json so that the compile script in the exporter packages can still be used independently from the top level package.json(073b01a). Hope that addresses this concern 🙂

[dyladan]

Looks good to me. It looks like the precompile is running npm protos which itself is running npm submodule, then after the compile is finished lerna run protos is run again. i think this isn't required anymore? Seems to only affect grpc exporter base and proto exporter base which both handle it in the precompile step. I think also the submodules are updated once per package instead of globally. Instead of --concurrency 1 it might make sense to just run once? I think the script is not doing anything package-specific.

I'm not sure I understood correctly: did you mean that I should split the protos script into submodule update and file generation script and then only trigger the submodule update once in the top-level package.json instead? 🤔

Yes. This just prevents duplicate submodule updates when the whole project is compiled

I pushed a change that does that but keeps the protos step in each package's package.json so that the compile script in the exporter packages can still be used independently from the top level package.json(073b01a). Hope that addresses this concern 🙂

Yeah it looks right to me

[pichlermarc]

I'll wait for #3712 to merge first and then add the logs client as well.
It will be easier to do it this way around as the diff is smaller for this PR.

[pichlermarc]

Added logs too, should be good to go. 🙂
@dyladan could you have another look? 🙂