Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: import public causes AssertionError #82

Closed
petermetz opened this issue Aug 13, 2021 · 7 comments · Fixed by #83
Closed

fix: import public causes AssertionError #82

petermetz opened this issue Aug 13, 2021 · 7 comments · Fixed by #83
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@petermetz
Copy link

This works fine:

import "models/health_check_response_pb.proto";

This one crashes protoc:

import public "models/health_check_response_pb.proto";

Logs:

> Executing task: yarn run proto:protoc-gen-ts <

yarn run v1.19.0
$ yarn run grpc_tools_node_protoc --plugin=protoc-gen-ts=../../node_modules/.bin/protoc-gen-ts --ts_out=grpc_js:./src/main/typescript/generated/proto/protoc-gen-ts/ --proto_path ./src/main/proto/generated/openapi/ --proto_path ./src/main/proto/generated/openapi/models/ --proto_path ./src/main/proto/generated/openapi/services/ ./src/main/proto/generated/openapi/models/*.proto ./src/main/proto/generated/openapi/services/*.proto
$ /******/node_modules/.bin/grpc_tools_node_protoc --plugin=protoc-gen-ts=../../node_modules/.bin/protoc-gen-ts --ts_out=grpc_js:./src/main/typescript/generated/proto/protoc-gen-ts/ --proto_path ./src/main/proto/generated/openapi/ --proto_path ./src/main/proto/generated/openapi/models/ --proto_path ./src/main/proto/generated/openapi/services/ ./src/main/proto/generated/openapi/models/health_check_response_pb.proto ./src/main/proto/generated/openapi/models/memory_usage_pb.proto ./src/main/proto/generated/openapi/services/default_service.proto

/******/node_modules/google-protobuf/google-protobuf.js:81
jspb.BinaryConstants.TWO_TO_32=4294967296;jspb.BinaryConstants.TWO_TO_52=4503599627370496;jspb.BinaryConstants.TWO_TO_63=0x7fffffffffffffff;jspb.BinaryConstants.TWO_TO_64=1.8446744073709552E19;jspb.BinaryConstants.ZERO_HASH="\x00\x00\x00\x00\x00\x00\x00\x00";goog.dom={};goog.dom.NodeType={ELEMENT:1,ATTRIBUTE:2,TEXT:3,CDATA_SECTION:4,ENTITY_REFERENCE:5,ENTITY:6,PROCESSING_INSTRUCTION:7,COMMENT:8,DOCUMENT:9,DOCUMENT_TYPE:10,DOCUMENT_FRAGMENT:11,NOTATION:12};goog.debug={};goog.debug.Error=function(a){if(Error.captureStackTrace)Error.captureStackTrace(this,goog.debug.Error);else{var b=Error().stack;b&&(this.stack=b)}a&&(this.message=String(a));this.reportErrorToServer=!0};goog.inherits(goog.debug.Error,Error);goog.debug.Error.prototype.name="CustomError";goog.asserts={};goog.asserts.ENABLE_ASSERTS=goog.DEBUG;goog.asserts.AssertionError=function(a,b){goog.debug.Error.call(this,goog.asserts.subs_(a,b));this.messagePattern=a};goog.inherits(goog.asserts.AssertionError,goog.debug.Error);goog.asserts.AssertionError.prototype.name="AssertionError";goog.asserts.DEFAULT_ERROR_HANDLER=function(a){throw a;};goog.asserts.errorHandler_=goog.asserts.DEFAULT_ERROR_HANDLER;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
Error [AssertionError]: Assertion failed
    at new goog.asserts.AssertionError (/******/node_modules/google-protobuf/google-protobuf.js:81:876)
    at Object.goog.asserts.doAssertFailure_ (/******/node_modules/google-protobuf/google-protobuf.js:82:257)
    at Object.goog.asserts.assert (/******/node_modules/google-protobuf/google-protobuf.js:83:83)
    at jspb.BinaryReader.readPackedField_ (/******/node_modules/google-protobuf/google-protobuf.js:398:249)
    at jspb.BinaryReader.readPackedInt32 (/******/node_modules/google-protobuf/google-protobuf.js:399:68)
    at Function.deserialize (/******/node_modules/protoc-gen-ts/src/compiler/descriptor.js:233:56)
    at /******/node_modules/protoc-gen-ts/src/compiler/plugin.js:193:151
    at jspb.BinaryReader.readMessage (/******/node_modules/google-protobuf/google-protobuf.js:385:329)
    at Function.deserialize (/******/node_modules/protoc-gen-ts/src/compiler/plugin.js:193:28)
    at Object.<anonymous> (/******/node_modules/protoc-gen-ts/src/index.js:22:45) {
  reportErrorToServer: true,
  messagePattern: 'Assertion failed'
}
--ts_out: protoc-gen-ts: Plugin failed with status code 1.
/******/node_modules/grpc-tools/bin/protoc.js:41
    throw error;
    ^

Error: Command failed: /******/node_modules/grpc-tools/bin/protoc --plugin=protoc-gen-grpc=/******/node_modules/grpc-tools/bin/grpc_node_plugin --plugin=protoc-gen-ts=../../node_modules/.bin/protoc-gen-ts --ts_out=grpc_js:./src/main/typescript/generated/proto/protoc-gen-ts/ --proto_path ./src/main/proto/generated/openapi/ --proto_path ./src/main/proto/generated/openapi/models/ --proto_path ./src/main/proto/generated/openapi/services/ ./src/main/proto/generated/openapi/models/health_check_response_pb.proto ./src/main/proto/generated/openapi/models/memory_usage_pb.proto ./src/main/proto/generated/openapi/services/default_service.proto

/******/node_modules/google-protobuf/google-protobuf.js:81
jspb.BinaryConstants.TWO_TO_32=4294967296;jspb.BinaryConstants.TWO_TO_52=4503599627370496;jspb.BinaryConstants.TWO_TO_63=0x7fffffffffffffff;jspb.BinaryConstants.TWO_TO_64=1.8446744073709552E19;jspb.BinaryConstants.ZERO_HASH="\x00\x00\x00\x00\x00\x00\x00\x00";goog.dom={};goog.dom.NodeType={ELEMENT:1,ATTRIBUTE:2,TEXT:3,CDATA_SECTION:4,ENTITY_REFERENCE:5,ENTITY:6,PROCESSING_INSTRUCTION:7,COMMENT:8,DOCUMENT:9,DOCUMENT_TYPE:10,DOCUMENT_FRAGMENT:11,NOTATION:12};goog.debug={};goog.debug.Error=function(a){if(Error.captureStackTrace)Error.captureStackTrace(this,goog.debug.Error);else{var b=Error().stack;b&&(this.stack=b)}a&&(this.message=String(a));this.reportErrorToServer=!0};goog.inherits(goog.debug.Error,Error);goog.debug.Error.prototype.name="CustomError";goog.asserts={};goog.asserts.ENABLE_ASSERTS=goog.DEBUG;goog.asserts.AssertionError=function(a,b){goog.debug.Error.call(this,goog.asserts.subs_(a,b));this.messagePattern=a};goog.inherits(goog.asserts.AssertionError,goog.debug.Error);goog.asserts.AssertionError.prototype.name="AssertionError";goog.asserts.DEFAULT_ERROR_HANDLER=function(a){throw a;};goog.asserts.errorHandler_=goog.asserts.DEFAULT_ERROR_HANDLER;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
Error [AssertionError]: Assertion failed
    at new goog.asserts.AssertionError (/******/node_modules/google-protobuf/google-protobuf.js:81:876)
    at Object.goog.asserts.doAssertFailure_ (/******/node_modules/google-protobuf/google-protobuf.js:82:257)
    at Object.goog.asserts.assert (/******/node_modules/google-protobuf/google-protobuf.js:83:83)
    at jspb.BinaryReader.readPackedField_ (/******/node_modules/google-protobuf/google-protobuf.js:398:249)
    at jspb.BinaryReader.readPackedInt32 (/******/node_modules/google-protobuf/google-protobuf.js:399:68)
    at Function.deserialize (/******/node_modules/protoc-gen-ts/src/compiler/descriptor.js:233:56)
    at /******/node_modules/protoc-gen-ts/src/compiler/plugin.js:193:151
    at jspb.BinaryReader.readMessage (/******/node_modules/google-protobuf/google-protobuf.js:385:329)
    at Function.deserialize (/******/node_modules/protoc-gen-ts/src/compiler/plugin.js:193:28)
    at Object.<anonymous> (/******/node_modules/protoc-gen-ts/src/index.js:22:45) {
  reportErrorToServer: true,
  messagePattern: 'Assertion failed'
}
--ts_out: protoc-gen-ts: Plugin failed with status code 1.

    at ChildProcess.exithandler (node:child_process:326:12)
    at ChildProcess.emit (node:events:365:28)
    at maybeClose (node:internal/child_process:1067:16)
    at Socket.<anonymous> (node:internal/child_process:453:11)
    at Socket.emit (node:events:365:28)
    at Pipe.<anonymous> (node:net:661:12) {
  killed: false,
  code: 1,
  signal: null,
  cmd: '/******/node_modules/grpc-tools/bin/protoc --plugin=protoc-gen-grpc=/******/node_modules/grpc-tools/bin/grpc_node_plugin --plugin=protoc-gen-ts=../../node_modules/.bin/protoc-gen-ts --ts_out=grpc_js:./src/main/typescript/generated/proto/protoc-gen-ts/ --proto_path ./src/main/proto/generated/openapi/ --proto_path ./src/main/proto/generated/openapi/models/ --proto_path ./src/main/proto/generated/openapi/services/ ./src/main/proto/generated/openapi/models/health_check_response_pb.proto ./src/main/proto/generated/openapi/models/memory_usage_pb.proto ./src/main/proto/generated/openapi/services/default_service.proto'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
The terminal process "bash '-c', 'yarn run proto:protoc-gen-ts'" terminated with exit code: 1.

Terminal will be reused by tasks, press any key to close it.
petermetz added a commit to petermetz/cacti that referenced this issue Aug 13, 2021
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes hyperledger-cacti#1189

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to petermetz/cacti that referenced this issue Aug 17, 2021
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes hyperledger-cacti#1189

Signed-off-by: Peter Somogyvari <[email protected]>
@thesayyn
Copy link
Owner

Hey. Thank you for reporting this.

I will try to take a look this week.

petermetz added a commit to petermetz/cacti that referenced this issue Aug 26, 2021
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes hyperledger-cacti#1189

Signed-off-by: Peter Somogyvari <[email protected]>
petermetz added a commit to hyperledger-cacti/cacti that referenced this issue Aug 27, 2021
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes #1189

Signed-off-by: Peter Somogyvari <[email protected]>
@thesayyn
Copy link
Owner

Update;

Upon closer look, I realized that public import directive transitively re-exports the symbols from dependencies. When you have two public imports with same package name they will conflict and lead to compiler errors.

TLDR: It is not so easy to solve this problem of re-exporting transitive dependencies.

// old.proto
syntax = "proto3";

import public "public.proto";
import public "public2.proto";

message OldMarker {
    importdirective.PublicMessage pubMessageField = 4;
    importdirective.PublicMessage2 pubMessageField2 = 5;
}
// public.proto
syntax = "proto3";

package importdirective;


message PublicMessage {
    bool pb = 1;
}
// public2.proto
syntax = "proto3";

package importdirective;

message PublicMessage2 {
    bool pb = 1;
}

this will lead to export conflicts. in order to work around this, we need to explicitly import symbols from transitive dependencies and re-export the symbols from them within a namespace whose name is identical to conflicting packages.

this is what the protoc compiler wants us to do. merge everything under a namespace if the protos has the same package directive hence it is safe to do because protoc guarantees that there can not be identically named symbols under the same name

@thesayyn
Copy link
Owner

But still, there are more cursed cases;

syntax = "proto3";

package importdirective;

import public "test/import/public.proto";
import public "test/import/public2.proto";

message OldMarker {
    importdirective.PublicMessage pubMessageField = 4;
    importdirective.PublicMessage2 pubMessageField2 = 5;
}

where we need to re-export everything that conflicts as well as the symbols in the proto within the same package.

@petermetz
Copy link
Author

@thesayyn Quite the can of worms indeed. :-) Thank you again for the investigation/fix efforts!

hanxu12 pushed a commit to hanxu12/cactus that referenced this issue Aug 29, 2021
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes hyperledger-cacti#1189

Signed-off-by: Peter Somogyvari <[email protected]>
@thesayyn thesayyn added bug Something isn't working help wanted Extra attention is needed labels Aug 31, 2021
@thesayyn
Copy link
Owner

Also seen as: protocolbuffers/protobuf-javascript#55 in js version. This is something that we definitely have to solve here.

@thesayyn thesayyn reopened this Aug 31, 2021
@thesayyn thesayyn mentioned this issue Oct 11, 2021
RafaelAPB pushed a commit to RafaelAPB/blockchain-integration-framework that referenced this issue Mar 9, 2022
Primary change:
--------------
The API server now contains a gRPC server in addition to what it
had before (HTTP+SocketIO servers) so that through this it can
provide the opportunity for plugins to expose their gRPC services
via the Cactus API server. It is possible for plugins to serve
their requests on multiple protocols at the same time which means
that this is a big step towards our goal of being properly multi-
protocol capable.

Secondary change(s):
-------------------
1. Custom protobuf-schema openapi generator template added because
of this issue: thesayyn/protoc-gen-ts#82
(we override the stock template to not use the public keyword)

TODO:
----

1. Implement streaming healthcheck endpiont with gRPC

2. Allow plugins to hook in their own gRPC service implementations.

Fixes hyperledger-cacti#1189

Signed-off-by: Peter Somogyvari <[email protected]>
@thesayyn
Copy link
Owner

thesayyn commented Mar 1, 2024

I am going to close this as not planned.

@thesayyn thesayyn closed this as completed Mar 1, 2024
@petermetz
Copy link
Author

@thesayyn Cool, thanks for the update! Hopefully the upstream repo adds support to it at some point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
2 participants