You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During analysis of DoFn types, when there's an issue with a type such as having an unencodable Exported field type, it can end up causing a message bad return type for <DoFn>: <type> but not including why it's a bad return type.
Eg The following can trigger this message through this analysis.
type myRet struct{
Error error // unencodable interface type as field type.
}
The complexity in fixing this though is that it may be possible that the wrapper type (myRet) has a coder registered for it, so it's permitted in this position. This might not be known at the functional Go type analysis level funcx operates on.
The ideal solution may be to change the analysis somewhat: The funcx analysis should return a list of suspect types, where they're coming from (this is critical), and why it's possibly a bad type, which a higher level validator can that inspect and validate against the higher level information (such as availability of coders for that type). This would prevent false rejections, and provide complete debugging information without iterating the pipeline one problem at a time.
But it would be sufficient to simply have the current message be more detailed about why the type is bad, instead of the current, less obvious situation.
Issue Priority
Priority: 3 (nice-to-have improvement)
Issue Components
Component: Python SDK
Component: Java SDK
Component: Go SDK
Component: Typescript SDK
Component: IO connector
Component: Beam YAML
Component: Beam examples
Component: Beam playground
Component: Beam katas
Component: Website
Component: Spark Runner
Component: Flink Runner
Component: Samza Runner
Component: Twister2 Runner
Component: Hazelcast Jet Runner
Component: Google Cloud Dataflow Runner
The text was updated successfully, but these errors were encountered:
What would you like to happen?
During analysis of DoFn types, when there's an issue with a type such as having an unencodable Exported field type, it can end up causing a message
bad return type for <DoFn>: <type>
but not including why it's a bad return type.Message comes from here: https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/funcx/fn.go#L468
Eg The following can trigger this message through this analysis.
The complexity in fixing this though is that it may be possible that the wrapper type (
myRet
) has a coder registered for it, so it's permitted in this position. This might not be known at the functional Go type analysis level funcx operates on.The ideal solution may be to change the analysis somewhat: The funcx analysis should return a list of suspect types, where they're coming from (this is critical), and why it's possibly a bad type, which a higher level validator can that inspect and validate against the higher level information (such as availability of coders for that type). This would prevent false rejections, and provide complete debugging information without iterating the pipeline one problem at a time.
But it would be sufficient to simply have the current message be more detailed about why the type is bad, instead of the current, less obvious situation.
Issue Priority
Priority: 3 (nice-to-have improvement)
Issue Components
The text was updated successfully, but these errors were encountered: