Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeDescriptor information gets lost when applying multiple DoFn on Composite Transform #20901

Open
damccorm opened this issue Jun 4, 2022 · 0 comments

Comments

@damccorm
Copy link
Contributor

damccorm commented Jun 4, 2022

While working on BEAM-12384 to preserve correctly the typeDescriptor of the Read transform. I noticed that ParDo passes the outputType information into the next transforms by taking the output of getOutputTypeDescriptor specified in the DoFn, but the subsequent transform does not get the full inputTypeDescriptor.

There are other open questions:

  1. How can we make this easier for users who rarely override the getOutputTypeDescriptor?
  2. Where the typeDescriptor should come in the SDF case, from the Restriction?

For more details:
#14854 (comment)

 

 

 

Imported from Jira BEAM-12420. Original Jira may contain additional context.
Reported by: iemejia.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant