(BEAM-12628) add ReflectDatum{Reader,Writer} option for AvroCoder

[clairemcginty]

Per discussion in https://www.mail-archive.com/[email protected]/msg24830.html ; a change in Beam 2.30.0 changed the default Avro decoding behavior to decode Avro string fields as org.apache.avro.util.Utf8s instead of java.lang.Strings, by switching from ReflectDatum{Reader,Writer} to SpecificDatum{Reader,Writer} (see gist for examples). This PR would add back the option to use {Reflect*} suite with an optional Boolean flag. I'm not sure how "stylistically" Beam this change is so let me know what you think/whether this change is appropriate to add.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

ValidatesRunner compliance status (on master branch)

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---					---
Java
Python	---					---
XLang						---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[brucearctor]

@RyanSkraba -- since you were commenting on the mail list and you filed (BEAM-12628)

[brucearctor]

@iemejia -- since you are esp. well versed in things AVRO. Any comments?

[RyanSkraba]

LGTM -- thanks so much for catching this!

[iemejia]

LGTM, Thanks @clairemcginty for bringing the issue and the clean solution / PR.
Thanks @RyanSkraba too for the suggestion +review.

[iemejia]

I merged it manually to squash the commits. Since we already cut the 2.32.0 release branch if you want this to go into the 2.32.0 version we will require to cherry pick it. Can you please do this @clairemcginty ?

@angoenka I put this in the list for 2.32.0 I think the regression is important enough to deserve to be cherry picked.

[iemejia]

Oups forgot to mention that probably it is worth to add this as an issue in the release notes

[clairemcginty]

hi @iemejia, thanks for the quick response! here's the cherry-pick pr: #15295

[angoenka]

Please create a cherry pick pr.
This issue has been present since 2.30 so it's not a regression.
I am in the process of creating an RC. If this RC fails or if I decided to create another one, I can pick it up.

[clairemcginty]

Please create a cherry pick pr.
This issue has been present since 2.30 so it's not a regression.
I am in the process of creating an RC. If this RC fails or if I decided to create another one, I can pick it up.

hi @angoenka , as linked above, here's the cherry-pick PR: #15295

[iemejia]

This issue has been present since 2.30 so it's not a regression.

@angoenka this is clearly a regression (a feature that worked and stopped working) because it breaks the default behavior of AvroCoder. The fact is that it was not discovered before does not change that. I understand the annoyance of having to do a new RC but it is probably for the best interest.