[BEAM-14048] [CdapIO] Add ConfigWrapper for building CDAP PluginConfigs

[Amar3tto]

Resolves #24961

CDAP plugins use their own configuration classes (ex. SalesforceSourceConfig, SalesforceStreamingSourceConfig) to set the required parameters to prepare and run pipeline in Source or Sink classes. All these config classes are extended from the base io.cdap.cdap.api.plugin.PluginConfig class.
As a first step in integration of CDAP plugins and Apache Beam, there should be implemented a mechanism to create an instance of the specific PluginConfig implementation.

For this purpose, suggested solution provides a ConfigWrapper<T> class. Apache Beam user creates an instance of this class by passing the PluginConfig specific class as a T parameter.
ConfigWrapper<T> object provides simple methods to instantiate the corresponding T object:

1. fromJsonFile(File file)
2. withParams(Map<String, Object> params)
3. setParam(String paramName, Object value)
4. build()

Apache Beam user could set required parameters for the CDAP plugin by passing the Map/JSON file or by manually setting each parameter.
After that, when the build() method was called, the ConfigWrapper<T> uses InstantiationUtils, which in turn uses Reflection API. InstantiationUtils class provides mechanisms for:

1. Creating an empty PluginConfig object bypassing the limitations of private constructors or the absence of no-args constructor.
2. Going over all Fields with the cdap.Name annotation and setting the appropriate parameter values for them.
   As a result ConfigWrapper<T> returns PluginConfig object which can be further used when integrating with the corresponding plugin.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[codecov]

Codecov Report

Merging #17051 (9df1c75) into master (8cda8a2) will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #17051      +/-   ##
==========================================
+ Coverage   73.96%   74.03%   +0.07%     
==========================================
  Files         671      679       +8     
  Lines       88245    89292    +1047     
==========================================
+ Hits        65267    66110     +843     
- Misses      21866    22079     +213     
+ Partials     1112     1103       -9     

Flag	Coverage Δ
python	83.64% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdks/go/pkg/beam/core/util/reflectx/types.go	0.00% <0.00%> (-94.74%)	⬇️
sdks/go/pkg/beam/core/util/reflectx/call.go	0.00% <0.00%> (-89.75%)	⬇️
sdks/go/pkg/beam/core/util/reflectx/structs.go	0.00% <0.00%> (-76.93%)	⬇️
sdks/go/pkg/beam/core/util/reflectx/functions.go	54.54% <0.00%> (-27.28%)	⬇️
sdks/go/pkg/beam/provision/provision.go	27.50% <0.00%> (-20.00%)	⬇️
...pache_beam/runners/interactive/interactive_beam.py	77.81% <0.00%> (-3.58%)	⬇️
...che_beam/runners/interactive/interactive_runner.py	90.84% <0.00%> (-2.73%)	⬇️
...ers/portability/fn_api_runner/watermark_manager.py	93.33% <0.00%> (-2.67%)	⬇️
sdks/python/apache_beam/utils/interactive_utils.py	92.68% <0.00%> (-2.44%)	⬇️
sdks/python/apache_beam/coders/typecoders.py	92.40% <0.00%> (-2.12%)	⬇️
... and 152 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8cda8a2...9df1c75. Read the comment docs.

[Amar3tto]

Run Java PreCommit

[Amar3tto]

Run Java PreCommit

[Amar3tto]

Run CommunityMetrics PreCommit

[Amar3tto]

Run CommunityMetrics PreCommit

[chamikaramj]

Run CommunityMetrics PreCommit

[Amar3tto]

Run CommunityMetrics PreCommit

[chamikaramj]

Thanks!

[chamikaramj]

Why was this needed ?

[Amar3tto]

This is a temporary solution until we deal with the vendored dependencies. (e.x. com.github.data-integrations:salesforce)

[chamikaramj]

Should we wait till dependency issue is resolved before committing the code ?
How would this work for release Beam ? Sounds like this just builds a local SNAPSHOT jar and probably not good as a permanent dependency for Beam.

[Amar3tto]

I agree that it's better to wait for the plugins to be published in the Maven Central Repository.

[Amar3tto]

retest this please

[Amar3tto]

Run Java PreCommit

[Amar3tto]

Run Python PreCommit

[chamikaramj]

Thanks.

[chamikaramj]

Should we wait till dependency issue is resolved before committing the code ?
How would this work for release Beam ? Sounds like this just builds a local SNAPSHOT jar and probably not good as a permanent dependency for Beam.

[chamikaramj]

Run CommunityMetrics PreCommit

[chamikaramj]

Thanks. LGTM.