Add spark applicationId #1778
Merged
Add spark applicationId #1778
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
rupal-bq
requested review from
pjfitzgibbons,
ps48,
kavithacm,
derek-ho,
joshuali925,
dai-chen,
YANG-DB,
mengweieric,
vmmusings,
Swiddis,
penghuo,
seankao-az,
MaxKsyunz,
Yury-Fridlyand,
anirudha,
forestmvey and
acarbonetto
as code owners
|
Is it possible to do an IT for that? |
Need to check. will address that in follow up PR. |
|
Could you elaborate a little what's the difference and why this change? |
In spark connector, we will be able to use different spark clients. In case of EMR client, stepId can be used to fetch the result from opensearch index but stepId concept is specific to EMR. So when we will use another client e.g. Livy client then we can use spark applicationId to get results from opensearch index because Livy response includes appId. This is basically just for adding alternate option to stepId if it's not emr in design of #1722 |
penghuo
approved these changes
Jul 3, 2023
dai-chen
approved these changes
Jul 5, 2023
matthewryanwells
pushed a commit
to Bit-Quill/opensearch-project-sql
that referenced
this pull request
Jul 7, 2023
* Initial spark application draft Signed-off-by: Rupal Mahajan <[email protected]> * Remove temp table Signed-off-by: Rupal Mahajan <[email protected]> * Add license header Signed-off-by: Rupal Mahajan <[email protected]> * Add scalastyle-config and update readme Signed-off-by: Rupal Mahajan <[email protected]> * Fix datatype for result and schema Signed-off-by: Rupal Mahajan <[email protected]> * Add test Signed-off-by: Rupal Mahajan <[email protected]> * Simplify code using toJSON.collect.toList Signed-off-by: Rupal Mahajan <[email protected]> * Add example in readme Signed-off-by: Rupal Mahajan <[email protected]> * Fix triple quotes issue Signed-off-by: Rupal Mahajan <[email protected]> * Update method name and description Signed-off-by: Rupal Mahajan <[email protected]> * Add applicationId Signed-off-by: Rupal Mahajan <[email protected]> --------- Signed-off-by: Rupal Mahajan <[email protected]>
matthewryanwells
pushed a commit
to Bit-Quill/opensearch-project-sql
that referenced
this pull request
Jul 7, 2023
* Initial spark application draft Signed-off-by: Rupal Mahajan <[email protected]> * Remove temp table Signed-off-by: Rupal Mahajan <[email protected]> * Add license header Signed-off-by: Rupal Mahajan <[email protected]> * Add scalastyle-config and update readme Signed-off-by: Rupal Mahajan <[email protected]> * Fix datatype for result and schema Signed-off-by: Rupal Mahajan <[email protected]> * Add test Signed-off-by: Rupal Mahajan <[email protected]> * Simplify code using toJSON.collect.toList Signed-off-by: Rupal Mahajan <[email protected]> * Add example in readme Signed-off-by: Rupal Mahajan <[email protected]> * Fix triple quotes issue Signed-off-by: Rupal Mahajan <[email protected]> * Update method name and description Signed-off-by: Rupal Mahajan <[email protected]> * Add applicationId Signed-off-by: Rupal Mahajan <[email protected]> --------- Signed-off-by: Rupal Mahajan <[email protected]> Signed-off-by: Matthew Wells <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Issues Resolved
#1722
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.