Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement saved query substitution for S3 integrations #1705

Merged
merged 10 commits into from
Apr 17, 2024

Conversation

Swiddis
Copy link
Collaborator

@Swiddis Swiddis commented Apr 16, 2024

Description

Activates the queries workflow with some simple example queries. It turns out that queries had hard refs to the tables and data sources they're defined with, so I also had to implement the ability to swap them out by adding it to the integration building process.

Issues Resolved

N/A

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <[email protected]>
@Swiddis Swiddis added enhancement New feature or request backport 2.x integrations Used to denote items related to the Integrations project labels Apr 16, 2024
@Swiddis Swiddis changed the title Add example queries to existing S3 integrations Implement saved query substitution for S3 integrations Apr 16, 2024
const builtInstance = await this.buildInstance(integration, refs, options);
return builtInstance;
}

// If we have a data source or table specified, hunt for saved queries and update them with the
// new DS/table.
substituteQueries(assets: SavedObject[], dataSource?: string, tableName?: string): SavedObject[] {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heart of the substitution updates: here's where we search for saved queries to update

workflows?: string[],
skipRedirect?: boolean
skipRedirect?: boolean,
dataSourceInfo?: { dataSource: string; tableName: string }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the front-end we need to add this pair of params to send to the back-end, so we do so. Todo: this method has a lot of parameters and should probably be broken up somehow.

@Swiddis
Copy link
Collaborator Author

Swiddis commented Apr 16, 2024

Going to have zombie lint failures, tracking issue #1363

@YANG-DB YANG-DB merged commit 0b58e64 into opensearch-project:main Apr 17, 2024
14 of 19 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/dashboards-observability/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/dashboards-observability/backport-2.x
# Create a new branch
git switch --create backport/backport-1705-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0b58e64baa0cd26df839a3ed46e038d61b031baf
# Push it to GitHub
git push --set-upstream origin backport/backport-1705-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/dashboards-observability/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1705-to-2.x.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Apr 17, 2024
* Add example queries for cloudfront

Signed-off-by: Simeon Widdis <[email protected]>

* Add Flint S3 label to haproxy

Signed-off-by: Simeon Widdis <[email protected]>

* Add query substitution method to build process

Signed-off-by: Simeon Widdis <[email protected]>

* Fix tests

Signed-off-by: Simeon Widdis <[email protected]>

* Update build API with new data source fields

Signed-off-by: Simeon Widdis <[email protected]>

* Update frontend to use new API term

Signed-off-by: Simeon Widdis <[email protected]>

* Use new fields in original integration creation request

Signed-off-by: Simeon Widdis <[email protected]>

* Fix broken data source usage ref

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
(cherry picked from commit 0b58e64)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
RyanL1997 pushed a commit to RyanL1997/dashboards-observability that referenced this pull request Apr 18, 2024
amsiglan pushed a commit to amsiglan/dashboards-observability that referenced this pull request Jun 7, 2024
…oject#1705)

* Add example queries for cloudfront

Signed-off-by: Simeon Widdis <[email protected]>

* Add Flint S3 label to haproxy

Signed-off-by: Simeon Widdis <[email protected]>

* Add query substitution method to build process

Signed-off-by: Simeon Widdis <[email protected]>

* Fix tests

Signed-off-by: Simeon Widdis <[email protected]>

* Update build API with new data source fields

Signed-off-by: Simeon Widdis <[email protected]>

* Update frontend to use new API term

Signed-off-by: Simeon Widdis <[email protected]>

* Use new fields in original integration creation request

Signed-off-by: Simeon Widdis <[email protected]>

* Fix broken data source usage ref

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
(cherry picked from commit 0b58e64)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x enhancement New feature or request integrations Used to denote items related to the Integrations project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants