[MD] Add data source register to savedObject (#1729) #2037

kristenTian · 2022-08-02T01:13:41Z

Signed-off-by: Kristen Tian [email protected]

Description

Add data source register to savedObject

Issues Resolved

#1729

Check List

[ x] Commits are signed per the DCO using --signoff

seraphjiang · 2022-08-02T01:17:01Z

src/plugins/data_source_management/server/plugin.ts

@@ -0,0 +1,43 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0


License header looks different compare to other TS file.

This is the header generated by the node scripts/precommit_hook --fix -- therefore I think this header should be sufficient (least possible). Let me know if this header need to be consistent.

Good to learn, thanks

Not sure this script is updated. I would suggest follow the best practices mentioned here https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.eslintrc.js#L31-L55

codecov-commenter · 2022-08-02T01:53:21Z

Codecov Report

Merging #2037 (5d8f3b3) into feature/multi-datasource (2abcc21) will not change coverage.
The diff coverage is n/a.

❗ Current head 5d8f3b3 differs from pull request most recent head c60b185. Consider uploading reports for the commit c60b185 to get more accurate results

@@                    Coverage Diff                    @@
##           feature/multi-datasource    #2037   +/-   ##
=========================================================
  Coverage                     67.49%   67.49%           
=========================================================
  Files                          3076     3076           
  Lines                         59144    59144           
  Branches                       8989     8989           
=========================================================
  Hits                          39919    39919           
  Misses                        17041    17041           
  Partials                       2184     2184

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

zhongnansu · 2022-08-02T03:06:26Z

src/plugins/data_source_management/server/saved_objects/data_source.ts

+      title: {
+        type: 'text',
+      },
+      type: {


What type could it be? "opensearch"?

data-source

Is it really needed? a field in mapping with one fixed value?

Good question we should not need this. Removed

noCharger · 2022-08-02T03:18:22Z

src/plugins/data_source_management/server/saved_objects/data_source.ts

+
+import { SavedObjectsType } from 'opensearch-dashboards/server';
+
+export const dataSource: SavedObjectsType = {


Shall we put data source saved object definition under src/plugins/data/server/saved_objects?

What's the benefit to introduce data source saved object dependency to data plugin?

What's the benefit to introduce data source saved object dependency to data plugin?

If I'm not mistaken, the only server-side logic is to register the saved object type. We can remove the entire server side plugin by registering data sources in data plugin. I recall @zengyan-amazon making similar comments in previous PRs.

Didn't see any benefit, I won't approve to add to data plugin unless I'm convinced with data point.

Didn't see any benefit, I won't approve to add to data plugin unless I'm convinced with data point.

Data points FYI: Index pattern is defined in https://github.com/opensearch-project/OpenSearch-Dashboards/tree/main/src/plugins/data/server/saved_objects instead of index pattern management plugin

Could you explain why it is good idea? Or we have to do this? This against single responsibility principle

I don't think it's a blocking comment, nor it against single responsibility principle. One of the benefits is to remove entire server side plugin and simplify code structure.

introduce new dependency to data plugin is huge concern to me. Leave data source here, it will only be registered when feature is enabled. There is no single line of code change need to make in data plugin.

I'm ok to create another server side code only plugin. But that's overkill.

introduce new dependency to data plugin is huge concern to me. Leave data source here, it will only be registered when feature is enabled. There is no single line of code change need to make in data plugin.

I'm ok to create another server side code only plugin. But that's overkill.

I think we are aligned that server side plugin is an overkill. Instead of saying it introduces a dependency on the data plugin, I would say creating a new service in the data plugin and allowing other plugins to rely on it is not a bad idea. We can publish the current code and leave the community to review.

Adding another plugin is overkill. For now. Leave that in DSM plugin is not.

Adding DS to data plugin will pollute the data plugin, it is bad idea to me. Even if we have to do that means data plugin is design in a bad way which doesn't allow extension without touch its code.

Adding another plugin is overkill. For now. Leave that in DSM plugin is not.

Adding DS to data plugin will pollute the data plugin, it is bad idea to me. Even if we have to do that means data plugin is design in a bad way which doesn't allow extension without touch its code.

Let's also comment on: #2052

noCharger · 2022-08-02T03:19:53Z

src/plugins/data_source_management/server/saved_objects/data_source.ts

+  namespaceType: 'agnostic',
+  hidden: false,
+  management: {
+    icon: 'apps', // todo: pending ux #2034


Can we add full url in todo?

noCharger · 2022-08-02T03:30:29Z

src/plugins/data_source_management/server/saved_objects/data_source.ts

+      title: {
+        type: 'text',
+      },
+      type: {


Do we need index for both title and type?

type is removed in the pr

noCharger · 2022-08-02T03:31:17Z

src/plugins/data_source_management/server/plugin.ts

+  private readonly logger: Logger;
+
+  constructor(initializerContext: PluginInitializerContext) {
+    this.logger = initializerContext.logger.get();


Since we imported logger, can we add some logs for each stage?

A common one, is on start log the plugin has started.

this part has been removed

kavilla · 2022-08-02T09:28:00Z

Would be great to start building in the config I think at least we would want to default to enabled to false. For example,

OpenSearch-Dashboards/src/plugins/telemetry/server/index.ts

Line 43 in d7004dc

enabled: true,

zengyan-amazon

should the data source mapping be registered in the dataSource plugin ?

kristenTian · 2022-08-02T20:29:21Z

should the data source mapping be registered in the dataSource plugin ?

didn't plan to have dataSource plugin this phase, but let's have this there

Signed-off-by: Kristen Tian <[email protected]>

…ch-project#2037) Signed-off-by: Kristen Tian <[email protected]>

* Instantiate credential management plugin code structure (#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (#2017) (#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (#2064) (#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]>

* Instantiate credential management plugin code structure (opensearch-project#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (opensearch-project#2017) (opensearch-project#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (opensearch-project#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (opensearch-project#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (opensearch-project#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (opensearch-project#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (opensearch-project#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (opensearch-project#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (opensearch-project#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (opensearch-project#2064) (opensearch-project#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (opensearch-project#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (opensearch-project#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (opensearch-project#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (opensearch-project#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (opensearch-project#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (opensearch-project#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (opensearch-project#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (opensearch-project#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (opensearch-project#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (opensearch-project#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (opensearch-project#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (opensearch-project#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (opensearch-project#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (opensearch-project#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (opensearch-project#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (opensearch-project#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (opensearch-project#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (opensearch-project#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (opensearch-project#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (opensearch-project#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (opensearch-project#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (opensearch-project#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (opensearch-project#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (opensearch-project#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (opensearch-project#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (opensearch-project#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (opensearch-project#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (opensearch-project#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (opensearch-project#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (opensearch-project#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (opensearch-project#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (opensearch-project#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]>

* Instantiate credential management plugin code structure (opensearch-project#1996) Signed-off-by: Kristen Tian <[email protected]> * Data source inside stack management setup (opensearch-project#2017) (opensearch-project#2030) Signed-off-by: Kristen Tian <[email protected]> * enable CI for feature branch (opensearch-project#2010) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add empty data source plugin (opensearch-project#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian <[email protected]> * [MD] Add initial credential management CRUD pages (opensearch-project#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Register Data source to savedObject & Update license header (opensearch-project#2037) Signed-off-by: Kristen Tian <[email protected]> * Move credential saved object to data source plugin (opensearch-project#2062) Signed-off-by: Louis Chu <[email protected]> Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian <[email protected]> * breadcrumbfix for datasource management (opensearch-project#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 <[email protected]> * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 <[email protected]> * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 <[email protected]> * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian <[email protected]> * Fix breadcrumb on listing page and update saved object mapping (opensearch-project#2069) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add data source step into IndexPattern with Mock switch (opensearch-project#2064) (opensearch-project#2086) Signed-off-by: Kristen Tian <[email protected]> * Add delete button for credential detailed page (opensearch-project#2067) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Link datasource to indexpattern (opensearch-project#2118) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add encrypt/decrypt module on data source plugin (opensearch-project#2120) Signed-off-by: Louis Chu <[email protected]> 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian <[email protected]> * Integrate index pattern with new data client (opensearch-project#2146) Signed-off-by: Kristen Tian <[email protected]> * Add noAuth to dataSource attributes (opensearch-project#2154) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - creation & listing - UI only (opensearch-project#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Louis Chu <[email protected]> Apply dataSource plugin as dependent for cm and dsm plugins (opensearch-project#2150) Signed-off-by: Kristen Tian <[email protected]> * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian <[email protected]> * [MD] Add client management module and register `getClient()` to route handler context (opensearch-project#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Make step info in index pattern creation dynamic (opensearch-project#2164) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Refactor for credential listing page & add loading effect (opensearch-project#2142) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integration sequal - replace data client placeholders (opensearch-project#2167) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Integrate with crypto module to decrpt password (opensearch-project#2170) Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] noAuth integration, credential & endpoint validation (opensearch-project#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 <[email protected]> * Refactoring validation message Signed-off-by: mpabba3003 <[email protected]> * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor search strategy to conditionally user datasource client (opensearch-project#2171) Signed-off-by: Kristen Tian <[email protected]> * adding relation between credential selection and no auth checkbox (opensearch-project#2175) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update getIndicesViaSearch with datasource (opensearch-project#2176) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add null header to child client spawn (opensearch-project#2188) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD][IP]Update button position & Add UT & Add skip option (opensearch-project#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian <[email protected]> * Update button position Signed-off-by: Kristen Tian <[email protected]> * Add skip option to allow use default os data source Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Refactor layout and validate input fields for listing and create pages (opensearch-project#2202) Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable datasource link in saveObjectManagement (opensearch-project#2209) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update configure data source per UX input (opensearch-project#2235) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Enable data source audit log to file (opensearch-project#2215) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Refactor of credential editing page layout & refactor backend field validation method (opensearch-project#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Tweak fetch data back to original (opensearch-project#2238) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Revamped UX for data source management (opensearch-project#2239) * revamped UX for data source management Signed-off-by: mpabba3003 <[email protected]> * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Credentials security redesign (opensearch-project#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Add step data source UI test (opensearch-project#2264) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD]Improve datasource server side error handling (opensearch-project#2236) Signed-off-by: Su <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource management new API change integrations (opensearch-project#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 <[email protected]> * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian <[email protected]> * Delete credential management Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * [MD] Datasource Management - Create data source - Unit tests (opensearch-project#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 <[email protected]> * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Fix update data source & block update endpint (opensearch-project#2364) Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * MD datasource management-datasource table-UTs (opensearch-project#2350) Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * reafctor based on PR 2334 comments to merge to main (opensearch-project#2375) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * removing invalid urls as CI fails on link checker for inavlid urls in git (opensearch-project#2376) Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Fix DS snapshot test Signed-off-by: Kristen Tian <[email protected]> * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian <[email protected]> * Address comments Signed-off-by: Kristen Tian <[email protected]> * Remove unnessacry check Signed-off-by: Kristen Tian <[email protected]> * Remove not needed check Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Kristen Tian <[email protected]> Signed-off-by: Zhongnan Su <[email protected]> Signed-off-by: Louis Chu <[email protected]> Signed-off-by: Yibo Wang <[email protected]> Signed-off-by: mpabba3003 <[email protected]> Signed-off-by: Su <[email protected]> Co-authored-by: Louis Chu <[email protected]> Co-authored-by: Zhongnan Su <[email protected]> Co-authored-by: Yan Zeng <[email protected]> Co-authored-by: Manideep Pabba <[email protected]> Co-authored-by: Yibo Wang <[email protected]> Signed-off-by: Sergey V. Osipov <[email protected]>

kristenTian requested a review from a team as a code owner August 2, 2022 01:13

seraphjiang approved these changes Aug 2, 2022

View reviewed changes

zhongnansu reviewed Aug 2, 2022

View reviewed changes

noCharger reviewed Aug 2, 2022

View reviewed changes

zengyan-amazon requested changes Aug 2, 2022

View reviewed changes

zhongnansu added the multiple datasource multiple datasource project label Aug 2, 2022

kristenTian force-pushed the OSD branch 3 times, most recently from 5d8f3b3 to c60b185 Compare August 2, 2022 22:54

kristenTian requested a review from zengyan-amazon August 2, 2022 22:54

Register Data source to savedObject & Update license header

086936e

Signed-off-by: Kristen Tian <[email protected]>

kristenTian force-pushed the OSD branch from c60b185 to 086936e Compare August 3, 2022 01:24

seraphjiang changed the title ~~Add data source register to savedObject (#1729)~~ [MD] Add data source register to savedObject (#1729) Aug 3, 2022

seraphjiang assigned kristenTian Aug 3, 2022

seraphjiang linked an issue Aug 3, 2022 that may be closed by this pull request

[MD]Add data-source model to saved object #1729

Closed

zengyan-amazon approved these changes Aug 3, 2022

View reviewed changes

zengyan-amazon merged commit 25a97cb into opensearch-project:feature/multi-datasource Aug 3, 2022

kristenTian added a commit to kristenTian/OpenSearch-Dashboards that referenced this pull request Sep 12, 2022

Register Data source to savedObject & Update license header (opensear…

76a5643

…ch-project#2037) Signed-off-by: Kristen Tian <[email protected]>

kristenTian added a commit to kristenTian/OpenSearch-Dashboards that referenced this pull request Sep 14, 2022

Register Data source to savedObject & Update license header (opensear…

9744151

…ch-project#2037) Signed-off-by: Kristen Tian <[email protected]>

kristenTian added a commit to kristenTian/OpenSearch-Dashboards that referenced this pull request Sep 14, 2022

Register Data source to savedObject & Update license header (opensear…

fb29ee5

…ch-project#2037) Signed-off-by: Kristen Tian <[email protected]>

kristenTian added a commit to kristenTian/OpenSearch-Dashboards that referenced this pull request Sep 15, 2022

Register Data source to savedObject & Update license header (opensear…

0f12808

…ch-project#2037) Signed-off-by: Kristen Tian <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MD] Add data source register to savedObject (#1729) #2037

[MD] Add data source register to savedObject (#1729) #2037

kristenTian commented Aug 2, 2022

seraphjiang Aug 2, 2022

kristenTian Aug 2, 2022

seraphjiang Aug 2, 2022

noCharger Aug 2, 2022

codecov-commenter commented Aug 2, 2022 •

edited

Loading

zhongnansu Aug 2, 2022

kristenTian Aug 2, 2022

zhongnansu Aug 2, 2022

kristenTian Aug 2, 2022

noCharger Aug 2, 2022

seraphjiang Aug 2, 2022

noCharger Aug 2, 2022

seraphjiang Aug 2, 2022 •

edited

Loading

noCharger Aug 2, 2022

noCharger Aug 2, 2022

seraphjiang Aug 2, 2022

noCharger Aug 2, 2022

seraphjiang Aug 2, 2022 •

edited

Loading

noCharger Aug 2, 2022 •

edited

Loading

noCharger Aug 2, 2022

noCharger Aug 2, 2022

kristenTian Aug 3, 2022

noCharger Aug 2, 2022

kavilla Aug 2, 2022

kristenTian Aug 2, 2022

kavilla commented Aug 2, 2022

zengyan-amazon left a comment

kristenTian commented Aug 2, 2022 •

edited

Loading


		import { SavedObjectsType } from 'opensearch-dashboards/server';

		export const dataSource: SavedObjectsType = {

[MD] Add data source register to savedObject (#1729) #2037

[MD] Add data source register to savedObject (#1729) #2037

Conversation

kristenTian commented Aug 2, 2022

Description

Issues Resolved

Check List

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 2, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seraphjiang Aug 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seraphjiang Aug 2, 2022 • edited Loading

Choose a reason for hiding this comment

noCharger Aug 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kavilla commented Aug 2, 2022

zengyan-amazon left a comment

Choose a reason for hiding this comment

kristenTian commented Aug 2, 2022 • edited Loading

codecov-commenter commented Aug 2, 2022 •

edited

Loading

seraphjiang Aug 2, 2022 •

edited

Loading

seraphjiang Aug 2, 2022 •

edited

Loading

noCharger Aug 2, 2022 •

edited

Loading

kristenTian commented Aug 2, 2022 •

edited

Loading