Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up unit testing CI by updating maxWorkers setting #884

Merged
merged 2 commits into from
Aug 9, 2023

Conversation

Swiddis
Copy link
Collaborator

@Swiddis Swiddis commented Aug 9, 2023

Description

I did some poking around on why our tests might be so slow in CI, and discovered that by default Jest limits itself to 50% of CPU resources. GH Actions has 2 cores, so setting it manually may make our tests run considerably faster.

This is mostly just experimental to see what the difference is.

Issues Resolved

N/A

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <[email protected]>
@Swiddis Swiddis added the testing label Aug 9, 2023
@codecov
Copy link

codecov bot commented Aug 9, 2023

Codecov Report

Merging #884 (7cbebcb) into main (6ec0967) will not change coverage.
Report is 3 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #884   +/-   ##
=======================================
  Coverage   43.72%   43.72%           
=======================================
  Files         313      313           
  Lines       18629    18629           
  Branches     4534     4534           
=======================================
  Hits         8146     8146           
  Misses       9932     9932           
  Partials      551      551           
Flag Coverage Δ
dashboards-observability 43.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@Swiddis
Copy link
Collaborator Author

Swiddis commented Aug 9, 2023

Results (comparing with the current run of #857):

Before:
Ubuntu: 18m  2s   Windows: 25m 14s   Mac: 21m 43s
Changed to maxWorkers=2:
Ubuntu: 11m 11s   Windows: 16m 38s   Mac: 23m 10s 
Changed to maxWorkers=100%:
Ubuntu: 12m 49s   Windows: 15m 59s   Mac: 20m 58s

It is faster, but not dramatically so, and since Mac doesn't see a speedup it won't really affect how quickly PRs get complete feedback. It might still be worth it just to have a faster result when making pushes.

Update 1: A quick check of the Mac logs shows that in both the old and new config, Mac is spending about 40 minutes of CPU time on tests. I suspect it may have already been splitting the workload, so you would expect to see no performance improvement. I'll also test with 100% to see if it happens to have 4 cores on-hand.

Update 2: 100% performs about the same for all 3, looks like Mac is just running slow and doesn't really see any speedups from this. Conclusion from before is the same, doesn't significantly change time for full CI but does give us a 5-10 minute shorter feedback loop on Unix and Windows.

Signed-off-by: Simeon Widdis <[email protected]>
@Swiddis Swiddis changed the title Set max CI test workers to 2 Speed up unit testing CI by updating maxWorkers setting Aug 9, 2023
@YANG-DB YANG-DB merged commit a39265a into opensearch-project:main Aug 9, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 9, 2023
* Set max CI workers to 2

Signed-off-by: Simeon Widdis <[email protected]>

* Try setting to 100% of workers

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
(cherry picked from commit a39265a)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Swiddis pushed a commit that referenced this pull request Aug 9, 2023
* Set max CI workers to 2



* Try setting to 100% of workers



---------


(cherry picked from commit a39265a)

Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
pjfitzgibbons pushed a commit that referenced this pull request Aug 10, 2023
* Set max CI workers to 2

Signed-off-by: Simeon Widdis <[email protected]>

* Try setting to 100% of workers

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
derek-ho added a commit that referenced this pull request Sep 6, 2023
* Bugfix filter not appearing, and loading on click

Signed-off-by: TackAdam <[email protected]>

* Fixed applications not working with change

Signed-off-by: TackAdam <[email protected]>

* Fixed refresh keeping the old pattern stored in the query

Signed-off-by: TackAdam <[email protected]>

* Add integration content

Signed-off-by: Simeon Widdis <[email protected]>

* Copy updated types

Signed-off-by: Simeon Widdis <[email protected]>

* Patterns working in Applications and Logs

Signed-off-by: TackAdam <[email protected]>

* Update mapping files to match current catalog versions

Signed-off-by: Simeon Widdis <[email protected]>

* Removed tempQuery from log patterns

Signed-off-by: TackAdam <[email protected]>

* Copy aws_elb integration from @danieldong51

Signed-off-by: Simeon Widdis <[email protected]>

* Remove nginx logo from gallery

Signed-off-by: Simeon Widdis <[email protected]>

* Remove extra logo from nginx

Signed-off-by: Simeon Widdis <[email protected]>

* Cypress test for panels working, bugfix on toast message

Signed-off-by: TackAdam <[email protected]>

* Removed unused function

Signed-off-by: TackAdam <[email protected]>

* Copy path sanitization from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* add docker-compose support  (#575)

* add capability to build the plugin and add it into the docker of the Opensearch-Dashboard

update the developer-guid to reflect the correct bootstraping process
add docker-compose support and tutorial

Signed-off-by: YANGDB <[email protected]>

* Update Dockerfile

* Update Dockerfile and adding .env

* Update Using-Docker.md

Co-authored-by: Joshua Li <[email protected]>
Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
Co-authored-by: Joshua Li <[email protected]>

* Merge Kibana backend from osints/dev into main (#565)

* Merge in kibana backend from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Add integration type to .kibana from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Re-add license header

Signed-off-by: Simeon Widdis <[email protected]>

* Fix integrations type

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Fixing bug with saved visualization querys from logs

Signed-off-by: TackAdam <[email protected]>

* code clean up

Signed-off-by: Eric Wei <[email protected]>

* update snapshots

Signed-off-by: Eric Wei <[email protected]>

* remove unused utils, color theme, and editor components

Signed-off-by: Eric Wei <[email protected]>

* add to index file

Signed-off-by: Eric Wei <[email protected]>

* Feature/prerelease code cleanup (#589)

* refactor naming

Signed-off-by: Eric Wei <[email protected]>

* refactor vis workspace

Signed-off-by: Eric Wei <[email protected]>

* config panel refactoring

Signed-off-by: Eric Wei <[email protected]>

* refactor workspace

Signed-off-by: Eric Wei <[email protected]>

---------

Signed-off-by: Eric Wei <[email protected]>

* Replace metrics slider with ResizableContainer (#602)

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Fixing cypress testing for apps

Signed-off-by: TackAdam <[email protected]>

* Fix availability not being displayed in panel

Signed-off-by: TackAdam <[email protected]>

* Fixed config data and layout

Signed-off-by: TackAdam <[email protected]>

* changed fix implementation to not break unit test

Signed-off-by: TackAdam <[email protected]>

* uncommented lines after bugfix

Signed-off-by: TackAdam <[email protected]>

* Delete missed comments

Signed-off-by: TackAdam <[email protected]>

* Prevent empty string being parsed

Signed-off-by: TackAdam <[email protected]>

* Pull changes from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Copy frontend from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Update nginx dashboard to remove errors

Signed-off-by: Simeon Widdis <[email protected]>

* address pr comments

Signed-off-by: Derek Ho <[email protected]>

* address pr comments

Signed-off-by: Derek Ho <[email protected]>

* Upgrade tough-cookie and semver (#619) (#627)

* Upgrade tough-cookie to fix CVE-2023-26136



* Add semver resolution to patch CVE-2022-25883



---------

Signed-off-by: Rupal Mahajan <[email protected]>
Co-authored-by: Rupal Mahajan <[email protected]>

* Merge frontend from osints/dev (#636)

* Remove unused NDJson parsing code

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused instance store

Signed-off-by: Simeon Widdis <[email protected]>

* Move repository logic to own class

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for IntegrationsRepository

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for kibana backend

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Clear repository between runs in testing

Signed-off-by: Simeon Widdis <[email protected]>

* fix tests and import

Signed-off-by: Derek Ho <[email protected]>

* add sanity test cypress test

Signed-off-by: Derek Ho <[email protected]>

* clean up code

Signed-off-by: Derek Ho <[email protected]>

* Add tests for integration builder

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Swap jest test to it

Signed-off-by: Simeon Widdis <[email protected]>

* {
  "attributes": {
    "description": "Nginx dashboard with basic Observability on access / error logs",
    "hits": 0,
    "kibanaSavedObjectMeta": {
      "searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[]}"
    },
    "optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
    "panelsJSON": "[{\"version\":\"2.5.0\",\"gridData\":{\"h\":8,\"i\":\"1f31e50b-06e3-41e6-972e-e4e5fe1a9872\",\"w\":48,\"x\":0,\"y\":0},\"panelIndex\":\"1f31e50b-06e3-41e6-972e-e4e5fe1a9872\",\"embeddableConfig\":{},\"panelRefName\":\"panel_0\"},{\"version\":\"2.5.0\",\"gridData\":{\"h\":9,\"i\":\"d91a8da4-b34b-470a-aca6-9c76b47cd6fb\",\"w\":24,\"x\":0,\"y\":8},\"panelIndex\":\"d91a8da4-b34b-470a-aca6-9c76b47cd6fb\",\"embeddableConfig\":{},\"panelRefName\":\"panel_1\"},{\"version\":\"2.5.0\",\"gridData\":{\"h\":15,\"i\":\"27149e5a-3a77-4f3c-800e-8a160c3765f4\",\"w\":24,\"x\":24,\"y\":8},\"panelIndex\":\"27149e5a-3a77-4f3c-800e-8a160c3765f4\",\"embeddableConfig\":{},\"panelRefName\":\"panel_2\"},{\"version\":\"2.5.0\",\"gridData\":{\"x\":0,\"y\":17,\"w\":24,\"h\":15,\"i\":\"4d8c2aa7-159c-4a1a-80ff-00a9299056ce\"},\"panelIndex\":\"4d8c2aa7-159c-4a1a-80ff-00a9299056ce\",\"embeddableConfig\":{},\"panelRefName\":\"panel_3\"},{\"version\":\"2.5.0\",\"gridData\":{\"x\":24,\"y\":23,\"w\":24,\"h\":15,\"i\":\"800b7f19-f50c-417f-8987-21b930531cbe\"},\"panelIndex\":\"800b7f19-f50c-417f-8987-21b930531cbe\",\"embeddableConfig\":{},\"panelRefName\":\"panel_4\"}]",
    "timeRestore": false,
    "title": "[NGINX Core Logs 1.0] Overview",
    "version": 1
  },
  "id": "96847220-5261-44d0-89b4-65f3a659f13a",
  "migrationVersion": {
    "dashboard": "7.9.3"
  },
  "references": [
    {
      "id": "3b49a65d-54d8-483d-a8f0-3d7c855e1ecf",
      "name": "panel_0",
      "type": "visualization"
    },
    {
      "id": "865e577b-634b-4a65-b9d6-7e324c395d18",
      "name": "panel_1",
      "type": "visualization"
    },
    {
      "id": "dc1803f0-b478-11ed-9063-ebe46f9ac203",
      "name": "panel_2",
      "type": "visualization"
    },
    {
      "id": "99acc580-b47a-11ed-9063-ebe46f9ac203",
      "name": "panel_3",
      "type": "visualization"
    },
    {
      "id": "01ea64d0-b62f-11ed-a677-43d7aa86763b",
      "name": "panel_4",
      "type": "visualization"
    }
  ],
  "type": "dashboard",
  "updated_at": "2023-02-26T23:44:09.855Z",
  "version": "WzczLDdd"
}

Signed-off-by: Derek Ho <[email protected]>

* Stub out sample integration data for front-end

Signed-off-by: Simeon Widdis <[email protected]>

* Use more realistic integration sample

Signed-off-by: Simeon Widdis <[email protected]>

* Add full traversal to fields list

Signed-off-by: Simeon Widdis <[email protected]>

* Add license to integration template

Signed-off-by: Simeon Widdis <[email protected]>

* add some unit testing framework

Signed-off-by: Derek Ho <[email protected]>

* fix up the testing name

Signed-off-by: Derek Ho <[email protected]>

* fix up table view and add a test

Signed-off-by: Derek Ho <[email protected]>

* add an added integration table test using dummy data

Signed-off-by: Derek Ho <[email protected]>

* add integration specific types and clean up code

Signed-off-by: Derek Ho <[email protected]>

* clean up code and use integration specific types

Signed-off-by: Derek Ho <[email protected]>

* Route integrations page to actual repository

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Remove unused import

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Use loading spinner

Signed-off-by: Simeon Widdis <[email protected]>

* add some changes to UI according to figma

Signed-off-by: Derek Ho <[email protected]>

* add license headers and hook up get to specific instance for added page

Signed-off-by: Derek Ho <[email protected]>

* add screenshots and fix up mappings and breadcrumbs

Signed-off-by: Derek Ho <[email protected]>

* Add failing test case for empty object

Signed-off-by: Simeon Widdis <[email protected]>

* Make template test pass

Signed-off-by: Simeon Widdis <[email protected]>

* use flyout according to figma and connect flyout input to backend creation

Signed-off-by: Derek Ho <[email protected]>

* implement delete

Signed-off-by: Derek Ho <[email protected]>

* fix up tests for changes

Signed-off-by: Derek Ho <[email protected]>

* get added page to working order

Signed-off-by: Derek Ho <[email protected]>

* Add template validator via Ajv

Signed-off-by: Simeon Widdis <[email protected]>

* Fix builder tests for new validation

Signed-off-by: Simeon Widdis <[email protected]>

* add unit test coverage

Signed-off-by: Derek Ho <[email protected]>

* update page and tests to remove added by for integration instances

Signed-off-by: Derek Ho <[email protected]>

* deleting an instance also deletes its children saved objects

Signed-off-by: Derek Ho <[email protected]>

* Add integration instance validation

Signed-off-by: Simeon Widdis <[email protected]>

* update components and snapshots with data-test-subj for cypress tests and add more comprehensive cypress test suite

Signed-off-by: Derek Ho <[email protected]>

* remove unecessary console logging

Signed-off-by: Derek Ho <[email protected]>

* Hotfix: Minor code cleanup

Signed-off-by: Simeon Widdis <[email protected]>

* dynamically handle id generation to allow for multiple instances of the same template to be created

Signed-off-by: Derek Ho <[email protected]>

* Update integration with fixed panels

Signed-off-by: Simeon Widdis <[email protected]>

* Apply minor behavior tweaks

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor repository for new format

Signed-off-by: Simeon Widdis <[email protected]>

* fix empty assets bug

Signed-off-by: Derek Ho <[email protected]>

* Write basic FS latest integration queries

Signed-off-by: Simeon Widdis <[email protected]>

* Update integration template type and validator

Signed-off-by: Simeon Widdis <[email protected]>

* Add template retrieval to repository fs

Signed-off-by: Simeon Widdis <[email protected]>

* Add copyright notice to repository_fs

Signed-off-by: Simeon Widdis <[email protected]>

* fix escape

Signed-off-by: Derek Ho <[email protected]>

* revert file

Signed-off-by: Derek Ho <[email protected]>

* fix test

Signed-off-by: Derek Ho <[email protected]>

* Stub composite repository class

Signed-off-by: Simeon Widdis <[email protected]>

* Apply minor condition refactor

Signed-off-by: Simeon Widdis <[email protected]>

* Fill in basic implementation for integration class

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for integration class

Signed-off-by: Simeon Widdis <[email protected]>

* Fix type errors on repository tests

Signed-off-by: Simeon Widdis <[email protected]>

* Patch: Handle files with invalid version formats

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor displayAssets to assets

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic getAssets implementation

Signed-off-by: Simeon Widdis <[email protected]>

* Add handling for invalid config error case

Signed-off-by: Simeon Widdis <[email protected]>

* Add handling for failed saved object loading

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic getStatic method

Signed-off-by: Simeon Widdis <[email protected]>

* Fix handling for missing statics

Signed-off-by: Simeon Widdis <[email protected]>

* Remove outdated repository classes

Signed-off-by: Simeon Widdis <[email protected]>

* Add docs for the integration class

Signed-off-by: Simeon Widdis <[email protected]>

* Hook up router to new static loading

Signed-off-by: Simeon Widdis <[email protected]>

* Clarify content-type docs

Signed-off-by: Simeon Widdis <[email protected]>

* Fix details page crashes due to outdated schema

Signed-off-by: Simeon Widdis <[email protected]>

* Infer mime type for statics

Signed-off-by: Simeon Widdis <[email protected]>

* Fix screenshot loading on integrations details page

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Remove mimetypes from nginx 1.0.0

Signed-off-by: Simeon Widdis <[email protected]>

* Add getSchemas method with tests

Signed-off-by: Simeon Widdis <[email protected]>

* Make minor modifications to integration typing

Signed-off-by: Simeon Widdis <[email protected]>

* Add integration deep check

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor integration builder

Signed-off-by: Simeon Widdis <[email protected]>

* Fix remap ID method

Signed-off-by: Simeon Widdis <[email protected]>

* Fix loading process

Signed-off-by: Simeon Widdis <[email protected]>

* Refactor tests for new integrations backend

Signed-off-by: Simeon Widdis <[email protected]>

* Fix type errors in test suite

Signed-off-by: Simeon Widdis <[email protected]>

* hook up datasource creation from flyout to backend

Signed-off-by: Derek Ho <[email protected]>

* remove nullable

Signed-off-by: Derek Ho <[email protected]>

* Add schema route to router

Signed-off-by: Simeon Widdis <[email protected]>

* Fix frontend request routing for new API

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Fix added integration errors

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Add an actual screenshot

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Remove unused assets folder

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Use Oui images

Signed-off-by: Simeon Widdis <[email protected]>

* Make integration instance deletion more robust

Signed-off-by: Simeon Widdis <[email protected]>

* Remove asset map (again

Signed-off-by: Simeon Widdis <[email protected]>

* Fix remap data source

Signed-off-by: Simeon Widdis <[email protected]>

* Correct deletion for actual 404 behavior

Signed-off-by: Simeon Widdis <[email protected]>

* Add display name field to integrations

Signed-off-by: Simeon Widdis <[email protected]>

* Apply display name through frontend

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Use display name on table view

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused code in flyout

Signed-off-by: Simeon Widdis <[email protected]>

* Stub mapping validation on frontend

Signed-off-by: Simeon Widdis <[email protected]>

* Add error message for failed validation result

Signed-off-by: Simeon Widdis <[email protected]>

* Fix request to proxy

Signed-off-by: Simeon Widdis <[email protected]>

* Add more detailed handling for missing schemas

Signed-off-by: Simeon Widdis <[email protected]>

* Apply integration type in validation

Signed-off-by: Simeon Widdis <[email protected]>

* Add full schema validation

Signed-off-by: Simeon Widdis <[email protected]>

* Add unit tests for mapping validation

Signed-off-by: Simeon Widdis <[email protected]>

* Fix type checking in unit tests

Signed-off-by: Simeon Widdis <[email protected]>

* create dataSource UI option

Signed-off-by: Derek Ho <[email protected]>

* Hotfix: Update route for listing added integrations

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Fix component tests

Signed-off-by: Simeon Widdis <[email protected]>

* Add basic status querying

Signed-off-by: Simeon Widdis <[email protected]>

* Add unit tests for availability queries

Signed-off-by: Simeon Widdis <[email protected]>

* create ss4o index from ui

Signed-off-by: Derek Ho <[email protected]>

* code cleanup

Signed-off-by: Derek Ho <[email protected]>

* use user input

Signed-off-by: Derek Ho <[email protected]>

* Hotfix: Sync package versions with osd

Signed-off-by: Simeon Widdis <[email protected]>

* Fix path-dependent tests on windows

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Repair template naming on index autocreation

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Allow wildcards in index pattern specification

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Remove redundant integrationType field

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Remove unused schema files

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Add sample data

Signed-off-by: Simeon Widdis <[email protected]>

* move integrations under core (pending ux)

Signed-off-by: Derek Ho <[email protected]>

* Fix handling for templates on create

Signed-off-by: Simeon Widdis <[email protected]>

* Merge in kibana backend from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Add integration type to .kibana from osints/dev

Signed-off-by: Simeon Widdis <[email protected]>

* Re-add license header

Signed-off-by: Simeon Widdis <[email protected]>

* hook up toast

Signed-off-by: Derek Ho <[email protected]>

* Update integration-instance type

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Fix types again

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Fix cypress

Signed-off-by: Simeon Widdis <[email protected]>

* Add sample data endpoint

Signed-off-by: Simeon Widdis <[email protected]>

* Add license checking to eslint

Signed-off-by: Simeon Widdis <[email protected]>

* Created elb integration

Signed-off-by: Daniel Dong <[email protected]>

* successfully hook up toast and block until done with for loop

Signed-off-by: Derek Ho <[email protected]>

* fix and fix test

Signed-off-by: Derek Ho <[email protected]>

* Added screenshot of elb integration`

Signed-off-by: Daniel Dong <[email protected]>

* Updated aws_elb descriptions and logos

Signed-off-by: Daniel Dong <[email protected]>

* Deleted .DS_Store

Signed-off-by: Daniel Dong <[email protected]>

* Added container in components

Signed-off-by: Daniel Dong <[email protected]>

* Merged markers

Signed-off-by: Daniel Dong <[email protected]>

* Hotfix: Rename installed integrations to added integrations

Signed-off-by: Simeon Widdis <[email protected]>

* Hotfix: Revert Nginx sample prettifying

Signed-off-by: Simeon Widdis <[email protected]>

* remove console.log

Signed-off-by: Derek Ho <[email protected]>

* updates cypress test to test create data source flow with toast

Signed-off-by: Derek Ho <[email protected]>

* update snapshot based on cypress changes

Signed-off-by: Derek Ho <[email protected]>

* get rid of extra line

Signed-off-by: Derek Ho <[email protected]>

* update according to figma

Signed-off-by: Derek Ho <[email protected]>

* Fix integrations type

Signed-off-by: Simeon Widdis <[email protected]>

* Sanitize file path for getting integration statics

Signed-off-by: Simeon Widdis <[email protected]>

* Update tests for merge

Signed-off-by: Simeon Widdis <[email protected]>

* make some UX improvements

Signed-off-by: Derek Ho <[email protected]>

* Add basic sample data loading

Signed-off-by: Simeon Widdis <[email protected]>

* moves integration on left nav, removes unused create flow

Signed-off-by: Derek Ho <[email protected]>

* hook up sample data to a standalone button

Signed-off-by: Derek Ho <[email protected]>

* fix tests

Signed-off-by: Derek Ho <[email protected]>

* implement length validation and update tests accordingly

Signed-off-by: Derek Ho <[email protected]>

* disable buttons while adding

Signed-off-by: Derek Ho <[email protected]>

* fix breadcrumbs and tests

Signed-off-by: Derek Ho <[email protected]>

* update according to figma

Signed-off-by: Derek Ho <[email protected]>

* maek some changes according to figma

Signed-off-by: Derek Ho <[email protected]>

* implement tabbing for both added/available and fields/assets

Signed-off-by: Derek Ho <[email protected]>

* remove status on template, change status to badge on instance and hook up type filtering for added

Signed-off-by: Derek Ho <[email protected]>

* remove actions

Signed-off-by: Derek Ho <[email protected]>

* convert to badges

Signed-off-by: Derek Ho <[email protected]>

* fix tests

Signed-off-by: Derek Ho <[email protected]>

* remove columns from table

Signed-off-by: Derek Ho <[email protected]>

* add linking to all assets

Signed-off-by: Derek Ho <[email protected]>

* fix unit and integration tests

Signed-off-by: Derek Ho <[email protected]>

* final fix up

Signed-off-by: Derek Ho <[email protected]>

* fix filtering

Signed-off-by: Derek Ho <[email protected]>

* final fix up

Signed-off-by: Derek Ho <[email protected]>

* fix tests

Signed-off-by: Derek Ho <[email protected]>

* add cateogry filter and fix tests

Signed-off-by: Derek Ho <[email protected]>

* fix cypress and unit tests

Signed-off-by: Derek Ho <[email protected]>

* Fix broken index template naming in sample creation

Signed-off-by: Simeon Widdis <[email protected]>

* Add priority to logs mapping and fix index pattern

Signed-off-by: Simeon Widdis <[email protected]>

* fix various UI and add validation

Signed-off-by: Derek Ho <[email protected]>

* add request metrics

Signed-off-by: Derek Ho <[email protected]>

* fix tests, remove console log

Signed-off-by: Derek Ho <[email protected]>

* add metrics to links and final UX fix up, test fixes

Signed-off-by: Derek Ho <[email protected]>

* add back installed

Signed-off-by: Derek Ho <[email protected]>

* Fix broken nginx panels

Signed-off-by: Simeon Widdis <[email protected]>

* address pr comments

Signed-off-by: Derek Ho <[email protected]>

* address pr comments

Signed-off-by: Derek Ho <[email protected]>

* Hotfix: Fix constants

Signed-off-by: Simeon Widdis <[email protected]>

* Add learn more link to index field

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused AWS ELB integration

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused ELB constants

Signed-off-by: Simeon Widdis <[email protected]>

* Remove redundant EUI spacing

Signed-off-by: Simeon Widdis <[email protected]>

* Fix snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* fix relative path

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: Derek Ho <[email protected]>
Signed-off-by: Derek Ho <[email protected]>
Signed-off-by: Daniel Dong <[email protected]>
Co-authored-by: Derek Ho <[email protected]>
Co-authored-by: Derek Ho <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* Release Notes 2.9.0

* Release Notes 2.9.0

---------

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Update Release Notes 2.9 w/ Documentation section (#652)

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Remove unused files (#662)

* remove unused files

Signed-off-by: Eric Wei <[email protected]>

* missing snapshots

Signed-off-by: Eric Wei <[email protected]>

---------

Signed-off-by: Eric Wei <[email protected]>

* Cypress ci integration (#443)

* add cypress parallel

Signed-off-by: Eric Wei <[email protected]>

* cypress workflow

Signed-off-by: Eric Wei <[email protected]>

* fix plugin crash issue

Signed-off-by: Eric Wei <[email protected]>

---------

Signed-off-by: Eric Wei <[email protected]>

* Increment version to 3.0.0.0 (#659)

Signed-off-by: opensearch-ci-bot <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>

* Remade all cypress test for visualizations charts

Signed-off-by: TackAdam <[email protected]>

* Remove job scheduler from cypress CI (#681)

* remove unused files

Signed-off-by: Eric Wei <[email protected]>

* missing snapshots

Signed-off-by: Eric Wei <[email protected]>

* remove job scheduler from CI artifact downloading

Signed-off-by: Eric Wei <[email protected]>

---------

Signed-off-by: Eric Wei <[email protected]>

* update readme to reflect recent changes (#691)

Signed-off-by: YANGDB <[email protected]>

* Add Integration request github issue template (#688)

Signed-off-by: YANGDB <[email protected]>
Co-authored-by: Joshua Li <[email protected]>

* adds back cypress watch and reload (#693)

Signed-off-by: Derek Ho <[email protected]>

* Fixes for Derek's comments

Signed-off-by: TackAdam <[email protected]>

* Added sample data load, moved shared constm removed waits

Signed-off-by: TackAdam <[email protected]>

* Code hygiene moved helpers out of constants

Signed-off-by: TackAdam <[email protected]>

* fix name change bug and modify test to test behavior (#724)

Signed-off-by: Derek Ho <[email protected]>

* Search bar fixes (#727)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* search bar fixes

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* Add AWS VPC Flow Log integration (#698)

* Add AWS VPC Flow Log integration

Signed-off-by: Haidong <[email protected]>

* init assets

Signed-off-by: Haidong <[email protected]>

* add more visualizations

Signed-off-by: Haidong <[email protected]>

* add readme for assets and aws vpc flow logs

Signed-off-by: Haidong <[email protected]>

---------

Signed-off-by: Haidong <[email protected]>
Co-authored-by: Haidong <[email protected]>

* Update snapshots (#734)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* update snapshot from previous PR

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* Fix vpc (#742)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* fix vpc flow logs with Simeon

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* Bump word-wrap from 1.2.3 to 1.2.4 (#723)

Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update vpc readme & references (#740)

* update the vpc flow log readme with a dashboard screenshot and update the sourceUrl to point to the correct place

Signed-off-by: YANGDB <[email protected]>

* update the vpc flow log readme with a dashboard screenshot and update the sourceUrl to point to the correct place

Signed-off-by: YANGDB <[email protected]>

* update author

Signed-off-by: Derek Ho <[email protected]>

* {
  "attributes": {
    "description": "Nginx dashboard with basic Observability on access / error logs",
    "hits": 0,
    "kibanaSavedObjectMeta": {
      "searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[]}"
    },
    "optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
    "panelsJSON": "[{\"version\":\"2.5.0\",\"gridData\":{\"h\":8,\"i\":\"1f31e50b-06e3-41e6-972e-e4e5fe1a9872\",\"w\":48,\"x\":0,\"y\":0},\"panelIndex\":\"1f31e50b-06e3-41e6-972e-e4e5fe1a9872\",\"embeddableConfig\":{},\"panelRefName\":\"panel_0\"},{\"version\":\"2.5.0\",\"gridData\":{\"h\":9,\"i\":\"d91a8da4-b34b-470a-aca6-9c76b47cd6fb\",\"w\":24,\"x\":0,\"y\":8},\"panelIndex\":\"d91a8da4-b34b-470a-aca6-9c76b47cd6fb\",\"embeddableConfig\":{},\"panelRefName\":\"panel_1\"},{\"version\":\"2.5.0\",\"gridData\":{\"h\":15,\"i\":\"27149e5a-3a77-4f3c-800e-8a160c3765f4\",\"w\":24,\"x\":24,\"y\":8},\"panelIndex\":\"27149e5a-3a77-4f3c-800e-8a160c3765f4\",\"embeddableConfig\":{},\"panelRefName\":\"panel_2\"},{\"version\":\"2.5.0\",\"gridData\":{\"x\":0,\"y\":17,\"w\":24,\"h\":15,\"i\":\"4d8c2aa7-159c-4a1a-80ff-00a9299056ce\"},\"panelIndex\":\"4d8c2aa7-159c-4a1a-80ff-00a9299056ce\",\"embeddableConfig\":{},\"panelRefName\":\"panel_3\"},{\"version\":\"2.5.0\",\"gridData\":{\"x\":24,\"y\":23,\"w\":24,\"h\":15,\"i\":\"800b7f19-f50c-417f-8987-21b930531cbe\"},\"panelIndex\":\"800b7f19-f50c-417f-8987-21b930531cbe\",\"embeddableConfig\":{},\"panelRefName\":\"panel_4\"}]",
    "timeRestore": false,
    "title": "[NGINX Core Logs 1.0] Overview",
    "version": 1
  },
  "id": "96847220-5261-44d0-89b4-65f3a659f13a",
  "migrationVersion": {
    "dashboard": "7.9.3"
  },
  "references": [
    {
      "id": "3b49a65d-54d8-483d-a8f0-3d7c855e1ecf",
      "name": "panel_0",
      "type": "visualization"
    },
    {
      "id": "865e577b-634b-4a65-b9d6-7e324c395d18",
      "name": "panel_1",
      "type": "visualization"
    },
    {
      "id": "dc1803f0-b478-11ed-9063-ebe46f9ac203",
      "name": "panel_2",
      "type": "visualization"
    },
    {
      "id": "99acc580-b47a-11ed-9063-ebe46f9ac203",
      "name": "panel_3",
      "type": "visualization"
    },
    {
      "id": "01ea64d0-b62f-11ed-a677-43d7aa86763b",
      "name": "panel_4",
      "type": "visualization"
    }
  ],
  "type": "dashboard",
  "updated_at": "2023-02-26T23:44:09.855Z",
  "version": "WzczLDdd"
}

Signed-off-by: Derek Ho <[email protected]>

* fix nginx references

Signed-off-by: Derek Ho <[email protected]>

* append minor changes

Signed-off-by: YANGDB <[email protected]>

* change to info directory

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>
Signed-off-by: Derek Ho <[email protected]>
Co-authored-by: Derek Ho <[email protected]>

* Fix snapshots from upstream OSD/OUI changes (#755)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* udpate snapshots from upstream

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* Remove link field from Nginx (#761)

* Remove redundant link

Signed-off-by: Simeon Widdis <[email protected]>

* Add local repository testing

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* simplify docker-compose.yml (#763)

* simplify docker-compose.yml

Signed-off-by: YANGDB <[email protected]>

* update docs regarding security

Signed-off-by: YANGDB <[email protected]>

* update docs security doc and restrictions

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* fix integration json loading parser so that it will trim unneeded empty spaces from the ndjson file and match the existing behavior of the savedObject import dashboard that works even when empty spaces appear in the ndjson file (#758)

Signed-off-by: YANGDB <[email protected]>

* fix docker-compose.yml dashboard issue with OPENSEARCH_HOSTS (#783)

Signed-off-by: YANGDB <[email protected]>

* Fail backport (#778)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* update fail backport

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* Add AWS ELB Integration (#772)

* Created aws_elb integration

Signed-off-by: Daniel Dong <[email protected]>

* Modified schemas

Signed-off-by: Daniel Dong <[email protected]>

* Modified user_agent to be an object in http

Signed-off-by: Daniel Dong <[email protected]>

* Added logo and dashboard

Signed-off-by: Daniel Dong <[email protected]>

* Updated static

Signed-off-by: Daniel Dong <[email protected]>

* Added more sample data

Signed-off-by: Daniel Dong <[email protected]>

* Added more sample data

Signed-off-by: Daniel Dong <[email protected]>

* Updated sourceUrl to be correct page

Signed-off-by: Daniel Dong <[email protected]>

* Updated dashboard image

Signed-off-by: Daniel Dong <[email protected]>

* Added image to README.md

Signed-off-by: Dan Dong <[email protected]>

* Updated matched_rule_priority to be int

Signed-off-by: Daniel Dong <[email protected]>

* Updated README.md with ingestion pipeline info

Signed-off-by: Dan Dong <[email protected]>

* Moved ingestion pipeline to new .md

Signed-off-by: Daniel Dong <[email protected]>

* Moved Ingestion material out of README.md

Signed-off-by: Daniel Dong <[email protected]>

---------

Signed-off-by: Daniel Dong <[email protected]>
Signed-off-by: Dan Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* Feature/metrics sidepanel update (#749)

* Metric Sidebar Refactor
- Move SearchBar into Sidebar
- show icons per-datasource
- compress metric-names (prometheus == "p..")

---------

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Add Apache Integration (#754)

* Modified map visualization to be region map instead of tiles

Signed-off-by: Daniel Dong <[email protected]>

* Added user agent mapping file

Signed-off-by: Daniel Dong <[email protected]>

* Added event.dataset field

Signed-off-by: Daniel Dong <[email protected]>

* removed event.dataset

Signed-off-by: Daniel Dong <[email protected]>

* Updated author

Signed-off-by: Daniel Dong <[email protected]>

* Added README.md

Signed-off-by: Daniel Dong <[email protected]>

* Added dashboard.png and logo.png

Signed-off-by: Daniel Dong <[email protected]>

* Updated search queries object in assets and another sample log

Signed-off-by: Daniel Dong <[email protected]>

* Updated geo fields in communication mappings

Signed-off-by: Daniel Dong <[email protected]>

* Fixed datetime format

Signed-off-by: Daniel Dong <[email protected]>

* Added geo to destination

Signed-off-by: Daniel Dong <[email protected]>

* Updated error search query object

Signed-off-by: Daniel Dong <[email protected]>

* Created Geo object in communications schema

Signed-off-by: Daniel Dong <[email protected]>

* Changed user_agent to be a subfield of http

Signed-off-by: Daniel Dong <[email protected]>

* Changed structure of logo and gallery

Signed-off-by: Daniel Dong <[email protected]>

* Removed user agent from logs component

Signed-off-by: Daniel Dong <[email protected]>

* Changed user_agent and os fields to be subobjects

Signed-off-by: Daniel Dong <[email protected]>

* Fixed geo object

Signed-off-by: Daniel Dong <[email protected]>

* Remoted user_agent from components

Signed-off-by: Daniel Dong <[email protected]>

* Added geo data to error log

Signed-off-by: Daniel Dong <[email protected]>

* Updated name

Signed-off-by: Daniel Dong <[email protected]>

* Added ingestion pipeline info

Signed-off-by: Daniel Dong <[email protected]>

* Updated README.md to remove ingestion pipeline

Signed-off-by: Dan Dong <[email protected]>

* Created INGESTION.md

Signed-off-by: Dan Dong <[email protected]>

* Updated INGESTION.md

Signed-off-by: Dan Dong <[email protected]>

---------

Signed-off-by: Daniel Dong <[email protected]>
Signed-off-by: Dan Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* Fix broken Nginx Integration data volume panel (#787)

* Fix broken nginx data volume panel

Signed-off-by: Simeon Widdis <[email protected]>

* Re-add sample data

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Updated link in sourceURL (#811)

Signed-off-by: Daniel Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* Make integration category and template lists dynamic (#782)

* Make category list dynamic

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused category constant

Signed-off-by: Simeon Widdis <[email protected]>

* Remove hardcoded integration template list

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Re-apply fix for VPC type (#809)

* Fix VPC type

Signed-off-by: Simeon Widdis <[email protected]>

* Add a test case

Signed-off-by: Simeon Widdis <[email protected]>

* Re-apply link fix and update test

Signed-off-by: Simeon Widdis <[email protected]>

* Revert "Re-apply link fix and update test"

This reverts commit f2ec20ddee7b4bc24deb570673a031e91ceb63aa.

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Integrations base path fix (#713)

* Fix behavior when base path is added

Signed-off-by: Simeon Widdis <[email protected]>

* Fix handling of base paths for the proxy

Signed-off-by: Simeon Widdis <[email protected]>

* Fix add integration flyout

Signed-off-by: Simeon Widdis <[email protected]>

* Remove fetch usages from proxy call

Signed-off-by: Simeon Widdis <[email protected]>

* Fix request query building

Signed-off-by: Simeon Widdis <[email protected]>

* Switch from props.http to coreRefs.http

Signed-off-by: Simeon Widdis <[email protected]>

* Revert "Switch from props.http to coreRefs.http"

This reverts commit f66db8328c7b7e831109cdd80ae486669aaf028a.

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Fix integration removal timing (#781)

* Fix integration removal timing

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused commented code

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Nginx Integration Panel Fix: Re-apply link removal (#807)

* Fix broken nginx data volume panel

Signed-off-by: Simeon Widdis <[email protected]>

* Re-add sample data

Signed-off-by: Simeon Widdis <[email protected]>

* Re-apply link fix and update test

Signed-off-by: Simeon Widdis <[email protected]>

* Re-apply formatting to local_repository.test.ts

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Remove dependence on integrations variable (#826)

Signed-off-by: Simeon Widdis <[email protected]>

* Fix index name collisions on integration sample creation. (#818)

* Add integration name to created index mapping

Signed-off-by: Simeon Widdis <[email protected]>

* Switch template naming to loosely reflect SS4O convention

Signed-off-by: Simeon Widdis <[email protected]>

* Split mapping creation by type for code clarity

Signed-off-by: Simeon Widdis <[email protected]>

* Re-introduce confusing result block

Signed-off-by: Simeon Widdis <[email protected]>

* Modify naming template to better match SS4O convention

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Changed log mappings to be apache-specific (#822)

Signed-off-by: Daniel Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* Changed logs mappings to be logs-elb (#821)

Signed-off-by: Daniel Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* update index template to avoid name collisions (#825)

* update index template to avoid name collisions
align s3 component template with all the other integrations

Signed-off-by: YANGDB <[email protected]>

* update k8s integration

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* Aws cloudfront integration (#779)

* new aws s3 integration

Signed-off-by: YANGDB <[email protected]>

* update s3 dashboard

Signed-off-by: YANGDB <[email protected]>

* remove `link` failing integration config field

Signed-off-by: YANGDB <[email protected]>

* add alias for each specific integration related log mapping template

Signed-off-by: YANGDB <[email protected]>

* fix aws prefix error

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping and sample.json

Signed-off-by: YANGDB <[email protected]>

* update integration.config.type field to match the index mapping schema container

Signed-off-by: YANGDB <[email protected]>

* update dashboard.png

Signed-off-by: YANGDB <[email protected]>

* update sample.json

Signed-off-by: YANGDB <[email protected]>

* aws cloud front integration update

Signed-off-by: YANGDB <[email protected]>

* update only relevant files

Signed-off-by: YANGDB <[email protected]>

* remove nginx's only relevant files

Signed-off-by: YANGDB <[email protected]>

* remove nginx's only relevant files

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update according to name collision component strategy

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* aws WAF integration (#788)

* aws cloud trail integration

Signed-off-by: YANGDB <[email protected]>

* update dashboard

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration update

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration update

Signed-off-by: YANGDB <[email protected]>

* add support for aws WAF (Web Application Firewall) logs integration

Signed-off-by: YANGDB <[email protected]>

* update dashboard.png

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update according to name collision component strategy

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* Aws rds Integration support (#741)

* update the vpc flow log readme with a dashboard screenshot and update the sourceUrl to point to the correct place

Signed-off-by: YANGDB <[email protected]>

* add support for observability on aws rds logs - using the pre-canned dashboards

Signed-off-by: YANGDB <[email protected]>

* update rds integration
Signed-off-by: YANGDB <[email protected]>

Signed-off-by: YANGDB <[email protected]>

* add dashboard.jpeg screenshot

Signed-off-by: YANGDB <[email protected]>

* append minor changes

Signed-off-by: YANGDB <[email protected]>

* append minor changes

Signed-off-by: YANGDB <[email protected]>

* fix sq-ip field

Signed-off-by: YANGDB <[email protected]>

* fix sq-ip field

Signed-off-by: YANGDB <[email protected]>

* update dashboard image

Signed-off-by: YANGDB <[email protected]>

* rearrange dashboard

Signed-off-by: YANGDB <[email protected]>

* update according pr comments

Signed-off-by: YANGDB <[email protected]>

* update schema mapping files including cloud mapping

Signed-off-by: YANGDB <[email protected]>

* update vpc flow log containers log name

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update according to name collision component strategy

Signed-off-by: YANGDB <[email protected]>

* remove none relevant fixes

Signed-off-by: YANGDB <[email protected]>


---------

Signed-off-by: YANGDB <[email protected]>

* Aws cloudtrail Integration (#773)

* aws cloud trail integration

Signed-off-by: YANGDB <[email protected]>

* update dashboard

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration update

Signed-off-by: YANGDB <[email protected]>

* aws cloud trail integration update

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* update according to name collision component strategy

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* aws s3 integration (#756)

* new aws s3 integration

Signed-off-by: YANGDB <[email protected]>

* update s3 dashboard

Signed-off-by: YANGDB <[email protected]>

* remove `link` failing integration config field

Signed-off-by: YANGDB <[email protected]>

* add alias for each specific integration related log mapping template

Signed-off-by: YANGDB <[email protected]>

* fix aws prefix error

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping

Signed-off-by: YANGDB <[email protected]>

* update cloud mapping and sample.json

Signed-off-by: YANGDB <[email protected]>

* update integration.config.type field to match the index mapping schema container

Signed-off-by: YANGDB <[email protected]>

* update dashboard.png

Signed-off-by: YANGDB <[email protected]>

* update sample.json

Signed-off-by: YANGDB <[email protected]>

* update index template to avoid name collisions

Signed-off-by: YANGDB <[email protected]>

* remove none relevant fixes

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* Modified Integrations Description (#853)

* Modified integration description

Signed-off-by: Daniel Dong <[email protected]>

* Modified integrations description in snapshot

Signed-off-by: Daniel Dong <[email protected]>

---------

Signed-off-by: Daniel Dong <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>

* update readme with the integration related docs and screen shorts (#852)

Signed-off-by: YANGDB <[email protected]>

* change unapplied cateogry filter to empty instead of x (#858)

Signed-off-by: Derek Ho <[email protected]>

* Fix failing CI (#863)

* Update outdated sidebar and searchbar snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Make backport only run on merged PRs

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* consolidate aws_s3-1.0.0.mapping.json schema to match all across (#851)

Signed-off-by: YANGDB <[email protected]>

* consolidate http-1.0.0.mapping.json schema to match all across (#850)

Signed-off-by: YANGDB <[email protected]>

* consolidate communication-1.0.0.mapping.json schema to match all across (#849)

Signed-off-by: YANGDB <[email protected]>

* Speed up unit testing CI by updating maxWorkers setting (#884)

* Set max CI workers to 2

Signed-off-by: Simeon Widdis <[email protected]>

* Try setting to 100% of workers

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Switch try it button to be secondary (#887)

Signed-off-by: Simeon Widdis <[email protected]>

* Add _refresh call to sync component creation before index creation (#857)

* Add _refresh call to sync component creation before index creation

Signed-off-by: Simeon Widdis <[email protected]>

* Switch to using error message

Signed-off-by: Simeon Widdis <[email protected]>

* Remove redundant catch-rethrow

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused imports

Signed-off-by: Simeon Widdis <[email protected]>

* Also wait for refresh on bulk data creation

Signed-off-by: Simeon Widdis <[email protected]>

* Update outdated sidebar and searchbar snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Make backport only run on merged PRs

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* update backport labels (#898)

Signed-off-by: Shenoy Pratik <[email protected]>

* update raw data (#891)

Signed-off-by: YANGDB <[email protected]>

* K8s integration (#829)

* create new k8s integration

Signed-off-by: YANGDB <[email protected]>

* update k8s integration

Signed-off-by: YANGDB <[email protected]>

* update k8s samples

Signed-off-by: YANGDB <[email protected]>

* update k8s samples

Signed-off-by: YANGDB <[email protected]>

* update k8s metadata schema components

Signed-off-by: YANGDB <[email protected]>

* update k8s metadata schema components

Signed-off-by: YANGDB <[email protected]>

* update k8s dashboard

Signed-off-by: YANGDB <[email protected]>

* update k8s dashboard image

Signed-off-by: YANGDB <[email protected]>

* update k8s dashboard image

Signed-off-by: YANGDB <[email protected]>

* update husky

Signed-off-by: YANGDB <[email protected]>

* revise readme docs

Signed-off-by: YANGDB <[email protected]>

* add OTEL semantic convention support for k8s using mapping aliases

Signed-off-by: YANGDB <[email protected]>

* update according to PR comments

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* consolidate cloud schema to match all across (#846)

Signed-off-by: YANGDB <[email protected]>

* Make OSD Bootstrap more sturdy in CI (#901)

* Apply updates from OSD

Signed-off-by: Simeon Widdis <[email protected]>

* Revert accidental flyout change

Signed-off-by: Simeon Widdis <[email protected]>

* Add bootstrap cache to CI

Signed-off-by: Simeon Widdis <[email protected]>

* Avoid nested directory traversal on cache hashing

Signed-off-by: Simeon Widdis <[email protected]>

* Tweak path traversal again

Signed-off-by: Simeon Widdis <[email protected]>

* Fix order of operations for cd

Signed-off-by: Simeon Widdis <[email protected]>

* Fix CDs again

Signed-off-by: Simeon Widdis <[email protected]>

* Debugging: Add ls

Signed-off-by: Simeon Widdis <[email protected]>

* Fix CDs (again)

Signed-off-by: Simeon Widdis <[email protected]>

* Fix directories for caching

Signed-off-by: Simeon Widdis <[email protected]>

* Update cache behavior based on actual bootstrap function

Signed-off-by: Simeon Widdis <[email protected]>

* Re-add plugin linking

Signed-off-by: Simeon Widdis <[email protected]>

* Make hash target smaller

Signed-off-by: Simeon Widdis <[email protected]>

* Add a restore key to the cache

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Changes Panel Titles to H2 and rearranges the details content to details panel (#918)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <[email protected]>

* update fail backport

Signed-off-by: Derek Ho <[email protected]>

* change panel titles to h2 and rearrange

Signed-off-by: Derek Ho <[email protected]>

* remove unecessary imports

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* update the README.md with Screenshots section

Signed-off-by: YANGDB <[email protected]>

* align all schema mapping components to contain the following metadata… (#927)

* align all schema mapping components to contain the following metadata info:
labels: list of the relevant labels that are associated with this mapping component or index template

Signed-off-by: YANGDB <[email protected]>

* align all schema mapping components to contain the following metadata info:
labels: list of the relevant labels that are associated with this mapping component or index template

Signed-off-by: YANGDB <[email protected]>

* align all schema mapping components to contain the following metadata info:
labels: list of the relevant labels that are associated with this mapping component or index template

Signed-off-by: YANGDB <[email protected]>

---------

Signed-off-by: YANGDB <[email protected]>

* Update label handling on frontend + Refactors (#931)

* Update label display on frontend

Signed-off-by: Simeon Widdis <[email protected]>

* Clear type errors in added_integration.tsx

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused imports

Signed-off-by: Simeon Widdis <[email protected]>

* Add missing labels

Signed-off-by: Simeon Widdis <[email protected]>

* Autoformat the whole repository

Signed-off-by: Simeon Widdis <[email protected]>

* Revert large sample data changes

Signed-off-by: Simeon Widdis <[email protected]>

* Undo _import autoescape

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Fix one more missing label change (#934)

* Update label display on frontend

Signed-off-by: Simeon Widdis <[email protected]>

* Clear type errors in added_integration.tsx

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused imports

Signed-off-by: Simeon Widdis <[email protected]>

* Add missing labels

Signed-off-by: Simeon Widdis <[email protected]>

* Autoformat the whole repository

Signed-off-by: Simeon Widdis <[email protected]>

* Revert large sample data changes

Signed-off-by: Simeon Widdis <[email protected]>

* Undo _import autoescape

Signed-off-by: Simeon Widdis <[email protected]>

* Modify snapshots

Signed-off-by: Simeon Widdis <[email protected]>

* Update label handling in table

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Feature/metrics multiline display (#899)

* Working multi line viz with new sql

Signed-off-by: Kavitha Conjeevaram Mohan <[email protected]>
Signed-off-by: Peter Fitzgibbons <[email protected]>

* Cleanup conflicts and fn. arguments

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Correct metric check fn.

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Clean Code

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Clean Code

Signed-off-by: Peter Fitzgibbons <[email protected]>

* merge dateTime convert function

Signed-off-by: Kavitha Conjeevaram Mohan <[email protected]>

* Update snapshots

Signed-off-by: Kavitha Conjeevaram Mohan <[email protected]>

* Fix cypress test 8_metrics_analytics.spec.js to match removal of "Recently Created" sidebar section.

Fix Metric search to be case-insensitive.

Signed-off-by: Peter Fitzgibbons <[email protected]>

* Fix several test snapshots.
Fix metrics_grid.test coreRefs mock

Signed-off-by: Peter Fitzgibbons <[email protected]>

---------

Signed-off-by: Kavitha Conjeevaram Mohan <[email protected]>
Signed-off-by: Peter Fitzgibbons <[email protected]>
Co-authored-by: Kavitha Conjeevaram Mohan <[email protected]>

* Increment version to 3.0.0.0 (#854)

Signed-off-by: opensearch-ci-bot <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>

* Refactor integration validation logic with a cleaner interface (#943)

* Refactor validation logic with a deeper interface

Signed-off-by: Simeon Widdis <[email protected]>

* Remove redundant test.

This test is unneeded after 12c4bcf

Signed-off-by: Simeon Widdis <[email protected]>

* Add tests for new validators

Signed-off-by: Simeon Widdis <[email protected]>

* Make better failure mode for invalid objects

Signed-off-by: Simeon Widdis <[email protected]>

* Convert validator methods to use result types

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* update logo related snapshots due to upstream changes (#950)

Signed-off-by: Derek Ho <[email protected]>

* Fix all yarn lints for server directory (#953)

* Apply yarn auto-lints

Signed-off-by: Simeon Widdis <[email protected]>

* Fix equalities

Signed-off-by: Simeon Widdis <[email protected]>

* Fix naming conventions

Signed-off-by: Simeon Widdis <[email protected]>

* Fix non-export issues

Signed-off-by: Simeon Widdis <[email protected]>

* Fix default exports

Signed-off-by: Simeon Widdis <[email protected]>

* Remove unused dependency

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* Update mocha (#960)

* Update mocha

Signed-off-by: Simeon Widdis <[email protected]>

* Update mocha to 10.2.0

Signed-off-by: Simeon Widdis <[email protected]>

* Revert mocha to 10.1.0 for OSD compat.

Signed-off-by: Simeon Widdis <[email protected]>

* Remove random added yaml dep

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>

* trace analytics collapse dashboard queries, align naming to industry conventions (#514)

* trace analytics v2 first remove expensive dashboard page

Signed-off-by: Derek Ho <[email protected]>

* get trace groups request working and mapping correctly

Signed-off-by: Derek Ho <[email protected]>

* get trace group filter fully working

Signed-off-by: Derek Ho <[email protected]>

* try some service map adjustments

Signed-off-by: Derek Ho <[email protected]>

* change to up down to fully show labels

Signed-off-by: Derek Ho <[email protected]>

* put dashboard in accordian at bottom of services and only make calls if it is open

Signed-off-by: Derek Ho <[email protected]>

* some changes to support dashboard content in accordian

Signed-off-by: Derek Ho <[email protected]>

* demo code

Signed-off-by: Derek Ho <[email protected]>

* fix tests

Signed-off-by: Derek Ho <[email protected]>

* remove dashboard from app analytics

Signed-off-by: Derek Ho <[email protected]>

* fix up with tests

Signed-off-by: Derek Ho <[email protected]>

* fix test

Signed-off-by: Derek Ho <[email protected]>

* revert only and extract to constant

Signed-off-by: Derek Ho <[email protected]>

* add auth const

Signed-off-by: Derek Ho <[email protected]>

* address self-PR review

Signed-off-by: Derek Ho <[email protected]>

* remove unecessary code

Signed-off-by: Derek Ho <[email protected]>

* change timeout to 25 seconds

Signed-off-by: Derek Ho <[email protected]>

* change to row in accordian

Signed-off-by: Derek Ho <[email protected]>

* update labels and tests

Signed-off-by: Derek Ho <[email protected]>

* update cypress test

Signed-off-by: Derek Ho <[email protected]>

* verify cypress tests work with changes

Signed-off-by: Derek Ho <[email protected]>

* remove accidental only

Signed-off-by: Derek Ho <[email protected]>

* address PR review comments

Signed-off-by: Derek Ho <[email protected]>

* make missing configuration more clear

Signed-off-by: Derek Ho <[email protected]>

* update snapshots

Signed-off-by: Derek Ho <[email protected]>

* update snapshot

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: Derek Ho <[email protected]>

* added code for the change of design of the sidebar, fixed overriding … (#928)

* added code for the change of design of the sidebar, fixed overriding spinner bug, changed font size

Signed-off-by: sumukhswamy <[email protected]>

* addressed PR comments

Signed-off-by: sumukhswamy <[email protected]>

* code for overriding pattern

Signed-off-by: sumukhswamy <[email protected]>

* updated snapshots

Signed-off-by: sumukhswamy <[email protected]>

---------

Signed-off-by: sumukhswamy <[email protected]>

* update search bar snapshots according to upstream changes (#964)

Signed-off-by: Derek Ho <[email protected]>

---------

Signed-off-by: TackAdam <[email protected]>
Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: YANGDB <[email protected]>
Signed-off-by: Eric Wei <[email protected]>
Signed-off-by: Peter Fitzgibbons <[email protected]>
Signed-off-by: Derek Ho <[email protected]>
Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: Derek Ho <[email protected]>
Signed-off-by: Daniel Dong <[email protected]>
Signed-off-by: opensearch-ci-bot <[email protected]>
Signed-off-by: Haidong <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Dan Dong <[email protected]>
Signed-off-by: Shenoy Pratik <[email protected]>
Signed-off-by: Kavitha Conjeevaram Mohan <[email protected]>
Signed-off-by: sumukhswamy <[email protected]>
Co-authored-by: TackAdam <[email protected]>
Co-authored-by: Simeon Widdis <[email protected]>
Co-authored-by: Adam Tackett <[email protected]>
Co-authored-by: Simeon Widdis <[email protected]>
Co-authored-by: YANGDB <[email protected]>
Co-authored-by: Joshua Li <[email protected]>
Co-authored-by: Eric Wei <[email protected]>
Co-authored-by: Anirudha (Ani) Jadhav <[email protected]>
Co-authored-by: Peter Fitzgibbons <[email protected]>
Co-authored-by: Shenoy Pratik <[email protected]>
Co-authored-by: Rupal Mahajan <[email protected]>
Co-authored-by: Daniel Dong <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: wanghd89 <[email protected]>
Co-authored-by: Haidong <[email protected]>
Co-authored-by: dependabot[bot] <49699333…
derek-ho pushed a commit that referenced this pull request Sep 6, 2023
* Set max CI workers to 2

Signed-off-by: Simeon Widdis <[email protected]>

* Try setting to 100% of workers

Signed-off-by: Simeon Widdis <[email protected]>

---------

Signed-off-by: Simeon Widdis <[email protected]>
amsiglan pushed a commit to amsiglan/dashboards-observability that referenced this pull request Jun 7, 2024
…roject#884) (opensearch-project#885)

* Set max CI workers to 2

* Try setting to 100% of workers

---------

(cherry picked from commit a39265a)

Signed-off-by: Simeon Widdis <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
(cherry picked from commit f3a4625)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants