opensearch-release-notes-2.0.0-rc1.md

OpenSearch and Dashboards 2.0.0-rc1 Release Notes

Release Highlights

• Document level alerting allows users to create monitors that can generate alerts per document.
• Lucene 9 is now used in OpenSearch
• The Geo Map Tiles in OpenSearch Dashboards are updated and now have a pipeline to update them more frequently.
• Document level security now supports term lookup queries.

OpenSearch Notifications

• OpenSearch 2.0.0 RC1 is the first official release with OpenSearch Notifications.
• Notifications consist of three plugins, notifications-core and notifications backend plugins for OpenSearch, and a notificationsDashboards frontend plugin for OpenSearch Dashboards.

Release Details

OpenSearch and OpenSearch Dashboards 2.0.0-rc1 include the following features, enhancements, bug fixes, infrastructure, documentation, maintenance, and refactoring updates.

OpenSearch Release Notes.

OpenSearch Dashboards Release Notes.

FEATURES

OpenSearch Index Management

• Adds shrink action to ISM (#326)
• Notification integration with IM (#338)

OpenSearch Index Management Dashboards Plugin

• Adds UI for shrink action (#176)

OpenSearch Job Scheduler

• Enables locking on an arbitrary lockID (164)

OpenSearch Notifications

• Use Notifications to manage your notification channel configurations in a centralized location and send messages to these channels.
• This release also adds notification-specific REST APIs for CRUD operations on channels and internal transport APIs for integrating/communicating with other plugins.

OpenSearch Performance Analyzer

• Adds setting to enable/disable Thread Contention Monitoring (#171)
• Fixes calculation of average thread blocked time and average thread waited time (#118)

OpenSearch SQL

• Add Kmeans and AD command documentation (#493)
• AD and Kmeans grammar edits (#500)
• Support more parameters for AD and KMEANS command, and update related documentation (#515)

ENHANCEMENT

OpenSearch Alerting

• Add automated migration for Destinations to Notification configs (#379)
• Integrate with Notifications plugin for Alerting backend (#401)
• Integrate Document Level Alerting changes (#410)
• Alias support for Document Level Monitors (#416)

OpenSearch Alerting Dashboards Plugin

• Implemented UX support for configuring doc level monitors. (#218)
• Integrate Alerting Dashboards with Notifications Plugin (#220)
• Added document column to alerts dashboard for doc level monitors. Adjusted alerts dashboard configuration to remove unused alert states for doc level monitors. Refactored style of alerts flyout based on UX feedback. (#223)

OpenSearch Anomaly Detection

• Changed usages of "master" to "clusterManager" in variable names (#504)

OpenSearch Common Utils

• Add SQL/PPL transport request/response models for SQL plugin (#155)
• Support sending email message via Notifications pass-through API (#158)

OpenSearch Cross Cluster Replication

• Change the "Master" nomenclature (#319)
• Replace checked-in ZIPs with dynamic dependencies (#335)
• Add support for build version qualifiers (#334)

OpenSearch Dashboards Visualizations

• Support advanced settings (#68)

OpenSearch Index Management Dashboards Plugin

• Add refresh button to rollup page (#132)
• Adding support to edit/create notifications using channels in IM (#181)

OpenSearch k-NN

• Manually add footer to engine files (#327)
• Integration with base OpenSearch 2.0 (#328)
• Remove remaining mapping type (#335)
• Adding documentation on codec versioning (#337)
• Adding clear cache test step (#339)
• Add size validation for Search Model API (#352)

OpenSearch Ml Commons

• Add circuit breaker trigger count stat.(#274)

OpenSearch Observability

• Remove button toggle and add stop button (#623)

OpenSearch Security

• Remove checked-in zip files (#1774)
• Introduce dfm_empty_overrides_all setting to enable role without dls/fls to override roles with dls/fls (#1735)
• Add depreciation notice to security tools (#1756)
• [Practice] Reverting changes (#1754)
• Renames securityconfig folder to config in bundle step and makes relevant changes (#1749)
• Updated issue templates from .github. (#1740)
• Updates Dev guide (#1590)
• List out test failures in CI log (#1737)
• Make Git ignore out/ directory (#1734)
• Fix data-stream name resolution for wild-cards (#1723)
• Remove support for JDK14 (#1720)
• Speeding up tests (#1715)
• Fix min_doc_count handling when using Document Level Security (#1714)
• Set the mapped security roles of the user so these can be used by the DLS privileges evaluator. Allow security roles to be used for DLS parameter substitution. Fixes opensearch-project/security/#1568 (#1588)
• Convert Plugin install to only build once (#1708)
• Upgrade to Gradle 7 (#1710)
• Move CodeQL into parallel workfow (#1705)
• Seperate BWC tests into parallel workflow (#1706)
• Fixes broken test due to unsupported EC using JDK-17 (#1711)
• Centralize version settings (#1702)
• Remove TransportClient auth/auth (#1701)
• Add new code hygiene workflow (#1699)
• Remove JDK8 from CI (#1703)
• Add CI check for demo script (#1690)
• Introduce BWC tests in security plugin (#1685)
• Correct the step name in CI (#1683)
• Add support for DLS Term Lookup Queries (#1541)

OpenSearch Security Dashboards Plugin

• Change 2.0-alpha1 to 2.0-rc1. (#946)
• Make Git ignore .idea/ folder (#944)
• Updated issue templates from .github. (#931)
• Bumps version of main to 2.0.0.0 (#928)

BUG FIXES

OpenSearch Alerting

• Completely fix docker pull and install plugin (#376)
• Make sure alerting is using the build script in its own repo (#377)
• Fix security test workflow (#407)
• Fixed a flaky test condition. (#375)
• Remove actionGet and fix minor bugs (#424)
• Fix UnsupportedOperation error while alert categorization in BucketLevel monitor (#428)

OpenSearch Alerting Dashboards Plugin

• Fixed a bug that was causing the UX to reset visual editor trigger conditions to their default values when a trigger name contained periods. (#204)
• Fixed a bug that was preventing the configured schedule from displaying when editing a monitor that was created through backend commands. (#197)
• Fixed bugs associated with alerts table, and addressed UX review feedback. (#222)
• Document level monitor UX bug fixes (#226)

OpenSearch Anomaly Detection

• Changed default description to empty string instead of null (#438)
• Fixed ADTaskProfile toXContent bug and added to .gitignore (#447)
• Fix restart HCAD detector bug (#460)
• Check if indices exist in the presence of empty search results (#495)

OpenSearch Anomaly Detection Dashboards

• Remove extra loading spinners (#238)
• Remove additional loading spinners and removing master from docker compose (#243)

OpenSearch Cross Cluster Replication

• Bugfix: 2.0 Changes for CI workflows (#354)
• Remove mapping types (#318)

OpenSearch Dashboards Reports

• Bump async from 3.2.0 to 3.2.3 in /dashboards-reports (#338)
• Bump moment from 2.29.1 to 2.29.3 in /dashboards-reports (#344)
• Bump minimist from 1.2.5 to 1.2.6 in /dashboards-reports (#321)

OpenSearch Dashboards Visualizations

• Bump async from 3.2.0 to 3.2.3 in /gantt-chart (#67)
• Bump moment from 2.29.1 to 2.29.2 in /gantt-chart (#66)
• Bump minimist from 1.2.5 to 1.2.6 in /gantt-chart (#57)
• Remove duplicated dependencies (#64)

OpenSearch Index Management

• Fix metadata migration logic error when update setting call failed (#328)
• Updates search text field to keyword subfield for policies and managed indices (#267)
• Fixes shard allocation checks (#335)
• BugFix: Notification integration issues (#339)
• Fixes flaky continuous transforms and shrink tests (#340)

OpenSearch Index Management Dashboards Plugin

• Fix rendering in transforms UI (#179)

OpenSearch k-NN

• Allow null value for params in method mappings (#354)

OpenSearch Ml Commons

• Support dispatching execute task; don't dispatch ML task again (#279)
• Fix NPE in anomaly localization (#280)
• Create model/task index with correct mapping (#284)

OpenSearch Observability

• Edit visualization time change (#617)
• Remove duplicated node dependencies (#620)
• Bug fixes for application analytics (#608)
• Fixes trace analytics invalid service map and increase span limit (#629)
• Adding legacy UI route for traces (#653)
• Fix change availability bug (#667)
• Fix test to check for empty event analytics (#669)
• Bump prismjs from 1.25.0 to 1.27.0 in /dashboards-observability (#508)
• Bump minimist from 1.2.5 to 1.2.6 in /dashboards-observability (#614)
• Bump moment from 2.29.1 to 2.29.2 in /dashboards-observability (#636)
• Bump async from 3.2.1 to 3.2.3 in /dashboards-observability (#654)

OpenSearch Performance Analyzer

• Fix EventLogFileHandlerTests flaky test (#178)
• Add retry for tests (#180)

OpenSearch Security

• Add signal/wait model for TestAuditlogImpl (#1758)
• Switch to log4j logger (#1751)
• Remove sleep when waiting for node closure (#1722)
• Remove explictt dependency on jackson-databind (#1709)
• Fix break thaat was missed during a merge (#1707)
• Revert "Replace opensearch class names with opendistro class names during serialization and restore them back during deserialization (#1278)" (#1691)
• Update to most recent verson of jackson-databind (#1679)
• Fixed rest status for the replication action failure with DLS/FLS and (#1677)
• Downgrade Gradle version (#1661)

OpenSearch Security Dashboards Plugin

• Fix broken nextUrl= parameter logic (#940)

OpenSearch SQL

• Bump async from 3.2.0 to 3.2.3 in /workbench (#559)
• Bump moment from 2.29.1 to 2.29.2 in /workbench (#546)
• Version Bump: spring-beans-5.2.19 -> spring-beans-5.2.20 (#527)
• Bug Fix, return default ID when log4j ThreadContext is empty (#538)
• Removed ES reference from build.gradle (#562)

INFRASTRUCTURE

OpenSearch Alerting

• Removed the Beta label from the bug report template. (#353)
• Update alerting with qualifier support in releases (#366)
• Use OpenSearch 2.0.0-alpha1 (#370)
• Add build qualifier default to alpha1 for 2.0.0 (#373)
• Remove JDK 14 and Add JDK 17 (#383)
• Updated issue templates from .github. (#382)
• Incremented version to 2.0-rc1. (#404)
• Replace checked-in ZIP for bwc tests with a dynamic dependency (#411)
• Update integTest gradle scripts to run via remote cluster independently (#418)

OpenSearch Alerting Dashboards Plugin

• Removed the Beta label from the bug report template. (#196)
• Updated issue templates from .github. (#205)

OpenSearch Anomaly Detection

• Reduced jacoco exclusions and added more tests (#446)
• Remove oss flavor (#449)
• Add auto labeler workflow (#455)
• Gradle 7 and Opensearch 2.0 upgrade (#464)
• Add support for -Dbuild.version_qualifier (#468)
• Changed forbiddenAPIsTest files and made relevant forbidden fixes (#450)
• Adding test-retry plugin (#456)
• Updated issue templates from .github. (#488)
• Removing job-scheduler zip and replacing with distribution build (#487)
• JDK 17 support (#489)
• Moving script file in scripts folder for file location standardization (#494)
• Removed rcf jar for 3.0-rc1 and fixed zip fetching for AD and JS (#500)
• Changed to rc1 and add tar to distribution download link (#503)
• Remove BWC zips for dynamic dependency (#505)

OpenSearch Anomaly Detection Dashboards

• Add auto labeler workflow (#205)
• Updated issue templates from .github. (#226)
• 2.0 version bump (#230)
• Change 2.0-alpha1 to 2.0-rc1. (#241)
• Update labeler to default backports to 2.x (#246)

OpenSearch Asynchronous Search

• Adding support for integration tests with remote cluster (#111)
• Remove support for JDK 8 (#114)
• Remove support for JDK 14 (#125)
• Updated issue templates from .github. (#126)
• Replace checked-in ZIP with a dynamic dependency (#133)

OpenSearch Common Utils

• Upgrade gradle artifacts to 7.3.3 (#135
• Update common-utils to depend on the OpenSearch repositories plugin (#137)
• Add sign-off option for version workflow PR (#143)
• Add qualifier default to alpha1 in build.gradle (#151)
• Update issue templates from github for bugs and features (#154)
• Remove support for JDK 14 (#159)

OpenSearch Cross Cluster Replication

• [CI] Add support for JDK 17 (#331)
• Remove support for JDK 8 (#330)
• Drop support for JDK 14 (#346)

OpenSearch Dashboards Reports

• Added missing zip for bwc tests (#329)
• Remove JDK14 from CI (#335)
• Updated issue templates from .github. (#328)
• Remove zips and download from remote at build time (#337)

OpenSearch Index Management

• Replace checked-in ZIPs with dynamic dependencies (#327)
• Only download JS zip when integTest is running (#334)

OpenSearch Job Scheduler

• Fix qualifier to be added as version number matching with core (152)
• Adding signoff option for version workflow PR (156)
• Add default alpha1 to JS qualifier (162)
• Remove hardcoding snapshot for JS in gradle (163)

OpenSearch k-NN

• Change minimum supported JDK version from 8 to 11 (#321)
• Adding jdk 17 to CI (#322)
• Adding build.version_qualifier (#324)
• Remove version from CMakeLists.txt (#325)
• Add support for knn to have qualifiers (#329)
• Applying build qualifier only to knn plugin version (#330)
• Remove hardcoding of version in knn CI (#334)
• Apply spotless on entire project (#336)
• Remove hardcoded URL (#338)
• Dropping support for JDK 14 (#344)
• Rename knnlib to lib (#345)
• Update knn with dynamic version assignment (#349)
• Updated issue templates from .github (#351)
• Incremented version to 2.0-rc1 (#363)

OpenSearch Ml Commons

• Drop support for JDK 14 (#267)
• Add UT/IT Coverage for action/models and action/tasks. (#268)
• Default qualifier to alpha1 and fix workflows (#269)
• Remove additional vars in build.gradle that are not used (#271)
• Add UT for Search transport action. (#272)
• Updated issue templates for bugs and features. (#273)
• Add more test to improve coverage of abstract search action(#275)
• Add UT for RestMLExecuteAction, and remove it out from the jacoco exclusive list. (#278)
• Add coverage badges (#281)
• Re-enable docker image tests for 2.0. (#288)

OpenSearch Observability

• Bwc update (#604)
• Event cypress tests (#611)
• Test 2.0 (#624)
• Updated panel flaky cypress tests (#633)
• Updated notebook cypress tests (#637)
• Updated events flyout ui, unskip jest tests (#638)
• Remove zips used by bwc tests (#648)
• Fix trace analytics cypress (#652)
• Event analytics jest tests (#651)
• 2.0 cypress tests (#658)
• Updated issue templates from .github. (#662)
• Removing add sample data test from panels cypress (#668)

OpenSearch SQL

• Fix bwc build issue with jdk17 (#520)
• Updated issue templates from .github. (#531)
• Removing JDK14 from CI (#547)
• Replace checked-in ZIP with a dynamic dependency (#514)

DOCUMENTATION

OpenSearch Alerting

• Add Document Level Alerting RFC (#388)
• Deprecate the Master nomenclature in 2.0 (#415)
• Add release notes for version 2.0.0-rc1 (#426)

OpenSearch Alerting Dashboards Plugin

• Add release notes for version 2.0.0-rc1 (#227)

OpenSearch Anomaly Detection

• Add Visualization integration RFC docs (#477)

OpenSearch Anomaly Detection Dashboards

• Update integ tests badge in README (#215)

OpenSearch Asynchronous Search

• Change master nomenclature (#116)

OpenSearch Common Utils

• Add release notes for version 2.0.0-rc1 (#162)

OpenSearch Dashboards Reports

• Remove master and whitelist text (#342)

OpenSearch Dashboards Visualizations

• Updated issue templates from .github. (#59)

OpenSearch Index Management

• Updated issue templates from .github. (#324)

OpenSearch Index Management Dashboards Plugin

• Updated issue templates from .github. (#168)

OpenSearch Job Scheduler

• Updated issue templates from .github. (165)

OpenSearch Observability

• Remove master and whitelist text (#657)

OpenSearch Performance Analyzer

• Updated issue templates from .github. (#177)
• Removing metrics which are not required now as were removed in OS 2.0 (#159)

OpenSearch SQL

• Change master nomenclature (#551)
• Change blacklist and whitelist nomenclature (#560)

MAINTENANCE

OpenSearch Alerting

• Upgrade kotlin to 1.16.10 (#356)
• Upgrade Alerting to 2.0 (#357)

OpenSearch Alerting Dashboards Plugin

• Bumped main branch version to 2.0 to align with OpenSearch-Dashboards. Added alpha1 qualifier to align with backend snapshot version. (#202)
• [Build] Bump plugin version to 2.0.0.0-rc1 (#213)
• Incremented version to 2.0-rc1. (#216)

OpenSearch Asynchronous Search

• Upgrades to Opensearch 2.0, Gradle 7.3 and fixes (#110)
• Add support for -Dbuild.version_qualifier (#115)
• Remove usage of mapping types (#119)
• Remove hardcoding of versions in workflow (#120)
• Add qualifier default as alpha (#123)
• Change 2.0-alpha1 to 2.0-rc1 (#131)

OpenSearch Dashboards Reports

• Bump version to 2.0.0 (#311)
• Support build version qualifier for reports-scheduler (#322)
• Bump to 2.0 alpha1 and gradle 7 (#325)
• Make sure qualifier is applied in 2.0.0 (#327)
• Change alpha1 to rc1 for first 2.0 release (#333)
• Change 2.0-alpha1 to 2.0-rc1. (#341)

OpenSearch Dashboards Visualizations

• Change alpha1 to rc1 for first 2.0 release (#65)
• Bump version to 2.0.0 (#56)
• Add alpha1 qualifiers for dashboards plugin (#58)

OpenSearch Index Management

• Upgrades Index Management to use 2.0.0-alpha1 of OpenSearch and dependencies (#318)
• Make sure qualifier default is alpha1 in IM (#323)
• Incremented version to 2.0-rc1. (#331)
• Non-inclusive nonmenclature update (#337)

OpenSearch Index Management Dashboards Plugin

• Upgrades IM Dashboard plugin to OpenSearch Dashboards 2.0 (#169)
• Bumps version to 2.0.0.0-rc1 (#172)
• Incremented version to 2.0-rc1. (#175)

OpenSearch Job Scheduler

• Removes usage of mapping types (155)
• Dropping support for jdk 14 (168)
• Incremented version to 2.0-rc1. (169)

OpenSearch Ml Commons

• Change 2.0-alpha1 to 2.0-rc1. (#282)
• Bump RCF version to 3.0-rc2.1 (#289)

OpenSearch Observability

• Bump plugins to 2.0 and support build.version_qualifier (#602)
• Add alpha1 qualifier and JDK 17 for backend (#607)
• Add alpha1 qualifiers for dashboards plugin (#616)
• Tweak build.gradle to have the correct qualifiers in 2.0.0 (#619)
• Change alpha1 to rc1 for first 2.0 release (#635)
• Change 2.0-alpha1 to 2.0-rc1. (#655)

OpenSearch Performance Analyzer

• Gradle 7, JDK related changes and OS 2.0 (#179)
• Add additional logs for Integration Tests (#182)
• Enable dependency license check and removing unused license (#183)
• Moving build script file here from opensearch build package (#184)
• Update directory names and remove jar for integTest (#187)
• Update PA directories from plugins to root (#189)
• Changes to add jdk17, remove jdk 8,14, OS 2.0 and upgrade to gradle 7 (#156)
• Update directory names (#166)
• Update PA directories from plugins to root (#168)

OpenSearch Security

• Incremented version to 2.0-rc1. (#1764)
• Upgrade to opensearch 2.0.0 alpha1 (#1741)
• Upgrade to OpenSearch 2.0.0 (#1698)
• Move to version 2.0.0.0 (#1695)

OpenSearch Security Dashboards Plugin

• Revert "Enforce authentication on api/status route by default (#943)" (#950)
• Enforce authentication on api/status route by default (#943)
• [Build] Restore osdVersion to 2.0.0 (#947)
• [Build] Bump to 2.0.0.0-rc1 (#941)

OpenSearch SQL

• Version 2.0 (#507)
• Removed changes introduced to support JDK8 (#513)
• Add JDK 17 support (#512)
• Upgrade OS Version to 2.0.0-alpha1-SNAPSHOT (#518)
• Add alpha1 qualifiers for dashboards plugin (#523)
• 2.0 build fix (#535)
• Change ODBC version to 1.4 for release (#542)
• Change workbench alpha1 to rc1 for first 2.0 release (#545)
• Change 2.0-alpha1 to 2.0-rc1. (#555)
• Replace checked-in ml-commons dependency for 2.0 (#563)
• Delete ml-commons zip file (#565)
• Bump ml-client to 2.0 (#568)

REFACTORING

OpenSearch Alerting

• Remove write Destination APIs (#412)
• Remove Alerting's notification subproject (#413)
• Skipping destination migration if alerting index is not initialized (#417)
• Fix Finding action naming and update release notes (#432)

OpenSearch Alerting Dashboards Plugin

• Temporarily disabled destination use in some cypress tests to resolve flakiness. (#214)
• Remove disabled buttons and update Destination flows to reflect read-only state (#221) ING

OpenSearch Common Utils

• Remove feature and feature_list usage for Notifications (#136)
• Rename references for Get Channels API for Notifications (#140)
• Remove allowedConfigFeatureList from GetPluginFeaturesResponse for Notifications (#144)
• Remove NotificationEvent Request, Response and SearchResults (#153)
• Add NotificationEvent to SendNotificationResponse and Removal of NotificationID (#156)

OpenSearch k-NN

• Refactor KNNCodec to use new extension point (#319)
• Refactor BWC tests into sub project (#359)

OpenSearch Ml Commons

• Removed RCF jars and updated to fetch RCF 3.0-rc2 from maven (#277)

OpenSearch Observability

• Modularize and cleanup traces (#601)
• Modularize and cleanup panel (#603)
• Modularize event Analytics live tail and fix bug (#647)
• Fix lint and modularize dashboard (#583)
• Modularize service and fix issues (#595)