The OpenSearch 2.5.0 release adds new tools and enhancements to help you advance your search, analytics, and observability workloads. This release includes the software’s first Debian distribution and first administrative user interface, along with support for multi-layered maps and the ability to analyze traces in the Jaeger schema. The release also includes indexing and search improvements for Lucene-based k-NN search functionality, and Security Analytics tools are now generally available. Following are some highlights for this release.
- You can now perform common administrative operations on your OpenSearch indexes, such as CRUD (Create, Read, Update, and Delete) functions, through an admin user interface.
- OpenSearch 2.5.0 lets you analyze trace data collected by the open-source Jaeger solution. Select Data Prepper or Jaeger as your trace data source as part of the OpenSearch Dashboards Observability feature.
- With this release, Security Analytics for OpenSearch and OpenSearch Dashboards is generally available, offering a number of tools to help users protect their data and infrastructure.
- You can build multi-layer maps from multiple data sources, combining data from different indexes into a single visualization to identify correlations and gain insights into geospatial data.
- New Debian distributions let you deploy OpenSearch and OpenSearch Dashboards directly on servers running Debian-based Linux distributions.
- Administrators can now view the health of their cluster at the awareness attribute level when shard allocation awareness is configured.
- You can now search your rollup indexes using query string search queries.
OpenSearch 2.5.0 includes the following experimental features. Experimental features are disabled by default. For instructions on how to enable them, see the version history (https://opensearch.org/docs/latest/version-history/) page which includes links to the documentation.
- Request-level durability allows you to deploy remote-backed storage on a per-index basis, supporting data durability for cloud-based backup and restore operations.
- As an enhancement to semantic search functionality, the model-serving framework now allows users to serve machine learning (ML) models on ML nodes that can take advantage of CUDA-compatible GPUs.
OpenSearch and OpenSearch Dashboards 2.5.0 includes the following feature, enhancement, bug fixes, infrastructure, documentation, maintenance and refactoring updates.
OpenSearch Release Notes.
OpenSearch Dashboards Release Notes.
- Add tooltip for hover (#132)
- Introduce tooltip fields to document layer specification (#124)
- Add support for WKT format (#165)
- Add global time filter bar to maps (#131)
- Support custom layer for maps (#150)
- Support geo_shape visualization in documents layer (#111)
- Introduce saved object plugin into maps plugin (#67)
- Add new layer functions for OpenSearch map layer (#66)
- Add map page and add basic layers panel (#40)
- Add layer panel component (#51)
- Add base map layer functions (#62)
- Query with geo bounding box (#148)
- Display search filters in map layer config panel (#130)
- Add log pattern table (#1187)
- Implementing search feature (#1286)
- Add support for raw Jaeger schema data (#150)
- Add index administrative operations (#537)
- Support for spotlessApply for Job Scheduler (#291)
- Add native memory circuit breaker. (#689)
- Implement secure transport action for get alerts and ack alerts. (#161)
- GetMappingsView API - index pattern/alias/datastream support. (#245)
- Createmappings api index pattern support. (#260)
- Refactoring | Updates overview stats components to use EUI/Stats loading component. (#194)
- Update chart legend font size and padding. (#196)
- YAML Rule Editor Support. (#201)
- Adds dynamic chart time unit based on the selected time span. (#204)
- Rule YAML preview. (#209)
- Feature/detector navigation to findings and alerts. (#210)
- Show surrounding documents when index pattern is available; Finding flyout UI polish. (#216)
- Rule flyout opening from Findings and Alerts page. (#219)
- Show success toast when detector is updated. (#224)
- Add chart tooltips. (#225)
- Add interactive legend into charts. (#226)
- Implement date/time picker on the overview page. (#232)
- Rule details flyout on create rule page. (#236)
- Add loading state for all tables visualizations on overview page. (#237)
- Toggle all rules button on detector edit page. (#239)
- Rule form validation on submit. (#264)
- Feature/charts should show the entire time range selected in the filter. (#265)
- Rule details flyout on detector create 4th step. (#266)
- More validations on YAML rule editor. (#279)
- Rule details flyout on detector view page. (#292)
- Improve rules view in detector details. (#310)
- Adds findings alerts legend in overview page. (#318)
- Feature/hide view docs button. (#320)
- Improved field mapping UX. (#330)
- Data source single select field. (#333)
- Suppressed unnecessary error toast for custom rules. (#338)
- Speed up cold start (#753)
- Support for indices clean-up after the test runs (625)
- Stopping replication before clean up of indices (635)
- Support other geopoint formats (#144)
- Add reorder handle inside layer panel (#116)
- Update show/hide icon (#114)
- Update basic layer settings (#107)
- Delete layer modal (#139)
- Add multi-layer support to map popup (#140)
- Support overriding maps config from OSD config yml file (#202)
- Use front-end report generation instead of chromium (#586)
- Added support for QueryStringQuery in rollups (#616)
- Extend SystemIndexPlugin for k-NN model system index (#630)
- Add Lucene specific file extensions to core HybridFS (#721)
- Add more parameters for text embedding model (#640)
- Add more pooling method and refactor (#672)
- Add ML task timeout setting and clean up expired tasks from cache (#662)
- Change only run on ml node setting default value to true (#686)
- Add filter option for query type (#88)
- Add retry mechanism for neural search inference (#91)
- Enable core branching strategy and make Neural Plugin as extensible plugin. (#87)
- Increasing thread sleep for tests to 100,000 (#294)
- Add more metrics for backend plugin (#1323)
- When excluding fields also exclude the term + .keyword (#2377)
- Update tool scripts to run in windows (#2371, #2379)
- Remove trimming of whitespace when extracting SAML backend roles (#2381, #2383)
- Add script for workflow version increment (#2374, #2386)
- Enhance the stability of SAML integ test (#1237, #1272)
- Change the reference branch to 2.5 for Cypress test (#1298)
- Add low-level create table and table exists API (#834)
- Add time window and window assigner (#950)
- Add valueOf() to Expression (#1055)
- Add Statement, QueryExecution and QueryManager (#845)
- Add Streaming Source Impl (#994)
- Add watermark generator (#959)
- Add stream context and window trigger (#958)
- Add micro batch streaming execution (#1044)
- Add Streaming Plan Implementation (#1068)
- Add CBRT to the V2 engine (#1081)
- Add CBRT function to the PPL (#1097)
- Add timeout option to SQL CLI tool. (#1076)
- Add Day_Of_Year Function To OpenSearch (#1128)
- Add Week_Of_Year Function To OpenSearch (#1127)
- Add Month_Of_Year Function To OpenSearch (#1129)
- Add Minute_Of_Day Function To SQL Plugin (#1207 #1214)
- Add Second_Of_Minute Function As An Alias Of The Second Function (#1231 #1237)
- Add Support For Legacy Syntax For Match Function In New Engine (#1090)
- Add MatchPhraseQuery As Alternate Syntax for Match_Phrase Function (#1103)
- Use query execution start time as the value of now-like functions. (#1047)
- Add Support for Alternate Legacy MULTIMATCH syntax (#1102)
- Add position() function to V2 engine (#1121)
- Add position() string function to PPL (#1147)
- Add support for wildcard_query function to the new engine (#156 #1108)
- Add reverse() string function to V2 SQL Engine(#1154)
- Add table write operator and builder (#1094)
- Add BETWEEN expression in v2 engine (#1163)
- Adding UTC_DATE, UTC_TIME, UTC_TIMESTAMP (#1193 #1198)
- Validate field and fields parameters in relevance search functions (#1067 #1199)
- Add
TIMEDIFFandDATEDIFFfunctions. (#131 #1195 #1234) - Add functions
ADDTIMEandSUBTIME. (#132 #1194 #1252) - Add Day_Of_Week Function As An Alias Of DayOfWeek (#190 #1228 #1239)
- Add Minute_Of_Hour Function As An Alias Of Minute Function (#1253)
- Add support for long value return for CEIL, CEILING and FLOOR math functions (#1205)
- Add Alternate Syntax For Match_Query And Other Functions (#1166)
- Add Support For
TIMEType in DAY_OF_YEAR Functions (#199 #1223 1#258) - Add Day_Of_Month Function As An Alias Of DayOfMonth (#1227 #1265)
- Add security patch for CVE-2020-15250 (#1095)
- Add security patch for CVE-2022-45868 (#1107)
- Add Hour_Of_Day Function As An Alias Of Hour (#1226 #1270)
- Added support for "nested" mappings. (#645)
- Create findingIndex bugfix. (#653)
- Fix bucket level monitor findings to support term aggs in query. (#666)
- Mappings traversal bug fix. (#669)
- Set message when building LegacyCustomWebhookMessage. (#670)
- Fix error message bug to show the correct destination ID that's missing. (#685)
- Fix percolator mapping error when having field name 'type'. (#726)
- [Forward port to main] removed duplicate popout icon and ran prettier (#379) (#382)
- Change detector out of time range modal warning into a callout warning (#384)
- Fix undefined entity list when heatmap is empty (#383)
- Updating multi-field mapping at follower (686)
- Fixing CI workflow to run on matrix.os (#292)
- Fix event analytics field blank margin (#174)
- Correct ppl learn more link (#154)
- Fixed PPL Error in containers & added tool tip to date picker (#179)
- Fix explorer dark mode issue and restructure scss (#157)
- Few fixes regarding issues for visualization rendering (#186)
- Fix bug with overriding patterns (#1298)
- QS to 6.5.3 (#1335)
- Bump json5 from 2.2.1 to 2.2.3 (#20)
- Upgrade dompurify to 2.4.1 (#587)
- Use advanced settings for leading wildcards in query for csv reports (#549)
- Upgrade json5 and glob-parent (#17)
- [FIX-149] Sanitise duration value (#150)
- [Bugfix] filter data streams from rollover alias requirement (#429)
- Add fix to fromXContent and toXContent in ModelGraveyard (#624)
- Allow mapping service to be null for scenarios of shard recovery from translog (#685)
- Add backward compatibility and validation checks to ModelGraveyard XContent bug fix (#692)
- Add Connection read timeout to prevent indefinite wait (#354)
- Remove jackson-databind and jackson-annotations (#587)
- Fix metrics tests and ClassNotFoundException when calling stats API (#546)
- Fixed aliases being returned in unmapped_index_fields. (#147)
- Fix vulnerability in yaml constructor. (#198)
- Fix flaky integration tests for security analytics. (#241)
- Fixed SecureFindingRestApiIT. Removed uppercasing of the detector type. (#247)
- Fix ci builds for security-analytics. (#253)
- Alerts and Findings overview table should have even height. (#250)
- Fix cypress flaky tests. (#261)
- Fetch only Rules matching Rule Types. (#262)
- Edit detector rules table paging goes to page the first page. (#270)
- Cypress windows tests fix. (#296)
- Wait for field mapping creation to succeed before detector creation API call. (#317)
- Fixed styling issues. (#322)
- Patch missing detector_id with data already on UI. (#328)
- Fix tenant label for custom tenant when both Global and Private tenants are disabled (#1277, #1280)
- Fix openid redirect issue to use base_redirect_url when nextUrl is absent (#1282, #1283)
- Add Notifications cluster permissions (#1290, #1291)
- Fix regression in jwt url parameter by awaiting async getAdditionalAuthHeader (#1292, #1296)
- Fix
FLOAT->DOUBLEcast. (#1025) - Fix error messaging from prometheus. (#1029 #1037)
- Add
queryfunction as alternate syntax toquery_stringfunction (#1010) - Deprecate span collector (#990)
- Back quote fix (#1041 #1050)
- Update DATE and TIME functions to parse string input as datetime (#991)
- Integ tests fix for arm64 (#1069)
- Fix history file usage in SQL CLI tool. (#1077)
- Update Jackson to 2.14.1 and fix dependency resolution issues (#1150)
- Change LIKE operator case-insensitive match (#1160)
- Fix arithmetic operator precedence (#1172 #1188)
- Fix back quoted alias of FROM subquery (#1189 #1208)
- Fix truncate function (#1197 #1213)
- Allow common keywords and scalar function name used as identifier (#1191 #1212)
- Suppress report uploading failure in CI. (#1180 #1220)
- Fixed error with single timestamp query (#1244 #1246 #1249)
- Support JOIN query on object field with unexpanded name (#1229 #1250)
- Bug fix for less than and greater than operators on @timestamp (#1267 #1272)
- Refactored the help text elements displayed when users access the destinations list page after destinations deprecation. (#413)
- Add branch constants in CI workflow (#345)
- Bump to 2.5.0; add constants in CI workflow (#346)
- Bump decode-uri-component (#359)
- Add windows env to integration test workflow (#390)
- Bump json5 to 2.2.3 (#393)
- Unit Test Coverage Threshold (#101)
- Change github actions file and add necessary files (#4)
- Add .whitesource configuration file (#1)
- Fix windows and macos CI (#569)
- Add benchmark workflow for queries with filters (#598)
- Fix failing codec unit test (#610)
- Update bwc tests for 2.5.0 (#661)
- Add release configs for lucene filtering (#663)
- Update backwards compatibility versions (#701)
- Update tests for backwards codecs (#710)
- Unit tests coverage for load/unload/syncup (#592)
- Add .whitesource configuration file (#626)
- Bump djl to 0.20 and add onnxruntime-gpu dependency (#644)
- Remove jackson-databind and jackson-annotations dependencies now coming from core (#652)
- Revert "Remove jackson-databind and jackson-annotations dependencies now coming from core" (#687)
- Adding backwards compatibility test for ml-commons plugin (#681)
- Change the inheritance of the BWC test file (#692)
- Include integration tests in windows and macOS workflow (#1375)
- Remove unnecessary scripts after repo split (#1372)
- Add bwc tests in distribution level (#1366)
- Include integration tests in windows and macOS workflow (#624)
- Remove unnecessary scripts after repo split (#622)
- Remove front end code(#620)
- Remove mac os from unit test platforms. (#211)
- Public Components Snapshot Tests. (#218)
- Sort alerts in descending order of timestamp by default. (#222)
- Filtered findings shown in alert details. (#229)
- Cypress checking on rule YAML content. (#248)
- Creating new object for alert condition initialization. (#255)
- Added windows to cypress test runs. (#259)
- Streamline rules request. (#281)
- Detector must have at least one alert set. (#289)
- Updated field mapping UX; disabled windows run for cypress. (#307)
- Improve alert condition input placeholders. (#308)
- Status chart colors update. (#309)
- Change ApplicationContext lifecycle to per node level (#822)
- Catalog to Datasource changes (#1027 #1049 #1086)
- Bump jackson to 2.14.0 (#1058)
- Add metadatalog interface and default in memory implementation (#974)
- Decouple function repository and DSL from IoC container for use anywhere (#1085)
- Move DataSourceServiceImpl to core module (#1084)
- Improve pushdown optimization and logical to physical transformation (#1091)
- Using jackson and jackson_databind version defined in OpenSearch (#1169 #1173)
- Add BWC tests for running against distribution bundle. (#1209)
- Added 2.5 release notes. (#743)
- Add 2.5.0 release notes. (#440)
- Updated MAINTAINERS.md format. (#388)
- Add 2.5.0 release notes. (#6)
- Add release notes for 2.5.0.0 (#155)
- 2.5 release note (#658)
- 2.5 release note (#561)
- Update MAINTAINERS.md format (#709)
- Updated MAINTAINERS.md format (#668)
- Updating maintainers list (#663)
- Add doc about how to setup GPU ML node (#677)
- Add 2.5.0 release notes. (#600)
- Adding release notes for 2.4.1 (#1343)
- Update 2.x to be the same as observability repo (#158)
- Release notes for 2.4.0 (#1259)
- Adding release notes for 2.4.1 (#1343)
- Added 2.5 release notes. (#268)
- Support opensearch-sql:run and update developer_guide doc (#1099)
- Updated MAINTAINERS.md to match recommended opensearch-project format. (#1224 #1233)
- [AUTO] Increment version to 2.5.0-SNAPSHOT (#629)
- Bumped version to 2.5. (#437)
- Increment version to 2.5.0-SNAPSHOT (#711)
- Bumped version to 2.5. (#571)
- Updated MAINTAINERS.md format (#108)
- Add Mingshi Liu as a maintainer (#112)
- Add config rule in release note for security category (#128)
- Increment from 2.x to 2.5.0 (#29)
- Add release note for 2.5.0 (#135)
- Bump version for ansi-regex, gs, glob-parent and update yarn.lock (#107)(#113)
- Bump version to 2.5.0 (#144)
- Increment version to 2.5.0-SNAPSHOT (#184)
- Fix all the compile warnings and detekt issues (#603)
- Unify test clean logic (#609)
- Security Workflow (#611)
- Bump 2.5.0 (#638)
- Updated MAINTAINERS.md format (#650)
- Change help text in "RestoreSnapshotAdvancedSettings" , documentation url in "IndexSettingsInput" (#365)
- Bump loader-utils to 1.4.1 to address vulnerability alert, specify OSD 2.4.0 (#381)
- Data stream selection popover auto adjust width (#412)
- Version bump 2.4.1 (#430)
- Feat: fix mac platform error (#518)
- Updated MAINTAINERS.md format. (#539)
- Bump to 2.5.0 (#540)
- Fix the codec94 version import statements (#684)
- Add integ test for index close/open scenario (#693)
- Make version of lucene k-nn engine match lucene current version (#691)
- Increment version to 2.5.0-SNAPSHOT (#632)
- Increment version to 2.5.0-SNAPSHOT (#513)
- Increment version to 2.5.0-SNAPSHOT (#76)
- Bumped version to 2.5. (#571)
- Increment version to 2.5.0-SNAPSHOT (#1205)
- Increment version to 2.5.0-SNAPSHOT (#1205)
- Increment version to 2.5.0-SNAPSHOT #528)
- Made minor changes to polish the UI. (#247)
- Bumped version to 2.5. (#297)
- Updated MAINTAINERS.md format. (#284)
- Bump json5 from 1.0.1 to 1.0.2. (#285)
- Remove experimental banner. (#303)
- Added exception check once the .opendistro-alerting-config index is being created. (#650)
- Set lastnotification time as created time for new bucket level monitor. (#675)
- Separate variables for alert and finding scheduled rollovers. (#705)
- Overriding defaultTitle with subject for SNS notifications. (#708)
- QueryIndex rollover when field mapping limit is reached. (#725)
- Added unwrapping exceptions from core. (#728)
- Change task worker node to list; add target worker node to cache (#656)
- Remove unused MLPredict Transport action from src (#94)
- Search returns detector type in CAPS fix and integration tests. (#174)
- Added dummy search when creating detector on the given indices. (#197)
- Updated network mappings. (#211)
- Updated windows mappings. (#212)
- Updated ad_ldap mappings. (#213)
- Removed create/delete queryIndex. (#215)
- Update Linux mappings. (#223)
- Changes to return empty search response for custom rules. (#231)
- Service Returns Unhandled Error Response. (#248)
- The SQL Plugin was rearranged into separate repositories (#640 #1263)
- Added SQL-Jdbc repository https://github.com/opensearch-project/sql-jdbc (#964)
- Added SQL-Jdbc maven artifact as a dependency (#2692)
- Added SQL-Odbc repository https://github.com/opensearch-project/sql-odbc (#965)
- Added SQL-Cli repository https://github.com/opensearch-project/sql-cli (#966)
- Added Workbench repository https://github.com/opensearch-project/dashboards-query-workbench (#1266)