Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport] [2.x] Expose DelimitedTermFrequencyTokenFilter (#9479) #9671

Merged
merged 1 commit into from
Aug 31, 2023
Merged

[Backport] [2.x] Expose DelimitedTermFrequencyTokenFilter (#9479) #9671

merged 1 commit into from
Aug 31, 2023

Conversation

reta
Copy link
Collaborator

@reta reta commented Aug 31, 2023

Backport of #9479 to 2.x

@reta reta added the backport PRs or issues specific to backporting features or enhancments label Aug 31, 2023
@reta reta force-pushed the backport/backport-9479-to-2.x branch from d6108ca to 843e7f8 Compare August 31, 2023 18:58
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change 843e7f8

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git]

@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change d6108ca

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git]

@russcam @reta
Expose DelimitedTermFrequencyTokenFilter (opensearch-project#9479)
350a1e0 
* Expose DelimitedTermFrequencyTokenFilter

Relates: opensearch-project#9413

This commit exposes Lucene's delimited term frequency token filter to be
able to provide term frequencies along with terms.

Signed-off-by: Russ Cam <[email protected]>

* fix format violations

Signed-off-by: Russ Cam <[email protected]>

* fix test and add to changelog

Signed-off-by: Russ Cam <[email protected]>

* Address PR feedback

- Add unit tests for DelimitedTermFrequencyTokenFilterFactory
- Remove IllegalArgumentException as caught exception
- Add skip to yaml rest tests to skip for version < 2.10

Signed-off-by: Russ Cam <[email protected]>

* formatting

Signed-off-by: Russ Cam <[email protected]>

* Rename filter

Signed-off-by: Russ Cam <[email protected]>

* update naming in REST tests

Signed-off-by: Russ Cam <[email protected]>

---------

Signed-off-by: Russ Cam <[email protected]>
(cherry picked from commit 1126d2f)
Signed-off-by: Andriy Redko <[email protected]>
@reta reta force-pushed the backport/backport-9479-to-2.x branch from 843e7f8 to 350a1e0 Compare August 31, 2023 19:41
@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change 350a1e0

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Aug 31, 2023

Codecov Report

Merging #9671 (350a1e0) into 2.x (a3548ba) will increase coverage by 0.07%.
Report is 2 commits behind head on 2.x.
The diff coverage is 86.00%.

@@             Coverage Diff              @@
##                2.x    #9671      +/-   ##
============================================
+ Coverage     70.83%   70.90%   +0.07%     
- Complexity    57737    57817      +80     
============================================
  Files          4773     4776       +3     
  Lines        273059   273105      +46     
  Branches      40192    40197       +5     
============================================
+ Hits         193417   193647     +230     
+ Misses        63072    62861     -211     
- Partials      16570    16597      +27
Files Changed Coverage Δ
...arch/telemetry/tracing/sampler/RequestSampler.java 66.66% <66.66%> (ø)
...ensearch/analysis/common/CommonAnalysisPlugin.java 91.15% <75.00%> (-0.92%) ⬇️
...elemetry/tracing/sampler/ProbabilisticSampler.java 82.35% <82.35%> (ø)
...mmon/DelimitedTermFrequencyTokenFilterFactory.java 100.00% <100.00%> (ø)
.../org/opensearch/telemetry/OTelTelemetryPlugin.java 100.00% <100.00%> (ø)
...search/telemetry/tracing/OTelResourceProvider.java 100.00% <100.00%> (ø)
...rg/opensearch/common/settings/ClusterSettings.java 93.18% <100.00%> (ø)
...va/org/opensearch/telemetry/TelemetrySettings.java 85.71% <100.00%> (+10.71%) ⬆️

... and 462 files with indirect coverage changes

@reta
Copy link
Collaborator Author

reta commented Aug 31, 2023

@msfroh please mind approving?

@msfroh msfroh merged commit 293da7e into opensearch-project:2.x Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msfroh msfroh msfroh approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@tlfeng tlfeng Awaiting requested review from tlfeng

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

Assignees
No one assigned
Labels
backport PRs or issues specific to backporting features or enhancments
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@reta @msfroh @russcam